CS 112 Assignment 3

Due: Wednesday, February 22 at 5:00pm

Due to the upcoming exam on Thursday, February 23, this assignment will be due by 5:00pm on Wednesday, February 22. After you submit your solutions, we will send a file with the formal solutions to you by e-mail, so that you can refer to them as you are studying for the exam. Your hardcopy submission for this assignment should include a cover sheet and printouts of four code files: illusions.m, grades.m, energy.m, and recognize.m. Your electronic submission is described in the section Uploading your saved work.

Reading

Getting Started: Download assign3_programs from cs112d

Uploading your saved work

• Browse through your directory and find the drop/assign03 folder
• Upload your assign3_programs folder into your drop/assign03 folder
• When done uploading, be sure to delete your renamed assign3_programs folder from the Desktop by dragging it to the trash can, and then empty the trash (Finder--> Empty Trash).
• Be sure to Exit out of MATLAB when you are done

When you are done with this assignment, you should have at least four code files stored in your assign3_programs folder: illusions.m, grades.m, energy.m, recognize.m.

Exercise 1: Seeing is believing

Synthetic images can be created in MATLAB to explore phenomena in visual perception. In this exercise, you will create one image of a perceptual illusion and a color stereo anaglyph to view 3-D surfaces. Your code for this exercise should be added to the illusions.m code file in the assign3_programs folder.

Let the sun shine in...

The image below on the left shows two cinder cones from the K'au Desert lava fields in Hawaii. The same image is shown on the right, flipped upside down (if you don't believe this, print a hardcopy and rotate the page upside down). As you can see, the cones in the left image appear as mounds with craters in the middle, while the same cones in the right image appear as craters with mounds in the middle. Shadows, highlights and smooth shading in an image help us to perceive the 3-D shape of surfaces. Knowing the direction of the illumination in the scene, such as the direction of the sun's rays, is important to this perceptual process. If we misperceive the direction of the light source, this can result in a misperception of 3-D shape. Our visual system evolved in an environment where light from the sun shines from above, as in the image on the left. When the image is turned upside down, the light source now comes up from below, but old habits die hard - we still think the surfaces are illuminated from above and misperceive their 3-D shape as a consequence.

The illusions.m code file contains a code statement that loads an image of a sphere illuminated from above. The image is stored in a 110x110 matrix named sphere1:

Add code to create a large matrix that is initially filled with a constant value of 0.8 (the light gray background of the sphere, seen in the corners of the above image) and create at least four non-overlapping copies of the sphere1 image in this larger matrix. Make half the copies upright and half upside-down, and display the resulting image with imshow. Do all of the spheres appear to have the same 3-D shape, protruding outward from the computer monitor? In addition to assuming that the illumination in a scene usually comes down from above, the human visual system also assumes that there is usually only one major source of illumination in the scene.

Two eyes are better than one...

Our two eyes view the world from slightly different perspectives that cause objects at different depths to appear at slightly different locations in the images seen by the left and right eyes. A color stereo anaglyph is a composite image that superimposes two images that mimic the views seen by the left and right eyes. The colors of the two superimposed images are adjusted so that one image (say, the left eye's image) is more apparent when viewed through a red filter, while the other (the right eye's image) is more apparent when viewed through a blue or green filter. If we view a color stereo anaglyph with glasses that have filters of different color for the two eyes, our visual system fuses the left and right images into a percept of 3-D structure.

There are many online galleries of color stereo anaglyphs, such as the one found here that includes the "blossoms" image shown below. In lab, we'll lend you a pair of red/blue glasses for viewing color stereo anaglyphs. The one shown below is best viewed with the red filter over the left eye and the blue filter over the right.

Vision scientists often use random-dot stereograms to probe aspects of human stereo vision through perceptual studies. The images in these stereograms consist of random patterns of black and white dots. The same dots appear in the left and right images of the stereogram, but the locations of the dots may be shifted to the left or right in one image relative to the other, mimicking the way objects at different depths appear at different locations in the two eyes. In the pictures below, the pair of black-and-white images on top form a stereogram. There is a square patch of dots in the center of the left image that is shifted slightly to the right in the right image. The dots surrounding this square patch appear at the same locations in the two images. If the stereogram is viewed in such a way that the left eye sees the left image and the right eye sees the right image, the two images will fuse into a 3-D percept of a flat background surface with a square floating in front of it, closer to the viewer. The picture shown below the stereogram is a color stereo anaglyph constructed from the left and right images. When viewed with red/blue glasses (red on the left and blue on the right), the anaglyph will also give rise to the same percept of a flat background surface with a square floating in front.

In this exercise, you will create your own random-dot stereogram and view it as a color anaglyph. The function makeDots, which is defined in the assign3_programs folder, creates a matrix of random black and white dots (0's and 1's). This function has two inputs corresponding to the number of rows and columns in the matrix. For example, the expression makeDots(200,200) creates and returns a 200x200 matrix of dots. First create a random-dot stereogram with the following steps:

• Create a matrix of random dots for the left image, using makeDots - select a size for the image somewhere between 200 and 400
• Create a matrix for the right image that is an exact copy of the left image
• Copy some patches of dots from the left image to the right image, shifting the position of the patch to the left or right by 1 to 6 pixels (larger shifts may make it more difficult to fuse the left and right images, but you're welcome to experiment). It is best to use fairly large patches (at least 25 rows and columns) and it is ok for the patches to overlap.

The final step is to create a color stereo anaglyph from your stereogram. We'll learn more about color later in the course, but for now, color images can be stored in a three-dimensional matrix, where the third dimension stores three values corresponding to the amount of red, green and blue that is combined to create the color that is displayed at each pixel. We will work with color images in which the values for red, green and blue range from 0.0 (no color) to 1.0 (maximum color). The following picture shows an enlarged version of a tiny portion of the central region of the color anaglyph shown above. You can see four colors in this picture: red, blue, magenta and black. Imagine overlaying the two black-and-white images that form the original stereogram. At locations where both patterns have a black dot, the color anaglyph also has a black dot. At locations where the left pattern has a white dot and the right pattern has a black dot (due to the shifting of the dots between the two images), the anaglyph has a red dot. Analogously, if the left pattern has a black dot at a location where the right pattern has a white dot, the anaglyph will have a blue dot. Finally, if both patterns have white dots at the same location, the anaglyph will have a magenta dot (the combination of red and blue). Carefully view the image below with the red/blue glasses - note that the red dots disappear when viewing the pattern through the red filter, and blue dots disappear when viewing through the blue filter.

Suppose the matrices storing the stereogram pair are named left and right and have size 200x200. The following code creates a color stereo anaglyph and displays it in a new figure window:

anaglyph = zeros(200,200,3);  % create a 3-D matrix
anaglyph(:,:,1) = left;       % copy the left image as red
anaglyph(:,:,3) = right;      % copy the right image as blue
figure
imshow(anaglyph);

Using the above code as a guide, create and display a color stereo anaglyph from the stereogram that you created, and of course, view it with your red/blue glasses to see the surfaces that you created!

Add comments to your illusions.m code file before submitting your assign3_programs folder.

Exercise 2: Working with gradesheets

The following table provides the semester grades for six CS112 students:

Name Homework Partic. Exam 1 Exam 2 Final Project Ann 83 90 76 80 88 Barb 79 80 80 81 70 Chelsea 90 90 92 98 94 Diana 72 75 63 75 73 Ella 88 90 78 89 82 Fiona 67 65 69 72 80

Open the file called grades.m. This file creates a matrix named scores that contains the information in the above table, and two cell arrays called names and measures that store the names of the students and the components of the grade. Add code to the grades.m code file to perform the tasks listed below. Try to write code that is as compact as possible but still readable and understandable.

• Plot the Exam 1 and Exam 2 scores for the six students on a single graph, with the y-axis ranging from 50 to 100.
• Add a new column to the scores matrix with each student's final course score. The course score is calculated in CS112 as follows: Homework counts 40%, Participation 5%, Exams are each 20% and the Final Project is 15%.
• Create a new vector avgScores with the average (across students) of each component of the final grade, as well as the average final course grade.
• Print the names of the students whose final course score is higher than the class average.
• Print the names of the grade components that received the highest and lowest average scores.
• Plot the final course grades for the six students on the same graph as the Exam 1 and 2 scores, and add a horizontal line across the graph whose height is the average final course grade (for reference).
• Add a title, axis labels and legend to the graph.

Problem 1: Energy production and consumption

In this problem, you will explore data on U.S. energy production and consumption using energy statistics available from the Energy Information Administration (EIA) within the U.S. Department of Energy. The assign3_programs folder contains two Excel spreadsheets, production.xls and consumption.xls, that were downloaded from the Energy Overview page at the EIA website. These spreadsheets contain both numerical and textual data related to the production and consumption of various energy sources over the years 1949-2006, which you can view by opening these files in Excel. Unfortunately, a separate MATLAB toolbox is needed to load data from complex spreadsheets such as this, and the public computers at Wellesley do not currently have this toolbox. We can, however, read Excel spreadsheets that contain only numerical data into MATLAB. The two files, produce.xls and consume.xls, contain most of the numerical data from the original EIA spreadsheets. These files can be read into MATLAB using the xlsread function, for example:

productionData = xlsread('produce.xls');

The contents of both produce.xls and consume.xls will be loaded into matrices with 58 rows corresponding to the years 1949-2006. The matrix created from produce.xls has 14 columns representing (1) year, (2) coal, (3) natural gas, (4) oil, (5) NGPL (natural gas plant liquids), (6) total fossil fuels, (7) nuclear, (8) hydroelectric, (9) geothermal, (10) solar, (11) wind, (12) biomass, (13) total renewable energy and (14) total energy produced. The matrix created from consume.xls contains 13 columns representing (1) year, (2) coal, (3) natural gas, (4) oil, (5) total fossil fuels, (6) nuclear, (7) hydroelectric, (8) geothermal, (9) solar, (10) wind, (11) biomass, (12) total renewable energy and (13) total energy consumed. The file population.mat in the assign3_programs contains a column vector of the U.S. population over the years 1949-2006.

Create a script file named energy.m that reads in the contents of the produce.xls, consume.xls and population.mat files and performs the following tasks:

Task 1: Plot the raw data

In a single figure window, plot the following data in one graph: amount of coal, gas, oil, NGLP, nuclear and total renewable energy produced, and the amount of coal, gas and oil consumed (note that the U.S. consumes all of the renewable energy sources that it produces). Plot this data as a function of the year. There should be 9 line plots drawn in a single plotting area. Complete this code without creating any additional variables - each call to the plot function should refer directly to the two matrices storing the data. Use one line style for all of the production data and a different line style for all of the consumption data, and use different colors for the plots. Add a title, axis labels and legend for the graph. Note that you can drag the corners of the figure window to expand its size, and drag the legend to a new location if desired.

Task 2: Graphical analyses of the data

Open a second figure window and create four graphs that display the following information for each year. In each case, the year can be plotted on the x axis. Use subplot to define a 2 x 2 grid of plotting areas for drawing the four graphs.

1. the fraction of total energy consumed that is produced in the U.S.
2. the amount of foreign oil that was needed to support the demand (i.e. the difference between oil consumed and produced)
3. the percentage of energy consumed that comes from renewable sources
4. the per capita total consumption of all energy sources combined

These observations are unfortunately a little disturbing...

Add comments to your energy.m code file.

Problem 2: Recognizing famous Wellesley alums

Wellesley College is proud to have some very distinguished alumnae! For this problem, you'll write a program to recognize the faces of three of our special graduates: Madeleine Albright '59, Hillary Clinton '69 and Pamela Melroy '83. The assign3_programs folder contains three face images whose identity is assumed to be known (albright.jpg, clinton.jpg, melroy.jpg) and three face images to be recognized by your program (face1.jpg, face2.jpg, face3.jpg). The file recognize.m contains initial code that loads the six face images into variables in the MATLAB workspace and displays them using subplot and imshow (the top three images are the known faces and the bottom images are the "unknown" faces):

In lecture, we explored the problem of fingerprint identification and discovered the shocking truth that Sohie is behind the recent string of crimes in the CS department! Given a partial fingerprint lifted from the crime scene, stored in the variable finger, and Sohie's partial fingerprint stored in the variable sohie, we calculated the average difference between the brightness values stored in the two images:

sohieDiff = mean(mean(abs(finger-sohie)));

After computing this difference for our other suspects Randy and Scott, we determined that Sohie's fingerprint was the closest match!

Recognition using the full image

Using the same strategy for calculating how well the pattern of brightnesses match between two images, first add code to the recognize.m code file to calculate the average difference between the face1 image and each of the three known face images stored in the variables albright, clinton and melroy. Then add code to determine which of the three known faces is the best match to face1 using the three average differences that you calculated. Print a message indicating the identity of face1. Then repeat this process for the unknown faces stored in face2 and face3. Tip: cutting and pasting, and then making small modifications to the copied code, will save a lot of time! Your program should be able to recognize each of the three faces correctly.

Recognition using partial images

Real face recognition systems face many challenges: hair styles change and faces appear with different expressions, poses and directions of gaze, and with different backgrounds and lighting. To cope with variations in hair styles and backgrounds, the recognition process is often based on a cropped area of the face that excludes the hair and background. For this part of the problem, you'll attempt to recognize the three faces using a cropped region of the faces that includes only the eyes, nose and mouth. The original images (the images in the top row of Figure 1 above, labeled with each person's name) were carefully constructed so that they have the same overall size, with the region around the eyes, nose and mouth covering roughly the same area of each image. Create a set of six new variables that each store a small region of one of the original face images containing only the eyes, nose and mouth. Use the same coordinates for the upper left and lower right corners of the rectangular region that you select for all six images. Use imtool to help identify appropriate coordinates (as you move your mouse over the image displayed with imtool, the (X,Y) coordinates of the mouse are displayed in the lower left corner of the display window - remember that the order of these coordinates should be reversed when specifying the row and column of the corresponding locations of the matrix storing the image). Using subplot and imshow, display the six cropped images, as shown in the example below (again consider copying, pasting and modifying code here!):

Repeat your code to recognize the cropped versions of the face1, face2 and face3 images (you can copy and paste the code that you created to recognize the full images, and then make modifications to this code to use the cropped images instead). Are you still able to recognize each of the three faces correctly?

Be sure to add comments to your recognize.m code file.