Assignment 6
Due: Thursday,
October 27
|
|
CS 332
Assignment 6
Due: Thursday, |
|
This assignment contains two programming problems on computing a 2D
velocity field, and a third problem about a motion illusion known as
reversed phi. Images and initial code for the programming problems
are contained in the /home/cs332/download/motion subdirectory
on the CS file server. After downloading this folder, set the Current Directory
in MATLAB to this folder.
This problem builds on previous methods that you explored for matching fingerprints and solving the stereo correspondence problem. In the fingerprint matching application (an extra credit problem on Assignment 2), a single partial fingerprint was compared to patches in a full, known fingerprint image to search for a good match. The measure used to assess the quality of the match was the sum of the absolute differences in intensity between the unknown fingerprint pattern and patches of the known fingerprint. In the case of stereo correspondence, every patch of the left image (excluding a border around the image) was compared to a set of horizontally displaced patches in the right image to find the best match and assign a stereo disparity to each location of the left image. In this context, the measure used to assess the quality of the match was the correlation between the pattern of zero-crossings in the left and right image patches.
In this problem, you will implement a strategy for motion measurement that combines
elements of the previous stereo and fingerprint matching solutions. The motion
folder contains a definition of the stereoMatch function from Assignment 4 that
uses the "sum of absolute differences" measure of similarity that was used in the fingerprint
matching problem. Use this function as a starting point to create a function named
motionMatch that computes the motion at each location in a sequence of images.
Your function should have four inputs, similar
to the stereoMatch function, representing the two images in a motion sequence,
an input that indicates the size of the image patches used for matching between the two images
(nsize) and an input range that specifies the range of displacements
of the patches in the horizontal and vertical directions to be considered by the function. For
each patch in the first image (ignoring a border around the image), the function should
find a patch in the second second image that is the best match, and record both the
horizontal and vertical displacements between the two patches. These displacements should be
recorded in two separate matrices that are provided as outputs of the function. The
testMatch.m script in the motion folder contains code to test your
motionMatch function that reads in two images from the motion folder,
shows the images as a movie, and creates and displays the true velocity field for the images.
In comments, there is code to run your motionMatch function on these two images.
Compare your results to the true velocity field. Add comments to your motionMatch
function that answer the questions, where do the errors occur in the results, and why
might you expect errors in these regions?
In this problem, you will write a function named computeVelocity whose input
includes the perpendicular components of motion derived from two images in a motion
sequence, and whose output is a 2D velocity field.
In class, we developed an algorithm to compute 2-D velocity from the perpendicular components
of motion, assuming that velocity is constant over extended regions in the image. Let
(Vx,Vy) denote the 2D velocity,
(uxi,uyi) denote the unit vector in the direction of
the gradient (i.e. perpendicular to an edge) at the ith image location, and
v⊥i denote the perpendicular component of velocity at this
location. In principle, from measurements of uxi, uyi and
v⊥i at two locations, we can
compute Vx and Vy by solving the following
two linear equations:
Vx ux1 + Vy uy1 =
v⊥1
Vx ux2 + Vy uy2 =
v⊥2
In practice, a better estimate of (Vx,Vy) can be obtained
by integrating information from many locations, and finding values for Vx and
Vy that best fit a large number of measurements of the perpendicular
components of motion. Because of error in the image measurements, it is not possible to find values for
Vx and Vy that exactly satisfy a large number of
equations of the form:
Vx uxi + Vy uyi = v⊥i
Instead, we compute Vx and Vy that minimize the difference
between the left- and right-hand sides of the above equation. In particular, we compute
a velocity (Vx,Vy) that minimizes the following expression:
∑[Vx uxi + Vy uyi -
v⊥i]2
where ∑ denotes summation over all locations i. To minimize this
expression, we compute the derivative of the above sum with respect to each of the two parameters
Vx and Vy, and set these derivatives to zero. This
analysis yields two linear equations in the two unknowns Vx and Vy:
a1 Vx + b1 Vy = c1
a2 Vx + b2 Vy = c2
where
a1 = ∑uxi2
b1 = a2 = ∑uxiuyi
b2 = ∑uyi2
c1 = ∑v⊥iuxi
c2 = ∑v⊥iuyi
The solution to this pair of equations is given as follows:
Vx = (c1b2 -
b1c2)/(a1b2 - a2b1)
Vy = (a1c2 -
a2c1)/(a1b2 - a2b1)
Your computeVelocity function will implement this solution.
The function getMotionComps, which is already defined in the motion
folder, computes the initial
perpendicular components of motion. This function has three inputs - the first two are
matrices containing the results of convolving two images with a Laplacian-of-Gaussian function.
It is assumed that there are small movements between the original images. The
third input to getMotionComps is a limit on the expected magnitude of the
perpendicular components of motion. This function has three outputs that are matrices containing
values of ux, uy and
v⊥. These quantities are computed only at the
locations of zero-crossings of the second input convolution. At locations that do not
correspond to zero-crossings, the value 0 is stored in the output matrices.
Your computeVelocity function should have the following header:
function [vx vy] = computeVelocity (ux, uy, vp, nsize, step, vlim)
The inputs ux, uy and vp are the three matrices that are returned by
the getMotionComps function. nsize is a neighborhood size for
integrating the motion components to compute the velocity at a particular location. To reduce the
amount of computation, velocities do not need to be computed at every location. Instead,
velocities should be computed at evenly spaced locations in the horizontal and vertical directions,
with the input step specifying the space between these locations.
Finally, vlim is a limit on the expected horizontal and vertical velocities that should
appear in the results. The two outputs of the computeVelocity function are matrices
of the same size as the input matrices, containing values for
Vx and Vy at the locations where velocity was
computed, and the value 0 elsewhere.
The computeVelocity function should step through the equally spaced image locations,
and at each location (x,y), it should
integrate all of the measurements of ux, uy and
v⊥ within a square region from
(x-nsize,y-nsize) to (x+nsize,y+nsize) and compute the coefficients
a1, a2, b1, b2, c1 and
c2 (remember to initialize these coefficients to
0 before accummulating
information for each new region). The velocity for the region should then be computed by solving for
Vx and Vy as shown above. If the absolute values of
both Vx and Vy are within the limit vlim,
then Vx and Vy
should be stored at the corresponding locations in the output matrices vx and
vy.
The motionTest.m script file contains two examples for testing your new function. The
first example uses images of a circle translating down and to the right. The second example,
which is initially in comments, uses a collage of four images of current and past Red Sox players,
where each subimage has a different motion, as shown by the red arrows on the image below:
Big Papi is shifting down and to the right, Manny is shifting right, Varitek and Lowell are
shifting left, and Coco Crisp is leaping up and to the left after a fly ball. For both examples,
the velocities computed by your computeVelocity function are displayed by
the displayV function in the motion folder, which uses the built-in
quiver function to display arrows. Your results for the Red Sox image should roughly
reflect the correct velocities within the four different regions of the image, but there will be
significant errors in some places. In comments in your code, answer the following
questions: Where do most of the errors in the results occur, and why might you expect
errors in these regions?
The results of your implementation will vary, depending on the size of the neighborhood used
to integrate measurements of the perpendicular components of motion. Run your
computeVelocity function with a larger and smaller neighborhood size and describe
the change in results. What are possible advantages or disadvantages of using a larger
or smaller neighborhood size for the computation of image velocity?
There is an interesting phenomenon known as reversed phi, in which observers perceive motion that is opposite in direction to the true displacement of an edge. Two images are constructed and alternated back and forth as a movie. If the edges are oriented vertically, then the first image has a single edge that is dark on the left and light on the right. To construct the second image, the edge is displaced to the right, and at the same time, its contrast is reversed, as shown below:
When the edge is displaced to the right (image 1 followed by image 2), observers perceive motion to the left. When the edge is displaced to the left while its contrast is reversed (image 2 followed by image 1), observers see motion to the right. Demonstrations of this general phenomenon can be viewed at http://www.michaelbach.de/ot/mot_reverse-phi/index.html. This percept is consistent with what we would expect from the strategy for computing the perpendicular components of motion described in lecture. Consider the convolutions of image 1 and image 2 with a Laplacian-of-Gaussian operator, and the signs of the derivatives of this convolution in the image and over time. When combining these derivatives to compute the components of motion, what direction of motion would result from this analysis? Explain your reasoning.
Submission details: Hand in a hardcopy of your motionMatch.m and
computeVelocity.m code files that contain your answers to the questions in Problems 1 and 2,
as well as your answer to problem 3.
Drop off an electronic copy of your code files by logging into puma,
connecting to your motion folder and executing the following command:
submit cs332 assign6 *.*