Questions about Affine transformation matrices

Some thought questions about affine transformations.

• Give an example of two rotations that are commutative. Give an example of two rotations that are not commutative.
• Are translations commutative?
• Do translations commute with scaling? With rotation?

Why Math?

In the past, some students have wondered why we're learning the matrix stuff and how that connects with the OpenGL stuff. It's a legitimate question. It doesn't seem to be necessary to using the OpenGL functions. Here are some reasons:

• Practical: While it may not be strictly necessary, it can be helpful to have an intuition of what goes on behind the scenes. It can help to explain why certain functions exist, such as glMultMatrixf(), and help you to understand the terminology of the reference books, such as glTranslatef(), which "multiplies the current matrix by a translation matrix." It will make you better graphics programmers.
• Conceptual: OpenGL has been the standard in graphics APIs for about 10 years. Will it still be the standard in another 10 years? Maybe. Probably not. However, whatever the new standard, I think it will still use homogeneous coordinates and affine transformations, defined as matrices. We try to teach you concepts that will last.
• Standard: As much as I've redesigned this course so that we can get to more fun things faster, there's still an expectation about what a computer graphics course is about, and the math is an important component. Some of you may go on in this field, go to grad school or work for Pixar, and I want you to have the solid ground that will take you there.
• Rigor: Learning the math can be hard, but undertaking intellectually demanding tasks is part of what college is about. So, even if this is the last experience you have with computer graphics, it's still good for you to have grappled with difficult concepts. (This is the "Brussels sprouts" reason).

Camera API

Additional Reading. If the following is confusing or you just want to hear a different explanation, feel free to consult the following:

• The Red Book, chapter 3
• Angel, sections 1.5-1.6 and 5.2-5.5
• Hearn and Baker, sections 7-3 and 7-10

There are two aspects to the camera API: the placement of the camera, and the shape of the camera. To understand the latter, we have to understand how cameras work.

The Pinhole Camera

This is a cutaway side view of a pinhole camera (in Latin, the "camera obscura," which means "dark room"). The z axis is imaginary, but the box on the left part of the picture is a real box: solid on all six sides. There is a tiny pinhole in the front of the box, where the circle is on the picture. The origin is placed at the pinhole, the y axis goes straight up through the front of the box, and the z axis points out into the scene. The depth of the box is d, and the height of the box is h. We also might care about the width of the box, but that's not visible on this picture, since we've eliminated that dimension.

Rays of light from the outside go through the hole and land on the image plane, which is the entire back of the box. For example, a ray of light from the top of the tree lands on the back of the box as shown. Because the hole is so tiny, only light from along a single ray can land on any spot on the back of the camera. So, in this example, only light from the top of the tree can land at that spot. This is why pinhole camera pictures are so sharp. Theoretically, they can provide infinite clarity, though in practice other issues arise (diffraction of light rays and the lack of enough photons). However in computer graphics, we only need the theoretical camera.

Pinhole cameras are simple and work really well. They're standard equipment for viewing solar eclipses. We only use lenses so that we can gather more light. A perfect pinhole camera only really allows one ray of light from each place in the scene, and that would make for a really dark image, or require very sensitive film (or retinas). Since the need to gather enough light is not important in computer graphics, the OpenGL model is of a pinhole camera.

Points to notice:

• The projection of an image (such as the tree) can be computed using similar triangles.
• The effects of perspective are manifest, such as things getting smaller with distance.
• Parallel lines seem to converge at a "vanishing point."
• One disadvantage of a pinhole camera is that the image is upside down on the film. (Your SLR camera has "through the lens" viewing, but uses additional lenses to flip the image right side up.) We'll see how to address this soon.

Computing Projection by similar triangles

Suppose we want to compute the projection of the top of the tree. We know the coordinates of the top of the tree, let them be (X,Y,Z). We want to know the coordinates of the projection, let them be (x,y,z). We can do this by similar triangles, using the two yellow triangles in the following figure:

The height and base of the big triangle are just Y and Z. The height and base of the little triangle are y and z. The base of the little triangle is known, because that's determined by the shape and size of our pinhole camera. In the figure, it's notated as "d." By similar triangles, we know:

y/d = Y/Z
y = d*Y/Z
y = Y/(Z/d)

Everything on the right hand side is known, so we can compute the projection of any point simply by knowing the depth of our pinhole camera and the location of the point.

As for the x coordinate, a similar argument holds.

The Synthetic Camera

There's a hassle with signs in the pinhole camera, such as whether the z value for the tree is negative. Also, the image ends up being upside down. In CG, since we're really only interested in the mathematics of projection, we use a synthetic camera, in which the image plane is placed on the same side of the origin as the scene.

• In CG, we can put the image plane in front of the focal point. This means the image is right side up.
• Mathematically, we'll make the origin be the focal point, with the camera pointing down the negative z axis.
• The image plane is the top of a frustum (a truncated pyramid). Run demos/camera/Frustum.py and spin it to see an example.
• The frustum is also our view volume. Anything outside the view volume is "clipped" away.
• Note that this also means that the CG system can't see infinitely far. That's because it needs to calculate relative depth, and it can't make infinitely fine distinctions.
• The projection is computed using a projection matrix
• Note that there is also an option of doing parallel projection rather than perspective projection. Parallel projection is useful in architectural drawings and the like. We will rarely use it.

Perspective Matrices and Perspective Division

For the same reason that we want to make all our affine transformations into matrix multiplications, we want to make projection into a matrix multiplication.

There are two kinds of projection available in OpenGL.

• Orthographic (Parallel). With this kind of projection, the view volume is a rectangular box, and we project just by squeezing out one dimension. As we mentioned earlier, this kind of projection is useful for architectural drawings and situations where there really isn't any perspective. If we align the direction of projection (DOP) with the Z axis, this kind of projection amounts to setting the z coordinate of all points to zero. Here's the orthographic projection matrix:

  1	0	0	0
  0	1	0	0
  0	0	0	0
  0	0	0	1

  Notice how it zeros out the z coordinate of the vertex. It's easier to come up with the matrix for that than to write it in HTML!

• Perspective. With this kind of projection, if we set up a frame where the origin is the center of projection (COP) and the image plane is parallel to the z=0 plane (the xy plane), we can compute the projection of each point using the similar triangles calculation, dividing each Y and X by Z/d.

The matrix for perspective projection isn't obvious. It involves using homogeneous coordinates and leaving part of the calculation undone.

The part of the calculation that is left undone is called perspective division. The idea is that the homogeneous coordinate (x,y,z,w) is the same as (x/w, y/w, z/w, 1): that is, we divide through by w. If w=1, this is a null operation, so it doesn't change our ordinary vertices. If, however, w has the value z/d, this perspective division accomplishes what we did earlier in our similar triangles, namely:

y = Y/(Z/d)

Therefore the perspective matrix is a matrix that accomplishes setting w=Z/d and leaves the other coordinates unchanged. Since the last row of the matrix computes w, all we need to do is put 1/d in z column of the w row. The perspective matrix, then, is just

Here's a PDF document that gives the math for projection

The OpenGL API for Camera Shape

Now that we know the basic theory, how do we get this to work in OpenGL? In OpenGL, to specify the camera shape, we have a choice:

1. We can specify the frustum directly using

   glFrustum(left,right,bottom,top, near,far);
   

   This is slightly more powerful than gluPerspective, but it can be unintuitive and therefore difficult. The first pair of arguments give the left and right edges of the top of the frustum along the X axis; the second pair are the same for the Y axis, and the last pair are the distance to the top and bottom of the frustum along the Z. The X and Y pairs are typically signed (positive and negative), while the Z pair must both be positive: distances measure from the origin.

   left and right are measured along the X axis, top and bottom along the Y axis.	Near and far are measured as positive distances along the negative z axis. Equivalently, there is a left-handed coordinate system in which they are the z coordinates.

2. Alternatively, we can specify the frustum indirectly using

   gluPerspective(fovy,aspect,near,far);
   

   This builds a frustum that is symmetrical around the Z axis. (So "left" and "right" are negatives of each other, such as -5 and +5, and similarly for "top" and "bottom".) The first argument is the "field of view y": the angle at the apex of the pyramid in the YZ plane. The second is the aspect ratio of the top of the frustum. The last two arguments are the same as for glFrustum. It looks like this:

   "Field of View Y" is the angle formed by the top and bottom of the frustum in the YZ plane.	the aspect ratio of the top of the frustum is width/height

The Meaning of FOVY

The first argument of gluPerspective() is called "fovy," which stands for "field of view, y." This is the angle between the upper and lower sides of the frustum.

Picture a pyramid, but not a normal square pyramid, but one where each slice through it is a rectangle. Picture it extreme, just to make this visualization easier. Now, think about the apex of the pyramid and the angle between opposite walls. There's a narrow angle between one pair of walls (the ones along the long sides of the rectangles) and a wider angle between the other pair. Thus, in the general case, there isn't just one angle for a frustum.

Since the frustum is aligned with the Z axis, we can refer to the angle in the YZ plane and the angle in the XZ plane. OpenGL arbitrarily chose to control the angle of the frustum using the angle in the YZ plane, hence the "field of view y." This is just the angle between the upper and lower (not left and right) sides of the frustum.

The FOVY is fairly intuitive with practice. Here are some things to keep in mind. (Try sketching to see what's going on.)

1. The greater the FOVY, the larger the frustum and therefore the view volume. (all else being equal)
2. The larger the view volume, the smaller the projection of anything in it. Therefore, things will appear small on the screen (all else being equal).
3. The closer the focal point (the center of projection or COP) to an object, the greater the angle that lines converge, so you get more noticeable perspective effects.
4. The perspective effect on an object depends on the relative depth of its front and back with respect to the COP. For example if the back of a cube is twice as far from the COP as the front, the image of the back will be half the size of the image of the front, so the perspective effect will be pronounced.
5. The farther the COP, the more parallel the projection lines, so you see less of a perspective effect.
6. The closer the image plane to the object, the larger its projection will be, and therefore the larger its image.
7. The human eye's FOV is approximately 90. A 50mm lens has a roughly 90 degree FOV, which is why photographers often choose that lens for natural scenes.
8. TW's camera setup uses a FOV of 90, and puts the camera very close to the scene. The result is that the image is big, but we get extreme perspective.
9. A zoom lens can also make the image big, but "flattens" the scene much more. A zoom lens has a small FOVY.
10. A wide-angle lens captures a lot of the scene, but makes everything small (so it'll fit). A wide-angle lens has a big FOVY.

The demos/camera/Perspective.py demo shows how much difference camera distance and shape can make. The two cubes are identical; what's different is the FOVY and the distance of the COP from the scene. You're encouraged to look at the code to see how the two cameras are set up. In particular, look at the code for leftDisplay() and rightDisplay, which sets up the left and right scenes in the demo.

Here's a screen shot of the demo:

• In the shot on the left, the FOVY=90 and camera is at z=2 and the image plane (the top of the frustum) is at z=1. The camera is 1 unit from the top of the frustum. The camera is 1.5 units from the near face of the cube and 2.5 units from the far face, so the projection of the far face is 3/5ths the size of the projection of the near face.
• In the shot on the right, the camera has been pulled back to be 3 units from the top of the frustum. The frustum has been left at z=1. Therefore, the FOVY=37 degrees. The top of the frustum is the same size and the same distance from the cube. However, the picture looks quite different.

The following are two diagrams from the side illustrating what is going on. The magenta cube is the wire cube, and it is centered around the origin, just as it is in the code. The z axis (blue arrow) goes horizontally to the left. The y axis (green arrow) goes up. The frustum is in red.

In this picture, the frustum has a FOVY=90.	In this picture, the frustum has a FOVY=37. The top of the frustum has been kept the same size by pulling back the camera to compensate.

In this course, we will typically use gluPerspective, but later we'll have occasion to use glFrustum for a special effect.

To modify the projection matrix, we have to do the following:

glMatrixMode(GL_PROJECTION);
glLoadIdentity();
gluPerspective(fovy,aspect,near,far);

Matrix Mode

Last time, we talked about the CTM. This is an almost entirely accurate concept, but it's not exactly implemented as a single matrix in OpenGL. Instead, it's implemented as two: the projection matrix and the modelview matrix. The first is about the shape of the camera, the second is about the transformations involved with placing objects in the scene and placing the camera.

To modify the shape of the camera, we switch to operating on the projection matrix. Then, since what we almost always want to do is to replace the current matrix with a different one, we load the identity matrix and then multiply by the desired projection matrix. That explains the sequence of calls at the end of the last section. To switch back to modifying the modelview matrix, we just do:

glMatrixMode(GL_MODELVIEW);

The OpenGL API for Camera Location

As we saw above, the perspective and orthographic projections work when the axis is aligned with the Z axis. This is actually the same as the initial coordinate system in OpenGL. But what if we don't want our scene/camera set up that way? We can either move the camera or move the scene: these are the same thing!

Therefore, to specify camera location, we have another choice:

1. Move our scene with respect to the camera using affine transformations. This is possible but annoying and unintuitive.
2. Place our camera using:

   glMatrixMode(GL_MODELVIEW);
   glLoadIdentity();
   gluLookAt(eyex,eyey,eyez, atx,aty,atz, upx,upy,upz);
   

• The "eye" point is the location of the focal point (also known as the "center of projection" or COP), as a point in space. Yet another standard term for this is the VRP: view reference point.
• The "at" point is the location of some point in the direction we want the camera to face. It doesn't even have to be in the view volume. It's only used to figure out where the camera is pointed. A standard term for the direction the camera is pointed is the VPN: view plane normal. (Here's our first encounter with a vector in OpenGL.)
• The "up" vector says which direction, projected onto the image plane, is the same as the vertical direction on the monitor. Note that it is a vector, not a point. A standard term for this is "VUP."

The VRP, VPN, VUP API

The OpenGL API is almost the same as another API for camera positioning that is standard in the field, namely:

• VRP: View Reference Point, the location of the view plane
• VPN: View Plane Normal, the orientation of the view plane
• VUP: View Up, the rotation of the view plane

Other Terminology

The following terms are commonly used for the kinds of motions for cameras and such.

• pan: rotating a fixed camera around a vertical axis
• tilt: rotating a fixed camera around a horizontal axis
• zoom: adjusting the lens to zoom in or out. (This adjusts the frustum.)
• roll: rotating a camera or a ship around a longitudinal axis
• pitch: same as tilt, but for ships and airplanes
• yaw: same as pan, but for ships and airplanes
• strafe: moving a camera along a horizontal axis; this terminology is used in video games I believe.

Camera Example

The demos/camera/BarnViews.py demo shows three different views of the barn, using the typical API for camera setup, and eschewing twCamera(). The key here is to look at the code for how to see the barn views are set up. Use the following keyboard callbacks:

• X to view from the positive X direction
• Y to view from the positive Y direction
• Z to view from the positive Z direction
• G to view from the diagonal of the bounding box, with random fov

Tutors

Nate Robbins (not me) has built a tutor for playing with these camera APIs. It is in a subdirectory of tw, namely:

~cs307/public_html/tw/tutors

twCamera

If we have time, we can talk about how the twCamera setup works. There are two modes:

• The "outside" viewpoint. This is the default. It figures out the radius of a bounding sphere and then sets up a frustum with a 90 degree FOVY that just fits the sphere. The aspect ratio matches that of the window, so there's no distortion. It sets up the COP so that the frustum is as close as possible to the scene but where the bounding sphere is tangent to the walls of the frustum. The COP is directly +z from the center of the bounding box. Finally, "near" and "far" are set up so that the top and bottom of the frustum are tangent to the bounding sphere.
• The "immerse" viewpoint. This is the same shape frustum as in the "outside" view, but it puts the COP at the center of the bounding box. "Far" is large enough so that nothing is cut off, and "Near" is small enough so that, hopefully, very little is cut off.

You can dump out a lot of information about the TW camera setup by doing the following:

twSetMessages(TW_ALL_MESSAGES);

Note that you may not use the twCamera() setup functions in your next assignment, but that's the only time this is forbidden. You may always use the TW functions when they are helpful and convenient.

Note also that I've never tried mixing the twCamera with plain OpenGL camera API, so if you decide to try it, let us know what you learn!

Forced Perspective

As long as we're setting up a camera, let's look at how to play tricks with it. In particular, we'll look at how forced perspective works. If we have time, we can look at the Lord of the Rings DVD that explains how they did some of this. Very cool.

Written by Scott D. Anderson
scott.anderson@acm.org

This work is licensed under a Creative Commons License.