CS307: User Interaction (Part 2)

Plan

• Review of mouse movement, click and drag
• Creative Scene/project. Please enter ideas here: CS 307 Spring 2023 Projects
• Quiz questions
• Exercise: drawing with the mouse
• Review of picking, normalized device coordinates,
             unprojection, raycasting
• Explore: zapping blocks
• Exercise: the Midas touch

Mouse Movement, Click and Drag

To implement click and drag, we need to handle three more mouse events:

document.addEventListener('mousemove', onMouseMove);
document.addEventListener('mousedown', onMouseDown);
document.addEventListener('mouseup', onMouseUp);

A dragging action combines mousedown and mousemove:

var isMouseDown = false;

function onMouseDown (event) {
    isMouseDown = true;
}
    
function onMouseUp (event) {
    isMouseDown = false;
}

function onMouseMove (event) {
    if (isMouseDown) {
        console.log("drag to (" + event.clientX + "," + event.clientY + ")";
    }
}    

We can also keep track of the mouse's previous and current locations as the mouse moves:

var oldMouse = {x:0, y:0};

function onMouseMove (event) {
    if (isMouseDown) {
        console.log("drag to (" + event.clientX + "," + event.clientY + ")"
                    + "from (" + oldMouse.x + "," + oldMouse.y + ")");
        oldMouse.x = event.clientX;
        oldMouse.y = event.clientY;
    }
}

How might you determine which direction the mouse moved?

We'll explore the detection of click-and-drag actions in this demo:

watching.html

Quiz Questions

I'll answer your questions

Picking, Normalized Device Coordinates, Unprojection, and Raycasting

Suppose we want to select or pick a particular element in a rendered scene, such as a component object, using a mouse click. We saw earlier that

• the location of the mouse is reported in window coordinates, with the origin at the upper left corner of the window
• we can subtract the offset of the target (the canvas) to get the mouse location in canvas coordinates

But how can we relate these mouse coordinates to the location of objects in the world coordinate frame of the scene?

• In the rendering process, a 3D scene is first projected to normalized device coordinates (NDC) with $x,y \in [-1,+1]$ and $y$ increasing up
• NDC has $z \in [0, 1]$ with $0$ corresponding to the near plane and $1$ corresponding to the far plane

  

• As we did in the earlier drawing exercise, the mouse coordinates can also be normalized to $x,y \in [-1,+1]$ with $y$ increasing up

  

Given the scene, camera, and normalized mouse coordinates, the THREE.Raycaster class can be used reverse the projection process, or unproject a location, to determine the 3D scene objects that project to this location

The following demo illustrates this idea. The scene is a collection of randomly positioned blocks. Clicking the mouse on a block, while holding down the shift key, removes the block from the scene.

blocks.html

The following exercise applies these ideas to turn the objects in a town to gold!