Daniel Haehn PRO
Hi, I am a biomedical imaging and visualization researcher who investigates how computational methods can accelerate biological and medical research.
Assignment 6
Due 10 / 28
Lecture 1
GPU Access!
Lecture 2
<canvas></canvas>
var c = document.createElement("canvas");
document.body.appendChild(c);
*.html
*.js
Lecture 3
Browser
Editor
Lecture 4
Web Developer Tools
Spector.js is a Web Developer Tool for WebGL
Lecture 5
Frustum
zNear
zFar
Viewport
Camera
(Eye)
Perspective Projection
Camera
Eye
Position (0,0,100)
Focus (0,0,0)
Scene
Center
Up (0,1,0)
Translate (Move in x,y)
Zoom (Move in z)
Rotate
X
Y
Z
c = new X.cube();
c2 = new X.cube();
c.center = [0, 0, 0];
c.lengthX
c.lengthY
c.lengthZ
gap
= 20
c2.center = [25, 0, 0];
= 5
c.color = [0, 0, 1];
c2.color = [1, 1, 1];
25, 0, 0
0, 0, 0
Z-Fighting
Lecture 6
easy
hard
XTK
Three.js
WebGL
limited
freedom
Complexity
Functionality
XTK
X.renderer3D
Renderer
Scene
Interaction
Loop
Three.js
THREE.WebGLRenderer
THREE.Scene
THREE.TrackballControls
Loop
Canvas
Canvas
Camera
THREE.PerspectiveCamera
Lighting
THREE.AmbientLight
THREE.DirectionalLight
ambientlight = new THREE.AmbientLight()
geometry = new THREE.BoxBufferGeometry
( 20, 20, 20 )
lengthX, lengthY, lengthZ
Geometry defines the shape!
material = new THREE.MeshStandardMaterial
( { color: 0xffffff } )
Material defines the appearance!
new THREE.Mesh( geometry, material )
cube =
Lecture 7
Wireframe
12 Triangles
V1
V2
V3
V4
V6
V5
(x, y, z)
(x, y, z)
(x, y, z)
(x, y, z)
(x, y, z)
(x, y, z)
Vertex
/ Vertices
Face
Face
Frame Buffer
Screen Space
width x height Pixels
Rasterization
var geometry = new THREE.Geometry();
geometry.vertices.push(
new THREE.Vector3(-10, 10, 0),
new THREE.Vector3(-10, -10, 0),
new THREE.Vector3(10, -10, 0)
);
geometry.faces.push( new THREE.Face3(0, 1, 2));
GPU
Vertex Shader
Fragment Shader
Viewport
From 3D..
..to 2D
Lecture 8
0, 0
width, height
renderer.domElement.onclick = function(e) {
pixel_coords = new THREE.Vector2(e.clientX, e.clientY);
};400, 200
400 , 200
Screen Space Coordinates
-1, -1
renderer.domElement.onclick = function(e) {
pixel_coords = new THREE.Vector2( e.clientX, e.clientY );
vp_coords = new THREE.Vector2(
( pixel_coords.x / window.innerWidth ) * 2 - 1, // X
- ( pixel_coords.y / window.innerHeight ) * 2 + 1 ); // Y
};-0.2, 0.3
-0.2
Viewport Coordinates
1, 1
0, 0
0.3
THREE.Raycaster
zNear
zFar
Ray
Position (x,y,z)
needs 2 points to define Ray
raycaster = new THREE.Raycaster();
raycaster.setFromCamera(vp_coords_near, camera);Fragment Shader
Vertex Shader
gl_Position
gl_FragColor
for every vertex
for every pixel
Lecture 9
XTK
X.renderer3D
X.cube
Three.js
THREE.WebGLRenderer
THREE.Scene
THREE.TrackballControls
THREE.PerspectiveCamera
THREE.AmbientLight
THREE.DirectionalLight
WebGL
gl.viewport
gl.createShader
gl.shaderSource
gl.compileShader
gl.getShaderInfoLog
gl.createProgram
gl.attachShader
gl.linkProgram
gl.useProgram
gl.createBuffer
gl.bindBuffer
gl.BufferData
gl.getAttribLocation
gl.vertexAttribPointer
gl.enableVertexAttribArray
gl.clearColor
gl.clear
gl.drawArrays
THREE.Geometry
THREE.Material
THREE.Mesh
V0
V1
V2
V3
V4
V5
Viewport Coordinates
-1, -1
1, 1
-0.5, -0.5
-0.5, 0.5
0.5, 0.5
0.5, -0.5
vertices = new Float32Array( [
-0.5, 0.5, 0.0, // V0
-0.5, -0.5, 0.0, // V1
0.5, 0.5, 0.0, // V2
0.5, 0.5, 0.0, // V3
-0.5, -0.5, 0.0, // V4
0.5, -0.5, 0.0 // V5
] );1. Initialize WebGL
2. Shaders
3. Create Geometry
4. Connect Shader with Geometry
5. Draw!
setup Canvas
setup GL Context
compile vertex shader
compile fragment shader
attach and link shaders
create vertices
create and bind buffer
put data in
unbind buffer
bind buffer
find vertex attribute in shader source
configure vertex attribute
enable vertex attribute array
clear viewport
clear color buffer
draw vertex arrays
Lecture 10
V0
V1
V2
V3
V4
V5
vertices = new Float32Array( [
-0.5, 0.5, 0.0, // V0
-0.5, -0.5, 0.0, // V1
0.5, 0.5, 0.0, // V2
0.5, 0.5, 0.0, // V3
-0.5, -0.5, 0.0, // V4
0.5, -0.5, 0.0 // V5
] );gl.TRIANGLES
V0
V1
V2
V3
V4
V5
vertices = new Float32Array( [
-0.5, 0.5, 0.0, // V0
-0.5, -0.5, 0.0, // V1
0.5, 0.5, 0.0, // V2
0.5, 0.5, 0.0, // V3
-0.5, -0.5, 0.0, // V4
0.5, -0.5, 0.0 // V5
] );gl.POINTS
V0
V1
V2
V3
V4
V5
vertices = new Float32Array( [
-0.5, 0.5, 0.0, // V0
-0.5, -0.5, 0.0, // V1
0.5, 0.5, 0.0, // V2
0.5, 0.5, 0.0, // V3
-0.5, -0.5, 0.0, // V4
0.5, -0.5, 0.0 // V5
] );gl.LINES
Rendering Primitives
V0
V1
V2
V3
V4
V5
vertices = new Float32Array( [
-0.5, 0.5, 0.0, // V0 // 0
-0.5, -0.5, 0.0, // V1, V4 // 1
0.5, 0.5, 0.0, // V2, V3 // 2
0.5, -0.5, 0.0 // V5 // 3
] );var indices = new Uint8Array( [ 0, 1, 2, // Triangle 1
2, 1, 3 ] ); // Triangle 20
1
2
3
Indexed Geometry
Before: 6 x 32 bits, Now: 4 x 32 bits
6 x 8 bits == 48 bits
We still save 16 bits.
We save 2 x 32 bits.
Lecture 11
Uniforms
createRectangle()
Rendering Loop
And, they move!
Fragment Shader
Vertex Shader
attribute vec3 position;
different data for each vertex
uniform vec3 offset;
different data for each gl.drawArray or gl.drawElements call
uniform vec4 color;
Lecture 12
Lecture 13
Transformations
Translate
Rotate
Scale
Matrix
1 0 0 t_x
0 1 0 t_y
0 0 1 t_z
0 0 0 1
x, y, z
x, y, z
new
Translation
Matrix
s_x 0 0 t_x
0 s_y 0 t_y
0 0 s_z t_z
0 0 0 1
x, y, z
x, y, z
new
Scaling
Matrix
r r r t_x
r r r t_y
r r r t_z
0 0 0 1
x, y, z
x, y, z
new
Rotation
var m = new Float32Array( [
r, r, r, 0,
r, r, r, 0,
r, r, r, 0,
t_x, t_y, t_z, 1
]);column-major ordering!
Lecture 14
0.5 0 0 0
0 0.5 0 0
0 0 0.5 0
0 0 0 1
2, 2, 2
?
cos(T) -sin(T) 0 0
sin(T) cos(T) 0 0
0 0 1 0
0 0 0 1
2, 2, 2
?
with T = Pi/2
= 90°
X
Y
Z
Scene
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
Scene Graph is a Tree-like Hierarchy!
*
*
*
Lecture 15
cos(T) -sin(T) 0 0
sin(T) cos(T) 0 0
0 0 1 0
0 0 0 1
2, 2, 2
with T = Pi/2
0
0
1
2 Pi
Pi
Pi/2
1.5 Pi
90°
1
1
-2
2
2
-1
Frame of Reference
X
Y
Z
World Frame
Frame of Reference
X
Y
Z
Object Frame
Frame of Reference
X
Y
Z
Eye Frame
Camera
(Eye)
cos(T) 0 sin(T) 0
0 1 0 0
-sin(T) 0 cos(T) 0
0 0 0 1
rotation around Y
cos(T/2)
sin(T/2) * 0
sin(T/2) * 1
sin(T/2) * 0
Matrix
Quaternion
Lecture 16
cos(T) -sin(T) 0 0
sin(T) cos(T) 0 0
0 0 1 0
0 0 0 1
2, 2, 2
with T = Pi/2
0
0
1
2 Pi
Pi
Pi/2
1.5 Pi
-1
-2, 2, 2
cos(T/2)
sin(T/2) * 0
sin(T/2) * 0
sin(T/2) * 1
Quaternion
w
x
y
z
~0.7
~0.7
0
0
~0.7
0
0
~0.7
0
2
2
2
w
x
y
z
*
Quaternion
?
~0.7
0
0
~0.7
0
2
2
2
*
~0.7
- 0
- 0
- ~0.7
Inverse
page 64, Gortler: 3D Computer Graphics
Rigid Body Transformation
Quaternion
Translation Vector
w
x
y
z
x
y
z
0
Arcball
Trackball
Controls
Natural Feeling
Spins twice as fast
Completely path independent
Lecture 17
var q = new THREE.Quaternion( sin( T / 2 ) * x ,
sin( T / 2 ) * y ,
sin( T / 2 ) * z ,
cos( T / 2 ) );0
0
1
2 Pi
Pi
Pi/2
1.5 Pi
-1
90°
180°
270°
360°
180°
around Y-axis
T is the angle
x y z is the axis
T = Math.PI
0, 1, 0
var q = new THREE.Quaternion( Math.sin( Math.PI / 2 ) * 0 ,
Math.sin( Math.PI / 2 ) * 1 ,
Math.sin( Math.PI / 2 ) * 0 ,
Math.cos( Math.PI / 2 ) );zNear
Ray
Position (x,y,z)
Invisible Plane
raycaster = THREE.Raycaster();
raycaster.setFromCamera(vp_coords_near, camera);
intersects = raycaster.intersectObject( invisiblePlane );Raycasting
Framebuffer (hidden)
1
2
3
4
( 1, 0, 0, 1 )
( 0, 1, 0, 1 )
( 1, 1, 0, 1 )
( 0, 0, 1, 1 )
( 0, 0, 0, 1 )
gl.readPixels
(r, g, b, a)
map to object ID
( 0, 1, 0, 1 )
2
Object Picking
Object Picking
Raycasting
More complex
Quick'n'Dirty
Very precise (Face picking)
Object precision
Occlusion not a problem
Objects can be hidden
Lecture 18
Lecture 19
r g b a
alpha
0: fully transparent
....
255: fully opaque
0: minimum
....
255: maximum
red
green
blue
#000000
#ffffff
minimum
maximum
Hex
#00000000
#ffffffff
minimum
maximum
Hex8
alpha
alpha
HSL
H:
S:
L:
Hue
Saturation
Lightness
0/360°
180°
240°
65%
40%
0%
100%
0%
100%
hsl(240, 65%, 40%)
rgb(36, 36, 168)
#2424a8
Gamut
range of colors in an imaging system
Lecture 20
Lecture 20
By Daniel Haehn
Lecture 20
Slides for CS460 Computer Graphics at UMass Boston. See https://cs460.org!
