Assignment 5

IEEE Visualization

starting Sunday at IEEEVis.org

Lecture 1

GPU Access!

http://madebyevan.com/webgl-water/

https://paveldogreat.github.io/WebGL-Fluid-Simulation/

Lecture 2

https://skilled.co/html-canvas/

var c = document.createElement("canvas");

document.body.appendChild(c);

*.html

*.js

https://goXTK.com

https://threejs.org

Lecture 3

https://indeed.com

Browser

Editor

Lecture 4

Web Developer Tools

Spector.js is a Web Developer Tool for WebGL

https://spector.babylonjs.com/

Lecture 5

Frustum

zNear

zFar

Viewport

Camera

(Eye)

Perspective Projection

Camera

Eye

Position (0,0,100)

Focus (0,0,0)

Scene

Center

Up (0,1,0)

Translate (Move in x,y)

Zoom (Move in z)

Rotate

c = new X.cube();

c2 = new X.cube();

c.center = [0, 0, 0];

c.lengthX

c.lengthY

c.lengthZ

gap

= 20

c2.center = [25, 0, 0];

= 5

c.color = [0, 0, 1];

c2.color = [1, 1, 1];

25, 0, 0

0, 0, 0

Z-Fighting

Lecture 6

easy

hard

XTK

Three.js

WebGL

limited

freedom

Complexity

Functionality

XTK

X.renderer3D

Renderer

Scene

Interaction

Loop

Three.js

THREE.WebGLRenderer

THREE.Scene

THREE.TrackballControls

Loop

Canvas

Canvas

Camera

THREE.PerspectiveCamera

Lighting

THREE.AmbientLight

THREE.DirectionalLight

ambientlight = new THREE.AmbientLight()

geometry = new THREE.BoxBufferGeometry

( 20, 20, 20 )

lengthX, lengthY, lengthZ

Geometry defines the shape!

material = new THREE.MeshStandardMaterial

( { color: 0xffffff } )

Material defines the appearance!

new THREE.Mesh( geometry, material )

cube =

Lecture 7

Wireframe

12 Triangles

(x, y, z)

Vertex

/ Vertices

Face

Frame Buffer

Screen Space

width x height Pixels

Rasterization

var geometry = new THREE.Geometry();
geometry.vertices.push(
	new THREE.Vector3(-10, 10, 0),
	new THREE.Vector3(-10, -10, 0),
	new THREE.Vector3(10, -10, 0)
);
geometry.faces.push( new THREE.Face3(0, 1, 2));

GPU

Vertex Shader

Fragment Shader

Viewport

From 3D..

..to 2D

Lecture 8

https://cs460.org/showcase/02/overview2.html

https://cs460.org/showcase/02/overview.html

0, 0

width, height

renderer.domElement.onclick = function(e) {
  
  pixel_coords = new THREE.Vector2(e.clientX, e.clientY);
  
};

400, 200

400 , 200

Screen Space Coordinates

-1, -1

renderer.domElement.onclick = function(e) {
  
  pixel_coords = new THREE.Vector2( e.clientX, e.clientY );
  vp_coords = new THREE.Vector2( 
    ( pixel_coords.x / window.innerWidth ) * 2 - 1, // X
    - ( pixel_coords.y / window.innerHeight ) * 2 + 1 ); // Y
};

-0.2, 0.3

-0.2

Viewport Coordinates

1, 1

0, 0

0.3

THREE.Raycaster

zNear

zFar

Ray

Position (x,y,z)

needs 2 points to define Ray

raycaster = new THREE.Raycaster();
raycaster.setFromCamera(vp_coords_near, camera);

Fragment Shader

Vertex Shader

gl_Position

gl_FragColor

for every vertex

for every pixel

Lecture 9

https://s2021.siggraph.org/

XTK

X.renderer3D

X.cube

Three.js

THREE.WebGLRenderer

THREE.Scene

THREE.TrackballControls

THREE.PerspectiveCamera

THREE.AmbientLight

THREE.DirectionalLight

WebGL

gl.viewport

gl.createShader

gl.shaderSource

gl.compileShader

gl.getShaderInfoLog

gl.createProgram

gl.attachShader

gl.linkProgram

gl.useProgram

gl.createBuffer

gl.bindBuffer

gl.BufferData

gl.getAttribLocation

gl.vertexAttribPointer

gl.enableVertexAttribArray

gl.clearColor

gl.clear

gl.drawArrays

THREE.Geometry

THREE.Material

THREE.Mesh

https://www.khronos.org/files/webgl/webgl-reference-card-1_0.pdf

Viewport Coordinates

-1, -1

1, 1

-0.5, -0.5

-0.5, 0.5

0.5, 0.5

0.5, -0.5

      vertices = new Float32Array( [
                                     -0.5,  0.5, 0.0, // V0
                                     -0.5, -0.5, 0.0, // V1
                                      0.5,  0.5, 0.0, // V2

                                      0.5,  0.5, 0.0, // V3
                                     -0.5, -0.5, 0.0, // V4
                                      0.5, -0.5, 0.0  // V5
                                    ] );

1. Initialize WebGL

2. Shaders

3. Create Geometry

4. Connect Shader with Geometry

5. Draw!

setup Canvas

setup GL Context

compile vertex shader

compile fragment shader

attach and link shaders

create vertices

create and bind buffer

put data in

unbind buffer

bind buffer

find vertex attribute in shader source

configure vertex attribute

enable vertex attribute array

clear viewport

clear color buffer

draw vertex arrays

Lecture 10

      vertices = new Float32Array( [
                                     -0.5,  0.5, 0.0, // V0
                                     -0.5, -0.5, 0.0, // V1
                                      0.5,  0.5, 0.0, // V2

                                      0.5,  0.5, 0.0, // V3
                                     -0.5, -0.5, 0.0, // V4
                                      0.5, -0.5, 0.0  // V5
                                    ] );

gl.TRIANGLES

      vertices = new Float32Array( [
                                     -0.5,  0.5, 0.0, // V0
        
                                     -0.5, -0.5, 0.0, // V1
        
                                      0.5,  0.5, 0.0, // V2

                                      0.5,  0.5, 0.0, // V3
        
                                     -0.5, -0.5, 0.0, // V4
        
                                      0.5, -0.5, 0.0  // V5
                                    ] );

gl.POINTS

      vertices = new Float32Array( [
                                     -0.5,  0.5, 0.0, // V0
                                     -0.5, -0.5, 0.0, // V1
        
                                      0.5,  0.5, 0.0, // V2
                                      0.5,  0.5, 0.0, // V3
        
                                     -0.5, -0.5, 0.0, // V4
                                      0.5, -0.5, 0.0  // V5
                                    ] );

gl.LINES

Rendering Primitives

http://antongerdelan.net/opengl/

      vertices = new Float32Array( [
                                     -0.5,  0.5, 0.0, // V0      // 0
                                     -0.5, -0.5, 0.0, // V1, V4  // 1
                                      0.5,  0.5, 0.0, // V2, V3  // 2
                                      0.5, -0.5, 0.0  // V5      // 3
                                    ] );

var indices = new Uint8Array( [ 0, 1, 2,     // Triangle 1 
                                2, 1, 3 ] ); // Triangle 2

Indexed Geometry

Before: 6 x 32 bits, Now: 4 x 32 bits

6 x 8 bits == 48 bits

We still save 16 bits.

We save 2 x 32 bits.

Lecture 11

Uniforms

createRectangle()

Rendering Loop

And, they move!

Fragment Shader

Vertex Shader

attribute vec3 position;

different data for each vertex

uniform vec3 offset;

different data for each gl.drawArray or gl.drawElements call

uniform vec4 color;

Lecture 12

1. Initialize WebGL

2. Shaders

3. Create Geometry

4. Connect Shader with Geometry

5. Draw!

1 Frame

multiple Frames

animate()

1 Rectangle

multiple Rectangles

createRectangle()

change Offsets!

multiple Rectangles

[ [ v_buffer, i_buffer, color, offset ], [ v_buffer, i_buffer, color, offset ] ]

objects =

function animate() {
  
  for(var o = 0; o < objects.length; o++) {
  
    var current_object = objects[o];
    
    var current_v_buffer = current_object[0];
    var current_i_buffer = current_object[1];
    var current_color = current_object[2];
    var current_offset = current_object[3];
    
    // pass data to shader
    
    gl.drawElements( gl.TRIANGLES ... )
    
  }
  
  requestAnimationFrame( animate );
  
}

Transformation

Lecture 13

Transformations

Translate

Rotate

Scale

Matrix

1 0 0 t_x

0 1 0 t_y

0 0 1 t_z

0 0 0 1

x, y, z

new

Translation

Matrix

s_x 0 0 t_x

0 s_y 0 t_y

0 0 s_z t_z

0 0 0 1

x, y, z

new

Scaling

Matrix

r r r t_x

r r r t_y

r r r t_z

0 0 0 1

x, y, z

new

Rotation

var m = new Float32Array( [
  r, r, r, 0, 
  r, r, r, 0, 
  r, r, r, 0, 
  t_x, t_y, t_z, 1
]);

column-major ordering!

Lecture 14

0.5 0 0 0

0 0.5 0 0

0 0 0.5 0

0 0 0 1

2, 2, 2

cos(T) -sin(T) 0 0

sin(T) cos(T) 0 0

0 0 1 0

0 0 0 1

2, 2, 2

with T = Pi/2

= 90°

Scene

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

Scene Graph is a Tree-like Hierarchy!

Lecture 15

cos(T) -sin(T) 0 0

sin(T) cos(T) 0 0

0 0 1 0

0 0 0 1

2, 2, 2

with T = Pi/2

2 Pi

Pi/2

1.5 Pi

90°

-2

-1

https://workshop.chromeexperiments.com/examples/gui/

Frame of Reference

World Frame

Frame of Reference

Object Frame

Frame of Reference

Eye Frame

Camera

(Eye)

cos(T) 0 sin(T) 0

0 1 0 0

-sin(T) 0 cos(T) 0

0 0 0 1

rotation around Y

cos(T/2)

sin(T/2) * 0

sin(T/2) * 1

sin(T/2) * 0

Matrix

Quaternion

Lecture 16

https://ayahea.github.io/cs460student/04/

cos(T) -sin(T) 0 0

sin(T) cos(T) 0 0

0 0 1 0

0 0 0 1

2, 2, 2

with T = Pi/2

2 Pi

Pi/2

1.5 Pi

-1

-2, 2, 2

cos(T/2)

sin(T/2) * 0

sin(T/2) * 1

Quaternion

~0.7

Quaternion

~0.7

- 0

- ~0.7

Inverse

page 64, Gortler: 3D Computer Graphics

Rigid Body Transformation

Quaternion

Translation Vector

Arcball

Trackball

Controls

http://3dvision.princeton.edu/courses/COS598/2014sp/slides/lecture15_webgl/WebGLPresentation.pdf

Natural Feeling

Spins twice as fast

Completely path independent

Lecture 17

var q = new THREE.Quaternion( sin( T / 2 ) * x , 
                              sin( T / 2 ) * y , 
                              sin( T / 2 ) * z , 
                              cos( T / 2 ) );

2 Pi

Pi/2

1.5 Pi

-1

90°

180°

270°

360°

180°

around Y-axis

T is the angle

x y z is the axis

T = Math.PI

0, 1, 0

var q = new THREE.Quaternion( Math.sin( Math.PI / 2 ) * 0 , 
                              Math.sin( Math.PI / 2 ) * 1 , 
                              Math.sin( Math.PI / 2 ) * 0 , 
                              Math.cos( Math.PI / 2 ) );

zNear

Ray

Position (x,y,z)

Invisible Plane

raycaster = THREE.Raycaster();
raycaster.setFromCamera(vp_coords_near, camera);
intersects = raycaster.intersectObject( invisiblePlane );

Raycasting

https://www.lighthouse3d.com/tutorials/opengl-selection-tutorial/

Framebuffer (hidden)

( 1, 0, 0, 1 )

( 0, 1, 0, 1 )

( 1, 1, 0, 1 )

( 0, 0, 1, 1 )

( 0, 0, 0, 1 )

gl.readPixels

(r, g, b, a)

map to object ID

( 0, 1, 0, 1 )

Object Picking

Raycasting

More complex

Quick'n'Dirty

Very precise (Face picking)

Object precision

Occlusion not a problem

Objects can be hidden

https://threejs.org/examples/webgl_decals.html

Lecture 18

r g b a

alpha

0: fully transparent

....

255: fully opaque

0: minimum

....

255: maximum

red

green

blue

#000000

#ffffff

minimum

maximum

Hex

#00000000

#ffffffff

minimum

maximum

Hex8

alpha

HSL

Hue

Saturation

Lightness

0/360°

180°

240°

65%

40%

100%

hsl(240, 65%, 40%)

rgb(36, 36, 168)

#2424a8

https://www.w3schools.com/colors/colors_hsl.asp

Gamut

range of colors in an imaging system

RateMyTalk.com/3WKSYF

Lecture 19

By Daniel Haehn

Lecture 19

Slides for CS460 Computer Graphics at UMass Boston. See https://cs460.org!

1,020

Daniel Haehn PRO

Hi, I am a biomedical imaging and visualization researcher who investigates how the study of brain connectivity and machine perception can help advance the understanding of biologically inspired artificial intelligence.

Lecture 19

More from Daniel Haehn