Graphics Processing Units (GPUs)

Parallel Algorithms

David Mayerich

Scalable Tissue Imaging and Modeling (STIM) Laboratory

Department of Electrical and Computer Engineering

Cullen College of Engineering

University of Houston

David Mayerich

STIM Laboratory, University of Houston

Vector Graphics

  • polygon: a series of line segments forming a closed planar shape

  • vertex: a point joining two line segments

  • transform: 2D or 3D linear operation applied to a series of vertices

David Mayerich

STIM Laboratory, University of Houston

\mathbf{R}= \begin{bmatrix} \cos{\theta} & -\sin{\theta}\\ \sin{\theta} & \cos{\theta} \end{bmatrix}
\mathbf{v}_1'= \mathbf{R}\mathbf{v}_1
\mathbf{v}_2'= \mathbf{R}\mathbf{v}_2
\mathbf{v}_3'= \mathbf{R}\mathbf{v}_3
\mathbf{v}_4'= \mathbf{R}\mathbf{v}_4
\mathbf{v}_5'= \mathbf{R}\mathbf{v}_5 = \begin{bmatrix} x_5'\\ y_5' \end{bmatrix} = \begin{bmatrix} \cos{\theta} & -\sin{\theta}\\ \sin{\theta} & \cos{\theta} \end{bmatrix} \begin{bmatrix} x_5\\ y_5 \end{bmatrix}
\mathbf{v}_6'= \mathbf{R}\mathbf{v}_6

Projections

  1. Describe the world as a series of vertices connected by lines​
  2. Create a virtual camera​
  3. Rotate all of the vertices to line the camera up with its position in the virtual world​
  4. Apply a projection to render the lines to a 2D plane (the screen)

David Mayerich

STIM Laboratory, University of Houston

Projections

David Mayerich

STIM Laboratory, University of Houston

  • orthographic projection: directly eliminate the z-axis component after rotating to the camera

\mathbf{P}= \begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 0 \end{bmatrix}
\dot{\mathbf{v}}_1=\mathbf{P}\mathbf{v}_1'
\dot{\mathbf{v}}_2=\mathbf{P}\mathbf{v}_2'
\dot{\mathbf{v}}_3=\mathbf{P}\mathbf{v}_3'
\dot{\mathbf{v}}_{n-1}=\mathbf{P}\mathbf{v}_{n-1}'
\dot{\mathbf{v}}_{n}=\mathbf{P}\mathbf{v}_{n}'
\cdots

  • perspective projection: scale the \((x,y)\) coordinates as a function of \(z\) (distance from the camera)

    • points must be represented as a 4D vector

\mathbf{P}= \begin{bmatrix} 1 & 0 & 0 & 0\\ 0 & 1 & 0 & 0 \\ 0 & 0 & -1 & 0\\ 0 & 0 & D & 0 \end{bmatrix}
\text{where } \mathbf{v}_n' = \begin{bmatrix} x_n'\\ y_n'\\ z_n'\\ w_n' \end{bmatrix}

Vector Graphics

David Mayerich

STIM Laboratory, University of Houston

// calculate view (V) and projection (P)

// allocate endpoints for lines
point p[2];

// for every polygon in the scene
for(int pi = 0; pi < P; pi++) {
  // for every vertex in the polygon
  for(int vi = 0; vi < Polys[pi].V; vi++) {
  	// calculate the screen point
    vertex p[1] = P * V * Polys[pi][vi];
    // if this is not the first poly point
    if(vi != 0)
      // draw a line between the points
      drawLine(p[0], p[1]);
    // shift points to draw the next line
    p[0] = p[1];
  }
  // connect the polygon
  drawLine(p[0], Polys[pi][0]);
}

Star Wars (Atari 1983), arcade-museum.com​

most of the work:

\(O(P \times V)\)

all \(P\) polygons

all \(V\) vertices

Shading (1980s)

  • Calculate a color value \(c=[c_r\, c_g\, c_b]\) for each point inside the polygon

  • Scan-line algorithm to fill pixels

  • Calculate illumination:

     

    • \(\mathbf{n}\) is the surface normal

    • \(\mathbf{l}\) is the direction to the light source

David Mayerich

STIM Laboratory, University of Houston

\dot{\mathbf{c}}=\mathbf{c}(\mathbf{n}\cdot\mathbf{l})
\theta = \mathbf{n}\cdot \mathbf{l}

Shading

David Mayerich

STIM Laboratory, University of Houston

// calculate view (V) and projection (P)
// stores vertices for the current triangle
vertex* t = new vertex[3];
// for every triangle
for(int ti = 0; ti < T; ti++) {
  // transform to screen coordinates
  t[0] = P * V * Mesh[ti][0];
  t[1] = P * V * Mesh[ti][1];
  t[2] = P * V * Mesh[ti][2];
  // calculate the normal
  vector n = normal(t[0], t[1], t[2]);
  // initialize light intensity to zero
  color l_sum = 0;
  // for each light in the scene
  for(int li=0; li<L; li++)
    // add its intensity
    l_sum += cos(n, Lights[li].p);
  // calculate the final color
  color c = Mesh[ti].color * l_sum;
  // draw all pixels in the triangle
  drawPixels(t, c);
}

I, Robot (Atari 1983), arcade-museum.com

all \(P\) pixels

all \(T\) triangles

all \(L\) lights

Texture Mapping (1990s)

  • texture mapping: map a 2D image onto the polygons of a 3D object

David Mayerich

STIM Laboratory, University of Houston

  • provides additional color and shading details without extra vertices

Voodoo2, Quake (1996)

GeForce2 MX, Creature (2001)

GeForce2 Go, Toy Soldier (2002)

GeForce2 Ti, Grove (2000)

Texture Mapping

David Mayerich

STIM Laboratory, University of Houston

// calculate view (V) and projection (P)
// stores vertices for the current triangle
vertex* t = new vertex[3];
// for every triangle
for(int ti = 0; ti < T; ti++) {
  // transform to screen coordinates
  t[0] = P * V * Mesh[ti][0];
  t[1] = P * V * Mesh[ti][1];
  t[2] = P * V * Mesh[ti][2];
  // calculate the normal
  vector n = normal(t[0], t[1], t[2]);
  // initialize light intensity to zero
  color l_sum = 0;
  // for each light in the scene
  for(int li=0; li<L; li++)
    // add its intensity
    l_sum += cos(n, Lights[li].p);
  
  // draw all pixels in the triangle
  drawPixels(t, Mesh[ti].texture, l_sum);
}

fragments

vertices

Darkened Skye, Boston Animation (2002)

Programmable Graphics Hardware

David Mayerich

STIM Laboratory, University of Houston

1998

1999

2000

2001

2002

2003

2004

DirectX 6
Microsoft
multitexture

DirectX 7
Microsoft

DirectX 8
Microsoft
shader model 1.x

DirectX 9
Microsoft
shader model 2.0 - 3.0

OpenGL
Silicon Graphics

(1998) Ocarina of Time

(1999) System Shock

(2000) Baulder's Gate II

(2002) Warcraft III

(2004) Fable

Geforce 256
nVidia
hardware texture
and lighting

Geforce 2
nVidia

Geforce 3
nVidia
programmable shading
(XBox)

Cg
nVidia
shader language

Radeon 9700
Radeon
floating point math

GeForce FX
nVidia
practical GPGPU

GLSL
shading
language

Voodoo
3dfx Interactive
texture mapping

Cg (C for Graphics)

  • GeForce 3 - first programmable shader model (using assembly)

  • GeForce FX - first practical card for GPGPU

    • floating point precision

    • Cg provides a high-level language
      for programming GPUs

David Mayerich

STIM Laboratory, University of Houston

Output varying(
  float2 position: POSITION,
  float4 color : COLOR,
  float2 texCoord : TEXCOORD0) {
    Output OUT;
    OUT.position = float4(position, 0, 1);
    OUT.color 	 = color;
    OUT.texCoord = texCoord;
    return OUT;
}

GeForce 3 Ti, Chameleon (2001)

GeForce FX 5200, Dawn (2003)

GPGPU - challenges and investments

General purpose GPU programming mapped mathematical problems to rendering

  • Calculate the matrix multiplication \(\mathbf{C} = \mathbf{A}\mathbf{B}\):
    • Store \(\mathbf{A}\) and \(\mathbf{B}\) as texture maps
    • Draw a rectangle on the screen that is exactly the size of \(\mathbf{C}\) in pixels
    • Write a pixel shader that multiplies the rows of \(\mathbf{A}\) with the columns of \(\mathbf{B}\)
    • Read the results from the screen buffer
       

  • Debugging was extremely difficult

  • You could get a PhD implementing reasonably complex algorithms using shading languages and rendering rectangles

  • Financial companies would hire game programmers to do market analysis

David Mayerich

STIM Laboratory, University of Houston

Compute Unified Device Architecture

  • GeForce 8800 (2006)

    • Tesla architecture

    • first CUDA compatible card

  • CUDA 1.0 SDK (2007)

    • 8800+ CUDA compatible

    • evolving compute capability (now 12.1)

  • Parallel model still based on GPUs

    • data parallel

    • single-instruction multiple data
      (maps to shader programs)

David Mayerich

STIM Laboratory, University of Houston

GeForce 8800, Froggy (2006)

Nvidia Tech Demos

David Mayerich

STIM Laboratory, University of Houston

GeForce 200, Medusa (2008)

GeForce GTX 460, Heaven (2010)

GeForce GTX TITAN, Ira (2013)

GeForce GTX 900, Apollo Lunar Landing (2015)

Modern Nvidia GPUs

  • CUDA Cores:

    • 32-element SIMD vector units

    • hyperthreading

  • Tensor Cores:

    • optimized matrix multiply-and-accumulate (MMA)

    • mixed-precision arithmetic (ex. FP64, FP32, FP16, FP8)

  • Ray Tracing Cores:

    • bounding volume hierarchy (BVH) acceleration

    • optimized intersection detection

David Mayerich

STIM Laboratory, University of Houston

CUDA API

C/C++

CUDA Cores

Tensor Cores

Ray Tracing Cores

cuBLAS
cuDNN

OptiX
DirectX
Vulkan

Shading Language

David Mayerich

STIM Laboratory, University of Houston