VIZBI 2022

Hello

...it is good to be back!

GPU Access!

http://madebyevan.com/webgl-water/

https://epok.tech/work/tendrils/

https://demo.marpi.pl/biomes/

https://demo.marpi.pl/ocean_desert/

https://www.newnaw.com/pub/js/webglglobe/worldelevation/​

https://www.larsberg.net/#/lubricious

https://paveldogreat.github.io/WebGL-Fluid-Simulation/

https://slicedrop.com

What has changed since 2013?

WebGL is everywhere!

There are a ton of frameworks and tutorials available!

...and The Metaverse is coming

support for 3D textures!

https://caniuse.com/webgl2

aka The Next OpenGL

WebGPU will be based on Vulkan

https://caniuse.com/webgpu

Image Volumes

Meshes

DTI Fibers

Radiology

0.1 - 1 mm^3 voxel size

< 1 Gigabyte

https://bioimagesuiteweb.github.io/webapp/viewer.html

https://brainbrowser.cbrain.mcgill.ca/volume-viewer

https://kitware.github.io/paraview-glance/app/

https://ohif.org/newsletters/2021-09-14-ohif-update-v3-public-beta-launch

Connectomics

Brain Connectivity at Synapse Level

nanometer resolution!

tera- or petabytes of data!

Drosophila Hemibrain Connectome

8x8x8nm^3 voxel size

~25,000 neurons

https://hemibrain-dot-neuroglancer-demo.appspot.com/#!gs://neuroglancer-janelia-flyem-hemibrain/v1.0/neuroglancer_demo_states/base.json

Google + Janelia

https://skilled.co/html-canvas/

WebGL is hard!

Frameworks make it easier!!

the most popular one!

XTK: the easiest!

brandnew + fast

Hello Cube!

Browser

Editor

Let's go!

<html>
  <head>
    <title>Hello Cube!</title>
    <style>
      html, body { 
        background-color:#000;
        margin: 0;
        padding: 0;
        height: 100%;
        overflow: hidden !important;  
      }
    </style>
    <script type="text/javascript" src="https://get.goXTK.com/xtk_edge.js"></script>
    <script type="text/javascript">
      window.onload = function() {

        var r = new X.renderer3D();
        r.init();

        var cube = new X.cube();

        r.add(cube);

        r.render();

      };
    </script>
  </head>
  <body>
  </body>
</html>

https://slides.com/haehn/vizbi#/31

easy

hard

XTK

Three.js

WebGL

limited

freedom

Complexity

Functionality

XTK

X.renderer3D

Renderer

Scene

Interaction

Loop

Canvas

Camera

Lighting

Three.js

THREE.WebGLRenderer

THREE.Scene

THREE.TrackballControls

Loop

Canvas

THREE.PerspectiveCamera

THREE.AmbientLight

THREE.DirectionalLight

12 Triangles

V1

V2

V3

V4

V6

V5

(x, y, z)

(x, y, z)

(x, y, z)

(x, y, z)

(x, y, z)

(x, y, z)

Y

X

Z

Vertex

/ Vertices

V1

V2

V3

V4

V6

V5

Vertex

/ Vertices

Face

Face

Edge

12 Triangles

3D

2D

?

12 Triangles

3D

2D

2560 x 1600 Pixels

width

height

Frame Buffer

Screen Space

width x height Pixels

Rasterization

Rendering Pipeline

JavaScript      3D     Screen Space

V1

V2

V3

(x, y, z)

(x, y, z)

(x, y, z)

V1

V2

(x, y, z)

(x, y, z)

V3

(x, y, z)

V1

V2

(x, y, z)

(x, y, z)

V3

(x, y, z)

var geometry = new THREE.Geometry();
geometry.vertices.push(
	new THREE.Vector3(-10, 10, 0),
	new THREE.Vector3(-10, -10, 0),
	new THREE.Vector3(10, -10, 0)
);
geometry.faces.push( new THREE.Face3(0, 1, 2));

Graphics Processing Unit (GPU)

Vertex Shader

3D World

Shape Assembly

Perspective Projection

V1

V2

V3

var geometry = new THREE.Geometry();
geometry.vertices.push(
	new THREE.Vector3(-10, 10, 0),
	new THREE.Vector3(-10, -10, 0),
	new THREE.Vector3(10, -10, 0)
);
geometry.faces.push( new THREE.Face3(0, 1, 2));

Graphics Processing Unit (GPU)

Vertex Shader

Fragment Shader

3D World

2D Space

Rasterization

Colorization

Frame Buffer / Viewport

Blending

Frame Buffer / Viewport

var geometry = new THREE.Geometry();
geometry.vertices.push(
	new THREE.Vector3(-10, 10, 0),
	new THREE.Vector3(-10, -10, 0),
	new THREE.Vector3(10, -10, 0)
);
geometry.faces.push( new THREE.Face3(0, 1, 2));

GPU

Vertex Shader

Fragment Shader

Viewport

From 3D..

..to 2D

Fragment Shader

Vertex Shader

gl_Position

gl_FragColor

for every vertex

for every pixel

attribute vec4 a_position;

void main() {
  gl_Position = a_position;
}
precision mediump float;

void main() {
  gl_FragColor = vec4(1., 1., 1., 1.);
}

X

Y

Z

3D?

It is all virtual.

Viewport

Camera

(Eye)

Frustum

zNear

zFar

Viewport

Camera

(Eye)

Perspective Projection

Camera

Scene

Camera

Eye

Position (x,y,z)

Focus (x,y,z)

Scene

Center

Up (x,y,z)

Camera

Eye

Position (0,0,100)

Focus (0,0,0)

Scene

Center

Up (0,1,0)

Translate (Move in x,y)

Zoom (Move in z)

Rotate

XTK

X.renderer3D

Renderer

Scene

Interaction

Loop

Canvas

Camera

Lighting

Three.js

THREE.WebGLRenderer

THREE.Scene

THREE.TrackballControls

Loop

Canvas

THREE.PerspectiveCamera

THREE.AmbientLight

THREE.DirectionalLight

Trackball Controls

Natural Feeling

<html>
  <head>
    <title>Hello Cube!</title>
    <style>
      body { 
        background-color:#000;
        margin: 0;
        padding: 0;
        height: 100%;
        overflow: hidden !important;  
      }
    </style>
    <script src="https://threejs.org/build/three.min.js" type="text/javascript"></script>
    <script src="https://threejs.org/examples/js/controls/TrackballControls.js" type="text/javascript"></script>
    <script type="text/javascript" src="http://get.goXTK.com/xtk_xdat.gui.js"></script>
    <script type="text/javascript">
      window.onload = function() {

        // setup scene
        var scene = new THREE.Scene();

        // configure camera
        var fov = 75;
        var ratio = window.innerWidth / window.innerHeight;
        var zNear = 1;
        var zFar = 10000;
        var camera = new THREE.PerspectiveCamera( fov, ratio, zNear, zFar );
        camera.position.set(0, 0, 100);

        // .. add WebGL Renderer
        var renderer = new THREE.WebGLRenderer();
        renderer.setSize(window.innerWidth, window.innerHeight);
        document.body.append(renderer.domElement);

        //.. setup lights
        ambientLight = new THREE.AmbientLight( 0x404040 );
        scene.add( ambientLight );

        light = new THREE.DirectionalLight( 0xffffff, 5.0 );
        light.position.set( 10, 100, 10 );
        scene.add( light );

        // add the CUBE
        var geometry = new THREE.BoxBufferGeometry(20, 20, 20);
        var material = new THREE.MeshStandardMaterial({ color: 0xffffff });
        var cube1 = new THREE.Mesh(geometry, material);

        scene.add(cube1);

        // and the trackball
        var controls = new THREE.TrackballControls(camera, renderer.domElement);

        // start rendering!
        render();


        // the rendering loop!
        function render() {

          controls.update();
          renderer.render(scene, camera);
          
          requestAnimationFrame(render);
        }

      };
    </script>
  </head>
  <body>
  </body>
</html>

https://slides.com/haehn/vizbi/#/60

<html>
  <head>
    <title>Cubes!</title>
    <style>
      html, body { 
        background-color:#000;
        margin: 0;
        padding: 0;
        height: 100%;
        overflow: hidden !important;  
      }
    </style>
    <script type="text/javascript" src="https://get.goXTK.com/xtk_edge.js"></script>
    <script type="text/javascript" src="http://get.goXTK.com/xtk_xdat.gui.js"></script>
    <script type="text/javascript">
      window.onload = function() {

        var r = new X.renderer3D();
        r.init();

        cube1 = new X.cube();
        cube1.color = [1, 1, 1]; // r g b
        cube1.center = [-30, 0, 0];

        var cube2 = new X.cube();
        cube2.color = [0, 0, 1];
        // cube2.lengthY += 0.01;
        // cube2.lengthZ += 0.01;

        r.add(cube1);
        r.add(cube2);

        r.render();

        var gui = new dat.GUI();
        gui.addColor(cube1, 'color');
        gui.add(cube1, 'lengthX', 0, 100).onChange(function() {
          cube1.modified();
        });
        gui.open();

      };
    </script>
  </head>
  <body>
  </body>
</html>

https://slides.com/haehn/vizbi/#/62

<html>
  <head>
    <title>Cubes!</title>
    <style>
      body { 
        background-color:#000;
        margin: 0;
        padding: 0;
        height: 100%;
        overflow: hidden !important;  
      }
    </style>
    <script src="https://threejs.org/build/three.min.js" type="text/javascript"></script>
    <script src="https://threejs.org/examples/js/controls/TrackballControls.js" type="text/javascript"></script>
    <script type="text/javascript" src="http://get.goXTK.com/xtk_xdat.gui.js"></script>
    <script type="text/javascript">
      window.onload = function() {

        // setup scene
        var scene = new THREE.Scene();

        // configure camera
        var fov = 75;
        var ratio = window.innerWidth / window.innerHeight;
        var zNear = 1;
        var zFar = 10000;
        var camera = new THREE.PerspectiveCamera( fov, ratio, zNear, zFar );
        camera.position.set(0, 0, 100);

        // .. add WebGL Renderer
        var renderer = new THREE.WebGLRenderer();
        renderer.setSize(window.innerWidth, window.innerHeight);
        document.body.append(renderer.domElement);

        //.. setup lights
        ambientLight = new THREE.AmbientLight( 0x404040 );
        scene.add( ambientLight );

        light = new THREE.DirectionalLight( 0xffffff, 5.0 );
        light.position.set( 10, 100, 10 );
        scene.add( light );

        // add the CUBE
        var geometry = new THREE.BoxBufferGeometry(20, 20, 20);
        var material = new THREE.MeshStandardMaterial({ color: 0xffffff });
        var cube1 = new THREE.Mesh(geometry, material);

        scene.add(cube1);

        // and ANOTHER CUBE
        var geometry = new THREE.BoxBufferGeometry(20, 20, 20);
        var material = new THREE.MeshStandardMaterial({ color: 0x0000ff });
        var cube2 = new THREE.Mesh(geometry, material);
        cube2.position.set(-30, 0, 0);

        scene.add(cube2);

        // and the trackball
        var controls = new THREE.TrackballControls(camera, renderer.domElement);

        lengthX = 20;
        var gui = new dat.GUI();
        gui.add(window, 'lengthX', 0, 100).onChange(function() {
          cube1.geometry = new THREE.BoxBufferGeometry(window.lengthX, 20, 20);
        });
        gui.open();

        // start rendering!
        render();


        // the rendering loop!
        function render() {

          controls.update();
          renderer.render(scene, camera);
          
          requestAnimationFrame(render);
        }

      };
    </script>
  </head>
  <body>
  </body>
</html>

https://slides.com/haehn/vizbi/#/63

Z-Fighting

3D Ultrasound Visualization

ultrasound.nrrd

https://danielhaehn.com/vizbi/

<html>
  <head>
    <title>3D Ultrasound!</title>
    <style>
      body { 
        background-color:#000;
        margin: 0;
        padding: 0;
        height: 100%;
        overflow: hidden !important;  
      }
    </style>
    <script type="text/javascript" src="https://get.goXTK.com/xtk_edge.js"></script>
    <script type="text/javascript" src="http://get.goXTK.com/xtk_xdat.gui.js"></script>
    <script type="text/javascript">
      window.onload = function() {

        var r = new X.renderer3D();
        r.init();

        var v = new X.volume();
        v.file = 'ultrasound.nrrd';

        r.add(v);
        r.render();


        // the onShowtime method gets executed after all files were fully loaded and
        // just before the first rendering attempt
        r.onShowtime = function() {

          //
          // The GUI panel
          //
          // (we need to create this during onShowtime(..) since we do not know the
          // volume dimensions before the loading was completed)
          
          var gui = new dat.GUI();
          
          // the following configures the gui for interacting with the X.volume
          var volumegui = gui.addFolder('Volume');
          // the indexX,Y,Z are the currently displayed slice indices in the range
          // 0..dimensions-1
          var sliceXController = volumegui.add(v, 'indexX', 0,
              v.range[0] - 1);
          var sliceYController = volumegui.add(v, 'indexY', 0,
              v.range[1] - 1);
          var sliceZController = volumegui.add(v, 'indexZ', 0,
              v.range[2] - 1);
          volumegui.open();
          
        };

      };
    </script>
  </head>
  <body>
  </body>
</html>

https://slides.com/haehn/vizbi/#/68

<html>
  <head>
    <title>3D Ultrasound!</title>
    <style>
      body { 
        background-color:#000;
        margin: 0;
        padding: 0;
        height: 100%;
        overflow: hidden !important;  
      }
    </style>
    <script src="https://threejs.org/build/three.min.js" type="text/javascript"></script>
    <script src="https://threejs.org/examples/js/controls/TrackballControls.js" type="text/javascript"></script>
    <script type="text/javascript" src="http://get.goXTK.com/xtk_xdat.gui.js"></script>

    <script src="https://threejs.org/examples/js/loaders/NRRDLoader.js" type="text/javascript"></script>
    <script src="https://threejs.org/examples/js/libs/fflate.min.js" type="text/javascript"></script>
    <script src="https://threejs.org/examples/js/misc/Volume.js" type="text/javascript"></script>
    <script src="https://threejs.org/examples/js/misc/VolumeSlice.js" type="text/javascript"></script>

    <script type="text/javascript">
      window.onload = function() {

        // setup scene
        var scene = new THREE.Scene();

        // configure camera
        var fov = 75;
        var ratio = window.innerWidth / window.innerHeight;
        var zNear = 1;
        var zFar = 10000;
        var camera = new THREE.PerspectiveCamera( fov, ratio, zNear, zFar );
        camera.position.set(0, 0, 100);

        // .. add WebGL Renderer
        var renderer = new THREE.WebGLRenderer();
        renderer.setSize(window.innerWidth, window.innerHeight);
        document.body.append(renderer.domElement);

        //.. setup lights
        ambientLight = new THREE.AmbientLight( 0x404040 );
        scene.add( ambientLight );

        light = new THREE.DirectionalLight( 0xffffff, 5.0 );
        light.position.set( 10, 100, 10 );
        scene.add( light );



        // LOAD NRRD FILE
        var gui = new dat.GUI();

        var loader = new THREE.NRRDLoader();
        loader.load( 'ultrasound.nrrd', function ( volume ) {

          var geometry = new THREE.BoxGeometry( volume.xLength, volume.yLength, volume.zLength );
          var material = new THREE.MeshBasicMaterial( { color: 0x00ff00 } );
          var cube = new THREE.Mesh( geometry, material );
          cube.visible = false;
          var box = new THREE.BoxHelper( cube );
          scene.add( box );
          box.applyMatrix4( volume.matrix );
          scene.add( cube );

          //z plane
          var sliceZ = volume.extractSlice( 'z', Math.floor( volume.RASDimensions[ 2 ] / 4 ) );
          scene.add( sliceZ.mesh );

          //y plane
          var sliceY = volume.extractSlice( 'y', Math.floor( volume.RASDimensions[ 1 ] / 2 ) );
          scene.add( sliceY.mesh );

          //x plane
          var sliceX = volume.extractSlice( 'x', Math.floor( volume.RASDimensions[ 0 ] / 2 ) );
          scene.add( sliceX.mesh );

          gui.add( sliceX, 'index', 0, volume.RASDimensions[ 0 ], 1 ).name( 'indexX' ).onChange( function () {

            sliceX.repaint.call( sliceX );

          } );
          gui.add( sliceY, 'index', 0, volume.RASDimensions[ 1 ], 1 ).name( 'indexY' ).onChange( function () {

            sliceY.repaint.call( sliceY );

          } );
          gui.add( sliceZ, 'index', 0, volume.RASDimensions[ 2 ], 1 ).name( 'indexZ' ).onChange( function () {

            sliceZ.repaint.call( sliceZ );

          } );

        });




        // and the trackball
        var controls = new THREE.TrackballControls(camera, renderer.domElement);

        // start rendering!
        render();


        // the rendering loop!
        function render() {

          controls.update();
          renderer.render(scene, camera);
          
          requestAnimationFrame(render);
        }

      };
    </script>
  </head>
  <body>
  </body>
</html>

https://slides.com/haehn/vizbi/#/69

daniel.nii

https://danielhaehn.com/vizbi/

<html>
  <head>
    <title>MRI!</title>
    <style>
      body { 
        background-color:#000;
        margin: 0;
        padding: 0;
        height: 100%;
        overflow: hidden !important;  
      }
    </style>
    <script type="text/javascript" src="https://get.goXTK.com/xtk_edge.js"></script>
    <script type="text/javascript" src="http://get.goXTK.com/xtk_xdat.gui.js"></script>
    <script type="text/javascript">
      window.onload = function() {

        var r = new X.renderer3D();
        r.init();

        var v = new X.volume();
        v.file = 'daniel.nii';

        r.add(v);


        r.camera.position = [0,0,1000];

        r.render();


        // the onShowtime method gets executed after all files were fully loaded and
        // just before the first rendering attempt
        r.onShowtime = function() {

          // activate volume rendering
          v.volumeRendering = true;
          v.lowerThreshold = 80;
          v.windowLower = 115;
          v.windowHigh = 360;
          v.opacity = 0.2;
          
        };

      };
    </script>
  </head>
  <body>
  </body>
</html>

https://slides.com/haehn/vizbi/#/71

<html>
  <head>
    <title>NiiVue!</title>
    <style>
      body { 
        background-color:#000;
        margin: 0;
        padding: 0;
        height: 100%;
        overflow: hidden !important;  
      }
    </style>
    <script type="text/javascript" src="https://niivue.github.io/niivue/features/niivue.umd.js"></script>
    <script type="text/javascript">
      window.onload = function() {

        var nv1 = new niivue.Niivue()
        nv1.attachTo('gl1')
        nv1.setSliceType(nv1.sliceTypeRender)

      };
    </script>
  </head>
  <body>
    <canvas id='gl1' style='height:100%;width:100%'></canvas>
  </body>
</html>

https://slides.com/haehn/vizbi/#/72

https://danielhaehn.com/vizbi/

t2_mri.nii

tracks.trk

tumor.vtk

<html>
  <head>
    <title>MRI+Fibers+Mesh!</title>
    <style>
      body { 
        background-color:#000;
        margin: 0;
        padding: 0;
        height: 100%;
        overflow: hidden !important;  
      }
    </style>
    <script type="text/javascript" src="https://get.goXTK.com/xtk_edge.js"></script>
    <script type="text/javascript" src="http://get.goXTK.com/xtk_xdat.gui.js"></script>
    <script type="text/javascript">
      window.onload = function() {

        var r = new X.renderer3D();
        r.init();

        var v = new X.volume();
        v.file = 't2_mri.nii';
        r.add(v);

        var f = new X.fibers();
        f.file = 'tracks.trk';
        r.add(f);

        var m = new X.mesh();
        m.file = 'tumor.vtk';
        r.add(m);

        r.camera.position = [0,0,1000];

        r.render();

      };
    </script>
  </head>
  <body>
  </body>
</html>

https://slides.com/haehn/vizbi/#/74

https://spector.babylonjs.com/

https://www.blackinxr.org/

https://aframe.io/

https://venturebeat.com/2021/09/16/rooom-raises-7m-for-multifaceted-3d-virtual-events-platform/

https://cs460.org

https://danielhaehn.com/papers/?franke2020modern

https://danielhaehn.com/papers/?baidak2021cellprofiler

DANIELHAEHN.com

bit.ly/webglquiz

VIZBI2022

By Daniel Haehn

VIZBI2022

Data visualization with WebGL

  • 1,139

More from Daniel Haehn