WebGL: Latest
Techniques

SIGGRAPH Asia 2018  |  Khronos BOF

4 December 2018

Course Presenters

Xavier Ho

CSIRO's Data61

xavier.ho@data61.csiro.au

Juan Miguel de Joya

United Nations International Telecommunication Union

juandejoya@gmail.com

Germain Rufflé

Google Daydream

germain.ruffle@gmail.com

Neil Trevett

NVIDIA, Khronos Group

ntrevett@nvidia.com

Mike Alger

Google Daydream

mikealger@google.com

Goals and Expectations

To present the latest work and techniques from the WebGL community

 

To cast a light on relevant standard groups (e.g. WebGPU)

 

To discuss effective user experience design in digital and augmented spaces

Technologists

UX Design

Standards

This presentation assumes a beginner's understanding of graphics rendering

Session Agenda

WebGL: latest and shiniest features

Optimisation techniques

Industry and community highlights

User experience design methods

Growing Adoption

WebGL 1.0 Applications

WebGL Public Wiki (Khronos), Long Live WebGL (Wired)

WebGL 1.0 (2011) was widely adopted in a variety of applications:

TensorFire: Neural Network on the web

WebGL 1.0 Adoption

Can I Use, WebGL Stats

WebGL 2.0 Adoptions

WebGL 2.0 is  growing!
 

WebGL Stats:
44-68% global (up from 37%, 2017)

Can I Use:
67-69% global (up from 29%, 2017)

Khronos WebGL 2.0 Webinar (Apr 2017),
WebGL2Fundamentals, Mozilla Hacks,
WebGL2 Basics (Real-time Rendering)

Progressive Extensions

WebGL 1.0

WebGL 2.0
feature guarantees

Many WebGL 2.0 features are available in WebGL 1.0 as extensions.

 

Engines will provide fallback solutions.

 

You can use WebGL 2.0 today!

WebGL Extension Registry (Khornos)

(go to website)

Friends of WebGL

glTF 2.0

Universal 3D file format, "JPEG for 3D" supports physically-based rendering

 

Good ecosystem support today: ARCore, Unity, Blender, and many more. A validator is available.

 

Find out more at the glTF session after this WebGL session :)

glTF Viewer

WebAssembly

Default compile target for Emscripten from any language (C++, Rust)
 

WebAssembly exposes Web APIs, including the DOM, Web Audio, WebGL

MDN, Unity Benchmarks, Emscripten Docs

Demos: Tanks!, wasm-particles

OffscreenCanvas

WebGL rendering off the main thread

 

Permits smooth rendering with CPU-intensive user interactions!

MDN, Google Developers

Demo: Three.js

CON

Difficult to synchronise exact rendering frame with user interactions

PRO

Near 1:1 API compatibility with <canvas> as progressive enhancement

GPU on the Web (proposals)

A new API for accessing GPU features on the web, based on designs from Vulkan, Metal, and Direct3D 12

 

Safari has an initial implementation behind flags. Chrome intends to impl.

ICS Media, W3C Community Group

 Demo: WebGPU, Compute (latest Safari)

WebGL 2.0 Compute Shaders (draft proposal)

Brings support for OpenGL ES 3.1 Compute Shaders, and GLSL 3.10

 

No browser currently implements this.
Chrome is working on an implementation.

WebGL 2.0 Compute Draft Proposal

Unity Compute Shader example :)

WEBGL_multiview Extension

Increases performance for web apps that are CPU-bound with multiple viewports, e.g. WebVR/WebXR

 

Compatible with WebGL 1.0

WebGL Aquarium Demo (Chromium only, setup instructions)

(If you're lucky.)

(Doesn't work on Mac when we tried yesterday.)

WebXR (experimental)

A single API to conquer all VR/AR on the web, including 6DOF and controller tracking (40% adoption)

 

Experimental binding to native mobile VR/AR APIs - available now

Mozilla Hacks,
Mixed Reality Blog, Can I Use

public_webgl@khronos.org

Join the discussion:

Optimisation Techniques

Profile, profile, profile

There are no one-trick-fits-all optimisation techniques, due to hardware differences

 

Instrument profiling often, automate if possible. Save screenshots.

 

And remember: good enough is good enough

Web Tracing Framework, Optimising WebGL Tips

Performance Tips - GL in general

  • Batch your draw calls into glDrawArrays(). (StackExchange)
  • Reduce redundant gl* calls. (see WebGL Inspector)
  • Favor switching bindings over writing data into buffer objects. (Khronos wiki)
  • Link shader programs infrequently, and at start if possible.
  • Clear colour & depth bits together; avoid unnecessary clearing.

Debugging and Optimising WebGL Applications (New Game 2011), FBOs (Song Ho), Porting Source to Linux (Valve)

Performance Tips - For the Web

  • Progressively enhance the user experience:
    • Use requestAnimationFrame() for draw calls
    • Later on: asynchronously load assets if you can
       
  • ​Respect the user's time
    • Optimise assets: download, storage, and access
    • Provide Level of Details (LOD) quality toggles
    • Cache, cache, cache (Babylon.js automated this)

IndexDB for WebGL (David Rousset), requestAnimationFrame (Paul Irish)

Performance Tips - WebGL 2 Textures

  • Turn on mipmapping. Memory is cheaper than time.
     
  • Immutable texture formats can be used from GPU memory immediately - use it wherever possible. (Samsung Developer)
     
  • Favor glBufferSubData() & glTexSubImage2D/3D() over glBufferData() & glTexImage2D/3D() even when the whole texture is updated. (Emscripten)

Optimisation for Game Assets Guide (Samsung Developers), WebGL2Samples, Using Textures (MDN)

Performance Tips - Instancing

Use instancing as much as you can.

  • Details and set dressing (trees, grass, buildings) and crowds/characters.
     
  • This in turn reduces draw calls as well.
     
  • Works with Transform Feedback to calculate particle position or skinned mesh.

WebGL Techniques (Google I/O 2011) , Crowd demo (tojicode), WebGL Instancing (tojicode), Instancing demo (three.js)

Transform Feedback

  • Transform Feedback: a vertex post-processing step for recording values output from the vertex process into the Buffer Objects.

    • Computations and data from shaders stay on GPU.

    • Post-transform attributes can be easily read from the buffer.

       

Transform Feedback: Usage

  • Particle Systems: composed of a large amount of small particles that simulate physically-driven phenomena, such as smoke, dust, fireworks, rain, etc.

     

Without Transform Feedback...

    1. WebGL copies particle contents of the VBO from GPU memory to CPU memory.

    2. Calculations on particle attributes.

    3. Reload information onto the GPU to be rendered.

 

Managing data transfer between GPU and CPU and redundant complex computation takes time and bandwidth!

With Transform Feedback...

    1. Connect w/ Transform Feedback Buffer.

    2. Apply further transformations on VBOs before rendering, all computations done on the GPU.

    3. No application involvement other than connecting buffers and setting state.

 

Good for doing complex computations on VBOs, offload CPU and memory read times.

Transform Feedback: WebGL

Transform Feedback: WebGL

Performance Tips - Shaders

WebGL Best Practices (MDN)

Performance Tips - Use WebGL 2.0!

  • Many new texture formats
  • 3D Textures / 2D Texture Arrays
  • Full non-power-of-2 texture support
  • Seamless cube maps
  • Multiple Render Targets (critical for deferred rendering paths)
  • Integer vertex attributes
  • Multisampled renderbuffers
  • Uniform Buffer Objects
  • ... and many more!

Using Emscripten? Turn on WebGL 2.0 by using -s USE_WEBGL2=1

Check out the WebGL 2 Examples by Tarek Sherif: 

https://github.com/tsherif/webgl2examples

State-of-the-Art WebGL 2.0 (SIGGRAPH Asia 2017)

Forward Rendering

Forward Rendering: pass geometry data into the vertex shader(s) for per-vertex operations, have these rasterized into fragments, do per-pixel operations via the fragment shader(s) before render.

  • Render primitive by lighting it according to all light sources. Each primitive is passed down linearly.
  • Heavy on performance, wastes a lot of fragment shader runs.

Deferred Rendering

Deferred Rendering: rendering is deferred until all primitives have been passed down the pipeline.

  • Two stages: geometry pass for geometry data stored as in multiple draw buffers; lighting pass to calculate final shading and lighting calculations based on buffered data.
  • Decouples expensive fragment processes to a later stage in the pipeline.

Deferred: Pros and Cons

Forward Rendering…

Pros

    1. Polygon data are passed per vertex and fragment, drawn over the screen.

    2. Preserves true transparency and provides anti-aliasing out of the box.

Cons

    1. Linear, heavy on performance

    2. Wasted fragment shader calculations

    3. Limited number of lights available for rendering.

Deferred Rendering…

Pros

    1. Actual fragment information as a screen pixel, only one fragment shader pass.

    2. Great for dynamic lights and optimizing for a larger amount of light sources and post-processing.

Cons

    1. Requires high bandwidth, memory.

    2. Requires transparency, anti-aliasing workarounds.


 

Deferred: Non-WebGL Cases

  • Real-time lighting via light linked list

  • Physically correct lighting models

  • SPU-based deferred shading

  • Extensive particle systems

  • Post-processing effects (bloom, FXAA, AO)

Multiple Draw Buffers

  • Available as WebGL 1.0 as extension, now core

  • Allow rendering to multiple framebuffers in one pass

  • Big performance boost for:

    • Deferred rendering

    • Multipass post-processing effects

    • Other techniques that use depth, color, normals, and so on into multiple buffers

  • Applications of Multiple Draw Buffers

Volumetrics: Non-WebGL Cases

  • Occluded scattering resulting in volumetric shadows and light shafts

  • Heterogeneous participating media

  • Sun/Particle interaction, physically-based sky and atmosphere simulation, large scale fog

    • Art directable clouds for Horizon Zero Dawn, Pixar’s The Good Dinosaur

Volumetrics: Non-WebGL Cases

  • Volumetric materials for character pipeline.

Volumetrics: WebGL Examples

  • Volumetric rendering simulates the propagation of light rays into a material’s volume (a.k.a a participating media), which allow for a broad range of rendering effects.

  • Volume ray marching is a technique where a ray is shot/cast from the camera into the material’s volume for rendering calculations.

Images: Nop Jiarathanakul, Almar Klein

Shading Language Updates


First line always starts with #version 300 es

  • Compiler needs to know if you’re using the new version.

Some of the new features:

See GLSL ES 3.00 Specification for the complete feature set.

Images: Shuai Shao, Trung Le, Patrick Cozzi

State of Industry Survey

State of: Libraries & Frameworks

  • Three.js:  r95 supports WebGL 2.0

  • Babylon.js: Microsoft's real-time 3D engine

  • PicoGL.js: minimal WebGL 2 rendering library

  • Unity/Unreal: uses Emscripten to port C++ code

  • PlayCanvas: Commercial game engine, supports most WebGL 2.0 feature

  • Shadertoy: Supports GLSL 3.00 fragment shader with WebGL 2

  • Blend4Web: Blender-interoperable engine, supports most WebGL 2.0 features

  • Verge3D: Blend4Web's founders' new framework

  • A-Frame: Open-sourced WebVR framework

  • Whitestorm.js: based on Three.js that simplify code, adds physics, and post-effects

State of: WebGL 2.0 Games

After the Flood [article][video]

  • Procedural clouds (3D textures)
  • Procedural water ripples/reflections
  • Animated trees
  • Dynamic lights
  • Leaf particle system
  • Alpha to coverage (antialiased foliage)
  • HDR rendering MSAA for correct blending of antialiased HDR surfaces.
  • Hardware PCF for shadow filtering
  • Compressed textures (DXT, PVR, ETC1)
  • Asynchronous asset streaming
  • Runtime lightmap baking
  • Planar mirrors for mirrored surfaces

State of: WebGL 2.0 Games

  • Voxels_WebGL2: Notch's current project [link]

    • ​Voxel engine running at 99fps

    • 60+km view distance

    • Real-time terrain generation

    • Modifiable voxels at runtime

State of: WebGL 2.0 Games

State of: Media

  • Facebook 3D Photos and Posts [link]

  • SketchFab 3D Remix Challenge [link]

  • Just A Reflektor (2013), Arcade Fire + Google Data Arts [link]

  • Project SUN (2017), Philip Schütte + Venice Art Projects [link]

  • Stream (2016), Ezra Miller + Houston's Day for Night Festival [link]

  • HTC Vive Projection Mapping in WebVR (2016), Hien Huynh + Marpi @ DanceHackDay [link]

  • Virtual Art Sessions, Google Tilt Brush team [link]

  • Codevember Examples (2018), Jaume Sanchez Elias [link]

  • ShaderToy Examples (2018), Inigo Quilez [link]

State of: Data Visualization

  • Data visualisation of > 2mil datasets at 60fps.

  • Cesium: 3D Geospatial Mapping & Analysis

    •  Web-based visualisation for time-dynamic autonomous car simulation [link]

  • Regl: Declarative and stateless WebGL

    • Point animation [link]

    • 360 photo [link]

  • Pixi.js: Fast, flexible, 2D WebGL renderer

    • How Uber Uses Psychological Tricks to Push Its Drivers’ Buttons, New York Times [link]

  • Luma.gl: GPU-powered data visualisation/computation

  • Deck.gl: Large-scale data visualisation

    • ​UK Road Safety Visualization [link]

    • 3D Indoor Scan Visualization [link]

  • Three.js: High-level scene graph

    • Bear 71 by Jam3 [link]

State of: Data Visualization

  • CryptoVoxels: Blockchain visualisation [link]

  • Illuminated City: Crime data visualisation [link]

  • Healthcare visualization

    • MRI obliquing with Pixpipe.io + WebGL2 [link]

    • A Web3D Serious Game for Human Anatomy Education [link]

  • Experience Insight:  NASA's JPL visualisation of the Mars Rover Insight [link]

  • TensorFlow.js: In-browser machine learning that is deployed and executed with WebGL2 [link]

  • Punk Office: 3D Avatars on the Web [link]

  • RDF Graphs visualisation: Visualisation of Global Cargo Ships [link]

  • Google Earth VR: Virtual reconstruction of Earth [link]

  • Shipmap: Visualisation of Global Cargo Ships [link]

  • D3.js and Three.js: GeoJSON interoperation [link]

State of: WebXR

  • Device adaptations are growing (96% for WebGL 1.0, 37% for WebGL 2.0, and growing).

  • WebVR Specification v2 Draft: [link], slated for Q3 2018 recommendation

  • Firefox v55 shipped WebVR by default (8th August, 2017)

Image: Don McCurdy

VR/AR: Non-WebGL Cases

  • Different setups

    • Mobile XR (Oculus Go/Quest, ARCore, ARKit)

    • Room-scale XR (Oculus Rift, Vive, Hololens, Magic Leap)

    • Full body tracking (HoloSuit)

  • Cinematic and/or narrative experiences

    • After Solitary, FRONTLINE/Emblematic Group [link]

    • Cycles, Walt Disney Animation Studios [link]

    • Age of Sail, Google Spotlight Stories [link]

  • Interaction/Gaming Experiences

    • Beat Saber, Beat Games [link]

    • Superhot, SuperHOT Team [link]

  • Virtual training/simulation

    • $480k military contract, 100K Hololens, Microsoft [link]

    • Pre-flight safety check, Qantas [link]

    • International Space Station Training, NASA [link]

VR/AR: Non-WebGL Cases

  • Art

    • Installation 036, Steve Teeps [link]

    • VR is for Artists, TIME [link]

  • Visualization

    • KPMG HoloLens Data Analytics tool, KPMG/Loook [link]

    • Virtual Data Visualization Platform, Virtualitics [link]

  • Media

    • Alive in Plasticland, High Fidelity [link]

    • Spider-Man: Into the Spider-Verse AR, Trigger Global, 8th Wall, AWS [link]

  • Medical/Health

    • Advanced Imaging and Medical Modeling Program, OSF Healthcare [link]

    • Smart Bionics Medium Article, The Luke Hand Project [link]

VR/AR: Non-WebGL Cases

Why WebXR?

Image: SantaWinsAgain

WebXR Frameworks

  • Frameworks include:

    • WebVR, Google’s JavaScript API for creating immersive VR experiences [demo]

    • A-Frame: Make WebVR with HTML and Entity-Component. [demo]

    • JSARtoolkit: Emscripten port of ARToolKit to JavaScript. [demo]

    • Argon.js: Adding augmented reality content to web applications [demo]

    • Awe.js: Mixed reality on the web [link]

Image: Mozilla

WebXR: It’s Happening

UX Design for Digital/Augmented Spaces

WebGL Futures

What’s Coming to WebGL?

Khronos WebGL Extension Registry

What’s Coming to WebGL? (cont.)

Image: Khronos Group

In Closing

WebGL 2.0 is growing fast -
we may hit 65%+ adoption next year! *

 

WebGL 1.0 already universal -

as seen in the state of industry survey


2018 has been a year of complementary Web API developments for WebGL.

2019 will hopefully see more WebGL extensions moving into implementation.

* based on linear projection, WebGL 2.0 may be universally supported by 2022.

Session slides available at

WebGL (this slidedeck):

https://tinyurl.com/sa2018-webgl

 

UX Design for Digital/Augmented Spaces:

https://slides.com/mikealger

Acknowledgements & Thanks

Neil Trevett

Emily Stearns

Angela Cheng

Kris Rose

Takashi Umezawa

Hitoshi Kasai

Kenneth Russell

Zhenyao Mo

Kai Ninomiya

Brandon Jones

Ricardo Cabello

Diego Marcos

Don McCurdy

Kevin Ngo

Tomasz Bednarz

Felix Maya

Jaume Sanchez Elias

Leonardo Carrion

Tarek Sherif

...and more!

Pol Jeremias

Inigo Quilez

Yang Gu

Jie Chen

Jeff Lang

Olli Etuaho

Gregg Tavares

Damon Hernandez

Will Eastcott

Josh Carpenter

Iker Jamardo

Google WebXR Team

MDN Contributors

Emscripten Team

Samsung Developers

 

WebGL: Latest Techniques | Khornos BOF, SIGGRAPH Asia 2018

By Xavier Ho

Made with Slides.com

WebGL: Latest Techniques | Khornos BOF, SIGGRAPH Asia 2018

SIGGRAPH Asia 2018 Khronos BOF

  • 2,426

More from Xavier Ho