WebGL: Latest
Techniques
SIGGRAPH Asia 2018 | Khronos BOF
4 December 2018
Course Presenters
Xavier Ho
CSIRO's Data61
xavier.ho@data61.csiro.au
Juan Miguel de Joya
United Nations International Telecommunication Union
juandejoya@gmail.com
Germain Rufflé
Google Daydream
germain.ruffle@gmail.com
Neil Trevett
NVIDIA, Khronos Group
ntrevett@nvidia.com
Mike Alger
Google Daydream
mikealger@google.com
Goals and Expectations
To present the latest work and techniques from the WebGL community
To cast a light on relevant standard groups (e.g. WebGPU)
To discuss effective user experience design in digital and augmented spaces
Technologists
UX Design
Standards
This presentation assumes a beginner's understanding of graphics rendering
Session Agenda
WebGL: latest and shiniest features
Optimisation techniques
Industry and community highlights
User experience design methods
Growing Adoption
WebGL 1.0 Applications
WebGL Public Wiki (Khronos), Long Live WebGL (Wired)
WebGL 1.0 (2011) was widely adopted in a variety of applications:
- Artists showcase (Sketchfab)
- Interactive journalism (NYT)
- Declarative maths (MathBox)
- Audio visualization (Google Experiments)
- Interactive graphics (Xmas Experiments)
- Engine compile target (Unity, Unreal)
-
Frameworks (Playcanvas, Turbulenz,
A-Frame, Three.js, Babylon.js)
WebGL 1.0 Adoption
WebGL 2.0 Adoptions
WebGL 2.0 is growing!
WebGL Stats:
44-68% global (up from 37%, 2017)
Can I Use:
67-69% global (up from 29%, 2017)
Khronos WebGL 2.0 Webinar (Apr 2017),
WebGL2Fundamentals, Mozilla Hacks,
WebGL2 Basics (Real-time Rendering)
Progressive Extensions
WebGL 1.0
WebGL 2.0
feature guarantees
Many WebGL 2.0 features are available in WebGL 1.0 as extensions.
Engines will provide fallback solutions.
You can use WebGL 2.0 today!
WebGL Extension Registry (Khornos)
Friends of WebGL
glTF 2.0
WebAssembly
MDN, Unity Benchmarks, Emscripten Docs
Demos: Tanks!, wasm-particles
OffscreenCanvas
WebGL rendering off the main thread
Permits smooth rendering with CPU-intensive user interactions!
Demo: Three.js
CON
Difficult to synchronise exact rendering frame with user interactions
PRO
Near 1:1 API compatibility with <canvas> as progressive enhancement
GPU on the Web (proposals)
A new API for accessing GPU features on the web, based on designs from Vulkan, Metal, and Direct3D 12
Safari has an initial implementation behind flags. Chrome intends to impl.
WebGL 2.0 Compute Shaders (draft proposal)
Brings support for OpenGL ES 3.1 Compute Shaders, and GLSL 3.10
No browser currently implements this.
Chrome is working on an implementation.
Unity Compute Shader example :)
WEBGL_multiview Extension
Increases performance for web apps that are CPU-bound with multiple viewports, e.g. WebVR/WebXR
Compatible with WebGL 1.0
WebGL Aquarium Demo (Chromium only, setup instructions)
(If you're lucky.)
(Doesn't work on Mac when we tried yesterday.)
WebXR (experimental)
A single API to conquer all VR/AR on the web, including 6DOF and controller tracking (40% adoption)
Experimental binding to native mobile VR/AR APIs - available now
public_webgl@khronos.org
Join the discussion:
Optimisation Techniques
Profile, profile, profile
There are no one-trick-fits-all optimisation techniques, due to hardware differences
Instrument profiling often, automate if possible. Save screenshots.
And remember: good enough is good enough
Performance Tips - GL in general
- Batch your draw calls into glDrawArrays(). (StackExchange)
- Reduce redundant gl* calls. (see WebGL Inspector)
- Favor switching bindings over writing data into buffer objects. (Khronos wiki)
- Link shader programs infrequently, and at start if possible.
- Clear colour & depth bits together; avoid unnecessary clearing.
Debugging and Optimising WebGL Applications (New Game 2011), FBOs (Song Ho), Porting Source to Linux (Valve)
Performance Tips - For the Web
-
Progressively enhance the user experience:
- Use requestAnimationFrame() for draw calls
-
Later on: asynchronously load assets if you can
-
Respect the user's time
- Optimise assets: download, storage, and access
- Provide Level of Details (LOD) quality toggles
- Cache, cache, cache (Babylon.js automated this)
IndexDB for WebGL (David Rousset), requestAnimationFrame (Paul Irish)
Performance Tips - WebGL 2 Textures
-
Turn on mipmapping. Memory is cheaper than time.
-
Immutable texture formats can be used from GPU memory immediately - use it wherever possible. (Samsung Developer)
- Favor glBufferSubData() & glTexSubImage2D/3D() over glBufferData() & glTexImage2D/3D() even when the whole texture is updated. (Emscripten)
Optimisation for Game Assets Guide (Samsung Developers), WebGL2Samples, Using Textures (MDN)
Performance Tips - Instancing
Use instancing as much as you can.
-
Details and set dressing (trees, grass, buildings) and crowds/characters.
-
This in turn reduces draw calls as well.
- Works with Transform Feedback to calculate particle position or skinned mesh.
WebGL Techniques (Google I/O 2011) , Crowd demo (tojicode), WebGL Instancing (tojicode), Instancing demo (three.js)
Transform Feedback
-
Transform Feedback: a vertex post-processing step for recording values output from the vertex process into the Buffer Objects.
-
Computations and data from shaders stay on GPU.
-
Post-transform attributes can be easily read from the buffer.
-
Transform Feedback: Usage
-
Particle Systems: composed of a large amount of small particles that simulate physically-driven phenomena, such as smoke, dust, fireworks, rain, etc.
Without Transform Feedback...
1. WebGL copies particle contents of the VBO from GPU memory to CPU memory.
2. Calculations on particle attributes.
3. Reload information onto the GPU to be rendered.
Managing data transfer between GPU and CPU and redundant complex computation takes time and bandwidth!
With Transform Feedback...
1. Connect w/ Transform Feedback Buffer.
2. Apply further transformations on VBOs before rendering, all computations done on the GPU.
3. No application involvement other than connecting buffers and setting state.
Good for doing complex computations on VBOs, offload CPU and memory read times.
Transform Feedback: WebGL
-
Caching and reusing GPU computation
-
Stateful particle systems
-
Non-WebGL implementations
-
Capture and reuse for geometric deformations
-
Terrain generation calculations
Transform Feedback: WebGL
-
Caching and reusing GPU computation
-
Stateful particle systems
-
Non-WebGL implementations
-
Capture and reuse for geometric deformations
-
Terrain generation calculations
Performance Tips - Shaders
-
Calculate early - per-draw, per-vertex, per-fragment (New Game 2011)
-
Use lower precision maths
-
Test often for precision artefacts (higher & lower)
-
Test often for precision artefacts (higher & lower)
-
Insert expensive maths between sampler reads (New Game 2011)
- Embrace the web: asynchronous shader compilation
WebGL Best Practices (MDN)
Performance Tips - Use WebGL 2.0!
- Many new texture formats
- 3D Textures / 2D Texture Arrays
- Full non-power-of-2 texture support
- Seamless cube maps
- Multiple Render Targets (critical for deferred rendering paths)
- Integer vertex attributes
- Multisampled renderbuffers
- Uniform Buffer Objects
- ... and many more!
Using Emscripten? Turn on WebGL 2.0 by using -s USE_WEBGL2=1
Check out the WebGL 2 Examples by Tarek Sherif:
State-of-the-Art WebGL 2.0 (SIGGRAPH Asia 2017)
Forward Rendering
Forward Rendering: pass geometry data into the vertex shader(s) for per-vertex operations, have these rasterized into fragments, do per-pixel operations via the fragment shader(s) before render.
- Render primitive by lighting it according to all light sources. Each primitive is passed down linearly.
- Heavy on performance, wastes a lot of fragment shader runs.
Deferred Rendering
Deferred Rendering: rendering is deferred until all primitives have been passed down the pipeline.
- Two stages: geometry pass for geometry data stored as in multiple draw buffers; lighting pass to calculate final shading and lighting calculations based on buffered data.
- Decouples expensive fragment processes to a later stage in the pipeline.
Deferred: Pros and Cons
Forward Rendering…
Pros
1. Polygon data are passed per vertex and fragment, drawn over the screen.
2. Preserves true transparency and provides anti-aliasing out of the box.
Cons
1. Linear, heavy on performance
2. Wasted fragment shader calculations
3. Limited number of lights available for rendering.
Deferred Rendering…
Pros
1. Actual fragment information as a screen pixel, only one fragment shader pass.
2. Great for dynamic lights and optimizing for a larger amount of light sources and post-processing.
Cons
1. Requires high bandwidth, memory.
2. Requires transparency, anti-aliasing workarounds.
Deferred: Non-WebGL Cases
-
Real-time lighting via light linked list
-
Physically correct lighting models
-
SPU-based deferred shading
-
Extensive particle systems
-
Post-processing effects (bloom, FXAA, AO)
- Destiny [SIGGRAPH 2013]
- Early History of Deferred Shading and
- Lighting [Rich Geldreich]
- Deferred Lighting Approaches [RTR 2009]
- Sunset Overdrive [SIGGRAPH 2014]
- Battlefield Series [EA DICE GDC 2011]
- Uncharted 4 [SIGGRAPH 2016]
- Killzone Shadowfall [Guerrilla Games]
Multiple Draw Buffers
-
Available as WebGL 1.0 as extension, now core
-
Allow rendering to multiple framebuffers in one pass
-
Big performance boost for:
-
Deferred rendering
-
Multipass post-processing effects
-
Other techniques that use depth, color, normals, and so on into multiple buffers
-
-
Applications of Multiple Draw Buffers
Volumetrics: Non-WebGL Cases
-
Occluded scattering resulting in volumetric shadows and light shafts
-
Heterogeneous participating media
-
Sun/Particle interaction, physically-based sky and atmosphere simulation, large scale fog
-
Art directable clouds for Horizon Zero Dawn, Pixar’s The Good Dinosaur
-
- Uncharted 4: A Thief’s End [Advances 2016]
- Call of Duty: Black Ops 3 [Advances 2016]
- Horizon Zero Dawn [Advances 2015]
- Frostbite Engine [Advances 2015]
- Pixar’s The Good Dinosaur [ACM DL]
Volumetrics: Non-WebGL Cases
-
Volumetric materials for character pipeline.
Volumetrics: WebGL Examples
-
Volumetric rendering simulates the propagation of light rays into a material’s volume (a.k.a a participating media), which allow for a broad range of rendering effects.
-
Volume ray marching is a technique where a ray is shot/cast from the camera into the material’s volume for rendering calculations.
Images: Nop Jiarathanakul, Almar Klein
Shading Language Updates
First line always starts with #version 300 es
Compiler needs to know if you’re using the new version.
Some of the new features:
-
- texture(sampler, uv).[rgba]
flat/smooth Interpolators
Flexible loop lengths in GLSL
Dynamic indexing
See GLSL ES 3.00 Specification for the complete feature set.
Images: Shuai Shao, Trung Le, Patrick Cozzi
State of Industry Survey
State of: Libraries & Frameworks
-
Three.js: r95 supports WebGL 2.0
-
Babylon.js: Microsoft's real-time 3D engine
-
PicoGL.js: minimal WebGL 2 rendering library
-
Unity/Unreal: uses Emscripten to port C++ code
-
PlayCanvas: Commercial game engine, supports most WebGL 2.0 feature
-
Shadertoy: Supports GLSL 3.00 fragment shader with WebGL 2
-
Blend4Web: Blender-interoperable engine, supports most WebGL 2.0 features
-
Verge3D: Blend4Web's founders' new framework
-
A-Frame: Open-sourced WebVR framework
-
Whitestorm.js: based on Three.js that simplify code, adds physics, and post-effects
State of: WebGL 2.0 Games
After the Flood [article][video]
- Procedural clouds (3D textures)
- Procedural water ripples/reflections
- Animated trees
- Dynamic lights
- Leaf particle system
- Alpha to coverage (antialiased foliage)
- HDR rendering MSAA for correct blending of antialiased HDR surfaces.
- Hardware PCF for shadow filtering
- Compressed textures (DXT, PVR, ETC1)
- Asynchronous asset streaming
- Runtime lightmap baking
- Planar mirrors for mirrored surfaces
State of: WebGL 2.0 Games
-
Voxels_WebGL2: Notch's current project [link]
-
Voxel engine running at 99fps
-
60+km view distance
-
Real-time terrain generation
-
Modifiable voxels at runtime
-
State of: WebGL 2.0 Games
State of: Media
-
Facebook 3D Photos and Posts [link]
-
SketchFab 3D Remix Challenge [link]
-
Just A Reflektor (2013), Arcade Fire + Google Data Arts [link]
-
Project SUN (2017), Philip Schütte + Venice Art Projects [link]
-
Stream (2016), Ezra Miller + Houston's Day for Night Festival [link]
-
HTC Vive Projection Mapping in WebVR (2016), Hien Huynh + Marpi @ DanceHackDay [link]
-
Virtual Art Sessions, Google Tilt Brush team [link]
-
Codevember Examples (2018), Jaume Sanchez Elias [link]
-
ShaderToy Examples (2018), Inigo Quilez [link]
State of: Data Visualization
-
Data visualisation of > 2mil datasets at 60fps.
-
Cesium: 3D Geospatial Mapping & Analysis
-
Web-based visualisation for time-dynamic autonomous car simulation [link]
-
-
Regl: Declarative and stateless WebGL
-
Pixi.js: Fast, flexible, 2D WebGL renderer
-
How Uber Uses Psychological Tricks to Push Its Drivers’ Buttons, New York Times [link]
-
-
Luma.gl: GPU-powered data visualisation/computation
-
Deck.gl: Large-scale data visualisation
-
Three.js: High-level scene graph
-
Bear 71 by Jam3 [link]
-
State of: Data Visualization
-
CryptoVoxels: Blockchain visualisation [link]
-
Illuminated City: Crime data visualisation [link]
-
Healthcare visualization
-
Experience Insight: NASA's JPL visualisation of the Mars Rover Insight [link]
-
TensorFlow.js: In-browser machine learning that is deployed and executed with WebGL2 [link]
-
Punk Office: 3D Avatars on the Web [link]
-
RDF Graphs visualisation: Visualisation of Global Cargo Ships [link]
-
Google Earth VR: Virtual reconstruction of Earth [link]
-
Shipmap: Visualisation of Global Cargo Ships [link]
-
D3.js and Three.js: GeoJSON interoperation [link]
State of: WebXR
Device adaptations are growing (96% for WebGL 1.0, 37% for WebGL 2.0, and growing).
WebVR Specification v2 Draft: [link], slated for Q3 2018 recommendation
Firefox v55 shipped WebVR by default (8th August, 2017)
Image: Don McCurdy
VR/AR: Non-WebGL Cases
-
Different setups
-
Mobile XR (Oculus Go/Quest, ARCore, ARKit)
-
Room-scale XR (Oculus Rift, Vive, Hololens, Magic Leap)
-
Full body tracking (HoloSuit)
-
-
Cinematic and/or narrative experiences
-
Interaction/Gaming Experiences
-
Virtual training/simulation
VR/AR: Non-WebGL Cases
-
Art
-
Visualization
-
Media
-
Medical/Health
VR/AR: Non-WebGL Cases
-
Academic Research
-
Putting yourself in the skin of a black avatar reduces implicit racial bias, Consciousness and Cognition (Vol 22, Issue 3; August 2013).
-
First Person Experience of Body Transfer in Virtual Reality, PLOS One (May 2010)
-
A Survey of Augmented Reality, Foundations and Trends in Human-Computer Interaction (Vol 8, Issue 2-3, March 2015)
-
Headset “Removal” for Virtual and Mixed Reality, Google Research and Daydream Labs (2017)
-
The Coming Age of Empathic Computing, Medium (2017)
-
-
Industry Standardization
-
OpenXR, Khronos Group
-
Why WebXR?
-
WebXR important part of ecosystem
-
Benefits of WebGL for widespread cross-compatibility and adoption
-
Works on every platform. WebVR specification does not distinguish what device you use.
-
-
WebVR becoming more readily available:
-
Firefox shipped WebXR for everyone for v55 (August 8th, 2017)
-
Image: SantaWinsAgain
WebXR Frameworks
-
Frameworks include:
-
WebVR, Google’s JavaScript API for creating immersive VR experiences [demo]
-
A-Frame: Make WebVR with HTML and Entity-Component. [demo]
-
JSARtoolkit: Emscripten port of ARToolKit to JavaScript. [demo]
-
Argon.js: Adding augmented reality content to web applications [demo]
-
Awe.js: Mixed reality on the web [link]
-
Image: Mozilla
WebXR: It’s Happening
-
VR Web Browser
-
WebVR NYC Snow Globe, Ronik Design
-
Broken Night VR, Eko
-
WebScreenVR, Samuel Cadillo
-
WebVR experiments, Google
-
Google I/O 2017 Prototype, Wayfair [Github]
-
Posture Recognition using Kinect, Azure IoT, ML and WebVR, Microsoft
-
BandExplorerVR, The Tin
-
Virtuleap WebVR Hackathon
WebGL Futures
What’s Coming to WebGL?
-
WEBGL_multiview - significant for WebVR performance
-
KHR_parallel_shader_compile - speed up experience initialization
-
EXT_float_blend - blending with 32-bit buffers
- EXT_texture_compression_bptc, EXT_texture_compression_rgtc - visually lossless texture compression
What’s Coming to WebGL? (cont.)
-
Compute Shader - WebGL 2.0 Compute
-
WebGPU
-
Geometry Shader (via ANGLE) - Intel is working on it 😻
-
Web Machine Learning - Depends on Compute Shader / WebGPU
- ... more features from OpenGL ES 3.1
Image: Khronos Group
In Closing
WebGL 2.0 is growing fast -
we may hit 65%+ adoption next year! *
WebGL 1.0 already universal -
as seen in the state of industry survey
2018 has been a year of complementary Web API developments for WebGL.
2019 will hopefully see more WebGL extensions moving into implementation.
* based on linear projection, WebGL 2.0 may be universally supported by 2022.
Session slides available at
Acknowledgements & Thanks
Neil Trevett
Emily Stearns
Angela Cheng
Kris Rose
Takashi Umezawa
Hitoshi Kasai
Kenneth Russell
Zhenyao Mo
Kai Ninomiya
Brandon Jones
Ricardo Cabello
Diego Marcos
Don McCurdy
Kevin Ngo
Tomasz Bednarz
Felix Maya
Jaume Sanchez Elias
Leonardo Carrion
Tarek Sherif
...and more!
Pol Jeremias
Inigo Quilez
Yang Gu
Jie Chen
Jeff Lang
Olli Etuaho
Gregg Tavares
Damon Hernandez
Will Eastcott
Josh Carpenter
Iker Jamardo
Google WebXR Team
MDN Contributors
Emscripten Team
Samsung Developers
WebGL: Latest Techniques | Khornos BOF, SIGGRAPH Asia 2018
By Xavier Ho
WebGL: Latest Techniques | Khornos BOF, SIGGRAPH Asia 2018
SIGGRAPH Asia 2018 Khronos BOF
- 2,652