webVR

Martin Schuhfuss | @usefulthink

about myself

  • creative developer @ubilabs
  • javascript <3
  • mostly WebGL, now WebVR
  • spare-time: playing with LEDs
    and professional lighting-equipment

WebVR,
what are you?

WebVR is..

a JS API to provide access to VR devices in your browser.

VR-Devices

  • VRDisplay: Head-Mounted Displays (HMD)
    Oculus Rift, HTC Vive, Gear VR, Daydream, Cardboard, ...
  • Tracked Controllers 
    Vive Controllers, Oculus Touch, Daydream Controller...

VR-Devices

  • independent image per eye
    (resolution in the range of 1024x1024)
  • 90° to 120° field of view
  • 6DOF Tracking: position and orientation in real-world space

VRDisplay

  • mostly just Gamepad-API
  • added 6DOF Tracking

VRControllers

WebVR is all about handling these VR-Devices

  • get current position / orientation
  • get rendering-properties (lens-distortion, frame-size, IPD, ..)
  • frame-timing (requestAnimationFrame())
  • setup the source and submit frames

WebVR is not about rendering

that is left entirely to WebGL.

(as of now, the only source a VRDisplay can present from is a <canvas>-Element, others – like <video> – are being discussed)

show me the codes!

get a VRDisplay

function getVRDisplay() {
  return navigator.getVRDisplays().then(displays => {
    if (displays.length < 1) {
      return Promise.reject(
          new Error('no displays found'));
    }
  
    return displays[0];
  });
}

start rendering to VRDevice

vrLinkElement.addEventListener('click', ev => {
  ev.preventDefault();

  // configure canvas according to vrDevice.getEyeParameters()

  vrDevice.requestPresent([{ source: canvasElement }])
     .then(() => vrDevice.requestAnimationFrame(renderloop));
});

(requestPresent must be called in response to a user gesture)

the renderloop

const frameData = new VRFrameData();

function renderloop() {
  vrDisplay.getFrameData(frameData);

  // render to canvas using camera-data from 
  // frameData

  vrDevice.submitFrame();
  vrDevice.requestAnimationFrame(renderloop);
});

what about this 'frameData'?

// calling:
vrDisplay.getFrameData(frameData);

// ...updates these 4x4-matrices you'll 
//    need for rendering:

// per-eye camera projection-matrix
frameData.leftProjectionMatrix;
frameData.rightProjectionMatrix;

// per-eye world-to-view transform-matrix
frameData.leftViewMatrix;
frameData.rightViewMatrix;

(for 3D-insiders: matrices are provided as Float32Array, compatible with gl.uniformMatrix4fv())

ok, and 'eyeParameters'?


const lParams = vrDisplay.getEyeParameters('left');
const rParams = vrDisplay.getEyeParameters('right');

canvas.width = Math.max(
    lParams.renderWidth, rParams.renderWidth) * 2;
canvas.height = Math.max(
    lParams.renderHeight, rParams.renderHeight);

// also contains the positions of the eyes relative
// to the headset center (both Float32Array):
lParams.offset; 
rParams.offset;

then there is the 'pose'

// the pose contains aggregated data from the 
// 6DOF-sensors
const pose = vrDisplay.getPose();

// all of the properties are vectors as Float32Arrays
pose.position;
pose.linearVelocity;
pose.linearAcceleration;

pose.orientation;
pose.angularVelocity;
pose.angularAcceleration;

// can be "reset" to set the point-of-origin 
// for position and orientation
vrDisplay.resetPose();

and finally, something called 'sittingToStandingTransform'

const pose = vrDisplay.getPose();

// all positions are given in so-called "sitting space", 
// for room-scale VR coordinates need to be transformed
// into real-world-space 

realWorldPos = matrixMultiply(
    pose.position, 
    vrDisplay.stageParameters.sittingToStandingTransform);

VR-controllers

const controller = navigator.getGamepads().item(1);

// VR-controllers have an additional pose like the HMD
controller.pose.position;
controller.pose.orientation;

// ...and vibration, dubbed "haptic actuators"
controller.hapticActuators[0]
  .pulse(intensity, duration);

// the rest is normal gamepad-stuff:
controller.axes[0];
controller.buttons[1].pressed;

however, you won't see all that very often.

frameworks and libraries

three.js

const renderer = new THREE.WebGLRenderer(...);
const scene = new THREE.Scene();
const camera = new THREE.PerspectiveCamera(...);

// ... add objects to the scene

function loop() {
  // ... update the scene

  renderer.render(scene, camera);
  requestAnimationFrame(loop);
}

the basic structure of any THREE.js application (without any WebVR).

three.js

// renderer, scene, camera initialized as before

const vrEffect = new THREE.VREffect(renderer);
const vrControls = new THREE.VRControls(camera);

function loop() {
  vrControls.update();

  // ... update the scene

  vrEffect.render(scene, camera);
  vrEffect.getVRDisplay().requestAnimationFrame(loop);
}

document.addEventListener('click', () => {
  vrEffect.requestPresent().then(() => {
    vrEffect.getVRDisplay().requestAnimationFrame(loop);
  });    
});

... and with WebVR

a-frame

  • built by the mozilla VR-team
  • easiest way to author WebVR-pages
  • easy to extend, built based on three.js
  • awesome developer-tools included <3

a-frame

<html>
  <head>
    <script src="https://aframe.io/releases/0.4.0/aframe.min.js"></script>
  </head>
  <body>
    <a-scene>
      <a-box color="#6173F4" opacity="0.8" depth="2"></a-box>
      <a-sphere radius="2" src="texture.png" position="1 1 0"></a-sphere>
      <a-sky color="#ECECEC"></a-sky>
    </a-scene>
  </body>
</html>

one of the demo-applications for a-frame.

ReactVR

  • now that's quite awesome
  • a bit similar to a-frame, but with react
  • also built on top of three.js
    (anyone else sees a pattern here?)
  • something with react-native coming?

exciting times ahead!

chrome

origin-trial in v56 (mobile), desktop in experimental builds

firefox

shipping in firefox-nightly and servo-builds

edge

currently in development

safari

well, fuck off safari

carmel

first dev-preview released last week

browser-support

windows

yes

linux

nope. not even steamOS.

mac

haha! no.

OS-Support

(that is current state of official support. There is the OSVR-project that tries to fix this)

carmel?

Why yes, that's an entirely new VR-first browser made by oculus for GearVR and (very likely) Oculus Rift

mid 2017

WebVR general availability: 

excited? Read more:

that's all for now.

Martin Schuhfuss | @usefulthink

Slides available online:

hhjs-webvr

By Martin Schuhfuss

hhjs-webvr

  • 1,524