webVR
Martin Schuhfuss | @usefulthink
about myself
- creative developer @ubilabs
- javascript <3
- mostly WebGL, now WebVR
- spare-time: playing with LEDs
and professional lighting-equipment
WebVR,
what are you?
WebVR is..
a JS API to provide access to VR devices in your browser.
VR-Devices
-
VRDisplay: Head-Mounted Displays (HMD)
Oculus Rift, HTC Vive, Gear VR, Daydream, Cardboard, ... -
Tracked Controllers
Vive Controllers, Oculus Touch, Daydream Controller...
VR-Devices
- independent image per eye
(resolution in the range of 1024x1024) - 90° to 120° field of view
- 6DOF Tracking: position and orientation in real-world space
VRDisplay
- mostly just Gamepad-API
- added 6DOF Tracking
VRControllers
WebVR is all about handling these VR-Devices
- get current position / orientation
- get rendering-properties (lens-distortion, frame-size, IPD, ..)
- frame-timing (requestAnimationFrame())
- setup the source and submit frames
WebVR is not about rendering
that is left entirely to WebGL.
(as of now, the only source a VRDisplay can present from is a <canvas>-Element, others – like <video> – are being discussed)
show me the codes!
get a VRDisplay
function getVRDisplay() {
return navigator.getVRDisplays().then(displays => {
if (displays.length < 1) {
return Promise.reject(
new Error('no displays found'));
}
return displays[0];
});
}
start rendering to VRDevice
vrLinkElement.addEventListener('click', ev => {
ev.preventDefault();
// configure canvas according to vrDevice.getEyeParameters()
vrDevice.requestPresent([{ source: canvasElement }])
.then(() => vrDevice.requestAnimationFrame(renderloop));
});
(requestPresent must be called in response to a user gesture)
the renderloop
const frameData = new VRFrameData();
function renderloop() {
vrDisplay.getFrameData(frameData);
// render to canvas using camera-data from
// frameData
vrDevice.submitFrame();
vrDevice.requestAnimationFrame(renderloop);
});
what about this 'frameData'?
// calling:
vrDisplay.getFrameData(frameData);
// ...updates these 4x4-matrices you'll
// need for rendering:
// per-eye camera projection-matrix
frameData.leftProjectionMatrix;
frameData.rightProjectionMatrix;
// per-eye world-to-view transform-matrix
frameData.leftViewMatrix;
frameData.rightViewMatrix;
(for 3D-insiders: matrices are provided as Float32Array, compatible with gl.uniformMatrix4fv())
ok, and 'eyeParameters'?
const lParams = vrDisplay.getEyeParameters('left');
const rParams = vrDisplay.getEyeParameters('right');
canvas.width = Math.max(
lParams.renderWidth, rParams.renderWidth) * 2;
canvas.height = Math.max(
lParams.renderHeight, rParams.renderHeight);
// also contains the positions of the eyes relative
// to the headset center (both Float32Array):
lParams.offset;
rParams.offset;
then there is the 'pose'
// the pose contains aggregated data from the
// 6DOF-sensors
const pose = vrDisplay.getPose();
// all of the properties are vectors as Float32Arrays
pose.position;
pose.linearVelocity;
pose.linearAcceleration;
pose.orientation;
pose.angularVelocity;
pose.angularAcceleration;
// can be "reset" to set the point-of-origin
// for position and orientation
vrDisplay.resetPose();
and finally, something called 'sittingToStandingTransform'
const pose = vrDisplay.getPose();
// all positions are given in so-called "sitting space",
// for room-scale VR coordinates need to be transformed
// into real-world-space
realWorldPos = matrixMultiply(
pose.position,
vrDisplay.stageParameters.sittingToStandingTransform);
VR-controllers
const controller = navigator.getGamepads().item(1);
// VR-controllers have an additional pose like the HMD
controller.pose.position;
controller.pose.orientation;
// ...and vibration, dubbed "haptic actuators"
controller.hapticActuators[0]
.pulse(intensity, duration);
// the rest is normal gamepad-stuff:
controller.axes[0];
controller.buttons[1].pressed;
however, you won't see all that very often.
frameworks and libraries
three.js
const renderer = new THREE.WebGLRenderer(...);
const scene = new THREE.Scene();
const camera = new THREE.PerspectiveCamera(...);
// ... add objects to the scene
function loop() {
// ... update the scene
renderer.render(scene, camera);
requestAnimationFrame(loop);
}
the basic structure of any THREE.js application (without any WebVR).
three.js
// renderer, scene, camera initialized as before
const vrEffect = new THREE.VREffect(renderer);
const vrControls = new THREE.VRControls(camera);
function loop() {
vrControls.update();
// ... update the scene
vrEffect.render(scene, camera);
vrEffect.getVRDisplay().requestAnimationFrame(loop);
}
document.addEventListener('click', () => {
vrEffect.requestPresent().then(() => {
vrEffect.getVRDisplay().requestAnimationFrame(loop);
});
});
... and with WebVR
a-frame
- built by the mozilla VR-team
- easiest way to author WebVR-pages
- easy to extend, built based on three.js
- awesome developer-tools included <3
a-frame
<html>
<head>
<script src="https://aframe.io/releases/0.4.0/aframe.min.js"></script>
</head>
<body>
<a-scene>
<a-box color="#6173F4" opacity="0.8" depth="2"></a-box>
<a-sphere radius="2" src="texture.png" position="1 1 0"></a-sphere>
<a-sky color="#ECECEC"></a-sky>
</a-scene>
</body>
</html>
one of the demo-applications for a-frame.
ReactVR
- now that's quite awesome
- a bit similar to a-frame, but with react
- also built on top of three.js
(anyone else sees a pattern here?) - something with react-native coming?
exciting times ahead!
chrome
origin-trial in v56 (mobile), desktop in experimental builds
firefox
shipping in firefox-nightly and servo-builds
edge
currently in development
safari
well, fuck off safari
carmel
first dev-preview released last week
browser-support
windows
yes
linux
nope. not even steamOS.
mac
haha! no.
OS-Support
(that is current state of official support. There is the OSVR-project that tries to fix this)
carmel?
Why yes, that's an entirely new VR-first browser made by oculus for GearVR and (very likely) Oculus Rift
mid 2017
WebVR general availability:
excited? Read more:
Basic Info-page: https://webvr.info/
The WebVR Spec: https://w3c.github.io/webvr
WebFundamentals on WebVR: https://developers.google.com/.../vr/
Oculus Dev: https://developer3.oculus.com/.../carmel-intro/
Mozilla's VRTeam: https://mozvr.com/ and https://blog.mozvr.com/
that's all for now.
Martin Schuhfuss | @usefulthink
Slides available online:
hhjs-webvr
By Martin Schuhfuss
hhjs-webvr
- 1,524