Panorama video with webGL

The Task

building an interactive panorama video experience, where users can navigate

+ Using videos, not just images.

++ We must deal with tablets as well!

Source videos

prototype before the shooting

shot with POSSIBLE London

test the raw videos on the spot

Spherical panorama

The equipment

6 separated wide

angled cameras

Ladybug 3 SDK

The tech solution

WebGL - using three.js library

Create a sphere -> add video as a texture

video = document.createElement('video');
video.src = 'testvideo.mp4';
videoTexture = new THREE.Texture(video);

geometry = new THREE.SphereGeometry(500,60,40);
material = new THREE.MeshBasicMateial({
   map = videoTexture
});
videoMesh = new THREE.Mesh(geometry, material);
scene.add(videoMesh).

function render() {
 ...
   videoTexture.needsUpdate = true;
...
}

Webgl support

iPad gyroscope

using ondeviceorientation!

iPad gyroscope

window.ondeviceorientation = function (event) {
    var alpha = (event.alpha),
	beta = (event.beta),
	gamma = (event.gamma);

    if (window.orientation == 0) {
	lon = gamma;
	lat = beta - 90;
    } else if (window.orientation == 90) {
	lon = -alpha;
	lat = -gamma - 90;
    } else if (window.orientation == -90) {
	lon = -alpha;
	lat = gamma - 90;
    }
}

Get Orientation Datas:

iPad gyroscope

function render() {
    lat = Math.max(-85, Math.min(85, lat));
    phi = THREE.Math.degToRad(90 - lat);
    theta = THREE.Math.degToRad(lon);

    camera.target.x = 500 * Math.sin(phi) * Math.cos(theta);
    camera.target.y = 500 * Math.cos(phi);
    camera.target.z = 500 * Math.sin(phi) * Math.sin(theta);

    camera.lookAt(camera.target);

    videoTexture.needsUpdate = true;
    renderer.render(scene, camera);
}

Add some code to the render...

++ 3D audio API

(in nutshells)

3d sound in WebGL

AudioPannerNode

define position / orientation / velocity of the sound.

html5rocks Mixing Positional Audio and WebGL & W3C WebAudioAPI

Thank you!

http://hozeee.github.io/

jozsef.szabadszallasi@possible.com