Péter Schmíz
Deputy CEO / CTO @ WTT Budapest
Morphing 3D objects with help of audio inputs
Sustainable Identities
Task: create a visualization which creates a unique identity and runs on tablet
Solution:
- WebGL (three.js)
- Web Audio API
- getUserMedia / WebRTC
3D part with three.js
- started with a sphere
- changed to icosahedron (has fancier name :D)
sphere
icosahedron
3D part with three.js
- add some material (MeshPhongMaterial)
- add some lights
3D part with three.js
And it's done :)
3D part with three.js
nope :)
How to create a blob out of the sphere?
f(x) = \int_{-\infty}^\infty
\hat f(\xi)\,e^{2 \pi i \xi x}
\,d\xi
3D part with three.js
We need something to move the vertex points,
something which is kind of random...
- vertex shaders?
- moving the actual vertex points?
- some "noise"?
3D part with three.js
PERLIN
NOISE
Perlin noise
"pseudorandom gradient vector"
"it seems random, but it isn't in reality"
Perlin noise
var pattern = [];
function generatePattern() {
var i = 0;
for (i; i < 256; i++) {
pattern[256 + i] = pattern[i];
}
}
function createRandomArray(elementNumber, min, max) {
var nums = [];
for (var element = 0; element < elementNumber; element++) {
nums[element] = randomNumber(min, max);
}
return nums;
}
function randomNumber(min, max) {
return (Math.round((max - min) * Math.random() + min));
}Perlin noise
while (l--) {
v = sphere.geometry.vertices[l];
n.copy(v);
n.normalize();
if (justCreation !== true) {
if (tempNoiseArray[l] === undefined) {
noise = APP.Render.Noise.noise(0.1 * v.x, 0.1 * v.y, 0.1 * v.z);
tempNoiseArray[l] = noise;
} else {
noise = tempNoiseArray[l];
}
noise = blobSteps * noise;
}
d = 8 + noise;
n.multiplyScalar(d);
v.copy(n);
}Perlin noise
...
A = pattern[X] + Y;
AA = pattern[A] + Z;
AB = pattern[A + 1] + Z;
B = pattern[X + 1] + Y;
BA = pattern[B] + Z;
BB = pattern[B + 1] + Z;
return lerp(w, lerp(v, lerp(u, grad(pattern[AA], x, y, z),
grad(pattern[BA], xMinus1, y, z)),
lerp(u, grad(pattern[AB], x, yMinus1, z),
grad(pattern[BB], xMinus1, yMinus1, z))),
lerp(v, lerp(u, grad(pattern[AA + 1], x, y, zMinus1),
grad(pattern[BA + 1], xMinus1, y, z - 1)),
lerp(u, grad(pattern[AB + 1], x, yMinus1, zMinus1),
grad(pattern[BB + 1], xMinus1, yMinus1, zMinus1))));Perlin noise
Demotimeee
Audio part
Now we have something we can use for generating blobs or potatoes.
We need an input method or a generator which will give us "numbers", so we can use them with our Perlin noise creator.
Audio part
-
access the microphone with Navigator.getUserMedia()
-
access the stream & data with Web Audio API
-
use analyser node to get frequency / volume datas
Audio part
Demotimeee
Dankeschön
Morphing 3D objects with help of audio inputs
By Péter Schmíz
Morphing 3D objects with help of audio inputs
Morhping 3D objects with the help of WebGL and Web Audio API. With the release of Android 5 now it’s possible to access the microphone on tablets. The practical use of this technique is showcased in this project.
