Motion
Detection
in JavaScript
Paulo Ávila - 4 July 2013
BarcelonaJS
Who's Paulo?
Brazil > California > Bilbao > Barcelona
Information Architect - Waytostay
JavaScript developer for ~10 years
NOT expert in video or motion graphics
... nor <canvas>
Agenda
- Questions
- The Challenge (overview)
- Detect Movement
- Video to Canvas
- Canvas basics (context)
- Photoshop
- Detect Motion (direction)
-
Applications
The Challenge
Perform real-time image
stabilization in the browser
stabilization in the browser
- Counteract motion
- Determine motion direction
- Detect movement*
The Hope
Motion Tracking in Adobe After Effects
(credit: Andrew Kramer)
<video>2<canvas>
No access to image data in video tag
Paint each frame onto <canvas>
var srcVideo = document.getElementById('src-video'),
srcCanvas = document.createElement('canvas'),
srcCtx = srcCanvas.getContext('2d');
srcCanvas.width = srcVideo.width;
srcCanvas.height = srcVideo.height;
function frameLoop() {
srcCtx.drawImage(srcVideo, 0, 0, srcVideo.width, srcVideo.height);
setTimeout(frameLoop, 1000 / 40);
}
<canvas>
context = canvas.getContext('2d');
context.getImageData(x, y, w, h);
context.createImageData(w, h);
imageData.data
-
Returns an array containing pixel data
-
Each series of 4 elements represents a pixel (rgba)
- 4px by 3px grid
- length = (4 * 3) * 4 = 48
[
r,g,b,a, r,g,b,a, r,g,b,a, r,g,b,a,
r,g,b,a, r,g,b,a, r,g,b,a, r,g,b,a,
r,g,b,a, r,g,b,a, r,g,b,a, r,g,b,a
]
Detecting Movement
function frameLoop() {
// Create a blank image data where the blended image (diff) will be drawn
diffImageData = srcCtx.createImageData(canvasWidth, canvasHeight);
// Get image data from the image drawn on the source canvas.
srcFrameImageData = srcCtx.getImageData(0, 0, canvasWidth, canvasHeight);
// Create an image if the previous image doesn’t exist (1st iteration)
if (!prevFrameImageData) {
prevFrameImageData = srcFrameImageData;
}
movementPxCount = compareFrames(diffImageData,
srcFrameImageData,
prevFrameImageData);
// Store the current frame's image data for the next iteration.
prevFrameImageData = srcFrameImageData;
setTimeout(frameLoop, 1000 / 40);
}
Comparing Frames
// diff between 2 frames' luma to be considered as movement (a.k.a. sensitivity)
var LUMA_DIFF_THRESHOLD = 20;
function compareFrames(deltaImgData, imgData1, imgData2) {
var imgDataSubPixels1 = imgData1.data,
imgDataSubPixels2 = imgData2.data,
deltaImgDataSubPixels = deltaImgData.data,
subPixelCount = imgDataSubPixels1.length;
// For each sub-pixel channel (rgba) of every pixel from the image data...
while (i < subPixelCount) {
// Get the grayscale (achromatic) value of the pixel.
pxLuminance1 = luma(imgDataSubPixels1[i],
imgDataSubPixels1[i + 1],
imgDataSubPixels1[i + 2]);
pxLuminance2 = luma(imgDataSubPixels2[i],
imgDataSubPixels2[i + 1],
imgDataSubPixels2[i + 2]);
// Get the difference in brightness of both pixels.
pxLuminanceDiff = Math.abs(pxLuminance1 - pxLuminance2);
// If brightness difference is "significant", set it to white, else black.
if (pxLuminanceDiff >= LUMA_DIFF_THRESHOLD) {
pxBinaryDiff = 255;
diffPxCount++;
} else {
pxBinaryDiff = 0;
}
// Write the difference in luminosity to the blended canvas context.
// We write the same value to all color channels to show it in white since
// there's no noticeable performance difference if we only use one color.
deltaImgDataSubPixels[ i ] = pxBinaryDiff; // r
deltaImgDataSubPixels[i + 1] = pxBinaryDiff; // g
deltaImgDataSubPixels[i + 2] = pxBinaryDiff; // b
deltaImgDataSubPixels[i + 3] = 255; // a
// Advance to the next pixel (a set of rgba channels).
i = (i + 4);
}
return diffPxCount;
}
function luma(r, g, b) {
return (r + g + b) / 3;
//return (r+r+r + g+g+g+g+g+g + b) / 10;
}
Photoshop
Visual demo of what's going on...
-
Blend mode: Difference
- Compares two layers
- Looks at their pixel color information
- Subtracts the darker of the two from the other
Direction of Motion
-
Get number of pixels
-
Shift the previous frame to one direction
-
Get number of pixels
-
If fewer pixels when shifted
- direction of shift is the same as motion
-
else
- direction of motion is opposite as the shift
Repeat for y axis
Applications
- Suggestions?
- Green screen backdrop
-
Any background backdrop
- Area detection: Game (credit: Romuald Quantin)
-
Accessibility programs
Keep in touch!
Google+: http://goo.gl/XbFFd
GitHub: https://github.com/demoive
Twitter: @demoive
Motion Detection in JavaScript
By Paulo Ávila
Motion Detection in JavaScript
- 2,357