How's it Work?

• USB webcam
• USB thumb drive
• Pushbutton, RGB LED
• JavaScript
• Johnny-Five!

How's it Work?

• `config.js`
• `av.js`
• `tweet.js`
• `sunriseMachine.js`
• `Recording.js`

module.exports = {
  autoMode           : 'sunrise', // Any property from `suncalc` or `false` to disable
  lat                : 43.34, // Your latitude (This is Vermont)
  long               : -72.64, // Your longitude (This is Vermont)
  postToTwitter      : false, // `true` will create posts of GIFs
  tweetBody          : 'It is a beautiful day!', // Text tweeted with GIFs
  captureFrequency   : 30000, // in ms (default one image every 30s)
  captureCount       : 20, // number of stills per montage
  calibrateCamera    : true, // calibrate camera exposure before shots
  calibrateLength    : 3000, // in ms (default 3 seconds)
  basedir            : '/mnt/sda1/captures', // where files get stored
  standbyColor       : '#0000ff', // LED color when no scheduled recording
  scheduledColor     : '#00ff00', // LED color when auto-recording is scheduled
  recordingColor     : '#ff0000', // LED color when recording session active
  consumer_key       : '', // Twitter
  consumer_secret    : '', // Twitter
  access_token_key   : '', // Twitter
  access_token_secret: '' // Twitter
};

Config

Default Behavior

• Auto-film the sunrise
• No Tweeting

Making Films

{
  captureFrequency   : 30000, // in ms (default one image every 30s)
  captureCount       : 20, // number of stills per montage
  calibrateCamera    : true, // calibrate camera exposure before shots
  calibrateLength    : 3000, // in ms (default 3 seconds)
  basedir            : '/mnt/sda1/captures', // where files get stored
  /* ... */
}

It's coming out all white

Making Films: The Nitty-Gritty

• Tessel 2: 64MB RAM
• Max resolution of capture: 320x240
• Following in the intelligent footsteps of @rwaldron and `tessel-av`
• Spawning child processes of `ffmpeg`

Tweeting Films: The Journey

• npm `twitter` module capable of supporting Twitter's Media API, but no precedent
• Bug in `twitter` (now fixed! Thanks @reconbot) considered non-200 HTTP 2xx responses as errors
• Uploading videos to Twitter is 3-step process
• And then...it turns out...

Tweeting Films: The Ugh

• After all that, FINALIZE media/upload API call was returning 400 (Invalid Content)
• Twitter will only accept h.264-encoded video
• OpenWRT build of `ffmpeg` doesn't have an h.264 encoder
• Now what?!

Animated GIFs, that's wut.

• `ffmpeg` can create animated GIFs
• Twitter will take animated GIFs
• But then...

Oh, for goodness' sake

<--- Last few GCs --->

   76889 ms: Mark-sweep 10.1 (18.5) -> 9.6 (18.5) MB, 431.5 / 0 ms [allocation failure] [GC in old space requested].
   77308 ms: Mark-sweep 9.8 (18.5) -> 9.3 (18.5) MB, 418.4 / 0 ms [allocation failure] [GC in old space requested].
   77728 ms: Mark-sweep 9.5 (18.5) -> 9.3 (18.5) MB, 419.8 / 0 ms [last resort gc].
   78122 ms: Mark-sweep 9.3 (18.5) -> 8.9 (18.5) MB, 393.8 / 0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x4ac24d59 <JS Object>
    1: /* anonymous */(aka /* anonymous */) [vm.js:39] [pc=0x5cbc09dc] (this=0x4ac08099 <undefined>,code=0x3c1598f1 <String[5]: Debug>)
    2: ensureDebugIsInitialized(aka ensureDebugIsInitialized) [util.js:194] [pc=0x5cbc07d4] (this=0x4ac08099 <undefined>)
    3: inspectPromise(aka inspectPromise) [util.js:200] [pc=0x5cbc0304] (this=0x4ac08099 <undefined>,p=0x3aaa675d <an Object with map 0x4a08333...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
Aborted

This is where it got hairy

• I assumed (wrongly) that `ffmpeg` was killing memory
• Then I assumed (wrongly) that uploading the file all in one chunk was killing memory
• Then I thought maybe some nested Promise was breaking everything...
• So I refactored all of my Twitter API code to be terrible...

Memory Crash

• Comes from logging `response` or `error` objects
• That means I can't log `error`s as they come back from `twitter` directly.
• So I refactored back to Promises.

ffmpeg

makes me feel dum

For some it's so easy

• Argument order matters
• There about fifty hundred things it can do
• It's very powerful
• It's very complicated

USB Cameras: The reckoning

/**
 * "Calibrate" a camera for a few seconds by letting it output to /dev/null
 */
module.exports.calibrate = function () {
  // ffmpeg -y -input_format yuyv422 -r 15 -i /dev/video0 -vframes 45 -f mp4 /dev/null
  var videoArgs = [];
  videoArgs.push('-y');
  videoArgs.push('-s', '320x240'); // Dimensions, respecting limited T2 memory
  videoArgs.push('-input_format', 'yuyv422'); // Native raw video format
  videoArgs.push('-r', '15'); // framerate
  videoArgs.push('-i', '/dev/video0'); // path to USB camera
  videoArgs.push('-vframes', '45'); // total frames to capture
  videoArgs.push('-f', 'mp4'); // Output format
  return ffmpeg(videoArgs, '/dev/null');
};

Capturing Stills

/**
 * Capture a single still image at 320x240 from the attached USB camera
 */
module.exports.captureStill = function (filepath) {
  // ffmpeg -y -v fatal -i /dev/video0 -s 320x240 -r 15 -q:v 2 -vframes 1 <filepath>
  var captureArgs = [];
  captureArgs.push('-s', '320x240'); // Dimensions, respecting limited T2 memory
  captureArgs.push('-r', '15'); // path to USB camera
  captureArgs.push('-i', '/dev/video0'); // path to USB camera
  captureArgs.push('-q:v', '2'); // JPG quality (1-31 where 1 is best)
  captureArgs.push('-vframes', '1'); // Total number of frames to capture
  return ffmpeg(captureArgs, filepath);
};

Making Movies

/**
 * Build an MP4 video from a collection of JPGs indicated by `glob`
 */
module.exports.videoFromStills = function (glob, outfile) {
  // ffmpeg -y -v fatal -s 320x240 -c:v h264 -f image2 -pattern_type \
  //   glob -framerate 6 -i capture*.jpg movie.mp4
  var stillArgs = [];
  stillArgs.push('-s', '320x240'); // Dimensions, respecting limited T2 memory
  stillArgs.push('-f', 'image2'); // FROM this input format...
  stillArgs.push('-pattern_type', 'glob'); // Use a glob to identify input
  stillArgs.push('-framerate', 6); // Framerate/frames per second
  stillArgs.push('-i', glob); // use this glob to find input files
  return ffmpeg(stillArgs, outfile);
};

Making GIFs

/**
 * Create an animated GIF from an MP4 video.
 */
module.exports.animatedGIFFromVideo = function (videofile, outfile) {
  // ffmpeg  -i video.mp4 -vf fps=6,scale=320:-1:flags=lanczos,palettegen palette.png
  var paletteArgs = [];
  paletteArgs.push('-y');
  paletteArgs.push('-i', videofile);
  paletteArgs.push('-vf');
  paletteArgs.push('fps=6,scale=320:-1:flags=lanczos,palettegen');
  return ffmpeg(paletteArgs, '/tmp/palette.png').then(paletteFile => {
    // ffmpeg -i video.mp4 -i palette.png -lavfi paletteuse output.gif
    console.log('made it back with ', paletteFile);
    var gifArgs = [];
    gifArgs.push('-y');
    gifArgs.push('-i', videofile);
    gifArgs.push('-i', paletteFile);
    gifArgs.push('-lavfi', 'paletteuse');
    return ffmpeg(gifArgs, outfile);
  });
};

Enclosure

• Finding an old book to sacrifice
• I'm terrible at soldering
• Didn't use long enough wires
• Coiling USB cable from camera atop T2 harshes WiFi reception

A speed prototype!

A speed prototype!

What's next.

• feature-sunrise-machine branch of j5ik
• It doesn't work offline (yet)
• Everything is in GMT
• Power consumption?
• Long-running process stability

The Sunrise Machine