Nodebots: Detecting Gestures with Node.js & Kinect

Why?

  • Worked with NodeBots for 1+ year
  • Wanted to explore the possibilities of Node.js
  • Because controlling a robot with a Kinect is pretty cool

The goals

  • Needed a way to use events to tell the robot to perform an action
  • Gestures seemed the best way to achieve that

What currently exists

Text

Basic Nodebot stack

  • node.js
  • socket.io
  • johnny-five

node-openni

  • A way to connect the Kinect to node.js
  • Very basic event for almost everything

What I needed

  • Support for a "skeleton"
  • Support for gestures

Gestures 101

  • Is the initial condition met ?
  • Is the condition still met ?
  • Is the final condition met ?

What do I need for gestures?

  • A Skeleton with an event when it changes
  • Know the position of the COM
  • Have a base unit

Introducing node-kinect

  • Gesture detection using a base class
  • Tracks the skeleton for an initial condition
  • Checks every 100 ms to see if the condition is still met
  • If the final condition is met trigger an event
  • Other classes are used to define the actual gestures

Code time !

Next few steps

  • Return a skeleton with easier coordinates
  • Create a npm package
  • Add more gestures

About me

@joel__lord

http://github.com/joellord

jlord@macadamian.com