Kinect

XBox + Kinect

face recognition

and

emotion analysis

 

 

 

Source Frome Goran Djonovic

WHAT IS KINECT?

 

Depth data

 

Kinect for xBox 360 SDKs (before 1.5) was

All about skeletal data

Kinect SDK 1.5
(Kinect for Windows)

  • Kinect studio (praise the lord)
  • Human interface guidelines
  • Face tracking SDK

  • Samples and Doc (again praise the lord)
  • Improved skeleton tracking
  • Performance and data quality enhancements
  • The Joint Orientation (Hierarchical Rotation
    Absolute Orientation)
  • Speech recognition improvement

 

 

 

FACE RECOGNITION

and analyzing

How to start

  • Download Kinect SDK 1.5 and development toolkit

You will get

  • FaceTrackData.dll
  • FaceTrackLib.dll


C++ application can directly use the face tracking libraries

In the other case, you should use a wrapper

Starting projects

Use:

  • Microsoft.Kinect.Toolkit
  • Microsoft.Kinect.Toolkit.FaceTracking

Initialize Sensor and face tracking

kinectSensor.ColorStream.Enable();
                
kinectSensor.DepthStream.Range = DepthRange.Near;
kinectSensor.DepthStream.Enable(rez80X60);
kinectSensor.SkeletonStream.EnableTrackingInNearRange = true;
kinectSensor.SkeletonStream.TrackingMode = SkeletonTrackingMode.Seated;
skeletonData = new Skeleton[6];
kinectSensor.Start();
faceTracker = new FaceTracker(kinectSensor);
var skeleton = skeletonData.FirstOrDefault(
                
s => s.TrackingState == SkeletonTrackingState.Tracked);
if (skeleton == null)
    return;
// only track face if all data are there


FaceTrackFrame faceFrame = faceTracker.Track(
kinectSensor.ColorStream.Format, colorPixelData,
kinectSensor.DepthStream.Format, depthPixelData,
skeleton);

Get3DShape()

This method will return 87 face  points in 3D

Emotion analyzer

First there were hits/clicks, then we got likes, but now we can have true emotions.

GetProjected3DShape()

returns the same collection but in a 2D referential corresponding to a 640×480 pixels image.

GetTriangles()

is a method that you can use in association with the Get3DShape method. It will return you a collection of triangles.

Get Animation Unit Coefficients()

 

returns the values

describing the facial expressions.

Jaw lowerer

Lip Corner Depressor

Upper Lip raiser

Lip Stretcher

Outer Brow Raiser

Neutral face

Let's talk about 

 

 

IDEAS

 

BUT FIRST

Be aware of technical abilities and limitations

Guidelines

        Match your user/client needs and mindset.