Kinect

XBox + Kinect

face recognition
and
emotion analysis
Source Frome Goran Djonovic
WHAT IS KINECT?

Depth data

Kinect for xBox 360 SDKs (before 1.5) was
All about skeletal data
Kinect SDK 1.5
(Kinect for Windows)
- Kinect studio (praise the lord)
- Human interface guidelines
-
Face tracking SDK
- Samples and Doc (again praise the lord)
- Improved skeleton tracking
- Performance and data quality enhancements
- The Joint Orientation (Hierarchical Rotation
Absolute Orientation) - Speech recognition improvement

FACE RECOGNITION
and analyzing
How to start
- Download Kinect SDK 1.5 and development toolkit
You will get
- FaceTrackData.dll
- FaceTrackLib.dll
C++ application can directly use the face tracking libraries
In the other case, you should use a wrapper
Starting projects
Use:
- Microsoft.Kinect.Toolkit
- Microsoft.Kinect.Toolkit.FaceTracking
Initialize Sensor and face tracking
kinectSensor.ColorStream.Enable();
kinectSensor.DepthStream.Range = DepthRange.Near;
kinectSensor.DepthStream.Enable(rez80X60);
kinectSensor.SkeletonStream.EnableTrackingInNearRange = true;
kinectSensor.SkeletonStream.TrackingMode = SkeletonTrackingMode.Seated;
skeletonData = new Skeleton[6];
kinectSensor.Start();
faceTracker = new FaceTracker(kinectSensor);
var skeleton = skeletonData.FirstOrDefault(
s => s.TrackingState == SkeletonTrackingState.Tracked);
if (skeleton == null)
return;
// only track face if all data are there
FaceTrackFrame faceFrame = faceTracker.Track(
kinectSensor.ColorStream.Format, colorPixelData,
kinectSensor.DepthStream.Format, depthPixelData,
skeleton);
Get3DShape()
This method will return 87 face points in 3D
Emotion analyzer
First there were hits/clicks, then we got likes, but now we can have true emotions.
GetProjected3DShape()
returns the same collection but in a 2D referential corresponding to a 640×480 pixels image.
GetTriangles()
is a method that you can use in association with the Get3DShape method. It will return you a collection of triangles.
Get Animation Unit Coefficients()
returns the values
describing the facial expressions.
Jaw lowerer
Lip Corner Depressor
Upper Lip raiser
Lip Stretcher
Outer Brow Raiser
Neutral face
Let's talk about
IDEAS
BUT FIRST
Be aware of technical abilities and limitations
Guidelines
Match your user/client needs and mindset.


