KINECT

face recognition and  emotion analysis

Goran Djonovic

my MISSION

Explain Kinect

EXPLAIN TEChnology

Talk about face recognition

Share ideas

Explain Kinect

WHAT IS KINECT?

Explain kinect

Explain Kinect

EXPLAIN KINECT

Depth data

EXPLAIN KINECT

KINECT SKDs:

1. Kinect SDK 1.0
2. Kinect SDK Beta
3. Kinect SDK Beta 2
4. Kinect SDK 1.5 (1.5.1, 1.5.2)
   
5. Kinect SKD 1.6
   

eXPLAIN KINECT

Kinect for xBox 360 SDKs (before 1.5) was

All about skeletal data

Explain Kinect

EXPLAIN Kinect

Kinect for xBox 360 SDKs (before 1.5)

• Track two users
• On distance from 0.8 to 2 meters
  
• No gesture support
• No near mode
• No face recognition
• Academic use only

EXPLAIN Kinect

Kinect SDK 1.5 (Kinect for Windows, February 21)

AND NOW IT'S COOL

EXPLAIN Kinect

Kinect SDK 1.5 (Kinect for Windows)

• Kinect studio (praise the lord)
• Human interface guidelines

• Face tracking SDK

• Samples and Doc (again praise the lord)
• Improved skeleton tracking 
• Performance and data quality enhancements
• The Joint Orientation (Hierarchical Rotation
  Absolute Orientation)
• Speech recognition improvement

explain KINECT

Explain Kinect

IS KINECT 

STILL A MYSTERY?

FACE RECOGNITION

How to start

• Download Kinect SDK 1.5 and development toolkit

You will get

• FaceTrackData.dll
• FaceTrackLib.dll

C++ application can directly use the face tracking libraries

In the other case, you should use a wrapper

Starting projects

Use:

Initialize Sensor and face tracking

kinectSensor.ColorStream.Enable();
kinectSensor.DepthStream.Range = DepthRange.Near;
kinectSensor.DepthStream.Enable(rez80X60);
kinectSensor.SkeletonStream.EnableTrackingInNearRange = true;
kinectSensor.SkeletonStream.TrackingMode = SkeletonTrackingMode.Seated;
skeletonData = new Skeleton[6];
kinectSensor.Start();
faceTracker = new FaceTracker(kinectSensor);

var skeleton = skeletonData.FirstOrDefault(
s => s.TrackingState == SkeletonTrackingState.Tracked);
if (skeleton == null)
    return;
// only track face if all data are there


FaceTrackFrame faceFrame = faceTracker.Track(
kinectSensor.ColorStream.Format, colorPixelData,
kinectSensor.DepthStream.Format, depthPixelData,
skeleton);

Get3DShape()

This method will return 87 face  points in 3D

Emotion analyzer

First there were hits/clicks, then we got likes, but now we can have true emotions.

GetProjected3DShape()

returns the same collection but in a 2D referential corresponding to a 640×480 pixels image.

GetTriangles()

is a method that you can use in association with the Get3DShape method. It will return you a collection of triangles.

Get Animation Unit Coefficients()

returns the values

describing the facial expressions.

Jaw lowerer

Lip Corner Depressor

Upper Lip raiser

Lip Stretcher

Outer Brow Raiser

Neutral face

Let's talk about 

IDEAS

Hegel: Absolute truth; the complete and ultimate product of reason.

BUT FIRST

Be aware of technical abilities and limitations

Guidelines

        Match your user/client needs and mindset.

(THE BEST) kinect event

TANDEM WORKSHOP

in Budapest

30 highly motivated students from 13 different countries

8 days

7 teams

TANDEM WORKSHOP BEST IDEA

IDEAS

product evaluation

market research

face ID

targeted marketing

....

Q&A

ANY QUESTIONS?

THANK YOU ALL