Who AM I


Developer, developer, developer!

at EXECOM

MICROSOFT STUDENT PARTNER

at Faculty of Technical Sciences

How can you contact me?

Follow Me on Twitter

@gdjonovic

How can you contact me?

SEND ME AN EMAIL

goran.djonovic@studentpartner.com

gdjonovic@execom.eu

OUR MISSION FOR TODAY


Explain Kinect

Motivate you to develop Kinect Apps

Talk about face recognition




first Lets



Explain kinect




Explain kinect

WHAT IS KINECT?

Explain kinect

YES, KINECT IS THE SENSOR

but what's in it?

how does it work?

Explain kinect


Explain Kinect

EXPLAIN KINECT

Depth data

EXPLAIN KInect

I don't care about tech spec I want to use it in an App

EXplain Kinect

USE THE KINECT SDK

but which one?


EXPLAIN KINECT




EXPLAIN KINECT

KINECT SKDs:

  1. Kinect SDK 1.0
  2. Kinect SDK Beta
  3. Kinect SDK Beta 2
  4. Kinect SDK 1.5 (1.5.1, 1.5.2)
  5. Kinect SKD 1.6 (4 days ago)


... and some/many other hacks 

eXPLAIN KINECT

Kinect for xBox 360 SDKs (before 1.5) was

All about skeletal data

... and it was fun

Explain Kinect

EXPLAIN Kinect

Kinect for xBox 360 SDKs (before 1.5)

  • Track two users
  • On distance from 0.8 to 2 meters
  • No gesture support
  • No near mode
  • No face recognition
  • Academic use only


... and it was fun

EXPLAIN Kinect

Kinect SDK 1.5 (Kinect for Windows, February 21)

AND NOW IT'S COOL



EXPLAIN Kinect

Kinect SDK 1.5 (Kinect for Windows)

  • Kinect studio (praise the lord)
  • Human interface guidelines

  • Face tracking SDK

  • Samples and Doc (again praise the lord)
  • Improved skeleton tracking 
  • Performance and data quality enhancements
  • The Joint Orientation (Hierarchical Rotation
    Absolute Orientation)
  • Speech recognition improvement

explain KINECT


Explain Kinect


IS KINECT STILL MYSTERY?

And now, do you want to



Develop kinect apps

THE FIRST and MOST IMPORTANT STEP IN SOFTWARE development is?

IDEA, IDEA, IDEA, IDEA!

Plato: An archetype of which a corresponding being in phenomenal reality is an imperfect replica.
Kant: A concept of reason that is transcendent but nonempirical.
Hegel: Absolute truth; the complete and ultimate product of reason.

Guidelines

Be aware of technical abilities and limitations




Guidelines

Be aware of technical abilities and limitations

Guidelines

Avoid direct sun light


Guidelines

        Match your user/client needs and mindset.

...other

Guidelines


  • Follow trends. Get to know what is already there 
  • Work with people with from different backgrounds and cultures
  • Any idea is potential gold. Don't trow it right away, polish it, work on it

while(true){

DEVELOP -> EVALUATE -> IMPROVE

}

Recent (THE BEST) kinect event

TANDEM WORKSHOP

in Budapest

30 highly motivated students from 13 different countries

8 days

7 teams

7 great ideas and countless more

 TANDEM WORKSHOP BEST IDEA




New interactions

  • Use finger to point 
  • Use legs to slide
  • Use characteristic moves
  • Suggestions


And of course...

Emotion analyzer

First there were hits/clicks, then we got likes, but now we can have true emotions.



ARE YOU INTERESTED IN HOW TO DO IT?  :)


Ok, Lets talk about code



FACE RECOGNITION

and analyzing

How to start

  • Download Kinect SDK 1.5 and development toolkit

You will get

  • FaceTrackData.dll
  • FaceTrackLib.dll


C++ application can directly use the face tracking libraries

In the other case, you should use a wrapper

Starting projects

Use:

  • Microsoft.Kinect.Toolkit
  • Microsoft.Kinect.Toolkit.FaceTracking

Initialize Sensor and face tracking

kinectSensor.ColorStream.Enable();
kinectSensor.DepthStream.Range = DepthRange.Near;
kinectSensor.DepthStream.Enable(rez80X60);
kinectSensor.SkeletonStream.EnableTrackingInNearRange = true;
kinectSensor.SkeletonStream.TrackingMode = SkeletonTrackingMode.Seated;
skeletonData = new Skeleton[6];
kinectSensor.Start();
faceTracker = new FaceTracker(kinectSensor);

Getting face frame 

void kinectSensor_AllFramesReady(object sender, AllFramesReadyEventArgs e)
{
    //you must track face on every frame
}

using (ColorImageFrame colorImageFrame = e.OpenColorImageFrame())
{
    if (colorImageFrame == null)
        return;
    colorImageFrame.CopyPixelDataTo(colorPixelData);
}
 
using (DepthImageFrame depthImageFrame = e.OpenDepthImageFrame())
{
    if (depthImageFrame == null)
        return;
    depthImageFrame.CopyPixelDataTo(depthPixelData);
}
 
using (SkeletonFrame skeletonFrame = e.OpenSkeletonFrame())
{
    if (skeletonFrame == null)
        return;
    skeletonFrame.CopySkeletonDataTo(skeletonData);
}

var skeleton = skeletonData.FirstOrDefault(
s => s.TrackingState == SkeletonTrackingState.Tracked);
if (skeleton == null)
    return;
// only track face if all data are there


FaceTrackFrame faceFrame = faceTracker.Track(
kinectSensor.ColorStream.Format, colorPixelData,
kinectSensor.DepthStream.Format, depthPixelData,
skeleton);

Get3DShape()

This method will return 87 face  points in 3D

GetProjected3DShape()

returns the same collection but in a 2D referential corresponding to a 640×480 pixels image.

GetTriangles()

is a method that you can use in association with the Get3DShape method. It will return you a collection of triangles.

Get Animation Unit Coefficients()

returns the values describing the facial expressions.



Jaw lowerer

Lip Corner Depressor

Upper Lip raiser

Lip Stretcher

Outer Brow Raiser

Neutral face

AND NOW THE DEMO

Q&A

ANY QUESTIONS?



THANK YOU ALL

I'm going home

Made with Slides.com