What is it?

Gaze is one part
but we can also track pupils

Gaze is one part
but we can also track pupils

Arousal ​

E.g., increases with fear-inducing images , surprise, prediction error, uncertainty​
​​

Attention & Mental effort ​

Changes with changing demands and task engagement​

​

Experimental advantage: Not under voluntary control​

​

Gaze is one part
but we can also track pupils

and blinks

and blinks

​ ​

• Disrupted in clinical conditions involving dopamine ​
  e.g., Parkinson’s = low blink rate; schizophrenia = high (Karsons, 1998; 1990)​
   

• Occur at breaks during reading and speech (Cummins, 2012; Nakano & Kitizawa, 2010; Orchard & Stern, 1991; Hall, 1945)​
   

• Modulate with task difficulty (cognitive load) ​
   

• ​Increase when tired (Johns, 2003)
   

• Fatigue and microsleep detection

Music and eye tracking

Music and eye-tracking

• Music reading (including sight-reading)​​
   
• Musical processing (difficulty, expertise, relation to acoustic features, attention,
  expectation, salience)​
  ​
• Subjective experience (altered states, absorption, mind-wandering, groove)​
  ​
• Social interaction (any combination of performers and audience)​
  ​
• Audiovisual interactions (prominence of sound vs. visuals in guiding attention)​
  ​
• Neurophysiological mechanisms of musical engagement (dopamine, norepinephrine)​

Applications in Human-Computer Interaction

How do we track eye movements?

Traditionally, using desktop eye-trackers in labs

but we can do better...

• Limited ecological validity
• Proprietary closed-sourced code

Eye-tracking with webcams in the browser

Pipeline overview

Demo

1. Ensure that you're connected to the LiveGuest network
    
2. Go to https://192.168.23.51:5000 on your browser

Mobile eye-tracking

The hardware

- 2 IR eye cameras 

- 1 worldview camera

- NeuralNet running on an android smartphone

Let's try it out!

take a pair of eye-tracking glasses from one of the volunteers.

We can also track multiple people now!