Teleconferencing in Spatial Audio

A thesis by Jackson Goode

Teleconferencing
is here to stay

But it's often a tiring affair...

• "Zoom fatigue" is real
• Reduced dimensionality
• Poor quality

Latency, network reliability, visual and audio fidelity can all contribute to a fatiguing experience

But software can also play a major role

Issues in Teleconferencing

Don't reinvent the wheel - instead focus on one critical component

How can the treatment of audio bring us to the goal of telepresence, and closer to realistic conversations

Replicate how real interactions take place

• Spatial model, spatial audio
  • Visual-aural coherency
  • Binaural audio
• Benefits from the literature
  • Lateralizing audio can improve intelligibility
  • Disentangle double-talk
  • And more: reduce cognitive load, improve comprehension, and is generally more favorable

Jitsi Meet

An open source video conferencing platform

WebAudio

An API in Javascript for working with audio

Implementation

• Capturing participants' audio streams (WebRTC)
• Head-related transfer functions via PannerNode
• Dynamic processing of participants

A short demonstration

There was support for four hypothesis of perceived metrics:

• Decreased cognitive effort
• Increased social presence
• Increased vocal intelligibility
• Increase in opinion score