Teleconferencing in Spatial Audio
A thesis by Jackson Goode
Teleconferencing
is here to stay
But it's often a tiring affair...
- "Zoom fatigue" is real
- Reduced dimensionality
- Poor quality
Latency, network reliability, visual and audio fidelity can all contribute to a fatiguing experience
But software can also play a major role
![](https://i.pinimg.com/originals/23/79/28/237928b0b59460604a26b455071749d8.gif)
Issues in Teleconferencing
Don't reinvent the wheel - instead focus on one critical component
How can the treatment of audio bring us to the goal of telepresence, and closer to realistic conversations
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1461358/images/8292395/pasted-from-clipboard.png)
Replicate how real interactions take place
-
Spatial model, spatial audio
- Visual-aural coherency
- Binaural audio
-
Benefits from the literature
- Lateralizing audio can improve intelligibility
- Disentangle double-talk
- And more: reduce cognitive load, improve comprehension, and is generally more favorable
![](https://media4.giphy.com/media/l0HlAgJTVaAPHEGdy/giphy.gif)
Jitsi Meet
An open source video conferencing platform
WebAudio
An API in Javascript for working with audio
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1461358/images/8269228/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1461358/images/8592271/Page_3.png)
Implementation
- Capturing participants' audio streams (WebRTC)
- Head-related transfer functions via PannerNode
- Dynamic processing of participants
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1461358/images/8591760/Orientations.png)
A short demonstration
Validating with
a user study
There was support for four hypothesis of perceived metrics:
- Decreased cognitive effort
- Increased social presence
- Increased vocal intelligibility
- Increase in opinion score
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1461358/images/8372226/pasted-from-clipboard.png)
And the future?
Toward a Telepresence of Sound
By jacksongoode
Toward a Telepresence of Sound
- 30