Web Audio API
History of Audio in the Web
- In the behinning there was Flash
- Plugin
- not secure
- ...Flash
- HTML5: <audio></audio>
- Cross Browser support by 2011
- No precise timing controls
- No ability to apply real-time effects
- No way to analyze sounds
- Web Audio API
- Awesome!
- The Best!
Web Audio API Support
What is Web Audio?
This specification describes a high-level JavaScript API for processing and synthesizing audio in web applications. The primary paradigm is of an audio routing graph, where a number of AudioNode objects are connected together to define the overall audio rendering. The actual processing will primarily take place in the underlying implementation (typically optimized Assembly / C / C++ code), but direct JavaScript processing and synthesis is also supported.
- W3C Abstract
Audio Routing Graph
Audio Context
Destination
Visualization
Gain
Analyser
BufferSource
User Control
Atwood's Law
Any application that can be written in JavaScript, will eventually be written in JavaScript.
- Jeff Atwood
Open DVS Demo
Indexed Db
- No storage limit
- Store objects (not just JSON)
- Supports circular references
- Transactional
- Async
- Key-value Store (not relational)
[.25, .5, .7, .8, .82, .7, .2, 0, -.2, -.5, -.7]| Sample Rate | Bit Depth | 4 minute song |
|---|---|---|
| 44,100 | 16bit | 21MB |
The Fourier Series
s_N(x) = \frac{A_0}{2} + \sum_{n=1}^N A_n\cdot \sin(\tfrac{2\pi nx}{P}+\phi_n), \quad \scriptstyle \text{for integer}\ N\ \ge\ 1.
- Takes a signal and represents it as a series of sin waves.
- Coverts between the "time domain" and "frequency domain"
Discrete Fourier Transform (DFT)
{\displaystyle X_{k}=\sum _{n=0}^{N-1}x_{n}\cdot (\cos(-2\pi k{\frac {n}{N}})+i\sin(-2\pi k{\frac {n}{N}})),\quad k\in \mathbb {Z} \,}
- Like the Fourier series but works with Discrete signals
- e.g. a digital sampled audio signal
Fast Fourier Transform (FFT)
- Any algorithm that can solve a DFT in
O(n\log n)
Joseph Fourier
1768 - 1830
- Used by .jpg to compress images
- Ised by .mp3 to compress audio
Open-DVS waveform / crossfade demo
E(|x+y|)
x, y \in [-1, 1]
E(|x|) = 0.5
E(|y|) = 0.5
X
Y
1981
Companies such as Roland, Yamaha, Korg and Kawai start discussing coming up with a standard protocol
MIDI Specification finalized
1983
MIDI Timeline
2015
Chrome implements the Web MIDI API
Web-MIDI API for Web browsers is the most significant advancement of MIDI since... MIDI itself
- The MIDI Association
1TTTCCCC 0AAAAAAA 0BBBBBBB
⏟
⏞
⏟
⏞
Message Type
- 000 - Note Off
- 001 - Note On
- 010 - Aftertouch
- 011 - Control Change
- 100 - Prog Change
- 101 - Chan Pressure
- 110 - Pitch Bend
- 111 - System Exclusive
Channel
1 - 16
Data Byte 1: 0 - 127
If type is "Note On" this represents the note number.
60 (0111100) === Middle C
Data Byte 2: 0 - 127
If Type is "Note On" this represents note velocity.
10110000 00000111 01000000
⏟
Control Change
⏞
Channel 1
⏟
111₂ === 7
Volume
1000000₂ === 64
⏞
Set channel 0 volume to 64/127
Open DVS MIDI Demo
Web RTC
- RTC === Real Time Communication
- Released by Google in 2011
- Standerdized by W3C and IETF
- Components
- getUserMedia: allows a web browser to access the camera and microphone and to capture media
- RTCPeerConnection: sets up audio/video calls
- RTCDataChannel: allows browsers to share data via peer-to-peer
Σ
∫
Open DVS Script Processor Node
Web Audio
By Rob McDiarmid
