Web Audio API
History of Audio in the Web
- In the behinning there was Flash
- Plugin
- not secure
- ...Flash
- HTML5: <audio></audio>
- Cross Browser support by 2011
- No precise timing controls
- No ability to apply real-time effects
- No way to analyze sounds
- Web Audio API
- Awesome!
- The Best!
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3565878/flash-player-logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3566029/adobe-dlm4-flash-installer.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3565895/adobe-flash-player-cartoon.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3566035/html5.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3566053/soundapicover.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3566063/evolution-grey.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3565834/web_audio_api_support.png)
Web Audio API Support
What is Web Audio?
This specification describes a high-level JavaScript API for processing and synthesizing audio in web applications. The primary paradigm is of an audio routing graph, where a number of AudioNode objects are connected together to define the overall audio rendering. The actual processing will primarily take place in the underlying implementation (typically optimized Assembly / C / C++ code), but direct JavaScript processing and synthesis is also supported.
- W3C Abstract
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3566158/open_quote.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3566159/close_quote.png)
Audio Routing Graph
Audio Context
Destination
Visualization
Gain
Analyser
BufferSource
User Control
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3566323/f02a629859c4.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3566326/zoom_file-icons_h1.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3566443/array-buffer.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3566454/media-knob.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3566465/Icon_EQ-512.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3566338/waap_0102.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3567139/traktor_02.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3625258/Scratch_Live_2.4_.4_Screenshot_.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3567141/2Deck.png)
Atwood's Law
Any application that can be written in JavaScript, will eventually be written in JavaScript.
- Jeff Atwood
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3566158/open_quote.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3566159/close_quote.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3566561/oaps_womens.png)
Open DVS Demo
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3569833/indexeddb.png)
Indexed Db
- No storage limit
- Store objects (not just JSON)
- Supports circular references
- Transactional
- Async
- Key-value Store (not relational)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3570154/types.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3570168/ff-premission.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3570654/waap_0108.png)
[.25, .5, .7, .8, .82, .7, .2, 0, -.2, -.5, -.7]
Sample Rate | Bit Depth | 4 minute song |
---|---|---|
44,100 | 16bit | 21MB |
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3570505/giphy.gif)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3570555/giphy__2_.gif)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3570556/giphy__3_.gif)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3570559/giphy__1_.gif)
The Fourier Series
- Takes a signal and represents it as a series of sin waves.
- Coverts between the "time domain" and "frequency domain"
Discrete Fourier Transform (DFT)
- Like the Fourier series but works with Discrete signals
- e.g. a digital sampled audio signal
Fast Fourier Transform (FFT)
- Any algorithm that can solve a DFT in
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3574672/Fourier_series_square_wave_circles_animation.gif)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3574673/Fourier_series_sawtooth_wave_circles_animation.gif)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3574758/3311485_orig.gif)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3574759/Fourier_transform_time_and_frequency_domains.gif)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3571715/fourier.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3574932/fourier.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3574933/Heat_eqn.gif)
Joseph Fourier
1768 - 1830
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3574905/lowpass.jpg)
- Used by .jpg to compress images
- Ised by .mp3 to compress audio
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3574879/giphy.gif)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3574926/foobar_fft.png)
Open-DVS waveform / crossfade demo
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3593004/Digital_wave-t.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3593020/Digital_wave.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3578999/midi.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3584919/rickroll.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3590758/unnamed.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3590769/steamworkshop_webupload_previewfile_239762349_preview.jpg)
1981
Companies such as Roland, Yamaha, Korg and Kawai start discussing coming up with a standard protocol
MIDI Specification finalized
1983
MIDI Timeline
2015
Chrome implements the Web MIDI API
Web-MIDI API for Web browsers is the most significant advancement of MIDI since... MIDI itself
- The MIDI Association
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3566158/open_quote.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3566159/close_quote.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3585172/can_I_use_web_midi.png)
1TTTCCCC 0AAAAAAA 0BBBBBBB
⏟
⏞
⏟
⏞
Message Type
- 000 - Note Off
- 001 - Note On
- 010 - Aftertouch
- 011 - Control Change
- 100 - Prog Change
- 101 - Chan Pressure
- 110 - Pitch Bend
- 111 - System Exclusive
Channel
1 - 16
Data Byte 1: 0 - 127
If type is "Note On" this represents the note number.
60 (0111100) === Middle C
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3585282/XGqc4.jpg)
Data Byte 2: 0 - 127
If Type is "Note On" this represents note velocity.
10110000 00000111 01000000
⏟
Control Change
⏞
Channel 1
⏟
111₂ === 7
Volume
1000000₂ === 64
⏞
Set channel 0 volume to 64/127
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3586244/cc_nums.png)
Open DVS MIDI Demo
Web RTC
- RTC === Real Time Communication
- Released by Google in 2011
- Standerdized by W3C and IETF
- Components
- getUserMedia: allows a web browser to access the camera and microphone and to capture media
- RTCPeerConnection: sets up audio/video calls
- RTCDataChannel: allows browsers to share data via peer-to-peer
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3589217/can_i_use_webRtc.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3592837/images.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/664662/images/3592838/imgres.png)
Σ
∫
Open DVS Script Processor Node
Web Audio
By Rob McDiarmid
Web Audio
- 976