Web Audio API









Chinmay Pendharkar
General Hackers September 2012

About Me..



    Acoustician / Audio Engineer


      Developer (Audio)


        Podcaster (WeBuildSG Live)


        Geek

        Audio + Web History



        Browsers in the 90s - #FAIL



        Plugins save the day?



        Flash, Java, .. (urgh!!)

        Plugin Free Web!


        Why?

        Lean(er) Stack
                Website > Browser > OS
                                Website > Plugin > Browser > OS


        Less breakable parts (theoretically)



         ↓ Reliance on single party (Adobe)

        Web + Audio


        Before

        Flash , Java (Urgh!!)



        After

        HTML5 Audio, WebAudio, WebRTC*, WebMIDI*


        * Ongoing W3C Standardization process

        HTML5 Audio
        <audio> / HTMLAudioElement()

        Part of the DOM Elements.

        Allows basic playback of Audio files.

        Basic functionality support in all Major browsers. 

         <audio controls="controls">
        Your browser does not support the <code>audio</code> element.
        <source src="https://dl.dropboxusercontent.com/u/77191118/DeepIntoIt.mp3" type="audio/wav">
</audio>

        HTML5 Audio Features


        Autoplay 
        (hello! 1990s called they want their website back)

        Loop 
        audioTag.loop = true;

        Speed   
        audioTag.playbackRate = 2;



        Stop horsing around..




        Give us the good stuff!!


        Web Audio API


        " JavaScript API for processing and synthesizing audio"


        JavaScript API


        Audio Processing is usually done at a lower level (C++/ASM)


        Realtime

        Web Audio API


        "signal routing graph paradigm"


        WebAudio VIPs


        AudioNode -  Everything is an AudioNode

        AudioContext -  God of WebAudio. Everything begins and ends here.  
        myContext = new AudioContext();
        Destination -  Main Audio output. Plug everything into this..  
        source = context.createOscillator();
source.connect(myContext.destination);

        Pitch 
        Volume 

        AudioNodes



        • Analyser
        • BiquadFilter
        • Buffer
        • BufferSource
        • ChannelMerger
        • ChannelSplitter
        • Convolver
        • Delay
        • DelayNode
        • DynamicsCompressor



        • GainNode
        • JavaScriptNode
        • MediaElementSource
        • MediaStreamDestination
        • MediaStreamSource
        • Oscillator
        • Panner
        • ScriptProcessor
        • WaveShaper
        • WaveTable

        Filter

         var audio = new Audio();
 audio.src = 'https://dl.dropboxusercontent.com/u/77191118/DeepIntoIt.mp3';
 source = context.createMediaElementSource(audio);
 var filter = context.createBiquadFilter();
 filter.type = filter.LOWPASS;
 filter.frequency.value = 440;

 source.connect(this.filter);
 filter.connect(context.destination);


        Pause/Play 

        Filter Cutoff 

        http://www.smartjava.org/examples/webaudio-filters/

        Convolver

        var source = context.createBufferSource();
var convolver = context.createConvolver();

source.connect(convolver);
convolver.connect(context.destination);

/*Load source audio and impulse response*/
var request = new XMLHttpRequest();
request.open("GET", "https://dl.dropboxusercontent.com/u/77191118/deep10.mp3", true);
request.responseType = "arraybuffer"; 
request.onload = function () {
    source.buffer = context.createBuffer(request.response, 2);
    var ir_request = new XMLHttpRequest();
    ir_request.open("GET", "https://dl.dropboxusercontent.com/u/77191118/church_ir.wav", true);
    ir_request.responseType = "arraybuffer"; 
    ir_request.onload = function () {
    convolver.buffer = context.createBuffer(ir_request.response, 2);
    source.start(0);
    }
    ir_request.send();
}
request.send();

        Hall :
        Cup :

        Demos


        Synth :
        http://synth.soundkeep.com/

        Sequencer :
        http://labs.dinahmoe.com/ToneCraft/

        WavePad :
        http://alxgbsn.co.uk/wavepad/

        Games :
        http://chrome.angrybirds.com/

        Resources


        http://www.html5audio.org/


        http://www.w3.org/wiki/HTML/Elements/audio


        https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html


        http://www.html5rocks.com/en/tutorials/webaudio/intro/

        What's coming?




        WebRTC (getUserMedia API)



        WebMIDI

        Final words


        Your mileage WILL vary.


        Watch out for Browser/OS dependencies for functionality.


        Watch out for codec issues (mp3/wav/ogg).


        Go Play!





        Questions?


        Made with Slides.com