Web Audio API









Chinmay Pendharkar
General Hackers September 2012

About Me..



    Acoustician / Audio Engineer


      Developer (Audio)


        Podcaster (WeBuildSG Live)



        Geek

        Audio + Web History



        Browsers in the 90s - #FAIL



        Plugins save the day?



        Flash, Java, .. (urgh!!)

        Plugin Free Web!


        Why?

        Lean(er) Stack
                Website > Browser > OS
                                Website > Plugin > Browser > OS


        Less breakable parts (theoretically)



         ↓ Reliance on single party (Adobe)

        Web + Audio


        Before

        Flash , Java (Urgh!!)



        After

        HTML5 Audio, WebAudio, WebRTC*, WebMIDI*


        * Ongoing W3C Standardization process

        HTML5 Audio


        <audio> / HTMLAudioElement()

        Part of the DOM Elements.

        Allows basic playback of Audio files.

        Basic functionality support in all Major browsers.

         <audio controls="controls">
                Your browser does not support the <code>audio</code> element.
                <source src="https://dl.dropboxusercontent.com/u/77191118/DeepIntoIt.mp3" type="audio/wav">
        </audio>

        HTML5 Audio Features


        Autoplay 
        (hello! 1990s called they want their website back)

        Loop 
        audioTag.loop = true;

        Speed  
        audioTag.playbackRate = 2; 



        Stop horsing around..




        Give us the good stuff!!


        Web Audio API


        " JavaScript API for processing and synthesizing audio"


        JavaScript API


        Audio Processing is usually done at a lower level (C++/ASM)


        Realtime

        Web Audio API


        "signal routing graph paradigm"


        WebAudio VIPs


        AudioNode Everything is an AudioNode

        AudioContext God of WebAudio. Everything begins and ends here. 
        myContext = new AudioContext();
        Destination Main Audio output. Plug everything into this.. 
        source = context.createOscillator();
        source.connect(myContext.destination); 

        Pitch 
        Volume 

        AudioNodes



        • Analyser
        • BiquadFilter
        • Buffer
        • BufferSource
        • ChannelMerger
        • ChannelSplitter
        • Convolver
        • Delay
        • DelayNode
        • DynamicsCompressor



        • GainNode
        • JavaScriptNode
        • MediaElementSource
        • MediaStreamDestination
        • MediaStreamSource
        • Oscillator
        • Panner
        • ScriptProcessor
        • WaveShaper
        • WaveTable

        Filter

         var audio = new Audio();
         audio.src = 'https://dl.dropboxusercontent.com/u/77191118/DeepIntoIt.mp3';
         source = context.createMediaElementSource(audio);
         var filter = context.createBiquadFilter();
         filter.type = filter.LOWPASS;
         filter.frequency.value = 440;
        
         source.connect(this.filter);
         filter.connect(context.destination);


        Pause/Play 

        Filter Cutoff 

        http://www.smartjava.org/examples/webaudio-filters/

        Convolver

        var source = context.createBufferSource();
        var convolver = context.createConvolver();
        
        source.connect(convolver);
        convolver.connect(context.destination);
        
        /*Load source audio and impulse response*/
        var request = new XMLHttpRequest();
        request.open("GET", "https://dl.dropboxusercontent.com/u/77191118/deep10.mp3", true);
        request.responseType = "arraybuffer"; 
        request.onload = function () {
            source.buffer = context.createBuffer(request.response, 2);
            var ir_request = new XMLHttpRequest();
            ir_request.open("GET", "https://dl.dropboxusercontent.com/u/77191118/church_ir.wav", true);
            ir_request.responseType = "arraybuffer"; 
            ir_request.onload = function () {
            convolver.buffer = context.createBuffer(ir_request.response, 2);
            source.start(0);
            }
            ir_request.send();
        }
        request.send();

        Hall :
        Cup :

        Demos


        Synth :
        http://synth.soundkeep.com/

        Sequencer :
        http://labs.dinahmoe.com/ToneCraft/

        WavePad :
        http://alxgbsn.co.uk/wavepad/

        Games :
        http://chrome.angrybirds.com/

        Resources


        http://www.html5audio.org/


        http://www.w3.org/wiki/HTML/Elements/audio


        https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html


        http://www.html5rocks.com/en/tutorials/webaudio/intro/

        What's coming?




        WebRTC (getUserMedia API)



        WebMIDI

        Final words


        Your mileage WILL vary.


        Watch out for Browser/OS dependencies for functionality.


        Watch out for codec issues (mp3/wav/ogg).


        Go Play!





        Questions?


        Made with Slides.com