github repo:

bit.ly/p5jsmusicviz

local server

loadSound('mySound.mp3')

Audio

Text

via http://www.npr.org/2014/04/09/300563606/what-does-sound-look-like

Time Domain

Sample Rate

i.e. 44.1kHz

44100 samples per second

Buffer of Samples

256

512

1024

samples

Let's look at a buffer

Oscilloscope: A buffer of samples over time

1.0

-1.0

0

Amplitude

      for (var i = 0; i < bufLength; i++) {
        x = inputBuffer[i];

          total += Math.max(Math.min(x/this.volMax, 1), -1);

          // SQUARE

          sum += Math.max(Math.min(x/this.volMax, 1), -1), 2) * Math.max(Math.min(x/this.volMax, 1), -1);
      }

      // MEAN
      var average = total/ bufLength;

      // ROOT ... then take the square root of the sum.
      var rms = Math.sqrt(sum / bufLength);

p5.Amplitude

Root Mean Square a buffer of samples

p5.AudioIn()

(microphone)

uses WebRTC Media Stream Track / getUserMedia API 

map(val, inMin, inMax, outMin, outMax)

amplitude.smooth(0.8);

amplitude = logarithmic

dB: -infinity to 0

gain:   0.  to   1.

0.1 sounds 1/2 as loud as 1.0

gainToDB: 20 * (Math.log(value) / Math.LN10);

Amplitude over Time

Array

Threshold

Peak Detection

threshold, cutoff, decayRate

Frequency Domain

Spectrum

FFT - Fast Fourier Transform

Frequency

Frequency

Hz = cycles per second

Human Hearing: 20Hz to 20,000Hz  (20 kHz)

44,100 sampling rate

/ 2 = Nyquist

22,500 

more about sampling rates and Neil Young (ht @infomuotoilu)

Frequency Spectrum

Frequency Spectrum

Sub Bass: 20-60Hz

Bass: 60-250 Hz

Low Midrange: 250-500 Hz

Midrange: 500Hz-2kHz

Upper Midrange: 2-4khz

Presence: 4-6khz

Brilliance: 6kHz - 20kHz

Frequency

= Logarithmic

Pitch !== Frequency

"Fundamental frequency"

Harmonics

example:

- thumb piano vs violin

- square wave vs sine wave

Fourier Transform

Time Domain --> Frequency Domain 

compression (jpeg, mp3)

Crick & Watson used it to decode DNA

via wikipedia

Amplitude + Phase

http://bl.ocks.org/jinroh/raw/7524988/

Taming the FFT

Short buffer - detect onsets

Long buffer - better for frequency precision

44100/1024 ~= 43Hz

Autocorrelation

Tmema (Golan Levin + Zach Lieberman), Jaap Blonk, Joan La Barbara

More info @ tmema

Autocorrelation

Compare the signal with itself shifted by x samples. If a lag of x samples correlates, then sampleRate / x may be the fundamental frequency. 

image via cnx.org pitch detection algorithms

Autocorrelation

pitch detection with lowpass filter

raw autocorrelation

Scheduling Events

.addCue(time, callback, value)

createAudio()

HTML5 <audio>

• p5.dom --> createAudio
• Loads without server
• Preload isn't necessary
• Can connect to web audio
• Cannot access raw buffer data

loadSound()

Web Audio API

• Can access raw buffer data
• PlaybackRate (i.e. mk-1)
• Sample-accurate timing

Two Ways to Play Sound

.setCue(time, callback, {val})

Lyrics

lrcgenerator.com

LRC file

p5.Font()

Examples w/ LRC to JSON parser by @justan

Echo Nest API Analyzer

API Key: developer.echonest.com

 more info 

  Further reading

Echo Nest API

Subtitle

curl -F "api_key=[YOURAPIKEY]" -F "filetype=mp3" -F
 "track=@[PATH]" "http://developer.echonest.com/api/v4/track/upload"

Echo Nest API

// Paste the ID and your API key into this GET request:

http://developer.echonest.com/api/v4/track/profile?api_key=[YOURAPIKEY]
  &format=json&id=[YOURTRACKID]&bucket=audio_summary

// click thru to analysis and save

thank you!

bit.ly/p5jsmusicviz

@therewasaguy