# Audio

Text

via http://www.npr.org/2014/04/09/300563606/what-does-sound-look-like

# Sample Rate

i.e. 44.1kHz

44100 samples per second

256

512

1024

samples

# Let's look at a buffer

Oscilloscope: A buffer of samples over time

1.0

-1.0

0

# Amplitude

for (var i = 0; i < bufLength; i++) {
x = inputBuffer[i];

total += Math.max(Math.min(x/this.volMax, 1), -1);

// SQUARE

sum += Math.max(Math.min(x/this.volMax, 1), -1), 2) * Math.max(Math.min(x/this.volMax, 1), -1);
}

// MEAN
var average = total/ bufLength;

// ROOT ... then take the square root of the sum.
var rms = Math.sqrt(sum / bufLength);

## p5.Amplitude

Root Mean Square a buffer of samples

# p5.AudioIn()

(microphone)

### 0.1 sounds 1/2 as loud as 1.0

gainToDB: 20 * (Math.log(value) / Math.LN10);

Array

### Peak Detection

threshold, cutoff, decayRate

# Frequency Domain

Spectrum

FFT - Fast Fourier Transform

# Frequency

### Hz = cycles per second

Human Hearing: 20Hz to 20,000Hz  (20 kHz)

/ 2 = Nyquist

22,500

# Frequency Spectrum

Sub Bass: 20-60Hz

Bass: 60-250 Hz

Low Midrange: 250-500 Hz

Midrange: 500Hz-2kHz

Upper Midrange: 2-4khz

Presence: 4-6khz

Brilliance: 6kHz - 20kHz

# Pitch !== Frequency

### Harmonics

example:

- thumb piano vs violin

- square wave vs sine wave

# Fourier Transform

### Time Domain --> Frequency Domain

compression (jpeg, mp3)

Crick & Watson used it to decode DNA

via wikipedia

Amplitude + Phase

http://bl.ocks.org/jinroh/raw/7524988/

# Taming the FFT

Short buffer - detect onsets

Long buffer - better for frequency precision

44100/1024 ~= 43Hz

# Autocorrelation

Tmema (Golan Levin + Zach Lieberman), Jaap Blonk, Joan La Barbara

# Autocorrelation

Compare the signal with itself shifted by x samples. If a lag of x samples correlates, then sampleRate / x may be the fundamental frequency.

# Autocorrelation

pitch detection with lowpass filter

raw autocorrelation

# Scheduling Events

## createAudio()

### HTML5 <audio>

• p5.dom --> createAudio
• Can connect to web audio
• Cannot access raw buffer data

### Web Audio API

• Can access raw buffer data
• PlaybackRate (i.e. mk-1)
• Sample-accurate timing

# Lyrics

## lrcgenerator.com

### LRC file

p5.Font()

Examples w/ LRC to JSON parser by @justan

# Echo Nest API Analyzer

API Key: developer.echonest.com

# Echo Nest API

## Subtitle

curl -F "api_key=[YOURAPIKEY]" -F "filetype=mp3" -F

# Echo Nest API

// Paste the ID and your API key into this GET request:

http://developer.echonest.com/api/v4/track/profile?api_key=[YOURAPIKEY]
&format=json&id=[YOURTRACKID]&bucket=audio_summary

// click thru to analysis and save

thank you!

bit.ly/p5jsmusicviz

@therewasaguy

By Jason Sigal

• 3,868