Sonification

What is Sonification?

...turning data into audio reveals changing variables to the listener through changes in sonic dimensions, such as frequency, pitch, amplitude, and location in the stereo field.

In musical contexts, data can map to these sonic dimensions, as well as higher-order musical dimensions, such as tempo, form, and timbre.

Source.

Any data can be "sonified"

I opened the raw data of this slides.html file with Audacity and exported it as .mp3.

WARNING! Turn your sound down, this is nasty!

Geiger counter

Some of the earliest work in sonification was done by an earthquake researcher named Hugo Benioff more than 70 years ago.

Benioff originally wanted to be an astronomer but switched careers when he discovered that astronomers sleep during the day and work at night. He joined Caltech’s Seismological Laboratory in 1924, and in 1932 invented the seismograph, which records tectonic activity on a roll of paper.

Sonification is all around us -- we hear it every day, from the “bleeeeep” of a microwave oven finishing its cycle, to the “bloop-blip” of a new message arriving on your phone, to the “crrrnk” of an error message on your laptop.

It’s also a staple of TV and film storytelling, with the “beep beep beep” of an operating theatre’s EKG machine, the “pong!” of a sonar in a submarine, or the “bing” of an arriving lift providing an essential sense of place.

An audio spirit level...

Death of the trees

Oakes et al. painstakingly surveyed thousands of trees across 50 vegetation plots, including five conifer species with over 30 documented variables per tree. While there was not an explicit time series element to the data, geographic latitude became a proxy for time in what is called a chronosequence: climate change had longer to impact the southern range of the forests, and so effectively, north to south told the temporal story of climate change's impacts on the forest composition.

Source.

Data sonification allows users to be able to hear information from cosmic objects as well as see it

Source

Literally, multi-dimensional

Layering the instruments on top of each other gives the observations an element of texture, Malec says. “It appealed to my musical sense because it was done in a harmonious way — it was not discordant.”

That was on purpose. “We wanted to create an output that was not just scientifically accurate, but also hopefully nice to listen to,” Arcand says. “It was a matter of making sure that the instruments played together in symphony.”

Astronomers including Díaz-Merced, who is blind, have used sonifications to study stars, solar wind and cosmic rays. And in experiments, Díaz-Merced has demonstrated that sighted astronomers can better pick out signals in datasets by analyzing audio and visual information together rather than relying on vision alone.

Dissonance?

DNA Sonification refers to the use of audio to convey the information content of a DNA sequence. Audio is created using the rules of gene expression and codons are played as musical notes.

TTTAGGGTTAGGGTTAGGGTTAGGGTTAGGGTTAGGGTTAGGGTTAGGGTTAGGGTTAGGGTTAGGGTTAGGGTTAGGGAGTAGGGTTAGGGTTAGGGTT

Patterns

Play My Weather

Multi-dimensional

The story of our planet's climate is recorded in ice over 800,000 years. The ice, over Lake Vostok in Antarctica, is 3 kilometers deep. A core sample of this mighty sheet of ice produces important geophysics data. With Ice Core Walk, you can take an audio tour and virtually walk down the data along the ice core.

Source

Ice Walk

Sonification with a contextualising narrative.

brianfoo.com

Brian has been creating interesting art works and projects, often responding to various data/archive material.

But hang on a minute!

Is there nothing new under the sun?

"Theme from Mission: Impossible" is the theme tune of the TV series Mission: Impossible. The theme was written and composed by Argentine composer Lalo Schifrin

_ _ ..

1966

Iannis Xenakis

In the late 1960s, composer, architect and theoretician Iannis Xenakis (1922-2001) began his research on stochastic synthesis: an approach to microsound synthesis that uses probability distributions to manipulate individual digital samples, as if they were indivisible elementary particles

The Bach Motif

A musical cryptogram is a cryptogrammatic sequence of musical symbols, a sequence which can be taken to refer to an extra-musical text by some 'logical' relationship, usually between note names and letters.

List of compositions with the theme "B-A-C-H"

Art of Fugue' Bach

1740

"The Missa Hercules dux Ferrariae is a setting of the Ordinary of the Mass composed by Josquin des Prez, and dedicated to Ercole d'Este I, Duke of Ferrara. The musical source material for the mass, the cantus firmus, is derived from the musical letters in the Duke's name, a technique called soggetto cavato.

1480

KONG

Activity: Explore the Data Sonification Archive

Activity: Explore some different "shapes" and imagine which your data might be a good fit for.

Activity: Hear Your Name As Music - Save the midi

"Tom Smith" in Bandlab

I put my name into Clarallel, downloaded the midi, dragged them into Bandlab and chose the instruments I wanted and added some effects.

Activity: Add your midi to an Instrument track and adapt

Watch this... Is it a debunking of sonification?

Information gets through...

A song, converted into midi, with all acoustic information removed, and you can still "hear" the lyrics?

Do it yourself with BearAudioTool

Why midi?

The Midi format is really useful because it isn't audio, it is a series of timed instructions to play an instrument (synthesizer), so it is very similar to your data.

The main messages are the NOTE ON and NOTE OFF messages.

And your data, regardless of the numbers, can be mapped onto audible notes, durations and velocities etc.

Aesthetics?

With so many sound design choices available, thinking carefully about aesthetic decisions is crucial.

Are acoustic or synthesized sounds used?
Are traditional musical instruments employed, and if so, from what culture and genre?
Do the authors intend the sonification to be used as an auditory graph, or to be experienced as a musical composition, gallery installation, soundwalk or alarm?
How is time integrated?