What is Sonification?
...turning data into audio reveals changing variables to the listener through changes in sonic dimensions, such as frequency, pitch, amplitude, and location in the stereo field.
In musical contexts, data can map to these sonic dimensions, as well as higher-order musical dimensions, such as tempo, form, and timbre.
Any data can be "sonified"
I opened the raw data of this slides.html file with Audacity and exported it as .mp3.
WARNING! Turn your sound down, this is nasty!
Some of the earliest work in sonification was done by an earthquake researcher named Hugo Benioff more than 70 years ago.
Benioff originally wanted to be an astronomer but switched careers when he discovered that astronomers sleep during the day and work at night. He joined Caltech’s Seismological Laboratory in 1924, and in 1932 invented the seismograph, which records tectonic activity on a roll of paper.
Sonification is all around us -- we hear it every day, from the “bleeeeep” of a microwave oven finishing its cycle, to the “bloop-blip” of a new message arriving on your phone, to the “crrrnk” of an error message on your laptop.
It’s also a staple of TV and film storytelling, with the “beep beep beep” of an operating theatre’s EKG machine, the “pong!” of a sonar in a submarine, or the “bing” of an arriving lift providing an essential sense of place.
An audio spirit level...
Death of the trees
Oakes et al. painstakingly surveyed thousands of trees across 50 vegetation plots, including five conifer species with over 30 documented variables per tree. While there was not an explicit time series element to the data, geographic latitude became a proxy for time in what is called a chronosequence: climate change had longer to impact the southern range of the forests, and so effectively, north to south told the temporal story of climate change's impacts on the forest composition.
Data sonification allows users to be able to hear information from cosmic objects as well as see it
Layering the instruments on top of each other gives the observations an element of texture, Malec says. “It appealed to my musical sense because it was done in a harmonious way — it was not discordant.”
That was on purpose. “We wanted to create an output that was not just scientifically accurate, but also hopefully nice to listen to,” Arcand says. “It was a matter of making sure that the instruments played together in symphony.”
Astronomers including Díaz-Merced, who is blind, have used sonifications to study stars, solar wind and cosmic rays. And in experiments, Díaz-Merced has demonstrated that sighted astronomers can better pick out signals in datasets by analyzing audio and visual information together rather than relying on vision alone.
DNA Sonification refers to the use of audio to convey the information content of a DNA sequence. Audio is created using the rules of gene expression and codons are played as musical notes.
Play My Weather
The story of our planet's climate is recorded in ice over 800,000 years. The ice, over Lake Vostok in Antarctica, is 3 kilometers deep. A core sample of this mighty sheet of ice produces important geophysics data. With Ice Core Walk, you can take an audio tour and virtually walk down the data along the ice core.
Sonification with a contextualising narrative.
Brian has been creating interesting art works and projects, often responding to various data/archive material.
But hang on a minute!
Is there nothing new under the sun?
"Theme from Mission: Impossible" is the theme tune of the TV series Mission: Impossible. The theme was written and composed by Argentine composer Lalo Schifrin
_ _ ..
In the late 1960s, composer, architect and theoretician Iannis Xenakis (1922-2001) began his research on stochastic synthesis: an approach to microsound synthesis that uses probability distributions to manipulate individual digital samples, as if they were indivisible elementary particles
The Bach Motif
A musical cryptogram is a cryptogrammatic sequence of musical symbols, a sequence which can be taken to refer to an extra-musical text by some 'logical' relationship, usually between note names and letters.
"The Missa Hercules dux Ferrariae is a setting of the Ordinary of the Mass composed by Josquin des Prez, and dedicated to Ercole d'Este I, Duke of Ferrara. The musical source material for the mass, the cantus firmus, is derived from the musical letters in the Duke's name, a technique called soggetto cavato.
Activity: Explore the Data Sonification Archive
Activity: Explore some different "shapes" and imagine which your data might be a good fit for.
Activity: Hear Your Name As Music - Save the midi
"Tom Smith" in Bandlab
I put my name into Clarallel, downloaded the midi, dragged them into Bandlab and chose the instruments I wanted and added some effects.
Activity: Add your midi to an Instrument track and adapt
Watch this... Is it a debunking of sonification?
Information gets through...
A song, converted into midi, with all acoustic information removed, and you can still "hear" the lyrics?
Do it yourself with BearAudioTool
See also: I Am Sitting In A Room
The Midi format is really useful because it isn't audio, it is a series of timed instructions to play an instrument (synthesizer), so it is very similar to your data.
The main messages are the NOTE ON and NOTE OFF messages.
And your data, regardless of the numbers, can be mapped onto audible notes, durations and velocities etc.
With so many sound design choices available, thinking carefully about aesthetic decisions is crucial.
- Are acoustic or synthesized sounds used?
- Are traditional musical instruments employed, and if so, from what culture and genre?
- Do the authors intend the sonification to be used as an auditory graph, or to be experienced as a musical composition, gallery installation, soundwalk or alarm?
- How is time integrated?
Things to consider-
As Slow As Possible
ORGAN2/ASLSP, the slowest and longest music piece ever, is being performed for 639 years in Halberstadt
Next tone change: 5 February 2022
Q: How might time interact with your data?
Now. Make it
Tools to try
Sonic Pi is a music live-coding tool that can import .csv files and play them
ScribbleAudio turns drawing gestures into music. Imagine how the trends in your data might be like these gestures.
TwoTone lets you add .csv data and turn it into music…
PixelSynth reads an image left-to-right.
Computorse generates music from parameters…
…as does MusicalAlgorithms.
…as does Aiva
p5Js example of sonifying weather .csv. Duplicate and adapt.
More related tools…
A brilliant collection of sonification concepts.