Musings with Bela

An Interactive Music System by Jackson Goode

MCT4045

Inspiration

Hardware

Software

Reflections

Inspiration

There were a number of papers that inspired me to seek out an EEG a day before our proposal

Alvin Lucier's
Music for Solo Performer (1965)

Luicer, 1965

  • Landmark piece for both EEG and sonification
  • Performance in passivity
  • Totally analog

Music for Sleeping
& Waking Minds (2011-2012)

Ouzounian et al., 2012

MoodMixer
(Leslie and Mullen, 2011)

A few others

Multimodal Brain Orchestra
(Le Grouz et al., 2010)

Towards a Dimension Space for Musical Devices
(Birnbaum et al., 2005)

Visualisations to think about the features of a IMS

  • Incredibly useful during the conceptualisation and design phase
  • As well as imagining points of strength and weakness in a system

Musings with Bela

A Framework for the Evaluation of Digital Musical Instruments (O'Modhrain, 2011)

Approach the evaluation of an IMS from every angle

  • As a designer, performer and an audience member
  • Assume these roles during construction (and evaluation)

Led me to consider how using an EEG might influence

  • Performability (must remain still, eyes closed?)
  • Visual presentation
  • Usability of raw EEG data 

Even Mr. Chafe is interested!

Hardware

Pretty simple design

Shock!

Awe!

3-axis accelerometer (analog)

Two, 1K potentiometers (analog)

Muse (2016)

  • 256Hz stream
  • 4 electrodes
  • Bluetooth Low Energy (BLE)
  • Good battery life
  • Hackable! (?)

The problems begin...

Total discontinued support for interfacing with earlier Muses - 3rd party software?

Software

Where I lost myself, found myself, and then got lost along the way

Attempting to avoid this...

Hamano et al., 2013

Clearly this could get complicated!

Pure Data

Ah, my love, Pure Data. Second only to Chris Chafe!

 

Far easier to use, but things can become an entangled mess very quickly.

Python

The Muse required something to pass its messages - surprise! it's not easy! :)

 

I spent the first week getting this interface working - to find out the EEG data is so difficult to work with.

uvicMuse

Enter uvicMuse, a random project posted on GitHub 4 months ago with zero stars. Hallelujah.

 

Featuring:

  • No documentation
  • No CLI

but...

  • A beautiful GUI
  • Native bluetooth support!

Barabadi & Fregeau, 2020

... made for Matlab

Thankfully it's written in Python where I can inject a malicious rootkit to convert the data stream to OSC.

Not so fast, Jackson!
 

uvicMuse is designed for Matlab.

What about Pure Data?

But what does this do?

I had to compile iemmatrix for the Bela
(this only took 4 hours to learn)

Morphing between tables

Uses the iemmatrix library to do operations on arrays without having to iterate through their index.

 

Critical CPU-wise when these operations have to happen quickly (accelerometer).

It sounds cool too!

Short grains can be interpolated in interesting ways

What about the EEG?

It modulates playback

I wish I could say there is some complex mapping but wrangling the data took time

Reflections

What went right... what went right?

A few problems

Conceptualise as much beforehand

  • Deciding how something ought to sound is difficult
    • Even more during design!
  • I love squashing bugs - big time waster
  • Working with audio in Pd imposes barriers with traditional programming routines

 

However

I learned a lot on the way!

  • Getting to know Pd through live audio signals creates much more responsive patches
  • Working with a micro-computer, linux, building externals, ssh, and more circuit building was super
     
  • Built a fully-closed system

fun

Works Cited

Birnbaum, David, et al. Towards a Dimension Space for Musical Devices. 2005.

Hamano, Takayuki, et al. ‘Generating an Integrated Musical Expression with a Brain-Computer Interface.’ NIME,
        2013, pp. 49–54.

Le Groux, Sylvain, et al. ‘Disembodied and Collaborative Musical Interaction in the Multimodal Brain Orchestra.’
        NIME, 2010, pp. 309–314.

Leslie, Grace, and Tim R. Mullen. ‘MoodMixer: EEG-Based Collaborative Sonification.’ NIME, Citeseer, 2011, pp. 296–
        299.

Lucier, A. 1972. Music for Solo Performer 1965. In Review 1972. London: Guildhall School of Music & Drama, 22–4.

O’Modhrain, Sile. ‘A Framework for the Evaluation of Digital Musical Instruments’. Computer Music Journal, vol. 35,
        Mar. 2011, pp. 28–42. ResearchGate, doi:10.1162/COMJ_a_00038.

Ouzounian, Gascia, et al. ‘To Be inside Someone Else’s Dream: On Music for Sleeping & Waking Minds’. New
        Interfaces for Musical Expression (NIME 2012), 2012, pp. 1–6.

Parvizi, Josef, et al. ‘Detecting Silent Seizures by Their Sound’. Epilepsia, vol. 59, no. 4, 2018, pp. 877–84. Wiley Online
        Library, doi:10.1111/epi.14043.