• "New Media Team Project," Senior Capstone
  
  
• "EEGJ," Interactive audio/visual experience
  
  
• DJ with your MIND (and other biosensors)
  
  
• Still a work in progress

• Plenty of contributed libraries
  
  
• Easy to learn, easy to use

• Focus on the music, worry about sound later
  
  
• Swap out instruments easily in software
  
  
• Plenty of DAW options (i.e. Ableton Live)
  
  
• There's a library for that: TheMidiBus

• Up to 4 bytes, usually: status, channel, and 2 data bytes
  
  
• Example: Note On/Off
• Status = 128/144 + channel (0 - 15)
• Channel = N/A
• Data1 = pitch (0-127)
• Data2 = velocity (0-127)
  
  
• Example: Channel Volume
• Status = 176 (any "channel message" status)
• Channel = channel (0-15)
• Data1 = 7
• Data2 = volume (0-127)

 import themidibus.* 
 MidiBus mb;

 void setup() {
   // See available MIDI ins/outs
   MidiBus.list();
   // MidiBus(PApplet sketch, int input, int output)
   mb = new MidiBus(this, -1, 1);
 }

 void draw() {
   // Play a note on channel 0 of pitch 60 (C) and velocity 80
   mb.sendNoteOn(0, 60, 127);
   delay(500);
   // Stop the note
   mb.sendNoteOff(0, 60, 127);
   delay(500);
 }

• Notes: A single tone plays for some duration
  
  
• Chords: Multiple notes play simultaneously
  
  
• Melody: Multiple notes play in sequence
  
  
• Need to manipulate Notes, Chords, and Melodies
  programmatically, in real time, as MIDI messages

• Processing library (almost) for making MIDI music
  
  
• Several types of objects:
• RiriNote - single note
• RiriChord - simultaneous notes
• RiriSequence - sequential notes
• RiriMessage - general MIDI message
  
  
• Uses an instance of TheMidiBus for sending MIDI messages
  
  
• Code: https://github.com/codenameriri/riri-framework

• Everything extends RiriObject, contains common properties
  
• Duration - length of the note/chord
• Repeats - how many times to play the note/chord
• Infinite - repeat forever
• Playing - currently playing
  
  
• All RiriObjects are Threads
• Used for timing and duration
• Needs tweaking, but it works well enough for now

• Has a channel, pitch, and velocity
  
  

 public void run() {
   while (running && counter < repeats) {
      // Send a noteOn message
      noteOn();
      // Sleep for the note's duration
      try {
         sleep((long) duration);
      } catch (Exception e) {
         println("Problem sleeping thread...");
         println(e.getMessage());
      }
      // Send a note off event and quit executing
      noteOff();
      if (!infinite) {
        counter++;
      }
   }
   running = false;
   counter = 0;}

• Has an ArrayList of RiriNotes, to be played simultaneously
• Duration set to that of the chord's longest RiriNote 

public void run() {
   while (running && counter < repeats) {
      // Play the notes in the chord
      for (int i = 0; i < notes.size(); i++) {
         notes.get(i).start();
      }
      // Sleep for the chord's duration
      try {
         sleep((long) duration);
      } catch (Exception e) {
         println("Problem sleeping thread...");
         println(e.getMessage());
      }
      // Stop the notes in the chord
      counter++;
   }
   running = false;
   counter = 0;
}

• Has an ArrayList of RiriObjects (notes or chords), played in order

 public void run() {
    // Loop through each note in the sequence
    while(running && counter < repeats) {
      int c = 0;
      while (c < notes.size()) {
        // Grab and play the current note in the sequence
        RiriObject currentNote = notes.get(c).clone();
        int wait = currentNote.duration() * currentNote.repeats();
        currentNote.start();
        // Sleep for the duration of the note
        try {
          //sleep((long) currentNote.duration());
          sleep((long) wait);
        } catch (Exception e) {
          println("Problem sleeping thread...");
          println(e.getMessage());
        }
        c++;
      }
      if (!infinite) {
        counter++;
      }
    }
    // If we're out of notes, stop executing
    running = false;
    counter = 0;
    println("Done");
  }

• Apply some rules to make it sound more "musical"
  
  
• Add structure for performing "songs"
  
  
• Use sensor data to determine what  sounds to play
  

• Keep things in time, maintain the rhythm
• Helper function: beatsToMils(1);
  
  
• Use scales that sound pleasant
• Ex) Pentatonic scale: I, II, III, V, VI
  
  
• Use common chord progressions
• Ex) Basic Song: I - IV - V - I
  
  
• Don't forget dynamics!
• Adjust the velocity of notes in a natural way

• Determine the time signature
• Beats per measure = 4
  
  
• Determine the length of each section
• Measures per "phase" = 8
  
  
• Determine the number of sections per song
• Phases = 4
  
  
• Adjust chords and scales based on position within the song

• Average data from 8 brain waves, map values
• -100 = Max "Relax"
• 0 = "Neutral"
• 100 = Max "Focus"
  
  
• Store last X values, use average to set global level
  
  
• Use absolute value of level (0 - 100) to determine "grain"
• Higher grain = more musical activity
  
  
• Other sensors:
• Pulse sensor = Tempo of the song
• Pressure sensors = Musical flourishes, Live effects

• Song #1 - Dynamic melody, constant interval
• Song #2 - Dynamic melody, dynamic interval
• Song #3 - Repeated notes and motifs
• Song #4 - Tempo shifting
  
  
• Code: https://github.com/codenameriri/test-song

• Music generated using (fake) brain data
  
• "Neutral" = a bit of everything, low intensity
• "Relax" = arpeggiator and synth pad
• "Focus" = drums and bass
  
  
• Code: https://github.com/codenameriri/test-music

• Officially package RiriFramework as a Processing Library
  
  
• Finish EEGJ for Imagine RIT in 2 weeks

Ben Centra

@TheBenCentra

blcentra@gmail.com

bencentra.com