As I mentioned last time, three of us from the Focusrite/Novation engineering team attended the Music Hackday event alongside the amazing Sonar festival in Barcelona. We learned lots, slept very little, and had a great time – here’s a quick summary!
Web APIs > everything else
It seems most of the hackday crowd are very web oriented, and we’d failed to make it easy for them to see how to use a MIDI controller (MIDI? What’s that? Is there a Node.js wrapper for it? …well, actually, yes, in fact there are two!) in their projects. Nonetheless, several of them did a great job.
We did point people towards the Web MIDI API and the essential Jazz Plugin, which was included in a particularly interesting hack – we’ll be exploring this further in the coming months.
Going mental
For our own hack, we decided to aim high and use the incredible Enobio EEG sensor from Starlabs. The team were very helpful getting us up and running with their OS X SDK, despite having so much else to do and having presented the Windows version!
We wanted to exploit the phenomenon known as Steady State Visually Evoked Potentials, or SSVEP. Broadly this means that if someone looks at a visual stimulus flashing at a rate between 3.5Hz and 75Hz, you can observe EEG activity at that frequency, with harmonics. We instantly thought of synthesizers and decided we’d build something that could sonify the EEG trace to play musical notes – effectively using Ross’s brain as an oscillator clocked by a flashing Launchpad.
Science is hard
After an hour or so with the sensor and some hacked Launchpad firmware generating outrageously nauseating flash patterns, we’d captured a number of test traces from Ross’s brain, but had seen little in the way of harmonic content in the spectrum analyser. I blame Ross’s brain.
At this point, we should probably have given up and decided to make something different, but instead we thought we’d give the EEG trace a helping hand and filter for the frequencies we were expecting. Combining this with a heart rate sensor to modulate the level, and pitch-shifting the output by an octave or two, we thought we’d be able to get something that was, while not truly “mind music”, at least in some way meaningfully related to brain activity.
DSP is hard
Ben set up an Audio Unit plugin and started integrating the Enobio SDK, discovering just how much fun* it is to add a dylib with dependencies to a component in a way that won’t bomb out at runtime.
Ross was having much more success modifying the Launchpad to do variable rate flashing, PWM for intensity and colour patterns. He spent time helping out some of the other hackers and got some great results (see later on).
At this point I lost the plot and charged off into DSP territory. Unfortunately I’d forgotten just how bad I am at DSP, and spent a long time troubleshooting the various buzzing, popping and aliasing introduced by my code. By 2am, we had something that made a noise in response to the heart-rate sample data, and decided to call it a day.
*really not much fun at all.
This brain has left the station
The next morning we felt like a result was just within reach – we’d use the heart rate sensor to modulate the processed (but horribly aliased) output from the EEG, in response to visual stimulus triggered from the Launchpad buttons. It might just work…
…but it didn’t. With an hour to go, we had nothing that made anything approaching a useful sound, we were struggling to normalise or beat-detect from the heart rate sensor, and had only one usable development machine after the ongoing runtime dependency nightmares had wasted hours.
We decided not to demo our hack (especially as it would mean monopolising one of the precious few Enobio sensors), and spent the remaining time helping our fellow hackers include Launchpads in their work, which was a lot of fun.
In hindsight, we’d have been much better off attacking the problem using something like Pure Data to get going quickly, wasting less time on dependencies, sample rate conversion and C++. I sincerely doubt I’ll be writing any DSP code in the near future either.
Hacks and demos
The hacks from the event are published here on Hacker League. Our favourite Launchpad hacks were:
- The giant lyrics display
- The singing cocktail maker, Sing4Drinks
- An online step sequencer
- The obligatory Whack-a-mole
…and some others that really made us smile:
We really enjoyed the event, meeting lots of inspiring people and learning about cutting-edge technologies. We’ll be back, and next time we’ll demo!
Focusrite Development 