Yesterday as part of a Kinect-based music project with some friends I started diving into the Java programming library that lets you directly create your own sounds, by building the signal yourself.
So I found myself really down in the wonderful low level playground where a little bit of computer programming lets me directly create vibrato (there are two kinds!), harmonics, pitch slide, echo, formants (for making vowel sounds) and all the other cool musical things our ears can hear.
This part is all just laying paint. The really interesting and fun part is when we start to make musical instruments out of all this, by using the Kinect to watch what your hands and fingers are doing, and try to create a really expressive and nicely controllable result.
We’ve already decided it’s not going to be all “waving your hands in the air”. Except for the Theremin, musical instruments usually involve touching something solid, and people are good at that. So in some ways it’s going to have things in common with the finger painting program I talked about recently.
Except of course that the thing we’ll be painting is music. 🙂
I’d like to pick up a Kinect for music and other multimedia hacking when I can spare the $150. I’ve played around a lot with the Wii Remote as an OSC controller, bridging it to MIDI and sending it to Ableton Live.
I lugged my MacBook Pro to an acoustic show once — I had the Wii Remote and numchuk set up to trigger a bass and snare drum respectively when you “hit” imaginary drum pads (i.e. when the internal accelerometers detected a sudden acceleration and stop). I brought a random audience member up on stage and had him play the virtual drums on a cover of the Jesus And Mary Chain’s “Just Like Honey”, which uses the old Ronettes “Be My Baby” beat that anybody can play easily.
It was cool to do simple audience participation like that. I’ve often thought that with a Kinect you could do even more — assign the frequency of a synth’s lowpass filter to audience movement, for example, so that the performance would be uniquely tied to the location and the reaction of the audience itself.
Very cool! I’m more interested in creating a musical instrument that I myself would play, rather than a way to evoke and respond to audience participation. Both are interesting directions.