Synesthesizer 1.0

As I've mentioned or alluded to in a few of my posts in the past couple of months, I've recently been working on a new talk for 2010. The talk is entitled 'Hearing Pictures' and is based around my work on a side hobby called 'Project Ludi'.

It's a little difficult to explain the long-term mission of Project Ludi in a short blog post, but in the first series of experiments on which I've honed in, I am seeking to create a system that can translate pictures into music. This is what I call the Synesthesizer -- a musical synthesizer that relies on synesthesia-inspired translation metaphors.

I'm only a few short months into what is a very long-term endeavor, and I've had precious few hours to spend on the project in that time, but I always set the benchmark definition of version 1.0 of the synesthesizer as 'a system that produces something approximating music', and even though I haven't invested nearly the amount of time as I'd like, I believe that the current Synesthesizer meets this definition. And thus, I feel I have Synesthesizer 1.0.

The Synesthesizer is a Flash 10 application that translates pictures into music in real-time. What I've included with this post is a rendering of the output of the application -- not the Synesthesizer itself. I'll let you judge the quality for yourself -- and please feel free to let me know what you think.

Now, of course, there is far more to do here, but what I found most amazing about this process was how easy it was to get to a point of generating 'music' instead of 'sound'. In effect, I apply something along the order of six rules to get to this point. The addition of each rule to the Synesthesizer brings us closer to the generation of music, instead of raw sound.

To get a bit more insight into the Synesthesizer, you should check out my talk, Hearing Pictures, which I'll be presenting at FITC Amsterdam in February and at FITC Toronto in April.

In this talk, I describe the process I followed to get the Synesthesizer to this point. Using the 1943 Hermann Hesse novel, The Glass Bead Game (also published under the title Magister Ludi, the protagonist of the novel, from whom Project Ludi derives its name), as a starting point, and proceeding through a discussion of synesthesia, the aural illusions of Professor Diana Deutsch, the Experiments in Musical Intelligence by Professor David Cope, the music of Tamarin Monkeys, and many other stepping points, I walk through the thought process required to assume an odd endeavor such as the cross-modal translation of pictures into music.

Share and enjoy!

-r