Evolved virtual keyboards

Mari’s comment on yesterday’s post about the Candy colored musical keyboard was: “I typed different words and listened.”

Which was absolutely brilliant!!!

Today I tried typing different words and phrases into the applet, and listened to the melodies that emerged. Mostly it sounded more or less like a parody of Schönberg, with “melodies” that seemed rather arbitrary.

Then I spent some time searching for words that would produce a strong lilt. Some words, like “once”, had a lovely melody, and occasionally an entire phrase, like “who am I”, sound as though it could be the melody for a song with that title.

But for the most part the results just didn’t add up. Since my keyboard layout was mapping to the 12 notes of the chromatic somewhat randomly (given the quirkiness of QWERTY), this was to be expected.

But what if we were to map the keys in a way that tried to optimize for mapping words to good melodies? For example, we could map letters, such as “s” or “d”, that often occur at the ends of words to the tonic or dominant note of a scale. And common letter combinations like “th” could map to pleasant intervals, such as the major third.

Of course it’s hard to know, a priori, whether a mapping will be good. So one possibility would be to evolve our keyboard, starting with keys mapped to notes at random. Then we would have the computer “type” a typical text, while we listened for whether interesting melodies emerge.

Then, just as Karl Sims evolved virtual creatures through genetic algorithms, we could incrementally change our mapping in random ways, and always choose the alternative that sounded best.

Maybe I’ll try something like that. :-)

2 Responses to “Evolved virtual keyboards”

  1. x says:

    Your colored keyboard reminds me of a poem by Rimbaud, who associated vowels with colors (but not musical tones). I wonder if Rimbaud and Nabokov agreed on color and letter associations (probably not).

    Voyelles

    A noir, E blanc, I rouge, U vert, O bleu : voyelles,
    Je dirai quelque jour vos naissances latentes :
    A, noir corset velu des mouches éclatantes
    Qui bombinent autour des puanteurs cruelles,

    Golfes d’ombre ; E, candeur des vapeurs et des tentes,
    Lances des glaciers fiers, rois blancs, frissons d’ombelles ;
    I, pourpres, sang craché, rire des lèvres belles
    Dans la colère ou les ivresses pénitentes ;

    U, cycles, vibrements divins des mers virides,
    Paix des pâtis semés d’animaux, paix des rides
    Que l’alchimie imprime aux grands fronts studieux ;

    O, suprême Clairon plein des strideurs étranges,
    Silence traversés des Mondes et des Anges :
    - O l’Oméga, rayon violet de Ses Yeux!

  2. Mari says:

    Have you typed your own name? It sounds very nice :) The way the key layout is now, many words seem to fall into tritones, but Japanese spelling (which has vowels in every other letters) sounds very different from English words :)

Leave a Reply