Performance by programming

There are many directions we can take when trying to build a programming language from the ground up specifically for the purpose of live performance for an audience that we expect to (1) be entertained by the process, and (2) understand reasonably well the performance by programming.

The approach that comes most clearly to my mind is to think in terms of a troupe of trained “actors” that know how to take commands. For example, if the performer wished to get those actors to line up in order, from shortest to tallest (essentially a sorting algorithm) then the programming language should allow the performer to tell each actor something like “if the person to the right of you is taller than you, swap places”.

Note that the language needs to deal gracefully with exceptions such as an actor being all the way to the right (so that there is nobody to that actor’s right). In such cases, an actor should know simply to gracefully ignore the command.

In the same spirit, the performer could issue a command like: “Follow the actor to the right of you. If nobody is to the right of you, then follow the leftmost actor” to get all the actors to form a circle.

If we think of actors as carrying colors, or numbers, or having the ability to play musical notes, then we can start to think of simple ways to command those actors in ways that would be interesting to an audience. For example, the performer can tell the actors “Go into a group with all the actors who hold the same color you do. Then everyone in your group sort yourselves from shortest to tallest, and then play your respective musical notes in order.”

I realize there are lots of details missing here (such as the actual syntax of the language) the basic idea has about the right feel to it.

Drunken master

I was introduced today to the music of The Strokes. As I watched their artful creation of a sound that looked accidental but clearly wasn’t, I realized I was seeing an example of an entire aesthetic. There are artists who put tremendous effort into making their work seem haphazard, while exercising so much control that the “accidental” results are invariably bravura and astonishing.

Jackie Chan is famous for this in the martial arts, through his various portrayals over the years of a “Drunken master” — a martial artist who careens around as though drunk, while somehow landing every throw and kick with deadly precision.

But the idea of the drunken master spans many genres in the arts. One that comes to mind is Gracie Allen from the old Burns and Allen show. On the surface she seemed daffy to the point of near insanity, and yet everything she “accidentally” said hit home with wondrous precision. Other examples are the aphorisms of Yogi Berra, the ingeniously sly “dumb blonde” performances of Judy Holliday, and most of the film performances of Peter Sellers.

It could be argued that the half-crazed Harlequin characters in American sitcoms are all drunken masters, from Ed Norton to Jim Ignatowski to Phoebe Buffay to Cosmo Kramer, as well as dozens of others. While their ostensibly saner friends struggle to find their mooring, these blissful agents of randomness always manage to effortlessly stumble upon the deeper truths that elude others.

In some way the drunken master is that ideal self we all secretly hope we have somewhere within us — the childlike part of us that bypasses mere reason to make direct contact with the sublime.

Computer programming as performance

Suppose we apply the structure of composition → researsal → performance to computer programming? This has been a topic of conversation recently between Murphy Stein, Vi Hart and myself.

Currently, computer programming is generally thought of as a strictly compositional activity. A programmer generally creates a program in relative isolation, over a period of time, iterating until the program’s behavior is ready to be experienced by others. The programming languages and environments that have evolved to support this activity very much privilege this way of working.

But what if the semantic constructs of programming — sequences of variable assignment, iteration, conditional execution, defining procedures, building hierarchies of objects — were part of some sort of live performance for an audience?

Suppose we were to design a form of programming from the ground up, specifically to be a performative medium? It wouldn’t replace programming as composition, but rather would complement it. Much of the program would still be pre-written, as a compositional stage. In such a paradigm, one would also expect a rehearsal stage.

This splitting of computer programming into two such different modalities is not as odd as it might sound. After all, this is what happens with plays and music. While a playwright shares the medium of words — the semantic level — with the actor, their modes of expression are wildly different. The playwright expresses through typed or written words on paper, whereas the actor uses voice, facial expression and body movement. What they share is the underlying meaning.

Similarly, a musical composer writes down notes on a staff, whereas a musician plays a physical instrument. What they have in common is, again, the underlying meaning.

A kind of performative programming is done now in the avant guarde music community. Performers manipulate programs written in Max/MSP (or its open-source counterpart PD) to create live variations in the procedure controlling an algorithmically generated or modified composition.

Personally, I’ve never found the experience of attending such concerts to be entirely satisfying, since Max/MSP was never written for this purpose. Max/MSP programming on stage usually remains relatively opaque to the audience, even if audience members can see the computer screen.

It would be fun to design a programming language and environment specifically to be a medium for performance in front of an audience — with the understanding that much prior composition has already been done to scaffold what the audience is seeing.

A programming modality that privileges performance would be useful not just for music, but also for dance or theatre. One could imagine an evolving performance acted out by robot actors or dancers. As the performance progresses, the human performer modifies the procedural behavior of the actors. The audience experience consists not only of observing the actions of the robot actors or dancers, but also of understanding the rules those “performers” are following.

Compose / rehearse / perform

Theatre, dance and musical performance generally share the following structure: There is an initial stage where the work is composed, then a period when it is rehearsed, and finally the work is performed one or more times in front of an audience.

Each of these three stages has its own distinct properties. Composition is often (although not always) done by one individual, generally over a long period of time. For example, a work that an audience will experience for ninety minutes might require months or even years to compose.

Researsal is interesting, in that it takes place in an intermediate time-frame — generally performative, and yet not at all optimized for an audience. For example, a group researsing a work of theatre or music might repeat the same passage many times.

The audience on some level senses all this prior effort at the time of actual performance, but those efforts cannot subsume the performance itself. In the implied contract with one’s audience, the considerable sweat and toil of composition and rehearsal need to remain offstage.

One of the exciting qualities of of this three part structure lies in the rich opportunities it affords for feedback and iteration. Either researsal or performance can lead to modification of the composition itself. Unlike, say, a film, a theatrical, dance or musical performance is never entirely fixed — each performance is unique, and is subtly influenced by that evening’s audience.

What if we apply this structure of composition/rehearsal/performance to other domains? That’s a topic for tomorrow.

My ISP had technical problems the day this post was created, so apologies if you are only first seeing it the following day.


This evening I was walking along the street in Greenwich Village, minding my own business, when suddenly my eyes were pleasantly assaulted by an exhibit in a nearby window — a retrospective of drawings by Al Hirschfeld.

Now, this next part will either make perfect sense to you, or it will seem completely nuts. For the next half hour, like a kid in a candy store, I stood on that sidewalk drifting happily from drawing to drawing, in a state of utter bliss, counting Ninas.

I remember exactly the day, when I was twelve years old, that I was first initiated into the mystery of Nina. On that fateful Sunday morning, somebody pointed out to me that in every brilliant Hirschfeld caricature on the cover of the New York Times Arts and Leisure section, there was a little number to the right of his signature. And they explained just what that number meant.

From that day forward my teenage soul was hooked. Each week I would wait faithfully for that weekend’s paper. When it arrived, the very first thing I did, before doing the crossword, before checking out movie reviews, even before reading letters to the editor in the NY Times Magazine section, was worship at the church of Nina.

I could explain more about this, but that’s what Google is for, right?

The King in Yellow

I have been reading The King in Yellow, Robert Chambers’ masterful collection of gothic short stories from 1895. It is odd to finally read something that was such a powerful root influence upon authors whose works I have known for years, from Lovecraft to Heinlein and Silverberg to Neil Gaiman and Stephen King. Also, quite evidently, Harlan Ellison.

A unifying theme behind some of the stories — the existence of a text that makes its reader go mad, simply through the sheer seductiveness of its ideas — seems to post deep questions about the nature of literature, communication and human interconnectedness. Could mere words and expressed thoughts indeed have the power to drive us insane, to separate us from whatever hold we may have upon objective (or at least collective) reality?

Whatever side you come down on in answering that question, the implications are profound. I suspect Jung would have responded in the affirmative, whereas most thinkers today would say no.

Could such a text in fact exist? I find the question interesting, but I’m not going to, you know, go crazy trying to figure it out. 🙂

Technology is easy.

It always amazes me when people say, upon hearing that I work with computers and technology: “oh, that must be so complicated!”

I can’t help wondering, when I think about how complicated we humans are, and how complex the relationships between us, just what these people are talking about.

A computer is, in fact, eminently learnable. If you put in the time to study a computer language, technology or interface protocol, you will indeed master the instrument. No matter how complex it might seem, technology is predictable — if you poke it in a particular way, it will respond the same thing every time.

People, in contrast, are utterly unpredictable, capable of immense generosity and equally immense venality, often in the space of a single breath. Just when you think you know what’s going on, human minds will charge off in some completely unexpected direction. It is both our tragedy and our glory.

To paraphrase a grand old quote: Technology is easy. People are hard.


I find it heartening that so many people weighed in on the question of children learning a second language through the aid of on-line communities, and related questions.

It’s becoming clear that this is not one question but a web of related questions. For example, experiences for teenagers (like the one John Nordlinger started around W.O.W.) are clearly going to be vastly different from experiences for six year olds, in almost every conceivable way.

Also, as Manooh points out, there might be much opportunity simply in making the right resource available to kids, and letting them build from there, as a community of peers. The wonderful thing about that, if we can get it to work, is that we might be able to achieve a transformative level of change out of an affordable investment in resources.

Personally I find myself drawn to creating the fun exploratory environment that promotes multilingual capability for children who are still in their language acquisition age range (seven and younger). There is something so cool about tapping into that powerful potential. Besides, it gives me more excuses to play with cute little fish characters. 🙂

By the way, people I know at Google have told me that they are not actually phasing out the experiments in Google Labs, but rather just reorganizing how they roll them out. As long as these wonderful experiments keep coming, I’m happy!

A creation of the mind of children

Steven Pinker once said that “Language is essentially a creation of the mind of children.” He also referenced, in his book “The Language Instinct”, a study that showed that when you try to teach children Esperanto (an early artificially constructed attempt at a “universal language”), the children spontaneously start to fix it — they immediately start to change the language itself rather than learning to speak it. Esperanto does not conform to the meta-grammar common to all natural languages. It is not naturally learnable by children.

Yesterday J. Peterson commented on my post about using on-line games as a way to get kids around the world to each other their respective languages. The comment focused on the idea of introducing a shared invented language — perhaps Esperanto, or Klingon, or Elvish. It’s not prohibitively difficult to create such a language. There are even books out there to help you along, such as “In the Land of Invented Languages” by Arika Okrent and “The Language Construction Kit” by Mark Rosenfelder.

Yet I see a problem with this approach: Such languages, unlike the hundreds of naturally evolved languages, are not “naturally learnable” by young minds. Children don’t just naturally learn an arbitrary language — they will only naturally learn languages that have the features found in natural languages like English, French, Chinese, Swahili, etc.

I could see getting a group of small children to devise a new shared natural language on-lline, much as a community of children spontaneously created Hawaiian Creole, or Nicaraguan Sign Language. Such a project could be very interesting, although it’s not immediately clear how one would go about doing that. I’m just not sure what advantage it would have over simply getting children to play shared games in which they learn each other’s natural language.

Invested engagement

In the last several days I have participated in several discussions that touch on the question of how to increase children’s knowledge of other languages by getting kids from different countries to communicate with each other on-line.

This topic is in some ways a response to the problematic nature of school-based teaching of a second language, which usually comes too late to take advantage of the “sweet spot” for proficiency in language acquisition (up to around age seven), while often devolving to teachers who are ill equipped for the task.

Putting aside for the moment the issues of security and provenance, what constitutes an effective way to use on-line social engagement between kids to promote mutual language acquisition? It came up during one of these discussions that mere conversation, such as Skype chat, seems not to work very well, whereas anecdotal evidence suggests that interacting with others by playing an on-line strategy game, such as “Worlds of Warcraft”, might be a much more effective way to learn elements of a second language.

This suggests that it is not merely engagement in conversation which leads to second language learning, but invested engagement — there needs to be something at stake.

Which leads to the question — could we construct an international on-line multi-player game that specifically promotes multilingual skills through invested engagement? And if so, how would we operationalize the question of how effective such a game is?

One strategy might be to build in design variations along various dimensions, instrument the game to measure effectiveness for language acquisition, and then gradually tune our design variables, in response to gathered data, so as to optimize the game for maximum language acquisition.

Another strategy might be to reward actions in the game that require multi-lingual proficiency — such as the ability to say a “magic spell” in two different languages, as in the recitation of the two spells: “place the magical cup upon the rock” and “coloque o copo mágico sobre a rocha”.

It might also be most effective to target such games toward kids seven years of age and younger, when facility for language acquisition is at its peak. This suggests, among other things, using spoken words and pictures in the game mechanic, rather than relying on written language.

There are so many possibilities — as well as unique challenges, such as the difficulty of building any computer software that must understand the speech of younger children.

Wouldn’t it be wonderful if such a project were successful, and on-line game play could lead to greater understanding between people around the world?