Ken's Blog

I realized after posting the 4D Pong game yesterday, and watching peoples’ reactions, that the tricky concept is hypervolume — the equivalent of volume, but in one higher dimension.

The problem is that we have no direct experience of navigating in that very rich space. It’s deceptively simple — go up/down, left/right, in/out, and also one other dimension, but that extra dimension is one for which we simply have no intuition, since we don’t live there.

Something as simple as playing Pong — moving a box around in a hypercube and bouncing off its hyper-walls (ie: cubes, which serve the same function as the square walls on a 3D cube) — is already stretching beyond what makes sense to most people on any intuitive level, even if they get what’s going on intellectually.

I’m starting now to think about what might be good scaffolding experiences, even simpler than something like Pong, by which one could gradually “level up” to having the intuition required to navigate with ease through a hypervolume.

Well ok, here’s what I have so far. It’s clearly a work in progress, and it’s already really clear that trying to play Pong in four dimensions is hard.

Try it for yourself, by clicking on the image below:

The classic computer game Pong has two distinctive dimensions: The dimension across the rectangular board, in which you move your paddle from side to side, and the dimension along the board, where the puck travels back and forth between you and your opponent.

It’s easy to generalize Pong to make it a 3D game. The rectangular board becomes a 3D box, and you let each player move their paddle not only side to side, but also up and down.

So far so good.

I thought it might be interesting to create 4D Pong. In this version each player can move their paddle within a three dimensional space, while the puck bounces back and forth into a fourth hyper-dimension.

This afternoon I implemented 4D Pong, which turned out to be not so difficult. What did turn out to be difficult was understanding what I was seeing after I was done. Even though I knew exactly what I had created, I still had trouble following the puck as it bounced off the walls of the hyper-cubic playing field.

Now I am wondering whether this was some sort of failure of visual design on my part, or whether it is just too visually difficult to follow the movement of a careening 4D puck as it bounces around inside a hypercube, careening off the board’s eight hyper-faces. If I manage to create a version where the puck’s movement makes a modicum of visual sense, I’ll post it tomorrow.

About a year ago, in the summer of 2010, while visiting Microsoft, and knowing that they had been worried about the rise of Google, I asked people at Microsoft why they didn’t just make their Bing database available to the general public. Giving people direct access to the underlying database is something that Google literally cannot do, since hoarding that reverse index of the entire Web is Google’s treasure — the one thing to which they cannot give away access.

But Microsoft is primarily a tools company. If they said “hey world, here’s our inverse index, use it to your heart’s content, using our software tools written in C# and managed code,” then lots of really smart people would use those tools and that database to write their own search algorithms. It would be a game changer, and Microsoft’s software would be at the very center of the new game.

The highest ranking Microsoft Vice President I spoke to about this told me “That’s a very interesting idea, but it’s above my pay grade.”

And so I dropped it. Maybe, I figured, something like that would be above everybody’s pay grade. Besides, this year I’ve gotten to know some very cool people at Google, so my personal loyalties have become more diverse.

When I recently visited Microsoft, one year later, I didn’t even bother to mention it.

On the recommendation of a friend, I started reading George R.R. Martin’s “A Game of Thrones”. I’m not sure I like it — in essence it’s “The Office” with broadswords — but there are moments where the deliberately archaic writing achieves a lovely poetry.

I especially like a line I just stumbled upon: “The ravens are gone from the rookery, and the stables are full of bones.” This sentence has an elegant air of wistfulness. It sounds like the reminiscence of a old knight who has known happier times. In fact it is spoken by a young boy, while recounting a dream.

One of the fun things about the fantasy genre is the permission it gives an author to blur the line between the actual and the poetically imagined. In a world where dragons are real, and a well timed magic spell might change the course of political history, there is no clear bright line between the dreams that trouble a character’s sleep and the waking world where he or she lives.

Unfortunately, this very freedom seems to create its own restrictions, by encouraging a set of genre conventions that are in some ways more rigid than the strictures of kitchen sink realism. Apparently when magic rules the world, then such considerations as class distinction — particularly the accident of one’s parentage — become as unyielding as fundamental laws of physics.

I wonder whether there is some sort of conservation law at work here. The more freedom of movement is allowed between dream and reality in a fictional world, the more rigid and unyielding is its social order. The castles in these worlds are lovely indeed, but the stables are full of bones.

A month is such an artificial construct. We don’t even give all our months the same number of days.

And yet … and yet. When the calendar makes that grand move from one page to the next, I always feel the motion, in a powerful way that my rational mind cannot quite justify.

It might simply be the power of labels. As long as it was still July, there was time remaining to do all my July things. The ball was in motion, the game afoot, the football not yet touched down within the end zone, the final juggling pin still in the air, the last fragile leaf of the season bravely hanging on to the tenuous tree of time.

But now, alas, to misquote Corinthians 13:11, “When I was in July, I spoke of July, I understood July, I thought of July: but when it became August, I put away July things.”

So sad. So very sad.

On the other hand, now I have a whole new month to do stuff! 🙂

There are many directions we can take when trying to build a programming language from the ground up specifically for the purpose of live performance for an audience that we expect to (1) be entertained by the process, and (2) understand reasonably well the performance by programming.

The approach that comes most clearly to my mind is to think in terms of a troupe of trained “actors” that know how to take commands. For example, if the performer wished to get those actors to line up in order, from shortest to tallest (essentially a sorting algorithm) then the programming language should allow the performer to tell each actor something like “if the person to the right of you is taller than you, swap places”.

Note that the language needs to deal gracefully with exceptions such as an actor being all the way to the right (so that there is nobody to that actor’s right). In such cases, an actor should know simply to gracefully ignore the command.

In the same spirit, the performer could issue a command like: “Follow the actor to the right of you. If nobody is to the right of you, then follow the leftmost actor” to get all the actors to form a circle.

If we think of actors as carrying colors, or numbers, or having the ability to play musical notes, then we can start to think of simple ways to command those actors in ways that would be interesting to an audience. For example, the performer can tell the actors “Go into a group with all the actors who hold the same color you do. Then everyone in your group sort yourselves from shortest to tallest, and then play your respective musical notes in order.”

I realize there are lots of details missing here (such as the actual syntax of the language) the basic idea has about the right feel to it.

I was introduced today to the music of The Strokes. As I watched their artful creation of a sound that looked accidental but clearly wasn’t, I realized I was seeing an example of an entire aesthetic. There are artists who put tremendous effort into making their work seem haphazard, while exercising so much control that the “accidental” results are invariably bravura and astonishing.

Jackie Chan is famous for this in the martial arts, through his various portrayals over the years of a “Drunken master” — a martial artist who careens around as though drunk, while somehow landing every throw and kick with deadly precision.

But the idea of the drunken master spans many genres in the arts. One that comes to mind is Gracie Allen from the old Burns and Allen show. On the surface she seemed daffy to the point of near insanity, and yet everything she “accidentally” said hit home with wondrous precision. Other examples are the aphorisms of Yogi Berra, the ingeniously sly “dumb blonde” performances of Judy Holliday, and most of the film performances of Peter Sellers.

It could be argued that the half-crazed Harlequin characters in American sitcoms are all drunken masters, from Ed Norton to Jim Ignatowski to Phoebe Buffay to Cosmo Kramer, as well as dozens of others. While their ostensibly saner friends struggle to find their mooring, these blissful agents of randomness always manage to effortlessly stumble upon the deeper truths that elude others.

In some way the drunken master is that ideal self we all secretly hope we have somewhere within us — the childlike part of us that bypasses mere reason to make direct contact with the sublime.

Suppose we apply the structure of composition → researsal → performance to computer programming? This has been a topic of conversation recently between Murphy Stein, Vi Hart and myself.

Currently, computer programming is generally thought of as a strictly compositional activity. A programmer generally creates a program in relative isolation, over a period of time, iterating until the program’s behavior is ready to be experienced by others. The programming languages and environments that have evolved to support this activity very much privilege this way of working.

But what if the semantic constructs of programming — sequences of variable assignment, iteration, conditional execution, defining procedures, building hierarchies of objects — were part of some sort of live performance for an audience?

Suppose we were to design a form of programming from the ground up, specifically to be a performative medium? It wouldn’t replace programming as composition, but rather would complement it. Much of the program would still be pre-written, as a compositional stage. In such a paradigm, one would also expect a rehearsal stage.

This splitting of computer programming into two such different modalities is not as odd as it might sound. After all, this is what happens with plays and music. While a playwright shares the medium of words — the semantic level — with the actor, their modes of expression are wildly different. The playwright expresses through typed or written words on paper, whereas the actor uses voice, facial expression and body movement. What they share is the underlying meaning.

Similarly, a musical composer writes down notes on a staff, whereas a musician plays a physical instrument. What they have in common is, again, the underlying meaning.

A kind of performative programming is done now in the avant guarde music community. Performers manipulate programs written in Max/MSP (or its open-source counterpart PD) to create live variations in the procedure controlling an algorithmically generated or modified composition.

Personally, I’ve never found the experience of attending such concerts to be entirely satisfying, since Max/MSP was never written for this purpose. Max/MSP programming on stage usually remains relatively opaque to the audience, even if audience members can see the computer screen.

It would be fun to design a programming language and environment specifically to be a medium for performance in front of an audience — with the understanding that much prior composition has already been done to scaffold what the audience is seeing.

A programming modality that privileges performance would be useful not just for music, but also for dance or theatre. One could imagine an evolving performance acted out by robot actors or dancers. As the performance progresses, the human performer modifies the procedural behavior of the actors. The audience experience consists not only of observing the actions of the robot actors or dancers, but also of understanding the rules those “performers” are following.

Theatre, dance and musical performance generally share the following structure: There is an initial stage where the work is composed, then a period when it is rehearsed, and finally the work is performed one or more times in front of an audience.

Each of these three stages has its own distinct properties. Composition is often (although not always) done by one individual, generally over a long period of time. For example, a work that an audience will experience for ninety minutes might require months or even years to compose.

Researsal is interesting, in that it takes place in an intermediate time-frame — generally performative, and yet not at all optimized for an audience. For example, a group researsing a work of theatre or music might repeat the same passage many times.

The audience on some level senses all this prior effort at the time of actual performance, but those efforts cannot subsume the performance itself. In the implied contract with one’s audience, the considerable sweat and toil of composition and rehearsal need to remain offstage.

One of the exciting qualities of of this three part structure lies in the rich opportunities it affords for feedback and iteration. Either researsal or performance can lead to modification of the composition itself. Unlike, say, a film, a theatrical, dance or musical performance is never entirely fixed — each performance is unique, and is subtly influenced by that evening’s audience.

What if we apply this structure of composition/rehearsal/performance to other domains? That’s a topic for tomorrow.

My ISP had technical problems the day this post was created, so apologies if you are only first seeing it the following day.