Ken's Blog

Today I am happily ensconced attending technical talks on the first day of the annual SIGGRAPH computer graphics conference in Vancouver, along with several thousand other like-minded people. There is nothing else quite like this event — the sheer number of great ideas, the combination of advanced mathematical and computational techniques in service of passionate artistic goals. I love the vast diversity to be found here. Some of the presented work aims for perfect realism, while other work creates artistic tools for far more impressionistic results.

What it all has in common is a perfect confluence of aesthetic expressiveness as a goal and top-notch mathematical engineering as a means. Talks are filled with technical terms like Principal Component Analysis, Single Value Decomposition and Hidden Markov Models, while the images one sees are of rays of light streaming through clouds, ocean waves at sunset, gracefully dancing figures, mysterious forests filled with dappled trees, and digital artworks that look like the marble sculptures, evocative pencil sketches, soft watercolors and sombre oil paintings of old masters.

It’s a bit of an arcane world — this use of shared mathematical language and computational expertise to describe the creation of things of aesthetic beauty — and it is a wonderful world, filled with passionate true believers.

Best of all, this year it is in the beautiful city of Vancouver, where the mountains meet the ocean, and the world itself seems made to inspire art.

Transitions between two ways of thinking can be complicated. For example, there are things you need to do when trying to introduce people to reasoning in four dimensions that would be unnecessary if you were talking to people who were already comfortable with four dimensional reasoning.

In that sense, the discussion about “how to enable people to think in four dimensions” reminds me of other discussions I’ve had recently about transitions. For example, this question about the (possible) transition to a world in which cars drive themselves.

The general reasoning here, which seems plausible to me, is that as computers become ever more powerful, computer driven cars will eventually become so much safer than human driven cars that the huge death toll caused by driver accidents will come to seem socially unacceptable, and eventually illegal — kind of like what has been happening with smoking in the workplace.

But the transition between a world with only human drivers and a world with only robot drivers is rather complicated. During this transition, robot drivers will need to correctly model and react to what those crazy human drivers are doing. And this can be far more difficult to compute than simply making rational decisions in a world that has only rationally acting robot cars.

Analogously, I suspect that even if we do manage to figure out how to transition people to reasoning in four dimensions, a lot of that effort will consist of relating everything back to the three dimensional reasoning people already understand (even if that isn’t the best way to think in four dimensions). I don’t really see any way out of this — we’re probably just going to have to do this the hard way.

I realized after posting the 4D Pong game yesterday, and watching peoples’ reactions, that the tricky concept is hypervolume — the equivalent of volume, but in one higher dimension.

The problem is that we have no direct experience of navigating in that very rich space. It’s deceptively simple — go up/down, left/right, in/out, and also one other dimension, but that extra dimension is one for which we simply have no intuition, since we don’t live there.

Something as simple as playing Pong — moving a box around in a hypercube and bouncing off its hyper-walls (ie: cubes, which serve the same function as the square walls on a 3D cube) — is already stretching beyond what makes sense to most people on any intuitive level, even if they get what’s going on intellectually.

I’m starting now to think about what might be good scaffolding experiences, even simpler than something like Pong, by which one could gradually “level up” to having the intuition required to navigate with ease through a hypervolume.

Well ok, here’s what I have so far. It’s clearly a work in progress, and it’s already really clear that trying to play Pong in four dimensions is hard.

Try it for yourself, by clicking on the image below:

The classic computer game Pong has two distinctive dimensions: The dimension across the rectangular board, in which you move your paddle from side to side, and the dimension along the board, where the puck travels back and forth between you and your opponent.

It’s easy to generalize Pong to make it a 3D game. The rectangular board becomes a 3D box, and you let each player move their paddle not only side to side, but also up and down.

So far so good.

I thought it might be interesting to create 4D Pong. In this version each player can move their paddle within a three dimensional space, while the puck bounces back and forth into a fourth hyper-dimension.

This afternoon I implemented 4D Pong, which turned out to be not so difficult. What did turn out to be difficult was understanding what I was seeing after I was done. Even though I knew exactly what I had created, I still had trouble following the puck as it bounced off the walls of the hyper-cubic playing field.

Now I am wondering whether this was some sort of failure of visual design on my part, or whether it is just too visually difficult to follow the movement of a careening 4D puck as it bounces around inside a hypercube, careening off the board’s eight hyper-faces. If I manage to create a version where the puck’s movement makes a modicum of visual sense, I’ll post it tomorrow.

About a year ago, in the summer of 2010, while visiting Microsoft, and knowing that they had been worried about the rise of Google, I asked people at Microsoft why they didn’t just make their Bing database available to the general public. Giving people direct access to the underlying database is something that Google literally cannot do, since hoarding that reverse index of the entire Web is Google’s treasure — the one thing to which they cannot give away access.

But Microsoft is primarily a tools company. If they said “hey world, here’s our inverse index, use it to your heart’s content, using our software tools written in C# and managed code,” then lots of really smart people would use those tools and that database to write their own search algorithms. It would be a game changer, and Microsoft’s software would be at the very center of the new game.

The highest ranking Microsoft Vice President I spoke to about this told me “That’s a very interesting idea, but it’s above my pay grade.”

And so I dropped it. Maybe, I figured, something like that would be above everybody’s pay grade. Besides, this year I’ve gotten to know some very cool people at Google, so my personal loyalties have become more diverse.

When I recently visited Microsoft, one year later, I didn’t even bother to mention it.

On the recommendation of a friend, I started reading George R.R. Martin’s “A Game of Thrones”. I’m not sure I like it — in essence it’s “The Office” with broadswords — but there are moments where the deliberately archaic writing achieves a lovely poetry.

I especially like a line I just stumbled upon: “The ravens are gone from the rookery, and the stables are full of bones.” This sentence has an elegant air of wistfulness. It sounds like the reminiscence of a old knight who has known happier times. In fact it is spoken by a young boy, while recounting a dream.

One of the fun things about the fantasy genre is the permission it gives an author to blur the line between the actual and the poetically imagined. In a world where dragons are real, and a well timed magic spell might change the course of political history, there is no clear bright line between the dreams that trouble a character’s sleep and the waking world where he or she lives.

Unfortunately, this very freedom seems to create its own restrictions, by encouraging a set of genre conventions that are in some ways more rigid than the strictures of kitchen sink realism. Apparently when magic rules the world, then such considerations as class distinction — particularly the accident of one’s parentage — become as unyielding as fundamental laws of physics.

I wonder whether there is some sort of conservation law at work here. The more freedom of movement is allowed between dream and reality in a fictional world, the more rigid and unyielding is its social order. The castles in these worlds are lovely indeed, but the stables are full of bones.

A month is such an artificial construct. We don’t even give all our months the same number of days.

And yet … and yet. When the calendar makes that grand move from one page to the next, I always feel the motion, in a powerful way that my rational mind cannot quite justify.

It might simply be the power of labels. As long as it was still July, there was time remaining to do all my July things. The ball was in motion, the game afoot, the football not yet touched down within the end zone, the final juggling pin still in the air, the last fragile leaf of the season bravely hanging on to the tenuous tree of time.

But now, alas, to misquote Corinthians 13:11, “When I was in July, I spoke of July, I understood July, I thought of July: but when it became August, I put away July things.”

So sad. So very sad.

On the other hand, now I have a whole new month to do stuff! 🙂

There are many directions we can take when trying to build a programming language from the ground up specifically for the purpose of live performance for an audience that we expect to (1) be entertained by the process, and (2) understand reasonably well the performance by programming.

The approach that comes most clearly to my mind is to think in terms of a troupe of trained “actors” that know how to take commands. For example, if the performer wished to get those actors to line up in order, from shortest to tallest (essentially a sorting algorithm) then the programming language should allow the performer to tell each actor something like “if the person to the right of you is taller than you, swap places”.

Note that the language needs to deal gracefully with exceptions such as an actor being all the way to the right (so that there is nobody to that actor’s right). In such cases, an actor should know simply to gracefully ignore the command.

In the same spirit, the performer could issue a command like: “Follow the actor to the right of you. If nobody is to the right of you, then follow the leftmost actor” to get all the actors to form a circle.

If we think of actors as carrying colors, or numbers, or having the ability to play musical notes, then we can start to think of simple ways to command those actors in ways that would be interesting to an audience. For example, the performer can tell the actors “Go into a group with all the actors who hold the same color you do. Then everyone in your group sort yourselves from shortest to tallest, and then play your respective musical notes in order.”

I realize there are lots of details missing here (such as the actual syntax of the language) the basic idea has about the right feel to it.

I was introduced today to the music of The Strokes. As I watched their artful creation of a sound that looked accidental but clearly wasn’t, I realized I was seeing an example of an entire aesthetic. There are artists who put tremendous effort into making their work seem haphazard, while exercising so much control that the “accidental” results are invariably bravura and astonishing.

Jackie Chan is famous for this in the martial arts, through his various portrayals over the years of a “Drunken master” — a martial artist who careens around as though drunk, while somehow landing every throw and kick with deadly precision.

But the idea of the drunken master spans many genres in the arts. One that comes to mind is Gracie Allen from the old Burns and Allen show. On the surface she seemed daffy to the point of near insanity, and yet everything she “accidentally” said hit home with wondrous precision. Other examples are the aphorisms of Yogi Berra, the ingeniously sly “dumb blonde” performances of Judy Holliday, and most of the film performances of Peter Sellers.

It could be argued that the half-crazed Harlequin characters in American sitcoms are all drunken masters, from Ed Norton to Jim Ignatowski to Phoebe Buffay to Cosmo Kramer, as well as dozens of others. While their ostensibly saner friends struggle to find their mooring, these blissful agents of randomness always manage to effortlessly stumble upon the deeper truths that elude others.

In some way the drunken master is that ideal self we all secretly hope we have somewhere within us — the childlike part of us that bypasses mere reason to make direct contact with the sublime.