Ken's Blog

When Roger Dannenberg, Robert Rowe and others started creating computer programs that would automatically accompany a live human musical performance well over twenty years ago, our current era of machine learning did not yet exist.

Their pioneering work was done before the development of Support Vector Machines, Convolutional Neural Nets and other powerful modern algorithmic tools of machine learning. These recent algorithmic techniques now underlie much software that we use every day, such as Google search and Google translate, and will soon be seen in our self-driving cars.

When we apply what these pioneers were trying to do to the various fields of live artistic performance (such as dance, acting, puppetry), we begin to see the modern possibilities for “smart instruments” — the piano that has “learned” your musical style, the virtual actor that embodies your unique body language, the paint brush which paints in ways that you might.

A lot of this has already been happening, in the work of Aaron Hertzmann and others, and we are poised for it to happen on a much larger scale. To be clear, we are not talking about machines replacing people. The live human performer is still very much in control, but is playing an instrument that has already been infused with styles of human performance, and can therefore be played at a higher level.

Through the use of smart instruments, a single talented performer can conduct an entire symphony, or direct a large virtual acting troupe. There is nothing mystical about this process — the human element is still very much present within these instruments. Such techniques are simply an enhanced way to distribute, and to gain the collective benefits from, old fashioned human talent.

Today we visited the Dennis Severs house. It’s quite a fascinating thing: An old London house turned into a physical re-creation, in enormous and highly personal detail, of a fictitious London family through several centuries. It’s very much like walking into a novel.

I really like this genre of art, other examples of which (more or less) are Punch Drunk’s Sleep No More and the wonderfully mysterious Museum of Jurassic Technology in Los Angeles. The basic idea is a rich novelistic fictional world presented in the form of an old fashioned house of curiosities.

The visitor is free to wander at will from room to room, examining the physical manifestations of fictional narratives, which are presented in enormous and idiosyncratic detail. The net result is an overwhelming sense of propinquity, an emergent feeling of presence not unlike the effect of reading Proust’s A la Recherche du Temps Perdu end to end.

I wonder how flexible this medium really is. For example, could one use the technique to create immersion into an everyday life of the 25th century? In other words, could such a delightfully Victorian approach to artistic engagement be effective as a form of speculative fiction?

I guess we would need to try it to find out.

Today we went to the Pompeii exhibit at the British Museum. Of course I had known about the famous events of A.D. 79, when Mount Vesuvius wiped out several cities, ejecting a spume of super-heated air that killed thousands of people in their homes within the span of a second.

But it’s different when you see it as something real and physical. The familiar elements of daily life are all there: lovely frescos on living room walls, surprisingly contemporary earrings and necklaces, cups and dishes, naughty erotic sculptures, tiny details that bridge the gap of almost two millennia. The delicately colored leaves in these ancient frescos, each so exquisitely rendered, could have been painted yesterday.

It struck me for the first time how little has changed in twenty centuries. Lives are still made of the same stuff — parents and children, people working on their gardens, young lovers, busy shopkeepers and rebellious teens. Those people could have been us.

And I realized what a strange contradiction is humanity. An individual life is so fleeting, a single leaf traced upon a wall. Yet the human project itself, this repeating cycle of lifetimes, creates endless variations through the flow of passing time.

We are painting a vast fresco throughout history, not with our art but with our selves. Yes, in many ways our lives are similar. Yet each life, so unique, so precious, is a thing of exquisite and irreplaceable beauty.

I had forgotten just how much I love London. Delays at Newark Airport, a long overnight flight, an endless line at customs followed by a crowded rush hour ride on the Piccadilly Line, all of it melted away the moment I stepped out into Covent Garden and remembered why I love this city.

Today I found myself in Brixton, Kennington, Tottenham, and various points about and in between. Riding on the top level of a double decker bus, you can really appreciate the sheer variety that is London. The youthful energy, multiethnicity and historic environs work together to create an almost alchemic sense of place.

Alas, I still don’t have the hang of the accent. Fortunately, people here seem to be very good at understanding my foreign dialect. 😉

To explore the possibilities of using permanent magnets together in different configurations, I’ve been spending the last few days writing various simulators. It turns out that HTML4 Canvas and JavaScript are great for this.

The below simulation is just built from basic magnetic dipole math. Once I had that in place, I simulated six little magnets in a hexagon pointing inward toward a stronger magnet pointing downward.

The field lines this creates are surprisingly beautiful, as you can see for yourself. Click on the image below to see an interactively viewable version:

 

Try to view it in Firefox or Chrome if you can, since Safari doesn’t properly show the delicacy of the lines.

On this very somber day, I thought it might help people to have something lighthearted to read. Which leads me to the following true story:

In a conversation with a friend recently, the topic came around to Pastafarianism. Some of you may be familiar with the Church of the Flying Spaghetti Monster. A fairly recently recent religion, it has one thing going for it: Its metaphysics is equally as grounded in verifiable evidence as are the metaphysics of all competing major religions.

But what about physics? Suppose there really is a Flying Spaghetti Monster? Would the existence of His Noodly Appendage be consistent with what we know of the physical laws of our Universe?

Eventually the conversation got around to various known physical phenomena. There are so many to consider: universal gravitation, the strong and weak nuclear forces, dark matter, black holes, the expansion of the universe.

And what about string theory? We couldn’t help but see the obvious geometric connection between strings and spaghetti. But how to reconcile such a view with the prevailing hypothesis that of the 9 or 10 spatial dimensions predicted by string theory, 6 or 7 of them actually consist of tiny circular loops?

We looked at each other in sudden enlightenment — it was so obvious once you saw it: Spaghettios!

To start off this semester, I took my class through a short history of computer graphics, by showing some of my favorite computer animations. Well most of them were computer animations, but there was one exception: The very first piece I showed was from Disney’s 1940 classic film “Fantasia”.

I told the class that I had first watched “Fantasia” when I was sixteen, and that from the moment I saw it, I knew that I wanted to make things like that. I didn’t know yet when I was sixteen that I would end up doing it with computer graphics. But really, that’s just a detail, isn’t it?

I love many things about “Fantasia”. But my favorite part is the brilliant “Night on Bald Mountain” sequence — animated by the great Bill Tytla — and that’s what we started with.

 

Why did I begin a class in computer graphics by showing something that wasn’t computer graphics? Because I wanted to make sure the class understood that art doesn’t start with any particular technique. It begins with the vision to want to make something transcendent, to create something that makes peoples’ lives richer.

The vision you have is more important than the question of which medium you end up using to achieve that vision. As it happens, I have ended up using computer graphics, but that’s really just an accident of history.

Lying in bed this morning, still half asleep, I found myself thinking back on a certain beloved TV show. It was one of those situation comedies that centers around little people and their lives, but in particular I was thinking about the sad-sack main character, who comes into work every day to verbally spar with his feisty secretary, never realizing that she is secretly in love with him.

Of course the audience knows the truth, and that’s what makes the show work. I was thinking about how eventually the actress who played the secretary left the show — one of those inevitable casting changes — and the opportunity was lost for these two people to find happiness together. Even though it was just a silly sitcom, the kind of show where you weren’t supposed to take anything too seriously, it still seemed sad to me, that missed connection.

I could see their faces so vividly in my mind, and little details like her dark pixie-cut hair, his badly fitting suit. Lying half awake, trying to remember the name of the show, and where else I had seen that actor and actress, I gradually realized that none of it was real.

There had never been such a show. My half-dreaming mind had made up the entire wistful reality in those brief moments between dream and wakefulness.

Before getting out of bed, I took a moment to mourn this show I would never see, this make-believe missed opportunity for happiness, and those two sad, funny and oddly appealing people who would never meet their true love, even in reruns.

This evening, in response to a question over email, I started wondering whether Javascript is fast enough to do simulations with many particles. So I wrote a little test program with 500 particles. Since every particle needs to interact with every other, the program needs to do more than a hundred thousand particle/particle comparisons at each animation frame.

I was happily surprised to discover that this performs just fine in my web browser. You can try it by clicking on the image below:

 

Then, just for fun, I started playing around with how the particles interact. I tried making them “sticky” so that instead of bouncing off each other they would clump together.

I was happy to see that the particles started forming and re-forming weird shapes, which make me think of alien creatures. Some of these creatures look a little scary. It’s a good thing they only exist inside a computer! Click on the image below to see the simulation:

Since discovering the Halbach array I have been thinking a lot about magnetic levitation.

Well actually, I’ve been thinking about magnetic levitation for years. I mean, who in their right mind wouldn’t want to think about magnetic levitation?

But in particular, the Halbach array is a shiny new toy that suggests all sorts of interesting approaches to magnetic levitation. To my surprise, as I have been reading about this stuff on-line, only some of those approaches seem to have been explored.

I may need to write more about this!