Ken’s Blog

Today I accompanied a friend to a store that sells kitchen furnishings. She and her boyfriend recently moved into a new house, and they are doing a massive renovation — hence the search for kitchen furnishings.

The salesman at the store asked my friend about the layout of her kitchen, and she said it might vary. The reason is that they are considering moving the position of one of the windows.

I found myself thinking about the immense amount of effort it takes to move a window. Essentially, you need to cover over one window, both inside and out.

Then you need make a new hole in the wall somewhere else, avoiding structural support elements of course. Finally you need to finish the second window, both inside and out.

To the naive observer, it will look as though the window has simply moved its location along the wall. Yet in fact it is an entirely different window.

So I asked myself the following question: Will there ever come a day when real and virtual realities become so thoroughly mixed that such massive reconstruction is no longer necessary? Will we ever get to the point where to move a window in our house we simply slide it along the wall to another location?

I realize that there are a lot of different conversations packed into that one deceptively simple question. But that’s rather the point, isn’t it? We might as well start those conversations now.

One thing you learn when you are developing procedural animation software is that when things go wrong, they can go very wrong. Unlike creatures in real life, a virtual creature can fail in very spectacular ways.

The head can fly off the body, knees can bend the wrong way, or parts of the body can simply disappear entirely. When this sort of thing happens, all you can do is start tracking down the bug, and hope that such symptoms won’t pop up in the next public demo.

Recently I managed to track down the cause of one insidious bug. Every once in a while, when my interactive mammoth moved its legs in a particular way, it’s entire upper body would vanish.

It turned out that the problem was a negative number under a square root. For those of you who don’t know, there is no real answer to the question “what is the square root of a negative number.” So when you try to take the square root of a negative number, you get back a result that is, quite literally, not a number.

Once those first “not a number” values show up, everything starts to break. Creatures move in impossible ways, body parts disappear, and general mayhem ensues.

When I next saw my colleagues at the lab, I felt somewhat triumphant. After many hours of searching, I had indeed traced the problem to a negative number under a square root sign.

Of course by this time I was somewhat tired and burnt out and maybe a little sleep deprived. So the way I explained it to my two colleagues was this: “I know why the mammoths went extinct.”

“Why?” asked one colleague.

“Because,” I explained, “there was a negative number under the square root sign.”

To my relief, they both understood completely.

The Jurassic Park franchise is now more than a quarter of a century old. I recently watched the latest installment not really expecting to see a movie, but rather a collection of very impressive special effects.

And I was not disappointed. What I saw didn’t feel like a movie in the traditional sense. I happily watched giant dinosaurs chasing people, and I didn’t feel a bit emotionally involved. And that was ok, because the dinosaurs looked completely awesome, as expected.

I found myself wondering why I felt so detached from all the chasing and chomping happening on the big screen. At some point it occurred to me that it was because of what I now think of as the “Jurassic Park superpower”.

Basically, any character who you care about in a Jurassic Park movie is completely invulnerable. The biggest baddest carnivorous dinosaur cannot harm such a character, no matter how hard it may try.

This includes any character who is a hero, or a heroine, or a cute child, or simply has a good self-deprecating sense of humor. Which of course leaves the bad guys.

Bad guys in these movies have a sort of anti-superpower. The iron clad rule in a Jurassic Park movie is that if you are the sort of low minded fool who willfully unleashes the mighty beasts of death and destruction (usually because you want to get rich), then said beasts will inevitably devour your sorry self before the end credits roll.

It would ruin everything if anyone on-screen realized they were actually in a Jurassic Park movie. None of the good guys would ever run away in terror, the bad guys would know better than to mess with Mother Nature in the first place, and the whole enterprise would fall apart.

Good thing they don’t know.

When I was a child I learned that in some traditional folklore, it was believed that inside of every human there is a miniature fully formed human, a homunculus, which is the true source of human intelligence. Sort of like having an operator inside a robot.

Of course this leads to a reductio ad absurdum. If you follow this concept to its logical conclusion, then it’s homunculi all the way down.

Yet we do something vaguely analogous in software. We build software that is based on other software, and below that is still other software. Eventually you get down to the actual processing hardware, but these days that is indeed many levels down.

The fascinating responses to yesterday’s post revived an old idea that I had pondered quite a few years ago. Suppose you made a simulated world, and populated it with virtual people. Then suppose each of these people was actually a colony of virtual beings, collectively charged with keeping their uber-individual functioning.

Now suppose you made this recursive. It would be fascinating to implement such a system, and try to understand its properties.

Precedents abound in literature, ranging from The Thirteenth Floor to the scene with Burt Reynolds and Tony Randall in Woody Allen’s extremely loose adaptation of Everything You Always Wanted to Know About Sex * But Were Afraid to Ask.

But suppose we were to try this sort of thing for real. I suspect we would quickly learn that there are non-obvious rules to working versions of such systems.

I still remember the day I first learned to use the Unix operating system command line. I was quite young at the time.

There is one command in particular that lets you make new directories to put files in. You can then travel into any of those directories and make subdirectories. And so on, in an ever expanding tree.

There was a moment, when I was first trying this, that I thought to myself “Wow, I can create worlds within worlds, entire vast Universes!” Looking back now, it seems like a naive thought, yet I can still recall that rapturous feeling, that heady rush of infinite possibility.

Today I put my little interactive woolly mammoth into a virtual world, and then I added a second mammoth. As soon as I saw that second mammoth beside the first one, I felt exactly that same heady rush.

It’s one thing to create a virtual creature. It is quite another to create an entire world of virtual creatures.

One creature is interesting. Two or more virtual creatures can interact with one another, play together, form relationships, become part of an ever expanding story.

Making worlds is fun.

Darn her stupid cleverness.
His vanity is so unattractive.
Her seriousness is so damn funny.
He is lonely, just like his friends.
She has quite an appetite for new diets.
His greatest flaw?  It’s his perfectionism.
She is happy only when she’s feeling miserable.
He is one of those people who simply loves to hate.
She has an old fashioned need to try the latest thing.
His reliance on superior strength is his greatest weakness.

When you work on a project by yourself, you are completely free to set your own rules and conventions. This can be extremely convenient.

After all, we are all very good at communicating with ourselves. For one thing, we know our own thoughts.

But when you bring another person into the mix, things get trickier. Everything needs to be clear — and not just inside your own head.

In the last few days I have needed to make the transition from writing a piece of software all on my own to bringing in a collaborator. Which means I’ve needed to go into my code and change a lot of things, in order to make everything easier to understand and communicate.

There was a part of me that felt grumpy about this. After all, I already knew exactly where everything was before this other person ever came on board.

But then I looked at the changes I had made, and realized that the entire system was now much cleaner, more robust, and far easier to maintain. Clearly there is something intrinsically better about code that is designed for collaboration.

I suspect that this principle generalizes to other parts of one’s life. Even to those parts that don’t involve software or computers.

I went to see a friend’s VR piece at Pioneer Works in Brooklyn. Then afterward some of us went out for beers.

Two very different experiences: Hanging out while experiencing an alternate sensory world, and then hanging out together in this sensory world.

I found myself wondering whether there could ever be a viable cross-over experience. Will we ever share a beer with friends in VR?

I’m not even sure at this point what such an experience would be like. But I am quite certain that we will recognize it when (or if) it ever happens.

Today I am visiting my mother. I showed her the interactive computer animated virtual mammoth I’ve been working on. Then my mother told me about the real elephants she saw a few weeks ago. She just recently came back from a two week trip to Africa (mostly Zambia, Zimbabwe and Botswana) with some friends. They spent time most of the time in the jungle amidst the lions, giraffes, elephants and hippopotamuses.

My mother described to me one particularly memorable incident. She was observing a large elephant that was standing at the edge of a river, eating some of the tall grass that grows in the water.

Another smaller elephant also saw the yummy grass, and sauntered up to join in the feast. When the second elephant got to the water’s edge, the first elephant pushed the smaller elephant firmly away with its trunk. The smaller elephant promptly beat a retreat.

Then, as my mother watched, the second elephant slowly returned to the water’s edge, step by quiet step, taking care to not disturb its larger acquaintance. In the end, both elephants were standing side by side, happily munching on the river grass.

My mother told me she was astonished at how human this whole interaction seemed. The second elephant had clearly figured out that the problem wasn’t that the first elephant didn’t want to share the grass, but rather that the first elephant simply didn’t like to be disturbed while it was eating.

When I create an interactive procedurally animated character, I usually start with a high level sketch, in words, of who the character is. Primarily I am looking for attitude and motivation.

It really boils down to a single question: “If I were that character, what would I do, and why would I do it?”

I’ve been writing a lot of code recently to create an interactive woolly mammoth that we will be showing at the big SIGGRAPH computer graphics conference in Vancouver this coming August. To make it all work, I’ve had to figure out lots of tricky mathematics and algorithmic detail.

There are also questions of anatomy and biomechanics that are specific to pachyderms. These questions have required research and study of bone and muscle structure, as well as much experimentation. For example, like I said yesterday, how exactly does an elephant move its trunk?

Yet at the end of the day, it all comes down to understanding the character. Here are the notes I wrote for myself when I started this. These notes serve as an indispensable guide to everything I am implementing:

The mammoth thinks of herself mainly as a friendly floating head, curious about the world around her. Her main means of interacting with the world is via her trunk, so she is primarily engaged in using her trunk to interact with the world in various ways, such as carrying objects, picking up food to put in her mouth, or affectionately nuzzling her friends.

Most of the time she not very aware of her body. When she wants to move her head, her body simply moves to make this happen, while using the minimal expenditure of energy.

She is not really aware of her tail. It functions mainly to swat away flies, without her really thinking about it.

To animate the mammoth, we mainly communicate with her head, telling her which objects or other beings she is interested in, and what tasks she is engaging in. For most tasks she will use her trunk. The mammoth’s trunk is her superpower, and she knows it.