Today over lunch I got into a conversation with Vi Hart and Marc ten Bosch about 4D.

At one point I said that with the advent of really low cost virtual reality (the Oculus Rift retails for only $300), it looks like it is finally possible to do something I’ve long wanted to do — reach out and manipulate four dimensional objects with my own hands.

As some of you know, both Vi and Marc are very interested in the fourth dimension, and they’ve both done really interesting things with it.

Even if you get all of the graphics and haptics right, a potential problem when trying to convey a sense of manipulating 4D objects with your hands is that our bodies exist in a 3D world.

As I described my preferred way around this, it turned out that Vi had had pretty much the same idea, which is best described by asking: “How would a two dimensional creature in Flatland manipulate a three dimensional object?”

Assuming the 2D creature can’t escape the plane of Flatland, he/she can still push on whatever part of a 3D object happens to intersect that plane. And this allows some rich possibilities.

For example, a cube passing through Flatland will appear to its denizens as a polygon (see the image below). A Flatlander gripping this polygon could squeeze, applying forces entirely within the plane, to force the cube to move upward in the third dimension — or else relax that grip, to allow the cube to fall downward.

Similarly, the part of a 4D object that intersects our 3D world will appear to us as some 3D shape. We can squeeze on this 3D shape — entirely within our own 3D world — to move the 4D shape in four dimensions.

The nice thing about this approach is that everything we do to control the 4D object is done entirely within our own 3D world. This can make the whole thing a lot easier and more intuitive to learn. Especially for little kids, since it’s a good bet they will be able to learn this stuff a lot more easily than we grownups ever could.

This topic might seem arcane, but one day everybody might be sharing an augmented reality view of the world. At that point, being able to directly manipulate four dimensional objects could become a very practical skill.

After all, there’s a lot of room to put things in four dimensions. If we learn our way around 4D, we won’t need to worry about running out of places to put our virtual stuff. 🙂


Today I tried iFly indoor sky diving. Basically, a big fan blows air up from below at anywhere from 120 to 170 miles per hour, while you surf the updraft, suspended above a net at an altitude of anything from two feet to about thirty five feet.

I loved it! And it made an interesting comparison to real skydiving.

The real thing is much wilder during free fall — you don’t have control over temperature or wind speed, since you are literally falling through the sky after having dropped out of an airplane.

Compared to that, the iFly experience is a bit like skydiving in your living room. Which is not necessarily a bad thing.

While you are “flying”, there are people all around the glass enclosure watching you. I had a great time waving at them, giving the thumbs up, and occasionally making a funny face.

The wonderful science fiction short story “The Menace from Earth” by Robert Heinlein describes a similar enterprise on a moon colony. Only on the moon they are able to make the whole thing a lot bigger, having only have 1/6 earth gravity to contend with.

I would imagine the experience Heinlein describes, with lots of room to swoop and glide, would be a lot closer to real flying. I guess we’re not going to find out until moon colonies have recreational facilities.

After the iFly experience, I found myself thinking “wouldn’t it be great to have one of these for my very own!” Imagine “flying” in one of these while wearing a wireless Oculus Rift. Now that could make for one cool video game experience.

Alas, the price for even the smallest and least expensive “vertical wind tunnel” starts at about $250K, and goes up rapidly from there into the millions.

On the other hand, suppose I were to design one from scratch…

Game Developer’s Conference

The Game Developer’s Conference has been surprising in a number of ways. I realize that talking about games has greatly evolved — it is no longer merely talking about games.

Discussions here have ranged from political advocacy to real life love stories to social interventions to philophical speculations to mathematical explorations to comedy routines to literary extrapolations.

Speaking of the latter, today Richard Evans and Emily Short presented their wonderful interaction fiction system Versu, which lets you converse with Jane Austen’s characters, and you can even mix them with characters from very different cultures and literary genres to see what will happen (imagine Mr. Collins at a dinner party with a character from “Office Space”).

At the end of their presentation, Richard (who was previously the A.I. designer for The SIMS), summed up the possibilities by asking: “Suppose Buffy the Vampire Slayer, Mr. Darcy and Queen Victoria were stuck in an elevator. What would they talk about?”

When he said this, I remember thinking to myself that the answer was obvious. They would all be saying: “That phone booth was a lot larger on the inside than it looked from the outside.”

On the bus

San Francisco has a completely different rhythm from New York. Proportionally, many more people here take the bus, which is kind of intense, as buses tend to get very full (a bus is a lot smaller than a NY subway car).

Everyone here just tunes out the intensity and goes into their own world. It’s one of those things, I think, that people accept about living in the beautiful City by the Bay.

The bus drivers here are godlike beings, brusk but fair, dispensing justice as they go. They have no room for politeness, and won’t hesitate to face down someone who tries to get on without a MUNI pass, but they seem to really look out for their charges.

If you don’t have a pass (if, say, you are a clueless out-of-towner like me), you need to pay two bucks cash. Exact change, if you please.

Today I got to the bus stop downtown and realized, to my chagrin, that I had no singles. I figured I would just pay the driver a five, since my friends who were waiting for me had made dinner reservations.

When I got on the bus, I showed the driver my five dollar bill, told him I didn’t have any singles, and asked if I could just pay with this.

“Exact change,” he said impassively.

“Could I just give you this five and you can keep the change?” I asked hopefully.

He stared at me. “Why would you want to do that?”

I had no time to think of any answer but the truth. “Because I really need to get there.”

With a tired shrug, he waved me in. He wasn’t interested in taking my five. Apparently, the fact that I had offered it was payment enough.

Mystery game

I am at the annual Game Developer’s Conference. Today over lunch we were discussing the vast difference between acting in movies/television/theatre and “acting” in computer games.

There is an enormous push in many top first-person games for ever greater levels of realism. This is reflected in many talks at GDC, which focus on things like hair, clothing, subsurface scattering for skin, eye rendering, and all the other subtleties required to get real-time rendering ever closer to photo-realism.

Yet there seems to be an almost willful blindness within the field about the state of “acting” in these games. For example, my friend pointed out that there were many moments in the recent high budget “L.A. Noire” from Rockstar Games when both the camera and the virtual actors went totally still.

The psychological effect was essentially that the characters had gone dead. Something like this would simply never happen in a linear film with real people.

I pointed out that there was, in fact, one commercially successful first-person game in which there was, in fact, no bad acting.

My friends were quite interested to hear what game I could be thinking of.

MYST!” I said.

Of course, this wasn’t such great news. After all, MYST came out twenty years ago.

Invisible shoes

Going through airport security today, I noticed that the man in front of me, an older gentleman who was dressed up in a beautiful suit, was having trouble removing his extremely high quality shoes.

Then, after we had gotten through, I saw that it was also taking him quite some time to put his shoes back on.

I said to him, “You know a great way for somebody to make a lot of money, with all this crazy security? Make invisible shoes.”

He thought about this for a moment. “But if they were invisible, you wouldn’t even need to take them off.”

“Yes,” I said, “exactly!”

The new electric pen

Last month I joined a KickStarter for the 3Doodler, a cool pen that lets you write directly in the air (see below). It’s essentially a 3D printer head stuffed into a pen, which works by extruding a thin liquid string of heated plastic that quickly cools after it hits the air, letting you draw wireframe 3D shapes. No computer is needed, because you are the computer.

Today I got a notice that they are fully funded (for many times their original target), and that I could expect delivery in a few months. I am very excited. Yet all this time there’s been something familiar about this thing, and just last night I figured out what it is.

I realized it reminds me of Edison’s Electric Pen. Invented by Thomas Alva Edison back in 1875, this ingenious device used an electric motor on the pen to push a needle rapidly back and forth. As you wrote with it, you would make lots of little perforations into a sheet of paper. The paper could then be used as a stencil to run off hundreds of copies of your original drawing:

It was indeed an ingenious invention, which allowed people for the first time to make large numbers of copies of their writing. Enthusiastic letters of support were written by such illustrious personages as the Reverend Charles Dodgson.

Alas, after only three years the pen had lost its market. By 1878 another newfangled invention — the typewriter — had started to become the dominant means of creating stencils.

We can look at the bright side: Even if the 3Doodler ends up failing, it will most likely end up ushering in some other cool and inspiring technology.


This week at a graphics conference I saw a nice little talk about crowd simulation. As I watched the presentation, it occurred to me that unless one does exhaustive measurements of real crowds, this is one of those situations where “success” just means that things look right: If people look at your simulation and believe they are seeing real crowd behavior, then you’re good.

So I decided to try my hand at crowd simulation, to figure out the simplest approach that would visually appear to act like actual crowds. It was surprisingly simple to do — within half an hour I had something reasonable, and then after another hour or so of tweaking, I was quite happy with the final result.

Taking my cue from the paper presentation I’d seen, my “test” was four different crowds of people trying to swap places — a crowd to the East swaps places with one to the West, while at the same time a crowd to the North swaps places with one the South.

As you may imagine, things can get chaotic. The simulation needs to convey a sense that people are streaming past each other intelligently, without bumping into one another.

For a first try I think I did pretty well. You can see the result by clicking on the below image.

Virtual beer

I was hanging out this evening with a group of top researchers in the fields of augmented and virtual reality. The conversation ranged from opinions about Google Glass to assessments of the Oculus Rift to various speculations about possible directions for augmented human perception.

It occurred to me at some point in the conversation that while just about everyone around the table was carrying a SmartPhone, nobody was looking at their phone. We were all too busy sitting around with a drink in our hand, while having a grand old time discussing the future.

So at some point I lifted my glass, and I said “We will know that augmented reality has truly arrived when we can experience it while sharing a beer.”

There was a thoughtful pause, as people weighed this thought in their minds. After all, you can’t really share a drink with a friend while you’re staring down at a little glowing rectangle in your hand.

Which is all just fine with me. For now, and perhaps for a while to come, the world is safe for non-augmented social drinking.