Author: admin

Since experiencing the Valve demo (and comparing notes with colleagues who had a similar reaction), I’ve found myself being a lot more inventive. Reaching that last one percent toward a sense of “this is real” has clearly convinced me, on some deep subliminal level, that this is something worth devoting brain cells to.

I have found myself coming up with new thoughts on visual languages for communicating in a shared virtual world, ideas for how to wander freely around a large open outdoor space while experiencing VR, for how to use detection of weight shift and changes in walk to make things happen in the virtual world, and lots more.

Let’s take just that last one. Wherever you go in a walk-around VR, you’re still standing on the ground. This limits what you can do convincingly. For example, taking off into the air like Superman might be problematic, because you can still clearly feel your feet on the ground, supporting your weight.

However, flying through the air like the Silver Surfer might be just fine, since you are supposed to be standing on something. Similar arguments can be made for riding on the back of a dragon in Pandora or a giant sandworm on Arrakis.

But then it becomes important to detect subtle shifts of weight. And you want to have that experience even if you were walking around before you took flight, which means that solutions that require you to stand still, like the Wii Fit, are inadequate.

Since I now know that it’s going to matter, I’m having a splendid time working out technological solutions that will be up to the job. I am looking forward to riding that sandworm.

Sometimes in conversation you may find
My disagreements placed upon the table
And then I try to really speak my mind
But rarely do I find that I am able
To truly put my money where my mouth is
Instead I merely hem and haw and fret,
At which point things can rapidly go south. Is
There some better strategy? I’ll bet
That if we both could read each other’s thought
Instead of trying to prevaricate
We might not find ourselves to be as fraught
With conversations breaking down. Oh wait,
      If you really knew just why I think you’re wrong
      You might not stick around for very long.

One of the criticisms leveled at virtual reality is that it can be isolating. Rather than being with other people, the argument goes, VR users are spending their time in a made-up world.

As Applin and Fischer and others have noted, this criticism contains within it a very misleading view of the human condition. In fact, we all live in a made-up world. With our extensive use of clothing, medicine, housing, utensils, written language and more, even the most “back to the earth” among us are living a highly virtualized existence.

So I don’t feel that I am betraying some key principle of “authenticity” as I let my mind wander over the possibilities that have occurred to me after experiencing the Valve VR demo — possibilities that would never have occurred to me after trying on “almost good enough” technologies, such as the Oculus Rift or the various CAVE environments I have visited through the years.

And I find that all of the experiences that occur to me are ones that actually draw me closer to other people. For example, I would love to be a dolphin for the day, going on a mini-vacation with a friend — who is also being a dolphin — as we explore the great barrier reef.

Unlike human scuba divers, my friend and I will be free to chat away as we visit one fantastical undersea wonder after another. And of course we will be able to swim a lot farther and faster. 🙂

Or perhaps my friend and I can be flying dragons swooping and gliding through the floating islands of James Cameron’s Pandora. Thinking of this last possibility, I am struck by the difference between a mere stereo iMax movie and the experience of true sensory immersion.

In point of fact, I remember thinking, when I first saw the beautiful and intricate visualizations of Middle Earth in Peter Jackson’s Ring trilogy, that one day even this will seem primitive, in the way most people now think of black and white silent films as primitive.

Now, for the first time, it is clear to me what kind of future alternative experience is coming along to make that happen.

This evening I invited two friends / colleagues to dinner who had never before met each other, because it just seemed that the two of them had a lot in common intellectually, and that they really needed to meet.

After the three of us settled down to dinner, I pretty much sat back and watched the two of them interact, and I was not disappointed. It was a joy to see two people meet for the first time who had independently spent many years pondering the same deep questions with a roving and intelligent mind.

I was reminded of that old folk belief that we each have a spiritual family scattered throughout the world. Not the family we were born into, but a family that shares a common soul. According to this belief, when we meet our spiritual brothers and sisters, we will recognize them at once.

Such an encounter can feel less like a first time meeting than a kind of reunion. Tonight I feel that I was witness to such a reunion.

time once washed away
these memories and dreams, now
the rain brings them back

Yesterday evening I stayed up late trying to fix a bug in my computer program. I had a general idea what was causing it, but try as I might, I couldn’t track it down. This went on for several hours.

I didn’t feel like going to sleep without having done anything useful. So just before I retired, I put in a little kludge — a completely hacked work-around — so that the bug wouldn’t show up.

It wasn’t a solution, and the bug was still lurking somewhere in the code, but my kludge was making sure the bug wouldn’t produce any symptoms.

When I woke up this morning, I realized that if I knew enough to make the kludge work, then I knew enough to solve the underlying problem. I went back and made a small change, and sure enough, the first thing I tried was successful. I had fixed the bug!

What I then realized was that the very fact that I had known enough to create my kludge meant that somewhere, in the back of my mind, I already knew what the problem was. The knowledge just hadn’t yet percolated to the front of my brain.

The take-away lesson seems to be this: If you can’t figure out the solution to a problem, it’s ok to create a temporary fix. Your kludged up patch does not need to be elegant. It just needs to work.

The very fact that you have gotten this far means that your mind is on its way to the real solution. The kludge is your bridge to get there.

Getting something working, even if via the “wrong” solution, may be just the hint you need to get you the rest of the way.

All of the examples to date of a natural language have involved only people’s brains and bodies — nothing else. Over dinner last night some colleagues and I discussed an interesting question: Can future evolution of natural language involve computers?

We already see very simple phenomena that suggest such a thing might be underway, such as the way the Twitter community adapted Chris Messina’s idea of hashtags — a wonderful example of folksonomy.

It could very well be that folksonomic tagging is currently the most direct cybernetic analog of natural language evolution. But somehow I feel that this way of evolving language is not quite rich enough.

And here we get to the crux of the biscuit. I assert that as augmented reality becomes more better, more ubiquitous and more perceptually blended into the everyday, human communication will “come back to the body”, and the use of computers to interact with each other will no longer seem like the strangely disembodied affair it is now.

At that point, evolution of natural language itself will start to shift into new directions, since even over cyberspace we will be able to make use of those parts of our minds that have evolved to respond well to subtle bodily clues such as eye gaze, bodily stance, head tilt and arm/hand/finger gestures.

How all this might relate to computational power-ups, such as the use of machine learning and computer vision, remains an open question.

Today I finally got to see the virtual reality demo at Valve Software. And it completely blew me away.

The standard I had been used to was what I had seen before — experiences on the order of the Oculus Rift, which people in our field have been seeing for many years, and which is far inferior.

What is different about this demo is that they have all the little details right, and this is an area in which all the little details really matter. As you move your head, the world moves around you as though you are really there, with none of those little delays that tell your subconscious that what you are seeing is fake.

On the contrary, wherever I looked, it simply felt as though I was in the place I was looking at, whether those objects were creatures, walls and floors, mountains or spaceships. Which was pretty remarkable, because some of the places I “visited” were utterly fantastical.

The very last “place” was a journey through a giant abstract world from the Demo Scene — beautiful and mysterious and utterly alien, yet somehow completely real. It was this demo, more than the more practical ones (e.g.: you are in a machine room, and you can stick your head inside the machine to see how it works), which really filled me with awe about the possibilities — the sense that our experience of possible universes is truly limited only by our imagination.

But perhaps the most important thing was the absolute confirmation that the details really do matter. In this demo, unlike all the others I have seen, a threshold has been crossed, and I have seen another world.

Every time I attend a performance of a Shakespeare play (and I have seen some of those plays many times) I am amazed all over agin by the brilliance of his work. From the intricate and inventive wordplay, to the depth of the characterizations, to the perfection of each plot and character arc, these plays are both astonishing and inspirational.

Some modern playwrights, such as Tom Stoppard, can reach moments of brilliance that remind me of the Bard, but for sheer continual inventiveness, musicality, psychological richness and depth of understanding of the human condition, nothing really comes close.

I imagine readers of Racine in the original french might feel a similar regard for that playwright’s perfect Alexandrine dramaturgy. But in English, I don’t think any talent rivals that of Shakespeare.

Which leads me to wonder: Could another playwright’s collected works ever be as good? Or was such a feat of inventiveness possible only in the Elizabethan Era?

I’ve been giving talks recently about my research to many people who are not really in my field. There’s a kind of “wow, gee whiz” quality to those talks, because the people watching and listening don’t quite know how I am doing things.

However intelligent or sophisticated the audience (and some of these audiences are extremely intelligent and sophisticated), and whatever the merits of the subject or the presentation, such a talk inevitably becomes at least part magic show. The energy in the room can tend toward the theatrical.

Today, on a visit to Microsoft Research, I am showing the same work to several people who have been thinking deeply about problems similar to the ones I am tackling. Their work is different from mine, but is certainly related.

I get both more and less from these people. On the one hand, they see right through the magic, so it doesn’t really work for them. And yet, because they can often reverse engineer what I’m showing them, just by watching my demo, they can provide unique insights into what I might try next.

It is not as though either of these audiences is better than the other — both are equally valuable. Yet they are differently valuable. Being able to show my work to both is really quite wonderful.

By the way, happy Bastille Day. 🙂