Ken’s Blog

One of the criticisms leveled at virtual reality is that it can be isolating. Rather than being with other people, the argument goes, VR users are spending their time in a made-up world.

As Applin and Fischer and others have noted, this criticism contains within it a very misleading view of the human condition. In fact, we all live in a made-up world. With our extensive use of clothing, medicine, housing, utensils, written language and more, even the most “back to the earth” among us are living a highly virtualized existence.

So I don’t feel that I am betraying some key principle of “authenticity” as I let my mind wander over the possibilities that have occurred to me after experiencing the Valve VR demo — possibilities that would never have occurred to me after trying on “almost good enough” technologies, such as the Oculus Rift or the various CAVE environments I have visited through the years.

And I find that all of the experiences that occur to me are ones that actually draw me closer to other people. For example, I would love to be a dolphin for the day, going on a mini-vacation with a friend — who is also being a dolphin — as we explore the great barrier reef.

Unlike human scuba divers, my friend and I will be free to chat away as we visit one fantastical undersea wonder after another. And of course we will be able to swim a lot farther and faster.

Or perhaps my friend and I can be flying dragons swooping and gliding through the floating islands of James Cameron’s Pandora. Thinking of this last possibility, I am struck by the difference between a mere stereo iMax movie and the experience of true sensory immersion.

In point of fact, I remember thinking, when I first saw the beautiful and intricate visualizations of Middle Earth in Peter Jackson’s Ring trilogy, that one day even this will seem primitive, in the way most people now think of black and white silent films as primitive.

Now, for the first time, it is clear to me what kind of future alternative experience is coming along to make that happen.

This evening I invited two friends / colleagues to dinner who had never before met each other, because it just seemed that the two of them had a lot in common intellectually, and that they really needed to meet.

After the three of us settled down to dinner, I pretty much sat back and watched the two of them interact, and I was not disappointed. It was a joy to see two people meet for the first time who had independently spent many years pondering the same deep questions with a roving and intelligent mind.

I was reminded of that old folk belief that we each have a spiritual family scattered throughout the world. Not the family we were born into, but a family that shares a common soul. According to this belief, when we meet our spiritual brothers and sisters, we will recognize them at once.

Such an encounter can feel less like a first time meeting than a kind of reunion. Tonight I feel that I was witness to such a reunion.

time once washed away
these memories and dreams, now
the rain brings them back

Yesterday evening I stayed up late trying to fix a bug in my computer program. I had a general idea what was causing it, but try as I might, I couldn’t track it down. This went on for several hours.

I didn’t feel like going to sleep without having done anything useful. So just before I retired, I put in a little kludge — a completely hacked work-around — so that the bug wouldn’t show up.

It wasn’t a solution, and the bug was still lurking somewhere in the code, but my kludge was making sure the bug wouldn’t produce any symptoms.

When I woke up this morning, I realized that if I knew enough to make the kludge work, then I knew enough to solve the underlying problem. I went back and made a small change, and sure enough, the first thing I tried was successful. I had fixed the bug!

What I then realized was that the very fact that I had known enough to create my kludge meant that somewhere, in the back of my mind, I already knew what the problem was. The knowledge just hadn’t yet percolated to the front of my brain.

The take-away lesson seems to be this: If you can’t figure out the solution to a problem, it’s ok to create a temporary fix. Your kludged up patch does not need to be elegant. It just needs to work.

The very fact that you have gotten this far means that your mind is on its way to the real solution. The kludge is your bridge to get there.

Getting something working, even if via the “wrong” solution, may be just the hint you need to get you the rest of the way.

All of the examples to date of a natural language have involved only people’s brains and bodies — nothing else. Over dinner last night some colleagues and I discussed an interesting question: Can future evolution of natural language involve computers?

We already see very simple phenomena that suggest such a thing might be underway, such as the way the Twitter community adapted Chris Messina’s idea of hashtags — a wonderful example of folksonomy.

It could very well be that folksonomic tagging is currently the most direct cybernetic analog of natural language evolution. But somehow I feel that this way of evolving language is not quite rich enough.

And here we get to the crux of the biscuit. I assert that as augmented reality becomes more better, more ubiquitous and more perceptually blended into the everyday, human communication will “come back to the body”, and the use of computers to interact with each other will no longer seem like the strangely disembodied affair it is now.

At that point, evolution of natural language itself will start to shift into new directions, since even over cyberspace we will be able to make use of those parts of our minds that have evolved to respond well to subtle bodily clues such as eye gaze, bodily stance, head tilt and arm/hand/finger gestures.

How all this might relate to computational power-ups, such as the use of machine learning and computer vision, remains an open question.

Today I finally got to see the virtual reality demo at Valve Software. And it completely blew me away.

The standard I had been used to was what I had seen before — experiences on the order of the Oculus Rift, which people in our field have been seeing for many years, and which is far inferior.

What is different about this demo is that they have all the little details right, and this is an area in which all the little details really matter. As you move your head, the world moves around you as though you are really there, with none of those little delays that tell your subconscious that what you are seeing is fake.

On the contrary, wherever I looked, it simply felt as though I was in the place I was looking at, whether those objects were creatures, walls and floors, mountains or spaceships. Which was pretty remarkable, because some of the places I “visited” were utterly fantastical.

The very last “place” was a journey through a giant abstract world from the Demo Scene — beautiful and mysterious and utterly alien, yet somehow completely real. It was this demo, more than the more practical ones (e.g.: you are in a machine room, and you can stick your head inside the machine to see how it works), which really filled me with awe about the possibilities — the sense that our experience of possible universes is truly limited only by our imagination.

But perhaps the most important thing was the absolute confirmation that the details really do matter. In this demo, unlike all the others I have seen, a threshold has been crossed, and I have seen another world.

Every time I attend a performance of a Shakespeare play (and I have seen some of those plays many times) I am amazed all over agin by the brilliance of his work. From the intricate and inventive wordplay, to the depth of the characterizations, to the perfection of each plot and character arc, these plays are both astonishing and inspirational.

Some modern playwrights, such as Tom Stoppard, can reach moments of brilliance that remind me of the Bard, but for sheer continual inventiveness, musicality, psychological richness and depth of understanding of the human condition, nothing really comes close.

I imagine readers of Racine in the original french might feel a similar regard for that playwright’s perfect Alexandrine dramaturgy. But in English, I don’t think any talent rivals that of Shakespeare.

Which leads me to wonder: Could another playwright’s collected works ever be as good? Or was such a feat of inventiveness possible only in the Elizabethan Era?

I’ve been giving talks recently about my research to many people who are not really in my field. There’s a kind of “wow, gee whiz” quality to those talks, because the people watching and listening don’t quite know how I am doing things.

However intelligent or sophisticated the audience (and some of these audiences are extremely intelligent and sophisticated), and whatever the merits of the subject or the presentation, such a talk inevitably becomes at least part magic show. The energy in the room can tend toward the theatrical.

Today, on a visit to Microsoft Research, I am showing the same work to several people who have been thinking deeply about problems similar to the ones I am tackling. Their work is different from mine, but is certainly related.

I get both more and less from these people. On the one hand, they see right through the magic, so it doesn’t really work for them. And yet, because they can often reverse engineer what I’m showing them, just by watching my demo, they can provide unique insights into what I might try next.

It is not as though either of these audiences is better than the other — both are equally valuable. Yet they are differently valuable. Being able to show my work to both is really quite wonderful.

By the way, happy Bastille Day.

Riding on the bus from Vancouver to Seattle today, I was listening to Simon and Garfunkel songs (an excellent activity for such a trip), when the line in the title of this post jumped out at me from the song “Faking It”.

Those of you who are up on your S&G arcana know that this line is part of a very sly and obscure reference in the song to Paul Simon’s fellow songwriter Donovan Leitch. What struck me about the phrase “I am the tailor’s face and hands” was my realization that it could be dropped into just about any conversation as a kind of ersatz Zen Koan.

Then it occurred to me that it would be really fun to cull other equally curious lines from old pop songs, such as “I wandered through my playing cards”, “to live outside the law, you must be honest”, “I told you when I came I was a stranger”, “the lock upon my garden gate’s a snail”, “the shadow boys are breaking all the laws”, or “blue songs are like tattoos” — I’m sure you can think of many more — and weave them into a dialogue.

Each such little snatch of lyric seems to hint at a mysterious personal story that remains just out of our view. Maybe they could be pieced together into a single consistent narrative, a revelation of secrets of the heart. If this were artfully done, the result might be a lovely story from some alternate and highly evocative universe.

I was having a conversation today with a colleague about the different kinds of students I have. I was explaining that students in my classes seem to fall into one of two categories (with some overlap).

There is the student who is taking the course out of sheer love of the subject. This student is eager to learn, and is continually looking for more opportunities to dive deeper, and to tackle something more challenging.

Then there is the student who is “taking it for the grade”. This student generally wants to know the minimum work needed to do to get an A, whether there will be a final exam, and how much the homeworks count in the course.

To me these are entirely different categories of people, and it’s hard for me to think of them in the same way. The ones who are in it for the love of learning are the students who make my heart sing. I think of them as fellow travelers and as future colleagues.

The others — well, they are just a necessary part of the landscape. Their presence in the class is really a consequence of the structure of our educational system.

On a physical level, these students are all inhabiting the same world. Yet on an intellectual and spiritual level — at least when it comes to my class — they may as well be living within entirely different universes.