Ken’s Blog

Every time I give a talk about our research and our mission, I make some kind of change. So in some sense the talk is a kind of evolving creature, growing a tentacle here, losing one there.

To do this, I generally go by the energy I get from the audience. Sometimes I feel that I have glossed over something important, and need to add a bit of connecting tissue. At other times I feel that I’ve belabored something inessential, which distracts — and therefore detracts — from the main points.

It’s a constant process of editing, and it feels very productive. Not just because it’s good to get out our message, but because for me it’s an essential part of understanding our message.

I suspect this is true for all of us: You don’t fully understand your own mission until you are able to clearly explain it to others. And at the end of the day, there’s nothing more satisfying than understanding your own mission.

If you want to assess the appeal
Of making the virtual real
The thing you must follow
In a novel or Holo
Is the way that it makes people feel

Today at Electronic Arts
They made me sign an NDA
I still have yet to figure out
Just what it is I shouldn’t say

The students in my graduate computer graphics class will show their shared metaroom final projects one week from today — the evening of Monday December 16. Today several groups have been at our lab working away to get their “one week to go” preliminary demos ready.

I can feel a really wonderful excitement as they work through creating these alternate worlds. Every once in a while, a student will pull me aside, hand me an Oculus Quest headset, and say “here, try this.”

And suddenly I am in another world. But a world in which we can all walk around together with our own feet, no wires attached.

I can’t wait to see their visions of the future come to life. Just one week to go.

I’ve been thinking about what to call our platform that allows multiple people to roam around a room together while seeing and hearing an alternate version of their shared physical reality. It occurs to me that it could be interesting to tie it into the seminal 1947 essay by Tolkien that I blogged about several months ago On Fairy Stories.

After all, Tolkien’s description of “Faërian Drama” within that essay is a very nice summary of the possibilities of fictional narrative using the tools we are developing. So why not look to it for inspiration when naming our project?

In that spirit, I think we might want to describe what we are doing in a way that produces an apt acronym: Shared Highly Immersive Reality Emulation

As our lab’s research explores shared worlds, we find it useful to look at movies for useful diegetic prototypes. For example, TRON, The Matrix and Ready Player One all tell essentially the same kind of story: People are beamed into an entirely virtual world, where they become completely separate from their actual physical bodies.

In contrast, the Jedi Council in the Star Wars universe tells the opposite story: Members of the Council always remain connected to their actual physical bodies. The people who are beaming in are simply enacting a full-body holographic version of Skype or Google Hangouts.

The essential point is that the Jedi Council does not posit any separate virtual world. The only world that exists is the one people inhabit with their physical bodies. They just happen to have a better version of Skype. I think our own lab’s research is much closer to this vision than it is to the vision within TRON and its descendants.

The Star Trek Holodeck is somewhere in between. There is a suggestion that people in the Holodeck remain attached to their physical bodies, but it is only a suggestion.

A different approach is found in the Visual Instrument and Sight Organ Replacement (VISOR) worn by Geordi La Forge in Star Trek the Next Generation. Geordi inhabits the same physical reality as everyone else, but he perceives that reality quite differently.

It is not entirely certain what Geordi sees when he looks at his fellow crew members. Perhaps he sees them as computer graphic reconstructions of themselves.

But it is clear that he can perceive things that are invisible to others. For example, Geordi can look at a wall and see the electrical wiring within it. I particularly like this diegetic prototype — partly because I suspect that something like it will become a part of our ordinary everyday reality within the next decade or so.

Last night I came up with my first political joke in a dream. That is, I was dreaming, and I told a political joke in my dream, and when I woke up I still thought my joke was funny.

The joke itself was fairly simple. It went something like this:

Q: Why do so many Americans hoping that RBG will stay healthy?

A: Because without her, our Supreme Court will become completely Ruthless.

I can’t remember whether my dream audience laughed. I don’t think there is any way to ask them.

Other than that, I’m not sure what this means, besides the sad fact that we Americans may have lost our last safe refuge. Apparently we can no longer escape the insanity of our national politics even when we are asleep.

We are setting up demos in our lab for visitors, so they can quickly get a taste of future reality. The Oculus Quest is great for this, but not quite perfect.

If you take off the headset, it exits the VR experience. When you go back in again, you need to use your controller to click on a menu in VR to get back in again.

That isn’t going to work for casual visitors. We want them to simply put on the headset and immediately be “in world”.

I solve part of the problem by putting a little piece of masking tape over the photosensor that the headset uses to detect that you’ve taken it off. That works great, but it has the problem that the display is always running, which will eventually burn out the headset’s display screen.

So today I added a little software hack. If the headset doesn’t move at all for ten seconds, the entire virtual world goes completely black. As soon as the headset moves again, it then immediately returns to full brightness.

In practice, this means that when somebody takes off the headset and puts it down on a table, the display goes black ten seconds later, thereby avoiding display screen burnout. The moment they grab the headset to put it back on, the display immediately goes back to normal. Yet behind the scenes, our software is still running, blissfully unaware that anything is amiss.

This all might seem trivial, but I think it’s going to make a big difference. When a random visitor stops by who might be inclined to support our lab’s work, we don’t want to have to tell them to wait. Even a slight delay, or fumbling with VR controllers, might mean we miss the chance to ever show them what the future looks like.

Also, do you really want a future where you’d need to click on some stupid menu just to make your glasses work properly? I know I don’t.

I’ve always wanted to be one of those people who gets their poem printed in The New Yorker magazine. And now at last, in a sort of unusual way, I have.

There is an article in the latest issue of The New Yorker by Patty Marx, entitled Taking Virtual Reality for a Test Drive. Near the end I am cited.

The entire citation consists of printing a poem that I once wrote on this blog about virtual reality, in the form of a haiku. I hereby reprint said poem, in its entirety:

when technology
shifts reality, will we
know the world has changed?

I think my poem points out a very valid truth. I could add that it is a timely truth, but in fact this particular truth has been equally timely over the entirety of human history.

In any case, I am pleased and honored to finally have a poem printed in The New Yorker. Thank you Patty!!!!

One of the difficult issues we wrestle with at our lab is the line between VR and not VR. If people are co-located, as they generally are in our work, is that still VR? Or is it actually an extreme version of mixed reality (MR)?

I don’t think it’s a very interesting question, as an intrinsic analysis of what we are doing. But in terms of how we capture people’s imagination, getting the terminology right could be very important.

If you want to change the world for the better, people need to understand what you are actually doing. It can be hard to get people on board if they think you are doing something else.