Ken’s Blog

Today on our Future Reality Lab blog I wrote about using VR to create parallel worlds. It’s a theme I’ve discussed in different ways on this blog.

Suddenly it occurred to me my simultaneous presence on both this blog and in our FRL blog is also a parallel world. Except that the worlds of these two blogs are parallel along a different dimension.

So here we have two parallel worlds being discussed in two parallel worlds. I wonder, what do you call it when you send a message out into the world that is parallel along multiple dimensions?

I know — a parallelogram!

There are primates who can easily pick up tools with their toes. Others can swing by their tails.

Humans cannot do these things. For one thing, our bodies are not constructed to do so.

But is this a limitation only of our bodies, or also of our brains? Do we have the latent brain capacity to pick things up with our toes or to swing from trees by a tail, if only we had the right body to do so?

Once we start using our full bodies to immerse ourselves into shared virtual worlds, this will start to become a practical question. Just how general is our brain’s ability to remap itself to a different body, if given the opportunity?

And if our brains do turn out to be adaptable in this way, what are the limitations of the mapping? Could we learn to be comfortable in the body of an amoeba, forming and extending temporary pseudopods as needed?

Could we adapt to an entirely alien body? That could be useful for exploration of other planets or deep sea environments.

It’s exciting to think that we humans might someday have the sensory experience of entering entirely alien bodies. But it is not yet clear whether such a thing is truly possible. I guess we will need to try it out and see.

As we move various aspects of our social life to shared immersive virtual reality, we don’t all need to present as the same size. It might be convenient, for various reasons, for somebody to appear 20 feet tall, while a group of five people might all fit on a tabletop.

I don’t know for sure whether this will happen, but it’s a reasonable thing to think about. After all, the next few years will be the first time in history when we will have the capability to hang out together everyday in a physical sense while defying the rules of nature.

We are used to seeing people at wildly different scales on what are now considered traditional media. Paintings, photographs, movie theaters, TV sets, smartphone screens, all of these forms of visual communication wreak havoc with the notion of immutable human scale.

Yet somehow our brains adjust to the sight of a movie star’s face being 30 feet tall at the cinema, while the friend we are chatting with on our smartphone has a face that is only two inches in height. None of it seems to bother us.

I suspect there is a part of our brains that automatically maps whatever we perceive as human to a “normal” size, on such a low level that we don’t even notice it happening. I see no reason why this perceptual transformation should not carry over into shared immersive worlds.

In any case, it’s an experiment worth carrying out. And it’s definitely something we are interested in trying in our Future Reality Lab at NYU.

Today at our lab’s weekly research meeting, I realized we are on the cusp of a new phase in our research. When you are doing research, it’s not only about what you might potentially develop, it’s also about whether you have the proper tools for that development to be practical.

After months of work, we are in sight of having a new set of tools that are far more powerful and flexible than anything we have had before. An analogy might help.

Imagine you are testing out a new automobile, and the only thing you can do is plot a course for the car beforehand, set it on the road, and see afterward whether the car has crashed. You might develop a good automobile using that approach, but your task will be very difficult.

Now compare that with the ability to actually get in the car, put your foot on the accelerator, and drive it yourself, turning the steering wheel as needed to travel to different places.

What we are developing now is basically like going from that first scenario to the second. Using these new tools, it’s going to be a lot more fun to do our research.

Also, we might end up traveling to places we never knew existed.

I was so proud of myself for getting my research to work yesterday using only the plastic lids of coffee cups. I happily showed my research result to people in the lab, and everybody liked it.

But then today I realized that I don’t actually need even the coffee cop lids. The whole thing works just fine even with virtual coffee cup lids. The physical parts are not really necessary.

I am not sure how I feel about this. On the one hand, it’s great to realize that there is a software-only solution. On the other hand, it was really fun to think I was able to do cutting edge research with cheap plastic coffee cup lids from our local take-out deli.

Oh well, at least the new technique will be eco-friendly.

Today I realized I needed a physical object of a certain size and shape for my research. I was thinking that in order to get the shape I wanted, I would need to run a 3D print job.

I really didn’t want to do that because it would be both complicated and expensive. But where was I to get a 3D shape that did exactly what I want?

Pondering this question, I went to our lab kitchen to make myself a coffee.

And that’s when I saw it: It turns out that the plastic coffee lids in our kitchen are exactly the right size and shape for my experiments.

There are other benefits as well: The total cost of materials for a plastic coffee lid is less than a penny.

I felt excited by this discovery, and newly energized to continue my research. In fact, I was so happy, I forgot to drink the coffee.

I had occasion today to look back at emails from 2008. It was a very weird experience.

Back then I was having near daily personal and professional conversations with people I have not spoken to for many years. Just reading those emails brought me back to a another time in my life, when my priorities were very different.

The most haunting email exchanges were the ones I was sharing with a close friend who has since passed away. Of course there was no way of knowing back then that tragedy was looming only a few short years away.

We are who we have always been, and yet we are also constantly in motion. Conversations between our past and our current selves can be a kind of time machine — a deeply revealing one — but they are never easy.

I am not sure I always have the courage it takes to hold such a conversation honestly. But I’d like to think I can rise to the occasion.

To test my little VR laboratory, I gave myself 10 minutes to implement the very beginnings of a VR geometric modeling system. It’s not much, but it’s a good start.

You can see me trying it out by clicking on the image below.

I am encouraged by the fact that it took so little time to get this far. I am looking forward to building out other capabilities, like snapping objects together, changing colors and textures, designing custom forms and creating animated creatures.

I don’t expect that I will end up with a modeler that will do everything. But I do expect to learn a lot in the process.

Besides, it’s really fun.

I have been having so much fun hanging out in my virtualized lab space, that I’m thinking of moving much of my research there. It wouldn’t be a physical move so much as a perceptual one.

After all, I will still be in the lab. Anybody who wants to join me can simply put on a VR headset, grab a pair of controllers, and join in the fun.

In this alternate version of our lab, we will all have super powers. We can create objects simply by waving our hands, change the appearance of the world around us at will, draw animations in the air. Or maybe, just for fun. going for a walk in the Taj Mahal.

Of course we will know that we are not the real world, but that shouldn’t be a problem. When you read a book or see a movie, you also know that what you are experiencing is not actually the real world.

Any sort of virtual experience, be it a book, a play, a film or a computer game, is both less and more than reality. Broadly speaking, these are all forms of literature, tools for sharing and exploring alternate worlds of our imagination.

I wonder what will happen as we get progressively better at blending the serious purpose of collaborative work with the tools of shared immersive experience. Will our collective understanding of reality change as well?

I guess we will find out.

Forty years ago today
The Wall was taken down
Today’s the anniversary
Of something quite profound

Why then did we bother
To go through all that fuss?
It makes me wonder what the hell
Is wrong with all of us