Your virtual body

As long as we will have apparent bodies in our virtual reality meetings and interactions with each other, there may not be any inherent reason that those bodies need to be anything like the ones we have now.

The issue here is not so much about any limitations on the body or the senses, but about potential limitations on the brain. We evolved to be the way we are as a set of functional responses to survival challenges in the physical world. We needed to eat, to procreate, to cooperate with each other, and in response our brains and bodies evolved together successfully (in the sense that our species is still around).

But what happens if you keep the brain pretty much the way it is, but rewrite the body? Surely at some point you will hit limitations on what the brain can handle. Our minds are pretty protean things, capable of learning all sorts of systems, but they still have limitations.

For example, we don’t do very well with a very high number of spatial dimensions (maybe we can handle four, but things start to get seriously wonky starting at about five). We can work out complex logical problems only with effort and training. We suck at probability.

But there are things we do spectacularly well. We can walk and dance and play the guitar. We pick up on incredibly subtle changes in facial expression and body language. We use language.

So how creative can we get before the mind is overwhelmed? How well would our minds accommodate having four arms? A lack of bilateral symmetry? Temporary limbs that appear only where and when we need them?

And what about vision? How well could our brains accommodate the ability to see equally well in all directions? At all locations in the world? At all scales? At different rates of time?

Once you take on the project of reconstructing the body and senses around the question of what the human mind could adapt to, you end up with a vast and fascinating space. It will be, to be sure, a space with some limitations, because the human brain, for all its protean qualities, has limits.

But it will be fun to understand what those limits are, and perhaps even to figure out ways to turn them into strengths, as we evolve our future cyber-bodies. After all, one of the qualities of the human mind is boundless curiosity.


I had several conversations earlier today that turned to the subject of self-righteousness. There is an unfortunate tendency among humans to know that we are right, and to feel it is our sacred mission to fix the error of other peoples’ ways.

The word “sacred” here is important. When we say that something is sacred, we usually believe, quite sincerely, that we are discussing some sort of inherent moral quality. But most of the time we are actually discussing tribalism. Somebody is doing or saying something that we perceive as an assault on our own tribal identity.

And I mean “tribal identity” in the broadest sense. It could refer to our nationality, our religion, our sexual orientation, or our choice of cuisine. Because our preferences in these things are generally the result of something other than purely rational thinking, those preferences are fragile. Since we cannot defend them by logic, we often resort to sanctimony.

It is hard to avoid the escalation from tribal identity to tribal warfare. To do so requires saying “Yes, this is what I believe, yet I respect that you believe something very different.” Cultural tribes, by their very nature, are constructed in a way that makes it difficult to say this.

I would like to think that I belong to the tribe of non-sanctimonious individuals. I aspire to look with respect and understanding at others with whom I disagree, and to accept that life among humans is never simple nor absolute, and that a person with different values from my own is as spiritual, in their way, as I am in mine.

The body electric

The opposite of what I was talking about yesterday is the place I visited this evening. Body Labs offers to scan your body. The resulting scan can then be animated, used for fashion, placed in a computer game or virtual world, and otherwise deployed as a visual proxy for your actual self.

There is an odd quality to the trend represented by this capability, since we have so many associations with the human body that are connected with its inherent physicality. For our body is the thing that grounds our mind to physical truth.

Bodies are earthy — they are our connection to the animal kingdom. Through them we smell, taste, feel pleasure and pain. We use them to dance, to grasp things, have sex, eat food and excrete waste.

Bodies are primal and they are messy. We love them and we hate them, and it is hard to think of our very identities without them.

But as our lives become more computer mediated, our perception of real bodies might be gradually replaced by their electronic simulacra. A virtual body is both more and less than a physical one. It can be idealized. It can exist in many places at once. It can fly and change shape. But it is not, and cannot be, a physical, animal thing.

A body in cyberspace may look, and perhaps one day even feel, exactly the same as a body in the physical world. Yet its meaning, in the most profound sense, is utterly different.

Voting with your pain

I had a long and wide-ranging conversation this last weekend with Esther Dyson, and at some point we got on the topic of where virtual reality might be going, and what its relationship will be to people seeing each other in person.

She made a very interesting point (not surprisingly). When people come to see her, perhaps to ask her to invest in their new venture, they get on a train or an airplane and see her in person.

It would be much easier for them just to get her on the phone, or Skype or talk via Google Hangouts. But they generally don’t do that.

Of course meeting in person has far higher bandwidth. You can pick up on subtle visual, verbal and facial cues when you are actually with somebody that might completely elude you using any currently known electronic means of communication.

But even if we solve those problems, and the future equivalents of the Skype chat become so ultra-high fidelity that they are perceptually indistinguishable from face to face conversation, a crucial difference will remain. The person who is going to see Esther is going to see her.

There are times, when it is important to you, that you choose to put your physical body on a bus or train or airplane, perhaps at the cost of significant physical discomfort. In this way you show, by your very presence, that it really matters. In a sense, you are are voting with your pain.

We only have one physical body, and it is precious to us. Where we choose to place that body, and whom we are with at the time, is a very important statement. Virtual reality, no matter how good it ever gets, is never going to replace that statement.

The whiteboard problem

There is no interface quite like a whiteboard. Leave your computer, put down your iPad, keep your Android phone in your pocket. When you want to have a real intellectual discussion with a colleague, the best thing to do is pick up a dry erase marker and start scribbling.

There is something wonderfully primal and simple about interacting with a whiteboard. As a medium of communication, it possesses a delightful transparency. The way you can draw something without needing to think about it, the expressiveness of your gestures and body language as you explain your drawing, the way the other person can jump in and add to or amend what you’ve written, these are properties not shared with any known computer interface.

But there is a problem with the whiteboard. When you are done, you can’t take it with you. If you want to have another conversation, you must erase what you’ve already drawn and start anew.

Sure you can take a photo before erasing, but that’s just a photo. It doesn’t have the liveness, the easy edibility, the visceral quality of the physical act of drawing, that makes a whiteboard sketch so powerful.

But that may change. Once we are having these conversations in shared virtual reality, any surface can be a whiteboard. And a whiteboard can be many whiteboards. You will be able to wind back the history of a whiteboard to any previous state, and even add missing details or corrections to that history.

The same surface will be able to serve as the shared location of multiple whiteboard-enhanced ongoing discussions. The same wall will shift to accommodate my conversation with you about politics in the morning, and my conversation with someone else about physics in the afternoon.

We won’t even need to be in the same room or city to scribble on a whiteboard together. This most old fashioned of modern tools may very well turn out to be the glue that connects us across distance, that helps us turn cyberspace into meaningful personal space.

In palaces of future memories

In several episodes of the new BBC series “Sherlock”, there are some wonderful visualizations of Sherlock Holmes going into the memory palace within his fabulous brain to retrieve huge volumes of detailed information he has systematically stored away through the years, each piece of data mapped to a specific location within a virtual building he has constructed in his mind.

The technique of the memory palace, or the “method of loci”, is often attributed to Simonides of Ceos, an ancient Greek lyric poet. This technique is still used today by memory champions to remember fantastically large amounts of information.

Once we have the power-up of being in an immersive virtual reality, perhaps the memory palace could become more accessible to us lesser mortals. We could use our own physical body, our movements and location, to navigate through vast amounts of information, entire libraries becoming available literally at our fingertips.

This would be much more intuitive than current methods of search, where we must type in sets of keywords and then scroll down lists of “hits” in hopes of finding what we want.

Once we have the full use of our body memory, we will be able to draw more completely upon our own brain / body connection to journey through the vast reaches of cyberspace. People will meet each other in data rich places, to which they will navigate without a second thought, perhaps arriving at their destination through a slight turn, a step to the left, a turn of the head, a gesture with the hand.

Future generations will build new kinds of communities, accessible not through keystrokes or screen taps, but by a kind of natural and body-centric form of travel that cyber-citizens of today can only vaguely imagine.

Let’s walk over here

If you and I are having a conversation in virtual reality, with the sensation that we are chatting face to face (or at least face to face with each other’s avatar), then the room itself can become part of the conversation.

If we want to focus on a particular kind of information search, we can wander over to the place in the room where search happens. As soon as we arrive there, data options start to assemble around us, and we can sift through them together to get what we need.

Another part of the room can be our science lab. Objects that we pick up and manipulate there become molecules, beakers, models of planets, ecosystems, neural pathways. We can change the properties of simulated experiments simply by picking up objects and gesturing.

Other parts of the room can serve other specialized roles, perhaps as a launchpad for shared travel to distant places or historical eras, or a place for creating and performing music, or a zen-like “quiet zone” for shared contemplation. Let’s walk over here, you might say, knowing that this short walk will imbue both you and your collaborator with new powers, suitable for the task at hand.

Of course there is no inherent technical reason for such geographic distinctions — after all, any part of the Holodeck is as powerful as any other — but these sorts of spatial conventions might be a nice way for people to organize their work together, just as we now organize some kinds of ideas on our desktop calendar, and others in our documents folder.

The difference is that we will be able to use our entire bodies — our hands, our eyes, a shift in our shoulders or torso to indicate a change in focus from one task to another — harnessing ways of communicating subtleties to each other that humans are particularly good at, which are not accessible to us when we are stuck in front of a computer screen.

Holodeck server

Now that we are getting things to work in our Virtual Reality set-up at NYU, we are thinking of how others could contribute. At core, what we have is the ability for two or more people to put on lightweight VR head-sets, with no wires trailing, and walking around together in an imaginary space where they can see each other as virtual beings.

We are starting to think of this as our own little Holodeck, and we realize that its success is going to be helped by people creating content for it. So we are working on creating a “Holodeck server”. Anybody who knows how to use some standard freely available game creation software tools, like Unity, can load a virtual game world onto their computer, which starts out looking just like our room at NYU.

From there they can customize anything they want — place virtual flying creatures, raise the ceiling, add a window onto Paris or the Moon, or do whatever strikes their fancy. The key is to make it very easy for people to create worlds of their own. Then anyone who enters our room and dons the headsets can experience that new world, becoming fully immersed in its reality.

One room starts to become many rooms, existing in parallel dimensions. All are folded into the same physical space, and each has a story to tell.

Their “Hamlet”

William Shakespeare wrote around thirty seven plays, more or less. The exact number is in some dispute, but most people seem to agree that this is a good ballpark estimate.

Some of the plays are rather obscure, and some are considered towering achievements. But one in particular seems to stand out — “Hamlet”. It’s not that others aren’t timeless classics. “Lear”, “Macbeth”, “Romeo and Juliet”, “Twelfth Night”, “Much Ado about Nothing”, the list can and does go on.

But “Hamlet” is different. Its hero is so modern, so resonant to us on many levels. His crisis of indecision, the oddity of his existential role: He is at times a royal who kills royalty, a lover who kills his love, a fool who kills a fool, and at each step we find ourselves in his head, sympathizing with him, loving him even more for his flaws.

If you hear a phrase from a Shakespeare play, the odds are pretty high that it is from “Hamlet”. That is how thoroughly it has seeped into our culture.

Many authors seem to have their “Hamlet” — that one work which is the apotheosis of what they are about. Woody Allen has “Annie Hall”, Oscar Wilde has “The Importance of Being Earnest”. If you think about it, half of the times we quote a line from a Woody Allen film, we are quoting “Annie Hall”. The same goes for Wilde and “Earnest”.

I wonder how many other creators have that one signature work, their “Yesterday”, or “Star Wars”, their “Taxi Driver” or “Luxo Jr”, one creation which most perfectly exemplifies and distills their particular genius.