Ken’s Blog

If only we could
Live forever, would our lives
Be better or worse?

There is no interface quite like a whiteboard. Leave your computer, put down your iPad, keep your Android phone in your pocket. When you want to have a real intellectual discussion with a colleague, the best thing to do is pick up a dry erase marker and start scribbling.

There is something wonderfully primal and simple about interacting with a whiteboard. As a medium of communication, it possesses a delightful transparency. The way you can draw something without needing to think about it, the expressiveness of your gestures and body language as you explain your drawing, the way the other person can jump in and add to or amend what you’ve written, these are properties not shared with any known computer interface.

But there is a problem with the whiteboard. When you are done, you can’t take it with you. If you want to have another conversation, you must erase what you’ve already drawn and start anew.

Sure you can take a photo before erasing, but that’s just a photo. It doesn’t have the liveness, the easy edibility, the visceral quality of the physical act of drawing, that makes a whiteboard sketch so powerful.

But that may change. Once we are having these conversations in shared virtual reality, any surface can be a whiteboard. And a whiteboard can be many whiteboards. You will be able to wind back the history of a whiteboard to any previous state, and even add missing details or corrections to that history.

The same surface will be able to serve as the shared location of multiple whiteboard-enhanced ongoing discussions. The same wall will shift to accommodate my conversation with you about politics in the morning, and my conversation with someone else about physics in the afternoon.

We won’t even need to be in the same room or city to scribble on a whiteboard together. This most old fashioned of modern tools may very well turn out to be the glue that connects us across distance, that helps us turn cyberspace into meaningful personal space.

In several episodes of the new BBC series “Sherlock”, there are some wonderful visualizations of Sherlock Holmes going into the memory palace within his fabulous brain to retrieve huge volumes of detailed information he has systematically stored away through the years, each piece of data mapped to a specific location within a virtual building he has constructed in his mind.

The technique of the memory palace, or the “method of loci”, is often attributed to Simonides of Ceos, an ancient Greek lyric poet. This technique is still used today by memory champions to remember fantastically large amounts of information.

Once we have the power-up of being in an immersive virtual reality, perhaps the memory palace could become more accessible to us lesser mortals. We could use our own physical body, our movements and location, to navigate through vast amounts of information, entire libraries becoming available literally at our fingertips.

This would be much more intuitive than current methods of search, where we must type in sets of keywords and then scroll down lists of “hits” in hopes of finding what we want.

Once we have the full use of our body memory, we will be able to draw more completely upon our own brain / body connection to journey through the vast reaches of cyberspace. People will meet each other in data rich places, to which they will navigate without a second thought, perhaps arriving at their destination through a slight turn, a step to the left, a turn of the head, a gesture with the hand.

Future generations will build new kinds of communities, accessible not through keystrokes or screen taps, but by a kind of natural and body-centric form of travel that cyber-citizens of today can only vaguely imagine.

If you and I are having a conversation in virtual reality, with the sensation that we are chatting face to face (or at least face to face with each other’s avatar), then the room itself can become part of the conversation.

If we want to focus on a particular kind of information search, we can wander over to the place in the room where search happens. As soon as we arrive there, data options start to assemble around us, and we can sift through them together to get what we need.

Another part of the room can be our science lab. Objects that we pick up and manipulate there become molecules, beakers, models of planets, ecosystems, neural pathways. We can change the properties of simulated experiments simply by picking up objects and gesturing.

Other parts of the room can serve other specialized roles, perhaps as a launchpad for shared travel to distant places or historical eras, or a place for creating and performing music, or a zen-like “quiet zone” for shared contemplation. Let’s walk over here, you might say, knowing that this short walk will imbue both you and your collaborator with new powers, suitable for the task at hand.

Of course there is no inherent technical reason for such geographic distinctions — after all, any part of the Holodeck is as powerful as any other — but these sorts of spatial conventions might be a nice way for people to organize their work together, just as we now organize some kinds of ideas on our desktop calendar, and others in our documents folder.

The difference is that we will be able to use our entire bodies — our hands, our eyes, a shift in our shoulders or torso to indicate a change in focus from one task to another — harnessing ways of communicating subtleties to each other that humans are particularly good at, which are not accessible to us when we are stuck in front of a computer screen.

Now that we are getting things to work in our Virtual Reality set-up at NYU, we are thinking of how others could contribute. At core, what we have is the ability for two or more people to put on lightweight VR head-sets, with no wires trailing, and walking around together in an imaginary space where they can see each other as virtual beings.

We are starting to think of this as our own little Holodeck, and we realize that its success is going to be helped by people creating content for it. So we are working on creating a “Holodeck server”. Anybody who knows how to use some standard freely available game creation software tools, like Unity, can load a virtual game world onto their computer, which starts out looking just like our room at NYU.

From there they can customize anything they want — place virtual flying creatures, raise the ceiling, add a window onto Paris or the Moon, or do whatever strikes their fancy. The key is to make it very easy for people to create worlds of their own. Then anyone who enters our room and dons the headsets can experience that new world, becoming fully immersed in its reality.

One room starts to become many rooms, existing in parallel dimensions. All are folded into the same physical space, and each has a story to tell.

William Shakespeare wrote around thirty seven plays, more or less. The exact number is in some dispute, but most people seem to agree that this is a good ballpark estimate.

Some of the plays are rather obscure, and some are considered towering achievements. But one in particular seems to stand out — “Hamlet”. It’s not that others aren’t timeless classics. “Lear”, “Macbeth”, “Romeo and Juliet”, “Twelfth Night”, “Much Ado about Nothing”, the list can and does go on.

But “Hamlet” is different. Its hero is so modern, so resonant to us on many levels. His crisis of indecision, the oddity of his existential role: He is at times a royal who kills royalty, a lover who kills his love, a fool who kills a fool, and at each step we find ourselves in his head, sympathizing with him, loving him even more for his flaws.

If you hear a phrase from a Shakespeare play, the odds are pretty high that it is from “Hamlet”. That is how thoroughly it has seeped into our culture.

Many authors seem to have their “Hamlet” — that one work which is the apotheosis of what they are about. Woody Allen has “Annie Hall”, Oscar Wilde has “The Importance of Being Earnest”. If you think about it, half of the times we quote a line from a Woody Allen film, we are quoting “Annie Hall”. The same goes for Wilde and “Earnest”.

I wonder how many other creators have that one signature work, their “Yesterday”, or “Star Wars”, their “Taxi Driver” or “Luxo Jr”, one creation which most perfectly exemplifies and distills their particular genius.

So here’s where all of this has been leading. When we look at how humans balance physically, we see at least four completely separate systems, all working together. Only one, the vestibular system. seems primarily focused on balance itself. The others — vision, proprioception and foot sensation — are used for other important purposes. But they got pulled in because balance is so important in a biped.

Given the importance of emotional balance, I’ll bet there are multiple subsystems at work, all contributing to our ability to concentrate, to focus, to remain on an even keel, to not react rashly in difficult or dangerous situations.

So maybe, if we start to look at it this way, we can eventually find separate mental subsystems, each with its own unique mechanism, that work together to help us maintain our mental balance.

If we can identify them, and understand better how each one works, then maybe we can strengthen them better, just as understanding the nature of vision or proprioception can help us find ways to improve or exercise those senses.

For centuries many cultures have studied variations on what is sometimes referred to as “mindfulness”. There are many forms of meditation and ritual practices which aim to increase our ability to, as Ram Das put it so eloquently, “Be here now”.

When we are tired, distracted, overwhelmed, our mind tends to go around in circles, darting from one place to the next, obsessing over what some person said to us last week, or that email we never returned, or a bill we haven’t payed. Lots of negative energy, buzzing around in our head like a swarm of locusts.

Practices that aim to increase mindfulness work to replace those useless and self-defeating thoughts with a calm focus on the present — on being here in this moment, breathing, aware, alive to the present.

As with any form of exercise, results appear only gradually over time. But exercise is not the same as knowledge. After all, we can become physically fit without knowing much about anatomy or biology.

So rather than talk about how to do such exercises, suppose we talk instead about how they work. What is the mechanism in our mind that we are training when we engage in mindful meditation? What psychic muscle, precisely, is becoming stronger?

To be continued…

Have you ever had one of those moments when you got mad and hit your limit, crossed over the line, completely lost it?

Maybe you were having a discussion that turned unexpectedly into an argument, maybe somebody made a remark and you suddenly felt defensive. Maybe some passive-aggressive person just managed to push your buttons.

In any case, the result is the same — blowing one’s stack, flipping one’s lid, going postal. In the calm reflection of hindsight, we usually wish we had not had such an extreme reaction, whatever the provocation. Of course by then it is too late.

In some sense such reactions are a result of losing one’s psychic balance. Your emotional center has suddenly been yanked in some unexpected direction, and you find yourself flailing, the niceties of civilized discourse abandoned as you grab wildly at anything you can in your struggle to regain control.

This sounds a lot like losing your physical balance, doesn’t it? Hmm. More tomorrow.

We have plans to use that pressure sensitive mat I wrote about in a recent post to help people who may have trouble with balance.

Because of this, I’ve been learning a lot about how people balance on their feet — and why we don’t just fall over. It turns out there are at least four distinct sensing systems we use for this.

Our eyes give us continual feedback about which direction is up. Meanwhile, our inner ear (the vestibular system) gives us inertial feedback. Our sense of proprioception (roughly speaking, knowing the positions of your arms, legs, torso, etc., without needing to look) helps us to know whether our body is in balance. And finally, we use touch sensations in our toes and the bottoms of our soles to know how our weight is distributed over our feet.

These are completely separate biological systems, which function through wildly different mechanisms. And yet they all operate in concert to help us stay upright. Nature is very good at being redundant when it’s important.

As you get older, each of these systems starts to work less well. Yet because of all this redundancy, you have a good chance of keeping your balance at even a quite advanced age, if you exercise properly (but not if you don’t exercise properly).

I’ve been thinking about this question of how we balance, and I think it generalizes to other aspects of the human condition. More tomorrow.