Ken's Blog

In several episodes of the new BBC series “Sherlock”, there are some wonderful visualizations of Sherlock Holmes going into the memory palace within his fabulous brain to retrieve huge volumes of detailed information he has systematically stored away through the years, each piece of data mapped to a specific location within a virtual building he has constructed in his mind.

The technique of the memory palace, or the “method of loci”, is often attributed to Simonides of Ceos, an ancient Greek lyric poet. This technique is still used today by memory champions to remember fantastically large amounts of information.

Once we have the power-up of being in an immersive virtual reality, perhaps the memory palace could become more accessible to us lesser mortals. We could use our own physical body, our movements and location, to navigate through vast amounts of information, entire libraries becoming available literally at our fingertips.

This would be much more intuitive than current methods of search, where we must type in sets of keywords and then scroll down lists of “hits” in hopes of finding what we want.

Once we have the full use of our body memory, we will be able to draw more completely upon our own brain / body connection to journey through the vast reaches of cyberspace. People will meet each other in data rich places, to which they will navigate without a second thought, perhaps arriving at their destination through a slight turn, a step to the left, a turn of the head, a gesture with the hand.

Future generations will build new kinds of communities, accessible not through keystrokes or screen taps, but by a kind of natural and body-centric form of travel that cyber-citizens of today can only vaguely imagine.

If you and I are having a conversation in virtual reality, with the sensation that we are chatting face to face (or at least face to face with each other’s avatar), then the room itself can become part of the conversation.

If we want to focus on a particular kind of information search, we can wander over to the place in the room where search happens. As soon as we arrive there, data options start to assemble around us, and we can sift through them together to get what we need.

Another part of the room can be our science lab. Objects that we pick up and manipulate there become molecules, beakers, models of planets, ecosystems, neural pathways. We can change the properties of simulated experiments simply by picking up objects and gesturing.

Other parts of the room can serve other specialized roles, perhaps as a launchpad for shared travel to distant places or historical eras, or a place for creating and performing music, or a zen-like “quiet zone” for shared contemplation. Let’s walk over here, you might say, knowing that this short walk will imbue both you and your collaborator with new powers, suitable for the task at hand.

Of course there is no inherent technical reason for such geographic distinctions — after all, any part of the Holodeck is as powerful as any other — but these sorts of spatial conventions might be a nice way for people to organize their work together, just as we now organize some kinds of ideas on our desktop calendar, and others in our documents folder.

The difference is that we will be able to use our entire bodies — our hands, our eyes, a shift in our shoulders or torso to indicate a change in focus from one task to another — harnessing ways of communicating subtleties to each other that humans are particularly good at, which are not accessible to us when we are stuck in front of a computer screen.

Now that we are getting things to work in our Virtual Reality set-up at NYU, we are thinking of how others could contribute. At core, what we have is the ability for two or more people to put on lightweight VR head-sets, with no wires trailing, and walking around together in an imaginary space where they can see each other as virtual beings.

We are starting to think of this as our own little Holodeck, and we realize that its success is going to be helped by people creating content for it. So we are working on creating a “Holodeck server”. Anybody who knows how to use some standard freely available game creation software tools, like Unity, can load a virtual game world onto their computer, which starts out looking just like our room at NYU.

From there they can customize anything they want — place virtual flying creatures, raise the ceiling, add a window onto Paris or the Moon, or do whatever strikes their fancy. The key is to make it very easy for people to create worlds of their own. Then anyone who enters our room and dons the headsets can experience that new world, becoming fully immersed in its reality.

One room starts to become many rooms, existing in parallel dimensions. All are folded into the same physical space, and each has a story to tell.

William Shakespeare wrote around thirty seven plays, more or less. The exact number is in some dispute, but most people seem to agree that this is a good ballpark estimate.

Some of the plays are rather obscure, and some are considered towering achievements. But one in particular seems to stand out — “Hamlet”. It’s not that others aren’t timeless classics. “Lear”, “Macbeth”, “Romeo and Juliet”, “Twelfth Night”, “Much Ado about Nothing”, the list can and does go on.

But “Hamlet” is different. Its hero is so modern, so resonant to us on many levels. His crisis of indecision, the oddity of his existential role: He is at times a royal who kills royalty, a lover who kills his love, a fool who kills a fool, and at each step we find ourselves in his head, sympathizing with him, loving him even more for his flaws.

If you hear a phrase from a Shakespeare play, the odds are pretty high that it is from “Hamlet”. That is how thoroughly it has seeped into our culture.

Many authors seem to have their “Hamlet” — that one work which is the apotheosis of what they are about. Woody Allen has “Annie Hall”, Oscar Wilde has “The Importance of Being Earnest”. If you think about it, half of the times we quote a line from a Woody Allen film, we are quoting “Annie Hall”. The same goes for Wilde and “Earnest”.

I wonder how many other creators have that one signature work, their “Yesterday”, or “Star Wars”, their “Taxi Driver” or “Luxo Jr”, one creation which most perfectly exemplifies and distills their particular genius.

So here’s where all of this has been leading. When we look at how humans balance physically, we see at least four completely separate systems, all working together. Only one, the vestibular system. seems primarily focused on balance itself. The others — vision, proprioception and foot sensation — are used for other important purposes. But they got pulled in because balance is so important in a biped.

Given the importance of emotional balance, I’ll bet there are multiple subsystems at work, all contributing to our ability to concentrate, to focus, to remain on an even keel, to not react rashly in difficult or dangerous situations.

So maybe, if we start to look at it this way, we can eventually find separate mental subsystems, each with its own unique mechanism, that work together to help us maintain our mental balance.

If we can identify them, and understand better how each one works, then maybe we can strengthen them better, just as understanding the nature of vision or proprioception can help us find ways to improve or exercise those senses.

For centuries many cultures have studied variations on what is sometimes referred to as “mindfulness”. There are many forms of meditation and ritual practices which aim to increase our ability to, as Ram Das put it so eloquently, “Be here now”.

When we are tired, distracted, overwhelmed, our mind tends to go around in circles, darting from one place to the next, obsessing over what some person said to us last week, or that email we never returned, or a bill we haven’t payed. Lots of negative energy, buzzing around in our head like a swarm of locusts.

Practices that aim to increase mindfulness work to replace those useless and self-defeating thoughts with a calm focus on the present — on being here in this moment, breathing, aware, alive to the present.

As with any form of exercise, results appear only gradually over time. But exercise is not the same as knowledge. After all, we can become physically fit without knowing much about anatomy or biology.

So rather than talk about how to do such exercises, suppose we talk instead about how they work. What is the mechanism in our mind that we are training when we engage in mindful meditation? What psychic muscle, precisely, is becoming stronger?

To be continued…

Have you ever had one of those moments when you got mad and hit your limit, crossed over the line, completely lost it?

Maybe you were having a discussion that turned unexpectedly into an argument, maybe somebody made a remark and you suddenly felt defensive. Maybe some passive-aggressive person just managed to push your buttons.

In any case, the result is the same — blowing one’s stack, flipping one’s lid, going postal. In the calm reflection of hindsight, we usually wish we had not had such an extreme reaction, whatever the provocation. Of course by then it is too late.

In some sense such reactions are a result of losing one’s psychic balance. Your emotional center has suddenly been yanked in some unexpected direction, and you find yourself flailing, the niceties of civilized discourse abandoned as you grab wildly at anything you can in your struggle to regain control.

This sounds a lot like losing your physical balance, doesn’t it? Hmm. More tomorrow.

We have plans to use that pressure sensitive mat I wrote about in a recent post to help people who may have trouble with balance.

Because of this, I’ve been learning a lot about how people balance on their feet — and why we don’t just fall over. It turns out there are at least four distinct sensing systems we use for this.

Our eyes give us continual feedback about which direction is up. Meanwhile, our inner ear (the vestibular system) gives us inertial feedback. Our sense of proprioception (roughly speaking, knowing the positions of your arms, legs, torso, etc., without needing to look) helps us to know whether our body is in balance. And finally, we use touch sensations in our toes and the bottoms of our soles to know how our weight is distributed over our feet.

These are completely separate biological systems, which function through wildly different mechanisms. And yet they all operate in concert to help us stay upright. Nature is very good at being redundant when it’s important.

As you get older, each of these systems starts to work less well. Yet because of all this redundancy, you have a good chance of keeping your balance at even a quite advanced age, if you exercise properly (but not if you don’t exercise properly).

I’ve been thinking about this question of how we balance, and I think it generalizes to other aspects of the human condition. More tomorrow.

I finally got around to seeing Movie 43, the puerile and unsettling sketch format comedy that opened last year to nearly universal hatred and condemnation (it scored 4% on Rotten Tomatoes, out of a possible 100%).

One thing that makes this unique film intriguing is its cast, which includes Anna Faris, Bobby Cannavale, Chloë Grace Moretz, Dennis Quaid, Elizabeth Banks, Emma Stone, Greg Kinnear, Halle Berry, Hugh Jackman, Justin Long, Kate Bosworth, Kate Winslet, Kieran Culkin, Kristen Bell, Liev Schreiber, Naomi Watts, Richard Gere, Seann William Scott, Seth MacFarlane, Stephen Merchant and Uma Thurman, among others.

I found the movie to be deeply offensive, highly disturbing, and completely wonderful. On the surface it seems to be a study in tastelessness, pure and simple. But if you scratch that surface, it’s not simple at all.

The movie has as its real target our American concept of ourselves as a culture of freedom and personal empowerment. In nearly every sketch, the spotlight is turned to the lie behind that presumption. We are actually — like most cultures — highly constrained by rigidly defined social conventions.

Yet unlike, say, the British, in America we like to pretend that we do not have a culture of envy, of obsessive one-upmanship, of petty acquisitiveness, of defining ourselves against a rigorously defined norm. In short, we are hypocrites.

This film punctures those hypocrisies one at a time, with a knife that is simultaneously blunt and deadly sharp. Given the ugly and weirdly self-righteous “burn the witch” mania I witnessed in the last week or so against Brian Williams, it’s refreshing to see a movie pointing out what should be obvious: We are all liars, and we are all fools. Our culture demands it — no, requires it.

How could such a film not be universally hated?

Happy Valentine’s Day. 🙂

Today I’ve been happily designing creatures for an alternate universe, to be experienced by people in shared virtual reality. This world is populated by mysterious mechanical creatures — possibly vehicles or robots — that are eerily reminiscent of the exotic denizens of our own world’s Precambrian explosion.

It just seemed to me that it might be a good idea to mash-up the visual tropes of Steampunk and the Precambrian era, both of which speak to our collective fascination with time and alternate worlds.

Below is a screen capture of an animated creature I made today. I call her Diana.