Author: admin

Most people are born with two arms and hands, with each hand containing four fingers plus an opposable thumb. Those hands are a remarkably flexible and protean way of interacting with the world.

From the time we are little we develop a powerful sense of proprioception. You can generally reach for something with wonderful accuracy even when your eyes are closed. After all, your brain has spent years and years learning your body, in all of its possible configurations.

But as technology starts to allow physical experience to become more visually virtualized, and we begin to make ever greater use of unseen helper robots, it might be possible to tinker with this basic architecture. Not by changing our physical armd and hands, but by changing our mind’s perception of them.

We might find it convenient, for example, to reach across a room to pick something up. We wouldn’t actually be stretching our arms across the room, but the illusion that we are doing so might be achieved by a combination of cyber-modified vision and helper robots that can pick up distant objects and place them in our grasp.

Today’s technologies allow us to do “impossible” things all the time. We move our bodies across oceans in a matter of hours, chat with people on the other side of the planet, and jot down our thoughts in a way that is potentially readable by billions of people (just as I am doing now). Because we are so used to performing these miraculous feats, we don’t realize how remarkable they are.

Similarly, emerging technologies will eventually allow us to reach across a room and pick up an object fifty feet away. And one day kids will be born into a world where that sort of thing is commonplace. They will think nothing of it.

In order for comments to show up on this blog, I need to approve the commenter. So if I’ve already approved a comment from you, then your subsequent comments will automatically be approved.

Which means that from time to time I get a comment from someone new. If a comment is not blatant spam, it’s usually a really cool comment, and I happily approve it. But every once in a while, not so much.

Just in the last day I got a comment that was truly offensive. Not so much because it was trying to be offensive, but because the person writing it was attempting make a joke based on puerile ignorance. It was the sort of nasty uninformed comment whose validity would not have survived a three second Google search.

And I didn’t approve it. Because this is, after all, a curated space, intended for respectful and informed discussion, and I don’t want that sort of ugly energy here.

That’s what YouTube comments are for.

Today I spent much of the day trading quips, ideas and demos with Vi Hart. And maybe a little scotch.

As the words and visuals went back and forth between us, I realized that she and I share a certain technique. We both look with great seriousness upon the concepts and inspirations that we first thought about as children, and use them in our work.

Of course that is not always an easy thing to do when you are a grownup, surrounded daily by the voices and thoughts of grownups. But it’s not impossible.

And I find that talking with Vi makes it a lot easier.

I am staying this evening at the lovely home of my dear old friends Ted and Ellen in Palo Alto. When I first met them, long ago, they were living in New York City. Midtown Manhattan, no less.

One day Ted was offered a fabulous job in California that he really couldn’t refuse, and so they needed to leave New York. We were talking this evening about that time years ago when they were about to depart New York, and had marked the occasion by convening one last gathering with their good friends in Manhattan.

This evening I reminded Ellen of something she’d said to me on that day long ago. “How does it feel,” I had asked her then, “to be leaving New York City.”

She’d thought about it, and had responded with one of the most quotable lines I’ve ever heard. “When you leave New York,” she told me, “you’re not going anywhere.”

Today, while attending a SIGCHI session, I learned of the term “diegetic prototype”, originally coined by David A. Kirby. It’s a concept with which I am very familiar, but I hadn’t actually known there was a name for it.

The basic idea is the “working example” of a future technology that you see in a science fiction movie. Some notable examples: the force field in Forbidden Plant, the Star Trek transporter, the robots in Star Wars, the flying skateboard in Back to the Future, the gestural interface in Minority Report and the interactive holographic displays in the recent Iron Man movies.

More than mere fictional constructs, these are aspirational objects meant to inspire audiences, a sort of stake in the cultural ground. They hold out the possibility, however remote, of a brighter and more exciting future here in the real world.

I am amazed that it has taken me so long to learn that there is a name for this method of approaching the future, since I have essentially structured my life around it. Since childhood I have generally thought in terms of imagining some exciting possibility for the future, prototyping it first in my mind as an attainable fantasy, and then going about the task of prototyping some version of it in reality.

I think one question that can be teased out here is the one of “how real is real enough?” From the point of view of Hollywood, it is sufficient that audiences see Tony Stark playing with his holographic display up on the big screen. But to me, diegetic prototyping is just a good first step toward making things happen in the real world.

Yesterday I wrote about an absurd encounter with telepresence robots. But then today I had an intriguing conversation about those contraptions.

A friend was telling me that an artist she knew was saying to people at this conference that they should attend the conference art show this evening. One of the people approached by this artist was “present” via one of those mobile telepresence robots.

As it happens, the gallery housing the art show is across the street from the conference center. So they ended up having a conversation about whether the robot would be capable of crossing the street to get to the gallery. Ostensibly this points to a limitation of the telepresence robots.

But suppose, I told my friend, that there were telepresence robots everywhere — much as we currently have electrical outlets everywhere. In that case, being “present” via telepresence robot would be a sort of superpower.

The telepresent person could instantly jump from the conference center to the gallery. In fact, they could instantly jump from anywhere to anywhere else.

It still wouldn’t be the same as really being there, but it might lead to some interesting new possibilities that we haven’t yet thought of.

Today on the first day of the SIGCHI conference, I was watching a session on virtual reality interfaces. But I wasn’t exactly there in person. The room was full, so a number of us were sitting on the floor in the hallway just outside the room, watching the talk on a monitor.

Every once in a while people would walk by, but it wasn’t a problem, because they generally walked really fast. But then something odd happened.

This year at SIGCHI, there are a number of people who are present only via telepresence robot. They are actually somewhere else in the world, and we see them as wheeled robots wandering about the conference, their face appearing as a disembodied image on an LCD screen.

At some point in the talk, one of those critters rolled by. And here’s the thing: They move really slowly. So all of a sudden we couldn’t see our virtual view of the live talk because, in physical reality, our sight line was blocked by the physical manifestaton of a person who was only virtually present.

We all waited patiently for the telepresent person to cross the screen, some of us smiling at the absurdity of it all. Afterward I leaned to the guy sitting next to me and said “That was a very postmodern moment.”

I am spending a rare two days just hanging out, spending time with wonderful old friends in their house deep in the woods, sharing meals and wine and fine conversation. This weekend has been the calm after one storm and before the next.

Workwise, I can’t exactly say I’m doing nothing, since I’m also programming. But it’s the fun type of programming, the kind I do when I want to relax. Sort of like when you pick up your guitar and work out a tune just to please yourself, as opposed to churning out a jingle on deadline.

I had nearly forgotten how wonderful it is simply to have time to chill, to let in some pure clean relaxing energy between the ears. I guess when you’re under the gun, charging forth on sheer adrenaline, you forget what it’s like to be truly calm and rested.

Of course you don’t really forget. Several times in the the last week or two I could see that I was mostly running on fumes. And sometimes, when I didn’t quite manage to hold it together, other people could see it as well.

But now my reserves are back up to a full charge, and I am ready to forge ahead and face whatever comes next, once more unto the breach dear friends. Until that is, it’s time to take another weekend off.

I’ve been watching Marvel’s Daredevil on Netflix. The basic premise is that the superhero is blind, and because he is blind, he uses his other senses, including hearing, touch, smell and proprioception, to a highly enhanced degree.

Our hero can focus on the changing sound of a heartbeat to hear somebody lurking in the shadows, or sense a bullet quickly enough to dodge it in time. He can stop an opponent with a well aimed twirl and kick even if that opponent is coming up from behind.

Since I was a kid I’ve thought of Daredevil as a wonderful premise. Not only does it turn the whole “disabilities” concept on its head, but it also gets kids thinking about talents and capabilities other than the obvious ones. The not so hidden message: You too can be a superhero, even if everyone around you thinks you are a misfit.

Watching this show now I find myself thinking about the future of virtual reality. Maria Lantin and I recently performed a dance duet in a downtown Manhattan gallery, both of us dancing while wearing VR headsets. I think this may have been the first time that a dance was performed by two people who could see the physical world only through VR.

During the entire dance we were both literally blind in the sense that our headsets blocked our view of the physical world around us. Yet we could both see an alternate view of that world, one that showed not only each other, but also choreography, sight lines, and hints as to what was behind us.

In a way we were experiencing a cyber-enabled version of Daredevil’s alternate perception of the world. During the performance I felt very good, as though I was seeing reality in a whole new way.

It’s possible I was getting a glimpse into the future.

I’ve come to realize that when I talk about our lab’s research, I’m generally trying to convey two concepts. One of them is very easy, but the other is a little tricky to get across.

The easy concept is that in the future we will all have those cyber-contact lenses, and will share physical reality just as we do now, except with enhanced senses — essentially the scenario Vernor Vinge describes in Rainbows End. Everybody gets this idea, perhaps because it is a straightforward extension of the cyber-enhanced physical existence we have today.

But the second concept is sometimes met with confusion: The concept that language and conversation itself will consequently evolve. For example, when I demonstrate ideas about how language might evolve to contain a more visual vocabulary, I will sometimes draw a stick figure of a bird. The bird I’ve drawn will then come to life, start to walk around and react to its environment.

People will then sometimes ask: “But isn’t that fake? You created something beforehand, and you are just making it appear as though you drew it now.” I think the confusion comes from the relationship between different parts of language.

When we talk with each other through verbal speech, we rely on most words to already exist. Otherwise we wouldn’t be able to hold a conversation. The bird is essentially a word in a different sort of language.

When I draw it, I’m not claiming to invent a new word on the spot, but rather to show how, in the future, we might converse with each other in a visually enhanced world after such a visually shareable language has had time to develop.

It’s a crucial concept, but alas one that is easy to misinterpret. I still haven’t figured out the best way to get this concept across in a clear and unambiguous way.

But I’m working on it.