Ken’s Blog

Continuing from yesterday…

As it happened, there was a bottle of soda on the table, within easy reach. Steve asked me to consider the question of whether this bottle was real or just an illusion created by augmented reality.

Of course we cannot ask that question today in any practical sense, because the underlying technology isn’t here yet. But sometime within the next decade, it may very well be.

“Suppose,” he said, “I were to reach out to take this bottle. If it is real, then I can take a swig of its contents. If it is an illusion, then I will go thirsty.”

He pointed out that this is different from the question of whether an image on a computer screen is “real”, because we already understand that such an image has no tangible substance. We never think that what is being represented might be part of our immediate physical world, and therefore we would never think to rely on its literal existence — for example, as a way to quench our thirst.

This sounded like a reasonable point, yet I was skeptical. It’s too easy to think of the technological advancements of one’s own time as being fundamentally different from the advancements that have come before, and I suspected Steve might be falling into this trap.

Precisely because any newly emerging technology is unfamiliar to us, we tend to credit it with outsized power. In contrast, we tend to dismiss the significance of advances from earlier times, because they are so familiar to us: Technological familiarity breeds technological contempt.

After all, we know quite well the cultural, social and psychological norms that bracket existing technologies, and therefore we understand the limits of their effect upon us. Yet we don’t have any knowledge of future cultural, social and psychological norms, so we tend to view future technological advances as being separate from any meaningful cultural context.

That is all well and good in principle, but objecting on principle was not good enough. I needed a more concrete argument. It wasn’t until the following morning that I worked through a counterexample that revealed the flaw in Steve’s logic.

More tomorrow.

I was having a conversation about Augmented Reality yesterday evening with my colleague Steve Feiner, who is one of the great pioneers of AR research. We were discussing changes in human perception that will accompany the coming age of wearables. In particular, we were debating whether those changes will be fundamentally different from the changes that have accompanied earlier technologies.

We both agreed that when wearable technology becomes mature, we will find ourselves seeing 3D objects in the world around us that are not actually there — other than in our perception. The debate centered around whether this difference between perception and reality will be fundamentally different from those provided by previous sensory interfaces that “defy reality”, such as, for example, the telephone or television or Skype.

Steve argued that the implications of the sensory illusions made possible by coming wearables would indeed be fundamentally different from the implications of previous sensory illusions. I argued that there was no fundamental difference — that in fact all such differences are determined by cultural forces and constraints, rather than by the nature of any specific technology.

I realize that this all may sound highly theoretical. But when we got down to cases, the discussion got interesting.

More tomorrow.

Nobody seems to remember phone numbers anymore. After all, why would they?

Back when I was a kid, in an era before phones had gone mobile, we all kept long lists of phone numbers in our heads. Sure, you could write down a phone number. But for people you knew it was more convenient just to memorize seven digits (the area code was usually easy).

With that knowledge in your head you could dial them from anywhere, on any phone that happened to be nearby. We didn’t think of this memorization as a chore. It was something we just took for granted, built into the fabric of “the way things are”, like remembering somebody’s name.

Speaking of which, after we enter the age of wearables, we won’t need to remember peoples’ names anymore. Just today I greeted a colleague, somebody I see all the time at conferences. We gave each other a big hug, and were genuinely glad to see each other. Except I couldn’t remember his name.

It didn’t really matter in that situation. On a social level, you only really need to remember somebody’s name when you are introducing them to a third person. Still, it would have been nice, and I found it somewhat distressing that the name of somebody I know and have liked for many years had managed to elude me.

But once we are all “wearing”, there won’t be any need for such a skill. The ability to keep peoples’ names in your head will come to be seen as one of those arcane skills, like typesetting with metal fonts or tying a proper cravat, which belong to a bygone age.

Saxophonic music reaching up to the Divine
Notes of purest heaven turning water into wine
Saxophonists swaying to the rhythm of the night
Music flying, colors sighing, dancing in the light
Saxophonists singing to the Angels with their sighs
Careful or they’ll melt you with the color of their eyes

Saxophonists playing everything from Bird to Tosca
Swirling round your spirit like the finest Ayahuasca
Saxophonists riffing with a rolling razzmatazz
Worshiping a goddess with the holy name of Jazz
Saxophonists singing to the Angels with their sighs
Careful or they’ll melt you with the color of their eyes

I was exchanging missives today with a dear friend, one whose thoughts invariably inspire me. Our correspondence had meandered into a discussion about how strangely self-destructive we humans can be.

That is indeed a sad thing, and I thought it might be pleasant to at least attempt to put a positive spin on it. So I ended up writing the following:

When faced with any situation in which the stakes seem to be high, people have an unfortunate tendency to do exactly the wrong thing. The upside of this is that it leads to all sorts of great plots for books and plays and movies. Where would literature be without the spectacular ability of people to act like complete idiots?

I wonder whether that counts as a saving grace?

Today marks the start of the second cycle of our Future Reality Lab blog. Two weeks ago we agreed that fourteen members of our lab would collectively maintain a daily blog, with each person posting once every two weeks.

Now that everyone has written their first post, I can see several distinct topic threads emerging. Some talk about language, others about telling stories with VR, still others about spatial audio or interactive animated characters or simply the philosophy underlying our Lab’s collective vision for the future.

As each lab member develops their theme, it will be fascinating to observe how these discussions influence one another. In the weeks to come, I look forward to seeing these narrative threads weave together, to create a beautiful fabric of new thoughts and ideas.

Because I keep a daily blog, I often find myself, over the course of my day, thinking of ideas for things I might want to write about. For the great majority of these ideas, an hour later I cannot recall them at all. Alas, they have fallen into that great “Pit of Forever Forgotten Fleeting Thoughts” (PFFFT) where they are destined for all eternity to remain.

Sure, I could have taken out a pen and paper and scribbled something down, or grabbed my SmartPhone and dictated my thoughts into it. But depending on where I am and what I am doing, such actions are often not an option.

But one can imagine some variant of augmented reality, perhaps involving wearables and subvocal speech, in which as soon as you get a thought in your head, you can instantly record it. In such a scenario, you could record such transient thoughts without the need for any real task switching that might interrupt whatever you are already doing.

So there is the potential there for many more ideas — the ones that spring spontaneously out of our heads in response to whatever is happening in the moment — to actually make it out into the world. Fewer ideas would end up going PFFFT, and more would end up in the intellectual space between us.

That can’t be a bad thing, can it?

It has been said that a culture defines itself by the stories it tells. I was reminded of this last night as I started to watch The Power of Myth, the miniseries from 1988, now on streaming Netflix, in which Bill Moyers interviews Joseph Campbell.

Campbell, who wrote many influential books including “The Hero with a Thousand Faces,” discusses his theories about the journey of the archetypal hero found in the mythologies of all cultures. I’m only a bit into the first episode, and already I am hooked.

And it got me thinking again about the relationship between stories and games. Many people use stories centered on a hero’s journey as a cultural and psychological touchstone, whether that hero be Hamlet, Luke Skywalker or Elizabeth Bennet.

Unlike stories, where one can only experience that journey vicariously, computer games allow the player to be the hero, directly making choices that effect the outcome. Yet in the general culture, Gordon Freeman, the Master Chief or Nathan Drake have not enjoyed the outsized recognition of, say, Huckleberry Finn or Emma Bovary or Katniss Everdeen.

One game character in particular, Lara Croft, has certainly entered the consciousness of the larger culture. Yet her outsize visibility is arguably related to the films starring Angelina Jolie, which brings us back to linear narrative.

Theoretically, a medium that allows us to walk the path of the hero’s journey for ourselves should have great power, as compared with a medium that merely asks us to watch. So why do linear narratives seem to be so much more influential than games in this regard?

Suppose we accept as a hypothesis that a culture defines itself through its versions of the hero’s journey. Why do stories of a hero’s journey seem to capture the imagination of the culture more effectively than games on the same theme?

Is this disparity simply due to the newness of the computer game as a cultural medium? And if so, should we expect this imbalance to change over time?

I was walking up Fifth Avenue this week, and when I got to around 10th Street I saw this. So I snapped a picture.

It looks a lot like what I saw when I visited the Amazon rainforest in Brazil. Different species of trees, but similar feeling.

It is good to know that there are still places in this urban jungle where you can see something that looks for all the world like, well, a jungle.

If you were to discover that the person inside the Mickey Mouse costume at Disney World was actually a robot, would that make a difference to you? Would you feel less comfortable with such a non-human “actor” posing for a photo with your child?

I imagine many people would start out by saying “Yes, of course it would make a difference!” When we think we are faced with a real human being, but then find we are dealing with a simulacrum, we tend to feel betrayed, at the very least.

Yet there is a subtlety here. That real live human inside the Mickey Suit is also a simulacrum. He or she has simply been hired to play a gig.

The actor in question does not necessary feel any actual emotional bond with your child. It’s all really a business transaction: A suit is worn, photos are taken, and at the end of the week an actor is paid.

So in this sense, might it not be more honest to have a robot portray Mickey? That would spare everybody involved the oddness of a situation whereby a total stranger goes through the pretense of caring about your child, only for the sake of a paycheck.

I realize this sounds cynical, but I’m not trying to be cynical. I’m genuinely trying to understand a difficult question about human nature:

Do we still prefer interaction with real humans rather than fake humans, even when that interaction is based on deception? And if so, why?