Month: November 2010

I started to write a post today about a cool conversation I recently had with a friend on an interesting math topic, until I realized it just wouldn’t work. It’s not that the concepts were out of anyone’s grasp — the ideas we were discussing were all very simple and logical and beautiful, and certainly not beyond the reach of anyone who reads this blog.

It was rather that the only way to discuss those ideas would require the notation of math. And I am all too acutely aware that most people have a very unfriendly relationship with mathematical notation. If you write something like e^x, or x + iy, most people will start to panic, and promptly set about finding somewhere else to be.

The ideas that these expressions represent are not at all difficult, but somehow the very fact that it’s “math” stops people dead — you end up discussing not the ideas, but rather the wall of incomprehension surrounding the way those ideas are expressed.

It’s a shame, because mathematical ideas have the kind of immense beauty found in so few things — a glorious sunset or a great Shakespearean sonnet come to mind. But unless something changes radically, that beauty will likely remain, for most people, out of reach.

Mari’s comment the other day, about extended embedded interaction technology to other senses, leads to a very good point. The more we build our interaction technologies into our own bodies, the more vulnerable we become to perception hacking.

It’s one thing if a computer screen is hacked, since the damage is localized. You might not be seeing a true representation of networked information, but the misinformation does not become conflated with the world around you. Once somebody else can hack into channels of perception that you use to see reality itself, the results can be far more damaging.

If you have technology that lets your eyes see things that aren’t there, or your ears hear things that don’t exist, or your fingers touch objects that are not real, you become open to the possibility that malicious software will be able to send false information through your enhanced senses, giving you a false impression of the reality around you.

And so an entirely new field might arise — a field of security that protects you from having your augmented reality replaced by a chimera. If you get used to seeing ambiscopic signs to give you directions, or to verifying the identities of people by looking at the information you see floating above their heads, you become vulnerable to this sort of hacking.

I don’t have any solution to this problem, since there is never a simple solution to the abuse of any new technology. But this is an issue that we will need to keep in mind, as we continue to augment the interface between brain and sensed reality.

There is a very solemn burial scene in the new Harry Potter movie. I know it’s supposed to be moving and deeply emotional, but the scene doesn’t work if you’ve been to an actual funeral in the recent past. One of the primary functions of such a scene is to comfort us with the knowledge that “it’s only a movie”, and that everything will be ok when we leave the theatre.

But of course if you already know that everything won’t be ok when you leave the theatre, then such scenes have the unintended effect of pulling you out of the fantasy of the movie, and bringing you back to sad events in your real life.

In contrast, gallows humor definitely helps. Today my friend Gerry told me about something that had happened at his father’s funeral some years ago. His father’s body was lying in its coffin in the funeral home, and Gerry was paying his last respects — looking at his father one last time and giving him a final highly emotional kiss.

Gerry told me that as he was walking away from the coffin, the only other person in the room — the funeral director — spoke to him. “I’m sorry to have to say this,” the funeral director explained, “but it will be a closed casket ceremony, so officially I need to ask you now the following question: `Is the person lying in that coffin actually your father?'”

The sudden transition from a deeply personal and emotional moment to something so absurd was a bit much for Gerry, which explains what he did next. He looked the funeral director squarely in the eye, and said “I’m afraid that’s a question you’re going to have to ask my mother.”

I had a wonderful lunch meeting this week with a colleague who does research involving localized sound synthesis. She told me that we now have the necessary computer power (thanks to Moore’s Law) to compute real-time localized sound. In other words, you can compute a synthetic soundscape for a listener’s two ears so that any sound can appear to originate from a particular location in space. This means, for example, that you can place a virtual radio in a room, and make it seem as though the sound is coming from the radio — even though there is no actual radio.

The only remaining bottleneck to this process is tracking the exact position and orientation of the listener’s head. Fortunately, that is one of the problems we are already tackling for ambiscopic interfaces. Which means that as we create virtual visual objects in the environment around you, we will also be able to have those objects make sounds.

So everyone will be able to walk around seeing and hearing virtual objects, but we still won’t be able to touch those objects. So it won’t quite be like holodeck we all saw in Star Trek, the Next Generation. But maybe there is something we can do about that.

There was a time, not all that long ago in history, when if you said that people would one day deliberately have their corneas burned by powerful lasers so they wouldn’t need to wear eyeglasses, everyone would think you were insane. But of course today we have Lasik surgery, and nobody thinks anything of it.

Similarly, if today you said that we will one day deliberately implant chips in our fingers to artificially stimulate the nerves in our fingertips, most people will just become uncomfortable and try to change the subject. But once there is an omnipresent virtual visual and auditory world that we all take for granted, coexisting with our physical reality, cultural values will start to shift.

Also, in just a few years from now the technique of embedding tiny devices that stimulate individual nerves in the fingers (which is vastly easier than implanting chips to stimulate entire arrays of brain neurons) will become a commercially viable proposition, and at that point the technology will start to make the familiar transition from a specialty operation to a high-end entitlement to a widely available consumer option, as have dental implants, liposuction and Lasik surgery in their time.

Some time in the next ten to fifteen years, we will all be able not just to see and hear, but to feel the virtual world around us.

A friend recently told me that Warner Brothers has decided to release a Buffy the Vampire Slayer movie, without the participation of Joss Whedon. For those of you who, like me, watched the seven seasons of Buffy in order, such an idea seems more than a little surreal. Note: if you haven’t seen all of Buffy, or if you’ve only seen an episode here or there, nothing I’m about to say will make much sense to you.

Buffy is one of the few true masterpieces to come out of the television era. Those of us who actually watched it from beginning to end — the way you’re supposed to read a novel — live in awe of Whedon’s genius. The depth and subtlety of the characters and their evolving relationships, the beauty and symmetry of the multi-year interlocking narrative arcs, the rock-steady hand on the many post-modern experiments Whedon embedded in the work, and the sheer brilliance and wit of his dialog, none of these are things that can could ever be replicated by hired hacks.

But I guess this is not surprising. These are, after all, the same folks who decided not to promote Brad Bird’s animated classic The Iron Giant in 1999 because they were too busy advertising one of the worst movies ever made — the shockingly inept film adaptation of The Wild Wild West. Ed Catmull has told me that he is still grateful to them for that, since Brad Bird promptly took a job at Pixar and never looked back.

Apparently this is all part of a larger plan at Warner Brothers. A disgruntled WB employee has secretly leaked to me their plans for forthcoming releases, and here are some of the highlights: After Buffy, Warner Brothers staff will write and release a new Beatles song. Then the studio will be issuing a new Jane Austen novel, followed by a collection of all new Borges short stories.

They will follow this with two new Shakespeare tragedies and a Chekov one-act, an all-new Tolkien mythology, and then, in 2012, Beethoven symphony number 10 in G minor, followed by Mozart symphanies number 42, 43 and 44. Finally — and this is where you really have to hand it to the brilliant folks at Warner Brothers — they are going to come out with a new Holy Bible.

When asked for comment about this last project, a Warner Brothers spokesperson indicated that the studio had originally invited God to submit a treatment, but that a development deal with the Divine Creator had fallen through at the last moment, which is when they decided to go with in-house writing talent.

God could not be reached for comment.

In 2003, like many New Yorkers, I attended a protest of our president’s decision to go to war against Iraq (I don’t think most of us were against our country being involved in a war — just in that war). The New York City police routed the crowd of protesters in a very odd way. We were shunted off into various side streets, and eventually quite a few of us found ourselves penned in, so that we couldn’t have left if we’d wanted to.

The mounted police showed up, whereupon policemen on horseback started to charge into the crowd. For the unfortunate people who happened to be in front, there was no way to avoid the kicking hooves of the horses. People were trapped, and some of those trapped people were injured.

The next day, national newspapers printed the police description of the incident. According to the official report, hostile protesters had started attacking the police horses, and the police had done their best to protect the helpless horses from the dangerous and unruly mob.

As you can imagine, it was a very strange experience to be reading something in a newspaper and knowing — from first hand experience — that it was simply untrue. And yet I was aware of the brilliance of the police version of the story. Everyone loves horses, and nasty ruffians (even fictional ruffians) who would attack any of these lovely creatures must be the bad guys.

That was seven years ago. Today the police couldn’t have gotten away with a stunt like this. Too many people in the crowd would be carrying SmartPhones, each with the ability to instantly upload images of what was really happening before the police would ever have a chance to take the phones away.

Which leads me to the question of privacy in an ambiscopic world. One objection to everyone having their own ambiscopic display, with anything that you can see instantly streamable to the information cloud, would be the loss of personal privacy within the public sphere. Wherever you go on a city street, somebody will be sure to be recording you, and those recordings can be pieced together to track your even movement — until you enter a private space not occupied by the prying electronic eyes of strangers.

But the incident I described suggests that this might be more of a good thing than a bad thing. Violent crime, acts of hate, police departments descending the slippery slope toward fascist methods, none of these things would be able to flourish in a world where the shared information world is shining a bright light upon the shared public space.

There would still be private spheres, and we would do well to protect them. But it could be argued that a democracy can best flourish when its shared public spaces are exposed to the light, not when they are shadowed in darkness and fear.

It is the eve before Thanksgiving in America, and I find my thoughts wandering to the nature of this most mysterious of holidays. I say “mysterious” because Thanksgiving is such an odd cultural contradiction. The word “thanksgiving” refers specifically to a religious pledge — the pledge to give thanks to God for the bounty from the land (actually, the original Pilgrims were giving thanks simply for having gotten through the winter).

Yet nobody I know experiences Thanksgiving as a religious occasion. I suspect there are many people in our country who do indeed experience it in religious terms. But I don’t know those people.

And so, for me and for all the people I know, Thanksgiving is simply a time to be with family, without any metaphysical overtones. Practically speaking, it is experienced not as a contract between the individual and the Divine, but rather as a contract between the individual and the State: Our government officially declares that we will get a four day weekend, and we unofficially promise that we will spend that time with our families.

We are not required to spend time with our families — the police will not come and arrest us if we don’t visit mom and dad. It’s more that we will feel sad if we don’t. In effect, family itself has become the religious impulse — an imperative imposed not by law, but by a collective inner belief system.

I think this is why Thanksgiving was my favorite holiday when I was a child. I understood that unlike the various religious holidays, Thanksgiving is the one time of the year when society is bound together by a common belief system. For nearly all of us, no matter what God we may worship, and even those among us who think religion is a big waste of time, do indeed share a common faith: a deep seated belief in family.

On the subject of eye contact, I’ve been thinking that it should be possible, without waiting for the future technology of electronic contact lenses, to create an ambiscope that does not place any visible impediment at all in front of your pupil — not even something on the order of eyeglasses.

The form factor I’m thinking of might look something like this:

 

The ear-piece that you see in the above image would extend out from a module that clips to one’s ear, much like the Jawbone hands-free cellphone. Collimated light from that small red tip would be projected toward the user’s eye.

One would think that this couldn’t work, since the rays of light are entering the user’s pupil from the wrong angle. But this is where a little nano-fabrication comes in handy.

Consider the humble contact lens:

 

Normally a contact lens, which rests upon its user’s cornea, uses refractive optics to make slight corrections to incoming light rays. But independently of this, we could also place diffractive patterns inside a contact lens, at a scale smaller than the wavelength of light. Effectively, we would be embedding a custom holographic optical element inside the contact lens.

Below is an image of a blazed nano-scale relief pattern that is currently used as a kind of diffractive mirror. When coherent laser light of the proper frequency encounters this element, the beam of light becomes coherently diffracted, and heads out again into a very different (and very specific) direction:

 

Similar sub-wavelength patterns embedded in a contact lens could be used to deflect laser light coming in from the side, so that the light ends up heading into the eye and toward the retina. The lens could be weighted on one side to keep it oriented properly, which is standard practice for astigmatism-correcting lenses.

Such a custom diffractive pattern would have essentially no effect on non-coherent light that heads straight into the eye, so the contact lens would not interfere at all with normal vision.

This is, in effect, a scaled down analog of the technology currently used for transparent rear-projection screens such as the DaLite Holo Screen. So unlike an electronic contact lens, this is technology that could be built today, given a standard nano-fabrication facility and a little engineering.

So many famous people died on this day in history — November 22. Influential novelists who died today include Aldous Huxley and C.S. Lewis, as well as Anthony Burgess and Jack London. From the world of the musical theatre, both Lorenz Hart and Arthur Sullivan died on this day.

So did Mae West and Mary Kay, two very different kinds of pioneering women, as well as Michael Hutchence from INXS, and even Mark Lenard, who played Mr. Spock’s dad.

Blackbeard the pirate died on this day — the same day of the year as Shemp from the Three Stooges. I’m not sure what that means, but it probably means something.

Yet when I think of November 22, I need to work to remember any of these fine and famous people. Because November 22, 1963 was the day that John Fitzgerald Kennedy died, and somehow for me, and for the history of my country, this was event that changed something fundamental in the nature of our society. In a very real way our nation’s history is divided into two — everything before then, and everything after. There was a kind of innocence that was lost in our culture on that day, and we have never quite managed to get it back.

Death divides our lives. And some deaths divide the lives of nations.

But every death creates ripples, which touch the lives of those still living.

There are several researchers who have, for quite a few years, bravely walked the walk and gone out into the world as cyborgs, wearing various generations of enhanced eyewear that allows them to see a net-connected cyber-reality as they roam about in the physical world. Perhaps the first to do this with a vengeance has been Steve Mann, now at the University of Toronto. At first his technology was large and ungainly, and when wearing his gear he looked like something from the Star Trek Borg. But through successive generations of refinement, his current eyewear is much more sleek:

 

In complementary work, Thad Starner at The Georgia Institute of Technology has been conducting research for many years related to the idea of walking around in one’s daily life while wearing an augmented reality headset. Here is a picture of Thad in his gear:

 

Both Steve and Thad are brilliant researchers, quite ahead of their time. While neither of them is incorporating the real-time head position/orientation tracking that would allow this to be the kind of thing we’ve been talking about, they are both looking seriously at the sociology of wearing a display and integrating it into one’s daily life.

Yet the thing that strikes me about both of these set-ups is that they interfere with eye contact. In both cases, you cannot look directly into the pupil of the person wearing the head-mounted display — the pupil is hidden by the display mechanism, which is literally right in the way.

I could be wrong, but something tells me that this is a show stopper for widespread adoption. Most people, when looking at another person face to face, want to see their eyes. It may or may not be true that the eyes are the window into the soul. but I suspect that retaining the ability to see other peoples’ eyes will be necessary for widespread acceptance of an ambiscopic future.