Ken's Blog

In 2003, like many New Yorkers, I attended a protest of our president’s decision to go to war against Iraq (I don’t think most of us were against our country being involved in a war — just in that war). The New York City police routed the crowd of protesters in a very odd way. We were shunted off into various side streets, and eventually quite a few of us found ourselves penned in, so that we couldn’t have left if we’d wanted to.

The mounted police showed up, whereupon policemen on horseback started to charge into the crowd. For the unfortunate people who happened to be in front, there was no way to avoid the kicking hooves of the horses. People were trapped, and some of those trapped people were injured.

The next day, national newspapers printed the police description of the incident. According to the official report, hostile protesters had started attacking the police horses, and the police had done their best to protect the helpless horses from the dangerous and unruly mob.

As you can imagine, it was a very strange experience to be reading something in a newspaper and knowing — from first hand experience — that it was simply untrue. And yet I was aware of the brilliance of the police version of the story. Everyone loves horses, and nasty ruffians (even fictional ruffians) who would attack any of these lovely creatures must be the bad guys.

That was seven years ago. Today the police couldn’t have gotten away with a stunt like this. Too many people in the crowd would be carrying SmartPhones, each with the ability to instantly upload images of what was really happening before the police would ever have a chance to take the phones away.

Which leads me to the question of privacy in an ambiscopic world. One objection to everyone having their own ambiscopic display, with anything that you can see instantly streamable to the information cloud, would be the loss of personal privacy within the public sphere. Wherever you go on a city street, somebody will be sure to be recording you, and those recordings can be pieced together to track your even movement — until you enter a private space not occupied by the prying electronic eyes of strangers.

But the incident I described suggests that this might be more of a good thing than a bad thing. Violent crime, acts of hate, police departments descending the slippery slope toward fascist methods, none of these things would be able to flourish in a world where the shared information world is shining a bright light upon the shared public space.

There would still be private spheres, and we would do well to protect them. But it could be argued that a democracy can best flourish when its shared public spaces are exposed to the light, not when they are shadowed in darkness and fear.

It is the eve before Thanksgiving in America, and I find my thoughts wandering to the nature of this most mysterious of holidays. I say “mysterious” because Thanksgiving is such an odd cultural contradiction. The word “thanksgiving” refers specifically to a religious pledge — the pledge to give thanks to God for the bounty from the land (actually, the original Pilgrims were giving thanks simply for having gotten through the winter).

Yet nobody I know experiences Thanksgiving as a religious occasion. I suspect there are many people in our country who do indeed experience it in religious terms. But I don’t know those people.

And so, for me and for all the people I know, Thanksgiving is simply a time to be with family, without any metaphysical overtones. Practically speaking, it is experienced not as a contract between the individual and the Divine, but rather as a contract between the individual and the State: Our government officially declares that we will get a four day weekend, and we unofficially promise that we will spend that time with our families.

We are not required to spend time with our families — the police will not come and arrest us if we don’t visit mom and dad. It’s more that we will feel sad if we don’t. In effect, family itself has become the religious impulse — an imperative imposed not by law, but by a collective inner belief system.

I think this is why Thanksgiving was my favorite holiday when I was a child. I understood that unlike the various religious holidays, Thanksgiving is the one time of the year when society is bound together by a common belief system. For nearly all of us, no matter what God we may worship, and even those among us who think religion is a big waste of time, do indeed share a common faith: a deep seated belief in family.

On the subject of eye contact, I’ve been thinking that it should be possible, without waiting for the future technology of electronic contact lenses, to create an ambiscope that does not place any visible impediment at all in front of your pupil — not even something on the order of eyeglasses.

The form factor I’m thinking of might look something like this:

 

The ear-piece that you see in the above image would extend out from a module that clips to one’s ear, much like the Jawbone hands-free cellphone. Collimated light from that small red tip would be projected toward the user’s eye.

One would think that this couldn’t work, since the rays of light are entering the user’s pupil from the wrong angle. But this is where a little nano-fabrication comes in handy.

Consider the humble contact lens:

 

Normally a contact lens, which rests upon its user’s cornea, uses refractive optics to make slight corrections to incoming light rays. But independently of this, we could also place diffractive patterns inside a contact lens, at a scale smaller than the wavelength of light. Effectively, we would be embedding a custom holographic optical element inside the contact lens.

Below is an image of a blazed nano-scale relief pattern that is currently used as a kind of diffractive mirror. When coherent laser light of the proper frequency encounters this element, the beam of light becomes coherently diffracted, and heads out again into a very different (and very specific) direction:

 

Similar sub-wavelength patterns embedded in a contact lens could be used to deflect laser light coming in from the side, so that the light ends up heading into the eye and toward the retina. The lens could be weighted on one side to keep it oriented properly, which is standard practice for astigmatism-correcting lenses.

Such a custom diffractive pattern would have essentially no effect on non-coherent light that heads straight into the eye, so the contact lens would not interfere at all with normal vision.

This is, in effect, a scaled down analog of the technology currently used for transparent rear-projection screens such as the DaLite Holo Screen. So unlike an electronic contact lens, this is technology that could be built today, given a standard nano-fabrication facility and a little engineering.

So many famous people died on this day in history — November 22. Influential novelists who died today include Aldous Huxley and C.S. Lewis, as well as Anthony Burgess and Jack London. From the world of the musical theatre, both Lorenz Hart and Arthur Sullivan died on this day.

So did Mae West and Mary Kay, two very different kinds of pioneering women, as well as Michael Hutchence from INXS, and even Mark Lenard, who played Mr. Spock’s dad.

Blackbeard the pirate died on this day — the same day of the year as Shemp from the Three Stooges. I’m not sure what that means, but it probably means something.

Yet when I think of November 22, I need to work to remember any of these fine and famous people. Because November 22, 1963 was the day that John Fitzgerald Kennedy died, and somehow for me, and for the history of my country, this was event that changed something fundamental in the nature of our society. In a very real way our nation’s history is divided into two — everything before then, and everything after. There was a kind of innocence that was lost in our culture on that day, and we have never quite managed to get it back.

Death divides our lives. And some deaths divide the lives of nations.

But every death creates ripples, which touch the lives of those still living.

There are several researchers who have, for quite a few years, bravely walked the walk and gone out into the world as cyborgs, wearing various generations of enhanced eyewear that allows them to see a net-connected cyber-reality as they roam about in the physical world. Perhaps the first to do this with a vengeance has been Steve Mann, now at the University of Toronto. At first his technology was large and ungainly, and when wearing his gear he looked like something from the Star Trek Borg. But through successive generations of refinement, his current eyewear is much more sleek:

 

In complementary work, Thad Starner at The Georgia Institute of Technology has been conducting research for many years related to the idea of walking around in one’s daily life while wearing an augmented reality headset. Here is a picture of Thad in his gear:

 

Both Steve and Thad are brilliant researchers, quite ahead of their time. While neither of them is incorporating the real-time head position/orientation tracking that would allow this to be the kind of thing we’ve been talking about, they are both looking seriously at the sociology of wearing a display and integrating it into one’s daily life.

Yet the thing that strikes me about both of these set-ups is that they interfere with eye contact. In both cases, you cannot look directly into the pupil of the person wearing the head-mounted display — the pupil is hidden by the display mechanism, which is literally right in the way.

I could be wrong, but something tells me that this is a show stopper for widespread adoption. Most people, when looking at another person face to face, want to see their eyes. It may or may not be true that the eyes are the window into the soul. but I suspect that retaining the ability to see other peoples’ eyes will be necessary for widespread acceptance of an ambiscopic future.

I was having a dinnertime conversation this evening, and to everyone’s surprise the “liberals” and “conservatives” were in agreement about the nature of the problem afflicting kids in our inner city schools, and the reason it is such a difficult problem to tackle.

The key was not to use loaded phrases like “cycle of poverty”. In the United States you don’t need money to succeed. Money helps, but it’s not the essential differentiator. Rather, you need a kind of inner fire and enough of a belief in the system that you’ll learn what you need to learn, and then apply those skills. Yes, this is more difficult during a recession, but the relative situation stays the same. The kid with the motivation and focus, who builds skills over time, is going to be more likely to succeed as he or she grows up.

But there is another phrase that we all could agree on — “cycle of broken culture”. If the parents have no faith in the system, and sees no point in their child entering that system, then the child is far more likely to reject the value system of skill building and achievement that the schools are trying to offer.

So what to do? You can’t tell a parent “You have bad parenting skills, and we are going to take your child away and make sure what is broken in you does not become broken in your child.” Yet to solve the problem of alienated parents producing alienated children who grow up to repeat the cycle, you must give children some exposure to other ways of being that their parents might not be offering them.

I think there are ways to frame practical approaches to this problem that will make equal sense to both liberals and conservatives. More later.

One question to ask when talking about a new way of looking at the world (literally), is “How do we get from here to there?” In particular, how do we develop applications for a technology that we have not yet finished creating?

One approach is to fake it. Or, to put it in a somewhat more dignified way, to create a functional prototype. In other words, we don’t need to actually build a device to be able to use it — we just need to build some other device that behaves the same way, albeit under controlled conditions.

For example, we can start to work out what a face-to-face eccescopic conversation between two people might be like through the use of head trackers and transparent projection screens. In particular, we can use some present-day technology, such as an Optitrack optical tracker, to measure with high accuracy the positions and orientations of the heads of two people, 200 times per second.

Both people will need to wear some sort of passive tracking markers, perhaps attached to a headband, but that’s ok — such markers won’t interfere with eye contact, and will serve just fine for a prototype.

In addition, we can use one of several types of transparent projection screens, such as the “CristalLine” rear projection screen from Woehburk, or the DaLite Holo Screen. Two people can look at each other through such a screen, while two projectors are projected onto the screen from opposite sides (so that each person sees only one of the rear projections).

Then we can use the tracked position of the two peoples’ heads to, for example, create the illusion of objects floating between the two participants, continually correcting the apparent position and orientation of those objects as each participant moves his/her head.

We can then use a gesture recognition system such as Microsoft Kinect to track free-hand gestures by the two participants. Eventually we would like miniaturized Kinect-like tracking devices to be built directly and unobtrusively into eccescopic headgear.

Of course this is not perfect. Not only do the two participants need to stay generally in one location, but they also cannot reach out and put their hands through the screen. Yet for prototyping what the experience of an eccescopic might feel like — and then implementing and testing out prototype applications — this isn’t such a bad place to begin.

By the way, does anybody think that the word “ambiscope” (which translates roughly into “device to look around”) is better than “eccescope”?

Most of the time when I look at people I see relatively little emotion on the surface. I suppose a society could not function if everyone were walking around on the verge of exploding. When we see people on the street or on the subway who look like they are likely to detonate at any moment, we tend to steer clear — for good reason.

Yet if you spend time with anyone, especially if you spend time with them during periods of stress or great loss, you come to realize that everyone, somewhere inside, has a bubbling cauldron of rage, fear and anxiety, of dark emotions lurking just below that apparently placid surface.

I suppose this is why people respond so powerfully to art that exposes the dark underside, such as Pinter’s The Homecoming, Dostoevsky’s Notes from Underground, Shirley Jackson’s The Lottery, Harlan Ellison’s I Have No Mouth and I must Scream, or just about anything by Kafka.

We seem to derive a peculiar pleasure from watching literary characters we care about and identify with, that we understand on some level as being us, as they approach an emotional abyss and proceed to fall off the edge, descending downward in unchecked flight to some or other existential hell.

I’m not sure what it is that fuels these volcanoes in our souls. Perhaps it is some remnant of the terrible and fearsome three year old within us, that unchecked raging infantile id we all embodied before we ever learned to layer over our raw desires with a mask of social agreeability.

Or perhaps it is simply the knowledge, always lurking around the corner, that our own existence is finite — that for all our struggles and sometime successes, death inevitably awaits.

In any case, I am glad when I see these glimpses of naked truth, when a spark of anger flashes unchecked, or some hidden despair surfaces and reveals itself — even for a moment. I am glad that we all must bear witness, from time to time, to the raging and often ugly sight of other souls in all their ungainly struggle.

Because those are the only times when we know for certain that we are not alone.

This week I visited the MIT Media Lab, where I talked to several of my friends on the faculty about eccescopy. I was surprised and rather delighted at the positive response. For example, Ramesh Raskar is fascinated by the possible form factors, and the ways they could be achieved.

Hiroshi Ishii feels that it is important to look at the entire Interface Ecology — how individuals and social networks communicate, and how an embodied face-to-face cyber-enhanced communication fits together with ideas about community.

And Pattie Maes has pretty much been doing this kind of thing anyway with her students, through things like smart light bulbs (which contain cameras and projectors), and position-tracked portable projectors that you carry around with you to visually augment physical objects in your environment (an idea that has been explored by Pierre Wellner, Hiroshi Ishii and Ramesh Raskar).

We all agreed that following Will Wright’s description of The SIMS 5 (“The game is already in the box … you just can’t open it yet”), is a useful way to frame things: While we are developing the physical support layer, we should be building applications as though that support layer is already available.

Today I was also told by a student about a book I am now going to try to read soon — the 2006 novel Rainbows End by Vernor Vinge. Apparently it is a world of the near future in which everyone wears a portable display that lets them see a cyber-enhanced world superimposed upon our real world.

Well, almost everyone wears a portable display. In the book there are some anti-technology rebels who insist on using good old fashioned computer screens. On some days I know just how they feel.

 

 

 

 

Today, for me, is a day for silence, and remembrance,
and above all for gratitude, beyond measure.