Month: August 2015

Coming home, after many weeks away, I recognize so much that is familiar. “Ah yes,” I tell myself, “this is my coffee, this is my New York Times in the morning.” There are so many familiar touchstones, from the walk along Washington Square North to the studied indifference of fellow New Yorkers, which was long ago raised to a high art.

And yet.

And yet, I find myself both here and not here, torn between the New Yorker I was six weeks ago, and the person I have become since then. My mind is filled with new people, new faces, new ideas and possibilities.

I will carry all this new experience within me, in my daily life here. But perhaps this time, after having gone away and come back, finding that my soul has been touched in transformational ways, I can no longer simply say “I am a New Yorker”.

Perhaps it would be more accurate to say “There is much that is New York within me. But there is more besides.”

When we talk to each other, we are using natural language. One of the qualities of natural language is that it is “naturally learnable” — it doesn’t need to be explicitly taught. Nearly 100% of children born into any society will, in the first seven or so years of life, master the grammar, and much of the vocabulary, of that society’s verbal speech.

Written language is something else entirely. Although it might seem to many of us to be essentially the same as verbal speech, it is not naturally learnable. Very few children will spontaneously learn to read and write. Rather, it is a skill that generally needs to be explicitly taught.

Our lab’s research has recently advanced to the point where we can put people into a shared simulation of social “future reality”. We can now (sort of) simulate those futuristic contact lenses which will allow us to graphically augment gesture: Other people will be able to see the shapes that you make in the air with your hands.

So it may at last be possible to find empirical answers to a question I’ve been thinking about for the last seven years or so: Will the shapes that we make in the air end up being more like natural language, or more like written language?

In other words, will the resulting visually enhanced speech evolve toward something that every child will learn, in the normal course of their growth and development? Or will will it be something that needs to be explicitly taught, the way that written language needs to be explicitly taught?

I strongly suspect that it will be a bit of both. In any case, it will be fun to do the research and see what happens.

One day, after we live in a world where cybernetic information from the cloud is superimposed on our view of other people, somebody is going to get the bright idea to superimpose mood and personality information on top of our view of other people.

There are good things and bad things about this. Among the good things, such a capability could provide a useful caution. If somebody has a short temper, or is sensitive to certain topics, or is just coming out of a bad encounter that could throw them off their game, you might be able to get fair warning, and adjust accordingly how you interact with that person.

Among the bad things, this sort of capability is prone to social media hacking. Your view of somebody might become colored by some person or group of people who dislike them, and are therefore prone to providing an unfairly negative or otherwise biased view of them.

And then of course there is the question of privacy.

I am quite confident that the sort of capability I am describing will be possible sometime in the near future. But I am not at all sure that it is something that will make our world a better place. Maybe we will end up collectively deciding to opt out of this particular superpower.

I had a conversation today on the subject of styles of teaching. When you are trying to impart some knowledge or understanding to a group of people, there are various strategies you can use.

One way to approach the task is what might be called the “mountaintop” strategy. In this approach, you are essentially standing on top of a mountain, and your message is aspirational: “If you pay attention, work hard, and be the best you can be, then you might some day be able to reach the height that I have reached.”

Another way to approach it is what might be called the “mountain climber” strategy. In this approach, you stand next to the student, while look up at the mountain. Essentially you are saying “Hey, this sure is a tall mountain. I’m going to try to climb it. Want to try it with me?”

There are advantages to both strategies, and each will work better for a particular kind of student. But I think it’s also true that each will work better for a particular kind of teacher.

For me, the second strategy works better as a teaching strategy, because it matches my personality. For one thing, I really like climbing mountains, especially old favorites that I have visited before. For another thing, when I go mountain climbing, I very much enjoy having company.

Suppose I am a dancer or an actor, and I am charged with giving a performance for an audience. In today’s world, if I want complete freedom of movement, current production practice requires me to use my naked eyes to look at the physical world around me.

But soon production practice will change, due to recent and imminent advances in technology. Performers will be able to see whatever they wish in the physical space around their bodies, without any loss of freedom of movement.

As this happens, entirely new visual languages will develop, to be used by performers. Dancers will see choreographic steps, lines of principle flow, and other useful indications in the space around them, while actors will perceive virtual sets, animated characters, dialog prompts, sight line hints, and other production cues.

Directors will have entirely new, far more visual, ways to communicate with their cast, just as choreographers will have new powerful tools for expressing their vision to the dance troupe that will realize that vision.

It can be difficult to predict what these visual production languages will look like. But it is going to be exciting to watch them develop.

Time passed in a blur. The One, the large creature with the power to communicate, would come by from time to time to share greetings, and she found these moments pleasurable. She detected in his mind other stirrings, ways of thinking that she could not quite understand. This would require further study.

The other large creatures were a complete mystery to her, although she could see that they were of the same tribe as the One. They did not communicate in any way that she could detect, yet clearly there was some sort of interaction between them.

Often this interaction felt like conflict, as though the creatures were preparing for war against each other, and this troubled her. A tribe does not war against itself.

Yet somehow she knew that no matter what happened, the One would protect her. Meanwhile, he made sure that there was always food. The days drifted by, and she fed, and she grew, and she was content.

A Vancouver walk
With a new friend. A perfect
Way to spend a day.

I just saw a great talk about the neuro-physics of human eyes. One take-away was the remarkable difference between what we believe we are paying attention to, and the actual movements our eyes make.

One basic difference is that we tend to believe that our gaze is continuous, because our minds construct an illusion of continuous change of gaze. In fact, our eyes more often change their gaze by darting around in an extremely fast and discontinuous way, through extremely rapid saccade movements.

I wonder whether this generalizes. Perhaps our thoughts work the same way. Maybe our subjective experience of continuous thought is also a trick of perception.

Perhaps the reality, below our level of conscious perception, is that our thoughts actually proceed through a series of mental saccades, jumping in wildly and in divergent ways from moment to moment, with the illusion of continuous thought merely an illusory construct overlayed on top.

Proof by induction:

Suppose, for the sake of argument, that one day somebody invents a machine that lets people travel back in time. That will be a big day in history — the day that travel back in time finally becomes possible!

Here’s the problem: Precisely because it will be such a big moment in history, sooner or later people from the future are going to want to come back and visit that historic day. In fact, since people will be traveling to the same destination from all different eras in the future, that particular moment in history is going to become awfully crowded.

And not all of those visitors from the future will be as careful as they should be. One of them, inevitably, is going to do something that will interfere with the sequence of events that resulted in the invention of the time machine.

When this happens, the time machine will no longer have been invented on that day. That entire future timeline will cease to exist, curious time traveling tourists and all.

Which means that we will enter an alternate timeline. In this new timeline, travel will only be invented sometime later.

But in this new timeline, the same sequence of events will happen, only at a later date.

And so, alas, QED. Nobody will ever invent that time machine.

We spent nearly an entire year building our Holojam demo for this week’s big SIGGRAPH conference. Yet it wasn’t until after we actually showed it — and got feedback from a large number of people — that I finally understood the significance of what we had built.

By “significance” I don’t mean long-term significance. As researchers, we were thinking about that quite a lot. I’m talking here about immediate significance.

All sorts of people, including effects animators, film directors and scientists, started to tell us things they would now be able to do, after having experienced our shiny new toy.

So here we were, completely focused on how our research could possibly impact things in twenty or thirty years from now, and we somehow missed the fact that it is also very likely to impact things right now.

Isn’t there a saying that sometimes you can’t see the trees for the forest? If not, there really should be.