Author: admin

The Spatial User Interaction workshop I am attending this weekend features many exciting new approaches to how people can interact with computers. Yet I’ve noticed an odd thing about some of these approaches.

They are cool, they are exciting, they are certainly thought provoking, but in some cases they just don’t work very well. Recognition of a user’s gestures is often error prone, or subject to noise, or ambiguous, or just too coarse for allow fine distinctions.

I’m beginning to think that there is some law of conservation at work here: The more radical is an idea for how people can interact with information, maybe the less likely that it will be truly useful.

I’m not saying that’s always the case, just that I see a pattern.

One example of this, which has become a bit of a joke among people in the user interfaces community, was the way the character played by Tom Cruise in the Stephen Spielberg film Minority Report held his arms up to direct things on the computer screen in front of him. The underlying ideas, which largely came from John Underkoffler, were indeed exciting.

Yet the way those ideas showed up in Spielberg’s direction, they didn’t really work on any practical level. What mere mortal could really hold their arms straight out in front of them for entire minutes at a time?

On the other hand, it looked very cool. 🙂

This morning I gave a keynote talk at the Spatial User Interaction workshop, in which I presented a kind of vision of the future. In the Q&A after the talk, people asked some really great and challenging questions.

In my answers to those questions, I realized that I kept returning to the same point: The most amazing thing about humans is our built in ability to communicate through natural language. Everything we do, build, create, comes out of that shared ability.

We so take it for granted that this superpower is “normal”, that we generally forget how astonishing it is.

To a sentient being that did not possess the ability to casually communicate their thoughts to each other, what we do every day, without even thinking about it, would seem like pure magic.

Consider not just our computer software, or our movies, books and plays, but the very clothing we wear and the buildings we inhabit, our bridges, roads, eyeglasses and coffee makers. These, and everything else we create, are basically outgrowths of our shared language instinct. Without it, none of these things would exist.

The fact that you are reading this right now and are immediately forming your own thoughts, theories and counter-theories in response, really is a marvel beyond compare.

Just didn’t want you to forget that. 🙂

Do not trust airplanes
One of those things took away
Someone dear to me

A stranger returned
Same of face, same of body
But not my true love

That last farewell kiss
Is as fresh now in my heart
As a new red rose

With a taste so sweet
And promise of a future
That was not to be

Do not trust airplanes
No, do not trust them. When they
Take your love away

The fire of a kiss,
Once so tender, leaves nothing
But a taste of ash

I’m going to be giving a presentation this Saturday. Much of it will be new, although I’ll be using elements from talks and demos I’ve given before.

I remain somewhat nonplussed by the mysterious alchemy that goes into preparing for one of these talks. On the one hand, the weeks leading up to it seem so busy, in an almost random way. I might spend hours — even days — on experiments that I end up throwing out, but then something that took only a few minutes ends up being exactly right.

Somewhere in the back of my mind, I believe, part of me knows exactly what will happen, how the actual talk will unfold, where all the beats will land. But that part of me isn’t telling the rest of me. Perhaps he doesn’t want to spoil the surprise.

As the days and then hours tick closer to the presentation, I finally start to see where I was going all the time, and only then does it fully make sense.

I used to think I should be fully planning these things out, in advance, that I was somehow being lazy or remiss by letting the pieces fall into place in such an apparently haphazard way.

Now I know better.

When I was a child I came upon the speculative fiction novel “October the First is too late” by the great astrophysicist Fred Hoyle. It’s a strange and stimulating book, about time itself folding like a pretzel and possibly coming to an end, which wreaks havoc on reality — particularly human reality.

For some reason my memory of that book always reminds me of another treatment of time itself in popular fiction that felt as weird and wonderful: The character of Emit Flesti in Wim Wenders’ “Far Away So Close”, a character played so memorably by Willem Defoe.

Of course today is, in fact, October the first. On this day of the year, a part of my mind invariably flashes the thought that Hoyle got it wrong: Time itself has not in fact come to an end, since we have safely made it to October. Feel free to breathe a sigh of relief.

But maybe getting it wrong isn’t the worst of sins. When you think about Defoe’s character, you realize that Wim Wenders didn’t just get it wrong about time Itself. He got it backwards.

I had the good fortune to be invited to attend a retirement celebration this evening for Fran Brill, one of the great Sesame Street muppeteers. The sheer talent in the room was breathtaking. Many of my favorite muppet characters were there — Miss Piggy, Big Bird, Grover, Snuffleupagus and more — all cleverly disguised in their human form.

The evening prominently featured a whirlwind review of her body of work through the decades. There is something marvelous about seeing the result of many years of creativity, artfully conveyed through a single evening’s worth of representative samples.

We are only human, and in any given month or year we can accomplish only so much. But the result of decade after decade of dedicated talent, focus and effort can be astounding.

There are times when I feel that I’m not getting enough done, and I lose heart — but then I see something like this. It reminds me of the power of human energy over time, and I am newly energized.

The takeaway here: Find something you love, just put one foot in front of another, and keep going. In the long run, it’s all going to be wonderful.

At this week’s faculty meeting of our Media and Games Network (MAGNET), we did a recap of the gender related events at last week’s Oculus Connect meeting. And we decided that we need to take a more pro-active stand on the need for men and women to work together toward gender parity in our field.

MAGNET already does a fair bit in this direction. It hosts Girls who Code, Black Girls Code, and other events that help to encourage young women to enter fields that require facility with software engineering.

But now that is going to be an explicit part of our mission — on the masthead, if you will. After all, why should the United States of America be a backwater among the industrial nations, capable of attracting only 50% of its potential workforce to 21st Century jobs?

Interestingly, due to vacations, sabbaticals, conferences, and other events, none of our female MAGNET faculty were in town this week. All of the people pushing for this new direction today were men.

Today I saw a delightful sign outside a restaurant. To emphasize the rustic nature of the establishment, the sign maker had built each individual letter out of hand-whittled wooden sticks.

Clearly a lot of work had gone into the effort, and the result was worth it. Just looking at that sign made some part of me want to abandon my city slicker ways and live off the land, like our ancestors did once upon a time.

But then I looked at the sign again, and I realized that the text on the bottom line was the restaurant’s phone number. At first I hadn’t noticed this little bit of culture shock because we don’t generally think about phone numbers any more.

But there it was, if you cared to look: an iconic symbol of modernity, represented by little pieces of rough-hewn hand-whittled wood.

And I couldn’t help but wondering: Maybe someday soon we will see that same beautifully old-fashioned sign, but with the phone number replaced by a URL — lovingly carved in rustic little hand-whittled wooden letters.

Or maybe not a URL — maybe a hashtag.

Or maybe whatever is going to come after the hashtag. Your guess is as good as mine.

A number of professions have recognized the issue I was discussing yesterday. Bringing women into a field that was historically dominated by men requires both men and women to work together.

For this reason, fields as diverse as anthropology, law, medicine and architecture have focused on doing just that. These professions don’t see this as an issue of “men versus women”. They see it as a way of getting talented young people into the field. And if you look at the gender balance in such fields over the past few decades, you see that this strategy has been working spectacularly well.

Facebook, the parent company of Oculus, has been doing this as well. Mark Zuckerberg has had the good sense to bring on Sheryl Sandberg as his COO. She has pointed out that men need to be part of the process of increasing participation by women — not as a handout to women, but to maximize their company’s ability to compete.

These examples of inter-gender cooperation are particularly impressive given that there are land mines everywhere. For example, women I know have privately told me that they have avoided commenting here in the last few days because they are afraid of on-line retaliation by trolls. After what I’ve seen, I’m afraid their fears are well founded.

And of course there are more subtle forms of bias, even among men who consider themselves enlightened. If a man asks a question at a conference in a knowing and slightly smart-ass tone, he is generally admired for his daring and chutzpah. But if a woman speaks in exactly the same tone, she risks coming across as “arrogant and smug”.

Based on the comments to yesterday’s post, it seems that Oculus was faced with a situation where practically no women were interested in developing for Virtual Reality. And that tells me something odd is going on here.

Among my female students (all of whom program computers), most are highly interested in V.R., and are eager to develop for it. I also know quite a few women who are currently working in V.R.

In one of the most exciting developments in the entire field, one of my colleagues at NYU is working on solving the problem of true spatially correct real-time audio reconstruction for V.R. About half of the grad students working with her on this are women.

As most of you know, V.R. itself is far from new — although its viability as a consumer product is indeed new. For over twenty years I have known female colleagues who have done research in and developed for virtual reality, including quite a few grad students.

So it seems we are confronting virtual realities of a different kind. On the one hand I have known, for many years, women who are highly skilled programmers who are also extremely interested in V.R. and have been working on it. On the other hand, when Oculus holds a conference, these women cannot be found.

I don’t know the answer to this mystery, but it might help to explain the tone of my previous two posts on this subject.

It also might help explain why I think Palmer Luckey can be a pivotal figure. The female V.R. researchers and developers are indeed out there. But many of them might be looking for a sign that they are welcome.

Unlike men, who never need to deal with this issue — and therefore can remain blissfully unaware that there even is an issue — I suspect many women in high technology know very well what happens when they show up somewhere they were not invited.