Only human

November 25th, 2015

Nearly every guy I know over 40 has back problems. This is not surprising. After all, the evolution of our primate ancestors to adapt to bipedal travel happened over an extremely small time scale, as these things go. You and I and all of civilization have happened along only quite recently.

In fact, the entire history of human civilization is just a tiny blip in evolutionary time. So it should not be surprising that the adaption of our spinal cords for upright posture is still very much a work in progress.

I think we have a similar problem with our human Theory of Mind. By far our biggest power-up as a species is our ability to model each others’ thoughts and emotions, and thereby, with the help of natural language, to use those mental models to form kinship groups that let us cooperate in ways that greatly increase the survival fitness of our species.

But this evolution too is a work in progress. When the exponential growth of civilization started to really take off — a mere five thousand or so years ago — our species was still in the process of gradually evolving its biological capacity for Theory of Mind.

To it’s not surprising that people have so much difficulty understanding each other. It’s not that there is anything wrong with us. It’s just that we are us. The brain of each human acts as an imperfectly evolved instrument for creating, and then acting on, a mental model what is going in the brains of other humans.

But that brain is still, as far as we know, the best goddamned instrument for doing such a thing. So while we bemoan (understandably) the tragedies of war, terror, irrational prejudice and unreasoned hatred, perhaps we can take a moment to remember that this is the best human brain we’re going to get, and that even in its imperfectly evolved form that brain is still pretty amazing.

Sure, we are often totally screwing up this whole civilization thing. But then again, we are only human.

Flight done right

November 24th, 2015

A year and a half ago, at the 2014 SIGGRAPH Conference in Vancouver, I first tried out Birdly, a kind of virtual reality experience in which you put on a VR headset and get strapped into a contraption that looks vaguely like a bird. When you flap your arms, you find yourself “flying” around San Francisco, like a bird soaring through the air.

Today I was thinking about this experience because our current “Future Reality” research at NYU involves using Tactonic Technology floor mats as a kind of magic flying carpet that lets you fly around in virtual worlds merely by shifting your weight — much as I imagined doing as a little kid, when I first learned about magic carpets, and wondered why I couldn’t have one.

This in turn made me think about the flying dreams that I still have from time to time, and which are always completely delightful. In my flying dreams, I am often soaring around cityscapes, swooping gracefully between buildings and generally having a grand old time.

And in all those dreams, I have never once hit anything. Whenever my flying self gets near to a building, I merely veer to the side, or cleanly soar up over it. In the moment, this seems ilke the most natural experience in the world.

In contrast, the first time I tried Birdly I kept crashing into buildings. That just didn’t seem right to me, since I know quite well that I can do better, from all the extensive flight time I have put in while in slumberland.

I guess that’s one of my motivations in wanting to use the Tactonic floor mats to allow us to soar through the air in our VR research at NYU. Why should my magical flying self ever need to crash into a building? It’s one of those situations where I know it can be done better, because I’ve seen it done better.

Even if I was sound asleep at the time. :-)

In another 50 years

November 23rd, 2015

In a recent post I remarked that I had been thinking about historical events every 50 years. Some students at Trinity college had observed that Alice’s Adventures in Wonderland was published exactly 150 years ago, and that Einstein’s theory of general relativity was first presented 100 years ago.

So I got to wondering what might have happened 50 years ago which similarly challenged our notions of time and space. I concluded that it was “Moore’s Law” — the prediction 50 years ago by Gordon Moore that the cost of computation will continue to decrease exponentially with time. This remarkable and prophetic statement has held true for the last half century, leading to profound and astonishing changes in our world.

But then the other day somebody asked me what might be happening right now — in the year 2015 — that people will one day look back upon as the next step in this legacy. Will it be virtual reality, neural implants, or something else that is so radical and paradigm changing that we are as yet unable to even recognize it?

I am open to suggestions.

Body relative

November 22nd, 2015

In our Future Reality research at NYU, we’ve been trying all sorts of things. One set of experiments has involved embodied conversations between people whereby we create the illusion that participants vary greatly in scale.

For example, in one experiment I found myself shrunk down to about 6 inches tall. I seemed to be standing on a table, so that the room around me seemed vast and cavernous, the walls and ceiling impossibly far away.

People who were of “normal” size appeared as huge, giant creatures who loomed against the sky, peering down at me as from a great height. The effect was quite compelling.

One question that comes up when you create such experiences is what, exactly, should the subjective experience be for each participant. After some experimentation, we came to an interesting conclusion.

If you find yourself virtually transformed into a six inch tall version of yourself standing on a table, then the most effective subjective experience does not create the impression that you have shrunk down in size. Rather, it creates the impression that the world itself has blown up to vast dimensions, and that you have not changed at all.

For example, the audio in the room will feel the most natural if it seems to be a vast chamber, hundreds of feet across. When the “normal” sized people speak, you should hear booming and echoing voices, as though the sound is traveling across a great distance, and bouncing off the walls of a giant space.

In effect, you are experiencing a different physics. From your perspective, sound still travels at 1100 feet per second. But because the entire world has become impossibly large, the time it takes for that sound to travel from one end of the room to that other is creating a noticable echo.

Meanwhile, the “normal” sized people should hear your voice as tinny and lacking bass, because you are a tiny being whose body contains no large resonant cavities. You will still sound normal to yourself, but to them you will sound like a small creature.

The take-away lesson is this: As we try to transform ourselves to travel through virtual worlds, our subjective experience will always be body relative. If we become tiny or huge in these alternate worlds, the most natural interface will be one that seems to keep us the same, while changing the entire universe around us.

My day

November 21st, 2015

I spent most of the day finishing up a 3D modeling software package that I started implementing a few days ago. Today was the day I added support for textures of various kinds.

To test everything out, I created a little “rogue’s gallery” of objects. Over the course of the day I kept going back to these objects, to make sure they still looked like they should. So I spent much of the day watching these shapes swirling around on my computer screen.

I didn’t realize how embedded this collection of shapes was in my mind until this evening, when I went to see some experimental theater. At the very beginning of the play, the performers handed out paper and markers, and asked us to draw an image that represented our day.

I ended up sketching the collection of animated objects that I’d been staring at for hours. Which is when I realized how much this particular collection of shapes had seeped into my brain.

So here, for your amusement, is my day — not a mere sketch on paper, but the computer graphic original, in all of its hopelessly nerdy glory:

In case you were wondering, the complete text on the bagel-like thing on the lower right says “This is not a bagel.” It is, I suppose, a sort of New Yorker’s version of Magritte.

Male bonding

November 20th, 2015

I went out for beers this evening with some fellow NYU profs, who all happened to be guys. Over the course of several hours our discussion touched upon a vast number of topics. Which is sort of the point of going out for beers.

Inevitably the conversation turned to ISIS. Whereupon one of my colleagues, a highly esteemed academic, asked whether any among us knew that ISIS has a magazine. Most people around the table expressed disbelief.

I guess we were all trying to wrap our heads around the idea that a nihilistic terrorist organization, bent on world distruction, would take time out to publish a periodical. “No, really,” he insited, “ISIS has its own magazine. It comes out regularly.”

He went on to ask whether any of us had ever read it. The topic of conversation seemed so absurd by this point that it didn’t even occur to me to give a serious answer.

“Sure, of course I read their magazine,” I offered, “but only for the short stories.”

I guess it was one of those jokes that only makes sense when you are out drinking with guys.

Still crazy

November 19th, 2015

I met my old lover on the street last night / She seemed so glad to see me, I just smiled / And we talked about some old times, and we drank ourselves some beers / Still crazy after all these years — Paul Simon

It’s odd to suddenly find myself oddly in the center of a technology movement that has gone viral. In my research I consciously try to work on things that other people won’t be thinking about for at least another ten years.

But the recent commercial focus on VR has sort of snuck up on me. When I was giving talks in 2010 about future VR wearables that track their user’s position using sensor fusion between inertial sensors and tiny cameras, I felt — perhaps a bit too smugly — that this was all far enough in the future that at least half my audience would just think I was crazy.

Now it turns out that extremely large companies are pouring ridiculous amounts of money into just those things. My “crazy vision of the future” talks of five years ago have become the business plans of today.

Sure, it’s nice to see your predictions come true, but it can also be a bit disconcerting. If you are doing research, you should not be spending your time in the present — you should be spending your time in the future.

Fortunately, I still have some ideas that everybody thinks are crazy. So at least I am doing something right.

Every six months or so

November 18th, 2015

I’ve started to notice that my research undergoes a paradigm shift about once every six months or so. I’m not sure why this happens at six month intervals, but I suspect it’s connected to the energy of a new semester.

In the summer of 2013 I discovered HTML5 and Javascript. I dropped Java like a rock, and I’ve never looked back.

In the winter of 2013-2014 I started the Chalktalk “magic drawing” project. It has pretty much been my personal mainstay ever since.

In fall of 2014 our research group at NYU started to explore untethered social VR, using only lightweight headsets. We have come to refer to this as “future Reality prototyping”. It was clearly the right way to go.

In spring of 2014-2015 my research group at NYU began to focus on artistically driven group projects shown in public. It quickly became obvious that this was really the way to go.

This fall — October and November 2015 — I realized that the best way to move things forward is to just give our technology freely away — both our technology and our know-how. This is turning out to be a spectacularly successful plan. We already have multiple partners from around the rorld for our various research project.

I wonder what the next paradigm shift will be.

L’éléphant dans la pièce

November 17th, 2015

I have been avoiding talking about l’éléphant dans la pièce for the last few days, because we have been so inundated by tragic accounts and assorted analyses of the recent horrific terrorist attacks. But there is one thought I would like to share.

When children are born, they are still capable of becoming anything. A child may grow up to be an artist, doctor, inventor, musician, scientist or architect, among many other possibilities. Every child born into this world is pure potential.

But children need to feel that they belong somewhere. If they are told that they are without worth or that their life has no meaning, they will seek out someplace where they are told otherwise.

Many children around the world have thus been led to dark places, from a White Supremicist movement in Wisconsin to an ISIS cell in Syria. The vulnerability that leads to recruitment into such hateful organizations begins with neglect, abandonment and prejudice.

Whatever our short-term response to terrorist threats, any meaningful longer term plan must include changing the conditions that turn young people into lost souls, easy pickings for recruitment to extremist ideologies.

I hope that amongst the billions of dollars that will undoubtedly be spent on new forms of warfare, some resources will be set aside for battling the conditions of neglect and economic abandonment of youth that make it all too easy for terrorist movements to thrive and grow.

Common interests

November 16th, 2015

Today Patrick Hebron gave a wonderful demonstration at NYU of some work he is doing that ties together object recognition and word recognition. He is using a very large database of 3D shapes of ordinary objects — everything from chairs to computers to airplanes to toilet bowls — and tagging each shape with sets of words.

One of the things he can do with his system is use words to explore the shapes associated with those words. He can even to use language to create novel in-between shapes. For example, I learned from his demo that if you make a shape part way between a chair and a toilet bowl, you get something that looks very much like a fancy modernist designer chair. Although I’m not sure exactly what that means.

Patrick then pointed out that his general approach could be generalized to multiple languages. A shape described in one language, perhaps English, could be associated with the same shape described in another language, perhaps Arabic.

At that point one person in the room said that this could be really interesting to the NSA. Another person said out that it could be really interesting to Noam Chomsky.

“How nice,” I said, “that the NSA and Noam Chomsky can have common interests.”