Ken’s Blog

The questions I’ve been asking the last two days were intended to get at something quite specific: The evolving relationship between advancing media technology and our experience of reality.

For a very long time, we didn’t need to think too much about the choice between “media” and “reality”. They were clearly different from each other in so many ways. Reading a novel is very different from making love to one’s partner. Each of these experiences brings something to your life that the other clearly does not.

For the first half century of the computer age, interactive media was, on a visceral level, no match for reality itself. The touch or caress of another human being, the deep emotions shared when two people look into each others’ eyes, there were the province of reality.

Sure, we would play our video games, watch our high definition movies, immerse ourselves in one cyber-enabled fantasy or another, but at the end of the day, we understood that it was all make believe. The real human being lying right next to us has the power to touch our souls that goes beyond the reach of mere technology.

But what if we are nearing a cusp? What if the intensely vivid quality once reserved for physical reality begins to seep into virtual shared experience? I’m not saying that this will happen any time soon, but I suspect that it may well happen in our life times.

And when it does, we may need to rethink a lot of our assumptions.

Suppose we start from the point Sharon made yesterday in her comment, and iterate from there:

Suppose you could wander freely throughout the world, with no restrictions on you at all, in the company of just one companion, whom you really like. The two of you would be totally free to explore the world, but without other people.

In the other extreme, suppose the two of you spent your entire life in a room together. Your interactions with other people would be virtual, pure exchanges of information. You both could have a great variety of friends, unlimited social connections, unbounded intellectual stimulation. But all from within that room.

If you had to choose one of these scenarios, which would you choose?

As I wrote yesterday’s post about walls, I realized that there are some complicated questions here, centering on the nature of freedom.

Suppose you could wander freely throughout the world, with no restrictions on you at all, but you were completely alone, with nobody else there. You would be totally free. But what would that freedom be worth?

In the other extreme, suppose you spent your entire life in a room. Your interactions with other people would be virtual, pure exchanges of information. You could have a great variety of friends, unlimited social connections, unbounded intellectual stimulation. But all from within that room.

Which would you choose?

“Something there is that doesn’t love a wall.”
      — Robert Frost

Time spent in shared virtual reality has changed my view of walls.

Until now, I always thought of a physical wall as something that creates limits. You walk up to a wall, and then you can go no further. Specifically, a wall prevents you from getting to what is on the other side of the wall.

If you are wearing a virtual reality headset and you walk up to a physical wall, it is still true that you cannot go through the wall. But the meaning of the wall is now different.

After all, why do you want to get to the other side of a wall? Because there is some place out there that you are interested in visiting, perhaps somebody you want to see or talk to, or some event you wish to attend.

But in VR, all of the meanings become scrambled. You can see people, talk to them, attend events, wherever you are. A wall — I mean a physical, immutable wall in the real world — becomes not a barrier, but merely a guidepost.

You walk up to that wall, you see it and feel it and touch it, and then you say “Ah, this is useful. I will use this to navigate, to help me orient myself on my way to where I want to go next. Maybe to visit a colleague. Or to drop in on a neighbor.”

After all, as Robert Frost once said, good fences make good neighbors. 😉

I’ve been trying to make some sort of excuse in my mind for Indiana. I don’t like to believe that people are just plain evil.

I have been listening hard to Governor Pence, parsing his words, trying to be fair, to see the other side, to make sense out of his statements that this law has just been blown out of proportion.

But hell, I come from a Jewish family. We’ve been here before, and this kind of “logic” is way too familiar.

Excuses are made, explanations are offered, people want you to see their side. They don’t want you to think they are being evil.

But in the end there is no excuse, and there is no religious exemption for discrimination.

This is just evil.

I was in a meeting with some fellow faculty today, in which we were trying to work out how best to evolve the curriculum at NYU to include what might be called “computational media”. Obviously this is a moving target. The computational media of ten years ago, of now, and of ten years from now are all quite different.

I think this is mainly because of Moore’s Law. Available compute power determines what sorts of forms of cyber-enabled expression will impact the culture in any given year. For example, desktop publishing really only started to become practical about thirty years ago. Similarly, wireless streaming of content (first songs, and then movies and TV), have only become practical in recent years. And we are just now hitting the wave of practical high quality consumer-level VR and AR.

But independent of all that, we needed to figure out what is the proper role of the University in all this. Trying to frame the issue in terms of a University’s fundamental purpose, I offered the following:

“We can divide the University’s mission into two parts. First, roughly speaking, we have research and teaching to understand the existing work of great dead people. And second, we have research and teaching to nurture the emerging work of future great dead people.”

As soon as the words had left my mouth, I realized, to my extreme embarrassment, that I had phrased that last part very awkwardly. Fortunately, my colleagues basically agreed with me.

That is, as soon as they could stop laughing.

Today I attended the celebration at NYU of Louis Nuremberg, the fourth NY recipient of the Abel prize in mathematics, a highly prestigious prize established in honor of the great group theorist Niels Henrik Abel.

The Abel prize is roughly the equivalent, in Mathematics, of the Nobel prize. In the thirteen years the prize has been in existence, four of the winners have been from our own NYU Courant Institute of Mathematical Sciences.

Which is amazing.

Because this was an important event, I thought it merited an original joke worthy of the occasion. So I casually buttonholed various colleagues, and said the following:

“You know, if it turns out that there are a prime number of people here, we could all join hands and form a circle. Then we’d be an Abelian group.”

Fortunately, this was just the right crowd for that sort of joke, and everybody thought it was very funny.

But I would definitely not recommend it for most parties.

It makes sense that there would be a rush to develop the most realistic, highest polygon count, most finely textured visions for virtual reality. After all, that’s the dream, right? The ability to create worlds, to remake reality itself, only perhaps better, richer, more full of wonder.

Yet as our little NYU Holodeck has started to become a reality, I find myself pulling strongly in a very different direction. I spent much of this weekend working on an animated stick figure. As I worked at it, I realized fairly early on that my inspiration was the xkcd webcomic, and its simple yet remarkably evocative characters.

It was a little over a year ago that I had the privilege of meeting Randall Munroe in person, having been a follower of xkcd for years. I showed him my software for making sketches come to life and turn into animated characters. At the time I felt a little like an aspiring songwriter playing a tune for Bob Dylan.

But he was very kind and helpful, and gave me some good suggestions. He helped me to realize that it’s not about making things more real, but about finding great expressiveness in a few strokes.

I think I’m channelling some of that energy now. I’m not so much interested in creating a physically accurate world, but rather a world filled with expressive characters, teeming with personalities and emotions and human ideas. Deliberately swinging away from realism — putting stick figures on the Holodeck — is one way to get closer to that.

Besides, it’s pretty cool to see a little stick figure man walking across a table. 🙂

At the recommendation of a colleague, I finally got around to reading Cory Doctorow’s 2003 sci-fi masterpiece Down and Out in the Magic Kingdom. I started reading it earlier today, and pretty much scarfed it down, start to finish, in a single gulp.

The most fun thing about this book, which takes place about 150 years in the future, is that all of the advanced technology — the cochlear implants, the built-in heads up displays, the casual use of finger and hand gestures as user interface, even the ability to upload our memories and download ourselves into new bodies — are not the focus.

Rather, these are just the starting point. They are simple givens, like automobiles or washing machines. The real focus of the book is on what people are like in such a reality, how they relate to each other — the world of friends, lovers, mentors and rivals, human concerns that are as old as time.

The same issues that tug at our hearts, the same mysterious bonds that tie us to each other, yet manifest themselves differently with every newly developed form of communication.

If only he hadn’t gotten the lyrics wrong to Rocky Raccoon. I’m still trying to figure out whether he did that on purpose.

Some day in the not too distant future, your conversations will leave a visible trail if you want them to. A discussion you had with a friend outside your apartment, that debate with a fellow student at school, your declaration of true love in a favorite restaurant.

You will be able to see any of these events as a visible trail in the air. Not only will your memories be there when you go to visit those places, but they will also be available in your VR travels, whenever you decide to put on your cyber-glasses and visit old haunts.

An episode of Black Mirror presented a dystopian version of this capability, but I remained unconvinced. The people in that story didn’t seem to understand the implications of the technology, and so as inhabitants of such a reality their behavior made no sense. The equivalent in our world would be talking to a reporter and then being surprised when your words show up in the newspaper.

Should the day come when we can all leave trails of our every conversation, I think people will be much more sophisticated about it. There will be laws against invading someone else’s memories, just as there are now laws against hacking into somebody’s computer.

To me the more interesting question is how we will handle our own trails. Will you choose to keep your memories in sight? Will you allow yourself to see only the happy ones? Or will you decide to stay away from some places all together, so you won’t have the burden of deciding?