Look at the bright side

Now that the next presidential election is gradually sneaking up on us, a memory from eight years ago has resurfaced. It was in November 2016, the morning that we all found out that you-know-who won the election.

My fellow NYU professors and I were standing around, feeling numb. I said to the group “I think this is going to be really bad for the country.”

“But look at the bright side,” one of my colleagues said. “It’s going to be great for comedy.”

Vernor Vinge

I was very sad to read about the passing of Vernor Vinge. He was truly a person of rare and great vision.

I know that in the world of ideas he is most known for his thoughtful essays about “the singularity” — the hypothetical moment in the future when general artificial intelligence will surpass human intelligence and will then continue to accelerate at an exponential rate.

But the work of his that had the most profound influence on me was Rainbows End, a novel about a future where ubiquitous mixed reality has reached a point of maturity. Reading that novel changed the course of my research.

That was when I stopped thinking of interactive computer graphics as something on a screen, and started to see it as something that will eventually simply be part of the world all around us.

Doing research

In my research at NYU I need to do a lot of math and to come up with all sorts of algorithms as I create user interfaces for extended reality. But I have come to realize that none of that is what I am actually researching.

My research is, as it has always been, about people. It’s the same as it was back in the day, when I first came up with what we now call shader languages.

It wasn’t so much about how to do something, but rather it was about what ends up working for human aesthetics. It’s not about the computer — it’s about us.

I am reminded of a visit many years ago to the NY Museum of Modern Art, when I went to see a career retrospective of Jackson Pollock. As his early work showed, he started the same way Picasso did, doing highly realistic and impressively faithful life drawings.

Then, through the years, he continued to experiment with gradually more abstract forms, over time building an entire visual language, trying different things to find out what worked and what didn’t. When I saw, compressed into a single exhibition, the years-long progression of his work, I realized that Jackson Pollock was up to the same thing that I was.

He was doing research.

Stinky tofu

Twenty years ago I visited Taipei. I was there because I’d been invited to give a talk at a technical conference.

While I was in Taipei I visited the national museum, which was breathtaking, and a former student of mine also took me on a tour of street vendors. On the way there, he told me about something called “stinky tofu”, which was considered a delicacy by his countrymen.

When we got to the place where all the street vendors were, I was overwhelmed by the powerful aroma of the stinky tofu. In particular, it smelled to me as though a thousand people had removed their shoes at the same moment on a particularly hot summer day.

As politely as I could, I asked if we could leave the area. Fortunately, my former student was very understanding.

I know for sure that this was twenty years ago, because exactly twenty years ago today, on March 19, 2004, during my visit to Taipei, the President of Taiwan, Chen Shui-bian, was shot — just one day before the presidential election. There was a great uproar, with huge crowds filling the streets, and thousands of people watching the continual TV news updates that seemed to be blaring from every storefront.

I remember the events of that day very well. But mostly I remember the stinky tofu.

On accident

It’s amazing to me how quickly natural language evolves. You can often spot changes that happened in your own lifetime.

When I was a kid, something either happened on purpose or else by accident. But now I see that things have changed.

Many people younger than me — adults — now say that something has either happened on purpose or on accident. The new way of saying it makes logical sense, but it’s still a little startling to realize that the version of English that I grew up with is already slightly out of date.

I wonder how many other changes there have been, even in my own lifetime, to our evolving language.

Defensive music listening

I thought would be a great day to get work done at the lab, because nobody is ever at the lab on weekends. But it turned out, sadly, that today was an exception.

Just when I was getting somewhere with my programming, somebody who had also come in on the weekend decided to eat his lunch to the accompaniment of loud music. Unfortunately, it was not music that I like, but you can’t really argue with somebody about their musical tastes.

So after a few minutes, I started playing Schoenberg’s piano music — in particular, Drei Klavierst├╝cke. Schoenberg’s piano music is pretty much the only music I can play while working, because its soothingly jagged rhythms and atonal chromatic shifts don’t pull me away from whatever I am focusing on.

So that guy had his musical zone and I had mine, and for a while we co-existed peaceably. And at some point I realized that I was engaging in an act of defensive music listening.

Super power

Many years ago I went scuba diving with a friend in the Florida Keys. The way it works there is that a guy takes you out on a little boat to where the reefs are, and then you go from the boat down into the water.

The boat was just big enough to take four passengers. My friend and I noticed that the other two passengers were deaf, and were speaking to each other in sign language, but we were too wrapped up in our own adventure to pay much attention.

Until, that is, we all started diving. My friend and I, who had been having a pleasant conversation on the boat, suddenly found ourselves mute. All we could do was point at the pretty fish and nod at one another.

The other two people, on the other hand, never stopped their conversation. While swimming among the fish and underwater coral reefs, they just kept chatting away.

I suddenly realized that in the water, they had a super power.