Doing research

In my research at NYU I need to do a lot of math and to come up with all sorts of algorithms as I create user interfaces for extended reality. But I have come to realize that none of that is what I am actually researching.

My research is, as it has always been, about people. It’s the same as it was back in the day, when I first came up with what we now call shader languages.

It wasn’t so much about how to do something, but rather it was about what ends up working for human aesthetics. It’s not about the computer — it’s about us.

I am reminded of a visit many years ago to the NY Museum of Modern Art, when I went to see a career retrospective of Jackson Pollock. As his early work showed, he started the same way Picasso did, doing highly realistic and impressively faithful life drawings.

Then, through the years, he continued to experiment with gradually more abstract forms, over time building an entire visual language, trying different things to find out what worked and what didn’t. When I saw, compressed into a single exhibition, the years-long progression of his work, I realized that Jackson Pollock was up to the same thing that I was.

He was doing research.

Stinky tofu

Twenty years ago I visited Taipei. I was there because I’d been invited to give a talk at a technical conference.

While I was in Taipei I visited the national museum, which was breathtaking, and a former student of mine also took me on a tour of street vendors. On the way there, he told me about something called “stinky tofu”, which was considered a delicacy by his countrymen.

When we got to the place where all the street vendors were, I was overwhelmed by the powerful aroma of the stinky tofu. In particular, it smelled to me as though a thousand people had removed their shoes at the same moment on a particularly hot summer day.

As politely as I could, I asked if we could leave the area. Fortunately, my former student was very understanding.

I know for sure that this was twenty years ago, because exactly twenty years ago today, on March 19, 2004, during my visit to Taipei, the President of Taiwan, Chen Shui-bian, was shot — just one day before the presidential election. There was a great uproar, with huge crowds filling the streets, and thousands of people watching the continual TV news updates that seemed to be blaring from every storefront.

I remember the events of that day very well. But mostly I remember the stinky tofu.

On accident

It’s amazing to me how quickly natural language evolves. You can often spot changes that happened in your own lifetime.

When I was a kid, something either happened on purpose or else by accident. But now I see that things have changed.

Many people younger than me — adults — now say that something has either happened on purpose or on accident. The new way of saying it makes logical sense, but it’s still a little startling to realize that the version of English that I grew up with is already slightly out of date.

I wonder how many other changes there have been, even in my own lifetime, to our evolving language.

Defensive music listening

I thought would be a great day to get work done at the lab, because nobody is ever at the lab on weekends. But it turned out, sadly, that today was an exception.

Just when I was getting somewhere with my programming, somebody who had also come in on the weekend decided to eat his lunch to the accompaniment of loud music. Unfortunately, it was not music that I like, but you can’t really argue with somebody about their musical tastes.

So after a few minutes, I started playing Schoenberg’s piano music — in particular, Drei Klavierstücke. Schoenberg’s piano music is pretty much the only music I can play while working, because its soothingly jagged rhythms and atonal chromatic shifts don’t pull me away from whatever I am focusing on.

So that guy had his musical zone and I had mine, and for a while we co-existed peaceably. And at some point I realized that I was engaging in an act of defensive music listening.

Super power

Many years ago I went scuba diving with a friend in the Florida Keys. The way it works there is that a guy takes you out on a little boat to where the reefs are, and then you go from the boat down into the water.

The boat was just big enough to take four passengers. My friend and I noticed that the other two passengers were deaf, and were speaking to each other in sign language, but we were too wrapped up in our own adventure to pay much attention.

Until, that is, we all started diving. My friend and I, who had been having a pleasant conversation on the boat, suddenly found ourselves mute. All we could do was point at the pretty fish and nod at one another.

The other two people, on the other hand, never stopped their conversation. While swimming among the fish and underwater coral reefs, they just kept chatting away.

I suddenly realized that in the water, they had a super power.

AI you forever

Suppose the time eventually came (and I acknowledge that this is still a long time away) when an AI is able to faithfully mimic your every response. In other words, your personal AI passes the Turing test with flying colors.

And then, in this far off future, you opt to designate that AI as your proxy. Whenever you are unavailable, you can trust your AI to step in and give the response that you would have given.

In time, people just take this arrangement for granted. As Abraham Lincoln once said “A difference that makes no difference is no difference.”

But then one day you die, and in your will you have designated your AI as your legal proxy in perpetuum. What happens then?

Do you legally live on forever? Do the courts step in and declare that you have overstepped?

By then I suspect that the courts may be filled with the AI proxies of jurists long deceased — or perhaps AIs that consist of well crafted amalgams of the finest legal minds from various historical eras. So those future courts might very well be biased in your favor.

Something to think about.

Eric Carmen

I was devastated to read that Eric Carmen has passed away. The man was a true musical genius.

Today I walked around NYC listening to Go All the Way on repeat, a song he penned as frontman for The Raspberries. It may very well possess the single best hook of any song in the modern pop music era.

And I say this as a devoted fan of the Beatles, ELO and other brilliant groups from that era. As Jeff Lynne might say, I can’t get it out of my head.