Ken's Blog

Many years ago I organized a morning panel at the SIGGRAPH computer graphics conference. The topic of my panel was what software tools people preferred to used to create their computer graphics.

My three invited panelists were all great pioneers in the field: Jim Blinn, Craig Reynolds and Tom Duff. As it happened, each at the time was strongly associated with a particular set of software tools, and they had all been using their very different toolsets to create amazing work.

Back then Jim did everything in FORTRAN — one of the oldest of computer languages. Craig worked in LISP, which is very different from FORTRAN in every way. Tom was known as one of the great creators of UNIX tools.

At the breakfast before the panel, I asked each of them what they thought was the best programming language. Not surprisingly, Jim said that you really don’t need anything other than FORTRAN. Craig said that LISP was the most powerful language for computer graphics.

I was expecting Tom to speak up for UNIX tools, so I wasn’t prepared for what came next. “A graduate student,” Tom replied, “is the highest level programming language.”

Shortly after that I quit my job in industry to become a professor at NYU, and I never looked back. To this day, I still don’t know whether that life decision was influenced by what Tom Duff told me over breakfast.

There is a perpetual problem one faces when designing interfaces for new communication technologies. This is true whether the technology is a smart phone, a pair of XR glasses, or an AI assistant.

The problem is that you generally know that the technology is coming well before it actually arrives. Which means that you want to figure out how to use it, but you cannot — because it does not yet exist.

That’s one reason I’m a big believer in emulation. Make a fake version of the thing, and play with that.

Cheat if you have to. In this particular case, cheating is a good thing.

Make something that looks and acts like smart phone, but with a hidden wire running to a powerful computer. Create fake XR glasses that act like the real thing, but only in your lab. Have a person behind the scenes pretend to be an AI assistant.

The future is indeed coming, but it’s not here yet. When you are trying to design user interfaces for the future, and that future has not yet arrived, emulation is your best friend.

Have you ever noticed that the U.S. Army uses somewhat odd words to define how individuals are positioned in its military hierarchy? Many of them are words that have entirely different meanings in other contexts.

For example, in civilian life if you can have a General idea that something is wrong, then you can have a Private discussion about it. Especially if it has become a Major issue.

OK, I admit that these are rank puns. Possibly deserving of corporal punishment.

I am finding that this new way of using Google — asking questions of Gemini — is much more fruitful than the old style of Google search. So far it has not really been a question of what Gemini is capable of, but rather a question of my learning how to ask the right questions.

I am reminded very much of my first experiences using Google in 1998. There was a learning curve there as well.

Eventually I acquired the skill needed to chain together a good set of keywords to properly narrow down my searches. Learning to work with Gemini is something like that, but on steroids.

We’ve certainly come a long way in 28 years. I wonder what will the process will be like 28 years later.

I am glad that in the end everyone in New York City came together in a respectful way to honor and pay their respects — both to the victims 25 years ago, and to the courageous firefighters who tried to help them.

Even Rudy Giuliani managed to move past his Islamophobic hate-mongering and share a respectful moment with our Mayor. For him that was a stretch goal, and I have to give him credit for putting aside the poisonous and deeply un-American MAGA playbook long enough to remember what actual patriotism looks like.

Many years ago I paid a visit to Apple Research Labs, back when they had a computer graphics research group. One of the people in that group was the late great Lance Williams.

At some point he and I both watched in bemusement as several people in the group got into a heated discussion about some arcane technical point. I could tell that Lance thought the whole topic was a waste of time, but he was too polite to say so.

Finally one of the people, exasperated by all the arguing over little details that probably didn’t really matter, said in frustration “Which came first, the chicken or the egg?”

Finally Lance spoke up. In a calm measured voice he said “It’s been well established that the egg came long before the chicken.”

That seemed to be the one thing everyone could agree about.

Tomorrow I am facing a big deadline. I hate deadlines. In fact, there are a few things I hate more in this world than deadlines.

But I love how much I get done when I am pushed to do so by a deadline. Some of my best work was only accomplished because, you guessed it, deadlines.

I guess you could call it a love-hate relationship.

I recently bought a wonderful little cup from a local craftsman in Seoul Korea. Its texture reminded me of my noise function. I keep it at the lab and use it for my daily espresso. Today I brought an avocado to the lab, and was struck by the similarity of the texture between cup and avocado. I took a photo, which you can see here.

It seems that noise is everywhere. Maybe it’s trying to tell us something.

Every once in a while, somebody says something in public that absolutely nails an important truth — one that actually matters. It doesn’t happen very often, but when it does, it is breathtaking.

This video by Ezra Klein is a beautiful example of that. Many of us had already understood what he is saying to be true, but it required a person of his talents to articulate it so perfectly.

Maybe there is hope for us after all.

I am working with my students on a research proposal to the National Science Foundation. The general idea of what we are up to is a sort of mixed-reality equivalent of Google Docs. Except instead of the shared “document” being just on everyone’s screen, it can also be in your physical space.

All of this would be using technology that doesn’t quite exist yet in the form that we’d like. But that’s why it’s research. 🙂

Google Docs is indeed a powerful tool for collaboration, and I don’t think we could be collaborating as well if we didn’t have it. And yet here we are, proposing to create something which may end up supplanting it in some ways.

I think that’s the way it is with tools. You use the best tools you can get your hands on, and sometimes you use them to make even better tools.

In a way you are building a kind of ladder of tools. You may eventually jump to those new tools, and abandon the old ones. But that is not a sign that those older tools have failed. It’s a sign that they have succeeded.