Ken's Blog

In science fiction movies, a repeating trope is the virtual window. You’ve probably seen this: In some distant future, people living in a tiny apartment have a magnificent view, which turns out to be completely artificial.

Essentially, their window is a hologram, which creates the illusion that the apartment’s inhabitants are surrounded by nature. When in fact they are probably living five levels underground in an overcrowded future megalopolis.

I have long wondered when the technology will exist to implement such a window for real. A simple engineering analysis, which I conduct every few years, tells me that we are still far from being to make that virtual window a reality.

But such a thing is indeed theoretically possible, given the right future breakthroughs in micro-optics. I wonder if or when we will ever get there.

This morning a student asked me why, when they create something with the help of AI tools, it doesn’t quite feel as though the result is entirely theirs. I responded like this:

In answer to your question, I think the reason that the project doesn’t feel completely “yours” is that it is not completely yours. To a large extent, when you work with an AI, you are working with the many other people whose expertise has gone into training the AI. You never meet these people, and you don’t know who they are, but the output of the AI reflects their work in addition to your own work. So in a sense you are actually engaged in a collaboration with a very large number of anonymous collaborators. It’s definitely not the same as the experience of building something all on your own.

I’m a little surprised that everyone doesn’t already know this.

In the mixed reality research at our lab at NYU we are starting to work with educational robot arms. One that we really like is the SO101. The robot provides continuous feedback about its position, is easy to work with, and is relatively inexpensive.

Another nice thing about the SO101 is that the people who make it provide all of the information that you need to 3D print your own parts, which also means that with a little bit of hacking, you can create your own digital double of the robot. A digital double lets us do research with the robot arm even if we don’t happen to have a physical robot arm handy.

So one of the first things I did was write a program to create an interactive visualization of the SO101 robot arm that runs in a Web browser. I managed to make it move around kind of like a dinosaur, and that looks very cool, so I thought I would share it HERE.

I gave an assignment this week to the students in my VR class that involved having them get an A.I. to create virtual worlds. So they are essentially being tasked with vibe coding.

What then is my relationship to the A.I.? I am specifying something to be done by A.I., but I am not actually communicating with any A.I. I am only communicating with human beings.

So you could say that if my students are vibe coding, then I am vibe vibing. I wonder whether that will become a thing.

Today I was hanging out with friends, and somebody asked if we liked oysters. I, being the vegetarian in the group, interpreted that differently from everyone else.

I said “I think we all like oysters, but I might like them in different sense.”

“What do you mean?” my friend asked.

“I mean I like for them to be happy.”

Today I showed Bruce Branit’s iconic short film World Builder to my VR class at NYU. Considering it was made nearly two decades ago, it is astonishing how prescient it is.

I think that’s because the world we are gradually entering, where A.I. does seemingly magical things for us, is actually an approximation of a world that had already existed. For millenia, designers and master have relied on assistants to help them realize their creations.

A.I. at its best will approximate that kind of relationship. The end goal of A.I. is to allow each of us to become a master creator, limited only by our own unique vision. That is the future we seen in Bruce Branit’s landmark film.

Which is why it resonates so profoundly. Also, it’s a really good film.

The offices of the animators at Pixar Animation Studios.

Those who know, know.

Today I am on a long flight. I am flying economy class.

If course it is uncomfortable.The seats are too small, there isn’t much legroom, and nobody is really happy to be here.

Perfect conditions, in fact, for getting programming done. The more you want to escape your immediate physical environment, the more motivated you are to escape into code.

I wonder, if you tally the productivity of programmers, and map it to where they happen to be, whether you would find a spike during long flights in the cheap seats.

Maybe, in their own way, the airlines are doing a service for our nation’s knowledge economy.

The act of livestreaming an execution, which the U.S. Pentagon has ordered for Dec 3, is of course beneath contempt. It seems like a deliberate attempt to position our government as a pack of savages, by publicly staging a pointedly disgusting and frankly vomit-inducing display of extreme violence porn.

But why would government officials debase themselves and our nation in this way? After all, this is just the sort of monstrosity that democracies find deeply offensive when it is done in such places as Iran and North Korea.

Maybe that’s the point. By now they realize that they have no chance of holding onto control of Congress in the midterms, given how thoroughly they have screwed up on every front. So their only remaining play is to flex their authoritarian muscles in preparation for what comes next.

The worry is that on November 4 they will declare the election that they just lost to have been rigged, and then begin the process of instituting martial law. By committing outrageous acts of public savagery now, they can signal that they will not let the normal rules of decorum get in their way.

It has often been said that is better to show rather than just to tell. In that spirit, sometimes I find it useful to put together little interactive visualizations on the Web to illustrate an idea.

Just this week I was trying to explain to some colleagues an idea I had about eyewear made from mirrors that lets you see to the left with your left eye, and to the right with your right eye. The question I was trying to address was how big your “visual aperture” can be. In other words, when is it like looking through a little peephole, and when is it like looking through a nice big pair of glasses?

Not surprisingly, the nearer you place the mirrors to your eyes (the “eye relief” of the device), the greater the visual aperture. But at some point the eye relief can become too small — if the mirrors get too near to your eyes it will make you uncomfortable.

The smallest comfortable eye relief is probably about 12mm (about half an inch). I created this little visualization to show how eye relief relates to visual aperture.