Ken’s Blog

Clouds are the most ordinary thing in the world. We see them every day, floating above us in the sky. Clouds are not something most people think about a lot.

Yet suppose you could only see a good cloudy day once in your life. Those soaring towers of light, the swirling patterns, the intricate feathers of alabaster writ large against the twilit sky.

You would be amazed and overwhelmed. The sight of a good cloud-filled sky is a thing of breathtaking beauty, if only we could remember to truly look.

Wouldn’t it be wonderful if we didn’t take our greatest treasures for granted? Think how much richer our lives would be.

Today I sat in on an on-line theater rehearsal process. The fact that this was happening was, of course, a result of the pandemic.

After all, a lot of actors are currently out of work. Rather than sitting on their hands, they are organizing to create on-line theatrical presentations.

All of the rehearsals, as well as the performance, are done over Zoom. The goal is not to replicate traditional theater, but rather to bring the concepts and skills of traditional theater to a new medium.

I don’t know where all of this is going, but it is exciting to see it in action. Just as cinema is related to theater, yet different, we may be seeing the emergence of a new art form.

Of course this is not entirely new. Yet there is a particular urgency now, and the tools are finally good enough to allow talented directors and actors to put on a production without the need for special equipment or high end technical expertise.

It feels good to be present at the birth of something like this. Looking back, people in future years may look upon 2020 as the year when on-line theater truly emerged as a distinct art form in its own right.

Some days you have things you must ponder
And there’s not really much you can say
And times when your mind will just wander
But today it was just a good day

There are days when the world is so rotten
That you want it to all go away
I am sure that there’s more I’ve forgotten
But today it was just a good day

As I have been going through my old Java applets, I am rediscovering things I had long forgotten. Mainly I am remembering how I would lift ideas from some of my projects and apply them to later projects.

For example, at one point I created an interactive animated character in the form of a desk lamp. I was basically doing a riff on Luxo Jr., but as an interactive Web character.

The very next thing I made was an interactive fish character with facial animation. I realize, looking back on it now, that to animate the fish swimming around, I just borrowed the movement of my Luxo lamp character.

I simply made the lamp invisible, and placed the fish where the lamp head would be. The lamp is still there moving around, except you can’t actually see it.

This simple trick created nice organic looking flowing movements for the fish character, because you should always to animate a character in a curved path, never in a straight line (straight line movements look incredibly fake). The trick ended up working really well.

The funny thing is that I had completely forgotten that I had done this, until I went back and revisited the Java applets. I wonder how many other things I’ve forgotten about my own work.

I suspect this sort of thing happens a lot. How many things about your own work have you forgotten?

From 1996 to 2013 I implemented lots and lots of 3D animated stuff in the Java programming language, which I made freely available to the world as Java applets. I implemented my own modeler and real-time software renderer and everything. It was really cool.

Unfortunately Oracle ended all that in mid-2013 when they changed things so that Java applets would be much more difficult for people to access. Sigh.

So starting in July 2013 I retooled and started implementing things in Javascript. That was wonderful because WebGL gave me the opportunity to implement my own hardware shaders, so I could do much more interesting things with real-time rendering (and also teach my students how to implement their own hardware shaders).

Unfortunately that transition left a very large back-catalog of my Java applets stranded. For most of them, I didn’t have the time or inclination to put in the many hours it would take to properly translate from one computer language to a very different one, although I did translate a few of them.

So now it’s 2020, and our lab is starting to transition over to creating VR worlds with the C++ programming language, because that’s the computer language you use with the Unreal game engine. And the Unreal game engine is an awesome software platform for making stuff in VR.

So I now find myself skipping entirely over the seven years of Javascript, and handing some of my old Java applets to grad students to translate into C++, so we can use them in VR. It’s an odd sort of leapfrogging, but I guess that’s the way things go.

Several years ago I created this image for a presentation about our research. All of the figures are from Chalktalk.

I rediscovered this image today, and thought it was cool. In any case, looking at it makes me feel happy.

Last night I did what many people have been doing of late. I participated in a social gathering on Zoom.

Everybody brought their intoxicating drink of choice, we had a far ranging discussion, and some creative background images were deployed to dramatic or comic effect. It was a fun time.

Yet it didn’t feel the same as it does when you go over to somebody’s house or hang out in a bar or coffee shop. There was still that feeling of looking at somebody through a window, as though you are visiting somebody in prison, and you’re all trying to pretend that everything is ok.

On a scale from zero to fully present, I would rate it somewhere between a four and a five, where fully present is ten. Which leads to an interesting question.

As technology advances, we will get progressively better at creating a sense of presence for conversations between people who are not in the same place. How will we measure our progress?

Can we develop a way to quantify presence? Will we have anything to go on, other than individual subjective intuition, to know whether we are making progress?

Perhaps we can eventually create a kind of “Turing test for presence.” That seems like a very good goal to aim for, and now seems like a good time to start.

I wrote yesterday about places to eat
Using VR in various ways
But if folks mostly want a location to meet
The future is VR cafés!

My post yesterday talked about virtual restaurants. The basic idea was to use the forthcoming technology of good mixed reality to virtually join together diners. All customers feel as though they are dining together in a restaurant, although each couple or family group is actually sitting at home.

This could be combined with complementary technologies, such as robotic equipment that prepares the food, and other robotic equipment that serves it. In this way, everybody in the virtual restaurant could partake of the same cuisine.

Delivery services from Whole Foods, Sprouts, and their equivalents could be packaged in such a way that food preparation at home can be entirely automated. So, for example, making a reservation to “dine out” on a Friday evening automatically prompts a set of orders to your food delivery provider of choice. By the time Friday arrives, all needed ingredients are in your house and ready to go.

Meanwhile, thanks to mixed reality, your robot “waiter” can have an avatar that suggests a particular cuisine: French, Italian, mediterranean. You can even get a Hobbit cuisine from the Shire as your virtual waitperson, if you are dining out for second breakfasts or elevensies.

If you are willing to pay a little extra, you can get a live human being to virtually wait on your table. In reality, this person may be operating from their own home, perhaps in Iowa or Kentucky. Technically that would be a form of remote computer-mediated puppetry.

A whole industry could be built around such remote personalized services. But that’s an entire topic unto itself.

As I mentioned yesterday, going to a restaurant can be a very compelling experience. But what if we can’t go to restaurants? We seem to be facing just such a situation now, during the coronavirus pandemic.

Perhaps we can take some elements of restaurants and bring them into the home. This may become much more feasible within the next several years.

Imagine that you and your loved one are dining at home. Perhaps you want to liven things up a bit. You both take out your trusty mixed reality glasses and voila! you are transported to a virtual restaurant.

All around you, other diners are eating, laughing, enjoying each others’ company. The mood is infectious.

Suddenly you feel more alive, your food tastes better, and everything is right in the world. That is the experience of the virtual restaurant.

It may seem different, yet one the most important things about restaurants has not changed: You and the other diners are all paying good money to provide entertainment for one another.