Month: May 2020

From 1996 to 2013 I implemented lots and lots of 3D animated stuff in the Java programming language, which I made freely available to the world as Java applets. I implemented my own modeler and real-time software renderer and everything. It was really cool.

Unfortunately Oracle ended all that in mid-2013 when they changed things so that Java applets would be much more difficult for people to access. Sigh.

So starting in July 2013 I retooled and started implementing things in Javascript. That was wonderful because WebGL gave me the opportunity to implement my own hardware shaders, so I could do much more interesting things with real-time rendering (and also teach my students how to implement their own hardware shaders).

Unfortunately that transition left a very large back-catalog of my Java applets stranded. For most of them, I didn’t have the time or inclination to put in the many hours it would take to properly translate from one computer language to a very different one, although I did translate a few of them.

So now it’s 2020, and our lab is starting to transition over to creating VR worlds with the C++ programming language, because that’s the computer language you use with the Unreal game engine. And the Unreal game engine is an awesome software platform for making stuff in VR.

So I now find myself skipping entirely over the seven years of Javascript, and handing some of my old Java applets to grad students to translate into C++, so we can use them in VR. It’s an odd sort of leapfrogging, but I guess that’s the way things go.

Several years ago I created this image for a presentation about our research. All of the figures are from Chalktalk.

I rediscovered this image today, and thought it was cool. In any case, looking at it makes me feel happy. 🙂

Last night I did what many people have been doing of late. I participated in a social gathering on Zoom.

Everybody brought their intoxicating drink of choice, we had a far ranging discussion, and some creative background images were deployed to dramatic or comic effect. It was a fun time.

Yet it didn’t feel the same as it does when you go over to somebody’s house or hang out in a bar or coffee shop. There was still that feeling of looking at somebody through a window, as though you are visiting somebody in prison, and you’re all trying to pretend that everything is ok.

On a scale from zero to fully present, I would rate it somewhere between a four and a five, where fully present is ten. Which leads to an interesting question.

As technology advances, we will get progressively better at creating a sense of presence for conversations between people who are not in the same place. How will we measure our progress?

Can we develop a way to quantify presence? Will we have anything to go on, other than individual subjective intuition, to know whether we are making progress?

Perhaps we can eventually create a kind of “Turing test for presence.” That seems like a very good goal to aim for, and now seems like a good time to start.

I wrote yesterday about places to eat
Using VR in various ways
But if folks mostly want a location to meet
The future is VR cafés!

My post yesterday talked about virtual restaurants. The basic idea was to use the forthcoming technology of good mixed reality to virtually join together diners. All customers feel as though they are dining together in a restaurant, although each couple or family group is actually sitting at home.

This could be combined with complementary technologies, such as robotic equipment that prepares the food, and other robotic equipment that serves it. In this way, everybody in the virtual restaurant could partake of the same cuisine.

Delivery services from Whole Foods, Sprouts, and their equivalents could be packaged in such a way that food preparation at home can be entirely automated. So, for example, making a reservation to “dine out” on a Friday evening automatically prompts a set of orders to your food delivery provider of choice. By the time Friday arrives, all needed ingredients are in your house and ready to go.

Meanwhile, thanks to mixed reality, your robot “waiter” can have an avatar that suggests a particular cuisine: French, Italian, mediterranean. You can even get a Hobbit cuisine from the Shire as your virtual waitperson, if you are dining out for second breakfasts or elevensies.

If you are willing to pay a little extra, you can get a live human being to virtually wait on your table. In reality, this person may be operating from their own home, perhaps in Iowa or Kentucky. Technically that would be a form of remote computer-mediated puppetry.

A whole industry could be built around such remote personalized services. But that’s an entire topic unto itself.

As I mentioned yesterday, going to a restaurant can be a very compelling experience. But what if we can’t go to restaurants? We seem to be facing just such a situation now, during the coronavirus pandemic.

Perhaps we can take some elements of restaurants and bring them into the home. This may become much more feasible within the next several years.

Imagine that you and your loved one are dining at home. Perhaps you want to liven things up a bit. You both take out your trusty mixed reality glasses and voila! you are transported to a virtual restaurant.

All around you, other diners are eating, laughing, enjoying each others’ company. The mood is infectious.

Suddenly you feel more alive, your food tastes better, and everything is right in the world. That is the experience of the virtual restaurant.

It may seem different, yet one the most important things about restaurants has not changed: You and the other diners are all paying good money to provide entertainment for one another.

The beauty of restaurants is that people pay good money to entertain each other. Think about it.

When you go to a restaurant, you are paying a lot more for the same food than you would pay to make it yourself at home. Ostensibly, the difference is that people are making it for you and serving it to you.

But I don’t think that’s quite it. The real appeal of a restaurant is that you are surrounded by other diners. You never meet them, but you call catch the excitement from each other of being someplace special.

Essentially, a restaurant plays to our instinct to get out of the cave and be in the presence of the larger tribe. When you see it this way, you realize that the diners at the different tables are collectively providing entertainment for each other.

In fact, if you really listen to the conversation in a crowded restaurant, you quickly realize that snatches of conversation at one table are soon repeated and incorporated into what is spoken at the next table over. People don’t consciously realize they are doing this, yet they do it nonetheless, and they do it a lot.

The effect of all this is that when we hang out at a restaurant, we feel smarter, wittier, more alive. Words flow with the wine, and everybody sparkles just a little bit more brightly than they do at home.

The beauty of restaurants is that people pay good money to entertain each other. Think about it.

For the first weeks of the coronavirus outbreak I was having difficulty balancing my time between the personal and professional. Early on I made a rule that I would keep work only on certain days of the week, and then only within a certain range of hours.

I thought that would solve the problem, but it didn’t. I found myself worrying about work when I should have been focusing on my personal life. I think the problem is that once work goes virtual, it is (ironically) always present in the room, wherever you are.

But now I think I have really solved the problem. Not only do I walk away from the computer outside of certain bounded hours, but I also walk away from the phone. Not only that, I power off my phone.

Once my phone is powered off at some point in the afternoon, I don’t even turn it on until the following morning. My feeling is that anything from the outside world that would require my attention at night can just as easily wait until morning.

There is no email so urgent that it can’t wait until morning. And if somebody phones me and really needs me to call them back, they can leave a voicemail message.

It was interesting to see how my subconscious self adapted to this change. At first I found myself reaching for my phone at random moments.

Then I would remember, with a sense of relief, that it was literally impossible for me to reach for my phone. Not only was my phone powered off, but it was sitting in a room somewhere else in the house, at a safe distance from my neurotic self.

Since I have adopted the practice of turning my phone off after work is done, life has gotten much better in every way. I am far more present with people I love, the people who are right there in the room with me. Not coincidentally, my level of anxiety has gone from very high to very low.

Turn off your phone. I highly recommend it.

Seeing everybody virtually through video software, while also catching up on lots of old movies, my mind goes to odd places. One of those places is the question of digital make-up.

One of the charming things about Hollywood movies is that they work with “found materials”. Cary Grant looks like Cary Grant, and Winona Ryder looks like Winona Ryder.

The particular quirks of nature that formed these unique human beings are factored into the writing, directing, lighting, editing and other choices made by filmmakers. If a different actor had been cast, a good filmmaker would have adjusted those choices.

Yet we entering an age where natural appearance matters less and less. At some point the digital alteration of appearance will become a mature technology.

At that point, all bets are off. If Paul Giamatti has the chops to play a Harrison Ford role, then why not? Even better, if he has the chops to play Humphrey Bogart well, then Bogie is back.

Beyond that, filmmakers will be able to create actor appearances that are tuned to the role, without needing to find somebody who happens to look just right. Screen acting will become a kind of digital puppetry, and nobody will care what the puppeteer happens to look like.

What will be the impact of all this on cinema? Will movies be better or worse? I have absolutely no idea.

There’s a sort of game that I’ve found myself playing with friends and colleagues lately. The basic idea is that we extrapolate from the current situation to see what it means for the future.

Specifically: What things, we ask, will not go back to the way they used to be? Now that millions of people are getting used to being home with their families, will they all just go back to working in offices?

Or will people resist going back to the relatively soulless world of long commutes and office parks? Will some other set of social and economic structures emerge, one that is more humane and family friendly?

Of course, after this outbreak is over, some things will go back to being pretty much the way they were before all of this ever started. But other things won’t.

And what will those things be? It’s an interesting question.