Ken’s Blog

Today is my first day of jury duty. Frankly, it’s not even clear that I should be telling you this, because one of the first things they tell you when you start jury duty is that you are not supposed to talk about jury duty.

I guess that makes it like Fight Club. But so far it’s as if in Fight Club absolutely nothing ever happened. Sort of the Zen version of Fight Club.

Imaging Brad Pitt and Edward Norton sitting around all day, doing nothing. Except at one point Brad says “The first rule of jury duty is you don’t talk about jury duty.” And that’s it. For the rest of the day nothing else happens and nobody says anything.

Except maybe at some point Helen Bonham Carter shows up, and gets confused about whether she is talking to Brad Pitt or Edward Norton.
Because, you know, nobody is saying anything.

So far that’s what jury duty is like. Except without the movie stars.

Today is the day after Leonard Bernstein’s 100th birthday. I didn’t write about it yesterday because everyone else was. I thought it would be nice to wait a day and help give the man a entire birthday weekend.

Like many people, I have been heavily influenced by the musical legacy of Mr. Bernstein. But I also have a memory of a more personal nature.

When I was still a small child, we would be given Scholastic Readers to read in school. The real purpose of these books was to help us improve our reading skills, and the way that was accomplished was quite clever.

Readings were organized into little stories on various topics that would be of interest to a kid. I always looked forward to the next one, but one story in particular has always stayed with me.

It had been written many years earlier by a piano teacher. She talked about a little boy who would come to lessons fresh from baseball practice. He’d show up at the lesson wearing his baseball uniform, carrying his bat and baseball mitt.

Then he would sit down and play, like nobody she had ever heard. That little boy was, of course, a young Leonard Bernstein.

I remember as a ten year old reading that, thinking about the famous musical legend. Before then I had only known of him as a gray haired man, the composer of the music for West Side Story (which I already loved by age ten).

But after reading that, I could also see him as that little boy Lenny, showing up for his piano lesson with dirt still on his pants from sliding into first base. And that change in perspective fundamentally changed my view about a number of things.

I thought to myself “that kid could be anybody, he could be me.” Reading about Leonard Bernstein as a boy dissolved the imaginary wall in my head between “famous person” and “real person”. We can’t all grow up to be Leonard Bernstein, but we can each grow up to do something unique and wonderful, something the world has never before seen.

It was a pretty good lesson.

I just saw Christopher Robin, the new live action Winnie the Pooh movie from Walt Disney Productions. In its way it is a very daring movie, because it violates conventional wisdom about how to tell a fanciful tale.

Story worlds like the ones of Harry Potter or Star Wars or The Lord of the Rings play it safe: They take place in a magical alternate universe, but in a self-consistent magical alternate universe.

In those story worlds, readers or viewers are asked to accept a reality that operates by a different set of rules from our own. Yet once that contract has been made, those rules become inviolable. The underlying reality of such story worlds remains very consistent.

Christopher Robin, on the other hand, is all about messing with our sense of reality, by forcing opposing and mutually inconsistent story worlds to clash head on. We’re not just talking about differing attitudes or moral codes here — we’re talking full-on incompatibility between fundamentally inconsistent Universes.

Which means the story becomes about its own metaphysical inconsistency. The audience is never permitted to forget the deep schism that lies within the very heart of the tale.

To my delight, I felt the filmmakers made it work beautifully. They took a deep risk (particularly given that this is a family movie), and they pulled it off.

You’ve got to admire people who can do something like that.

Sitting in an airport early this morning, waiting for my flight, I am struck by how much an airport is a place of absence. Nobody is here because they want to be in an airport. They are here because they want to be someplace else.

Unlike, say, a train station, the place we want to be when we are waiting for a flight is invariably very far from where we currently are. Our minds are focused on that far off destination, which means our minds are very distant from our bodies.

Also, given the nature of air travel these days, we often spend much more time waiting for a flight than we do waiting for a train. So those of us who need to travel by air can find ourselves spending a lot of time with our minds and our bodies far away from each other.

Fortunately, rather than passively waiting to be somewhere else, our minds can also choose to spend some of this strange liminal time in places that do not even have a physical existence, doing things that are far more pleasant. Like posting to a blog. 🙂

We let ourselves cry at a movie because we know it’s not real. We allow ourselves to absorb the tragedy of Hamlet because we know those are just actors on a stage. We give ourselves over to the world of a book because we know it’s all just words on a page.

We use make-believe to allow ourselves access to emotional landscapes that we would find treacherous in real life. Once the frame around the narrative has been established, we are then safe to explore the complex emotions within.

This need to establish a clear frame is crucial. One of the things I have not liked about much VR content over the last several years is that the frame has not been worked out with sufficient clarity.

The idea of a “movie for an audience of one” just doesn’t sit right with me. When an entire narrative is so insistently focused on you the individual viewer, it’s hard for you to create and maintain the proper frame in your mind.

I am hoping that as more creators experiment with other approaches to VR storytelling, such as the “theater on the Holodeck” approach we took for CAVE, we will all get better at constructing the necessary frame — one that is so effective, audiences will simply forget it is there.

The process of discovering best practices for creating that frame will not happen overnight. It will require learning an appropriate visual language for telling stories in VR.

And that’s ok. After all, it took cinema quite a few years to work out its own visual language.

Mere hours before my flight was scheduled to depart for the SIGGRAPH conference in Vancouver, my wonderful Google Pixel Phone decided to stop recharging. Which left me without a viable phone during a major professional conference.

Fortunately several trusty students helped me transfer all of my data to a different Android phone. Once I put my SIM card into the new phone, the identity swap was complete. So for more than a week I have been using a different phone and a very different version of the Android operating system.

And every second of it was utter misery.

In the immortal words of Joni Mitchell, “You don’t know what you’ve got till it’s gone.” Using the substitute phone felt bad, really bad. Like they’d paved Paradise and put up a snarky bot.

I have now located a USBC cable that connects properly with my original Google Pixel Phone (apparently those USBC connectors are finicky). The Pixel Phone now recharges properly, and I have happily moved my SIM card back into its rightful home.

Today a friend asked me what it had been like to be stuck with a substitute phone for a week. I replied right off the top of my head, without really thinking. Yet on reflection I think my description was eerily accurate.

“It was,” I had said, “like sleeping with a total stranger. Everything felt wrong.”

Very often when I am in conversation with a colleague, one of us asked “Do you know —?” Half the time I am not sure, because the name sounds vaguely familiar but I can’t associate it with a face.

But the moment I see a face, I generally know right away if he or she is somebody I’ve actually met. And then from that image I can recall all sorts of other potentially useful information about the person in question.

Theoretically, if I want to show what somebody looks like I could just take out my phone and speak a name into it. In practice that generally fails because speech to text software doesn’t know to interpret what I am saying as a person’s name.

I wonder whether there is a Paftan (Putting a face to a name) app that is optimized for just this sort of search. Rather than a database of general speech utterances, its machine learning algorithm would be optimized for recognizing peoples’ names when you speak into your phone, and its search results would consist entirely of images of human faces.

Of course the conversation with my colleague might go even better if my phone is also loaded with Pantaf, the complementary app. 🙂

I had a thought yesterday about dystopian predictions for how Artificial Intelligence might bring about the end of the world as we know it. The thought was a sort of corrective to the cultural misinformation about A.I. generally promoted by Science Fiction movies and TV shows.

But it wasn’t really a very comforting corrective. My thought went something like this:

If A.I. ends up wiping out humanity, it won’t be because Sarah Connor was defeated by Skynet. It will be because Mickey Mouse was defeated by the brooms.

When you show a movie or a traditional animation — even a computer generated animation — the work of “rendering” the image has already been done. Every audience sees exactly the same result.

But this is not true for many films made for virtual reality, particularly those made for room-scale virtuality (one in which an audience member’s position in the scene can change). For those films, the actual rendering of the scene happens only at the moment the film is viewed.

And this creates an opportunity. Rather than thinking of a story world as fixed and immutable, we can design the world of a VR story as something that can be evolved over time by audiences.

In the spirit of Star Wars Uncut, the visual and sound-track decisions in a VR film can be opened up to crowd sourcing. Ideas for backgrounds, character appearance, lighting, pictures on walls, music tracks, all of these elements can be made subject to change.

For example, audience members can propose variations, and those variations can then be voted upon by the public. Those variations that prove the most popular become incorporated into the official presentation.

There is no one correct way to do such a thing. Many variations on this theme are possible, and it is not yet clear to me which of those variations would be the most successful. Maybe that could also be crowd sourced. 🙂

Consider the difference between an establishing shot in film and an establishing shot in live theater. In the former, the camera might pan over a city or a village. Then we cut to street level, and the audience understands that they have just entered the human scale of our story.

In terms of literal point of view, the audience is the camera. They were hovering in the sky, high above our locale. And then they, the audience as camera, become repositioned at street level in the same village.

In the case of live theater, the tools and therefore the rhetoric are different. Perhaps the audience sees, on the stage, a miniature of a village. There might be a little puppet walking through this miniature.

Artifice is perfectly welcome in theater, so our audience might even see the puppeteers holding the strings of this little puppet, but knows to ignore those puppeteers. The fact that the presence of such figures is artfully revealed yet ignored is all part of the fun.

The next scene a full size actor walks on stage, and the audience understands that this is the same character, now seen in close-up. The effect is roughly the same as that cinematic sequence of shots, but not exactly the same, because the mental model is different.

When we watch successive shots of a movie, our point of view is literally altered. When we watch scene transitions in theater, we are rather presented with multiple representations of the same subject, which we continue to observe from a fixed point of view.

When presenting theater in shared virtual reality, using the brand new methods that our group showed this past week at SIGGRAPH 2018, it is not yet clear what mental model to use. In terms of technique, there are no limitations: We can choose to transport each audience member’s point of view, as in cinema, or choose instead to present variously transformed representations of that world to a fixed audience, as in theater.

I think the relevant questions to ask concern not technique, but rather the audience’s understanding of its relationship to a fictional world. Will each audience member be asked to project herself into an alternate world, or will she be asked to interpret stylized representations of that alternate world as seen from a fixed location? Or will there be a third way, something unattainable in either cinema or live theater?

I don’t have definitive answers to these questions. But over the course of the next year, my collaborators and I are going to do our best to explore the possibilities.