Ken's Blog

An interesting question has recently come up in our lab’s research: Suppose you could perfectly reproduce the sensation of being in the physical presence of another person, so that remote conversation with that person is indistinguishable from the presence of a person who is physically with you. In what ways would it continue to matter that the person you are talking to is not actually in the same room with you?

To be clear, I am not suggesting that we are yet at the point where this is possible. The detection of physical presence involves subtleties of vision, audio, touch, vibration, even smell, that may be well beyond our current level of technology.

Yet I think the question remains valid for any level of technology, whether past, present or future. For example, we are perfectly willing to have a telephone conversation with somebody, even though our remote interlocutor is not visible to us. This is because we understand the nature of phone technology, and we find ourselves able to use that technology to communicate with each other despite its limitations.

I have no doubt that it will always matter to us whether or not somebody is actually with us in the same physical room. Hence the question: After technology has advanced to the point where sensory reproduction at a distance can be absolutely perfect, what will remain as the essential difference between actual physical presence and the mere illusion of physical presence?

This evening we wrapped our production of Holojam in Wonderland at Snug Harbor. Everyone on our crew was exhausted but satisfied.

There were many things we might have done differently, given more time to workshop or a larger budget, but this production was a valid statement of where we are now, and a shot across the bow declaring our vision for future theater. We had just staged four full days of a multi-actor, multi-audience live magical realist play, with everyone in the same room and wearing untethered VR headsets.

As we wrapped, I realized that all the questions we’d been too busy to ask were now once again on the table. Having seen what is possible, we will now need to set about making it even better.

As I said to one of my collaborators this evening, because I was sure she was old enough to get the reference: This wasn’t our Shea Stadium. This was our Hamburg.

Today was the first full day of showing our virtual reality theater experience to the public – the very first day’s performances of a new kind of live theater. And sure enough, for one of today’s performances the New York Times showed up.

After that performance (which I thought went well, but we will just have to see if the Times agrees), I said to our director that I thought there was an irony here.

He asked what I meant, so I explained. “Here we are,” I said, “creating a shared experience that is all about being together in the immediate moment. Yet it is only culturally validated when the Press shares it with the world.”

Our director just laughed. “Ken,” he said, “theater has always been like that.”

Eighth Street, last shoe store,
Going out of business sale.
End of an era.

It’s amazing to think that the Space Age is 60 years old today, this being the 60th anniversary of the launching of Sputnik 1. Although it seems that nobody is thinking that much about outer space anymore, other than in science fiction movies and TV shows.

We seem to have shifted our gaze inward toward computer simulated possibilities, rather than outward toward the possibilities of the Universe itself. I’m not sure that this is a good thing.

This week we will be publicly premiering a theater piece in which the actors appear as digital avatars. It’s all live performance, with actors and audience members in the same room and in their actual positions. But everything is seen digitally.

Looking at this one way, each actor is a puppeteering digital puppet, which just happens to be the same size and in the same location as the actor herself. Looked at another way, the actor is simply wearing an elaborate digital costume.

So it seems that performance in VR raises some interesting definitional questions. For example, what is the boundary between puppetry and costume?

This is certainly not an entirely new question. After all, when Sweetums first showed up in Jim Henson’s The Frog Prince in 1971, many children were undoubtedly asking themselves the same question: Is that a puppet like Kermit, or is that a guy in a costume like the Cowardly Lion?

Virtual Reality theater ups the game a bit, because the “costume” in question can take on surreal qualities. Take for example, a VR performer dressed up in a digital ostrich costume. Unlike such a costume in the physical world, in VR the character’s knees can bend backwards.

Maybe there is no good answer here. We might need to come up with a different vocabulary to discuss the relatively new realm of VR theater.

Today a tragic mass murder in Las Vegas.

Meanwhile so many Americans are enduring terrible suffering in Puerto Rico.

Meanwhile our State Department is being slapped down for the “transgression” of trying to avoid war with North Korea — an avoidable war that would certainly cause the horrific deaths of a vast number of innocent people.

Meanwhile…

Meanwhile…

Meanwhile.

I am overwhelmed with sorrow.

October comes awake to weave its spell of ancient darkness on the morn
The eldritch sounds of autumn whisper soft as rumors of a wildling born
All about the forest creatures listen for a melody enthralling
We shall sing the ancient songs, for now at last October has come calling

Cast your spell about the scarlet oak, encircle thrice and thrice again
Spirits gathered here have come from far beyond the simple dreams of men
Dance within the hidden place and laugh to see the sacred tower falling
Remember well the ancient songs, for now at last October has come calling

Yesterday Stephan’s comment made a valid point about the sense in which mathematical notation is one dimensional. I think the disagreement, if there is one, is about which semantic level to focus on.

There are statements that are certainly true, yet not at all useful in particular contexts. For example, it is certainly true that humans are made of atoms, but that fact doesn’t provide very much insight about why Romeo and Juliet was a tragedy.

Similarly, I think the statement “all mathematical statements can e expressed as a one dimensional string,” while certainly true, is not useful in most contexts. Clearly it is useful at the meta-level, where Gödel’s incompleteness theorem resides.

But when you are using mathematical notation to communicate some concept or relationship to a fellow human being, you are rarely operating on that meta-level. In such cases, which by far the great majority of cases, you want to maximize for readability and clarity of thought, and your math notation should ideally express how multiple dimensions of ideas interact with each other.

After all, it is certainly true that if I send you a digital photograph of my cat, the transmitted data can be represented by a one dimensional array of pixel values. And as Stephan points out, such a representation is perfectly adequate for performing a Fourier Transform, digital convolution, or various other mathematical operations.

Yet if we insist on keeping things at that level of interpretation, you may never realize that you are looking at a picture of my cat.