Month: November 2023

There has been much debate in recent years about the relative merits of optical see-through XR (as in the Hololens or Magic Leap) versus video passthrough XR (as in the Vision Pro or Quest Pro). I have a very strong preference.

It comes down to the question of immersion. Do I feel that the XR objects that I am interacting with are immersed within my world?

In the case of optical see-through, objects are visually clipped to a somewhat small area in front of me. If I turn my head to the side by even a moderate amount, the object that I was just looking at will disappear.

That is not the case with video passthrough. A virtual object in video passthrough is visible throughout your entire field of view. However you turn your head, you can see that it is still there off to the side.

On a visceral level, these are a fundamentally different experiences. Objects in optical see-through feel transient and ephemeral. In contract, objects in video passthrough are persistent, the way real objects are — you get the feeling that they are still there even when you are not looking at them.

To me this makes all the difference.

Today is Savion Glover’s 50th birthday. Thinking of that great dancer/choreographer always reminds me of one of my odder cinematic experiences.

In 2006 I went out to the movies with some friends to see Happy Feet, an animated film about dancing penguins (although not the first — Mary Poppins got there first). I enjoyed the movie ok, but the thing I really loved about the movie was the tap dancing.

Now here is the weird part. Every time the lead penguin danced, it was obvious to me that I was watching the tap dancing of Savion Glover. It was as though I was watching the man himself — visualized as a penguin, but still inimitably himself.

Sure enough, his name was listed in small print buried somewhere in the end credits. He had not only choreographed the dances, but they had motion-captured the man himself.

I enjoyed it, but there was something odd about it. The one thing that delighted me about that movie — the dancing — was only credited in a kind of “blink and you’ve missed it” way.

And don’t even get me started on Being John Malkovich and Phil Huber…

When a new technology replaces an old technology, there is a transition period during which people who remember the old technology may miss it. For example, I am sure that there were people during the time of the advent of talkies who missed the very different kind of expressive power of silent movies. But that generation is long gone.

I wonder what technologies of today will experience that temporary burst of nostalgia for a transitional generation. There might, for example, in our own lifetimes, be a group of people who will still remember when people drove cars as opposed to cars driving themselves. Later generations will wonder why anybody would ever want such a thing, and there would be no easy way to explain it to them.

One of the stranger aspects of being human is the way that what is going on in one person’s mind affects what is going on in the minds of other people. On a physical level of course, we are all completely separate beings, each with our own body and our own personal space.

Yet when people get together, all sorts of strange transferences occur. It’s as though there is some sort of extra dimension, parallel to this one, in which bodies and brains are connected by a psychic cable.

My mood affects your mood, and your mood in turn affects the mood of the next people you meet. This mental transference can be either good or bad, but it is rarely negligible.

I guess it is a large part of what makes us human.

Bodies are weird. There are many things about the human form that seem completely capricious and arbitrary.

If we were an alien species on a different planet, we might have a completely different type of body. I suspect that on a distant planet somewhere there is an alien contemplating its own body.

And I suspect it might be having exactly the same thought that I am having. It’s nice to think that we have something in common.

When I am programming, there is a point at which everything is working. I’ve caught all the bugs I can think of texting for, and the code can safely be distributed for use by others.

But that’s when I start polishing. It makes no difference to anybody else, but to me it matters.

I double check all the naming and style conventions, find and remove any extraneous code, and add more little comments here and there. Functionally, these changes make no difference, but on some level they matter.

I suspect this is similar to the mindset of someone who polishes the parts of their motorcycle that nobody else can see. It may be just a conversation between you and yourself, but it’s a conversation that matters.

We had an interesting discussion this morning about my posts of the last two days. The question on the table was whether an AI-enhanced literary work is still a book.

We concluded that it comes down to the intention of the author. If your goal is to use words to make a painting, then you are making a painting. If your goal is to use words to create an interactive game experience, then you are creating a game.

But if your goal is to tell a story with words, and those words are subsequently enhanced through automated means, then your book is still a book. For example, an e-book on a Kindle is still a book, and an audiobook with A.I. narration is still a book.

Intentionality matters.

Continuing the topic from yesterday’s post…

The reason it is still a book is that there is still only a single author. Contrast this with a movie or a theater production, which is a collaboration between an entire team of professionals with complementary skills.

Such creations require not just a screenplay, but also a production designer, director, camera operator, lighting designer, sound designer, and so forth, as well as a cast and crew.

But a book is the product of a writer’s mind. What you read emerges from that writer’s imagination and ability to tell a story.

In my “future book” scenario there is still only you and the writer. Yes, there is also Artificial Intelligence, but that is a tool, like a camera or the process of printing and book binding, not another human artist.

Eventually we will come to see A.I. as just another tool, like a movie camera or a paint brush. What really matters is the judgement and skill, for better or worse, of the person who wields that paint brush.

If you put on your XR glasses and walk into it, is it still a book? If the pages you are looking for appear as you wander through the room, is it still a book?

If the illustrations come to life when you play with them, is it still a book? If you can ask it a question, and it immediately puts together a little movie to act out the answer for you, is it still a book?

I say yes.

There are sunsets every day. They are so common that we completely forgot about them.

And yet each sunset is beautiful beautiful and priceless, a crown jewel of nature’s miracles.

I wonder whether we could all learn not to take such miracles for granted. That would indeed be a miracle.