Month: June 2017

There is a well-known schism in the world of computer graphics. People who create computer graphics for games tend to think differently from people who create computer graphics for movies and animation.

In a three dimensional X,Y,Z coordinate system, people who work in game development usually think of the Z coordinate as pointing up toward the sky. In contrast, people who create computer graphics for movies and animation often think of the Y coordinate as pointing up toward the sky.

I think this is due to the fundamental problem each is trying to solve. A person who is designing a computer game starts out by thinking about strategy: How does a character move about in the world? Does the character need to go over a bridge? Storm a castle? Escape from a room?

These are the sorts of questions you tend to ask while looking at a map. So the first two coordinates a game designer thinks about — the X and the Y — are the ones that describe the geography of the game world. The Z coordinate is then added later for height above the terrain, to create a 3D game experience.

In contrast, a filmmaker asks questions in terms of the final image: How will this composition look in my movie? Is this a head shot or an establishing shot? Are these two characters looking at each other?

These are the sorts of questions you tend to ask while looking at a movie screen. So the first two coordinates a film designer thinks about — the X and the Y — are the ones that describe the image on that screen. The Z coordinate is then added later for depth into and out of the screen, to add visual layers to the image.

Practically speaking, this has sometimes led to some difficulty in the business. Some software is designed with the “Z is up” convention and other software is designed with the “Y is up” convention.

Everything works great until you try to use these various software programs together. Then again, whoever said art was supposed to be easy?

Unexpected time
Just for me and for my thoughts
Five hour flight delay

If you’ve just seen a man
With a beard on his chin

And dark shades on his eyes
Who is crossing the street

Wearing dark capri pants
And a Starfleet dress shirt

Who texts as he carries
A large Sousaphone

Then you can be sure
That you’re in Seattle.

There is a tendency among computer gamers to find the most energy efficient way to play. For example, if you watch a beginner play Wii Tennis, you may see a lot of wild arm swinging. Yet an expert player will barely move the controller. For this particular game, as for most computer games, less movement enables greater control.

This suggests that the standard input control designs for computer games may not produce the best long term results for physically immersive Virtual and Mixed Reality games. To see why, we need to go back to earlier and more established forms of human entertainment.

Consider the guitar. In order to play the guitar properly, you need to use a mix of larger and smaller muscles. A proficient guitar player will use her body and arms to keep the guitar in an optimal position as well as for placing her left hand at the optimal location and orientation on the fret board. The fine motor control of her fingers are not tasked with any of this work, but rather are free to press against the fret board in the most energy efficient way, thereby allowing maximum control and dexterity.

A similar separation of tasks between large muscles and small muscles is seen in most musical instruments, from piano to trombone to viola. A well designed instrument allows its user to make optimal use of various parts of the body’s musculature, with the large muscles shouldering the burden of strength work, and the small muscles performing the fine motor tasks.

We see the same thing with well designed sports equipment, such as a tennis racket, or a football. An expert player will use complementary muscle groups at all times.

This suggests that if we are going to be evolving from the disembodies realm of games on screen to the more active and embodied realm of VR/MR, we should be designing games that make full use of the body. Those games may be more difficult to learn than “wrist flicking” games, but I suspect that in the long run they are going to be a lot more satisfying, and will have a better chance of becoming widely adopted in the years to come.

I had dinner last night here in Seattle with an old friend at an excellent vegan Heavy Metal bar / restaurant called Highline. It was pretty awesome.

Diners around us were wearing punk clothes, and the restaurant played excellent death metal music while we ate yummy vegan comfort food. It was all delicious, but probably not healthy.

I ordered a giant vegan BBQ “pulled pork” po-boy made with Seitan. And lots of other yummy stuff too.

When the huge sandwich arrived, I took a moment to behold its decadent plant-sourced magnificence. Just before taking the first bite I told my friend: “It makes sense that in a vegan death metal restaurant, they would worship Seitan.”

Today, after giving a research talk at Oculus about educational uses of Virtual Reality, I was having a conversation with some of my colleagues there. One of them related an interesting anecdote about VR.

It seems that one day not that long ago, when he and a colleague at Oculus were taking a taxi, the driver overheard their conversation about Virtual Reality and volunteered that she used VR at home. Eventually it came out that she used it to watch pornography.

My colleague, intrigued, asked her whether this was something she did when she was alone. Oh no, she replied, she experienced VR porn together with her boyfriend.

It seems that they did this while they were having intimate relations. They would each don a VR headset, and watch a different sexual fantasy, while at the same time engaging in similar activities with one another.

He and his colleague expressed surprise at this unexpected use case. Their driver replied that lots of her friends used it in the same way.

When he told me this story, I told him that it didn’t seem surprising at all. After all, this is exactly what happened in the early days of VHS.

Until the late 1970s, pornographic films were mainly watched furtively by men in seedy movie theaters. It was not an activity people wanted to be caught doing.

But it turned out that the advent of home video led to a far larger market: Couples who liked to watch porn together in the privacy of their own homes.

Women would not have been caught dead going into a seedy porn theater, but watching erotic films at home with their boyfriend was a turn-on. It was a way to set the mood in a safe and comfortable environment.

As with many emerging media, pornography turned out to be a market leader. Once enough people had purchased the VHS players, they were more likely to rent other sorts of movies as well, and the industry grew rapidly.

So I told my colleague that from an historical perspective, this use case was only to be expected. In fact, it is great news for the Virtual Reality industry, and it bodes very well indeed for future market penetration.

I spent the afternoon today visiting an old friend at Valve Software. Since he is working on a Virtual Reality game, and I am doing research that involves Virtual Reality technology (much of which is made by Valve Software), we spent quite a bit of time talking about the nature of the experience of being in VR.

As Janet Murray has noted, new forms of media can be difficult to properly assess, because you need to disentangle novelty from inherent value. Lots of experiences can seem fun and exciting simply because you are trying them for the first time, but a medium cannot survive for very long on novelty alone.

For example, audiences may have been astonished back at the dawn of the twentieth century when Georges Méliès showed actors appearing and disappearing on screen in a puff of smoke. But we are now long past the point in cinema history where such a sight would grab the attention of a movie-going public.

I told my friend that to help me think about such issues, I sometimes try to imagine that I am a student at Hogwarts, learning magic alongside Harry Potter. To anyone reading J.K. Rowling’s books, or watching the films based on those books, the life of such a student must seem pretty extraordinary — conjuring up potions and magical creatures, playing Quidditch, conversing with ghosts and assorted mythical beings.

But here’s the thing: If you actually are that student, none of it seems extraordinary. In fact, it is the definition of ordinary. This is just your regular life, and to get by you need to do your homework and pass your exams just like all the other Hogwarts students you know.

It’s the same for all of those magical worlds. If you are actually in Starfleet flying through the galaxy on a starship, your life might seem pretty ordinary to you. It’s only when we peer into such a world through the magic window of fantasy that we experience a sense of wonder.

One could imagine somebody from a society quite different from ours, looking in on us, as we go about our everyday lives, flying in our airplanes, communicating with our SmartPhones, having all the knowledge of the world at our fingertips. Such an onlooker might think “Wow, it must be so marvelous to have such powers. Those people surely exist in a continual state of delight!”

Alas, it doesn’t work that way. You can’t marvel at Harry Potter and also be Harry Potter at the same time.

How strange that Yahoo is gone. Today, in fact, was the last day of its existence.

It was such a huge and influential company in its time. Back in the dotcom boom Yahoo was the very symbol of internet success.

It’s a bit like seeing a mighty oak falling in the forest. You know that the once invincible tree is hollow inside, yet you still stand in awe at the sight of a fallen giant.

I guess it’s good to remember that no corporation, no matter its reach or power, is immune from such a fate. Digital Equipment Corporation, Compaq, Sun Microsystems, TWA, Eastern and Pan Am, EF Hutton and PaineWebber, Standard Oil, Woolworths and Westinghouse, so many more, were towering giants in their day, seemingly invulnerable. Yet they are all gone.

I look at Microsoft, Google, Amazon, Apple and Facebook, and ponder that possibility. As indestructable as they may seem now, in twenty years any of them might be just a memory.

There are things one is proud of for the right reasons, and things one is proud of for the wrong reasons. This is probably an example of the second sort of pride, and I’m ok with that.

At our AR in Action conference last week, at some point I entered the auditorium and somebody suggested that I join the artist’s panel. Which, as it happens, was already in progress.

The atmosphere felt very serious. It was clear that the artists on the panel had been engaged in a heated debate about the relationship between Augmented Reality and art.

It was also clear that people had strong opinions on the subject. But since I hadn’t been in the room for that discussion, I wasn’t quite prepared for the very first question one of the other panelists threw at me.

Firmly grasping her hand-held microphone and fixing me with a steely gaze, my fellow panelist wished to know where I stood on the issue. “How,” she asked me, “do you make art out of AR?”

I said the very first thing that came into my head, which was probably just as well. I told her: “You add a ‘t'”.

The other day I outlined the progression of how wearables will be integrated into everyday society in the coming years. Several readers raised very important issues about privacy.

Whenever you are dealing with very large groups of people — whatever the historical era — you can no longer rely on family connections and tribal kinship to guarantee trust. So if you are dealing with anything of value, you need a way to protect it.

Otherwise everyone ends up in a state of fear. Societies that are socially broken are often characterized by roving bands of marauders going into homes and stealing at will. Which leads to the very opposite of freedom.

In other words, in any large and heterogeneous population, if you are going to have doors, then you need to have locks. Freedom requires security.

We already have the technology required to protect against the sort of unwanted tracking, monitoring, reporting, surveillance, etc. that Sally and Adrian warned about in their thoughtful comments. One-way encryption can already provide the required level of security and anonymity.

The problem is not a lack of technology, but rather a general lack of awareness of the importance of putting that technology in place. Alas, humans tend not to deal with problems until those problems slap us in the face.

We don’t think about terrorism until we’ve been bombed, and we don’t think about cybersecurity until we’ve been hacked. It’s just human nature to ignore the open barn door until after the horse has gone missing.

Fortunately, the introduction of the sort of pervasive wearable technology that I described will be gradual. There will indeed be incidents, breaches of privacy, theft of property, but initially not at a mass scale, because the technology will only gradually come up to speed.

The first incidents to reach general consciousness will therefore work as a kind of trigger to our societal immune system. As our citizenry becomes aware of the stakes, we will learn to understand the difference, in the context of wearables, between a locked door and an unlocked door.

And then people will start buying locks. Good ones.