Ken's Blog

A colleague of mine, one who has great insights about things, said something surprising to me today. She told me “VR that doesn’t involve moving your body is stupid.”

I disagree with that statement, but I think it reveals something more interesting than a simple difference of opinion. I think it shows that she and I, although we both work in what is usually called “VR research”, are actually working in somewhat different fields.

To her, and to many good researchers, virtual reality is primarily an exploration about the possible ways we can connect with our bodies. I completely agree that this is a great problem to explore, and there are many exciting questions to ask in that direction.

Yet for me, the most interesting questions about VR center about how people relate to each other. Such questions can involve moving one’s body, but they don’t need to. They can, for example, focus on how we tell each other stories.

The promise of VR is, after all, the promise of sensory immersion into other worlds. Those alternate worlds can potentially affect us in profound ways, and those ways can be intellectual, emotional, spiritual or physical.

From my perspective, learning how to use VR to help achieve a profound connection with another human being, even when we are both simply sitting down in one place, is a valid direction for exploration.

It is not that I am against the vast space of how we might use VR to connect with possible worlds through the movement of our bodies. That is indeed an incredibly rich area for exploration.

It is more that I think that as a researcher I prefer to think less in terms of “What is the right direction for VR?”, and more in terms of “What are the many possible directions?” For me (and I speak only for myself here), the latter question is much more exciting.

This evening I went to see a multimedia show in the really great dome in Montreal at the Société des Arts Technologiques. The dome is fifty feet across, and it is a thing of wonder.

There are no chairs in the dome. Instead, the floor is covered with beanbags.

When you come in, you just pick the nearest open spot and lie back to watch the show. I am a friend of one of the artists, so I came early and was able to pick my spot.

So for the first time in my life I achieved a childhood dream: I got to sit in the very center of the dome. Always before I had to sit off to one side or another, and I had always wondered what it would be like to sit dead center.

One of the great things about the exact center of a dome is that when you start to talk, your voice sounds very strange. The sound of your voice takes 50 milliseconds (1/20 second) to reach the dome surface and then return to you, and that happens the same way in all directions.

So it sounds like your voice is talking back to you 1/20 second after anything you say. It is a strange and wonderful sensation. If you’ve never tried to stand in the center of a dome, I highly recommend it.

Of course the best part was that I got an undistorted view of everything they showed. It was wonderful, and now I can day that I have fulfilled my childhood dream of watching a show from the very center of a dome.

I’ve been thinking about the specter of members of Congress grilling Mark Zuckerberg recently. And I’ve been thinking about how absurd that was.

Don’t get me wrong. I don’t think of Zuckerberg as a “good actor”. As far as I can tell, his position is essentially amoral: Facebook is in the business of maximizing its profits by selling ads.

As CEO of Facebook, he will likely take whatever course of action best furthers that agenda. This includes whatever actions (like politely testifying to Congress) which succeed in portraying Facebook as “the good guys.”

But here’s the odd thing: Zuckerberg is quite open about his essentially venal agenda. He does not hide the fact that his loyalty is to his shareholders, and that advertisers — not the general public — are his customers.

Yet Congress does not have such a clear sense of itself. This is the same Congress which passed a law last year which gave Comcast, Verizon and Time Warner fairly unfettered rights to go through your entire web history without your permission, gathering all of that info for monetary gain.

Their argument (this is so stupid that I’m having trouble believing it even as I type this) was that Google and Facebook do it, so why not Comcast, Verizon and Time Warner? Of course the answer to that is very obvious:

You can go through your life perfectly well these days without using either Facebook or Google. But unless you are extremely unusual, you cannot go through life these days without access to the internet.

Was the Republican majority that voted for that horrific policy cognizant of how stupid and misguided it is? Were they simply snowed by slick industry lobbying? Were they paid off by the telecoms to vote in a corrupt way?

My worry is that the real reason is just that they are stupid and ignorant. And if that is the case, then what are they doing trying to get the best of Mark Zuckerberg?

Oh right, I remember now: They were being idiots.

Would it make sense on any level to think about an Internet for non-humans? What would the internet for a dog look like? A cat? An elephant?

I find this to be an interesting mental experiment because it forces us to realize the many ways that other sentient beings on this planet are different from us. They don’t merely look different — different species think in radically different ways.

Which means that many of our core assumptions underlying good design practice for the Internet might need to be thrown out and started afresh. To me this doesn’t seem to be such a bad idea.

Sometimes you need to shake things up a bit.

In our discussion about what would constitute “reasonable” privacy in future augmented reality, my colleagues and I settled on some core principles. To review, the question on the table was the following: Suppose you allow yourself to see a synthetic overlay through your augmented reality glasses which is socially unacceptable to others. Under what circumstances is that ok?

The conclusion we reached was that it all comes down to whether you’ve taken reasonable steps to lock down your private info. An example of not taking such steps would be if you just left your AR overlay content lying around on the table, just waiting for somebody to slip on your AR glasses.

The key here is the word “reasonable”. For example, if you lock the door to your apartment, and somebody still breaks in, the law generally acknowledges that you were not negligent. The “bad actor” who broke into your apartment was violating well established norms — both cultural and legal.

The same principle will apply to the question of “What can I see privately in my AR glasses?” If you’ve taken reasonable steps to protect your privacy, then it is not your fault if somebody else violates the law.

I think most people would agree that hate speech is problematic. The vandal who spray paints a swastika on a synagogue door is engaging the entire community in an ethical challenge.

These days, most communities would respond to such a challenge in a very negative way. Such an act would be labeled hate speech, and there would be consequences for the perpetrator.

But what about “speech” that is intended for nobody but oneself? Suppose, for example, you enhance your (slightly in the future) augmented reality glasses to draw a virtual swastika on the front door of every synagogue you can see — an intervention intended for your eyes only.

Have you committed an ethical violation of community standards? Have you, in fact, even engaged your community in any way?

Today I had a rousing debate with some colleagues about these very questions. We didn’t come up with any simple solutions, but we did work out some basic principles.

More tomorrow.

It is so peaceful
Knowing that I cannot do
Anything but wait

I was having a conversation with a colleague and the phrase “transparent process” came up. It’s a great phrase, and it strikes to the heart of some interesting cultural questions.

For example, why is there a rich general shared culture of music, or of cooking, or of gardening or acting or writing, but not so much of computer programming or architecture? There are many answers to this question, but I suspect at least part of it has to do with transparent process.

The process of getting into music or cooking or gardening or acting or writing — and of many other crafts and skills as well — is quite transparent. Even a beginning musician or writer understands the basic process, and is able to perceive and absorb the ideas of advanced practitioners.

Yet many fields — particularly those we think of as the “technical fields” — don’t seem to offer this level of transparency. Most people can pick out a melody on a musical keyboard, yet most people cannot write even the simplest of computer programs.

This is not for lack of trying. There have been many attempts to create a transparent onboarding process for budding programmers. And yet it is arguable that these efforts have failed, at least in comparison with efforts to show that “anybody can cook” or “anybody can play the piano”.

I wonder whether this is due to an inherent opacity somewhere in the process of learning the so-called “technical fields”, or to cultural bias. Or perhaps it is due to something else entirely.

In particular, we’ve had a long running split in the computer world between “compute it” and “capture it”. In my own work in texturing, it has often come down to “generate a procedural texture” or “scan a texture image”.

Yet like most dichotomies, that turns out to be a simplification. In practice, people will scan a texture image and then use that image as source material for a procedure.

For example, you might use Photoshop to paint an image of “here is where the forest should go.” Then the places where you painted green will be used by a computer program to grow synthetic trees.

So in the best cases it’s not really “procedure versus data,” but more “procedure using data.” Now we are just entering a new regime where this partnership is really taking off.

That’s because of recent rapid advances in machine learning. The beauty of machine learning is that it builds a procedure from data. The more examples of existing data you give it, the better will be the procedure that it can build.

Machine learning isn’t a panacea — it will only show answers to new things that are similar to the things you’ve already showed it. But it’s a lot better than anything we’ve had before.

For solving completely new problems, we still need human brains creating procedures. Computers don’t know how to do that yet. And maybe they never will.

Which may not be a bad thing. 🙂

This whole argument about “procedure versus data” is perhaps a bit of a red herring. Long before computers, the two modes of operation formed a complementary set.

For example, you probably know a musician who has an encyclopedic memory for songs. You name pretty much any song, and he or she will remember that song on the spot and play it for you.

And you may know a musician who is a great improviser. You name a musical style, and he or she will be able to immediately riff in that style and create something new, something that has never been heard before.

In my experience, one rarely finds a high level of development of these two complementary skills within the same individual. And that makes sense, since each kind of skill takes not only native talent but many hours of time and practice to learn and develop.

But why should these be seen as two separate skills? Isn’t there some place where they meet, and build upon each other? More on this tomorrow.