A different kind of Augmented Reality

Today I enjoyed a luxurious six hours of uninterrupted time on my flight from Los Angeles to Montreal (actually more, because of flight delays).

There was nobody except me and my computer and my music.

Sure, there were people all around me in the plane, but I was able to tune them out, and focus on the many things I wanted to work on on my MacBook.

It definitely helped that I had in my Bose Hearphones. All of the outside sounds faded away to a distant dream, and I was free to float in my own world, listening to my favorite songs and getting lots of work done.

As the flight approached Montreal, I switched the music to Leonard Cohen.
Which I suppose is obvious, but it was also completely wonderful, just Leonard and me.

When people think about augmented reality these days, they usually think about enhancing what you can see with your eyes. But maybe, when we think about the possibilities of augmented reality, we should also be thinking about what you can hear with your ears.

Snap chats

Snap Inc is tucked away inconspicuously less than a block from Venice Beach in L.A. The company’s presence here is almost totally stealth — if you didn’t already know where it was, you would be hard pressed to find it.

I was here to visit for just one whirlwind day (flying in from NYC late last night, then flying off to Quebec City for the UIST conference tomorrow morning). This morning I gave a guest lecture, and then, over the course of the day, met with a lot of very cool and brilliant people.

I hadn’t been completely sure what to focus on for my talk, so I had given a sort of overview of various thoughts and experiments. Then, after meeting with all those great people one on one and in small group discussions, I developed a better understanding of what I should have focused on in my talk.

I kept thinking, all through the day, “Ah, if only I had shown that demo, or described this project.” But of course, when I’d given my talk in the morning, all of those great conversations had not yet happened. And outside of a Ted Chiang story, nobody can actually see the future.

So I consoled myself by noting that those very thoughts meant that this is probably just the beginning of a fruitful collaboration. And the next time I come here for a visit, I will probably know exactly what to lecture about. 🙂

Future conversational pinging

In 2017 it’s pretty obvious when someone isn’t paying attention to the person they are with, due to some on-line distraction. You see this all the time: somebody in a restaurant is staring at their phone, while their companion across the dinner table either looks on in dismay, or else gives up and takes out their own phone.

But if wearables take off, then this overt social signaling of inattention will no longer be available. People will look right at you, while secretly scrolling through Facebook, or reading their email, or just checking out the latest funny cat pics.

In response, their conversational companions might develop a new sort of defensive strategy, which might be called “conversational pinging”. Rather than just talking, and assuming one’s companion is paying attention because they are looking right at you, we might evolve techniques to periodically “ping” our conversant, to make sure he is really paying attention.

This isn’t something we generally need to do now, because when in 2017 when someone is looking right at us, they literally cannot be checking their Facebook page. But once wearables start to take over, all bets will be off, and pinging strategies will start to evolve and grow more sophisticated.

Such pings might consist of mentioning a surprising name or topic that is sure to elicit a reaction — unless the other person is not paying attention. Or deliberately slipping a bad joke into the conversation, and acting as though you just said something funny. If the person laughs anyway, you can be pretty sure it means they weren’t really paying attention.

Or else it means they have a really bad sense of humor. In which case you probably shouldn’t keep talking to them anyway.

Bonsai roots

This evening a friend took me to see Uncle Bonsai, an edgily funny deconstructionist alt-folk group that has been playing, recording and touring on and off for more than thirty five years.

At one point during the between-song banter, guitarist Andrew Ratshin started to wax rhapsodic about Wonderama, a kids’ TV show that reached its peak of popularity about half a century ago. Gazing misty eyed into the middle distance, he said “The show was never the same after Sonny Fox left.” **

He then turned to the audience and asked “How many people remember Sonny Fox?” A surprisingly large number of hands shot in the air.

At which point somebody from the audience shouted “Bob McAllister was better!” A hush fell over the crowd, and there was a moment of eerie silence.

Ratshin’s bandmate Arni Adler then turned to fix the heretic with an icy stare of disdain and disbelief. “After the show,” she said, “we’re taking this outside.”

** Sonny Fox left the show in 1967

The comedy of the commons

Generally speaking, people are pretty good at keeping our lab’s kitchen clean. They wash their own dishes, put food away in the fridge, throw out their garbage after meals.

But people aren’t always perfect, and sometimes somebody leaves some paper plates on the table, or an unwashed mug ends up sitting in the sink. This makes nobody happy.

The other morning I came in to find some napkins and paper plates on the table. On top of them was a big handwritten paper sign. It said: “It’s rude to leave your stuff lying around.”

I could understand the frustration of the person who found the napkins and paper plates lying on the table. And I could see how they would want to send a clear message to the perpetrator.

But I also could see what this whole scene might look like from the perspective of an outside visitor to our lab. “Who are these people?” she might ask herself. “Are they engaged in some sort of ongoing domestic war? Do they even like each other?”

I did what I usually do when I see stuff lying around. I picked it all up off the table and I threw it in the trash — including the handwritten paper sign.

For me this is a typical response. When I see the occasional unwashed dish or mug in the lab sink, I just wash it and put it in the drying rack.

There is a part of me that wonders whether I am being an enabler, whether maybe there is a better path. But if that “better path” is to shame people, I’d prefer to stick to the path I’m on.

After all, as a faculty member it’s part of my job to invite visitors to our lab, so I always need to see what our lab looks like from the perspective of those visitors. And when you look at it that way, the occasional dirty mug in the sink is a lot more acceptable than evidence of shaming.

That song in your head

Have you ever gone through an entire day, and then realized, only toward the end of the day, that all day long a particular song had been running through your head? That happened to me today.

The funny thing is that it has happened to me lots of times before. And I generally don’t realize, until the end of the day, that I have been playing the same song in my head for hours on end.

I suspect that this may be a widespread phenomenon. And I also suspect that it’s a sort of “pearl in the oyster” sort of thing.

In other words, somebody says a random phrase, or you have a particular experience, which gets a bit of remembered song lyric into your head. You don’t really take note of that initial experience, but your brain latches on to that moment, like the grain of sand in an oyster, and builds upon it.

Before you know it, the song is playing in your head. Except you don’t realize, until much later in the day, that you’ve been playing, on an endless loop, that song in your head.

The most productive decision

Yesterday, for the first time in a long time, I took the entire day off. I stayed home, did all of the New York Times crossword puzzles (between Saturday and Sunday there are a lot of puzzles), binged on Netflix, raided the fridge, binged on Netflix some more, and just generally goofed off.

Toward the evening I wandered into the lab and worked for a few hours. I didn’t really get much done, but I sort of got clear in my head what I really wanted to get done.

Then I went home, fell asleep by 11pm, woke up early this morning, and headed back to the lab. By 8am I was at work, and I ended up having one of the most productive days I’ve had in years.

I tackled bugs in my software that I had been sure would be forever unsolvable. And they fell apart like tissue paper, transformed into clean working code before my eyes, because I finally had the clarity of mind to approach them from a fresh viewpoint. I dove into project reports and research descriptions that I had been procrastinating over and avoiding. And they turned out to be easy to write — fun, in fact!

I am quite certain that the events of these two days were strongly linked. The choice to deliberately take an entire day off was probably the most productive decision I could have made.

Killing your own supporters, for fun and profit

So now the creepy reality show idiot has decided to go out of his way to create a two-tiered system of healthcare. The poor, the infirm — many of the very people who voted for him, expecting something much better than they got — are the victims here.

Most people who have money, power, privilege, will be just fine. In other words, everyone he knows.

But if you aren’t the “right sort” of American — if you were not born to privilege and power, if you’re a working class citizen struggling to get by — then you won’t be covered for maternity leave, or for pre-existing conditions, or for mental health issues, or if you sustain a serious injury.

The message is extremely clear and unambiguous: If you are not a person of privilege in this country, then you are now considered a lesser human being. Your life simply isn’t important, and nobody really cares if you or your family get sick or die.

I wonder how many of the people who voted for this moron realize what he is doing to them and their children?

The fourth wall, revisited

In theater, the direction toward the audience is called “the fourth wall”. Most of the time, the characters on stage are not aware that we are there, watching them. Only in experimental theater and works inspired by Vaudeville do they actually look out across the footlights and acknowledge our presence.

In cinema it is rare indeed for a character to breach the fourth wall. The only person I can think of in cinema history who was able to do it well was Groucho Marx. Bugs Bunny had an easy time of it as well, but he had the superpower of being a toon.

Every time a fictional character acknowledges us, the “magic circle” of storytelling is weakened. Without this magic circle firmly in place, the entire trust relationship underlying our willing suspension of disbelief is prone to collapse.

One trick that is sometimes used in cinema is to erect an inner fourth wall. This was done rather famously in The Ring. When a nightmarish figure crawls out of a TV screen within the movie we are watching, we vicariously share the horror of a character in the story. as he experiences the unexpected collapse of the safe distance between “media” and “reality”.

As we move to more immersive media, such as virtual reality, I wonder how that will change our relationship with the fourth wall. Imagine, for example, some future where VR experience, within which we are watching a movie of The Ring. Within the movie, we see a nightmarish figure crawl out of a TV set, and we witness the mounting horror of the unfortunate character who was watching that TV set.

But then we see that same nightmarish figure crawling out of the screen that we are watching, and start to head our way. I wonder what we will think.