Ken’s Blog

I’ve noticed, in recent discussions with my students about the potential future of virtual reality, that the “Matrix” question comes up rather frequently.

I mean, the question of whether there is any way to know if you are experiencing an excellent computer simulation of reality, rather than reality itself.

In the original film, Neo was given the choice of taking a blue pill or a red pill. If you take the blue pill, then you remain blissfully aware that you are living within an illusion. But if you take the red pill, then you end up waking up to the reality outside the emulation.

Which leads to the following question: It you suspect you are in the Matrix, and you really really want to take the red pill, what would be your best strategy to figure out whether the world you see around you is just a simulation?

I suspect the answer would have something to do with the topics I discussed yesterday. Some things are much much harder to emulate than others, so your best bet might be to figure out what is the most computationally expensive thing to emulate, and then test for flaws in that.

Then again, if there is an A.I. agent monitoring your experience, intent on keeping you on a blue pill diet, then it can simply warp your perception of whatever experiment you try to perform, thereby maintaining the illusion of a perfect emulation.

So maybe you would need to design an experiment that takes such an A.I. agent into account. Which might not be so easy. 🙂

I was having a spirited debate with a colleague about where the brain ends. In particular, we were discussing where you can say “this part is the human body”, and “this part is the human brain.”

My colleague was arguing for an inclusive definition — the body is a simulator of itself, and the brain cannot function without the feedback from that body. One example he gave was the chemical activity in our gastric system. When we’re nervous or agitated, the nerves to our brain receive signals from all that chemical activity.

Eventually we both clarified that what we were really discussing was the question of what parts of the body, if any, could be effectively emulated — replaced by a simulation — without adversely affecting our brain’s ability to process the resulting data. We both agreed that it probably wouldn’t be too difficult to computationally simulate the chemical reactions in the stomach that trigger signals in our nervous system when we are agitated or upset.

But we disagreed about the nerve pathways that run from the body up into the brain itself. I felt that it would be extremely difficult to replace those pathways, and the various sorts of processing that goes on along them, by computer emulation.

Thinking back on the conversation, it occurs to me that we might really have been discussing Moore’s Law. Assuming computers continue to get twice as fast every eighteen months, when will it become feasible to emulate the signals from the stomach to the enteric nervous system? And then at what later date will we be able to emulate the parasympathetic nervous system? And then the entire autonomic nervous system?

At what point will we be able to emulate the cerebellum? The optic nerve? Specific language centers in our brain?

We could map out a rough timeline, given that Moore’s Law holds, when it could be possible to replace more and more of our experience of reality with a computer emulation.

I’m not saying that this would be a good thing to do. I’m just saying that at some point in the future, parts of it will become possible to do. And whatever side of the ethical debate you are on, you might be well advised to know where the important points are along that timeline.

I suspect that there are key central functions of your brain — including higher level cognition that we associate with our conscious sense of self — which would be far off to the right on that emulation timeline, perhaps several hundred years beyond your lifetime.

Unless, of course, the other parts of you become emulated well enough. In that case, your conscious mind might still be around when it becomes possible to emulate your entire brain. I wonder what your opinion will be then.

Like many Netflix users, I’m watching various TV shows in parallel. As I mentioned a few weeks ago, one of them is “The Gilmore Girls”. Another is “Dr. Who”, and a third is “Marvel’s Agents of Shield.”

It’s hard to choose a favorite, when you’re ping-ponging between Joss Whedon, Steven Moffat and Amy Sherman-Paladino. I would love to have all three of them over for dinner sometime. If that ever happens, I promise I will record their glittering conversation for posterity.

One thing I’ve notice is that in the most important sense, they are all essentially the same show. Well, not literally. One is concerned with the dating life of a single mom and her teenage daughter, another is concerned with saving the world from alien menaces, and the third is concerned with, um, saving the world from alien menaces.

But really, underneath all that, they are pretty much exactly the same: A ragtag group of very smart non-conformists, deprived of anything most of us would recognize as a normal family, have banded together to build their own idiosyncratic version of family. All of which is accompanied by rapid-fire quips, conversations filled with obscure pop culture references, and the kind of “us against them” sardonic humor that would be instantly recognizable to any middle school kid who has ever gotten beaten up before lunch.

Come to think of it, I think I’ve just described half the TV shows out there.

A few days ago, as I was putting the final touches on that interactive logo I talked about in yesterday’s post (well, it’s interactive if you’re not using Firefox), I decided to explore the space a bit more.

So I implemented a kind of ‘Lego version’: When you draw any collection of lines, those lines are converted into links of a flexible interactive shape that you can then play with. I showed this to my computer graphics class, and we used it to make little stick figure creatures as part of a discussion of principles of animation.

One thing led to another over the next two days, and now this little animated stick figure program has grown into a new direction for our lab that ties together research ideas in interactive drawing, procedural animation, recursive graphs and virtual reality.

I was very surprised when all of these connections started showing up, and also very pleased. That’s one of the things I like about research. Through some mysterious alchemy that I don’t completely understand, just ‘playing around’ can lead to exciting — and often unexpected — breakthroughs.

I am fortunate enough to be affiliated with a research consortium of brilliant and highly inventive individuals, the Communications Design Group (CDG). Recently we started talking about what our logo should be. Not surprisingly, lots of people jumped in with fun and innovative suggestions.

One of our colleagues, Patrick Dubroy, created quite a few designs, all of them wonderful. Each design emphasized a different positive aspect of our group. One design in particular, which he called “connections” (see below), highlighted the way everything we do is interconnected:

A number of us looked at this and had the same thought: that a design so visually evocative is practically begging to be interactive. So several people in the group tried their hand at making it so.

Here was my attempt to turn Patrick’s logo design into a fun toy that you can play with.

When I first saw Christopher Nolan’s Interstellar, I was a little non-plussed. It’s certainly a gorgeous film, a true space opera beautifully shot in the grand tradition.

But with its themes of five dimensional beings and alternate realities, of how the power of the universe is mystically connected to the power of personal feelings, it was all a little beyond my grasp.

But then this weekend I found the key. I came upon a cultural artifact from more than forty years ago, a long lost gem, a beacon of clarity interred (perhaps by design) deep within the vastness of YouTube.

I realized that this video which spoke to me — quite literally — must be a message, sent by wise and incomprehensibly advanced five dimensional creatures from our distant future, and planted into the recent past of our own time line, to elegantly underscore the connections between spaceflight, alternate realities, and personal alienation that were to later find their way into Interstellar.

I speak, of course, of this.

“You’re … you’re not supposed to be here.” Charlotte was truly appalled.

“Why not? If characters can talk about the book they are in, why can’t an author talk to his characters?”

“But if you’re here, then who is writing this? You can’t be in two places at once.”

“Of course I can. I can do whatever I want. This is fiction. I’m sorry if you are disturbed by any of this.”

“You seem very smug about it all,” Sheldon said. “Isn’t there anything that would disturb you?”

“I can’t see how,” said the author. “I mean, after all, this entire world is my creation.”

“Not according to the cat,” said Charlotte’s mom. “If your characters have free will, then you’re not really in charge. We are.”

“Nicely put,” said the cat.

“Thank you,” said Charlotte’s mom, looking very pleased with herself.

“That’s only part of it,” came a voice from the door.

“And who are you?” asked Charlotte.

“I’m the author, of course.”

Charlotte looked from one author to the other. They looked quite identical, except for the fact that the first author was looking very pale. “You didn’t expect this, did you?” she asked him.

“Not really. I guess this means…”

“It means,” said the cat, “that you’ve become a character in the book. So you are no longer the author.”

“But wait,” said Charlotte’s mom. “If entering the scene makes someone a character, then aren’t they both characters now?”

“Oh no,” said Charlotte’s dad. “Looks like we’re going to get an author infestation. It’s like the stateroom scene, but with recursion.”

“The stateroom?” Sheldon said quizzically. “Is that physics?”

“Marx brothers,” said Charlotte’s mom. “You really should pay more attention to the classics.”

Just then a third author showed up at the door. “That’s only part…”, he began, but when he noticed the first two authors, he lapsed into an embarrassed silence.

“Dear,” said Charlotte’s mother to her husband, “it looks as though you were right. I have a feeling it’s going to get awfully crowded in here. Is there nothing we can do?”

“Wait,” said Charlotte, “I think this is all happening because the book is in limbo. And I just remembered something.”

“What’s that?” Sheldon asked. Just then a fourth author walked in. Seeing the other three, he didn’t even bother trying to announce himself. The third author stepped aside politely to give him space. The room was beginning to become crowded.

“I remembered that I have the power to change the book.” She took the piece of paper out of her pocket, the one with the altered title page, and unfolded it. “Does anybody have a pen?”

“I do,” said the first author.

“Thanks,” said Charlotte. “I hope you don’t mind my messing with your work.”

The author chuckled, “It wouldn’t be the first time you’ve done that.”

“True enough.” Charlotte thought for a moment, and then wrote something down. Suddenly all the authors vanished from the room.

“I’m not a ghost anymore!” said Charlotte’s mom. “And neither are you dear,” she added, looking at her husband.

“Hey, I’m not either,” said Sheldon. “How did you do that?”

Charlotte shrugged. “I just found the right title. The rest of the book pretty much writes itself from there.”

“What’s the new title?”

Charlotte read from the page. “Sheldon, who is not a Ghost; A Love Story.”

Sheldon beamed. “That is a very good title. And looking at you right now, I find myself extremely happy that I am no longer a ghost.”

Charlotte blushed. “I quite agree. Nothing like being able to decide for yourself what story you are in. And I guess we really have the cat to thank. I don’t think I would have figured any of this out otherwise.”

“Say,” said Charlotte’s dad, “Where is that cat anyway?” They all looked around, but the cat was nowhere to be found.

“Oh, I’m pretty sure I know where we will be able to find her,” Charlotte laughed.

“Where?”

“In the sequel, of course.”

fin

“Sheldon,” said the cat, “you do know you’re a fictional character, don’t you?”

“Yes, I believe we’ve established that. And I can live with it. I mean, in the broad sense of the word ‘live’.”

“Then you must know that as a ghost you are serving as a metaphor.”

Sheldon rolled his eyes. “Oh great, now I’m a metaphor. You sure I can’t be a simile, or an allegory, or maybe a synechdoche?”

“Actually,” said Charlotte, “I think at this point Sheldon is more of a pataphor.”

“That’s going a bit far, pumpkin, don’t you think?” said Charlotte’s mother.

“Yes, exactly!” said Charlotte. “I’m glad you agree. In fact, this whole thing has gone too far — haven’t any of you noticed? What started as a discussion about our existence has somehow taken over our existence.”

“My head hurts,” said Charlotte’s dad. “Are you saying we’re trapped in this conversation?”

“It’s worse than that,” Charlotte said, shaking her head sadly. “I think we’ve become the conversation.”

“Well, I guess the cat’s out of the bag,” said Sheldon.

“Now Sheldon,” Charlotte admonished. “Lets stay away from catachresis.”

Charlotte’s father looked confused. “Are you saying the cat is having a crisis?”

“No dad, it’s a word that means … oh, never mind. It’s not important. What’s important is that Sheldon is stuck being a ghost because of some metaphorical imperative. That doesn’t seem very fair to me.”

“Are you unhappy being a ghost?” Charlotte’s mother asked Sheldon gently.

“Well, I can’t say it’s all that great. The whole thing about not being able to eat a meal — that’s a serious bummer right there.”

“And from what the cat has explained,” Charlotte said “this is all because Sheldon is stuck between two fictional universes.”

“That’s only part of it,” came a voice from the door.

They all turned to look. “Who are you?” asked Charlotte.

“I’m the author, of course.”

“First of all,” said the cat, addressing Charlotte’s mom, “I disagree with your basic premise.”

“Which premise?”

“The premise that a fictional character cannot be at fault. An author cannot simply dictate actions. Characters must be free to make good choices and bad choices. Otherwise, how can they learn and grow?”

“But wait,” Charlotte said, “isn’t it actually the author making those choices?”

“I’m surprised at you Charlotte,” said the cat, “You are the one who studies PolySocial Reality. The ‘you’ who interacts with me is not the same ‘you’ who interacts with your mother. So it is meaningless to speak of a single person, let alone a single author. PolySocial Reality exists just as much within us as between us. As Whitman said: ‘I am large. I contain multitudes.'”

“Or in other words,” Sheldon said, “it’s turtles all the way down.”

“Yes, exactly. It’s all described very well in Minsky’s ‘Society of Mind’. The reader recognizes within the author’s characters that same fragmentation and multiplicity of state that she sees within her own mind. Why else would she ever care what a character does, and why would she bother to keep reading?”

“Well, there’s plot!” Charlotte said. “I mean, Sheldon is a ghost. Wouldn’t a reader be a little curious about why?”

“I’m certainly curious about why,” Sheldon added. “And that has to count for something. If you’re such a smart cat, tell me why I am a ghost.”

“Of course it’s not your fault, pumpkin,” said Charlotte’s mom. “It couldn’t be your fault.”

“By definition,” added Charlotte’s dad.

“What do you mean, by definition?”

Charlotte’s parents looked at each other. “You see,” said Charlotte’s mom with an understanding smile, “Nothing that happens can ever really your fault, because you’re a fictional character.”

“You’re not writing all of this,” added Charlotte’s dad, “You are being written. We all are.”

“Yes,” Sheldon said, interrupting, “but who is writing us?”

“The writer,” Charlotte’s parents said in unison.

Charlotte laughed. “Sheldon, I guess you could have seen that one coming.”

“I suppose so,” he said, “but none of it seems very fair.”

“What’s not fair about it?”

“Well for one thing, even Philip K. Dick didn’t tell his characters they were in a fictional universe until the last page of the book. Whereas we just sort of jumped right into the deep end of the pool.”

“I think it’s all going swimmingly,” said the cat.

“Maybe it’s all to the good,” Charlotte said, ignoring the cat. “I mean, we’re not going to be able to figure out the whole story of why you’re a ghost, unless we confront the creator.”

“And you think playing ping pong against the fourth wall is how we’re going to do that?”

“Yeah, pretty much. Sooner or later we’re going to smash right through that fourth wall, and then we’ll see who is on the other side of the looking glass.”

“Mixing metaphors much?” asked the cat.

“Do you have a better theory about what to do?” Charlotte asked the cat.

“In fact,” said the cat, “I do.”