Ken's Blog

I saw a great talk this evening by Eero Simoncelli, who gave a broad overview of the research going on in his computational neuroscience lab at NYU. He started with a diagram showing the overlap between three fields: Perception, Engineering and Physiology. The first is the science of what exactly we are able to see, hear, taste, etc. The second helps us create practical tools based on those perceptions. The third dives into the brain itself to understand the mechanism inside us that causes perception to happen the way it does.

The key to studying perception is finding metamers — two or more things that seem the same to our senses even though we know they are different. The classic example of this is how we see colors. In 1853 the great mathematician Hermann Grassmann, working in the field of Perception, showed that everything we see, no matter how rich its color spectrum, is reduced in our perception to just red, green and blue — three colors. Which means that humans can’t see the difference between many objects in the world which actually have wildly different spectra. In a sense, we all suffer from metameric blindness.

By 1931 engineers had codified this knowledge into the CIE color standards that underlie the technology of modern movies, photography, television and computer displays. Yet it wasn’t until 1987 — more a century after Grassmann’s perceptual results — that physiologists in D.A. Baylor’s lab at Stanford were finally able to measure the behavior of a single color receptor cell, showing the actual mechanism inside our bodies that makes all this happen.

It had taken 134 years to go from knowing exactly what our brain is doing, to knowing exactly how our brain is doing it.

This was a great story, but it nearly got waylaid right at the beginning, when a prof in the audience who studies Physiology objected to putting all three fields on an equal footing. “After all,” he said, “Perception is completely subsumed by Physiology”.

Eero said the comment reminded him of that famous New Yorker magazine cover A New Yorker’s View of the World, a cartoon showing that to a New Yorker, the rest of the world seems small and insignificant. Which may or may not be true, but now we know that if you’re Physiologist, the rest of science seems small and insignificant.

After the talk I complimented Eero on a great presentation, and I told him it could also go the other way: Maybe Physiology is completely subsumed by Perception. After all, without our perception we wouldn’t be able to study anybody’s physiology.

Eero replied, sensibly enough, that both fields are subsumed by Physics.

I was about to point out that to a Creationist, Physics would be subsumed by Engineering. But then I thought better of it.

There is something thrilling about live performance. No matter how many hundreds of millions of dollars have gone into the production of a feature film, a movie can never replicate the delicious immediacy of a live performer on stage.

You know, when you are attending a performance, be it a play, opera, music concert, magic act or circus, that this is really happening, right here and now. And also that things might unexpectedly go wrong — or spectacularly right. When you see a movie, you know going in that no such real time uncertainties are possible, unless maybe the projector breaks down.

But a performance is also not quite the same as a conversation. Our real lives are largely unscripted. Dinner with a friend is very much a real time improvisation, whereas the person you see up on the stage is saying words what were worried over by William Shakespeare, or an arrangement of notes carefully composed by Ludwig van Beethoven.

A live performance is, in a sense, a supercharged now, an experience that takes place in the real-time present, yet is infused by non-real-time inventions created in the past.

There is nothing inherently better or worse about live performance. It’s just different. While a movie can offer a kind of perfection — each shot carefully worked over and edited until a precise effect is achieved — a play offers us the messiness of still-open possibility.

But even that is sometimes not enough. This evening a friend and I had planned to see a movie together. But in the end we opted for reality, with no supercharging at all. We went to a nearby restaurant to share a meal and great conversation. Nothing scripted, only the pure improvisation of living in the moment.

After all, what can be more thrilling than life itself?

A friend and I were talking about Shakespeare. I’m a fan, he’s not. He was wondering aloud whether the hyper around the Bard could ever be tested objectively. “Maybe,” he said, “people only like Shakespeare because they know it’s Shakespeare. I wonder whether they’d still like the plays if they’d never heard of him.”

I pointed out that Shakespeare had a number of contemporaries who wrote on many of the same themes, in the same iambic pentameter, with all the same conventions. And yet we don’t watch many of their plays. Just as we don’t listen to all that much music by J.S. Bach’s contemporaries. Maybe, I posited, Shakespeare was just so good that his work transcends its own time.

“Too bad,” my friend said, “that there’s no way to test that hypothesis.”

“Christopher Marlowe was a contemporary of Shakespeare,” I told him. “Do you think you would know whether you’re watching a play by Shakespeare or by Marlowe?”

“No,” he replied, “I don’t think I’ve seen enough Shakespeare that I would know which was which.”

“I’m guessing,” I said, “that most people would be in the same boat as you. So I think you’d have a sample to work with.”

My friend perked right up. “Yes,” he said, “I think we could get enough people to do a proper blind comparison.”

I was glad to hear that we had a plan. I wonder how it will turn out. Maybe next we can try the same thing with Mozart and Salieri.

An article in today’s New York Times reported a research result that has been circling recently: Researchers believe they have figured out why we sleep. In scientific circles this has long been a puzzlement, given that for much of our recent evolutionary history, sleep would seem to be the worst of all possible Darwinian survival traits. After all, when you’re passed out and snoring on the veldt, you’re pretty much prime snackage for the nearest saber toothed tiger.

So what evolutionary advantage is conferred by those mysterious eight or so hours of thanatomorphism? The latest empirical evidence indicates that sleep is the occasion for flushing out the dead cells and accreted chemical poisons that we generate in the course of a day’s thinking. Night time, my friends, seems to be the right time for Mother Nature to dump our mental chamber pot.

But what if they have it all wrong? I mean, the major impetus for this research is the burning mystery of why we sleep at all, and why such odd behavior didn’t make us all die out thirty five centuries ago in equatorial Africa.

Let’s turn it around. What is the singular trait of human beings? It is our oddly cerebral way of dealing with the world. Other species have all sorts of intelligence, but we seem to be the only ones who survive by making elaborate and logistically intricate plans.

But as we now know, intelligence comes in many forms, and most of it is not quite at the conscious level. Haven’t you ever been stuck on a crossword puzzle, and then you woke up the next morning and simply knew the answers?

Maybe when we dream, we’re a lot smarter than we are when we are awake. Maybe Slumberland is where we actually solve the problems that need solving in order for us to survive. Perhaps the major survival function of our conscious selves is to take credit for solutions we’ve already worked out in our sleep, and for which our conscious mind then basks in undeserved applause.

“But I usually can’t remember my dreams!” you object. Of course you can’t. That symphony you wrote last night when you were dead asleep, that triple integral you solved, that sublime spiritual awakening to the vast and perfectly connected oneness of the Universe, you can’t remember these things because you’re not smart enough. Your waking brain simply doesn’t have the capacity.

Don’t worry about it. Go and get some sleep.

Have you ever noticed the emotional power of song lyrics that don’t quite make sense if you just read the words?

A certain level of crypticness seems to actually make some songs more emotionally effective, in a way that would never be true for prose. There is much obscurity in the lyrics of many amazingly powerful songs, like Nirvana’s “Smells like Teen Spirit”, Cat Power’s “Lived in Bars”, Tom Waits’ “Time”, or almost any song from Dylan’s golden period. And then of course there’s “Stairway to Heaven”.

For many of the songs we love, a simple recitation of the words might elicit a confused shrug. Yet in combination with that particular music, the result can be overwhelmingly moving. So what exactly is going on here?

Perhaps the very obscurity of these lyrics, when combined with the emotional sense-making of the music, acts as a trigger for our own mind to add the missing pieces. By keeping explicit meaning hidden, the songwriter creates a kind of grandly protean canvas upon which each of us can project our own inner emotional landscape.

Of course, that’s just one theory.

There comes a point in every project when the project starts to assume its true form. Until then you are mostly going on instinct, feeling your way along as you push up hill, because the thing isn’t quite right yet.

But then at some point things turn a corner. The unfortunate decisions you’d made at first start to fall away. Some of those were the decisions that got you thinking about the project in the first place, before you quite knew where it was headed. Now they’re just holding things back.

Right about now I’m at such a turning point. The first weeks of the project I’m working on were mostly spent building tools, trying things out, demoing to friends and colleagues and listening to their criticisms.

One thing that’s great about those demos is that you often hear the same criticisms from different people — people who don’t even know each other. And usually they are about features that you’d fallen in love with.

How easily we are blinded by love for our own creations. That is why we often need other people to point out when they are just in the way. As Faulkner wisely said, “Kill your darlings.”

This evening I attended a talk by one of my favorite writers, George Saunders. In the course of a wide ranging conversation with the audience, he mentioned something a writer gets to do that we can’t normally do in real life: See things from inside somebody else’s head.

In particular, the writer can give us a privileged view of the minds of people we would not at all like in real life. In fact, we might go out of our way to them. But when we encounter such a prickly soul as a literary character, we find ourselves inside his head, and therefore cannot help but understand how he feels. And so we like him.

To do the same thing as an actor requires an analogous kind of skill. Think about the actors and actresses who have made you like a character you would have found repulsive in real life. These actors give us a window into the human soul that the average leading man or leading lady could never provide.

These days I am working intensely on a software project. When I get into this mode, I have a particular, and somewhat eccentric, way of working.

Rather than use a software version control program (like a normal person would), every day I copy the entire software base, all the lines of code, into a new folder. So I end up with a trail of folders, each somewhat different from the last, and each named for the date I was working on it.

This is not nearly as flexible as version control software. For example, if I get stuck, I can’t simply “roll back” to what I was doing an hour before. And yet I find it very satisfying. Recently I’ve started to understand why.

A day is a very natural marker of progress. In the morning the Sun goes up, in the evening it goes down, and that day will never come again. For example, when you take a cross-country journey, you tend to think of the places you visited on any given day. In a sense, that city belongs to that day. If one day you found yourself in some fascinating city, then that day becomes important in your memory.

And I’ve come to realize that by giving each day its own little narrative, I am somehow putting the days into a kind of competition with each other. On Wednesday I added this cool feature. Then on Friday I fixed that nasty bug.

I realize it’s not very logical, but it works. By asking each day to bring me something new or exciting, I think I’m getting a lot more done.

I went to a 3D movie recently where the person operating the project was clearly unfamiliar with the technology. This was apparent in several ways. For one thing, for the first ten minutes of the movie, there was no polarizing filter over the projector.

When this happens, you always see two images whether or not you are wearing the 3D glasses. In fact, until the projectionist puts the filter on, there is no point in putting on the glasses. You might as well enjoy the much brighter view you get by watching the movie without the glasses.

Because I am a basically positive person, I would like to think that at least some of the very high ticket price (3D movies are expensive!) will go to remedial training for this hapless projectionist.

After about ten minutes, someone in the projection booth finally put the filter on the projector. So I put on my 3D glasses, and then everything started to look really funny. Near things looked far away, and far things looked near. The filter was on backwards!

Since I know a little bit about 3D movie technology, I know that the two images are each polarized at 45^o — the left image slants to the right, and the right image slants to the left.

So I figured that if the glasses were turned upside down, its two filters would each slant the other way, and everything would look ok. I tried it, wearing the glasses upside down, and suddenly the 3D movie looked just fine. I’m sure I looked silly wearing upside down 3D glasses, but fortunately movie theaters are dark.

I told the friend I came with to flip her glasses upside down. Then we told our neighbors in all directions, and suggested they spread the word. Eventually this inverted fashion spread throughout much of the theater, and many of us watched the movie with our 3D glasses turned upside down.

After about half an hour, somebody in the projection booth figured out that there was a problem. They took out the filter, flipped it around, and put it into the projection path the right way. So I turned my 3D glasses right side up and kept watching.

But here’s the weird thing: Most of the people around me never turned their 3D glasses right side up again. They just continued to watch the rest of the movie the wrong way.

What on earth were they thinking?

I was having a conversation today with a colleague about a particular tension in technical advancement of the arts. In particular, we were discussing distribution of two different kinds of talent.

There are people who are incredibly good at inventing, and other people who are incredibly good at performing. But rare is the person who can do both with world class skill.

For example, the person who invents a new kind of movie camera is probably not going to have the skill of a Spielberg, and the inventor of a better guitar is not likely to be able to play like Segovia.

The result is a kind of lag in innovation: A new tool first needs to fall into the hands of the best practitioners. Only then can the experience of those practitioners inform further innovation.

Occasionally the same individual is able to evolve something technically while also bringing best practices to the medium. For example, Bach was inventing new musical forms even as he was using them to compose work of incomparable genius.

But Bach was the rare exception. Alas, for the most part there seems to be a law of conservation of talent at work.