Author: admin

A few days ago I had the pleasure of attending the NYU ITP New Instruments for Musical Expression event, in which students perform original music on novel electronic musical instruments that they themselves have invented.

While there was a tremendous variety among the various instruments, one theme in particular seemed to emerge: In the hallowed tradition of the Theremin — perhaps the very first electronic musical instrument, invented in the 1920s by Léon Theremin — many of the students designed their instruments around the idea of a single large sliding control.

In Theremin’s original instrument, the performer’s hand forms part of an oscillating capacitive circuit. As the performer makes subtle adjustments to her hand position, the frequency of oscillation changes over time. The Theremin then transforms this human-modulated signal into an audible musical tone.

The Theremin can make beautiful sounds when properly played, but it is notoriously difficult to operate. Yet its basic mode of operation is easy to adopt: Move your hand position to modulate an electronically generated musical tone.

On the night of the show, I saw students doing this in various ways. One instrument used a single long resistive strip, another contained an infrared optical distance measuring circuit, yet another relied on an ultrasonic distance detector.

In every case, the fundamental act of playing the instrument consisted of moving one’s hand through space. To make music, the musician/inventor would modulate pitch by changing the position of his/her hand over time.

Despite the fact that they used different underlying mechanisms, all of these instruments reminded me a bit of the Theremin. Each was built around the same fundamental idea of forming a melody by moving one’s hand in space to vary pitch. Consequently, the music they made all shared a certain lovely simplicity that I found quite charming.

At some point I turned to the person next to me, and said “I think we are hearing a new musical genre.”

“What’s that?” he asked.

I was ready with my shiny new word. “Thereminimalism!”

Fast forward to that hypothetical time in the not-too-distant future when everyone is “wearing”. So rather than seeing the world through our naked eyes, we will all be able to see, through our future contact lenses or implants, some computer-mediated transformation of that world.

The extent to which this visually transformed world differs from the literal world will ultimately not be a technological question, but rather a question that centers on individual and collective values, as we have discussed here in earlier posts.

When such transformations become possible, you will be able to “dial in” a preferred age to show the world. For example, someone in their forties can choose to appear as their twenty-something self in a party situation, and then revert back to a truer appearance to take a business meeting, if that is desired.

You should also be able to project forward, running plausible simulations of what you might look like in ten or twenty years, and then choose, at times, to show that face to the world.

It will also work the other way: When you talk to a person in their seventies, you might opt, for whatever reason, to see them as they looked when they were twenty or thirty.

It’s not clear to me what this capability implies from a social, cultural or ethical perspective. But it might be worth thinking about.

Our various human senses do a very good job of complementing each other. Our sense of sight provides enormous bandwidth, while our hearing allows us to detect and locate events all around us. Touch lets us accurately assess the texture and solidity of objects, and permits us to use our amazing fingers and hands to manipulate tools.

One difference between our respective senses is in the scale of time at which each operates. For example, to simulate our sense of vision accurately, you need to flash about 100 different images in front of the eyes every second. Of course, that’s just a rough approximation, but it’s reasonably accurate. For example, 30 images per second isn’t quite enough, whereas 300 images per second would be overkill.

In contrast, to simulate our sense of touch you need to provide about 1000 changes in haptic sensation per second. If you only provide 300, then the things you are simulating will always feel spongy and soft, whereas 3000 haptic sensations per second feels pretty much the same as 1000.

To create a perceptually perfect simulation of sound, you need to go up to about 100,000 vibrations per second. If you try to get away with 30,000, you end up losing phase information at the high frequencies, which makes some objects sound like they aren’t quite coming from the right location. On the other hand, anything above 100,000 vibrations per second would pretty much be pointless.

I find it interesting that we’ve got these three nice round numbers: 100, 1000 and 100,000, for visual, haptic and auditory “samples per second”, respectively.

So what happened to 10000? We seem to have skipped right over that one. I’m pretty sure we won’t find it in our sense of smell, since smell works at a very slow time scale, compared with most of our other senses.

Maybe in the future, after we have figured out a practical way to interface directly to the brain, we will discover that some high-tech version of mind reading requires 10000 neural samples per second.

But it might be a while before we know whether that is true or not. By my reckoning, we will probably need to wait about seven trillion neural samples, more or less, before we find out.

It was not too long ago in our nation’s history when something like the following would have been inconceivable:

https://www.youtube.com/watch?v=95KTrtzOY-g

For reasons that I have not yet fully worked out, being in a nation where stuff like this can happen makes me incredibly happy.

It occurs to me that the “Three famous people” game that I described yesterday can be played the other way. Rather than the challenge being to find the answer, the challenge can be to come up with a good question.

It can be surprisingly difficult to find two famous people with exactly *one* other famous person obviously connecting them.

For example, yesterday Sharon suggested “Tom Hanks” and “Neil Young”. I thought she meant Jonathan Demme, since he connects them through the film “Philadelphia”. But it turns out she meant Daryl Hannah, which definitely works as a good answer at the moment.

So maybe we can design a kind of crowd sourcing game: Contestants post two famous names, and various people out there on the internet try to guess who is the third famous person clearly connected to both of them.

Your goal as a contestant is to post two names that will result in nearly everybody guessing the same third name. The greater the unanimity of response, the higher your score.

By the way, we should probably ban the use of famous people who strongly evoke one person in particular (e.g.: Stan Laurel).

Come to think of it, I wonder whether we can use this general method for turning around any guessing game: Given any given guessing game, use the Crowd to create another game, one which measures the quality not of the answers, but of the questions.

There are some pairs of people who are so indelibly tied together that if I name one, you’ll probably think of the other. We could even make a game of it. For example, if I say “Spencer Tracy” — assuming you know who Spencer Tracy was — you would immediately say “Katherine Hepburn”.

But some very famous people are not clearly tied to just one person. For example, if I say “John Lennon”, do you say “Paul McCartney” or “Yoko Ono”?

So it might be interesting to make the following variant of this game: I name two famous people, and you need to figure out what famous third person is associated with both of them.

Let’s give some examples. In each case, you need to name a third famous person that both have in common:

(1) Bill Clinton and Tommy Lee Jones
(2) Peter Bogdanovich and Bruce Willis
(3) Heath Ledger and Taylor Swift

I’m guessing that many people will be solve one or another of the above, but most people won’t be able to solve all three. ^†

I suspect that if you design this game properly, you can learn quite a lot about a person by which answers they get right and which they don’t.

Can you come up with interesting examples for the “Three famous people” game?

† To clarify, by “solve” I mean that you don’t need to check your answer with a search engine, because you know for sure. 🙂

You can send a sample of your DNA to ancestry.com or one of its competitors, and they will tell you where your forebears came from, your ethnic mix going back many generations, and all sorts of things about your family tree that might end up surprising you. Clearly the people who subscribe to such services would like to know the truth about themselves and where their genes came from.

Yet when it comes to other things, we might not be so eager to know the truth about ourselves. I’m thinking in particular about our attitudes about race, gender and other groups of people that society likes to lump together.

As I watch the various responses to the tragedies in Ferguson and on Staten Island, I think that one of the tricky aspects of trying to discuss all of this is that most people don’t realize that they themselves harbor prejudices. Almost anybody you talk to will assure you that they themselves are not prejudice. Yet they are perfectly comfortable with the thought that a very large number of other people are indeed prejudiced.

Suppose we had the equivalent of genetic testing, but one that gives you a map of your prejudices, racial or otherwise, that you yourself never knew you were harboring.

I would argue that a conversation in the U.S. about race could only be helped by such self-knowledge. After all, there is no shame in harboring prejudices. All of us have irrational emotional responses to things, and many of those responses stem from times in our early childhood when we had no control over whatever nutty ideas our all-powerful parents may have been feeding us.

The real question is what you actually do and say, how you treat and speak about others, whatever your inner demons may be. It can only help to learn about prejudices in your own soul that you never knew you had. To speak truly about anything, you must first know the truth about yourself.

Now that the BBC miniseries Black Mirror is available on streaming Netflix, a lot of us are getting to see it for the first time. I had been hearing for months about the third episode, “The Entire History of You”, and it truly was worth the wait.

For my money it’s an incredibly well written story. And unlike much of what you see on American television, not everything is spelled out in a patronizingly explicit way. The more I thought about the characters afterward, and what was left implied but never said about their relationship history, the more perfectly it all came together.

Spoiler alert: Stop here if you want to know nothing at all about the episode before seeing it, although I’m not going to give anything away that you won’t find out in the first few minutes of the show.

When you and I have a conversation, I can choose what to tell you and what not to tell you. But more than that, I can skew the truth in artful ways, usually to avoid saying something that might be hurtful to you or someone else. “The Entire History of You” raises the question of what human relationships would be like if everyone had total recall. So one thing that strikes me about this question is how much it highlights the importance of ambiguity.

Not only is our entire social existence predicated on a delicate dance around truth, but natural language itself supports this dance very well. There is a consensus among linguists that one of the richest features of natural language is its powerful ability to modulate degrees of ambiguity of meaning. Note, by the way, that this is exactly the opposite of what we generally want from computer programming languages.

So if everyone had perfect total recall, we would be faced with a socially impossible situation: The very way that we have evolved as a species to think and to communicate with each other, to be able to co-exist with each other as humans, would no longer work.

After all, happiness is nothing more than good health and a bad memory. And if I had total recall, I could tell you who first said that. 🙂

I’ve noticed, in recent discussions with my students about the potential future of virtual reality, that the “Matrix” question comes up rather frequently.

I mean, the question of whether there is any way to know if you are experiencing an excellent computer simulation of reality, rather than reality itself.

In the original film, Neo was given the choice of taking a blue pill or a red pill. If you take the blue pill, then you remain blissfully aware that you are living within an illusion. But if you take the red pill, then you end up waking up to the reality outside the emulation.

Which leads to the following question: It you suspect you are in the Matrix, and you really really want to take the red pill, what would be your best strategy to figure out whether the world you see around you is just a simulation?

I suspect the answer would have something to do with the topics I discussed yesterday. Some things are much much harder to emulate than others, so your best bet might be to figure out what is the most computationally expensive thing to emulate, and then test for flaws in that.

Then again, if there is an A.I. agent monitoring your experience, intent on keeping you on a blue pill diet, then it can simply warp your perception of whatever experiment you try to perform, thereby maintaining the illusion of a perfect emulation.

So maybe you would need to design an experiment that takes such an A.I. agent into account. Which might not be so easy. 🙂

I was having a spirited debate with a colleague about where the brain ends. In particular, we were discussing where you can say “this part is the human body”, and “this part is the human brain.”

My colleague was arguing for an inclusive definition — the body is a simulator of itself, and the brain cannot function without the feedback from that body. One example he gave was the chemical activity in our gastric system. When we’re nervous or agitated, the nerves to our brain receive signals from all that chemical activity.

Eventually we both clarified that what we were really discussing was the question of what parts of the body, if any, could be effectively emulated — replaced by a simulation — without adversely affecting our brain’s ability to process the resulting data. We both agreed that it probably wouldn’t be too difficult to computationally simulate the chemical reactions in the stomach that trigger signals in our nervous system when we are agitated or upset.

But we disagreed about the nerve pathways that run from the body up into the brain itself. I felt that it would be extremely difficult to replace those pathways, and the various sorts of processing that goes on along them, by computer emulation.

Thinking back on the conversation, it occurs to me that we might really have been discussing Moore’s Law. Assuming computers continue to get twice as fast every eighteen months, when will it become feasible to emulate the signals from the stomach to the enteric nervous system? And then at what later date will we be able to emulate the parasympathetic nervous system? And then the entire autonomic nervous system?

At what point will we be able to emulate the cerebellum? The optic nerve? Specific language centers in our brain?

We could map out a rough timeline, given that Moore’s Law holds, when it could be possible to replace more and more of our experience of reality with a computer emulation.

I’m not saying that this would be a good thing to do. I’m just saying that at some point in the future, parts of it will become possible to do. And whatever side of the ethical debate you are on, you might be well advised to know where the important points are along that timeline.

I suspect that there are key central functions of your brain — including higher level cognition that we associate with our conscious sense of self — which would be far off to the right on that emulation timeline, perhaps several hundred years beyond your lifetime.

Unless, of course, the other parts of you become emulated well enough. In that case, your conscious mind might still be around when it becomes possible to emulate your entire brain. I wonder what your opinion will be then.