Month: December 2014

The other day a friend showed me the cool app on her phone that lets her monitor her beloved puppy when she is away from home. She has puppy-cams installed throughout her apartment, and she can see not only where he is and what he’s up to, but which bit of toy or furniture he may have been chewing up.

I told her that it reminded me of an academic paper I saw some years ago, that not only allowed an animal companion’s activities and location to be tracked remotely, but also create new activities. For example, you could initiate a game of fetch with the press of a computer key.

And I remembered that when I had first seen that paper presented, I had thought it a bit one-sided. After all, why should the humans have all the fun?

Don’t you it might be cool to create an app that can be used by a dog or a cat? Fido could watch the location of his beloved human throughout the day, as that human goes from one adorable human activity to another. Princess could initiate fun activities for her human friend at any moment of the day.

Which leads to the following question: If you were a dog or a cat, what fun activity would you want to start your adorable human doing, at the touch of a paw?

Many of us who find ourselves programming in HTML5 — basically using Javascript to create dynamic or otherwise interestingly interactive web pages — start out with something very simple. By “simple”, I mean that all of our Javascript code (other than code already built into the browser) fits right in the web page itself.

Eventually this wonderfully minimal approach starts to get unwieldy. As you develop your program, the code in that one file gets longer and longer, and more of a potential mess. Eventually it starts to make sense to split up your code into multiple files, just as in the physical world you might hang your clothes in the closet or place your dishes in the cupboard.

But on some level I think you never give up on that wonderful moment — when you wrote your first simple and elegant program that fits entirely into a single HTML file. Sort of the web programming equivalent of a haiku.

I just saw Inherent Vice, the new film from P.T. Anderson based on the Pynchon novel. Hands down the best new release, in my opinion, since Michael Haneke’s Amour came out two years ago. I wouldn’t be surprised if Joaquin Phoenix wins an Oscar for his astonishing performance.

But one thing kind of unsettled me. Owen Wilson played a key role in this film, and like everyone else in the cast, he was excellent. Except whenever I see Owen Wilson on screen, I tend to think of the other Anderson.

Just as Robert De Niro was muse to Martin Scorsese, and John Wayne to Howard Hawks, Owen Wilson has long been the on-screen avatar for Wes Anderson. So as I watched, part of me kept waiting for Bill Murray or Jason Schwartzman to show up.

Oh well, it could have been worse. If Malcolm McDowell had appeared in the movie, maybe I would have started thinking about Lindsay Anderson.

Suppose, through some future technology, everyone could dial in their mood, deciding how they feel at any given moment of the day. What would such a world be like?

If you need to focus for an exam, or pay attention in class, or maintain your poise during that potentially nerve-wracking job interview, just choose the appropriate settings. If later that evening you want to get frisky with your partner, no problem — you can both always be in the mood if you want to.

If you have a fear of flying, you can set yourself, pre-flight, to Zen-like calm. Or maybe you want the high that can come from drinking, without its debilitating side-effects. Just set your mood knob to “elation”.

Of course there would be limitations. The human brain has evolved built-in mechanisms at a very low level that compensate for any sustained deviation from the norm.

For example, if you try to set your mood knob to ecstatic for too many hours at a stretch, your brain will work progressively harder and harder to bring you back down. You could eventually crash, and go into a depressive tailspin.

But let’s say that these future technologies come with sensible brakes, so they don’t allow you to tweak your mood in ways that would actually cause you harm. Sort of the way an elevator will bring you up or down on command, but won’t let you crash through the roof or the floor.

Would people who have access to such a technology be fundamentally changed by it?

This week I sat in on some really wonderful thesis presentations by Masters students in the NYU Music Technology program. The sheer amount of intellectual energy and inventiveness on display was very inspiring.

One trend I noticed was that some students would start with a traditional acoustic instrument they really know and love, such as a trumpet or a classical guitar, and convert it into a musical controller. The result would not make any direct use of the sounds that the instrument produces naturally.

Rather, the student would use those sounds as data, to be input into a computer synthesizer. A new sound would then be computed — perhaps one that could only be created with computer assistance. The result, however radically different, would retain a subtlety and expressiveness that is characteristic of the original musical instrument.

This general approach reminds me of other recent trends in computer mediated performance. For example, there are similarities to the way Andy Serkis “performed” his own body in the Lord of the Rings films, to create a digital Gollum. You never saw the actor himself, but only the computer-transformed result of his performance. Essentially, he was using technology to puppeteer his own body.

In a sense, an artist’s use of any tool — from the paintbrush to the piano — is a kind of puppetry. And as computers continue to become more powerful, new kinds of puppetry will continue to emerge, allowing us to use our brains and bodies to create ever more powerful forms of aesthetic expression.

After all, what is the piano, but a complex industrial innovation, enabled by advanced technology, which interposes itself artificially between musician and nature, in order to give the performer a greater power of expression?

The lyrics to the wildly popular theme song “Let It Go” from the recent Disney film Frozen have puzzled many people. What exactly is the song trying to say?

It occurs to me that one valid way to interpret it is as an avowal of Buddhist tathagatagarbha. Specifically, the song expresses a mindful path to contentment based upon acceptance of what is, rather than on a craving for what is not.

Excerpting lyrics from the song, we uncover something that sounds very much like a succession of Zen koans:

snow glows white on the mountain
a kingdom of isolation
distance
makes everything seem small
and the fears that once controlled me
can’t get to me at all
no right, no wrong
the past is in the past
let it go, let it go
i am one with the wind and sky
here i stand
in the light of day
let the storm rage on
the cold never bothered me anyway
 

That’s pretty cool. Also the fact that this is the very first Disney song in history with lyrics that include the word “fractal”.

The tragic stories keep piling on, one atop the other. One that has recently cycled back to public consciousness was the teenage boy shot to death by a cop — in the kid’s mom’s bathroom.

This was in 2012 right here in New York City. The police department says they thought the young man might have a gun, and that he was running from the police. But surveillance video shows the teenager calmly walking to and entering his mother’s apartment building.

A group of cops then ran into the building, guns drawn, without a warrant. When they burst into his mother’s bathroom, they found the teen trying to dump some marijuana down the toilet.

One cop, thinking the teenager might be reaching for a gun, shot him dead in the chest where he stood, while his six year old brother and grandmother looked on. Afterward, the cops took his grandmother down to the precinct for seven hours of questioning, not letting her lawyer talk to her for about two hours.

No gun was ever found.

What I have been trying to imagine is how this scene would have played out if the kid had been white. Would the police officer have shot the blue eyed youngster right in front of his tousle headed six year old brother, and his sweet old white grandma? Would the police department then have interrogated the old lady, who wasn’t charged with any crime, for seven hours straight, right after she’d just seen her grandson shot dead?

I’m trying to imagine whether that policeman would have pulled the trigger — whether he would have been emotionally capable of pulling that trigger. I’m trying to imagine whether, if the kid had been white, such a scenario — such a now horribly familiar scenario — could ever ever happen, even once, in these United States of America.

And I cannot.

A few days ago I had the pleasure of attending the NYU ITP New Instruments for Musical Expression event, in which students perform original music on novel electronic musical instruments that they themselves have invented.

While there was a tremendous variety among the various instruments, one theme in particular seemed to emerge: In the hallowed tradition of the Theremin — perhaps the very first electronic musical instrument, invented in the 1920s by Léon Theremin — many of the students designed their instruments around the idea of a single large sliding control.

In Theremin’s original instrument, the performer’s hand forms part of an oscillating capacitive circuit. As the performer makes subtle adjustments to her hand position, the frequency of oscillation changes over time. The Theremin then transforms this human-modulated signal into an audible musical tone.

The Theremin can make beautiful sounds when properly played, but it is notoriously difficult to operate. Yet its basic mode of operation is easy to adopt: Move your hand position to modulate an electronically generated musical tone.

On the night of the show, I saw students doing this in various ways. One instrument used a single long resistive strip, another contained an infrared optical distance measuring circuit, yet another relied on an ultrasonic distance detector.

In every case, the fundamental act of playing the instrument consisted of moving one’s hand through space. To make music, the musician/inventor would modulate pitch by changing the position of his/her hand over time.

Despite the fact that they used different underlying mechanisms, all of these instruments reminded me a bit of the Theremin. Each was built around the same fundamental idea of forming a melody by moving one’s hand in space to vary pitch. Consequently, the music they made all shared a certain lovely simplicity that I found quite charming.

At some point I turned to the person next to me, and said “I think we are hearing a new musical genre.”

“What’s that?” he asked.

I was ready with my shiny new word. “Thereminimalism!”

Fast forward to that hypothetical time in the not-too-distant future when everyone is “wearing”. So rather than seeing the world through our naked eyes, we will all be able to see, through our future contact lenses or implants, some computer-mediated transformation of that world.

The extent to which this visually transformed world differs from the literal world will ultimately not be a technological question, but rather a question that centers on individual and collective values, as we have discussed here in earlier posts.

When such transformations become possible, you will be able to “dial in” a preferred age to show the world. For example, someone in their forties can choose to appear as their twenty-something self in a party situation, and then revert back to a truer appearance to take a business meeting, if that is desired.

You should also be able to project forward, running plausible simulations of what you might look like in ten or twenty years, and then choose, at times, to show that face to the world.

It will also work the other way: When you talk to a person in their seventies, you might opt, for whatever reason, to see them as they looked when they were twenty or thirty.

It’s not clear to me what this capability implies from a social, cultural or ethical perspective. But it might be worth thinking about.

Our various human senses do a very good job of complementing each other. Our sense of sight provides enormous bandwidth, while our hearing allows us to detect and locate events all around us. Touch lets us accurately assess the texture and solidity of objects, and permits us to use our amazing fingers and hands to manipulate tools.

One difference between our respective senses is in the scale of time at which each operates. For example, to simulate our sense of vision accurately, you need to flash about 100 different images in front of the eyes every second. Of course, that’s just a rough approximation, but it’s reasonably accurate. For example, 30 images per second isn’t quite enough, whereas 300 images per second would be overkill.

In contrast, to simulate our sense of touch you need to provide about 1000 changes in haptic sensation per second. If you only provide 300, then the things you are simulating will always feel spongy and soft, whereas 3000 haptic sensations per second feels pretty much the same as 1000.

To create a perceptually perfect simulation of sound, you need to go up to about 100,000 vibrations per second. If you try to get away with 30,000, you end up losing phase information at the high frequencies, which makes some objects sound like they aren’t quite coming from the right location. On the other hand, anything above 100,000 vibrations per second would pretty much be pointless.

I find it interesting that we’ve got these three nice round numbers: 100, 1000 and 100,000, for visual, haptic and auditory “samples per second”, respectively.

So what happened to 10000? We seem to have skipped right over that one. I’m pretty sure we won’t find it in our sense of smell, since smell works at a very slow time scale, compared with most of our other senses.

Maybe in the future, after we have figured out a practical way to interface directly to the brain, we will discover that some high-tech version of mind reading requires 10000 neural samples per second.

But it might be a while before we know whether that is true or not. By my reckoning, we will probably need to wait about seven trillion neural samples, more or less, before we find out.