Speaking to computers

In 1966 the computer in Star Trek responded to spoken commands from Captain Kirk and his crew. At this time this was considered wonderfully futuristic, and seemed like something we might see in the 23rd century.

Now of course we are living in a world where computers respond to speech. My little Google Pixel Phone — and the Cloud with which it communicates — is able to turn my speech to text, and in many cases is also able act upon the result.

So have we achieved the sine qua non of human to computer communication? Is this the interface to end all interfaces, this ability to simply tell computers what we would like them to do?

I don’t really know the answer. I am just throwing the question out there.

Maybe I should just ask Alexa.

Short talks are harder

Tomorrow morning I am supposed to give a four minute talk. And that seems hard.

I’m used to giving hour long talks, or 30 minute talks, or even 10 minute talks. But there is something about a four minute talk that brings things to a whole other level.

When you only have four minutes, every moment counts. I like showing live demos in my talks, but in that amount of time, your live demo needs to work right on cue, without fail.

For one thing, you won’t have time to try your demo again. For another thing, if it doesn’t work, you’ve disrupted the rhythm of the entire experience.

On the other hand, if you manage to get a four minute talk right, it’s wonderful thing. When every second counts, what you say can have a lot of impact.

I’m reminded of Blaise Pascal’s famous apology: “Je n’ai fait celle-ci plus longue que parce que je n’ai pas eu le loisir de la faire plus courte.”**

** I have made this longer than usual because I have not had time to make it shorter.

A day in Kingston

I am spending the day with friends up at their farm in Kingston NY. It is as different from Manhattan as you can imagine.

Today is a cool rainy day of conversation, walking in the countryside, a few inspired games of ping pong, and picking salad fresh from the garden.

Also several varieties of locally sourced wine. Maybe a game chess as well, but only after a few glasses of that wine. 🙂

Novel forms of virtual reality

Today we hosted the external advisors to our big collaborative research consortium at NYU. Six of us on the NYU faculty, together with our awesome grad students, showed the various VR related research projects we’ve been working on.

Afterward, our advisors gave their feedback. And at that point I began to see a rift in what different people mean by “virtual reality”.

All of our advisors are legendary in their fields, and all have deep insight that comes from many decades of research. Yet they disagreed with each other on the basic premise of VR on a surprisingly fundamental level.

Some of our advisors clearly see the goal of VR as involving the literal re-creation of sensory reality. Their criterion for “success” can roughly be stated as follows: Even if someone is not physically present in the room with me, I an able to see and hear them exactly as though they are right there in front of me.

Yet some of our other advisors have what might be thought of as a more nuanced take. To them, successful VR is not about literal transmission of physical presence, but rather high quality transmission of emotional and psychological presence.

I tend to side with the latter group. But then again, I’m old fashioned. To me, some of the most profound experiences of virtual reality can be achieved simply by curling up with a really great novel.

Dream workshop

Last night I dreamt I was in a professional workshop with some colleagues. All the other participants in the workshop were people I know in real life, either as collaborators or as potential collaborators.

The workshop itself was very entertaining, and people had lots of intriguing ideas. Of course it wasn’t until I woke up that I realized it had all been a dream.

The surprising thing was that even after I awoke, I felt that the idea that had been discussed in the workshop were work pursuing. I realize, of course, that all of those ideas were actually coming from my own brain, but I also think that those ideas were informed by my internal model of these particular colleagues.

So here we have something interesting: I internalized the thinking of smart people I know. Then I dreamt about cool ideas that, in my dream, appeared to come from them.

In a way those ideas did come from them. My theory of mind of those colleagues became tangled up with their own unique thought patterns, resulting in something different from anything they might actually have thought of, or that I might have thought of without their influence.

I wonder whether there something here to pursue. Perhaps lucid dreaming as an approach to collaborative research?

Useful procrastination

In recent weeks I’ve been working on various parts of a fairly large software project. One problem with large software projects is that it can be hard to combine all the pieces, even if you’ve designed all the pieces to make that integration easier.

For the last week or two I’ve been trying to work out the best way to merge two particular parts of this large project. I get a little closer every day, but the thing is still rolling around in my head, and I think it’s a bit too soon for that race to the finish line.

Today, knowing that I am still not quite ready to complete that big software task, I started another project. It’s just a little side project, something I knew I could get done within a single day.

As I was finishing up this little project, it occurred to me that I’m engaging in a kind of “useful procrastination”. To avoid doing something useful, I end up doing something else that’s useful.

Sure, it’s still a form of procrastination. But at least I have something to show for it. 🙂

The future history of AI

There is a long list of things that everybody knew artificial intelligence would never do, until it did.

A few of the better known items on this list are chess, accurate handwriting recognition, ping-pong, Go, speech-to-text, and natural sounding text-to-speech. Others, such as the self-driving car, may have temporarily hit a bump on the road (so to speak), but are sure to follow.

One interesting side note: I am using speech-to-text on my Pixel phone to write this post, and it is working flawlessly.

What might we be able to predict about future tasks which are still out of reach of AI, but which it will achieve at some point? In particular, in what historical order will particular capabilities fall within the purview of AI?

It would be fun, based on what we know now, to chart a future timeline, and then come back sometime later and revisit it to see how much we got right and how much we got wrong.

Mood app part 2

Yesterday’s post notwithstanding, I was looking back over my photos today. One was a photo I took a few months back of a sketch I had made on the whiteboard.

And it occurred to me that if there were an app that could transport me from the image on the left to the image on the right, I would definitely use it. Which is not to say it wouldn’t be dangerous… 😉

Mood app

If you could dial up any mood you wanted, at any time, would you do it? Or is the entire prospect of such a thing too dangerous for mere humans?

We have all experienced moments when we are filled with melancholy. And the reasons for such moments are often obscure, even to ourselves.

If, hypothetically, you had an app on your phone that you could use to set your mood, would that be desirable? I could certainly see ways it could be convenient.

There are situations where it is better for everyone if you are able to catch a certain mood. Perhaps a moment calls for you to be happy, or thoughtful, or amorous, or hungry, or receptive to a good laugh, or highly attuned to the feelings of others, to give just a few examples.

Yet I have a feeling that such an app would be highly dangerous. It would be offering a kind of false shortcut to a sense of enlightenment — one that would not have been earned.

And therefore, eventually, for many people it would become a drug. Perhaps it is good that we need to struggle with our moods, to negotiate with them and work through what they mean.

That struggle is an important part of who we are. If we try to take a shortcut around that, we may be removing something essential from ourselves, and we may end up losing far more than we have gained.

Dance VR puppetry

This evening I went to see end of semester student performances at the NYU Dance Department. One thing that really struck me was the way the dance students, who are all supremely talented, tried to push their bodies into new forms of movement that were beyond what we would usually think of as being within the range of human bodily expression.

Watching this, it occurred to me that these young artists could really benefit from combining their talents with the infinite possibilities of virtual reality. If they could think of the target of their expression not as the literal human body, but rather as a kind of unbounded puppetry that could be controlled by their physical movements, I feel that these brilliant young people would come up with new kinds of dance that would be beyond what we might generally consider possible today.

Of course it is impossible to say how such an art form would develop. But I am extremely interested in the possibility of our lab working with their department to explore the possibilities.