Ken’s Blog

Back in February 2014 I wrote a post on Future Language. What I meant by that is how language itself will evolve in a future where ubiquitous mixed and augmented reality will be an everyday part of life.

Children growing up in such a world will create shared visual representations of thought by gesturing in the air with their hands. To children born into that reality, this will simply be taken for granted, the way we now take for granted the ability to text or speak on the telephone.

Such forms of visual communication will not replace verbal speech. Rather, they will augment it, allowing speech itself to be used in new ways — much as phone and text have not replaced speech, but rather have extended its reach, allowing it to be used and shared in ways that have altered the way we communicate.

Since my initial post, we are four years nearer to that reality. So this seems like an auspicious time to delve more deeply into the topic.

In the coming days I will go into more detail about how visually augmented speech will evolve, and what that change will mean.

Since today is the 27th day of the month, I find my thoughts drifting toward mathematical patterns. That’s because 27 happens to be 3 raised to the power of 3.

Which suggests the idea of raising a number to the power of itself. If we do this with integers, we get a series that starts: 1, 4, 27, 256, 3125, 46656, 823543, 16777216, 387420489, 10000000000 …

But we don’t need to do this with integers only. We might just as well raise 1.5 to the power of 1.5 (in which case we get a result between 1.8 and 1.9).

If we try it with negative numbers, for example -1.5, things start to get more complex. And what if we start with complex numbers?

If we consider the entire complex number plane, this operation gets very interesting. If you are mathematically inclined, you might want to explore the question: What is the shape formed by raising every complex number to the power of itself?

Sometimes when I’m programming I look back at old code that I wrote long ago and I am surprised. I say to myself “I wonder what was going on in that guy’s mind.”

There are times when I think “Would, he really didn’t have a clue, did he? I’m just going to have to fix this now.”

Then there are other times when I look at code that I wrote some time ago and I think “This guy is so much smarter than I am. I have no idea how he figured out how to do that.”

I’m not sure what it all means. Is what I am describing some failure of long term memory? Or is it just the fact that we use multiple parts of our mind when we do something like programming a computer?

I know that technically I’m talking about a single-person activity. But sometimes it sure feels a hell of a lot like collaborating with somebody I don’t quite know.

There are tasks we never get around to doing because they seem overwhelming. Then there are other tasks we break down into little pieces, doing a little bit every day.

I freely admit that there are quite a few tasks that for me fall squarely into the first category. In fact I may never get around to doing them. I look up at that mountain and all I see is insurmountable height.

On the other hand I have practices that fall very much into the second category. For example, most mornings I wake up very early and head to the lab. Before anybody else shows up I have already put in a solid two hours of programming.

If you were to up all of the time I spend programming every year, it comes to quite a lot. And yet it doesn’t seem like a lot, because I divide the work into those manageable little chunks.

And it doesn’t even seem like work, because I love programming. Perhaps one definition of what we love is whatever we make sure to do a little bit every day.

Sometimes we take umbrage at what people say, even though what they said was completely inoffensive. Perhaps the encounter has triggered some trauma from our past. In such cases, we are not really dealing with the reality before us, but with demons from our own mind.

What if you could load an app on your phone that would record and map your emotional response to various things that people said to you? Eventually, as technology advances, such an app could measure such things as facial expression, vocal timbre, heart rate, blood pressure, skin conductivity, posture, pupil dilation and gaze saccades, to name just some of the many physiological indicators of mood.

With this data, your app could correlate your emotional responses with the objective reality of what was actually said to you. It could then search for and highlight discrepancies between input and response.

The resulting analysis could help you to better understand your own emotional responses, and perhaps to modify them over time. You might end up living a happier and less needlessly stressful life.

I wonder whether such an app would be popular.

On the train back to NYC yesterday after a visit up to the country, I pointed my phone out the window and took this photo.

The resulting image is a perfect reflection of my state of mind at that particular moment.

Behind every compelling story there is a psychological structure that the audience senses but may not explicitly acknowledge or be aware of. When I think of the songwriting team of Lennon and McCartney I think of such a structure.

Fundamentally we are talking about two men, each of whose creative energy arises from his formative experience as a young boy in working class Liverpool.

The young Paul McCartney was a happy child who felt loved by his mother and was eager to communicate that feeling of love and security to the entire world. For John Lennon it was very different.

John’s energy is that of a man who had lost his mother when still just a boy, and thereafter always felt slightly unmoored. His brooding and intense lyrics suggest a man searching for love but never quite sure that it exists.

I think it is the combination of those two complementary energies which creates the powerful psychological underpinning that audiences respond to in the brilliant songwriting of these two young geniuses. Between them, their themes run the gamut from love and security on the one hand to constant doubtful searching for love on the other.

In the human drama, there are few more compelling narratives than the two faces of our eternal search for love.

One of the stranger things about existence is that everything seems normal. You are born, you grow up surrounded by fellow creatures much like yourself, you become socialized.

So a lot of things just fall under that category of “normal”, because that’s the way it is and that’s the way it has always been. As far as you can tell, that’s the way it will always continue to be.

But every once in a while you catch yourself looking at something “normal”, and find yourself thinking “wait, this is weird.” Recently I have been thinking that way about sleep.

Imagine visitors from another planet who have no concept of sleep. Unlike us, their biology does not require it.

Such beings would not need to spend roughly one third of every day in an unconscious and essentially helpless state. They might be amazed to see that we, fellow intelligent beings with whom they have just held a perfectly nice conversation, are suddenly collapsing all around.

Our visitors would never dream. They wouldn’t even have the concept of dreaming until we have described it to them.

I am trying to imagine how sleep-prone creatures like us would seem to such observers. I’m pretty sure they would find us to be intriguingly non-normal.

Some people worry that as techniques of procedural animation develop, the result will be less work for animators. I beg to disagree.

The reason any of this is an issue is a consequence of the forthcoming wearables, which will soon replace SmartPhones. Wearables lead inevitably to ubiquitous augmented reality, which will pretty much demand the presence of interactive procedurally animated characters.

I’ll explain. In order for responsive augmented reality characters to work properly, they will need to be driven not by traditional animation techniques, but rather by procedural animation. But that doesn’t mean animators will be out of a job.

In fact, when animated characters go completely procedural, they will need to be trained by great animators. So the animators will not be creating animations individually, but rather will be “training by example”, functioning essentially as acting coaches for a new breed of virtual actor.

This means that the value created by the animator will be monetized in the form of licensable I.P., rather than by payment for animation services on a specific production. This will be great for the animator.

That’s because it’s always better to be paid for use of one’s property, rather than relying on an hourly wage. A good animator will have the opportunity to make a lot more money, because her creative output will be able to be used on many productions — and she will not even need to be involved in those productions.

This evening I got a dance lesson from a friend who really knows how to dance. It was wonderful and fun and enlightening, but also humbling.

The experience made me realize how much I generally live within my head, and not within my body. I wonder how many experiences I am missing by not having my full body involved in the processes of my mind.

Something my friendly dance instructor said really stuck with me. She told me “I need to teach you not to think.”

Yes, I can see that. There is ancient wisdom in the body. Alas, it is a particular form of wisdom that the mind can all too easily forget.

I am reminded of Zen and the Art of Archery: You can’t think your way to learning how not to think.