Category: Uncategorized

In Mashhuda’s comment from my blog entry the other day about universal programming literacy, she contrasts the chair made by a skilled carpenter with the pre-fab kit that the consumer buys from IKEA and assembles at home. I think this might be a misleading analogy to bring to the question of “consumer level” programming.

The major factor missing from the IKEA-assembling consumer’s experience is not technical expertise or deep knowledge of the tools of the trade, but rather any sort of design process. The IKEA chair kit is deliberately designed to avoid any design decisions on the part of the consumer. And we can argue – as Bill Buxton has, consistently and articulately – that the major hurdle to learning good programming is not the mastery of the technical tools, but rather an effective knowledge of design process.

I think a better analogy is with cooking. Millions of home cooks do a reasonable job of making good and original recipes. Often they start with a recipe that they learned from a book or neighbor, and then they iteratively refine this recipe over time, often creating something fresh and original.

To anyone who knows both programming and cooking, it is clear that both involve algorithmic thinking – the ability to produce a well defined result from a series of procedural steps. The major difference is that cooking requires direct action, whereas programming requires instructing the computer to perform actions by proxy.

When I described the UPL question to my colleague Natalie Jeremijenko, she quite sensible asked me “Why will people be doing it? What purpose will it serve them?” And I think she was exactly right. After all, Bill Buxton’s questions about design process are not primarily about the tool, but about the purpose – design begins not by picking up your tools but looking at the problem you aim to solve.

By this reasoning, the tools that allow millions of people to program in a powerful way are going to be those that allow those people to achieve goals which really matter to them. For example, very many people are motivated to create recipes and to compose original music, and in both cases many of those people become quite good at cooking or composing. The populations that can do these things well is much larger than is the population that can program.

I suspect that the reason for this is not that either creating recipes or composing music is an inherently easier task than programming, but rather that each is a communicative task, understood to be a direct way for one person to emotionally connect and bond with another. I think we are going to see a kind of programming tool effectively embraced by large parts of the population only when that tool is shown to be effective in allowing emotional connection and bonding between people.

Today I am spending the day in Miami, at the house of my cousin Ben. At the moment I am hanging out with Ben’s big friendly dog Sasha. Jast as I typed those words, Sasha let out a deep and soulful sigh, as though she knew I was writing about her. I suspect she is wondering why I am typing on this stupid computer rather than running around the house like a reasonable human should, throwing a bone for her to fetch over and over again.

One of my first encounters with Russian names was the time I read Dostoevsky’s great last novel The Brothers Karamazov. It’s an amazing book, full of powerful ideas, larger than life characters and intense emotions, spiritual struggles and debates about belief and the limits of free will. It took a while for me to get through the whole thing. I remember that one weekend I was reading it while alternately watching successive films in the Hellraiser series, where some cable station was running them back to back in a marathon. I can’t really explain why, but the two experiences went together extremely well.

But if you’re going to read it (with or without Clive barker movies), here is a friendly warning: I remember I was on about page one hundred and sixty before I suddenly realized that Alexei was the same person as Lyosha. I’ve just now looked it up on Wikipedia, and it seems that at different times this character is variously referred to as Alyosha, Alyoshechka, Alyoshenka, Alexeichik, Lyosha, and Lyoshenka. There’s a similar kind of deal going on with the other characters. No wonder it took me a while to realize that there were four brothers.

For weeks after reading the novel, my head was filled with all the ideas to be found between its covers. In fact, it became somewhat difficult to concentrate on the here and now. What finally cured me of my Karamazosis was the simple act of renting and sitting down to watch the 1958 Hollywood film version. Yul Brynner did a perfect job of capturing the powerful and smoldering Dmitri, but William Shatner as Alexei was – how to say this delicately – unfortunate. Kind of killed the mood. Well, the good new was that after weeks of wandering in a haze of Dostoevesky’s powerful thoughts, I managed to snap out of it in under two hours, thanks to the magic of Hollywood.

I never tire of walking around Manhattan. It truly is the infinite city. There is an astonishing variety of culture and a tremendous energy to this place. Everyone seems to be on a mission of some sort, some tryst with destiny.

And for me, having lived here for years, there is that added quality that the place is highly layered. I walk down a street and remember the store that used to be at this spot on Broadway, the restaurant that used to be on that corner over there. Although the store and the restaurant are long gone, sometimes replaced several times over, they are still there in my memory, and I can feel them as I walk by.

There is the street just outside the bar where I kissed that girl for the first time. The girl and the bar are both gone, and yet the moment, and the kiss, are right there, exactly where I left them.

Today I am revisiting the topic of my January 21st post Learning from Children, in which I talked about the possibility of Universal Programming Literacy (UPL). But today, rather than talk about how to achieve such a thing, I’d like to talk about whether it’s a good idea at all.

It is clear that we live in a world in which, even in the most literate and computer savvy of societies, only a small percentage of the population programs computers. It is reasonable to ask the question “Is there a scenario in which pretty much everyone in society programs computers?” I don’t think it comes down to ability. Just about anybody can understand how to program, if taught the right way. I know this from first-hand experience. I think the more salient issue is utility, and therefore motivation.

One argument against UPL posits that programming is like carpentry: We all sit on chairs, yet a society only needs a relatively small percentage of its population to be able to build chairs. Some people may learn carpentry as a hobby, but it’s not necessary for your child to be able to build furniture in order to succeed in society. Therefore there is no compelling pressure from parents and communities to make sure every child learns how to be a functional carpenter.

A counter-argument in favor of UPL – and this is very hard to make in the context of most of today’s programming languages (C++, Java, Python, C#, etc) – is that programming can evolve to something more like written language: A communicative tool that helps provide the glue by which people form into communities. This has been the role served by written literacy since the Web took off in the mid ’90s: The fact that millions of people can read and write has allowed the Web to enable a grass roots kind of communication, in which communities can form themselves around common goals and interests, rather than needing to depend upon trained experts.

Those of us who program know that our skill provides us with an enormous increase in our ability to take advantage of the power of computers – the computer becomes a fantastically powerful and extremely protean servant. We are able to instruct the computer to do things for us that require millions of custom operations. For example: go off and search through millions of items, see which ones match what I want, combine them in this particular way with these other items, and bring me back the best result. Very powerful stuff indeed.

And yet, if it turns out that the number of people who would benefit from directly wielding such power is small, then programming is indeed akin to carpentry, and then there is no point in trying to promote UPL. On the other hand, if it turns out that such power can be fruitfully applied by people in such diverse fields as sociology, economics, community activism, literature, politics, music, poetry, and so on, then we’re talking about something that is more like written literacy, and then there is a reason to pursue UPL.

My current sense is that the problem is still ill-posed, because the kind of programming language that would empower most people to do the things that matter to them does not currently exist. Most existing programming languages are rather brittle and are quite focused on relatively mathematical operations. Those kinds of operations are very useful for those of us who do things like computer graphics, physics, statistical analysis, and other tasks related to mainstream computer science, but are not really useful for modeling the sorts of problems that come up when you’re looking at history, community activism, or the works of Charlotte Bronte.

A small number of special purpose programming languages are widely adopted by people who don’t think of themselves as programmers. In each case (unlike C#, Java, C++, etc), the programming environment is not brittle – mistakes don’t take down the whole system – and the visual representation of the language itself maps well to the problem being tackled. Some examples of this are Excel and similar spreadsheet languages for modeling finance problems, and Max MSP and similar visual data-flow languages for building procedural music.

My sense is that if any programming language is going to succeed in the broader arena of human endeavor, it’s going to have some important qualities in common with these special purpose languages: (i) The programming environment is very forgiving of mistakes: Whatever fool thing you choose to do, you can’t crash the program; (ii) Things are “always running”: There won’t be the sort of compile/run cycle that traditional programs rely on now. Rather, you will be making continual interventions into a world that continues to exist at all times; (iii) The program itself appears to model the problem you’re solving: In some sense, the “code” appears to be intuitively superimposed upon, or even identified with, the problem space that is being modeled (think Excel).

It could be that even with all these qualities, it simply doesn’t make sense for many millions of people to be interacting with computers by programming them – maybe programming is really carpentry and that’s that. But I think that in order to do experiments that aim to find out one way or another, we need to be developing and testing these sorts of guiding principles.

Today I went with my friend Jaron to the San Francisco MOMA. We had gone to see the wonderful Olafur Eliasson exhibition, which was indeed awesome. But for me the real highlight was wandering through the museum’s permanent collection afterward and coming face to face with one of my favorite paintings – René Magritte’s masterful painting Les Valeurs Personnelles (Personal Values).

As you can see, this painting is based on a simple but very effective visual trick: Innocuous objects from daily life are blown up to monumental scale, and then placed back into a personal setting, where they completely dominate and become shocking.

To me this painting has always been the purest expression of Magritte’s quest to get us to see that the mystery and unknowableness of existence is embodied in even the most ordinary things. His paintings were not trying to solve the mystery – I don’t believe he thought such a solution to be possible – but to engage with it, to regain a sense of awe in the face of existence, thought and perception.

Magritte’s views on this topic had always influenced my own work. In my computer generated images there has often been a somewhat surreal quality – ordinary objects seen in surprising ways that make you look at them in a new light – reflecting my own belief that to see the world through the prism of perception is always to be confronted by mystery. For example, the following image from one of my papers on texture synthesis is clearly influenced by Magritte:

So to come upon that particular painting on this particular day was wonderful. I’ve been wrestling lately with a need to bridge the gap between the ordinary on the one hand, and the sense on the other hand that each moment can be a source of infinite mystery and adventure. This daily blog has been part of my attempt to build that bridge. Each day that we get to be here is infinitely precious, and yet most days we somehow seem not to notice.

Every time you wake up in the morning and create something new, something that has never before existed in this world, especially if you build it out of the most ordinary of materials, you are speaking to that mystery. And maybe that’s the best we can ever do. As Arthur O’Shaughnessy so aptly put it: “We are the music makers, and we are the dreamers of dreams.”

Today, for reasons that cannot be revealed here, turns out to be an astonishingly auspicious day to post the complete version of The Heleniad.

I sing this to the young lady in the front row. You others all may listen.

     THE HELENIAD

Canto the first

   And so Miss Helenius
   Feeling most curious
   Not quite anonymous
   Yet not yet eponymous
   Intent on the spurious
   Though nothing injurious
   In a moment unserious
   Set out on a lark

   Like brazen young Theseus
   Or better, Prometheus
   Whose tales still fire us
   And often inspire us
   To passions erroneous
   If not quite felonious
   But somewhat delirious
   And never too dark


§  The heart can bedazzle us
   And this may be perilous
   For the heart is perfidious
   And somewhat insidious
   Those moments that capture us
   Can make us feel rapturous
   Till hopes grow innumerous
   Quite out of the blue

   The place was commodious
   The music melodious
   The night serendipitous
   For adventure precipitous
   As the Gods like to play with us
   And so have their way with us
   So he, feeling humorous,
   Did pick up her cue


§  The girl was vivacious
   And somewhat voracious
   The liquor contagious
   The boy felt outrageous
   Their mood grew gregarious
   Till, feeling hilarious,
   They descended the palace
   And took to the street

   In this vast megalopolis
   Our winter metropolis
   What is our true purpose?
   Do our moments usurp us?
   For when things become amorous
   Then our hearts, feeling glamorous,
   May drink from the Chalice
   And so be complete


Canto the second

   Their talk was far-ranging, the rhythm was changing
   And rhyme rearranging out there in the night
   Their thoughts began drifting, for something was shifting
   A curtain was lifting, a song taking flight

   And so then she kissed him, and yes she did bind him
   The wall was behind him and yes yes they said
   This flower of the mountain, like the girls Andalusian
   Perhaps an illusion, her lips were so red

   Her arms were around him, her body imploring
   The boy, now adoring, returned her caress
   Say yes mountain flower and the wind somewhere blowing
   Their hearts madly going and yes I will Yes.


§  Moments may sway us, but kisses betray us,
   For fate won't obey us, and oft goes astray
   Twas fateful that meeting, two hearts fiercely beating
   But alas, joy is fleeting when stolen away

   In a turn most appalling the darkness came calling
   For a curse was befalling, a thing of their fears
   And a figure demonic - it was almost iconic -
   In voice monotonic said: "Seventeen years!"

   It was all rather vexing, and sorely perplexing,
   This grim specter hexing their love most sublime
   "What be you?" they wondered, then the night air was sundered
   As the dark figure thundered: "The demon - of Time!"


§  "Je parlerai en français
   C'est la langue de la vérité
   Pour vous dire ce que je sais"
   Ainsi le démon a dit

   "Il est temps pour vous d'entendre
   C'est le moment de comprendre
   Vos rêves que je veux prendre"
   Alors le démon a ri

   La jeune fille a pensée
   "Dix-sept ans" a-t-elle répété
   "Cela fait beaucoup d'années
   Et la vie est brève"

   Le garçon a dit "Mon amour
   Je te parle de tout mon coeur
   Tu sais que je t'adore
   Est ce que tout était un rêve?"

   En un jour une vie peut changer
   Tout est ré-arrangé
   Un rêve est dérangé
   Et l'amour s'est dissout

   Le nuit était très sombre
   Le monde était dans l'ombre
   Qu'est ce qui est dans un nombre?
   Tristement, peut-être tout.


Epilogue

   There's a room in my soul where the old shattered dreams
   Lie in pieces all over the floor
   Where the stillness of time shades the windows, it seems
   And a demon stands guard at the door

   But sometimes a memory lights in my mind
   And it shines in the soft attic air
   And a strange kind of music plays sweetly and kind
   That I let myself hear, if I dare

   This flower of the mountain, this girl Andalusion,
   This force I could not understand
   Yes your touch Miss Helenius was, in conclusion
   The caress of a Theremin hand

   But like delicate fragrance of madeleines dipped
   Into lime-flowers long gone away
   Your succulent kiss so deliciously sipped
   Beguiles me even today

Have you ever had the experience of realizing that you’ve been dealing with some traumatic experience in your past – perhaps your distant past – by shutting off some parts of your thinking or feeling? I guess we all do that – one of the ways we deal pain is by shutting down, staying in the safe places. Sometimes you don’t just refrain from touching the hot stove which once burned you. Instead you stay away from stoves in general.

Well, I started this blog in order to try to make sense of an unexpected emotional encounter which told me that I had been operating too much in this “safe” mode. I didn’t quite have a handle on it, but I knew, through what I now perceive as a lucky accident of fate, that there were some doors and windows in the old psyche that I had been keeping shut, that had grown maybe one or two coats of cobwebs too many, and which I had allowed to stay that way out of some sense that it might be dangerous to pry them open.

A blog is a good way to start taking those steps – to start to angle up sideways to asking some hard questions. People who know you and like you are around, and supportive, and you can take weeks or even months to gradually work a question around to where you are staring it square in the face. And by the time you do, maybe it’s not so scary or difficult anymore.

And along the way, the opportunities for creative expression are simply awesome. Stories, discussions, epic poetry, drawings and animations, maybe an opera or two – the things you can create are utterly limitless. Of course they are anyway, but we have so many ways of stopping ourselves from achieving those possibilities, that something to force us to open up those windows and fling open some doors is a good start.

I just finished reading, for the first time, Philip K. Dick’s novel The Man in the High Castle. The initial premise is completely wild in a good way (I won’t spoil it for you), but as I read it I found myself wondering “where can he go with this?” It seemed as though he was writing himself into a corner.

Then, in the very last scene, there was a revelation that was so unexpected, so utterly brilliant in its perfectly logical nuttiness, that it took my breath away. A simple line of dialog spoken by one character to another, and the book I was holding in my hands, that I had just spent several amused and curious hours reading, suddenly took on a completely new and far more interesting meaning.

The experience of encountering such a completely unexpected ending calls into question the implied contract between author and reader. What are the rules when we pick up a novel, meet some characters, start to care about them? What does the author owe to our relationship with these characters? And what does it mean when the author suddenly pulls the rug out from under that relationship?

I have had some bad experiences with sudden literary revelations that shift reality and therefore my relationship to characters, such as the ambitious but misfired film Identity. On the other hand, I have had very positive experiences. Among the films I have seen, Christopher Nolan’s Memento, Charlie Kaufman’s Adaptation, and several works of Hitchcock come to mind.

But I don’t think I’ve ever experienced anything like the quietly monumental jolt I got in the last scene of The Man in the High Castle. If you have not yet read it, you are in for a treat.

This evening I went to see Ben Katchor’s The SlugBearers of Kayrol Island or The Friends of Dr. Rushower, a theatre piece in which live actors are completely integrated with animation projected onto huge movable flat panels that constitute the walls of the stage.

I’ve seen things before that try to do things like that, but what struck me about this production is that finally somebody got it right. From the opening image, in which the audience sees a huge hand drawn animation of a telephone, and then notices a real-life telephone. As the actress playing a main character walks onto the stage to pick up the real phone, we see a cartoon hand reach for the animated phone.

And suddenly, in one moment, we realize that the animated phone is actually about the young woman’s state of mind, her sense of anticipation at the mystery and possibility of a fateful phone call from a stranger.

In that one simple gesture at the start of the play, it is established to the audience that the animated walls will be showing a kind of internal reflection of the thoughts of the characters, and always in a subtle and somewhat indirect way.

This approach leads to a kind of new medium, where live action is subliminally underscored and commented upon by animation. There are similarities to the work that Bob Sabiston did for the Richard Linklater film Waking Life, but with some crucial differences.

One of those crucial differences is that this is a live performance. There is always the possiblity of some level of improvisation, of the unexpected performative moment, and so we get the best of both worlds – the abstraction and compression of animation, together with the immediacy and excitement of live acting.

Seeing things like this makes me very happy.

The other day I went with a friend to see a movie in a fairly large movie theatre. Just as my friend and I were getting up to leave we happened to see, mounted high up on the back wall, just below the projection booth, a big bright display that was streaming text messages in red LED lights. And all the text was backwards, reversed left to right. We could make out a message that was saying something like “Thank you for using the reflection message system” (I can’t remember the exact words).

After about ten seconds I suddenly realized what I was looking at, and why it was there, and that I was seeing something incredibly clever. I asked my friend if she could figure out what it was for. To my surprise she could not, so I told her. Shortly after that I met up with some other friends – really smart friends – and told them what we had seen, and asked them if they could figure out why it was there. And they couldn’t either.

So today I posed the same question to an entire room full of really smart computer science graduate students. To my amazement not one of them could figure out the purpose of the thing I was describing, no matter how long I gave them to work it out.

So I guess I must have been in some sort of unusual space in my head, that I was able to realize right away what I was seeing in that movie theatre. Or, far more likely, I just got lucky.

Can you figure it out?