Ken’s Blog

The other day I posted about gathering the tools to convert a frame-by-frame sequence of my computer graphic images to a Quicktime movie. This gives me a kind of doorway into making animated films, but it’s a door that goes only one way.

What if I want to make computer graphic films that incorporate the real world itself? What if I want to mix live-action film-making with computer graphics and stop-motion? In order to do all that, I need the door to open both ways.

Today I figured out what I need to do to close the loop — to start with a video that was shot with a camcorder, and pull it, frame by frame, into my own computer programs.

You might ask why I would bother to do this when there are perfectly good movie editing programs out there. The problem is that those movie editing programs will only let me do what they were written to let me do. If I want to try out one of my own crazy techniques for film making, whether that technique is a different way of blending live action with computer animation, or a way of skipping through time in an odd sort of way, I can’t program somebody’s movie editing program to let me do that.

But once I’ve got everything sitting in my own computer program, frame by frame, then I can pretty much do whatever I want.

It’s like having a shiny new toy. And pretty soon I hope to combine this shiny new toy with my other one — the 3D printer sitting next to my couch. Hopefully I’ll have a few fun things to show in the days to come!

Today I was in a discussion in which people were talking about the future of apps for iPads and other smart tablets. The topic came up of how to customize apps for each individual user. “Imagine,” one person said, “a custom version of Google search for New Yorkers.”

Immediately my mind went to an important culturally defining moment in 1976, when New Yorkers were confronted by the image that perhaps resonated more strongly with their sense of self than any other single image. I am referring, of course, to the following iconic cover that March of the New Yorker magazine, created by the incomparable Saul Steinberg:

 

And that’s when it occurred to me, that all these years later, the killer iPad app for us New Yorkers would not be a custom version of Google search.

No, it would be a very special custom version of Google Maps. 🙂

It continually surprises me how easily we allow ourselves to slip into relationships with fictitious people. Give the average person a novel to read, or a TV series to follow, and they will find themselves drawn into the passion of lives that never existed.

This will happen even though they know for certain that the people in these books and on these screens do not exist. In many cases, those people cannot exist, at least in the reality we inhabit. You will never actually meet a Hobbit, or a Vampire (at least, I hope for your sake you won’t). Yet it takes very little to make you care about them, to share in their pain, their joy and sorrow, their triumphs and their tragedies.

I think the actual cause of this transference is the evolutionary history of language itself. Since humans evolved spoken language, large portions of our perception of the world around us has become channeled through linguistic perception. Yes, we can appreciate the inanimate world without the benefit of language, but the moment other people enter the equation, we instinctively reach for words and sentences to make sense of our emotional response to them.

And this means that even people who consist of nothing but words (such as the characters in a novel) will feel real to us, since so much of the cognitive apparatus we already use in assessing other people will be fully engaged. Intellectually, we know that the characters in a novel do not exist. But our emotions tell us otherwise.

Recently I wrote about the future of augmented reality — about how some time in the next few years technology will allow us to see the world not literally as it is, but rather as merely one aspect of the info-verse that will perpetually surround us and dwell among us, infusing even our most casual social encounters.

Yet in light of our species’ odd relationship with language, was I really talking about anything new — anything that is not in fact many thousands of years old? We have always lived within virtual reality, from the moment the first early humans began talking to each other in generative grammatical languages.

In fact, to be human is to be incapable of having a social relationship with another human completely outside of language. We perceive one another through a cloud of symbols, as translated by the virtual-reality screen of the unspoken verbal descriptions within our own minds.

In any way that matters on a social or interpersonal level, we already live our entire lives within virtual reality.

I was talking with a colleague from Taiwan today, who asked me whether I had ever been there. Yes, in fact, I recalled, I was in Taipei during their completely crazy 2006 election. For those of you who don’t follow Taiwanese politics, the incumbent Green party’s platform was a strong declaration of independence from mainland China, whereas the Blue party advocated finding a graceful way to just avoid the whole issue.

If you’ve ever looked at the two countries on a map, you would know that the outcome of any outright military conflict would be more or less reminiscent of the Marv Newland classic animation Bambi Meets Godzilla, with Taiwan playing the part of Bambi.

As you can imagine, the Green party was not favored to be reelected. However, the day before the election, both the president and the vice president were reported wounded in simultaneous assassination attempts. Tensions ran high on election day, and the vote ended up being split right down the middle. At the end of the day, the wounded Green party president and VP returned to office, riding (just barely) on an outpouring of emotional support.

There were many on the Blue side who said that it was a set-up job — that the assassination attempts had been faked, engineered by the Green party to gain votes. The day after the election, as I walked the streets of Taipei, police were holding back thousands upon thousands of protesters. It was like being in “The Year of Living Dangerously”, only without Linda Hunt and what’s-his-name.

That night I attended a formal dinner, where one could argue that I’d enjoyed one or two drinks too many. I was sitting to the right of a pleasant fellow from the Taiwanese government, when suddenly I had an inspiration. I told him, rather giddily, that Taiwan was playing out the full spectrum of politics, according to my academic field, computer graphics.

“What do you mean?” he asked.

“Well,” I explained happily, “you’ve got a Green party and a Blue party, right?”

“Yes,” he nodded, still not getting where I was going.

“Don’t you see?” I went on, “It’s perfect. Right over there, just north of the Taiwan Strait, you’ve got the Red party. So between you, you can make any color you want!”

The man just stared at me for a long moment, then ever so politely excused himself from the conversation, and began talking to the person on his left.

Perhaps I am not cut out for a career in diplomacy.

The first time I posted the butterfly-to-fractal movie, it was as an animated GIF. That was because the program I wrote just created individual Jpeg images, and I didn’t have any conversion software handy other than Gimp, which lets you convert a sequence of Jpeg images to animated GIFs.

Then a friend offered to compose music for the animation, at which point animated GIFs just wouldn’t cut it. To set an animation to music, you need something in a legitimate format like Quicktime, which lets you scrub forward and back over the animation, so you can do proper timing.

After a little looking around, I found the wonderful free program VirtualDub, which lets you convert a sequence of Jpeg images to an AVI movie, and another wonderful free program Oxelon Media Converter, which lets you convert an AVI movie in to a Quicktime MOV file.

So thanks to the kind generosity of the open software movement, I am able to present you with the butterfly film as a legitimate movie, both in a slow one minute version, and a fast thirty second version.

I can’t decide which version I like better. The fast/short one is peppier, but the slow/long one is more mysterioso, and gives you more time to look at the individual visions along the butterfly’s journey.

Does anyone care to weigh in?

Continuing from yesterday…

My first experiment was to 3D-print a Koch snowflake. The Koch snowflake is a lovely fractal, which is made by following this very simple recipe:

1. Start with a triangle.
2. Glue a 1/3 scale version of the original shape onto the middle of each edge.
3. Repeat step (2) with the new smaller edges, until you either get bored or die of exhaustion.

You can see the first few steps of the progression here:

 

As you can see, it rather quickly turns into something that looks like a snowflake. I wrote a computer program to craft one of these babies in a format that would print onto my little home PP3DP printer. The result looked like this:

Actually, not everything in the decoration you see in that photo comes from the Koch snowflake itself. The PP3DP printer, like many 3D printers, builds things by adding them in layers, from bottom to top. To make sure things don’t collapse, it starts with a loose backing, which you can later peel off. You can get a better sense of this by looking at the printer in the act of printing a snowflake (that’s my couch in the background):

 

I decided to keep the backing on, because I like the way it visually framed the fractal, and also because it gave me an easy place to attach a string, for hanging each snowflake onto the tree.

To give the decoration more of that nice wispiness that helps make snowflakes so lovely, I decided to replace each of the triangles with a Sierpinski triangle. Like the Koch snowflake, the Sierpinski triangle can also be made by following a very simple recipe:

1. Start with a triangle.
2. Replace all triangles with three 1/2 scale triangles, each tucked into one of the bigger triangle’s vertices.
3. Repeat step (2) with the new smaller triangles, until you either get bored or die of exhaustion.

You can see the first few steps of the progression here:

 

When I replaced each triangle of the Koch snowflake with a Sierpinski triangle, this is the first version that came out of the 3D printer:

 

At this point I realized that there was a bug in my program, which was causing many parts of the fractal to be left out (although I think it still looks cool). I fixed the bug before heading to the party, but alas, in my excitement I forgot to take another photo before I left.

I can assure you there is now a lovely Sierpinski-Koch snowflake dangling from the festive tree in my friends’ house. But if you want to see it in person, you might just need to get yourself invited to their party next year. 🙂

Today I was invited to a holiday party where the hosts asked us to bring not food nor drink but rather decorations for their tree.

I took this as a challenge. I have a 3D printer, damn it, said I to myself, and I plan to use it. I spent much of the day writing various Java programs, trying this or that, running experiments on my 3D printer, until I hit upon a strategy that would produce a truly good holiday tree decoration.

And after all, what use is a 3D printer, this technology of the future, this glimpse at things that may yet be, if not for printing cool holiday decorations to be draped upon trees?

Eventually, after many false starts, I found inspiration from the Koch Curve and the Sierpinsky Gasket — two venerable fractal shapes that never fail to surprise and delight.

If you’ve never heard of these things — or if you just want to see the cool 3D fractal shapes I created for this party — tune in tomorrow.

There were a number of interesting suggestions in response to the butterfly-to-fractal animation I posted the other day. One of them was an offer by a composer to create an accompanying score — I’m very excited about that!!

Another idea, since the piece is semi-nonfigurative and somewhat mysterious in places (several people described it as an “animated Rorschach test”), was to open it up to audience annotation.

This is an intriguing notion — create a base piece, and then use that original work merely as the starting point for an entire community to layer meaning over it. Eventually, these annotations might start to form conversations with each other, building meanings and mutual interactions far beyond any ideas contained in the original work.

The question comes up of how best to structure such a community-building enterprise. What would the result look like? An animation? A storybook? An evolving 3D landscape?

At this point I don’t have answers to these questions. If you do, I welcome any and all ideas!

Inspired by both the MirrorFugue research of Xiao Xiao and the various explorations by Vi Hart into everything from mathemusical palindromic canons to candy button musical instrumentals, today I decided to create a virtual machine to help you compose musical palindromes.

In particular, I thought it would be an interesting constraint to incorporate elements of time reversal (eg: palindromes), instruments controlled by scores on paper tape, and candy buttons. Putting this all together, I ended up with a virtual player piano controlled by two copies of a candy button strip. The two strips get played together in reverse order to create a palindromic score.

Since these are virtual candy buttons, I took the liberty of expanding on the three classic flavors of cherry, lemon and lime, replacing them with a full palette consisting of, respectively, cherry, peach, orange, lemon, lime, blueberry, plum and raspberry. It turns out that when you bring these particular eight flavors into the virtual world, each one corresponds exactly to a note on the musical scale. Which makes them very easy to work with in making musical scores.

In addition to being incredibly delicious.

The recent buzz around Harry Potter has reminded me of an odd experience I had back when the series was new. It was sometime after the launch of the first book in the series, well before “Harry Potter” had become a household name and a record breaking best seller. A guy who worked in a bookstore in Seattle recommended that I get this new book by an unknown British author, which, he said, was starting to fly off the shelves [no, dear reader, not literally].

I bought the book — “Harry Potter and the Sorcerer’s Stone” — took it home and read it, almost in one sitting. I was soon so smitten with all things Harry that I was practically speaking Parseltongue (not to be confused with Python, another language capable of casting wondrous magic spells).

As I read this delightful book, the only thing that bothered me was the eponymous phrase “Sorcerer’s Stone”. As this magical object was described in the book, I couldn’t help but notice that it sounded awfully familiar. In fact it was — in every way I could think of — an exact description of the famed Philosopher’s Stone, the long sought after rock that could turn lead into gold and allow its alchemic possessor to achieve immortality.

“Why,” I asked myself, “is this author, J. K. Rowling, going through all of this trouble to write about something everybody knows about, and then giving it the wrong name?” After all, the Philosopher’s Stone already had a wonderfully resonant name, richly steeped in history and lore. Calling it something as ridiculous as “The Sorcerer’s Stone” just made it sound, well, dumb.

It wasn’t until later that I found out that that actual title of the book, as it was originally published in England, was indeed “Harry Potter and the Philosopher’s Stone”. When the book made its way across the Atlantic, American book publishers insisted that the name be changed from its perfectly good proper title to this perversely idiotic American variant.

The reasoning, apparently, was twofold: (1) no American could possibly have heard of the “Philosopher’s Stone”, and (2) a big word like “Philosopher” in the title of a book for our children would scare off us monosyllabic knuckle-dragging Yanks.

Harrumph. It’s a wonder those publishers even considered us capable of the arcane act of reading words on paper.

It makes perfect sense that Rowling subsequently demanded so much control over the film versions, given that she now knew she was dealing with idiots and lunatics. Ever since this incident, I’ve found myself wondering how many other perfectly good books from overseas may have been damaged in “translation” by overly patronizing American book publishers.

Here are some other “American” versions I’m glad we never had to see in bookstores:

• “A Story about Two Towns”, by C. Dickens
• “Frank’s Stone,” by M. Shelley
• “The Study that Somebody Painted Red”, by A. C. Doyle
• “Loud Winds in High Places”, by E. Bronte
• “Sense and, um, Sensing,” by J. Austen

Perhaps you can think of one or two more.