Fun with interactive diagrams

I confess that I feel a little like a kid in a candy shop making interactive diagrams in html5.

Today, for my graduate computer graphics class, I adding little interactive diagrams to illustrate the course notes for my lecture on ray tracing.

I am sure I will continue tweaking, and I’ll probably add more diagrams to the notes, but already I think what I have there adds a lot more than if I had just made some pictures to go along with the text.

If you’re curious, you can check out those course notes HERE.

Transparency in augmented conversations

I had an interesting conversation with a friend this evening about the hypothetical future where people will be able to communicate with each other by creating and manipulating virtual objects that float in the air between them.

One question that immediately came up in the discussion is how transparent such interfaces will be. If you and I are having such a cyber-enhanced face to face conversation, and you are looking at a menu of possible things to create or manipulate, will I be able to see your choices?

At one extreme, each participant would have a perfectly private dashboard, invisible to the other person. At the other extreme, all such choices would be visible to all parties, and would become, in effect, part of the conversation.

Each of these two approaches has its advantages and its disadvantages. I wonder which one will become more of a norm than the other.

The limits of physical experience

I attended a wonderful talk today on natural interfaces. These are, roughly speaking, human-computer interfaces that allow their user to interact in ways that are relatively natural for people (think, for example, of the way we interact with the world around us when no computers are involved).

And it got me thinking. As advancing technology gets us ever closer to being able to simulate any physical experience at all, how far can our brains go in accepting extremely non-human experiences? Could we learn to feel perfectly natural inhabiting the body of a spider? Of a millipede?

The brain is a very protean instrument, known to be capable of astonishing feats of learning. But are there limits to the brain’s ability to remap physical existence itself? For example, could it ever make sense to a human mind to find its physical body seeminglz transformed into a billowing sheet of cloth? Or into a puff of smoke?

Smart instruments

When Roger Dannenberg, Robert Rowe and others started creating computer programs that would automatically accompany a live human musical performance well over twenty years ago, our current era of machine learning did not yet exist.

Their pioneering work was done before the development of Support Vector Machines, Convolutional Neural Nets and other powerful modern algorithmic tools of machine learning. These recent algorithmic techniques now underlie much software that we use every day, such as Google search and Google translate, and will soon be seen in our self-driving cars.

When we apply what these pioneers were trying to do to the various fields of live artistic performance (such as dance, acting, puppetry), we begin to see the modern possibilities for “smart instruments” — the piano that has “learned” your musical style, the virtual actor that embodies your unique body language, the paint brush which paints in ways that you might.

A lot of this has already been happening, in the work of Aaron Hertzmann and others, and we are poised for it to happen on a much larger scale. To be clear, we are not talking about machines replacing people. The live human performer is still very much in control, but is playing an instrument that has already been infused with styles of human performance, and can therefore be played at a higher level.

Through the use of smart instruments, a single talented performer can conduct an entire symphony, or direct a large virtual acting troupe. There is nothing mystical about this process — the human element is still very much present within these instruments. Such techniques are simply an enhanced way to distribute, and to gain the collective benefits from, old fashioned human talent.


Today we visited the Dennis Severs house. It’s quite a fascinating thing: An old London house turned into a physical re-creation, in enormous and highly personal detail, of a fictitious London family through several centuries. It’s very much like walking into a novel.

I really like this genre of art, other examples of which (more or less) are Punch Drunk’s Sleep No More and the wonderfully mysterious Museum of Jurassic Technology in Los Angeles. The basic idea is a rich novelistic fictional world presented in the form of an old fashioned house of curiosities.

The visitor is free to wander at will from room to room, examining the physical manifestations of fictional narratives, which are presented in enormous and idiosyncratic detail. The net result is an overwhelming sense of propinquity, an emergent feeling of presence not unlike the effect of reading Proust’s A la Recherche du Temps Perdu end to end.

I wonder how flexible this medium really is. For example, could one use the technique to create immersion into an everyday life of the 25th century? In other words, could such a delightfully Victorian approach to artistic engagement be effective as a form of speculative fiction?

I guess we would need to try it to find out.


Today we went to the Pompeii exhibit at the British Museum. Of course I had known about the famous events of A.D. 79, when Mount Vesuvius wiped out several cities, ejecting a spume of super-heated air that killed thousands of people in their homes within the span of a second.

But it’s different when you see it as something real and physical. The familiar elements of daily life are all there: lovely frescos on living room walls, surprisingly contemporary earrings and necklaces, cups and dishes, naughty erotic sculptures, tiny details that bridge the gap of almost two millennia. The delicately colored leaves in these ancient frescos, each so exquisitely rendered, could have been painted yesterday.

It struck me for the first time how little has changed in twenty centuries. Lives are still made of the same stuff — parents and children, people working on their gardens, young lovers, busy shopkeepers and rebellious teens. Those people could have been us.

And I realized what a strange contradiction is humanity. An individual life is so fleeting, a single leaf traced upon a wall. Yet the human project itself, this repeating cycle of lifetimes, creates endless variations through the flow of passing time.

We are painting a vast fresco throughout history, not with our art but with our selves. Yes, in many ways our lives are similar. Yet each life, so unique, so precious, is a thing of exquisite and irreplaceable beauty.


I had forgotten just how much I love London. Delays at Newark Airport, a long overnight flight, an endless line at customs followed by a crowded rush hour ride on the Piccadilly Line, all of it melted away the moment I stepped out into Covent Garden and remembered why I love this city.

Today I found myself in Brixton, Kennington, Tottenham, and various points about and in between. Riding on the top level of a double decker bus, you can really appreciate the sheer variety that is London. The youthful energy, multiethnicity and historic environs work together to create an almost alchemic sense of place.

Alas, I still don’t have the hang of the accent. Fortunately, people here seem to be very good at understanding my foreign dialect. 😉

Lines of force

To explore the possibilities of using permanent magnets together in different configurations, I’ve been spending the last few days writing various simulators. It turns out that HTML4 Canvas and JavaScript are great for this.

The below simulation is just built from basic magnetic dipole math. Once I had that in place, I simulated six little magnets in a hexagon pointing inward toward a stronger magnet pointing downward.

The field lines this creates are surprisingly beautiful, as you can see for yourself. Click on the image below to see an interactively viewable version:


Try to view it in Firefox or Chrome if you can, since Safari doesn’t properly show the delicacy of the lines.

Spaghetti theory

On this very somber day, I thought it might help people to have something lighthearted to read. Which leads me to the following true story:

In a conversation with a friend recently, the topic came around to Pastafarianism. Some of you may be familiar with the Church of the Flying Spaghetti Monster. A fairly recently recent religion, it has one thing going for it: Its metaphysics is equally as grounded in verifiable evidence as are the metaphysics of all competing major religions.

But what about physics? Suppose there really is a Flying Spaghetti Monster? Would the existence of His Noodly Appendage be consistent with what we know of the physical laws of our Universe?

Eventually the conversation got around to various known physical phenomena. There are so many to consider: universal gravitation, the strong and weak nuclear forces, dark matter, black holes, the expansion of the universe.

And what about string theory? We couldn’t help but see the obvious geometric connection between strings and spaghetti. But how to reconcile such a view with the prevailing hypothesis that of the 9 or 10 spatial dimensions predicted by string theory, 6 or 7 of them actually consist of tiny circular loops?

We looked at each other in sudden enlightenment — it was so obvious once you saw it: Spaghettios!