Ken’s Blog

One of the papers at the SIGGRAPH conference showed how you can replace the fancy expensive compound lens in a digital camera with a really cheap lens. Of course when you do this you get all sorts of optical aberrations — chromatic aberration, spherical aberration, field curvature, and so on.

But if you have a powerful enough computer, and you know exactly what sorts of errors your cheap lens is introducing into the image, then you can post-process the captured image to get an impressively good result.

I thought this paper was a great example of the progressive virtualization of our physical environment. More and more of the things we think of as being part of the physically built world around us are being augmented — and in some cases replaced — by virtual components.

From the ringing of your phone, (simulating the sound of a long-gone technology) to the electric motor that drives your steering wheel (haptically simulating the direct mechanical linkage of an earlier error), to similar innovations too numerous to mention, every year we make our physical environment just a little bit more virtual.

***

Oddly enough, the very moment I finished writing the above, an old friend came over to say hello. He has a company called PaintScaping. They specialize in projecting digital make-believe content onto real world walls and other surfaces, matching the lighting, shadows, and 3D relief so perfectly that the resulting images seem like they are part of the physical world itself.

Maybe it’s a sign.

Like a number of people in my field, I have developed techniques that ended up being named for me (through no fault of my own).

The other night, just as people we gathering for the SIGGRAPH conference, I was introduced by friends to somebody who decided to make an impromptu joke of the occasion. “Your parents must have had a great sense of humor,” he said, “to name you after a well known technique in computer graphics.”

It could have been an awkward moment, but life is too short for awkward moments. “That’s nothing,” I replied, “You should meet my brother Fourier.”

As it happened I had dinner with my Mom last night. I wanted to tell her the story, but as I began I realized she would have no idea who “Fourier” was.

So I adapted the tale to the audience. Everything was the same until my reply at the end, which had now become: “That’s nothing. You should meet my brother Kleenex.”

I am happy to report that my Mom found the entire episode very funny.

Today at the SIGGRAPH conference I surprised an older colleague, by suddenly recalling an old memory that involved him, from when I was about eleven years old.

I had just joined the Boy Scouts. One Saturday our troup volunteered to help with paper recycling, which mainly involved loading many bundles of discarded newspapers and magazines into a big dumpster. Amid all of the trash, I happened upon a discarded issue of a magazine for electrical engineers (which of course I had never heard of). Curious, I started leafing through it.

In it I found the coolest article: an ingenious way of making computer graphics look fully three dimensional, as though objects were floating in space. No 3D glasses required.

I remember thinking, as I looked through the article, that this — using technology to make wondrous and magical things — was exactly what I wanted to do when I grew up.

I remember wanting to meet the person who did this work, but I did not meet the man himself until quite a few years later (when I was all grown up and doing computer graphics myself). By then the memory of that Saturday had receded to somewhere far in the back of my mind.

Until today, when I said hello to this colleague and suddenly the events of that long ago day came flooding back. I told him my story about finding that article, and how it was what first inspired me to want to do computer graphics.

He seemed a little taken aback, but very happy.

I’m very excited about the upcoming SIGGRAPH conference. But the emailed “Important Information For SIGGRAPH 2013 Attendees” has me a little non-plussed. Here is ther policy on cameras and recording:

No cameras or recording devices are permitted at SIGGRAPH 2013. Abuse of this policy will result in the loss of the individual’s registration credentials.

Now unless I am mistaken, essentially every SIGGRAPH attendee has a camera and recording device in their pocket at all times. Which means that about 25000 people — statistically 100% of attendees — are in continual violation of this policy.

What happens when they realize this, and take away the registration credentials of everyone? Should we then start our own conference?

A colleague of mine at Microsoft told me today about the recent send-off party for their director of research, who is being promoted. Because the guest of honor is a big fan of Star Trek, my colleague told me that they had decided to celebrate by watching the man’s favorite ST episode.

“City on the Edge of Forever,” I said.

My colleague just looked at me blankly. “Wait, how could you know that?” he asked. “Did somebody already tell you about about this?”

“No,” I said, “but it had to be ‘City on the Edge of Forever’.”

My colleague continued to look dubious, wondering whether I was putting him on by pretending to be clairvoyant.

So I tried to explain it. “Look,” I said, “I know roughly how old he is, so I know he would have been watching original Trek when he was around fifteen years old. I also know he’s a very intelligent guy, and that he still takes ST quite seriously almost half a century later. Which means that his favorite episode would have been the best one, the one that perfectly intertwines love, fate, loyalty, and speculative fiction on a metaphysical level.”

“Also,” I continued, as if that weren’t enough, “the one with an original script by Harlan Ellison.”

I’m not sure my colleague quite believed that I had worked all this out from first principles. But I am sure that it’s as obvious to some of you reading this as it was to me.

I am very impressed by our president’s recent short speech reflecting on the death of Trayvon Martin. The whole subject is such a political minefield, and of course he will be criticized in some quarters for saying anything at all. Yet everything he said was very true, and very sad, and these are truths that need to be heard.

I was especially impressed by the way he managed to convey the pain of systematic prejudice, the terrible truth of what so many in our society must endure every day — and that he managed to all that not with anger, but with clarity and understanding.

And even a note of hopefulness.

A cool feature of the modern World Wide Web that you use every day, whether or not you realize it, is its system of “Cascading Style Sheets”. The basic power that CSS gives to web designers is the ability to separate style from content. When you create a web page, CSS lets you focus just on the content. Then, somewhere else, you can describe the style, or “look” of the page.

This means, for example, that if you want to upgrade to a fancier font, or change the color of all your text box borders, or make the corners of all your buttons a little rounder, you don’t need to mess with (and potentially damage) the place where you wrote your valuable web content. Conversely, it makes it a lot easier to do things like run computer programs behind the scenes that update charts and other data. The output of those programs doesn’t need to know anything about the style or look of your web page.

I’ve been involved for a number of years in studying how to use games and other interactive media to help make learning for kids more fun and effective. A game or interactive experience is not only more engaging than a textbook, but it can also monitor how well a student understands what’s going on, and can continually adjust things to keep each learner in their own personal zone of best learning experience.

Not surprisingly, different kids respond to different approaches — there is no “one size fits all” when it comes to learning. There are young people who respond best to concrete examples, and others who do best with abstraction or symbolic approaches. Some prefer and learn best from things that are highly visual, or require speed and dexterity. Some learn best by doing things cooperatively, others through competition, and still others work best when working things through on their own.

Perhaps we should be working to define a system of Cascading Style Sheets for learning: The same subject areas, be they math, science, language comprehension, history or music, would be accessible to each learner via that learner’s own personal style filter. One student would go through a subject as a first-person shooter, whereas another could experience the same subject as a set of contemplative puzzles, while others could work through the same material as a set of cooperative building activities.

It would be interesting to work out, in this context, what form a CSS type of specification would take. What is the interface, in interactive learning experiences, between subject and style? And what is the best way to describe that interface?

I was at a workshop where roughly half the participants self-identified as designers and the other half self-identified as computer scientists. The goal was to try to find ways for the two cultures to understand each other and work together better.

At one point, after a designer had talked about his process of creating ideas via sketching, people were invited to ask questions that the panel would then discuss. One designer asked the following question: “How would one reconcile our practice in these conceptual stages of design, where we need to use sketching, with the practice of computer programming? In code everything needs to be so precise, so black and white. It’s all ones and zeroes.”

I happened the next person whose turn it was to ask a question. Before I asked my own question I couldn’t help commenting on the previous one. “It’s funny,” I said, “Everything I make is sketching, and everything I sketch is made by writing code.”

The modern internet is built atop a set of layers of protocols which are collectively known as TCP/IP. The essential idea behind this system is that anything you get over the internet, such as a document from a Web page, is that all of the data is broken down into tiny packets, and each packet is labeled. The label tells the packet (1) where it is in the larger document, and (2) where the final destination is (ie: where your computer is on the internet).

The beauty of this approach is that all of those packets can take different routes to get to you. In effect, your document “pours” over the internet like a kind of liquid, with each packet acting like a tiny “molecule” that travels along whichever path happens to be least congested. This turns out to be a really good solution, and it’s one of the reasons the internet continues to work even when there is heavy traffic.

The way that each of these packets is intelligently handed from intermediate server to server on its way to you is called “packet switching”.

Several years ago somebody explained to me why a lot of deliveries in Europe are done by truck rather than train, although one would think that trains would be more efficient by virtue of larger speed, greater fuel efficiency and absence of traffic lights. It turns out that a freight car often needs to sit waiting at borders and other switching stations, sometimes for hours or more, until it can be coupled to another train. There just aren’t enough tracks to be completely flexible.

In contrast, putting your product on lots of trucks allows you to deliver it across Europe more efficiently. Each truck driver is free to take whatever route he/she likes, as long as the load gets to its final destination. This is essentially same highly granular packet switching that makes the internet.

We might see the same sort of change in the next ten years or so, as we switch over to self-driving cars. Once driving routes can be computer controlled, automobiles will be able to coordinate with each other, forming an optimizing packet switching traffic network. Those annoying traffic jams that last for hours — the bane of many of today’s commuters — will be a thing of the past.

In his talk yesterday Bill Gates made a similar point about the relationship between education and credentialing. Today there is really only one way to build academic credentials with good provenance: Enroll at an accredited university, and take its degree granting curriculum. He posited that as more on-line options become available, credentialing may begin to decentralize. Education consumers may be able to pick and choose, building their personal credential portfolio from a combination of on-line and brick-and-mortar vendors.

If this scenario comes to fruition, then educational credentialing — the way you will show a potential employer that you are qualified for your next job — will operate via a kind of granular packet switching.

One can see similar patterns in the recent evolution of personal music collections, casual written communication, and even how people arrange to meet for afternoon coffee. I wonder whether this emerging meme is an inevitable consequence of evolving information technology.

For quite a while, one of my favorite examples of the unintended consequences of technological change has been the historical relationship between the invention of musical recording and live musical performance. Today that history came up in a surprising and intriguing context.

At the dawn of the age of commercial recorded music, many top musicians refused to go into recording studios and cut records, because they were afraid that audiences would then no longer be interested in hearing their live performances. That’s not how things worked out. As we now know, the artists who made recordings were able to build a much larger following. Demand for their live performances consequently increased.

Today I attended a talk by Bill Gates. During the Q&A, somebody asked him what he thought would be the long term effects of the growing practice of on-line education. He mentioned the historical precedent of music recording, and for me that was an “aha” moment. If music performance is any precedent, the recording and wide distribution of lectures will not kill the live lecture, but rather will result in a greater demand for live lectures by good teachers, as a result of the following they will develop among inspired learners. Bad teachers won’t fare as well.

On the other hand, the history of musical recording and distribution is still being written.