Author: admin

He was running fast – as fast as he’d ever remembered running. Sweat was pouring down into his eyes, making everything before him smear into a blur. Darting into one narrow alleyway and out of another, he thought a few times he’d given it the slip – the Beast – but then he’d hear its heavy footfalls behind him again. Where was he? The signs were all in Japanese – Tokyo, maybe. Asakusa district. But the streets were deserted, which was impossible. No time to think about that now. He didn’t look back – that would just slow him down, the hideous thing would catch up with him and that would be the end. Wait – did he even know what the Beast looked like? Had he ever looked back? Something about that last thought seemed wrong. “This has happened before,” he heard himself saying aloud, although he could swear he hadn’t moved his lips. That was the last thought he had before he felt a claw attach itself to his left ankle. He tried to shake free but the claw dug in and pulled savagely backward, drawing his snared foot with it. Needles of pain shot up his leg. He began screaming even before he started to fall…

Clay woke up covered in sweat.

So the dreams were coming back. He hadn’t had one of those since the first months after she… He stopped the thought before it had time to fully form, with a discipline born of long practice. He tried sitting up, and promptly realized he had a raging hangover. Slowly, deliberately, he swung his legs to the floor and managed to get his weight under him, to stand up. It took all his concentration to make it to the bathroom, to get his head under the shower, one hand pressed against the cool tile for support, the other hand turning on the cold water tap.

It felt good. Icy cold, damned good. Brain function began to return, clarity restored, a clarity he’d obviously been trying his darnedest to avoid the night before. He didn’t remember the exact sequence, but it seemed that at some point last night he’d given up trying to keep the memories out, had poured himself a drink, only the one, just to take the edge off. But of course it was never only the one, was it?

He dried off his hair, and with a purposeful air walked back into the bedroom. One of the two whiskey bottles was empty, but the other was only half finished – good thing he’d passed out when he had or he’d be feeling a lot worse right about now. He looked at the labels, impressed. Both bottles were Macallan thirty year old single malt – pure liquid gold. How the hell had he come into possession of such riches? Most likely the misplaced gratitude of a wealthy former client. Back in the day, when he’d still had wealthy clients. Before…

He could feel the thought stop cold in his head as he walked over to the sink, ran the tap, and carefully poured out the remaining whiskey. How odd to be pouring over four hundred dollars worth of liquor down the drain – more money than he had left in the bank, last time he’d checked. But now he had something better than money – something he hadn’t had in a long time – an interesting case.

It took him only twenty minutes to shower the rest of his body, shave badly, pull on his old suit and head out the door. Another twenty to get to his destination. When he arrived he looked dubiously at the dilapidated old store front. The rotten smell emanating from the grocery store next door was definitely not helping his lingering hangover. He couldn’t figure out which looked more out of place here, the ancient doorbell beneath the faded sign on the glass door, or the fluffy white cat sleeping in the window. He declared it a tie. Trying to look as dignified as possible, he rang the bell.

The front half of the doorbell promptly fell off the door. It hit the ground with a loud ringing thud that seemed to go right through his aching head. He stared down balefully, wondering if this had been a good idea after all. Just then the door opened and a young brown haired man opened the door.

“Welcome to SunMoon detective services. We solve your cases, night or day. How may I help you?” The young man smiled helpfully. “Please don’t mind the doorbell – it does that.”

Clay remembered the importance of first impressions. He thought of various ways of expressing regret over the doorbell incident, decided in the end to pretend it had never happened. Squaring his shoulders, he looked the young man in the eye and was about to speak, when suddenly he realized he was seeing double. Well, almost double. Two young women had just appeared. They looked almost identical, except that one had bright orange curls and the other possessed the blackest head of straight hair he’d ever seen. Both women had one eye hidden by her hair, so that between them they had only two eyes visible. For a moment he had the oddest sensation that he was looking into the eyes of a single person.

“Oh my,” he said to himself, “One person, two heads.” His own head started to ache with renewed vigour. Then he got a hold of himself, and started again.

“My name is Clayton Adam Terransky, private investigator, and I have an interesting case for you.”

This is the first part (of thirty parts) of a Nanowrimo story that my collaborator and I will be writing, as one post every day, throughout the month of November.

It really didn’t seem like that amazing a place when you first saw it. The sign on the glass door was old (but could have been older), its text faded (but still legible). To the left of the street entrance was an old grocery store whose food was nearly all rotten or infested with some sort of bug — to the right, a restaurant that never got any customers. A fluffy white cat was sleeping in the window. From the outside, the agency looked more like a dentist’s office. The only clue to its true nature was the small, badly-drawn symbol on the aforementioned old, faded sign — the sun, eclipsed by the moon, and a simple phrase in plain text: SunMoon detective services.

Of course it was a run-down place, but that was probably a good thing. Those girls, the SunMoon detectives, were so smart that a place any nicer would have flooded them with customers and undoubtedly burnt them out within a month and a half. They liked it here. Or rather, they didn’t particularly dislike it. None of their neighbours talked to them much, and they had to do the occasional odd job to pay the rent, but when they got their cases, they solved every one. Whether it was lost cat or a murderer on the lam, they could find it for you.

The self-proclaimed ‘Sun’ half of their service was exceedingly bored at the moment. Her office was littered with bad drawings of smiling faces and rainbows. On her desk, in place of the customary picture of a loved one, she had a newspaper clipping from the last time they’d solved a really big case a few years ago. Her computer ran a screensaver of a bustling city during the day, but it was of no interest to her at the moment. Right now she was wrapped up in one of her doodles, trying desperately to draw a lion basking in the sun purely from memory. Or at least that was the picture in her head. To anyone else the drawing looked a lot more like an anorexic cat right after being run over.

But that wasn’t the kind of thought she would have. She was the sun, and the Sun should be bright, productive and cheery. Oh! Productive! That’s right, she was supposed to be doing something worthwhile. She straightened herself up, pushing orange curls off of her right eye, and wrote in large letters on the top of her doodle-filled page: NOTES. Smiling and having accomplished her goal, she slouched again and continued drawing her lion. Julia Strype, after all, wouldn’t be caught dead doing something non-productive.

Speaking of non-productive, in an adjacent office, one door down from Julia the ‘Sun’, sat the ‘Moon’, although this title was not exactly self-proclaimed. She couldn’t remember exactly how she had gotten to be the ‘Moon’, but suffice to say it was not her idea. Her room was far less cheery than that of her partner. On the wall were miscellaneous prints of waterfalls and mountainous landscapes, neither of which particularly ignited her interest, and on her desk was a faded newspaper clipping from when they were still Genius Teen Detectives. On the back wall there was a colourful dartboard, festooned at the moment with about a dozen magnet darts, none of which were even vaguely close to the center of the board.

She glanced over quickly at her computer’s screensaver, which displayed a number of tropical fish swimming across the screen, back and forth, back and forth. Quite mesmerizing, she thought, if you stared at it long enough. Right now it was the angelfish. Sighing, she pushed her straight black hair out of her left eye and threw another dart, missing the dartboard completely. Maybe it was her lack of depth perception… why did they have these stupid haircuts, anyway? Concealing one eye each from the rest of the world only debilitated them, didn’t it? Anyway, all things considered it probably hadn’t been her idea. She decided to throw her next dart straight up into the air and catch it, but it ended up hitting the door behind her and landing, broken, on the ground. She sighed again, frustrated.

It had been months since their last case, and their last case had been way too easy. They needed something big. Umbry Stykes, the ‘Moon’ detective, really hated being bored. A whole world full of crime, and yet it seemed there weren’t any interesting cases they could get their hands on. Friggin’ FBI and the police and all the other acronyms got all the good ones. Maybe the days of the PI were really over. Maybe they should shorten their name. S and M? Oh wait, that didn’t sound too good at all. She wrinkled her nose, stretched and stood up, walking out into the lobby. She was greeted by an apathetic college guy and a sleeping cat. She patted it (the sleeping cat, that is, NOT the college guy) on the head and it purred. The college guy, on the other hand, didn’t even acknowledge her presence.

“Any calls, Lindsay?” She asked, trying to act as if she hadn’t come out here for the express purpose of knowing if there were any calls.

“What do you think?”

Umbry sighed. “Good point.” She gave the fluffy cat one last scratch behind the ears and left. The cat turned lazily to gaze up at Lindsay. He smiled back at it before going back to looking at funny pictures of other cats. Oh, how he loved cats.

A few minutes later Julia emerged from her office. She scratched the white cat’s ear and rubbed its belly, and the cat purred loudly in contentment. She smiled.

“Any calls, Lindsay?”

“What do you think?”

“…Oh.” Somewhat disappointed, she gave the cat one last belly rub and left for her office, closing the door behind her.

Lindsay rolled his eyes. “What weirdos. I swear, as soon as I get a better job than this…” He leaned back in his chair and yawned, pushing shaggy brown hair out of his eyes. The cat stared at him for a second through half-open eyes and then curled back up in a ball to continue sleeping, apparently having had enough excitement for one day. He grinned in appreciation at the feline’s economy of expression, and was about to continue perusing his catalogue of comical cats when the front doorbell rang. Startled by the unexpected sound, he jumped out of the chair, quickly scooping up the mess of doodles, darts and candy wrappers and sweeping them into the desk drawer. If he’d been able to get to the door faster, he could have told his customer that their bell was very old. Now he could only watch as half of the bell detached itself from the glass door, making a very depressing muffled ringing as it hit the ground. Bashfully, he massaged his temples, and proceeded to flip his long bangs away from his face. He put on an obviously fake ghost of a smile. The cat, now awake again, watched this unexpected bustle of activity with clinical interest.

“Welcome to SunMoon detective services. We solve your cases, night or day. How may I help you?”

Halloween eve in New York City, and the last day before my writing partner and I begin our November novel. In this pause between two projects, I have time to reflect with nostalgia on something we used to have in NYC. It’s not something everybody cares about, and I realize I’m going to sound hopelessly old fashioned in some circles for being so gauche as to mention this. But I can remember a time – not all that long ago really, when New York City actually had a democratically elected mayor.

Now of course we have the illusion of an election. Everybody here is going through the motions, pretending it’s a real election, pretending that there is any doubt as to the outcome. But of course it’s not a real election. It’s like some poor sap being pushed into the ring with a raging gorilla, and told to fight a fair fight.

Well, almost like that. Except in this case, the gorilla weighs around sixteen times as much as his opponent.

And the odd thing is that none of this is about who is the better candidate. There are good things and bad things to say about both the incumbent mayor Mike Bloomberg and his challenger Bill Thompson. Each has done commendable things during his political tenure, and each has stumbled on occasion. But that’s not what this election is about, not even a little.

This election is about three hundred and fifty million dollars – around one third of a billion bucks. That’s how much our current Mayor, a billionaire worth around $17 billion, will have spent on his three runs for office by election day next Tuesday. By comparison, the Obama presidential campaign spent less than twice that much to reach an electorate approximately one hundred times larger.

Now don’t get me wrong. I’m not saying that’s a lot of money. On the contrary, it’s hardly anything at all – chump change really – if you are Mike Bloomberg and so happen to have $17 billion in the bank. By way of comparison, let’s say you were running for mayor, and you decided to self-finance your campaign. Suppose you had, say, $10,000 in your bank account (times being hard and all). Well, if you spent the same proportion of your personal wealth as our current mayor has this time around, the election would cost you less than a hundred bucks – about the cost of a nice dinner for two in Manhattan, if you order wine and dessert, and don’t have to pay for parking.

So basically all our mayor is doing, in terms of his own personal spending, is going out for dinner with a lady friend and maybe getting a nice merlot and the blueberry pie. He’s not even taking the car.

But from the point of view of us ordinary mortals the situation is quite different. Bloomberg has top ad agencies, production companies, store fronts in Manhattan filled with teams of campaign workers, the services of the best professionals money can buy, all working around the clock, all focused on trying to discredit Bill Thompson. Almost every day I get a fancy flyer in my mailbox from the Bloomberg reelection campaign. And these are no ordinary flyers. They are like nothing you’ve ever seen before in an election. The production quality on these things makes even the polished Obama campaign literature look like it was hand cranked on a used mimeograph machine by some sweaty old guy in a basement.

Somewhere there are suppliers of fancy paper, exotic inks, custom illustrations and high class glossy photography, as well as an entire Letterman-show full of writers, who are thriving despite the bad economy, just to make those flyers that keep landing in my mailbox. And every one of these lovely flyers does the same thing – attack Bill Thompson with the intensity of a pack of feral dogs ripping into a downed calf.

I’m starting to wonder whether Bill Thompson isn’t actually some sort of saint – a holy man with angel wings and the moral discipline of a Mahatma Gandhi. Otherwise, by now we would surely all be convinced the man was a raving pornographic child molester, given the sheer volume of vitriol being hurled at him by the Bloomberg campaign.

Don’t get me wrong. Our incumbent mayor has achieved some fine things at City Hall. But this is crazy. The Thompson campaign is completely outgunned, shouted down at every turn by Bloomberg’s shockingly over-financed operation. The challenger is unable to get any message at all out to the voters. Anything he might have to say has been overwhelmed by the solid wall of media blitz that is the Bloomberg campaign.

No, Mr. Bloomberg is not breaking any laws by doing this. The fault lies with our election laws, which are so screwed up that they indeed allow wealthy people to buy elections. And to be fair, it wouldn’t work if Bloomberg were an atrocious mayor. But nonetheless, this is not an election about the merits – it is not about which of the two candidates is better. That question has been effectively buried under an avalanche of lopsided spending. This election is about one thing: a sixteen to one spending ratio.

And so I find myself asking the following question: If you believe in the idea of fair elections, can you vote for someone who is deliberately, ostentatiously subverting the process? And if you were to pull the lever for that guy, knowing he was effectively buying your vote, could you still tell yourself that you live in a democracy?

That’s pretty much it for the new techniques developed for combining hand drawn and 3D animation for the Wild Things test. The only thing left to talk about is shadows. Here we cheated, in a really outrageous way – but it paid off.

When you create a computer graphic scene, you specify a number of light sources. The computer program calculates, for each pixel in the image, where is the visible 3D point at that pixel, and from that it calculates which of your light sources are able to illuminate that point, and which are in shadow. After all, not every light source can reach every point in the scene. Sometimes there are objects in the way that block the light from some light source or another – thereby creating shadows.

In order to make Max and his dog feel as though they were part of the 3D scene – even though they were really hand-drawn animated characters – it was very important that they cast shadows. Otherwise they would have appeared to look like they were just floating in front of the scene.

Of course, Max and his dog were not really 3D objects in the scene. So we couldn’t just throw some sort of algorithm at the problem of what shape their shadows should take – there is, quite literally, no mathematical solution to that problem. Fortunately, we had animators who were perfectly happy to draw the outline of a shadow. And here is where we cheated. Just as we had the animator draw the outline of a character, and then used a computer paint program to fill in that character’s colors, similarly, we asked the animator to draw the outline of the shadow that Max or his dog should cast onto the 3D scene.

In other words, we relied on the animator’s talent to figure out where the shadow should go. Once we knew the shape of the shadow in any given frame of the animation, we used that shape to suppress the lighting from the key light source in the 3D computer graphic lighting. The visual result was the same as if we’d had a magic computer graphic algorithm to cast true shadows onto the scene.

Note that we were not painting a shadow onto the scene. Rather, we were invoking the same computer graphics techniques that we used to light and shade the 3D background – except we were giving the animator a chance to add shadows to this 3D shaded scene.

On a philosophical level, this created a very interesting interaction between animator and scene. 2D hand drawings were being used to reach in and directly modify the physics of a 3D computer graphic simulation – in particular, blocking 3D light sources at selected pixels. Effectively, we were casting actual shadows from non-existent objects.

The results were spectacularly successful, as you can see from watching the Wild Things test.

The only final note – and it is an important one – is that we were very careful throughout the production to choose the colors for the 3D computer graphic background and the image-processed hand-drawn characters that would mesh together perfectly. I can’t overemphasize how important this is when making a film that combines work from two very different media.

Examined in hindsight, our little test for “Where the Wild Things Are” represented a new way to look at computer animation. It wasn’t the result a single technique, or even a single approach, but rather a mash-up of complementary techniques and approaches, a way of mixing the old and the new, of using the computer as a tool in a very different way. That little test floated around the industry in the following years, and ended up influencing many things that were to come after, from “Who Framed Roger Rabbit” to the “Toy Story” films and beyond. I would argue that the success of this test proved the point of what my friend Lance Williams used to point out, around the time we were first bringing Max and his dog to life: “Computer graphics,'” he would say, “is limited only by your imagination.”

One thing that animators can do very quickly and accurately is draw lines on paper. And there are a lot of line drawings involved in making an animation, so you don’t want to make any extra work for the animator. We wanted to give the animators an easy way to convey to the computer, through their drawings, what a fully shaded and rounded-looking character would look like. The following image of the back of Max’s head will give you an idea of what we came up with.

The artist would draw something like the image on the left – indicating the shape of the characters, as well as the outline of regions where the character should be bright or dark. Once we scanned in this image, we could start to do our magic on it. Christine Chang implemented a paint program that allowed an operator to fill each of these regions with a different color or shade, as shown in the image on the right.

But how do you go from that image to something that looks fully smooth and rounded? My basic approach was to use my fast blurring technique to blur out the regions inside the character. First, in software we clean up the painted image by removing the outlines and setting everything outside the shape to black:

Then we apply those fast blurs I talked about yesterday. In the two images that follow, we smear first horizontally, and then vertically:

But like I said yesterday, one blur isn’t good enough – the result doesn’t look quite as smooth as we would like. So we just smear again, first horizontally and then vertically:

Now it’s starting to look good. We had to smear four times to get that result (twice horizontally and twice vertically) but that’s ok, since the technique is fast.

But this result is clearly not yet what we want – the shape itself is blurry, not just the internal details. So next we use the silhouette of the original unblurred shape to trim the result – just like using a cookie cutter:

It’s almost there now, but not quite. Max’s head is too dark around the perimeter. But why? Because when we blurred everything, the black background color bled into the shape, creating an unwanted vignetting effect. We need to need to get rid of that vignette.

Fortunately, we know exactly how much black has bled into the shape at every pixel – exactly the same amount by which the silhouette becomes darker at that pixel when we blur the silhouette. And that gives a solution to this problem: At every pixel inside the shape, we need to divide by the brightness of the blurred silhouette. Most places this won’t change anything – the blurred silhouette is white almost everywhere. But near the perimeter, the result will get brighter by just the right amount:

Aha! Now it’s starting to look like a rounded 3D version of Max’s head. All that remains is to add the character to the background. In the real test, that background was the 3D computer graphic room, but here I’m just going to drop him into a white background. Also, of course, in the actual test Glen Keane draw the entire body of Max, not just the back of his head. 🙂

So there you have it – we are almost done with the series. Mostly all that remains is to talk about shadows and a few little details about color, which I’ll discuss tomorrow to wrap up.

In my own personal experience doing this, the part of the above recipe that was a true revelation for me was in the step where I realized I just needed to divide by the blurry silhouette to get rid of the vignetting around the edges. That was the first time I realized that I could do any arithmetic I wanted on entire images, treating them just like numbers that can add, subtract, multiply and divide. There was a sense of freedom in realizing this, and it led me to start thinking more out of the box about images and the infinite possibilities of computer graphics.

On one level, what Richard Taylor was asking for was easy. Below is an example of blurring an image. On the left is the original, and on the right is the blurred version.

By the way, for those of you who don’t know, this is an image of Lena Sjööblom. She was originally the Playboy playmate of the month in November 1972. Her image started to be used by appreciative computer vision researchers at the University of Southern California, and she ended up becoming the standard test picture for image processing research. Needless to say, the original photo showed quite a bit more of Lena. If they had used the entire image, the field of image processing might have taken quite a different turn. You can read the whole story here.

Blurring is easy when you use a camera. Just push the lens out of focus, and you get a nice blurry image. That’s because every point of the original image gets smeared out over many points of the resulting blurry image. I could have done it that way in computer software, but that would have taken a very long time. Computers then were a lot slower than they are now. The image below shows the problem I was facing:

If you want to blur an image, then every pixel of the original image (left) needs to get added to many pixels of the blurred image (right). This is a serious problem if you’re image is typical size – say 1000 × 1000, or a million pixels. If your blur size is 30 × 30 pixels, then every pixel in the original image needs to get added to about 1000 pixels in the blurred image. We’re talking about a billion or so operations here. Back when we were doing the Wild Things test, that was way too much computation to be practical.

I needed to come up with something faster. The breakthrough came when I had the idea of smearing. If you think of the value at one pixel of an image, you can do the following to smear that pixel value out in a horizontal direction. First, copy the pixel value to another image, but offset to the left. Also copy the negative of the pixel value to this other image, but offset to the right. You can see this represented in the image below, in the transition from A to B:

Now here comes the secret sauce. Sum up all of the values in the result, starting from the left and going all the way to the right. What you end up with is the pattern in C above – a smearing out of this one pixel value over an entire region. What’s cool about this is that it doesn’t matter how far apart you separate the positive and negative values – the amount of computation stays the same.

Of course it’s not all that useful to blur out a single pixel. But the nice thing is that if you start with more than one pixel value, everything still works. In the image below we have two non-zero pixels. Applying the same trick (going from A to B in the image), we get some positive values and some negative values. If we sum everything up from left to right (going from B to C) we end up with a blurred version of the original.

This trick works no matter how many pixels you start with in the original image. In fact, you can start with any image at all, apply the same trick, and you end up with a smeared out version of your original image. And the amount of computation doesn’t increase as the smearing gets bigger.

This gave me a way to create nicely blurred images without requiring too much computation. I could just do this smearing trick twice in the horizontal direction, and twice in the vertical direction, and end up with a really nicely blurred image – without needing to wait too long for the result.

So how did this little fast blurring trick let me create images of Max and his dog that looked rounded and 3D? We’ll get to that tomorrow.

Today a friend – and reader of this blog – told me that when I describe how we did the Wild Things test, I should go into more detail about the technology. I objected that many people who read this are not technical in that way. But he pointed out that it would be a shame to water it down, when there are a number of people out there who really want to know the techniques. So I’m going to go for it, but before I do that I’m going to make sure you get the background, so that everything is sufficiently motivated.

First, a little history – let’s go back in time a bit, to the making of TRON. There were a lot of brilliant people behind the visual ideas in TRON. One of them was the art director, Richard Taylor. Richard was coming to TRON fresh from having worked for the legendary production company Robert Abel and Associates. While at Abel, Richard had perfected a technique he called the “candy apple glow” – which became a kind of signature look for the many award winning commercial spots created by Abel and Associates through the years.

The basic idea of the candy apple glow was to take a white silhouette image of an object, blur the hell out of it, and then slap the image of the original object on top. The result looked like a kind of corona surrounding the object. Edges remained crisp and well defined, but the entire object would seem bathed in an unearthly angelic halo.

It was a very successful look, much sought after by ad agencies, but rather difficult and expensive to achieve. In those days, the only way to composite images together was to run them through an optical printer – a big, cumbersome and expensive machine that reprinted film from one reel onto another, allowing you to apply a simple special effect each time you ran the film through the printer. In order to make the candy apple glow, quite a few steps through the optical printer were required, each one requiring another run of film through the optical printer.

First you needed to make a silhouette image of the object, white against black, then you needed to print a blurred version of that white against black silhouette. Then you needed to subtract the original silhouette from the blurry one – which required another run through the optical printer. Then you had to print the original image, adding it to the glowing white outline – yet another run through the optical printer.

The results looked great, but all those multiple passes through the optical printer were slow and expensive, and film costs ended up being very high – especially if you made a mistake anywhere in the process and had to do it all over.

So one day Richard said to those of us at MAGI, “Could you create the candy apple glow look in computer software?”

That turned out to be a fateful question…

3D computer graphics is a great way to make things look real. After all, the techniques under the hood are basically simulating the physics of how real cameras capture real life: the shapes of objects, how a camera moves around a scene, the way light shines on surfaces. But there can be such a thing as being too realistic.

That’s one of the problems we were fighting in our Wild Things test. Here we had all of this fancy computer software that we had painstakingly tuned to convince people they were looking at reality, and now what we wanted was to convince people they were looking into a magical storybook. Not the same thing at all.

Fortunately, we had learned while making TRON that when you’re combining computer graphics with other things (like Jeff Bridges in a weirdly glowing spandex unitard), the trick is to modify the look of everything, so that it all meets in the middle. In the case of TRON the live action footage of the actors was deliberately given an eerie grainy look, and made to look hand-tinted – like something out of Fritz Lang’s “Metropolis” – and this same processing was done to the computer graphics backgrounds.

That, together with the “everything has red or blue glowing lines” motif, was very effective in marrying the foreground actors to the background computer graphics. The cranked-up graininess and stylized color palette masked any differences between computer graphics and physical props, and made you believe you were looking into some sort of consistently strange alternate universe – very different from the brightly lit, almost clinical, look of the scenes in TRON that were located in the “real” world outside the computer.

For Wild Things we were going for a very different look, but the principle was the same: marry foreground and background by visually stylizing both the computer graphics and the hand-drawn animated characters in a mutually consistent way. Today I’m going to talk about how we made the computer graphics backgrounds look like something out of a storybook. Then tomorrow I’ll talk about how we made the animated characters look like they were three dimensional.

When you look at a real object lit by a light source, you immediately see that the light only directly affects the portion of the object that faces the light. The portion of the object that faces away from the light source remains dark. So generally cinematic lighting revolves around using a bright light in the front (the “key light”) to highlight the roundness of each shape, and a softer backlight (the “fill light”) to give definition to the shape’s silhouette. Using such a technique, the bedknobs on Max’s bed might have looked something like this:

That looks very nice, but it doesn’t look particularly magical. When you look at Maurice Sendak’s drawing style, you eventually realize that he had a wonderful trick of keeping you a bit confused about where the light comes from. In his drawings, light seems to come from everywhere and nowhere all at the same time. So to make this test, I modified our shading system in several ways. For one thing, I changed it so that the highlight on an object didn’t need to be in the right place. You could make an object brighter on the left, while having the highlight seem to come from the right.

I also added an option to change the way objects in our 3D system react to light. Instead of only the part facing the light source getting brighter, I rigged it so that the light could seem to “wrap around” the object, bathing it in a softer light. This made the objects seem to glow a bit, like they do in Sendak’s illustrations. Finally, to capture the deliciously ominous feeling of the deep shadows Sendak adds to his illustrations, I added a feature that’s really impossible in reality – lights that can make things darker, instead of brighter. Using all of these techniques together, you could literally paint objects with light. So the bedknob on Max’s bed ended up looking more like this:

The difference is subtle but essential. The first bedknob looks like a simple but plausible 3D object. The second bedknob looks both real and not real, all at the same time. When an entire room is lit this way, the result looks not so much like a real place, but rather like an imagined idea of a real place. In other words, like the storybook version.

Here is a frame from the final test. With these lighting and shading tricks in place, the 3D scene seems to be slightly surreal, with a sense of drama and otherworldliness that would be hard to achieve using “physically correct” lighting:

The question still remained of how to take Glen Keane’s wonderful drawings of Max and the dog – which were really just pencil lines on paper – and make them appear to be rounded and three dimensional, so that they would look like they belonged together with the 3D backgrounds. And of course there was the little problem of how to make these pencil drawings cast believable shadows into a computer graphic scene.

Good topics for tomorrow!

The process we came up with to do the Wild Things test ended up being called Synthamation. In concept it was very simple – but God was in the details.

It all started with an animatic provided by the Disney team – a shot by shot view of the 3D background, created as a series of hand-drawn sketches. Then our artists at MAGI (Chris Wedge and Jan Carlée) worked from this animatic to create 3D models of everything – the room, hallway, bed, staircase, the little table with the lamp on it seen in the last shot.

Chris and Jan then computer animated a camera path through their 3D scene that matched the successive viewpoints of the hand drawn animatics. The result was rendered out frame by frame, not as a fully shaded background scene, but rather as a series of computer generated line drawings (called a “pencil test”). These pencil test frames of the 3D animated scene were printed out on big sheets of paper, so that the Disney artists could use them as a guide to rotoscope the frame by frame animation of the characters of Max and his dog.

Rotoscoping in those days was a process whereby an animator draws on paper while looking through a big piece of slanted glass. The animator sees the paper containing his own drawing through the glass, while simultaneously seeing another piece of paper containing a pencil test frame reflected by the glass. To the animator, it looks as though the two pieces of paper are superimposed on each other. So he can always see the computer generated pencil test of the background image, but the drawing he makes actually goes onto a clean white sheet of paper.

Here is where our team came up with a clever trick. When we rendered the pencil test, we added dummy versions of Max and his dog. These were only to guide the animator – they would not appear in the final animation. These dummy versions were really simple – each was just composed of a few simple shapes, as you can see in the image below. But that was enough to show the animator (in this case, Glen Keane) where to draw the characters for each successive frame of the animation.

The dummy characters also served another, more subtle purpose. In the final animation the hand-drawn characters are moving all around the scene – getting closer or further away, or running behind or in front of the staircase. The dummy characters allowed our programmers to know how far away to place the animated characters when they were finally composited into the scene. For example, in the final animation Max’s dog runs behind the staircase when he runs down the stairs – just like the dummy version of the dog.

When the animator was done, he had created various big stacks of drawings. Some of these drawings were of Max and some were of Max’s dog. We digitized each drawing into the computer at high resolution, using an Eikonix flatbed scanner. Once we had all the pieces scanned into the computer, all that remained to do was combine those pieces together in software, and add shading and lighting.

That last bit is not as simple as it might seem. We wanted the hand-drawn animated characters to appear not flat, but rather rounded and three dimensional – as though they were being lit along with the rest of the 3D scene. We also wanted them to cast shadows onto the floors and walls of the 3D computer generated background. And the background itself needed to have a kind of magical storybook appearance. To do all that, we needed to invent a few new techniques.

But that’s a topic for tomorrow.

Seeing Spike Jonze’s excellent film version of Maurice Sendak’s “Where the Wild Things Are” got me thinking back to the time that I helped get John Lasseter started in computer graphics.

I know that sounds completely weird, little old me helping to get the director of “Luxo Jr” and “Toy Story” into computer graphics, but it turns out to be true. It all happened in the months after Disney’s release of TRON, which unfortunately had not been a raging box office success. Nonetheless, since I was the young “coming up with crazy new ways to do things” guy at MAGI SynthaVision – the Westchester based computer graphics production house where we did the cool light cycles, game grid and more for TRON – I was flown out to Buena Vista, California to meet with Disney’s brilliant young animation director John Lasseter, where the two of us brainstormed ideas.

I was itching to try out some new techniques for making flat-shaded characters look rounded and 3D, and other techniques for making 3D graphics look more like hand illustrated storybooks. Meanwhile John wanted to do something that combined hand-drawn characters with the sorts of 3D worlds he’d seen in TRON. Together we came up with a suitably harebrained scheme.

I would lead a team at MAGI back in New York, where we would create a shaded 3D background animation, while John supervised a team over in California – with the great Glen Keane doing the character animation – to create hand-drawn animated characters that the MAGI team would magically integrate into the 3D backgrounds, with matching lighting, shadows and camera moves. None of the commercial 3D graphics software everyone now takes for granted existed back then, so we pretty much had to come up with new and sometimes unexpected ways to do everything.

Fortunately, I was working with an incredible group of fellow young turks, including Josh Pines, Christine Chang, Chris Wedge, Jan Carlée and Carl Ludwig. Every one of these people went on to amazing careers. Josh ended up going to ILM, where he revolutionized the process whereby film and computer graphics are combined together, so that you can’t tell which is which (for which he won a Technical Academy Award). Christine ended up at Don Bluth Studios in Ireland, and Chris, Jan and Carl went on to co-found Blue Sky Productions, makers of “Ice Age”, “Robots” and other visually stunning films.

Our Wild Things test was delightfully successful, and John became completely smitten with the possibilities of computer graphics. There is a video of our little test up on YouTube. Unfortunately, soon after the test was finished there was a political regime change at Disney headquarters, and John Lasseter – the fair-haired boy of the outgoing regime – ended up getting fired, for the crime of continuing to push this weird new computer graphics stuff (hard to believe now but true). Fortunately, John was able to take our little “Where the Wild Things Are” test over to Ed Catmull at LucasFilm as a calling card, where Ed (who may be the smartest guy I know) immediately brought him on as creative director of what would soon become Pixar. The rest, as they say, is – well, you know.

Some of the techniques we came up with in doing that test were extremely cool, and I don’t think they’ve been properly described anywhere. So I’m going to take the next few days to describe them here. Think of it as a rare window into the old “wild west” days of computer graphics, when computers ran slow, programmers ran fast, and we all just made it up as we went along, with nothing to go on but some pixels and a dream.