Ken's Blog

When we try to describe the new immersive cinematic medium that our Future Reality Lab is creating, exemplified by CAVE at SIGGRAPH 2018, it’s difficult to properly convey the idea to people who haven’t seen it.

We can’t just say something like this: “It’s like going to the movies crossed with theater crossed with being in a video game.” If we do that, people usually get the wrong idea about what we’re up to. They think that it’s a video game, or that it’s like Facebook Spaces or High Fidelity, when it’s really very different from any of those things.

I find myself imagining a futurist in 1926, in days of future past, trying to explain the soon to be released film The Jazz Singer. Maybe this visionary could say: “It’s like a movie, but with sound.”

But that wouldn’t work, because movies already had sound. Pretty much every movie you went to back then had a score, and a very talented organist playing along.

Maybe the futurist could say: “It’s like a movie, but people are talking.” But that wouldn’t work, because actors already talked in movies. And the intertitles were right up there on the screen, to tell you what they were saying.

Maybe it would have worked best if the futurist had said: “It’s like a movie crossed with radio.” That might have sounded completely crazy to people in 1926, but they probably would have gotten the idea.

We need to find our equivalent, for what we are doing in 2019, to “It’s like a movie crossed with radio.”

Come Saturday morning, I find myself sadly reading about the death of Andre Previn. Which makes me think of Dory Previn, which makes me think of the song Come Saturday Morning, for which she wrote the lyrics, which makes me think of the movie The Sterile Cuckoo, for which that was the theme song, which makes me think of Liza Minelli, who starred in that movie.

The film is a tender coming of age story in which Liza plays a young woman who falls in love with a man who seems also to be in love with her. But then it turns out he’s more attracted to men than to women.

Liza then went on to superstardom in Cabaret, a musical drama set in Germany between the Wars in which she plays a young woman who falls in love with a man who seems also to be in love with her. But then it turns out he’s more attracted to men than to women.

During the years when both these films were produced and released, Liza Minelli was married to singer/songwriter Peter Allen. He seemed also to be in love with her. But then it turned out he was more attracted to men than to women.

Once, in my younger days, I took a girl I liked to a Peter Allen concert. At some point in the middle of the concert the young man sitting on the other side of me, to my great surprise, made a pass at me.

I thought it was very rude of him. Regardless of his theories about my preferences, it was obvious I was there with someone else.

Now, many years later, as I look back on the semiotics of all this, a different theory comes to mind. Perhaps that young man wasn’t insulting my date, but subtly complimenting her.

Perhaps that was just his roundabout way of suggesting that the young woman beside me was the Liza Minelli of our generation.

Our final step adds the real magic sauce: Simulating both wind and coriolis effects. Both of these will be done via shameless cheats.

In fact, we are not going to simulate the physics at all. Instead, we are just going to simulate the appearance of wind and coriolis effects.

We do this by adding just two lines to our shader code:

   T.z += .1 * uTime;    // MOVING TEXTURE FAKES WIND.
   T.y += noise(.5 * P); // UP/DOWN MOTION FAKES CORIOLIS.

The first line of code simply slides the entire texture perpendicular to the screen. This has the effect of creating smooth yet unpredictable changes over time — exactly the visual impression we would get from wind forces across the planet surface.

The second line of code makes different parts of the texture travel up and down in latitude as the texture slides past the planet. The wispy cloud texture flows upward in some places, while flowing downward in other places — exactly the visual impression we would get from coriolis forces across the planet surface.

We have now rebuilt the complete cloud-roiled planet texture, step by step. The code looks slightly different from the original code, but it does exactly the same thing. You can see the live animation, together with the editable code that generates it, by CLICKING HERE.

A cloudy planet where the clouds don’t move is boring. After all, one of the fun things about clouds is that they move and shift around and change shape over time.

The next step doesn’t look very different from the previous one, except that now it is starting to move:

So far the movement is very simple. The cloudy planet is just spinning.

But after all, as we know from real life, a spinning planet leads to coriolis forces, and those forces result in far more interesting cloud movements.

If you look at the code, you can see that all I’ve done is use sine and cosine functions to make the x (left-to-right) direction and the z (back-to-front) direction spin around each other. You can see all of this running live, together with the code that generated it, by CLICKING HERE.

Today we deal with the problem of the cloud cover not being wispy. The trick we do that is to combine textures together.

Rather than just use a turbulence texture, we use that turbulence texture to highly distort a stripe pattern (like the way things look wavy when seen through a mirror that isn’t flat). The result doesn’t look like stripes, because those stripes are so highly distorted, but the result ends up having the wispier look we were aiming for:

In the code, the stripes are created by a sin function. I put the new code right after the old code, so that you can compare them to see the difference. You can see the live version, with the code that generates it, by CLICKING HERE.

Fortunately, we can do better than simple noise texture. Back when I came up with noise, I also used it to build some other interesting functions. One of them was the turbulence function, which creates a kind of fractal texture.

Here you can see I’ve replaced noise with turbulence:

Now the texture looks a lot more like cloud cover. But it doesn’t yet have that delicate wind-blown wispy quality, and it doesn’t yet animate. You can see it live, together with the code that generates it, by CLICKING HERE.

The first thing we can do is throw on a simple noise texture to use as our cloud cover. As you can see, that by itself is pretty boring.

It doesn’t really look anything like cloud cover. For one thing, it’s lacking that wonderful wispy quality that clouds have. For another, it’s not animated.

In order to do that, we are going to be a lot more subtle in our use of procedural textures. Meanwhile, you can see this example live, together with the editable code that generates it, by CLICKING HERE.

I thought it would be interesting, for those who are interested in these sorts of things, to do a step by step breakdown of how I made that animated cloud texture for the cloudy planet I showed the other day. I plan to do that over the next several days.

Taking things step by step, first we start with a planet that has no cloud cover at all. All the lights are there, and it’s got the right watery surface color — just no texture.

This is where we will begin. To see this running live together with the code that generates it, CLICK HERE.

A lot of wonderful people have passed away recently, and each death has made the world a poorer place. But Stanley Donen, who passed away this week at the age of 94, has a special place in my heart.

Donen directed On the Town, Royal Wedding, Singin’ in the Rain, Seven Brides for Seven Brothers, Funny Face, The Pajama Game , Damn Yankees, Charade, Two for the Road, Bedazzled and The Little Prince, among his many films.

From the time I was a small child, a Stanley Donen film was, to me, the epitome of Hollywood magic. Not the new kind of magic we see today, created with digital special effects, but an older kind — the kind they don’t produce any more — made of pure fairy dust and moonbeams.

As a reward for all the joy he gave to this world, I hope that Mr. Donen is now reunited with Gene and Fred. I can picture them together, dancing with grace and style across a ceiling up in Heaven, or splashing happily through a celestial rain puddle, all in glorious Technicolor.

While I was preparing for that class where I created the orange, I spent a little time just noodling around with shader code. I didn’t have any particular goal in mind other than to have fun and see what would come out.

What came out is something that looks like this, except animated (it’s a lot cooler when you see it animated):

The code that creates this fits in just a page of text. You can see both the code and the animation it creates if you click HERE.

If you’re looking at the shader code, the most interesting section is that four line stretch surrounded by blank lines. That’s where all of the swirling cloud texture is created.

If you like programming, feel free to try editing the code and see what happens. If your changes make the whole thing stop running, you can always just refresh the page and try again.