Ken's Blog

Here is the diagram I started to draw on the whiteboard at Google yesterday, although on the whiteboard I draw only the surrounding square and the 26 letters.

In the above image, you can see me in the process of drawing a letter “b”: I draw a stroke first to the upper left (where the “b” is located in the alphabet) and then veer to the right (where the “b” is located within its little cluster of letters).

Some very frequently occurring characters, like a, e, i, l, o and t, are just a single straight stroke. The others are all bent shapes.

The space character is just a click, and the capital letters you see in the dictionary are special characters: D for Delete, C for Caps, A for Alt keyboard (eg: most punctuation) and E for Enter.

Sketchtext is a variant of the Quikwriting system I wrote over twenty years ago, but with a particular emphasis on being able to sketch text in VR, in situations where you want to be able to spell things out without needing to look at your pen.

The view you are seeing is the tutorial view. It’s a very easy system to learn, because everything goes around the circle in alphabetical order, so when you’re using it, you don’t really need to see the dictionary.

I wonder if it will be better than Morse Code. 🙂

Today I was visiting Google, and at the very start of the meeting one of the Googlers — a young man I had never before met — started talking about the problem of writing text in VR.

As it happens, on the flight on the way over, I had implemented yet another gestural text entry system. Implementing these has been a hobby of mine for many years.

So I jumped up and scribbled my system on the whiteboard. Then he jumped up and scribbled his system on the whiteboard. Excited conversation and much scribbling ensued.

After a few minutes, we both sat down, feeling quite pleased with the exchange. Then we looked around the table and realized that everyone else had just been watching us.

My host — the person who had actually invited me — politely suggested that we make introductions before starting the meeting. “OK,” I said, feeling somewhat sheepish, “now that we’ve finished nerding out.”

I am not convinced that for creating text we will want to use keyboard, either real or virtual, in a future reality where millions of people wander around together in shared Virtual and Augmented reality. Perhaps we will simply move away from the use of text altogether.

After all, speech-to-text is now quite reliable, and faster than typing in many cases. Still, there is something appealing about using our hands rather than our mouths to create text. It allows us to work with text while continuing our conversation with other humans, which is very useful for collaboration.

Because of the recent emergence of VR at the consumer level, a lot of people are now thinking about the text input question. But what properties should a “virtual VR/AR keyboard” have?

One of the great things about using your hands to type on a QWERTY keyboard is that you don’t need to look at your hands. You can keep talking with other people, maintain eye contact, be able to absorb their body language, all while typing away.

I suspect that we will continue to value those two constraints: (1) the ability to continue talking with people while creating text, and (2) not needing to look at your hands while you are creating text. Exactly what form that will take, as VR and AR continue to go mainstream, only time will tell.

I saw Ready Player One with my nephews. It was delightful. My nephew Jonathan and I laughed out loud in all the same places.

I liked it so much, that I went to see it again, and this time I treated my grad students to see it. Both times in glorious 3D.

In addition to the obvious fact that Mark Rylance is one of the great wonders of the world, this is Spielberg in top pop form. He is having fun here, like he did in Minority Report and Catch Me if you Can, celebrating popular culture for its sheer visual magic.

I was curious to see what the critics thought, so I went on-line and read the reviews. And what I discovered was, for the most part, pure venom.

Many critics seem incensed, indignant, left sputtering in outraged at the very idea of a Spielberg film that is simply fun, a pop confection designed mainly to entertain and delight.

Perhaps some of it is their feeling of horror that modern pop culture might be something worthy of celebrating, simply for the sake of celebrating a phenomenon that many people find delightful. But why the extreme degree of venom?

I suspect it has something to do with critics’ feeling that they are the gatekeepers of culture. If you’ve ever listened to a classical music maven decry The Beatles and all that their influence has wrought, you probably know what I mean.

Fortunately, the movie itself has the courage to simply celebrate the sublime beautiful nuttiness of modern pop/nerd culture. If you haven’t seen this movie yet, I recommend you get yourself to a theater and see it.

Preferably in 3D.

Let’s review. We’re looking for a way to find a regular simplex — the simplest symmetric shape with flat boundaries — and we want our method to work no matter how many dimensions it has.

We already saw that to get a one dimensional simplex — a line — you go up to two dimensions and draw a line between the two points (1,0) and (0,1), keeping only the parts where all the coordinates are positive.

To get a two dimensional simplex — an equilateral triangle — you go up to three dimensions and draw a plane between the three points (1,0,0), (0,1,0) and (0,0,1), keeping only the parts where all the coordinates are positive.

It turns out this trick works in any number of dimensions. For example, suppose we want to get the three dimensional simplex — a regular tetrahedron.

You go up to four dimensions and draw a volume between the four points (1,0,0,0), (0,1,0,0), (0,0,1,0) and (0,0,0,1), keeping only the parts where all the coordinates are positive. In this picture, the x,y,z,w axes — all at right angles to each other — are in blue, and the resulting simplex is bounded by thick black lines:

Wait, what was that? Draw a volume???

Well, sure. Math doesn’t care how many dimensions you have. It doesn’t even care whether you can picture in your head the thing you’re talking about. It just does exactly what you tell it to.

And now, thanks to the statistically inspired math my brother showed me, I have a way to create a regular simplex in any number of dimensions — even a hundred dimensions, or a million. It may not make any visual sense to me, but I can describe it exactly.

And when you think about it, that’s pretty cool.

The logic of this way of looking at a regular simplex starts to become clear when we consider the regular simplex in two dimensions: the equilateral triangle.

Suppose there are three horses in the race, and the odds of each horse winning are, respectively, x, y and z. Since it is certain that one of the three horses will win, we know that:

      x + y + z = 1

But just like the race with two horses, we also know that the odds of any particular horse winning are at least zero. After all, there is no such thing as a negative probability. Well ok, sometimes there is, but not in horse races.

To my brother, the corresponding picture would look something like this:

In this three dimensional diagram, the diagonal plane indicated in red shows all the places where x+y+z=1. The equilateral triangle outlined in black shows all of the places within this plane where x, y and z are all at least zero.

Which means that triangle shape represents all the possible sets of odds for our horse race. It is also, not by coincidence, the shape of the regular simplex in two dimensions.

We’re starting to see a pattern here. Does this pattern continue to work for higher dimensions? We will look at that question tomorrow.

Because his work centers on statistics, my brother works a lot with probability. And the fundamental question of probability is: “Given several things that might happen, what are the odds that any particular one of them will happen?”

If two horses are racing (and if we don’t allow a tie), then we know for sure that one of those horses is going to win, and the other will lose. If the odds of the first horse winning are x, and the odds of the second horse winning are y, then:

      x + y = 1

where 1 is certainty (or 100%, as people often call it).

To my brother, this situation could be pictured something like this:

In the picture, x goes to the right, and y goes up. All of the possible outcomes of the horse race are along the diagonal black line that goes from bottom right to top left.

Further to the lower right (where x=1) it’s more likely that the first horse will win. Further to the upper left (where y=1) it’s more likely that the second horse will win.

That diagonal black line is exactly how my brother thinks of a regular simplex in one dimension — the shape that I drew as a horizontal line in yesterday’s post. For him, a one dimensional regular simplex isn’t just a line. It’s a long diagonal slice of a square.

The reason for this will become more clear when we go up one more dimension and talk about the three dimensional regular simplex — a triangle. Which we will do tomorrow.

In the world of computer graphics, it gets more and more difficult to place the vertices of a simplex as the number of dimensions increases. In one dimension, it’s easy. You can just draw a line from x = -1 to x = +1.

For two dimensions it’s a little more difficult, but still pretty easy. To create a regular triangle you go around a circle, marking off a vertex every third of the way around.

Starting with three dimensions it starts to get harder. To create a regular tetrahedron you need to resort to some fancy tricks. One trick is to first make a cube, and then use only half of its vertices. The edges of the tetrahedron are just diagonals of the cube’s faces.

But once you get to four dimensions, those tricks don’t really work anymore. Which is very discouraging if you like doing things with simplex shapes.

But then my brother told me how they make simplex shapes in his field, and it is completely awesome. More tomorrow.

It turns out that my brother and I, in our work, both deal with a mathematical object called a simplex. A simplex is just the simplest thing with straight sides in any particular number of dimensions.

For example, in one dimension a simplex is a line. In two dimensions it is a triangle, and in three dimensions it is a tetrahedron (a three sided pyramid).

In each case, the number of vertices (or corner points) is one more than the number of dimensions. For example, a line has two vertices (one on each end), a triangle has three, and a tetrahedron has four.

In my work in computer graphics, I have often needed to create a simplex shape. But once things go beyond two dimensions, it can get hard to figure out where to put the vertices.

Or at least I thought so, until I talked about simplex shapes with my brother. Because he comes from the world of statistics, he had a whole different way of looking at the shape of a simplex. More tomorrow.

The other day I spent time with my brother, and as usual we ended up discussing mathematics. We both use math in our work, but because our work is very different, he and I end up using it differently.

My brother develops powerful techniques for analyzing and identifying DNA. For example, after the attack on the World Trade Center, our federal government turned to him to help them properly identify what remained of the victims.

Meanwhile, I just do computer graphics. So you can see why I think of my brother as the Mycroft in the family.

Because he uses math for statistical analysis, and I use it to make pictures, it would seem that we would find little common ground. Yet it turns out that we are often thinking about the same mathematical objects. We just think about them very differently.

Tomorrow I will give an example.