Ken's Blog

I love it in movies and TV shows when something happens in a character’s dream, and then afterward shows up in the character’s real life. In fact, that happened in the most recent episode of Only Murders in the Building.

In a rational sense, of course this is impossible. Yet when we see it on the screen it seems to make perfect sense. Technically, such a moment is transforming what we had thought was a true-to-life fictional world into a fantasy fictional world.

But isn’t that the point? When we watch something on the screen, we are indeed receiving it the way we receive a dream. On a sensory level, the end result of the techniques of camerawork and editing is not like real life. It is like our dreams.

So when we see something impossible happen, such as a dream foreshadowing reality, we accept it, and we even celebrate this momentary escape from the strictures of real life.

Because after all, isn’t that why we go to the movies?

Pretty much everybody I know has a smart phone, and everybody I know has access to a Web browser. That has very important implications for how we think about and use smart phones and Web browsers. Ubiquity creates community.

Most people I know do not have a virtual reality headset, and even fewer have mixed reality headsets. So there isn’t that assumption of community between everyone.

I’ve been wondering what things might be like when everyone we know has a pair of mixed reality glasses, and we can make that assumption of community. At that point, I am guessing that there will be entirely new types of apps that everyone will use for such devices, much as most of us now use Google Maps on our phones.

But what will those apps be? Maybe we can start to think about that now.

To teach the principles of ray tracing with reflection and refraction in fragment shaders for my computer graphics students, I implemented a little scene of glass spheres bouncing off each other (click on the image below to try it).

It runs just fine on my MacBook Pro. But oddly, it has all sorts of artifacts when I try to run it on my Android Pixel Phone.

I wonder what is causing the difference.

There is currently an enormous disparity within the U.S. in the money spent per child on K-12 education. Some fortunate children get high quality books, computers and well paid teachers. Other children barely get workable bathrooms.

In a rational society, we would all understand that every child has a fundamental right to a high quality K-12 education. Whatever is needed to support that right would be viewed as a necessity, regardless of the economic circumstance of that child’s family.

Of course that would cost a lot of money, which would largely be paid by taxes. But in the long run, this would be the best investment we could make with our taxes. The benefits to our nation of a generation of children who can each become an effective engine of the economy would pay for itself many times over.

Unfortunately, we do not live in a rational society.

Eventually we will get past the novelty of mixed reality. You will just put on your smart glasses in the morning (or not), the way you now slip your smart phone in your pocket in the morning (or not). It won’t be something you particularly think about.

But it will mean that things that we can’t do now will become possible. In one fun example, we will be able to build and play with 3D puzzles that couldn’t exist in the physical world.

I’ve begun working on designing these sorts of puzzles on my mixed reality headset. I will let you know if I come up with anything really good.

I usually agree with everything John Carmack says. But this time I don’t.

He recently wrote that Meta is making a mistake by bringing out a consumer level headset that focuses so much on mixed reality, rather than virtual reality. But I think he has it backwards.

Apple’s Vision Pro is clearly aspirational. Millions of people are not going to run out and spend their money on a headset that costs $3500.

But Meta’s Quest 3, at $500, is a very different value proposition. It is priced to be a popular Christmas stocking stuffer. This thing is meant for a mass market.

John Carmack thinks this is a mistake. I disagree. The only way these head worn devices will survive in the long run will be to jump entirely out of the category of “device for hardcore gamers.”

Adding good color pass-through and a depth sensor at an affordable price makes this the first really practical headset in an entirely different category: , one that is focused not on escapism, but on bringing enhancements to our actual lives.

And I think that is exactly the right move.

Back in the old days, even famous people could appear only as a hand-rendered drawing, painting, etching or sculpture. If a person was dead, you could never be sure exactly how they really looked when alive.

Then, after around 1839, images of famous people could be captured photographically, without the need for manual rendering. Long even after someone was dead, you could indeed see exactly how they used to look, by looking at their photo.

Eventually this ability to capture, for all time, the fragile and fleeting nature of life extended to movement, when moving pictures were introduced.

Now that we are all getting used to Generative AI, it is not difficult to envision a time when the equivalent of a “photograph” will be a lifelike 3D representation of a person that you can have a conversation with. This virtual person will respond to any questions you have with the general manner and opinions of the original.

During such interactions, the person’s facial movements, body language and vocal inflections will be faithfully reproduced. You will quickly be able to get an accurate sense of their personality, even if they died long before you were born.

I wonder what this form of representation will be called. Maybe it will just be called a photograph.

Jim Dale once said “You cannot learn anything from success, you only learn from failure.” And today I experienced a failure — something professional that I put a lot of work into, which I thought was going to be accepted, ended up being rejected.

There was a moment of numbness, followed by about an hour of sadness and unhealthy snackage. But then things changed.

I started getting to work, building things in places where I had lazily neglected to build things. Suddenly I am finding myself industrious, on mission, productive.

None of these useful things would have happened had I received good news rather than bad news. I most likely would have coasted along on that success, until eventually getting tripped up by my entitled laziness.

Jim Dale was right.

Today, around the corner from my Manhattan apartment, two men in hassidic garb had a little table set up.

As I walked past, one of them said to me “Excuse me, are you Jewish?”

Responding in kind, I asked him “Are you Jewish?”

“Yes,” he replied.

“Funny,” I said, “you don’t look Jewish.”

And then I continued on my way.

Magic is a funny thing. When we see a magician perform, there is a mutual understanding between performer and audience that what is being presented is an artful illusion.

We also eagerly embrace fictional representations of real magic, like Harry Potter or Bewitched. In a way, this is the same thing: We are watching performers pretend that magic is real, yet there is an unspoken understanding that it is all make-believe.

But what if we were to encounter real magic? Would that completely disrupt our understanding of reality?

Or would we simply relabel it “physics”? In other words, is the term “real magic” an oxymoron? Is magic, by definition, that which cannot actually exist in reality?