Month: October 2023

Today I unpacked my new Met Quest 3, and started playing with it. It is even better than I had thought it would be, in every way.

Coincidentally, about an hour before the Quest 3 arrived, I was meeting with one of my students, and the topic of extended reality came up. I told him about something that I used to say in talks I gave about 20 years ago.

I would say back then that one day we would be able to seamlessly integrate CGI into our everyday lives. And I would add that whenever that day finally game, it would be the beginning of the age of computer graphics.

Welcome to the dawn of the age of computer graphics.

Today I was flying into LaGuardia airport, and on a whim I took out my smartphone and opened up Google maps. From that point until landing, I happily tracked all the locations that we were flying over, identifying many interesting places in the vicinity of New York City that I had never before thought about.

It was definitely an enriching experience. I only wonder why I had never thought to do it before.

I realize that in a few years the smartphone won’t be necessary. I will just be able to look out of the window of a plane while wearing my smart glasses, and the map, with the style and filters of my choosing, will be superimposed directly on reality itself.

Currently there is a general idea that either you are conversing with a human or else you are conversing with a chatBot that is faking being a human. But what will happen after that binary distinction no longer makes sense?

Eventually, people will use A.I. to augment their conversation in more granular ways. I may start a thought but then let my A.I., which has been trained on my writings and thought processes, fill out the details.

This is already happening to some extent. Students are using ChatGPT as a helper when writing essays. The resulting essay isn’t entirely written by either student or computer, but rather by a combination of the two.

At some point, when the technology is there to support it, this mode of operation will begin to work its way into even the most casual and spontaneous conversations. I wonder whether that will be a good thing or a bad thing.

I love it in movies and TV shows when something happens in a character’s dream, and then afterward shows up in the character’s real life. In fact, that happened in the most recent episode of Only Murders in the Building.

In a rational sense, of course this is impossible. Yet when we see it on the screen it seems to make perfect sense. Technically, such a moment is transforming what we had thought was a true-to-life fictional world into a fantasy fictional world.

But isn’t that the point? When we watch something on the screen, we are indeed receiving it the way we receive a dream. On a sensory level, the end result of the techniques of camerawork and editing is not like real life. It is like our dreams.

So when we see something impossible happen, such as a dream foreshadowing reality, we accept it, and we even celebrate this momentary escape from the strictures of real life.

Because after all, isn’t that why we go to the movies?

Pretty much everybody I know has a smart phone, and everybody I know has access to a Web browser. That has very important implications for how we think about and use smart phones and Web browsers. Ubiquity creates community.

Most people I know do not have a virtual reality headset, and even fewer have mixed reality headsets. So there isn’t that assumption of community between everyone.

I’ve been wondering what things might be like when everyone we know has a pair of mixed reality glasses, and we can make that assumption of community. At that point, I am guessing that there will be entirely new types of apps that everyone will use for such devices, much as most of us now use Google Maps on our phones.

But what will those apps be? Maybe we can start to think about that now.

To teach the principles of ray tracing with reflection and refraction in fragment shaders for my computer graphics students, I implemented a little scene of glass spheres bouncing off each other (click on the image below to try it).

It runs just fine on my MacBook Pro. But oddly, it has all sorts of artifacts when I try to run it on my Android Pixel Phone.

I wonder what is causing the difference.

There is currently an enormous disparity within the U.S. in the money spent per child on K-12 education. Some fortunate children get high quality books, computers and well paid teachers. Other children barely get workable bathrooms.

In a rational society, we would all understand that every child has a fundamental right to a high quality K-12 education. Whatever is needed to support that right would be viewed as a necessity, regardless of the economic circumstance of that child’s family.

Of course that would cost a lot of money, which would largely be paid by taxes. But in the long run, this would be the best investment we could make with our taxes. The benefits to our nation of a generation of children who can each become an effective engine of the economy would pay for itself many times over.

Unfortunately, we do not live in a rational society.

Eventually we will get past the novelty of mixed reality. You will just put on your smart glasses in the morning (or not), the way you now slip your smart phone in your pocket in the morning (or not). It won’t be something you particularly think about.

But it will mean that things that we can’t do now will become possible. In one fun example, we will be able to build and play with 3D puzzles that couldn’t exist in the physical world.

I’ve begun working on designing these sorts of puzzles on my mixed reality headset. I will let you know if I come up with anything really good.

I usually agree with everything John Carmack says. But this time I don’t.

He recently wrote that Meta is making a mistake by bringing out a consumer level headset that focuses so much on mixed reality, rather than virtual reality. But I think he has it backwards.

Apple’s Vision Pro is clearly aspirational. Millions of people are not going to run out and spend their money on a headset that costs $3500.

But Meta’s Quest 3, at $500, is a very different value proposition. It is priced to be a popular Christmas stocking stuffer. This thing is meant for a mass market.

John Carmack thinks this is a mistake. I disagree. The only way these head worn devices will survive in the long run will be to jump entirely out of the category of “device for hardcore gamers.”

Adding good color pass-through and a depth sensor at an affordable price makes this the first really practical headset in an entirely different category: , one that is focused not on escapism, but on bringing enhancements to our actual lives.

And I think that is exactly the right move.

Back in the old days, even famous people could appear only as a hand-rendered drawing, painting, etching or sculpture. If a person was dead, you could never be sure exactly how they really looked when alive.

Then, after around 1839, images of famous people could be captured photographically, without the need for manual rendering. Long even after someone was dead, you could indeed see exactly how they used to look, by looking at their photo.

Eventually this ability to capture, for all time, the fragile and fleeting nature of life extended to movement, when moving pictures were introduced.

Now that we are all getting used to Generative AI, it is not difficult to envision a time when the equivalent of a “photograph” will be a lifelike 3D representation of a person that you can have a conversation with. This virtual person will respond to any questions you have with the general manner and opinions of the original.

During such interactions, the person’s facial movements, body language and vocal inflections will be faithfully reproduced. You will quickly be able to get an accurate sense of their personality, even if they died long before you were born.

I wonder what this form of representation will be called. Maybe it will just be called a photograph.