Ken's Blog

I was at a professional dinner this week in which our hosts set the scene by lowering the lights and running a giant monitor which showed a video of a fireplace. It was kind of like that yule log video, but without the yule.

After dinner there were speeches, which at some point I started tuning out, as one does. Instead I focused on the fireplace videos.

First I set out to determine how long the video loop was, and exactly when it repeated. That turned out to be easy, because at some point a little spark floated up from one of the logs, froze in place for a moment mid-screen, and promptly vanished.

Counting off the seconds in my head, I determined that this little moment was repeating every thirty seconds. Which means it must have been repeating after exactly thirty seconds, because nobody makes a 29 second or 31 second video loop.

This was great, because it meant I could make a game out of it. As each loop repeated, I counted off seconds in my head, seeing how close I could get to reaching a count of 30 at exactly the right moment.

By the time the speeches ended, I had gotten really good at this. Using the fireplace video as my guide, I could now mentally count off seconds at a rate of exactly one second per count. I am sure this is a skill that will come in handy on many occasions.

Not having listened to the speeches, I am not in a position to judge whether they were good. But the other people in the room seemed really happy.

Which means that the other people in the room had really liked the speeches. Either that, or they had also been mentally counting off the seconds until the speeches were finished.

I was harried today, it is true
And this blog was one more thing to do
Yet I value you most
So dear readers, I post!
I would never forget about you 🙂

One of the hardest lessons to learn is the importance of ignoring what other people think about your research. It’s not that you shouldn’t be aware of their opinions. It’s more that you need to learn to trust your own instincts more.

The problems we work on in our lab are generally so outside of the norm that many people mislabel our work. Other people are simply confused about what we are doing.

I consider that a good thing. It means there is a good chance we are working on something that will actually be relevant in five or ten years.

I can’t speak for other fields, but I’ve learned a truism in my own field of research: If everybody understands what you are working on, you are probably working on the wrong thing.

At the lab today I was using my little procedural modeling system to create some computer graphic shapes. These are things that will go into my virtual reality world. While I was doing this, I was listening with one ear to a conversation of some of my colleagues.

They were discussing snacks to get for a little end-of-week get-together tomorrow. We do those here from time to time, and it cheers people up, especially toward the end of the semester when work can get very intense.

One of my colleagues said that we should get baby carrots. It took me about 10 seconds to modify the thing I was creating — changing the proportions, the color and the lighting — to make it look like a first approximation to a baby carrot.

“Look,” I said, “a baby carrot!”

It doesn’t look exactly like a baby carrot. But for 10 seconds of work, it’s not too bad.

I was talking today with somebody who works with people who have various kinds of ASD. She was describing how walking around in an urban environment can be difficult for people on the spectrum.

Perhaps a good way to think about it is as an I/O problem. Signals from reality — such things, as bright lights, sudden noises and rapid movement — can be processed by most people without causing a crisis.

But it your brain is wired to process input differently, then those same signals can be overwhelming. They tend to dominate your experience of a situation, making everything more difficult.

For this reason, many people on the spectrum enjoy hanging out in virtual worlds, such as Second Life. You are still interacting with people and enjoying their company, but you can place a more comfortable and manageable filter on the sensory experience itself.

This made me think of my own experiments in creating a parallel version of our lab that maps one-to-one to the physical lab. While our lab has many visual distractors — like junk piled on desks and lighting in all sorts of odd places — my parallel virtual version can be very visually clean and calming.

In wonder whether the sensation of walking around in the physical world will become easier for people on the spectrum in a few years. You just put on your extended reality glasses in the morning, dial down the noise in your visual field to a comfortable level before leaving the house, and you’re good to go.

I wonder whether there is a connection between disparate experiences that simulate discomfiting situations in the service of entertainment. On the one hand, video games, sky diving and horror movies are three examples of activities that don’t pose any actual danger, yet create the illusion that danger is lurking just around the corner.

Having gone sky diving myself, I can assure you that it is quite safe. Yet dropping out of an airplane from an altitude 12,000 feet certainly makes you feel as though you are putting your life on the line.

On the other hand, in most comedy films and plays, we see other people in extremely discomfiting situations. In that moment, we would not wish to be those people, and yet we can derive great pleasure from laughing at their predicament.

I wonder whether the first-person simulated discomfort of thrill experiences and the third-person simulated discomfort of comedy are somehow connected. Is there a section of our brain that seeks to be thrown out of balance?

Maybe in each case the ultimate payoff is the tame: The serotonin rush which comes from realizing that everything is, after all, ok.

Today on our Future Reality Lab blog I wrote about using VR to create parallel worlds. It’s a theme I’ve discussed in different ways on this blog.

Suddenly it occurred to me my simultaneous presence on both this blog and in our FRL blog is also a parallel world. Except that the worlds of these two blogs are parallel along a different dimension.

So here we have two parallel worlds being discussed in two parallel worlds. I wonder, what do you call it when you send a message out into the world that is parallel along multiple dimensions?

I know — a parallelogram!

There are primates who can easily pick up tools with their toes. Others can swing by their tails.

Humans cannot do these things. For one thing, our bodies are not constructed to do so.

But is this a limitation only of our bodies, or also of our brains? Do we have the latent brain capacity to pick things up with our toes or to swing from trees by a tail, if only we had the right body to do so?

Once we start using our full bodies to immerse ourselves into shared virtual worlds, this will start to become a practical question. Just how general is our brain’s ability to remap itself to a different body, if given the opportunity?

And if our brains do turn out to be adaptable in this way, what are the limitations of the mapping? Could we learn to be comfortable in the body of an amoeba, forming and extending temporary pseudopods as needed?

Could we adapt to an entirely alien body? That could be useful for exploration of other planets or deep sea environments.

It’s exciting to think that we humans might someday have the sensory experience of entering entirely alien bodies. But it is not yet clear whether such a thing is truly possible. I guess we will need to try it out and see.

As we move various aspects of our social life to shared immersive virtual reality, we don’t all need to present as the same size. It might be convenient, for various reasons, for somebody to appear 20 feet tall, while a group of five people might all fit on a tabletop.

I don’t know for sure whether this will happen, but it’s a reasonable thing to think about. After all, the next few years will be the first time in history when we will have the capability to hang out together everyday in a physical sense while defying the rules of nature.

We are used to seeing people at wildly different scales on what are now considered traditional media. Paintings, photographs, movie theaters, TV sets, smartphone screens, all of these forms of visual communication wreak havoc with the notion of immutable human scale.

Yet somehow our brains adjust to the sight of a movie star’s face being 30 feet tall at the cinema, while the friend we are chatting with on our smartphone has a face that is only two inches in height. None of it seems to bother us.

I suspect there is a part of our brains that automatically maps whatever we perceive as human to a “normal” size, on such a low level that we don’t even notice it happening. I see no reason why this perceptual transformation should not carry over into shared immersive worlds.

In any case, it’s an experiment worth carrying out. And it’s definitely something we are interested in trying in our Future Reality Lab at NYU.

Today at our lab’s weekly research meeting, I realized we are on the cusp of a new phase in our research. When you are doing research, it’s not only about what you might potentially develop, it’s also about whether you have the proper tools for that development to be practical.

After months of work, we are in sight of having a new set of tools that are far more powerful and flexible than anything we have had before. An analogy might help.

Imagine you are testing out a new automobile, and the only thing you can do is plot a course for the car beforehand, set it on the road, and see afterward whether the car has crashed. You might develop a good automobile using that approach, but your task will be very difficult.

Now compare that with the ability to actually get in the car, put your foot on the accelerator, and drive it yourself, turning the steering wheel as needed to travel to different places.

What we are developing now is basically like going from that first scenario to the second. Using these new tools, it’s going to be a lot more fun to do our research.

Also, we might end up traveling to places we never knew existed.