Month: November 2019

I was talking today with somebody who works with people who have various kinds of ASD. She was describing how walking around in an urban environment can be difficult for people on the spectrum.

Perhaps a good way to think about it is as an I/O problem. Signals from reality — such things, as bright lights, sudden noises and rapid movement — can be processed by most people without causing a crisis.

But it your brain is wired to process input differently, then those same signals can be overwhelming. They tend to dominate your experience of a situation, making everything more difficult.

For this reason, many people on the spectrum enjoy hanging out in virtual worlds, such as Second Life. You are still interacting with people and enjoying their company, but you can place a more comfortable and manageable filter on the sensory experience itself.

This made me think of my own experiments in creating a parallel version of our lab that maps one-to-one to the physical lab. While our lab has many visual distractors — like junk piled on desks and lighting in all sorts of odd places — my parallel virtual version can be very visually clean and calming.

In wonder whether the sensation of walking around in the physical world will become easier for people on the spectrum in a few years. You just put on your extended reality glasses in the morning, dial down the noise in your visual field to a comfortable level before leaving the house, and you’re good to go.

I wonder whether there is a connection between disparate experiences that simulate discomfiting situations in the service of entertainment. On the one hand, video games, sky diving and horror movies are three examples of activities that don’t pose any actual danger, yet create the illusion that danger is lurking just around the corner.

Having gone sky diving myself, I can assure you that it is quite safe. Yet dropping out of an airplane from an altitude 12,000 feet certainly makes you feel as though you are putting your life on the line.

On the other hand, in most comedy films and plays, we see other people in extremely discomfiting situations. In that moment, we would not wish to be those people, and yet we can derive great pleasure from laughing at their predicament.

I wonder whether the first-person simulated discomfort of thrill experiences and the third-person simulated discomfort of comedy are somehow connected. Is there a section of our brain that seeks to be thrown out of balance?

Maybe in each case the ultimate payoff is the tame: The serotonin rush which comes from realizing that everything is, after all, ok.

Today on our Future Reality Lab blog I wrote about using VR to create parallel worlds. It’s a theme I’ve discussed in different ways on this blog.

Suddenly it occurred to me my simultaneous presence on both this blog and in our FRL blog is also a parallel world. Except that the worlds of these two blogs are parallel along a different dimension.

So here we have two parallel worlds being discussed in two parallel worlds. I wonder, what do you call it when you send a message out into the world that is parallel along multiple dimensions?

I know — a parallelogram!

There are primates who can easily pick up tools with their toes. Others can swing by their tails.

Humans cannot do these things. For one thing, our bodies are not constructed to do so.

But is this a limitation only of our bodies, or also of our brains? Do we have the latent brain capacity to pick things up with our toes or to swing from trees by a tail, if only we had the right body to do so?

Once we start using our full bodies to immerse ourselves into shared virtual worlds, this will start to become a practical question. Just how general is our brain’s ability to remap itself to a different body, if given the opportunity?

And if our brains do turn out to be adaptable in this way, what are the limitations of the mapping? Could we learn to be comfortable in the body of an amoeba, forming and extending temporary pseudopods as needed?

Could we adapt to an entirely alien body? That could be useful for exploration of other planets or deep sea environments.

It’s exciting to think that we humans might someday have the sensory experience of entering entirely alien bodies. But it is not yet clear whether such a thing is truly possible. I guess we will need to try it out and see.

As we move various aspects of our social life to shared immersive virtual reality, we don’t all need to present as the same size. It might be convenient, for various reasons, for somebody to appear 20 feet tall, while a group of five people might all fit on a tabletop.

I don’t know for sure whether this will happen, but it’s a reasonable thing to think about. After all, the next few years will be the first time in history when we will have the capability to hang out together everyday in a physical sense while defying the rules of nature.

We are used to seeing people at wildly different scales on what are now considered traditional media. Paintings, photographs, movie theaters, TV sets, smartphone screens, all of these forms of visual communication wreak havoc with the notion of immutable human scale.

Yet somehow our brains adjust to the sight of a movie star’s face being 30 feet tall at the cinema, while the friend we are chatting with on our smartphone has a face that is only two inches in height. None of it seems to bother us.

I suspect there is a part of our brains that automatically maps whatever we perceive as human to a “normal” size, on such a low level that we don’t even notice it happening. I see no reason why this perceptual transformation should not carry over into shared immersive worlds.

In any case, it’s an experiment worth carrying out. And it’s definitely something we are interested in trying in our Future Reality Lab at NYU.

Today at our lab’s weekly research meeting, I realized we are on the cusp of a new phase in our research. When you are doing research, it’s not only about what you might potentially develop, it’s also about whether you have the proper tools for that development to be practical.

After months of work, we are in sight of having a new set of tools that are far more powerful and flexible than anything we have had before. An analogy might help.

Imagine you are testing out a new automobile, and the only thing you can do is plot a course for the car beforehand, set it on the road, and see afterward whether the car has crashed. You might develop a good automobile using that approach, but your task will be very difficult.

Now compare that with the ability to actually get in the car, put your foot on the accelerator, and drive it yourself, turning the steering wheel as needed to travel to different places.

What we are developing now is basically like going from that first scenario to the second. Using these new tools, it’s going to be a lot more fun to do our research.

Also, we might end up traveling to places we never knew existed.

I was so proud of myself for getting my research to work yesterday using only the plastic lids of coffee cups. I happily showed my research result to people in the lab, and everybody liked it.

But then today I realized that I don’t actually need even the coffee cop lids. The whole thing works just fine even with virtual coffee cup lids. The physical parts are not really necessary.

I am not sure how I feel about this. On the one hand, it’s great to realize that there is a software-only solution. On the other hand, it was really fun to think I was able to do cutting edge research with cheap plastic coffee cup lids from our local take-out deli.

Oh well, at least the new technique will be eco-friendly.

Today I realized I needed a physical object of a certain size and shape for my research. I was thinking that in order to get the shape I wanted, I would need to run a 3D print job.

I really didn’t want to do that because it would be both complicated and expensive. But where was I to get a 3D shape that did exactly what I want?

Pondering this question, I went to our lab kitchen to make myself a coffee.

And that’s when I saw it: It turns out that the plastic coffee lids in our kitchen are exactly the right size and shape for my experiments.

There are other benefits as well: The total cost of materials for a plastic coffee lid is less than a penny.

I felt excited by this discovery, and newly energized to continue my research. In fact, I was so happy, I forgot to drink the coffee.

I had occasion today to look back at emails from 2008. It was a very weird experience.

Back then I was having near daily personal and professional conversations with people I have not spoken to for many years. Just reading those emails brought me back to a another time in my life, when my priorities were very different.

The most haunting email exchanges were the ones I was sharing with a close friend who has since passed away. Of course there was no way of knowing back then that tragedy was looming only a few short years away.

We are who we have always been, and yet we are also constantly in motion. Conversations between our past and our current selves can be a kind of time machine — a deeply revealing one — but they are never easy.

I am not sure I always have the courage it takes to hold such a conversation honestly. But I’d like to think I can rise to the occasion.

To test my little VR laboratory, I gave myself 10 minutes to implement the very beginnings of a VR geometric modeling system. It’s not much, but it’s a good start.

You can see me trying it out by clicking on the image below.

I am encouraged by the fact that it took so little time to get this far. I am looking forward to building out other capabilities, like snapping objects together, changing colors and textures, designing custom forms and creating animated creatures.

I don’t expect that I will end up with a modeler that will do everything. But I do expect to learn a lot in the process.

Besides, it’s really fun. 🙂