The software that I let you play with yesterday and the week before is not an isolated thing. It’s part of a larger set of ideas that we’ve been discussing in our Future Reality Lab at NYU about what the world might look like after everyone has those fully functional XR glasses about ten years from now.
Ever since I was a little kid and read Harold and the Purple Crayon, I thought to myself “Why can’t we just sketch things in the world to make them?” After playing around with this idea for a few years, I reached the conclusion that sketching in 3D space is really difficult, whereas sketching images of 3D things on a 2D surface is much easier and more natural.
So the general thought is that one day everyone will just be able to make quick and easy 2D sketches of whatever 3D thing they are trying to describe. Given the right technology to support all of this, that thing will then show up in the world and come to life for everyone to see.
Assuming everyone remembers to bring their glasses.