For the last two years I have been working, with my students, on two related projects. One, called Holojam, allows people to walk around together in the same physical room, wearing very lightweight VR headsets, to share a kind of radical augmented reality: We are all physically together, but we are all visually sharing a fantasy world, as though we have entered the Star Trek Holodeck.
The other project, called Chalktalk, allows drawings to come to life, sort of like a real-life version of Harold and the Purple Crayon.
For months I have been working on putting those two projects together. Technically it has been very complicated. I needed to completely rewrite large portions of Chalktalk, build my own virtual reality modeling software and renderer, and learn more about web sockets and other bits and pieces of internet plumbing than I ever thought I’d need to know.
And just yesterday, it finally all started working together. I am able to draw things in Chalktalk and then see them come magically to life floating in the air before me. Other people can also share the experience with me.
The basic components are now all there, for example, to allow somebody to give a Chalktalk lecture in virtual reality for a group of people. Each person can each walk around the 3D animated chalkboard and look at it from his or her own perspective.
There are still a few wrinkles to work out, but things are now in the home stretch. And it feels great!