In a discussion today about the future of virtual and augmented reality experiences, Alan Kay told me that what the medium really needs is the equivalent of classical music. It took me a few minutes more of conversion to work through the thought and understand the full dimensions of what he was suggesting.
Basically, he was positing that things will really get interesting when the audience of works created in such media are literate, in the way that audiences for classical music tend to be literate. It is not sufficient that audiences just think “this is cool”. They need to understand the language of what is going on well enough to appreciate why something is working.
Of course this sort of expectation of audience literacy is not limited to classical music. It is found in other genres, including various types of jazz, theater, poetry and computer games.
I was struck by how similar Alan’s observation was to something Marvin Minsky told me in 2003. When I raised the subject of the potential benefits of everybody learning to program — and computer languages that might make such a project easier — he said: “Computer programming doesn’t need a shared grammar. It needs a shared literature.”
I think the existence of classical augmented reality experiences depends on a stable and well-established notation for recording them. Text and musical notation work great for plays, operas, symphonies, etc. Dance has been a bit more challenging.
How would you record an augmented reality experience for performance a century or two from now?
That’s a very good question!
Maybe you will be one of the people who helps us start to work out the answer. 🙂
Re Minsky’s observation: it might be fun to attempt to compile a list of links to beautiful open source code.
I think it’d go more the route of film and fine arts. There’s no notations to reproduce the works, the works stand for themselves.
Before that, don’t we need the counterpart of Eisenstein to start identifying and naming the primitives/grammars/etc? Classical music took a few hundred years to turn into classical music…
I think this is an incorrect analogy. In VR we have invented the motion picture camera and the projector but we have not invented movies yet. There was a language of cinema that had to be created – the idea that the camera could be a full participant in the dialog on screen, the concept of a cut in time and space, the close up and even the music score were invented to create a sense of engagement and emotional involvement within the observer. Before this (like now) we had technical tour de force with the camera as a purely objective observer relaying the information to the recipient with minimal context and no emotion. The language of film does not require a literate user to understand it. This is the wonder and perhaps the failure of this medium – it is hard to create but easy to consume. VR has the distinction of immediately providing an extraordinarily immersive experience – which is often hard in film. What it lacks is a story.