Many people I know — myself included — first got excited about binaural recording when we heard the Binaural Haircut recording. The effect is truly remarkable.
The more you learn about binaural recording, the more you appreciate the difference between recording a binaural experience and synthesizing one. In the first case, the scene cannot change when you move your head. In the second case it can — and that turns out to make a big difference..
You can create a great binaural recording without any computation. All you need is a good quality dummy head and high quality audio recording equipment.
But if you want the sound to change in response to a user moving their head, then dummy head recording won’t suffice. You’ll need to go the much more difficult route of building a computational model of the soundscape, and then generating or otherwise modifying all of the sounds dynamically on a computer.
It find it fascinating how analogous this is to the equivalent visual situation: If you want to create a realistic stereoscopic image, you just need to take two side by side photographs. But if you want the scene to change in response to a person’s movements, you need to create and manipulate some sort of computer model of the scene.
In a sense, the ability to record 3D reality is a kind of teaser. It gives us a taste of what virtual reality could be like, but only a taste.
In order to interact with that virtual reality, to truly enter another world on a sensory level, we need to do a lot more work. The good news is that when you put in that work, you learn that it was well worth the effort.