Audio virtual reality

Let’s say you and I are having a conversation and we want to include our friend, who happens to be somewhere else.

Let’s also say that you and I are both wearing earphones (so we have an audio input leading into each of our two ears). This allows each of us to have a binaural input capability. In layman’s terms, this means that a properly constructed sound could appear to us as coming from any particular location around us — in front or back of us, above, below, left, right, or any angle in between.

With this binaural capability (and the right computer software to back it up), the apparatus we each wear could analyze and then re-synthesize a very high quality representation of the voice of the person we want to include — which will seem to come from some exact location in the room.

Nothing that I’m saying is beyond today’s technology — it could all be done with commodity equipment. But I suspect that our culture’s single-minded focus on the visual has distracted us from all of the cool things we could be doing with audio.

In particular, I’d love to hear what it would be like to have a third “virtually present” person in a conversation, accurately represented in pure spatial audio form. Perhaps, without the distraction of imperfectly formed video or computer graphics, the person would appear, on a psychological level, to be fully present in the room.

Or maybe not. In any case, it’s certainly something worth finding out.

4 Responses to “Audio virtual reality”

  1. Dagmar says:

    I had the fun experience of the dummy head technology once and it is distracting, because you just turn your head and want to see where the noise comes from.
    So I would turn my head and talk to the air…
    I believe it would be a good experience, when you talk to someone you know in “real” life so your imagination could put a “full Ken” for example in the room, but for a person you don’t know it might be different.
    Just think of someone you can’t really judge or you are afraid for, and you can’t see the eyes of this person while he is talking. But in the end that gives you at least the opportunity to imagine this person smiling at you or as a small little something you don’t need to be afraid for. ;-)

  2. Mari says:

    Back in early 90s I sat in Tokyo univ. acoustics lab where they simulated a “hair cut” for me. You could hear the scissors snapping your hair from one ear all around to the other. It was very cool :) They were working on creating phone booth inside a disco to cancel out the noise, where your head is likely to be steady. Don’t know what happened to those projects…

  3. AJ says:

    Hi Ken,

    Fascinating idea, and you thought of it several years before it was obvious that virtual reality is indeed coming soon to the masses. My team is developing stock and custom binaural audio for use in virtual reality. We expect the world of digital video conferencing is going to be heavily impacted by the exact techniques you described in this post. Great insight!

Leave a Reply