Skynect

The purchase of Skype by MIcrosoft has got me thinking about the possibilities of putting Skype together with Kinect (now that they are owned by the same corporate entity).

Yes, I realize these two things seem very different — one is a kind of internet phone chat and the other is a fancy game controller. But to me they seem to converge in a very interesting way, and I suspect the folks at Microsoft have already thought about this.

In the late 1990s the NSF funded a project called “The National Tele-Immersion Initiative”. Led by such top minds as Jaron Lanier and Henry Fuchs, the goal was to enable people at remote locations to perceive each other as though they were in the same room. Back then this was a very difficult proposition indeed. Computers were slower, digital cameras had poor resolution, and just about every other piece of the supporting infrastructure from projector to network was in a state that today we would call primitive.

Yet they succeeded in creating a working demo, as you can see from this image of the prototype in action:



At the root of that success was a combination of digital video chat and 3D tracking of head position. Once you can digitally capture and transmit a 3D representation a person’s head and face in real-time, then you can remotely compute what that head and face would look like from different points of view.

This means that each remote observer is free to move their own head (and hence vary their point of view), while your system reconstructs what each person will look like to the other. Note how different this is from video chat, in which you can see the other person from only a single point of view (wherever their video camera happens to be).

What the NTI accomplished in 1999 is exactly what you can get by combining Skype with Kinect. The two technologies together, when properly combined, can give you a seamless 3D representation of a person who is far away, as though that person is right there in front of you.

What this means is that you can probably expect video chat in the next few years that will seem far more real and immersive, with a much greater sense that the person you are speaking with is present.

Of course none of this should be surprising. After all, Jaron Lanier was a partner architect for Microsoft’s Kinect. 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *