Today Patrick Hebron gave a wonderful demonstration at NYU of some work he is doing that ties together object recognition and word recognition. He is using a very large database of 3D shapes of ordinary objects — everything from chairs to computers to airplanes to toilet bowls — and tagging each shape with sets of words.
One of the things he can do with his system is use words to explore the shapes associated with those words. He can even to use language to create novel in-between shapes. For example, I learned from his demo that if you make a shape part way between a chair and a toilet bowl, you get something that looks very much like a fancy modernist designer chair. Although I’m not sure exactly what that means.
Patrick then pointed out that his general approach could be generalized to multiple languages. A shape described in one language, perhaps English, could be associated with the same shape described in another language, perhaps Arabic.
At that point one person in the room said that this could be really interesting to the NSA. Another person said out that it could be really interesting to Noam Chomsky.
“How nice,” I said, “that the NSA and Noam Chomsky can have common interests.”