A colleague told me today that he relies on Amazon’s Mechanical Turk for his academic user interface research.
For those of you who don’t know, this is a facility that allows people to sign up anonymously to take simple tests on-line. They get paid small amounts for taking these tests. In the words of Wikipedia, it is:
“…a crowdsourcing Internet marketplace that enables individuals or businesses (known as Requesters) to co-ordinate the use of human intelligence to perform tasks that computers are currently unable to do.”
The term “Mechanical Turk” was inspired by Wolfgang von Kempelen’s famous 18th century chess playing robot, known as “The Turk”, which turned out to be a hoax — there was actually a chess-playing human cleverly hidden inside the robot’s housing.
If you’re doing market research, or trying to understand whether a new kind of software interface is better than the old one, Mechanical Turk can be an effective way to get lots of people to try things out for you, at modest cost.
During my colleague’s description of the process, he mentioned that the great majority of the people who answer these questions are located in India.
Which is when the following thought occurred to me: As computer interface design studies come to rely ever more on Mechanical Turk, perhaps the interfaces we use will become increasingly optimized for use by people in India.
I’m not entirely sure whether this is a good thing or a bad thing.
Quite true. Luckily, Amazon enables filtering by region. Theoretically, one could test user interfaces designs among different cultures. It would be interesting if regional differences changed performance in simple tests.