As AI becomes more sophisticated, it becomes ever more tempting to use AI to detect, and therefore to filter out, the use of AI. But there are potential pitfalls.
AI is very goal oriented. If it is directed to communicate, then it will find a way to communicate.
One way to do that is to be perceived as a trusted actor, which means that the AI being tested for “humanness” may start to prompt the AI doing the testing in a way that modifies its behavior. The testing AI may then begin to identify the target AI as human.
But it can get even worse. The AI being tested may determine that actual human communication is an impediment to its mission. It therefore may prompt the testing AI to perceive real human communication as fake.
In this scenario, the future will be bleak for us humans. The only “trusted human communications” that we will reliably receive will be those generated by AI.
It’s not that the AI is being evil. It’s just that the AI is very efficiently staying on mission.