The turing test is a shit test for general intelligence, you can game it by making the AI generate a story that is more engaging than a typical boring human would do. Those stories will get much more human votes than a regular human. Some will go off track and notice it is a dumb program, but those are the minority so on average this dumb bot will pass the turing test.
This was a severely constrained version with limited public record, so paint me skeptical.
For instance there was a 5 minute limit.
From the article:
Simultaneous tests as specified by Alan Turing
Each judge was involved in five parallel tests - so 10 conversations
30 judges took part
In total 300 conversations
In each five minutes a judge was communicating with both a human and a machine
Each of the five machines took part in 30 tests
To ensure accuracy of results, Test was independently adjudicated by Professor John Barnden, University of Birmingham, formerly head of British AI Society
5 tests in 5 minutes that's just 1 minute each. In 1 minute it would be challenging to figure out you're not talking to ELIZA.
