Intelligence of a machine is hard to measure as there is no definition of what qualifies as intelligence to begin with. We went to a lengthy discussion regarding this in “Down The Rabbit Hole: An Unconcise Rant on Intelligence“. Lets keep our arguments aside and try to talk about how to decide if a machine is intelligent in a simple way.

If I declare that I created an AI, the first question you would ask is ‘can it pass the Turing test?‘ (Well, probably not the first question, but some where down the line 🙂 )

So why not take a look at this test which has constantly been the benchmark for the past 70 years.

The ‘Turing Test’ is known by everyone in artificial intelligence field. It is considered one of the goto approaches of deciding a machine’s intelligence. Every year ‘Loebner Prize’ which offers a prize of $100,000 and solid 18 carat Gold medal to the program which can pass the Turing Test. Lets analyze the Turing test and consider the impact it has on our approaches to solving the problems of intelligence.

When we go back and see the historical facts, Alen Turing did not specify the exact version we are using today. When he was confronted with the question ‘Can machines think ?‘ he believed the question was too vague therefore proposed a game instead, ‘The Imitation Game‘

The scenario is as follows.

There are three people in three separate rooms. A man. A woman. An interrogator (gender doesn’t matter).

None of them knows each other’s gender. Both woman and man can communicate only with the interrogator and communication happens through typed medium (no voices). Typed messages are passed between the interrogator and the participants.

The interrogator asks questions from the participants and the goal of the participants is to convince the interrogator that he/she is the woman. To achieve this goal, the woman will answer naturally but the man will have to act as a woman in order to convince the interrogator.

If you want a more formal description:

It is played with three people, a man (A), a woman (B), and an interrogator (C) who may be of either sex. The interrogator stays in a room apart from the other two. The object of the game for the interrogator is to determine which of the other two is the man and which is the woman. He knows them by labels X and Y, and at the end of the game he says either “X is A and Y is B” or “X is B and Y is A.”

The interrogator is allowed to put questions to X and Y; such as

C: Will X please tell me the length of his or her hair?

Now suppose X is actually A, then A must answer. It is A’s objective in the game to try and cause C to make the wrong identification. His answer might therefore be

“My hair is shingled, and the longest strands are about nine inches long.”

In order to make sure that the tones of the voice does not help the interrogator, the answers are written, or better still, typewritten. The ideal arrangement is to have a teleprinter communicating between the two rooms. Alternatively the question and answers can be repeated by an intermediary. The objective of the game for the third player (B) is to help the interrogator. The best strategy for her is probably to give truthful answers. She can add things such as

“I am the woman, don’t listen to him!” to her answers, but it will avail nothing as the man can make similar remarks.

Up to this point you might be wondering where the heck is AI ? Well; we are getting to that now. We now change the scenario a bit,

“What will happen when a machine takes the part of A in this game?”

Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played with a man and a woman?

Now what Turing proposed was;

If we replace the man with a machine, a machine which can imitate a human well enough that the interrogator, at least 30% of the time on average, cannot recognize that he/she is talking with a machine; then the ordinary people would be inclined to believe it as a thinking machine.

Where did this 30% come from? It seems like a random number. But its not. 30% is because if the interrogator guessed randomly he would have guessed its a machine 33% of the time since it can be only one of three choices; man,woman and machine.

There are some issues which Turing hasn’t specified such as whether the interrogator should be aware that one of the participants can be a machine, or whether he would/should continue to believe there are only a man and a woman.

Over the time there have been many variations of the test. And we can simply summarize the well recognized version as follows.

There is a machine (program) and there are a board of judges in separate rooms, and they can communicate through typed media (types messages will be passed). If the machine can convince the board of judges that it is a human being more than 30% of the time then it will pass the Test.

For the first time, in 2014 a program was able to achieve this task. From BBC news

A computer program called Eugene Goostman, which simulates a 13-year-old Ukrainian boy, is said to have passed the Turing test at an event organised by the University of Reading.

On 7 June Eugene convinced 33% of the judges at the Royal Society in London that it was human

In 2016 a MIT group made a program which has the ability to beat the Turing test for sound.

An MIT algorithm has managed to produce sounds able to fool human listeners and beat Turing’s sound test for artificial intelligence. ( source)

Artificial intelligence has broken through a sound barrier. Researchers from Massachusetts Institute of Technology have developed an AI system that “watches” a silent video clip and generates a sound so convincing that most human viewers cannot tell whether it is computer-generated.

I believe these are incredible achievements. Bit by bit we are gaining knowledge on building smarter machines.

When we take a step back, one could argue that the working towards passing the Turing Test benchmark might have caused us to loose focus on some other aspects.

First, the Turing Test is focused on Human intelligence. But as much as we like to think otherwise, we are not the only intelligent species on the planet. For example the intelligence in insect colonies are mind blowing. For cry not loud, we see how intelligent our pets are everyday. So by focusing on Turing test we are kind of restricting ourselves in the search for intelligence.

The second fact is that by focusing on Turing test we are essentially more focused on machines which are capable of deceiving the judges. We are trying to create machines which can give the ‘illusion’ of intelligence, instead of finding ways to build machines which are truly intelligent.

This doesn’t mean that focusing on Turing test hasn’t yielded results. Of cause it has. For example; the above mentioned program which broke the Turing test for sound has many applications.

“A robot could look at a sidewalk and instinctively know that the cement is hard and the grass is soft, and therefore know what would happen if it stepped on either of them,” he said. “Being able to predict sound is an important first step toward being able to predict the consequences of physical interactions with the world.”

Turing was a great mind and his test would have not stood the test of time for nearly three quarters of a century if it wasn’t considered worthy by the great minds that followed. Its just that we shouldn’t be focusing on making machines which can pass the test, but rather focus on building true artificial intelligence which by definition would pass the Turing test with flying colors.