An artificial intelligence system at MIT has taken an IQ test designed for a young child. The results, written up by MIT Review, revealed that its IQ was equivalent to that of a four-year-old. Increasingly machines are getting better at specific tasks such as playing chess, recognising pictures and making complex commutations.
But general intelligence is still proving elusive for most of them. The MIT machine – dubbed ConceptNet4 – was put through its paces by researchers from the University of Illinois, led by Stellan Ohlsson. The IQ test they chose, known as the Wechsler Preschool and Primary Scale of Intelligence, is commonly used in US schools.
It is designed for young children and measures intelligence in five categories:
information – questions such as "Where can you find a penguin?"
vocabulary – questions such as "What is a house?"
word reasoning – children are given three clues such as "You can see through it", "It is square", "You can open it".
similarities – questions such as "Pen and pencil are both….?"
comprehension – questions such as "Why do people shake hands?"
The questions had to be modified for the computer and some programming was necessary to allow the machine to understand what it was being asked.
It did well on vocabulary and similarities, had an average score on information and scored poorly in the word reasoning and comprehension categories.
"The ConceptNet scored a [mark] that is average for a four-year-old child, but below average for five to seven-year-olds," the researchers concluded.
A lot depended on how the machine interpreted the questions. For instance, in answer to the question ‘Why do we shake hands?", the machine produced the result "epileptic fit".
But when the team reduced the question to just "shake hand", the computer came back with more relevant answers such as "flirt", "thanks" and "meet friend".
Sometimes its answers appeared completely illogical – in response to the question "Where can you find a teacher?", it came back with "piano" and "band". The researchers could not explain such anomalies but did suggest ways that the test could be improved – for instance, inputting questions using natural language via a virtual assistant such as Siri or Cortana.
The history of AI research stretches back to the 1950s. In the early days, computers were given knowledge bases and worked things out using logic and reasoning. That method of teaching computers has been surpassed in recent decades by a new era of machine learning, where AIs are given large quantities of data from which they can learn. The researchers concluded that both approaches might be useful in future.
"In general, recent successes in AI have been mostly learning-driven and the era of knowledge-driven AI appears to have passed. Perhaps knowledge bases that are a hybrid of the two paradigms will play a role in the next round of AI progress."