DeepMind has introduced a new form of memory neural network called a ‘differentiable neural computer’, and shown that it can learn to use memory to answer questions about complex, structured data, including artificially generated stories, family trees, and even a map of the London Underground.
Google DeepMind, the artificial intelligence acquisition by the search engine giant that recently shocked the world by beating the best human player at the ancient game Go, and improving text-to-speech dramatically, has made another breakthrough in the development of AI. The company founded by Demis Hassabis, has now created an advanced machine learning system that combines a neural network with conventional computer memory.
The researchers at DeepMind, including Hassabis, call their creation a “differentiable neural computer”, or DNC. The system can learn from examples like neural networks, but they can also store complex data like computers. The team demonstrated the system planning the best route between distant stations on the London Underground or working out relationships between relatives on family trees.
Before the creation of the DNC, neural networks have only been able to access the data contained within their own network. DeepMind says their system provides neural networks with access to previously incompatible external data, such as text encoded in conventional digital form.
As the diagram above shows, the neural network controller receives external inputs and, based on these, interacts with the memory using read and write operations known as ‘heads’ in the DNC. To help the controller navigate the memory, DNC stores ‘temporal links’ to keep track of the order things were written in, and records the current ‘usage’ level of each memory location.
In the publication, the researchers showed that a DNC can learn on its own to write down a description of an arbitrary graph and answer questions about it. When they described the stations and lines of the London Underground, they asked the DNC to answer questions like, “Starting at Bond street, and taking the Central line in a direction one stop, the Circle line in a direction for four stops, and the Jubilee line in a direction for two stops, at what stop do you wind up?” Or, the DNC could plan routes given questions like “How do you get from Moorgate to Piccadilly Circus?”
In the future the DNC more useful in the real world than existing AI systems, it will need to be expanded to access far larger memories.
"We hope that DNCs provide both a new tool for computer science and a new metaphor for cognitive science and neuroscience: here is a learning machine that, without prior programming, can organise information into connected facts and use those facts to solve problems," write the researchers.