Nick Bostrom’s book Superintelligence might just be the most debated technology book of the year. Since its release, big names in tech and science, including Stephen Hawking and Elon Musk, have warned of the dangers of artificial intelligence. We don’t know exactly when AI will rival human intelligence, many experts believe there is a good chance it will happen during the 21st century.
He suggests that when AI reaches a human level of intelligence, it may very rapidly move past humans as it takes over its own development. The concept has long been discussed and is often described as an “intelligence explosion”, a term coined by computer scientist IJ Good fifty years ago. Good described the process like this:
“Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man, however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an ‘intelligence explosion,’ and the intelligence of man would be left far behind.”
Bostrom says that once this happens, if we aren’t prepared, superintelligent AI might wipe us out as it acts to achieve its goals. He draws the analogy to humans redeveloping various ecosystems and, in the process, causing animal extinctions.
“If we think about what we are doing to various animal species, it’s not so much that we hate them,” Bostrom told IEEE Spectrum. “For the most part, it’s just that we have other uses for their habitats, and they get wiped out as a side effect.”