Along with AI’s remarkable achievements and continuing rapid expansion into new domains comes greater concern over the ethical problems surrounding advanced AI systems. Incidents like last year’s shutdown of AI-powered Genderify and Yann LeCun’s “exit” from Twitter after heated discussions regarding Duke University’s PULSE AI photo recreation model underscore the ongoing controversies regarding biases and errors embedded in the design and deployment of AI systems. Google AI’s recent dismissal of its Ethical AI team co-lead Timnit Gebru poses a more serious question: do tech giants and large research institutions even want to find solutions?
A paper in the Journal of Artificial Intelligence Research goes a step further, warning that it could become fundamentally impossible to control a superintelligent AI (a computer program or a programmed robot that is much more intelligent than humans in almost any field). A preprint of the paper was uploaded to arXiv in 2016, and now, with the topic becoming more urgent than ever, an expanded version has been published with additional presentation details, references, and comments about alternative scenarios.
Suppose an AI system with superhuman intelligence could learn independently and, through the Internet, had access to all of humanity’s data. The question considered in the paper Superintelligence Cannot be Contained: Lessons from Computability Theory is whether such a master AI system, controlling all online machines worldwide, would work to cure cancer, establish world peace and prevent a climate disaster — or would it instead simply scrap humanity and take over the Earth?
Universidad Autonoma de Madrid Professor and the paper’s first author Manuel Alfonseca told Synced via email, “Our conclusion is that a superintelligence cannot be controlled in all circumstances. It can be controlled in some cases, but not all.” Alfonseca worked on the study with researchers from the Max-Planck Institute for Human Development in Germany, University of California San Diego, IMDEA Networks Institute and the University of Chile.
The team posits that superintelligent AI poses a fundamentally different problem than systems typically studied under the umbrella of “robot ethics” because such a superintelligence is multi-faceted and therefore potentially capable of mobilizing a diversity of resources to achieve objectives that are potentially incomprehensible to humans, let alone controllable.
The researchers looked at two different mechanisms proposed by Oxford philosopher Nick Bostrom in considering how a superintelligent AI might be controlled. One option is to limit the superintelligence’s abilities in order to prevent it from doing harm to humans, for example by walling it off from the Internet and all other technical devices so it has no contact with the outside world. This approach however would render the superintelligent AI significantly less powerful.
The other option explored is motivating a superintelligent AI from the outset only to pursue goals that are in the best interests of humanity, for example by programming ethical principles into it. The researchers however demonstrate that these and other contemporary and historical ideas for controlling superintelligent AI have their limits.
The team conceived a theoretical containment algorithm that might ensure a superintelligent AI could not harm people under any circumstances by simulating AI behaviours and halting those considered harmful. Their analysis however concluded that under the current computing paradigm such an algorithm cannot be built.
“If you break the problem down to basic rules from theoretical computer science, it turns out that an algorithm that would command an AI not to destroy the world could inadvertently halt its own operations. If this happened, you would not know whether the containment algorithm is still analyzing the threat, or whether it has stopped to contain the harmful AI. In effect, this makes the containment algorithm unusable,” says Iyad Rahwan, another author on the paper and the director of the Max-Planck Institute for Human Development in Berlin.
Based on the researchers’ calculations, the containment problem is incomputable, and no single algorithm will be able to find a solution for determining whether an AI would produce harm to the world.
The study focuses on superintelligent AI, a goal that remains a long way off. Alfonseca however believes it will also be impossible to fully control artificial general intelligence (AGI) agents (computer programs or programmed robots that are as intelligent as people in a wide variety of fields). As such systems would be equivalent to human beings, he says, we can ask the question: Is it possible to control a human being in all circumstances?
Experts estimate we may need another century or more to reach AGI, and nobody can make any provision about superintelligence, Alfonseca says. “It’s even possible that we won’t even reach AGI ever. In that case, our theorem will become just a theoretical exercise.”
The paper Superintelligence Cannot be Contained: Lessons from Computability Theory is available on the Journal of Artificial Intelligence Research website.