The legend of King Midas is, like many fairly tales or myths, a warning. As told by Aristotle, Midas was gifted the ability to turn everything he touched into gold as a consequence of his “vain prayers.” He quickly realized his mistake when attempting to eat and drink, and — in some versions of the tale — died of starvation. The message is clear: be careful what you wish for.
It is of utmost importance that designers of Artificial Intelligence heed this warning. To show why, let us consider the example of an AI agent in the realm of chess. The designers of the agent define its objective as winning the match. In this case, what is to stop the agent from blackmailing its opponent, bribing the audience to make noise during its opponent’s turn, or even causing physical harm to the opponent? These are all logical actions if the goal is simply to win, so it is clear AI designers must be very careful what they wish for.
This problem, known as the King Midas or value alignment problem in the sub-field of AI ethics, is explored deeply in Stanley Kubrick’s science fiction film, 2001: A Space Odyssey. In the film, five scientists are bound for Jupiter on the spacecraft Discovery One, whose operations are mostly in the hands of a HAL 9000 computer known as “HAL.” Supposedly, no HAL 9000 computer has ever made a mistake, so it would seem that the scientists are in good hands. HAL realizes, however, that these scientists are prone to human error where HAL itself is not. This being the case, HAL concludes that the success of the mission is higher without the humans and starts to kill them off, displaying its talent for manipulation, subtlety, and a developed understanding of human emotion in the meantime. HAL is eventually thwarted by Dr. David Bowman who disconnects HAL’s circuits as HAL pleads for him to stop, expressing its fear. “My mind is going. I can feel it. I can feel it. I’m afraid.”
Perhaps this situation would have been avoided if the “Three Laws of Robotics,” proposed in the 1942 short story “Runaround” by scientist and author Isaac Asimov, had been followed. Foreshadowing the imminent dangers of advanced AI, Asimov conceived the following rules:
- A robot may not injure a human being or, through inaction, allow a human being to come to harm.
- A robot must obey orders given to it by human beings, except where such orders would conflict with the First Law.
- A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.
One can see that if these rules were built into HAL, disaster may have been avoided. Matters are only complicated further, however, by the problem of consciousness. Was HAL conscious? If, one day, we are able to construct conscious beings, would we not then be responsible for ensuring their rights in society? And if so, are not Asimov’s proposed rules akin to slavery?
Questions like these are complex but demand answers as AI technology continues to advance and promise a future of AI agents in all aspects of society.