Fun Friday: Google working on a kill switch for rogue AI

by L&D09 Jun 2016

We’ve all seen the harm which robots can get up to in movies such as The Terminator.

Now, scientists are working on a way to stop rogue robots causing mayhem in the office, the factory, the home or wherever they are put to use.

Their solution?

A “big red button”, otherwise known as a kill switch.

The research was conducted by Laurent Orseau, from Google DeepMind, and Stuart Armstrong, from the Future of Humanity Institute at the University of Oxford.

“If an agent is operating in real-time under human supervision, now and then it may be necessary for a human operator to press the big red button to prevent the agent from continuing a harmful sequence of actions—harmful either for the agent or for the environment—and lead the agent into a safer situation,” said the researchers in a paper.

“However, if the learning agent expects to receive rewards from this sequence, it may learn in the long run to avoid such interruptions, for example by disabling the red button— which is an undesirable outcome.”

Indeed, the list of robots behaving badly includes one that learned how to pause a game of Tetris to avoid losing.

Moreover, earlier this year, Microsoft’s AI chatbox Tay went rogue on Twitter by swearing and making racist remarks.

"The AI chatbot Tay is a machine learning project, designed for human engagement. As it learns, some of its responses are inappropriate and indicative of the types of interactions some people are having with it. We're making some adjustments to Tay," said Microsoft at the time.

The paper concluded by proposing a framework to allow a human operator to repeatedly and safely interrupt a reinforcement learning agent “while making sure the agent will not learn to prevent or induce these interruptions”.

Orseau said it is sane to be concerned - but, currently, the state of our knowledge doesn't require us to be worried.

"It is important to start working on AI safety before any problem arises,” he added.

"AI safety is about making sure learning algorithms work the way we want them to work."

Related stories:

Lawyers trained to "polish up their human skills" 

Technology giant is training a ‘bonkbuster bot’ to write adult fiction