What does it mean to have human control over intelligent machines?

We seek to find out when and how human beings should take control over intelligent machines based on artificial intelligence and advanced robotics. To this end, we are conducting experiments in a virtual, game-like environment, which was originally designed by the Pentagon for the "Darpa Subterranean Challenge". This is a competition in which disaster relief in tunnels and caves is provided by robots. The Pentagon has granted us access to this virtual environment for our experiments. We are also cooperating with the Swiss army so as to benefit from their experience.

In recent years, progress in the areas of artificial intelligence and robotics has been enormous. Computer programs and algorithms are now capable of learning, and robots can perform tasks independently. Less and less human involvement is required for machines to function properly, and humans are often only active in the background. As a result, machines can drive, fly, or even kill completely independently. In light of this development, our aim is to define effective human control over machines.

Our project aims to sharpen the concept of control over intelligent machines, so as to define when and how human control is required. Accordingly, we aspire to provide legislators with concrete ideas for laws and norms regarding the control over intelligent machines, as is the case with the legal and technical norms currently applied to conventional cars.

We intend to approach manufacturers of modern machines, e.g. robots, that function on the basis of artificial intelligence. Our findings should show them how to design their products in terms of control, in order to guarantee their ethical and legal operation.

Meaningful Human Control of Security Systems – Aligning International Humanitarian Law with Human Psychology