Who is responsible?

The idea

The people and organisations that build and use AI are responsible for what it does, because a machine cannot be held responsible and AI should serve human interests.

The timetable mix-up

A school's AI timetable tool puts two exams in the same hour for Class 8. Who is responsible for putting this right?

• The AI tool, because it made the mistake
• The people who built it and the school that chose to use it
• Nobody, because it was just an accident

Who should act?

In each card, an AI system causes a problem. Tap the people who should put it right.

There is no group for the machine. Think about why. Tap a group, read the reason, then press Next card.

A machine cannot answer for itself

Every problem had a person or an organisation behind it: someone who built the tool, chose it, or used it. The machine only followed its data and its rules.

So we ask: Who made it? Who checks it? Who fixes mistakes? Who answers if harm happens? AI is meant to serve people, so people must stay in charge of important decisions.

People and organisations that build and use AI are accountable for it, because a machine cannot be held responsible.

Notes

People and organisations that build and use AI are accountable for it, because a machine cannot be held responsible.

This lesson is about who should act and check. Who is blamed or punished is for the law and its rules to decide.

Check yourself

A delivery app's route tool keeps sending riders down a closed road. Riders complain. Who should look into it and fix the fault?

A clinic's AI suggests which patient needs urgent care first. How should the doctors use it?

A tool says Joseph's essay was copied. His teacher is about to give him zero. What should happen?

• The route tool, which will fix itself over time. A tool does not read complaints or decide to improve. People must read them and make the changes.
• The riders, who should know better by now. Riders trust the app for directions. Reporting the fault helps, but fixing it belongs to the people who build and run the app.
• The company that runs it, with the builders — correct. Yes. They can read the complaints, find the cause and change the tool. Those who build and run a system answer for it.
• As help, with the doctors making the final decision — correct. Yes. The AI can spot patterns quickly, but the doctors know their patients and stay responsible for each decision.
• Follow it every time, because it is faster. Fast is not the same as right. A mistake could harm a patient, and the AI cannot answer for it.
• Ignore it, because machines are no use in a hospital. AI can help doctors by spotting patterns in reports quickly. The mistake is trusting it blindly, not using it.
• Give the zero, since the tool was trained on many essays. A large training set does not make a tool right every time. Such tools can be wrong. A zero changes a student's marks, so a person must check the evidence.
• The teacher reads the essay, talks with Joseph, then decides — correct. Yes. The tool can raise a question, but a person who can listen and weigh the evidence must decide.
• Run the tool again until it gives the answer she expects. That is not checking. It only searches for the answer she wants. A real review looks at the evidence.