Stretch tasks Bias, error and responsibility

Round 1: Quiz Blitz

1. It copies the patterns in past decisions, including unfair ones Trained on twenty years of decisions, it learns everything in them, including any discrimination.
2. Its training data was mostly lighter-skinned faces A system is only as good as the variety in its training data.
3. False The maths faithfully reproduces whatever is in the data. Looking neutral is exactly what makes AI bias dangerous.
4. Correlation is not understanding Doctors photographed worrying moles beside a ruler, so rulers correlated with cancer. The AI had no idea why.
5. Training on data that fairly represents everyone; Testing its accuracy separately for different groups Representative data and checking results group by group are the two main defences.
6. It produces likely-sounding text, not checked facts Generated text is statistically likely, not necessarily true.
7. True AI outputs are confident, not guaranteed correct.
8. Past decisions in the world contain unfairness → Those decisions become the training data → The system learns the patterns in the data → Its outputs repeat the bias while looking neutral Nobody has to intend it: unfairness flows from the world, through the data, into the system.
9. It is unclear who is responsible: the bank, the programmers or the data supplier Responsibility is spread across many people, and the law is still catching up.
10. Patterns in the past hiring decisions it was trained on If past hires were mostly men, the system learned to prefer CVs that looked like theirs.
11. bias Bias usually comes from training data that does not represent everyone fairly.
12. Test how accurate it is for different groups of people Overall accuracy can hide a system that works well for most people and badly for some.

Round 2: Quick Fire

1. • It learns the patterns in its training data
   • The data reflects past unfairness, or does not represent everyone
   Model answer

   An AI system learns the patterns in its training data. If that data reflects past unfairness, or mostly contains one group of people, the system reproduces that bias, even though nobody intended it.

2. • It produces statistically likely text, not checked facts
   • So it can be confidently wrong, or make things up
   Model answer

   A chatbot produces text that is statistically likely, not facts it has checked. It can state false things with complete confidence, or make them up, so anything important must be checked against a real source.

3. • The AI linked rulers with cancer, because doctors put rulers in photos of worrying moles
   • It did not understand skin cancer, so it would fail on photos taken a different way
   Model answer

   The AI noticed that photos with a ruler were more likely to show cancer, because doctors put a ruler in the picture when they were worried. It had no understanding of skin cancer itself, so it would fail completely on photos taken a different way.

4. • The owner
   • The manufacturer
   • The programmers
   • The company that supplied the training data
   • The regulator that approved it
   Model answer

   The manufacturer of the car, and the programmers who wrote its software.

Round 3: Big Think

1. • It will be less accurate for people with darker skin
   • Innocent people could be wrongly identified, stopped or arrested
   • Its output looks neutral, so officers may trust it too much
   • It would treat some groups unfairly
   • It is unclear who is accountable for mistakes
   Model answer

   Because the training data was mostly light-skinned faces, the system will be less accurate for people with darker skin and make more mistakes for them. Innocent people could be wrongly matched to suspects, then stopped or even arrested. The output looks neutral and mathematical, so officers may trust it more than they should, and the result is that some groups are treated unfairly. When it goes wrong, it is also unclear who is accountable.

2. • An AI is consistent: it is not affected by mood or tiredness
   • Its decisions can be audited and measured
   • It learns bias from the data it is trained on
   • Its output looks neutral, so bias is hidden and harder to challenge
   • It finds correlations without understanding the context
   • It is unclear who is accountable for a wrong decision
   • People still choose the data and the goals
   • A reasoned conclusion
   Model answer

   It is true that an AI is consistent: it is not affected by mood or tiredness, and its decisions can be audited and measured in a way a human's feelings cannot. However, having no feelings does not make it fair. It learns from training data, which reflects past decisions, so it can learn bias from them. Its output looks neutral and mathematical, so the bias is hidden and harder to challenge. It finds correlations without understanding the context, and when it gets something wrong it is unclear who is accountable. People also still choose the data and the goals it is given. Overall, the claim is wrong: an AI can be more consistent than a person, but it is only as fair as the data and choices behind it.

This or That

1. Its training data The system learns patterns from data, and data comes from a world that contains bias.
2. Its output looks neutral and mathematical People trust numbers, so unfair output that looks mathematical is less likely to be questioned.
3. No: it copies the patterns in its data The system is doing exactly what it was built to do: reproduce the patterns in its training data.
4. There is no settled answer yet The owner, the maker, the programmer, the data supplier or the regulator: the law has not caught up.
5. Faces of every skin tone, in large numbers When some groups are missing from the data, the system works less accurately for them.
6. Spotting a link without knowing why The skin cancer system linked rulers in photos to cancer without understanding cancer at all.
7. A wrong decision can harm a real person When AI decisions affect people's lives, someone must be answerable when they go wrong.
8. Statistically likely A generated answer is likely rather than verified, so it can be confidently wrong.

Memory Match

• A 2 Bias: Systematic unfairness in a system's output
• B 5 Correlation: Two things that tend to happen together
• C 6 Accountability: Someone being answerable for a decision
• D 3 Self-driving car: Raises the question of who is at fault in a crash
• E 1 Representative data: Data that includes every group fairly
• F 4 Confident output: Sounds certain, but may still be false