Responsible AI: Fairness, Safety, Privacy and the Law

Why responsibility is part of engineering

AI systems now help decide who gets a loan, a job interview, a medical test or a university place. When they go wrong, real people are harmed, often people with the least power to complain.

None of these systems was built to discriminate. Each learned from data that reflected an unequal world, or was judged by a measure that hid who it failed. Responsible AI is not a separate subject added at the end: it is part of doing the engineering properly.

Measuring fairness

"Fair" has several precise meanings, each comparing a different number between groups:

In the example in the figure, both groups have the same precision, 75%, so by one definition the model is fair. But group B's applicants who would repay are approved only 60% of the time, against 90% for group A.

Where bias comes from, and what to do

Bias can enter at every stage (Level 3): historical bias in the world the data records, representation bias when some groups are scarce in the data, measurement bias when a label is a poor stand-in (arrests are not the same as crimes), and deployment bias when a model is used on people unlike those it learned from.

Engineers have three places to intervene:

• Before training: collect better data, rebalance or reweight groups, remove proxies.
• During training: add a fairness constraint or penalty to the loss.
• After training: adjust decision thresholds per group, as the lab above lets you do.

Each has costs, often a little overall accuracy, and some, such as treating groups differently, are restricted by law in some settings. Just as important are the non-technical steps: involving the people affected, auditing results by group before and after release, and being willing to decide that a system should not be built at all.

Explaining decisions

People affected by a decision often have a right to an explanation, and engineers need explanations to find bugs.

• Interpretable models, such as small decision trees or linear models, can be read directly.
• Feature importance asks how much each input matters overall, for example by shuffling one feature and seeing how much worse the model gets.
• Local explanations such as LIME (2016) and SHAP (2017) estimate how much each feature pushed one particular decision up or down.
• Saliency maps highlight the pixels of an image that most affected a vision model's answer.
• Counterfactual explanations say what would have had to be different: "If your income had been £57,500 instead of £38,000, the loan would have been approved."

Explanations of complex models are approximations, and a convincing explanation can still be wrong. They help, but they do not replace testing.

Privacy

Models can leak the data they were trained on. In 2021 researchers extracted names, phone numbers and email addresses that GPT-2 had memorised word for word from its training text. And "anonymised" data is often not anonymous: in 2008, researchers re-identified people in a dataset of film ratings released by Netflix by matching it against public reviews.

Two techniques protect privacy by design:

• Differential privacy adds carefully measured random noise to results, so that no answer reveals much about any one person. A parameter ε (epsilon) sets the trade-off: smaller ε means more noise and more privacy. The 2020 US Census used it.

• Federated learning trains a shared model without collecting the raw data: each phone improves the model on its own data and sends back only the changes, which are averaged on the server. Phone keyboards learn to predict the next word this way.

The simplest rule of all is data minimisation: do not collect, keep or train on personal data you do not need.

Robustness and adversarial examples

In 2014 researchers showed that adding a carefully designed pattern of noise, too faint for a person to see, made an image classifier change its answer for a photo of a panda from "panda" to "gibbon", with 99% confidence. These adversarial examples work against almost every kind of neural network, and in 2018 a few stickers on a real stop sign were enough to make a vision system read it as a speed limit sign.

Defences include adversarial training (adding such examples to the training data), checking whether an input looks unlike the training data, and never relying on one model alone where safety matters. The deeper lesson is that a model can score brilliantly on a test set yet fail on inputs that would never fool a person.

Safety and alignment

Alignment is the problem of making AI systems pursue what their designers and users actually intend, not just what they were literally rewarded for (Level 13's reward hacking). As systems become more capable, the questions get more serious:

• Will a model tell people what they want to hear rather than what is true?
• Could it help someone cause serious harm, for example with cyber attacks or dangerous chemistry?
• Does it behave the same when it is not being tested?

Developers evaluate models for dangerous capabilities before release, pay red teams to attack them, train them to refuse harmful requests, and study their insides with interpretability research, looking for the circuits that implement behaviours. In November 2023, 28 countries and the EU signed the Bletchley Declaration on AI safety at Bletchley Park, and the UK set up a government AI Safety Institute, since renamed the AI Security Institute, to test advanced models.

AI and the law

Laws already apply to AI, and new ones are arriving.

• Data protection: the GDPR, which the UK keeps in its own version, gives people rights over their personal data, including rights around decisions made solely by automated means that have legal or similarly significant effects, and a right to meaningful information about how such decisions are made.
• Equality law forbids discrimination by protected characteristics, whether a person or an algorithm does it.
• The EU AI Act, in force since August 2024 with its rules applying in stages, sorts AI systems by risk:

Uses with unacceptable risk, such as social scoring by governments, have been banned since February 2025. High-risk uses, such as screening job applicants, credit scoring, marking exams and safety components of medical devices, must meet requirements for data quality, documentation, human oversight and accuracy. Limited-risk systems must be transparent: people must be told they are talking to a chatbot, and deepfakes must be labelled.

Copyright is still being argued in courts and parliaments: whether training on copyrighted books, art and music without permission is lawful, and who owns what an AI produces.

AI and society

• Work: AI automates some tasks and assists with others. Most jobs are bundles of tasks, so many will change more than disappear, but the benefits and costs will not fall evenly.
• Information: cheap fluent text, images and voices make misinformation and scams easier, and make provenance (Level 12) more important.
• Energy and resources: the International Energy Agency estimated that data centres used about 415 terawatt-hours of electricity in 2024, around 1.5% of the world's electricity, with AI a fast-growing share; cooling also uses water.
• Power: training frontier models costs so much that few organisations can do it, which concentrates influence.
• Benefits: AI already helps detect disease in scans, predict protein structures, translate between languages, make technology accessible to people with disabilities, and speed up science.

Engineers do not decide all of these questions alone, but they decide many of the details that settle them.

Hands-on: auditing a model

Fairness metrics for two groups

# Outcomes of a loan model for two groups of applicants.
# tp: would repay, approved; fn: would repay, refused; fp: approved but did not repay; tn: refused, did not repay
groups = {"group A": {"tp": 360, "fn": 40, "fp": 120, "tn": 480},
          "group B": {"tp": 120, "fn": 80, "fp": 40, "tn": 760}}

print(f"{'':8} {'approved':>9} {'TPR':>6} {'FPR':>6} {'precision':>10}")
for name, g in groups.items():
    n = sum(g.values())
    selection = (g["tp"] + g["fp"]) / n                 # demographic parity compares this
    tpr = g["tp"] / (g["tp"] + g["fn"])                 # equal opportunity compares this
    fpr = g["fp"] / (g["fp"] + g["tn"])                 # equalised odds compares TPR and this
    precision = g["tp"] / (g["tp"] + g["fp"])           # predictive parity compares this
    print(f"{name:8} {selection:9.0%} {tpr:6.0%} {fpr:6.0%} {precision:10.0%}")

Explaining one decision

weights = {"income (thousands)": 0.08, "years at address": 0.3, "missed payments": -1.2}
bias = -4.0

def score(person):
    return bias + sum(weights[k] * person[k] for k in weights)

applicant = {"income (thousands)": 38, "years at address": 2, "missed payments": 1}
s = score(applicant)
print(f"score {s:.2f}: {'approved' if s >= 0 else 'refused'}")
print("contribution of each feature:")
print(f"  {'starting point':20} {bias:+6.2f}")
for k, w in weights.items():
    print(f"  {k:20} {w * applicant[k]:+6.2f}")
print("smallest single change that would approve this application:")
for k, w in weights.items():
    target = applicant[k] - s / w                 # the value of this feature that brings the score to 0
    if target >= 0:
        print(f"  {k}: {applicant[k]} -> {target:.1f}")
    else:
        print(f"  {k}: no possible value is enough on its own")

Differential privacy in one line of noise

import random

random.seed(3)
def laplace(scale):
    # the difference of two exponential random numbers has a Laplace distribution
    return random.expovariate(1 / scale) - random.expovariate(1 / scale)

true_count = 142            # e.g. how many students in a survey said they had been bullied
for epsilon in (0.1, 0.5, 1.0, 5.0):
    answers = [true_count + laplace(1 / epsilon) for _ in range(1000)]
    typical_error = sum(abs(a - true_count) for a in answers) / len(answers)
    print(f"epsilon {epsilon:3}: one released answer {answers[0]:7.1f}; typical error {typical_error:5.1f}")

Adding or removing one student changes the true count by at most 1, and with ε = 0.5 the noise is typically about 2, so nobody can tell from the answer whether any one student took part.

Try this

1. Change group B's numbers so that its true positive rate matches group A's. What happens to its precision?
2. Give the applicant 0 missed payments. Is the loan approved now? What does the counterfactual say?

Level check

• Describe real cases where AI systems caused harm, and why
• Define demographic parity, equal opportunity, equalised odds and predictive parity, and calculate them
• Explain why fairness definitions can conflict
• Describe where bias enters and three places to mitigate it
• Compare ways of explaining a model's decisions, including counterfactuals
• Explain differential privacy and federated learning
• Explain adversarial examples and defences against them
• Explain what alignment means and how developers test for safety
• Describe the EU AI Act's risk tiers and data protection rights over automated decisions
• Discuss AI's effects on work, information, energy and power

Key terms you must be able to define

Demographic parity

Equal shares of each group receiving the positive decision.

Equal opportunity

Equal true positive rates: deserving people in each group are approved equally often.

Equalised odds

Equal true positive and false positive rates across groups.

Predictive parity

Equal precision across groups: a positive decision means the same for everyone.

Counterfactual explanation

A statement of the smallest change to the inputs that would have changed the decision.

SHAP

A method that estimates how much each feature pushed one prediction up or down.

Differential privacy

Adding measured noise to results so that no answer reveals much about any individual.

Federated learning

Training a shared model on many devices, sending model updates instead of raw data.

Data minimisation

Collecting and keeping only the personal data a task needs.

Adversarial example

An input altered, often invisibly, to make a model give a wrong answer.

Alignment

Making AI systems pursue what people actually intend.

Red teaming

Deliberately attacking a system to find harmful behaviour before release.

EU AI Act

European law that regulates AI systems according to their level of risk.

Flashcards

Go further: notes for future AI engineers

The impossibility result, in one line

Precision, recall and the false positive rate are linked by the base rate p, the share of a group that is truly positive:

FPR = p ÷ (1 − p) × (1 − precision) ÷ precision × TPR

If two groups have the same precision and the same TPR but different base rates p, the right-hand side differs, so their false positive rates must differ. Equal precision and equal error rates cannot all hold at once unless the base rates are equal or the model is perfect. You can check it with the hands-on numbers: group A has p = 0.4, group B has p = 0.2.

Model collapse of the commons

As more of the web is written by AI, future models trained on web text learn partly from earlier models, and rare voices, languages and ideas risk being drowned out. Careful data curation, provenance labels and preserving human-written archives are becoming part of responsible AI too.

Misconceptions to avoid