MskProdComputing
Back to the topic

The mark scheme starts on a new page, so you can print pages one and two for a student and keep the last page yourself. Everything prints in black on white whichever theme you are using.

Data Analytics · F201 TA4

F201 TA4: Legal and Ethical Issues in Data Management

Section A: 10 multiple choice questions, 10 marks. Section B: 3 written questions, 14 marks. Total 24 marks.

Name

Class

Date

Mark  / 24

Section A

Circle the letter of the correct answer. One mark each.

  1. 1A model infers that a customer is probably pregnant. What is that data?1
    1. ANot personal data, because it was inferred
    2. BSpecial category personal data
    3. CThe company's own intellectual property only
    4. DAnonymous statistical data
  2. 2Why does removing ethnicity from the features not remove racial bias?1
    1. AModels always discriminate
    2. BIt does remove the bias
    3. COther fields such as postcode act as proxies for it
    4. DThe field is still stored elsewhere
  3. 3Under the UK GDPR, a person has the right not to be subject to:1
    1. AA solely automated decision with significant effects, without safeguards
    2. BHaving their data stored in the cloud
    3. CAny automated processing at all
    4. DBeing included in a training set
  4. 4A recruitment model trained on past hiring decisions downgrades certain applicants. What is this?1
    1. AData poisoning
    2. BSampling bias
    3. CA programming error
    4. DHistoric bias learned from the training data
  5. 5What is the black box problem?1
    1. AThe most accurate models are often the least inspectable
    2. BModels cannot store their training data
    3. CEncrypted models cannot be audited
    4. DModels fail without warning
  6. 6Why should model performance be measured per group?1
    1. AThe law requires equal accuracy
    2. BOverall accuracy can hide a model that works for some people and not others
    3. CTo increase the sample size
    4. DTo reduce training time
  7. 7What is data poisoning?1
    1. ADeleting a training set
    2. BOverfitting a model
    3. CCorrupting training data so the model behaves as the attacker wants
    4. DIntroducing noise for privacy
  8. 8What is a dark pattern?1
    1. AAn interface that steers users away from the privacy-protecting choice
    2. BAn unexplainable model
    3. CA hidden data processing activity
    4. DA security vulnerability
  9. 9What is model drift?1
    1. AThe model's predictions becoming more confident
    2. BTraining data being lost over time
    3. CA model gradually using more memory
    4. DPerformance declining as the world the model was trained on changes
  10. 10Which is the strongest argument for a slightly less accurate but explainable model?1
    1. AAccuracy does not matter
    2. BExplainable models are always faster
    3. CThe law bans complex models
    4. DA decision that cannot be explained cannot be defended or appealed

Section B

Answer in the space provided. The number of marks tells you how many separate points to make.

  1. 1StateState two sources of bias in a machine learning model.2
  2. 2ExplainExplain why data inferred by a model about an individual is still subject to data protection law.4
  3. 3DiscussAn insurer plans to use social media activity and shopping data to set premiums. Discuss the legal and ethical problems with this, and what safeguards would be needed.8

Mark scheme

Section A

1B2C3A4D5A6B7C8A9D10D

Section B

1 State two sources of bias in a machine learning model. 2

One mark for each of the following, to a maximum of 2.

  • One valid source
  • A second, different source

Model answer. Historic bias, where the training data records past unfair human decisions, and sampling bias, where a group is under-represented in the training data.

2 Explain why data inferred by a model about an individual is still subject to data protection law. 4

One mark for each of the following, to a maximum of 4.

  • Inferred information relates to an identifiable person
  • It therefore meets the definition of personal data
  • Inferring it is itself processing
  • A consequence, such as special category duties or rights applying

Model answer. Personal data is any information relating to an identified or identifiable living person, and that definition does not depend on how the information arose. A model's inference that a particular customer is likely to be pregnant, in financial difficulty or of a given religious belief relates directly to that individual, so it is personal data in exactly the way a declared field would be. Generating the inference is itself processing, which needs a lawful basis and must be transparent. The consequences are substantial: where the inference concerns health, religion or sexual orientation it is special category data requiring an additional condition, the individual can request access to it and dispute its accuracy, and because a prediction is not a fact the organisation must be careful not to present it as one.

3 An insurer plans to use social media activity and shopping data to set premiums. Discuss the legal and ethical problems with this, and what safeguards would be needed. 8

One mark for each of the following, to a maximum of 8.

  • Purpose limitation and the original collection purpose are addressed
  • Transparency is addressed
  • Special category data by inference is addressed
  • Automated decision making rights are addressed
  • Discrimination or proxy variables are addressed
  • Accuracy and the harm of a wrong inference are addressed
  • Specific safeguards are proposed
  • A judgement weighing the insurer's interest against the harm

Model answer. The insurer has a genuine interest here, since pricing risk accurately is what insurance is, and a model that prices more precisely can lower premiums for lower-risk customers. But the proposal has serious problems. The data was not collected for this: posts and purchases were made for social and commercial purposes, and reusing them to price insurance is a different and incompatible purpose, which engages purpose limitation and means the insurer cannot simply help itself to data that happens to be accessible. Transparency compounds it, because customers would have no realistic idea that their shopping history was affecting their premium, and a privacy notice that technically mentions it does not make the processing fair. The inferences themselves are the legal centre of the problem: shopping and social data are highly revealing, and a model will infer pregnancy, mental health, alcohol consumption and religious observance whether or not it is asked to, which means the insurer would be processing special category data and needs a specific condition for doing so. Because the outcome, a price or a refusal, has a significant effect on the individual, the rules on solely automated decision making apply, so there must be meaningful human involvement, an explanation of the logic and a route to contest the result. Discrimination is the ethical core. Removing protected characteristics from the model does not help, because postcode, store, brand and vocabulary are proxies, and under the Equality Act it is the outcome that matters: a model that prices higher for an ethnic group or a disabled customer is unlawful however innocent the feature list looks. Accuracy makes it worse rather than better, because a wrong inference, a gift bought for someone else, a parody account, produces a real financial penalty the customer cannot see, understand or appeal. The safeguards would have to be substantial: a Data Protection Impact Assessment before anything is built; the feature set cut to data the customer knowingly provided for insurance purposes; explicit exclusion of inferred special category data; testing of error and pricing rates per protected group with unequal outcomes treated as a defect; human review and a published appeal route for every adverse decision; and an audit trail allowing any individual price to be reconstructed and explained. On balance, the social media element should not proceed at all, because the purpose mismatch and the unavoidable inference of sensitive characteristics cannot be safeguarded away, while a narrow use of data the customer supplies for the policy itself can be defended provided the safeguards above are real rather than documentary.