MskProdComputing
Back to the topic

AI · 3.2

Datasets, Splits and Bias: Good Data In, Good AI Out

Section A: 10 multiple choice questions, 10 marks. Section B: 3 written questions, 13 marks. Total 23 marks.

Name

Class

Date

Mark  / 23

Section A

Circle the letter of the correct answer. One mark each.

  1. 1What is the test set for?1
    1. AOne final, honest estimate of how the model performs on unseen data
    2. BStoring data that has been cleaned
    3. CLearning the model's parameters
    4. DChoosing the best settings during development
  2. 2What is the validation set used for?1
    1. AGiving the final score that is reported to users
    2. BLearning the model's parameters from examples
    3. CChoosing settings and comparing models while building
    4. DFilling in values that are missing from the data
  3. 3Which is an example of data leakage?1
    1. APhotos of the same patient appear in both the training and test sets
    2. BThe data is shuffled before splitting
    3. CThe validation set is used to choose settings
    4. DThe training set is larger than the test set
  4. 4Why should time series data not be shuffled before splitting?1
    1. ATime series data cannot be split into different sets
    2. BThe model would learn from the future to predict the past
    3. CShuffling makes the test set much too small to be useful
    4. DShuffling deletes some of the data points in the series
  5. 51 in 200 emails is a phishing attack. A model never flags anything. What is its accuracy?1
    1. A0.5%
    2. B50%
    3. C99.5%
    4. D100%
  6. 6A dataset has a missing age for one student. Which is a sensible fix?1
    1. ASet it to the largest age in the data
    2. BFill it with the median age, or drop the row
    3. CSet it to 0
    4. DDelete the age column from every row
  7. 7A face recognition system was trained mostly on photos of lighter-skinned men. Which bias is this?1
    1. AData leakage
    2. BMeasurement bias
    3. CLabel noise
    4. DSampling bias
  8. 8A recruiting model learns from ten years of past hiring decisions that favoured men. Which bias is this?1
    1. ASampling bias
    2. BClass imbalance
    3. CLeakage
    4. DHistorical bias
  9. 9Why should accuracy be checked for each subgroup of users?1
    1. AChecking subgroups raises the model's overall accuracy
    2. BSmaller subgroups always get a higher accuracy than average
    3. CA good overall score can hide worse results for a small group
    4. DThe law requires every group to get exactly equal accuracy
  10. 10What does random.seed(7) do before random.shuffle?1
    1. AStops the shuffle from changing anything
    2. BPicks seven random items
    3. CShuffles the list seven times
    4. DMakes the shuffle the same every time the program runs

Section B

Answer in the space provided. The number of marks tells you how many separate points to make.

  1. 1ExplainExplain why data is split into training, validation and test sets.4
  2. 2EvaluateA bank trains a fraud detector on data where 0.5% of transactions are fraud. It reports 99.5% accuracy. Evaluate this claim.5
  3. 3DescribeDescribe two different ways bias can enter the data used to train an AI system. Give an example of each.4

Mark scheme

Section A

1A2C3A4B5C6B7D8D9C10D

Section B

1 Explain why data is split into training, validation and test sets. 4

One mark for each of the following, to a maximum of 4.

  • The model must be judged on data it has not seen
  • Training set: learns the parameters
  • Validation set: tune settings or compare models
  • Test set: used once for an honest final estimate

Model answer. A model has to be judged on data it has never seen, otherwise we only measure how well it memorised. The training set is used to learn the parameters. The validation set is used during development to choose settings and compare models. The test set is kept unseen until the end and used once, so it gives an honest estimate of how the model will perform on new data.

2 A bank trains a fraud detector on data where 0.5% of transactions are fraud. It reports 99.5% accuracy. Evaluate this claim. 5

One mark for each of the following, to a maximum of 5.

  • A model that never predicts fraud also scores 99.5%
  • So accuracy mostly measures the imbalance
  • Recall: how much real fraud is caught
  • Precision: how many flagged transactions really are fraud
  • A fix: resampling, class weights or more fraud examples

Model answer. The claim is meaningless on its own. Because only 0.5% of transactions are fraud, a model that never predicts fraud at all would also score 99.5% accuracy while catching nothing, so accuracy here mostly measures the imbalance. The bank should report recall, the share of real fraud that is caught, and precision, the share of flagged transactions that really are fraud. The model could also be improved by oversampling fraud cases or weighting mistakes on them more heavily.

3 Describe two different ways bias can enter the data used to train an AI system. Give an example of each. 4

One mark for each of the following, to a maximum of 4.

  • A first type of bias named and described
  • A valid example of it
  • A second, different type named and described
  • A valid example of the second

Model answer. Sampling bias happens when the data does not represent the people the system will be used on. For example, face analysis systems trained mostly on lighter-skinned faces were found in 2018 to misclassify darker-skinned women far more often. Historical bias happens when data accurately records an unfair past. For example, a recruiting tool trained on ten years of mostly male CVs learned to mark down CVs containing the word women's.