MskProdComputing
Back to the topic

The mark scheme starts on a new page, so you can print pages one and two for a student and keep the last page yourself. Everything prints in black on white whichever theme you are using.

Python · Advanced 9

Machine Learning from Scratch

Section A: 10 multiple choice questions, 10 marks. Section B: 5 written questions, 21 marks. Total 31 marks.

Name

Class

Date

Mark  / 31

Section A

Circle the letter of the correct answer. One mark each.

  1. 1What is the difference between supervised and unsupervised learning?1
    1. AUnsupervised is faster
    2. BSupervised needs a human watching the program run
    3. CSupervised only works with numbers
    4. DSupervised uses labelled examples; unsupervised has no labels
  2. 2Why is data split into a training set and a test set?1
    1. ATo measure how well the model works on data it has not seen
    2. BTo stop the data being copied
    3. CTo store the data in two places
    4. DTo make the program run twice as fast
  3. 3What is overfitting?1
    1. AThe test set is too large
    2. BThe training takes too long
    3. CThe model fits the training data, noise included, so does badly on new data
    4. DThe model is too small to learn anything
  4. 4How does the k-nearest neighbours algorithm classify a new item?1
    1. ABy averaging all the labels
    2. BBy the most common label among the k closest items in the training data
    3. CBy drawing a straight line through the data
    4. DBy choosing the label at random
  5. 5Why should features be scaled before using k-nearest neighbours?1
    1. ASo that a feature with large values does not dominate the distance
    2. BTo reduce the amount of data
    3. CTo make the labels numbers
    4. DTo remove the test set
  6. 6In gradient descent, what happens if the learning rate is too large?1
    1. AIt gives exactly the same answer
    2. BIt takes a bit longer
    3. CIt finds the answer faster
    4. DThe steps overshoot, and the error can grow instead of shrinking
  7. 7Why can a single perceptron not learn XOR?1
    1. AA perceptron cannot learn anything
    2. BThe two classes cannot be separated by a single straight line
    3. CXOR needs a negative number
    4. DIt has too many inputs
  8. 8What does a model with a precision of 0.9 tell you?1
    1. AIt is right nine times in ten about everything
    2. BIt was trained on 90 items
    3. CNine in ten of the items it flagged as positive really were
    4. DIt found nine in ten of all the positives
  9. 9A disease test always says 'healthy'. Only 5 in 100 people are ill. What is wrong with saying the test is 95% accurate?1
    1. AThe accuracy is high but the test finds none of the ill patients (recall 0)
    2. BAccuracy cannot be calculated
    3. CThere is nothing wrong
    4. DThe accuracy is actually 5%
  10. 10A loan model is accurate overall but wrong far more often for a small group. What is the most likely cause?1
    1. AThe computer made a calculation error
    2. BThe small group was under-represented in the training data
    3. CThe learning rate was too low
    4. DThe model was too complicated

Section B

Answer in the space provided. The number of marks tells you how many separate points to make.

  1. 1ExplainExplain the difference between supervised learning and unsupervised learning.4
  2. 2ExplainExplain why the data used to train a machine learning model is kept separate from the data used to test it.3
  3. 3CalculateA classifier is tested on 200 items. The confusion matrix is shown. Calculate the accuracy, the precision and the recall of the classifier. Show your working.4
                          predicted positive   predicted negative
actually positive           40                   20
actually negative           10                  130
  4. 4DescribeDescribe how the k-nearest neighbours algorithm decides which class a new item belongs to.4
  5. 5DiscussA bank wants to use a machine learning model to decide automatically whether to approve loans. Discuss the benefits and the ethical issues of doing so.6

Mark scheme

Section A

1D2A3C4B5A6D7B8C9A10B

Section B

1 Explain the difference between supervised learning and unsupervised learning. 4

One mark for each of the following, to a maximum of 4.

  • In supervised learning the training data is labelled with the correct answers
  • The model learns the link between the inputs and the labels so it can predict the label for new data
  • In unsupervised learning the data has no labels
  • The model finds patterns or groups (such as clusters) in the data by itself

Model answer. In supervised learning the training data is labelled with the correct answers, and the model learns the relationship between the inputs and the labels so that it can predict the label for new data. In unsupervised learning the data has no labels, and the model finds patterns or groups, such as clusters, in the data by itself.

2 Explain why the data used to train a machine learning model is kept separate from the data used to test it. 3

One mark for each of the following, to a maximum of 3.

  • The test data has not been seen by the model
  • So it shows how well the model works on new data (how well it generalises)
  • Testing on the training data could hide overfitting, where the model has only memorised the examples

Model answer. The test data has not been seen by the model while it was being trained, so the results show how well the model will work on new data, which is how well it generalises. If the model was tested on the data it was trained on, the result could hide overfitting, where the model has only memorised the training examples and would do badly on new data.

3 A classifier is tested on 200 items. The confusion matrix is shown. Calculate the accuracy, the precision and the recall of the classifier. Show your working. 4

One mark for each of the following, to a maximum of 4.

  • Accuracy = (40 + 130) / 200 = 0.85 (85%)
  • Precision = TP / (TP + FP) = 40 / 50
  • Precision = 0.8 (80%)
  • Recall = TP / (TP + FN) = 40 / 60 = 0.67 (66.7%)

Model answer. The accuracy is the number of correct answers divided by the total, (40 + 130) / 200 = 170 / 200 = 0.85, or 85%. The precision is TP / (TP + FP) = 40 / (40 + 10) = 40 / 50 = 0.8, or 80%. The recall is TP / (TP + FN) = 40 / (40 + 20) = 40 / 60 = 0.67, or 66.7%.

4 Describe how the k-nearest neighbours algorithm decides which class a new item belongs to. 4

One mark for each of the following, to a maximum of 4.

  • The distance from the new item to every item in the training data is calculated
  • The k items that are closest (the nearest neighbours) are selected
  • The class of each of those k items is counted
  • The new item is given the class that is most common among them (majority vote)

Model answer. The distance from the new item to every item in the training data is calculated, using the values of its features. The k items that are closest to it, its nearest neighbours, are selected. The class of each of those k items is counted, like a vote, and the new item is given the class that is most common among them.

5 A bank wants to use a machine learning model to decide automatically whether to approve loans. Discuss the benefits and the ethical issues of doing so. 6

One mark for each of the following, to a maximum of 6.

  • Decisions can be made very quickly, and for a large number of applications, at low cost
  • The model is consistent, applying the same rule to every applicant without tiredness or personal prejudice
  • The model learns from past data, so any bias in the data (such as unfair past decisions or too few people from some groups) is copied into its decisions
  • It may be very hard to explain why a person was refused, so they cannot challenge or understand the decision
  • Personal data is used, which raises privacy and data protection issues
  • A reasoned conclusion: it can be used, provided the model is tested for fairness for each group and a human can review and overturn decisions

Model answer. The model could make decisions very quickly for a large number of applications at a low cost, and it would apply the same rule to every applicant, without tiredness or personal prejudice. However, it learns from past data, so any bias in that data, such as unfair earlier decisions or too few examples from some groups of people, would be copied into its decisions. It may also be very hard to explain why a particular person was refused, so they could not understand or challenge the decision, and the model uses a great deal of personal data, which raises privacy and data protection issues. The bank could use it, provided that the model is tested for fairness for each group of people, and that a person is able to review and overturn its decisions.