Data Analytics F201 Paper D: Big Data and Machine Learning
A full length F201 paper built around a supermarket chain: the five Vs, stream and edge processing, transfer times, replication, types of machine learning, a confusion matrix to calculate from, overfitting, the law on loyalty data, the digital divide, PUE, and a 10 mark discussion of personalised offers.
Instructions
- Time allowed
- 1 hour 30 minutes
- Total marks
- 60
- Calculator
- A calculator may be used.
- Answer all the questions.
- The number of marks for each question is shown in brackets. Use it as a guide to how much to write.
- The clock at the top counts down from the real time allowed. When it reaches zero the paper is locked and marked, as it would be collected in the exam hall.
- The 10 mark question is an extended response. In the real exam it is marked in levels; here every relevant point you make earns a mark, up to 10.
- Several questions print a results table: use its numbers. Show your working in every calculation.
Questions
Answer every question, then finish the paper to have it marked. Marking as you go teaches you far less.
| Volume | Velocity | Variety | Veracity | Value | |
|---|---|---|---|---|---|
| Millions of transactions a day add terabytes of data every month | |||||
| Fridge sensors send a reading every second, which must be acted on at once | |||||
| The data includes tables, JSON from the app, product photos and customer reviews | |||||
| Some loyalty card sign-ups contain made-up names and wrong ages | |||||
| The data is only worth keeping if it improves decisions, such as stock levels |
Tick one box in each row.
- LMillions of transactions a day add terabytes of data every month: Volume
- LFridge sensors send a reading every second, which must be acted on at once: Velocity
- LThe data includes tables, JSON from the app, product photos and customer reviews: Variety
- LSome loyalty card sign-ups contain made-up names and wrong ages: Veracity
- LThe data is only worth keeping if it improves decisions, such as stock levels: Value
Amount is volume, speed is velocity, mixed forms are variety, uncertain accuracy is veracity, and usefulness is value.
- LWeather affects what people buy, such as barbecue food and ice cream in hot weather
- LSo demand for these products can be predicted from the forecast
- LSo stores can order the right stock, reducing waste and empty shelves
What people buy depends on the weather: sales of barbecue food, salad and ice cream rise in hot weather and soup sells in the cold. Combining past sales with past weather lets FreshCart predict demand from the forecast, so each store orders the right stock, wasting less fresh food and selling out less often.
- LStream processing handles each reading as it arrives
- LSo a failing fridge raises an alert at once
- LA nightly batch would find the fault hours later, after the food had spoiled
Stream processing handles each reading the moment it arrives, so if a fridge starts to warm up, staff are alerted within seconds and can move the food. A nightly batch would only spot the problem the next morning, by which time the food would have to be thrown away.
- LProcessing data close to where it is created, such as on a device in the store, rather than in a central data centre
- LBenefit: much less data is sent over the network, because only summaries or alerts are sent
- LBenefit: faster responses, and the store keeps working if its connection fails
Edge processing analyses data close to where it is produced, on a small computer in the store, instead of sending every reading to a central data centre. Only alerts and summaries need to be sent, so far less data crosses the network, and the store can react immediately, even if its internet connection is down.
Tick one box.
- LC Time series database
A time series database is optimised for timestamped measurements.
- LCorrect answer, full marks: 88.9 hours
- LMethod: Converts to bits: 40 x 10^12 x 8 = 3.2 x 10^14 bits
- LMethod: Divides by 10^9 to give 320,000 seconds
40 TB = 40 x 10^12 x 8 = 3.2 x 10^14 bits. 3.2 x 10^14 / 10^9 = 320,000 seconds. 320,000 / 3600 = 88.9 hours, nearly four days.
- LIf one data centre fails, the data is still available from another
- LAnd requests can be served from a centre close to the user, or shared between centres, so access is faster
If one data centre fails or loses power, the data is still available from another, so the website and the stores keep working. Requests can also be served from the nearest centre or spread between them, which makes access faster.
| Supervised | Unsupervised | Reinforcement | |
|---|---|---|---|
| Predicting whether a customer will cancel their delivery subscription, learning from past customers who did and did not | |||
| Grouping customers by what they buy, without being told what the groups are | |||
| Learning to plan delivery routes by being rewarded for faster deliveries |
Tick one box in each row.
- LPredicting whether a customer will cancel their delivery subscription, learning from past customers who did and did not: Supervised
- LGrouping customers by what they buy, without being told what the groups are: Unsupervised
- LLearning to plan delivery routes by being rewarded for faster deliveries: Reinforcement
Learning from labelled outcomes is supervised, finding groups without labels is unsupervised, and learning from rewards is reinforcement.
| Training set | Validation set | Test set | |
|---|---|---|---|
| The data the model learns from | |||
| Used to tune the model and to compare versions of it | |||
| Used once, at the end, to estimate how well the model will perform on new data |
Tick one box in each row.
- LThe data the model learns from: Training set
- LUsed to tune the model and to compare versions of it: Validation set
- LUsed once, at the end, to estimate how well the model will perform on new data: Test set
The model learns from the training set, is tuned on the validation set and is judged once on the test set.
Figure 2: the fraud model's results on 1,000 test orders
| Predicted fraud | Predicted genuine | |
|---|---|---|
| Actually fraud | 40 | 20 |
| Actually genuine | 10 | 930 |
- L97.0 %
- L80.0 %
- L66.7 %
Accuracy = (40 + 930) / 1000 = 97.0%. Precision = 40 / (40 + 10) = 80.0%. Recall = 40 / (40 + 20) = 66.7%.
- LA third of fraudulent orders are not caught: 20 of the 60
- LThese cost FreshCart money, so it may need to catch more, even if that flags more genuine orders
A recall of 66.7% means the model misses a third of fraudulent orders: 20 of the 60 got through. Each costs FreshCart money, so it may want to make the model flag more orders, accepting that more genuine ones will be checked by mistake.
Tick one box.
- LD Recall
Recall is the proportion of real fraud the model catches.
- LThe model is overfitting
- LIt has learned the training data, including its noise, too closely, so it does not generalise to new data
- LFix: more training data, a simpler model, fewer features, regularisation or stopping training earlier
The gap shows overfitting: the model has learned the details and noise of its training data so closely that it does not generalise to orders it has not seen. Training it on more data, or using a simpler model with fewer features, would make it learn the general pattern instead.
- LLoyalty card data identifies customers, so it is personal data under the Data Protection Act 2018 and UK GDPR
- LPurpose limitation: data collected for one purpose cannot simply be used for a new, incompatible one
- LIt needs a lawful basis, such as consent, and must tell customers how their data is used (transparency)
Loyalty card data identifies customers and their shopping habits, so it is personal data. Under purpose limitation, data collected to award points cannot simply be reused for a different purpose such as setting prices; FreshCart needs a lawful basis for the new use, such as customers' consent, and must tell them clearly how their data will be used.
| Privacy and Electronic Communications Regulations | Equality Act 2010 | Computer Misuse Act 1990 | Data Protection Act 2018 | |
|---|---|---|---|---|
| Cookies that track customers on the website need their consent | ||||
| A pricing model offers worse prices to customers of one ethnic group | ||||
| An employee accesses the customer database without permission | ||||
| Customer data is kept for longer than necessary |
Tick one box in each row.
- LCookies that track customers on the website need their consent: Privacy and Electronic Communications Regulations
- LA pricing model offers worse prices to customers of one ethnic group: Equality Act 2010
- LAn employee accesses the customer database without permission: Computer Misuse Act 1990
- LCustomer data is kept for longer than necessary: Data Protection Act 2018
Cookies are covered by PECR, discrimination by the Equality Act, unauthorised access by the Computer Misuse Act, and keeping data too long by data protection law.
- LSome customers do not have a smartphone or the skills to use the app, such as some older or poorer people
- LThis is the digital divide
- LSo they pay more for the same shopping, which is unfair, and those groups can least afford it
Not everyone has a smartphone or feels confident using an app, particularly some older people and people on low incomes. This digital divide means they would pay more for the same shopping, which is unfair, and falls hardest on the customers who can least afford it.
- LCorrect answer, full marks: 1.5
- LMethod: Divides the total energy by the IT equipment's energy
PUE = total energy / IT equipment energy = 1.8 / 1.2 = 1.5.
- LAll the energy is used by the IT equipment, with none used for cooling, lighting or other overheads
Every unit of energy would go to the IT equipment, with nothing used for cooling or other overheads.
Tick two boxes.
- LC Electronic waste from servers that are replaced
- LD The electricity used to run and cool data centres
- LNo marks if more than 2 boxes are ticked
Energy use and e-waste are environmental costs. Faster deliveries and personalised offers are benefits.
- LBenefit: personalised offers that customers value, increasing loyalty and sales
- LBenefit: more accurate stock levels, reducing food waste and empty shelves
- LCombining the sources gives a fuller picture, but brings challenges of variety and veracity
- LCamera data and shopping habits are personal data, some possibly special category; a lawful basis, transparency and a DPIA are needed
- LCustomers may feel watched: in-store cameras tracking them is intrusive
- LBias: a model may treat some groups unfairly, which could breach the Equality Act
- LSolely automated decisions with significant effects need safeguards such as human review
- LSecurity: a breach of combined data would be very damaging
- LEnvironmental: training models and storing data use energy and cause e-waste
- LA justified recommendation
The plan offers real benefits. Personalised offers that match what each customer buys make the loyalty scheme more valuable to them and increase sales, and predicting demand from sales history lets each store hold the right stock, cutting food waste and empty shelves. Combining loyalty, online and in-store data gives a far fuller picture than any one source, though the variety of formats and doubtful accuracy of some sign-up data must be dealt with first. The risks are mostly legal and ethical. Shopping habits and camera images are personal data, and images used to identify people are biometric, special category data, so FreshCart needs a lawful basis, must tell customers clearly what it does and should carry out a data protection impact assessment. Many customers would find cameras that track them around the store intrusive, which could damage trust. A model trained on past data may treat some groups less favourably, which could breach the Equality Act, so its decisions should be tested for bias, and any decision with a significant effect on a customer should not be made solely by the model. Combining so much data also makes a breach far more damaging, so access must be tightly controlled. Environmentally, storing the data and training models use a great deal of electricity in data centres, adding to the chain's carbon footprint. I recommend that FreshCart goes ahead with stock forecasting, which uses aggregated sales data and carries little privacy risk, and offers personalised discounts only to customers who opt in, using loyalty and online data but not in-store cameras, with regular bias testing, human oversight and an efficient, low-PUE data centre.
This is an original paper modelled on the real OCR F201 past papers: the same structure, the same balance of topics, the same command words and mark tariffs, 60 marks in 1 hour 30 minutes, exactly as the real paper allows. It is not a past paper. Real past papers are the copyright of the exam board and cannot be republished here, so links to the official papers on the board's own site are on the exam papers page.