Data Analytics F201 Paper B: Big Data and Machine Learning

Machine learning is a system that learns patterns from data in order to make predictions or decisions, improving as it is given more examples, rather than following rules written explicitly by a programmer.

Supervised learning is trained on labelled examples, where each input comes with the correct answer, and learns to predict that answer for new data. Unsupervised learning is given unlabelled data and finds structure within it, such as clusters or anomalies, without being told what to look for.

Accuracy is (TP + TN) / all = (152 + 3342) / 4000 = 3494 / 4000 = 0.874, about 87%. Precision is TP / (TP + FP) = 152 / (152 + 468) = 152 / 620 = 0.245, about 25%. Recall is TP / (TP + FN) = 152 / (152 + 38) = 152 / 190 = 0.80, exactly 80%.

Only 190 of the 4,000 patients actually have the condition, so the classes are heavily imbalanced. A model that simply predicted negative for every patient would be right 3,810 times out of 4,000, an accuracy of about 95%, which is higher than the real model achieves, while finding not a single case. On an imbalanced problem accuracy largely measures the imbalance rather than the model, which is why it should never be quoted alone: precision and recall separate the two things that matter here, how much of the condition the model finds and how often its positive calls are right.

A test set is data held back from training entirely, used at the very end to estimate how the model will perform on cases it has never seen. It should be used once because every time a decision is made on the basis of the test score, which model to keep, which setting to use, information from the test set leaks into those choices. After a few rounds the model has effectively been tuned to that particular sample, and the score it produces measures how well it fits the test set rather than how it will behave on new patients. Tuning belongs on a separate validation set, which can be consulted as often as needed.

Medical imaging, where a single CT or MRI study is hundreds of megabytes and a trust produces thousands a week, which stresses volume above all. Bedside monitoring, where every patient on a ward streams heart rate, oxygen saturation and blood pressure continuously, which stresses velocity because the value of a deteriorating trend is in acting on it now. Free text clinical notes and letters, which stress variety and veracity together: they hold a great deal of the clinically useful information and none of it is structured, abbreviations and spellings vary by author, and the same condition is described five ways.

Overfitting is when a model learns its training data too closely, capturing not only the real pattern but the noise and accidents of that particular sample. The symptom is a model that performs almost perfectly on data it has seen and markedly worse on anything new, because it has memorised rather than generalised. One remedy is to reduce the model's capacity, using fewer features or a simpler, pruned model, because a model with less room to fit cannot fit the noise. A second is to train on more data, since noise does not repeat while the real pattern does, so additional examples dilute the accidents. Alongside both, holding back a validation set and stopping training when validation error stops improving detects the problem as it starts rather than after deployment.

Imaging is the trust's volume problem: a single study runs to hundreds of megabytes, a busy trust produces thousands a week, and the retention periods for clinical images are measured in decades, so the archive grows into petabytes and long ago passed what a single machine can hold. That forces distributed, replicated object storage, and it makes moving the data the binding constraint rather than storing it, because transferring a petabyte over any ordinary link takes weeks. The practical response is to keep the images in one place and send the processing to them, running analysis on machines in the same facility, with GPU capacity alongside the storage, and to tier older studies onto cheaper, slower media rather than keeping everything on the fastest tier.

Patient records are special category data, so processing needs an additional condition as well as a lawful basis. The records were collected to provide care rather than to train a model, which engages purpose limitation. And because this is large scale processing of special category data, a Data Protection Impact Assessment is required before it begins.

Under the UK GDPR a person has the right not to be subject to a decision based solely on automated processing where it produces legal or similarly significant effects, and a decision about someone's clinical care plainly qualifies. Where such processing is permitted at all, three safeguards apply. There must be meaningful human involvement, which means a clinician who genuinely reviews the case and can reach a different conclusion, not one who signs off a queue of scores. The patient must be able to obtain an explanation of the logic involved, which in practice rules out a model whose contributing factors cannot be surfaced. And there must be a route to contest the outcome and have it reconsidered. In a hospital this points firmly towards using the model as decision support that flags patients for review, never as a system that allocates or withholds care by itself.

First, an account belonging to a former employee is still active with full rights, which in a trust means someone no longer employed can read and delete patient data; the fix is an enforced leaver process that disables accounts on the last working day, backed by a quarterly review of all active accounts against the staff list. Second, least privilege is not being applied: an intern and a marketing assistant can export all data, and developers and analysts can read payroll, none of which those roles require. Permissions should be granted through roles, each cut to the minimum the job needs, with any exception time limited and justified. Third, almost every role can delete records and export the entire dataset, so a single compromised account could exfiltrate or destroy the trust's data. Export and deletion should be separated from ordinary read access, limited to named roles, require a second person's approval, and be written to an audit log that is actually reviewed rather than merely retained.

The first is the electricity of training. A substantial model occupies many accelerators continuously for days or weeks, and that cost is not paid once: the published run is preceded by many failed experiments and followed by retraining each time the data or the task shifts. The second is inference, which is counter-intuitive because a single prediction is tiny; multiplied across every patient scored every day for years, the cumulative energy of running a model routinely exceeds the energy of training it. Behind both sit costs that do not appear on the electricity bill: cooling, which can add a large fraction again depending on the facility's efficiency, fresh water consumed by evaporative cooling, and the mined metals and e-waste of a hardware refresh cycle measured in a few years. The mitigations are mostly about doing less work: fine-tune a pretrained model rather than training from scratch, choose the smallest model that answers the question, schedule flexible training when and where the grid is cleanest, and delete the intermediate datasets a training run leaves behind.

The case for deployment is real. A recall of 80% means the model identifies four fifths of the patients who go on to be readmitted within thirty days, and a readmission is both a poor outcome for the patient and an expensive one for the trust. If that list lets a discharge team make a follow-up call, arrange a district nurse visit or check that medication has actually been collected, even a modest reduction in readmissions pays for the work several times over, and it targets scarce follow-up capacity at the patients most likely to need it rather than spreading it evenly. The case against is largely about the other three quarters. With precision at 25%, the model flags 620 patients to find 152, so 468 people are followed up who were never going to be readmitted. That is not harmless: it consumes the same scarce staff time the model is supposed to target, and if the follow-up is intrusive it is an intervention in the lives of people who did not need one. The 38 missed cases matter too, and more subtly, because a list of flagged patients invites staff to treat the unflagged ones as safe, which is a false assurance the model never offered. Two further objections are more serious than the arithmetic. If the trust's historic care was less effective for a deprived or minority group, the model has learned that as though it were clinical fact, so performance must be measured separately for each group and an unequal error rate treated as a defect rather than a footnote. And clinicians cannot act on a score with no reason attached, nor can the trust defend a decision it cannot explain, so a model whose contributing factors can be surfaced is worth more here than a marginally more accurate opaque one. Legally the trust needs a Data Protection Impact Assessment before deployment, an additional condition for processing special category data, transparency with patients, and no solely automated decision with significant effects. My recommendation is to deploy, but as decision support in a limited pilot rather than across the trust, and only once four conditions are met: the impact assessment is complete, error rates have been measured and found comparable across patient groups, the model's main contributing factors are shown alongside each score, and a clinician makes every decision that follows. The pilot should run on one or two wards with readmission rates measured against a comparable ward that does not use it, because the only question that actually matters is whether patients are readmitted less, and that is not answered by the model's accuracy at all.