Data Analytics F200 Paper B: Fundamentals of Data Analytics

Quantitative data is numerical and can be calculated with, such as the number of missed bin collections per week. Qualitative data is descriptive rather than numerical, such as residents' written comments explaining why they were dissatisfied with the service.

Census and Office for National Statistics population data, and published government open data such as the index of multiple deprivation.

A1001 appears twice, identically, so any total will double count it. The country column holds one real category written five ways: UK, U.K., United Kingdom, GB and uk, which would split every grouping five ways. The totals are formatted inconsistently, some with a currency symbol, one without and one with a space after it, so a naive import will read them as text rather than numbers; the dates are similarly inconsistent, and 03/01/2026 is ambiguous about whether it is January or March. Finally there are suspect values: a total of 0.00 on a completed order and a blank customer reference on A1004.

Deduplication needs a definition before it needs code: the council should decide what makes two rows the same, here the order reference together with the date and total, review a sample of the proposed matches, then keep one record, merging in the most complete value for each field and preserving both source identifiers. Deleting every row with a blank field is the wrong approach because missing values are rarely random. If the blank customer references all come from one leisure centre whose till does not capture them, deleting those rows removes that centre from the analysis entirely, and the resulting report says that site is quiet when in fact it was excluded. Blanks should be kept and explicitly flagged as unknown, and the pattern of where they occur investigated, because that pattern is often itself the finding.

Three things break it. The identifiers do not match: sales holds 204 while support holds C-204, so a direct join on those columns returns nothing at all, and the keys have to be normalised before anything else happens. The two tables also contain different people: Mihai, 455, appears only in sales and Hargreaves, C-902, only in support, so the answer depends on whether the council wants customers in both systems, in either, or in one specific system, and that is a decision somebody has to make rather than a technical detail. The most dangerous problem is the definitional one. Sales treats a customer as active if they have ordered in the last twelve months, while support treats them as active unless their account has been closed, so Olawale is lapsed in one system and TRUE in the other without either being wrong. Joining them produces a precise number that answers no question. The council has to agree a single written definition of 'active', apply it to both sources, and record it in a data dictionary before counting anything.

Accuracy is whether a value matches reality, so a recorded address is accurate if the resident actually lives there. Completeness is whether anything that should be present is missing, such as records with no postcode. Consistency is whether two systems holding the same fact agree with each other. Timeliness is whether the data is current enough for the decision being made, since a six month old occupancy figure cannot support a decision about next week.

In ETL the data is extracted, transformed to clean and reshape it, and only then loaded, so nothing but modelled data ever reaches the warehouse and storage stays small. The cost is that any field discarded on the way in has to be fetched again from the source if it later turns out to be needed. In ELT the data is extracted, loaded raw, and transformed inside the warehouse using its own compute, so the original remains available and a transformation can simply be rerun when a definition changes. That costs more storage and processing, which is why cheap cloud storage is the reason ELT has become the common choice.

Both servers start the day at about the same point, around 40 to 42 milliseconds, and server B stays there: it drifts gently from 40 to 52 milliseconds over the twelve hours, which is effectively flat. Server A tracks it until about midday and then diverges sharply, climbing from roughly 52 milliseconds to 168 by the end of the period, four times its morning figure and more than three times server B at the same moment. Since both serve the same site, the cause is specific to server A rather than to demand. The next step is to check whether the two are genuinely receiving the same share of traffic, and then to look at server A's own resources over the same hours, because a steady rise through the day that does not recover is the signature of something accumulating, such as a memory leak, an unreleased connection pool or a log file filling the disk. Restarting it and watching whether the curve starts again from 40 would confirm that quickly.

The right of access, the right to rectification, and the right to erasure.

A Data Protection Impact Assessment is a documented exercise that identifies what personal data a proposed activity will process, why, what could go wrong for the people in it, and what will be done to reduce that risk. It has to be carried out before the processing starts, not written up afterwards, because its purpose is to change the design. One is legally required wherever processing is likely to result in a high risk to individuals, which includes large scale processing of special category data, systematic monitoring of a public area, profiling with significant effects, and combining datasets collected for different purposes, all of which a council joining service records would be doing.

Authentication establishes who somebody is, using something they know, have or are, such as a password with a second factor or a certificate. Authorisation then decides what that identity is allowed to do, usually through a role, so an officer in waste services can read waste records but not housing ones and cannot delete anything. The two fail differently and have to be fixed differently: an HTTP 401 means the system does not recognise you at all, while a 403 means it recognises you perfectly well and is refusing anyway.

The analyst should open with the conclusion rather than the method, because the committee is deciding, not reviewing: one sentence saying what the data shows and what it implies for the decision. Everything after that should be in the committee's own units, residents affected, weeks of delay and pounds, rather than percentages of a sample or statistical terms. There should be one chart, carrying one message, large enough to read from the back of the room; if a chart needs a paragraph to interpret, it is the wrong chart. Uncertainty should be stated in plain words rather than hidden or faked, 'somewhere between four and seven thousand households' rather than '5,412', because a figure that implies precision the data cannot support destroys trust the moment somebody tests it. The presentation should end with a specific recommendation and what it would cost, since an analysis that recommends nothing leaves the committee exactly where it started. The method, the sources and the assumptions belong in a written appendix, available to anyone who asks and in front of nobody who does not.

A council is likely to have a lawful basis for this without consent, because supporting vulnerable residents falls within its public task, but a basis for each dataset separately is not automatically a basis for combining them. Waste, housing and leisure records were collected under different statutory functions, so purpose limitation requires the council to show that the new use is compatible with the original collection and to be able to point to the specific function it serves. Data minimisation then constrains the design: the instinct is to join everything available, but a model built on three or four indicators is both more lawful and more likely to work than one built on forty, and every extra field widens the exposure if the dataset leaks. The sensitivity of the result matters more than the sensitivity of the inputs. None of these sources is a health record, yet joined they infer health, disability, financial difficulty and household composition, and inference is processing, so the council would be handling special category data whether or not any field is labelled that way. Because this is large scale processing of sensitive inferences about vulnerable people, a Data Protection Impact Assessment is mandatory and must be completed before anything is built. Residents must also be told, in a published privacy notice written to be read rather than to be compliant, what is being combined and why, with a route to object. Beyond the law there is a question the law does not settle. Residents would be surprised to learn that their bin collections and their gym attendance had been joined to score their household, and that surprise matters for a council, which depends on people reporting difficulties honestly; a service that feels like surveillance gets told less. A false positive risks stigmatising a family and a false negative gives false assurance, and the people affected have the least power to challenge either. The safeguards follow from that. Analysts should work on pseudonymised extracts with the linking key held separately, and only the support team should see identified records, under role based access on least privilege with every access logged. The combined dataset needs a named owner, a documented retention period with automatic deletion, and a scheduled review of whether it is actually producing better outcomes rather than just more flags. Most importantly the output must prompt a human conversation rather than trigger an automated consequence, because a solely automated decision with a significant effect on someone engages their right to human involvement and an explanation. On balance the council can proceed, but only with the impact assessment completed first, the field list cut to the minimum that works, and a standing commitment that an officer, not a score, decides what happens to a household.