F200 TA5: Job Roles, Skills and Attributes

A data engineer is responsible for building and maintaining the pipelines that extract, transform and load data so that it arrives reliably and on time. A data analyst is responsible for answering business questions from that data, through querying, analysis, visualisation and reporting to the people who make the decisions.

A correlation shows only that two variables move together, which can happen for several reasons other than one causing the other. A third factor may drive both: ice cream sales and drowning deaths rise and fall together across the year, but neither causes the other, because hot weather causes both more ice cream buying and more swimming. The relationship can also run the opposite way to the one assumed, or be coincidence in a small sample. Establishing cause needs a mechanism and ideally a controlled experiment, not a correlation coefficient.

The analyst should open with the finding in one sentence, not with how it was found: complaints run at roughly three times the normal rate for three days after every release, which is about 120 extra contacts per release. A single line chart of daily complaints with the release dates marked makes the pattern obvious without any explanation, and that should be the only chart in the room. The figures should be given in the management team's own units, complaints and support hours rather than standard deviations, and the uncertainty stated plainly: the pattern holds across the last eight releases, but it shows association rather than cause, and some of the rise may be users contacting support about a change rather than a defect. The analyst should then close with a specific recommendation, such as categorising the complaints from the next two releases to separate defects from confusion, because that distinguishes a testing problem from a communication one and costs nothing to find out. The method and the query should be available as an appendix for anyone who wants it, but not presented.

The technical floor is well defined. An analyst needs SQL, because almost all organisational data sits behind it, together with enough spreadsheet modelling to build something a colleague can use and enough Python or R to automate anything done more than twice. They need visualisation skill, which is mostly the judgement to pick the right chart and resist the decorative one, and enough statistics to know that a mean is misleading on a skewed distribution, that a correlation is not a cause, and that a difference in a small sample may be nothing at all. Underneath those sits data preparation, which is the largest single part of the actual job: finding the data, understanding what each field really means, reconciling two systems that disagree and deciding what to do about missing values, all of which demands patience as much as technique. The personal attributes matter as much as the tools. Attention to detail is what makes an analyst chase a total that is one per cent out instead of rounding it, and that one per cent is quite often the finding. Scepticism is the discipline of distrusting the convenient result, including their own, and checking hardest when the answer is what everybody hoped for. Commercial awareness decides which questions are worth the week it takes to answer them, and ethical judgement decides which ones should not be answered at all. Weighing these, the most important is communication. Every other skill produces a result that exists only inside the analyst's head or their notebook, and a correct analysis that a decision maker does not understand changes nothing, while a clearly explained partial answer changes behaviour the same afternoon. Technical skills can be taught quickly and are increasingly assisted by tooling; the ability to turn a result into a sentence a director will act on, with its uncertainty stated honestly, is the part that makes the whole role worth funding.