The Programming Project (NEA)

1 Explain why success criteria must be measurable. 3

One mark for each of the following, to a maximum of 3.

• The evaluation assesses the finished system against them
• A vague criterion cannot be shown to have been met or not met
• A measurable criterion allows objective evidence to be presented

Model answer. The success criteria written during the analysis stage are what the finished system is evaluated against, so they determine whether the evaluation can be written convincingly at all. A criterion such as the system will be easy to use is subjective, and there is no evidence that could settle whether it was achieved, so the evaluation of it becomes an unsupported opinion. A measurable criterion such as a new user can complete a booking in under sixty seconds without assistance can be tested directly by observing a real user with a stopwatch, which produces objective evidence that can be presented and judged. Writing testable criteria at the start therefore makes both the testing and the evaluation possible, and it also clarifies during design exactly what the system has to achieve.

2 Describe what should be included in each development cycle of the project. 5

One mark for each of the following, to a maximum of 5.

• A statement of what is being built and why it comes next
• Annotated code explaining the techniques used
• Evidence of it working, such as screenshots with the input used
• Testing of that component including cases that should fail
• A review describing what worked, what did not and what was changed

Model answer. Each development cycle should begin with a clear statement of what is being built in that iteration and why it is the sensible next step, which demonstrates that the development was planned rather than arbitrary. The code produced should then be presented with annotation explaining the techniques used and why they were chosen over alternatives, rather than simply pasted in. Evidence that it works must follow, in the form of screenshots that show both the input given and the output produced, since a screenshot of output alone proves nothing. That component should then be tested, including deliberately with data that should be rejected, and the results shown. Finally the cycle should close with a review that honestly describes what worked, what did not, and what was consequently changed, with evidence of the refinement. It is this review and refinement, repeated across cycles, that evidences a genuinely iterative process rather than a single linear attempt.

3 Explain why identifying real stakeholders is important for the project. 4

One mark for each of the following, to a maximum of 4.

• Stakeholders provide the requirements the system must meet
• Their needs justify design decisions during the design stage
• They can be observed using the system during usability testing
• The evaluation depends on their feedback and on the criteria derived from them

Model answer. Real stakeholders matter because three separate sections of the project depend on them. In analysis, they are the source of the requirements: interviewing them establishes what the system actually needs to do rather than what the developer imagines, and the success criteria are derived from those needs. In design, their particular characteristics justify decisions, so choosing large controls and minimal text can be defended by reference to the specific users rather than being asserted as good practice in general. In testing and evaluation, they are the people who use the system while the developer observes, which reveals usability problems the developer cannot possibly see because they already know where everything is, and their feedback is the evidence against which the finished system is judged. A project with an invented stakeholder has nothing genuine to test against and loses marks in all of these areas.

4 Explain what should be included in the evaluation section and why an honest evaluation scores more highly. 5

One mark for each of the following, to a maximum of 5.

• Each success criterion is addressed in turn with evidence
• It states whether each was fully met, partly met or not met
• Usability feedback from stakeholders is reported, including negative feedback
• Limitations and possible further developments are described
• An evaluation with no identified weaknesses appears not to have been genuinely carried out

Model answer. The evaluation should work through each success criterion established during analysis in turn, presenting the evidence relating to it and stating clearly whether it was fully met, partly met or not met, with an explanation where it fell short. It should then report the usability feedback obtained from stakeholders who actually used the system, including the criticism they gave and what was changed in response to it. Finally it should describe the limitations of the finished system honestly, and set out realistic further developments with an explanation of how each would be implemented. Honesty scores more highly because the marks are awarded for the quality of the judgement rather than for the quality of the outcome being described. An evaluation claiming that every criterion was fully met and every user was entirely satisfied is not credible, since no software project achieves that, and it demonstrates either that the evaluation was not genuinely carried out or that the student cannot identify weaknesses in their own work. Identifying a specific shortcoming, explaining why it occurred and describing precisely how it would be addressed shows exactly the critical understanding the mark scheme rewards.

5 A student is choosing a problem for their programming project. Explain what makes a problem suitable. 5

One mark for each of the following, to a maximum of 5.

• It must be genuinely solvable by a computational approach
• It must have sufficient technical depth to demonstrate skill
• It should involve data structures, algorithms and persistence rather than trivial operations
• It must have real stakeholders whose needs can be researched
• It must be achievable within the available time

Model answer. A suitable problem must first be genuinely amenable to a computational solution, meaning it can be defined precisely, the necessary data is obtainable and the rules governing it can be stated explicitly. It must then have enough technical depth to allow the student to demonstrate real skill, because the mark band is capped by the difficulty of what is attempted, so a flawlessly implemented but trivial program cannot reach the highest marks. In practice this means the problem should naturally require non trivial data structures, meaningful algorithms, validated input and some form of persistent storage, rather than being a sequence of simple calculations. There must be real stakeholders who can be interviewed at the analysis stage and observed at the testing stage, since the requirements, usability testing and evaluation all depend on them. Finally the problem must be achievable within the time available, because an over ambitious project that is left incomplete loses marks across development, testing and evaluation, and a well executed project of moderate ambition scores far better than an unfinished one of great ambition.