Producing Robust Programs
Defensive design, validation, authentication, maintainability, and how to design a test plan that actually finds the bugs rather than confirming what you already believe.
Defensive design
Defensive design means writing a program that continues to work correctly even when things go wrong: bad input, unexpected use, or a missing file.
Anticipating misuse
Ask what a user might do that you did not intend:
- Typing letters where a number is expected
- Entering a date of birth in the future
- Leaving a field blank
- Entering an enormous number
- Pressing buttons in an unexpected order
- Entering SQL or script code into a text field
Input validation
Validation checks that input is sensible and in the expected format before the program uses it.
| Check | What it does | Example |
|---|---|---|
| Range check | Value falls between limits | Age between 0 and 120 |
| Type check | Data is the expected type | Age is a whole number |
| Presence check | Something has been entered | Surname is not blank |
| Length check | Right number of characters | Password at least 8 characters |
| Format check | Matches a required pattern | Email contains an at sign |
| Look up check | Value is one of a permitted list | Year group is 7, 8, 9, 10 or 11 |
age = input("Enter your age: ")
while not age.isdigit() or int(age) < 0 or int(age) > 120:
print("Please enter a whole number between 0 and 120.")
age = input("Enter your age: ")
age = int(age)This performs a type check and a range check, and loops until the input is acceptable rather than crashing.
Validation checks that data is sensible. Verification checks that it was entered correctly, for example by asking for a password twice or by asking the user to confirm. A date of birth of 01/01/1900 is valid but may not be correct.
Authentication
Authentication confirms the user is who they claim to be.
- Usernames and passwords
- Two factor authentication, where a code is sent to a separate device
- Biometrics such as fingerprint or face recognition
- Security questions
Passwords should be stored hashed, not in plain text, so a stolen database does not reveal them.
Maintainability
Code that is easy to understand and change later.
- Meaningful identifiers.
totalPricenottp - Comments explaining why something is done, not restating what the line obviously does
- Indentation showing structure clearly
- Subroutines so each task is separated
- Constants instead of unexplained numbers scattered through the code
- White space grouping related lines
Programs are read far more often than they are written, and are usually maintained by someone other than the original author. Code that cannot be understood cannot safely be changed.
- Validation checks: presence, type, range, length, format and lookup.
- Validation checks that data is sensible. It cannot know whether it is true.
- Verification asks the user again, for example with a confirm password box, to catch typing slips.
- Three kinds of evidence: something you know, have or are.
- Two factors together are far stronger than one: a stolen password alone is not enough.
- Passwords should be stored as hashes, never as typed, so a leak does not reveal them.
Testing
The two kinds of testing
Iterative testing happens during development. Each part is tested as it is written, and corrected before moving on. Problems are found early, when they are cheap to fix.
Final testing, sometimes called terminal testing, happens when development is complete. The whole program is tested against the original requirements to confirm it does everything it was supposed to.
The three kinds of error
| Error | What it is | Example |
|---|---|---|
| Syntax error | The code breaks the rules of the language, so it will not run at all | Missing colon, misspelled keyword, unclosed bracket |
| Logic error | The program runs, but produces the wrong result | Using + where * was intended, or > where >= was needed |
| Runtime error | The program crashes while running | Dividing by zero, opening a file that does not exist |
The computer finds them for you. Logic errors are dangerous precisely because the program runs perfectly and quietly gives the wrong answer.
Test data
You must be able to choose test data of each type and say what it proves.
| Type | Meaning | Example for ages 11 to 18 |
|---|---|---|
| Normal | Typical values that should be accepted | 14 |
| Boundary | Values at the very edge of what is allowed | 11 and 18, and also 10 and 19 |
| Invalid or erroneous | Values of the right type but outside the allowed range | 25 |
| Erroneous | Values of the wrong type entirely | "fourteen", or a blank entry |
Boundary data is the most valuable, because it is exactly where the off by one errors live. If the check should be >= 11 but was written > 11, only a test with the value 11 will reveal it.
Writing a test plan
A test plan is a table:
| Test | Test data | Type | Expected result | Actual result |
|---|---|---|---|---|
| 1 | 14 | Normal | Accepted | Accepted |
| 2 | 11 | Boundary | Accepted | Accepted |
| 3 | 10 | Boundary | Rejected with a message | Rejected |
| 4 | 25 | Invalid | Rejected with a message | Rejected |
| 5 | "abc" | Erroneous | Rejected with a message | Program crashed |
The last row is the point of testing. Test 5 has found a real fault.
A test with no expected result proves nothing, because you have no standard to compare against. This is worth a mark in test plan questions.
- Syntax error: the code breaks the rules of the language, so it will not run at all.
- Runtime error: the program starts, then crashes when it meets something it cannot do.
- Logic error: it runs without complaint but gives the wrong answer, so only testing finds it.
- Test data for an age from 13 to 19, on a number line.
- Normal data is typical and should be accepted.
- Boundary data sits on and just outside each edge, because that is where mistakes hide.
- A test plan records expected against actual. A failed test shows what to fix.
Key terms you must be able to define
- Defensive design
- Designing a program to keep working correctly even when it is given unexpected input or used incorrectly.
- Validation
- Checking that input data is sensible and in the expected format before it is used.
- Range check
- A validation check that a value falls between an upper and a lower limit.
- Presence check
- A validation check that data has actually been entered.
- Format check
- A validation check that data matches a required pattern.
- Authentication
- Confirming that a user is who they claim to be.
- Maintainability
- How easily a program can be understood and changed by another programmer later.
- Iterative testing
- Testing carried out during development, as each part is written.
- Final testing
- Testing the complete program against its original requirements once development is finished.
- Syntax error
- An error that breaks the rules of the programming language so the program will not run.
- Logic error
- An error where the program runs but produces an incorrect result.
- Boundary data
- Test data at the very edge of the acceptable range, where errors are most likely.
- Erroneous data
- Test data of the wrong type entirely, which should be rejected.
Flashcards
Say the definition out loud before you flip the card. Retrieving it from memory is what builds recall, reading it again is not.
Reach the top grade on this topic
Two things reliably separate the top answers.
Name the specific validation check. Not "check the input is right", but "a range check to confirm the age is between 11 and 18, and a type check to confirm a whole number has been entered".
Choose boundary data that is actually on the boundary. For a range of 11 to 18, the boundary values are 10, 11, 18 and 19. Testing 5 and 30 proves far less, because they are nowhere near the point where the logic changes.
Give every test an expected result. A test plan without expected results scores badly, because there is nothing to compare the actual result against.
Be precise about error types. A syntax error stops the program running at all. A logic error lets it run and produce the wrong answer. Students frequently give a logic error as an example of a syntax error.
- Design a five row test plan with normal, boundary and erroneous data and expected results
- Write a validation loop that rejects both wrong types and out of range values
- Explain three ways to make a program maintainable, each with a reason
- Classify any given error correctly as syntax, logic or runtime
Mistakes that cost marks here
- Giving a logic error as an example of a syntax error. A missing colon is syntax, using plus instead of times is logic.
- Choosing boundary data that is not on the boundary.
- Writing a test plan with no expected results.
- Saying validation checks data is 'correct'. It checks data is sensible. Verification checks it was entered correctly.
- Listing 'use comments' as the only way to improve maintainability, without meaningful names, indentation and subroutines.
- Describing authentication as validation. They are different: authentication confirms identity, validation checks data.
A robust program assumes the user will do the worst possible thing at the worst possible moment. This is not pessimism, it is professionalism. The most expensive bugs in history were all inputs that nobody thought anyone would ever enter.
Knowledge check
Exam-style questions
Five written questions in the style of the real paper. Write a full answer, then let the marker check it against the mark scheme.
- LA presence check to ensure something has been entered
- LA format check to ensure it contains an at sign and a domain
A presence check could be used to ensure that the field has not been left blank. A format check could also be applied, confirming that the entry contains an at sign followed by a domain name with a full stop, since an email address without those parts cannot be valid.
- LA syntax error breaks the rules of the programming language
- LThe program will not run at all until it is corrected
- LA logic error means the program runs but produces an incorrect result
- LGives valid examples of each, such as a missing colon and using the wrong operator
A syntax error occurs when the code breaks the rules of the programming language, for example a missing colon at the end of an if statement or an unclosed bracket. Because the code is not valid, the program will not run at all until the error is corrected, and the development environment will usually point to the offending line. A logic error is different: the code is completely valid and the program runs without complaint, but the instructions do not do what the programmer intended, so the output is wrong. An example would be writing total = total - price instead of total = total + price when calculating a shopping basket, or using a greater than sign where greater than or equal to was needed. Logic errors are more dangerous because nothing alerts the programmer to them, and they can only be found through careful testing.
- LIncludes normal data within the range
- LIncludes boundary data at the edge of the accepted range
- LIncludes boundary data just outside the accepted range
- LIncludes erroneous data of the wrong type
- LStates the type of each piece of test data
- LStates an expected result for every test
Test 1 uses the value 20, which is normal data well within the accepted range, and the expected result is that the value is accepted and processed. Test 2 uses the value 50, which is boundary data at the very top of the accepted range, and the expected result is that it is accepted, since the range is inclusive. Test 3 uses the value 51, which is boundary data just outside the accepted range, and the expected result is that it is rejected with an error message asking the user to try again. Test 4 uses the entry "hot", which is erroneous data of the wrong type entirely, and the expected result is that the program rejects it with an error message rather than crashing. A fifth useful test would be -51, checking the lower boundary from the outside, since an error at one end of a range does not guarantee an error at the other.
- LUse meaningful variable and subroutine names
- LSo that another programmer can understand the purpose without tracing the logic
- LUse comments to explain sections of code
- LUse consistent indentation to show the structure
- LUse subroutines so each task is separated and can be located easily
- LUse constants rather than unexplained numbers appearing throughout the code
The first way is to use meaningful identifiers. Naming a variable totalPriceIncludingVat rather than t means another programmer can understand what the value holds immediately, without tracing back through the code to work it out, and the same applies to subroutine names that describe the task they perform. The second way is to use comments and consistent indentation. Comments should explain why something is being done, particularly where the reason is not obvious, rather than restating what the line clearly does, and consistent indentation makes the structure of loops and selection statements visible at a glance so the flow of the program can be followed quickly. The third way is to break the program into subroutines and use named constants. Separating each task into its own subroutine means a programmer looking for a particular piece of behaviour knows exactly where to find it and can change it without risk to the rest of the program, and replacing values such as 0.2 with a named constant like VAT_RATE means the value appears once, is self explanatory, and can be updated in a single place if the rate ever changes.
- LUsers may enter data that is the wrong type or outside a sensible range
- LWithout validation the program may crash or produce incorrect results
- LValidation checks the input is sensible before it is used, allowing the program to ask again
Input validation is needed because users cannot be relied upon to enter data in the expected form. Someone might type letters instead of digits, enter a negative age, leave the field blank, or enter an impossibly large number, whether by accident or deliberately. Without validation the program would attempt to use that input directly, which could cause it to crash when it tries to convert text into a number, or to continue running and produce meaningless results such as calculating a negative average. Validation checks the input against sensible criteria, such as a type check that a whole number has been entered and a range check that it falls between 0 and 120, and rejects anything that fails with a clear error message so the user can try again. This makes the program robust, meaning it keeps working correctly regardless of what the user does.