Testing, Unit Tests and Test-Driven Development

1 Describe the difference between black box testing and white box testing. 4

One mark for each of the following, to a maximum of 4.

• In black box testing the tester does not know, or does not use, the code
• Tests are chosen from the specification, checking that the right output is given for an input
• In white box testing the tester knows the code
• Tests are chosen so that every path, branch or statement in the code is executed

Model answer. In black box testing the tester has no knowledge of the code and chooses tests from the specification, checking that each input gives the output that the requirements say it should. In white box testing the tester can see the code, and chooses tests so that every path, branch or statement in it is executed at least once.

2 A company develops a new mobile app. State, with a reason, whether each of the following is an example of alpha testing, beta testing or acceptance testing. (a) Employees of the company who did not write the app use it for a week and report faults. (b) A group of 500 members of the public are given the app before its public release and report faults. (c) The client who ordered the app uses it in their own business and then signs to confirm that it does what they asked. 3

One mark for each of the following, to a maximum of 3.

• (a) Alpha testing, because it is carried out within the organisation, by staff
• (b) Beta testing, because it is carried out by real users before release
• (c) Acceptance testing, because the client confirms that it meets the requirements they ordered

Model answer. (a) is alpha testing, because it is carried out by staff inside the company, who are not the programmers, before anyone outside sees the app. (b) is beta testing, because a group of real users outside the company try the app in their own environment before the general release. (c) is acceptance testing, because the client checks that the app does what they asked for, and accepts it.

3 A function is_palindrome(text) returns True if the text reads the same forwards and backwards, ignoring case, and False otherwise. Write a unittest test case class with at least three tests that use normal, boundary and erroneous data. Include the function as well. 5

One mark for each of the following, to a maximum of 5.

• Imports unittest, and a class that inherits from unittest.TestCase
• Test methods whose names start with test
• A normal data test, such as racecar or Level, with assertTrue or assertFalse
• A boundary test, such as an empty string or one character
• A test for erroneous or unusual data, using assertRaises or a check of the expected result

import unittest

def is_palindrome(text):
    cleaned = text.lower()
    return cleaned == cleaned[::-1]

class TestIsPalindrome(unittest.TestCase):
    def test_normal(self):
        self.assertTrue(is_palindrome("racecar"))
        self.assertTrue(is_palindrome("Level"))
        self.assertFalse(is_palindrome("python"))

    def test_boundary(self):
        self.assertTrue(is_palindrome(""))
        self.assertTrue(is_palindrome("a"))
        self.assertFalse(is_palindrome("ab"))

    def test_erroneous(self):
        with self.assertRaises(AttributeError):
            is_palindrome(12321)

suite = unittest.defaultTestLoader.loadTestsFromTestCase(TestIsPalindrome)
result = unittest.TestResult()
suite.run(result)
print(result.testsRun, "tests,", len(result.failures) + len(result.errors), "problems")

4 Explain how test driven development works, and give one advantage of working in this way. 4

One mark for each of the following, to a maximum of 4.

• A test is written first, for a feature that does not yet exist
• The test is run and fails, which shows that it can detect the absence of the feature
• The simplest code needed to make the test pass is then written, and the code is tidied up afterwards
• An advantage, such as all of the code being covered by tests, or tests protecting later changes

Model answer. In test driven development a test for a small piece of behaviour is written first, before the code exists. The test is run and fails, which shows that the test can detect the missing feature. The simplest code that makes the test pass is then written, and once it passes the code is refactored, which means tidied up with the tests still passing. The cycle repeats for the next feature. An advantage is that all of the code is covered by tests, so later changes can be made with confidence, because the tests show straight away if anything has been broken.

5 A programmer fixes a fault in the calculation of delivery charges in an online shop. Explain why the programmer should carry out regression testing before the shop is updated. 3

One mark for each of the following, to a maximum of 3.

• Regression testing repeats the earlier tests on the program after it has been changed
• A change in one part of a program can cause a fault in a part that was working
• It confirms that the delivery charge fix has not affected, for example, the basket total or the payment

Model answer. Regression testing means repeating the earlier tests on the program after it has been changed. A change to one part of a program can cause a fault in another part that was working before, for example the fix to the delivery charge might alter the basket total or the payment. By repeating the tests, the programmer confirms that everything that worked before still works before the update is released.