Exercise 3: Testing

This exercise will help you practice and demonstrate your ability to write code that is testable, to apply simple test suite adequacy measures, and to reason about regressions. Note that we will revisit testing again in a future exercise that requires you to build tools that can assist in the testing process. Unlike previous exercises, this exercise will not require you to submit (much) valid code. Instead, you will be reasoning about programs and test suites. You should submit your answers via a PDF to CourSys.

Task 1

You have been given the following code by a former friend:

class Cat {
public:
  Cat() {
    catDB.register(this);
  }
    
  void eat() { policy.eat(); }

private:
  EatingPolicy policy;
  static CatDB catDB;
};

a) Explain why this code is challenging to test.

b) Rewrite the code in order to make it more testable.

Task 2

You have been provided code that should compute a simple safety condition for a program.

int
willNotExplode(int a, int b, int c) {
  if ((3 < a*b) & (7 < b*c) & (5 < a*c)) {
    return 1;
  }
  return 0;
}

a) Construct a test suite with the smallest possible number of tests that will provide MC/DC coverage for this code. For each test, show (1) the inputs, (2) the clause outcomes, and (3) the result.

b) Is this test suite good or bad? Explain yourself.

c) This given code avoids short circuiting evaluation. Why does that matter? How would you adapt to it?

Task 3

You have been provided with the following code along with a test suite, and you must determine whether the test suite is good or bad. To do so, you are using mutation testing.

int
countEven(const std::vector<int>& numbers) {
  size_t count = 0;
  for (size_t i = 0, end = numbers.size(); i < end; ++i) {
    if (numbers[i] % 2 == 0) {
      ++count;
    }
  }
  return count;
}

The test suite includes the following inputs for numbers (and appropriate oracles):

• []
• [0, 1, 2, 3, 4, 5]
• [0, 2, 4, 6]
• [1, 3, 5, 7]

Some of the included mutants are:

1. i < end → i != end in the loop guard
2. size count = 0 → size count = 1 at the start
3. (numbers[i] % 2 == 0) → (numbers[i] % 2 == 1)
4. (numbers[i] % 2 == 0) → (numbers[i] % 2 != 0)

Answer the following questions:

a) What effect does mutation (1) have on the mutation score and why?

b) What effect does mutation (2) have on the mutation score and why?

c) What effect do the remaining tests have on the mutation score and why?

Task 4

You are given the following classes and tests:

The suite of tests initially passes. An arrow from one component to another denotes that the source of the arrow depends upon (uses) the target of the arrow. Lack of an arrow indicates one component is not directly affected by another. This is the only information that you initially have about the structure of the classes and tests. Suppose that a change was made to class C.

a) What is the minimum set of tests that you must re-execute in order to know that the test suite still passes after the change?

b) If you had additional knowledge about the classes, might there be fewer tests? Why or why not?

c) Suppose that the lack of an arrow no longer conveys the absence of information. Does this change your answers? If so, how? If not, why not?

Submitting

Submit a PDF containing your responses to all questions via CourSys.