11. Assertions, Pre-conditions, and Testing

11.1. Specifying Functions with Pre and Post Conditions

Programming is a precise activity, and so very often it is important that we define the behaviour of a function precisely. One way to do this is to use function contracts. A function contract is essentially a comment that specifies two things:

  • The function pre-conditions, i.e. what must be true before the function is called.
  • The function post-conditions, i.e. what will be true after the function returns. This assumes that the pre-condition was true before the function was called.

You can think about the contract informally as the function saying this:

If you promise to call only when the pre-condition is true, then I promise to make the post-condition is true.

But if the pre-condition is not true, I make no promises: anything could happen!

For example:

// Pre: n is not 0
// Post: returns 1 / n
double invert(int n) {
        return 1 / n;
}

The contract for invert says that you should only call invert when n is 0. If n equals 0, then there is no guarantee what the function will do, i.e. invert(0) could do anything because passing 0 to it violates the pre-condition.

11.2. The assert Macro

assert is a macro that comes with the C++ standard library that tests if a boolean expression is true or false. For example:

int n;
cin >> n;
assert(n > 0);
cout << "n is greater than 0";

Here, assert tests if the expression n > 0 is either true or false. If n > 0 is true then assert(n > 0) does nothing. However, if it is false, then assert(n > 0) immediately crashes the program with a message telling you on what line the assertion failed.

In general, assert(cond) tests if cond is true. If it isn’t, then assert immediately crashes the program. This forces you to deal with bugs right away, which helps prevent them from causing problems later in your program where it might be much harder to figure out what’s gone wrong.

It’s not hard to implement your assert-like function:

void my_assert(bool b) {
  if (b == false) {
     error("assertion failure");
  }
}

Clearly, if the boolean expression b is false, then the program is halted immediately (recall that the error function throws a runtime_exception which, if uncaught, crashes the program).

You could certainly use my_assert in your programs instead of assert. But the advantage of using assert is that it tells you the file and line number of assertion that failed. It can do this because it is not actually implemented as a C++ function, but is instead a macro that is handled by the pre-processor. Macros are textually inserted into your source code before compiling, and so they can do a few tricks functions can’t do like determine what line they are on. In general, though, there are very few examples where a macro is preferred over a function, and so we won’t cover macros in any detail in this course.

11.3. Checking Pre-conditions

In practice, assert is used to test if a condition is true at some point in a program. When used wisely, this is a very good way of finding and preventing bugs.

For example, consider this simple function:

// Pre: n is not 0
// Post: returns 1 / n
double invert(int n) {
        return 1 / n;
}

The pre-condition for invert requires that n not be equal to 0. But the function doesn’t check for that: if you call invert(0) then it tries to evaluate 1 / 0, which makes no sense.

A better approach is to use assert to automatically test the pre-condition every time the function is called:

// Pre: n is not 0
// Post: returns 1 / n
double invert(int n) {
  assert(n != 0);
        return 1 / n;
}

Now if you try to evaluate invert(0), the program will crash with an assertion error. In this case, it means that some other part of the program passed bad data to invert, which is a helpful hint about where to start looking for the bug.

11.4. Turning Assertions Off

Like any piece of code, assertions take time to execute and so a program that calls assert a lot might run slower than you’d like. This is often fine when you are developing new code because the extra checking provided by the assertions is often well worth the slow-down.

However, when your program is ready to be released to the world, your assertions should never be failing. But they are still in your code and still take time to run, and so they slow your program down.

You can, of course, get rid of assertions by deleting (or commenting) them all out of your program. But there are two problems with this. First, your program might be big, consisting of dozens of files and hundreds of functions. Deleting assertions in such a large program is tedious and time consuming, and this prone to error. Second, you may want to add the assertions back in at a later time. For instance, a user might report a bug that you didn’t find during development, and while debugging it you may find it helpful to put the assertions back in. Adding assertions to a large program is tedious and time- consuming, especially when there’s time pressure to fix the bug as soon as possible.

So instead, you can “turn off” all calls to assert using the following trick:

#define NDEBUG        // causes asserts to be ignored by the compiler
                // must come before inclusion of assert.h
#include "assert.h"

NDEBUG is a pre-processor variable that can be used to turn assertions on or off. When it is defined, calls to assert are not compiled into your program, and so they take up zero time or space in your running program. Of course, the assert commands still remain in your source code, and so you can always turn them back on by commenting-out the NEDBUG define.

11.5. The check_pre Macro

A subtle problem with assert is that it doesn’t throw an exception. This means you cannot use try and catch to deal with assertion failures.

11.6. Automated Testing

Another common use of assert is to write test cases for testing code. For instance, suppose you want to test if this function works as expected:

// Pre: a >= 0 and b >= 0 c >= 0
// Post: Returns a string consisting of a 'a's, b 'b's,
//       and c 'c's. All the 'a's come first, followed by
//       all the 'b's, and then all the 'c's.
// Example:
//     make_abc_string(1, 2, 3) returns "abbccc"
//     make_abc_string(2, 2, 0) returns "aabb"
//     make_abc_string(0, 0, 0) returns ""
//     make_abc_string(1, -2, 3) throws an error
string make_abc_string(int a, int b, int c) {
  check_pre(a >= 0);
  check_pre(b >= 0);
  check_pre(c >= 0);
  return string(a, 'a') + string(b, 'b') + string(c, 'c');
}

Testing is a tedious business when you do it by hand. Thus, it’s wise to automate your testing whenever possible, and the simplest way to do that is to use assert.

Here’s a test function you could write that automatically tests make_abc_string:

void make_abc_string_test() {
  Trace t("make_abc_string");
  assert(make_abc_string(1, 2, 3) == "abbccc");
  assert(make_abc_string(2, 2, 0) == "aabb");
  assert(make_abc_string(0, 0, 0) == "");

  try {
    make_abc_string(1, -2, 3);
    assert(false);  // fail on purpose if the previous function
                    // call doesn't throw an error
  } catch (runtime_error e) {
    // if you get here, the test passed, so do nothing
  }

  try {
    make_abc_string(9, 0, -4);
    assert(false);  // fail on purpose if the previous function
                    // call doesn't throw an error
  } catch (runtime_error e) {
    // if you get here, the test passed, so do nothing
  }
}

int main()
{
  make_abc_string_test();
}

The idea behind this kind of testing function is that if any test fails then the program immediately crashes. This lets you deal with error immediately, and provides a concrete example of input that causes a problem (e.g. you can trace through the function by hand using the test case data). If all the tests pass, then the function runs to completion.

Notice the use of the Trace object t at the top of the function. This prints a message when the function begins and when it ends, which is useful feedback to show that the function has been run.

Note

Trace is not a standard part of C++, but is instead implemented in std_lib_cmpt125.h.

Notice how try and catch are used to check that an exception is thrown when bad data is given to these functions. Since an exception being thrown means the test has passed, we do nothing in the catch part.

11.7. Advice for Choosing Good Test Cases

It’s not always easy to figure out the best cases to test a function with, or how many test cases to use. You get better at it with practice, but when you are first starting out it is helpful to keep the following advice in mind:

  • Choose a variety of test cases that test for different kinds of problems. Every test case should have a reason why it is there. Quality is more important than quantity in testing.
  • Extreme values often make good test cases. For example, extreme values:
    • for strings: the empty string, strings of length 1, strings consisting of entirely the same character;
    • for integers: -1, 0, 1 and values around INT_MAX and INT_MIN;
    • for files: an empty file, a file with a single character, files with or without a final \n, files consisting entirely of whitespace.
  • Generally, you want as few test case as possible that thoroughly test a function.
  • Generally, simpler test cases are more useful than complex test cases. The simpler the test, the easier it is to trace through a function that fails it.