Assertions, Pre-conditions, and Testing ======================================= Specifying Functions with Pre and Post Conditions ------------------------------------------------- Programming is a precise activity, and so very often it is important that we define the behaviour of a function precisely. One way to do this is to use function **contracts**. A function contract is essentially a comment that specifies two things: - The function **pre-conditions**, i.e. what must be true before the function is called. - The function **post-conditions**, i.e. what will be true after the function returns. This assumes that the pre-condition was true before the function was called. You can think about the contract informally as the function saying this: If you promise to call only when the pre-condition is true, then I promise to make the post-condition is true. But if the pre-condition is *not* true, I make no promises: anything could happen! For example:: // Pre: n is not 0 // Post: returns 1 / n double invert(int n) { return 1 / n; } The contract for ``invert`` says that you should only call ``invert`` when ``n`` is 0. If ``n`` equals 0, then there is no guarantee what the function will do, i.e. ``invert(0)`` could do anything because passing 0 to it violates the pre-condition. The assert Macro ---------------- ``assert`` is a macro that comes with the C++ standard library that tests if a boolean expression is ``true`` or ``false``. For example:: int n; cin >> n; assert(n > 0); cout << "n is greater than 0"; Here, ``assert`` tests if the expression ``n > 0`` is either ``true`` or ``false``. If ``n > 0`` is ``true`` then ``assert(n > 0)`` *does nothing*. However, if it is ``false``, then ``assert(n > 0)`` immediately crashes the program with a message telling you on what line the assertion failed. In general, ``assert(cond)`` tests if ``cond`` is ``true``. If it isn't, then ``assert`` immediately crashes the program. This forces you to deal with bugs right away, which helps prevent them from causing problems later in your program where it might be much harder to figure out what's gone wrong. It's not hard to implement your assert-like function:: void my_assert(bool b) { if (b == false) { error("assertion failure"); } } Clearly, if the boolean expression ``b`` is ``false``, then the program is halted immediately (recall that the ``error`` function throws a ``runtime_exception`` which, if uncaught, crashes the program). You could certainly use ``my_assert`` in your programs instead of ``assert``. But the advantage of using ``assert`` is that it tells you the file and line number of assertion that failed. It can do this because it is not actually implemented as a C++ function, but is instead a *macro* that is handled by the pre-processor. Macros are textually inserted into your source code before compiling, and so they can do a few tricks functions can't do like determine what line they are on. In general, though, there are very few examples where a macro is preferred over a function, and so we won't cover macros in any detail in this course. Checking Pre-conditions ----------------------- In practice, ``assert`` is used to test if a condition is true at some point in a program. When used wisely, this is a very good way of finding and preventing bugs. For example, consider this simple function:: // Pre: n is not 0 // Post: returns 1 / n double invert(int n) { return 1 / n; } The pre-condition for ``invert`` requires that ``n`` not be equal to 0. But the function doesn't check for that: if you call ``invert(0)`` then it tries to evaluate ``1 / 0``, which makes no sense. A better approach is to use ``assert`` to automatically test the pre-condition every time the function is called:: // Pre: n is not 0 // Post: returns 1 / n double invert(int n) { assert(n != 0); return 1 / n; } Now if you try to evaluate ``invert(0)``, the program will crash with an assertion error. In this case, it means that some other part of the program passed bad data to ``invert``, which is a helpful hint about where to start looking for the bug. Turning Assertions Off ---------------------- Like any piece of code, assertions take time to execute and so a program that calls ``assert`` a lot might run slower than you'd like. This is often fine when you are developing new code because the extra checking provided by the assertions is often well worth the slow-down. However, when your program is ready to be released to the world, your assertions should never be failing. But they are still in your code and still take time to run, and so they slow your program down. You can, of course, get rid of assertions by deleting (or commenting) them all out of your program. But there are two problems with this. First, your program might be big, consisting of dozens of files and hundreds of functions. Deleting assertions in such a large program is tedious and time consuming, and this prone to error. Second, you may want to add the assertions back in at a later time. For instance, a user might report a bug that you didn't find during development, and while debugging it you may find it helpful to put the assertions back in. Adding assertions to a large program is tedious and time- consuming, especially when there's time pressure to fix the bug as soon as possible. So instead, you can "turn off" all calls to ``assert`` using the following trick:: #define NDEBUG // causes asserts to be ignored by the compiler // must come before inclusion of assert.h #include "assert.h" ``NDEBUG`` is a pre-processor variable that can be used to turn assertions on or off. When it is defined, calls to ``assert`` are *not* compiled into your program, and so they take up zero time or space in your running program. Of course, the ``assert`` commands still remain in your source code, and so you can always turn them back on by commenting-out the ``NEDBUG`` define. The check_pre Macro ------------------- A subtle problem with ``assert`` is that it doesn't throw an exception. This means you cannot use ``try`` and ``catch`` to deal with assertion failures. Automated Testing ----------------- Another common use of ``assert`` is to write test cases for testing code. For instance, suppose you want to test if this function works as expected:: // Pre: a >= 0 and b >= 0 c >= 0 // Post: Returns a string consisting of a 'a's, b 'b's, // and c 'c's. All the 'a's come first, followed by // all the 'b's, and then all the 'c's. // Example: // make_abc_string(1, 2, 3) returns "abbccc" // make_abc_string(2, 2, 0) returns "aabb" // make_abc_string(0, 0, 0) returns "" // make_abc_string(1, -2, 3) throws an error string make_abc_string(int a, int b, int c) { check_pre(a >= 0); check_pre(b >= 0); check_pre(c >= 0); return string(a, 'a') + string(b, 'b') + string(c, 'c'); } Testing is a tedious business when you do it by hand. Thus, it's wise to automate your testing whenever possible, and the simplest way to do that is to use ``assert``. Here's a test function you could write that automatically tests ``make_abc_string``:: void make_abc_string_test() { Trace t("make_abc_string"); assert(make_abc_string(1, 2, 3) == "abbccc"); assert(make_abc_string(2, 2, 0) == "aabb"); assert(make_abc_string(0, 0, 0) == ""); try { make_abc_string(1, -2, 3); assert(false); // fail on purpose if the previous function // call doesn't throw an error } catch (runtime_error e) { // if you get here, the test passed, so do nothing } try { make_abc_string(9, 0, -4); assert(false); // fail on purpose if the previous function // call doesn't throw an error } catch (runtime_error e) { // if you get here, the test passed, so do nothing } } int main() { make_abc_string_test(); } The idea behind this kind of testing function is that if any test fails then the program immediately crashes. This lets you deal with error immediately, and provides a concrete example of input that causes a problem (e.g. you can trace through the function by hand using the test case data). If all the tests pass, then the function runs to completion. Notice the use of the ``Trace`` object ``t`` at the top of the function. This prints a message when the function begins and when it ends, which is useful feedback to show that the function has been run. .. note:: ``Trace`` is not a standard part of C++, but is instead implemented in ``std_lib_cmpt125.h``. Notice how ``try`` and ``catch`` are used to check that an exception is thrown when bad data is given to these functions. Since an exception being thrown means the test has passed, we do nothing in the ``catch`` part. Advice for Choosing Good Test Cases ----------------------------------- It's not always easy to figure out the best cases to test a function with, or how many test cases to use. You get better at it with practice, but when you are first starting out it is helpful to keep the following advice in mind: - Choose a variety of test cases that test for different kinds of problems. Every test case should have a reason why it is there. Quality is more important than quantity in testing. - *Extreme values* often make good test cases. For example, extreme values: - for strings: the empty string, strings of length 1, strings consisting of entirely the same character; - for integers: -1, 0, 1 and values around ``INT_MAX`` and ``INT_MIN``; - for files: an empty file, a file with a single character, files with or without a final ``\n``, files consisting entirely of whitespace. - Generally, you want as few test case as possible that thoroughly test a function. - Generally, simpler test cases are more useful than complex test cases. The simpler the test, the easier it is to trace through a function that fails it.