Property Testing

property testing is a sometimes useful approach to testing that says you should test that your function has properties of interest

consider the trim function:

// returns a copy of s with all leading and trailing
// spaces removed
string trim(const string& s);

some properties of trim include:

1. if s consists of 0 or more spaces, then trim(s) == ""
2. for any string s, trim(trim(s)) == trim(s)
3. for any string s, trim(reverse(s)) == reverse(trim(s)) (where reverse(x) is a function that returns the reverse of string x)

the idea of property testing is to write tests that give us confidence that these properties hold

it’s often possible to generate random test cases for such properties

for example, string(n, ' ') returns a string of n spaces, and so you could write test code for the first property like this:

for(int i = 0; i < 100; i++) {     // test code for property 1
    int n = random_nonneg_int();
    string all_spaces(n, ' ');
    if (trim(all_spaces) == "") {
        // test passed
    } else {
        // test failed on s
    }
}

this tries 100 randomly-generated test inputs on trim

this style of testing encourages you to think about the high-level properties of your code

if you choose good properties, it can be quite effective

of course, like all testing, property testing can’t promise to catch all problems, and hand-crafted test cases are still useful for testing important inputs

on the plus side:

• it is usually easy to run many more tests (since they are being generated automatically)
• many programmers find it interesting and useful to come up with properties of their functions

Property Testing

here’s some code that can be used to randomly test some properties of trim:

const string possible_chars = "abcdefghijklmnopqrstuvwxyz0123456789~!@#$%^*()_+";

char rand_char() {
    int r = rand() % possible_chars.size();
    return possible_chars[r];
}

// returns random string of characters; 50% chance a character is a space
// (since spaces are important when testing trim)
string rand_string(int max_len = 10) {
    int size = rand() % (max_len + 1);
    string result(size, ' ');
    for(int i = 0; i < size; i++) {
        if (rand() % 2 == 0) {
            result[i] = rand_char();
        }
    }
    return result;
}

// returns a new string that is the character-by-character reverse of s
string reverse(string s) {
    reverse(begin(s), end(s));
    return s;
}

void trim_property_test() {
    srand (time(NULL));
    const int num_tests = 100;
    for(int i = 0; i < num_tests; i++) {
        string s = rand_string();
        cout << "s = \"" << s << "\"\n";
        assert(trim_bug1(trim_bug1(s)) == trim_bug1(s));
        assert(trim_bug1(reverse(s)) == reverse(trim_bug1(s)));
    }
    cout << num_tests << " random tests passed\n";
}