Objects First

If you're going to revel with another programmer,
it's a good idea to choose one who verifies his or her programs before installing them!

Special Values

We have seen that it is wise to include values at the boundaries of classes as special cases. This is because the operators used for comparison, >, <, >=, < have the 'special' property that programmers are likely to include or exclude the = sign.

There are other special values which it is always wise to include in test data sets because of their special properties:

zeroes or nulls - these have the property that they don't generate a change in value for addition or subtraction and
ones - these don't generate changes in values for multiplication and division.

These values (or their equivalents) should

always be included in test sets and
never chosen as the representative of a class.

Mathematically, these special values are known as the identities for various operations. 0 is the identity for addition because:

x + 0 = x

for any value of x. Similarly, 1 is an identity for multiplication, since

x * 1 = x

for any value of x.

In general, any number system, eg the complex numbers, has identities under operations that are permitted on its members.

Testing the null case

All of this clearly suggests that whenever functions operate on collections of data, eg a function that sorts a collection of numbers into order, must always be tested with the empty set. (You can view the empty set as a boundary value or as a special case, but you must not forget it!)

Thus the test data for the function:

int sort( int data[], int n );

should include data sets for which n is 0 (and 1). In this case, the specification of the function will tell you what should happen when sort( d, 0 ); is called. The specification could be written in two ways:

int sort( int data[], int n ); /* Pre-condition: (n > 0) && (data != NULL) / / Post-condition: returns sorted array in data and error code as return value */	In this case, the pre-condition states that it's the caller's responsibility to ensure that `n>0`. This means that the behaviour of `sort` is not specified for `n<=0` - thus any behaviour (including a program crash) is in accordance with the specification. However, our design strategy requires that the implementation of the function should have an assertion checking each pre-condition. Thus the test data should verify that the assertion is raised for `n<=0`, when the program is running in "debug" mode. Naturally, what happens after the assertion is raised is irrelevant - most implementations of `assert` will terminate once the "assertion failed" message is printed out, because further computation is assumed, quite reasonably, to be pointless.
int sort( int data[], int n ); /* Pre-condition: (data != NULL) / / Post-condition: returns sorted array in data and n as return value, with n<0 indicating an error; n=0 on input returns 0 and does nothing */	Here, the behaviour for *n=0* is specified. (It is often useful to have your library functions handle null cases in a defined and predictable way as many algorithms will reduce a problem to a null case. This could avoid large numbers of statements checking pre-conditions in users' programs.)
Both of these examples are acceptable coding practice. The important thing is that the function's behaviour is unambiguously specified: in the first case, it is unambiguous about where the responsibility for *n=0* cases lies - with the caller!

Output

Output (return values from functions, etc) should also be divided into equivalence classes. Values of inputs which generate representatives of the output equivalence classes should be included in the test data also. Thus output values are treated just like input values.

Summary

For each input to your function or program under test, you should determine a test data set which includes:

a representative of each equivalence class, including classes of inputs which raise errors,
boundary values for each class,
special values, in particular,
- nulls or zeroes,
- unities or ones,
- other special cases.

You must also verify that values that violate pre-conditions do, in fact, raise assertions as required by the design strategy. This ensures that certain errors in code which calls the function under test are detected when the system is being tested (and the assertions are still "compiled in").

Test data is best placed in program tables or files. This ensures that it is easy to expand the test set, when:

bugs caused by incorrect equivalence class analysis are discovered,
specifications change, changing the equivalence classes,
and even,
your users require certain cases to be tested because they are particularly important or common (you chose another representative of the class, but you can't convince your user - so it's simpler just to add another line to your test driver file or data structure!)

This last point illustrates a very common fallacy in approaches to testing. This approach requires you to spend a lot of time testing commonly occurring or important cases. Very often, all the commonly occurring and important cases will fall into a very small number of equivalence classes! Thus most of the time spent testing multiple values of these cases is completely wasted! There's also the factor, that, their frequency or importance tends to ensure that quite a bit of effort is put into getting them correct.

It's the uncommon and rare case that you forget to test that will arise at the most inconvenient time - or provide the loop-hole for the unscrupulous!

Thorough and careful analysis of the tests which must be performed is the only way to eliminate the timebombs from your code!

Key terms

identities: Values which when applied to a value by an operator don't change the original value, eg 1 when combined with the multiply operator. In general, if x op I = x for all x and some operator op and value I, then we say "I is an identity under op".