Code coverage

From Wikipedia, the free encyclopedia

Code coverage is a measure used in software testing. It describes the degree to which the source code of a program has been tested. It is a form of testing that looks at the code directly and as such comes under the heading of white box testing.

Code coverage techniques were amongst the first techniques invented for systematic software testing. The first published reference was by Miller and Maloney in Communications of the ACM in 1963.

There are a number of different ways of measuring code coverage, the main ones being:

Statement Coverage - Has each line of the source code been executed and tested?
Condition Coverage - Has each evaluation point (such as a true/false decision) been executed and tested?
Path Coverage - Has every possible route through a given part of the code been executed and tested?

Safety critical applications are often required to demonstrate that testing achieves 100% of some form of code coverage.

Some of the coverage criteria above are connected; for instance, path coverage implies both condition and statement coverage. Statement coverage does not imply condition coverage, as the code (in the C programming language) below shows:

void foo(int bar)
{
   printf("This is "); 
   if (bar < 0)
   {
      printf("not ");
   }
   printf("a positive integer.\n");
   return;
}

If the function "foo" were called with variable "bar = -1", statement coverage would be achieved. Condition coverage, however, would not.

Full path coverage, of the type described above, is usually impractical or impossible. Any module with a succession of $n$ decisions in it can have up to $2 n$ paths within it; loop constructs can result in an infinite number of paths. Many paths may also be infeasible, in that there is no input to the program under test that can cause that particular path to be executed. However, a general-purpose algorithm for identifying infeasible paths has been proven to be impossible (such an algorithm could be used to solve the halting problem). Techniques for practical path coverage testing instead attempt to identify classes of code paths that differ only in the number of loop executions, and to achieve "basis path" coverage the tester must cover all the path classes.

Usually the source code is instrumented and run through a series of tests. The resulting output is then analysed to see what areas of code have not been exercised, and the tests are updated to include these areas as necessary. Combined with other code coverage methods the aim is to develop a rigorous yet manageable set of regression tests.

Code coverage is ultimately expressed as a percentage, as in "We have tested 67% of the code." The meaning of this depends on what form(s) of code coverage have been used, as 67% path coverage is more comprehensive than 67% statement coverage.

The value of code coverage as a measure of test quality is debated (see external links).