Test harness

In software testing, a test harness^{[note 1]} or automated test framework is a collection of software and test data configured to test a program unit by running it under varying conditions and monitoring its behavior and outputs. It has two main parts: the test execution engine and the test script repository.

Test harnesses allow for the automation of tests. They can call functions with supplied parameters and print out and compare the results to the desired value. The test harness is a hook to the developed code, which can be tested using an automation framework.

A test harness should allow specific tests to run (this helps in optimising), orchestrate a runtime environment, and provide a capability to analyse results.

The typical objectives of a test harness are to:

• Automate the testing process.
• Execute test suites of test cases.
• Generate associated test reports.

A test harness may provide some of the following benefits:

• Increased productivity due to automation of the testing process.
• Increased probability that regression testing will occur.
• Increased quality of software components and application.
• Ensure that subsequent test runs are exact duplicates of previous ones.
• Testing can occur at times that the office is not staffed (e.g. at night)
• A test script may include conditions and/or uses that are otherwise difficult to simulate (load, for example)

An alternative definition of a test harness is software constructed to facilitate integration testing. Where test stubs are typically components of the application under development and are replaced by working component as the application is developed (top-down design), test harnesses are external to the application being tested and simulate services or functionality not available in a test environment. For example, if you're building an application that needs to interface with an application on a mainframe computer but none is available during development, a test harness may be built to use as a substitute. A test harness may be part of a project deliverable. It’s kept outside of the application source code and may be reused on multiple projects. Because a test harness simulates application functionality — it has no knowledge of test suites, test cases or test reports. Those things are provided by a testing framework and associated automated testing tools.

Notes

• Agile Processes in Software Engineering and Extreme Programming, Pekka Abrahamsson, Michele Marchesi, Frank Maurer, Springer, Jan 1, 2009