Test Anything Protocol

From Wikipedia, the free encyclopedia

The Test Anything Protocol (TAP) is a general purpose format for transmitting the result of test programs to a thing which interprets and takes action on those results. Though it is language agnostic, it is primarily used by Perl modules.

TAP's general format is:

   1..N
   ok 1 Description # Directive
   # Diagnostic
   ....
   ok 47 Description
   ok 48 Description
   more tests....

For example, a test file's output might look like:

   1..4
   ok 1 - Input file opened
   not ok 2 - First line of the input valid
   ok 3 - Read the rest of the file
   not ok 4 - Summarized correctly # TODO Not written yet

TAP separates the test program from the thing which interprets the results using simple text pipelining. This allows one to run several tests isolated in different processes together in one harness. It allows one to write tests in different languages and run them together with unified results. It also allows the runner of the tests, rather than the author of the tests, to decide how to display the results. The results can be transformed by the runner into other formats (for example, XML) or even cause actions to happen (a smoke test server can send email on failure).

TAP is designed to be human readable. The simple "ok", "not ok" output and descriptions do not require a parser or even extensive knowledge of TAP for the basic intention of each line to be understood by the user.

TAP is designed to be portable. Its plain text format means it will work with any language or operating system.

TAP is designed to be streamable. The results can be interpreted a line at a time without having to wait for the test to complete.

Contents 1 Specification 2 History 3 TAP Parsers 4 TAP Producers

[edit] Specification

Despite its being almost 20 years old and in very wide use, it is only now being formally specified. This is to allow for adoptation by other languages, the writing of additional parsers, and extension of TAP. The current spec is the TAP module on CPAN. This spec is under development and incomplete. Where the spec is lacking, the behavior of Test::Harness with the existing Perl core tests and, to a lesser extent, CPAN module tests, should be used as a guide.

A formal grammar is in development.

[edit] History

TAP came into being with the first version of Perl (1987) although it wasn't called anything then. For most of its existence the only TAP parser was Perl's core test harness (t/TEST) written by Larry Wall. When Perl 5 introduced modules, Test::Harness was written by Tim Bunce and Andreas König allowing Perl module authors to take advantage of TAP. Testing has become very popular in Perl in the last few years and Perl authors working with other languages wishing to take their tools with them have begun to write TAP parsers and TAP-based testing libraries in other languages.

[edit] TAP Parsers

These are libraries which parse TAP and display the results.

• Test::Harness is the oldest and most complete TAP parser. It is limited in how it displays TAP. Though it most often runs tests written in Perl, it can launch any process which generates TAP. Most of the TAP spec is taken from the behavior of Test::Harness.
• The t/TEST parser contained in the Perl source code.
• TAPx::Parser is a new and more flexible parser being written by Curtis "Ovid" Poe.
• Test::Run is a fork of Test::Harness being written by Shlomi Fish.
• TAP::Harness is a rewrite of Test::Harness by Michael G. Schwern. Work in progress and as of November 2006, not really usable.

[edit] TAP Producers

These are libraries for writing tests which output TAP.

• Test::More is the most popular testing module for Perl.
• libtap is a TAP parser written in C.
• Test.Simple is a port of the Perl Test::Simple and Test::More modules to JavaScript by David Wheeler.
• test-harness.php A TAP parser for PHP.
• PyTAP A beginning TAP implementation for Python.