TestNG

TestNG is a unit testing framework for the Java programming language inspired from JUnit and NUnit. The design goal of TestNG is to cover all categories of tests: unit, functional, end-to-end, integration, etc., with more powerful and easy-to-use functionalities.

Features

TestNG's main features include:

1. Annotation support.
2. Support for parameterized and data-driven testing (with @DataProvider and/or XML configuration).
3. Support for multiple instances of the same test class (with @Factory)
4. Flexible execution model. TestNG can be run either by Ant via build.xml (with or without a test suite defined), or by an IDE plugin with visual results. There isn't a TestSuite class, while test suites, groups and tests selected to run are defined and configured by XML files.
5. Concurrent testing: run tests in arbitrarily big thread pools with various policies available (all methods in their own thread, one thread per test class, etc.), and test whether the code is multithread safe.
6. Embeds BeanShell for further flexibility.
7. Default JDK functions for runtime and logging (no dependencies).
8. Dependent methods for application server testing.
9. Distributed testing: allows distribution of tests on slave machines.

Data Provider

A data provider in TestNG is a method in a test class, which provides an array of varied actual values to dependent test methods.

Example:

	//This method will provide data to any test method that declares that its Data Provider is named "test1". 
	@DataProvider(name = "provider1")
	public Object[][] createData1() {
		return new Object[][] { 
			{ "Cedric", new Integer(36) },
			{ "Anne", new Integer(37) }
		};
	}
 
	// This test method declares that its data should be supplied by the Data Provider named "test1".
	@Test(dataProvider = "provider1")
	public void verifyData1(String n1, Integer n2) {
		System.out.println(n1 + " " + n2);
	}
 
	// A data provider which returns an iterator of parameter arrays.
	@DataProvider(name = "provider2")
	public Iterator<Object[]> createData() {
		return new MyIterator(...);
	}	
 
	// A data provider with an argument of the type java.lang.reflect.Method.
	// It is particularly useful when several test methods use the same 
	// provider and you want it to return different values depending on 
	// which test method it is serving. 
	@DataProvider(name = "provider3")
	public Object[][] createData(Method m) {
		System.out.println(m.getName()); 
		return new Object[][] { new Object[] { "Cedric" } };
	}

The returned type of a data provider can be one of the following two types:

• An array of array of objects (Object[][]) where the first dimension's size is the number of times the test method will be invoked and the second dimension size contains an array of objects that must be compatible with the parameter types of the test method.
• An Iterator<Object[]>. The only difference with Object[][] is that an Iterator lets you create your test data lazily. TestNG will invoke the iterator and then the test method with the parameters returned by this iterator one by one. This is particularly useful if you have a lot of parameter sets to pass to the method and you don't want to create all of them upfront.

Tool Support

TestNG is supported, out-of-the-box or via plug-ins, by each of the three major Java IDEs - Eclipse, IntelliJ IDEA, and NetBeans. It also comes with a custom task for Apache Ant and is supported by the Maven build system. The Hudson continuous integration server has built-in support for TestNG and is able to track and chart test results over time. Most Java code coverage tools, such as Cobertura, work seamlessly with TestNG.

Reporting

TestNG generates test reports in HTML and XML formats. The XML output can be transformed by the Ant JUnitReport task^[3] to generate reports similar to those obtained when using JUnit. Since version 4.6, TestNG also provides a reporter API^[4] that permits third-party report generators, such as ReportNG^[5] and TestNG-XSLT,^[6] to be used.

Comparison with JUnit

What are the differences between the two seemingly competing Java tools, TestNG and JUnit? What are the pros and cons? Which one to choose?... These FAQs have been quite hot and debated throughout the decade. Both camps have strong grounds and supporters. Stackoverflow discussions reflect this controversy.^[7][8][9]

Mkyong's article has made a concise and accessible comparison of the two on several aspects, including: annotation support, exception test, ignore/disabling, timeout test, suite and grouping, parameterized test, dependency test, etc.^[10]

Annotations

In JUnit 4, the @BeforeClass and @AfterClass methods have to be declared as static. TestNG does not have this constraint.

TestNG has provided three additional setup/teardown pairs for the suite, test and groups, i.e. @BeforeSuite, @AfterSuite, @BeforeTest, @AfterTest, @BeforeGroup and @AfterGroup.

Parameterized test

This feature is implemented in both tools, however in quite different ways.

TestNG has basically two ways for providing varing parameter values to a test method: by setting the testng.xml, and by defining a @DataProvider method.

In JUnit 4, @RunWith and @Parameters are used together to facilitate parameterized tests, while the @Parameters method has to return List[] with all the actual values, which will be fed into a dedicated class constructor as an argument.

Conclusion

JUnit is often shipped with mainstream IDEs by default, which contributes to its wider popularity. However, TestNG's goal is much wider, which includes not only unit testing, but also support of integration and acceptance testing, etc. It's hard to say which one is better or more suitable, which depends on use contexts and requirements. Going on the competition, the arguments will continue.

See also

• List of unit testing frameworks
• xUnit

References

External links

• TestNG Home page
• TestNG makes Java unit testing a breeze
• TestNG Tutorials and Example
• TestNG Presentation
• IBM Developerworks In pursuit of code quality: JUnit 4 vs. TestNG Why TestNG is still the better framework for large-scale testing