Oracle (software testing)

From Wikipedia, the free encyclopedia
For other uses, see oracle (disambiguation).

An oracle is a mechanism used by software testers and software engineers for determining whether a test has passed or failed.[1] It is used by comparing the output(s) of the system under test, for a given test case input, to the outputs that the oracle determines that product should have. The term was first used and defined in William Howden's Introduction to the Theory of Testing.[2] Additional work on different kinds of oracles was explored by Elaine Weyuker.[3] Oracles are often separate from the system under test.[4] Method postconditions are commonly used as automated oracles in automated class testing.[5] The oracle problem is often much harder than it seems, and involves solving problems related to controllability and observability.[6]

Common oracles include:

  • specifications and documentation,[7][8]
  • other products (for instance, an oracle for a software program might be a second program that uses a different algorithm to evaluate the same mathematical expression as the product under test)
  • a heuristic oracle that provides approximate results or exact results for a set of a few test inputs,[9]
  • a statistical oracle that uses statistical characteristics,[10]
  • a consistency oracle that compares the results of one test execution to another for similarity,[11]
  • a model-based oracle that uses the same model to generate and verify system behavior,[12]
  • or a human being's judgment (i.e. does the program "seem" to the user to do the correct thing?).[4]

References

  1. ↑ Kaner, Cem; A Course in Black Box Software Testing, 2004
  2. ↑ Miller, Edward; and Howden, William E.; Software Testing and Validation Techniques, Long Beach, CA: IEEE Computer Society Press, 1978 (first edition), pp. 16-19
  3. ↑ Weyuker, Elaine J.; "The Oracle Assumption of Program Testing", in Proceedings of the 13th International Conference on System Sciences (ICSS), Honolulu, HI, January 1980, pp. 44-49
  4. ↑ 4.0 4.1 Jalote, Pankaj; An Integrated Approach to Software Engineering, Springer/Birkhäuser, 2005, ISBN 0-387-20881-X
  5. ↑ Meyer, Bertrand; et al.; "Programs that Test Themselves", IEEE Computer, 42:9 (September 2009), pp. 46-55
  6. ↑ Ammann, Paul; and Offutt, Jeff; "Introduction to Software Testing", Cambridge University Press, 2008, ISBN 978-0-521-88038-1
  7. ↑ Peters, Dennis K. (1995). Generating a Test Oracle from Program Documentation (M. Eng. thesis). McMaster University. CiteSeerX: 10.1.1.69.4331. 
  8. ↑ Peters, Dennis K.; Parnas, David L. "Generating a Test Oracle from Program Documentation". Proceedings of the 1994 International Symposium on Software Testing and Analysis. ISSTA. ACM Press. pp. 58–65. 
  9. ↑ Hoffman, Douglas; Heuristic Test Oracles, Software Testing & Quality Engineering Magazine, 1999
  10. ↑ Mayer, Johannes; and Guderlei, Ralph; Test Oracles Using Statistical Methods
  11. ↑ Hoffman, Douglas; Analysis of a Taxonomy for Test Oracles, Quality Week, 1998
  12. ↑ Robinson, Harry; Finite State Model-Based Testing on a Shoestring, STAR West 1999

Bibliography

  • Binder, Robert V. (1999). "Chapter 18 - Oracles" in Testing Object-Oriented Systems: Models, Patterns, and Tools, Addison-Wesley Professional, 7 November 1999, ISBN 978-0-201-80938-1
This article is issued from Wikipedia. The text is available under the Creative Commons Attribution/Share Alike; additional terms may apply for the media files.