Codec listening test

From Wikipedia, the free encyclopedia

It has been suggested that this article or section be merged with ABX test. (Discuss)

A codec listening test is a scientific study designed to compare two or more lossy audio codecs, usually with respect to perceived fidelity and/or compression efficiency.

Most tests take the form of a double-blind comparison, referred to as "ABX" or "ABC/HR" testing. In an ABX test, the listener has to identify an unknown sample X as being A or B, with A (the original) and B (the encoded version) available for reference. The outcome of a test must be statistically significant. This setup ensures that the listener is not biased by his/her expectations, and that the outcome is not likely to be the result of chance. In an ABC/HR test, if sample X can be identified reliably, the listener can assign a score as a subjective judgement of the quality.

If sample X cannot be determined reliably with a low p-value in a predetermined number of trials, then the null hypothesis cannot be rejected and it cannot be proven that there is a perceptible difference between samples A and B. This usually indicates that the encoded version will actually be transparent to the listener.

There are various software packages available for individuals to perform this type of testing themselves with minimal assistance.

[edit] Results

2005 – "guruboolez"'s ~180 kbit/s classical music test [1] - Vorbis clear winner, MPC second.
2005 – Sebastian Mares' 128 kbit/s test [2]
2004 – Roberto Amorim's second 128 kbit/s test [3] - between 14 and 27 listeners, depending on sample. MPC and Vorbis tied for first.
2003 – Roberto Amorim's first 128 kbit/s test [4] - between 14 and 29 listeners, depending on sample. AAC, MPC, Vorbis, and WMA tied for first.
2002 – ff123's second 128 kbit/s test [5]
2001 – ff123's 128 kbit/s test [6]