ABX test
From Wikipedia, the free encyclopedia
An ABX test is a method of comparing two kinds of sensory stimuli to identify detectable differences. A subject is presented with two known samples (sample A, the reference, and sample B, an alternative). Of these two samples, one is selected as the unknown sample (sample X), which the subject identifies as being either A or B. If sample X cannot be determined reliably with a low p-value in a predetermined number of trials, then the null hypothesis cannot be rejected and it cannot be proven that there is a perceptible difference between samples A and B.
They can easily be performed as double-blind trials, eliminating any possible unconscious influence from the researcher or supervising technician.
ABX tests are commonly used in evaluations of digital audio data compression methods; sample A is typically a lossless sample without any compression applied, and sample B is a compressed version of A. Audible compression artifacts that indicate a shortcoming in the compression algorithm can be identified with subsequent testing. ABX tests can also be used to compare the different degrees of fidelity loss between two different audio formats at a given bitrate.
Contents |
[edit] Hardware tests
ABX test equipment utilizing relays to switch between two different hardware paths can help determine if there are perceptual differences in cables and components. Video, audio and digital transmission paths can be compared. If the switching is microprocessor controlled, double-blind tests are possible.
Loudspeaker level and line level audio comparisons could be performed on an ABX test device offered for sale as the "ABX Comparator" by QSC Audio Products from 1998 to 2004. Other hardware solutions have been fabricated privately by individuals or organizations for internal testing.
[edit] Confidence
If one ABX trial is performed, a confidence level of only 50% is achieved, the same as flipping a coin. With increasing trial iterations, increasingly accurate results are obtained. Since the limited number of tests, sometimes a 70 % results in favor of “A=X” by example, does not means that is true. It is due to the statisticall probability of biased results when the samples are not infinite. All the ABX tests must be done with at least a 95% Confidence. The company QSC, in the ABX Comparator user manual, recommended a minimum of ten listening trials in each round of tests, as this allows a 95% level of confidence in results. A 95% confidence level means the results are statistically significant.[1] QSC also recommended that no more than 25 trials be performed, as listener fatigue can set in and skew tests.
| Number of trials | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Minimum number correct | 8 | 8 | 9 | 9 | 10 | 11 | 11 | 12 | 12 | 13 | 14 | 14 | 15 | 15 | 16 | 17 |
[edit] Software tests
The foobar2000 and the Amarok audio players support software-based ABX testing, the latter using a third-party script. More ABX software can be found at the archived PCABX website.
