Holm-Bonferroni method

From Wikipedia, the free encyclopedia

In statistics, the Holm-Bonferroni method ^[1] performs more than one hypothesis test simultaneously. It is named after Sture Holm and Carlo Emilio Bonferroni.

Suppose there are k hypotheses to be tested and the overall type 1 error rate is α. Start by ordering the p-values and comparing the smallest p-value to α/k. If that p-value is less than α/k, then reject that hypothesis and start all over with the same α and test the remaining k - 1 hypothesis, i.e. order the k - 1 remaining p-values and compare the smallest one to α/(k - 1). Continue doing this until the hypothesis with the smallest p-value cannot be rejected. At that point, stop and accept all hypotheses that have not been rejected at previous steps.

Here is an example. Four hypotheses are tested with α = 0.05. The four unadjusted p-values are 0.01, 0.03, 0.04, and 0.005. The smallest of these is 0.005. Since this is less than 0.05/4, hypothesis four is rejected. The next smallest p-value is 0.01, which is smaller than 0.05/3. So, hypothesis one is also rejected. The next smallest p-value is 0.03. This is not smaller than 0.05/2. Therefore, hypotheses one and four are rejected while hypotheses two and three are not rejected.

The Holm-Bonferroni method is an example of a closed test procedure ^[2]. As such, it controls the familywise error rate for all the k hypotheses at level α in the strong sense. Each intersection is tested using the simple Bonferroni test.

It is also possible to define a weighted version. Let p₁,..., p_k be the unadjusted p-values and let w₁,..., w_k be a set of corresponding positive weights that add to 1. Without loss of generality, assume the p-values and the weights are all ordered such that p₁/w₁ ≤ p₂/w₂ ≤ ... ≤ p_k/w_k. The adjusted p-value for the first hypothesis is q₁ = min{1,p₁/w₁}. Inductively, define the adjusted p-value for hypothesis i by q_i=min{1,max{q_i-1,(w_i + ... + w_k)×p_i/w_i}}. A hypothesis is rejected at level α if and only if its adjusted p-value is less than α. In the earlier example using equal weights, the adjusted p-values are 0.03, 0.06, 0.06, and 0.02. This is another way to see that using α = 0.05, only hypotheses one and four are rejected by this procedure.

[edit] References

^ Holm, S (1979): "A simple sequentially rejective multiple test procedure", Scandinavian Journal of Statistics, 6:65-70
^ Marcus R, Peritz E, Gabriel KR (1976): "On closed testing procedures with special reference to ordered analysis of variance", Biometrika 63: 655-660