Power in multiple testing

Abstract

Multiple hypotheses testing in the context of a correlation matrix is used to compare the statistical power of the original Bonferroni with six modified Bonferroni procedures which control the overall Type I error rate. Three definitions of statistical power are considerated : 1) detecting at least one true relationship, 2) detecting all true relationships, and 3) the average power to detect true relationships. Simulation results show no difference between the seven methods in detecting at least one true relationship; but all six modified Bonferroni procedures are more powerful than the original Bonferroni procedure to detect all true relationship power and average power. Among the six midified Bonferroni procedures, small difference were observed, with the Holm procedure having the lowest power and the Holland-Copenhaver (step-up) procedure having the highest power.