×

Generalizations of the familywise error rate. (English) Zbl 1072.62060

Summary: Consider the problem of simultaneously testing null hypotheses \(H_1,\dots,H_s\). The usual approach to dealing with the multiplicity problem is to restrict attention to procedures that control the familywise error rate (FWER), the probability of even one false rejection. In many applications, particularly if \(s\) is large, one might be willing to tolerate more than one false rejection provided the number of such cases is controlled, thereby increasing the ability of the procedure to detect false null hypotheses. This suggests replacing control of the FWER by controlling the probability of \(k\) or more false rejections, which we call the \(k\)-FWER.
We derive both single-step and stepdown procedures that control the \(k\)-FWER, without making any assumptions concerning the dependence structure of the \(p\)-values of the individual tests. In particular, we derive a stepdown procedure that is quite simple to apply, and prove that it cannot be improved without violation of control of the \(k\)-FWER. We also consider the false discovery proportion (FDP) defined by the number of false rejections divided by the total number of rejections (defined to be 0 if there are no rejections). The false discovery rate proposed by Y. Benjamini and Y. Hochberg [J. R. Stat. Soc., Ser. B 57, 289–300 (1995; Zbl 0809.62014)] controls \(E(FDP)\). Here, we construct methods such that, for any \(\gamma\) and \(\alpha\), \(P\{FDP>\gamma\}\leq \alpha\). Two stepdown methods are proposed. The first holds under mild conditions on the dependence structure of \(p\)-values, while the second is more conservative but holds without any dependence assumptions.

MSC:

62J15 Paired and multiple comparisons; multiple testing
62G10 Nonparametric hypothesis testing

Citations:

Zbl 0809.62014
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B 57 289–300. · Zbl 0809.62014
[2] Benjamini, Y. and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Ann. Statist. 29 1165–1188. · Zbl 1041.62061 · doi:10.1214/aos/1013699998
[3] Genovese, C. and Wasserman, L. (2004). A stochastic process approach to false discovery control. Ann. Statist. 32 1035–1061. · Zbl 1092.62065 · doi:10.1214/009053604000000283
[4] Hochberg, Y. and Tamhane, A. (1987). Multiple Comparison Procedures . Wiley, New York. · Zbl 0731.62125
[5] Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scand. J. Statist. 6 65–70. · Zbl 0402.62058
[6] Hommel, G. (1983). Tests of the overall hypothesis for arbitrary dependence structures. Biometrical J. 25 423–430. · Zbl 0524.62027
[7] Hommel, G. and Hoffman, T. (1988). Controlled uncertainty. In Multiple Hypotheses Testing (P. Bauer, G. Hommel and E. Sonnemann, eds.) 154–161. Springer, Heidelberg. · Zbl 0591.62071 · doi:10.1007/BF01894737
[8] Korn, E., Troendle, J., McShane, L. and Simon, R. (2004). Controlling the number of false discoveries: Application to high-dimensional genomic data. J. Statist. Plann. Inference 124 379–398. · Zbl 1074.62070 · doi:10.1016/S0378-3758(03)00211-8
[9] Perone Pacifico, M., Genovese, C., Verdinelli, I. and Wasserman, L. (2004). False discovery control for random fields. J. Amer. Statist. Assoc. 99 1002–1014. · Zbl 1055.62105 · doi:10.1198/0162145000001655
[10] Sarkar, S. (1998). Some probability inequalities for ordered \(\mmMTP_2\) random variables: A proof of the Simes conjecture. Ann. Statist. 26 494–504. · Zbl 0929.62065 · doi:10.1214/aos/1028144846
[11] Sarkar, S. and Chang, C. (1997). The Simes method for multiple hypothesis testing with positively dependent test statistics. J. Amer. Statist. Assoc. 92 1601–1608. · Zbl 0912.62079 · doi:10.2307/2965431
[12] Simes, R. (1986). An improved Bonferroni procedure for multiple tests of significance. Biometrika 73 751–754. · Zbl 0613.62067 · doi:10.1093/biomet/73.3.751
[13] van der Laan, M., Dudoit, S. and Pollard, K. (2004). Multiple testing. Part III. Procedures for control of the generalized family-wise error rate and proportion of false positives. Working Paper Series, Paper 141, Div. Biostatistics, Univ. California, Berkeley. · Zbl 1166.62379
[14] Victor, N. (1982). Exploratory data analysis and clinical research. Methods of Information in Medicine 21 53–54.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.