×

Controlling the familywise error rate in functional neuroimaging: a comparative review. (English) Zbl 1121.62645

Summary: Functional neuroimaging data embodies a massive multiple testing problem, where 100 000 correlated test statistics must be assessed. The familywise error rate, the chance of any false positives is the standard measure of Type I errors in multiple testing. In this paper we review and evaluate three approaches to thresholding images of test statistics: Bonferroni, random field and the permutation test. Owing to recent developments, improved Bonferroni procedures, such as Hochberg’s methods, are now applicable to dependent data. Continuous random field methods use the smoothness of the image to adapt to the severity of the multiple testing problem. Also, increased computing power has made both permutation and bootstrap methods applicable to functional neuroimaging. We evaluate these approaches on \(t\) images using simulations and a collection of real datasets. We find that Bonferroni-related tests offer little improvement over Bonferroni, while the permutation method offers substantial improvement over the random field method for low smoothness and low degrees of freedom. We also show the limitations of trying to find an equivalent number of independent tests for an image of correlated test statistics.

MSC:

62P10 Applications of statistics to biology and medical sciences; meta analysis
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Hochberg Y, Multiple comparison procedures (1987) · doi:10.1002/9780470316672
[2] Worsley K, Journal of Cerebral Blood Flow & Metabolism 12 pp 900– (1992) · doi:10.1038/jcbfm.1992.127
[3] Friston KJ, Journal of Cerebral Blood Flow & Metabolism 11 pp 690– (1991) · doi:10.1038/jcbfm.1991.122
[4] Storey JD., Journal of the Royal Statistical Society, Series B 64 pp 479– (2002) · Zbl 1090.62073 · doi:10.1111/1467-9868.00346
[5] McShane LM., Proceedings of the International Biometrics Society
[6] Korn DL, Controlling the number of false discoveries: application to high-dimensional genomic data (2001)
[7] Westfall PH, Resampling-based multiple testing: examples and methods for p-value adjustment (1993)
[8] Tong Y., Probability inequalities in multivariate distributions (1980) · Zbl 0455.60003
[9] Holm S., Scandinavian Journal of Statistics 6 pp 65– (1979)
[10] Hochberg Y., Biometrika 75 pp 800– (1988) · Zbl 0661.62067 · doi:10.1093/biomet/75.4.800
[11] Simes RJ., Biometrika 73 pp 751– (1986) · Zbl 0613.62067 · doi:10.1093/biomet/73.3.751
[12] Benjamini Y, Journal of the Royal Statistical Society, Series B, Methodological 57 pp 289– (1995)
[13] Holland BS, Biometrics 43 pp 417– (1987) · Zbl 0654.62068 · doi:10.2307/2531823
[14] Benjamini Y, Annals of Statistics 29 (4) pp 1165– (2001) · Zbl 1041.62061 · doi:10.1214/aos/1013699998
[15] Sarkar SK., Annals of Statistics 30 (1) pp 239– (2002) · Zbl 1101.62349 · doi:10.1214/aos/1015362192
[16] Sarkar SK., Recent advances in multiple testing (2002)
[17] Worsley KJ, Human Brain Mapping 4 pp 58– (1995) · doi:10.1002/(SICI)1097-0193(1996)4:1<58::AID-HBM4>3.0.CO;2-O
[18] Cao J, Lecture Notes in Statistics pp 169– (2001)
[19] Petersson KM, Philosophical Transactions of the Royal Society, Series B 354 pp 1261– (1999) · doi:10.1098/rstb.1999.0478
[20] Adler RJ., The geometry of random fields (1981)
[21] Worsley KJ, Journal of Cerebral Blood Flow & Metabolism 12 (6) pp 900– (1992) · doi:10.1038/jcbfm.1992.127
[22] Poline JB, NeuroImage 5 (2) pp 83– (1997) · doi:10.1006/nimg.1996.0248
[23] Worsley KJ, Human Brain Mapping 8 pp 98– (1999) · doi:10.1002/(SICI)1097-0193(1999)8:2/3<98::AID-HBM5>3.0.CO;2-F
[24] Poline J-B, Journal of Computer Assisted Tomography 19 pp 788– (1995) · doi:10.1097/00004728-199509000-00017
[25] Worsley KJ., 8th International Conference on Functional Mapping of the Human Brain
[26] NeuroImage 16 (2) pp 779– (2002)
[27] Kiebel S, NeuroImage 10 pp 756– (1999) · doi:10.1006/nimg.1999.0508
[28] Worsley KJ., Annals of Statistics 23 pp 640– (1995) · Zbl 0898.62120 · doi:10.1214/aos/1176324540
[29] Sampson PD, Journal of the American Statistical Association 87 pp 108– (1992) · doi:10.1080/01621459.1992.10475181
[30] Hayasaka S, 8th International Conference on Functional Mapping of the Human Brain
[31] NeuroImage 16 (2) pp 1062– (2002)
[32] Dinov ID, Journal of Computer Aided Tomography 24 (1) pp 128– (2000) · doi:10.1097/00004728-200001000-00024
[33] Good P., Permutation tests. A practical guide to resampling methods for testing hypotheses (1994) · Zbl 0815.62027
[34] Peasarin F., Multivariate permutation tests: with applications in biostatistics (2002)
[35] DOI: 10.1007/978-1-4899-4541-9 · doi:10.1007/978-1-4899-4541-9
[36] Davison AC, Bootstrap methods and their application (1997) · doi:10.1017/CBO9780511802843
[37] Nichols TE, Human Brain Mapping 15 pp 1– (2001) · doi:10.1002/hbm.1058
[38] Scheffé H., Annals of Mathematical Statistics 14 pp 304– (1947)
[39] Belmonte M, IEEE Transactions on Medical Imaging 20 pp 243– (2001) · doi:10.1109/42.918475
[40] Liu C, Abstracts of ENAR Meeting of the International Biometric Society
[41] Brammer MJ, Magnetic Resonance Imaging 15 pp 763– (1997) · doi:10.1016/S0730-725X(97)00135-5
[42] Locascio JJ, Human Brain Mapping 3 pp 168– (1997) · doi:10.1002/(SICI)1097-0193(1997)5:3<168::AID-HBM3>3.0.CO;2-1
[43] Bullmore E, Human Brain Mapping 12 pp 61– (2001) · doi:10.1002/1097-0193(200102)12:2<61::AID-HBM1004>3.0.CO;2-W
[44] Fadili J, NeuroImage 15 pp 217– (2001) · doi:10.1006/nimg.2001.0955
[45] Holmes AP, NeuroImage 7 (4) pp S754– (1999)
[46] Proceedings of Fourth International Conference on Functional Mapping of the Human Brain
[47] Henson RNA, Cerebral Cortex 12 pp 178– (2002) · doi:10.1093/cercor/12.2.178
[48] Marshuetz C, Journal of Cognitive Neuroscience 12 (2) pp 130– (2000) · doi:10.1162/08989290051137459
[49] Watson JDG, Cerebral Cortex 3 pp 79– (1993) · doi:10.1093/cercor/3.2.79
[50] Phan KL, Biological Psychiatry 53 pp 211– (2003) · doi:10.1016/S0006-3223(02)01485-3
[51] Holmes AP, Journal of Cerebral Blood Flow & Metabolism 16 (1) pp 7– (1996) · doi:10.1097/00004647-199601000-00002
[52] Stoeckl J, NeuroImage 13 pp S259– (2001) · doi:10.1016/S1053-8119(01)91602-7
[53] Singh KD, NeuroImage 19 pp 1589– (2003) · doi:10.1016/S1053-8119(03)00249-0
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.