×

Posterior predictive model checking in hierarchical models. (English) Zbl 1033.62027

Summary: Model checking is a crucial part of any statistical analysis. Hierarchical models present special problems because assumptions made about the distribution of unobservable parameters are difficult to check. We review some approaches to model checking and apply posterior predictive model checking to a hierarchical normal-normal model analysis of data from educational testing experiments in eight schools. Then we carry out a simulation study to investigate the difficulties in model checking for hierarchical models. It turns out that it is very difficult to detect violations of the assumptions made about the population distribution of the parameters unless the extent of violation is huge or the observed data have small standard errors.

MSC:

62F15 Bayesian inference
62P15 Applications of statistics to psychology

Software:

BayesDA
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Bayarri, S.; Berger, J., P-values for composite null models, J. Amer. Statist. Assoc., 95, 1127-1142 (2000) · Zbl 1004.62022
[2] Belin, T. R.; Rubin, D. B., The analysis of repeated-measures data on schizophrenic reaction times using mixture models, Statist. Med., 14, 747-768 (1995)
[3] Box, G. E.P., Sampling and Bayes’ inference in scientific modelling and robustness, J. Roy. Statist. Soc. A, 143, 383-430 (1980) · Zbl 0471.62036
[4] Carlin, B. P.; Louis, T. A., Bayes and Empirical Bayes Methods for Data Analysis (1996), Chapman & Hall: Chapman & Hall London · Zbl 0871.62012
[5] Dempster, A. P.; Ryan, L. M., Weighted normal plots, J. Amer. Statist. Assoc., 80, 845-850 (1985)
[6] Dey, D. K.; Gelfand, A. E.; Vlachos, P. K.; Swartz, T. B., A simulation-intensive approach for checking hierarchical models, Comput. Sci. Statist., 28, 162-171 (1997)
[7] Gelman, A.; Carlin, J. B.; Stern, H. S.; Rubin, D. B., Bayesian Data Analysis (1995), Chapman & Hall: Chapman & Hall London
[8] Gelman, A.; Meng, X. L.; Stern, H. S., Posterior predictive assessment of model fitness via realized discrepancies, Statistica Sinica, 6, 733-807 (1996), with discussion · Zbl 0859.62028
[9] Gilks, W. R.; Richardson, S.; Spiegelhalter, D. J., Markov Chain Monte Carlo in Practice (1996), Chapman & Hall: Chapman & Hall London · Zbl 0832.00018
[10] Glickman, M. E.; Stern, H. S., A state-space model for National Football League scores, J. Amer. Statist. Assoc., 93, 25-35 (1998) · Zbl 0915.62078
[11] Guttman, I., The use of the concept of a future observation in goodness-of-fit problems, J. Roy. Statist. Soc. B, 29, 83-100 (1967) · Zbl 0158.37305
[12] Robins, J. M.; van der Vaart, A.; Ventura, V., The asymptotic distribution of \(p\)-values in composite null models, J. Amer. Statist. Assoc., 95, 1143-1172 (2000) · Zbl 1072.62522
[13] Rubin, D. B., Estimation in parallel randomized experiments, J. Ed. Statist., 6, 377-401 (1981)
[14] Rubin, D. B., Bayesianly justifiable and relevant frequency calculations for the applied statistician, Ann. Statist., 12, 1151-1172 (1984) · Zbl 0555.62010
[15] Sinharay, S., 1998. A look at the application of posterior predictive model checking in hierarchical Bayesian models. Unpublished Technical Report, Department of Statistics, Iowa State University, Ames, IA.; Sinharay, S., 1998. A look at the application of posterior predictive model checking in hierarchical Bayesian models. Unpublished Technical Report, Department of Statistics, Iowa State University, Ames, IA.
[16] Stern, H.S., Cressie, N., 1995. Bayesian and constrained Bayesian inference for extremes in epidemiology. 1995 Proceedings of the Epidemiology Section, American Statistical Association, pp. 11-20.; Stern, H.S., Cressie, N., 1995. Bayesian and constrained Bayesian inference for extremes in epidemiology. 1995 Proceedings of the Epidemiology Section, American Statistical Association, pp. 11-20.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.