×

A predictive view of Bayesian clustering. (English) Zbl 1090.62023

Summary: This work considers probability models for partitions of a set of \(n\) elements using a predictive approach, i.e., models that are specified in terms of the conditional probability of either joining an already existing cluster or forming a new one. The inherent structure can be motivated by resorting to hierarchical models of either parametric or nonparametric nature. Parametric examples include the product partition models (PPMs) and the model-based approach of A. Dasgupta and A.E. Raftery [J. Am. Stat. Assoc. 93, 294–302 (1998; Zbl 0906.62105)], while nonparametric alternatives include the Dirichlet process, and more generally, the species sampling models (SSMs). Under exchangeability, PPMs and SSMs induce the same type of partition structure. The methods are discussed in the context of outlier detection in normal linear regression models and of (univariate) density estimation.

MSC:

62F15 Bayesian inference
62H30 Classification and discrimination; cluster analysis (statistical aspects)
65C60 Computational problems in statistics (MSC2010)

Citations:

Zbl 0906.62105

Software:

mclust; AS 136
PDFBibTeX XMLCite
Full Text: DOI Link

References:

[1] Aitkin, M.; Tunnicliffe Wilson, G., Mixture models, outliers, and the EM algorithm, Technometrics, 22, 325-331 (1980) · Zbl 0466.62034
[2] Antoniak, C. E., Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems, Ann. Statist., 2, 1152-1174 (1974) · Zbl 0335.60034
[3] Arratia, R.; Barbour, A. D.; Tavaré, S., Poisson process approximations for the Ewens sampling formula, Ann. Appl. Probab., 2, 519-535 (1992) · Zbl 0756.60006
[4] Banfield, J. D.; Raftery, A. E., Model-based Gaussian and non-Gaussian clustering, Biometrics, 49, 803-821 (1993) · Zbl 0794.62034
[5] Barry, D.; Hartigan, J. A., Product partition models for change point problems, Ann. Statist., 20, 260-279 (1992) · Zbl 0780.62071
[6] Barry, D.; Hartigan, J. A., A Bayesian analysis for change point problems, J. Amer. Statist. Assoc., 88, 309-319 (1993) · Zbl 0775.62065
[7] Belsley, D. A.; Kuh, E.; Welsch, R. E., Regression DiagnosticsIdentifying Influential Data and Sources of Collinearity (1980), Wiley: Wiley New York · Zbl 0479.62056
[8] Blackwell, D.; MacQueen, J. B., Ferguson distributions via Pólya urn schemes, Ann. Statist., 1, 353-355 (1973) · Zbl 0276.62010
[9] Brunner, L., Chan, A., James, L., Lo, A.Y., 2001. Weighted Chinese restaurant processes and Bayesian mixture models. Unpublished manuscript.; Brunner, L., Chan, A., James, L., Lo, A.Y., 2001. Weighted Chinese restaurant processes and Bayesian mixture models. Unpublished manuscript.
[10] Bush, C. A.; MacEachern, S. N., A semiparametric Bayesian model for randomised block designs, Biometrika, 83, 275-285 (1996) · Zbl 0864.62052
[11] Cifarelli, D. M.; Melilli, E., Some new results for Dirichlet priors, Ann. Statist., 28, 1390-1413 (2000) · Zbl 1105.62303
[12] Crowley, E. M., Product partition models for normal means, J. Amer. Statist. Assoc., 92, 192-198 (1997) · Zbl 0889.62011
[13] Cruz, F.; Loschi, R.; Iglesias, P.; Arellano-Valle, R., A Gibbs sampling scheme to product partition modelan application to change-point problems, Comput. Oper. Res., 30, 463-482 (2003) · Zbl 1012.62096
[14] Damien, P.; Wakefield, J. C.; Walker, S. G., Gibbs sampling for Bayesian non-conjugate and hierarchical models using auxiliary variables, J. Roy. Statist. Soc. Ser. B, 61, 331-344 (1997) · Zbl 0913.62028
[15] Dasgupta, A.; Raftery, A. E., Detecting features in spatial point processes with cluster via model-based clustering, J. Amer. Statist. Assoc., 93, 294-302 (1998) · Zbl 0906.62105
[16] Dempster, A. P.; Laird, N. M.; Rubin, D. B., Maximum likelihood from incomplete data via the EM algorithm (with discussion), J. Roy. Statist. Soc. Ser. B, 39, 1-37 (1977) · Zbl 0364.62022
[17] Devroye, L.; Lugos, G., Combinatorial Methods in Density Estimation (2001), Springer: Springer New York
[18] Diaconis, P.; Freedman, D., On the consistency of Bayes estimates (with discussion), Ann. Statist., 14, 1-67 (1986) · Zbl 0595.62022
[19] Diaconis, P.; Kemperman, J., Some new tools for Dirichlet priors, (Bernardo, J. M.; Berger, J. O.; Dawid, A. P.; Smith, A. F.M., Bayesian Statistics 5. Proceedings of the Fourth Valencia International Meeting (1996), Oxford University Press: Oxford University Press Oxford), 97-106
[20] Escobar, M. D.; West, M., Bayesian density estimation and inference using mixtures, J. Amer. Statist. Assoc., 90, 577-588 (1995) · Zbl 0826.62021
[21] Ewens, W., The sampling theory of selectively neutral alleles, Theoret. Population Biol., 3, 87-112 (1972) · Zbl 0245.92009
[22] Ferguson, T. S., A Bayesian analysis of some nonparametric problems, Ann. Statist., 1, 209-230 (1973) · Zbl 0255.62037
[23] Fraley, C.; Raftery, A. E., How many clusters? Which clustering methods? Answers via model-based cluster analysis, Comput. J., 41, 578-588 (1998) · Zbl 0920.68038
[24] Fraley, C.; Raftery, A. E., Model-based clustering discriminant analysis and density estimation, J. Amer. Statist. Assoc., 97, 611-631 (2002) · Zbl 1073.62545
[25] Fraley, C., Raftery, A.E., 2002b. MCLUST: software for model-based clustering, density estimation and discriminant analysis. Technical Report, Department of Statistics, University of Washington.; Fraley, C., Raftery, A.E., 2002b. MCLUST: software for model-based clustering, density estimation and discriminant analysis. Technical Report, Department of Statistics, University of Washington. · Zbl 1073.62545
[26] Hartigan, J. A., Partition models, Comm. Statist. Part A—Theory Methods, 19, 2745-2756 (1990)
[27] Hartigan, J. A.; Wong, M. A., A k-means clustering algorithm, Appl. Statist., 28, 100-108 (1979) · Zbl 0447.62062
[28] Hennig, C., Fixed point clusters for linear regressioncomputation and comparison, J. Classification, 19, 249-276 (2002) · Zbl 1017.62057
[29] Hennig, C., Clusters, outliers, and regressionfixed point clusters, J. Multivariate Anal., 86, 183-212 (2003) · Zbl 1020.62051
[30] Hocking, R. R., Methods and Applications of Linear ModelsRegression and the Analysis of Variance (1996), Wiley: Wiley New York
[31] Hocking, R. R.; Pendleton, O. J., The regression dilemma, Comm. Statist. Ser. A, 12, 497-527 (1983) · Zbl 0513.62066
[32] Ishwaran, H.; James, L. F., Gibbs sampling methods for stick-breaking priors, J. Amer. Statist. Assoc., 96, 161-173 (2001) · Zbl 1014.62006
[33] Ishwaran, H., James, L.F., 2003a. Generalized weighted Chinese restaurant processes for species sampling mixture models. Statist. Sinica 13, 1211-1235.; Ishwaran, H., James, L.F., 2003a. Generalized weighted Chinese restaurant processes for species sampling mixture models. Statist. Sinica 13, 1211-1235. · Zbl 1086.62036
[34] Ishwaran, H., James, L.F., 2003b. Some further developments for stick-breaking priors: finite and infinite clustering and classification. Sankhya, Ser. A 65, 577-592.; Ishwaran, H., James, L.F., 2003b. Some further developments for stick-breaking priors: finite and infinite clustering and classification. Sankhya, Ser. A 65, 577-592. · Zbl 1193.62106
[35] Jain, S.; Neal, R. M., A split-merge Markov chain Monte Carlo procedure for the Dirichlet process mixture model, J. Comput. Graphic. Statist., 13, 158-182 (2004)
[36] Jorgensen, M., Influence-based diagnostics for finite mixture models, Biometrics, 46, 1047-1058 (1990)
[37] Kong, A.; Liu, J. S.; Wong, W. H., Sequential imputations and Bayesian missing data problems, J. Amer. Statist. Assoc., 89, 278-288 (1994) · Zbl 0800.62166
[38] Korwar, R. M.; Hollander, M., Contributions to the theory of Dirichlet processes, Ann. Probab., 1, 705-711 (1973) · Zbl 0264.60084
[39] Liu, J. S., Nonparametric hierarchical Bayes via sequential imputations, Ann. Statist., 24, 911-930 (1996) · Zbl 0880.62038
[40] Lo, A. Y., On a class of Bayesian nonparametric estimatesI. Density estimates, Ann. Statist., 12, 351-357 (1984) · Zbl 0557.62036
[41] Lo, A.Y., Brunner, L.J., Chan, A.T., 1998. Weighted Chinese restaurant processes and Bayesian mixture models. Technical Report, Department of Information and Systems Management, Hong Kong University of Science and Technology.; Lo, A.Y., Brunner, L.J., Chan, A.T., 1998. Weighted Chinese restaurant processes and Bayesian mixture models. Technical Report, Department of Information and Systems Management, Hong Kong University of Science and Technology.
[42] Loschi, R.; Cruz, F., An analysis of the influence of some prior specifications in the identification of change points via product partition model, Comput. Statist. Data Anal., 39, 477-501 (2002) · Zbl 1101.62317
[43] MacEachern, S. N.; Müller, P., Estimating mixture of Dirichlet process models, J. Comput. Graphic. Statist., 7, 2, 223-338 (1998)
[44] MacEachern, S. N.; Clyde, M.; Liu, J. S., Sequential importance sampling for nonparametric Bayes modelsthe next generation, Canad. J. Statist., 27, 251-267 (1999) · Zbl 0957.62068
[45] McLachlan, G. J.; Basford, K. E., Mixture ModelsInference and Applications to Clustering (1988), Marcel Dekker: Marcel Dekker New York
[46] McLachlan, G. J.; Peel, D., Finite Mixture Models (2000), Wiley: Wiley New York · Zbl 0963.62061
[47] Muliere, P., Secchi, P., 1995. A note on a proper Bayesian Bootstrap. Technical Report 18, Dipartamento di Economia Politica e Metodi Quantitativi, Università degli Sudi di Pavia.; Muliere, P., Secchi, P., 1995. A note on a proper Bayesian Bootstrap. Technical Report 18, Dipartamento di Economia Politica e Metodi Quantitativi, Università degli Sudi di Pavia.
[48] Müller, P., Quintana, F.A., 2004. Nonparametric Bayesian data analysis. Statist. Sci. 19 (1), 95-110.; Müller, P., Quintana, F.A., 2004. Nonparametric Bayesian data analysis. Statist. Sci. 19 (1), 95-110. · Zbl 1057.62032
[49] Murtagh, F.; Raftery, A. E., Fitting straight lines to point patterns, Pattern Recognition, 17, 479-483 (1984)
[50] Neal, R. M., Markov chain sampling methods for Dirichlet process mixture models, J. Comput. Graphic. Statist., 9, 249-265 (2000)
[51] Pitman, J., 1996. Some developments of the Blackwell-MacQueen Urn scheme. In: Ferguson, T.S., Shapeley, L.S., MacQueen, J.B. (Eds.), Statistics, Probability and Game Theory. Papers in Honor of David Blackwell, Haywar, California: IMS Lecture Notes—Monograph Series, pp. 245-268.; Pitman, J., 1996. Some developments of the Blackwell-MacQueen Urn scheme. In: Ferguson, T.S., Shapeley, L.S., MacQueen, J.B. (Eds.), Statistics, Probability and Game Theory. Papers in Honor of David Blackwell, Haywar, California: IMS Lecture Notes—Monograph Series, pp. 245-268.
[52] Quintana, F. A.; Iglesias, P. L., Bayesian clustering and product partition models, J. Roy. Statist. Soc. Ser. B, 65, 557-574 (2003) · Zbl 1065.62115
[53] Quintana, F. A.; Newton, M. A., Computational aspects of nonparametric Bayesian analysis with applications to the modeling of multiple binary sequences, J. Comput. Graphic. Statist., 9, 4, 711-737 (2000)
[54] Redner, R. A.; Walker, H. F., Mixture densities maximum likelihood and the EM algorithm, SIAM Rev., 26, 195-202 (1984) · Zbl 0536.62021
[55] Richardson, S.; Green, P. J., On Bayesian analysis of mixtures with an unknown number of components (with discussion), J. Roy. Statist. Soc. Ser. B, 59, 731-792 (1997) · Zbl 0891.62020
[56] Roeder, K., Density estimation with confidence sets exemplified by superclusters and voids in the galaxies, J. Amer. Statist. Assoc., 85, 617-624 (1990) · Zbl 0704.62103
[57] Rolin, J.-M., 1992. Some useful properties of the Dirichlet process. Technical Report 9207, Center for Operations Research and Econometrics, Université Catholique de Louvain.; Rolin, J.-M., 1992. Some useful properties of the Dirichlet process. Technical Report 9207, Center for Operations Research and Econometrics, Université Catholique de Louvain.
[58] Schwarz, G., Estimating the dimension of a model, Ann. Statist., 6, 461-464 (1978) · Zbl 0379.62005
[59] Scott, D., Parametric statistical modeling by minimum integrated square error, Technometrics, 43, 274-285 (2001)
[60] Scott, D. W., Multivariate Density EstimationTheory Practice and Visualization (1992), Wiley: Wiley New York
[61] Sethuraman, J., A constructive definition of Dirichlet priors, Statist. Sinica, 4, 639-650 (1994) · Zbl 0823.62007
[62] Silverman, B. W., Density Estimation for Statistics and Data Analysis (1986), Chapman & Hall/CRC: Chapman & Hall/CRC London · Zbl 0617.62042
[63] Stephens, M., Bayesian analysis of mixture models with an unknown number of components—alternative to reversible jump methods, Ann. Statist., 28, 40-74 (2000) · Zbl 1106.62316
[64] Walker, S.G., Damien, P., 1996. Sampling a Dirichlet process mixture model. Technical Report, Business School, University of Michigan.; Walker, S.G., Damien, P., 1996. Sampling a Dirichlet process mixture model. Technical Report, Business School, University of Michigan.
[65] Yeung, K. Y.; Fraley, C.; Murua, A.; Raftery, A.; Ruzzo, L., Model-based clustering and data transformations for gene expression data, Bioinformatics, 17, 10, 977-987 (2001)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.