×

The maximum likelihood prior. (English) Zbl 0927.62023

Summary: Consider an estimate \(\theta^*\) of a parameter \(\theta\) based on repeated observations from a family of densities \(f_\theta\) evaluated by the Kullback-Leibler loss function \(K(\theta,\theta^*)=\int\log(f_\theta/f_{\theta^*})f_\theta\). The maximum likelihood prior density, if it exists, is the density for which the corresponding Bayes estimate is asymptotically negligibly different from the maximum likelihood estimate. The Bayes estimate corresponding to the maximum likelihood prior is identical to maximum likelihood for exponential families of densities. In predicting the next observation, the maximum likelihood prior produces a predictive distribution that is asymptotically at least as close, in expected truncated Kullback-Leibler distance, to the true density as the density indexed by the maximum likelihood estimate.
It frequently happens in more than one dimension that maximum likelihood corresponds to no prior density, and in that case the maximum likelihood estimate is asymptotically inadmissible and may be improved upon by using the estimate corresponding to a least favorable prior. As in L. D. Brown, Ann. Stat. 7, 960-994 (1979; Zbl 0414.62011), Ann. Math. Stat. 42, 855-903 (1971; Zbl 0246.62016), the asymptotic risk for an arbitrary estimate “near” maximum likelihood is given by an expression involving derivatives of the estimator and of the information matrix. Admissibility questions for these “near ML” estimates are determined by the existence of solutions to certain differential equations.

MSC:

62F15 Bayesian inference
62C15 Admissibility in statistical decision theory
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] AITCHISON, J. 1975. Goodness of prediction. Biometrika 62 547 554. Z. JSTOR: · Zbl 0339.62018 · doi:10.1093/biomet/62.3.547
[2] AMARI, S. 1982. Differential geometry of curved exponential families: curvatures and information loss. Ann. Statist. 10 357 387. Z. Z. · Zbl 0507.62026 · doi:10.1214/aos/1176345779
[3] BERNARDO, J. M. 1979. Reference posterior densities for Bayesian inference with discussion. J. Roy. Statist. Soc. Ser. B 41 113 147. Z. JSTOR: · Zbl 0428.62004
[4] BERGER, J. O. and BERNARDO, J. M. 1992. On the development of the reference prior method. In Z Bayesian Statistics 4. Proceedings of the Fourth Valencia International Meeting J. M.. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds. 35 60. Clarendon Press, Oxford. Z.
[5] BROWN, L. D. 1971. Admissible estimators, recurrent diffusions, and insoluble boundary value problems. Ann. Math. Statist. 42 855 903. Z. · Zbl 0246.62016 · doi:10.1214/aoms/1177693318
[6] BROWN, L. D. 1979. A heuristic method for determining admissibility of estimators with applications. Ann. Statist. 7 960 994. Z. · Zbl 0414.62011 · doi:10.1214/aos/1176344782
[7] CLARKE, B. and BARRON, A. 1994. Jeffrey s prior is asy mptotically least favourable under entropy risk. J. Statist. Plann. Inference 41 37 60. Z. · Zbl 0820.62006 · doi:10.1016/0378-3758(94)90153-8
[8] GHOSH, J. K. 1994. Higher Order Asy mptotics. IMS, Hay ward, CA. Z.
[9] GHOSH, J. K. and SUBRAMANy AM, K. 1974. Second-order efficiency of maximum likelihood estimators. Sankhy a Ser. A 36 325 358. Z. · Zbl 0321.62035
[10] HARTIGAN, J. A. 1964. Invariant prior densities. Ann. Math. Statist. 35 836 845. Z. · Zbl 0151.23003 · doi:10.1214/aoms/1177703583
[11] HARTIGAN, J. A. 1965. The asy mptotically unbiased density. Ann. Math. Statist. 36 1137 1152. Z. · Zbl 0133.42106 · doi:10.1214/aoms/1177699988
[12] HARTIGAN, J. A. 1983. Bay es Theory. Springer, New York. Z. · Zbl 0537.62007
[13] JEFFREy S, H. 1946. An invariant form of the prior probability in estimation problems. Proc. Roy. Soc. London Ser. A 186 453 461. Z. · Zbl 0063.03050 · doi:10.1098/rspa.1946.0056
[14] JEFFREy S, H. 1961. Theory of Probability. Oxford Univ. Press. Z.
[15] JOHNSON, R. A. 1967. An asy mptotic expansion for posterior distributions. Ann. Math. Statist. 38 1899 1907. Z. · Zbl 0157.46802 · doi:10.1214/aoms/1177698624
[16] KOMAKI, F. 1996. On asy mptotic properties of predictive distributions. Biometrika 83 229 313. Z. JSTOR: · Zbl 0864.62007 · doi:10.1093/biomet/83.2.299
[17] LEVIT, B. YA. 1982. Minimax estimation and positive solutions of elliptic equations. Theory Probab. Appl. 27 563 586. Z. · Zbl 0518.62031 · doi:10.1137/1127063
[18] LEVIT, B. YA. 1983. Second-order availability and positive solutions of the Schrodinger equaẗion. Lecture Notes in Math. 1021 372 385. Springer, Berlin. Z. · Zbl 0585.62061
[19] LEVIT, B. YA. 1985. Second-order asy mptotic optimality and positive solutions of Schrodinger’s ëquation. Theory Probab. Appl. 30 333 363. Z. · Zbl 0608.62035 · doi:10.1137/1130039
[20] MCCULLAGH, P. 1987. Tensor Methods in Statistics. Chapman Hall, London. Z. · Zbl 0732.62003
[21] PERKS, W. 1947. Some observations on inverse probability, including a new indifference rule. J. Inst. Actuaries 73 285 334.Z. · Zbl 0031.06001
[22] PFANZAGL, J. and WEFELMEy ER, W. 1978. A third-order optimal property of the maximum likelihood estimator. J. Multivariate Anal. 8 1 29. Z. · Zbl 0376.62019
[23] RAO, C. R. 1961. Asy mptotic efficiency and limiting information. Proc. Fourth Berkeley Sy mp. Math. Statist. Probab. 1 531 546. Univ. California Press, Berkeley. · Zbl 0156.39802
[24] STEIN, C. 1956. Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. Proc. Third Berkeley Sy mp. Math. Statist. Probab. 1 197 206. Univ. California Press, Berkeley. Z. · Zbl 0073.35602
[25] STRASSEN, H. 1977. Asy mptotic expansions for Bay es procedures. In Recent Developments in Z. Statistics Barra, J. L., eds. 9 35. North-Holland, Amsterdam. Z.
[26] TAy LOR, M. E. 1997. Partial Differential Equations III. Nonlinear Equations. Springer, New York. Z.
[27] WELCH, B. L. and PEERS, H. W. 1963. On formulae for confidence points based on integrals of weighted likelihoods. J. Roy. Statist. Soc. Ser. B 25 318 329. JSTOR: · Zbl 0117.14205
[28] NEW HAVEN, CONNECTICUT 06520
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.