×

Asymptotically efficient recursive estimation for incomplete data models using the observed information. (English) Zbl 1093.62523

Summary: For a recursive maximum-likelihood estimator with step lengths decaying as \(1/n\), an adaptive matrix needs to be incorporated to obtain asymptotic efficiency. Ideally, this matrix should be chosen as the inverse Fisher information matrix, which is usually very difficult to compute for incomplete data models. In this paper we give conditions under which the observed information can be incorporated into the recursive procedure to yield an efficient estimator, and we also investigate the finite sample properties of these estimators by simulation.

MSC:

62F10 Point estimation
62F12 Asymptotic properties of parametric estimators
PDFBibTeX XMLCite
Full Text: DOI EuDML

References:

[1] Behboodian, J., Information matrix for a mixture of two normal distributions, J. Statist. Comput. Simul., 1, 295-314 (1972) · Zbl 0247.62012 · doi:10.1080/00949657208810024
[2] Csörgö, M.; Fischler, R., Some examples and results in the theory of mixing and random-sum central limit theorems, Period. Math. Hungar., 3, 41-57 (1973) · Zbl 0269.60017 · doi:10.1007/BF02018460
[3] Dattatreya, GR; Kanal, LN, Estimation of mixing probabilities in multiclass finite mixtures, IEEE Trans. Syst. Man Cybern., 20, 149-158 (1990) · Zbl 0692.62031 · doi:10.1109/21.47817
[4] Dattatreya, GR; Kanal, LN, Asymptotically efficient estimation of prior probabilities in multiclass finite mixtures, IEEE Trans. Inform. Theory, 37, 482-489 (1991) · Zbl 0738.62066 · doi:10.1109/18.79904
[5] Dempster, AP; Laird, NM; Rubin, DB, Maximum likelihood from incomplete data via the EM algorithm, J. Royal Statist. Soc., 39, 1-22 (1977) · Zbl 0364.62022
[6] Dubins, LE; Freedman, DA, A sharper form of the Borel-Cantelli lemma and the strong law, Ann. Math. Statist., 36, 800-807 (1965) · Zbl 0168.16901
[7] Fabian, V., On asymptotically efficient recursive estimation, Ann. Statist., 6, 854-866 (1978) · Zbl 0378.62031
[8] Fabian, V., The local asymptotic minimax adaptive property of a recursive estimate, Stat. Prob. Letters, 6, 383-388 (1988) · Zbl 0663.62039 · doi:10.1016/0167-7152(88)90096-X
[9] Hall, P.; Heyde, CC, Martingale limit theory and its applications (1980), New York: Academic Press, New York · Zbl 0462.60045
[10] Hall, P.; Patil, P., On the efficiency of on-line density estimators, IEEE Trans. Inform. Theory, 40, 1504-1512 (1994) · Zbl 0809.62030 · doi:10.1109/18.333864
[11] Holst, U., Recursive estimation of quantiles using recursive kernel density estimators, Seq. Anal., 6, 219-237 (1987) · Zbl 0626.62083 · doi:10.1080/07474948708836128
[12] Holst, U.; Lindgren, G., Recursive estimation in mixture models with Markov regime, IEEE Trans. Inform. Theory, 37, 1683-1690 (1991) · Zbl 0739.62068 · doi:10.1109/18.104334
[13] Kazakos, D., Recursive estimation of prior probabilities using a mixture, IEEE Trans. Inform. Theory, 23, 203-211 (1977) · Zbl 0382.93058 · doi:10.1109/TIT.1977.1055693
[14] Kushner, HJ; Clark, DS, Stochastic approximation methods for constrained and unconstrained systems (1978), New York: Springer-Verlag, New York · Zbl 0381.60004
[15] Ljung, L., Strong convergence of a stochastic approximation algorithm, Ann. Statist., 6, 680-696 (1978) · Zbl 0402.62060
[16] Ljung, L.; Söderström, T., Theory and practice of recursive identification (1983), Cambridge: MIT Press, Cambridge · Zbl 0548.93075
[17] Major, P.; Révész, P., A limit theorem for the Robbins-Monro approximation, Z. Wahrsch. verw. Geb., 27, 79-87 (1973) · Zbl 0253.62046 · doi:10.1007/BF00736010
[18] Polyak, BT; Juditsky, AB, Acceleration of stochastic approximation by averaging, SIAM J. Control Optim., 30, 838-855 (1992) · Zbl 0762.62022 · doi:10.1137/0330046
[19] Ruppert, D., Almost sure approximations to the Robbins-Monro and Kiefer-Wolfowitz processes with dependent noise, Ann. Prob., 10, 178-187 (1982) · Zbl 0485.62083
[20] Ruppert, D.; Ghosh, BK; Sen, PK, Stochastic approximation, Handbook of sequential analysis (1991), New York: Marcel Dekker, New York · Zbl 0753.62046
[21] Schwabe, R., Strong representation of an adaptive stochastic approximation procedure, Stoch. Proc. Appl., 23, 115-130 (1986) · Zbl 0614.62107 · doi:10.1016/0304-4149(86)90019-0
[22] Stout, WF, Almost sure convergence (1974), New York: Academic Press, New York · Zbl 0321.60022
[23] Titterington, DM, Recursive parameter estimation using incomplete data, J. Royal Statist. Soc. B, 46, 257-267 (1984) · Zbl 0556.62061
[24] Weinstein, E.; Feder, M.; Oppenheim, AV, Sequential algorithms for parameter estimation based on the Kullback-Leibler information measure, IEEE Trans. Acoust. Speech Signal Process, 38, 1652-1654 (1990) · Zbl 0699.62079 · doi:10.1109/29.60089
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.