×

On the adaptive control of a partially observable binary Markov decision process. (English) Zbl 0712.93063

Advances in computing and control, Sel. Pap. Int. Conf. Adv. Commun. Control Syst., Baton Rouge/LA (USA) 1988, Lect. Notes Control Inf. Sci. 130, 217-228 (1989).
[For the entire collection see Zbl 0708.00019.]
Some initial steps are taken toward the development of a methodology for the optimal adaptive control of markov chains with incomplete state observations and unknown model parameters. One set of problems for which some results are available, when all the parameters are known, are those involving quality control and machine maintenance/replacement. These results are reviewed; in particular, the adaptive control of a problem with simple structure is studied: the two-state binary replacement problem. An enforced certainty equivalent adaptive control algorithm is formulated, with a recursive parameter estimator of the stochastic gradient type. Initial results are given in the direction of using the ODE method of analysis to show convergence and optimality of the combined estimation and control scheme. However, the presence of feedback makes a complete analysis difficult. The principal technical contribution of the paper is the derivation of some (uniformly geometric) ergodic properties for an augmented state process. More recent work by the authors substantially improve on the results of this paper.

MSC:

93E20 Optimal stochastic control
93C40 Adaptive control/observation systems
90B25 Reliability, availability, maintenance, inspection in operations research

Citations:

Zbl 0708.00019