×

Numerical computation of mean passage times and absorption probabilities in Markov and semi-Markov models. (English) Zbl 0618.90094

This paper discusses an efficient method to compute mean passage times and absorption probabilities in Markov and semi-Markov models. It uses the state reduction approach introduced by W. Grassmann [J. Oper. Res. Soc. 36, 1041-1050 (1985; Zbl 0568.60089)] for the computation of the stationary distribution of a Markov model. The method is numerically stable and has a simple probabilistic interpretation. It is especially stressed that the natural frame for the state reduction method is rather semi-Markov theory than Markov theory.

MSC:

90C40 Markov and semi-Markov decision processes
60K15 Markov renewal processes, semi-Markov processes

Citations:

Zbl 0568.60089
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Cinlar E (1969) Markov renewal theory. Adv Appl Prob 1:123–187 · Zbl 0212.49601 · doi:10.2307/1426216
[2] Cinlar E (1975) Introduction to stochastic processes. Englewood Cliffs NJ · Zbl 0341.60019
[3] Gaede KW (1977) Zuverlässigkeit, Mathematische Modelle. Carl Hauser Verlag, München · Zbl 0375.60003
[4] Grassmann WK (1985a) The factorization of queueing equations, and their interpretation. J Oper Soc, to appear · Zbl 0568.60089
[5] Grassmann WK (1985b) ThePH X /M/c queue. Dept of Comp Sci University of Saskatchevan, Saskatoon
[6] Grassmann WK (1985c) Solving queueing equations using probability theory. Dpt of Comp Sci University of Saskatchevan, Saskatoon
[7] Grassmann WK, Taksar MI, Heyman DP (1985a) Regenerative analysis and steady state distributions for Markov chains. Op Res 33:1107–1116 · Zbl 0576.60083 · doi:10.1287/opre.33.5.1107
[8] Howard RA (1960) Dynamic programming and Markov processes. Cambridge, Mass · Zbl 0091.16001
[9] Kohlas J (1982) Stochastic methods of Operations Research. Cambridge London · Zbl 0505.90022
[10] Kohlas J (1985) Numerical computation of mean passage times and absorption probabilities in Markov and semi-Markov-models. Institute for Automation and Operations Research, University of Fribourg, Switzerland
[11] Kohlas J (1987) Zuverlässigkeit und Verfügbarkeit: Mathematische Modelle, Methoden und Algorithmen. Teubner, Stuttgart · Zbl 0628.90015
[12] Pyke R (1961a) Markov renewal processes. Definitions and preliminary properties. Ann Math Stat 32:1231–1242 · Zbl 0267.60089 · doi:10.1214/aoms/1177704863
[13] Pyke R (1961b) Markov renewal processes with finitely many states. Ann Math Stat 32:1243–1259 · Zbl 0201.49901 · doi:10.1214/aoms/1177704864
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.