Advanced Search

Query:

Fill in the form and click »Search«...

Format:

Display: entries per page entries

Zbl 1117.60073
Heidergott, B.; Vázquez-Abad, F.J.
Measure-valued differentiation for random horizon problems. (English)
[J] Markov Process. Relat. Fields 12, No. 3, 509-536 (2006). ISSN 1024-2953

Summary: This paper deals with sensitivity analysis (gradient estimation) of random horizon cost functions of Markov chains. More precisely, we consider general state-space Markov chains and the random horizon is given through a hitting time of the chain onto a predefined set. The cost of interest is an expectation of a functional of the stopped process. This encompasses a wide range of models, such as the Gambler's ruin problem and performance evaluation for stationary queueing networks. We work within the framework of measure-valued differentiation and provide a general condition under which the gradient of the random horizon performance can be obtained in a closed form expression. For several scenarios, which occur typically in applications, we subsequently provide sufficient conditions for our general condition to hold. We illustrate our results with a series of examples. Eventually, we discuss unbiased sensitivity estimators and establish a new unbiased estimator for the gradient of stationary Markov chains.

MSC 2000:: *60J27 Markov chains with continuous parameter
90C31 Sensitivity, etc.

Keywords: weak derivatives; gradient estimation; optimization; Markov chains

Comment on this Item PDF MathML XML ASCII DVI PS BibTeX Online Ordering Journal

	Scientific prize winners of the ICM 2010
	Overhang
	Lie groups, physics and geometry. An introduction for physicists, engineers and chemists.

Advanced Search

Zentralblatt MATH Berlin [Germany]