Banks, Jeffrey S.; Sundaram, Rangarajan K. Denumerable-armed bandits. (English) Zbl 0771.90100 Econometrica 60, No. 5, 1071-1096 (1992). It is well known that finite-armed bandit problems with independent arms and discounting are solved by dynamic allocation (or Gittins) index policies. This result is generalized to the case of a countably infinite number of arms presenting necessary and sufficient conditions for the existence of optimal policies. In addition, structural results are proved concerning the “survival” of arms. For stationary bandits (each arm has the same prior) special properties are shown. Possible applications of these bandit models are discussed including job search, matching and voting problems. Reviewer: G.Hübner (Hamburg) Cited in 22 Documents MSC: 90C40 Markov and semi-Markov decision processes 90B35 Deterministic scheduling theory in operations research 91B40 Labor market, contracts (MSC2010) 91B12 Voting theory Keywords:Gittins index; finite-armed bandit problems; independent arms; discounting; dynamic allocation; countably infinite number of arms; existence of optimal policies; stationary bandits; job search; matching; voting PDFBibTeX XMLCite \textit{J. S. Banks} and \textit{R. K. Sundaram}, Econometrica 60, No. 5, 1071--1096 (1992; Zbl 0771.90100) Full Text: DOI Link