Hostname: page-component-7b7567c8c-tnkh4 Total loading time: 0 Render date: 2026-11-16T11:21:21.954Z Has data issue: false hasContentIssue false

ON THE IDENTIFICATION AND MITIGATION OF WEAKNESSES IN THE KNOWLEDGE GRADIENT POLICY FOR MULTI-ARMED BANDITS

Published online by Cambridge University Press:  13 September 2016

James Edwards
Affiliation:
STOR-i Centre for Doctoral Training, Lancaster University Lancaster LA1 4YF, UK E-mail: j.edwards4@lancaster.ac.uk
Paul Fearnhead
Affiliation:
Department of Mathematics and Statistics, Lancaster University, Lancaster LA1 4YF, UK E-mail: p.fearnhead@lancaster.ac.uk
Kevin Glazebrook
Affiliation:
Department of Management Science, Lancaster University, Lancaster LA1 4YX, UK E-mail: k.glazebrook@lancaster.ac.uk
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the 'Save PDF' action button.

The knowledge gradient (KG) policy was originally proposed for online ranking and selection problems but has recently been adapted for use in online decision-making in general and multi-armed bandit problems (MABs) in particular. We study its use in a class of exponential family MABs and identify weaknesses, including a propensity to take actions which are dominated with respect to both exploitation and exploration. We propose variants of KG which avoid such errors. These new policies include an index heuristic, which deploys a KG approach to develop an approximation to the Gittins index. A numerical study shows this policy to perform well over a range of MABs including those for which index policies are not optimal. While KG does not take dominated actions when bandits are Gaussian, it fails to be index consistent and appears not to enjoy a performance advantage over competitor policies when arms are correlated to compensate for its greater computational demands.

Information

Type
Research Article
Copyright
Copyright © Cambridge University Press 2016