Hostname: page-component-74656f9884-8r8nv Total loading time: 0 Render date: 2026-11-10T05:20:10.249Z Has data issue: false hasContentIssue false
  • English
  • Français

A MARKOV CHAIN CHOICE PROBLEM

Published online by Cambridge University Press:  10 December 2012

Sheldon M. Ross
Sheldon M. Ross*
Affiliation:
Epstein Department of Industrial and Systems Engineering, University of Southern California Los Angeles, CA 90089, USA E-mail: smross@usc.edu
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the 'Save PDF' action button.

Consider two independent Markov chains having states 0, 1, and identical transition probabilities. At each stage one of the chains is observed, and a reward equal to the observed state is earned. Assuming prior probabilities on the initial states of the chains it is shown that the myopic policy that always chooses to observe the chain most likely to be in state 1 stochastically maximizes the sequence of rewards earned in each period.

Information

Type
Research Article
Information
Probability in the Engineering and Informational Sciences , Volume 27 , Issue 1 , January 2013 , pp. 53 - 55
Copyright
Copyright © Cambridge University Press 2013

References

REFERENCES

1.Koole, G., Liu, Z. & Righter, R. (2001). Optimal transmission policies for noisy channels. Operations Research 49(6): 892–899.CrossRefGoogle Scholar
2.Ahmad, S.H.A., Liu, M., Javidi, T., Zhao, Q. & Krishnamachari, B. (2009). Optimality of myopic sensing in multichannel opportunistic access. IEEE Transactions on Information Theory 55(9): 4040–4050.CrossRefGoogle Scholar
3.Zhao, Q., Krishnamachari, B. & Liu, K. (2008). On myopic sensing in multichannel opportunistic access: structure, optimality, and performance. IEEE Transactions Wireless Communications 7(12): 5431–5440.CrossRefGoogle Scholar