Hostname: page-component-7b7567c8c-7pwfn Total loading time: 0 Render date: 2026-11-12T18:57:53.352Z Has data issue: false hasContentIssue false
  • English
  • Français

Stopping Probabilities for Patterns in Markov Chains

Part of:Stochastic processesMarkov processes

Published online by Cambridge University Press:  30 January 2018

Renato Jacob Gava and
Danilo Salotti
Renato Jacob Gava*
Affiliation:
Universidade de São Paulo
Danilo Salotti*
Affiliation:
Fundação Educacional Inaciana Padre Sabóia de Medeiros
*
∗ Postal address: Instituto de Matemática e Estatística, Universidade de São Paulo, Rua do Matão 1010, CEP 05508-090, São Paulo, SP, Brazil. Email address: gava@ime.usp.br
∗∗ Postal address: Centro Universitário da FEI, Av. Humberto de Alencar Castelo Branco 3972, CEP 09850-901, São Bernardo do Campo, SP, Brazil. Email address: dsalotti@fei.edu.br
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the 'Save PDF' action button.

Consider a sequence of Markov-dependent trials where each trial produces a letter of a finite alphabet. Given a collection of patterns, we look at this sequence until one of these patterns appears as a run. We show how the method of gambling teams can be employed to compute the probability that a given pattern is the first pattern to occur.

Keywords

Stopping timewaiting timegambling teammartingaleMarkov chainpatternstopping probability

MSC classification

Primary:60J10: Markov chains (discrete-time Markov processes on discrete state spaces)
Secondary:60G42: Martingales with discrete parameter

Information

Type
Research Article
Information
Journal of Applied Probability , Volume 51 , Issue 1 , March 2014, pp. 287 - 292
Copyright
© Applied Probability Trust 

Footnotes

Research supported by FAPESP fellowship 2012/01432-9.

References

Blom, G. and Thorburn, D. (1982). How many random digits are required until given sequences are obtained? {J. Appl. Prob.} 19, 518–531.CrossRefGoogle Scholar
Chrysaphinou, O. and Papastavridis, S. (1990). The occurrence of sequence patterns in repeated dependent experiments. {Theory Prob. Appl.} 35, 145–152.CrossRefGoogle Scholar
Feller, W. (1971). An Introduction to Probability Theory and Its Applications, Vol. 1, 2nd edn. John Wiley, New York.Google Scholar
Fu, J. S. and Chang, Y. M. (2002). On probability generating functions for waiting time distributions of compound patterns in a sequence of multistate trials. J. Appl. Prob. 39, 70–80.CrossRefGoogle Scholar
Gerber, H. U. and Li, S.-Y. R. (1981). The occurrence of sequence patterns in repeated experiments and hitting times in a Markov chain. Stoch. Process. Appl. 11, 101–1086.CrossRefGoogle Scholar
Glaz, J., Kulldorff, M., Pozdnyakov, V. and Steele, J. M. (2006). Gambling teams and waiting times for patterns in two-state Markov chains. J. Appl. Prob. 43, 127–140.CrossRefGoogle Scholar
Han, Q. and Aki, S. (2000). Waiting time problems in a two-state Markov chain. Ann. Inst. Statist. Math. 52, 778–789.Google Scholar
Li, S.-Y. R. (1980). A martigale approach to the study of occurrence of sequence patterns in repeated experiments. Ann. Prob. 8, 1171–1176.CrossRefGoogle Scholar
Pozdnyakov, V. (2008). On occurrence of patterns in Markov chain: method of gambling teams. Statist. Prob. Lett. {78}, 2762–2767.CrossRefGoogle Scholar
Pozdnyakov, V. and Kulldorff, M. (2006). Waiting times for patterns and a method of gambling teams. Amer. Math. Monthly {113}, 134–143.CrossRefGoogle Scholar
Schwager, S. J. (1983). Run probabilities in sequences of Markov-dependent trials. J. Amer. Statist. Assoc. {78}, 168–180.Google Scholar
Williams, D. (1991). Probability and Martigales. Cambridge University Press.Google Scholar