Hostname: page-component-7b7567c8c-7pwfn Total loading time: 0 Render date: 2026-11-16T08:41:55.795Z Has data issue: false hasContentIssue false

Extension of Wald's first lemma to Markov processes

Published online by Cambridge University Press:  14 July 2016

George V. Moustakides*
Affiliation:
University of Patras
*
∗ Postal address: Department of Computer Engineering and Informatics, University of Patras, 26 500 Patras, Greece. Email address: moustaki@cti.gr.

Abstract

Let ξ0,ξ1,ξ2,… be a homogeneous Markov process and let S n denote the partial sum S n = θ(ξ1) + … + θ(ξn ), where θ(ξ) is a scalar nonlinearity. If N is a stopping time with 𝔼N < ∞ and the Markov process satisfies certain ergodicity properties, we then show that 𝔼S N = [limn→∞𝔼θ(ξn )]𝔼N + 𝔼ω(ξ0) − 𝔼ω(ξN ). The function ω(ξ) is a well defined scalar nonlinearity directly related to θ(ξ) through a Poisson integral equation, with the characteristic that ω(ξ) becomes zero in the i.i.d. case. Consequently our result constitutes an extension to Wald's first lemma for the case of Markov processes. We also show that, when 𝔼N → ∞, the correction term is negligible as compared to 𝔼N in the sense that 𝔼ω(ξ0) − 𝔼ω(ξN ) = o(𝔼N).

Information

Type
Research Papers
Copyright
Copyright © Applied Probability Trust 1999 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable