Hostname: page-component-74656f9884-8r8nv Total loading time: 0 Render date: 2026-11-10T06:18:50.820Z Has data issue: false hasContentIssue false
  • English
  • Français

Partially observable semi-Markov reward processes

Part of: Special processes Operations research and management science

Published online by Cambridge University Press:  14 July 2016

Yasushi Masuda
Yasushi Masuda*
Affiliation:
University of California, Riverside
*
∗ Postal address: Graduate School of Management, University of California, Riverside, CA 92521, USA.
Get access Rights & Permissions [Opens in a new window]

Abstract

The main objective of this paper is to investigate the conditional behavior of the multivariate reward process given the number of certain signals where the underlying system is described by a semi-Markov process and the signal is defined by a counting process. To this end, we study the joint behavior of the multivariate reward process and the multivariate counting process in detail. We derive transform results as well as the corresponding real domain expressions, thus providing clear probabilistic interpretation.

Keywords

COUNTING PROCESSMULTIVARIATE PROCESS

MSC classification

Primary: 60K20: Applications of Markov renewal processes (reliability, queueing networks, etc.)
Secondary: 90B25: Reliability, availability, maintenance, inspection

Information

Type
Research Papers
Information
Journal of Applied Probability , Volume 30 , Issue 3 , September 1993 , pp. 548 - 560
Copyright
Copyright © Applied Probability Trust 1993 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

ÇlInlar, E. (1969) Markov renewal theory. Adv. Appl. Prob. 1, 123–187.Google Scholar
Csenki, A. (1991) The joint distribution of sojourn times in finite semi-Markov processes. Stoch. Proc. Appl. 39, 287–299.Google Scholar
Howard, R. (1971) Dynamic Probabilistic Systems, Vol. 2: Semi-Markov and Decision Analysis. Wiley, New York.Google Scholar
Janssen, J., (ed.) (1986) International Symposium on Semi-Markov Processes and Their Applications. Semi-Markov Models: Theory and Applications. Plenum Press, New York.Google Scholar
Keilson, J. (1969) On the matrix renewal function for Markov renewal processes. Ann. Math. Statist. 40, 1901–1907.Google Scholar
Keilson, J. and Nunn, W. R. (1979) Laguerre transform as a tool for the numerical solution of integral equation of convolution type. Appl. Math. Comput. 5, 313–359.Google Scholar
Kulkarni, V. G. (1989) A new class of multivariate phase type distributions. Operat. Res. 37, 151–158.Google Scholar
Kulkarni, V. G., Nicola, V. F. and Trivedi, K. S. (1987) The completion time of a job on multimode systems. Adv. Appl. Prob. 19, 932–954.Google Scholar
Masuda, Y. and Sumita, U. (1987) Analysis of counting process associated with semi-Markov process: number of entries into a subset of state space. Adv. Appl. Prob. 19, 767–783.Google Scholar
Masuda, Y. and Sumita, U. (1991) A multivariate reward process defined on a semi-Markov process and its first-passage-time distributions. J. Appl. Prob. 28, 360–373.Google Scholar
Smith, W. L. (1955) Regenerative stochastic processes. Proc. R. Society London A 232, 6–31.Google Scholar
Sumita, U. (1984) Matrix Laguerre transform. Appl. Math. Comput. 15, 1–28.Google Scholar