In this article, we introduce Mallows processes, defined to be continuous-time cadlag processes with Mallows distributed marginals. We show that such processes exist and that they can be restricted to have certain natural properties. In particular, we prove that there exist regular Mallows processes, defined to have their inversion numbers ${\mathrm{Inv}}_j(\sigma )=|\{i\in [j-1] \, : \, \sigma (i)\gt \sigma (\,j)\}|$ be independent increasing stochastic processes with jumps of size $1$. We further show that there exists a unique Markov process which is a regular Mallows process. Finally, we study properties of regular Mallows processes and show various results on the structure of these objects. Among others, we prove that the graph structure related to regular Mallows processes looks like an expanded hypercube where we stacked $k$ hypercubes on the dimension $k\in [n]$; we also prove that the first jumping times of regular Mallows processes converge to a Poisson point process; we eventually provide results regarding the Markov property of the discrete jumping chain corresponding to a Mallows process.

Recent work on random growth models with light-tailed Markov-modulated additive shocks has shown that irreducible modulation yields tail behavior resembling an exponential distribution. We show that with reducible modulation the tail behavior more generally resembles an Erlang distribution. Our main technical contribution is a theorem on the order of a real pole of the inverse of a holomorphic matrix-valued function with reducible Metzler structure. In a special affine case, the theorem recovers the Rothblum index theorem. Applying this result together with a Tauberian theorem, we characterize the Erlang shape parameter in two models of Markov-modulated random growth.

Fluid queues governed by birth–death processes have been used to analyze buffer dynamics and stability behavior of the fluid flow systems. However, most existing studies primarily focus on classical single-ended queues, often ignore double-ended queue flow dynamics, or rely heavily on simulation-based approaches. Specifically, the study of the fluid flow systems modulated by double-ended queues subject to the catastrophic failure and subsequent repair processes is challenging and interesting and has not yet received attention in the literature. Even when such systems are considered, explicit closed-form analytic expressions for equilibrium buffer content distributions and related performance measures are rarely available. To overcome these limitations, this article investigates a fluid flow system regulated by a double-ended queue and exposed to catastrophic failures with subsequent repair processes. Such a driven queue can be equivalently represented as a one-dimensional bilateral birth–death process, namely a continuous-time randomized random walk on the integers with catastrophic failures and repairs. The stability condition for the fluid occupancy in the credit buffer is rigorously established, and explicit closed-form analytical expressions for both the probability density function and the cumulative distribution function of the buffer content in the equilibrium regime are determined. These analytic results provide deeper insight into the steady-state behavior of the system and enable the derivation of several vital performance measures of practical interest. Furthermore, graphical illustrations are presented to highlight the influence of the system parameters on the performance descriptors of the fluid content, thereby enhancing the interpretability and applicability of the proposed fluid queueing system.

In this paper we propose a new efficient algorithm to compute the value function for zero-sum stopping games featuring two players with opposing interests. This can be seen as a game version of the ‘forward algorithm’ for (one-player) optimal stopping problems, first introduced by Irle (2006) for discrete-time Markov chains and later revisited by Miclo and Villeneuve (2021) for continuous-time Markov processes on general state spaces. This paper focuses on a game driven by a homogeneous continuous-time Markov chain taking values in a finite state space and also discusses the number of iterations needed. Illustrated computational implementations for a few particular examples are also provided.

This paper introduces a novel expectation-maximization (EM) algorithm for estimating general phase-type (PH) distributions from left-truncated and right-censored (LTRC) data, a common challenge in survival analysis. The proposed algorithm is highly efficient with computational complexity that scales with the number of nonzero elements in the generator matrix. This feature makes the estimation of high-dimensional, sparse PH models computationally tractable and enables the practical use of the computationally intensive extended information criterion for model selection. Numerical experiments demonstrate its significant speed advantage over a modern benchmark and the applicability of PH models to complex lifetime data.

Recently, there has been much progress in understanding stationary measures for colored (also called multi-species or multi-type) interacting particle systems, motivated by asymptotic phenomena and rich underlying algebraic and combinatorial structures (such as nonsymmetric Macdonald polynomials). In this paper, we present a unified approach to constructing stationary measures for most of the known colored particle systems on the ring and the line, including (1) the Asymmetric Simple Exclusion Process (multi-species ASEP, or mASEP); (2) the $q$-deformed Totally Asymmetric Zero Range Process (TAZRP) also known as the $q$-Boson particle system; (3) the $q$-deformed Pushing Totally Asymmetric Simple Exclusion Process ($q$-PushTASEP). Our method is based on integrable stochastic vertex models and the Yang–Baxter equation. We express the stationary measures as partition functions of new ‘queue vertex models’ on the cylinder. The stationarity property is a direct consequence of the Yang–Baxter equation. For the mASEP on the ring, a particular case of our vertex model is equivalent to the multiline queues of Martin (Stationary distributions of the multi-type ASEP, Electron. J. Probab. 25 (2020), 1–41). For the colored $q$-Boson process and the $q$-PushTASEP on the ring, we recover and generalize known stationary measures constructed using multiline queues or other methods by Ayyer, Mandelshtam and Martin (Modified Macdonald polynomials and the multispecies zero range process: II, Algebr. Comb. 6 (2022), 243–284; Modified Macdonald polynomials and the multispecies zero-range process: I, Algebr. Comb. 6 (2023), 243–284) and Bukh and Cox (Periodic words, common subsequences and frogs, Ann. Appl. Probab. 32 (2022), 1295–1332). Our proofs of stationarity use the Yang–Baxter equation and bypass the Matrix Product Ansatz (used for the mASEP by Prolhac, Evans and Mallick (The matrix product solution of the multispecies partially asymmetric exclusion process, J. Phys. A. 42 (2009), 165004)). On the line and in a quadrant, we use the Yang–Baxter equation to establish a general colored Burke’s theorem, which implies that suitable specializations of our queue vertex models produce stationary measures for particle systems on the line. We also compute the colored particle currents in stationarity.

We study a two-dimensional discounted optimal stopping zero-sum (or Dynkin) game related to perpetual redeemable convertible bonds expressed as game (or Israeli) options in a model of financial markets in which the behaviour of the ex-dividend price of a dividend-paying asset follows a generalized geometric Brownian motion. It is assumed that the dynamics of the random dividend rate of the asset paid to shareholders are described by the mean-reverting filtering estimate of an unobservable continuous-time Markov chain with two states. It is shown that the optimal exercise (conversion) and withdrawal (redemption) times forming a Nash equilibrium are the first times at which the asset price hits either lower or upper stochastic boundaries being monotone functions of the running value of the filtering estimate of the state of the chain. We rigorously prove that the optimal stopping boundaries are regular for the stopping region relative to the resulting two-dimensional diffusion process and that the value function is continuously differentiable with respect to the both variables. It is verified by means of a change-of-variable formula with local time on surfaces that the optimal stopping boundaries are determined as a unique solution to the associated coupled system of nonlinear Fredholm integral equations among the couples of continuous functions of bounded variation satisfying certain conditions. We also give a closed-form solution to the appropriate optimal stopping zero-sum game in the corresponding model with an observable continuous-time Markov chain.

We study a variant of the classical Markovian logistic SIS epidemic model on a complete graph, which has the additional feature that healthy individuals can become infected without contacting an infected member of the population. This additional ‘self-infection’ is used to model situations where there is an unknown source of infection or an external disease reservoir, such as an animal carrier population. In contrast to the classical logistic SIS epidemic model, the version with self-infection has a non-degenerate stationary distribution, and we derive precise asymptotics for the time to converge to stationarity (mixing time) as the population size becomes large. It turns out that the chain exhibits the cutoff phenomenon, which is a sharp transition in time from one to zero of the total variation distance to stationarity. We obtain the exact leading constant for the cutoff time and show that the window size is of constant (optimal) order. While this result is interesting in its own right, an additional contribution of this work is that the proof illustrates a recently formalised methodology of Barbour, Brightwell and Luczak (2022), ‘Long-term concentration of measure and cut-off’, Stochastic Processes and their Applications 152, 378–423, which can be used to show cutoff via a combination of concentration-of-measure inequalities for the trajectory of the chain and coupling techniques.

We study continuous-time Markov chains on the nonnegative integers under mild regularity conditions (in particular, the set of jump vectors is finite and both forward and backward jumps are possible). Based on the so-called flux balance equation, we derive an iterative formula for calculating stationary measures. Specifically, a stationary measure $\pi(x)$ evaluated at $x\in\mathbb{N}_0$ is represented as a linear combination of a few generating terms, similarly to the characterization of a stationary measure of a birth–death process, where there is only one generating term, $\pi(0)$. The coefficients of the linear combination are recursively determined in terms of the transition rates of the Markov chain. For the class of Markov chains we consider, there is always at least one stationary measure (up to a scaling constant). We give various results pertaining to uniqueness and nonuniqueness of stationary measures, and show that the dimension of the linear space of signed invariant measures is at most the number of generating terms. A minimization problem is constructed in order to compute stationary measures numerically. Moreover, a heuristic linear approximation scheme is suggested for the same purpose by first approximating the generating terms. The correctness of the linear approximation scheme is justified in some special cases. Furthermore, a decomposition of the state space into different types of states (open and closed irreducible classes, and trapping, escaping and neutral states) is presented. Applications to stochastic reaction networks are well illustrated.

This paper characterizes irreducible phase-type representations for exponential distributions. Bean and Green (2000) gave a set of necessary and sufficient conditions for a phase-type distribution with an irreducible generator matrix to be exponential. We extend these conditions to irreducible representations, and we thus give a characterization of all irreducible phase-type representations for exponential distributions. We consider the results in relation to time-reversal of phase-type distributions, PH-simplicity, and the algebraic degree of a phase-type distribution, and we give applications of the results. In particular we give the conditions under which a Coxian distribution becomes exponential, and we construct bivariate exponential distributions. Finally, we translate the main findings to the discrete case of geometric distributions.

For a continuous-time phase-type (PH) distribution, starting with its Laplace–Stieltjes transform, we obtain a necessary and sufficient condition for its minimal PH representation to have the same order as its algebraic degree. To facilitate finding this minimal representation, we transform this condition equivalently into a non-convex optimization problem, which can be effectively addressed using an alternating minimization algorithm. The algorithm convergence is also proved. Moreover, the method we develop for the continuous-time PH distributions can be used directly for the discrete-time PH distributions after establishing an equivalence between the minimal representation problems for continuous-time and discrete-time PH distributions.

We consider time-inhomogeneous ordinary differential equations (ODEs) whose parameters are governed by an underlying ergodic Markov process. When this underlying process is accelerated by a factor $\varepsilon^{-1}$, an averaging phenomenon occurs and the solution of the ODE converges to a deterministic ODE as $\varepsilon$ vanishes. We are interested in cases where this averaged flow is globally attracted to a point. In that case, the equilibrium distribution of the solution of the ODE converges to a Dirac mass at this point. We prove an asymptotic expansion in terms of $\varepsilon$ for this convergence, with a somewhat explicit formula for the first-order term. The results are applied in three contexts: linear Markov-modulated ODEs, randomized splitting schemes, and Lotka–Volterra models in a random environment. In particular, as a corollary, we prove the existence of two matrices whose convex combinations are all stable but are such that, for a suitable jump rate, the top Lyapunov exponent of a Markov-modulated linear ODE switching between these two matrices is positive.

The problem of reservation in a large distributed system is analyzed via a new mathematical model. The target application is car-sharing systems. This model is motivated by the large station-based car-sharing system in France called Autolib’. This system can be described as a closed stochastic network where the nodes are the stations and the customers are the cars. The user can reserve a car and a parking space. We study the evolution of the system when the reservation of parking spaces and cars is effective for all users. The asymptotic behavior of the underlying stochastic network is given when the number N of stations and the fleet size M increase at the same rate. The analysis involves a Markov process on a state space with dimension of order $N^2$. It is quite remarkable that the state process describing the evolution of the stations, whose dimension is of order N, converges in distribution, although not Markov, to a non-homogeneous Markov process. We prove this mean-field convergence. We also prove, using combinatorial arguments, that the mean-field limit has a unique equilibrium measure when the time between reserving and picking up the car is sufficiently small. This result extends the case where only the parking space can be reserved.

The embedding problem of Markov chains examines whether a stochastic matrix$\mathbf{P} $ can arise as the transition matrix from time 0 to time 1 of a continuous-time Markov chain. When the chain is homogeneous, it checks if $ \mathbf{P}=\exp{\mathbf{Q}}$ for a rate matrix $ \mathbf{Q}$ with zero row sums and non-negative off-diagonal elements, called a Markov generator. It is known that a Markov generator may not always exist or be unique. This paper addresses finding $ \mathbf{Q}$, assuming that the process has at most one jump per unit time interval, and focuses on the problem of aligning the conditional one-jump transition matrix from time 0 to time 1 with $ \mathbf{P}$. We derive a formula for this matrix in terms of $ \mathbf{Q}$ and establish that for any $ \mathbf{P}$ with non-zero diagonal entries, a unique $ \mathbf{Q}$, called the ${\unicode{x1D7D9}}$-generator, exists. We compare the ${\unicode{x1D7D9}}$-generator with the one-jump rate matrix from Jarrow, Lando, and Turnbull (1997), showing which is a better approximate Markov generator of $ \mathbf{P}$ in some practical cases.

Continuous-time Markov chains are frequently used to model the stochastic dynamics of (bio)chemical reaction networks. However, except in very special cases, they cannot be analyzed exactly. Additionally, simulation can be computationally intensive. An approach to address these challenges is to consider a more tractable diffusion approximation. Leite and Williams (Ann. Appl. Prob. 29, 2019) proposed a reflected diffusion as an approximation for (bio)chemical reaction networks, which they called the constrained Langevin approximation (CLA) as it extends the usual Langevin approximation beyond the first time some chemical species becomes zero in number. Further explanation and examples of the CLA can be found in Anderson et al. (SIAM Multiscale Modeling Simul. 17, 2019).

In this paper, we extend the approximation of Leite and Williams to (nearly) density-dependent Markov chains, as a first step to obtaining error estimates for the CLA when the diffusion state space is one-dimensional, and we provide a bound for the error in a strong approximation. We discuss some applications for chemical reaction networks and epidemic models, and illustrate these with examples. Our method of proof is designed to generalize to higher dimensions, provided there is a Lipschitz Skorokhod map defining the reflected diffusion process. The existence of such a Lipschitz map is an open problem in dimensions more than one.

By the technique of augmented truncations, we obtain the perturbation bounds on the distance of the finite-time state distributions of two continuous-time Markov chains (CTMCs) in a type of weaker norm than the V-norm. We derive the estimates for strongly and exponentially ergodic CTMCs. In particular, we apply these results to get the bounds for CTMCs satisfying Doeblin or stochastically monotone conditions. Some examples are presented to illustrate the limitation of the V-norm in perturbation analysis and to show the quality of the weak norm.

Birth–death processes form a natural class where ideas and results on large deviations can be tested. We derive a large-deviation principle under an assumption that the rate of jump down (death) grows asymptotically linearly with the population size, while the rate of jump up (birth) grows sublinearly. We establish a large-deviation principle under various forms of scaling of the underlying process and the corresponding normalization of the logarithm of the large-deviation probabilities. The results show interesting features of dependence of the rate functional upon the parameters of the process and the forms of scaling and normalization.

This paper investigates tail asymptotics of stationary distributions and quasi-stationary distributions (QSDs) of continuous-time Markov chains on subsets of the non-negative integers. Based on the so-called flux-balance equation, we establish identities for stationary measures and QSDs, which we use to derive tail asymptotics. In particular, for continuous-time Markov chains with asymptotic power law transition rates, tail asymptotics for stationary distributions and QSDs are classified into three types using three easily computable parameters: (i) super-exponential distributions, (ii) exponential-tailed distributions, and (iii) sub-exponential distributions. Our approach to establish tail asymptotics of stationary distributions is different from the classical semimartingale approach, and we do not impose ergodicity or moment bound conditions. In particular, the results also hold for explosive Markov chains, for which multiple stationary distributions may exist. Furthermore, our results on tail asymptotics of QSDs seem new. We apply our results to biochemical reaction networks, a general single-cell stochastic gene expression model, an extended class of branching processes, and stochastic population processes with bursty reproduction, none of which are birth–death processes. Our approach, together with the identities, easily extends to discrete-time Markov chains.

A comparison theorem for state-dependent regime-switching diffusion processes is established, which enables us to pathwise-control the evolution of the state-dependent switching component simply by Markov chains. Moreover, a sharp estimate on the stability of Markovian regime-switching processes under the perturbation of transition rate matrices is provided. Our approach is based on elaborate constructions of switching processes in the spirit of Skorokhod’s representation theorem varying according to the problem being dealt with. In particular, this method can cope with switching processes in an infinite state space and not necessarily of birth–death type. As an application, some known results on the ergodicity and stability of state-dependent regime-switching processes can be improved.

We consider an SIR (susceptible $\to$ infective $\to$ recovered) epidemic in a closed population of size n, in which infection spreads via mixing events, comprising individuals chosen uniformly at random from the population, which occur at the points of a Poisson process. This contrasts sharply with most epidemic models, in which infection is spread purely by pairwise interaction. A sequence of epidemic processes, indexed by n, and an approximating branching process are constructed on a common probability space via embedded random walks. We show that under suitable conditions the process of infectives in the epidemic process converges almost surely to the branching process. This leads to a threshold theorem for the epidemic process, where a major outbreak is defined as one that infects at least $\log n$ individuals. We show further that there exists $\delta \gt 0$, depending on the model parameters, such that the probability that a major outbreak has size at least $\delta n$ tends to one as $n \to \infty$.