We study stationary distributions in the context of stochastic reaction networks. In particular, we are interested in complex balanced reaction networks and the reduction of such networks by assuming that a set of species (called non-interacting species) are degraded fast (and therefore essentially absent from the network), implying that some reaction rates are large relative to others. Technically, we assume that these reaction rates are scaled by a common parameter N and let $N\to\infty$. The limiting stationary distribution as $N\to\infty$ is compared with the stationary distribution of the reduced reaction network obtained by elimination of the non-interacting species. In general, the limiting stationary distribution could differ from the stationary distribution of the reduced reaction network. We identify various sufficient conditions under which these two distributions are the same, including when the reaction network is detailed balanced and when the set of non-interacting species consists of intermediate species. In the latter case, the limiting stationary distribution essentially retains the form of the complex balanced distribution. This finding is particularly surprising given that the reduced reaction network could be non-weakly reversible and might exhibit unconventional kinetics.

We study Langevin-type algorithms for sampling from Gibbs distributions such that the potentials are dissipative and their weak gradients have finite moduli of continuity not necessarily convergent to zero. Our main result is a non-asymptotic upper bound on the 2-Wasserstein distance between a Gibbs distribution and the law of general Langevin-type algorithms based on a Liptser–Shiryaev-type condition for change of measures and Poincaré inequalities. We apply this bound to show that the Langevin Monte Carlo algorithm can approximate Gibbs distributions with arbitrary accuracy if the potentials are dissipative and their gradients are uniformly continuous. We also propose Langevin-type algorithms with spherical smoothing for distributions whose potentials are not convex or continuously differentiable and show their polynomial complexities.

We study quasi-stationary distributions and quasi-limiting behaviour of Markov chains in general reducible state spaces with absorption. First, we consider state spaces that can be decomposed into two successive subsets (with communication possible in a single direction), differentiating between three situations, and characterize the exponential order of magnitude and the exact polynomial correction, called the polynomial convergence parameter, for the leading-order term of the semigroup for large time. Second, we consider general Markov chains with finitely or countably many communication classes by applying the first results iteratively over the communication classes of the chain. We conclude with an application of these results to the case of denumerable state spaces, where we prove existence for a quasi-stationary distribution without assuming irreducibility before absorption, but only aperiodicity, existence of a Lyapunov function, and existence of a point with almost surely finite return time.

Designing efficient and rigorous numerical methods for sequential decision-making under uncertainty is a difficult problem that arises in many applications frameworks. In this paper we focus on the numerical solution of a subclass of impulse control problems for the piecewise deterministic Markov process (PDMP) when the jump times are hidden. We first state the problem as a partially observed Markov decision process (POMDP) on a continuous state space and with controlled transition kernels corresponding to some specific skeleton chains of the PDMP. We then proceed to build a numerically tractable approximation of the POMDP by tailor-made discretizations of the state spaces. The main difficulty in evaluating the discretization error comes from the possible random jumps of the PDMP between consecutive epochs of the POMDP and requires special care. Finally, we discuss the practical construction of discretization grids and illustrate our method on simulations.

We study a continuous-time mutually catalytic branching model on the $\mathbb{Z}^{d}$. The model describes the behavior of two different populations of particles, performing random walk on the lattice in the presence of branching, that is, each particle dies at a certain rate and is replaced by a random number of offspring. The branching rate of a particle in one population is proportional to the number of particles of another population at the same site. We study the long time behavior for this model, in particular, coexistence and noncoexistence of two populations in the long run. Finally, we construct a sequence of renormalized processes and use duality techniques to investigate its limiting behavior.

We prove new results about comparing the efficiency of general state space Markov chain Monte Carlo algorithms that randomly select a possibly different reversible method at each step (previously known only for finite state spaces). We also provide new, simpler, more accessible proofs of key results, and analyse numerous examples. We provide a full proof of the formula for the asymptotic variance for real-valued functionals on $\varphi$-irreducible reversible Markov chains, first introduced by Kipnis and Varadhan (1986, Commun. Math. Phys. 104, 1–19). Given two Markov kernels P and Q with stationary measure $\pi$, we say that the Markov kernel P efficiency-dominates the Markov kernel Q if the asymptotic variance with respect to P is at most the asymptotic variance with respect to Q for every real-valued functional $f\in L^2(\pi)$. Assuming only a basic background in functional analysis, we prove that for two reversible Markov kernels P and Q, P efficiency-dominates Q if and only if the operator $\mathcal{Q}-\mathcal{P}$, where $\mathcal{P}$ is the operator on $L^2(\pi)$ that maps $f\mapsto\int f(y)P(\cdot,\mathrm{d}y)$ and similarly for $\mathcal{Q}$, is positive on $L^2(\pi)$, i.e. $\langle f,\left(\mathcal{Q}-\mathcal{P}\right)f\rangle\geq0$ for every $f\in L^2(\pi)$ (previous proofs for general state spaces use technical results from monotone operator function theory). We use this result to show that under mild conditions, sandwich variants of data augmentation algorithms efficiency-dominate the original algorithm. We also provide other easy-to-check sufficient conditions for efficiency dominance, some of which are generalized from the finite state space case. We also provide a proof based on that of Tierney (1998, Ann. Appl. Prob. 8, 1–9) that Peskun dominance is a sufficient condition for efficiency dominance for reversible kernels. Using these results, we show that Markov kernels formed by random selection of other ‘component’ Markov kernels will always efficiency-dominate another Markov kernel formed in this way, as long as the component kernels of the former efficiency-dominate those of the latter. These results on the efficiency dominance of combining component kernels generalizes the results on the efficiency dominance of combined chains introduced by Neal and Rosenthal (2024, J. Appl. Prob. 62, 188–208) from finite state spaces to general state spaces.

In this paper, we solve an exit probability game between two players, each of whom controls a linear diffusion process. One player controls its process to minimize the probability that the difference of the processes reaches a low level before it reaches a high level, while the other player aims to maximize the probability. By solving the Bellman–Isaacs equations, we find the sub-value and sup-value functions of the game in explicit forms, which are twice continuously differentiable. The optimal plays associated with the sub-value and sup-value are also found explicitly.

In this paper we are concerned with susceptible–infected–removed (SIR) epidemics with vertex-dependent recovery and infection rates on complete graphs. We show that the hydrodynamic limit of our model is driven by a nonlinear function-valued ordinary differential equation consistent with a mean-field analysis. We further show that the fluctuation of our process is driven by a generalized Ornstein–Uhlenbeck process. A key step in the proofs of the main results is to show that states of different vertices are approximately independent as the population $N\rightarrow+\infty$.

The relevation model is a fundamental tool in reliability engineering for assessing the effectiveness of redundancy allocation in coherent systems. In this study, we address the problem of allocation of relevations for one or two nodes in a coherent system with independent components to enhance system reliability. We establish results concerning the usual stochastic and hazard rate orders for coherent systems. Moreover, we illustrate our findings with a range of examples and counterexamples. In addition, we conduct a simulation-based study and a real data analysis to further illustrate the application of our results. Lastly, we study the case of the minimal repair policy in detail.

Past research has indicated that the covariance of the stochastic gradient descent (SGD) error done via minibatching plays a critical role in determining its regularization and escape from low potential points. Motivated by some new research in this area, we prove universality results by showing that noise classes that have the same mean and covariance structure of SGD via minibatching have similar properties. We mainly consider the SGD algorithm, with multiplicative noise, introduced in previous work (Wu et al (2016) Int. Conf. on Machine Learning, PMLR, pp. 10367–10376), which has a much more general noise class than the SGD algorithm done via minibatching. We establish non-asymptotic bounds for the multiplicative SGD algorithm in the Wasserstein distance. We also show that the error term for the algorithm is approximately a scaled Gaussian distribution with mean 0 at any fixed point.

The marked Hawkes risk process is a compound point process where the occurrence and amplitude of past events impact the future. Since data in real life are acquired over a discrete time grid, we propose a strong discrete-time approximation of the continuous-time risk process obtained by embedding from the same Poisson measure. We then prove trajectorial convergence results in both fractional Sobolev spaces and the Skorokhod space, hence extending the theorems proven in Huang and Khabou ((2023). Stoch. Process. Appl. 161, 201–241) and Kirchner ((2016). Stoch. Process. Appl. 126(8), 2494–2525). We also provide upper bounds on the convergence speed with explicit dependence on the size of the discretization step, the time horizon, and the regularity of the kernel.