Background and motivation. Due to the continuous demand for reducing computational costs of Monte Carlo simulation in various fields of application, variance reduction methods have attracted significant interest for decades. In particular, adaptive Monte Carlo methods, an essential theme of this thesis, aim at running the Monte Carlo simulation concurrently with a parameter search procedure for variance reduction using a common set of random realisations, in contrast to sequential variance reduction approaches that separate the parameter search phase and the primary simulation phase. Monte Carlo simulation along with variance reduction techniques has gained substantial attention in structural reliability analysis since the 1980s, aiming at improving the estimation of the probability of failure of a structural system, which is the primary line of application we investigate.
This thesis consists of six chapters contributing to the theory and application of Monte Carlo methods and variance reduction techniques, with the first three establishing general probabilistic frameworks and the latter three investigating the application in the context of structural reliability analysis. Summaries of motivation and contributions of each chapter are outlined as follows.
Sampling and change of measure on simplices. Monte Carlo methods have been studied extensively for numerical integration over the unit hypercube along with a variety of variance reduction techniques for accelerating the convergence of the central limit theorem, whereas not much attention has been given to Monte Carlo integration on more challenging domains, such as polytopes. Simplices are the fundamental domain when integrating over convex polytopes, with various numerical techniques such as exact integration methods and cubature algorithms developed for integrating over simplices in the literature. The aim of this chapter is to establish a novel framework of Monte Carlo integration over simplices, from sampling to variance reduction. We first develop a uniform sampling method on the standard simplex consisting of two independent procedures. Then, with a view towards variance reduction by importance sampling, we construct theories on change of measure on each of the two independent elements in the developed sampling technique, with discussions on the potential for adaptive implementation.
Batching adaptive variance reduction. Although frequent recalibration of the parameter is not required for adaptive Monte Carlo methods, an initialisation step is still required in some instances. The length of such a preliminary phase has often been left unspecified for the user to determine on a case-by-case basis. For example, a pilot run is conducted for a random duration prior to the simulation in preparing learning rates of stochastic approximation in a framework of adaptive importance sampling [Reference Kawai1]. This uncertain element may possibly be even fatal in realistic finite-budget situations, since the pilot run may take most of the budget, or possibly use up all of it. This chapter aims to resolve the issue of an ad hoc initialisation step by adopting a batching procedure into a framework of adaptive variance reduction and providing an implementable formula of the batch-wise learning rate in the parameter search algorithm [Reference Song and Kawai5]. We provide an implementable formula of the batch-wise learning rate in the parameter search which minimises an upper bound of the theoretical variance of the empirical batch mean. By setting up predetermined timings for periodically updating relevant problem parameters instead of conducting a preliminary run to prepare the constant learning rate, we attempt to both avoid the opportunity loss of variance reduction and eliminate the randomness in the length of the pilot run.
Dynamic finite-budget allocation of stratified sampling with adaptive variance reduction by strata. Stratified sampling is an effective variance reduction method that accelerates Monte Carlo simulation by sampling reasonable shares of points from mutually exclusive and exhaustive strata. Adaptive importance sampling is generally difficult to apply under stratification, since the original form of strata may not be retained in the course of changing the probability measure for formulating importance sampling. In this chapter, by wisely formulating the base framework on the unit hypercube, we are able to apply combined adaptive importance sampling and control variates [Reference Kawai2] to stratified random elements, without needing to monitor the differences in stratum computational costs. We also develop and analyse a dynamic finite-budget allocation scheme for stratified sampling, which updates the budget allocation only occasionally yet effectively takes full account of decreasing stratum variances, while simultaneously estimating the stratum means and updating the variance reduction parameters throughout [Reference Song and Kawai6].
Adaptive stratified sampling for structural reliability analysis. Various Monte Carlo variance reduction techniques have been employed in improving the estimation of the probability of failure in the context of structural reliability analysis. Stratified and Latin hypercube sampling has been shown to be effective due to its intrinsic flexibility in dealing with structural systems with sparse and possibly multiple failure modes. We adopt the framework of stratified sampling with dynamic budget allocation and variance reduction by strata [Reference Song and Kawai6] developed in the previous chapter into this field of application. By combining the advantages of stratified sampling, importance sampling and control variates, the proposed approach [Reference Song and Kawai4] is suitable and effective for a variety of structural reliability problems with sparse structures, which is demonstrated through a variety of numerical examples in this chapter.
Adaptive stratified directional importance sampling. Among various Monte Carlo methods in estimating the probability of failure for structural reliability analysis, both directional simulation and importance sampling have gained significant attention. Directional simulation is a radial sampling method in the polar coordinate system, aiming at improving the estimation by suppressing the randomness in the radial distance, on which the monotonicity condition is required for justifying a deterministic numerical procedure such as root finding. This chapter proposes an adaptive framework [Reference Song and Kawai3] and establishes its easy-to-implement algorithm that is purely simulation based and hence does not depend on such a monotonicity condition, with the unit hypersphere stratified on the direction and importance sampling applied on the radial length by strata, adopting the dynamic budget allocation scheme presented in the third chapter.
Monte Carlo and variance reduction methods for structural reliability analysis: methodological aspects. Monte Carlo simulation and variance reduction techniques have attracted constant interest in structural reliability analysis for estimating the probability of failure of time-invariant problems, with various developments continuously appearing in the literature. Existing reviews and benchmark studies often specialise in certain topics, aiming at summarising and comparing a few selected approaches in detail, usually from an implementation point of view. To provide a comprehensive guidebook on methodological aspects of Monte Carlo methods in this context, in this chapter, we provide a broad and exhaustive overview of existing formulations and techniques, and survey numerical approaches that make use of the methodologies of Monte Carlo simulation and variance reduction techniques [Reference Song and Kawai7]. For each Monte Carlo technique employed in structural reliability analysis, we review its general formulation, sub-categories and variants, as well as its combined use with other simulation techniques or surrogate models, along with insightful summaries of developments of numerical approaches in the literature.
