1. Introduction
1.1. Background
Approximate uniform sampling from convex bodies is an important algorithmic primitive and has consequently been the focus of a large body of research. The first algorithm for this problem that ran in time polynomial in the ambient dimension of the convex body was given in the seminal work of Dyer, Frieze and Kannan [Reference Dyer, Frieze and Kannan4] and was based on a nearest-neighbour random walk on a discrete grid inside the convex body. This resulted in the first polynomial time (in the ambient dimension and
$1/\epsilon$
) algorithm for obtaining a
$(1 \pm \epsilon)$
-factor approximation for the volume of an input convex body
${\mathbb R}^n$
, given access to a so-called ‘well-guaranteed membership oracle’Footnote 1 for the body.
Following the success of the Markov chain Monte Carlo approach for the problem in the work of Dyer, Frieze and Kannan, there have been many attempts at designing other random walks for sampling from convex bodies with provably better mixing times; for a partial list, see [Reference Chen, Dwivedi, Wainwright and Yu2, Reference Cousins3, Reference Kannan, Lovász and Simonovits7, Reference Kannan and Narayanan8, Reference Laddha, Lee and Vempala10–Reference Lee and Vempala12, Reference Lovász14, Reference Lovász and Simonovits15, Reference Mangoubi and Vishnoi18, Reference Narayanan20]. The goal in this line of work is twofold. First, the Markov chain should be such that each step of the random walk can be implemented easily, under reasonable regularity conditions on the convex body, and second, the number of steps the resulting Markov chain takes to come
$\epsilon$
-close to the stationary distribution (in total variation distance) should be bounded by a small polynomial in n (the ambient dimension),
$1/\epsilon$
, and the ratio of the in-radius and the diameter of the body.
One such random walk is the so-called ‘Hit-and-Run’ walk, first proposed in the work of Smith [Reference Smith21]. Given a convex body K, the samples
$x_1, x_2, \dots$
of this walk are generated as follows. Given
$x_i = x$
, choose a direction u on the unit sphere
$\mathbb{S}^{n-1}$
uniformly at random, and let
$\ell$
be the unique chord of K through x along the direction u. Then, pick a random point y from the uniform measure on this chord and set
$x_{i+1} = y$
. A simplified version of this is the coordinate Hit-and-Run (CHR) random walk, in which instead of sampling the direction u from the unit sphere, it is sampled instead from the discrete uniform distribution on the canonical basis vectors
$e_1, e_2, \dots, e_n$
. This restriction can make the CHR random walk easier to implement than the original Hit-and-Run walk in certain settings [Reference Haraldsdóttir, Cousins, Thiele, Fleming and Vempala6].
Lovász [Reference Lovász14] analysed the mixing time of the Hit-and-Run random walk from a ‘warm start’, that is, a starting distribution whose Radon–Nikodym derivative with respect to the uniform distribution on K is bounded above by some given quantity M, and showed that with such a starting distribution, the Hit-and-Run walk mixes to within
$\epsilon$
total variation distance of the uniform distribution on a convex body K in time
${\mathop{\mathrm{poly}}\left({{n, R, M}}\right)}$
(here, it is assumed that K is contained with the Euclidean ball of radius R and contains within it the standard Euclidean ball of radius 1). However, despite its applicability in practice [Reference Fallahi, Skaug and Alendal5, Reference Haraldsdóttir, Cousins, Thiele, Fleming and Vempala6], establishing a similar polynomial mixing time bound on for the CHR walk has remained an open problem (see, e.g, [Reference Lee and Vempala13, pp. 23–24]).
1.2. Our contribution
In this paper, we prove that under regularity conditions similar to those in [Reference Lovász14], the CHR walk has a polynomial mixing time from a warm start. More formally, let
$B_\infty$
denote the unit ball
$\left\{{x \in {\mathbb R}^n {\;\textbf{:}\;} {\Vert x \Vert_{\infty}} \leq 1}\right\}$
in
${\Vert {\cdot} \Vert_{\infty}}$
. As usual, for any positive real r, the notation
$r\cdot B_\infty$
denotes the set
$\left\{{r\cdot x {\;\textbf{:}\;} x \in B_\infty}\right\}$
. For
$M \geq 1$
, we say that a probability distribution
$\mu$
is ‘M-warm’ with respect to another probability distribution
$\pi$
if
$\mu$
is absolutely continuous with respect to
$\pi$
and if the Radon–Nikodym derivative of
$\mu$
with respect to
$\pi$
is pointwise bounded above by M. Our main result is then the following (see Theorem 6.1 for a more quantitative statement).
Theorem 1.1. Let
$K \subset {\mathbb R}^n$
be a closed convex body such that
$B_\infty \subseteq K \subseteq R\cdot B_\infty$
, for some
$R \geq 1$
. Let
$\mu_0$
be a probability distribution on K that is M-warm with respect to the uniform distribution
$\pi_K$
on K. Starting with the initial ‘warm start’ distribution
$\mu_0$
, let
$\mu_k$
denote the probability distribution of the point obtained after
$k \geq 1$
steps of the CHR walk for K. Then, for any
$\epsilon > 0$
and
$k \geq \tilde{O}\left({\frac{n^{7}R^{4}M^{4}}{\epsilon^{4}}}\right)$
, we have
$d_{TV}(\mu_k, \pi_K) \leq {\epsilon}.$
(Here
$d_{TV}$
denotes the total variation distance between probability measures.)
It follows that starting from a M-warm start, the mixing time
$T_{CHR}(\epsilon)$
of the CHR random walk on the body K is bounded by a polynomial in n, M, R and
$1/\epsilon$
. The precise form of the polynomial is given in Theorem 6.1. Noting that the unit cube in n dimensions is contained in a Euclidean ball of diameter
$\sqrt{n}$
, this result also implies a similar result for convex sets sandwiched using Euclidean balls, with a slightly worse dependence on n (i.e.
$n^9$
).
Concurrent and independent work by Laddha and Vempala [Reference Laddha and Vempala9] has a main result that is similar to ours: they also give a polynomial mixing time from a warm start for the CHR chain. Their results hinge upon directly proving a novel isoperimetric inequality with respect to axis aligned moves for convex sets that are cubes, and then extending this to arbitrary convex sets by tiling them using cubes. The bounds provided by Laddha and Vempala are in terms of sandwiching ratios S involving Euclidean balls, and have the form
$\tilde{O}(n^9 S^2)$
rather than
$\tilde{O}(n^7R^4)$
of this paper. Depending on the body, one or the other bound can be better – it is possible to construct ‘tube-like’ bodies for which
$\tilde{O}(n^7R^4)$
is larger, and cube-like bodies with a small sandwiching ratio for which
$\tilde{O}(n^9 S^2)$
is larger.
There exists an affine transformation that can make the sandwiching ratio using Euclidean balls as small as O(n) (such as rotating to the isotropic position), and even
$\tilde{O}(\sqrt{n})$
if one is willing to discard an insignificant amount of mass (see the discussion on page 410 of Lovász and Vempala [Reference Lovász and Vempala16]). However, we believe that the situations where CHR would be most useful are those where there is an underlying set of basis vector directions that has a statistically relevant interpretation. CHR also readily lends itself to a random walk on the lattice points of a convex set. However, rounding the body using non-diagonal linear transformations would not preserve the axis directions. Similarly, non-unimodular linear transformations do not preserve the lattice. For these reasons, we do not include a detailed discussion of pre-processing the body through such affine transformations.
1.3. Technical overview and discussion
In this section, we fix a convex body
$K \in {\mathbb R}^n$
, and assume that
$R \geq 1$
is such that
$B_\infty \leq K \leq R\cdot B_\infty$
. One of the main ingredients of Lovász’s proof of the fast mixing of the Hit-and-Run walk [Reference Lovász14] is the following ‘overlap property’ of the Hit-and-Run walk. For
$v \in K$
, let
$\mathcal{H}_v$
denote the probability distribution of the point sampled by executing one step of the Hit-and-Run walk on K started from v. Lovász [Reference Lovász14] showed that when
$u, v \in K$
are ‘close’, the probability distributions
$H_u$
and
$H_v$
have total variation distance bounded away from 1. The ‘closeness’ condition was defined in [Reference Lovász14] in terms of a combination of the Euclidean distance and the ‘cross-ratio distance’ (derived from the Hilbert metric on K). The second ingredient is an appropriate isoperimetric inequality, which in the case of [Reference Lovász14] was a new isoperimetric inequality in terms of the Hilbert metric. These two ingredients were then combined in [Reference Lovász14] to obtain lower bounds on the s-conductance (see equation (2.2)) of the Hit-and-Run random walk for all
$0 < s < 1/2$
, from which the mixing time bound from a warm start follows via a result of Lovász and Simonovits [Reference Lovász and Simonovits15] (see Theorem 2.1).
Unfortunately, the overlap property fails for the CHR chain. To see this, denote by
$\mathcal{C}_v$
the probability distribution of the point sampled by executing one step of the CHR walk on K started from v. It is easy to see then that whenever
$x, y \in K$
are such that
$x - y$
has a non-zero component along each of the coordinate directions,
$\mathcal{C}_x$
and
$\mathcal{C}_y$
have disjoint supports, so that
$d_{TV} (\mathcal{C}_x , \mathcal{C}_y) = 1$
. To circumvent this difficulty, we consider in Section 4 a multi-step variant of the CHR chain, which we call the Gaussian chain. We then establish an analogue of the overlap property (see Definition 3.1 for a precise definition of the property in our setting) for this chain. The method for passing from this version of the overlap property to a lower bound on s-conductance, via an isoperimetric inequality, is similar to the one used by Lovász [Reference Lovász14], and is outlined in Section 3. An isoperimetric inequality due to Lovász and Simonovits [Reference Lovász and Simonovits15] (see Theorem 2.1) is then combined with the results in Section 3, and with a comparison argument between the Gaussian chain and the actual CHR chain, to give a lower bound on the s-conductance of the latter (see Section 5). The mixing time result follows immediately from this lower bound on the s-conductance using Theorem 2.1 due to Lovász and Simonovits [Reference Lovász and Simonovits15]; the details are provided in Section 6.
The isoperimetric inequality with respect to the cross-ratio distance used in [Reference Lovász14] was an important ingredient in the optimised run-time bounds obtained therein for the Hit-and-Run walk. An extension of this inequality was later derived by Lovász and Vempala [Reference Lovász and Vempala17], who used it to prove fast mixing for the Hit-and-Run walk while substantially relaxing the requirement of ‘warm start’ used in [Reference Lovász14]. Unfortunately, the CHR walk does not appear to share with the Hit-and-Run walk the nice geometric properties that enabled the use of these improved isoperimetric inequalities. Optimising the mixing time bounds obtained for CHR in this paper, and relaxing the ‘warm start’ condition, both therefore remain interesting open problems.
2. Preliminaries
Notation. We will denote positive universal constants that are less than 1 by c and positive universal constants that are greater than 1 by C.
2.1. The CHR walk
Let
$K\subseteq {\mathbb R}^n$
be a convex body. Let
$e_1, \dots, e_n$
denote the canonical basis of
${\mathbb R}^n$
. The law of the CHR walk can then be expressed as follows. Fix v belonging to the interior of K. To implement one step of the CHR walk, do the following. First, choose
$i \in [n]$
uniformly at random, and let
$\ell$
be the cord
$K \cap (v + e_i {\mathbb R}).$
Then, choose u uniformly at random from
$\ell$
(with respect to the
$1-$
dimensional Lebesgue measure
$\lambda_1$
on
${\mathbb R}$
). Thus, the transition probability densities (with respect to
$\lambda_1$
) are given by
\begin{equation*} \mathcal P_{vu} = \begin{cases} \frac{1}{n|K \cap (v + e_i {\mathbb R})|} & \text{$u \in K \cap (v + e_i {\mathbb R})$ for some $i \in [n]$,}\\ 0 & \text{ otherwise.} \end{cases}\end{equation*}
2.2. Markov schemes and isoperimetry
In this section, we collect some definitions and results about Markov chains that will be used in our proofs. We mostly follow the terminology and notation of Lovász and Simonovits [Reference Lovász and Simonovits15].
For any measurable
$S \subseteq {\mathbb R}^n$
, we denote by
${\mathrm{vol}\left({S}\right)}$
its Lebesgue volume, and by
$\pi_S$
the uniform probability distribution on S. Thus,
$\pi_{S}(A)=\mathrm{vol}(S\cap A)/\mathrm{vol}(S)$
for any measurable A. Fix a measurable set
$S \subseteq {\mathbb R}^n$
of finite volume and let
$\mathcal{S}$
denote the set of measurable subsets of S. A Markov scheme on S is a map
$P\;:\;S \times \mathcal{S} \rightarrow [0,1]$
such that (i) for every
$u \in S$
,
$P(u, \cdot)$
is a probability measure on S, and (ii) for every
$A \in \mathcal{S}$
, the function
$P(\cdot, A) : S \rightarrow [0,1]$
is measurable. Given a Markov scheme P and a probability distribution
$\mu_0$
on S, A Markov chain
$M = M(\mu, P)$
is a sequence
$X_0, X_1, \dots, X_k,\dots$
of random points in S such that
$X_0 \sim \mu_0$
and for
$i \geq 1$
,
$X_i \sim P(X_{i-1}, \cdot)$
. We denote by
$\mu_{k, P}$
the probability distribution of the random variable
$X_k$
.
A probability measure Q on S is said to be stationary with respect to a Markov scheme P on S if for all measurable
$A\subseteq S$
,
Such a Q is said to be reversible with respect to P if for all measurable
$A, B \subseteq S$
,
Note that if Q is reversible with respect to P, then it is also stationary with respect to Q.
The ergodic flow
$\Phi_{P,Q}$
of a Markov scheme P on S with respect to a probability distribution Q on S is defined as
Note that when Q is stationary with respect to P, one can easily check that [Reference Lovász and Simonovits15, p. 366]
By abuse of notation, we extend the definition of ergodic flow to pairs of measurable sets:
The s-conductance [Reference Lovász and Simonovits15, p. 367]
$\Phi_s$
, where
$0 \leq s < 1/2$
, of a Markov scheme P on S with respect to a probability distribution Q on S is defined as
Quantitative estimates involving the conductance of Markov chains have been an important ingredient of various results proving mixing time upper bounds for Markov chains. In our proof, we will use the following estimate, stated in this form by Lovász and Simonovits [Reference Lovász and Simonovits15].
Theorem 2.1
([
Reference Lovász and Simonovits15
, Corollary 1.6]). Let S be a measurable subset of
${\mathbb R}^n$
of finite volume. Let P be a Markov scheme on S, and let
$\mu_0$
and Q be probability distributions on S such that Q is stationary with respect to P. Assume that Q is atom-free, that is,
$Q(\left\{{u}\right\}) = 0$
for all
$u \in S$
. Fix
$0 < s < 1/2$
, and define
For each
$k \geq 0$
, let
$\mu_{k, P}$
denote the distribution of the kth element in the Markov chain
$(\mu_0, P)$
. Then, for all non-negative integers k,
$${d_{TV}}({\mu _{k,P}},Q) \le \left( {1 + {{{{(1 - \Phi _s^2/2)}^k}} \over s}} \right)\cdot{H_s},$$
where
$\Phi_s$
is the s-conductance of P with respect to Q.
We will also need the following isoperimetric inequality.
Theorem 2.2
([
Reference Lovász and Simonovits15
, Corollary 2.7]). Let
$\delta > 0$
be arbitrary. Fix any norm
${\Vert {\cdot} \Vert_{\ell}}$
on
${\mathbb R}^n$
. Let
$K \subseteq {\mathbb R}^n$
be a convex body and let
$K_1$
and
$K_2$
be disjoint measurable subsets of K such that for all
$u \in K_1$
and
$v \in K_2$
, one has
${\Vert {u - v} \Vert_{\ell}} \geq \delta$
. Suppose that the diameter of K in the
${\Vert {\cdot} \Vert_{\ell}}$
-norm is at most D (i.e.,
${\Vert {x - y} \Vert_{\ell}} \leq D$
for all
$x, y \in K$
). Then,
2.3. Volume of the robust interior
Given a closed convex body K, we define, for
$r > 0$
,
The following proposition follows directly from [Reference Dyer, Frieze and Kannan4, Proposition 2].
Proposition 2.3. Suppose that
$\epsilon \in (0, 1/\sqrt{n})$
and
$K \subset {\mathbb R}^n$
is a closed convex body such that
$B = B_2 \subseteq K$
. Then
$(1-\epsilon\sqrt{n})K \subseteq K_{\epsilon}$
. In particular,
${\mathrm{vol}\left({{K_\epsilon}}\right)} \geq (1-\epsilon\sqrt{n})^n{\mathrm{vol}\left({{K}}\right)}$
.
In our setting, we will need the following proposition which improves upon Proposition 2.3 in the case when K is guaranteed to contain the
${\Vert {\cdot} \Vert_{\infty}}$
-ball
$B_\infty$
instead of just the
${\Vert {\cdot} \Vert_{2}}$
-ball
$B_2$
. The proof follows in an essentially identical fashion to that of [Reference Dyer, Frieze and Kannan4, Proposition 2], but we include it for completeness.
Proposition 2.4. Suppose that
$\epsilon \in (0, 1)$
and
$K \subset {\mathbb R}^n$
is a closed convex body such that
$B_\infty \subseteq K$
. Then
$(1-\epsilon)K \subseteq K_{\epsilon}$
. In particular,
${\mathrm{vol}\left({{K_\epsilon}}\right)} \geq (1-\epsilon)^n {\mathrm{vol}\left({{K}}\right)}$
.
Proof. We only need to show that
${\Vert {z - x} \Vert_{\infty}} > \epsilon$
for all
$x \in (1-\epsilon)K$
and
$z \in K^c$
. Fix any
$z \in K^c$
. Now, by the separating hyperplane theorem, there exists a non-zero vector v such that
$v^Tz > 1$
and
$v^Ty \leq 1$
for all
$y\in K$
. Since
$B_\infty \subseteq K$
, the latter inequality implies that
${\Vert {v} \Vert_{1}} \leq 1$
. Further, for any
$x \in (1-\epsilon)K$
, the same inequality gives
$v^T{x} \leq (1-\epsilon)$
. But then we have
3. Overlaps and conductance
The next definition and the results following it in this section are based on a method proposed by Lovász [Reference Lovász14, Section 5].
Definition 3.1 (
$(\epsilon, \delta, \nu)$
-overlap property) Let K be a convex body in
${\mathbb R}^n$
. A Markov scheme P on K is said to have the
$(\epsilon, \delta, \nu)$
-overlap property with respect to K
′, where K
′ is a convex subset of K, if (i)
${\mathrm{vol}\left({{K'}}\right)} \geq {\mathrm{vol}\left({{K}}\right)}(1 - \epsilon)$
and (ii) for all
$u, v \in K'$
satisfying
${\Vert {u - v} \Vert_{2}} \leq \delta$
, we have
Lemma 3.2 Let
$\epsilon, \delta, \nu \in (0, 1/2)$
. Let K be a convex body in
${\mathbb R}^n$
of
${\Vert {\cdot} \Vert_{2}}$
-diameter at most
$D \geq 2\delta$
and let P be a Markov scheme on K which is reversible with respect to the uniform distribution
$\pi_K$
on K. Suppose also that P has the
$(\epsilon, \delta, \nu)$
-overlap property with respect to K’. Then, the
$\epsilon$
-conductance of P with respect to
$\pi_K$
is at least
$\frac{\nu\delta}{4(D-\delta)}$
. In fact, if
$S_1, S_2$
is any arbitrary partition of K into disjoint measurable subsets, and we denote
$K'\cap S_1$
by
$T_1$
and
$K'\cap S_2$
by
$T_2$
, then we have
\begin{align*} \Phi_{P, \pi_K}(S_1) \geq \max \left\{{\Phi_{P, \pi_K}(S_1, T_2), \Phi_{P, \pi_K}(S_2, T_1) }\right\} & \geq \left(\frac{\nu\delta}{4(D-\delta)}\right)\min\left\{{\pi_K(T_1), \pi_K(T_2)}\right\}\!,\\ & \geq \left(\frac{\nu\delta}{4(D-\delta)}\right)\min\left\{{\pi_K(S_1) - \epsilon, \pi_K(S_2) - \epsilon}\right\}\!. \end{align*}
Proof. The proof follows the same method as that of Lovász [Reference Lovász14, Sections 5 and 6]. Let
$S_1, S_2$
be any arbitrary partition of K into disjoint measurable subsets, and let the convex subset
$K' \subseteq K$
be as in the definition of the
$(\epsilon, \delta, \nu)$
-overlap property, with
$T_1 \,:\!=\, S_1 \cap K'$
,
$T_2 \,:\!=\, S_2 \cap K'$
. Note that since
$\pi_K(K') \geq 1 - \epsilon$
, one has
Now, consider the following subsets of
$T_1$
and
$T_2$
:
It then follows from the definition of the overlap property that
for if not, then for such x and y satisfying
${\Vert {x-y} \Vert_{2}} \leq \delta$
, we would have
which is a contradiction. We can therefore apply the isoperimetric inequality of Theorem 2.2 to the subsets
$S^{\prime}_1$
and
$S^{\prime}_2$
of the convex body K
′ to get
Note that since P is reversible with respect to
$\pi_K$
, we have
$\Phi_{P, \pi_K}(S_1, T_2) = \Phi_{P, \pi_K}(T_2, S_1)$
. We thus have
\begin{equation} \Phi_{P, \pi_K}(S_1, T_2) = \int\limits_{T_2}P(u, S_1) \pi_K(du) \geq \int\limits_{T_2 - S^{\prime}_2}P(u, S_1) \pi_K(du) \geq \frac{\nu}{2}\pi_K(T_2-S^{\prime}_2). \end{equation}
Similarly, we get
\begin{equation} \Phi_{P, \pi_K}(S_2, T_1) = \int\limits_{T_1}P(u, S_2) \pi_K(du) \geq \int\limits_{T_1 - S^{\prime}_1}P(u, S_2) \pi_K(du) \geq \frac{\nu}{2}\pi_K(T_1-S^{\prime}_1). \end{equation}
Consider now the condition:
The rest of the proof is divided into two cases:
Case 1 : The condition in equation(3.7)is false. In this case, we must have
$\pi_K(S^{\prime}_1
) \leq \frac{1}{2} \pi_K(T_1)$
or
$\pi_K(S^{\prime}_2) \leq \frac{1}{2}\pi_K(T_2)$
. Using equation (3.5) or equation (3.6) in the respective cases, we immediately get
\begin{align} & \max\left\{{\Phi_{P, \pi_K}(S_1, T_2), \Phi_{P, \pi_K}(S_2, T_1)}\right\} \nonumber \\ & \quad \geq \frac{\nu}{4}\min\left\{{\pi_K(T_1), \pi_K(T_2)}\right\} \stackrel{\text{equation (3.1)}}{\geq} \frac{\nu}{4}\min\left\{{\pi_K(S_1) - \epsilon, \pi_K(S_2) -\epsilon}\right\}, \end{align}
and the required inequality follows because of the assumption
$D \geq 2\delta$
.
Case 2 : The condition in equation(3.7)is true. By adding the inequalities in equations (3.5) and (3.6), and using the fact that the sets
$T_1$
and
$T_2$
form a disjoint partition of K
$^{\prime}$
, we have
Now, using equation (3.7) followed by equation (3.1) in the isoperimetric inequality in equation (3.4), we get
\begin{align*} \pi_K(K' - (S^{\prime}_1 \cup S^{\prime}_2)) & \geq \frac{\delta}{D - \delta}\min\left\{{\pi_K(T_1), \pi_K(T_2)}\right\}\\ & \geq \frac{\delta}{D - \delta}\min\left\{{\pi_K(S_1) - \epsilon, \pi_K(S_2) - \epsilon}\right\}. \end{align*}
Substituting this into equation (3.9), we again obtain the required inequality.
4. The Gaussian walk
In this section, we analyse an auxiliary Markov chain with Gaussian steps. In the next section, this chain will be compared with the CHR chain to establish lower bounds on the s-conductance of the latter.
Let
$N(0, \sigma^2)$
denote the Gaussian distribution on
${\mathbb R}$
with mean 0 and variance
$\sigma^2$
, and let
$\gamma$
denote its density with respect to the Lebesgue measure. The Gaussian walk on a convex body K is then defined as follows. For some
$t \in \mathbb{N}$
, suppose that we are given
$x_t = v \in K$
. To generate
$x_{t+1}$
, do the following. Choose
$i \in [n]$
uniformly at random, and
$\kappa \sim N(0, \sigma^2)$
. Let
$u = v + \kappa e_i$
. If
$u \in K$
, accept the proposed transition and set
$x_{t+1} = u$
. Else, the proposal is rejected and
$x_{t+1} = v$
.
Thus, the transition probability densities
$\mathcal{G}_{vu}$
(with respect to the one-dimensional Lebesgue measure) are given by
\begin{equation*} \mathcal{G}_{vu} = \begin{cases} \frac{\gamma(|u - v|)}{n} & \text{ if $u \neq v$ and $u \in K \cap (v + e_i {\mathbb R})$ for some $i \in [n]$,}\\ 0 & \text{ otherwise.} \end{cases}\end{equation*}
In addition, the rejection probability, that is, the probability that
$x_{t+1} = v$
, conditional on
$x_t = v$
, is given by
Note that like the CHR chain, the Gaussian chain is also reversible with respect to the uniform probability measure
$\pi_K$
on K.
Given a positive integer
$\tau$
, let
$\mathcal{G}_v^{(\tau)}$
denote the probability distribution of
$x_\tau$
given that
$x_0 = v,$
and where, for
$0 \leq t \leq \tau - 1$
,
$x_{t+1}$
is generated from
$x_t$
according to the Gaussian chain
$\mathcal{G}_{x_t}$
.
We now set up some notation. Given a multi-index
$\mathbb{I}=(i_1, \dots, i_n) \in \mathbb{N}^{n}$
, we define
$\mathcal{G}_{v,\mathbb{I}}$
to be the unique Gaussian distribution centered at
$v \in {\mathbb R}^n$
, whose covariance matrix
$\Sigma_\mathbb{I}$
with respect to the canonical basis is the diagonal matrix with
$i_j \sigma^2$
as the
$j^{th}$
diagonal entry. We define
$\mathcal M_{n,\tau}$
to be the set of all such multi-indices
$\mathbb{I} = (i_1, i_2, \dots, i_n) \in \mathbb{N}^n$
for which
$\sum_j i_j = \tau$
. We say that such an
$\mathbb{I}$
is of full rank if the covariance matrix
$\Sigma_I$
has full rank, that is, if all of the
$i_k$
are at least 1. Further, for
$\mathbb{I} \in \mathcal M_{n,\tau}$
, we define
$\lambda_\mathbb{I} \,:\!=\, {\binom{\tau}{\mathbb{I}}}n^{-\tau},$
where
${\binom{\tau}{\mathbb{I}}}$
is the multinomial coefficient
$\frac{\tau!}{\prod_k i_k!}.$
Observe that
We then have the following lemma.
Lemma 4.1 Fix
$\sigma > 0$
,
$n \geq 2$
and
$\tau = \tau(n) \,:\!=\, \lceil 20 n \ln n\rceil$
. Let
$K \in {\mathbb R}^n$
be a convex body and let
$v \in K$
be a point such that
$\inf_{z \in \partial K} \|v - z\|_{\infty} > 100 \sigma \ln n.$
Define the measures
$\mathcal{G}_v^{\tau}$
and
$\mathcal{G}_{v, \mathbb{I}}$
as above, in terms of
$N(0, \sigma^2)$
. Then,
$$d_{TV}\left({\mathcal{G}_v^{(\tau)}, \sum_{\substack{\mathbb{I} \in \mathcal M_{n,\tau}\\\mathbb{I}\text{ of full rank}}} \lambda_\mathbb{I} \mathcal{G}_{v,\mathbb{I}}}\right) \leq 2 n^{-5}.$$
Proof. Let
$\mathcal{H}$
denote the probability measure
$\left(\sum_{\mathbb{I} \in \mathcal M_{n,\tau}} \lambda_\mathbb{I} \mathcal{G}_{v,\mathbb{I}}\right)$
, and
$\mathcal{H}'$
the measure
$(\!\sum_{\substack{\mathbb{I} \in \mathcal M_{n,\tau}\\\mathbb{I}\text{ of full rank}}} \lambda_\mathbb{I} \mathcal{G}_{v,\mathbb{I}})$
. Consider the following natural coupling between
$\mathcal{G}_{v}^{(\tau)}$
and
$\mathcal{H}$
, for
$0 \leq t \leq \tau$
. Set
$x_0 = x^{\prime}_0 = v$
. For
$0 \leq t \leq \tau - 1$
, the pair
$(x_{t+1}, x^{\prime}_{t+1})$
, given the pair
$(x_t, x^{\prime}_t)$
, is generated as follows. Pick
$i \in [n]$
u.a.r. and
$\kappa \sim N(0, \sigma^2)$
, and set
$x^{\prime}_{t+1} = x^{\prime}_{t} + \kappa e_i$
. Further
$x_{t+1} = x_t + \kappa e_i$
if that point is in K, otherwise
$x_{t+1} = x_{t}$
. As before, in the latter case we say that the update at time
$t + 1$
was rejected. Let E be the event that none of the updates at times
$1\leq t \leq \tau$
are rejected.
By definition, we have
$x_\tau \sim \mathcal{G}_v^{(\tau)}$
and
$x^{\prime}_\tau \sim \mathcal{H}$
. Further, when the event E occurs, we also have
$x_\tau = x^{\prime}_\tau$
. It therefore follows from standard coupling arguments that
We now lower bound
${\mathbb{P}}[E]$
. For
$1 \leq i \leq n$
, let
$A_i$
denote the number of times an update along the coordinate direction i is proposed in the above coupling. Then, by a standard Chernoff bound (see Fact A.1), and recalling that
$\tau = {\left\lceil{20 n \ln n}\right\rceil}$
, we have
We now recall that
$x_0 = x^{\prime}_0 = v$
satisfies
$\inf_{z \in \partial K} \|v - z\|_{\infty} > 100 \sigma \ln n$
. Thus, conditioned on the event that each
$A_i$
is at most
$50\ln n$
, standard Gaussian tail bounds imply that with high probability, each of the
$x^{\prime}_j$
,
$0 \leq j \leq \tau$
, is actually contained in K, so that the event E occurs. To establish this claim formally, let
$A_{i,t}$
, where
$1 \leq i \leq n$
and
$1 \leq t \leq \tau$
, denote the number of times, up to time t, that an update along the coordinate direction i is performed in the above coupling (thus,
$A_i = A_{i, \tau}$
). Note that conditioned on the
$A_{i,t}$
, the jth coordinate
$x^{\prime}_{s,j}$
of
$x^{\prime}_s$
has the distribution
$N(v_j, A_{j,s}\sigma^2)$
. Further, if we have
$\vert x^{\prime}_{s,j} - v_j\vert \leq 100 \sigma \ln n$
for all
$1 \leq s \leq \tau$
and
$1 \leq j \leq n$
, then the event E occurs. Using a union bound, we thus have
\begin{multline*} {\mathbb{P}}[\lnot E\, \big\vert\, A_{i,t}, \text{ where } 1\leq i \leq n, 1 \leq t \leq \tau]\\
\begin{aligned} &\leq \sum_{j = 1}^n\sum_{s = 1}^\tau {\mathbb{P}}[\vert x^{\prime}_{s,j} - v_j\vert > 100 \sigma \ln n\, \big\vert\, A_{i,t}, \text{ where } 1\leq i \leq n, 1 \leq t \leq \tau]\\ &= \sum_{j = 1}^n\sum_{s = 1}^\tau {\mathbb{P}}_{Z \sim N(v_j, A_{j,s}\sigma^2)}[\vert Z - v_j\vert > 100 \sigma \ln n]. \end{aligned} \end{multline*}
Now, using the fact that
$A_{j,s} \leq A_j$
for all
$1 \leq j \leq n$
and
$1 \leq s \leq \tau$
, and then applying Fact A.2 to bound each term in the sum above, we get
Together, equations (4.2), (4.3) and (4.4) imply that
Finally, we note that
\begin{equation} d_{TV}({\mathcal{H'}, \mathcal{H}}) \;\;\leq \sum_{\substack{\mathbb{I} \in \mathcal M_{n,\tau}\\\mathbb{I}\text{ not of full rank}}} \lambda_\mathbb{I} \;\; \leq \;\; n\cdot\left({1- \frac{1}{n}}\right)^\tau \leq n^{-19}. \end{equation}
The claim now follows from equations (4.5) and (4.6) and the triangle inequality for
$d_{TV}$
.
Lemma 4.2 Let
$\sigma > 0, n \geq 2, \tau = \tau(n)$
and the convex body
$K \in {\mathbb R}^n$
be as in the statement of Lemma 4.1. Let
$v, u \in K$
be points such that
-
•
$\|v - u\|_2 \leq \sigma,$
-
•
$\inf_{z \in \partial K} \|v - z\|_{\infty} > 100 \sigma \ln n,$
and -
•
$\inf_{z \in \partial K} \|u - z\|_{\infty} > 100 \sigma \ln n.$
Define the measures
$\mathcal{G}_u^{(\tau)}, \mathcal{G}_v^{(\tau)}$
and
$\mathcal{G}_{u, \mathbb{I}},\mathcal{G}_{v, \mathbb{I}}$
as above, in terms of
$N(0, \sigma^2)$
. Then,
Proof. Let us fix an
$\mathbb{I} \in \mathcal M_{n,\tau}$
such that
$\mathbb{I}$
has full rank. As
$\mathbb{I}$
has full rank, it follows that
$\sigma^2I \preccurlyeq \Sigma_\mathbb{I}$
and
$\Sigma_\mathbb{I}^{-1} \preccurlyeq \sigma^{-2}I$
. We then have
Here, the first inequality is Pinsker’s inequality (see Fact A.3), the equality is a direct calculation of the Kullback–Leibler divergence between two Gaussian distributions with the same covariance matrix, and the second inequality is from the observation above that
$\Sigma_\mathbb{I}^{-1} \preccurlyeq \sigma^{-2}I$
. On the other hand, from Lemma 4.1(applied separately to
$\mathcal{G}_{u, \mathbb{I}}$
and
$\mathcal{G}_{v, \mathbb{I}}$
) and the triangle inequality for
$d_{TV}$
, we have
$$d_{TV}\!\left({\mathcal{G}_{v, \mathbb{I}}, \mathcal{G}_{u, \mathbb{I}}}\right) \leq 4\, n^{-5} + \sum_{\substack{\mathbb{I} \in \mathcal M_{n,\tau}\\\mathbb{I}\text{ of full rank}}} \lambda_\mathbb{I} d_{TV}\left({\mathcal{G}_{u,\mathbb{I}}, \mathcal{G}_{v,\mathbb{I}}}\right).$$
The claim now follows from equation (4.7) since the sum of the
$\lambda_\mathbb{I}$
over all
$\mathbb{I}\in \mathcal M_{n,\tau}$
of full rank is at most 1.
It follows quite easily from Lemma 4.2 that the multi-step Gaussian chain has the overlap property discussed in Section 3. In the next section, we discuss this in more detail, and compare the Gaussian chain with the CHR chain to obtain a lower bound on the s-conductance of the latter.
5. The s-conductance of CHR
Our goal in this section is to the give a lower bound on the s-conductance of the CHR chain, by comparing it with the Gaussian chain studied in the previous section.
Let the Markov scheme corresponding to CHR be denoted by
$\mathcal{C}$
. We also carry forward the notation
$\mathcal{G}$
and
$\mathcal{G}^{(\tau)}$
for the Gaussian scheme and its iterates, respectively, introduced in the previous section.
Lemma 5.1 There are universal constants
$C> 1$
and
$0 < c < 1$
such that the following is true. Let
$\sigma > 0, n \geq 2, \tau = \tau(n) = {\left\lceil{20n\ln n}\right\rceil}$
be as in the statement of Lemma 4.1, and let
$K \in {\mathbb R}^n$
be a convex body such that for some
$R \geq 1$
, it is the case that
$B_\infty \subseteq K \subseteq R\cdot B_\infty$
. Set
${\epsilon} \,:\!=\, Cn\sigma\ln n$
, and assume that
$\sigma$
is chosen so that
$\epsilon, \sigma < 1/2$
. Define the chain
$\mathcal{G}^{(\tau)}$
as in Section 4, in terms of
$N(0, \sigma^2)$
. Then, for any measurable set S contained in K, such that
${\epsilon} < \pi_K(S)$
we have
Proof. Recall from Definition 3.1 that a Markov scheme
$\mathcal P$
on K is said to have the
$(\epsilon, \delta, \nu)$
-overlap property with respect to K
$^{\prime}$
if K
$^{\prime}$
is a measurable convex subset of K such that (i)
${\mathrm{vol}\left({{K'}}\right)} \geq {\mathrm{vol}\left({{K}}\right)}(1 - \epsilon)$
, and further, (ii) for all
$u, v \in K'$
satisfying
${\Vert {u - v} \Vert_{2}} \leq \delta$
, we have
Set
$C = 100$
. Now, from Proposition 2.4, we see that
${\mathrm{vol}\left({{K_{C\sigma\ln n}}}\right)} \geq (1-C\sigma\ln n)^n{\mathrm{vol}\left({{K}}\right)} \geq (1-\epsilon){\mathrm{vol}\left({{K}}\right)}$
. On the other hand, Lemma 4.2 implies that for any
$u, v \in K_{C\sigma\ln n}$
such that
${\Vert {u - v} \Vert_{2}} \leq \sigma$
, we have
$d_{TV}\left({\mathcal{G}^{(\tau)}(u, \cdot), \mathcal{G}^{(\tau)}(v, \cdot)}\right) \leq 3/4$
. It follows that
$\mathcal{G}^{(\tau)}$
has the
$({\epsilon} , \sigma, 1/4)-$
overlap property with respect to
$K_{C\sigma \ln n}$
.
Now, we apply Lemma 3.2 to the Markov scheme
$G^{(\tau)}$
to see that (note that the
${\Vert {\cdot} \Vert_{2}}$
-diameter of K is at most
$2R\sqrt{n}$
)
Setting
$c = 1/32$
gives us the lemma.
To use the above result for obtaining a lower bound on the s-conductance of the CHR chain, we need the following simple observation comparing the ergodic flows of the single step CHR chain and the single step Gaussian chain (note that what was analysed in the previous lemma was an iterate of the latter).
Observation 5.2 Let
$\sigma > 0, n \geq 2, \tau = \tau(n) = {\left\lceil{20 n \ln n}\right\rceil}\!, R \geq 1$
and the convex body
$K \in {\mathbb R}^n$
be as in the statement of Lemma 5.1. Define the Gaussian scheme
$\mathcal{G}$
as in Section 4, in terms of
$N(0, \sigma^2)$
. Then, for any measurable subset S of K, we have
Proof. The measures
$\mathcal{G}_x$
and
$\mathcal{C}_x$
corresponding to transitions from any point
$x \in S$
for the two schemes
$\mathcal{G}$
and
$\mathcal{C}$
, respectively, are absolutely continuous with respect to each other at all points
$y \neq x$
(in particular, for all
$y \in K - S$
) in the support
$\mathrm{supp}(\mathcal{G}_x)$
of
$\mathcal{G}_x$
. Further, the Radon–Nikodym derivative of
$\mathcal{C}_x$
with respect to
$\mathcal{G}_x$
at any
$y \neq x$
in
$\mathrm{supp}(\mathcal{G}_x)$
is bounded below by
$(\sqrt{2\pi} \sigma/(2R))$
. The claim now follows by integration.
We now have all the ingredients to give a lower bound on the s-conductance of the CHR chain.
Lemma 5.3 Let
$n \geq 2, R \geq 1$
and the convex body
$K \in {\mathbb R}^n$
be as in the statement of Lemma 5.1. Then, for any s satisfying
$0 < s < 1/2$
, the s-conductance
$\Phi_s$
of the CHR chain
$\mathcal{C}$
on K satisfies
Here, the constants implicit in the
$\Omega(\!\cdot\!)$
notation are independent of n, R and K.
Proof. Let the constants C and c be as in the statement of Lemma 5.1. Set
$\sigma = s/(Cn \ln n) \leq 1/2$
, so that the quantity
$\epsilon$
in Lemma 5.1 satisfies
$\epsilon = s < 1/2$
. Let
$\tau(n) = {\left\lceil{20n \ln n}\right\rceil}$
, and define the schemes
$\mathcal{G}$
and
$\mathcal{G}^{(\tau)}$
in terms of
$N(0, \sigma^2)$
, again as in the statement of Lemma 5.1.
Let S be an arbitrary measurable subset of K. Consider the following experiment. Let
$x_0$
be sampled from
$\pi_K$
, and let
$x_0, x_1, \dots, x_\tau$
be a walk of length
$\tau$
according to the scheme
$\mathcal{G}$
. For
$0 \leq i < \tau$
, let
$E_i$
denote the event that
$x_i \in S$
and
$x_{i+1} \in K - S$
. Similarly, let E denote the event that
$x_0 \in S$
and
$x_\tau \in K - S$
. Note that
${\mathbb{P}}[E] \leq \sum_{i=0}^{\tau-1}{\mathbb{P}}[E_i]$
. On the other hand, since
$\mathcal{G}$
is reversible with respect to
$\pi_K$
, we have
$x_i \sim \pi_K$
for each
$0 \leq i < \tau$
, so that
${\mathbb{P}}[E_i] = \Phi_{\mathcal{G}, \pi_K}(S, K - S)$
. Similarly,
${\mathbb{P}}[E] = \Phi_{\mathcal{G}^{(\tau)}, \pi_K}(S, K - S)$
. Thus, we get that
Combining this with Observation 5.2, we therefore obtain
where c
$^{\prime}$
is an absolute positive constant.
Finally, applying Lemma 5.1, we then see that for any measurable subset
${S \subset K}$
such that
$\pi(S) > s$
, we have
Combining this with equation (5.2), we see that for every measurable
$S \subseteq K$
such that
$s < \pi_K(S) \leq 1/2$
, we have
\begin{align} \Phi_{\mathcal{C}, \pi_K}(S, K - S) & \geq \left(\frac{c\cdot c' \cdot \sigma^2}{\tau \cdot R^2\sqrt{n}}\right) \left(\min\left\{{\pi_K(S), \pi_K(K-S)}\right\} - s\right) \nonumber\\ & \geq \left(\frac{c'' s^2}{R^2 n^{3.5} \ln^3 n}\right) \left({\pi_K(S) -s}\right), \end{align}
where c
$^{\prime\prime}$
is an absolute positive constant. The claim follows immediately from equation (5.4).
6. Mixing time of CHR
We now restate and prove our main theorem (Theorem 1.1). Given the lower bound on the s-conductance of the CHR scheme derived in Lemma 5.3, the proof follows immediately from Theorem 2.1
Theorem 6.1 There exists a positive constant C such that the following is true. Fix
$\epsilon \in (0, 1/2), n \geq 2$
and
$R \geq 1$
. Let
$K \subset {\mathbb R}^n$
be a closed convex body such that
$B_\infty \subseteq K \subseteq R\cdot B_\infty$
. Suppose that
$\mu_0$
is absolutely continuous with respect to
$\pi_K$
and that its Radon–Nikodym derivative with respect to
$\pi_K$
is bounded above by
$M \geq 1$
. Then, for any
$k \geq {\left\lceil{\frac{C M^4 R^4 n^7 \ln^6 n\ln(2M/\epsilon)}{{\epsilon}^4}}\right\rceil}$
, we have
$d_{TV}(\mu_{k, \mathcal{C}}, \pi_K) \leq {\epsilon}.$
Proof. Let
$0 < s < 1/2$
be arbitrary. We apply Theorem 2.1 to the CHR scheme
$\mathcal{C}$
and its stationary distribution
$\pi_K$
. Let
$H_s$
be as defined in Theorem 2.1. The absolute continuity constraint imposed on
$\mu_0$
with respect to
$\pi_K$
then implies that
$H_s \leq M\cdot s$
. Let
$\mu_k$
denote the distribution after k iterations of
$\mathcal{C}$
starting with the initial distribution
$\mu_0$
. Theorem 2.1, along with the bound on
$\Phi_s(\mathcal{C})$
obtained in Lemma 5.3 then implies that (for some absolute constant c)
Choosing
$s = \epsilon/(2M)$
, we then see that for any
$k \geq {\left\lceil{\frac{16 M^4 R^4n^7 \ln^6 n\ln(2M/\epsilon)}{c{\epsilon}^4}}\right\rceil}$
, we have
$d_{TV}(\mu_{k, \mathcal{C}}, \pi_K) \leq {\epsilon}.$
Acknowledgements
HN and PS acknowledge support from the Department of Atomic Energy, Government of India, under project no. RTI4001, the Ramanujan Fellowship of SERB, and the Infosys foundation, through its support for the Infosys-Chandrasekharan virtual center for Random Geometry. Additionally, PS acknowledges support from Adobe Systems Incorporated via a gift to TIFR, and HN acknowledges support from a Swarna Jayanti fellowship. The contents of this paper do not necessarily reflect the views of the funding agencies listed above.
A. Some standard inequalities
In this section, we collect a few standard inequalities that we use at various places in the main body of the paper.
Fact A.1 (Chernoff bound, see, e.g., [19, Theorem 4.1]) Let
$X_1, X_2, \dots, X_n$
be independent Bernoulli random variables, and let
$X \,:\!=\, \sum_{i=1}^n X_i$
. Define
$\mu \,:\!=\, {\mathbb E}[X]$
. Then, for any
$\delta > 0$
,
In particular, when
$\delta = 1$
,
${\mathbb{P}}[X > 2\mu] < \exp(\!-\mu/3).$
Fact A.2 (Gaussian tail bound) Fix
$t \geq \sigma > 0$
and let
$X \sim N(0, \sigma^2)$
. Then,
Proof. Define
$s \,:\!=\, t/\sigma \geq 1$
. Let
$Z \sim N(0, 1)$
. Then,
\begin{align*} {\mathbb{P}}[{\left|{X}\right|} \geq t] & = {\mathbb{P}}[{\left|{Z}\right|} \geq s] = \sqrt{\frac{2}{\pi}}\int\limits_{s}^{\infty}\exp(-x^2/2)dx \leq \sqrt{\frac{2}{\pi s^2}}\int\limits_{s}^{\infty}x\exp(-x^2/2)dx \nonumber \\ &= \sqrt{\frac{2}{\pi s^2}}\exp(-s^2/2). \end{align*}
The claimed inequality follows since
$s \geq 1$
.
Fact A.3 (Pinsker’s inequality, see, e.g., [1, Theorem 4.19]) Let P and Q be probability measures on the same
$\sigma$
-field, such that P is absolutely continuous with respect to Q. Let
$D_{KL}(P\Vert Q)$
denote the Kullback–Leibler divergence between P and Q. Then







