2.1 Background on nilpotent Lie groups and Lie algebras

Let N be a connected, simply connected k-step nilpotent Lie group with Lie algebra $\mathfrak {n}$ with $n+1$ generators. Let $\Gamma $ be a cocompact lattice in N. The quotient $N/\Gamma := M$ is then a compact nilmanifold on which the left action of $N$ is given by translations. Denote by $\mu $ the N-invariant measure on M.

For $j = 1,\ldots , k$ , let $\mathfrak {n}_j$ denote the descending central series of $\mathfrak {n}$ :

(4) $$ \begin{align} \mathfrak{n}_1 = \mathfrak{n}, \mathfrak{n}_2 = [\mathfrak{n},\mathfrak{n}], \ldots, \mathfrak{n}_j = [\mathfrak{n}_{j-1},\mathfrak{n}], \ldots, \mathfrak{n}_k \subset Z(\mathfrak{n}) \end{align} $$

where $Z(\mathfrak {n})$ is the center of $\mathfrak {n}$ . In this setting, there exists a strong Malcev basis through the filtration $(\mathfrak {n}_{i})_{i=1}^k$ strongly based at the lattice $\Gamma $ (see [Reference Corwin and GreenleafCG90]). That is, given a basis

$$ \begin{align*}\mathcal{F} = \{\xi^{(1)}, \eta_1^{(1)},\ldots, \eta_{n_1}^{(1)}, \ldots,\eta_1^{(k)},\ldots \eta_{n_k}^{(k)} \}\end{align*} $$

with $\xi := \xi ^{(1)} \in \mathfrak {n}_1 \backslash \mathfrak {n}_2$ and $\eta _i^{(l)} \in \mathfrak {n}_l \backslash \mathfrak {n}_{l+1} $ for $i = 1,\ldots , n_l$ , we have the following assertions.

1. (1) If we drop the first l elements of the basis, we obtain a basis of a subalgebra $\mathfrak {n}$ of codimension l.

2. (2) For each j, the elements in order $\eta _1^{(j)},\ldots \eta _{n_j}^{(j)},\ldots \eta _1^{(k)},\ldots \eta _{n_k}^{(k)}$ form a basis of an ideal $\mathfrak {n}_j$ of $\mathfrak {n}$ .

3. (3) The lattice $\Gamma $ is generated by $\{x, y_{1}^{(1)}, \ldots , y_{n_k}^{(k)} \}$ with

   $$ \begin{align*}x := \exp(\xi),\quad y_{n_i}^{(j)}:= \exp(\eta_{n_i}^{(j)}).\end{align*} $$

For any nilpotent Lie algebra $\mathfrak {n}$ , there exists a codimension- $1$ subalgebra $\mathfrak {I}$ where

$$ \begin{align*}\mathfrak{n} = \mathbb{R} \xi\oplus \mathfrak{I}. \end{align*} $$

Then $\mathfrak {I}$ is an ideal and $[\mathfrak {n},\mathfrak {n}] \subseteq \mathfrak {I}$ (see [Reference HumphreysH73, Ch. 3, p. 12] and [Reference Corwin and GreenleafCG90, Lemma 1.1.8]). For convenience, we write dimension $a = \dim (\mathfrak {I}) = n_1+\cdots + n_k$ and set $n = n_1$ .

2.2 Nilmanifolds and nilflows

Every nilmanifold M is a fiber bundle over a torus. In fact, the group $N^{ab} = N/[N,N]$ is abelian, connected and simply connected, hence isomorphic to $\mathbb {R}^{n+1}$ , and $\Gamma ^{ab} = \Gamma /[\Gamma , \Gamma ]$ is a lattice in $N^{ab}$ . Thus, we have a natural projection $pr_1: M \rightarrow \mathbb {T}^{n+1}$ .

We introduce two fibrations of the nilmanifold M. Let $M_2 \simeq N_2/\Gamma _2$ with $N_2 = \exp (\mathfrak {n}_2)$ and its lattice $\Gamma _2$ . Then there exists an exact sequence

(5) $$ \begin{align} 0 \rightarrow M_2 \rightarrow M \xrightarrow {pr_1} \mathbb{T}^{n+1} \rightarrow 0. \end{align} $$

Another fibration arises from the canonical homomorphism $N \rightarrow N/N' \approx \langle \exp \xi \rangle $ . For $\theta \in \mathbb {T}^1$ , the fiber $M_\theta ^a = {pr_2}^{-1}(\theta )$ is a local section of the nilflow on M:

(6) $$ \begin{align} 0 \rightarrow M_\theta^a \rightarrow M \xrightarrow {pr_2} \mathbb{T}^{1} \rightarrow 0. \end{align} $$

On the nilmanifold M, the nilflow $\phi ^t_X$ generated by $X \in \mathfrak {n}$ is the flow obtained by the restriction of this action to the one-parameter subgroup $(\exp tX)_{t \in \mathbb {R}}$ of N, with

$$ \begin{align*}\phi^t_X(x) = x\exp(tX), \quad x \in M, \ t \in \mathbb{R}.\end{align*} $$

The projection $\bar {X}$ of X is the generator of a linear flow $\psi _{\bar {X}} := \{\psi ^t_X\}_{t \in \mathbb {R}}$ on $\mathbb {T}^{n+1} \approx \mathbb {R}^{n+1} \backslash \bar {\Gamma }$ defined by

$$ \begin{align*}\psi^t_{\bar{X}}(x_1,\ldots,x_{n+1}) = (x_1+tv_1,\ldots ,x_{n+1}+tv_{n+1}).\end{align*} $$

The canonical projection $ {pr_1}:M \rightarrow \mathbb {T}^{n+1}$ intertwines the flows $\phi ^t_X$ and $\psi ^t_{\bar {X}}$ .

We recall the following result.

Consider the set of indices

$$ \begin{align*} J &:= \{(i,j) \mid 1 \leq i \leq n_j, 1 \leq j \leq k \},\\ J^+ &:= \{(i,j) \mid 1 \leq i \leq n_1, \ j=1 \},\\ J^{-} &:= \{(i,j) \mid 1 \leq i \leq n_j ,\ j> 1 \},\\ J^{-2} &:= \{(i,j) \mid 1 \leq i \leq n_j ,\ j > 2 \}. \end{align*} $$

Let $\alpha = (\alpha _i^{(j)}) \in \mathbb {R}^{J}$ and $X:= X_{\alpha }$ be a vector field on M defined

(7) $$ \begin{align} X_{\alpha}:= \log\left[x^{-1}\exp\left(\sum_{(i,j) \in J}\alpha^{(j)}_i\eta^{(j)}_i\right)\right]\!, \quad x = \exp(\xi), \end{align} $$

and equivalently we write

(8) $$ \begin{align} X_{\alpha}:= -\xi+\sum_{(i,j) \in J}\alpha^{(j)}_i\eta^{(j)}_i. \end{align} $$

For $\theta \in \mathbb {T}^1$ , let $M_\theta ^a = {pr_2}^{-1}(\theta )$ denote a fiber over $\theta \in \mathbb {T}^1$ of the fibration $pr_2$ . The transverse section $M_\theta ^a$ of the nilflow $\{\phi ^t_{X_{\alpha }}\}_{t\in \mathbb {R}}$ corresponds to

$$ \begin{align*} &\bigg\{ \Gamma \exp(\theta \xi) \exp\bigg( \sum_{i=1}^a{s_i \eta_i } \bigg) \mid (s_i)_{i=1}^a \in \mathbb{R}^a \bigg\}\\&\quad= \bigg\{ \Gamma \exp\bigg(e^{ad(\theta \xi)} \sum_{i=1}^a{s_i \eta_i } \bigg)\exp(\theta \xi) \mid (s_i)_{i=1}^a \in \mathbb{R}^a \bigg\}. \end{align*} $$

Lemma 2.3. The flow $(\phi ^t_{X_{\alpha }})_{t\in \mathbb {R}}$ on M is isomorphic to the suspension of its first return map $\Phi _{\alpha , \theta }: M_\theta ^a \rightarrow M_\theta ^a$ . For every $(i,j) \in J$ and $r \geq 1$ , there exists a polynomial $p_{i,r}^{(j)}(\alpha , \mathbf {s})$ for $\mathbf {s} = (s_i)_{i=1}^a \in \mathbb {R}^a$ such that the rth return map $\Phi _{\alpha , \theta }^r$ is written as follows.

On the coordinates of  $\ \mathbf {s} = (s^{(j)}_i)$ for $\Gamma \exp (\theta \xi )\exp (\sum _{(i,j)\in J} s^{(j)}_i\eta ^{(j)}_i) \in M_\theta ^a$ ,

(9) $$ \begin{align} \Phi_{\alpha, \theta}( \mathbf{s} ) = \Gamma \exp(\theta \xi)\exp &\bigg(\sum_{(i,j) \in J} (s^{(j)}_i + \alpha^{(j)}_i)\eta^{(j)}_i \notag \\ &\quad \ {} + \bigg[\sum_{(i,j) \in J} s^{(j)}_i\eta^{(j)}_i , X_{\alpha}\bigg] + \sum_{(i,j) \in J^{2-}} p_i^{(j)}(\alpha, \mathbf{s})\eta^{(j)}_i\bigg), \end{align} $$

and for $r \in \mathbb {N}$ ,

(10) $$ \begin{align} \Phi^r_{\alpha, \theta}( \mathbf{s}) = \Gamma \exp(\theta \xi)\exp &\bigg(\sum_{(i,j) \in J} (s^{(j)}_i + r\alpha^{(j)}_i)\eta^{(j)}_i \notag \\ &\quad + \bigg[\sum_{(i,j) \in J} s^{(j)}_i\eta^{(j)}_i , rX_{\alpha}\bigg] + \sum_{(i,j) \in J^{2-}} p_{i,r}^{(j)}(\alpha, \mathbf{s})\eta^{(j)}_i\bigg). \end{align} $$

Proof. By (7), we have

$$ \begin{align*} \exp\bigg(\sum_{(i,j)\in J } s^{(j)}_i\eta^{(j)}_i\bigg)\exp(X_{\alpha}) & = \exp\bigg(\sum_{(i,j) \in J} s^{(j)}_i\eta^{(j)}_i\bigg)x^{-1} \exp\bigg(\sum_{(i,j) \in J} \alpha^{(j)}_i\eta^{(j)}_i\bigg)\\ & = x^{-1}\exp (e^{\text{ad}(\xi)}{\sum_{(i,j) \in J} s^{(j)}_i\eta^{(j)}_i }) \exp\bigg(\sum_{(i,j) \in J} \alpha^{(j)}_i\eta^{(j)}_i\bigg). \end{align*} $$

By the Baker–Campbell–Hausdorff formula, there exist polynomials $p_i^{(j)}(\alpha , s)$ with

$$ \begin{align*} \exp\bigg(\sum_{(i,j)\in J } s^{(j)}_i\eta^{(j)}_i\bigg)\exp(X_{\alpha}) & = x^{-1}\exp\bigg( \sum_{(i,j) \in J} (s^{(j)}_i + \alpha^{(j)}_i)\eta^{(j)}_i \\ &\quad + \bigg[\sum_{(i,j) \in J} s^{(j)}_i\eta^{(j)}_i , X_{\alpha}\bigg] + \sum_{(i,j) \in J^{2-}} p_i^{(j)}(\alpha, \textbf{s})\eta^{(j)}_i\bigg). \end{align*} $$

Since $x \in \Gamma $ , we conclude

$$ \begin{align*} &\Gamma \exp(\theta \xi)\exp\bigg(\sum_{(i,j) \in J} s^{(j)}_i\eta^{(j)}_i\bigg)\exp(X_{\alpha})\\ &\quad= \Gamma\exp(\theta \xi) \exp\bigg( \sum_{(i,j) \in J} (s^{(j)}_i + \alpha^{(j)}_i)\eta^{(j)}_i + \bigg[\sum_{(i,j) \in J} s^{(j)}_i\eta^{(j)}_i , X_{\alpha}\bigg]\\ &\qquad + \sum_{(i,j) \in J^{2-}} p_i^{(j)}(\alpha, \textbf{s})\eta^{(j)}_i\bigg). \end{align*} $$

The formula implies that $t =1$ is a return time of the restriction of the flow to $M_\theta ^a \subset M$ . The formula for $r \in \mathbb {N}$ follows by induction.

2.3 Kirillov theory and classification

Kirillov theory yields the complete classification of irreducible unitary representations of N. All the irreducible unitary representations of nilpotent Lie groups are parametrized by the coadjoint orbits $\mathcal {O} \subset \mathfrak {n}^*$ . A polarizing (or maximal subordinate) subalgebra for $\Lambda \in \mathfrak {n}^*$ is a maximal isotropic subspace $\mathfrak {m} \subset \mathfrak {n}$ which is a subalgebra of $\mathfrak {n}$ . It is well known that for any $\Lambda \in \mathfrak {n}^*$ there exists a polarizing subalgebra $\mathfrak {m}$ for a nilpotent Lie algebra $\mathfrak {n}$ (see [Reference Corwin and GreenleafCG90, Theorem 1.3.3]). Let $\mathfrak {m}$ be a polarizing subalgebra for given linear form $\Lambda \in \mathfrak {n}^*$ . Then the character $\chi _{\Lambda ,\mathfrak {m}}: \exp {\mathfrak {m}}\rightarrow S^1$ is defined by

$$ \begin{align*}\chi_{\Lambda,\mathfrak{m}}(\exp Y) = e^{2\pi \iota \Lambda(Y) }.\end{align*} $$

Given the pair $(\Lambda ,\mathfrak {m})$ , we associate the unitary representation

$$ \begin{align*}\pi_{\Lambda} = \text{Ind}_{\exp \mathfrak{m}}^{N}(\chi)\end{align*} $$

where the induced representation $\pi _{\Lambda }$ is defined by

$$ \begin{align*}\pi_{\Lambda}(x)f(g) = f(g\cdot x) \quad \text{for } x \in N \text{ and } f \in H_{\pi_{\nu,\mathfrak{m}}}.\end{align*} $$

These unitary representations are irreducible up to equivalence, and all unitary irreducible representations are obtained in this way. It is known that $\Lambda $ and $\Lambda '$ belong to the same coadjoint orbit if and only if $\pi _{\Lambda ,\mathfrak {m}} $ and $\pi _{\Lambda ',\mathfrak {m}'} $ are unitarily equivalent and $\pi _{\Lambda ,\mathfrak {m}}$ is irreducible whenever $\mathfrak {m}$ is maximal subordinate for $\Lambda $ . We write $\pi _\Lambda \simeq \pi _{\Lambda '}$ if $\Lambda $ and $\Lambda '$ are in the same coadjoint orbit.

Since the action of N on M preserves the measure $\mu $ , we obtain a unitary representation $\pi $ of N. The regular representation of $L^2(M)$ of N decomposes as a countable direct sum (or direct integral) of the irreducible, unitary representation $H_\pi $ , which occurs with at most finite multiplicity

(11) $$ \begin{align} L^2(M,d\mu) = \bigoplus_{\pi} H_{\pi}. \end{align} $$

The derived representation $\pi _{*}$ of a unitary representation $\pi $ of N on a Hilbert space $H_\pi $ is a representation of the Lie algebra $\mathfrak {n}$ on $H_\pi $ defined as follows. For every $X \in \mathfrak {n}$ ,

(12) $$ \begin{align} \pi_{*}(X) = \lim_{t\rightarrow 0}(\pi (\exp tX) - I )/t. \end{align} $$

We recall that a vector $v \in H_\pi $ is of $C^\infty $ -vectors in $H_\pi $ for representation $\pi $ if the function $g \in N \mapsto \pi (g)v \in H_\pi $ is of class $C^\infty $ as a function on N with values in a Hilbert space.

Definition 2.4. The space of Schwartz functions on $\mathbb {R}$ with values of $C^\infty $ vectors for the representation $\pi '$ on $H'$ is denoted $\mathcal {S}(\mathbb {R}, C^\infty (H'))$ . It is endowed with the Fréchet topology induced by the family of seminorms

$$ \begin{align*}\{\|\cdot\|_{i,j,Y_1,Y_2,\ldots,Y_m} \mid i,j,m\in \mathbb{N} \text{ and } Y_1,\ldots,Y_m \in \mathfrak{n}'\}\end{align*} $$

and defined as follows: for all $f \in \mathcal {S}(\mathbb {R}, C^\infty (H'))$ ,

$$ \begin{align*}\|f\|_{i,j,Y_1,Y_2,\ldots,Y_m} := \sup_{t \in \mathbb{R}} \|(1+t^2)^{j/2}\pi_{*}'(Y_1)\ldots\pi_{*}'(Y_m)f^{(i)}(t)\|_{H'},\quad t\in \mathbb{R}.\end{align*} $$

Lemma 2.5. [Reference Flaminio and ForniFF07, Lemma 3.4]

As a topological vector space

$$ \begin{align*}C^\infty(H_\pi) = \mathcal{S}(\mathbb{R}, C^\infty(H'))\end{align*} $$

where $\mathcal {S}(\mathbb {R}, C^\infty (H'))$ is Schwartz space.

Suppose that $\mathfrak {n} = \mathbb {R} X\oplus \mathfrak {I} $ with its codimension- $1$ ideal $\mathfrak {I}$ , and $N = \mathbb {R} \ltimes N'$ with a normal subgroup $N'$ of N. Let $\pi '$ be a unitary irreducible representation of $N'$ on a Hilbert space $H'$ . Each irreducible representation $H_\pi $ is unitarily equivalent to $L^2(\mathbb {R},H',\,dx)$ , and the derived representation of $\pi _*$ of the induced representation $\pi = \text {Ind}_{N'}^N(\pi ')$ has the following description.

For $f \in L^2(\mathbb {R},H',\,dx)$ , the group $\mathbb {R}$ acts by translations and its representation is polynomial in the variable x. For any $Y \in \mathfrak {n}'$ ,

(13) $$ \begin{align} (\pi_{*}(Y)f)(x) &= \sum_{j=0}^{d_Y} \frac{1}{j!} \pi^{\prime}_{*}(\text{ad}_X^j(Y)f(x)\nonumber\\ &= \iota P_Y(x)f(x) = \iota\sum_{j=0}^{d_Y} \frac{1}{j!} (\Lambda\circ\text{ad}_X^jY)x^jf(x). \end{align} $$

We define its degree $d_Y \in \mathbb {N}$ with respect to the representation $\pi _{*}(Y)$ to be the degree of the polynomial. Let $(d_1,\ldots , d_a)$ be the degrees of the elements $(Y_1, \ldots , Y_a)$ , respectively. The degree of the representation $\pi $ is defined as the maximum of the degrees of the elements of any basis.

3.1 Distributions and Sobolev space

Let $L^2(M)$ be the space of complex-valued, square integrable functions on M. Given an ordered basis $\mathcal {F}$ of $\mathfrak {n}$ , the transverse Laplace–Beltrami operator is a second-order differential operator defined by

$$ \begin{align*}\Delta_{\mathcal{F}} = -\sum_{i=1}^a Y_i^2,\quad Y_i \in \mathfrak{I}.\end{align*} $$

For any $\sigma \geq 0$ , let $|\cdot |_{\sigma ,\mathcal {F}}$ be the transverse Sobolev norm defined as follows: for all functions $f \in C^\infty (M)$ , let

$$ \begin{align*}|f|_{\sigma,\mathcal{F}} := \|(I+\Delta_{\mathcal{F}})^{({\sigma}/{2})}f\|_{L^2(M)}.\end{align*} $$

Equivalently,

$$ \begin{align*}|f|_{\sigma,\mathcal{F}} = \bigg(\|f\|^2_2+ \sum_{1\leq m \leq \sigma}\|Y_{j_1}\cdots Y_{j_m}f\|_2^2\bigg)^{{1}/{2}}, \quad Y_{j_m} \in \mathfrak{I}.\end{align*} $$

The completion of $C^\infty (M)$ with respect to the norm $|\cdot |_{\sigma ,\mathcal {F}}$ is denoted $W^\sigma (M,\mathcal {F})$ and the distributional dual space (as a space of functional with values in $H'$ ) to $W^\sigma (M)$ is denoted

$$ \begin{align*}W^{-\sigma}(M,\mathcal{F}): = (W^{\sigma}(M,\mathcal{F}))'.\end{align*} $$

We denote by $C^\infty (H_\pi )$ the space of $C^\infty $ vectors of the irreducible unitary representation $\pi $ . Following the notation in (11), let $W^\sigma (H_\pi ) \subset H_\pi $ be the Sobolev space of vectors, endowed with the Hilbert space norm in the maximal domain of the essential self-adjoint operator $(I+\pi _{*}( \Delta _{\mathcal {F}}))^{{\sigma }/{2}}$ . That is to say, for every $f \in C^\infty (H_\pi )$ and $\sigma>0$ ,

$$ \begin{align*}|f|_{\sigma,\mathcal{F}} := \bigg( \int_{\mathbb{R}} \|(1+\pi_{*}(\Delta_{\mathcal{F}}))^{{\sigma}/{2}}f(x)\|^2_{H'}\,dx \bigg)^{1/2}\end{align*} $$

where $\pi _{*}(\Delta _{\mathcal {F}})$ is determined by derived representations.

3.2 A priori estimates

The distributional obstruction to the existence of solutions of the cohomological equation

$$ \begin{align*}Xu = f, \quad f\in C^\infty(H_\pi),\end{align*} $$

in an irreducible unitary representation $H_\pi $ is the normalized X-invariant distribution.

Definition 3.1. For any $X \in \mathfrak {n}$ , the space of X-invariant distributions for the representation $\pi $ is the space $\mathcal {I}_X(H_\pi )$ of all distributional solutions $D \in D'(H_\pi )$ of the equation $\pi _{*}(X)D = XD = 0$ . Let

$$ \begin{align*}\mathcal{I}^\sigma_X(H_\pi): = \mathcal{I}_X(H_\pi) \cap W^{-\sigma}(H_\pi)\end{align*} $$

be the subspace of invariant distributions of order at most $\sigma \in \mathbb {R}^+$ .

By [Reference Flaminio and ForniFF07], each invariant distribution D has Sobolev order equal to $1/2$ , that is, $D \in W^{-\sigma }(H_\pi )$ for any $\sigma> 1/2$ .

For all $\sigma> 1$ , let $\mathcal {K}^\sigma (H_\pi ) = \{f \in W^\sigma (H_\pi ) \mid D(f) = 0 \in C^\infty (H_\pi ), \text { for any } D \in W^{-\sigma }(H_\pi ) \}$ be the kernel of the X-invariant distribution on the Sobolev space $W^\sigma (H_\pi )$ . Green’s operator $G_X:C^\infty (H_\pi ) \rightarrow C^\infty (H_\pi )$ with

$$ \begin{align*}G_Xf(t) = \int_{-\infty}^{t}f(s)\,ds\end{align*} $$

is well defined on the kernel of distribution $\mathcal {K}^\infty (H_\pi )$ on $C^\infty (H_\pi )$ . In fact, for any $f \in \mathcal {K}^\infty (H_\pi )$ , we have $\int _{\mathbb {R}}f(t)\,dt = 0 \in C^\infty (H')$ and

$$ \begin{align*}G_Xf(t) = \int_{-\infty}^{t}f(s)\,ds = -\int_{t}^{\infty}f(s)\,ds \in C^\infty(\mathbb{R},H').\end{align*} $$

Now we define the generalized (complex-valued) invariant distribution on smooth vector $C^\infty (H_\pi )$ .

Lemma 3.2. The invariant distribution is generalized in the following sense. For every invariant distribution D, there exists a linear functional $\ell : C^\infty (H_\pi ) \rightarrow \mathbb {C}$ such that for every function $f \in C^\infty (H_\pi ) \subset L^2(\mathbb {R}, H')$ ,

(14) $$ \begin{align} D(f) = \int_{\mathbb{R}} \ell ( f(t) )\,dt. \end{align} $$

Furthermore, $\ell \in W^{-s}(H_\pi )$ for $s> 1/2$ and

(15) $$ \begin{align} \int_{\mathbb{R}} \ell ( f(t) )\,dt = \ell \bigg( \int_{\mathbb{R}} f(t)\,dt \bigg). \end{align} $$

Proof. We construct a linear functional $\ell $ in (14) as follows. Let $\chi \in C_0^\infty (\mathbb {R})$ be a smooth function with compact support with unit integral over $\mathbb {R}$ . Given an invariant distribution $D \in \mathcal {I}_X(H_\pi )$ , let us define $\ell (v) = D (f_v)$ for $f_v = \chi v$ and $v\in C^\infty (H')$ .

Firstly, we prove that $\ell $ is well defined. Let $\chi _1 \neq \chi _2 \in C_0^\infty (\mathbb {R})$ be functions with compact support such that $\int _{\mathbb {R}} \chi _1(t)\,dt = \int _{\mathbb {R}} \chi _2(t)\,dt = 1$ . Note that there exists $\psi \in C_0^\infty (\mathbb {R})$ such that $\chi _1 - \chi _2 = \psi '$ with $\psi (t) = \int _{-\infty }^t (\chi _1(x) - \chi _2(x))\,dx.$

Then we have

$$ \begin{align*}\chi_1(t) v - \chi_2(t) v = \frac{d}{dt}(\psi(t)v) = \pi_*(X)(\psi(t)v) \in C^\infty(H_\pi),\end{align*} $$

and $\chi _1(t) v - \chi _2(t) v$ is an X-coboundary for every $v \in C^\infty (H')$ . Hence, $D(\chi _1(t) v - \chi _2(t) v) = 0$ , which implies that $D(\chi _1(t)v) = D(\chi _2(t)v)$ . Therefore, $\ell (v)$ does not depend on the choice of $\chi $ and the functional $\ell $ is well defined.

Next, we verify that $\ell $ is a distribution on $C^\infty (H')$ . It suffices to prove that $\ell $ is bounded and continuous. For $v \in C^\infty (H')$ and $s> 1/2$ ,

$$ \begin{align*}|\ell(v)| = |D(\chi(t) {v})| \leq \|D\|_{-s}\|\chi(t)({v})\|_{W^s(H')}.\end{align*} $$

In the representation, $\pi _*(Y_i)$ acts as multiplication of the polynomial $p_i(t)$ on $L^2(\mathbb {R},H')$ . By definition of the Sobolev norm in representation, there exists a non-zero constant

$$ \begin{align*}C:= C(\chi, p_1,\ldots, p_s) = \displaystyle \max_{\substack {t \in \mathbb{R}, j_1 + \cdots j_d = s \\ 0 \leq j_i \leq s, 1\leq i \leq d \leq s}}\{\chi(t)p^{j_1}_1(t)\cdots p^{j_d}_d(t)\}\end{align*} $$

such that

$$ \begin{align*}\|\chi(t)({v})\|_{W^s(H_{\pi})} \leq C\|{v}\|_{W^s(H')}.\end{align*} $$

Hence, for $\ell \in W^{-s}(H')$ ,

$$ \begin{align*}\|\ell\|_{-s}: = \sup_{\|v\| = 1} \dfrac{|\ell(v)|}{\|v\|_{W^s(H')}} \leq C \|D\|_{-s}.\end{align*} $$

Therefore, $\ell $ is continuous on $C^\infty (H')$ .

To prove equality (15), for any $f \in C^\infty (H_\pi )$ , we observe $f(t) - \chi (t)( \int _{\mathbb {R}} f(x)\,dx )$ has zero average, hence it is a coboundary with smooth transfer function. Since distribution D is invariant under translation, we obtain

$$ \begin{align*}D\bigg(f - \chi(t)\bigg( \int_{\mathbb{R}} f(x)\,dx \bigg)\bigg) = 0\end{align*} $$

and

$$ \begin{align*}D(f) = D \bigg(\chi(t) \int_{\mathbb{R}} f(x)\,dx \bigg) = \ell \bigg( \int_{\mathbb{R}} f(t)\,dt \bigg).\end{align*} $$

Furthermore,

$$ \begin{align*}\int_{\mathbb{R}} \ell (f(t))\,dt &= \int_{\mathbb{R}} D(f_\chi(t))\,dt = \int_{\mathbb{R}} \bigg(\int_{\mathbb{R}} \chi(x)f(t)\,dx\bigg)\,dt\\& = \int_{\mathbb{R}} f(t) \bigg(\int_{\mathbb{R}} \chi(x)\,dx\bigg)\,dt.\end{align*} $$

Since D is an invariant distribution for translation and $\int _{\mathbb {R}} \chi = 1$ , we conclude

$$ \begin{align*} \int_{\mathbb{R}} \ell (f(t))\,dt = \int_{\mathbb{R}} f(t)\,dt = D(f) = \ell \bigg( \int_{\mathbb{R}} f(t)\,dt \bigg).\\[-42pt] \end{align*} $$

Let $\mathcal {O}$ be any coadjoint orbit of maximal rank. For all $(X,Y)\in \mathfrak {n} \times \mathfrak {n}_{k-1}$ and $\Lambda \in \mathcal {O}$ , the skew-symmetric bilinear form

$$ \begin{align*}B_\Lambda(X,Y) = \Lambda([X,Y])\end{align*} $$

does not depend on the choice of linear form $\Lambda \in \mathcal {O}$ .

Let

(16) $$ \begin{align} &\delta_{\mathcal{O}}(X,Y) := |B_\Lambda(X,Y)|\quad \text{for any } \Lambda \in \mathcal{O}, \nonumber\\ &\delta_{\mathcal{O}}(X) := \max\{ \delta_{\mathcal{O}}(X,Y) \mid Y \in \mathfrak{n}_{k-1} \text{ and } \|Y\|=1 \}. \end{align} $$

Here we recall estimates for Green’s operator.

Lemma 3.3. [Reference Flaminio and ForniFF07, Lemma 2.5]

Let $X \in \mathfrak {n}$ and $Y \in \mathfrak {n}_{k-1}$ be any operator such that $B_\Lambda (X,Y) \neq 0$ . The derived representation $\pi _*$ of the Lie algebra $\mathfrak {n}$ satisfies

(17) $$ \begin{align} \pi_{*}(X) = \frac{d}{dx}, \quad \pi_{*}(Y) = 2\pi \iota B_\Lambda(X,Y) x \text{Id}_{H'} \text{ on }L^2(\mathbb{R}, H',\,dx). \end{align} $$

Theorem 3.4. [Reference Flaminio and ForniFF07, Theorem 3.6]

Let ${\delta _{\mathcal {O}}}:=\delta _{\mathcal {O}}(X)>0$ , and let $\pi $ be an irreducible representation of $\mathfrak {n}$ on a Hilbert space $H_\pi $ . If $f \in W^s(H_\pi )$ , $s>1$ , and $D(f) = 0$ for all $D \in \mathcal {I}_X(H_\pi )$ , then $G_Xf \in W^r(H_\pi )$ , for all $r < (s-1)/k$ , and there exists a constant $C := C(X,k,r,s)$ such that

$$ \begin{align*}|{ G_Xf}|_{r,\mathcal{F}} \leq C \max\{1,\delta_{\mathcal{O}}^{-(k-1)r-1)} \} |{f}|_{s,\mathcal{F}}.\end{align*} $$

3.3 Rescaling method

The renormalization dynamics is defined as the action of the diagonal subgroup of the Lie group on the deformation space. Let $\rho : = (\rho _1,\ldots , \rho _a) \in (\mathbb {R}^{+})^a$ be any vector with rescaling condition $\sum _{i=1}^a {\rho _i} = 1$ . Then there exists a one-parameter subgroup $\{A_t^{\rho } \}$ of the Lie group of $SL(a+1,\mathbb {R})$ defined as follows:

(18) $$ \begin{align} A_t^{\rho}(X,Y_1,\ldots, Y_a) = (e^tX, e^{-\rho_1t}Y_1, \ldots, e^{-\rho_at} Y_a). \end{align} $$

The renormalization group $\{A_t^{\rho } \}$ preserves the set of all adapted bases. However, it is not a group of automorphisms of the Lie algebra since generated elements in the center $Z(\mathfrak{n})$ are not preserved in higher steps. Therefore, on higher-step nilmanifolds, the dynamics induced by the renormalization group on the deformation space is trivial (it has no recurrent orbits).

Let $(d_1,\ldots , d_i)$ be the degrees of the elements $(Y_1, \ldots , Y_i)$ , respectively. For any $\rho = (\rho _1, \ldots , \rho _a ) \in \mathbb {R}^a$ , let

(19)

For all $i = 1, \ldots , a$ , denote

(20) $$ \begin{align} \Lambda_i^{(j)}(\mathcal{F}) := (\Lambda\circ\text{ad}^j(X))(Y_i). \end{align} $$

This is the coefficient appearing in (13). Set

(21) $$ \begin{align} |\Lambda(\mathcal{F})| :=\sup_{(i,j): 1\leq i \leq a, 0 \leq j \leq d_i}\bigg|\frac{\Lambda_i^{(j)}(\mathcal{F})}{j!}\bigg|. \end{align} $$

Let $\mathfrak {U}(\mathfrak {n})$ be the enveloping algebras of $\mathfrak {n}$ . The generator $\delta $ is the derivation on $\mathfrak {U}(\mathfrak {n}')$ obtained by extending the derivation $\text {ad}(X)$ of $\mathfrak {n}'$ to $\mathfrak {U}(\mathfrak {n}')$ . From the nilpotency of $\mathfrak {n}$ it follows that for any $L \in \mathfrak {U}(\mathfrak {n}')$ there exists a first integer $[L]$ such that $\delta ^{[L]+1}L = 0$ .

Recall that $\mathfrak {I}$ is a codimension- $1$ ideal of $\mathfrak {n}$ used in §2.3.

Proof. Firstly, we fix elements X and Y as stated in Lemma 3.3. For convenience, we normalize the constant of $\pi _*(Y)$ by 1. That is, there exists $X, Y \in \mathfrak {n}$ such that

$$ \begin{align*}\pi_{*}(X) = \frac{d}{dx}, \quad \pi_{*}(Y) = x.\end{align*} $$

We now replace the expansion of $\pi _{*}(L) := \sum _{j=0}^i \frac {1}{j!} \Lambda _L^{(j)}(\mathcal {F})x^j$ by choosing elements $Q_i$ in the enveloping algebra $\mathfrak {U}(\mathfrak {n})$ .

For the coefficient of top degree, denote $Q_i = ({1}/{i!})\text {ad}_X^i(L) \in \mathfrak {n}$ . For degree $i-1$ , we set $Q_{i-1} = \text {ad}_X^{i-1}(L) - Q_iY \in \mathfrak {U}(\mathfrak {n})$ such that

$$ \begin{align*}\pi_{*}(Q_{i-1}) = \pi_{*}(\text{ad}_X^{i-1}(L)) - \pi_{*}(Q_i)\pi_{*}(Y) = {\Lambda_L^{(i-1)}(\mathcal{F})}.\end{align*} $$

Repeating this process up to degree 0, there exists $ Q_{l} \in \mathfrak {U}(\mathfrak {n})$ such that for $0 < l <i$ ,

$$ \begin{align*}\pi_{*}(Q_{l}) = \pi_{*}(\text{ad}_X^{l}(L)) - \frac{1}{l!}\sum_{j=l+1}^{i} \pi_{*}(Q_{j})\pi_{*}(Y)^{j-l}\end{align*} $$

and

$$ \begin{align*} \pi_{*}(Q_{0}) = \pi_{*}(L) - \frac{1}{l!}\sum_{l=1}^{i} \pi_{*}(Q_{l})\pi_{*}(Y)^l = {\Lambda_L^{(j)}(\mathcal{F})}.\\[-42pt] \end{align*} $$

Remark 3.10. For two self-adjoint operators A and B, $A \geq B$ if and only if $A - B \geq 0$ . Suppose that A and B are bounded operators and commute. Then $A \geq 0$ and $B \geq 0$ implies that $AB \geq 0$ . By the spectral theorem, there exists a unique, self-adjoint square root $A^{1/2} = \int _{\sigma (A)} x^{1/2} dE_A(x)$ . Since A and B commute, $A^{1/2}$ commutes with B. Then

$$ \begin{align*}\langle AB u, u\rangle = \langle A^{1/2}B u, A^{1/2}u\rangle = \langle B A^{1/2}u, A^{1/2}u \rangle \geq 0.\end{align*} $$

Also, if $0 \leq A \leq B$ , then $A^2 \leq B^2$ ( $B^2-A^2 = B(B-A) + (B-A) A $ ).

Recall that for any positive operators A and B,

(22) $$ \begin{align} AB + BA \leq A^2+B^2 \text{ and } (A+B)^2 \leq 2(A^2 + B^2). \end{align} $$

Lemma 3.11. For any $r\geq 1$ and $a \geq 1$ , there exists constant $C(a,r)>0$ such that

(23) $$ \begin{align} \|\pi_{*}(\Delta(t)^{2r})u\| \leq C(a,r)\bigg\|\sum_{i=1}^a\pi_{*}(Y_i(t)^{4r})u\bigg\|. \end{align} $$

Proof. It suffices to prove that there exists $C = C(a,r)>0$ such that

(24) $$ \begin{align} \Delta(t)^{2r} \leq C\bigg(\sum_{i=1}^aY_i(t)^{4r}\bigg) \end{align} $$

since this implies $\Delta (t)^{4r} \leq C^2(\sum _{i=1}^aY_i(t)^{4r})^2$ by the remark.

We prove (24) by induction. If $r =1$ , then

$$ \begin{align*}\Delta(t)^{2} = \bigg(\sum_{i=1}^aY_i(t)^2\bigg)^2 = \bigg(\sum_{i=1}^aY_i(t)^4 + \sum_{i \neq j}^aY_i(t)^2Y_j(t)^2\bigg).\end{align*} $$

By (22), for each i and j,

$$ \begin{align*}Y_i(t)^2Y_j(t)^2 + Y_j(t)^2Y_i(t)^2 \leq Y_i(t)^4+Y_j(t)^4.\end{align*} $$

Then there exists $C_0 = (a+1)$ such that

(25) $$ \begin{align} \Delta(t)^{2} \leq C_0\bigg(\sum_{i=1}^aY_i(t)^4\bigg). \end{align} $$

Assume that the statement holds for large r. Then, since $\Delta (t)^{2}$ and $\sum _{i=1}^aY_i(t)^4$ are positive, by (25), there exists $C_1(a,r)$ such that

$$ \begin{align*} \Delta(t)^{2(r+1)} &\leq C_1(a,r)\bigg(\sum_{i=1}^aY_i(t)^{4r}\bigg)(\Delta(t)^{2}) \\ &\leq C_1(a,r)(a+1)\bigg(\sum_{i=1}^aY_i(t)^{4r}\bigg)\bigg(\sum_{i=1}^aY_i(t)^4\bigg). \end{align*} $$

Note that the following inequality holds: for any $r \geq 1$ ,

(26) $$ \begin{align} Y_i(t)^{4r}Y_j(t)^4 + Y_i(t)^4Y_j(t)^{4r} \leq Y_i(t)^{4(r+1)}+Y_j(t)^{4(r+1)}. \end{align} $$

This inequality is proved by showing that

$$ \begin{align*} (Y_i(t)^4 &- Y_j(t)^4) (Y_i(t)^{4r} - Y_j(t)^{4r} )\\[3pt] &= (Y_i(t)^4 - Y_j(t)^4)^2 \bigg(\sum_{l=0}^{r-1}(Y_i(t)^4)^{l}(Y_j(t)^4)^{r-1-l} \bigg) \geq 0. \end{align*} $$

Since $Y_i(t)^4, Y_j(t)^4$ are all positive, the last inequality holds.

Then, by (26),

$$ \begin{align*} &\bigg(\sum_{i=1}^aY_i(t)^{4r}\bigg)\bigg(\sum_{i=1}^aY_i(t)^4\bigg) \\[3pt] &\quad = \bigg(\sum_{i=1}^aY_i(t)^{4r+4}+ \sum_{i < j}^a(Y_i(t)^{4r}Y_j(t)^{4}+Y_j(t)^{4r}Y_i(t)^{4})\bigg) \\[3pt] &\quad \leq (a+1)\bigg(\sum_{i=1}^aY_i(t)^{4r+4}\bigg). \end{align*} $$

Setting $C_2(a,r) = C_1(a,r)(a+1)^2$ ,

$$ \begin{align*}\Delta(t)^{4(r+1)} \leq C_2(a,r)\bigg(\sum_{i=1}^aY_i(t)^{4(r+1)}\bigg).\end{align*} $$

Therefore, induction holds and we finish the proof.

For the cohomological equation $X(t)u = f$ , denote its Green’s operator $G_{X(t)}$ . The following theorem states an estimate for the rescaled version of Theorem 3.4.

Theorem 3.12. For $r>1$ , let $s> r(k+1)+1/4$ . For any $f \in \mathcal {K}^s(M) $ , there exists $C_{r,k,s}>0$ such that the following holds: for any $t> 0$ ,

Proof. First, we estimate the bound on Green’s operator with the Sobolev norm.

By Lemma 3.3, there exists a rescaled operator $Y(t) \in \mathfrak {n}_{k-1}$ with

$$ \begin{align*}\pi_{*}(Y(t)) = 2\pi \iota \delta_{\mathcal{O}}(t) x \text{Id}_{H'}\end{align*} $$

where $\delta _{\mathcal {O}}(t) =\delta _{\mathcal {O}} e^{-\rho _Y t}>0$ .

By the Cauchy–Schwarz inequality, for any $l \in \mathbb {N}$ ,

$$ \begin{align*} &\|\pi_{*}(Y(t))^lG_{X(t)}f\|^2 \\[3pt] &\leq \int_{0}^{\infty}\bigg(|2\pi {\delta_{\mathcal{O}}(t)}x|^l \int_{x}^\infty e^{-t}\|f(s)\|_{H'}\,ds \bigg)^2\,dx\\[3pt] &\quad+ \int_{-\infty}^{0}\bigg(|2\pi {\delta_{\mathcal{O}}(t)}x|^l \int_{-\infty}^xe^{-t}\|f(s)\|_{H'}\,ds \bigg)^2\,dx\\[-6pt] \\ & \leq \int_{0}^{\infty}|2\pi {\delta_{\mathcal{O}}(t)}x|^{2l} \int_{x}^\infty e^{-2t}\|f(s)\|^2_{H'}\,dsdx\\[3pt] &\quad+ \int_{-\infty}^{0}|2\pi {\delta_{\mathcal{O}}(t)}x|^{2l} \int_{-\infty}^x e^{-2t}\|f(s)\|^2_{H'}\,dsdx. \end{align*} $$

If $\alpha>1$ , then by Hölder’s inequality,

(27) $$ \begin{align} \int_{-\infty}^x \|f(s)\|^2_{H'}\,ds \leq \bigg(\int_{-\infty}^x{(1+ 4\pi^2\delta_O(t)^2s^2)^{-\alpha}}\,ds\bigg) \|(I-Y(t)^2)^{{\alpha}/{2}}f \|^2. \end{align} $$

For all $\alpha>1$ , we set

(28) $$ \begin{align} C_{\alpha,l}^2 & = \int_{0}^{\infty} (2\pi x)^{2l}\bigg(\int_{x}^{\infty}(1+(4\pi^2 s^2))^{-(l+\alpha)}\,ds\bigg)\,dx \notag\\[3pt] &\quad + \int_{-\infty}^{0} (2\pi x)^{2l}\bigg(\int_{-\infty}^{x}(1+(4\pi^2 s^2))^{-(l+\alpha)}\,ds\bigg)\,dx < \infty. \end{align} $$

By Hölder’s inequality and change of variables for $ x' = {\delta _{\mathcal {O}}(t)}x$ and $s' = {\delta _{\mathcal {O}}(t)}s$ ,

(29) $$ \begin{align} \begin{split} \|\pi_{*}(Y(t))^lG_{X(t)}f\| &\leq C_{\alpha,l} \bigg(\frac{e^{-t}}{\delta_{\mathcal{O}}(t)}\bigg)\|\pi_{*} (I-Y(t)^2)^{{(l+\alpha)}/{2}}f \|\\[3pt] & = C_{\alpha,l}e^{-(1-\rho_Y)t}\bigg(\frac{1}{{\delta_{\mathcal{O}}}}\bigg)\|\pi_{*} (I-Y(t)^2)^{({l+\alpha})/{2}}f \|. \end{split} \end{align} $$

Let $E_x:C^\infty (H_\pi ) \rightarrow C^\infty (H')$ be the linear operator defined by $E_xf = f(x)$ (cf. Lemma 2.5). Then the action of $\pi _{*}(Y(t))$ on $C^\infty (H_\pi )$ can be rewritten as

(30) $$ \begin{align} E_x\pi_{*}(Y(t)) = (2\pi \iota x \delta_{\mathcal{O}}(t))^jE_x \quad \text{for all } j \in \mathbb{N} \text{ and } x \in \mathbb{R}. \end{align} $$

Let $L(t)$ be a rescaled element of $L \in \mathcal {F}$ . By definition of representation (13) and by Lemma 3.8 for rescaled basis $\mathcal {F}(t)$ , for any $L(t)$ there exists $Q_j(t) \in \mathfrak {U}(\mathfrak {n})$ such that

(31) $$ \begin{align} E_x\pi_{*}(L(t)) = \sum_{j=0}^{[L]} \frac{(2\pi \iota x \delta_{\mathcal{O}}(t) )^j}{j!} \pi^{\prime}_{*}(Q_{j}(t))E_x. \end{align} $$

For Green’s operator $G_{X(t)}$ , since $E_xG_{X(t)} = \int _{-\infty }^{x}E_s \,ds$ for all $x \in \mathbb {R}$ ,

(32) $$ \begin{align} E_x\pi_{*}(L(t))G_{X(t)} &= \sum_{j=0}^{[L]} \frac{(2\pi \iota x \delta_{\mathcal{O}}(t))^j}{j!} \pi^{\prime}_{*}(Q_{j}(t))E_xG_{X(t)}\nonumber\\[3pt] & = \sum_{j=0}^{[L]} \frac{(2\pi \iota x \delta_{\mathcal{O}}(t))^j}{j!} \int_{-\infty}^{x}\pi^{\prime}_{*}(Q_{j}(t))E_s \,ds\nonumber\\[3pt] & = \sum_{j=0}^{[L]} \frac{(2\pi \iota x \delta_{\mathcal{O}}(t))^j}{j!} \int_{-\infty}^{x}E_s\pi_{*}(Q_{j}(t))\,ds\nonumber\\[-6pt] \nonumber \\ &= \sum_{j=0}^{[L]} \frac{(2\pi \iota x \delta_{\mathcal{O}}(t))^j}{j!} E_xG_{X(t)}(\pi_{*}(Q_{j}(t)))\nonumber\\ & = E_x\sum_{j=0}^{[L]} \frac{1}{j!} \pi_{*}(Y(t))^jG_{X(t)}(\pi_{*}(Q_{j}(t))). \end{align} $$

Combining (29) and (32), there exists a constant $C^{\prime }_{\alpha ,j}>0$ such that

$$ \begin{align*} \|\pi_{*}(L(t))G_{X(t)}(f)\| & {{\kern-1pt}\leq{\kern-3pt}} \sum_{j=0}^{[L]} \frac{1}{j!}\|\pi_{*}(Y(t))^jG_{X(t)}(\pi_{*}(Q_{j}(t))f)\|\\ & {\leq{\kern-3pt}} \sum_{j=0}^{[L]} C^{\prime}_{\alpha,j}e^{-(1-\rho_Y)t} \bigg(\frac{1}{{\delta_{\mathcal{O}}}}\bigg)^{j+1}\|\pi_{*} (I-Y(t)^2)^{(({j+\alpha})/{2})}\pi_{*} (Q_{j}(t)f)\|. \end{align*} $$

Note that by binomial formula, there exists $R_{j}(t) \in \mathfrak {U}(\mathfrak {n})$ such that

(33) $$ \begin{align} \pi_*(L(t))^{2r} = \bigg(\sum_{j=0}^{[L]} \frac{1}{j!}\pi_{*}(Y(t)^j)\pi_{*}(Q_{j}(t))\bigg)^{2r} := \sum_{j=0}^{2r[L]}\pi_{*}(Y(t)^j)\pi_{*}(R_{j}(t)). \end{align} $$

In particular, $R_j(t)$ is product of $Q_i(t)$ s and $[R_{j}(t)] = 2r([L]+1)-j$ . Specifically, to compute the transverse Laplacian, here we assume that $L(t) = (Y_i(t))^{4r}$ for each element $Y_i \in \mathcal {F}$ . Then, by (33), for any $\alpha>1$ ,

(34) $$ \begin{align} &\|\pi_{*}(Y_i(t)^{4r})G_{X(t)}(f)\| \nonumber\\ & \quad \leq \sum_{j=0}^{4r[Y_i]} C({\alpha,i,j,r})e^{-(1-\rho_Y) t} \bigg(\frac{1}{ {\delta_{\mathcal{O}}}}\bigg)^{j+1}\|\pi_{*} (I-Y(t)^2) ^{(({j+\alpha})/{2})}\pi_{*}(R_{j}(t))f \|\nonumber\\ &\quad \leq C(\alpha,i,r)e^{-(1-\rho_Y) t} \max\{1, {{\delta_{\mathcal{O}}}}^{-4r[Y_i]+1}\}|f|_{\alpha + 4r([Y_i] + 1),\mathcal{F}(t)}. \end{align} $$

Therefore, combining with Lemma 3.11,

$$\begin{align*}&\|\pi_*(\Delta(t)^{2r})G_{X(t)}(f)\|\\&\quad \leq C(a,r)\sum_{i=1}^{a} \|\pi_*(Y_i(t)^{4r})G_{X(t)}(f)\| \\&\quad \leq C(a,r,\alpha)\sum_{i=1}^{a} e^{-(1-\rho_Y) t} \max\{1, {{\delta_{\mathcal{O}}}}^{-4r([Y_i]+1)}\}|f|_{\alpha + 4r([Y_i] + 1),\mathcal{F}(t)} \\&\quad \leq C(a,r,\alpha)e^{-(1-\rho_Y) t} \max\{1, {{\delta_{\mathcal{O}}}}^{-4r(k+1)}\}|f|_{\alpha + 4r(k + 1),\mathcal{F}(t)}. \end{align*}$$

Then, there exists $C' = C'(a,\alpha ,r,X)> 0$ such that

$$ \begin{align*} |{G_{X(t)}f}|_{4r,\mathcal{F}(t)} \leq C' e^{-(1-\rho_Y)t} \max\{1,{\delta^{-4r(k+1)}_{\mathcal{O}}}\} |{f}|_{\alpha+4(k+1)r,\mathcal{F}(t)}. \end{align*} $$

By interpolation, for all $s> r(k+1)+1/4$ , there exists a constant $C_{r,s}: = C_{r,s}(k,X)>0$ such that

$$ \begin{align*} |{G_{X(t)}f}|_{r,\mathcal{F}(t)} \leq C_{r,s} e^{-(1-\rho_Y)t} \max\{1,{\delta^{-4r(k+1)}_{\mathcal{O}}} \} |{f}|_{s,\mathcal{F}(t)}. \end{align*} $$

By the choice of Y, we obtain $\rho _Y = \unicode{x3bb} _{\mathcal {F}}$ , which finishes the proof.

3.4 Scaling of invariant distributions

We introduce the Lyapunov norm and compare bounds between the Sobolev dual norm and the Sobolev Lyapunov norm of invariant distribution in every irreducible, unitary representation.

For all $t \in \mathbb {R}$ and $\unicode{x3bb} := \unicode{x3bb} _{\mathcal {F}}(\rho )$ defined in (19), let the operator $U_t: L^2(\mathbb {R},H') \rightarrow L^2(\mathbb {R},H')$ be the unitary operator defined as follows:

(35) $$ \begin{align} (U_tf)(x) = e^{-({\unicode{x3bb}}/{2})t}f(e^{-\unicode{x3bb} t}x). \end{align} $$

We will compare the norm estimate of invariant distributions with respect to scaled basis

$$ \begin{align*}|D|_{-r,\mathcal{F}(t)} = \sup_{f \in W^r(H_{\pi})}\{|D(f)|:|f|_{r,\mathcal{F}(t)} =1\}\end{align*} $$

by the unscaled norm $|D|_{-r,\mathcal {F}}$ .

Theorem 3.13. For $r \geq 1$ and $s> r(k+1)$ , there exists a constant $C_{r,s}>0$ such that for all $t>0$ , the following bound holds:

$$ \begin{align*}{|U_tf|_{r, \mathcal{F}(t)}} \leq C_{r,s}{|f|_{s, \mathcal{F}}}.\end{align*} $$

Proof. Assume the same hypothesis for $L \in \mathcal {F}$ and $L(t)$ in the proof of Theorem 3.12. By Lemma 3.8, there exists an $(i-j+1)$ th-order $Q_j \in \mathfrak {U}(\mathfrak {n})$ with

$$ \begin{align*} {U_t^{-1}L(t)U_t } & = x^iQ_i + e^{-\unicode{x3bb} t}x^{i-1}Q_{i-1} + e^{-2\unicode{x3bb} t}x^{i-2}Q_{i-2}+\cdots + e^{-i \unicode{x3bb} t}Q_o. \end{align*} $$

Then there exists $C>0$ such that

$$ \begin{align*} \|U_t^{-1}L(t)U_t f\| & \leq \sum_{j=0}^i \|e^{-(i-j)\unicode{x3bb} t}x^jQ_jf\| \\ & \leq C \max_{0 \leq j \leq i} \|Y^jQ_jf\|\\ & \leq C|f|_{[L]+1,\mathcal{F}}. \end{align*} $$

Since $[L] \leq k$ , by unitarity

(36) $$ \begin{align} |{U_tf}|_{1, \mathcal{F}(t)} \leq C_1 |{f}|_{k+1,\mathcal{F}}. \end{align} $$

Hence, for any $s> r(k+1)$ ,

(37) $$ \begin{align} |{U_tf}|_{r, \mathcal{F}(t)} \leq C_{r,s} |{f}|_{s,\mathcal{F}}. \end{align} $$

Theorem 3.14. For $r \geq 1$ and $s> r(k+1)$ , there exists a constant $C_{r,s}>0$ such that for any $\unicode{x3bb}>0$ and $t>0$ , the invariant distribution defined in (14) satisfies

$$ \begin{align*}|D|_{-s, \mathcal{F}} \leq C_{r,s}e^{-({\unicode{x3bb}}/{2})t}|D|_{-r, \mathcal{F}(t)}.\end{align*} $$

Proof. Recall the functional $\ell $ defined in the Lemma 3.2. For $f \in C^\infty (H_\pi )$ ,

$$ \begin{align*} \begin{split} D(U_tf) & = \int_{\mathbb{R}} \ell (e^{-({\unicode{x3bb}}/{2})t}f(e^{-\unicode{x3bb} t}x))\,dx \\ &= \int_{\mathbb{R}} e^{-({\unicode{x3bb}}/{2})t}\ell(f(e^{-\unicode{x3bb} t}x))\,dx \\[-2pt] & = e^{({\unicode{x3bb}}/{2})t} \int_{\mathbb{R}} \ell(f(y))\,dy \\[-2pt] & = e^{({\unicode{x3bb}}/{2})t}D(f). \end{split} \end{align*} $$

Then, by unitarity (37),

$$ \begin{align*}|D|_{-r, \mathcal{F}(t)} = \sup_{f\neq 0} \frac{|D(f)|}{|{f}|_{r, \mathcal{F}(t)}} = \sup_{f\neq 0} \frac{|D(U_tf)|}{|{U_tf}|_{r, \mathcal{F}(t)}} \geq \sup_{f\neq 0} \frac{e^{({\unicode{x3bb}}/{2})t}|D(f)|}{C_{r,s}|{f}|_{s, \mathcal{F}}} = C^{-1}_{r,s}e^{({\unicode{x3bb}}/{2})t}|D|_{-s, \mathcal{F}},\end{align*} $$

and

$$ \begin{align*} |D|_{-s, \mathcal{F}} \leq C_{r,s}e^{-({\unicode{x3bb}}/{2})t}|D|_{-r, \mathcal{F}(t)}.\\[-42pt] \end{align*} $$

Definition 3.15. (Lyapunov norm)

For any basis $\mathcal {F}$ and all $\sigma> 1/2$ , define the Lyapunov norm

(38)

The following lemma is immediate from the definition of the norm.

Lemma 3.16. [Reference Flaminio and ForniFF14, Lemma 4.15]

For all $t \geq 0$ and $\sigma> 1/2$ , we have

We conclude this section by introducing a useful inequality that follows from the Theorem 3.14:

(39) $$ \begin{align} C_{r, s}^{-1}{|D|_{-s,\mathcal{F}}} \leq \|D\|_{-r,\mathcal{F}} \leq |D|_{-r,\mathcal{F}}. \end{align} $$

4.1 Sobolev a priori bounds

Assume $\mathcal {F}(t) = (X(t),Y(t))$ is a rescaled basis. For any $x \in M$ , let $\phi _{x,t} : \mathbb {R} \times \mathbb {R}^a \rightarrow M$ be the local embedding defined by

$$ \begin{align*}\phi_{x,t}(\tau,\textbf{s}) = x \exp(\tau X(t))\prod_{i=1}^{a}\exp(s_{i}{Y_{i}(t)}), \quad \textbf{s} = (s_i)_{i=1}^a.\end{align*} $$

Lemma 4.1. For any $x \in M$ , $t \geq 0$ , and $f \in C^{\infty }(M)$ , we have

$$ \begin{align*} &\partial_{s_i}f\circ \phi_{x,t}(\tau,\mathrm{\mathbf{s}}) = S_i f\circ\phi_{x,t}(\tau,\mathrm{\mathbf{s}}), \quad S_i = Y_{i}(t)+\sum_{l>i}^aq_l(\mathrm{\mathbf{s}},t)Y_{l}(t) \in \mathfrak{n}, \end{align*} $$

where q is polynomial in s of degree at most $k-1$ and $|q_l(\mathrm{\mathbf{s}},t)| \leq |q_l(\mathrm{\mathbf{s}},0)|$ for all $t \geq 0$ .

Proof. For $h \in \mathbb{R}$ and $1 \leq i \leq a$ , let $\textbf {s}+h_i$ denote a sequence with $(\textbf {s}+h_i)_i = s_i +h$ and $(\textbf {s}+h_i)_j = s_j$ if $i \neq j$ . By definition,

$$ \begin{align*}\partial_{s_i}f\circ \phi_{x,t}(\tau,\textbf{s}) = \lim_{h\rightarrow 0}\frac{f\circ \phi_{x,t}(\tau,\textbf{s}+h_i)- f\circ \phi_{x,t}(\tau,\textbf{s})}{h},\end{align*} $$

and we plan to rewrite $f\circ \phi _{x,t}(\tau ,\textbf {s}+h_i)$ in a suitable manner for differentiation.

Fix $i$ . Then for $j >i$ ,

$$ \begin{align*} & \exp((s_{i}+h){Y_{i}(t)})\exp(s_{j}{Y_{j}(t)}) \\ &\quad = \exp(s_{i}Y_{i}(t))\exp(hY_{i}(t))\exp(s_{j}{Y_{j}(t)})\\ &\quad = \exp(s_{i}Y_{i}(t))\exp(e^{ad(hY_{i}(t)) }s_{j}{Y_{j}(t)})\exp(hY_{i}(t))\\ &\quad = \exp(s_{i}Y_{i}(t))\exp(s_{j}Y_{j}(t))\exp\bigg(\sum_{n=1}^\infty{\frac{1}{n!}ad^n_{hY_{i}(t)} }s_{j}{Y_{j}(t)}\bigg)\exp(hY_{i}(t)). \end{align*} $$

By the Baker–Campbell–Hausdorff formula, we set

$$\begin{align*} & \exp((s_{i}+h){Y_{i}(t)})\exp(s_{j}{Y_{j}(t)})\\ &\quad = \exp(s_{i}Y_{i}(t))\exp(s_{j}Y_{j}(t))\exp\Big(h(Y_{i}(t)+[Y_i(t),s_{j}Y_{j}(t)]) + O(h^2)\sum_{m>j}Y_m\Big).\end{align*}$$

Choose $j = i+1$ and observe that all the terms of h are combined into an exponential. Iteratively, we will repeat this process from $j = i+1$ to a until all the terms of h are pushed back. That is, we conclude

$$ \begin{align*} \phi_{x,t}(\tau,\textbf{s}+h_i) &= \phi_{x,t}(\tau,\textbf{s})\exp(h(Y_{i}(t)+[Y_i(t),s_{i+1}Y_{i+1}(t)] \\&\enspace+[[Y_i(t),s_{i+1}Y_{i+1}(t)],s_{i+2}Y_{i+2}(t)]\\&\enspace + \cdots + [\cdots[Y_i(t),s_{i+1}Y_{i+1}(t)],\cdots],s_aY_a(t)] ] )+ O(h^2)\sum_{m>i+1}^{a}Y_m. \end{align*} $$

For convenience, we write coefficient function $q_l(\textbf{s},t)$ in polynomial degree at most k for $\textbf{s}$ such that

$$ \begin{align*}\phi_{x,t}(\tau,\textbf{s}+h_i) = \phi_{x,t}(\tau,\textbf{s})\exp\left(h\left(Y_{i}(t)+\sum_{l>i}^aq_l(\textbf{s},t)Y_{l}(t)\right) + O(h^2)\sum_{m>i+1}^{a}Y_m\right).\end{align*} $$

We conclude the proof by choosing $S_i = Y_{i}(t)+\sum _{l>i}^aq_l(\textbf{s},t)Y_{l}(t)$ . Also, commutation between rescaled elements yields $[Y_i(t),s_{j}Y_{j}(t)] = s_{j}e^{-\rho_{ij} t}Y_{k}(t)$ for some $\rho_{ij} >0$ . This implies that the term $q_l(\textbf{s},t)$ for each $l >i$ includes exponential terms with negative exponent so that it decreases for $t \geq 0$ .

Let $\triangle _{\mathbb {R}^a}$ be the Laplacian operator on $\mathbb {R}^a$ given by

$$ \begin{align*}\triangle_{\mathbb{R}^a} = - \sum_{i=1}^a\frac{\partial^2}{\partial s_i^2}.\end{align*} $$

Given an open set $O \subset \mathbb {R}^a$ containing the origin, let $\mathcal {R}_O$ be the family of all a-dimensional symmetric rectangles $R \subset [-(1/2), 1/2]^a \cap O$ that are centered at the origin. The inner width of the set $O \subset \mathbb {R}^a$ is the positive number

$$ \begin{align*}w(O) = \sup \{\text{Leb}(R) \mid R \in \mathcal{R}_O \},\end{align*} $$

where Leb is Lebesgue measure on R. The width function of a set $\Omega \subset \mathbb {R} \times \mathbb {R}^a$ containing the line $\mathbb {R} \times \{0\}$ is the function $w_\Omega : \mathbb {R} \rightarrow [0,1]$ defined as follows:

$$ \begin{align*}w_\Omega(\tau):= w(\{\textbf{s}\in \mathbb{R}^a \mid (\tau,\textbf{s}) \in \Omega\}),\quad \text{for all } \tau \in \mathbb{R}.\end{align*} $$

Definition 4.2. Consider the family $\mathcal {O}_{x,t,T}$ of open sets $\Omega \subset \mathbb {R} \times \mathbb {R}^a$ satisfying the two conditions

$$ \begin{align*}[0,T]\times \{0\} \subset \Omega \subset \mathbb{R} \times [-\tfrac{1}{2}, \tfrac{1}{2}]^a\end{align*} $$

and $\phi _{x,t}$ is injective on the open set $\Omega \subset \mathbb {R}^a$ . The average width of the orbit segment of the rescaled nilflow $\{\phi _{x,t}(\tau ,0) \mid 0 \leq t \leq T\}$ ,

(40) $$ \begin{align} w_{\mathcal{F}(t)}(x, T) := \sup_{\Omega \in \mathcal{O}_{x,t,T} }\bigg(\frac{1}{T}\int_{0}^{T}\frac{ds}{w_\Omega(s)}\bigg)^{-1}, \end{align} $$

is a positive number.

The following lemma is derived from the standard Sobolev embedding theorem under the rescaling argument.

Lemma 4.3. [Reference Flaminio and ForniFF14, Lemma 3.7]

Let $I \subset \mathbb {R}$ be an interval, and let $\Omega \subset \mathbb {R} \times \mathbb {R}^a$ be a Borel set containing the segment $I \times \{0\} \subset \mathbb {R} \times \mathbb {R}^a$ . For every $\sigma>a/2$ , there is a constant $C_s>0$ such that for all functions $F \in C^{\infty }(\Omega )$ and all $\tau \in I$ , we have

$$ \begin{align*}\bigg(\int_I |F(\tau,0)|\,d\tau \bigg)^2 \leq C_\sigma \bigg(\int_I \frac{d\tau}{w_\Omega(\tau)} \bigg) \int_{\Omega}|(I-\triangle_{\mathbb{R}^a})^{{\sigma}/{2}}F(\tau,\textbf{s})|\,d\tau d\textbf{s}.\end{align*} $$

The following theorem indicates the bound of the ergodic average of the scaled nilflow $\phi _{X(t)}^\tau $ with width function on general nilmanifolds (see also [Reference Flaminio, Forni and TanisFFT16, Theorem 5.2] for twisted horocycle flows).

Theorem 4.4. For all $\sigma> a/2 $ , there is a constant $C_{\sigma }>0$ such that

$$ \begin{align*}\bigg|\frac{1}{T} \int_{0}^{T}f\circ \phi_{X(t)}^\tau(x)\,d\tau\bigg| \leq {C_\sigma}{T^{-{1}/{2}}w_{\mathcal{F}(t)}(x,T)^{-{1}/{2}}}|f|_{\sigma,\mathcal{F}(t)}.\end{align*} $$

Proof. Recall that for any self-adjoint operators A and B,

$$ \begin{align*}(A+B)^2 \leq 2(A^2+B^2).\end{align*} $$

Since $|s_i| \leq \tfrac 12$ and $t \geq 1$ , by Lemma 4.1, each polynomial $q_{j}$ is bounded in $\textbf {s}$ and t. Then, essentially by skew-adjointness of $Y_i(t)$ , there exists a large constant $C>1$ with

$$ \begin{align*} -S_i^2 &= -\left(Y_i(t) +\sum_{l>i}^aq_l(\textbf{s},t)Y_{l}(t)\right)^2\\ & \leq -C\sum_{j=i}^a Y_j({t})^2. \end{align*} $$

Since operators on both sides are essentially self-adjoint,

$$ \begin{align*}\bigg(I-\sum_{i=1}^a S_i^2\bigg)^{{\sigma}/{2}} \leq C^{({\sigma}/{2})}\bigg(I - \sum_{i=1}^{a}Y_i(t)^2\bigg)^{{\sigma}/{2}}.\end{align*} $$

Thus, there is a constant $C_\sigma>0$ such that

(41) $$ \begin{align} \|(I-\triangle_{\mathbb{R}^a})^{{\sigma}/{2}}f\circ\phi_{x,t}\|_{L^2(\Omega)}^2 \leq C_\sigma \|(I-\Delta_{{\mathcal{F}(t)}})^{{\sigma}/{2}}f\|_{L^2(M)}^2. \end{align} $$

By Lemma 4.3, we can see that for $\sigma>a/2$ , setting $F(\tau ,0) = f\circ \phi _{X(t)}^\tau (x)$ ,

$$ \begin{align*} \bigg|\frac{1}{T} \int_{0}^{T}f\circ \phi_{X(t)}^\tau(x)\,dt \bigg|^2 & = \bigg(\frac{1}{T} \int_{0}^{T}|F(\tau,0)\,d\tau|^2 \bigg)^2\\ & \leq C_\sigma \frac{1}{T}\bigg(\frac{1}{T}\int_0^T \frac{ds}{w_\Omega(s)} \bigg) \int_{\Omega}|(I-\triangle_{\mathbb{R}^a})^{{\sigma}/{2}}F(\tau,\textbf{s})|\,d\tau d\textbf{s}\\ & \leq C_\sigma T^{-1}w_{\mathcal{F}(t)}(x,T)^{-1}\|(I-\Delta_{\mathcal{F}(t)})^{{\sigma}/{2}}f\|_{L^2(M)}^2.\\[-42pt] \end{align*} $$

5.1 Almost periodic points

Let $X_{\alpha }$ be the vector field on M defined in (7). Recall formula (8):

$$ \begin{align*}X_{\alpha}:= \xi+\sum_{(i,j) \in J}\alpha^{(j)}_i\eta^{(j)}_i.\end{align*} $$

Let us introduce special type of condition for the Lie algebra $\mathfrak {n}$ required for width estimates.

Definition 5.1. The nilpotent Lie algebra $\mathfrak {n}$ satisfies the transversality condition if there exists a basis $(X_\alpha , Y)$ of $\mathfrak {n}$ such that

(42) $$ \begin{align} \langle \mathfrak{G}_\alpha \rangle + \text{Ran}(\text{ad}_{X_{\alpha}})+C_{\mathfrak{I}}(X_{\alpha}) = \mathfrak{n} \end{align} $$

where $\mathfrak {G}_\alpha = (X_\alpha , Y^{(1)}_i)_{1 \leq i \leq n}$ is a set of generator, $\text {Ran}(\text {ad}_{X_{\alpha }})=\{Y \in \mathfrak {I} \mid Y= \text {ad}_{X_{\alpha }}(W), W \in \mathfrak {I} \}$ and $C_{\mathfrak {I}}(X_{\alpha }) = \{Y \in \mathfrak {I} \mid [Y,X_{\alpha }] = 0\}$ is the centralizer.

It is clear that the set of generators is included neither in the range of $\text {ad}_{X_{\alpha }}$ nor in the centralizer $C_{\mathfrak {I}}(X_{\alpha })$ . We will restrict $\mathfrak {n}$ to satisfying the condition (42) in the rest of sections.

Recall that $M_\theta ^a$ denotes the fiber at $\theta \in \mathbb {T}^1$ of the fibration $pr_2: M \rightarrow \mathbb {T}^1$ . $\Phi _{\alpha ,\theta }$ denotes the first return map of the nilflow $\{\phi _{X_{\alpha }}^t\}$ to the transverse section $M_\theta ^a$ , and $\Phi ^r_{\alpha ,\theta }$ denotes the rth iterate of the map $\Phi _{\alpha ,\theta }$ . Let G denote the nilpotent Lie group with its lattice $\Gamma $ defining $M_\theta ^a = \Gamma \backslash G$ . G acts on $M_\theta ^a$ by right action and the action of G extends to $M_\theta ^a \times M_\theta ^a$ .

Define a map $\psi _{\alpha ,\theta }^{(r)} : M_\theta ^a \rightarrow M_\theta ^a \times M_\theta ^a$ given by $\psi _{\alpha ,\theta }^{(r)}(x) = (x,\Phi ^r_{\alpha ,\theta }(x))$ . By its definition, the map $\Phi ^r_{\alpha ,\theta }$ commutes with the action of the centralizer $C_{G} = \exp (C_{\mathfrak {I}}(X_{\alpha })) \subset G$ and its action on the product $M_\theta ^a \times M_\theta ^a$ commutes with $\psi _{\alpha ,\theta }^{(r)}$ . That is, for $c \in C_{G}$ and $x = \Gamma g$ ,

(43) $$ \begin{align} \psi_{\alpha,\theta}^{(r)}(xc) = (xc,\Phi^r_{\alpha,\theta}(xc)) = (xc, \Phi^r_{\alpha,\theta}(x)c) = \psi_{\alpha,\theta}^{(r)}(x)c. \end{align} $$

Then the quotient map is well defined on

(44) $$ \begin{align} \Psi_{\alpha,\theta}^{(r)} := M_\theta^a/C_{G} \longrightarrow M_\theta^a \times M_\theta^a/C_{G}. \end{align} $$

Setting. (i) In $M_\theta ^a \times M_\theta ^a$ , we set the diagonal $\Delta = \{(x,x) \mid x \in M_\theta ^a\}$ which is isomorphic to $M_\theta ^a $ by identifying $(x,x)$ with $x \in M_\theta ^a$ . Given $(x,x) \in \Delta $ , the tangent space of the diagonal is $ T_{(x,x)}\Delta := \{(v,v) \mid v \in T_{x} M_\theta ^a\}$ , and its normal space is defined by $(T_{(x,x)}\Delta )^\perp = \{(v,-v) \mid v \in T_x M_\theta ^a \} = T_{(x,x)}\Delta ^\perp $ . On the tangent space at $(x,x) \in M_\theta ^a\times M_\theta ^a$ , it splits as

$$ \begin{align*}T_{(x,x)}(M_\theta^a \times M_\theta^a) = T_{(x,x)}\Delta \oplus (T_{(x,x)}\Delta)^\perp.\end{align*} $$

For any $w_1, w_2 \in T_{x}M_\theta ^a$ ,

(45) $$ \begin{align} (w_1,w_2) = (1/2)(w_1+w_2,w_1+w_2) + (1/2)(w_1-w_2, -(w_1-w_2)). \end{align} $$

(ii) Given $x = \Gamma h_1, y = \Gamma h_2 \in M_\theta ^a$ , define a set $\Delta _{(x,y)} = \{(xg,yg) \mid g \in G \} \subset M_\theta ^a \times M_\theta ^a$ for $(xg,yg) = (\Gamma h_1 g, \Gamma h_2 g)$ and $\Delta _{(x,y)}^\perp = \{(xg,yg^{-1}) \mid g \in G \}$ that contains $(x,y)$ . For $\psi _{\alpha ,\theta }^{(r)}(x) = (x,\Phi ^r_{\alpha ,\theta }(x))$ , its tangent space in $M_\theta ^a \times M_\theta ^a$ is decomposed as

$$ \begin{align*}T_{(x,\Phi^r_{\alpha,\theta}(x))}(M_\theta^a \times M_\theta^a) = T_{(x,\Phi^r_{\alpha,\theta}(x))}\Delta_{(x,\Phi^r_{\alpha,\theta}(x))} \oplus (T_{(x,\Phi^r_{\alpha,\theta}(x))}\Delta_{(x,\Phi^r_{\alpha,\theta}(x))})^\perp.\end{align*} $$

Then the tangent space of the diagonal is $T_{(x,\Phi ^r_{\alpha ,\theta }(x))}\Delta _{(x,\Phi ^r_{\alpha ,\theta }(x))} = \{(v,d_x\Phi ^r_{\alpha ,\theta }(v) ) \mid v \in T_x M_\theta ^a\}$ and its normal space is identified as

$$ \begin{align*}(T_{(x,\Phi^r_{\alpha,\theta}(x))}\Delta_{(x,\Phi^r_{\alpha,\theta}(x))})^\perp = T_{(x,\Phi^r_{\alpha,\theta}(x))}\Delta_{(x,\Phi^r_{\alpha,\theta}(x))}^\perp.\end{align*} $$

By identification in (45), for $w_1= v$ and $w_2 = -d_x\Phi ^r_{\alpha ,\theta }(v)$ , we write

(46) $$ \begin{align} &(T_{(x,\Phi^r_{\alpha,\theta}(x))}\Delta_{(x,\Phi^r_{\alpha,\theta}(x))})^\perp \nonumber\\ &\quad=\{(1/2)(v-d_x\Phi^r_{\alpha,\theta}(v)),-(1/2)(v-d_x\Phi^r_{\alpha,\theta}(v)) \mid v \in T_x M_\theta^a\}. \end{align} $$

(iii) Now define the orthogonal projection $ \pi : M_\theta ^a \times M_\theta ^a \rightarrow M_\theta ^a \times M_\theta ^a$ along the direction of the diagonal. That is, for $(x,y) \in M_\theta ^a \times M_\theta ^a$ , there exists $(x',y')$ such that $\pi (x,y) = (x', y') \in \Delta _{(x,y)} \cap \Delta _{(x,x)}^\perp .$ Then

(47) $$ \begin{align} T_{\pi(x,y)} \Delta_{\pi(x,y)} = T_{(x,y)}\Delta_{(x,y)}, \quad T_{\pi(x,y)} \Delta_{\pi(x,y)}^\perp = T_{(x,y)}\Delta_{(x,y)}^\perp. \end{align} $$

Define a map $F^{(r)}: M_\theta ^a \rightarrow M_\theta ^a \times M_\theta ^a$ given by $F^{(r)} = \pi \circ \psi _{\alpha ,\theta }^{(r)}$ . In the local coordinate, by identification (46) and (47),

(48) $$ \begin{align} d_xF^{(r)}(v) = (1/2)(v-d_x\Phi^r_{\alpha,\theta}(v)),-(1/2)(v-d_x\Phi^r_{\alpha,\theta}(v)), \quad v \in T_x M_\theta^a. \end{align} $$

By (43) and the definition of $F^{(r)}$ , we have $F^{(r)}(xc) = F^{(r)}(x)c$ for $c \in C_G$ . Then for all $r \in \mathbb {Z}$ , $F^{(r)}$ induces a quotient map $F_C^{(r)}: M_\theta ^a / C_G \rightarrow M_\theta ^a \times M_\theta ^a / C_G$ . From (48), the range of the differential $DF_C^{(r)}$ is determined by $I-D\Phi ^r_{\alpha ,\theta }$ (refer to Figure 1).

In the next lemma, we verify the range of the differential map $DF_C^{(r)}$ .

Proof. Recall that $\Phi ^r_{\alpha ,\theta }$ is the rth return map on $M_\theta ^a$ . We find the differential in the direction of each $Y^j_i$ for fixed i and j. For $x \in M$ , set a curve $\gamma ^x_{i,j}(t) = x\exp (tY_i^{(j)})\exp (rX_{\alpha }). $ Note that

$$ \begin{align*} \exp({tY_i^{(j)}})\exp(rX_{\alpha}) & = \exp(rX_{\alpha})\exp(-rX_{\alpha})\exp({tY_i^{(j)}})\exp(rX_{\alpha})\\ & = \exp(rX_{\alpha}) \exp(e^{-r (\text{ad}_{X_{\alpha}})}({tY_i^{(j)}})) \end{align*} $$

and

$$ \begin{align*}\dfrac{d}{dt}\gamma^x_{i,j}(t)\mid_{t=0}\, = e^{-r (\text{ad}_{X_{\alpha}})}({Y_i^{(j)}}).\end{align*} $$

By definition, $ ({\partial \Phi ^r_{\alpha , \theta }}/{\partial s_{i}^{(j)}})(x) = {d}/{dt}(\gamma ^x_{i,j}(t))\mid _{t=0}$ and we have $I-D\Phi ^r_{\alpha ,\theta } = I-\sum _{(i,j) \in J}({\partial \Phi ^r_{\alpha , \theta }}/{\partial s_{i}^{(j)}})$ . Then

(49) $$ \begin{align} (I-D\Phi^r_{\alpha,\theta})\bigg(\sum_{(i,j) \in J} s_{i}^{(j)}Y_{i}^{(j)}\bigg) = \bigg[r(\text{ad}_{X_{\alpha}})\bigg(\sum_{k=0}^\infty \frac{(-1)^k}{(k+1)!}(\text{ad}_{X_{\alpha}})^k\bigg)\bigg]\bigg(\sum_{(i,j) \in J} s_{i}^{(j)}Y_{i}^{(j)}\bigg).\end{align} $$

Therefore, the range of $I-D\Phi ^r_{\alpha ,\theta }$ is contained in $\text {Ran}(\text {ad}_{X_{\alpha }})$ .

Conversely, $({1-e^{-\text {ad}_{X_{\alpha }}}})/{\text {ad}_{X_{\alpha }}} = \sum _{k=0}^\infty ({(-1)^k})/({(k+1)!})(\text {ad}_{X_{\alpha }})^k$ is invertible and

(50) $$ \begin{align} (I-D\Phi^r_{\alpha,\theta})\bigg(\bigg(\frac{1-e^{-\text{ad}_{X_{\alpha}}}}{\text{ad}_{X_{\alpha}}}\bigg)^{-1}\bigg(\sum_{(i,j) \in J} s_{i}^{(j)}Y_{i}^{(j)}\bigg)\bigg) = r(\text{ad}_{X_{\alpha}})\bigg(\sum_{(i,j) \in J} s_{i}^{(j)}Y_{i}^{(j)}\bigg). \end{align} $$

Therefore, we conclude that the range of $I-D\Phi ^r_{\alpha ,\theta }$ is $ \text {Ran}(\text {ad}_{X_{\alpha }})$ .

If $\sum _{(i,j) \in J} s_{i}^{(j)}Y_{i}^{(j)} \in C_{\mathfrak {I}}(X_{\alpha })$ , then $(I-D\Phi ^r_{\alpha ,\theta })(\sum _{(i,j) \in J} s_{i}^{(j)}Y_{i}^{(j)}) = 0$ and the kernel of $I-D\Phi ^r_{\alpha ,\theta }$ is $C_{\mathfrak {I}}(X_{\alpha })$ . That is, $I-D\Phi ^r_{\alpha ,\theta }$ is bijective on $\mathfrak {I}/C_{\mathfrak {I}}(X_{\alpha }) $ . Thus, by (49), the Jacobian of $I-D\Phi ^r_{\alpha ,\theta }$ is non-zero constant, and this concludes the proof.

Setting (continued). (iv) Set the submanifold $\mathcal {S} \subset M_\theta ^a \times M_\theta ^a$ that consists of the diagonal $\Delta $ and the coordinates of generators in normal (transverse) directions. Denote its quotient $\mathcal {S}_C = \mathcal {S}/C_G \subset M_\theta ^a \times M_\theta ^a / C_G$ . Then, following Lemma 5.3, we obtain the transversality of $F_C^{(r)}$ to $\mathcal {S}_C$ . For every $p \in (F_C^{(r)})^{-1}(\mathcal {S}_C)$ , the transversality holds on the tangent space:

(51) $$ \begin{align} T_{F_C^{(r)}(p)}\mathcal{S}_C + DF_C^{(r)}(T_p M_\theta^a/C_G) = T_{F_C^{(r)}(p)}(M_\theta^a \times M_\theta^a/C_G). \end{align} $$

(v) Denote the Lebesgue measure $\mathcal {L}^{a+1} (= \mathrm {vol}_{M})$ on the nilmanifold M and the conditional measure $\mathcal {L}_\theta ^{a} (= \mathrm {vol}_{M_\theta ^a})$ on the transverse manifold $M_\theta ^a$ . On the quotient space $M_\theta ^a/C_G$ , we write the measure $\mathcal {L}_\theta ^{c} (= \mathrm {vol}_{M_\theta ^a/C_G})$ . Similarly, we set the conditional measure $\mu _\theta ^{a}(= \mathrm {vol}_{M_\theta ^a \times M_\theta ^a})$ on the product manifold and $\mu _\theta ^{c}(= \mathrm {vol}_{M_\theta ^a \times M_\theta ^a/C_G})$ on its quotient space.

Denote the image of $F^{(r)}$ by $M_{\theta ,r}^a:= F^{(r)}(M_\theta ^a) \subset M_\theta ^a \times M_\theta ^a$ and $M_{\theta ,r,C}^a:= F_C^{(r)}(M_\theta ^a/C_G)$ . We write its conditional Lebesgue measure $\mu _{\theta ,r}^{a}:= \mu _\theta ^{a}|_{M_{\theta ,r}^a}$ and $\mu _{\theta ,r}^{c}:= \mu _\theta ^{c}|_{M_{\theta ,r,c}^a}$ , respectively.

For any open set $U_{\mathcal {S}_C} \subset M_\theta ^a \times M_\theta ^a / C_G$ , we write the pushforward measure $(F_C^{(r)})_* \mathcal {L}_\theta ^{c}$ as

$$ \begin{align*} (F_C^{(r)})_* \mathcal{L}_\theta^{c}(U_{\mathcal{S}_C} \cap M_{\theta,r,C}^a) & = \mathcal{L}_\theta^{c}((F_C^{(r)})^{-1}(U_{\mathcal{S}_C} \cap M_{\theta,r,C}^a))\\ & = \int_{U_{\mathcal{S}_C}} \sum_{x \in (F_C^{(r)})^{-1}(\{z\}), z \in U_{\mathcal{S}_C}} \frac{1}{\mathrm{Jac}(F_C^{(r)}(x)) } d \mathrm{vol}_{M_{\theta,r,c}^a}(z). \end{align*} $$

By the compactness of $M_\theta ^a$ (or $M_\theta ^a/C_G$ ), the above expression is finite. By Lemma 5.3, the Jacobian of $F_C^{(r)}$ is constant and $(F_C^{(r)})_* \mathcal {L}_\theta ^{c} = \mu _{\theta ,r}^{c}$ is Lebesgue.

By invariance of the action of the centralizer, for any neighborhood $U_{\mathcal {S}} \in M_\theta ^a \times M_\theta ^a$ with $U_{\mathcal {S}_C} = U_{\mathcal {S}}/C_G$ ,

(52) $$ \begin{align} \mu_{\theta,r}^{c}(U_{\mathcal{S}_C} \cap M_{\theta,r,C}^a) = \mu_{\theta,r}^{a}(U_{\mathcal{S}}\cap M_{\theta,r}^a), \end{align} $$

and by definition of conditional measure,

(53) $$ \begin{align} \mu_{\theta,r}^{a}(U_{\mathcal{S}}\cap M_{\theta,r}^a) = \mu_{\theta}^{a}(U_{\mathcal{S}}). \end{align} $$

Let d be a distance function in $M_\theta ^a \times M_\theta ^a$ , and we abuse notation d for the induced distance on $M_\theta ^a \times M_\theta ^a/C_G$ . Let $U_{\delta } = \{ z \in M_\theta ^a \times M_\theta ^a \mid d(z,\mathcal {S}) < \delta \}$ be a $\delta $ -tubular neighborhood of $\mathcal {S}$ and $U_{\delta ,C} = \{ z \in M_\theta ^a \times M_\theta ^a/C_G \mid d(z,\mathcal {S}_C) < \delta \}$ be its quotient.

Define the almost-periodic set (set of rth close return) on the diagonal

(54) $$ \begin{align} AP^{r}(\mathcal{U}_\delta) := \{ x \in M_\theta^a \mid d(F^{(r)}(x),\mathcal{S}) < \delta \}. \end{align} $$

Since $F^{(r)}$ commutes with $C_G$ , $AP^{r}(\mathcal {U}_\delta )/C_G = \{ x \in M_\theta ^a/C_G \mid d(F_C^{(r)}(x),\mathcal {S}_C) < \delta \}$ and $ \mathcal {L}_\theta ^{a}(AP^{r}(\mathcal {U}_\delta )) = \mathcal {L}_\theta ^{c}(AP^{r}(\mathcal {U}_\delta )/C_G) $ .

Figure 1

Illustration of displacement $F^{(r)}$ in the product $M_\theta ^a \times M_\theta ^a$ and comparison with the uniform expanding map.

The following volume estimate of almost-periodic set holds.

Lemma 5.4. Let $U_{\delta ,C}$ be any tubular neighborhood of $\mathcal {S}_C$ in $M_\theta ^a \times M_\theta ^a/C_G$ . For all $r \in \mathbb {Z} \backslash \{0\}$ , the conditional measure $ \mathrm {vol}_{M_\theta ^a}$ of $AP^{r}(\mathcal {U}_\delta )$ is given as follows:

$$ \begin{align*}\mathcal{L}_\theta^{a}(AP^{r}(\mathcal{U}_\delta)) = \mu_{\theta,r}^{c}(U_{\delta,C} \cap M_{\theta,r,C}^a).\end{align*} $$

Recall that $F_C^{(r)}: M_\theta ^a/C_G \rightarrow M_{\theta ,r,C}^a$ has non-zero constant Jacobian if $r \neq 0$ by Lemma 5.3 and it is a local diffeomorphism. Thus, by the transversality of $F_C^{(r)}$ , in a small tubular neighborhood U, $F_C^{(r)}$ is covering.

Set $N_r(z) = \#\{x \in M_\theta ^a/C_G \mid F_C^{(r)}(x) = z\}$ as the number of pre-images of $F_C^{(r)}$ . The number $N_r(z)$ is independent of the choice of $z \in U\cap M_{\theta ,r,C}^a$ since Jacobian is constant and the degree of the map is invariant (see [Reference Dubrovin, Fomenko and NovikovDAS, §3]).

We now introduce the volume estimate of the $\delta $ -neighborhood $U_{\delta ,C}$ .

Proposition 5.6. The following volume estimate holds: for any $r \neq 0$ , there exists $C:= C(M_\theta ^a)>0$ such that

$$ \begin{align*}\mu_{\theta,r}^{c}(U_{\delta,C}\cap M_{\theta,r,C}^a) < C\delta.\end{align*} $$

Proof. Let $U \subset M_\theta ^a \times M_\theta ^a/C_G$ be a tubular neighborhood of $\mathcal {S}_C$ that contains $U_{\delta ,C}$ with the following condition:

(55) $$ \begin{align} \mathrm{vol}_{M_\theta^a \times M_\theta^a/C_G}(U_{\delta,C}) = \delta \mathrm{vol}_{M_\theta^a \times M_\theta^a/C_G}(U). \end{align} $$

If $U\cap M_{\theta ,r,C}^a = \emptyset $ , then there is nothing to prove since $U_{\delta ,C}\cap M_{\theta ,r,C}^a = \emptyset $ . Assume $z \in U\cap M_{\theta ,r,C}^a$ and let $\{J_k\}_{k \geq 1}$ be connected components of $(F_C^{(r)})^{-1}(U \cap M_{\theta ,r,C}^a)$ . We first claim that $F_C^{(r)}|_{J_k}$ is injective.

Given $z \in U\cap M_{\theta ,r,C}^a$ , assume that there exist $x_1 \neq x_2 \in J_k$ for some k such that $z = F_C^{(r)}|_{J_k}(x_1) = F_C^{(r)}|_{J_k}(x_2)$ . Let $\gamma : [0,1] \rightarrow J_k$ be a path that connects $\gamma (0) = x_1$ and $\gamma (1) = x_2$ . Set the lift of the path $\tilde \gamma = F_C^{(r)}|_{J_k} \circ \gamma : [0,1] \rightarrow U$ . Then $\tilde \gamma (0) = \tilde \gamma (1) = z$ and $\tilde \gamma $ is a loop in U. Since U is simply connected, $\tilde \gamma $ is contractible and there exists a homotopy of the path $g_s: [0,1] \rightarrow U$ such that $g_0 = \tilde \gamma $ is homotopic to a constant loop $g_1 = c$ by fixing two end points $F_C^{(r)}|_{J_k}(x_1) = F_C^{(r)}|_{J_k}(x_2) = z$ for $s \in [0,1]$ .

Note that $F_C^{(r)}|^{-1}_{J_k} \circ g_s$ is a lift of homotopy $g_s$ , and a lift of $g_0$ is $\gamma = F_C^{(r)}|^{-1}_{J_k}(\tilde \gamma )$ with fixed end points $x_1$ and $x_2$ . By homotopy continuity, $g_s$ also keeps the same end points $x_1$ and $x_2$ fixed for all $s \in [0,1]$ . Since $g_1$ is constant loop and its lift should be a single point, $\gamma $ is homotopic to a constant. Since the end points of $\gamma $ are fixed, they have to be constant, but this leads a contradiction. Therefore, we have $x_1= x_2$ .

By injectivity of $F_C^{(r)}|_{J_k}$ , we obtain

$$ \begin{align*}(F_C^{(r)})^{-1}(U) = (F_C^{(r)})^{-1}(U \cap M_{\theta,r,C}^a) = \bigcup_{k=1}^{N_r} J_k.\end{align*} $$

Furthermore, we obtain the following equality:

(56) $$ \begin{align} \mathrm{vol}_{M_\theta^a/C_G}(J_k) = \frac{ \mathrm{vol}_{M_{\theta,r,C}^a}(U \cap M_{\theta,r,C}^a)}{\mathrm{Jac}(F_C^{(r)}|_{J_k})} = \frac{ \mathrm{vol}_{M_{\theta,r,C}^a}(U \cap M_{\theta,r,C}^a)}{\mathrm{Jac}(F_C^{(r)})}. \end{align} $$

Since the volume of $M_\theta ^a/C_G$ is a finite,

(57) $$ \begin{align} N_r\bigg(\frac{ \mathrm{vol}_{M_\theta^a \times M_\theta^a/C_G}(U)}{\mathrm{Jac}(F_C^{(r)})}\bigg) &= N_r\bigg(\frac{ \mathrm{vol}_{M_{\theta,r,C}^a}(U \cap M_{\theta,r,C}^a)}{\mathrm{Jac}(F_C^{(r)})}\bigg) \end{align} $$

(58) $$ \begin{align} &\qquad\qquad\qquad\qquad\, \, \, = \sum^{N_r}_{k=1} \mathrm{vol}_{M_\theta^a/C_G}(J_k) < \infty. \end{align} $$

Assume that $(F_C^{(r)})^{-1}(U_{\delta ,C}) = \bigcup _{k=1}^{N_r} (F_C^{(r)}|_{J_k})^{-1}(U_{\delta ,C})$ . Then by (56) and definition of conditional measure,

$$ \begin{align*} \mathrm{vol}_{M_\theta^a/C_G}((F_C^{(r)})^{-1}(U_{\delta,C})) &= \sum^{N_r}_{k=1} \mathrm{vol}_{M_\theta^a/C_G}({(F_C^{(r)}|_{J_k})^{-1}}(U_{\delta,C})) \\ &= N_r \bigg(\frac{ \mathrm{vol}_{M_{\theta,r,C}^a}(U_{\delta,C} \cap M_{\theta,r,C}^a)}{{\mathrm{Jac}(F_C^{(r)})}}\bigg) \\ & = N_r\bigg(\frac{ \mathrm{vol}_{M_\theta^a \times M_\theta^a/C_G}(U_{\delta,C})}{\mathrm{Jac}(F_C^{(r)})}\bigg). \end{align*} $$

By the last equality, together with condition (55),

(59) $$ \begin{align} \mathrm{vol}_{M_\theta^a/C_G}((F_C^{(r)})^{-1}(U_{\delta,C})) = N_r\bigg(\frac{ \delta \mathrm{vol}_y{M_\theta^a \times M_\theta^a/C_G}(U)}{\mathrm{Jac}(F_C^{(r)})}\bigg). \end{align} $$

Therefore, combining (57) and (59), there exists $C>0$ such that

$$ \begin{align*} \mu_{\theta,r}^{c}(U_{\delta,C}) = (F_C^{(r)})_* \mathrm{vol}_{M_\theta^a/C_G}(U_{\delta,C}) = \mathrm{vol}_{M_\theta^a/C_G}((F_C^{(r)})^{-1}(U_{\delta,C})) < C\delta.\\[-40pt] \end{align*} $$

Definition 5.7. For any basis $Y = \{Y_1,\ldots Y_a \}$ of codimension- $1$ ideal $\mathfrak {I}$ of $\mathfrak {n}$ , let $I:=I(Y)$ be the supremum of all constants $I' \in (0,\tfrac 12)$ such that, for any $x \in M$ , the map

(60) $$ \begin{align} \phi_x^Y : (s_1,\ldots,s_a) \mapsto x \exp\bigg(\sum_{i=1}^{a}s_i{Y_i}\bigg) \in M \end{align} $$

is a local embedding (injective) on the domain

$$ \begin{align*}\{\textbf{s} \in \mathbb{R}^a \mid |s_i| < I' \text{ for all }i = 1,\ldots,a\}.\end{align*} $$

For any $x, x' \in M$ , set local distance $d_*$ (measured locally in the Lie algebra) on the transverse section $M_\theta ^a$ along the $Y_i$ direction by $d_{Y_i}({x,x'}) = |s_i|$ if there is $\textbf {s}: = (s_1,\ldots , s_a) \in [-I/2,I/2]^a $ such that

$$ \begin{align*}x' = x\exp\bigg(\sum_{i=1}^as_iY_i\bigg),\end{align*} $$

otherwise $d_{Y_i}({x,x'}) =I$ .

Recall the projection map $pr_1 : M \rightarrow \mathbb {T}^{n+1}$ onto the base torus. On the transverse manifold, for all $\theta \in \mathbb {T}^1$ , let $pr_\theta : M_\theta ^a \rightarrow \mathbb {T}^n$ be the restriction to $M_\theta ^a$ . Then, by applying formula (10) to projection to the base torus,

(61) $$ \begin{align} d_{Y_i}(pr_\theta(\Phi_{\alpha,\theta}^r(x)),pr_\theta(x)) = r\alpha_i, \quad 1\leq i \leq n. \end{align} $$

For any $L \geq 1$ , $r \in \mathbb {Z}$ , $x \in M_\theta ^a $ and given scaling factor $\rho = (\rho _1,\ldots , \rho _a) \in [0,1)^a$ , we define

(62) $$ \begin{align} \begin{aligned} \epsilon_{r,L} &:= \max_{1\leq i \leq n}\min\{I, L^{\rho_i} d_{Y_i}(\Phi^r_{\alpha, \theta}(x),x) \},\\ \delta_{r,L}(x) &:= \max_{n+1 \leq i \leq a}\min\{I, L^{\rho_i}d_{Y_i}(\Phi^r_{\alpha, \theta}(x),x) \}. \end{aligned} \end{align} $$

We note that the distance $d_{Y_i}(\Phi ^r_{\alpha , \theta }(x),x)$ on the generator level does not depend on the choice of x. For this reason, we split the cases $\epsilon _{r,L}$ and $\delta _{r,L}(x)$ to control the distance separately on higher steps.

The condition $\epsilon _{r,L} < \epsilon < I$ and $\delta ' < \delta _{r,L}(x) < \delta < I$ are equivalent to saying

(63) $$ \begin{align} \Phi_{\alpha,\theta}^r(x) = x \exp\bigg(\sum_{i=1}^as_i{Y_i}\bigg) \end{align} $$

for some vectors $\textbf {s}: = (s_1,\ldots , s_a) \in [-I/2,I/2]^a $ such that

$$ \begin{align*} & |s_i| < \epsilon L^{-\rho_i}\quad \text{for all } i \in \{1,\ldots, n\}, \\ & |s_i| < \delta L^{-\rho_i}\quad \text{for all } i \in \{n+1,\ldots, a\}, \\ & |s_j|> \delta' L^{-\rho_j}\quad \text{for some } j \in \{n+1,\ldots, a\}. \end{align*} $$

For every $r \in \mathbb {Z} \backslash \{0\}$ and $j \geq 0$ , let ${AP^r_{j,L}} \subset M$ be a set defined as follows:

(64) $$ \begin{align} {AP_{j,L}^{r}} = \begin{cases} \emptyset & \text{ if } \epsilon_{r,L}>\frac{I}{2},\\ (\delta_{r,L})^{-1}\bigg((2^{-(j+1)}I, 2^{-j}I]\bigg) & \text{ otherwise.} \end{cases} \end{align} $$

In the next lemma, the Lebesgue measure of the set of almost-periodic points ${AP_{j,L}^{r}}$ on M is estimated by the volume of the $\delta $ -neighborhood $\mathcal {U}_\delta $ .

Lemma 5.8. For all $r \in \mathbb {Z} \backslash \{0\}$ , $j \in \mathbb {N}$ , $L \geq 1$ , the $(a+1)$ dimensional Lebesgue measure of the set $AP^r_{j,L}$ can be estimated as follows: there exists $C>0$ such that

$$ \begin{align*}\mathcal{L}^{a+1}(AP^r_{j,L}) \leq \frac{C I^{a-n}}{2^{j(a-n)}} L^{-\sum_{i=n+1}^a\rho_i}.\end{align*} $$

Proof. Without loss of generality, we assume that $AP^r_{j,L} \neq \emptyset $ .

By Tonelli’s theorem,

(65) $$ \begin{align} \mathcal{L}^{a+1}(AP^r_{j,L}) = \int_{0}^1 \mathcal{L}_\theta^{a}(AP^r_{j,L} \cap M_\theta^a)d\theta. \end{align} $$

Recall the definition $AP^{r}(\mathcal {U}_\delta )$ in (54). Choose $\delta = {I^{a-n}}/{2^{j(a-n)}} L^{-\sum _{i=n+1}^a\rho _i}$ and set $ \mathcal {U}_{\delta }^{L,j}:= \mathcal {U}_{\delta }$ . Then we claim that $ {AP_{j,L}^{r}} \cap M_\theta ^a \subset AP^{r}(\mathcal {U}_{\delta }^{L,j})$ .

By identification of $x \in M_\theta ^a$ with $ (x,x)$ in the diagonal $\Delta \subset M_\theta ^a \times M_\theta ^a$ , the local distance $d_{Y_i}(\Phi ^r_{\alpha , \theta }(x),x)$ is identified by the distance function d in the product $M_\theta ^a \times M_\theta ^a$ . Thus, $x \in AP^r_{j,L} \cap M_\theta ^a$ implies that $d(F^{(r)}(x),\mathcal {S}) < \delta $ . That is, $AP^r_{j,L} \cap M_\theta ^a \subset AP^r(\mathcal {U}_\delta ^{L,j})$ . By Lemma 5.4, the volume estimate follows

$$ \begin{align*} \mathcal{L}_\theta^{a}(AP^r_{j,L} \cap M_\theta^a) &\leq \mathcal{L}_\theta^{a}(AP^{r}(\mathcal{U}_\delta^{L,j})) = \mu_{\theta,r}^{c}(\mathcal{U}_{\delta,C}^{L,j} \cap M_{\theta,r,C}^a). \end{align*} $$

Finally, by Proposition 5.6,

$$ \begin{align*}\mu_{\theta,r}^{c}(\mathcal{U}_{\delta,C}^{L,j} \cap M_{\theta,r,C}^a) \leq \frac{CI^{a-n}}{2^{j(a-n)}} L^{-\sum_{i=n+1}^a\rho_i}.\end{align*} $$

Thus the proof follows from formula (65).

5.2 Expected width bounds

We prove a bound on the average width of an orbit on a nilmanifold with respect to a scaled basis. This subsection follows along the lines of [Reference Flaminio and ForniFF14, §5.2]. To complete of the proof, we repeat the similar arguments in nilmanifolds under transverse conditions.

For $L\geq 1$ and $r \in \mathbb {Z} \backslash \{0\}$ , let us consider the function

(66) $$ \begin{align} h_{r,L} = \sum_{j=1}^{\infty} \min\bigg\{2^{j(a-n)},\bigg(\frac{2}{\epsilon_{r,L}}\bigg)^n \bigg\} \chi_{{AP_{j,L}^r}}. \end{align} $$

Define the cut-off function $J_{r,L} \in \mathbb {N}$ by the formula

(67) $$ \begin{align} J_{r,L} := \max\bigg\{j\in \mathbb{N} \mid 2^{j(a-n)} \leq \bigg(\frac{2}{\epsilon_{r,L}}\bigg)^n \bigg\}. \end{align} $$

The function $h_{r,L}$ is

(68) $$ \begin{align} h_{r,L} = \sum_{j=1}^{J_{r,L}}2^{j(a-n)}\chi_{{AP_{j,L}^r}} + \sum_{j>J_{r,L}}\bigg(\frac{2}{\epsilon_{r,L}}\bigg)^n\chi_{{AP_{j,L}^r}}. \end{align} $$

For every $L\geq 1$ , let $\mathcal {F}_{\alpha }^{(L)}$ be the rescaled strongly adapted basis

(69) $$ \begin{align} \mathcal{F}_{\alpha}^{(L)} & = (X_{\alpha}^{(L)},Y_1^{(L)},\ldots,Y_a^{(L)}) = (LX_{\alpha},L^{-\rho_1}Y_1,\ldots,L^{-\rho_a}Y_a). \end{align} $$

For $(x,T) \in M \times \mathbb {R}$ , let $w_{\mathcal {F}_{\alpha }^{(L)}}(x,T)$ denote the average width of the (scaled) orbit segment

$$ \begin{align*}\gamma^T_{X_{\alpha}^{(L)}}(x) : = \{ \phi_{X_{\alpha}^{(L)}}^t(x) \mid 0 \leq t \leq T \}.\end{align*} $$

We prove a bound for the average width of the orbit arc in terms of the following function:

(70) $$ \begin{align} H_{L}^T := 1 + \sum_{|r|=1}^{[TL]}h_{r,L}. \end{align} $$

Definition 5.9. For $t \in [0,T]$ , we define a set of points $\Omega (t) \subset \{t\}\times \mathbb {R}^a$ as follows.

Case 1. If $\phi ^t_{X_{\alpha }^{(L)}}(x) \notin \bigcup _{|r|=1}^{[TL]}\bigcup _{j>0} AP^r_{j,L}$ , let $\Omega (t)$ be the set of all points $(t,s_1,\ldots ,s_a)$ such that

$$ \begin{align*}|s_i| < I/4, \ \ i \in \{1,\ldots,a\}.\end{align*} $$

If $\phi ^t_{X_{\alpha }^{(L)}}(x) \in \bigcup _{|r|=1}^{[TL]}\bigcup _{j>0} AP^r_{j,L}$ , then we consider two subcases.

Case 2-1. If $\phi ^t_{X_{\alpha }^{(L)}}(x) \in \bigcup _{|r|=1}^{[TL]}\bigcup _{j>J_{r,L}} AP^r_{j,L}$ , let $\Omega (t)$ be the set of all points $(t,\textbf {s})$ such that

$$ \begin{align*} &|s_i| < \tfrac{1}{4}\min_{1\leq |r| \leq [TL]}\min_{j>J_{r,L}}\{\epsilon_{r,L}:\phi^t_{X_{\alpha}^{(L)}}(x) \in AP^r_{j,L} \} \quad \text{for } i \in \{1,\ldots,n\},\\ &|s_i| < \frac{I}{4} \quad \text{for} \ i \in \{n+1,\ldots,a\}. \end{align*} $$

Case 2-2. If $\phi ^t_{X_{\alpha }^{(L)}}(x) \in \bigcup _{|r|=1}^{[TL]}\bigcup _{j\leq J_{r,L}} AP^r_{j,L}\backslash \bigcup _{|r|=1}^{[TL]}\bigcup _{j>J_{r,L}} AP^r_{j,L}$ , let l be the largest integer such that

$$ \begin{align*}\phi^t_{X_{\alpha}^{(L)}}(x) \in \bigcup_{|r|=1}^{[TL]}\bigcup_{ l \leq j\leq J_{r,L}} AP^r_{j,L}\backslash \bigcup_{|r|=1}^{[TL]}\bigcup_{j>J_{r,L}} AP^r_{j,L},\end{align*} $$

and let $\Omega (t)$ be the set of all points $(t, \textbf {s})$ such that

$$ \begin{align*} &|s_i| < \frac{I}{4} \quad \text{for } i \in \{1,\ldots,n\},\\ &|s_i| < \frac{I}{4}\frac{1}{2^{l+1}} \quad \text{for} \ i \in \{n+1,\ldots,a\}. \end{align*} $$

We set

$$ \begin{align*}\Omega := \bigcup_{t \in [0,T]} \Omega(t) \subset [0,T] \times [-I/4,I/4]^a \subset [0,T] \times \mathbb{R}^a.\end{align*} $$

Lemma 5.10. The restriction to $\Omega $ of the map

(71) $$ \begin{align} (t,\textbf{s})\in \Omega \mapsto x\exp(tX_{\alpha}^{(L)})\exp\bigg(\sum_{i=1}^as_i Y_i^{(L)}\bigg) \end{align} $$

is injective.

Proof. Assume there exist $t'\geq t$ and $(t,s_1,\dotsc,s_n) \neq (t',s'_1, \dotsc, s'_n)$ such that

(72) $$ \begin{align} \phi_{X_{\alpha}^{(L)}}^t(x) \exp\bigg(\sum_{i=1}^as_i{Y_i^{(L)}}\bigg) = \phi_{X_{\alpha}^{(L)}}^{t'}(x) \exp\bigg(\sum_{i=1}^as^{\prime}_i{Y_i^{(L)}}\bigg). \end{align} $$

By considering the projection on the base torus, we have the following identity:

(73) $$ \begin{align} (t,s_1,\ldots,s_n) \text{ mod } \mathbb{Z}^{n+1} & = pr_1(\phi_{X_{\alpha}^{(L)}}^t(x))\nonumber \\ &= pr_1(\phi_{X_{\alpha}^{(L)}}^{t'}(x)) = (t',s^{\prime}_1,\ldots,s^{\prime}_n) \text{ mod } \mathbb{Z}^{n+1}, \end{align} $$

which implies $t \equiv t'$ modulo $\mathbb {Z}$ . As $\phi _{X_{\alpha }}^{tL} = \phi ^t_{X_{\alpha }^{(L)}}$ , the number $r_0 = t' - t$ is a non-negative integer satisfying $r_0 \leq TL$ ; hence $r_0 \leq [TL]$ .

If $r_0=0$ , then $t' = t$ and $s^{\prime }_i = s_i$ for all $1\leq i \leq a$ . Then injectivity is obtained by definition of I. Assume that $r_0\neq 0$ . Let $p,q \in M_\theta ^a$ . Then we have

$$ \begin{align*}p: = \phi^t_{X_{\alpha}^{(L)}}(x), \quad q: = \phi^{t'}_{X_{\alpha}^{(L)}}(x) \Longrightarrow q = \Phi_{\alpha,\theta}^{r_0}(p).\end{align*} $$

From identity (72) we have

(74) $$ \begin{align} q & = p \exp\bigg(\sum_{i=1}^as_i{Y_i^{(L)}}\bigg)\exp\bigg(-\sum_{i=1}^as^{\prime}_i{Y_i^{(L)}}\bigg) \nonumber\\ & = p\exp\bigg( \sum_{i=1}^a(s_i'-s_i + P_i(s_i,s_i'))L^{-\rho_i}Y_i \bigg) \end{align} $$

where $P_i$ is polynomial expression following from the Baker–Cambell–Hausdorff formula.

Note that $P_i(s_i,s_i') = 0$ if $i = 1,\ldots , n$ and $|P_i(s_i,s_i')| \leq \sum _{l = 1}^\infty 1/2|s_ls_l'|^l$ for $i>n$ . Since $|s_i|, |s_i'| \leq {I}/{4} \ll 1$ ,

$$ \begin{align*}q = p\exp\bigg( \sum_{i=1}^a(s_i'-s_i + \epsilon_i)L^{-\rho_i}Y_i \bigg) \quad \text{for some } \epsilon_i \in [0,I_\epsilon)\end{align*} $$

where $I_\epsilon = \sum _{l=1} ({I}/{4})^l = {I}/({4-I}) < I/3.$ Thus for all $i \in \{1,\ldots ,a\}$ ,

(75) $$ \begin{align}L^{\rho_i} d_{Y_i}(p, \Phi_{\alpha,\theta}^{r_0}(p)) = L^{\rho_i}|(s_i'-s_i + \epsilon_i)L^{-\rho_i}| \leq |s_i'|+|s_i| + |\epsilon_i|\end{align} $$

and

$$ \begin{align*}\epsilon_{r_0,L} = \max_{1\leq i \leq n}L^{\rho_i} d_{Y_i}(p, \Phi_{\alpha,\theta}^{r_0}(p)) \leq \max_{1\leq i \leq n} \{|s_i| +|s_i'|+|\epsilon_i|\} \leq \tfrac{5}{6}I.\end{align*} $$

For the same reason, from formula (74) we also obtain that

$$ \begin{align*}\delta_{r_0,L}(p) = \delta_{-r_0,L}(q) < I/2.\end{align*} $$

By defining $j_0 \in \mathbb {N}$ as the unique non-negative integer such that

$$ \begin{align*}\frac{I}{2^{j_0+1}} \leq \delta_{r_0,L}(p) \leq \frac{I}{2^{j_0}}\end{align*} $$

and by the Definition 5.7, we have $p \in AP^{r_0}_{j_0,L}$ and $q \in AP^{-r_0}_{j_0,L}$ .

If $j_0> J_{r_0,L} = J_{-r_0,L},$ then $p,q \in \bigcup _{|r|=1}^{[TL]}\bigcup _{j \geq J_{r,L}}$ . It follows that the sets $\Omega (t)$ and $\Omega (t')$ are both defined in case 2-1. Hence by (75),

$$ \begin{align*}\epsilon_{r_0,L} \leq \max_{1\leq i \leq n} \{|s_i| +|s_i'|+|\epsilon_i|\} \leq \tfrac{5}{6}\epsilon_{r_0,L},\end{align*} $$

which is a contradiction.

If the map in formula (71) fails to be injective at points $(t,s)$ and $(t',s')$ with $t \geq t'$ , then there are integers $r_0 \in [1,TL], j_0 \in [1,J(|r_0|)]$ and $\theta \in \mathbb {T}^1$ such that the points p and q satisfy

$$ \begin{align*}q = \Phi_{\alpha,\theta}^{r_0}(p), \quad p,q \notin \bigcup_{|r|=1}^{[TL]}\bigcup_{ j> J_{r,L}} AP^r_{j,L}.\end{align*} $$

In this case, the sets $\Omega (t)$ and $\Omega (t')$ are both defined according to case (2-2). Let $l_1$ and $l_2$ as the largest integers such that

$$ \begin{align*}p \in \bigcup_{|r|=1}^{[TL]}\bigcup_{l_1 \leq j \leq J_{r,L}} AP^r_{j,L} \quad \text{and} \quad q \in \bigcup_{|r|=1}^{[TL]}\bigcup_{l_2 \leq j \leq J_{r,L}} AP^r_{j,L}.\end{align*} $$

In case (2-2), we have

$$ \begin{align*}|s_i| < \frac{I}{4} \frac{1}{2^{l_1+1}}, \quad |s_i'| < \frac{I}{4} \frac{1}{2^{l_2+1}},\quad \text{for all } i\in \{n+1,\ldots,a\},\end{align*} $$

which also leads to a contradiction because $l_1,l_2> j_0$ deduce the contradiction

$$ \begin{align*}\frac{I}{2^{j_0+1}}\leq \delta_{r_0,L}(p) &\leq \max_{ i \geq n+1} \{|s_i| +|s_i'|+|\epsilon_i|\}\\ &< \frac{1}{4}\frac{I}{2^{l_1+1}} + \frac{1}{4}\frac{I}{2^{l_2+1}} + \bigg(\frac{1}{32}\bigg) I^2/2^{l_1+l_2+2} \leq \frac{3}{4} \frac{I}{2^{j_0+1}}.\end{align*} $$

Hence, the injectivity is proved.

We simply re-prove the bound on the averaged width (see [Reference Flaminio and ForniFF14, Lemma 5.5]) in the general settings (under transversality conditions) by combining with Lemma 5.10.

Lemma 5.11. For all $x \in M$ and for all $T, L \geq 1$ we have

$$ \begin{align*}\frac{1}{w_{\mathcal{F}_{\alpha}^{(L)}}(x,T)} \leq \bigg(\frac{2}{I} \bigg)^a \frac{1}{T}\int_{0}^{T}H^{T}_{L}\circ \phi^{t}_{X_{\alpha}^{(L)}}(x)\,dt.\end{align*} $$

Proof. The width function $w_{\Omega }$ of the set $\Omega $ is given by

(76) $$ \begin{align} w_\Omega(t) = \begin{cases} \bigg(\dfrac{I}{2}\bigg)^a & \text{case 1,} \\ \bigg(\dfrac{I}{2}\bigg)^a\bigg(\dfrac{\min_{1 \leq |r| \leq TL}\{\epsilon_{r,L}\}}{2}\bigg)^n & \text{case 2-1,} \\ \bigg(\dfrac{I}{2}\bigg)^a2^{-(a-n)(l+1)} & \text{case 2-2}, \end{cases} \end{align} $$

and implies that

(77) $$ \begin{align} \frac{1}{w_\Omega(t)} \leq \begin{cases} \bigg(\dfrac{2}{I}\bigg)^{a-n} & \text{case 1,} \\ \bigg(\dfrac{2}{I}\bigg)^{a-n} \displaystyle\sum_{|r|=1}^{[TL]}\sum_{j>J_{r,L}}\frac{2^{n}\chi_{AP_{j,L}^r}(\phi_{X_{\alpha}^{(L)}}^t(x))}{(\epsilon_{r,L})^n} & \text{case 2-1,} \\ \bigg(\dfrac{2}{I}\bigg)^a \displaystyle\sum_{|r|=1}^{[TL]}\sum_{j>J_{r,L}}2^{(j+1)(a-n)}\chi_{AP_{j,L}^r}(\phi_{X_{\alpha}^{(L)}}^t(x)) & \text{case 2-2}. \end{cases} \end{align} $$

By the definition of the function $H_{L}^T$ in formula (70), we have

(78) $$ \begin{align} \frac{1}{w_\Omega(t)} \leq \bigg(\frac{2}{I}\bigg)^a H_{L}^T\circ \phi^{t}_{X_{\alpha}^{(L)}}(x)\quad \text{for all } t \in [0,T]. \end{align} $$

From the definition (40) of the average width of the orbit segment $\{x\exp {(tX_{\alpha }^{(L)})} \mid 0 \leq t \leq T\}$ , we have the estimate

$$ \begin{align*}\frac{1}{w_{\mathcal{F}_{\alpha}^{(L)}}(x,T)} \leq \frac{1}{T}\int_{0}^{T}\frac{dt}{w_{\Omega}(t)} \leq \bigg(\frac{2}{I}\bigg)^a\frac{1}{T}\int_{0}^{T}H_{L}^T\circ \phi^{t}_{X_{\alpha}^{(L)}}(x)\,dt. \\[-42pt] \end{align*} $$

Lemma 5.12. For all $r \in \mathbb {Z} \backslash \{0\}$ and for all $L \geq 1$ , the following estimate holds:

$$ \begin{align*}\bigg|\int_{M} h_{r,L}(x)\,dx\bigg| \leq CI^{a-n}(1+J_{r,L})L^{-\sum_{i=n+1}^a\rho_i}.\end{align*} $$

Proof. It follows from Lemma 5.8 that for $r \neq 0$ and for all $j \geq 0$ , the Lebesgue measure of the set $AP^r_{j,L}$ satisfies the following bound:

(79) $$ \begin{align} \mathcal{L}^{a+1}(AP^r_{j,L}) \leq \frac{C I^{a-n}}{2^{j(a-n)}} L^{-\sum_{i=n+1}^a\rho_i}. \end{align} $$

From formula (68), it follows that

$$ \begin{align*} \int_{M} h_{r,L}(x)\,dx \leq 1& + \sum_{j=1}^{J_{r,L}}2^{j(a-n)} \mathcal{L}^{a+1}({AP_{j,L}^r})\\ & + \sum_{j>J_{r,L}} \frac{2^n\mathcal{L}^{a+1}({AP_{j,L}^r})}{(\epsilon_{r,L})^n}. \end{align*} $$

By the estimate in formula (79), we immediately have that

$$ \begin{align*}\sum_{j=1}^{J_{r,L}} 2^{j(a-n)}\mathcal{L} ^{a+1}(AP_{j,L}^r)\leq C I^{a-n}J_{r,L} L^{-\sum_{i=n+1}^a\rho_i}.\end{align*} $$

By the definition of the cut-off in formula (67) we have the bound

$$ \begin{align*}\frac{2^{n-(J_{r,L}+1)(a-n)}}{(\epsilon_{r,L})^n} \leq 1,\end{align*} $$

and by an estimate on a geometric sum

$$ \begin{align*} \sum_{j>J_{r,L}} \frac{2^n\mathcal{L}^{a+1}({AP_{j,L}^r})}{(\epsilon_{r,L})^n} &\leq \frac{2^{n-(J_{r,L}+1)(a-n)}}{(\epsilon_{r,L})^n}CI^{a-n}L^{-\sum_{i=n+1}^a\rho_i} \\ &\leq CI^{a-n}L^{-\sum_{i=n+1}^a\rho_i}.\\[-38pt] \end{align*} $$

5.3 Diophantine estimates

We review the simultaneous Diophantine condition.

Definition 5.13. A vector $\alpha \in \mathbb {R}^n\backslash \mathbb {Q}^n$ is simultaneously Diophantine of exponent $\nu \geq 1$ , say $\alpha \in DC_{n,\nu }$ if there exists a constant $c(\alpha )>0$ such that, for all $r\in \mathbb {N} \backslash \{0\} $ ,

$$ \begin{align*}\min_i\|r \alpha_i\| = d(r\alpha, \mathbb{Z}^n) = \|r\alpha\| \geq \frac{c(\alpha)}{r^{({\nu}/{n})}}.\end{align*} $$

Definition 5.14. For any basis $\bar {Y}:= \{ \bar {Y}_1, \ldots , \bar {Y}_n\} \subset \mathbb {R}^n$ , let $\bar {I}:= \bar {I}(\bar {Y})$ be the supremum of all constants $\bar {I'}>0$ such that the map

$$ \begin{align*}(s_1,\ldots,s_n) \rightarrow \exp\bigg(\sum_{i=1}^ns_i\bar{Y}_i\bigg) \in \mathbb{T}^n\end{align*} $$

is a local embedding on the domain

$$ \begin{align*}\{\textbf{s} \in \mathbb{R}^n \mid |s_i| < \bar{I}' \text{ for all } i=1,\ldots,n \}.\end{align*} $$

For any $\theta \in \mathbb {R}^n$ , let $[\theta ] \in \mathbb {T}^n$ be its projection onto the torus $\mathbb {T}^n := \mathbb {R}^n/\mathbb {Z}^n$ and let

$$ \begin{align*}|\theta|_1 = |s_1|, \ldots, |\theta|_i = |s_i|, \ldots, |\theta|_n = |s_n|,\end{align*} $$

if there is $\textbf {s}:=(s_1,\ldots , s_n) \in [-\bar {I}/2, \bar {I}/2]^n$ such that

$$ \begin{align*}[\theta] = \exp\bigg(\sum_{i=1}^ns_i\bar{Y}_i\bigg) \in \mathbb{T}^n;\end{align*} $$

otherwise we set $|\theta |_1 = \cdots = |\theta |_n = \bar {I}$ .

Definition 5.15. (Counting for close return time)

Let $\sigma = (\sigma _1,\ldots ,\sigma _n) \in (0,1)^n$ be such that $\sigma _1+\cdots +\sigma _n = 1$ . For any $\alpha = (\alpha _1,\ldots ,\alpha _n) \in \mathbb {R}^n$ , for any $N \in \mathbb {N}$ and every $\delta>0$ , let

$$ \begin{align*}R_\alpha(N,\delta) = \{r\in [-N,N] \cap \mathbb{Z} \mid |r\alpha|_1\leq \delta^{\sigma_1},\ldots, |r\alpha|_n\leq \delta^{\sigma_n}\}.\end{align*} $$

For every $\nu>1 $ , let $D_n(\bar {Y},\sigma , \nu ) \subset (\mathbb {R}\backslash \mathbb {Q})^n$ be the subset defined as follows: the vector $\alpha \in D_n(\bar {Y},\sigma , \nu )$ if and only if there exists a constant $C(\bar {Y},\sigma , \alpha )>0$ such that for all $N \in \mathbb {N}$ for all $\delta>0$ ,

(80) $$ \begin{align} \# R_\alpha (N,\delta) \leq C(\bar{Y}, \sigma, \alpha)\max\{N^{1-({1}/{\nu})}, N\delta\}. \end{align} $$

The Diophantine condition for counting return time implies a standard simultaneous Diophantine condition.

Lemma 5.16. [Reference Flaminio and ForniFF14, Lemma 5.9]

Let $\alpha \in D_n$ . For all $r \in \mathbb {Z}\backslash \{0\}$ , we have

$$ \begin{align*}\max\{|r\alpha|_1,\ldots, |r\alpha|_n\} \geq \min\bigg\{\frac{\bar{I}^2}{4}, \frac{1}{[1+C(\bar{Y},\sigma,\alpha)]^{2\nu}} \bigg\}\frac{1}{|r|^\nu}.\end{align*} $$

For any vector $\sigma = (\sigma _1, \ldots , \sigma _n) \in (0,1)^n$ such that $\sigma _1+\cdots +\sigma _n = 1$ , let

$$ \begin{align*}m(\sigma) = \min\{\sigma_1,\ldots, \sigma_n\} \quad \text{and} \quad M(\sigma) = \max\{\sigma_1,\ldots, \sigma_n\}.\end{align*} $$

Lemma 5.17. [Reference Flaminio and ForniFF14, Lemma 5.12]

For all bases $\bar {Y} \subset \mathbb {R}^n$ , for all $\sigma = (\sigma _1, \ldots , \sigma _n) \in (0,1)^n$ such that $\sigma _1+\cdots +\sigma _n = 1$ and for all $\nu \geq 1$ , the inclusion

$$ \begin{align*}DC_{n,\nu} \subset D_n(\bar{Y},\sigma,\nu)\end{align*} $$

holds under the assumption

$$ \begin{align*}\mu \leq \min\bigg\{\nu, \bigg[\frac{M(\sigma)}{\nu}+1-\frac{1}{n}\bigg]^{-1}, \bigg[\frac{1}{\nu} + \bigg(1-\frac{2}{n}\bigg)\bigg(1-\frac{m(\sigma)}{M(\sigma)}\bigg) \bigg]^{-1}\bigg\}.\end{align*} $$

The set $D_n(\bar {Y},\sigma ,\nu )$ has full measure if

(81) $$ \begin{align} \frac{1}{\nu} < \min\bigg\{ [M(\sigma)n]^{-1},1-\bigg(1-\frac{2}{n}\bigg)\bigg(1-\frac{m(\sigma)}{M(\sigma)}\bigg) \bigg\}. \end{align} $$

In dimension $1$ , the vector space has unique basis up to scaling. The following result is immediate.

Lemma 5.18. [Reference Flaminio and ForniFF14, Lemma 5.13]

For all $\nu \geq 1$ the following identity holds:

$$ \begin{align*}DC_{1,\nu} = D_1(\nu).\end{align*} $$

Let $\mathcal {F}_\alpha := (X_\alpha , Y)$ be a basis and let $\bar {Y} = \{\bar {Y}_1,\ldots , \bar {Y}_{n} \} \in \mathbb {R}$ denote the projection of the basis of codimension- $1$ ideal $\mathfrak {I}$ onto the abelianized Lie algebra $\bar {\mathfrak {n}} := \mathfrak {n}/[\mathfrak {n},\mathfrak {n}] \approx \mathbb {R}^{n}$ . For $\rho = (\rho _1,\ldots ,\rho _a) \in [0,1)^a$ , we write a vector of scaling exponents

$$ \begin{align*}\bar{\rho} = (\rho_1,\ldots,\rho_n), \quad |\bar{\rho}| = \rho_1+\cdots + \rho_n.\end{align*} $$

Let $\alpha _1 = (\alpha ^{(1)}_1, \ldots , \alpha ^{(1)}_{n} ) \in D_n(\bar {Y}, \bar {\rho }/|\bar {\rho }|,\nu )$ . For brevity, let $C(\bar {Y}, \bar {\rho }/|\bar {\rho }|,\alpha _1)$ denote the constant appearing in (80) for $\alpha _1 \in D_n(\bar {Y}, \bar {\rho }/|\bar {\rho }|,\nu )$ . Let

(82 $$ \begin{align} C(\alpha_1) = 1+C(\bar{Y}, \bar{\rho}/|\bar{\rho}|,\alpha_1). \end{align} $$

We prove the upper bound on the cut-off function in formula (67). Let $I = I(Y)$ and $\bar {I} = \bar {I}( \bar {Y})$ be the positive constant introduced in Definitions 5.7 and 5.14. We observe that $I \leq \bar {I} $ since the basis $\bar {Y}$ is the projection of the basis $Y \subset \mathfrak {n}'$ and the canonical projection commutes with the exponential map. Then the following logarithmic upper bound holds.

Lemma 5.19. [Reference Flaminio and ForniFF14, Lemma 5.14]

For every $\rho \in [0,1)^a$ , for every $\nu \leq 1/|\bar {\rho }|$ and for every $\alpha \in D_n(\bar {Y}, \bar {\rho }/|\bar {\rho }|,\alpha )$ , there exists a constant $K>0$ such that for all $T \geq 1$ and for all $r \in \mathbb {Z} \backslash \{0\}$ , the following bound holds:

$$ \begin{align*}J_{r,L} \leq K\{1+\log^+[I(Y)^{-1}]+\log C(\alpha_1)\}(1+\log|r|).\end{align*} $$

Proof. By Lemma 5.16 and by the definition of $\epsilon _{r,L}$ in formula (62), it follows that, for all $T>0, L\geq 1$ and for all $r \in \mathbb {Z} \backslash \{0\}$ , we have

$$ \begin{align*}\epsilon_{r,L} \geq \max_{1\leq i \leq n}\min\{I, |r\alpha_1|_i\} \geq \min\bigg\{I, \frac{\bar{I}^2}{4}, \frac{1}{[1+C(\alpha_1)]^{2\nu}} \bigg\}\frac{1}{|r|^\nu}.\end{align*} $$

It follows by the bound above and by the definition of the cut-off function (67) that

$$ \begin{align*}J_{r,L} \leq \frac{n}{a-n}(3\log2+3\log^{+}(1/I)+2\nu \log[1+C(\alpha_1)]+\nu\log|r| ).\\[-40pt] \end{align*} $$

Assume that there exists $\nu \in 1/|\bar {\rho }|$ such that $\alpha _1 \in D_n(\bar {Y},\bar {\rho }/|\bar {\rho }|,\nu )$ . For brevity, we introduce the following notation:

(83) $$ \begin{align} \mathcal{H}(Y,\rho,\alpha) = 1+ I(Y)^{a-n}C(\alpha_1)\{1+\log^+[I(Y)^{-1}]+\log{C(\alpha_1)} \}. \end{align} $$

Theorem 5.20. [Reference Flaminio and ForniFF14, Theorem 5.15]

For every $\rho \in [0,1)^a$ , for every $\nu \leq 1 / |\bar {\rho }|$ such that $\alpha _1 = \alpha _i^{(1)} \in D_n(\bar {Y},\bar {\rho },\nu ) $ there exists a constant $K'>0$ such that for all $T>0$ and for all $L \geq 1$ , the following bound holds:

$$ \begin{align*}\bigg|\int_{M}H_{L}^T(x)\,dx\bigg| \leq K'\mathcal{H}(Y,\rho,\alpha)(1+T)(1+\log^+ T+\log L)L^{1-\sum_{i=1}^a\rho_i}.\end{align*} $$

Proof. By the definition of $H_{L}^T$ in formula (70), the statement follows from Lemmas 5.12 and 5.19. In fact, for all $r \in \mathbb {Z} \backslash \{0\}$ and $j \geq 0$ , by definition (64) the set $AP_{j,L}^{r}$ is non-empty only if $\epsilon _{r,L}<({I}/{2})$ . Since $\nu \leq 1/|\bar {\rho }|$ , it follows from the definition of the Diophantine class $D_n$ that

$$ \begin{align*}\# \{r \in [-TL,TL] \cap \mathbb{Z}\backslash \{0\} \mid AP_{j,L}^{r} \neq \emptyset \} \leq C(\bar{Y}, \sigma, \alpha_1)(1+T)L^{1-|\bar{\rho}|}.\end{align*} $$

Hence, the statement follows from Lemmas 5.12 and 5.19.

The main idea of the proof above follows from Lemma 5.12 which is based on the estimate to the upper bound of the measure of the almost periodic set $AP^r_{j,L}$ . In Lemma 5.8, this bound is independent of choice of transverse section $M_\theta ^a$ .

5.4 Width estimates along orbit segments

In this subsection we introduce the definition of good points, which is crucial in controlling the average width estimate.

Definition 5.21. For any increasing sequence $(T_i)$ of positive real numbers, let $h_i \in [1,2]$ denote the ratio $\log {T_i}/[\log T_i]$ for every $T_i \geq 1$ . Set $N_i = [\log T_i]$ and $T_{j,i} = e^{jh_i}$ for integer $j \in [0,N_i]$ .

Let $\zeta>0$ and $w>0$ . A point $x \in M$ is $(w,T_i,\zeta )$ -good for the basis $\mathcal {F}_\alpha $ if, setting $y_i = \phi _{X_{\alpha }}^{T_i}(x)$ , then for all $i \in \mathbb {N}$ and for all $0 \leq j \leq N_i$ ,

$$ \begin{align*}w_{\mathcal{F}_\alpha^{(T_{j,i})}}(x,1) \geq w/T_i^\zeta, \quad w_{\mathcal{F}_\alpha^{(T_{j,i})}}(y_i,1) \geq w/T_i^\zeta.\end{align*} $$

Under the transversality condition, the following proof is still valid and extendable from the quasi-abelian case. For the completion of the subsection, we repeat the proof of the following lemma.

Lemma 5.22. [Reference Flaminio and ForniFF14, Lemma 5.18]

Let $\zeta>0$ be fixed and let $ (T_i)$ be an increasing sequence of positive real numbers satisfying the condition

(84) $$ \begin{align} \Sigma((T_i),\zeta) := \sum_{i\in \mathbb{N}}(\log T_i)^2T_i^{-\zeta} < \infty. \end{align} $$

Let $\rho \in [0,1)$ with $\sum \rho _i =1$ . Then the Lebesgue measure of the complement of the set $\mathcal {G}(w, (T_i), \zeta )$ of $(w,(T_i),\zeta )-$ good points is bounded above. That is, $\text { there exists } K>0$ such that

$$ \begin{align*}\mathrm{meas}( \mathcal{G}(w, (T_i), \zeta)^c) \leq K \Sigma((T_i),\zeta)[1/I(Y)]^a \mathcal{H}(Y,\rho,\alpha)w.\end{align*} $$

Proof. For all $i \in \mathbb {N}$ and for all $j = 0, \ldots , N_i$ , let

$$ \begin{align*}\mathfrak{S}_{j,i} = \{z \in M : w_{\mathcal{F}_\alpha^{(T_{j,i})}}(z,1) < T_i^\zeta/w \}.\end{align*} $$

By definition we have

(85) $$ \begin{align} \mathcal{G}(w,(T_i),\zeta)^c = \bigcup_{i\in \mathbb{N}} \bigcup_{j=0}^{N_i}(\mathfrak{S}_{j,i} \cup \phi_{X_{\alpha}}^{-T_i}(\mathfrak{S}_{j,i})). \end{align} $$

By Lemma 5.11 for all $z \in \mathfrak {S}_{j,i}$ we have

$$ \begin{align*}(I/2)^aT_i^\zeta/w < \int_{0}^1 H^1_{T_{j,i}}\circ \phi_{X_\alpha^{(T_{j,i})}}^\tau(z)\,d\tau = \frac{1}{T_{j,i}}\int_{0}^{T_{j,i}}H^1_{T_{j,i}}\circ \phi_{X_{\alpha}}^ \tau(z)\,d\tau.\end{align*} $$

It follows that

$$ \begin{align*}\mathfrak{S}_{j,i} \subset \mathfrak{S}(j,i) := \bigg\{ z \in M : \sup_{J>0} \frac{1}{J}\int_{0}^JH^1_{T_{j,i}}\circ \phi_{X_{\alpha}}^\tau(z)d\tau > (I/2)^a T_i^\zeta/w \bigg\}.\end{align*} $$

By the maximal ergodic theorem, the Lebesgue measure $\mathrm {meas}(\mathfrak {S}_{j,i})$ of the set $\mathfrak {S}(j,i) $ satisfies the inequality

$$ \begin{align*}\mathrm{meas}(\mathfrak{S}_{j,i}) \leq \mathrm{meas}(\mathfrak{S}(j,i)) = (2/I)^a (w/T_i^\zeta) \int_{M} H^1_{T_{j,i}}(z)\,dz.\end{align*} $$

Let $\mathcal {H} = \mathcal {H}(Y,\rho ,\nu )$ denote the constant defined in formula (83). By Theorem 5.20, since by hypothesis $\nu \leq 1/|\bar {\rho }|$ and $\alpha \in D_n(\bar {\rho }/|\bar {\rho }|,\nu )$ , there exists a constant $K':= K'(a,n,\nu )>0 $ such that the following bound holds:

$$ \begin{align*}\bigg|\int_{M} H^1_{T_{j,i}}(z)\,dz \bigg| \leq K' \mathcal{H}(1+\log T_{j,i}).\end{align*} $$

Hence, by the definition of the $T_{j,i}$ , we have

(86) $$ \begin{align} N_i \leq \log T_i \leq N_i +1, \ \ \log T_{j,i} \leq 2j. \end{align} $$

Thus, for some constant $K'' $ , we have

$$ \begin{align*}\mathrm{meas}(\mathfrak{S}_{j,i}) \leq K''(2/I)^a \mathcal{H} w(1+j)T_i^{-\zeta}.\end{align*} $$

By (86), for some constant $K'''>0$ ,

$$ \begin{align*}\mathrm{meas}\bigg(\bigcup_{j=0}^{N_i} \mathfrak{S}_{j,i} \cup \phi_{X_{\alpha}}^{-T_i}(\mathfrak{S}_{j,i})\bigg) \leq K''' (2/I)^a\mathcal{H} w(\log T_i)^2 T_i^{-\zeta}.\end{align*} $$

By sub-additivity of the Lebesgue measure, we derive the bound

$$ \begin{align*}\mathrm{meas}\bigg(\bigcup_{i\in \mathbb{N}}\bigcup_{j=0}^{N_i} \mathfrak{S}_{j,i} \cup \phi_{X_{\alpha}}^{-T_i}(\mathfrak{S}_{j,i})\bigg) \leq K''' \Sigma((T_i),\zeta)\mathcal{H} w.\end{align*} $$

By formula (85), the above estimate concludes the proof.

6.1 Coboundary estimates for rescaled basis

Recall the definition of degree of $Y_{i}^{(m)}$ and

$$ \begin{align*}d_{i}^{(m)} = \begin{cases} k-m\quad & \text{for all } 1 \leq m \leq k-1, \\ 0, \quad & m = k. \end{cases} \end{align*} $$

For any linear functional $\Lambda $ , the degree of the representation $\pi _{\Lambda }$ only depends on its coadjoint orbits. We denote the scaling vector $\rho \in ({\mathbb {R}^+})^J$ such that

$$ \begin{align*}\sum_{(i,j) \in J} \rho_{i}^{(j)} = 1 \quad \text{and}\quad \rho_{i}^{(j)} = 0\quad \text{for } \deg(Y_i^{(j)})=0.\end{align*} $$

Assume that the number of bases of $\mathfrak {n}$ with degree $k-m$ is $n_m$ . Define

(87) $$ \begin{align} S_{\mathfrak{n}}(k) &:= (n_1-1)(k-1) + n_2 (k-2) + \cdots + n_{k-1}, \end{align} $$

(88) $$ \begin{align} \delta(\rho) &:= \min_{\substack{1 \leq m \leq k-1 \\ 1 \leq i,j \leq n_m }}\{\rho_{i}^{(m)}-\rho_{j}^{(m+1)}, \ \rho_{i}^{(m)}-\rho_{i}^{(m+1)} \}. \end{align} $$

We have $\delta (\rho ) \leq \unicode{x3bb} (\rho )$ . This inequality is strict unless one has homogeneous scaling

$$ \begin{align*} \rho^{(j)}_i = \frac{d_j}{S_{\mathfrak{n}}}\quad \text{for } j \leq k-1. \end{align*} $$

Lemma 6.3. There exists a constant $C>0$ such that, for all $r \in \mathbb {R}^+$ and for any function $f \in W^r(H_{\mathcal {O}}) $ , we have

$$ \begin{align*}\sum_{(m,i) \in J}|[X_{\alpha}(t),Y^{(m)}_i(t)]f|_{r,\mathcal{F}_{\alpha,\Lambda}(t)} \leq Ce^{t(1-\delta(\rho))}|f|_{r+1,\mathcal{F}_{\alpha,\Lambda}(t)}.\end{align*} $$

Proof. For all $(m,i) \in J$ , we have

$$ \begin{align*}[X_{\alpha}(t),Y_i^{(m)}(t)] = \sum_{l\geq 1} c_l^{(m+1)}e^{t(1-\rho_{i}^{(m)}+\rho_{l}^{(m+1)} )}Y_l^{(m+1)}(t). \end{align*} $$

We note that $c^{(j)}_l = 0$ for $j = k$ and for some l, which is determined by the commutation relation. Setting $C = \max _{(i,j)\in J^+ }\{|c_i^{(j)}|\} $ ,

$$ \begin{align*} \sum_{(m,i) \in J}|[X_{\alpha}(t),Y^{(m)}_i(t)]f|_{r,\mathcal{F}_{\alpha,\Lambda}(t)} \leq Ce^{t(1-\delta(\rho))}\sum_{(m,i) \in J^+}|Y^{(m)}_i(t)f|_{r,\mathcal{F}_{\alpha,\Lambda}(t)}.\\[-46pt] \end{align*} $$

For $x \in M$ , let $\gamma _x$ be the Birkhoff average operator

$$ \begin{align*}\gamma_x^T(f) = \frac{1}{T}\int_{0}^Tf\circ\phi_{X_{\alpha}}^s(x)\,ds.\end{align*} $$

Consider the decomposition of the restriction of linear functional $\gamma _x$ to $W^r_0(H_{\mathcal {O}},\mathcal {F}_{\alpha ,\Lambda }(t))$ as an orthogonal sum $\gamma _x = D(t)+R(t) \in W^{-r}_0(H_{\mathcal {O}},\mathcal {F}_{\alpha ,\Lambda }(t))$ of an $X_{\alpha }$ -invariant distribution $D(t)$ and an orthogonal complement $R(t)$ .

Theorem 6.4. Let $r> (k + 1)(a/2 + 1) + 1/4 $ . For $g \in W^r(H_{\mathcal {O}},\mathcal {F}_{\alpha ,\Lambda }(t))$ and for all $t \geq 0$ , there exists a constant $C_r^{(1)}>0$ such that

(89) $$ \begin{align} |R(t)(g)| &\leq C_r^{(1)}e^{(1-\delta(\rho)-(1-\unicode{x3bb}))t}\max\{1,\delta_{\mathcal{O}}^{-4(r-1)}\} \notag \\ &\quad \times T^{-1}(w_{\mathcal{F}_{\alpha,\Lambda}(t)} (x,1)^{-({1}/{2})}+w_{\mathcal{F}_{\alpha,\Lambda}(t)}(\phi_{X_{\alpha}}^T(x),1)^{-({1}/{2})} )|{g}|_{r,\mathcal{F}_{\alpha,\Lambda}(t)}. \end{align} $$

Proof. Fix $t \geq 0$ and set $ D = D(t), R = R(t)$ for convenience. Let $g \in W^r(H_{\mathcal {O}},{\mathcal F}_{\alpha,\Lambda}(t))$ . We write $g = g_D + g_R$ , where $g_R$ is the kernel of $X_{\alpha }$ -invariant distributions and $g_D$ is orthogonal to $g_R$ in $W^r(H_{\mathcal {O}},{\mathcal F}_{\alpha,\Lambda}(t))$ . Then $g_R$ is a coboundary and $R(g_D)=0$ . Let $f = G^{X_\alpha (t)}_{X_\alpha ,\Lambda } (g_R)$ . From $D(g_R)=0$ ,

(90) $$ \begin{align} |R(g)| & = |R(g_D + g_R)| = |R(g_R)| = |\gamma_x(g_R) - D(g_R)| = |\gamma_x(g_R)|. \end{align} $$

By the Gottschalk–Hedlund argument,

(91) $$ \begin{align} \begin{split} |\gamma_x(g_R)| & = \bigg|\frac{1}{T}\int_0^Tg\circ\phi_{X_{\alpha}}^s(x)\,ds\bigg| = \frac{1}{T}|f\circ\phi^T_{X_{\alpha}}(x) - f(x)|\\ & \leq \frac{1}{T}(|f(x)| + |f\circ\phi^T_{X_{\alpha}}(x)|). \end{split} \end{align} $$

By Theorem 4.4 and Lemma 6.3, for any $\tau> a/2+1$ , there exists a positive constant $C_r$ such that for any $z \in M$ ,

(92) $$ \begin{align} |f(z)| \leq \frac{C_r}{w_{\mathcal{F}_{\alpha,\Lambda}(t)}(z,1)^{{1}/{2}}}(Ce^{(1-\delta(\rho))t}|f|_{\tau,\mathcal{F}_{\alpha,\Lambda}(t)}+|g|_{\tau-1,\mathcal{F}_{\alpha,\Lambda}(t)} ). \end{align} $$

By Theorem 3.12, if $r> (k+1)\tau +1/4$ , then

$$ \begin{align*}|f|_{\tau,\mathcal{F}_{\alpha,\Lambda}(t)} \leq C_{r,k,\tau} e^{-(1-\unicode{x3bb})t} \max\{1, \delta_{\mathcal{O}}^{-4\tau(k+1)}\}|g_R|_{r,\mathcal{F}_{\alpha,\Lambda}(t)}.\end{align*} $$

By orthogonality, we have $|g_R|_{r,\mathcal {F}_{\alpha ,\Lambda }(t)} \leq |g|_{r,\mathcal {F}_{\alpha ,\Lambda }(t)}$ .

Corollary 6.5. For every $r> (k+1)(a/2+1)+1/4 $ , there is a constant $C_r^{(2)}>0$ such that the following holds for every $\mathcal {O} \in \widehat I_0$ and every $x \in M$ :

$$ \begin{align*}|R|_{-r, \mathcal{F}_{\alpha,\Lambda}} \leq C_r^{(2)} [1/I(Y_\Lambda)]^{a/2}\max\{1,\delta_{\mathcal{O}}^{-4(r-1)} \}T^{-1}.\end{align*} $$

Proof. For all $x \in M$ , we have

$$ \begin{align*}w_{\mathcal{F}_{\alpha,\Lambda}}(x,1) \geq \bigg(\frac{I(Y_\Lambda)}{2}\bigg)^a.\end{align*} $$

It follows from Theorem 6.4 applied to the orthogonal decomposition of $\gamma _x = D(0)+R(0)$ .

6.2 Bounds on ergodic averages in an irreducible subrepresentation

In this subsection we derive the bounds on ergodic averages of nilflows for functions in a single irreducible subrepresentation.

For brevity, let us set

(93) $$ \begin{align} C_r(\mathcal{O}) = (1+\delta_{\mathcal{O}}^{-4(r-1)}). \end{align} $$

Proposition 6.6. Assume $(T_i)_{i \in \mathbb{N}}$ is an increasing sequence of positive real numbers $ \geq 1$ . Let $0 < w < I(Y)^a$ and $\zeta>0$ . Given $r> (k+1)(a/2+1)+1/4 $ , there exists a constant $C_r(\rho )$ such that for every $\mathcal {G}(w, (T_i), \zeta )$ -good point $x \in M$ and all $f \in W^r(H_{\mathcal {O}},\mathcal {F})$ , we have

(94) $$ \begin{align} \bigg|\frac{1}{T_i}\int_{0}^{T_i}f\circ \phi_{X_{\alpha}}^t(x)\,dt \bigg| \leq C_{r}(\rho)C_r(\mathcal{O})w^{-1/2}{T_i}^{-\delta(\rho)+\zeta+\unicode{x3bb}/2} |{f}|_{r,\mathcal{F}_\alpha}. \end{align} $$