2.1 Non-flag examples

In the flag case, Green and Tao’s counting lemma computes averages of the form $\mathbb{E}_{\boldsymbol{x}}\prod_{i=1}^t \phi(\psi_i(\boldsymbol{x}))$ where $\phi= f \circ g$ is a suitably ‘irrational’ nilsequence in some filtered nilmanifold $(G/\Gamma, G_\bullet)$ . Roughly speaking, if $G_\bullet$ is the ‘correct’ filtered nilmanifold for the polynomial sequence g (i.e. if g cannot be written as a polynomial sequence on cosets of some smaller filtered subnilmanifold), then the distribution of $g^\Psi(\boldsymbol{x}) := (g(\psi_1(\boldsymbol{x})), \ldots, g(\psi_t(\boldsymbol{x})))$ in $G^t/\Gamma^t$ may be determined from the data $\{G_\bullet, \Psi\}$ .

The flag counting lemma says that if g is irrational in $G_\bullet$ , then $g^\Psi$ equidistributes in $G^\Psi / G^\Psi \cap \Gamma^t$ , and so the average $\mathbb{E}_{\boldsymbol{x}} \prod_{i=1}^t\phi(\psi_i(\boldsymbol{x}))$ is asymptotically equal to the integral $\int_{G^\Psi/G^\Psi\cap \Gamma^t}f^{\otimes t}$ .

When moving to non-flag systems of forms, not only does $g^\Psi$ not necessarily equidistribute on the Leibman group, but the notion of irrationality originally considered by Green and Tao does not capture enough information about the polynomial sequence g to determine its distribution. The following example appears in [Reference Green and TaoGT20] and [Reference TaoTao20].

Example 2.2 Let $\Psi:\mathbb{Z}^2 \to \mathbb{Z}^4$ be our non-flag system of linear forms from above:

\[\Psi(x,y) = (\psi_1(x,y),\psi_2(x,y),\psi_3(x,y),\psi_4(x,y))=(y, 2x+2y,x+3y,x).\]

Then, in $\mathbb{R}^4$ ,

\begin{align*}V_\Psi &= \mathrm{span}\{(0,2,1,1),(1,2,3,0)\} = \ker\{(3,0,-1,1), (2,-1,0,2)\},\\V_\Psi^2 &= \mathrm{span}\{(0,4,1,1),(1,4,9,0),(0,4,3,0)\} = \ker\{(24,3,-4,-8)\},\end{align*}

and $V_\Psi^3 = \mathbb{R}^4$ .

Let $\alpha, \beta\in \mathbb{R}$ be such that $\{1,\alpha, \beta\}$ satisfy no nontrivial linear relations over $\mathbb{Q}$ . Let $f: \mathbb{R}^2/\mathbb{Z}^2 \to \mathbb{C}$ be smooth and let $g(n) = (\alpha n, \beta n^2)$ determine a polynomial sequence on $\mathbb{R}^2$ . Then (suppressing in our notation the quotient $\mathbb{R}^2 \to \mathbb{R}^2/\mathbb{Z}^2$ ) we have that $\phi(n):=f(\alpha n, \beta n^2)$ is an irrational nilsequence in the filtered Lie group $\mathbb{R}^2$ with filtration $G_1 = \mathbb{R}^2$ , $G_2 = 0 \times \mathbb{R}$ , essentially because the linear part $(\alpha,0)n$ of g(n) equidistributes in $G_1/G_2\Gamma \cong (\mathbb{R}/\mathbb{Z})\times 0$ , and the quadratic part $(0,\beta)n^2$ equidistributes in $G_2/G_2\cap \Gamma \cong 0 \times \mathbb{R}/\mathbb{Z}$ . (We refer the reader to [Reference Green and TaoGT10, Definition A.6] for a formal definition of the irrationality of a polynomial sequence.)

We would like our counting lemma to determine the average

(1) \begin{equation}\mathbb{E}_{x,y} (f\circ g)(\psi_1(x,y))(f\circ g)(\psi_2(x,y))(f\circ g)(\psi_3(x,y))(f\circ g)(\psi_4(x,y)).\end{equation}

To do so we find where $g^\Psi(x,y):=(g(\psi_1(x,y)),\ldots,g(\psi_4(x,y)))$ distributes within in $(\mathbb{R}^2/\mathbb{Z}^2)^4$ whereupon the average (1) may be computed by the relatively easy task of integrating over a subtorus.

Here the Leibman group is

\[\{(a, b): a \in V_\Psi, b \in V_\Psi+V_\Psi^2\},\]

suitably interpreted as a subgroup of $(\mathbb{R}^2)^4$ . On the other hand,

\[g^\Psi(x,y)= ((0,2,1,1)\alpha x+ (1,2,3,0)\alpha y, (0,4,1,1)\beta x^2 + (1,4,9,0)\beta y^2 +2(0,4,3,0)\beta xy)\]

is constrained to the following subgroup of $G^\Psi$ :

\[\{(a, b): a \in V_\Psi, b \in V_\Psi^2\}.\]

Since $V_\Psi^2 \subsetneq V_\Psi+V_\Psi^2$ we have that $g^\Psi$ takes values in a strict subgroup of the Leibman group and so will certainly not equidistribute there. (Note, on the other hand, that if the flag condition was satisfied by the system of linear forms, i.e. $V_\Psi \leq V_\Psi^2$ so $V_\Psi^2 = V_\Psi+V_\Psi^2$ , then the subgroup is equal to the Leibman group and indeed Green and Tao’s counting lemma holds.)

Now define $\tilde g(n) = (\alpha n , \gamma n + \beta n^2)$ , where $\{1,\alpha,\beta,\gamma\}$ is a $\mathbb{Q}$ -linearly independent set. Then $\tilde g$ is irrational in the same filtration as g and in this instance $\tilde g^\Psi$ is not constrained to the same subgroup of $G^\Psi$ as $g^\Psi$ was. In fact, $\tilde g^\Psi$ equidistributes in $G^\Psi$ . The upshot is that in the non-flag setting, the irrationality of a polynomial sequence in $G_\bullet$ is not enough to determine the distribution of $g^\Psi$ .

In light of this example, one might conjecture in general that the subgroup of $G^t$ on which $g^\Psi$ equidistributes may be expressed in terms of the sequence of subgroups on which the distinct degree parts of g equidistribute. In our example with $g(n) = (\alpha,0)n +(0,\beta)n^2$ and $\tilde g(n) = (\alpha, \gamma)n + (0,\beta)n^2$ , we have $(\alpha,0)n$ equidistributing in $\mathbb{R}\times 0$ whereas $(\alpha,\gamma)n$ equidistributes in $\mathbb{R}^2$ and indeed this precisely accounts for how the distribution of $g^\Psi$ differs from that of $\tilde g^\Psi$ . It turns out that when the nilmanifold in question is abelian, one can indeed compute the distribution of $g^\Psi$ in $G^t/\Gamma^t$ if the distribution of the coefficients of g are known.Footnote ⁴ However, this is not the case in the nonabelian setting, as the following example of linear sequences on the Heisenberg group demonstrates.

Example 2.3. Let G be the Heisenberg group with filtration determined by $G_2 = [G,G]$ . Let $\Psi, \psi_i, V$ be as above and let $\Gamma$ be the obvious lattice on G. Let $v_1 = (1,2,3,0)$ and $v_2=(0,2,1,1)$ so $V_\Psi = \mathrm{span}(v_1,v_2)$ . Let

\[g_1 = \begin{pmatrix}1\quad & \alpha\quad & \gamma \\0\quad & 1\quad & \beta \\0\quad & 0\quad & 1 \\\end{pmatrix}, \quad \tilde g_1 = \begin{pmatrix}1\quad & \alpha\quad & \frac{1}{2}\alpha \beta \\0\quad & 1\quad & \beta \\0\quad & 0\quad & 1 \\\end{pmatrix},\]

where $1,\alpha, \beta, \alpha\beta, \gamma$ satisfy no nontrivial linear relations over $\mathbb{Q}$ . Then both $g(n)=g_1^n$ and $\tilde g(n) = \tilde g_1^n$ equidistribute in $G/\Gamma$ ; however, as we now show, $g^\Psi$ distributes differently to $\tilde g^\Psi$ in $G^4/\Gamma^4$ . (Furthermore, both polynomial sequences are irrational by Definition A.6 of [Reference Green and TaoGT10].)

Both $g^\Psi$ and $\tilde g^\Psi$ take values in the Leibman group $G^\Psi$ . Define $\eta$ on $G^\Psi \leq G^4$ by

\[\eta(h_1,h_2,h_3,h_4) = w \cdot (h_{1,2}, h_{2,2}, h_{3,2}, h_{4,2}),\]

Where

\[h_i = \begin{pmatrix}1\quad & h_{i,0}\quad & h_{i,2} \\0\quad & 1\quad & h_{i,1} \\0\quad & 0\quad & 1\end{pmatrix} \in G\]

for each i and $w = (24,3,-4,-8) \in {V_\Psi^2}^\perp$ . An easy but mundane computation (using crucially that $w \in {V_\Psi^2}^\perp$ ) reveals that, in fact, $\eta$ is a homomorphism from $G^\Psi$ to $\mathbb{R}$ . We compute (with slight abuse of notation where we use a single matrix with vector-valued entries to denote a 4-tuple of matrices)

\[g^\Psi(x,y) = g_1^{\Psi(x,y)} = g_1^{x v_1 + yv_2} = \begin{pmatrix}1\quad & \alpha(xv_1 + yv_2)\quad & \gamma(xv_1 + yv_2) + \binom{xv_1 + yv_2}{2}\alpha \beta \\0\quad & 1\quad &\beta(xv_1 + yv_2) \\0\quad & 0\quad & 1\end{pmatrix}\]

and so

\begin{align*}\eta(g^\Psi(x,y)) &= w\cdot v_1\left(\gamma - \tfrac{1}{2}\alpha \beta\right)x + w \cdot v_2 \left(\gamma -\tfrac{1}{2} \alpha \beta\right)y \\&\quad + w\cdot v_1^2\left(\tfrac{1}{2}\alpha \beta\right)x^2 + 2 w \cdot v_1v_2\left(\tfrac{1}{2}\alpha \beta\right) xy + w \cdot v_2^2 \left(\tfrac{1}{2}\alpha \beta\right)y^2.\end{align*}

Since $w \in {V_\Psi^2}^\perp$ , we have that $\eta(g^\Psi(x,y)) = w\cdot v_1(\gamma -\tfrac{1}{2}\alpha \beta)x + w \cdot v_2 (\gamma - \tfrac{1}{2} \alpha \beta)y$ , where $w\cdot v_1,w\cdot v_2 \ne 0$ . Of course, completing the same computation with $\tilde g$ yields the same result with $\tfrac{1}{2}\alpha \beta$ in place of $\gamma$ and so, in particular, $\eta(\tilde g^\Psi(x,y)) = 0$ . That is, $\tilde g^\Psi$ is constrained to the subgroup of $G^\Psi$ given by $\ker \eta$ whereas $g^\Psi$ is not. (As an application of our qualitative counting lemma, Theorem 2.16, we show in Example 2.19 that $g^\Psi$ in fact equidistributes on $G^\Psi / G^\Psi \cap \Gamma^4$ and $\tilde g^\Psi$ equidistributes on $\ker\eta / \ker \eta \cap \Gamma^4$ .)

2.2 Moving to the Lie algebra

Let us now interpret the previous example in the Lie algebra $\mathfrak{g}$ of the Heisenberg group G. Recall that $\log:G\to \mathfrak{g}$ is a bijection given by

\[\log \begin{pmatrix}1\quad & a\quad & c \\0\quad & 1\quad & b \\0\quad & 0\quad & 1\end{pmatrix} = \begin{pmatrix}0\quad & a\quad & c-\frac{1}{2}ab \\0\quad & 0\quad & b \\0\quad & 0\quad & 0\end{pmatrix}.\]

For convenience and brevity we use the coordinates $(a,b,c-\tfrac{1}{2}ab)$ to denote the latter matrix in the previous sentence. The Lie bracket in these coordinates is

(2) \begin{equation}[(a,b,c),(x,y,z)] = (0,0,ay-bx).\end{equation}

The algebra $\mathfrak{g}$ inherits a filtration $\mathfrak{g}_0=\mathfrak{g}_1=\mathfrak{g}$ and $\mathfrak{g}_2 = [\mathfrak{g},\mathfrak{g}] = \{(0,0,\mathbb{R})\}$ . In these coordinates, we have that $p(n) := \log g(n) = (\alpha,\beta,\gamma - \tfrac{1}{2}\alpha\beta)n$ and $\tilde p(n) := \log \tilde g(n) = (\alpha,\beta,0)n$ .

Let us digress slightly and consider a general polynomial sequence p on some Lie algebra $\mathfrak{g}$ with linear forms $\Psi$ whereupon we are interested in determining where $p^\Psi$ equidistributes within $\mathfrak{g}^t$ .

It will prove convenient to deal with $\mathfrak{g} \otimes \mathbb{R}^t$ rather than $\mathfrak{g}^t$ , where the map from the former to the latter on elementary tensors is given by $\varphi: a \otimes v \mapsto(v_1a,v_2a,\ldots,v_ta)$ . This gives a vector space isomorphism and we may obtain a Lie algebra isomorphism by endowing $\mathfrak{g} \otimes \mathbb{R}^t$ with a Lie bracket as follows:

(3) \begin{align}[a\otimes v,b\otimes w] &:= \varphi^{-1}([(v_1a,\ldots,v_ta),(w_1b,\ldots,w_tb)]) \nonumber\\&= \varphi^{-1}(([v_1a,w_1b],\ldots,[v_ta,w_tb])) \nonumber \\&= \varphi^{-1}((v_1w_1[a,b],\ldots,v_tw_t[a,b])) \nonumber \\&= [a,b]\otimes vw,\end{align}

where multiplication in the second entry is conducted coordinatewise. With this setup, and henceforth suppressing our $\varphi$ notation and implicitly identifying $\mathfrak{g}^t$ with $\mathfrak{g}\otimes\mathbb{R}^t$ , if $p(n)=\sum a_in^i$ , then $p^\Psi(\boldsymbol{x}):=(p(\psi_1(\boldsymbol{x})),\ldots,p(\psi_t(\boldsymbol{x}))) = \sum a_i\otimes \Psi^i(\boldsymbol{x}) \in \mathfrak{g} \otimes \mathbb{R}^t$ .

Returning to our example, $\tilde p^\Psi$ is contained in the subspace $(\mathbb{R},\mathbb{R},0)\otimes V_\Psi$ at all inputs. We claim that $(\mathbb{R},\mathbb{R},0)\otimes V_\Psi + \mathfrak{g}_2 \otimes V_\Psi^2$ is the smallest Lie algebra containing the subspace $(\mathbb{R},\mathbb{R},0)\otimes V_\Psi$ . Indeed, we first observe that $(\mathbb{R},\mathbb{R},0)\otimes V_\Psi + \mathfrak{g}_2 \otimes V_\Psi^2$ is closed under the Lie bracket and so is certainly a Lie algebra which contains $(\mathbb{R},\mathbb{R},0)\otimes V_\Psi$ . On the other hand, for $a,b,x,y\in \mathbb{R}$ and $v,w\in V_\Psi$ ,

\[[(a,b,0)\otimes v, (x,y,0)\otimes w ] := [(a,b,0),(x,y,0)]\otimes vw = (0,0,ay-bx)\otimes vw,\]

and elements of this form span $\mathfrak{g}_2 \otimes V_\Psi^2$ .

We show later as an application of our qualitative counting lemma, Theorem 2.16, in Example 2.19 that in fact $\tilde p^\Psi$ equidistributes in $(\mathbb{R},\mathbb{R},0)\otimes V_\Psi + \mathfrak{g}_2 \otimes V_\Psi^2$ (modulo an appropriate lattice). Importantly, $(\mathbb{R},\mathbb{R},0)\otimes V_\Psi + \mathfrak{g}_2 \otimes V_\Psi^2$ is a proper subalgebra of $\mathfrak{g} \otimes V_\Psi + \mathfrak{g}_2 \otimes V_\Psi^2 = \log G^\Psi$ and indeed $p^\Psi$ equidistributes in $\log G^\Psi$ (again, modulo an appropriate lattice).

2.3 Notions of irrationality

In this subsection we restrict our attention to polynomial sequences p with $p(0) = 0 \in\mathfrak{g}$ . In particular, notions of irrationality are only defined for polynomial sequences satisfying this property.

We begin by introducing some definitions, in particular that of a rational structure on $\mathfrak{g}$ . A rational structure on $\mathfrak{g}$ is a rational Lie subalgebra $\mathfrak{g}_\mathbb{Q}$ of $\mathfrak{g}$ such that the vector space isomorphism $\mathfrak{g} \cong \mathfrak{g}_\mathbb{Q} \otimes \mathbb{R}$ holds. Then, let $\mathfrak{g}_\mathbb{Q}^\ast\subset\mathfrak{g}^\ast$ be the linear maps on $\mathfrak{g}$ which map $\mathfrak{g}_\mathbb{Q}$ to $\mathbb{Q}$ . In the coordinates in our Heisenberg example above, we have implicitly used the rational structure $\mathbb{Q}^3 \leq \mathbb{R}^3$ . In what follows, let $\mathrm{Hom}_\mathbb{Q}(\mathfrak{g},\mathbb{R})$ denote the Lie algebra homomorphisms from $\mathfrak{g}$ to $\mathbb{R}$ which map $\mathfrak{g}_\mathbb{Q}$ to $\mathbb{Q}$ ; we refer to elements of $\mathrm{Hom}_\mathbb{Q}(\mathfrak{g},\mathbb{R})$ as horizontal characters on $\mathfrak{g}$ .Footnote ⁵ Finally, many statements in this document are made with respect to a rational structure $\mathfrak{g}_\mathbb{Q}$ on $\mathfrak{g}$ ; at times the underlying rational structure may not be referenced explicitly.

In the following we invoke Lie’s third theorem to glean the simply connected Lie group G from the Lie algebra $\mathfrak{g}$ .

It is not at all a priori clear that the above is well defined (in particular, that such a lattice exists and, furthermore, that the property of equidistribution is independent of the choice of such lattice). This is addressed in Appendix A.2.

The following theorem of Leibman is our main tool for determining qualitative equidistribution. It says that the above two definitions are equivalent.

We deduce the above statement from Leibman’s original theorem [Reference LeibmanLei05] in Theorem A.8.

For appropriately (i.e. linearly) irrational p, we show in the proof of Theorem 2.16 that $p^\Psi$ is additively irrational in an explicit subalgebra of $\mathfrak{g}^t$ , whereupon we have determined its distribution by Theorem 2.8. Before defining linear irrationality, we transfer Green and Tao’s notion of irrationality [Reference Green and TaoGT10, Definition A.6] to the Lie algebra. For the purposes of our current discussion, we give the following qualitative translation in which one removes the complexity requirement on ‘i-horizontal characters’ [Reference Green and TaoGT10, Definition A.5] and takes the limit as $N\to \infty$ .

(A direct translation works with functions that map $\Gamma$ to $\mathbb{Z}$ , but the above is equivalent and more natural in the qualitative setting.) By basic Lie theory one may interpret this in the Lie algebra with filtration $\mathfrak{g}_\bullet := (\log G_i)_{i=1}^s$ .

The following definition is what one obtains when one lifts qualitative Green–Tao irrationality to the Lie algebra. We call it ‘filtration irrationality’.

It is standard [Reference Corwin and GreenleafCG90, Theorem 5.1.8] that for any lattice $\Gamma \leq G$ , $\mathrm{span}_\mathbb{Q} \log\Gamma$ is a rational structure in $\mathfrak{g}$ . With the rational structure $\mathfrak{g}_\mathbb{Q}$ on $\mathfrak{g}$ chosen in this way, one may check that every l as in Definition 2.11 induces a function $\eta$ on $G_i$ with the properties in Definition 2.9 (and, conversely, every $\eta$ induces an $l\in \mathfrak{g}_\mathbb{Q}^\ast$ which vanishes on $\mathfrak{h}_i$ ).

In the examples in the previous subsection, we were somewhat vague about the notion of irrationality. The reader may wish to return to Examples 2.2 and 2.3 to verify that the polynomial sequences there are filtration irrational and, thus, ultimately that filtration irrationality is insufficient to determine the distribution of $p^\Psi$ in the non-flag setting.

The notion of linear irrationality does not refer to the Lie algebra structure on $\mathfrak{g}$ and so may be defined analogously for vector spaces with fixed rational structures. We note that linear irrationality is robust under rational (vector space) automorphisms.

In our Heisenberg example above, recalling that $\alpha$ , $\beta$ , $\gamma$ and 1 are linearly independent over $\mathbb{Q}$ , we have that $\tilde p(n) =(\alpha,\beta,0)n$ is linearly irrational in $S_\bullet = (S_1:=\mathbb{R} \times \mathbb{R} \times 0,S_2 := 0)$ , but is not linearly irrational in $(\mathbb{R}^3,0)$ , or in $(\mathbb{R}\times \mathbb{R} \times 0 , 0\times 0\times \mathbb{R})$ . Define

\[p'(n) = (\alpha, \beta, 2\alpha+\beta)n + (0,0,\gamma)n^2, \quad p''(n) = (\alpha, \beta, 2\alpha+ \beta+1/3)n + (0,0,\gamma)n^2.\]

Then p’ is linearly irrational in $(\ker(2,1,-1),0 \times 0 \times \mathbb{R})$ . On the other hand, p” is not linearly irrational in any sequence of subspaces of $\mathfrak{g}$ because there is $l \in \mathfrak{g}_\mathbb{Q}^\ast$ with $0 \ne l(a_1)\in \mathbb{Q}$ .

In general, any polynomial sequence which has a coefficient $a_i$ with $0\ne l(a_i) \in \mathbb{Q}$ for some $l \in \mathfrak{g}_\mathbb{Q}^\ast$ is not linearly irrational in any sequence of subspaces. If, on the other hand, all coefficients $\{a_i\}_i$ of p satisfy the property that $l(a_i)\in \mathbb{Q}$ implies $l(a_i)=0$ , then p is linearly irrational in the sequence of rational subspaces $S_\bullet=(S_i)_{i=1}^s$ defined by $S_i = (a_i^\perp \cap \mathfrak{g}_\mathbb{Q}^\ast)^\perp$ .

Linear irrationality is a stronger notion than filtration irrationality in the sense that if p is linearly irrational with respect to some $S_\bullet$ , then there exists some $\mathfrak{g}'_\bullet$ (potentially on a subalgebra $\mathfrak{g}'\leq \mathfrak{g}$ ) with respect to which p is filtration irrational (see Lemma 3.4). The converse is not true, as is demonstrated by the example of pʹʹ above which is filtration irrational with respect to the filtration given by $\mathfrak{g}_1 = \mathfrak{g}$ and $\mathfrak{g}_2= [\mathfrak{g},\mathfrak{g}] = (0,0,\mathbb{R})$ ; indeed, for nontrivial $l\in \mathfrak{g}_\mathbb{Q}^\ast$ which vanishes on $\mathfrak{h}_1 = \mathfrak{g}_2$ , we have that $l((\alpha,\beta, 2\alpha+\beta + 1/3))$ is a nonzero rational linear combination of $\alpha$ and $\beta$ , which cannot lie in $\mathbb{Q}$ since $\alpha$ and $\beta$ have been chosen to be rationally independent (and $(0,0,\gamma)$ is trivially filtration irrational in $\mathfrak{g}_2$ ). Finally, also note that if p is adapted to a rational filtration $\mathfrak{g}_\bullet$ and is linearly irrational with respect to $S_\bullet$ then we automatically have that $S_i \leq \mathfrak{g}_i$ for all i and so $[S_i,S_j] \subset \mathfrak{g}_{i+j}$ for all i,j.

Unfortunately the notions of linear and filtration irrationality are not quite as simple in the quantitative setting. For example, a polynomial sequence may be quantitatively filtration/linearly irrational in two distinct sequences of subspaces; see Example 4.2. The relationship between the two notions of irrationality is also not so clear cut in the quantitative setting (though they are related; see Lemma 7.1). For this reason, we explicitly keep track of the sequence of subspaces with respect to which we are asserting quantitative linear irrationality and independently the filtered Lie algebra with respect to which we are asserting quantitative filtration irrationality.

2.4 The counting lemma

We can now prove a qualitative counting lemma for polynomial sequences on linear patterns. Again we restrict our attention to polynomial sequences with $p(0) = 0\in \mathfrak{g}$ ; in the general case, one simply obtains equidistribution on a coset of the algebra studied in this section. Taking for granted our algebraic preliminaries, Theorem 2.16 is rather straightforward.

The qualitative counting lemma may be stated informally as follows: if a polynomial sequence p is linearly irrational in $S_\bullet $ , then $p^\Psi$ equidistributes in $\mathfrak{g}^\Psi(S_\bullet)$ . We offer a more explicit description of $g^\Psi(S_\bullet)$ in § 3.

We are ready to prove the qualitative counting lemma.

Proof. We use Theorem 2.8 and so show that $p^\Psi$ is additively irrational in $\mathfrak{g}^\Psi(S_\bullet)$ . Let $p(n) := {\sum}_{i=1}^s a_in^i$ . Let $\eta \in\mathrm{Hom}_\mathbb{Q}(\mathfrak{g}^\Psi(S_\bullet), \mathbb{R})$ .Footnote ⁶ In $\mathfrak{g}^\Psi(S_\bullet) \leq \mathfrak{g}\otimes\mathbb{R}^t$ , we have that

(4) \begin{equation}\eta(g^\Psi(\boldsymbol{x})) = {\sum}_{i=1}^s \eta(a_i\otimes \Psi(\boldsymbol{x})^i) = {\sum}_{m \in\mathcal{M}}{\sum}_{i=1}^s \eta(a_i\otimes v_{m,i})m(\boldsymbol{x}),\end{equation}

where the $\{v_{m,i}\}_{m\in \mathcal{M}}$ span $V_\Psi^i$ as in § 1.3. By Lemma 2.15, if $\eta(g^\Psi(\boldsymbol{x})) \in \mathbb{Q}$ for all $\boldsymbol{x} \in \mathbb{Z}^D$ , then each $\eta(a_i \otimes v_{m,i}) \in \mathbb{Q}$ . But each $\eta(\cdot \otimes v_{m,i})$ lies in $\mathfrak{g}_\mathbb{Q}^\ast$ and so by the linear irrationality of $a_i$ in $S_i$ we have that, in fact, $\eta(a_i\otimes v_{m,i}) = 0$ . The kernel of a rational homomorphism is a rational subspace and so $\eta(S_i \otimes v_{m,i})=0$ , since $S_i$ is necessarily the smallest rational subspace containing $a_i$ . Next, the $\{v_{m,i}\}_{m \in \mathcal{M}}$ span $V_\Psi^i$ , so we have $\eta(S_i \otimes V_\Psi^i)= 0$ for all i. Furthermore, since $\eta$ is a Lie algebra homomorphism to an abelian Lie algebra, $\eta$ vanishes on the smallest Lie algebra containing the subspaces $S_i \otimes V_\Psi^i$ and so $\eta$ is trivial on $\mathfrak{g}^\Psi(S_\bullet)$ . This completes the proof.

Note that Theorem 2.16 yields equidistribution statements for polynomial sequences which are not filtration irrational.

We are also able to prove the equidistribution claims we made in Examples 2.2 and 2.3, and in § 2.2.

Moving to Example 2.3, we saw in § 2.2 that the relevant polynomial sequences may be transferred to the Lie algebra as $p(n) = (\alpha, \beta,\gamma-\tfrac{1}{2}\alpha \beta)n$ and $\tilde p(n) = (\alpha, \beta, 0)n$ . Theorem 2.16 says that $p^\Psi$ equidistributes in the smallest Lie algebra containing $\mathbb{R}^3 \otimes V$ , which is $\mathbb{R}^3 \otimes V_\Psi + (0\times 0\times \mathbb{R}) \otimes V_\Psi^2=\mathfrak{g} \otimes V_\Psi + \mathfrak{g}_2 \otimes V_\Psi^2$ . We also obtain that $\tilde p^\Psi$ equidistributes in $(\mathbb{R}\times\mathbb{R} \times 0)\otimes V_\Psi + \mathfrak{g}_2 \otimes V_\Psi^2$ . One may furthermore check that this subalgebra maps to the subgroup $\ker \eta$ from Example 2.3 under the exponential map.

In each of the examples above, computing explicitly the Lie algebra $\mathfrak{g}^\Psi(S_\bullet)$ is not a difficult task. Of course, this may not be the case in general. We address this in § 3.

2.5 Factorisation

We have demonstrated how to obtain an equidistribution result for polynomial sequences which satisfy a special property: linear irrationality. Ultimately, one wants to understand the equidistribution of arbitrary polynomial sequences (we again focus on p with $p(0)=0$ ). The purpose of this section is to show (Proposition 2.28) that one may write an arbitrary p as a product $p = p'\ast r$ where p’ is linearly irrational and r is rational (and, as before, $\ast$ is group multiplication conducted in the Lie algebra). Then, roughly speaking, one may partition $\mathbb{Z}^D$ into appropriately spaced subprogressions where we understand the equidistribution of p on each one.

Part of the power of Green and Tao’s counting lemma is that it requires a surprisingly small amount of data from a polynomial sequence g to determine the distribution of $g^\Psi$ in the flag case. In the non-flag case, we require more data to determine distribution and so the task of reducing a general nilsequence to something that our counting lemma can handle (i.e. factorising for linear irrationality) is inevitably a little more involved. This subsection may be viewed as a warm up for the quantitative factorisation theory in § 5.

Linear irrationality is a linear algebraic notion and indeed one may, by natural linear algebraic arguments, write an arbitrary $a \in \mathfrak{g}$ as the sum of a linearly irrational element and a rational element. This is the content of the upcoming lemma. We do not provide a proof immediately because Lemma 2.20 is a special case of the more general Lemma 2.22, which is proven shortly afterwards.

One may apply this to each coefficient of an arbitrary polynomial sequence p to obtain an additive decomposition $p = p' + r$ , where p’ is linearly irrational and r is rational. This additive decomposition does not, however, immediately determine a factorisation with respect to group multiplication, which is ultimately what is required.

To later obtain our multiplicative factorisation, we need a stronger additive decomposition, Lemma 2.22, of which Lemma 2.20 is a special case. First we need the following definition of linear irrationality relative to a subspace.

In what follows we use $T_\mathbb{Q}$ to denote the space $T \cap \mathfrak{g}_\mathbb{Q}$ , and similarly for other subspaces. If T is rational with respect to $\mathfrak{g}_\mathbb{Q}$ , then $T_\mathbb{Q}$ is a rational vector space of full dimension (and indeed topologically dense) inside T.

Next, we may assume that $U \cap T$ has positive dimension, otherwise again a is already linearly irrational in U by its linear irrationality in $U\cap T$ . We claim that the elements of $\mathcal{B}$ remain linearly independent when restricted to maps on $U \cap T$ . Indeed, if not, choose a nontrivial linear combination of the elements of $\mathcal{B}$ which is the trivial map on $U \cap T$ to obtain a contradiction with the linear irrationality of a in $U\bmod T$ . Furthermore, since U,T are rational subspaces, so too is $U\cap T$ and so $\mathcal{B}$ is linearly independent when restricted to a set of maps on the $\mathbb{Q}$ -vector space $(U \cap T)_\mathbb{Q}$ . Thus, the map from $(U\cap T)_\mathbb{Q}$ to $\mathbb{Q}^{|\mathcal{B}|}$ given by $x \mapsto (l(x))_{l\in \mathcal{B}}$ is full rank and, in particular, surjective. Therefore, there exists $a_r\in (U \cap T)_\mathbb{Q}$ such that $l(a) =l(a_r)$ for all $l \in \mathcal{B}$ and, thus, for all $l \in L$ .

Note that since $a_r \in U_\mathbb{Q}$ , the subspace of $\mathfrak{g}_\mathbb{Q}^\ast$ of elements which are trivial on $U^\perp$ and such that $l(a-a_r) \in \mathbb{Q}$ is also equal to L, and for all of these elements $l(a-a_r) = 0$ . Ultimately, letting $l \in \mathfrak{g}_\mathbb{Q}^\ast$ and writing $l = l'+ l''$ where l’ is trivial on $U_\mathbb{Q}^\perp$ and l” is trivial on $U_\mathbb{Q}$ , we see that if $l(a-a_r)\in \mathbb{Q}$ then $l'(a-a_r) \in \mathbb{Q}$ , so that $l'\in L$ , and so $l(a-a_r)=0$ . Thus $a-a_r$ is linearly irrational in $\ker\{ l\in \mathfrak{g}_\mathbb{Q}^\ast : l(a-a_r) \in \mathbb{Q} \} \leq U$ .

Note that the above lemma does not depend at all on the Lie bracket structure and so holds in vector spaces.

In what follows, let $(p)_i$ denote the ith coefficient of a polynomial sequence p. Applying Lemma 2.22 to each coefficient of a polynomial sequence we obtain the following.

Now we pursue a factorisation with respect to the $\ast$ operation. We need the fact that the set of polynomial maps adapted to $\mathfrak{g}_\bullet$ form a group. This result (in the Lie group, and in greater generality) is originally due to Leibman [Reference LeibmanLei02]. See also [Reference Green and TaoGT12, Proposition 6.2] for a related statement and different proof. The result that we need in the Lie algebra is the following more obvious lemma, which follows easily from the Baker–Campbell–Hausdorff formula (we omit the proof).

\[p = p_1 \ast r_1 \mod \mathfrak{g}_2.\]

Set $T^{(2)}_\bullet := (\mathfrak{g}_2,\mathfrak{g}_2,\ldots, \mathfrak{g}_2)$ . Then $p \ast r_1^{-1}$ is a polynomial sequence adapted to $\mathfrak{g}_\bullet$ by Lemma 2.26, and is linearly irrational in $(S_\bullet^{(1)} + T^{(2)}_\bullet) \bmod T^{(2)}_\bullet$ .Footnote ⁷ Next apply Corollary 2.25 with $U_\bullet =S^{(1)}_\bullet + T^{(2)}_\bullet$ and $T_\bullet = T^{(2)}_\bullet$ to $p\ast r_1^{-1}$ to find $p_2$ which is linearly irrational in some $S^{(2)}_\bullet$ and $r_2$ which is adapted to $(S^{(1)}_\bullet \cap T^{(2)}_\bullet)_\mathbb{Q}$ such that $p\ast r_1^{-1} = p_2 + r_2$ . By the Baker–Campbell–Hausdorff formula and the fact that $[\mathfrak{g},\mathfrak{g}_2] \subset \mathfrak{g}_3$ we then have $p\ast r_1^{-1}= p_2 \ast r_2 \bmod \mathfrak{g}_3$ and so

\[p = p_2 \ast r_2' \mod \mathfrak{g}_3,\]

where $r_2':= r_2 \ast r_1$ is rational by Lemma 2.27. Continue in this way to eventually find $p':=p_s$ , $S'_\bullet := S^{(s)}_\bullet$ and $r:= r_s'$ .

5.1 Additive decomposition for linear irrationality

In this subsection we prove an additive decomposition of vector space elements into linearly irrational, rational and small parts (Proposition 5.4), and as a corollary obtain a similar decomposition for polynomial sequences (Corollary 5.5).

The following decomposition of vector space elements into linearly irrational, rational and error terms allows us to perturb an element which is linearly irrational in a quotient space by elements of the quotient to obtain an element which is linearly irrational in the original space. Note that this result includes the case where the quotient space is trivial (that is, $T=U$ in the upcoming notation), whereupon we obtain a decomposition of an arbitrary element in the vector space.

Let $L_1$ be the $1\times \dim U$ matrix whose entries correspond to l with respect to ${\mathcal{X}}'$ . This is the $m=1$ (i.e. base) case of the following induction. By induction on m, we may suppose that we have an $m \times \dim U$ integer matrix $L=L_m$ whose rows are linearly independent, integer linear maps $U \to \mathbb{R}$ with respect to the basis ${\mathcal{X}}':= {\mathcal{X}}'_{U,{\mathcal{X}}}$ and which possess the following properties. First, the projection of the rows are linearly independent as maps in $(U\cap T)^\ast$ . Furthermore, for each row l, we have that l has complexity $ A^{O(1)}$ with respect to ${\mathcal{X}}'$ and that $\|l \cdot a\|_{\mathbb{R}/\mathbb{Z}} \leq A^{O(1)}\varepsilon$ . We show in the following that either:

1. (i) there are elements $a_s \in U\cap T$ , $a_r \in (U \cap T)_\mathbb{Q}$ such that with respect to ${\mathcal{X}}'$ , $a_r$ is $ A^{O(1)}$ -rational, $\|a_s\|_\infty = A^{O(1)}\varepsilon$ and $a- a_r- a_s$ is $(A,\varepsilon)$ -linearly irrational in $\ker L \leq U$ (i.e. the proof is complete upon observing that with respect to ${\mathcal{X}}$ , $a_s$ is still $A^{O(1)}\varepsilon$ -small and $a_r$ is still $A^{O(1)}$ -rational); or

2. (ii) we may find some $L_{m+1}$ which fits the description above with $m+1$ in place of m.

Note that in situation (i), we set $U' = \ker L$ . In situation (ii), the constants implicit in the O(1) notation may change when we pass from m to $m+1$ . However, this can occur only finitely many times since $\dim (U\cap T) = O(1)$ and the rows of each $L_m$ are linearly independent in $(U\cap T)^\ast$ . In particular, we must, at some point, find ourselves in situation (i), completing the proof. It remains to prove the above claim.

Let $L = L_m$ be as above. For $i=1, \ldots, m$ , we have by supposition that $l_i\cdot a = n_i +e_i$ , for some integer $n_i$ and $0 < |e_i| < \varepsilon$ . We may write this simultaneously for $i=1,\ldots ,k $ as $L a = n + e$ where n,e are vectors consisting of the $n_i,e_i$ , respectively.

We claim that there is a solution to $Lx = n$ with $x \in U\cap T$ and such that each entry of x with respect to ${\mathcal{X}}'$ lies in $({1}/{A^{O(1)}})\mathbb{Z}$ ; we let this be $a_r$ . Let M be the change of basis matrix from ${\mathcal{X}}''={\mathcal{X}}''_{{\mathcal{X}}, U\cap T}$ to ${\mathcal{X}}'$ , so LM is the linear map corresponding to L where input vectors are given with respect to ${\mathcal{X}}''$ . Note that M is an integer matrix since $\langle {\mathcal{X}}' \rangle = U\cap \langle {\mathcal{X}} \rangle \supset {\mathcal{X}}''$ and, furthermore, complexity bounds ensure that M has entries of size at most $A^{O(1)}$ . Recall that by the induction statement L has rank m as a family of linear maps restricted to $U \cap T$ . Thus, LM has rank m. Let L’ be the invertible square submatrix of LM obtained by choosing m linearly independent columns. Then it is a standard linear algebra fact that the simultaneous equations $L'x' = n$ are solvable in $({1}/{|\det L'|})\mathbb{Z}$ . Observing that $|\det L'| = A^{O(1)}$ and then padding the solution x’ with $\dim (U\cap T) - m$ zero entries to form an appropriate vector $\tilde x$ yields a solution $LM\tilde x = n$ in $(({1}/{A^{O(1)}})\mathbb{Z})^{\dim U\,\cap\, T}$ . Setting $x= M\tilde x$ proves the claim made at the beginning of the paragraph.

Next, we claim that there is a solution to $Lx = e$ with $x \in U\cap T$ and $\|x\|_\infty =A^{O(1)}\varepsilon$ . This may be proven, for example, by choosing L’ as above, setting $x' = L'^{-1}e$ , upper bounding $\|L'^{-1}\|_\infty$ directly using Cramer’s rule and then augmenting x’ with zero entries to form a vector $\tilde x$ with $LM\tilde x = e$ . Let $a_s = M\tilde x \in U\cap T \subset U$ so that $La_s = e$ and $\|a_s\|_\infty = A^{O(1)}\varepsilon$ (where here the $\ell_\infty$ norm is of course with respect to the basis ${\mathcal{X}}'$ ).

Now, $a_p := a-a_r-a_s$ is contained in $\ker L$ . If $a_p$ is $(A,\varepsilon)$ -linearly irrational in $\ker L$ , then we are in situation (i) and we are done, so suppose not. Then there is some nontrivial integer linear map l of complexity at most A in $\ker L$ such that $\|l(a_p)\|_{\mathbb{R}/\mathbb{Z}} < \varepsilon$ . In addition, it follows from definitions that $\ker L$ is of complexity $A^{O(1)}$ and so by Lemma 4.10 we may find an integer linear map $\tilde l\in U^\ast$ of complexity $ A^{O(1)}$ such that $\tilde l|_{\ker L} = A^{O(1)}l$ . Furthermore, since l is nontrivial on $\ker L$ , we have that $\tilde l$ does not lie in the row span of L. We claim that, in fact, (the projection of) the rows of L and $\tilde l$ are linearly independent in $(U\cap T)^\ast$ . Suppose not for a contradiction with the $(A^c,A^c\varepsilon)$ -linear irrationality of a in $U\bmod T$ . Then there is some linear combination $b\tilde l + {\sum}_{i=1}^m b_il_i =0\in(U \cap T)^\ast$ , where $b \ne 0$ because $\{l_1,\ldots,l_m\}$ are linearly independent in $(U\cap T)^\ast$ . In fact, since the entries of each $l_i, \tilde l$ are integers of size $A^{O(1)}$ , one may check, e.g., by standard Gaussian elimination that we can insist that the constants $b_i,b$ are themselves integers of size $ A^{O(1)}$ (with a different constant O(1) which may depend on the dimension of the space). For these $b,b_i$ , let $l' = b\tilde l + {\sum}_{i=1}^m b_il_i \in (U \cap T)^\perp \subset U^\ast$ . Then $l'(a) = l'(a_p)$ (since $a_r,a_s\in U \cap T$ ), $l'(a_p) = b\tilde l(a_p)$ (since $a_p \in \ker L$ ), and $b\tilde l(a_p) = bA^{O(1)}l(a_p)$ (by the construction of $\tilde l$ and since $a_p \in \ker L$ ). Thus,

\[\|l'(a)\|_{\mathbb{R}/\mathbb{Z}} = \|bA^{O(1)}l(a_p)\|_{\mathbb{R}/\mathbb{Z}} \leq bA^{O(1)}\|l(a_p)\|_{\mathbb{R}/\mathbb{Z}} < A^{O(1)}\varepsilon.\]

However, l’ is of complexity $A^{O(1)}$ with respect to ${\mathcal{X}}'$ . For c chosen suitably large, this contradicts the $(A^c, A^c\varepsilon)$ -linear irrationality of a in $U\bmod T$ . Thus, we have that (the projection of) the rows of L and $\tilde l$ are linearly independent in $(U \cap T)^\ast$ .

Finally, let C be the smallest positive integer such that $C\tilde l (a_{r}) \in \mathbb{Z}$ ; note that $C = A^{O(1)}$ . Then we compute

\begin{align*}\|C\tilde l(a)\|_{\mathbb{R}/\mathbb{Z}} &\leq \|C\tilde l (a_{p})\|_{\mathbb{R}/\mathbb{Z}} + \|C\tilde l(a_{r})\|_{\mathbb{R}/\mathbb{Z}} +\|C\tilde l(a_{s})\|_{\mathbb{R}/\mathbb{Z}}\\& \leq CA^{O(1)}\varepsilon + 0 + CA^{O(1)}\varepsilon \leq A^{O(1)}\varepsilon.\end{align*}

One may then form $L_{m+1}$ by adding the row $C\tilde l$ to the bottom of $L=L_m$ . This puts us in situation (ii) and completes the proof.