2.1 Infinite-dimensional symmetric spaces

In this section, we recall definitions and facts about infinite-dimensional Riemannian symmetric spaces. By a Riemannian manifold, we mean a (possibly infinite-dimensional) smooth manifold modeled on some real Hilbert space with a smooth Riemannian metric. For a background on infinite-dimensional Riemannian manifolds, we refer to [Reference LangLan99] or [Reference PetersenPet06].

Let $(M,g)$ be a Riemannian manifold, a symmetry at a point $p\in M$ is an involutive isometry $\sigma _p\colon M\to M$ such that $\sigma _p(p) = p$ , and the differential at p is $-\operatorname {\textrm {Id}}$ . A Riemannian symmetric space is a connected Riemannian manifold such that, at each point, there exists a symmetry. See [Reference DuchesneDuc15a, §3] for more details.

We will be interested in infinite-dimensional analogs of symmetric spaces of non-compact type. If $(M,g)$ is a symmetric space of non-positive sectional curvature without local Euclidean factor, then for any point $p\in M$ , the exponential $\exp \colon T_pM\to M$ is a diffeomorphism and, if d is the distance associated to the metric g, then $(M,d)$ is a CAT(0) space [Reference DuchesneDuc15a, Proposition 4.1]. So, such a space M has a very pleasant metric geometry and in particular, it has a visual boundary $\partial M$ at infinity. If M is infinite-dimensional, then $\partial M$ is not compact for the cone topology.

Let us describe the principal example of an infinite-dimensional Riemannian symmetric space of non-positive curvature.

A totally geodesic subspace of a Riemannian manifold $(M,g)$ is a closed submanifold N such that for any $x\in N$ and $v\in T_xN\setminus \{0\}$ , the whole geodesic with direction v is contained in N. All the simply connected non-positively curved symmetric spaces that will appear in this paper are totally geodesic subspaces of the space $\operatorname {\textrm {P}}^2(\mathcal {H})$ described in Example 2.2.

A Lie triple system of $\operatorname {\textrm {S}}^2(\mathcal {H})$ is a closed linear subspace $\mathfrak {p}<\operatorname {\textrm {S}}^2(\mathcal {H})$ such that for all $X,Y,Z\in \mathfrak {p}$ , $[X,[Y,Z]]\in \mathfrak {p}$ , where the Lie bracket $[X,Y]$ is simply $XY-YX$ . The totally geodesic subspaces N of $\operatorname {\textrm {P}}^2(\mathcal {H})$ containing $\operatorname {\textrm {Id}}$ are in bijection with the Lie triple systems $\mathfrak {p}$ of $\operatorname {\textrm {S}}^2(\mathcal {H})$ . This correspondence is given by $N=\exp (\mathfrak {p})$ . See [Reference de la HarpedlH72, Proposition III.4].

All totally geodesic subspaces of $\operatorname {\textrm {P}}^2(\mathcal {H})$ are symmetric spaces as well and satisfy a condition of non-positivity of the curvature operator. This condition of non-positivity of the curvature operator allows a classification of these symmetric spaces [Reference DuchesneDuc15a, Theorem 1.8]. In this classification, all the spaces that appear are the natural analogs of the classical finite-dimensional Riemannian symmetric spaces of non-compact type.

The isometry group of a finite-dimensional symmetric space is a real algebraic group and thus has a Zariski topology; this is no more available in infinite dimension. Let $(M,g)$ be an irreducible symmetric space of finite dimension and non-positive sectional curvature, and let $G\leq \operatorname {\mathrm{Isom}}(M)$ . It is well known that the group G is Zariski dense if and only if there is neither a fixed point at infinity nor an invariant totally geodesic strict subspace (possibly reduced to a point). Thus, following the ideas in [Reference Caprace and MonodCM09], we say that a group G acting by isometries on a (possibly infinite-dimensional) Riemannian symmetric space of non-positive curvature ${\mathcal X}$ is geometrically dense if G has no fixed point in $\partial {\mathcal X}$ and no invariant closed totally geodesic strict subspace in ${\mathcal X}$ .

2.2 The Riemannian symmetric spaces ${\mathcal X}_{{\mathbf K}}(p,\infty )$

Throughout the paper, $\textbf {H}$ denotes the division algebra of the quaternions, and $\mathcal {H}$ is a separable Hilbert space over ${\textbf K}=\textbf {R}$ , ${\mathbf{C}}$ , or $\textbf {H}$ of infinite dimension. In the latter case, the scalar multiplication is understood to be on the right. We denote by $\operatorname {\mathrm{L}}(\mathcal {H})$ the algebra of all bounded ${\textbf K}$ -linear operators of $\mathcal {H}$ , and $\operatorname {\textrm {GL}}(\mathcal {H})$ is the group of all bounded invertible ${\textbf K}$ -linear operators with bounded inverse. Using the real Hilbert space $\mathcal {H}_{\mathbf {R}}$ underlying $\mathcal {H}$ , one can consider $\operatorname {\textrm {GL}}(\mathcal {H})$ as a closed subgroup of $\operatorname {\textrm {GL}}(\mathcal {H}_{\mathbf {R}})$ . We denote by $A^*$ the adjoint of $A\in \operatorname {\mathrm{L}}({\mathcal H})$ . In particular, when ${\textbf K}=\textbf {R}$ , $A^*= ^t\!A$ .

Let $p\in \mathbf{N}$ . We fix an orthonormal basis $(e_i)_{i\in \mathbf{N}}$ of the separable Hilbert space $\mathcal {H}$ , and we consider the Hermitian form

$$ \begin{align*}Q(x)=\sum_{i=1}^p\overline x_ix_i-\sum_{i>p}\overline x_ix_i,\end{align*} $$

where $x=\sum e_ix_i$ . The isometry group of this quadratic form will be denoted $\operatorname {{\textrm O_{\mathbf {K}}}}(Q)$ or equivalently $\operatorname {{\textrm O_{\mathbf {K}}}}(p,\infty )$ .

The intersection of $\operatorname {{\textrm O_{\mathbf {K}}}}(p,\infty )$ and the orthogonal group of $\mathcal {H}$ is isomorphic to $\operatorname {{\textrm O_{\mathbf {K}}}}(p)\times \operatorname {{\textrm O_{\mathbf {K}}}}(\infty )$ , where $\operatorname {{\textrm O_{\mathbf {K}}}}(p)$ (respectively $\operatorname {{\textrm O_{\mathbf {K}}}}(\infty )$ ) is the orthogonal group of the separable Hilbert space of dimension p (respectively of infinite dimension). Then the quotient

$$ \begin{align*}{\mathcal X}={\mathcal X}_{{\mathbf K}}(p,\infty)= \operatorname{{\textrm O_{\mathbf{K}}}}(p,\infty)/(\operatorname{{\textrm O_{\mathbf{K}}}}(p)\times\operatorname{{\textrm O_{\mathbf{K}}}}(\infty))\end{align*} $$

has a structure of an infinite-dimensional irreducible Riemannian symmetric space of non-positive curvature. This can be seen by the identification of ${\mathcal X}_{{\mathbf K}}(p,\infty )$ with the set

$$ \begin{align*}\mathcal{V}=\{V\leq{\mathcal H},\ \dim_{{\mathbf K}}(V)=p,\ Q|_V>0\}.\end{align*} $$

The group $\operatorname {{\textrm O_{\mathbf {K}}}}(p,\infty )$ acts transitively on $\mathcal {V}$ (by Witt’s theorem) and the stabilizer of the span of the p first vectors is $\operatorname {{\textrm O_{\mathbf {K}}}}(p)\times \operatorname {{\textrm O_{\mathbf {K}}}}(\infty )$ . Moreover, the subgroup $\operatorname {\textrm {O}}_{{\mathbf K}}^2(p,\infty )=\operatorname {{\textrm O_{\mathbf {K}}}}(p,\infty )\cap \operatorname {\textrm {GL}}^2({\mathcal H})$ also acts transitively on $\mathcal {V}$ and thus

$$ \begin{align*}{\mathcal X}_{{\mathbf K}}(p,\infty)\simeq \mathcal{V}\simeq \operatorname{\textrm{O}}_{{\mathbf K}}^2(p,\infty)/(\operatorname{{\textrm O_{\mathbf{K}}}}(p)\times\operatorname{\textrm{O}}_{{\mathbf K}}^2(\infty)).\end{align*} $$

The stabilizer of the origin in the action of $\operatorname {\textrm {O}}_{{\mathbf K}}^2(p,\infty )$ on $\operatorname {\textrm {P}}^2({\mathcal H}_{\mathbf {R}})$ is exactly $\operatorname {{\textrm O_{\mathbf {K}}}}(p)\times \operatorname {\textrm {O}}_{{\mathbf K}}^2(\infty )$ and the orbit of $\operatorname {\textrm {O}}_{{\mathbf K}}^2(p,\infty )$ in $\operatorname {\textrm {P}}^2({\mathcal H}_{\mathbf {R}})$ is a totally geodesic subspace [Reference DuchesneDuc13, Proposition 2.3]. Thus ${\mathcal X}_{{\mathbf K}}(p,\infty )$ has a structure of a simply connected non-positively curved Riemannian symmetric space.

Observe that when ${\textbf K}=\textbf {R}$ or ${\mathbf{C}}$ , homotheties act trivially on ${\mathcal X}_{{\mathbf K}}(p,\infty )$ and thus the group $\operatorname {\mathrm{PO}}_{{\mathbf K}}(p,\infty )$ , defined to be $\operatorname {\textrm {O}}_{{\mathbf K}}(p,\infty )/\{\unicode{x3bb} \operatorname {\textrm {Id}},\ |\unicode{x3bb} |=1\}$ , acts by isometries on ${\mathcal X}_{{\mathbf K}}(p,\infty )$ . Moreover, it is proved in [Reference DuchesneDuc13, Theorem 1.5] that this is exactly the isometry group of ${\mathcal X}_{{\mathbf K}}(p,\infty )$ when ${\textbf K}=\textbf {R}$ .

The spaces ${\mathcal X}_{{\mathbf K}}(p,\infty )$ , with p finite, are very special among infinite-dimensional Riemannian symmetric spaces: they have finite rank, which is p. This means there are totally geodesic embeddings of $\textbf {R}^p$ in ${\mathcal X}_{{\mathbf K}}(p,\infty )$ but there are no totally geodesic embeddings of $\textbf {R}^q$ for $q>p$ . Furthermore, every infinite-dimensional irreducible Riemannian symmetric space of non-positive curvature operator and finite rank arises in this way [Reference DuchesneDuc15a].

This finite-rank property gives some compactness on $\overline {{\mathcal X}}={\mathcal X}\cup \partial {\mathcal X}$ for a weaker topology [Reference Caprace and LytchakCL10, Remark 1.2]. Moreover, we have the following important property.

The boundary at infinity $\partial {\mathcal X}_{{\mathbf K}}(p,\infty )$ has a structure of a spherical building, which we now recall. We refer to [Reference Abramenko and BrownAB08] for general definitions and facts about buildings, and to [Reference DuchesneDuc13] for the specific case in which we are interested. The space $\partial {\mathcal X}_{{\mathbf K}}(p,\infty )$ has a natural structure of a simplicial complex (of dimension $p-1$ ): a simplex (of dimension $r-1$ ) in $\partial {\mathcal X}_{{\mathbf K}}(p,\infty )$ is defined by a flag $(V_1\subsetneq \cdots \subsetneq V_r)$ , where all the $V_i$ are non-zero totally isotropic subspaces of ${\mathcal H}$ . In particular, vertices of $\partial {\mathcal X}_{{\mathbf K}}(p,\infty )$ correspond to totally isotropic subspaces. A simplex A is contained in a simplex B if all the subspaces appearing in the flag A also appear in the flag B.

Each vertex has a type, which is a number between $1$ and p given by the dimension of the corresponding isotropic subspace. More generally, the type of a cell is the finite increasing sequence of dimensions of the isotropic subspaces in the associated isotropic flag.

2.3 Hermitian symmetric spaces

Let $(M,g)$ be a Riemannian manifold (possibly of infinite dimension). An almost complex structure is a $(1,1)$ -tensor J such that for any vector field X, $J(J(X))=-X$ . A triple $(M,g,J)$ , where $(M,g)$ is a Riemannian manifold and J is an almost complex structure, is a Hermitian manifold if for all vector fields $X,Y$ , $g(J(X),J(Y))=g(X,Y)$ . If $(M,g,J)$ is a Hermitian manifold, we define a 2-form $\omega $ by the formula $\omega (X,Y)=g(J(X),Y)$ . A Kähler manifold is a Hermitian manifold such that $d\omega =0$ and $\omega $ is the Kähler form on M.

Let $(M,g,J)$ be a Hermitian manifold and $\nabla $ be the Levi-Civita connection associated to the Riemannian metric g. The almost-complex structure J is parallel if $\nabla J=0$ , that is, if for all vector fields $X,Y$ , $\nabla _X(JY)=J(\nabla _XY)$ . This parallelism condition implies that $\omega $ is parallel as well, that is, for all vector fields $X,Y,Z$ , $(\nabla _X\omega )(Y,Z)=0$ . Because $d\omega (X,Y,Z)=(\nabla _X\omega )(Y,Z)-(\nabla _Y\omega )(X,Z)+(\nabla _Z\omega )(X,Y)$ , the parallelism condition $\nabla J=0$ implies that $\omega $ is closed.

Let N be the Nijenhuis (2,0)-tensor on M, that is, for all vector fields $X,Y$ ,

$$ \begin{align*}N(X,Y)=2([X,Y]-[J(X),J(Y)]-J[J(X),Y]-J[X,J(Y)]).\end{align*} $$

The parallelism of J implies that this tensor vanishes. An almost-complex structure J with vanishing Nijenhuis tensor is called a complex structure. Thus a parallel almost-complex structure is a complex structure.

Definition 2.7. Let $(M,g)$ be a simply connected Riemannian symmetric space of non-positive sectional curvature. The symmetric space M is said to be a Hermitian symmetric space if it admits a Hermitian almost-complex structure J that is invariant under symmetries. This means that for any $p,q\in M$ ,

$$ \begin{align*}d\sigma_p\circ J_{\sigma_p(q)}\circ d\sigma_p=J_q\end{align*} $$

on the tangent space $T_qM$ . One also says that the symmetries are holomorphic.

Let us recall some notation in $\operatorname {\textrm {P}}^2(\mathcal {H})$ . We denote by o the origin in $\operatorname {\textrm {P}}^2(\mathcal {H})$ , that is, the identity $\operatorname {\textrm {Id}}$ of ${\mathcal H}$ . The symmetry $\sigma _o$ at the origin is the map $x\mapsto x^{-1}$ . The action $\tau $ of $\operatorname {\textrm {GL}}^2({\mathcal H})$ on $\operatorname {\textrm {P}}^2({\mathcal H})$ is given by $\tau (g)(x)=gx^tg$ . In particular, one has the relation

$$ \begin{align*}\sigma_o\circ\tau(g)=\tau(g^{-1})\circ\sigma_o.\end{align*} $$

The exponential map $\exp \colon \operatorname {\mathrm{L}}^2({\mathcal H})\to \operatorname {\textrm {GL}}^2({\mathcal H})$ is a local diffeomorphism around the origin and it induces a diffeomorphism $\exp \colon \operatorname {\textrm {S}}^2({\mathcal H})\to \operatorname {\textrm {P}}^2({\mathcal H})$ . In particular, we identify the tangent space at the origin $T_o\operatorname {\textrm {P}}^2({\mathcal H})$ with the Hilbert space $\operatorname {\textrm {S}}^2({\mathcal H})$ . The isotropy group of the origin, that is, the fixator of o, is $\operatorname {\textrm {O}}^2({\mathcal H})$ . It acts also on $\operatorname {\textrm {S}}^2({\mathcal H})$ by $g\cdot v=gv^tg$ and one has $g\exp (v)^tg=\exp (gv^tg)$ for all $v\in \operatorname {\textrm {S}}^2({\mathcal H})$ and $g\in \operatorname {\textrm {O}}^2({\mathcal H})$ . If K is a subgroup of $\operatorname {\textrm {O}}^2({\mathcal H})$ , we denote by $K^*$ its image in the isometry group of $\operatorname {\textrm {S}}^2({\mathcal H})$ .

The following proposition is a mere extension of a classical statement in finite dimension to our infinite-dimensional setting (see for example [Reference HelgasonHel01, Proposition VIII.4.2]). It can be proved with the same methods.

In the remainder of this section, we exhibit the complex structures J on two classes of Hermitian symmetric spaces we will use later in the paper. Thanks to Proposition 2.8, it suffices to find $J_0$ with the required properties. The complex structures we describe are all the natural analogs of the corresponding complex structures in finite dimension.

Below, we use orthogonal decompositions ${\mathcal H}=V\oplus W$ and block decompositions for elements of $\operatorname {\mathrm{L}}({\mathcal H})$ . When we write $g=\Big [\begin {smallmatrix} A &B\\ C & D \end {smallmatrix}\Big ]$ , this means that $A=\pi _V\circ g|_V\in \operatorname {\mathrm{L}}(V)$ , $B=\pi _V\circ g|_W\in \operatorname {\mathrm{L}}(W,V)$ , $C=\pi _W\circ g|_V\in \operatorname {\mathrm{L}}(V,W)$ , and $D=\pi _W\circ g|_W\in \operatorname {\mathrm{L}}(W)$ .

2.3.1 The Hermitian symmetric space ${\mathcal X}_{\mathbf{C}}(p,\infty )$

Let $\mathcal {H}$ be a complex Hilbert space of infinite dimension. We denote by ${\mathcal H}_{\mathbf {R}}$ the underlying real Hilbert space. Let $V,W$ be closed orthogonal complex subspaces of dimension $p\in \mathbf{N}\cup \{\infty \}$ and $\infty $ such that ${\mathcal H}=V\oplus W$ . Let $I_{p,\infty }=\operatorname {\textrm {Id}}_V\oplus -\operatorname {\textrm {Id}}_W$ . Thus,

$$ \begin{align*}\operatorname{\textrm{O}}^2_{\mathbf{C}}(p,\infty)=\{g\in\operatorname{\textrm{GL}}^2({\mathcal H}),\ g^*I_{p,\infty}g=I_{p,\infty}\}.\end{align*} $$

The symmetric space ${\mathcal X}_{\mathbf{C}}(p,\infty )$ is the $\operatorname {\textrm {O}}^2_{\mathbf{C}}(p,\infty )$ -orbit of the identity in $\operatorname {\textrm {P}}^2({\mathcal H}_{\mathbf {R}})$ , that is, the image under the exponential map of the Lie triple system

$$ \begin{align*}\mathfrak p=\left\{\left[\begin{array}{@{}ll}0&A\\A^*&0\end{array}\right],\ A\in\operatorname{\textrm{L}}^2(W,V)\right\}.\end{align*} $$

The complex structure is induced by the endomorphism $J_0$ of $\mathfrak p$ defined by $J_0\Big [\begin {smallmatrix}0&A\\A^*&0\end {smallmatrix}\Big ]=\Big [\begin {smallmatrix}0&iA\\-iA^*&0\end {smallmatrix}\Big ]$ . Because the stabilizer of $\operatorname {\textrm {Id}}_{{\mathcal H}_{\mathbf {R}}}$ in $\operatorname {\textrm {O}}_{\mathbf{C}}(p,\infty )$ is given by all the operators that can be expressed as $\Big [\begin {smallmatrix}P&0\\0&Q\end {smallmatrix}\Big ]$ with $P\in \operatorname {\textrm {O}}^2_{\mathbf{C}}(V)$ and $Q\in \operatorname {\textrm {O}}^2_{\mathbf{C}}(W)$ , $J_0$ satisfies the conditions of Proposition 2.8.

2.3.2 The Hermitian symmetric space ${\mathcal X}_{\mathbf {R}}(2,\infty )$

Let ${\mathcal H}$ be a real Hilbert space of infinite dimension. Let $V,W$ be closed orthogonal subspaces of dimension two and $\infty $ such that ${\mathcal H}=V\oplus W$ . Let $I_{2,\infty }=\operatorname {\textrm {Id}}_V\oplus -\operatorname {\textrm {Id}}_W$ . Thus,

$$ \begin{align*}\operatorname{\textrm{O}}^2_{\mathbf{R}}(2,\infty)=\{g\in\operatorname{\textrm{GL}}^2({\mathcal H}),\ ^tgI_{2,\infty}g=I_{2,\infty}\}.\end{align*} $$

The symmetric space ${\mathcal X}_{\mathbf {R}}(2,\infty )$ is the $\operatorname {\textrm {O}}^2_{\mathbf {R}}(2,\infty )$ -orbit of the identity in $\operatorname {\textrm {P}}^2({\mathcal H})$ , that is, the image under the exponential map of the Lie triple system

$$ \begin{align*}\mathfrak p=\left\{\left[\begin{array}{@{}ll}0&A\\{}^tA&0\end{array}\right],\ A\in\operatorname{\textrm{L}}(W,V)\right\}.\end{align*} $$

Fix some orthonormal basis $(e_1,e_2)$ of V and let $I=\Big [\begin {smallmatrix}0&-1\\1&0\end {smallmatrix}\Big ]$ . This element belongs to the group $\operatorname {\textrm {SO}}_{\mathbf {R}}(2)$ , which is commutative. The complex structure is defined by $J_0\Big [\begin {smallmatrix}0 &A \\ {}^t A &0 \end {smallmatrix}\Big ]=\Big [\begin {smallmatrix}0 &IA \\-^{t}(IA)&0\end {smallmatrix}\Big ]$ . Because the stabilizer of $\operatorname {\textrm {Id}}_{\mathcal H}$ in the identity component of $\operatorname {\textrm {O}}^2_{\mathbf {R}}(2,\infty )$ is given by operators of the form $\Big [\begin {smallmatrix}P &0\\0 &Q\end {smallmatrix}\Big ]$ , with $P\in \operatorname {\textrm {SO}}_{\mathbf {R}}(2)$ and $Q\in \operatorname {\textrm {O}}^2_{\mathbf {R}}(\infty )$ , let us denote by $\operatorname {\textrm {O}}^{+}_{\mathbf {R}}(2,\infty )$ the set of elements in $\operatorname {\textrm {O}}_{\mathbf {R}}(2,\infty )$ of the form $\Big [\begin {smallmatrix} A &B\\ C & D \end {smallmatrix}\Big ]$ , where A has positive determinant. So there is a $\operatorname {\textrm {O}}^{+}_{\mathbf {R}}(2,\infty )$ -invariant complex structure on ${\mathcal X}_{\mathbf {R}}(2,\infty )$ . Let us denote by $\operatorname {\mathrm{PO}}_{\mathbf {R}}^+(2,\infty )$ the image of $\operatorname {\textrm {O}}^{+}_{\mathbf {R}}(2,\infty )$ under the quotient map $\operatorname {\textrm {O}}_{\mathbf {R}}(2,\infty )\to \operatorname {\mathrm{PO}}_{\mathbf {R}}(2,\infty )$ .

2.4 Tube-type Hermitian symmetric spaces

In finite dimension, the class of Hermitian symmetric spaces splits into two classes: those of tube type and those that are not of tube type. This distinction is important for the approach we use to understand maximal representations. For a definition in finite dimension, we refer to [Reference Burger, Iozzi and WienhardBIW09]. Let us briefly recall that if ${\mathcal X}$ is a finite-dimensional Hermitian symmetric space, a chain is the boundary (as a subset of the Shilov boundary of ${\mathcal X}$ ) of a maximal tube-type subspace. By definition, if ${\mathcal X}$ is of tube type, there is a unique maximal tube-type subspace: ${\mathcal X}$ itself. However, if ${\mathcal X}$ is not of tube type, chains lie in a unique $\operatorname {\mathrm{Isom}}({\mathcal X})$ -orbit and it yields a new incidence geometry: the chain geometry (see [Reference PozzettiPoz15, §3]). Let us give an ad hoc definition of tube-type Hermitian symmetric spaces in infinite dimension.

3.1 Algebraic subgroups of bounded operators of Hilbert spaces

Algebraic subgroups of finite-dimensional linear Lie groups are well understood and equipped with a useful topology: the Zariski topology. In infinite dimension, some new and baffling phenomena may appear. For example, one-parameter subgroups may be non-continuous. In [Reference Harris and KaupHK77], Harris and Kaup introduced the notion of linear algebraic groups and showed that they behave nicely with respect to the exponential map. In particular, the exponential map is a local homeomorphism and any point sufficiently close to the origin lies in some continuous one-parameter subgroup.

Let $A,B$ be two real Banach algebras and let $G(A)$ be the set of all invertible elements of A. A map $f\colon A\to B$ is a homogeneous polynomial map of degree n if there is a continuous n-linear map $f_0\in \operatorname {\mathrm{L}}^n(A,B)$ such that for any $a\in A$ , $f(a)=f_0(a,\ldots ,a)$ . Now, a map $f\colon A\to B$ is polynomial if it is a finite sum of homogeneous polynomial maps. Its degree is the maximum of the degrees that appear in the sum.

Let ${\mathcal H}$ be a real Hilbert space. The Banach algebras we will use are $\operatorname {\mathrm{L}}({\mathcal H})$ endowed with the operator norm and the field of real numbers $\textbf {R}$ . The group of invertible elements in $\operatorname {\mathrm{L}}({\mathcal H})$ is $\operatorname {\textrm {GL}}({\mathcal H})$ .

Definition 3.1. A subgroup G of $G(A)$ is an algebraic subgroup if there is a constant n and a set $\mathcal {P}$ of polynomial maps of degrees at most n on $A\times A$ such that

$$ \begin{align*}G=\{g\in G(A);\ P(g,g^{-1})=0,\text{ for all } P\in \mathcal{P}\}.\end{align*} $$

Observe that $\mathcal {P}$ may be infinite but the degrees of its elements are uniformly bounded. The main result of [Reference Harris and KaupHK77] is the fact that an algebraic subgroup is a Banach Lie group (with respect to the norm topology) and that the exponential map gives a homeomorphism in a neighborhood of the identity.

In finite dimension, linear algebraic groups of $\operatorname {\textrm {GL}}_n(\textbf {R})$ are given by polynomial equations in matrix coefficients. We generalize this notion to subgroups of $\operatorname {\textrm {GL}}(\mathcal {H})$ .

Definition 3.4. Let ${\mathcal H}$ be a real Hilbert space. A matrix coefficient is a linear map $f\colon \operatorname {\mathrm{L}}({\mathcal H})\to \textbf {R}$ such that there are $x,y\in {\mathcal H}$ such that $f(L)=\langle Lx,y\rangle $ for any $L\in \operatorname {\mathrm{L}}({\mathcal H})$ . A homogeneous polynomial map P on $\operatorname {\mathrm{L}}({\mathcal H})\times \operatorname {\mathrm{L}}({\mathcal H})$ of degree d is standard if there is an orthonormal basis $(e_n)_{n\in \mathbf{N}}$ of ${\mathcal H}$ , non-negative integers $m,l$ such that $d=m+l$ , and families of real coefficients $(\unicode{x3bb} _i)_{i\in \mathbf{N}^{2m}}$ , $(\mu _j)_{j\in \mathbf{N}^{2l}}$ such that for all $(M,N)\in \operatorname {\mathrm{L}}({\mathcal H})\times \operatorname {\mathrm{L}}({\mathcal H})$ , $P(M,N)$ can be written as an absolutely convergent series

where $P_i(M)\!=\!\prod _{k\!=\!0}^{m-1}\langle Me_{i_{2k}},e_{i_{2k+1}}\rangle $ for $i\in \mathbf{N}^{2m}$ and similarly $P_j(N)\!=\!\prod _{k\!=\!0}^{l-1}\langle Ne_{i_{2k}},e_{i_{2k+1}}\rangle $ for $j\in \mathbf{N}^{2l}$ .

A polynomial map is standard if it is a finite sum of standard homogeneous polynomial maps. A subgroup of $\operatorname {\textrm {GL}}({\mathcal H})$ is a standard algebraic subgroup if it is an algebraic subgroup defined by a family of standard polynomials.

Let H be a standard algebraic subgroup of $\operatorname {\textrm {GL}}({\mathcal H})$ . If E is a finite-dimensional subspace of ${\mathcal H}$ , we denote by $H_E$ the subgroup of elements $g\in H$ such that $g(E)=E$ , $g|_{E^{\bot }}=\operatorname {\textrm {Id}}$ . We identify $H_E$ with an algebraic subgroup of $\operatorname {\textrm {GL}}(E)$ . By a strict algebraic group H, we mean that H is algebraic and $ H\neq \operatorname {\textrm {GL}}({\mathcal H})$ .

3.2 Exterior products

Let ${\mathcal H}_{{\mathbf K}}$ be a Hilbert space over ${\textbf K}$ with Hermitian form Q of signature $(p,\infty )$ . We denote by ${\mathcal H}$ the underlying real Hilbert space and by $(\cdot ,\cdot )$ the real quadratic form $\Re (Q)$ .

The exterior product ${\textstyle \bigwedge }^k\mathcal {H}$ has a natural structure of pre-Hilbert space and there is a continuous representation $\pi _k\colon \operatorname {\textrm {GL}}(\mathcal {H})\to \operatorname {\textrm {GL}}({\textstyle \bigwedge }^k\mathcal {H})$ given by the formula $\pi _k(g)(x_1\wedge \cdots \wedge x_k)=gx_1\wedge \cdots \wedge gx_k$ . An orthonormal basis of ${\textstyle \bigwedge }^k\mathcal {H}$ is given by $(e_{i_1}\wedge \cdots \wedge e_{i_k})_{i\in \mathcal {I}}$ , where $\mathcal {I}$ is the set of elements $i\in \mathbf{N}^k$ such that $i_1<\cdots <i_k$ and $(e_i)$ is an orthonormal basis of $\mathcal {H}$ . In other words, if $\langle \cdot ,\cdot \rangle $ is the scalar product on ${\mathcal H}$ , then the bilinear form applied to two vectors $x_1\wedge \cdots \wedge x_k$ and $y_1\wedge \cdots \wedge y_k$ is given by the Gram determinant $\det ( \langle x_i,y_j\rangle _{i,j=1..k})$ . As usual, the completion of ${\textstyle \bigwedge }^k\mathcal {H}$ is denoted $\overline {{\textstyle \bigwedge }^k\mathcal {H}}$ .

The space ${\textstyle \bigwedge }^k\mathcal {H}$ is also endowed with a quadratic form built from Q. One defines $(x_1\wedge \cdots \wedge x_k,y_1\wedge \cdots \wedge y_k)$ to be $\det ((x_i,y_j))$ . As soon as $k\geq 2$ , this quadratic form is non-degenerate of signature $(\infty ,\infty )$ and extends continuously to $\overline {{\textstyle \bigwedge }^k\mathcal {H}}$ . Moreover, $\pi _k(\operatorname {{\textrm O_{\mathbf {K}}}}(Q))$ preserves this quadratic form.

Lemma 3.9. Let $(e_i)_{i\in \mathcal I}$ be an orthonormal basis of $\mathcal {H}$ and let $v,w$ be vectors of ${\textstyle \bigwedge }^k \mathcal {H}$ . There are families $(\unicode{x3bb} _i)$ and $(\mu _i)$ such that for any $g\in \operatorname {\mathrm{GL}}(\mathcal {H})$ ,

is a standard polynomial in g.

Proof of Proposition 1.5

We have seen in Example 3.5 that stabilizers of points at infinity are standard algebraic subgroups. Assume $\mathcal Y$ is a strict totally geodesic subspace of ${\mathcal X}$ . Without loss of generality, we assume that $o\in \mathcal Y$ and thus $\mathcal Y$ corresponds to some Lie triple system $\mathfrak {p}< \operatorname {\textrm {S}}^2({\mathcal H})$ . Let $\mathfrak {k}=\overline {[\mathfrak {p},\mathfrak {p}]}$ and $\mathfrak m$ be the Lie algebra $\mathfrak {k}\oplus \mathfrak {p}\leq \operatorname {\mathrm{L}}^2({\mathcal H})$ . Because $\mathfrak m$ is a Lie algebra, $G=\exp (\mathfrak m)$ is a subgroup of $\operatorname {\textrm {GL}}^2({\mathcal H})$ that is generated by transvections along geodesics in $\mathcal Y$ . In particular, for any $h\in \operatorname {{\textrm O_{\mathbf {K}}}}(p,\infty )$ , h normalizes G if and only if h preserves $\mathcal Y$ . Because $G=\exp (\mathfrak m)$ , h normalizes G if and only if it stabilizes $\mathfrak m$ under the adjoint action (that is, $\operatorname {\textrm {Ad}}(h)(\mathfrak m)=\mathfrak m$ , which means that for any $X\in \mathfrak m$ , $hXh^{-1}\in \mathfrak m$ ). Because $\mathfrak m$ is a closed subspace of $\operatorname {\mathrm{L}}^2({\mathcal H})$ , we have the splitting $\operatorname {\mathrm{L}}^2({\mathcal H})=\mathfrak m\oplus \mathfrak m^{\bot }$ . So, h stabilizes $\mathfrak m$ if and only if for any $X\in \mathfrak m$ and $Y\in \mathfrak m^{\bot }$ , $\langle hXh^{-1},Y\rangle =0$ , where $\langle \ ,\ \rangle $ is the Hilbert–Schmidt scalar product. Finally, because the map $(M,N)\mapsto \langle MXN,Y\rangle $ is bilinear on $\operatorname {\mathrm{L}}({\mathcal H})\times \operatorname {\mathrm{L}}({\mathcal H})$ , H is an algebraic subgroup of $\operatorname {{\textrm O_{\mathbf {K}}}}(p,\infty )$ . It remains to show that these bilinear maps are standard. Let $(e_n)$ be an orthonormal basis of ${\mathcal H}$ and let $E_{i,j}=e_i\otimes e_j^*$ be the associated orthonormal basis of $\operatorname {\mathrm{L}}^2({\mathcal H})$ , that is, $E_{i,j}(x)=\langle x,e_j\rangle e_i$ . Thus, let us write $X=\sum _{i,j}X_{i,j}E_{i,j}$ and $Y=\sum _{i,j}X_{i,j}E_{i,j}$ to obtain

$$ \begin{align*}\langle MXN,Y\rangle=\sum_{i,j,k,l}X_{i,j}Y_{k,l}\langle ME_{i,j}N,E_{k,l}\rangle,\end{align*} $$

where

$$ \begin{align*} \langle ME_{i,j}N,E_{k,l}\rangle&=\operatorname{\textrm{Trace}}((ME_{i,j}N)^*E_{k,l})\\ &=\sum_{n\in\textbf{N}}\langle N^*E_{i,j}M^*E_{k,l}(e_n),e_n\rangle\\ &=\langle N^*E_{j,i}M^*(e_k),e_l\rangle\\ &=\langle E_{j,i}M^*(e_k),N(e_l)\rangle\\ &=\langle e_i,M^*(e_k)\rangle\langle e_j,N(e_l)\rangle\\ &=\langle M(e_i),e_k\rangle\langle e_j,N(e_l)\rangle. \end{align*} $$

The absolute convergence of the series can be proven with the same arguments as in Example 3.5.(1).

Let ${\mathcal H}$ be a Hilbert space over ${\textbf K}$ with a non-degenerate Hermitian form Q of signature $(p,\infty )$ with $p\in \mathbf{N}$ . For a finite-dimensional non-degenerate subspace $E\subset {\mathcal H}$ of Witt index p, we denote by ${\mathcal X}_E\subset {\mathcal X}_{{\mathbf K}}(p,\infty )$ the subset of isotropic subspaces of E of dimension p. This corresponds to a standard embedding of ${\mathcal X}_{{\mathbf K}}(p,q)\hookrightarrow {\mathcal X}_{{\mathbf K}}(p,\infty )$ , where $(p,q)$ is the signature of the restriction of Q to E. Let $\mathcal {E}$ be the collection of all such finite-dimensional subspaces. We conclude this section with a lemma that shows that the family $({\mathcal X}_E)_{E\in \mathcal {E}}$ is cofinal among finite-dimensional totally geodesic subspaces.

4.1 Maps from strong boundaries

Let G be a locally compact, second countable group acting continuously by isometries on ${\mathcal X}_{{\mathbf K}}(p,\infty )$ , where p is finite. This section is dedicated to the analysis of Furstenberg maps also known as boundary maps from a measurable boundary of G to the geometric boundary $\partial {\mathcal X}_{{\mathbf K}}(p,\infty )$ , or more precisely, to some specific part of this boundary. We use a suitable notion of a measurable boundary of a group introduced in [Reference Bader and FurmanBF14, §2]. This definition (see Definition 4.7) is a strengthening of previous versions introduced by Furstenberg [Reference FurstenbergFur73] and Burger and Monod [Reference Burger and MonodBM02].

Let us recall that a Polish space is a topological space which is separable and completely metrizable. By a Lebesgue G-space, we mean a standard Borel space (that is, a space and a $\sigma $ -algebra given by some Polish space and its Borel $\sigma $ -algebra), equipped with a Borel probability measure and an action of G which is measurable and preserves the class of the measure. We denote by $\textbf {P}(\Omega )$ the space of probability measures on a standard Borel space $\Omega $ . This is a Polish space for the topology of weak convergence.

Before going on, let us give a few examples of fiberwise isometric actions. These examples are closed to measurable fields of metric spaces that appear in [Reference Bader, Duchesne and LécureuxBDL16] and are simpler versions of fiberwise isometric actions that will appear in the proof of Theorem 1.7.

Definition 4.5. Let A and B be Lebesgue G-spaces. Let $\pi :A\to B$ be a measurable G-equivariant map. We say that $\pi $ is relatively isometrically ergodic if each time we have a G-equivariant Borel map $p:Y\to Z$ of standard Borel G-spaces, which admits a fiberwise isometric action, and measurable G-maps $A\to Y$ and $B\to Z$ such that the following diagram commutes:

then there exists a measurable G-map $\phi \colon B\to Y$ which makes the following diagram commutative.

Remark 4.6. If a G-Lebesgue space B is such that the first projection $\pi _1\colon B\times B\to B$ is relatively isometrically ergodic, then it is isometrically ergodic. Indeed, if Y is a separable metric G-space and $f:B\to Y$ is G-equivariant, then it suffices to apply relatively isometric ergodicity to the map $\tilde {f}\colon (b,b')\mapsto f(b')$ and the trivial fibration $Y\to \{\ast \}$ .

Actually, relative isometric ergodicity yields a measurable map $\phi \colon B\to Y$ such that for almost all $(b,b')$ , $\phi (b)=f(b')$ , and thus f, is essentially constant.

Let B be a Lebesgue G-space. We use the definition of amenability for actions introduced by Zimmer, see [Reference ZimmerZim84, §4.3]. The action $G\curvearrowright B$ is amenable if for any compact metrizable space M on which G acts continuously by homeomorphisms, there is a measurable G-equivariant map $\phi \colon B\to \textbf {P}(M)$ .

The next example shows that every countable group admits a strong boundary.

Existence of Furstenberg maps is already known. In the next section, we show that we can specify the type of points in the essential image and get that these points are essentially opposite. Let us recall that an isometric action of a group $\Gamma $ on a CAT(0) space ${\mathcal X}$ is non-elementary if there is neither invariant flat subspace (possibly reduced to a point) nor a global fixed point at infinity.

We will also rely on results obtained in [Reference Bader, Duchesne and LécureuxBDL16]. Unfortunately, [Reference Bader, Duchesne and LécureuxBDL16] was written before the final version of [Reference Bader and FurmanBF14] and a slightly different language was used there. Group actions on measurable metric fields were used there and here we just described fiberwise isometric actions. In the following proposition, we establish the relation between these two notions. We refer to [Reference Bader, Duchesne and LécureuxBDL16, §3] for definitions, notation, and a discussion about measurable metric fields. Roughly speaking, a measurable metric field over a Lebesgue $\Gamma $ -space $\Omega $ is a collection $\textbf {X}=(X_{\omega })_{\omega \in \Omega }$ of metric spaces $(X_{\omega },d_{\omega })$ where one moves from one such metric space to another one in a measurable fashion. It admits a $\Gamma $ -action if for all $\omega \in \Omega , g\in \Gamma $ , there is an isometry $\sigma (g,\omega )\colon X_{\omega }\to X_{g\omega }$ that satisfies the cocycle relation $\sigma (gg',\omega )=\sigma (g,g'\omega )\circ \sigma (g',\omega )$ almost surely.

Remark 4.12. With this lemma, for any two Lebesgue $\Gamma $ -spaces $A, B$ with a $\Gamma $ -factor map, that is, a measurable surjective $\Gamma $ -map $\pi \colon A\to B$ , the relative isometric ergodicity of $\pi $ , as stated in [Reference Bader, Duchesne and LécureuxBDL16, Definition 25], follows from Definition 4.5 above. Actually, if $\textbf {X}$ is a metric field over B, and $B_0$ , $p\colon X\to B_0$ are given by Lemma 4.11, this relative ergodicity is reflected in the following diagram where $A_0=\pi ^{-1}(B_0)$ .

This allows us to use freely the results from [Reference Bader, Duchesne and LécureuxBDL16].

4.2 Equivariant maps to the set of maximal isotropic subspaces under Zariski density

As before, let ${\mathcal H}$ be a Hilbert space over ${\textbf K}$ with a Hermitian form Q of signature $(p,q)$ with $p<q$ and $q\in \mathbf{N}\cup \{\infty \}$ . We fix some locally compact second countable group G with a continuous action by isometries on ${\mathcal X}_{{\mathbf K}}(p,q)$ and strong boundary B.

We denote by $\mathcal {I}_k$ the space of totally isotropic subspaces of $\mathcal {H}$ of dimension $k\leq p$ . Following the end of §2.2, this space can be identified with a type of vertices of the spherical building structure on $\partial {\mathcal X}_{p,q}$ . When recalling that the signature of the Hermitian form will seem to help comprehension, we will include it in our notation, and denote the space of totally isotropic subspaces as $\mathcal I_k(p,q)$ . Let us observe that if $p,q$ are finite, then $\mathcal I_k(p,q)$ can be identified with some homogeneous space $G/P$ , where $G=\operatorname {{\textrm O_{\mathbf {K}}}}(p,q)$ and P is a parabolic subgroup. In that case, we endow $G/P$ with the corresponding $\sigma $ -algebra and the unique G-invariant measure class on it (see e.g. [Reference Bekka, de la Harpe and ValetteBdlHV08, Appendix B]). Observe that when $p=k=1$ (and q is finite or infinite), then $\mathcal I_1(1,q)$ is merely the visual boundary $\partial {\mathcal X}_{{\mathbf K}} (1,q)$ of the hyperbolic space ${\mathcal X}_{{\mathbf K}} (1,q)$ of dimension q over ${\textbf K}$ . For the application to maximal representations, the space $\mathcal {I}_p$ of totally isotropic subspaces of maximal dimension plays an important role.

For example, if ${\mathcal H}$ is a finite-dimensional complex vector space, ${\mathcal X}_{\mathbf{C}} (p,q)$ is a complex manifold admitting a bounded domain realization whose Shilov boundary can be $\operatorname {\mathrm{SU}}(p,q)$ -equivariantly identified with $\mathcal {I}_p$ . The purpose of this section is to associate to geometrically dense or Zariski-dense representations $\rho $ an equivariant boundary map with values in the set of maximal isotropic subspaces $\mathcal {I}_p$ (Theorem 1.7).

First, we prove opposition for boundary maps to $\mathcal {I}_k$ under Zariski density.

In the remainder of this section, our goal is to show the existence of maps from a strong boundary B to $\mathcal {I}_p$ . We begin our discussion by observing that for every k, there is a natural fiberwise isometric action of ${\Gamma }$ over $\mathcal {I}_k$ : we denote by $\mathcal V_k$ the space of subspaces V of $\mathcal {H}$ with dimension p such that $Q|_V$ is non-negative and $\ker (Q|_V)$ has dimension k. We endow both $\mathcal I_k$ and $\mathcal V_k$ with the induced topologies coming from the corresponding Grassmannians $\mathcal {G}_k,\mathcal {G}_p$ of subspaces of dimension k and p in $\mathcal {H}$ . Let us recall that a complete and separable distance on the Grassmannian $\mathcal {G}_m$ of all subspaces of dimension m is given by

$$ \begin{align*}d(V,W)^2=\sum_{i=1}^m\alpha_i^2,\end{align*} $$

where $\alpha _1,\ldots ,\alpha _m$ are the principal angles between V and $W\in \mathcal {G}_m$ . This topology also coincides with the Wisjman topology of the hyperspace $2^{\mathcal H}$ .

The natural projection

$$ \begin{align*}\begin{array}{@{}lll} \pi\colon \mathcal V_k&\to& \mathcal{I}_k\\ V&\mapsto &\ker(Q|_V) \end{array}\end{align*} $$

is continuous. For $V_0\in \mathcal {I}_k$ , the fiber $\pi ^{-1}(V_0)$ can be identified with a symmetric space ${\mathcal X}(V_0^{\bot }/V_0,Q)$ that we define in the following lines. The kernel of Q restricted to $V_0^{\bot }$ is exactly $V_0$ and thus Q defines a strongly non-degenerate Hermitian form on $V_0^{\bot }/V_0$ of signature $(p-k,q)$ . So, we define ${\mathcal X}(V_0^{\bot }/V_0,Q)$ to be the symmetric space associated to that Hermitian form, that is, the collection of positive subspaces of $V_0^{\bot }/V_0$ of dimension $p-k$ . The metric on ${\mathcal X}(V_0^{\bot }/V_0,Q)$ is given by the hyperbolic principal angles [Reference DuchesneDuc13, §3.1]. The preimages of such a positive subspace under the projection $V_0^{\bot }\to V_0^{\bot }/V_0$ are in bijective correspondence with the elements in the fiber of $\mathcal V_k$ above $V_0$ .

Recall that $V_0,W_0\in \mathcal {I}_k$ are opposite if the restriction of Q to $V_0+W_0$ is non-degenerate and thus has signature $(k,k)$ . If $V_0,W_0$ are opposite, then $\mathcal {H}=V_0\oplus W_0^{\bot }$ because $W_0^{\bot }$ has codimension k and $V_0\cap W_0^{\bot }=\{0\}$ . So, there is a bijective correspondence $\sigma _{V_0,W_0}\colon {\mathcal X}(V_0^{\bot }/V_0,Q)\to {\mathcal X}(W_0^{\bot }/W_0,Q)$ given by the formula

$$ \begin{align*}\sigma_{V_0,W_0}(V)=(V\cap W_0^{\bot})+W_0\end{align*} $$

for $V\in \pi ^{-1}(V_0)$ . This map is well defined because $V\cap W_0^{\bot }$ has dimension $p-k$ and is positive definite for Q. The inverse is given by

$$ \begin{align*}\sigma^{-1}_{V_0,W_0}(W)=\sigma_{W_0,V_0}(W)=(W\cap V_0^{\bot})+V_0.\end{align*} $$

Proof. Because $V_0$ and $W_0$ are opposite, we have the following orthogonal decomposition:

$$ \begin{align*}\mathcal{H}=(V_0\oplus W_0)\oplus^{\bot}(V_0^{\bot}\cap W_0^{\bot}),\end{align*} $$

and the restriction of Q to $V_0^{\bot }\cap W_0^{\bot }$ is non-degenerate of signature $(p-k,\infty )$ . In particular, $V_0^{\bot }=V_0\oplus (V_0^{\bot }\cap W_0^{\bot })$ and thus the quotient map induces an isomorphism $(V_0^{\bot }/V_0,Q)\simeq (V_0^{\bot }\cap W_0^{\bot },Q)$ as spaces with Hermitian forms.

Now, if $V\in \pi ^{-1}(V_0)$ is written $V=V_0+V'$ , where $V'=V\cap W_0^{\bot }$ , then $V'\subset V_0^{\bot }\cap W_0^{\bot }$ . In particular, $\sigma _{W_0,V_0}(V)=V'+W_0$ . By construction of the metric through hyperbolic principal angles, the following map are isometries:

Finally, $\sigma _{W_0,V_0}$ is an isometry being the composition of two isometries.

In particular, $\sigma _{V_0,W_0}$ maps flat subspaces to flat subspaces.

Proof of Theorem 1.7 in the Zariski-dense case

In this proof, we freely use measurable metric fields thanks to Lemma 4.11. We know from Theorem 4.10 that there exists a ${\Gamma }$ -map to the visual boundary $\partial {\mathcal X}(p,\infty )$ and, by ergodicity, we get a ${\Gamma }$ -equivariant map $\phi \colon B\to \mathcal {I}_k$ for some $k\geq 1$ . Assume that k is a maximal such integer. If $k=p$ , we are done. Assume then that $k<p$ . Denoting $V_b\in \mathcal {I}_k$ for $\phi (b)$ and $X_b={\mathcal X}(V_b^{\bot }/V_b,Q)$ , we obtain a measurable field of (non-trivial) CAT(0) spaces $\textbf {X}=\{X_b\}$ with a $\Gamma $ -isometric action. Thanks to [Reference DuchesneDuc13, Theorem 1.8], either there is an invariant section of the metric field ${\partial {\textbf X}}=\{\partial X_b\}$ or there is a $\Gamma $ -equivariant Euclidean subfield $\textbf {F}=\{F_b\}$ with $F_b\subset {X}_b$ for almost all $b\in B$ .

In the first case, the stated result easily follows: to every point in $\partial X_b$ , one can associate a (non-trivial) totally isotropic flag in $V_b^{\bot }/V_b$ , and we can choose the totally isotropic subspace of maximal dimension in such a flag (whose dimension is essentially constant by ergodicity) to define a totally isotropic subspace $V^\prime _b$ of $\mathcal {H}$ strictly containing $V_b$ . Thus, we get a $\Gamma $ -map $\phi ^\prime \colon B\to \mathcal {I}_{k^\prime }$ for $k^\prime>k$ , which contradicts the maximality of k.

To conclude the proof, it is enough to show that, under our hypotheses, there cannot exist a $\Gamma $ -equivariant Euclidean subfield. So let us assume that there exists such a subfield. In other words, we have a $\Gamma $ -map $\psi _0\colon B\to F$ , where F is the Polish space constructed from $\textbf {F}$ thanks to Lemma 4.11, such that $\psi _0(b)$ is a flat in $X_b$ . Let us merely denote $F_b$ for $\psi _0(b)$ . Note that the map $b\mapsto \dim (F_b)$ is measurable [Reference Bader, Duchesne and LécureuxBDL16, Lemma 14] and hence $F_b$ is essentially of constant dimension. Among all possible such maps $\psi _0$ , we choose one such that this dimension, say $k_0$ , is minimal.

Let us denote $\sigma _{b,b^\prime }=\sigma _{\phi (b),\phi (b')}$ the correspondence isometry from $ X_b$ to $ X_{b^\prime }$ defined above.

We claim first that $\sigma _{b',b}(F_{b^\prime })$ is parallel to $F_b$ . The proof of this statement is very similar to that of [Reference Bader, Duchesne and LécureuxBDL16, Theorem 34]. We will explain the proof quickly and refer to [Reference Bader, Duchesne and LécureuxBDL16] for more details (in particular, about measurability of the various maps which appear during the proof).

Consider the function $f_{b,b'}$ defined on $F_b$ by $f_{b,b'}(x)=d(x,\sigma _{b',b}(F_{b^\prime }))$ (recall that both $F_b$ and $\sigma _{b',b}(F_{b^\prime })$ are flat subspaces of $X_b$ ). Then, $f_{b,b'}$ is a convex function on the Euclidean space $F_b$ . Using proposition 4 from [Reference Bader, Duchesne and LécureuxBDL16], we see that four cases are possible for $f_{b,b'}$ , which are described below. By the arguments from the proof of [Reference Bader, Duchesne and LécureuxBDL16, Theorem 34], these four conditions are measurable and ${\Gamma }$ -invariant, so that one of them happens almost surely.

The first case is when $f_{b,b'}$ does not attain its infimum m. In that case, one can consider the sequence $E_n^{b,b'}$ of a subset of $F_b$ defined as $E_n^{b,b'}=\{x\mid f(x)\leq m+1/n\}$ . By [Reference DuchesneDuc13, Proposition 8.10], this sequence of subsets gives a $\Gamma $ -map $\xi :B\times B\to \partial F$ , where $\partial F$ is the Borel space associated to the metric field $\partial \textbf {F}$ such that $\xi (b,b')\in \partial F_b$ for almost all $(b,b')$ . Because $\partial \textbf {F}$ is a metric field with a $\Gamma $ -action, using relative isometric ergodicity, we see that $\xi $ does not in fact depend on $b'$ , and therefore we have a map $\xi : B\to \partial X$ (where $\partial X$ is the Borel space associated to the metric field $\partial \textbf X$ ) such that $\xi (b)\in \partial X_b$ . Now $X_b={\mathcal X}(V_b^{\bot }/V_b,Q)$ has a boundary, which is a spherical building where cells correspond to totally isotropic flags in $V_b^{\bot }/V_b$ . Therefore, to a point in the boundary, one can associate a totally isotropic subspace $W\subset V_b^{\bot }/V_b$ , which we can lift to a totally isotropic space $\overline {W}$ containing $V_b$ in ${\mathcal H}$ . Thus the map $\xi $ gives rise to a map $B\to \mathcal I_{k'}$ with $k'>k$ , which contradicts the assumption on k.

If $f_{b,b'}$ attains its minimum m, let $Y=f_{b,b'}^{-1}(m)$ , which depends on b and $b'$ . The second case is when Y is bounded. Then one can consider its circumcenter $y(b,b')$ . The map $(b,b')\mapsto y(b,b')$ is measurable [Reference DuchesneDuc13, Lemma 8.7] and $\Gamma $ -equivariant. By relative isometric ergodicity, it does not in fact depend on $b'$ . So there is a $\Gamma $ -map $x\colon B\to X$ such that $x(b)\in X_b$ . In particular, $\{x(b)\}$ is a Euclidean subfield of $X_b$ and by minimality of $k_0$ , $F_b=\{x_b\}$ and thus we get parallelism of $\sigma _{b,b'}(F_{b'})$ and $F_b$ because any two points are parallel as Euclidean subspaces of dimension zero.

In the third case, one can write $Y=E\times T$ , where E is subflat and T is bounded. Let t be the circumcenter of T and let $E'=E\times \{t\}$ . Then $E'$ is a subflat of some dimension d which, by ergodicity, does not depend on $(b,b')$ . Now the set of subspheres of dimension d of a $k_0$ -dimensional Euclidean space is a metric field with a $\Gamma $ -invariant metric [Reference Bader, Duchesne and LécureuxBDL16, Lemma 20]. By relative isometric ergodicity, the map $(b,b')\mapsto \partial E'$ does not depend on $b'$ . By the second part of [Reference Bader, Duchesne and LécureuxBDL16, Lemma 20], the set of Euclidean subsets of $X_b$ , whose boundary is $\partial E'$ , is again a metric field with a $\Gamma $ -invariant metric. Thus, relative isometric ergodicity again allows us to conclude that the map $(b,b')\mapsto E'$ does not depend on $b'$ . In other words, we get a map which associates to b a subflat of $F_b$ . Because we assumed the dimension of $F_b$ to be minimal, this map must be equal to $\psi _0$ . This means that $f_{b,b'}$ is constant on $F_b$ , and therefore $F_b$ and $\sigma _{b',b}(F_b^\prime )$ are parallel.

In the last case, one can write $Y=E\times T$ , where T is unbounded, but $\partial T$ has a center. Then we get a map which associates to $(b,b')$ the center of $\partial T$ , which is a point in $\partial X_b$ . We conclude by the same argument as in the first case.

This concludes the proof of the claim: $F_b$ is (almost surely) parallel to $\sigma _{b',b}(F_{b'})$ . The set of flats parallel to $F_b$ is a metric field with a $\Gamma $ -invariant metric. Therefore, one can apply again relative isometric ergodicity to prove that the map $(b,b')\mapsto \sigma _{b',b}(F_{b'})$ does not depend on $b'$ . In other words, we get a map $\psi _1$ such that for almost all $b^\prime $ , $\sigma _{b,b^\prime }(\psi _0(b^\prime ))=\psi _1(b)$ . Let us denote $G_b=\psi _1(b)$ . One has $\sigma _{b,b'}(F_{b'})=G_b$ and because $\sigma _{b',b}=\sigma _{b,b'}^{-1}$ , one also has $\sigma _{b,b'}(G_{b'})=F_{b}$ . Thus, $\sigma _{b,b'}$ maps the flat equidistant to $F_{b'}$ and $G_{b'}$ maps the flat equidistant to $F_b$ and $G_b$ .

Up to replacing $\psi _0(b)$ by the flat equidistant to $\psi _0(b)$ and $\psi _1(b)$ , we may assume that $\psi _0(b)=\psi _1(b)$ and thus $\sigma _{b,b^\prime }(F_{b^\prime })=F_b$ for almost all $(b,b^\prime )\in B\times B$ . Let us recall that points in $X_b$ are positive definite subspaces $W\subset V_b^{\bot }/V_b$ and we denote by $\overline {W}$ the preimage of W under the quotient map $V_b^{\bot }\to V_b^{\bot }/V_b$ . Let us denote $\psi (b)=\overline {F_b}\subset \mathcal {V}_k$ , where $\overline {F_b}$ is the collection of $\overline {W}$ for $W\in F_b$ . Because $\sigma _{b,b^\prime }(F_{b^\prime })=F_b$ , if $W\in F_b$ , then $W'=\sigma _{b',b}(W)\in F_{b'}$ satisfies $\overline {W}=V_b+(\overline {W}\cap \overline {W'})$ . In particular, $\operatorname {\textrm {Span}}(\psi (b))\cap \operatorname {\textrm {Span}}(\psi (b'))\neq \{0\}$ for almost all $(b,b')$ . Moreover, the dimension of this intersection is essentially constant by ergodicity.

So, there is $b_0\in B$ such that for almost every b, $\operatorname {\textrm {Span}}(\psi (b_0))\cap \operatorname {\textrm {Span}}(\psi (b))\neq \{0\}$ and this set, $B^\prime $ , of full measure can be assumed to be $\Gamma $ -invariant. Let $\ell _b$ be the line corresponding to $\operatorname {\textrm {Span}}(\psi (b))$ in $\Lambda ^d\mathcal {H}$ , where d is the dimension of $\operatorname {\textrm {Span}}(\psi (b))$ . In particular, for all $b\in B'$ , $\ell _b$ is in the kernel of the map

$$ \begin{align*}\begin{array}{@{}lll} \Lambda^d\mathcal{H}&\to&\Lambda^{2d}\mathcal{H}\\ v&\mapsto&v\wedge v_{b_0}, \end{array}\end{align*} $$

where $v_{b_0}$ is a fixed non-trivial vector in $\ell _{b_0}$ . As in the proof of Proposition 4.14, the closure of the span of $\{\psi (b)\}_{b\in B'}$ is a non-trivial $\Gamma $ -invariant subspace in $\Lambda ^d\mathcal {H}$ . Thanks to Lemma 3.10, we have a contradiction with the Zariski-density assumption.

The statement about transversality is a direct consequence of Proposition 4.14.

4.3 Low-rank cases

In this subsection, we prove that if the rank of the target is at most two, then Zariski density can be relaxed to geometric density to obtain the desired boundary map. The difference between the rank-1 or -2 cases and the general case comes from the complexity of the relative positions of finitely many points in $\mathcal I_k$ for $k\leq p$ . This complexity increases with p but remains manageable in small ranks.

It is shown in [Reference Monod and PyMP14, Proposition 5.5] that geometric density implies irreducibility (in the real case, but the proof works over ${\mathbf{C}}$ and $\textbf {H}$ as well). So we deduce straightforwardly the following.