2.1 The holomorphic chamber

Let T be a maximal torus of K, with Lie algebra $\mathfrak {t}$ . Its dual $\mathfrak {t}^{*}$ can be seen as the subspace of $\mathfrak {g}^{*}$ fixed by T. Let us denote by $\mathfrak {g}^{*}_{e}$ the set formed by the elliptic elements: In other words, $\mathfrak {g}^{*}_{e}:=\mathrm {Ad}^{*}(G)\cdot \mathfrak {t}^{*}$ .

Following [Reference Weinstein38], we consider the invariant open subset $\mathfrak {g}^{*}_{se}=\{\xi \in \mathfrak {g}^{*}\,\vert \, G_{\xi } \ \mathrm {is\ compact} \}$ of strongly elliptic elements. It is nonempty since the groups G and K have the same rank.

We start with the following basic facts.

Proof. The first point is due to the fact that every compact subgroup of G is conjugate to a subgroup of K.

Let $\xi \in \mathcal {C}_{G/K}^{0}$ . There exists $\epsilon>0$ so that

$$ \begin{align*}\langle \xi +\eta,g z\rangle \geq 0,\quad \forall g\in G,\quad \forall \|\eta\|\leq \epsilon. \end{align*} $$

It implies that $|\langle \eta ,g z\rangle | \leq \langle \xi , z\rangle $ , $\forall g\in G_{\xi }$ and $\forall \|\eta \|\leq \epsilon $ . In other words, the adjoint orbit $G_{\xi }\cdot z\subset \mathfrak {g}$ is bounded. For any $g=e^{X} k$ , with $(X,k)\in \mathfrak {p}\times K$ , a direct computation shows that $\|gz\|=\|e^{X} z\|\geq \|[z,X]\|=\|X\|$ . Then, there exists $\rho>0$ such that $\|X\|\leq \rho $ if $e^{X} k\in G_{\xi }$ for some $k\in K$ . It follows that the stabilizer subgroup $G_{\xi }$ is compact.

Let $\wedge ^{*}\subset \mathfrak {t}^{*}$ be the weight lattice: By definition, $\alpha \in \wedge ^{*}$ if and only if $i\alpha $ is the differential of a character of T. Let $\mathfrak {R}\subset \wedge ^{*}$ be the set of roots for the action of T on $\mathfrak {g}\otimes \mathbb {C}$ . We have $\mathfrak {R}=\mathfrak {R}_{c}\cup \mathfrak {R}_{n}$ , where $\mathfrak {R}_{c}$ and $\mathfrak {R}_{n}$ are, respectively, the set of roots for the action of T on $\mathfrak {k}\otimes \mathbb {C}$ and $\mathfrak {p}\otimes \mathbb {C}$ . We fix a system of positive roots $\mathfrak {R}^{+}_{c}$ in $\mathfrak {R}_{c}$ , and we denote by $\mathfrak {t}^{*}_{\geq 0}$ the corresponding Weyl chamber.

We have $\mathfrak {p}\otimes \mathbb {C}=\mathfrak {p}^{+}\oplus \mathfrak {p}^{-}$ , where the K-module $\mathfrak {p}^{\pm }$ is equal to $\ker (\mathrm {ad}(z)\mp i)$ . Let $\mathfrak {R}^{\pm ,z}_{n}$ be the set of roots for the action of T on $\mathfrak {p}^{\pm }$ . The union

(6) $$ \begin{align} \mathfrak{R}^{+}=\mathfrak{R}^{+}_{c}\cup\mathfrak{R}^{+,z}_{n} \end{align} $$

defines then a system of positive roots in $\mathfrak {R}$ . We notice that $\mathfrak {R}^{+,z}_{n}$ is the set of roots $\beta \in \mathfrak {R}$ satisfying $\langle \beta , z\rangle =1$ . Hence, $\mathfrak {R}^{+,z}_{n}$ is invariant relatively to the action of the Weyl group $W=N(T)/T$ .

Let us recall the following classical fact concerning the parametrization of the G-orbits in $\mathcal {C}_{G/K}^{0}$ via the holomorphic chamber

$$ \begin{align*}\mathcal{C}_{\mathrm{hol}}:=\{\xi\in\mathfrak{t}^{*}_{\geq 0},(\xi,\beta)>0,\ \forall \beta\in \mathfrak{R}^{+,z}_{n}\}. \end{align*} $$

The elliptic coadjoint orbits of G, i.e., those contained in $\mathfrak {g}^{*}_{e}$ , are parameterized by the Weyl chamber $\mathfrak {t}^{*}_{\geq 0}$ . Thus, we have a projection $\mathrm {p}: \mathfrak {g}^{*}_{e}\to \mathfrak {t}^{*}_{\geq 0}$ , defined by the relations $G\xi \cap \mathfrak {t}^{*}_{\geq 0}=\{ \mathrm {p}(\xi )\}$ , and that induces a bijection $\mathfrak {g}^{*}_{e}/G\simeq \mathfrak {t}^{*}_{\geq 0}$ .

Proof. Let us choose some maximal strongly orthogonal system $\Sigma \subset \mathfrak {R}^{+,z}_{n}$ . The real span $\mathfrak {a}$ of the $X_{\beta },\beta \in \Sigma $ is a maximal abelian subspace of $\mathfrak {p}$ . Hence, any element $g\in G$ can be written $g=k e^{X(t)} k^{\prime }$ with $X(t)=\sum _{\beta \in \Sigma }t_{\beta } X_{\beta }$ and $k,k^{\prime }\in K$ . We get

(7) $$ \begin{align} gz=k\left(z +\sum_{\beta\in\Sigma}(\mathrm{cosh}(t_{\beta})-1)h_{\beta} +\sum_{\beta\in\Sigma}\mathrm{sinh}(t_{\beta})Y_{\beta}\right) \end{align} $$

and

$$ \begin{align*}\langle \xi,g z\rangle=\langle k^{-1}\xi,z\rangle + \sum_{\beta\in\Sigma}(\mathrm{cosh}(t_{\beta})-1) \langle k^{-1}\xi,h_{\beta}\rangle. \end{align*} $$

For any $\xi \in \mathcal {C}_{\mathrm {hol}}$ , we define $c_{\xi } := \min _{\beta \in \mathfrak {R}^{+,z}_{n}}\langle \xi ,h_{\beta }\rangle>0$ . Let $\pi :\mathfrak {k}^{*}\to \mathfrak {t}^{*}$ be the projection. We have $\langle k^{-1}\xi ,z\rangle =\langle \pi (k^{-1}\xi ),z\rangle $ and $\langle k^{-1}\xi ,h_{\beta }\rangle =\langle \pi (k^{-1}\xi ),h_{\beta }\rangle $ . The convexity theorem of Kostant tell us that $\pi (k^{-1}\xi )$ belongs to the convex hull of $\{w\xi , w\in W\}$ . It follows that $\langle k^{-1}\xi ,z\rangle \geq \langle \xi ,z\rangle>0 \quad \mathrm {and}\quad \langle k^{-1}\xi ,h_{\beta }\rangle \geq c_{\xi }>0$ for any $k\in K$ . We obtain then that $\langle \xi ,g z\rangle \geq \frac {1}{2}\min (\langle \xi ,z\rangle , c_{\xi }) e^{\|X(t)\|}$ for any $\xi \in \mathcal {C}_{\mathrm {hol}}$ , where $\|X(t)\|=\sup _{\beta } |t_{\beta }|$ . From equation (7), it is not difficult to see that there exists $C>0$ such that $\|gz\|\leq C e^{\|X(t)\|}$ for any $g=k e^{X(t)} k^{\prime }\in G$ .

Let $\mathcal {K}$ be a compact subset of $\mathcal {C}_{\mathrm {hol}}$ . Take $c_{\mathcal {K}}=\frac {1}{2C}\min (\min _{\xi \in \mathcal {K}}\langle \xi ,z\rangle , \min _{\xi \in \mathcal {K}}c_{\xi })>0$ . The previous computations show that $\langle \xi ,g z\rangle \geq c_{\mathcal {K}} \|g z\|,\ \forall g\in G,\ \forall \xi \in \mathcal {K}$ .

The following result is needed in §4.1.

Proof. Let $(\xi _{n})$ be a sequence of $\mathcal {C}_{G/K}^{0}$ converging to $\xi _{\infty }\in \mathcal {C}_{G/K}^{0}$ . Let $\xi ^{\prime }_{n} =\mathrm {p}(\xi _{n})$ and $\xi ^{\prime }_{\infty } =\mathrm {p}(\xi _{\infty })$ : We have to prove that the sequence $(\xi ^{\prime }_{n})$ converges to $\xi ^{\prime }_{\infty }$ . We choose elements $g_{n},g_{\infty }\in G$ such that $\xi _{n}=g_{n}\xi ^{\prime }_{n},\forall n$ and $\xi _{\infty }=g_{\infty }\xi ^{\prime }_{\infty }$ .

First, we notice that $-b(\xi _{n},\xi _{n})=\|\xi ^{\prime }_{n}\|^{2}$ ; hence, the sequence $(\xi ^{\prime }_{n})$ is bounded. We will now prove that the sequence $(g_{n})$ is bounded. Let $\epsilon>0$ such that $\langle \xi _{\infty }+\eta ,gz\rangle \geq 0$ , $\forall g\in G$ , $\forall \|\eta \|\leq \epsilon $ . If $\|\xi -\xi _{\infty }\|\leq \epsilon /2$ , we write $\xi =\frac {1}{2}(\xi _{\infty }+ 2(\xi -\xi _{\infty }))+\frac {1}{2}\xi _{\infty }$ , and then

$$ \begin{align*}\langle\xi,gz\rangle= \frac{1}{2}\langle\xi_{\infty}+ 2(\xi-\xi_{\infty}),gz\rangle +\frac{1}{2}\langle\xi_{\infty},gz\rangle\geq \frac{1}{2}\langle\xi_{\infty},gz\rangle, \quad \forall g\in G. \end{align*} $$

Now we have $\langle \xi ^{\prime }_{n},z\rangle =\langle \xi _{n},g_{n}z\rangle \geq \frac {1}{2}\langle \xi _{\infty },g_{n}z\rangle $ if n is large enough. This shows that the sequence $\langle \xi _{\infty },g_{n}z\rangle $ is bounded. If we use Lemma 2.3, we can conclude that the sequence $(g_{n})$ is bounded.

Let $(\xi ^{\prime }_{\phi (n)})$ be a subsequence converging to $\ell \in \mathfrak {t}^{*}_{\geq 0}$ . Since $(g_{\phi (n)})$ is bounded, there exists a subsequence $(g_{\phi \circ \psi (n)})$ converging to $h\in G$ . From the relations $\xi _{\phi \circ \psi (n)}=$ $g_{\phi \circ \psi (n)}\xi ^{\prime }_{\phi \circ \psi (n)},\forall n\in \mathbb {N}$ , we obtain $\xi _{\infty }=h\ell $ . Then $\ell =\mathrm {p}(\xi _{\infty })= \xi _{\infty }^{\prime }$ . Since every subsequence of $(\xi ^{\prime }_{n})$ has a subsequential limit to $\xi _{\infty }^{\prime }$ , then the sequence $(\xi ^{\prime }_{n})$ converges to $\xi ^{\prime }_{\infty }$ .

2.2 The cone $\Delta _{\mathrm {hol}}(\tilde {G},G)$ is closed

We suppose that $G/K$ is a complex submanifold of a Hermitian symmetric space $\tilde {G}/\tilde {K}$ . In other words, $\tilde {G}$ is a reductive real Lie group such that $G\subset \tilde {G}$ is a closed connected subgroup preserved by the Cartan involution, and $\tilde {K}$ is a maximal compact subgroup of $\tilde {G}$ containing K. We denote by $\tilde {\mathfrak {g}}$ and $\tilde {\mathfrak {k}}$ the Lie algebras of $\tilde {G}$ and $\tilde {K}$ , respectively. We suppose that there exists a $\tilde {K}$ -invariant element $z\in \mathfrak {k}$ such that $\mathrm {ad}(z)\vert _{\tilde {\mathfrak {p}}}$ defines a complex structure on $\tilde {\mathfrak {p}}$ : $(\mathrm {ad}(z)\vert _{\tilde {\mathfrak {p}}})^{2}= -Id_{\tilde {\mathfrak {p}}}$ .

Let $\mathcal {C}_{\tilde {G}/\tilde {K}}\subset \tilde {\mathfrak {g}}^{*}$ be the closed invariant cone associated to the Hermitian symmetric space $\tilde {G}/\tilde {K}$ . We start with the following key fact.

Let $\tilde {T}$ be a maximal torus of $\tilde {K}$ , with Lie algebra $\tilde {\mathfrak {t}}$ . The $\tilde {G}$ -orbits in the interior of $\mathcal {C}_{\tilde {G}/\tilde {K}}$ are parametrized by the holomorphic chamber $\tilde {\mathcal {C}}_{\mathrm {hol}}\subset \tilde {\mathfrak {t}}^{*}$ . The previous lemma says that the projection $\pi _{\mathfrak {g},\tilde {\mathfrak {g}}}(\tilde {\mathcal {O}})$ of any $\tilde {G}$ -orbit $\tilde {\mathcal {O}}\subset \mathcal {C}_{\tilde {G}/\tilde {K}}^{0}$ is the union of G-orbits $\mathcal {O}\subset \mathcal {C}^{0}_{G/K}$ . So it is natural to study the following object:

(8) $$ \begin{align} \Delta_{\mathrm{hol}}(\tilde{G},G):=\left\{(\tilde{\xi},\xi)\in \tilde{\mathcal{C}}_{\mathrm{hol}}\times\mathcal{C}_{\mathrm{hol}}; \ G\xi\,\subset \,\pi_{\mathfrak{g},\tilde{\mathfrak{g}}}\big(\tilde{G}\tilde{\xi}\,\big)\right\}. \end{align} $$

Here is a first result.

2.3 Rational and weakly regular points

Let $(M,\Omega )$ be a symplectic manifold. We suppose that there exists a line bundle $\mathcal {L}$ with connection $\nabla $ that prequantizes the $2$ -form $\Omega $ : In other words, $\nabla ^{2}=- i\, \Omega $ . Let K be a compact connected Lie group acting on $\mathcal {L}\to M$ , and leaving the connection invariant. Let $\Phi _{K}:M\to \mathfrak {k}^{*}$ be the moment map defined by Kostant’s relations

(9) $$ \begin{align} L_{X}-\nabla_{X}=i\langle \Phi_{K},X\rangle,\quad \forall X\in\mathfrak{k}. \end{align} $$

Here $L_{X}$ is the Lie derivative acting on the sections of $\mathcal {L}\to M$ .

Remark that relations (9) imply, via the equivariant Bianchi formula, the relations

(10) $$ \begin{align} \iota(X_{M})\Omega=- d\langle \Phi_{K},X\rangle,\quad \forall X\in\mathfrak{k}, \end{align} $$

where $X_{M}(m) :=\frac {d}{dt}\vert _{t=0}e^{-tX}m$ is the vector field on M generated by $X\in \mathfrak {k}$ .

When $\xi \in \mathfrak {k}^{*}$ is a weakly regular value of $\Phi _{K}$ , the constant rank theorem tells us that $\Phi _{K}^{-1}(\xi )$ is a submanifold of M stable under the action of the stabilizer subgroup $K_{\xi }$ . We see then that the reduced space

(11) $$ \begin{align} M_{\xi}:=\Phi_{K}^{-1}(\xi)/K_{\xi} \end{align} $$

is a symplectic orbifold.

Let $T\subset K$ be a maximal torus with Lie algebra $\mathfrak {t}$ . We consider the lattice $\wedge :=\frac {1}{2\pi }\ker (\exp :\mathfrak {t}\to T)$ and the dual lattice $\wedge ^{*}\subset \mathfrak {t}^{*}$ defined by $\wedge ^{*}=\hom (\wedge ,\mathbb {Z})$ . We remark that $i\eta $ is a differential of a character of T if and only if $\eta \in \wedge ^{*}$ . The $\mathbb {Q}$ -vector space generated by the lattice $\wedge ^{*}$ is denoted by $\mathfrak {t}^{*}_{\mathbb {Q}}$ : The vectors belonging to $\mathfrak {t}^{*}_{\mathbb {Q}}$ are designed as rational. An affine subspace $V\subset \mathfrak {t}^{*}$ is called rational if it is affinely generated by its rational points.

We also fix a closed positive Weyl chamber $\mathfrak {t}^{*}_{\geq 0}$ . For each relatively open face $\sigma \subset \mathfrak {t}^{*}_{\geq 0}$ , the stabilizer $K_{\xi }$ of points $\xi \in \sigma $ under the coadjoint action does not depend on $\xi $ and will be denoted by $K_{\sigma }$ . The Lie algebra $\mathfrak {k}_{\sigma }$ decomposes into its semisimple and central parts $\mathfrak {k}_{\sigma }=[\mathfrak {k}_{\sigma },\mathfrak {k}_{\sigma }]\oplus \mathfrak {z}_{\sigma }$ . The subspace $\mathfrak {z}_{\sigma }^{*}$ is defined to be the annihilator of $[\mathfrak {k}_{\sigma },\mathfrak {k}_{\sigma }]$ or, equivalently, the fixed point set of the coadjoint $K_{\sigma }$ action. Notice that $\mathfrak {z}_{\sigma }^{*}$ is a rational subspace of $\mathfrak {t}^{*}$ and that the face $\sigma $ is an open cone of $\mathfrak {z}_{\sigma }^{*}$ ,

We suppose that the moment map $\Phi _{K}$ is proper. The convexity theorem [Reference Atiyah1, Reference Guillemin and Sternberg10, Reference Kirwan16, Reference Sjamaar35, Reference Lerman, Meinrenken, Tolman and Woodward22] tells us that $\Delta _{K}(M):=\mathrm {Image}(\Phi _{K})\, \bigcap \, \mathfrak {t}_{\geq 0}^{*}$ is a closed, convex, locally polyhedral set.

We will use the following remark in the next sections.

2.4 Weinstein’s theorem

Let $\tilde {a}\in \tilde {\mathcal {C}}_{\mathrm {hol}}$ . Consider the Hamiltonian action of the group G on the coadjoint orbit $\tilde {G}\tilde {a}$ . The moment map $\Phi _{G}^{\tilde {a}}:\tilde {G}\tilde {a}\to \mathfrak {g}^{*}$ corresponds to the restriction of the projection $\pi _{\mathfrak {g},\tilde {\mathfrak {g}}}$ to $\tilde {G}\tilde {a}$ . In this setting, the following conditions holds:

1. 1. The action of G on $\tilde {G}\tilde {a}$ is proper.

2. 2. The moment map $\Phi _{G}^{\tilde {a}}$ is a proper map since the map $\langle \Phi _{G}^{\tilde {a}},z\rangle $ is proper (see Lemma 2.3).

Conditions 1 and 2 impose that the image of $\Phi _{G}^{\tilde {a}}$ is contained in the open subset $\mathfrak {g}^{*}_{se}$ of strongly elliptic elements [Reference Paradan31]. Thus, the G-orbits contained in the image of $\Phi _{G}^{\tilde {a}}$ are parametrized by the following subset of the holomorphic chamber $\mathcal {C}_{\mathrm {hol}}$ :

$$ \begin{align*}\Delta_{G}(\tilde{G}\tilde{a}):=\mathrm{Image}(\Phi_{G}^{\tilde{a}})\, \bigcap \, \mathfrak{t}^{*}_{\geq 0}. \end{align*} $$

We notice that $\Delta _{\mathrm {hol}}(\tilde {G},G)=\bigcup _{\tilde {a}\in \tilde {\mathcal {C}}_{\mathrm {hol}}}\{\tilde {a}\}\times \Delta _{G}(\tilde {G}\tilde {a})$ .

Like in Definition 2.7, an element $\xi \in \mathfrak {g}^{*}$ is a weakly regular value of $\Phi _{G}^{\tilde {a}}$ if for all $m\in (\Phi _{G}^{\tilde {a}})^{-1}(\xi )$ we have $\dim \mathfrak {g}_{m}=\min _{x\in \tilde {G}\tilde {a}}\dim (\mathfrak {g}_{x})$ . We denote by $\Delta _{G}(\tilde {G}\tilde {a})^{0}$ the set of elements $\xi \in \Delta _{G}(\tilde {G}\tilde {a})$ that are weakly regular for $\Phi _{G}^{\tilde {a}}$ .

Proof. We recall briefly the arguments of the proof (see [Reference Weinstein38] or [Reference Paradan31][§2]). Under Conditions 1 and 2, one checks easily that $Y_{\tilde {a}}:=(\Phi _{G}^{\tilde {a}})^{-1}(\mathfrak {k}^{*})$ is a smooth K-invariant symplectic submanifold of $\tilde {G}\tilde {a}$ such that

(12) $$ \begin{align} \tilde{G}\tilde{a}\simeq G\times_{K} Y_{\tilde{a}}. \end{align} $$

The moment map $\Phi _{K}^{\tilde {a}}:Y_{\tilde {a}}\to \mathfrak {k}^{*}$ , which corresponds to the restriction of the map $\Phi _{G}^{\tilde {a}}$ to $Y_{\tilde {a}}$ , is a proper map. Hence, the convexity theorem tells us that $\Delta _{K}(Y_{\tilde {a}}):=\mathrm {Image}(\Phi _{K}^{\tilde {a}})\bigcap \mathfrak {t}^{*}_{\geq 0}$ is a closed, convex, locally polyhedral set. Thanks to the isomorphism (12), we see that $\Delta _{G}(\tilde {G}\tilde {a})$ coincides with the closed convex subset $\Delta _{K}(Y_{\tilde {a}})$ . The proof is completed.

The next lemma is used in §4.1.

3.1 Definition

We return to the framework of §2.1. We recall the notion of holomorphic discrete series representations associated to a Hermitian symmetric spaces $G/K$ . Let us introduce

$$ \begin{align*}\mathcal{C}_{\mathrm{hol}}^{\rho}:=\left\{ \xi\in\mathfrak{t}^{*}_{\geq 0} \vert\ (\xi,\beta)\geq (2\rho_{n},\beta) , \ \forall \beta\in \mathfrak{R}_{n}^{+,z}\right\}, \end{align*} $$

where $2\rho _{n}=\sum _{\beta \in \mathfrak {R}_{n}^{+,z}}\beta $ is W-invariant.

We will be interested in the following subset of dominant weights:

$$ \begin{align*}\widehat{G}_{\mathrm{hol}}:=\wedge^{*}_{+}\bigcap \mathcal{C}_{\mathrm{hol}}^{\rho}. \end{align*} $$

Let $\mathrm {Sym}(\mathfrak {p})$ be the symmetric algebra of the complex K-module $(\mathfrak {p},\mathrm {ad}(z))$ .

The set $V^{G}_{\lambda },\lambda \in \widehat {G}_{\mathrm {hol}}$ corresponds to the holomorphic discrete series representations associated to the complex structure $\mathrm {ad}(z)$ .

3.2 Restriction

We come back to the framework of §2.2. We consider two compatible Hermitian symmetric spaces $G/K\subset \tilde {G}/\tilde {K}$ , and we look after the restriction of holomorphic discrete series representations of $\tilde {G}$ to the subgroup G.

Let $\tilde {\lambda }\in \widehat {\tilde {G}}_{\mathrm {hol}}$ . Since the representation $V^{\tilde {G}}_{\tilde {\lambda }}$ is discretely admissible relatively to the circle group $\exp (\mathbb {R} z)$ , it is also discretely admissible relatively to G. We can be more precise [Reference Jakobsen and Vergne15, Reference Martens24, Reference Kobayashi21]:

Proposition 3.3. We have an Hilbertian direct sum

$$ \begin{align*}V^{\tilde{G}}_{\tilde{\lambda}}\vert_{G}=\bigoplus_{\lambda\in \widehat{G}_{\mathrm{hol}}} m_{\tilde{\lambda}}^{\lambda} \ V^{G}_{\lambda}, \end{align*} $$

where the multiplicity $m_{\tilde {\lambda }}^{\lambda } :=[V^{G}_{\lambda }:V^{\tilde {G}}_{\tilde {\lambda }}]$ is finite for any $\lambda $ .

The Hermitian $\tilde {K}$ -vector space $\tilde {\mathfrak {p}}$ , when restricted to the K-action, admits an orthogonal decomposition $\tilde {\mathfrak {p}}=\mathfrak {p}\oplus \mathfrak {q}$ . Notice that the symmetric algebra $\mathrm {Sym}(\mathfrak {q})$ is an admissible K-module.

In [Reference Jakobsen and Vergne15], H. P. Jakobsen and M. Vergne obtained the following nice characterization of the multiplicities $[V^{G}_{\lambda }:V^{\tilde {G}}_{\tilde {\lambda }}]$ . Another proof is given in [Reference Paradan31], §4.4.

3.3 Discrete analogues of $\Delta _{\mathrm {hol}}(\tilde {G},G)$

We define the following discrete analogues of the cone $\Delta _{\mathrm {hol}}(\tilde {G},G)$ :

(13) $$ \begin{align} \Pi_{\mathrm{hol}}^{\mathbb{Z}}(\tilde{G},G):=\left\{(\tilde{\lambda},\lambda)\in \widehat{\tilde{G}}_{\mathrm{hol}}\times\widehat{G}_{\mathrm{hol}}\; \ [V^{G}_{\lambda}:V^{\tilde{G}}_{\tilde{\lambda}}]\neq 0 \right\}, \end{align} $$

and

(14) $$ \begin{align} \Pi_{\mathrm{hol}}^{\mathbb{Q}}(\tilde{G},G):=\left\{(\tilde{\xi},\xi)\in \tilde{\mathcal{C}}_{\mathrm{hol}}\times\mathcal{C}_{\mathrm{hol}}\; \ \exists N\geq 1,\ (N\xi,N\tilde{\xi})\in \Pi_{\mathrm{hol}}^{\mathbb{Z}}(\tilde{G},G)\right\}. \end{align} $$

We have the following key fact.

3.4 Riemann–Roch numbers

We come back to the framework of §2.3.

We associate to a dominant weight $\mu \in \wedge ^{*}_{+}$ the (possibly singular) symplectic reduced space $M_{\mu }:=\Phi _{K}^{-1}(\mu )/K_{\mu }$ and the (possibly singular) line bundle over $M_{\mu }$ :

$$ \begin{align*}\mathcal{L}_{\mu}:=\left(\mathcal{L}\vert_{\Phi_{K}^{-1}(\mu)}\otimes \mathbb{C}_{-\mu}\right)/K_{\mu}. \end{align*} $$

$\underline {\mbox {Suppose first that }\mu \mbox { is a weakly regular value of }\Phi _{K}.}$ Then $M_{\mu }$ is an orbifold equipped with a symplectic structure $\Omega _{\mu }$ , and $\mathcal {L}_{\mu }$ is a line orbi-bundle over $M_{\mu }$ that prequantizes the symplectic structure. By choosing an almost complex structure on $M_{\mu }$ compatible with $\Omega _{\mu }$ , we get a decomposition $\wedge \textbf { T}^{*} M_{\mu }\otimes \mathbb {C} = \oplus _{i,j}\wedge ^{i,j} \textbf { T}^{*} M_{\mu }$ of the bundle of differential forms. Using Hermitian structures in the tangent bundle $\textbf { T} M_{\mu }$ of $M_{\mu }$ and in the fibers of $\mathcal {L}_{\mu }$ , we define a Dolbeaut–Dirac operator

$$ \begin{align*}D^{+}_{\mu} : \mathcal{A}^{0,+}(M_{\mu},\mathcal{L}_{\mu})\longrightarrow \mathcal{A}^{0,-}(M_{\mu},\mathcal{L}_{\mu}), \end{align*} $$

where $\mathcal {A}^{i,j}(M_{\mu },\mathcal {L}_{\mu })=\Gamma (M_{\mu },\wedge ^{i,j} \textbf { T}^{*} M_{\mu }\otimes \mathcal {L}_{\mu })$ .

$\underline {\mbox {Suppose that }\mu \notin \Delta _{K}(M).}$ Then $M_{\mu }=\emptyset $ , and we take $RR(M_{\mu },\mathcal {L}_{\mu })=0$ .

$\underline {\mbox {Suppose now that }\mu \in \Delta _{K}(M)\mbox { is not (necessarily) a weakly regular value of }\Phi _{K}.}$ Take a small element $\epsilon \in \mathfrak {t}^{*}$ such that $\mu +\epsilon $ is a weakly regular value of $\Phi _{K}$ belonging to $\Delta _{K}(M)$ . We consider the symplectic orbifold $M_{\mu +\epsilon }$ : If $\epsilon $ is small enough,

$$ \begin{align*}\mathcal{L}_{\mu,\epsilon}:=\left(\mathcal{L}\vert_{\Phi_{K}^{-1}(\mu+\epsilon)}\otimes \mathbb{C}_{-\mu}\right)/K_{\mu+\epsilon}. \end{align*} $$

is a line orbi-bundle over $M_{\mu +\epsilon }$ .

We have the following important result (see §3.4.3 in [Reference Paradan and Vergne34]).

We can now introduce the following definition.

Definition 3.8. Let $\mu \in \wedge ^{*}_{+}$ . We define

$$ \begin{align*}\mathcal{Q}(M_{\mu},\Omega_{\mu})= \begin{cases} 0 \hspace{26.5mm} \mathrm{if }\quad \mu\notin \Delta_{K}(M),\\ RR(M_{\mu+\epsilon},\mathcal{L}_{\mu,\epsilon})\hspace{4mm} \mathrm{if }\quad \mu\in \Delta_{K}(M). \end{cases} \end{align*} $$

Above, $\epsilon $ is chosen small enough so that $\mu +\epsilon \in \Delta _{K}(M)$ is a weakly regular value of $\Phi _{K}$ .

Let $n\geq 1$ . The manifold M, equipped with the symplectic structure $n\Omega $ , is prequantized by the line bundle $\mathcal {L}^{\otimes n}$ : The corresponding moment map is $n\Phi _{K}$ . For any dominant weight $\mu \in \wedge ^{*}_{+}$ , the symplectic reduction of $(M,n\Omega )$ relatively to the weight $n\mu $ is $(M_{\mu },n\Omega _{\mu })$ . Like in Definition 3.8, we consider the following Riemann–Roch numbers

$$ \begin{align*}\mathcal{Q}(M_{\mu},n\Omega_{\mu})= \begin{cases} 0 \hspace{33.4mm} \mathrm{if }\quad \mu\notin \Delta_{K}(M),\\ RR(M_{\mu+\epsilon},(\mathcal{L}_{\mu,\epsilon})^{\otimes n})\hspace{4mm} \mathrm{if }\quad \mu\in \Delta_{K}(M)\ \mathrm{and} \ \|\epsilon\|<<1. \end{cases} \end{align*} $$

The Kawasaki–Riemann–Roch formula shows that $n\geq 1\mapsto \mathcal {Q}(M_{\mu },n\Omega _{\mu })$ is a quasi-polynomial map [Reference Vergne37, Reference Loizides23]. When $\mu $ is a weakly regular value of $\Phi _{K}$ , we denote by $\mathrm {vol}(M_{\mu }):=\frac {1}{d_{\mu }}\int _{M_{\mu }}(\frac {\Omega _{\mu }}{2\pi })^{\frac {\dim M_{\mu }}{2}}$ the symplectic volume of the symplectic orbifold $(M_{\mu },\Omega _{\mu })$ . Here, $d_{\mu }$ is the generic value of the map $m\in \Phi _{K}^{-1}(\mu )\mapsto \mathrm {cardinal}(K_{m}/K^{0}_{m})$ .

The following proposition is a direct consequence of the Kawasaki–Riemann–Roch formula (see [Reference Loizides23] or §1.3.4 in [Reference Paradan30]).

3.5 Quantization commutes with reduction

Let us explain the “quantization commutes with reduction” theorem proved in [Reference Paradan31].

We fix $\tilde {\lambda }\in \widehat {\tilde {G}}_{\mathrm {hol}}$ . The coadjoint orbit $\tilde {G}\tilde {\lambda }$ is prequantized by the line bundle $\tilde {G}\times _{K_{\tilde {\lambda }}}\mathbb {C}_{\tilde {\lambda }}$ , and the moment map $\Phi _{G}^{\tilde {\lambda }}:\tilde {G}\tilde {\lambda }\to \mathfrak {g}^{*}$ corresponding to the G-action on $\tilde {G}\times _{K_{\tilde {\lambda }}}\mathbb {C}_{\tilde {\lambda }}$ is equal to the restriction of the map $\pi _{\mathfrak {g},\tilde {\mathfrak {g}}}$ to $\tilde {G}\tilde {\lambda }$ .

The symplectic slice $Y_{\tilde {\lambda }}=(\Phi _{G}^{\tilde {\lambda }})^{-1}(\mathfrak {k}^{*})$ is prequantized by the line bundle $\mathcal {L}_{\tilde {\lambda }}:=\tilde {G}\times _{K_{\tilde {\lambda }}}\mathbb {C}_{\tilde {\lambda }}\vert _{Y_{\tilde {\lambda }}}$ . The moment map $\Phi _{K}^{\tilde {\lambda }}:Y_{\tilde {\lambda }}\to \mathfrak {k}^{*}$ corresponding to the K-action is equal to the restriction of $\Phi _{G}^{\tilde {\lambda }}$ to $Y_{\tilde {\lambda }}$ .

For any $\lambda \in \widehat {G}_{\mathrm {hol}}$ , we consider the (possibly singular) symplectic reduced space

$$ \begin{align*}\mathbb{X}_{\tilde{\lambda},\lambda}:=(\Phi_{K}^{\tilde{\lambda}})^{-1}(\lambda)/K_{\lambda}, \end{align*} $$

equipped with the reduced symplectic form $\Omega _{\tilde {\lambda },\lambda }$ , and the (possibly singular) line bundle

$$ \begin{align*}\mathbb{L}_{\tilde{\lambda},\lambda}:=\left(\mathcal{L}_{\tilde{\lambda}}\vert_{(\Phi_{K}^{\tilde{\lambda}})^{-1}(\lambda)}\otimes \mathbb{C}_{-\lambda}\right)/K_{\lambda}. \end{align*} $$

Thanks to Definition 3.8, the geometric quantization $\mathcal {Q}(\mathbb {X}_{\tilde {\lambda },\lambda },\Omega _{\tilde {\lambda },\lambda })\in \mathbb {Z}$ of those compact symplectic spaces $(\mathbb {X}_{\tilde {\lambda },\lambda },\Omega _{\tilde {\lambda },\lambda })$ are well-defined even if they are singular. In particular, $\mathcal {Q}(\mathbb {X}_{\tilde {\lambda },\lambda },\Omega _{\tilde {\lambda },\lambda })=0$ when $\mathbb {X}_{\tilde {\lambda },\lambda }=\emptyset $ .

The following theorem is proved in [Reference Paradan31].

Theorem 3.10. Let $\tilde {\lambda }\in \widehat {\tilde {G}}_{\mathrm {hol}}$ . We have an Hilbertian direct sum

$$ \begin{align*}V^{\tilde{G}}_{\tilde{\lambda}}\vert_{G}=\bigoplus_{\lambda\in \widehat{G}_{\mathrm{hol}}} \mathcal{Q}(\mathbb{X}_{\tilde{\lambda},\lambda},\Omega_{\tilde{\lambda},\lambda}) \ V^{G}_{\lambda}. \end{align*} $$

It means that, for any $\lambda \in \widehat {G}_{\mathrm {hol}}$ , the multiplicity of the representation $V^{G}_{\lambda }$ in the restriction $V^{\tilde {G}}_{\tilde {\lambda }}\vert _{G}$ is equal to the geometric quantization $\mathcal {Q}(\mathbb {X}_{\tilde {\lambda },\lambda },\Omega _{\tilde {\lambda },\lambda })$ of the (compact) reduced space $\mathbb {X}_{\tilde {\lambda },\lambda }$ .

Remark 3.11. Let $({\tilde {\lambda },\lambda })\in \widehat {\tilde {G}}_{\mathrm {hol}}\times \widehat {G}_{\mathrm {hol}}$ . Theorem 3.10. shows that

$$ \begin{align*}\left[V^{G}_{n\lambda} : V^{\tilde{G}}_{n\tilde{\lambda}}\right]= \mathcal{Q}(\mathbb{X}_{\tilde{\lambda},\lambda},n\Omega_{\tilde{\lambda},\lambda}) \end{align*} $$

for any $n\geq 1$ .

4.1 Proof of Theorem A

The set $\Delta _{\mathrm {hol}}(\tilde {G},G)$ is equal to $\bigcup _{\tilde {a}\in \tilde {\mathcal {C}}_{\mathrm {hol}}}\{\tilde {a}\}\times \Delta _{G}(\tilde {G}\tilde {a})$ . We define

$$ \begin{align*}\Delta_{\mathrm{hol}}(\tilde{G},G)^{0}:=\bigcup_{\tilde{a}\in \tilde{\mathcal{C}}_{\mathrm{hol}}} \{\tilde{a}\} \times \Delta_{G}(\tilde{G}\tilde{a})^{0}. \end{align*} $$

We start with the following result.

The main purpose of this section is the proof of the following theorem.

Theorem 4.2. For any rational element $(\tilde {\mu },\mu )$ of the holomorphic chamber $\tilde {\mathcal {C}}_{\mathrm {hol}}\times \mathcal {C}_{\mathrm {hol}}$ , the following statements hold:

• • If $\mu \in \Delta _{G}(\tilde {G}\tilde {\mu })^{0}$ , then $(\tilde {\mu },\mu )\in \Pi ^{\mathbb {Q}}_{\mathrm {hol}}(\tilde {G},G)$ .

• • If $(\tilde {\mu },\mu )\in \Pi ^{\mathbb {Q}}_{\mathrm {hol}}(\tilde {G},G)$ , then $\mu \in \Delta _{G}(\tilde {G}\tilde {\mu })$ .

In other words, we have the following inclusions:

$$ \begin{align*} \Delta_{\mathrm{hol}}(\tilde{G},G)^{0}\, \bigcap\, \tilde{\mathfrak{t}}^{*}_{\mathbb{Q}} \times \mathfrak{t}^{*}_{\mathbb{Q}} \quad \underset{(1)}{\Large{\subset}}\quad \Pi_{\mathrm{hol}}^{\mathbb{Q}}(\tilde{G},G) \quad \underset{(2)}{\Large{\subset}}\quad \Delta_{\mathrm{hol}}(\tilde{G},G). \end{align*} $$

Lemma 4.1 and Theorem 4.2 gives the important corollary.

Now we can terminate the proof of Theorem A.

Thanks to Proposition 3.5, we know that $\Pi _{\mathrm {hol}}^{\mathbb {Q}}(\tilde {G},G)$ is a $\mathbb {Q}$ -convex cone. Since $\Delta _{\mathrm {hol}}(\tilde {G},G)$ is a closed subset of $\tilde {\mathcal {C}}_{\mathrm {hol}}\times \mathcal {C}_{\mathrm {hol}}$ (see Proposition 2.6), we can conclude, by a density argument, that $\Delta _{\mathrm {hol}}(\tilde {G},G)$ is a closed convex cone of $\tilde {\mathcal {C}}_{\mathrm {hol}}\times \mathcal {C}_{\mathrm {hol}}$ .

4.2 The affine variety $\tilde {K}_{\mathbb {C}}\times \mathfrak {q}$

Let $\tilde {\kappa }$ be the Killing form on the Lie algebra $\tilde {\mathfrak {g}}$ . We consider the $\tilde {K}$ -invariant symplectic structures $\Omega _{\tilde {\mathfrak {p}}}$ on $\tilde {\mathfrak {p}}$ , defined by the relation

$$ \begin{align*}\Omega_{\tilde{\mathfrak{p}}}(\tilde{Y},\tilde{Y}^{\prime})=\tilde{\kappa}(z,[\tilde{Y},\tilde{Y}^{\prime}]),\quad \forall \tilde{Y},\tilde{Y}^{\prime}\in\tilde{\mathfrak{p}}. \end{align*} $$

We notice that the complex structure $\mathrm {ad}(z)$ is adapted to $\Omega _{\tilde {\mathfrak {p}}}$ : $\Omega _{\tilde {\mathfrak {p}}}(\tilde {Y},\mathrm {ad}(z)\tilde {Y})>0$ if $\tilde {Y}\neq 0$ .

We denote by $\Omega _{\mathfrak {q}}$ the restriction of $\Omega _{\tilde {\mathfrak {p}}}$ on the symplectic subspace $\mathfrak {q}$ . The moment map $\Phi _{\mathfrak {q}}$ associated to the K-action on $(\mathfrak {q}, \Omega _{\mathfrak {q}})$ is defined by the relations $\langle \Phi _{\mathfrak {q}}(Y),X\rangle = \frac {-1}{2}\tilde {\kappa }([X,Y],[z,Y])$ , $\forall (X,Y) \in \mathfrak {p}\times \mathfrak {q}$ . In particular, $\langle \Phi _{\mathfrak {q}}(Y),z\rangle = \frac {-1}{2}\|Y\|^{2}$ , $\forall Y \in \mathfrak {q}$ , so the map $\langle \Phi _{\mathfrak {q}},z\rangle $ is proper.

The complex reductive group $\tilde {K}_{\mathbb {C}}$ is equipped with the following action of $\tilde {K}\times K$ : $(\tilde {k},k)\cdot a= \tilde {k} a k^{-1}$ . It has a canonical structure of a smooth affine variety. There is a diffeomorphism of the cotangent bundle $\textbf { T}^{*}\tilde {K}$ with $\tilde {K}_{\mathbb {C}}$ defined as follows. We identify $\textbf { T}^{*}\tilde {K}$ with $\tilde {K}\times \tilde {\mathfrak {k}}^{*}$ by means of left-translation and then with $\tilde {K}\times \tilde {\mathfrak {k}}$ by means of an invariant inner product on $\tilde {\mathfrak {k}}$ . The map $\varphi :\tilde {K}\times \tilde {\mathfrak {k}}\to \tilde {K}_{\mathbb {C}}$ given by $\varphi (a,X)=a e^{iX}$ is a diffeomorphism. If we use $\varphi $ to transport the complex structure of $\tilde {K}_{\mathbb {C}}$ to $\textbf { T}^{*}\tilde {K}$ , then the resulting complex structure on $\textbf { T}^{*}\tilde {K}$ is compatible with the symplectic structure on $\textbf { T}^{*}\tilde {K}$ so that $\textbf { T}^{*}\tilde {K}$ becomes a Kähler Hamiltonian $\tilde {K}\times K$ -manifold (see [Reference Hall11], §3). The moment map relative to the $\tilde {K}\times K$ -action is the proper map $\Phi _{\tilde {K}} \oplus \Phi _{K} : \textbf { T}^{*}\tilde {K} \to \tilde {\mathfrak {k}}^{*}\oplus \mathfrak {k}^{*}$ defined by $\Phi _{\tilde {K}}(\tilde {a},\tilde {\eta })=-\tilde {a}\tilde {\eta }$ and $\Phi _{K}(\tilde {a},\tilde {\eta })=\pi _{\mathfrak {k},\tilde {\mathfrak {k}}}(\tilde {\eta })$ . Here $\pi _{\mathfrak {k},\tilde {\mathfrak {k}}}: \tilde {\mathfrak {k}}^{*} \to \mathfrak {k}^{*}$ is the projection dual to the inclusion $\mathfrak {k} \hookrightarrow \tilde {\mathfrak {k}}$ of Lie algebras.

Finally, we consider the Kähler Hamiltonian $\tilde {K}\times K$ -manifold $\textbf { T}^{*}\tilde {K}\times \mathfrak {q}$ , where $\mathfrak {q}$ is equipped with the symplectic structure $\Omega _{\mathfrak {q}}$ . Let us denote by $\Phi : \textbf { T}^{*}\tilde {K}\times \mathfrak {q}\to \tilde {\mathfrak {k}}^{*}\oplus \mathfrak {k}^{*}$ the moment map relative to the $\tilde {K}\times K$ -action:

(15) $$ \begin{align} \Phi(\tilde{a},\tilde{\eta},Y)=\left(-\tilde{a}\tilde{\eta},\pi_{\mathfrak{k},\tilde{\mathfrak{k}}}(\tilde{\eta})+\Phi_{\mathfrak{q}}(Y)\right). \end{align} $$

Since $\Phi $ is proper map, the convexity theorem tells us that

$$ \begin{align*}\Delta(\textbf{T}^{*}\tilde{K}\times \mathfrak{q}):=\mathrm{Image}(\Phi)\bigcap \, \tilde{\mathfrak{t}}^{*}_{\geq 0}\times \mathfrak{t}^{*}_{\geq 0} \end{align*} $$

is a closed convex locally polyhedral set.

We consider now the action of $\tilde {K}\times K$ on the affine variety $\tilde {K}_{\mathbb {C}}\times \mathfrak {q}$ . The set of highest weights of $\tilde {K}_{\mathbb {C}}\times \mathfrak {q}$ is the semigroup $\Pi ^{\mathbb {Z}}(\tilde {K}_{\mathbb {C}}\times \mathfrak {q})\subset \tilde {\wedge }^{*}_{+}\times \wedge ^{*}_{+}$ consisting of all dominant weights $(\tilde {\lambda },\lambda )$ such that the irreducible $\tilde {K}\times K$ -representation $V_{\tilde {\lambda }}^{\tilde {K}}\otimes V_{\lambda }^{K}$ occurs in the coordinate ring $\mathbb {C}[\tilde {K}_{\mathbb {C}}\times \mathfrak {q}]$ . We denote by $\Pi ^{\mathbb {Q}}(\tilde {K}_{\mathbb {C}}\times \mathfrak {q})$ the $\mathbb {Q}$ -convex cone generated by the semigroup $\Pi ^{\mathbb {Z}}(\tilde {K}_{\mathbb {C}}\times \mathfrak {q})$ : $(\tilde {\xi },\xi )\in \Pi ^{\mathbb {Q}}(\tilde {K}_{\mathbb {C}}\times \mathfrak {q})$ if and only if $\exists N\geq 1$ , $N(\tilde {\xi },\xi )\in \Pi ^{\mathbb {Z}}(\tilde {K}_{\mathbb {C}}\times \mathfrak {q})$ .

The following important fact is classical (see Theorem 4.9 in [Reference Sjamaar35]).

A direct application of the Peter–Weyl theorem gives the following characterization:

(16) $$ \begin{align} (\tilde{\lambda},\lambda)\in\Pi^{\mathbb{Z}}(\tilde{K}_{\mathbb{C}}\times \mathfrak{q})&\Longleftrightarrow \left[ V^{\tilde{K}}_{\tilde{\lambda}}\vert_{K}\otimes V^{K}_{\lambda}\otimes \mathrm{Sym}(\mathfrak{q}) \right]^{K}\neq 0\\ &\Longleftrightarrow \left[ V^{K}_{\lambda^{*}} : V^{\tilde{K}}_{\tilde{\lambda}}\vert_{K}\otimes \mathrm{Sym}(\mathfrak{q}) \right]\neq 0\nonumber\\ &\Longleftrightarrow (\tilde{\lambda},\lambda^{*})\in \Pi^{\mathbb{Z}}_{\mathfrak{q}}(\tilde{K},K).\nonumber \end{align} $$

4.3 Proof of Theorem B

Consider the semigroup $\Pi ^{\mathbb {Z}}_{\mathfrak {q}}(\tilde {K},K)$ of $\tilde {\wedge }^{*}_{+} \times \wedge ^{*}_{+}$ (see Definition 1.3) and the $\mathbb {Q}$ -convex cone $\Pi ^{\mathbb {Q}}_{\mathfrak {q}}(\tilde {K},K):=\{(\tilde {\xi },\xi )\in \tilde {\mathfrak {t}}^{*}_{\geq 0}\times \mathfrak {t}^{*}_{\geq 0} \; \ \exists N\geq 1, N(\tilde {\xi },\xi )\in \Pi ^{\mathbb {Z}}_{\mathfrak {q}}(\tilde {K},K)\}$ .

The Jakobsen–Vergne theorem says that $\Pi _{\mathrm {hol}}^{\mathbb {Z}}(\tilde {G},G)=\Pi ^{\mathbb {Z}}_{\mathfrak {q}}(\tilde {K},K)\,\bigcap \, \widehat {\tilde {G}}_{\mathrm {hol}}\times \widehat {G}_{\mathrm {hol}}$ . Hence, the convex cone $\Pi _{\mathrm {hol}}^{\mathbb {Q}}(\tilde {G},G)$ is equal to $\Pi ^{\mathbb {Q}}_{\mathfrak {q}}(\tilde {K},K)\cap \tilde {\mathcal {C}}_{\mathrm {hol}}\times \mathcal {C}_{\mathrm {hol}}$ . Thanks to equation (16), we know that $(\tilde {\xi },\xi )\in \Pi ^{\mathbb {Q}}_{\mathfrak {q}}(\tilde {K},K)$ if and only if $(\tilde {\xi },\xi ^{*})\in \Pi ^{\mathbb {Q}}(\tilde {K}_{\mathbb {C}}\times \mathfrak {q})$ . The density results obtained in Proposition 4.4 and Corollary 4.3 gives finally Theorem B.

4.4 Proof of Theorem C

We denote by $\bar {\mathfrak {q}}$ the K-vector space $\mathfrak {q}$ equipped with the opposite symplectic form $-\Omega _{\mathfrak {q}}$ and opposite complex structure $-\mathrm {ad}(z)$ . The moment map relative to the K-action on $\bar {\mathfrak {q}}$ is denoted by $\Phi _{\bar {\mathfrak {q}}}=-\Phi _{\mathfrak {q}}$ .

Lemma 4.5. Any element $(\tilde {\xi },\xi )\in \tilde {\mathfrak {t}}^{*}_{\geq 0}\times \mathfrak {t}^{*}_{\geq 0}$ satisfies the equivalence

$$ \begin{align*}(\tilde{\xi},\xi^{*})\in \Delta(\textbf{T}^{*}\tilde{K}\times \mathfrak{q}) \Longleftrightarrow \xi\in \Delta_{K}(\tilde{K}\tilde{\xi}\times \overline{\mathfrak{q}}). \end{align*} $$

At this stage, we know that $\Delta _{G}(\tilde {G}\tilde {\mu })=\Delta _{K}(\tilde {K}\tilde {\mu }\times \overline {\mathfrak {q}})\cap \mathcal {C}_{\mathrm {hol}}$ . Hence, Theorem C will follow from the next result.

Proof. Let $z^{*}\in \tilde {\mathfrak {t}}^{*}$ such that $\langle z^{*},\tilde {X}\rangle = -\tilde {\kappa }(z,\tilde {X})$ , $\forall \tilde {X}\in \tilde {\mathfrak {g}}$ . Consider the coadjoint orbit $\tilde {\mathcal {O}}=\tilde {G} z^{*}$ equipped with its canonical symplectic structure $\Omega _{\tilde {\mathcal {O}}}$ : The symplectic vector space $\textbf { T}_{z^{*}}\tilde {\mathcal {O}}$ is canonically isomorphic to $(\tilde {\mathfrak {p}},-\Omega _{\tilde {\mathfrak {p}}})$ . In [Reference McDuff26], McDuff proved that $(\tilde {\mathcal {O}},\Omega _{\tilde {\mathcal {O}}})$ is diffeomorphic, as a $\tilde {K}$ -symplectic manifold, to the symplectic vector space $(\tilde {\mathfrak {p}},-\Omega _{\tilde {\mathfrak {p}}})$ (see [Reference Deltour6, Reference Deltour8] for a generalization of this fact). McDuff’s results show in particular that $\mathrm {Image}(-\Phi _{\tilde {\mathfrak {p}}})=\pi _{\tilde {\mathfrak {g}},\tilde {\mathfrak {k}}}(\tilde {\mathcal {O}})$ . Our proof is completed if we check that $\pi _{\tilde {\mathfrak {g}},\tilde {\mathfrak {k}}}(\tilde {\mathcal {O}})\subset \mathcal {C}_{\tilde {G}/\tilde {K}}$ : In other words, if $\langle \pi _{\tilde {\mathfrak {g}},\tilde {\mathfrak {k}}}(\tilde {g}_{0}\, z^{*}),\tilde {g}_{1}z\rangle \geq 0$ , $\forall \tilde {g}_{0},\tilde {g}_{1}\in \tilde {G}$ . But

$$ \begin{align*} 2\langle\pi_{\tilde{\mathfrak{g}},\tilde{\mathfrak{k}}}(\tilde{g}_{0}\, z^{*}),\tilde{g}_{1}\,z\rangle &= \langle\tilde{g}_{0}\, z^{*},\tilde{g}_{1}z+\Theta(\tilde{g}_{1})z\rangle\\ &= -\tilde{\kappa}(z,\tilde{g}_{0}^{-1}\tilde{g}_{1}\, z)-\tilde{\kappa}(z,\tilde{g}_{0}^{-1}\Theta(\tilde{g}_{1})z). \end{align*} $$

With equation (7) in hand, it is not difficult to see that $-\tilde {\kappa }(z,\tilde {g}\, z)\geq 0$ for every $\tilde {g}\in \tilde {G}$ . We thus verified that $\pi _{\tilde {\mathfrak {g}},\tilde {\mathfrak {k}}}(\tilde {\mathcal {O}})\subset \mathcal {C}_{\tilde {G}/\tilde {K}}$ .

5.1 Admissible elements

We choose maximal tori $\tilde {T}\subset \tilde {K}$ and $T\subset K$ such that $T\subset \tilde {T}$ . Let $\mathfrak {R}_{o}$ and $\mathfrak {R}$ be, respectively, the set of roots for the action of T on $(\tilde {\mathfrak {g}}/\mathfrak {g})\otimes \mathbb {C}$ and $\mathfrak {g}\otimes \mathbb {C}$ . Let $\tilde {\mathfrak {R}}$ be the set of roots for the action of $\tilde {T}$ on $\tilde {\mathfrak {g}}\otimes \mathbb {C}$ . Let $\mathfrak {R}^{+}\subset \mathfrak {R}$ and $\tilde {\mathfrak {R}}^{+}\subset \tilde {\mathfrak {R}}$ be the systems of positive roots defined in equation (6). Let $W,\tilde {W}$ be the Weyl groups of $(T,K)$ and $(\tilde {T},\tilde {K})$ . Let $w_{o}\in W$ be the longest element.

We start by introducing the notion of admissible elements. The group $\hom (U(1),T)$ admits a natural identification with the lattice $\wedge :=\frac {1}{2\pi }\ker (\exp :\mathfrak {t}\to T)$ . A vector $\gamma \in \mathfrak {t}$ is called rational if it belongs to the $\mathbb {Q}$ -vector space $\mathfrak {t}_{\mathbb {Q}}$ generated by $\wedge $ .

We consider the $\tilde {K}\times K$ -action on $N:=\textbf { T}^{*} \tilde {K} \times \mathfrak {q}$ . We associate to any subset $\mathcal {X}\subset N$ , the integer $\dim _{\tilde {K}\times K}(\mathcal {X})$ (see equation (5)).

If $\gamma \in \mathfrak {t}$ , we denote by $\mathfrak {R}_{o}\cap \gamma ^{\perp }$ the subsets of weight vanishing against $\gamma $ . We start with the following lemma whose proof is left to the reader (see §6.1.1 of [Reference Paradan32]).

The next result is a direct consequence of the previous lemma.

5.2 Ressayre’s data

We consider an admissible element $(\tilde {w}\gamma ,\gamma )$ . The submanifold of $N\simeq \tilde {K}_{\mathbb {C}} \times \mathfrak {q}$ fixed by $(\tilde {w}\gamma ,\gamma )$ is $N^{(\tilde {w}\gamma ,\gamma )}= \tilde {w} \tilde {K}_{\mathbb {C}}^{\gamma } \times \mathfrak {q}^{\gamma }$ . There is a canonical isomorphism between the manifold $N^{(\tilde {w}\gamma ,\gamma )}$ equipped with the action of $\tilde {w}\tilde {K}^{\gamma } \tilde {w}^{-1}\times K^{\gamma }$ with the manifold $\tilde {K}_{\mathbb {C}}^{\gamma } \times \mathfrak {q}^{\gamma }$ equipped with the action of $\tilde {K}^{\gamma } \times K^{\gamma }$ . The tangent bundle $(\textbf { T} N\vert _{N^{(\tilde {w}\gamma ,\gamma )}})^{(\tilde {w}\gamma ,\gamma )>0}$ is isomorphic to $N^{\gamma _{w}}\times \tilde {\mathfrak {k}}_{\mathbb {C}}^{\gamma>0}\times \mathfrak {q}^{\gamma >0}$ .

The choice of positive roots $\mathfrak {R}^{+}$ (resp. $\tilde {\mathfrak {R}}^{+}$ ) induces a decomposition $\mathfrak {k}_{\mathbb {C}}=\mathfrak {n}\oplus \mathfrak {t}_{\mathbb {C}}\oplus \overline {\mathfrak {n}}$ (resp. $\tilde {\mathfrak {k}}_{\mathbb {C}}=\tilde {\mathfrak {n}}\oplus \tilde {\mathfrak {t}}_{\mathbb {C}}\oplus \overline {\tilde {\mathfrak {n}}}$ ), where $\mathfrak {n}=\sum _{\alpha \in \mathfrak {R}^{+}}(\mathfrak {k}\otimes \mathbb {C})_{\alpha }$ (resp. $\tilde {\mathfrak {n}}=\sum _{\tilde {\alpha }\in \tilde {\mathfrak {R}}^{+}}(\tilde {\mathfrak {k}}\otimes \mathbb {C})_{\tilde {\alpha }}$ ). We consider the map

$$ \begin{align*}\rho^{\tilde{w},\gamma}: \tilde{K}_{\mathbb{C}}^{\gamma} \times \mathfrak{q}^{\gamma} \longrightarrow \hom \left(\tilde{\mathfrak{n}}^{\tilde{w}\gamma>0}\times \mathfrak{n}^{\gamma>0}\,,\, \tilde{\mathfrak{k}}_{\mathbb{C}}^{\gamma>0}\times\mathfrak{q}^{\gamma>0}\right) \end{align*} $$

defined by the relation

$$ \begin{align*}\rho^{\tilde{w},\gamma}(\tilde{x},v): (\tilde{X},X)\longmapsto ((\tilde{w} \tilde{x})^{-1}\tilde{X}-X\, ;\, X\cdot v) \end{align*} $$

for any $(\tilde {x},v)\in \tilde {K}_{\mathbb {C}}^{\gamma } \times \mathfrak {q}^{\gamma }$ .

Since the linear map $\rho ^{\tilde {w},\gamma }(\tilde {x},v)$ commutes with the $\gamma $ -actions, we obtain the following necessary conditions.

Lemma 5.8. Relation (B) is equivalent to

(17) $$ \begin{align} \sum_{\stackrel{\alpha\in\mathfrak{R}^{+}}{\langle\alpha,\gamma\rangle> 0}}\langle\alpha,\gamma\rangle= \sum_{\stackrel{\tilde{\alpha}\in\tilde{\mathfrak{R}}^{+}}{\langle\tilde{\alpha},\tilde{w}_{0}\tilde{w}\gamma\rangle> 0}}\langle\tilde{\alpha},\tilde{w}_{0}\tilde{w}\gamma\rangle. \end{align} $$

Proof. First, one sees that $\operatorname {Tr}_{\gamma }(\mathfrak {q}^{\gamma>0})=\operatorname {Tr}_{\gamma }(\tilde {\mathfrak {p}}^{\gamma >0})-\operatorname {Tr}_{\gamma }(\mathfrak {p}^{\gamma >0})= \sum _{\stackrel {\tilde {\alpha }\in \tilde {\mathfrak {R}}^{+}_{n}}{\langle \tilde {\alpha },\gamma \rangle >0}}\langle \tilde {\alpha },\gamma \rangle -\sum _{\stackrel {\alpha \in \mathfrak {R}^{+}_{n}}{\langle \alpha ,\gamma \rangle >0}}\langle \alpha ,\gamma \rangle $ , and $\operatorname {Tr}_{\gamma }(\tilde {\mathfrak {k}}_{\mathbb {C}}^{\gamma>0})=\operatorname {Tr}_{\tilde {w}\gamma }(\tilde {\mathfrak {k}}_{\mathbb {C}}^{\tilde {w}\gamma >0})=\operatorname {Tr}_{\tilde {w}\gamma }(\tilde {\mathfrak {n}}^{\tilde {w}\gamma >0})+ \sum _{\stackrel {\tilde {\alpha }\in \tilde {\mathfrak {R}}^{+}_{c}}{\langle \tilde {\alpha },\tilde {w}_{0}\tilde {w}\gamma \rangle >0}}\langle \tilde {\alpha },\tilde {w}_{0}\tilde {w}\gamma \rangle $ . Relation $(B)$ is equivalent to

(18) $$ \begin{align} \operatorname{Tr}_{\gamma}(\mathfrak{n}^{\gamma>0})+\sum_{\stackrel{\alpha\in \mathfrak{R}^{+}_{n}}{\langle\alpha,\gamma\rangle>0}}\langle\alpha,\gamma\rangle= \sum_{\stackrel{\tilde{\alpha}\in \tilde{\mathfrak{R}}^{+}_{n}}{\langle\tilde{\alpha},\gamma\rangle>0}}\langle\tilde{\alpha},\gamma\rangle +\sum_{\stackrel{\tilde{\alpha}\in \tilde{\mathfrak{R}}^{+}_{c}}{\langle\tilde{\alpha},\tilde{w}_{0}\tilde{w}\gamma\rangle>0}}\langle\tilde{\alpha},\tilde{w}_{0}\tilde{w}\gamma\rangle. \end{align} $$

Since $\tilde {\mathfrak {R}}^{+}_{n}$ is invariant under the action of the Weyl group $\tilde {W}$ , the right-hand side of equation (18) is equal to $\sum _{\stackrel {\tilde {\alpha }\in \tilde {\mathfrak {R}}^{+}}{\langle \tilde {\alpha },\tilde {w}_{0}\tilde {w}\gamma \rangle>0}}\langle \tilde {\alpha },\tilde {w}_{0}\tilde {w}\gamma \rangle $ . Since the left-hand side of equation (18) is equal to $\sum _{\stackrel {\alpha \in \mathfrak {R}^{+}}{\langle \alpha ,\gamma \rangle>0}}\langle \alpha ,\gamma \rangle $ , the proof of the lemma is complete.

5.3 Cohomological characterization of Ressayre’s data

Let $\gamma \in \mathfrak {t}$ be a nonzero rational element. We denote by $B\subset K_{\mathbb {C}}$ and by $\tilde {B}\subset \tilde {K}_{\mathbb {C}}$ the Borel subgroups with Lie algebra $\mathfrak {b}=\mathfrak {t}_{\mathbb {C}}\oplus \mathfrak {n}$ and $\tilde {\mathfrak {b}}=\tilde {\mathfrak {t}}_{\mathbb {C}}\oplus \tilde {\mathfrak {n}}$ . Consider the parabolic subgroup $P_{\gamma }\subset K_{\mathbb {C}}$ defined by

(19) $$ \begin{align} P_{\gamma}=\{g\in K_{\mathbb{C}}, \lim_{t\to\infty}\exp(-it\gamma)g\exp(it\gamma)\ \mathrm{exists}\}. \end{align} $$

Similarly, one defines a parabolic subgroup $\tilde {P}_{\gamma }\subset \tilde {K}_{\mathbb {C}}$ .

We work with the projective varieties $\mathcal {F}_{\gamma }:=K_{\mathbb {C}}/P_{\gamma }$ , $\tilde {\mathcal {F}}_{\gamma }:=\tilde {K}_{\mathbb {C}}/\tilde {P}_{\gamma }$ and the canonical embedding $\iota : \mathcal {F}_{\gamma }\to \tilde {\mathcal {F}}_{\gamma }$ . We associate to any $\tilde {w}\in \tilde {W}$ , the Schubert cell

$$ \begin{align*}\tilde{\mathfrak{X}}^{o}_{\tilde{w},\gamma}:= \tilde{B}[\tilde{w}]\subset \tilde{\mathcal{F}}_{\gamma} \end{align*} $$

and the Schubert variety $\tilde {\mathfrak {X}}_{\tilde {w},\gamma }:=\overline {\tilde {\mathfrak {X}}^{o}_{\tilde {w},\gamma }}$ . If $\tilde {W}^{\gamma }$ denotes the subgroup of $\tilde {W}$ that fixes $\gamma $ , we see that the Schubert cell $\tilde {\mathfrak {X}}^{o}_{\tilde {w},\gamma }$ and the Schubert variety $\tilde {\mathfrak {X}}_{\tilde {w},\gamma }$ depend only of the class of $\tilde {w}$ in $\tilde {W}/\tilde {W}^{\gamma }$ .

On the variety $\mathcal {F}_{\gamma }$ , we consider the Schubert cell $\mathfrak {X}^{o}_{\gamma }:= B[e]$ and the Schubert variety $\mathfrak {X}_{\gamma }:=\overline {\mathfrak {X}^{o}_{\gamma }}$ .

We consider the cohomologyFootnote ¹ ring $H^{*}(\tilde {\mathcal {F}}_{\gamma },\mathbb {Z})$ of $\tilde {\mathcal {F}}_{\gamma }$ . If Y is an irreducible closed subvariety of $\tilde {\mathcal {F}}_{\gamma }$ , we denote by $[Y]\in H^{2n_{Y}}(\tilde {\mathcal {F}}_{\gamma },\mathbb {Z})$ its cycle class in cohomology: Here $n_{Y}=\mathrm {codim}_{\mathbb {C}}(Y)$ . Let $\iota ^{*}:H^{*}(\tilde {\mathcal {F}}_{\gamma },\mathbb {Z})\to H^{*}(\mathcal {F}_{\gamma },\mathbb {Z})$ be the pull-back map in cohomology. Recall that the cohomology class $[pt]$ associated to a singleton $Y=\{pt\}\subset \mathcal {F}_{\gamma }$ is a basis of $H^{\mathrm {max}}(\mathcal {F}_{\gamma },\mathbb {Z})$ .

To an oriented real vector bundle $\mathcal {E}\to N$ of rank r, we can associate its Euler class $\mathrm {Eul}(\mathcal {E})\in H^{2r}(N,\mathbb {Z})$ . When $\mathcal {V}\to N$ is a complex vector bundle, then $\mathrm {Eul}(\mathcal {V}_{\mathbb {R}})$ corresponds to the top Chern class $c_{p}(\mathcal {V})$ , where p is the complex rank of $\mathcal {V}$ , and $\mathcal {V}_{\mathbb {R}}$ means $\mathcal {V}$ viewed as a real vector bundle oriented by its complex structure (see [Reference Bott and Tu5], §21).

The isomorphism $\mathfrak {q}^{\gamma>0}\simeq \mathfrak {q}/ \mathfrak {q}^{\gamma \leq 0}$ shows that $\mathfrak {q}^{\gamma>0}$ can be viewed as a $P_{\gamma }$ -module. Let $[\mathfrak {q}^{\gamma>0}]= K_{\mathbb {C}}\times _{P_{\gamma }}\mathfrak {q}^{\gamma >0}$ be the corresponding complex vector bundle on $\mathcal {F}_{\gamma }$ . We denote simply by $\mathrm {Eul}(\mathfrak {q}^{\gamma>0})$ the Euler class $\mathrm {Eul}([\mathfrak {q}^{\gamma>0}]_{\mathbb {R}})\in H^{*}(\mathcal {F}_{\gamma },\mathbb {Z})$ .

The following characterization of Ressayre’s data was obtained in [Reference Paradan32], §6. Recall that $\mathfrak {R}_{o}$ denotes the set of weights relative to the T-action on $(\tilde {\mathfrak {g}}/\mathfrak {g})\otimes \mathbb {C}$ .

5.4 Parametrization of the facets

We can finally describe the Kirwan polyhedron $\Delta (\textbf{T}^{*}\tilde {K}\times \mathfrak {q})$ (see [Reference Paradan32], § 6).

Theorem 5.10. An element $(\tilde {\xi },\xi )\in \tilde {\mathfrak {t}}_{\geq 0}^{*}\times \mathfrak {t}_{\geq 0}^{*}$ belongs to $\Delta (\textbf{T}^{*}\tilde {K}\times \mathfrak {q})$ if and only if

$$ \begin{align*}\langle \tilde{\xi},\tilde{w}\gamma\rangle+\langle \xi,\gamma\rangle\geq 0 \end{align*} $$

for any Ressayre’s datum $(\gamma ,\tilde {w})\in \mathfrak {t}\times \tilde {W}$ .

Theorem 5.10 and Theorem B permit us to give the following description of the convex cone $\Delta _{\mathrm {hol}}(\tilde {G},G)$ .

Theorem 5.11. An element $(\tilde {\xi },\xi )$ belongs to $\Delta _{\mathrm {hol}}(\tilde {G},G)$ if and only if $(\tilde {\xi },\xi )\in \tilde {\mathcal {C}}_{\mathrm {hol}}\times \mathcal {C}_{\mathrm {hol}}$ and

$$ \begin{align*}\langle \tilde{\xi},\tilde{w}\gamma\rangle\geq \langle \xi,w_{0}\gamma\rangle \end{align*} $$

for any $(\gamma ,\tilde {w})\in \mathfrak {t}\times \tilde {W}$ satisfying the following conditions:

• • $\gamma $ is nonzero and rational.

• • $\mathrm {Vect}(\mathfrak {R}_{o}\cap \gamma ^{\perp })=\mathrm {Vect}(\mathfrak {R}_{o})\cap \gamma ^{\perp }$ .

• • $[\mathfrak {X}_{\gamma }]\cdot \iota ^{*}([\tilde {\mathfrak {X}}_{\tilde {w},\gamma }])\cdot \mathrm {Eul}(\mathfrak {q}^{\gamma>0})= k[pt],\ k\geq 1$ in $H^{*}(\mathcal {F}_{\gamma },\mathbb {Z})$ .

• • $\sum _{\stackrel {\alpha \in \mathfrak {R}^{+}}{\langle \alpha ,\gamma \rangle> 0}}\langle \alpha ,\gamma \rangle = \sum _{\stackrel {\tilde {\alpha }\in \tilde {\mathfrak {R}}^{+}}{\langle \tilde {\alpha },\tilde {w}_{0}\tilde {w}\gamma \rangle > 0}}\langle \tilde {\alpha },\tilde {w}_{0}\tilde {w}\gamma \rangle $ .