1.1.1 Involutions

In our setting we may view $G({\mathbb {R}})$ as the fixed-point group $G^\sigma $ , where $\sigma $ is an anti-holomorphic involution $\sigma $ of G. Fix such an involution $\sigma $ , and choose a compatible Cartan involution $\theta $ for G. This means: $\theta $ is an algebraic involution, commuting with $\sigma $ , and $G({\mathbb {R}})^\theta $ is a maximal compact subgroup of $G({\mathbb {R}})$ . Set $K=G^\theta $ and $K({\mathbb {R}})=G({\mathbb {R}})^\theta =K^\sigma $ . Set $\mathfrak {g}=\mathrm {Lie}(G)$ , and let $\mathfrak {s}$ be the $(-1)$ -eigenspace of $\theta $ acting on $\mathfrak {g}$ , where we use the notation $\sigma , \theta $ also for the automorphisms of $\mathfrak {g}$ induced by the above involutions. Set $\mathfrak {g}({\mathbb {R}})=\mathrm {Lie}(G({\mathbb {R}}))$ .

1.1.2 Nilpotent orbits in $\mathfrak {g}^*$

Let $\mathcal {N}$ be the set of nilpotent elements of $\mathfrak {g}$ . Several of the invariants below are related to $\mathcal {N}$ , but are more naturally defined as subsets of the dual vector space $\mathfrak {g}^\ast $ . Therefore we need to pass to $\mathfrak {g}^*$ , and it is important to do this as independently of a choice of isomorphism $\mathfrak {g}\simeq \mathfrak {g}^*$ as possible.

First we define the nilpotent cone in $\mathfrak {g}^*$ as follows. We say an element X of $\mathfrak {g}^*$ is nilpotent if its G-orbit is a (weak) cone, i.e. closed under multiplication by $\mathbb C^*$ , and we say X is regular if $\dim _{\mathrm {Cent}(G)}(X)=\mathrm {rank}(G)$ . Let $\mathcal {N}^*$ be the set of nilpotent elements. The principal nilpotent orbit ${\mathcal {O}}_p\in \mathcal {N}^*$ is the set of regular nilpotent elements, and $\overline {{\mathcal {O}}_p}=\mathcal {N}^*$ .

The statements of the main results below do not require the choice of an isomorphism $\mathfrak {g}\simeq \mathfrak {g}^*$ . However such a choice appears in the proofs, so we spell this out. Having a $G({\mathbb {R}})$ -invariant bilinear form on $\mathfrak {g}({\mathbb {R}})$ is equivalent to having a $G({\mathbb {R}})$ -equivariant isomorphism $\mathfrak {g}({\mathbb {R}})\simeq \mathfrak {g}^*({\mathbb {R}})$ . Alternatively, this is equivalent to having a non-degenerate bilinear form on $\mathfrak {g}$ which is both $G(\mathbb C)$ - and Galois-invariant. The restriction of the form to each simple factor is unique up to a nonzero constant. For example we can use the Killing form on $\mathfrak {g}_{\mathrm {der}}=[\mathfrak {g}, \mathfrak {g}]$ , extended arbitrarily by a nondegenerate form on the center.

Given such a nondegenerate bilinear form $\kappa $ , we define $\psi _\kappa : \mathfrak {g}\rightarrow \mathfrak {g}^*$ as usual:

$$ \begin{align*}\psi_\kappa(X)(Y)=\kappa(X,Y)\quad (X,Y\in\mathfrak{g}). \end{align*} $$

Using $\psi _\kappa $ we shall transfer to $\mathfrak {g}^\ast $ several notions usually defined on $\mathfrak {g}$ : the asymptotic cone, the Kostant section, and the Kostant–Sekiguchi correspondence. In each case the resulting notions are independent of the choice of $\kappa $ . See Sections 3.3 and 3.5. In Section 3.1 we need to be more precise: we choose $\kappa $ to be the Killing form (or any nondegenerate invariant form which is negative definite on the Lie algebra of a maximal compact subgroup).

1.1.3 Representations

We need to pass back and forth between representations of $G({\mathbb {R}})$ and $(\mathfrak {g},K)$ -modules. See [Reference Vogan22, Chapter 0] for details.

Suppose $\pi $ is an irreducible admissible representation of $G({\mathbb {R}})$ on a Hilbert space. The underlying $(\mathfrak {g},K)$ -module is on the space of K-finite vectors in V; we denote it by $\pi _K$ . Conversely, if $\pi $ is an irreducible $(\mathfrak {g},K)$ -module, it can be realized as the K-finite vectors of a Hilbert space representation, which we refer to as the Hilbert space globalization of $\pi $ .

The notion of equivalence of $(\mathfrak {g}, K)$ -modules is defined in such a way that two equivalent $(\mathfrak {g}, K)$ -modules have equivalent Hilbert space globalizations, and that any two equivalent admissible representations of G have equivalent underlying $(\mathfrak {g}, K)$ -modules. Therefore if $\pi $ refers to an equivalence class of irreducible Hilbert space representations of $G({\mathbb {R}})$ (as opposed to an actual representation in the equivalence class), then it makes sense to write $\pi _K$ for the corresponding equivalence class of $(\mathfrak {g}, K)$ -modules.

1.2.1 Associated variety and Gelfand–Kirillov dimension

Suppose $\pi $ is an irreducible $(\mathfrak {g},K)$ -module. The associated variety of $\pi $ , denoted $\mathrm {AV}(\pi )$ , is a closed, K-invariant set in $\mathfrak {s}^*$ ; so it is a finite union of nilpotent K-orbits on $\mathfrak {s}^*$ . For the definition, and many properties, see [Reference Vogan23, Section 5].

A related invariant is the associated variety of the annihilator $I_\pi $ of $\pi $ in the universal enveloping algebra, denoted $\mathrm {AV}_{\mathrm {ann}}(\pi )$ . This is a subset of $\mathcal {N}^* \subset \mathfrak {g}^*$ , and is the closure of a single complex nilpotent orbit: see [Reference Vogan23, Section 1, Theorem 4.7]. Both $\mathrm {AV}(\pi )$ and $\mathrm {AV}_{\mathrm {ann}}(\pi )$ depend only on the equivalence class of $\pi $ .

The Gelfand–Kirillov dimension $\mathrm {GK}(\pi )$ of $\pi $ can be defined in several different ways [Reference Vogan21]; one of them is

$$ \begin{align*}GK(\pi)=\dim(\mathrm{AV}(\pi))=\frac12\dim(\mathrm{AV}_{\mathrm{ann}}(\pi)). \end{align*} $$

All dimensions are complex unless otherwise stated.

We define the associated variety of a Hilbert space representation to be that of its underlying $(\mathfrak {g},K)$ -module.

1.2.2 Wavefront set

Now suppose $\pi $ is an irreducible admissible representation of $G({\mathbb {R}})$ on a Hilbert space $\mathcal H$ . The wave-front set $\mathrm {WF}(\pi )$ is an analytic invariant determined by the singularity of the global character of $\pi $ near the identity. See [Reference Howe10, Reference Barbasch and Vogan7, Reference Harris, He and Ólafsson9] for the definition. The set $\mathrm {WF}(\pi )$ is a closed $G({\mathbb {R}})$ -invariant subset of the nilpotent cone in $i\mathfrak {g}({\mathbb {R}})^*$ . As such, it is the union of a finite number of nilpotent $G({\mathbb {R}})$ -orbits. Again, this depends only on the equivalence class of $\pi $ .

We define the wave front set of a $(\mathfrak {g},K)$ -module to be that of its Hilbert space globalization.

1.2.3 Whittaker models

Suppose again that $\pi $ is an irreducible admissible representation of $G({\mathbb {R}})$ on a Hilbert space $\mathcal H$ . We consider the corresponding actions of $G({\mathbb {R}})$ on the space $\mathcal H^\infty $ of smooth vectors in $\mathcal H$ , and on its dual $\mathcal H^{-\infty }$ . Let $(B, \eta )$ be a pair satisfying conditions (i)-(ii) of the Introduction; we shall say $(B, \eta )$ is a Whittaker pair. We say $\pi $ has a $(B,\eta )$ -Whittaker model if there exists a nonzero element v in $\mathcal H^{-\infty }$ such that $\pi (n)(v)=\eta (n)v$ for all n in the nilradical $N({\mathbb {R}})$ of $B({\mathbb {R}})$ . See [Reference Matumoto12]. This condition depends only on the $G({\mathbb {R}})$ -conjugacy class of the pair $(B, \eta )$ . It is equivalent to the existence of a G-equivariant embedding of $\mathcal H^\infty $ into the space of smooth sections of the equivariant line bundle over $G({\mathbb {R}})/N({\mathbb {R}})$ determined by $\eta $ (for the quasi-regular action of G on sections); the image of that embedding is a geometric realization of (the smooth vectors of) $\pi $ . This realization is the origin of the terminology “Whittaker model for $\pi $ ”.

If $\mathfrak {w}$ is the Whittaker datum corresponding to $(B, \eta )$ , i.e. the $G({\mathbb {R}})$ -conjugacy class of the pair $(B, \eta )$ , we say $\pi $ has a $\mathfrak {w}$ -Whittaker model.

We write $\mathrm {Wh}(\pi )$ for the set of Whittaker data $\mathfrak {w}$ such that $\pi $ has a $\mathfrak {w}$ -Whittaker model. If $\pi $ has a unique Whittaker datum $\mathfrak {w}$ we write $\mathrm {Wh}(\pi )=\mathfrak {w}$ .

We define the Whittaker data of a $(\mathfrak {g},K)$ -module to be that of its Hilbert space globalization.

1.4.1 Discrete series representations

Recall $G({\mathbb {R}})$ has discrete series representations if and only if it has a compact Cartan subgroup, i.e. there exists a $\theta $ -stable Cartan subgroup T of G such that $T({\mathbb {R}}) \cap G$ is compact. Throughout the paper, we assume $G({\mathbb {R}})$ has discrete series representations, we fix a Cartan subgroup $T \subset G$ satisfying the conditions above, and we let $\mathfrak {t}$ denote the Lie algebra of T, which is a Cartan subalgebra of $\mathfrak {g}$ .

Choose a positive system $\Delta ^+$ for the roots of ${\mathfrak {t}}$ in $\mathfrak {g}$ , and let $\rho \in \mathfrak {t}^\ast $ be the half-sum of positive roots. Let $\Lambda \subset \mathfrak {t}^\ast $ be the set of elements $\lambda \in \mathfrak {t}^\ast $ which are regular (i.e. $\lambda $ does not vanish on any coroot) and such that $\lambda -\rho $ is the differential of a character of $T({\mathbb {R}})$ . The set $\Lambda $ does not depend on the choice of $\Delta ^+$ , and is preserved by the action of the Weyl group $W=W(\mathfrak {g}, {\mathfrak {t}})$ on ${\mathfrak {t}}^\ast $ .

Each discrete series representation of $G({\mathbb {R}})$ has an infinitesimal character or Harish-Chandra parameter, which is an element of $\Lambda /W$ . More precisely, suppose $\lambda \in \Lambda $ and $\zeta $ is a character of the center $Z({\mathbb {R}})$ of $G({\mathbb {R}})$ whose differential agrees with $\lambda -\rho $ on the intersection $T({\mathbb {R}}) \cap Z({\mathbb {R}})$ . Then the pair $(\lambda , \zeta )$ determines, uniquely up to unitary equivalence, a discrete series representation $\pi _{\lambda , \zeta }$ of $G({\mathbb {R}})$ : see [Reference Adams and Vogan5, Section 4]. Every discrete series representation arises in this way. The infinitesimal character of $\pi _{\lambda , \zeta }$ is defined to be the image of $\lambda $ in $\mathfrak {t}^\ast /W$ , and the central character is $\zeta $ . See [Reference Adams and Vogan4, Section 8] and [Reference Adams and Vogan5] for more on the parametrization.

There is a simple criterion for genericity in terms of this parametrization. Let us say a set of positive roots $\Delta ^+$ , or the corresponding Weyl chamber, is large if every simple root is non-compact. For $\lambda \in \Lambda $ , let $\Delta ^+(\lambda )$ be the set of roots $\alpha $ for which the coroot $\alpha ^{\vee }$ satisfies $\langle \alpha ^{\vee },\lambda \rangle>0$ . Suppose $\pi \simeq \pi _{\lambda ,\zeta }$ is a discrete series representation; then $\pi $ is large if and only if $\Delta ^+(\lambda )$ is large. See [Reference Vogan21, Section 6]. In the terminology of Shelstad [Reference Shelstad20, Section 2.3], if B is the corresponding Borel subgroup of G corresponding to $\Delta ^+(\lambda )$ , then $(B, T)$ is a fundamental Borel pair of Whittaker type.

1.4.2 Discrete series L-packets

By definition an L-packet of discrete series representations for $G({\mathbb {R}})$ is precisely the set of discrete series with a given infinitesimal character and a given central character. Each L-packet is a finite set, and each discrete series representation belongs to a single L-packet.

The group W has an action on the set of equivalence classes of discrete series representations, given by $w\cdot \pi _{\lambda ,\zeta }=\pi _{w\lambda ,\zeta }$ . This action preserves, and is transitive on, each L-packet $\Pi $ . Furthermore the stabilizer of any $\pi \in \Pi $ is the normal subgroup $W_K=\mathrm {Norm}_K(T)/T$ of W. For all this see [Reference Adams and Vogan5, Section 4].

To discuss the generic elements in a discrete series L-packet let us introduce the adjoint group $G_{\mathrm {ad}}=G/Z(G)$ , and set $K_{\mathrm {ad}}=(G_{\mathrm {ad}})^\theta $ . The Weyl groups of G and $G_{\mathrm {ad}}$ are canonically isomorphic, so we can identify $W_{K_{\mathrm {ad}}}$ with a subgroup of $W=W(G,T)$ . It is immediate from the definition that for $g\in \mathrm {Norm}_G(T)$ , the image of g in W is in $W_{K_{\mathrm {ad}}}$ if and only if ${\mathrm {int}}(g)$ (conjugation by g) commutes with $\theta $ , or equivalently $g{^{-1}} \theta (g)\in Z(G)$ .

2.2.1 Definition of the two groups, and a canonical isomorphism

Let $Z=Z(G)$ be the center of G, and let $G_{\mathrm {ad}}$ be the complex reductive group $G/Z$ . Then $Z({\mathbb {R}})=Z^\sigma $ is the center of $G({\mathbb {R}})$ . We set $G({\mathbb {R}})_{\mathrm {ad}}=G({\mathbb {R}})/Z({\mathbb {R}})$ . On the other hand $\sigma $ factors to an automorphism of $G_{\mathrm {ad}}$ , still denoted $\sigma $ , and we let $G_{\mathrm {ad}}({\mathbb {R}})=(G_{\mathrm {ad}})^\sigma $ . The projection $G\rightarrow G_{\mathrm {ad}}$ restricts to a map $G^\sigma \rightarrow (G/Z)^\sigma $ , with kernel $Z^\sigma $ . This induces a canonical injection $(G^\sigma )_{\mathrm {ad}}\hookrightarrow (G_{\mathrm {ad}})^\sigma $ , and we define

$$ \begin{align*}Q_\sigma(G)=(G_{\mathrm{ad}})^\sigma/(G^\sigma)_{\mathrm{ad}}=G_{\mathrm{ad}}({\mathbb{R}})/G({\mathbb{R}})_{\mathrm{ad}}. \end{align*} $$

This finite group may be viewed as a group of outer automorphisms of $G({\mathbb {R}})$ .

A similar discussion applies to the involution $\theta $ . Then $Z^\theta $ is the center of $K=G^\theta $ and $\theta $ factors to $G_{\mathrm {ad}}$ . We define $(G^\theta )_{\mathrm {ad}}=G^\theta /Z^\theta $ and

$$ \begin{align*}Q_\theta(G)=(G_{\mathrm{ad}})^\theta/(G^\theta)_{\mathrm{ad}}. \end{align*} $$

This is a group of outer automorphisms of K.

An important fact is that $Q_\sigma (G)$ and $Q_\theta (G)$ are canonically isomorphic. This is a generalization of the well known fact that $G({\mathbb {R}})/G({\mathbb {R}})_0\simeq K/K_0$ where the subscript $0$ denotes identity component.

We shall prove this in Section 2.2.3, using an interpretation of $Q_\sigma (G)$ and $Q_\theta (G)$ in terms of group cohomology which we spell out in the next section. Before that, let us point out that the groups $Q_\sigma (G)$ , $Q_\theta (G)$ are isomorphic with the group $W_{K_{\mathrm {ad}}}/W_K$ of Section 1.4.

By definition every element w in $W_{K_{\mathrm {ad}}}$ has a representative g in $(G_{\mathrm {ad}})^\theta $ ; the image of g in $Q_\theta (G)=(G_{\mathrm {ad}})^\theta /(G^\theta )_{\mathrm {ad}}$ depends only on w. This induces a group homomorphism $\psi \colon W_{K_{\mathrm {ad}}} \to Q_\theta (G)$ .

Because of Lemmas 2.5 and 2.6, when describing the actions in Theorem 2.1, we shall realize $Q(G)$ as $W_{K_{\mathrm {ad}}}/W_K$ , as $Q_\sigma (G)$ or as $Q_\theta (G)$ , depending on the situation.

2.2.2 Connection with group cohomology

The groups $Q_{\sigma }(G)$ and $Q_{\theta }(G)$ are instances of the following construction: if $\tau $ is an involutive automorphism of G, then it preserves Z and induces an involution of $G_{\mathrm {ad}}$ , still denoted $\tau $ ; we may then set $Q_{\tau }(G)=(G_{\mathrm {ad}})^{\tau }/(G^{\tau })_{\mathrm {ad}}$ , where $(G^{\tau })_{\mathrm {ad}}$ is the image of $G^{\tau }$ in $G_{\mathrm {ad}}$ . In this situation $Q_{\tau }(G)$ has a simple interpretation in Galois cohomology, which we now recall.

Let us first set up some general notation. When A is a group and $\tau $ is an involutive automorphism of A, consider the cohomology sets $H^0(\tau , A)$ and $H^1(\tau , A)$ attached to the action of $\mathbb Z/2\mathbb Z$ on A by $\tau $ . We shall view $H^0(\tau , A)$ as the fixed-point-set $A^{\tau }$ , and $H^1(\tau ,A)$ as the quotient of $A^{-\tau } = \{ a \in A, \tau (a)=a^{-1}\}$ by the equivalence relation $a \sim x a \tau (x^{-1})$ for all $x \in A$ . The set $H^1(\tau , A)$ has a distinguished point $1$ , coming from the identity of A; but in general it is just a pointed set, and not a group if A is nonabelian.

We now take $A=G$ , and continue to assume $\tau $ is an involutive automorphism. Then the short exact sequence $1\rightarrow Z \rightarrow G \rightarrow G_{\mathrm {ad}}\rightarrow 1$ gives rise to a long exact sequence of pointed sets:

(2.1) $$ \begin{align} 1\rightarrow Z^\tau \rightarrow G^\tau \rightarrow G_{\mathrm{ad}}^{\tau} \overset{\psi_\tau}{\longrightarrow} H^1(\tau,Z)\to H^1(\tau,G)\rightarrow H^1(\tau,G_{\mathrm{ad}}). \end{align} $$

The connecting map $ \psi _\tau \colon G_{\mathrm {ad}}^{\tau } \rightarrow H^1(\tau ,Z)$ is defined as follows: if $\gamma \in G_{\mathrm {ad}}^{\tau }$ is the coset $gZ$ , then $g\tau (g^{-1}) \in Z^{-\tau }$ , and $\psi _\tau $ sends $\gamma $ to the class of $g \tau (g^{-1})$ in $H^1(\tau , Z)$ . The kernel of $\psi _\tau $ is $(G^\tau )_{\mathrm {ad}}$ . Since (2.1) is exact we deduce that $\psi _\tau $ induces a canonical bijection

(2.2) $$ \begin{align} Q_{\tau}(G) \overset{\sim}{\longrightarrow} \ker\left(H^1(\tau,Z)\overset{\varphi_\tau}{\longrightarrow} H^1(\tau,G)\right)\end{align} $$

where $\varphi _\tau $ is induced by the inclusion $Z^{-\tau } \hookrightarrow G^{-\tau }$ .

2.2.3 Proof of Lemma 2.5

The proof of the Lemma uses Galois cohomology for the compact real form. Recall $\theta \sigma = \sigma \theta $ , and $G^{\sigma \theta }$ is a compact real form of G. The involutions $\sigma $ and $\theta $ coincide on $G^{\sigma \theta }$ , therefore $H^1(\sigma , G^{\sigma \theta })$ and $H^1(\theta , G^{\sigma \theta })$ are the same set, which we will denote by $H(\ast , G^{\sigma \theta })$ . Now the inclusion $G^{\sigma \theta } \hookrightarrow G$ induces canonical maps

(2.3) $$ \begin{align} H^{1}(\sigma, G) \longleftarrow H(\ast, G^{\sigma\theta}) \longrightarrow H^{1}(\theta, G).\end{align} $$

By [Reference Adams and Taïbi3, Corollary 4.4 and Corollary 4.7], both of these maps are bijections. Replacing G by Z we get bijections $H^{1}(\sigma , Z) \leftarrow H(\ast , Z^{\sigma \theta }) \rightarrow H^{1}_{\theta }(\Gamma , Z)$ . These fit with the bijections (2.3) into a commutative diagram

where $\varphi _\sigma $ , $\varphi _\theta $ are the maps in (2.2).

Let us now prove part (a) of Lemma 2.5. Let $\gamma $ be an element of $(G_{\mathrm {ad}})^\sigma $ ; what we need to show is that there exists $\gamma _0 \in (G^\sigma )_{\mathrm {ad}}$ such that $\gamma \gamma _0^{-1}$ is $\theta $ -invariant.

Let g be an element of G such that $\gamma = gZ$ ; then $z=g \sigma (g^{-1})$ is an element of $Z^{-\sigma }$ . Let $\zeta $ be its class in $H^{1}(\theta , Z)$ . Let $\zeta _c$ be the inverse image of $\zeta $ under the bijection $H(\ast , Z^{\sigma \theta })\to H^{1}(\sigma , Z) $ , and let $z_c\in (Z^{\sigma \theta })^{-\sigma }$ be a representative of $\zeta _c$ . By definition there exists $z_0 \in Z$ such that $z = z_0 z_c \sigma (z_0^{-1})$ . Since $\zeta _c$ is in the kernel of the map $H^1(\ast , Z^{\sigma \theta }) \to H^1(\ast , G^{\sigma \theta })$ , there must exist $k \in G^{\sigma \theta }$ such that $z_c = k \sigma (k^{-1})$ . We see that $z = g \sigma (g)^{-1}$ is equal to $z_0 k \sigma ((z_0k)^{-1})$ , in other words

$$\begin{align*}z_0^{-1} k^{-1} g \in G^{\sigma}.\end{align*}$$

Let $\gamma _0$ be the image of $z_0^{-1} k^{-1} g$ in $G_{\mathrm {ad}}$ . Then $\gamma _0 \in (G^{\sigma })_{\mathrm {ad}}$ , and $\gamma _0$ is equal to the image of $k^{-1} g$ in $G_{\mathrm {ad}}$ . The element $\kappa =\gamma \gamma _0^{-1}$ of $G_{\mathrm {ad}}$ is therefore $\sigma $ -invariant, and since $\kappa =kZ$ , we also have $(\theta \sigma )(\kappa )=\kappa $ ; we deduce that $\theta (\kappa ) = \kappa $ , which concludes the proof of (a).

To show that when $g\in (G_{\mathrm {ad}})^\sigma \cap (G_{\mathrm {ad}})^\theta $ is a representative of $x \in Q_{\sigma }(G)$ , the image of g in $Q_\theta (G)$ depends only on x, it is enough to show that $(G^\sigma )_{\mathrm {ad}} \cap (G_{\mathrm {ad}})^\theta \subset (G^\theta )_{\mathrm {ad}}$ . This follows from [Reference Adams and Taïbi3, Proposition 5.4], applied to the action of G on $G_{\mathrm {ad}}$ and the obvious involutions. We conclude that taking $x \in Q_{\sigma }(G)$ to the image $\varphi (x)$ of g in $Q_{\theta }(G)$ gives a well-defined map $\varphi \colon Q_{\sigma }(G)\to Q_{\theta }(G)$ , which is clearly a group homomorphism. Exchanging the roles of $\sigma $ and $\theta $ , we get a well-defined map $Q_{\theta }(G) \to Q_{\sigma }(G)$ , which is the inverse of $\varphi $ ; therefore $\varphi $ is an isomorphism. Assertion (b) follows.

2.3.1 On representations

The adjoint action of $G_{\mathrm {ad}}^\sigma $ on $G({\mathbb {R}})$ induces an action on $G({\mathbb {R}})$ -representations, which descends to an action of $Q_\sigma (G)$ on equivalence classes of $G({\mathbb {R}})$ -representations. If q is an element of $Q_\sigma (G)$ , we write $\pi \mapsto \pi ^q$ for the action of q on equivalence classes of $G({\mathbb {R}})$ -representations.

Similarly the actions of $G_{\mathrm {ad}}^\theta $ on $\mathfrak {g}$ and K induce an action of $Q_\theta (G)$ on equivalence classes of $(\mathfrak {g},K)$ -modules, for which we use similar notation.

These two actions are intertwined by the isomorphism of Lemma 2.5 (this follows immediately from the definitions of the actions and of that isomorphism):

There is a similar compatibility statement with the third description of $Q(G)$ as $W_{K_{\mathrm {ad}}}/W_K$ , and the corresponding action on equivalence classes of discrete series representations. Suppose $q \in Q_{\sigma }(G)$ , and let w be the element of $W_{K_{\mathrm {ad}}}/W_K$ that corresponds to q using the isomorphisms of Lemma 2.5 and 2.6. Suppose $\pi =\pi _{\lambda , \zeta }$ is the equivalence class of a discrete series representation of $G({\mathbb {R}})$ ; inspecting the definitions in [Reference Adams and Vogan5, Section 4] shows that $w\cdot \pi = \pi ^q$ .

As a result, if $\Pi $ is an L-packet of discrete series for $G({\mathbb {R}})$ , we have compatible actions on $\Pi $ of $Q(G)=W_{K_{\mathrm {ad}}}/W_K$ , of $Q_\sigma (G)$ , and of $Q_{\theta }(G)$ . These actions are all simply transitive by Lemma 1.2.

2.3.2 On Whittaker data and generic representations

Suppose $\tilde {q} \in G_{\mathrm {ad}}^\sigma $ . For every Whittaker pair $(B, \eta )$ as in Section 1.2.3, the pair $\tilde {q}\cdot (B, \eta )=\big (\tilde {q} B \tilde {q}^{-1}, \eta \circ (x \mapsto \tilde {q} x \tilde {q}^{-1})\big )$ is again a Whittaker pair, which is conjugate to $(B, \eta )$ if and only if $\tilde {q} \in (G^\sigma )_{\mathrm {ad}}$ . This determines an action of $Q_\sigma (G)$ on the set of Whittaker data for $G({\mathbb {R}})$ , which we shall denote by $(q, \mathfrak {w}) \mapsto q \cdot \mathfrak {w}$ . This action is simply transitive: see [Reference Adams and Kaletha2, Lemma 3.1.3] for an elementary argument.

Now let $\pi $ be a generic discrete series representation of $G({\mathbb {R}})$ . Let q be an element of $Q_\sigma (G)$ , with a representative $\tilde {q}$ in $G_{\mathrm {ad}}^\sigma $ . For every Whittaker pair $(B, \eta )$ such that $\pi $ has a $(B, \eta )$ -Whittaker model, it is immediate from the definitions that the representation $\pi ^{\tilde g}$ is again a discrete series representation, and that it has a $\tilde {q}\cdot (B, \eta )$ -Whittaker model. Passing to equivalence classes we see that $\pi ^q$ is generic, and satisfies $\mathrm {Wh}(\pi ^q)=q \cdot \mathrm {Wh}(\pi )$ . Therefore the map $\pi \mapsto \mathrm {Wh}(\pi )$ is $Q_\sigma (G)$ -equivariant.

2.3.3 On real nilpotent orbits

To define the action of $Q_{\sigma }(G)$ on (4), first observe that $G_{\mathrm {ad}}$ acts on $\mathfrak {g}$ by the adjoint action. This action preserves ${\mathcal {O}}_{p}$ , and its restriction to $G_{\mathrm {ad}}({\mathbb {R}})$ preserves ${\mathcal {O}}_{p}\cap i\mathfrak {g}({\mathbb {R}})^*$ . Since elements of a given $G({\mathbb {R}})_{\mathrm {ad}}$ -orbit are in the same $G({\mathbb {R}})$ -orbit, this induces an action of $Q_{\sigma }(G)=G_{\mathrm {ad}}({\mathbb {R}})/G({\mathbb {R}})_{\mathrm {ad}}$ on $({\mathcal {O}}_{p}\cap i\mathfrak {g}({\mathbb {R}})^*)/G({\mathbb {R}})$ .

2.3.4 On regular nilpotent K-orbits

Next, $Q(G)$ , realized as $Q_\theta (G)$ , is a group of automorphisms of $(\mathfrak {g},K)$ . It induces actions on $\mathfrak {s}^*$ , ${\mathcal {O}}_{p}$ and ${\mathcal {O}}_{p}\cap \mathfrak {s}^*$ , and on $(\mathfrak {g},K)$ -modules.

The proof of (a) is exactly the same as that of Proposition 2.1(a), replacing $\sigma $ by the Cartan involution $\theta $ .

As for (b), it follows directly from the definition of the associated variety. This uses filtrations of the universal enveloping algebra of $\mathfrak {g}$ which are all invariant under $G_{\mathrm {ad}}^{\theta _{\mathrm {ad}}}$ : see for instance the Introduction of [Reference Vogan23]. Inspecting the definitions in loc. cit. it is clear that the map $\pi \mapsto \mathrm {AV}(\pi )$ , taking a generic discrete series $(\mathfrak {g},K)$ -module (or rather its equivalence class) to its associated variety, is equivariant under $Q_{\theta }(G)$ , proving (b).

3.5.1 The Sekiguchi correspondence for principal nilpotent orbits

Let us spell out the special case of the Sekiguchi correspondence which is of interest here. We follow [Reference Adams and Vogan6, Section 2].

Suppose $E_{\mathbb {R}}\in i\mathfrak {g}({\mathbb {R}})$ is a principal nilpotent element. We may choose $H_{\mathbb {R}}, F_{\mathbb {R}} \in \mathfrak {g}$ such that $(H_{\mathbb {R}},E_{\mathbb {R}},F_{\mathbb {R}})$ is an $\mathrm {SL}(2)$ -triple and $E_{\mathbb {R}}$ is contained in $i\mathfrak {g}({\mathbb {R}})$ , which implies $H_{\mathbb {R}}\in \mathfrak {g}({\mathbb {R}})$ . After conjugating by $G({\mathbb {R}})$ we may further assume $\theta (E_{\mathbb {R}})=-F_{\mathbb {R}}$ . Set

$$\begin{align*}E_\theta=\frac12(E_{\mathbb{R}}-F_{\mathbb{R}}+H_{\mathbb{R}})\quad F_\theta=\frac12(-E_{\mathbb{R}}+F_{\mathbb{R}}+H_{\mathbb{R}}). \end{align*}$$

Then

$$ \begin{align*}\mathrm{KS}(G({\mathbb{R}})\cdot E_{\mathbb{R}})=K\cdot E_\theta. \end{align*} $$

Computing the map in the other direction is similar. Suppose $E_\theta \in \mathfrak {s}$ is a principal nilpotent element. Choose $F_\theta , H_\theta \in \mathfrak {g}$ such that $(E_\theta , H_\theta , F_\theta )$ is an $SL(2)$ -triple with $F_\theta \in \mathfrak {s}$ , which implies $H_\theta \in {\mathfrak {k}}$ . After conjugating by K we may further assume $\sigma (E_\theta )=F_\theta $ . Define

(3.3) $$ \begin{align} E_{\mathbb{R}}=\frac12(E_\theta-F_\theta-H_\theta)\quad F_{\mathbb{R}}=\frac12(-E_\theta+F_\theta-H_\theta). \end{align} $$

Then

(3.4) $$ \begin{align} \mathrm{KS}(K\cdot E_\theta)=G({\mathbb{R}})\cdot E_{\mathbb{R}}\quad\mathrm{KS}(K\cdot F_\theta)=G({\mathbb{R}})\cdot(-E_{\mathbb{R}}). \end{align} $$

We need the corresponding result in $\mathfrak {g}^*$ . As discussed in Section 1.1.2 this uses a choice of an isomorphism $\psi \colon \mathfrak {g}\to \mathfrak {g}^*$ , but the result is independent of this choice. That is for $X\in {\mathcal {O}}_p\subset \mathfrak {g}^*$ we have

$$ \begin{align*}\mathrm{KS}(K\cdot X)=\psi(\mathrm{KS}(K\cdot \psi^{-1}(X))), \end{align*} $$

independent of the choice of $\psi $ .

3.5.2 A simple proof of Proposition 3.3

Let us fix a generic discrete series representation $\pi $ and let $\lambda $ be a representative for its Harish-Chandra parameter. As in Section 3.1 let $\kappa $ be the Killing form and $\psi \colon \mathfrak {g}_{\mathrm {der}}\to \mathfrak {g}_{\mathrm {der}}^*$ be the corresponding isomorphism.

Consider the principal nilpotent element $F_\pi $ defined in (3.1), so by Lemma 3.1 $\mathrm {AV}(\pi )=K\cdot \psi (F_\pi )$ . We use this as $F_\theta $ in the previous subsection: define $E_\theta = \sum _{\alpha \in S(\lambda )}X_\alpha $ and $H_\theta = [E_\theta , F_\theta ]$ ; then $E_\theta \in \mathfrak {s}$ and $(E_\theta , H_\theta , F_\theta )$ is an $\mathrm {SL}(2)$ -triple. Define another $\mathrm {SL}(2)$ -triple $(E_{\mathbb {R}}, F_{\mathbb {R}}, H_{\mathbb {R}})$ by

$$\begin{align*}\begin{aligned} F_{\mathbb{R}}&=\frac12(-E_\theta+F_\theta-H_\theta)\\ E_{\mathbb{R}}&=\frac12(E_\theta-F_\theta-H_\theta)\\ H_{\mathbb{R}}&=E_\theta+F_\theta. \end{aligned}\end{align*}$$

By the discussion in the previous section we have

$$ \begin{align*}\mathrm{KS}(K\cdot \psi(F_\theta))=\psi(\mathrm{KS}(K\cdot F_\theta))=\psi(G({\mathbb{R}})\cdot (-E_{\mathbb{R}}))=G({\mathbb{R}})\cdot \psi(-E_{\mathbb{R}}).\end{align*} $$

We need to show this equals $\mathrm {WF}(\pi )$ . By Corollary 3.5, proving $\mathrm {Wh}(\pi )=\mathfrak {w}_{\psi (-E_ {\mathbb {R}})}$ is enough; by Lemma 3.4 this is equivalent to: $\mathcal {K}(\psi (-E_{\mathbb {R}}))$ meets $G({\mathbb {R}})\cdot \lambda $ .

We compute $\mathcal {K}(\psi (-E_{\mathbb {R}}))=\psi (\mathcal {K}(-E_{\mathbb {R}}))\ni -E_{\mathbb {R}}-F_{\mathbb {R}}=H_\theta $ (by (3.3)). Thus $\mathcal {K}(\psi (-E_{\mathbb {R}}))$ meets $\psi (H_\theta )$ . Since $\psi (H_{\theta })$ and $\lambda $ are in the same Weyl chamber, by Proposition 3.2 we conclude $\mathcal {K}(\psi (-E_{\mathbb {R}}))$ meets $\lambda $ as required.

3.5.3 A simple description of the wavefront set

As we saw the correspondence $\pi \leftrightarrow \mathrm {WF}(\pi )$ can be described using the asymptotic cone of the semisimple orbit of the Harish-Chandra parameter. We used this in the proof of Proposition 3.3. Now, the results of Sections 3.1, 3.5.1 and 3.5.2 provide another description of $\mathrm {WF}(\pi )$ from the Harish-Chandra parameter: we can compose the maps (1) $\leftrightarrow $ (3) $\leftrightarrow $ (4). This gives the following rather concrete conclusion. As in Sect. 3.1 we define $\psi :\mathfrak {g}_{\mathrm {der}}\simeq \mathfrak {g}_{\mathrm {der}}^*$ using the Killing form.

Lemma 3.9. Let $\pi $ be a generic discrete series representation. Fix a representative $\lambda \in {\mathfrak {t}}^*$ of the Harish-Chandra parameter of $\pi $ . Define $F_\theta =F_\pi $ as in (3.1), and choose $E_\theta $ , $H_\theta $ such that $(H_\theta ,E_\theta ,F_\theta )$ is an $\mathrm {SL}(2)$ -triple with $\sigma (E_\theta )=F_\theta $ . Then

$$ \begin{align*}\mathrm{WF}(\pi)=G({\mathbb{R}})\cdot \psi(F_{\mathbb{R}}), \quad \text{where } F_{\mathbb{R}} = \frac i2(-E_\theta+F_\theta-H_\theta)\in\mathcal{N}\cap i\mathfrak{g}({\mathbb{R}}). \end{align*} $$

There is a partial converse to Lemma 3.9 if we consider discrete series within a single L-packet. Fix an L-packet $\Pi $ of discrete series, and consider a regular semisimple orbit $\Omega $ in ${\mathcal {O}}_{p} \cap i\mathfrak {g}({\mathbb {R}})^\ast $ ; then we can determine the generic representation $\pi \in \Pi $ with $\mathrm {WF}(\pi )=\Omega $ , as follows. Fix $\xi _{\mathbb {R}} \in \Omega $ , write $\xi _{\mathbb {R}}=F_{\mathbb {R}}^\ast $ with $F_{\mathbb {R}} \in \mathfrak {g}$ and let $E_{\mathbb {R}}, H_{\mathbb {R}} \in \mathfrak {g}$ be such that $(E_{\mathbb {R}}, H_{\mathbb {R}}, F_{\mathbb {R}})$ is an $\mathrm {SL}(2)$ -triple with $\theta (E_{\mathbb {R}})=-F_{\mathbb {R}}$ . Set

$$ \begin{align*}H_\theta=E_{\mathbb{R}}-F_{\mathbb{R}}. \end{align*} $$

Then $H_\theta $ is a regular semisimple element, and belongs to ${\mathfrak {k}}$ . Define a Cartan subalgebra ${\mathfrak {t}}$ of $\mathfrak {g}$ by ${\mathfrak {t}}=\mathrm {Cent}_{\mathfrak {g}}(H_\theta )$ , and a set of positive roots by

$$ \begin{align*}\Delta^+=\{\alpha\in\Delta({\mathfrak{t}},\mathfrak{g})\mid \alpha(H_\theta)>0\}. \end{align*} $$

Then $\pi $ is the discrete series representation in $\Pi $ whose Harish-Chandra parameter is dominant for $\Delta ^+$ .