2.1 Discriminant locus

For $\lambda \in \Lambda$ with $\langle \lambda ,\lambda \rangle <0$, we define

\[ H_{\lambda}:=\{[x]\in\Omega_{\Lambda};\,\langle x,\lambda\rangle=0\}. \]

Then $H_{\lambda }$ is a non-zero divisor on $\Omega _{\Lambda }$. For any root $d\in \Delta _{\Lambda }$, we have the relation

(2.1)\begin{equation} H_{d}=\Omega_{\Lambda\cap d^{\perp}}. \end{equation}

The discriminant locus of $\Omega _{\Lambda }$ is the reduced divisor of $\Omega _{\Lambda }$ defined by

\[ {\mathcal{D}}_{\Lambda}:=\sum_{d\in\Delta_{\Lambda}/\pm1}H_{d}. \]

We define the $O(\Lambda )$-invariant Zariski open subset $\Omega _{\Lambda }^{0}$ of $\Omega _{\Lambda }$ by

\[ \Omega_{\Lambda}^{0}:=\Omega_{\Lambda}\setminus{\mathcal{D}}_{\Lambda}. \]

We set

\[ \bar{\mathcal{D}}_{\Lambda}:={\mathcal{D}}_{\Lambda}/O(\Lambda), \quad {\mathcal{M}}_{\Lambda}^{0}:=\Omega_{\Lambda}^{0}/O(\Lambda)={\mathcal{M}}_{\Lambda}\setminus\bar{\mathcal{D}}_{\Lambda}. \]

2.2 Some subloci of ${\mathcal {D}}_{\Lambda }$

We define the decomposition $\Delta _{\Lambda }=\Delta ^{+}_{\Lambda }\amalg \Delta ^{-}_{\Lambda }$ by

\[ \Delta^{+}_{\Lambda}:=\{d\in\Delta_{\Lambda}, d/2\in\Lambda^{\lor}\}, \quad \Delta^{-}_{\Lambda}:=\{d\in\Delta_{\Lambda}, d/2\not\in\Lambda^{\lor}\}. \]

Then $\Delta ^{\pm }_{\Lambda }$ are $O(\Lambda )$-invariant. We define the $O(\Lambda )$-invariant reduced divisors ${\mathcal {D}}^{\pm }_{\Lambda }$ on $\Omega _{\Lambda }$ and the corresponding divisors $\bar {\mathcal {D}}^{\pm }_{\Lambda }$ on ${\mathcal {M}}_{\Lambda }$ by

\[ {\mathcal{D}}^{\pm}_{\Lambda}:=\sum_{d\in\Delta^{\pm}_{\Lambda}/\pm1}H_{d}, \quad \bar{\mathcal{D}}^{\pm}_{\Lambda}:={\mathcal{D}}^{\pm}_{\Lambda}/O(\Lambda). \]

Proposition 2.1 Let $d\in \Delta _{\Lambda }^{+}$. Let $i\colon \Omega _{\Lambda \cap d^{\perp }}=H_{d}\hookrightarrow \Omega _{\Lambda }$ be the inclusion. Then the following equalities of divisors on $\Omega _{\Lambda \cap d^{\perp }}$ hold:

\[ i^{*}({\mathcal{D}}_{\Lambda}^{+} - H_{d}) = {\mathcal{D}}_{\Lambda\cap d^{\perp}}^{+}, \quad i^{*}{\mathcal{D}}_{\Lambda}^{-} = {\mathcal{D}}_{\Lambda\cap d^{\perp}}^{-}. \]

Proof. Let $\delta \in \Delta _{\Lambda }\setminus \{\pm d\}$. Then $H_{d}\cap H_{\delta }\not =\emptyset$ if and only if $L:=\textbf {Z}d+\textbf {Z}\delta$ is negative-definite. Since the Gram matrix of $L$ with respect to the basis $\{d,\delta \}$ is given by $-2\binom {1\,a}{a\,1}$ where $a=-\langle \delta ,d/2\rangle \in \textbf {Z}$, we conclude that $H_{d}\cap H_{\delta }\not =\emptyset$ if and only if $a=0$, that is, $\delta \in \Delta _{\Lambda }\cap d^{\perp }=\Delta _{\Lambda \cap d^{\perp }}$. Since $\Delta _{\Lambda }^{\pm }\cap d^{\perp }=\Delta _{\Lambda \cap d^{\perp }}^{\pm }$, we get

\begin{gather*} i^{*}({\mathcal{D}}_{\Lambda}^{+}-H_{d}) = \sum_{\delta\in(\Delta_{\Lambda}^{+}\setminus\{\pm d\})/\pm1}i^{*}H_{\delta} = \sum_{\delta\in\Delta_{\Lambda}^{+}\cap d^{\perp}/\pm1}i^{*}H_{\delta} = \sum_{\delta\in\Delta_{\Lambda\cap d^{\perp}}^{+}/\pm1}H_{\delta} = {\mathcal{D}}_{\Lambda\cap d^{\perp}}^{+},\\ i^{*}{\mathcal{D}}_{\Lambda}^{-} = \sum_{\delta\in\Delta_{\Lambda}^{-}/\pm1}i^{*}H_{\delta} = \sum_{\delta\in\Delta_{\Lambda}^{-}\cap d^{\perp}/\pm1}i^{*}H_{\delta} = \sum_{\delta\in\Delta_{\Lambda\cap d^{\perp}}^{-}/\pm1}H_{\delta} = {\mathcal{D}}_{\Lambda\cap d^{\perp}}^{-}. \end{gather*}

This proves the result.

For $d\in \Delta _{\Lambda }$, we define a non-empty Zariski open subset $H^{0}_{d}\subset H_{d}$ by

\[ H_{d}^{0}:=H_{d}\left\backslash \bigcup_{\delta\in\Delta_{\Lambda}\setminus\{\pm d\}}H_{\delta}.\right. \]

We set

\[ {\mathcal{D}}^{0,\pm}_{\Lambda}:=\sum_{d\in\Delta^{\pm}_{\Lambda}/\pm1}H^{0}_{d}, \quad {\mathcal{D}}_{\Lambda}^{0} := {\mathcal{D}}^{0,+}_{\Lambda}+{\mathcal{D}}^{0,-}_{\Lambda} = \sum_{d\in\Delta_{\Lambda}/\pm1}H_{d}^{0}. \]

Then $\Omega _{\Lambda }^{0}\amalg {\mathcal {D}}_{\Lambda }^{0}$ is a Zariski open subset of $\Omega _{\Lambda }$, whose complement has codimension at least $2$ when $r(\Lambda )\geq 4$ and is empty when $r(\Lambda )\leq 3$.

2.3 Characteristic Heegner divisor

Set

\[ \varepsilon_{\Lambda}:=\{12-r(\Lambda)\}/2. \]

We define the characteristic Heegner divisor of $\Omega _{\Lambda }$ as the reduced divisor

\[ {\mathcal{H}}_{\Lambda} = {\mathcal{H}}_{\Lambda}(\varepsilon_{\Lambda},\textbf{1}_{\Lambda}) := \sum_{\lambda\in\Lambda^{\lor}/\pm1, \lambda^{2}=\varepsilon_{\Lambda}, [\lambda]=\textbf{1}_{\Lambda}}H_{\lambda}, \]

where $[\lambda ]:=\lambda +\Lambda \in A_{\Lambda }$. Since $\textbf {1}_{\Lambda }$ is $O(q_{\Lambda })$-invariant, ${\mathcal {H}}_{\Lambda }$ is $O(\Lambda )$-invariant. Since $\varepsilon _{\Lambda }\geq 0$ when $r(\Lambda )\leq 12$, we get ${\mathcal {H}}_{\Lambda }=0$ if $r(\Lambda )\leq 12$.

Proposition 2.2 Let $d\in \Delta _{\Lambda }^{+}$. Let $i\colon \Omega _{\Lambda \cap d^{\perp }}=H_{d}\hookrightarrow \Omega _{\Lambda }$ be the inclusion. Then the following equality of divisors on $\Omega _{\Lambda \cap d^{\perp }}$ holds:

\[ i^{*}{\mathcal{H}}_{\Lambda}=2 {\mathcal{H}}_{\Lambda\cap d^{\perp}}. \]

Proof. Since $r(\Lambda )\leq 21$, we get $\varepsilon _{\Lambda }\geq -\frac {9}{2}$. Set $\Lambda ':=\Lambda \cap d^{\perp }$. Since $d\in \Delta _{\Lambda }^{+}$, we deduce from [Reference Finashin and KharlamovFK08, Proposition 3.1] that $\Lambda$ and $\Lambda '\oplus \textbf {Z}d$ have the same invariants $(r,l,\delta )$. Hence we get the orthogonal decomposition $\Lambda =\Lambda '\oplus \textbf {Z}d$. Let $\lambda \in \Lambda ^{\lor }$ be such that $\lambda ^{2}=\varepsilon _{\Lambda }$ and $[\lambda ]=\textbf {1}_{\Lambda }$. Then we can write $\lambda =\lambda '+a(d/2)$, where $\lambda '\in (\Lambda ')^{\lor }$ and $a=-\langle \lambda ,d\rangle \in \textbf {Z}$. Since $[\lambda ]=\textbf {1}_{\Lambda }$, we get $a\equiv 1\bmod 2$. Hence $a=2k+1$ for some $k\in \textbf {Z}$ and $\lambda '\in \textbf {1}_{\Lambda '}$. Since $H_{\lambda '}\not =\emptyset$ if and only if $(\lambda ')^{2}<0$, we get $i^{*}H_{\lambda }=H_{\lambda '}\not =\emptyset$ if and only if $0>(\lambda ')^{2}=\lambda ^{2}+a^{2}/2=(\varepsilon _{\Lambda }+\frac {1}{2})+2k(k+1)$. Since $\varepsilon _{\Lambda }\geq -\frac {9}{2}$ and hence $-2\leq ({-1-\sqrt {-2\varepsilon _{\Lambda }}})/{2}< ({-1+\sqrt {-2\varepsilon _{\Lambda }}})/{2}\leq 1$, we see that $(\varepsilon _{\Lambda }+\frac {1}{2})+2k(k+1)<0$ if and only if $k=0,-1$. This proves that

(2.2)\begin{equation} i^{*}H_{\lambda}=H_{\lambda'}\not=\emptyset \Longleftrightarrow \lambda=\lambda'\pm(d/2). \end{equation}

When $\lambda =\lambda '\pm (d/2)$, we get $(\lambda ')^{2}=\varepsilon _{\Lambda }+\frac {1}{2}=\varepsilon _{\Lambda '}$. This, together with (2.2), yields that

\begin{align*} i^{*}{\mathcal{H}}_{\Lambda} &= \sum_{\lambda\in\Lambda^{\lor}/\pm1, \lambda^{2}=\varepsilon_{\Lambda}, [\lambda]=\textbf{1}_{\Lambda}}i^{*}H_{\lambda} = \sum_{\lambda'\in(\Lambda')^{\lor}/\pm1, (\lambda')^{2}=\varepsilon_{\Lambda'}, [\lambda']=\textbf{1}_{\Lambda'}}i^{*}H_{\lambda'\pm(d/2)}\\ &= 2\sum_{\lambda'\in(\Lambda')^{\lor}/\pm1, (\lambda')^{2}=\varepsilon_{\Lambda'}, [\lambda']=\textbf{1}_{\Lambda'}}H_{\lambda'} = 2 {\mathcal{H}}_{\Lambda'}. \end{align*}

This proves the proposition.

3.1 $2$-elementary $K3$ surfaces

A $K3$ surface $X$ equipped with a holomorphic involution $\iota \colon X\to X$ is called a $2$-elementary $K3$ surface if $\iota$ is anti-symplectic:

(3.1)\begin{equation} \iota^{*}|_{H^{0}(K_{X})}=-1. \end{equation}

The possible deformation types of $2$-elementary $K3$ surfaces were determined by Nikulin (see [Reference Alexeev and NikulinAN06, § 2.3] and the references therein). Let $\alpha \colon H^{2}(X,\textbf {Z})\cong {\Bbb L}_{K3}$ be an isometry of lattices. Set $H^{2}(X,\textbf {Z})_{\pm }:=\{l\in H^{2}(X,\textbf {Z});\,\iota ^{*}l=\pm l\}$ and

(3.2)\begin{equation} M:=\alpha(H^{2}(X,\textbf{Z})_{+}), \quad \Lambda:=M^{\perp}=\alpha(H^{2}(X,\textbf{Z})_{-}). \end{equation}

Then $M\subset {\Bbb L}_{K3}$ must be a primitive $2$-elementary Lorentzian sublattice. Conversely, for any primitive $2$-elementary Lorentzian sublattice $M\subset {\Bbb L}_{K3}$, there exists a $2$-elementary $K3$ surface with (3.2). For a $2$-elementary $K3$ surface $(X,\iota )$, the isometry class of $H^{2}(X,\textbf {Z})_{+}$ is called the type of $(X,\iota )$. By an abuse of notation, the sublattice itself $\alpha (H^{2}(X,\textbf {Z})_{+})\subset {\Bbb L}_{K3}$ is also called the type of $(X,\iota )$. Then there is a one-to-one correspondence between the deformation types of $2$-elementary $K3$ surfaces and the triplets $(r,l,\delta )$. Since the latter consists of $75$ points, there exist $75$ mutually distinct deformation types of $2$-elementary $K3$ surfaces. For a given primitive $2$-elementary Lorentzian sublattice $M\subset {\Bbb L}_{K3}$, the moduli space of $2$-elementary $K3$ surfaces of type $M$ is given as follows.

Let $(X,\iota )$ be a $2$-elementary $K3$ surface of type $M$ and let $\alpha \colon H^{2}(X,\textbf {Z})\cong {\Bbb L}_{K3}$ be an isometry satisfying (3.2). Since $H^{2,0}(X,\textbf {C})\subset H^{2}(X,\textbf {Z})_{-}\otimes \textbf {C}$ by (3.1), we get

\[ \pi_{M}(X,\iota,\alpha):=[\alpha(H^{2,0}(X,\textbf{C}))]\in\Omega_{\Lambda}^{0}. \]

Its $O(\Lambda )$-orbit is called the Griffiths period of $(X,\iota )$ and is denoted by

\[ \bar{\pi}_{M}(X,\iota):=O(\Lambda)\cdot\pi_{M}(X,\iota,\alpha)\in{\mathcal{M}}_{\Lambda}^{0}. \]

By [Reference YoshikawaYos04, Theorem 1.8] and [Reference YoshikawaYos13, Proposition 11.2], the coarse moduli space of $2$-elementary $K3$ surfaces of type $M$ is isomorphic to ${\mathcal {M}}_{\Lambda }^{0}$ via the period map $\bar {\pi }_{M}$. In the rest of this paper we identify the point $\bar {\pi }_{M}(X,\iota )\in {\mathcal {M}}_{\Lambda }^{0}$ with the isomorphism class of $(X,\iota )$.

3.2 The Torelli map for $2$-elementary $K3$ surfaces

3.2.1 The set of fixed points

For a $2$-elementary $K3$ surface $(X,\iota )$ of type $M$, set

\[ X^{\iota}:=\{x\in X;\,\iota(x)=x\}. \]

Then $X^{\iota }=\emptyset$ if and only if $M$ is exceptional, that is, $M\cong {\Bbb U}(2)\oplus {\Bbb E}_{8}(2)$. In this case, the quotient $X/\iota$ is an Enriques surface and $(X,\iota )$ is the universal covering of an Enriques surface endowed with the non-trivial covering transformation. When $M$ is non-exceptional, by Nikulin [Reference NikulinNik83, Theorem 4.2.2], we have

(3.3)\begin{equation} X^{\iota}=C^{(g(M))}\amalg E_{1}\amalg\cdots\amalg E_{k(M)} \end{equation}

for $M\not \cong {\Bbb U}(N)\oplus {\Bbb E}_{8}(2)$ $(N=1,2)$ and we have $X^{\iota }=C_{1}^{(1)}\amalg C_{2}^{(1)}$ for $M\cong {\Bbb U}\oplus {\Bbb E}_{8}(2)$, where $C^{(g)}$ is a projective curve of genus $g$ and $E_{i}\cong \mathbf{P}^{1}$ and

\[ g(M):=\{22-r(M)-l(M)\}/2, \quad k(M):=\{r(M)-l(M)\}/2. \]

Since $r(\Lambda )=22-r(M)$ and $l(\Lambda )=l(M)$, we have the relations

\[ g(M)=\{r(\Lambda)-l(\Lambda)\}/2, \quad k(M)=\{22-r(\Lambda)-l(\Lambda)\}/2. \]

As we defined $g(\Lambda )=(r(\Lambda )-l(\Lambda ))/2$ in Proposition 1.1, we have $g(M)=g(\Lambda )$. Notice that, when $M\cong {\Bbb U}(2)\oplus {\Bbb E}_{8}(2)$ and hence $X^{\iota }$ is empty, $g(M)=g(\Lambda )=1$ has no geometric meaning.

3.2.2 The Torelli map

For $g\geq 0$, let ${\frak S}_{g}$ be the Siegel upper half-space of degree $g$ and let ${\rm Sp}_{2g}(\textbf {Z})$ be the symplectic group of degree $2g$ over $\textbf {Z}$. We define

\[ {\mathcal{A}}_{g}:={\frak S}_{g}/{\rm Sp}_{2g}(\textbf{Z}). \]

The Satake compactification of ${\mathcal {A}}_{g}$ is denoted by ${\mathcal {A}}_{g}^{*}$.

For a $2$-elementary $K3$ surface $(X,\iota )$ of type $M$, the period of $X^{\iota }$ is denoted by $\Omega (X^{\iota })\in {\mathcal {A}}_{g(M)}$. We define a map $\bar {J}_{M}^{0}\colon {\mathcal {M}}_{\Lambda }^{0}\to {\mathcal {A}}_{g(M)}$ by

\[ \bar{J}_{M}^{0}(X,\iota) = \bar{J}_{M}^{0}(\bar{\pi}_{M}(X,\iota)) := \Omega(X^{\iota}). \]

Let $\Pi _{\Lambda }\colon \Omega _{\Lambda }\to {\mathcal {M}}_{\Lambda }$ be the projection. The Torelli map is the $O(\Lambda )$-equivariant holomorphic map $J_{M}^{0}\colon \Omega _{\Lambda }^{0}\to {\mathcal {A}}_{g(M)}$ defined by

\[ J_{M}^{0}:=\bar{J}_{M}^{0}\circ\Pi_{\Lambda}|_{\Omega_{\Lambda}^{0}}. \]

In Theorem 3.3 below, we will extend $J_{M}^{0}$ to a certain Zariski open subset of $\Omega _{\Lambda }$ containing $\Omega _{\Lambda }^{0}\cup {\mathcal {D}}_{\Lambda }^{0}$ and prove its compatibility with respect to the inclusion $\Omega _{\Lambda \cap d^{\perp }}=H_{d}\hookrightarrow \Omega _{\Lambda }$ for $d\in \Delta _{\Lambda }$. For this, we introduce a stratification of $\Omega _{\Lambda }$.

3.2.3 A stratification of $\Omega _{\Lambda }$

For a primitive sublattice $L\subset \Lambda$ generated by $\Delta _{L}$, we define

\[ H_{L}:=\bigcap_{d\in\Delta_{L}}H_{d}, \quad H_{L}^{0}:=H_{L}\left\backslash\bigcup_{\delta\in\Delta_{\Lambda}\setminus\Delta_{L}}H_{\delta}.\right. \]

Then $H_{L}\not =\emptyset$ if and only if $L$ is negative-definite, that is, $L$ is a root lattice. If $H_{L}\not =\emptyset$, then $H_{L}^{0}$ is a non-empty dense Zariski open subset of $H_{L}$. By definition, it is obvious that if $r(L)=r(L')$ and $L\not =L'$, then $H_{L}^{0}\cap H_{L'}^{0}=\emptyset$. Set

\[ \Omega_{\Lambda}^{k}:=\amalg_{L\subset\Lambda, r(L)=k}H_{L}^{0}, \quad \Omega_{\Lambda}^{\geq k}:=\amalg_{l\geq k}\Omega_{\Lambda}^{l}=\bigcup_{L\subset\Lambda, r(L)=k}H_{L}, \]

where $L$ runs over the set of all primitive root sublattices of $\Lambda$ of rank $k$. Then $\Omega _{\Lambda }^{\geq k}$ is a Zariski closed subset of $\Omega _{\Lambda }$ of pure codimension $k$, and $\Omega _{\Lambda }\setminus \Omega _{\Lambda }^{\geq k+1}$ is a dense Zariski open subset of $\Omega _{\Lambda }$, whose complement has codimension $k+1$. We have

\[ {\mathcal{D}}_{\Lambda}=\Omega_{\Lambda}^{\geq 1}=\sum_{d\in\Delta_{\Lambda}/\pm1}H_{d}, \quad \Omega_{\Lambda}^{0}=\Omega_{\Lambda}\setminus{\mathcal{D}}_{\Lambda}. \]

For a root lattice ${\Bbb K}$, let ${\Bbb K}(\Lambda )$ be the set of primitive sublattices of $\Lambda$ isometric to ${\Bbb K}$. Since a root lattice of rank $2$ is either ${\Bbb A}_{1}^{\oplus 2}$ or ${\Bbb A}_{2}$, we have

\[ \Omega_{\Lambda}^{\geq2}=\Bigg(\bigcup_{L\in{\Bbb A}_{1}^{\oplus2}(\Lambda)}H_{L}\Bigg)\cup\Bigg(\bigcup_{L\in{\Bbb A}_{2}(\Lambda)}H_{L}\Bigg). \]

We set

\[ {\mathcal{D}}_{\Lambda}^{1,+}:=\bigcup_{L\in{\Bbb A}_{1}^{\oplus2}(\Lambda)}H_{L}^{0}, \quad {\mathcal{D}}_{\Lambda}^{1,-}:=\bigcup_{L\in{\Bbb A}_{2}(\Lambda)}H_{L}^{0}. \]

Then ${\mathcal {D}}_{\Lambda }^{1,+}\amalg {\mathcal {D}}_{\Lambda }^{1,-}=\Omega _{\Lambda }^{2}$ and we have the stratification

(3.4)\begin{equation} \Omega_{\Lambda}\setminus\Omega_{\Lambda}^{\geq3} = \Omega_{\Lambda}^{0}\amalg{\mathcal{D}}_{\Lambda}^{0}\amalg{\mathcal{D}}_{\Lambda}^{1,+}\amalg{\mathcal{D}}_{\Lambda}^{1,-} \end{equation}

such that ${\mathcal {D}}_{\Lambda }\setminus \Omega _{\Lambda }^{\geq 3}={\mathcal {D}}_{\Lambda }^{0}\amalg {\mathcal {D}}_{\Lambda }^{1,+}\amalg {\mathcal {D}}_{\Lambda }^{1,-}$ is a divisor of $\Omega _{\Lambda }\setminus \Omega _{\Lambda }^{\geq 3}$.

Lemma 3.1 If $d\in \Delta _{\Lambda }^{+}$, then $H_{d}\setminus \Omega _{\Lambda }^{\geq 3}=\Omega _{\Lambda \cap d^{\perp }}^{0}\amalg {\mathcal {D}}_{\Lambda \cap d^{\perp }}^{0}$.

Proof. The proof consists of four steps.

Step 1. By the definition of ${\mathcal {D}}_{\Lambda }^{0}$, we have $H_{d}^{0}=H_{d}\cap {\mathcal {D}}_{\Lambda }^{0}$. Since $d\in \Delta _{\Lambda }^{+}$, we have $H_{d}\cap H_{\delta }\not =\emptyset$ for $\delta \in \Delta _{\Lambda }\setminus \{\pm d\}$ if and only if $\delta \in \Delta _{\Lambda \cap d^{\perp }}$. Hence $H_{d}\cap {\mathcal {D}}_{\Lambda }^{0}=H_{d}^{0}=H_{d}\setminus \bigcup _{\delta \in \Delta _{\Lambda }\setminus \{\pm d\}}H_{\delta } =H_{d}\setminus \bigcup _{\delta \in \Delta _{\Lambda \cap d^{\perp }}}H_{\delta }=\Omega _{\Lambda \cap d^{\perp }}^{0}$.

Step 2. Assume $H_{d}\cap {\mathcal {D}}_{\Lambda }^{1,-}\not =\emptyset$. Then there exist $L\in {\Bbb A}_{2}(\Lambda )$ and $[\eta ]\in H_{d}\cap H_{L}^{0}$. By the definition of $H_{L}^{0}$, we have $d\in \Delta _{L}$. Since $L\cong {\Bbb A}_{2}$, there exists $\delta \in \Delta _{L}$ with $\langle d,\delta \rangle =1$. Since $d\in \Delta _{\Lambda }^{+}$, this yields the contradiction $\langle d,\delta \rangle =2\langle (d/2),\delta \rangle \in 2\textbf {Z}$. This proves $H_{d}\cap {\mathcal {D}}_{\Lambda }^{1,-}=\emptyset$.

Step 3. Assume $H_{d}\cap {\mathcal {D}}_{\Lambda }^{1,+}\not =\emptyset$. Then there exists $L\in {\Bbb A}_{1}^{\oplus 2}(\Lambda )$ with $[\eta ]\in H_{d}\cap H_{L}^{0}$. By the definition of $H_{L}^{0}$, we get $d\in \Delta _{L}$. Hence $\Delta _{L}=\{\pm d,\pm \delta \}$ for some $\delta \in \Delta _{\Lambda \cap d^{\perp }}$. Thus

\begin{align*} H_{d}\cap{\mathcal{D}}_{\Lambda}^{1,+} &= \bigcup_{L\in{\Bbb A}_{1}^{\oplus2}(\Lambda), d\in L}H_{d}\cap H_{L}^{0} = \bigcup_{\delta\in\Delta_{\Lambda\cap d^{\perp}}} \biggl\{ (H_{d}\cap H_{\delta})\Big\backslash \bigcup_{\epsilon\in\Delta_{\Lambda} \setminus\{\pm d,\pm\delta\}}H_{\epsilon}\biggr\}\\ &=\bigcup_{\delta\in\Delta_{\Lambda\cap d^{\perp}}} \biggl\{ (H_{d}\cap H_{\delta})\Big\backslash \bigcup_{\epsilon\in\Delta_{\Lambda\cap d^{\perp}}\setminus\{\pm\delta\}}H_{\epsilon} \biggr\} = {\mathcal{D}}_{\Lambda\cap d^{\perp}}^{0}. \end{align*}

Since $d\in \Delta _{\Lambda }^{+}$, the third equality follows from the fact that $H_{d}\cap H_{\epsilon }\not =\emptyset$ for $\epsilon \in \Delta _{\Lambda }\setminus \{\pm d\}$ if and only if $\epsilon \in \Delta _{\Lambda \cap d^{\perp }}$. This proves $H_{d}\cap {\mathcal {D}}_{\Lambda }^{1,+}={\mathcal {D}}_{\Lambda \cap d^{\perp }}^{0}$.

Step 4. Since $H_{d}\cap \Omega _{\Lambda }^{0}=\emptyset$, the result follows from steps 1–3 and (3.4).

3.2.5 Inclusion of lattices and the Torelli map

Recall that for $d\in \Delta _{\Lambda }^{+}$, the equality of sets $H_{d}\setminus \Omega _{\Lambda }^{\geq 3}=\Omega _{\Lambda \cap d^{\perp }}^{0}\cup {\mathcal {D}}_{\Lambda \cap d^{\perp }}^{0}$ follows from Lemma 3.1. By Lemma 3.2 (1), $\Omega _{\Lambda }^{0}\cup {\mathcal {D}}_{\Lambda }^{0}\cup {\mathcal {D}}_{\Lambda }^{1,+}$ is a Zariski open subset of $\Omega _{\Lambda }\setminus \Omega _{\Lambda }^{\geq 3}$.

Theorem 3.3 $J_{M}^{0}$ extends to a holomorphic map from $\Omega _{\Lambda }^{0}\cup {\mathcal {D}}_{\Lambda }^{0}\cup {\mathcal {D}}_{\Lambda }^{1,+}$ to ${\mathcal {A}}_{g}^{*}$.

Proof. Set $n:=\dim \Omega _{\Lambda }$. Let $[\eta ]\in {\mathcal {D}}_{\Lambda }^{0}\cup {\mathcal {D}}_{\Lambda }^{1,+}$. By Lemma 3.2 (1), there is a neighborhood $U$ of $[\eta ]$ in $\Omega _{\Lambda }$ such that either $U\setminus ({\mathcal {D}}_{\Lambda }^{0}\cup {\mathcal {D}}_{\Lambda }^{1,+})\cong \Delta ^{*}\times \Delta ^{n-1}$ or $U\setminus ({\mathcal {D}}_{\Lambda }^{0}\cup {\mathcal {D}}_{\Lambda }^{1,+})\cong (\Delta ^{*})^{2}\times \Delta ^{n-2}$. By Borel [Reference BorelBor72], $J_{M}^{0}$ extends to a holomorphic map from $U$ to ${\mathcal {A}}_{g}^{*}$. Since $[\eta ]\in {\mathcal {D}}_{\Lambda }^{0}\cup {\mathcal {D}}_{\Lambda }^{1,+}$ is an arbitrary point, we get the result.

Remark 3.4 By Lemma 3.2 (2), Borel's extension theorem does not apply to $J_{M}^{0}$ near ${\mathcal {D}}_{\Lambda }^{1,-}$. This explains why $J_{M}$ does not extend to $\Omega _{\Lambda }\setminus \Omega _{\Lambda }^{\geq 3}$ in general.

Denote the extension of $J_{M}^{0}$ by

\[ J_{M}\colon\Omega_{\Lambda}^{0}\cup{\mathcal{D}}_{\Lambda}^{0}\cup{\mathcal{D}}_{\Lambda}^{1,+}\to{\mathcal{A}}_{g}^{*}, \]

and call it again the Torelli map. By [Reference YoshikawaYos13, Theorem 2.5], the equality

\[ J_{M}|_{H_{d}^{0}}=J_{[M\perp d]}|_{\Omega_{\Lambda\cap d^{\perp}}^{0}} \]

holds for all $d\in \Delta _{\Lambda }$. The following refinement is crucial for the proof Theorem 0.1.

Theorem 3.5 If $d\in \Delta _{\Lambda }^{+}$, then

(3.5)\begin{equation} J_{M}|_{H_{d}\setminus\Omega_{\Lambda}^{\geq3}} = J_{[M\perp d]}|_{\Omega_{\Lambda\cap d^{\perp}}^{0}\cup{\mathcal{D}}_{\Lambda\cap d^{\perp}}^{0}}. \end{equation}

Proof. Since both $J_{M}|_{H_{d}\setminus \Omega _{\Lambda }^{\geq 3}}$ and $J_{[M\perp d]}|_{\Omega _{\Lambda \cap d^{\perp }}^{0}\cup {\mathcal {D}}_{\Lambda \cap d^{\perp }}^{0}}$ are holomorphic maps from $\Omega _{\Lambda \cap d^{\perp }}^{0}\cup {\mathcal {D}}_{\Lambda \cap d^{\perp }}^{0}$ to ${\mathcal {A}}_{g}^{*}$ by Lemma 3.1 and Theorem 3.3, it suffices to prove the equality on $\Omega _{\Lambda \cap d^{\perp }}^{0}$. Since this was proved in [Reference YoshikawaYos13, Theorem 2.5], we get the result.

3.3 Hyperelliptic linear system

This subsection is the technical basis for §§ 6 and 7. We advise the reader to skip it for the moment and return when necessary. We prepare some tools to realize a given 2-elementary $K3$ surface as a double cover of $\mathbf{P}^2$ or a Hirzebruch surface.

We will use the following notation. For $n\geq 0$ let

\[ \textbf{F}_{n}=\textbf{P}(\mathcal{O}_{\textbf{P}^1}\oplus\mathcal{O}_{\textbf{P}^1}(n)) \]

be the $n$th Hirzebruch surface, equipped with the natural projection $\pi :\mathbf{F}_{n}\to \mathbf{P}^1$. When $n>0$, denote by $\Sigma \subset \mathbf{F}_{n}$ its unique $(-n)$-section. We write $L_{a,b}$ for the line bundle on $\mathbf{F}_{n}$ of $\pi$-degree $a$ with $(L_{a,b}, \Sigma )=b$. In particular, we have $\pi ^{\ast }\mathcal {O}_{\mathbf{P}^1}(1)\simeq L_{0,1}$, $\mathcal {O}_{\mathbf{F}_{n}}(\Sigma )\simeq L_{1,-n}$ and $K_{\mathbf{F}_{n}}\simeq L_{-2,-2+n}$.

Let $(X, \iota )$ be a 2-elementary $K3$ surface. A line bundle $L$ on $X$ with $(L, L)=2d>0$ is called hyperelliptic [Reference Saint-DonatSai74] if the linear system $|L|$ contains a smooth hyperelliptic member. In that case, $L$ is base point free and every smooth member of $|L|$ is hyperelliptic of genus $d+1$. The associated morphism

(3.6)\begin{equation} \phi_L : X \to |L|^{\vee} \simeq \textbf{P}^{d+1} \end{equation}

is generically two-to-one onto its image, mapping a smooth member of $|L|$ to a rational normal curve in a hyperplane of $|L|^{\vee }$. According to Saint-Donat [Reference Saint-DonatSai74, § 5], we have the following possibilities for the image surface $\phi _L(X)$.

1. (I) $\phi _L(X)$ coincides with $|L|^{\vee }\simeq \mathbf{P}^2$: this is the case $d=1$.

2. (II) $\phi _L(X)$ is a Veronese surface in $|L|^{\vee }\simeq \mathbf{P}^5$: this happens when $L=2L'$ for $L'\in {\rm Pic}(X)$ with $(L', L')=2$.

3. (III) $\phi _L(X)$ is a rational normal scroll, that is, the embedding image of $\mathbf{F}_n$ by a line bundle $L_{1,m}$ with $m>0$ and $n+2m=d$.

4. (IV) $\phi _L(X)$ is a cone over a rational normal curve, that is, the image of $\mathbf{F}_d$ by the bundle $L_{1,0}$. In this case $\phi _L$ lifts to a morphism $X\to \mathbf{F}_d$, and we must have $2\leq d\leq 4$.

We now assume that the hyperelliptic bundle $L$ is $\iota$-invariant, in the sense that there exists an isomorphism

\[ \iota^{\ast}L\simeq L. \]

Although $\iota$ may not necessarily act on $L$ equivariantly, it does so on the morphism (3.6). We then obtain an $\iota$-equivariant morphism

(3.7)\begin{equation} \phi : X\to Y \end{equation}

with $Y=\mathbf{P}^2$ in cases (I) and (II), and $Y=\mathbf{F}_n$, $\mathbf{F}_d$ in cases (III) and (IV).

It will be useful to have a purely lattice-theoretic method for finding an $\iota$-invariant hyperelliptic bundle. This is based on the following lemmas. Denote by $H_+=H^2(X, \textbf {Z})_+$ the invariant lattice of $(X, \iota )$.

Proof. We first show that $L$ is base point free. Otherwise, by [Reference MayerMay72, Proposition 5] the linear system $|L|$ would be of the form $|(d+1)F|+\Gamma$ where $F$ is a smooth elliptic curve and $\Gamma$ is a $(-2)$-curve with $(F, \Gamma )=1$. Since $\iota$ acts on $|L|$, the class of $F$ is $\iota$-invariant and then would violate assumption $({\rm b})$. Hence $L$ is free, and a general member $C\in |L|$ is smooth and irreducible of genus $d+1$. We show that $C$ is hyperelliptic. Since $g(C)=2$ when $d=1$, we may assume $d>1$. Let $E$ be a divisor as in assumption $({\rm a})$. Since $h^0(-E)=0$, we have $h^0(E)\geq 2$ by the Riemann–Roch inequality. Consider the exact sequence

(3.8)\begin{equation} 0 \to H^0(E-L) \to H^0(E) \to H^0(E|_C) \to \cdots \end{equation}

Since $(E-L, L)<0$, we have $h^0(E-L)=0$ by the nefness of $L$. Hence $h^0(E|_C)\geq h^{0}(E)\geq 2$ and $E|_C$ gives a $g^1_2$ on $C$.

Conditions (a) and (b) are purely arithmetic. It is always possible to meet the nefness condition by the following lemma.

Thus we can obtain an $\iota$-equivariant morphism (3.7) by just finding a vector in $H_+$ with the arithmetic conditions (a) and (b). The $\iota$-equivariant Weyl group action as in Lemma 3.7 would then carry this vector to a class of hyperelliptic bundles.

We are interested in when $\iota$ acts by the covering transformation of $\phi :X\to Y$, in which case $(X, \iota )$ may be recovered from $Y$ and the branch curve of $\phi$. Let $g$ denote the genus of the main component of the fixed curve $X^{\iota }$.

This criterion is coarse, but will suffice for our purpose.

4.1 Siegel modular forms

Recall that the line bundle on ${\mathcal {A}}_{g}$ associated with the automorphic factor ${\rm Sp}_{2g}(\textbf {Z})\ni \binom {A\,B}{C\,D}\mapsto \det (C\Omega +D)\in {\mathcal {O}}({\frak S}_{g})$, $\Omega \in {\frak S}_{g}$, is called the Hodge line bundle on ${\mathcal {A}}_{g}$ and is denoted by ${\mathcal {F}}_{g}$ in this paper. A holomorphic section of ${\mathcal {F}}_{g}^{\otimes q}$ is identified with a Siegel modular form of weight $q$, and ${\mathcal {F}}_{g}^{\otimes q}$ is equipped with the Hermitian metric $\|\cdot \|_{{\mathcal {F}}_{g}^{\otimes q}}$ called the Petersson norm: for any Siegel modular form $S$ of weight $q$, we define $\|S(\Omega )\|_{{\mathcal {F}}_{g}^{\otimes q}}^{2}:=(\det \Im \,\Omega )^{q}|S(\Omega )|^{2}$.

In this paper, the following Siegel modular forms on ${\frak S}_{g}$ play crucial roles:

\begin{gather*} \chi_{g}(\Omega)^{8}:=\prod_{(a,b)\,{\rm even}}\theta_{a,b}(\Omega)^{8},\\ \Upsilon_{g}(\Omega):=\chi_{g}(\Omega)^{8}\sum_{(a,b)\,{\rm even}}\theta_{a,b}(\Omega)^{-8}. \end{gather*}

Here

\[ \theta_{a,b}(\Omega):=\sum_{n\in\textbf{Z}^{g}}\exp\{\pi\sqrt{-1}{}^{t}(n+a)\Omega(n+a)+2\pi\sqrt{-1}{}^{t}(n+a)b\} \]

is the theta constant with even characteristic $(a,b)$, where $a,b\in \{0,1/2\}^{g}$ and $4{}^{t}ab\equiv 0\bmod 2$. For $g=0$, we set $\chi _{0}=\Upsilon _{0}=1$. Note that $\Upsilon _{g}(\Omega )$ is the elementary symmetric polynomial of degree $2^{g-1}(2^{g}+1)-1=(2^{g-1}+1)(2^{g}-1)$ in the even theta constants $\theta _{a,b}(\Omega )^{8}$. By [Reference IgusaIgu72, Corollary, p. 176, and Theorem 3, p. 182], $\chi _{g}^{8}$ (respectively, $\Upsilon _{g}$) is a Siegel modular form of weight $2^{g+1}(2^{g}+1)$ (respectively, $2(2^{g}-1)(2^{g}+2)$). The locus of vanishing theta-null $\theta _{{\rm null},g}$ is the reduced divisor on ${\mathcal {A}}_{g}$ defined by $\chi _{g}$

\[ \theta_{\textrm{null},g}:=\{[\Omega]\in{\mathcal{A}}_{g};\,\chi_{g}(\Omega)=0\}. \]

For the proof of Theorem 0.1(1) and (2), we need an estimate for the vanishing order of $\Upsilon _{g}$ for certain ordinary singular families of curves.

Lemma 4.2 Let $p\colon {\mathcal {C}}\to \Delta$ be an ordinary singular family of curves of genus $g>0$ with irreducible $C_{0}:=p^{-1}(0)$. Namely, $p\colon {\mathcal {C}}\to \Delta$ is a proper surjective holomorphic function from a complex surface ${\mathcal {C}}$ to the unit disc $\Delta$ without critical points on ${\mathcal {C}}\setminus C_{0}$ and with a unique, non-degenerate critical point on $C_{0}$. Assume that $\chi _{g}(\Omega (C_{t}))=0$ and $\Upsilon _{g}(\Omega (C_{t}))\not =0$ for all $t\in \Delta ^{*}$ and that $\chi _{g-1}(\Omega (\hat {C}_{0}))\not =0$, where $\hat {C}_{0}$ is the normalization of $C_{0}$. Then there exists $h(t)\in {\mathcal {O}}(\Delta )$ such that

\[ \log\|\Upsilon_{g}(\Omega(C_{t}))\|^{2} = (2^{2g-2}-1)\log|t|^{2}+\log|h(t)|^{2}+O(\log\log|t|^{-1}) \quad (t\to0). \]

Proof. We follow [Reference YoshikawaYos13, Proof of Lemma 4.1]. For $\Omega \in {\frak S}_{g}$, write $\Omega =\binom {z\,{}^{t}\omega }{\omega \,Z}$, where $z\in {\frak H}$, $\omega \in \textbf {C}^{g-1}$, $Z\in {\frak S}_{g-1}$. For $t\in \Delta ^{*}$, we can express

\[ \Omega(C_{t})=\left[\frac{\log t}{2\pi i}A+\psi(t)\right]\in{\mathcal{A}}_{g}, \quad A= \begin{pmatrix} 1 & {}^{t}\textbf{0}_{g-1}\\ \textbf{0}_{g-1} & O_{g-1} \end{pmatrix}, \]

where $\psi (t)$ is a holomorphic function on $\Delta$ with values in complex symmetric $g\times g$ matrices such that $\psi (0)=\binom {\psi _{0}\,{}^{t}\omega _{0}}{\omega _{0}\, Z_{0}}$, $Z_{0}\in {\frak S}_{g-1}$, $\Omega (\hat {C}_{0})=[Z_{0}]\in {\mathcal {A}}_{g-1}$.

By the assumption $\chi _{g}(\Omega (C_{t}))=0$, $\Upsilon _{g}(\Omega (C_{t}))\not =0$ for all $t\in \Delta ^{*}$ and Lemma 4.1, $C_{t}$ has a unique effective even theta characteristic for $t\in \Delta ^{*}$. By fixing a marking of a reference curve, there is a unique even pair $(a,b)$, $a,b\in \{0,1/2\}^{g}$, such that $\theta _{a,b}(\Omega (C_{t}))=0$ on $\Delta$ and $\theta _{c,d}(\Omega (C_{t}))\not =0$ on $\Delta ^{*}$ for all even $(c,d)\not =(a,b)$. Write $a=(a_{1},a')$ and $b=(b_{1},b')$. If $a_{1}=0$, then we get $\theta _{a',b'}(Z_{0})=0$ for the even pair $(a',b')$, $a',b'\in \{0,1/2\}^{g-1}$, by Yoshikawa [Reference YoshikawaYos13, (4.4)], which contradicts the assumption $\chi _{g-1}(Z_{0})=\chi _{g-1}(\Omega (\hat {C}_{0}))\not =0$. Thus $a_{1}=1/2$.

By [Reference YoshikawaYos13, (4.3), (4.4)], there is a holomorphic function $F_{a,b}(\zeta ,\omega ,Z)$ such that

\[ \prod_{(c,d)\not=(a,b)}\theta_{c,d}(\Omega) = (e^{\pi iz/4})^{2^{2(g-1)}-1}\,F_{a,b}(e^{\pi iz},\omega,Z). \]

Hence there is a holomorphic function $\phi (\zeta ,\omega ,Z)$ such that

\[ \Upsilon_{g}(\Omega)=(e^{2\pi iz})^{2^{2(g-1)}-1}\,\phi(e^{\pi iz},\omega,Z). \]

Since $\Upsilon _{g}(\Omega )$ is a Siegel modular form and hence $\Upsilon _{g}(\Omega +A)=\Upsilon _{g}(\Omega )$, $\phi (\zeta ,\omega ,Z)$ is an even function in $\zeta$. There exists a holomorphic function $h(t)\in {\mathcal {O}}(\Delta )$ such that

\[ \Upsilon_{g}((\log t/2\pi i)A+\psi(t))=t^{2^{2g-2}-1}h(t). \]

This, together with [Reference YoshikawaYos13, (4.7)], implies the result.

As a consequence of Lemma 4.2, we get the following result.

Lemma 4.3 Let $M\subset {\Bbb L}_{K3}$ be a primitive $2$-elementary Lorentzian sublattice and set $\Lambda =M^{\perp }$. Let $\gamma \colon \Delta \to {\mathcal {M}}_{\Lambda }$ be a holomorphic curve with $\gamma (\Delta ^{*})\subset {\mathcal {M}}_{\Lambda }^{0}$ intersecting $\bar {\mathcal {D}}_{\Lambda }^{0}$ transversally at $\gamma (0)\in \bar {\mathcal {D}}_{\Lambda }^{0}$. Let $\tilde {\gamma }\colon \Delta \to \Omega _{\Lambda }$ be its lift with $\gamma (t^{2})=\Pi _{\Lambda }\circ \tilde {\gamma }(t)$ and let $d\in \Delta _{\Lambda }^{-}$ be such that $\tilde {\gamma }(0)\in H_{d}$. If $\chi _{g}({J}_{M}(\tilde {\gamma }(t)))=0$ and $\Upsilon _{g}({J}_{M}(\tilde {\gamma }(t)))\not =0$ for all $t\in \Delta ^{*}$ and if $\chi _{g-1}(J_{[M\perp d]}(\tilde {\gamma }(0)))\not =0$, then

\[ {\rm ord}_{t=0}\tilde{\gamma}^{*}(J_{M}^{*}\Upsilon_{g})\geq2(2^{2(g-1)}-1). \]

Proof. Let $\bar {J}_{M}\colon {\mathcal {M}}_{\Lambda }^{0}\cup \bar {\mathcal {D}}_{\Lambda }^{0}\to {\mathcal {A}}_{g}^{*}$ be the extension of $\bar {J}_{M}^{0}\colon {\mathcal {M}}_{\Lambda }^{0}\to {\mathcal {A}}_{g}$. By [Reference YoshikawaYos13, Theorem 2.3(1) and (2)], there is an ordinary singular family of curves $p\colon {\mathcal {C}}\to \Delta$ of genus $g$ with irreducible $C_{0}$ and with period map $\bar {J}_{M}\circ \gamma$. By Lemma 4.2, we get

\[ {\rm ord}_{t=0}\gamma^{*}(\bar{J}_{M}^{*}\Upsilon_{g})\geq2^{2(g-1)}-1, \]

which, together with $J_{M}(\gamma (t^{2}))=\bar {J}_{M}(\tilde {\gamma }(t))$, yields the result.

4.2 Automorphic forms on $\Omega _{\Lambda }$

Let $M\subset {\Bbb L}_{K3}$ be a primitive $2$-elementary Lorentzian sublattice and set $\Lambda =M^{\perp }$ as before. Let $q\in \textbf {Z}_{>0}$ be such that ${\mathcal {F}}_{g}^{\otimes q}$ extends to a very ample line bundle on ${\mathcal {A}}_{g}^{*}$. Let $i\colon \Omega _{\Lambda }^{0}\cup {\mathcal {D}}_{\Lambda }^{0}\hookrightarrow \Omega _{\Lambda }$ be the inclusion and define $\lambda _{M}^{q}$ as the trivial extension of $J_{M}^{*}{\mathcal {F}}_{g(M)}^{\otimes q}$ from $\Omega _{\Lambda }^{0}\cup {\mathcal {D}}_{\Lambda }^{0}$ to $\Omega _{\Lambda }$, that is,

\[ \lambda_{M}^{q}:=i_{*}{\mathcal{O}}_{\Omega_{\Lambda}^{0}\cup{\mathcal{D}}_{\Lambda}^{0}}(J_{M}^{*}{\mathcal{F}}_{g(M)}^{\otimes q}). \]

Since $\Omega _{\Lambda }\setminus (\Omega _{\Lambda }^{0}\cup {\mathcal {D}}_{\Lambda }^{0})$ has codimension $2$ in $\Omega _{\Lambda }$, $\lambda _{M}^{q}$ is an $O(\Lambda )$-equivariant invertible sheaf on $\Omega _{\Lambda }$. On $\Omega _{\Lambda }^{0}$, $\lambda _{M}^{q}$ is equipped with the Hermitian metric

\[ \|\cdot\|_{\lambda_{M}^{q}}:=J_{M}^{*}\|\cdot\|_{{\mathcal{F}}_{g}^{\otimes q}}. \]

Fix $l_{\Lambda }\in \Lambda \otimes \textbf {R}$ with $\langle l_{\Lambda },l_{\Lambda }\rangle \geq 0$. Define $j_{\Lambda }(\gamma ,\cdot )\in {\mathcal {O}}^{*}_{\Omega _{\Lambda }}$, $\gamma \in O(\Lambda )$ and $K_{\Lambda }(\cdot )\in C^{\infty }(\Omega _{\Lambda })$ by

\[ j_{\Lambda}(\gamma,[\eta]) := \frac{\langle\gamma(\eta),l_{\Lambda}\rangle}{\langle\eta,l_{\Lambda}\rangle}, \quad K_{\Lambda}([\eta]):=\frac{\langle\eta,\bar{\eta}\rangle}{|\langle\eta,l_{\Lambda}\rangle|^{2}}. \]

Let $p,q\in \textbf {Z}$. Then $F\in H^{0}(\Omega _{\Lambda },\lambda _{M}^{q})$ is called an automorphic form on $\Omega _{\Lambda }$ for $O(\Lambda )$ of weight $(p,q)$ if it satisfies the following functional equation on $\Omega _{\Lambda }$:

(4.1)\begin{equation} F(\gamma\cdot[\eta])=j_{\Lambda}(\gamma,[\eta])^{p} \gamma(F([\eta])), \quad \forall\,\gamma\in O(\Lambda). \end{equation}

The notion of automorphic forms on $\Omega _{\Lambda }^{+}$ for $O^{+}(\Lambda )$ of weight $(p,q)$ is defined in the same way. In the rest of this paper the vector space of automorphic forms on $\Omega _{\Lambda }$ for $O(\Lambda )$ of weight $(p,q)$ is identified with the vector space of automorphic forms on $\Omega _{\Lambda }^{+}$ for $O^{+}(\Lambda )$ of weight $(p,q)$ via the restriction map

\[ H^{0}(\Omega_{\Lambda},\lambda_{M}^{q})\ni F\to F|_{\Omega_{\Lambda}^{+}} \in H^{0}(\Omega_{\Lambda}^{+},\lambda_{M}^{q}). \]

We define the Petersson norm of an automorphic form $F$ on $\Omega _{\Lambda }$ for $O(\Lambda )$ of weight $(p,q)$ as the $O(\Lambda )$-invariant $C^{\infty }$ function on $\Omega _{\Lambda }^{0}$ defined as

\[ \|F([\eta])\|^{2}:=K_{\Lambda}([\eta])^{p}\cdot\|F([\eta])\|_{\lambda_{M}^{q}}^{2}. \]

6.1 Proof of Theorem 6.1: the strategy

Let us first explain the outline of the proof of Theorem 6.1, reducing it to the construction of certain elliptic curves.

As the first step we see the irreducibility of $\bar {\mathcal {H}}_{\Lambda }$, which holds in a wider range.

Proof. This is restated as the property that vectors $l\in \Lambda ^{\vee }$ with $(l, l)=\varepsilon _{\Lambda }$ and $[l]= {\boldsymbol 1}_{\Lambda }$ are all equivalent under $O^+(\Lambda )$. Consider the vector $l'=2l$ in $\Lambda$. Since $ {\boldsymbol 1}_{\Lambda }$ is of order $2$, $l'$ is primitive in $\Lambda$ and satisfies

\[ {\rm div}(l')=2, \quad [l'/{\rm div}(l')] = {\boldsymbol 1}_{\Lambda}\in A_{\Lambda}, \quad (l', l')=4\varepsilon_{\Lambda}. \]

When $(r, l)\ne (9, 9)$, the lattice $\Lambda$ contains $\mathbb {U}\oplus \mathbb {U}$, and then we can resort to the Eichler criterion (cf. [Reference ScattoneSca87]) which says that the $O^+_0(\Lambda )$-equivalence class of a primitive vector $l'\in \Lambda$ depends only on the norm $(l', l')$ and the element $[l'/{\rm div}(l')]\in A_{\Lambda }$. Hence the above vectors $l'=2l$ are all $O^+_0(\Lambda )$-equivalent. When $(r, l)=(9, 9)$, $\bar {\mathcal {H}}_{\Lambda }$ is defined by $(-2)$-vectors $l'\in \Lambda$ with $\Lambda =\textbf {Z}l'\oplus (l')^{\perp }$. The isometry class of $(l')^{\perp }$ is uniquely determined by [Reference NikulinNik80], namely $(l')^{\perp }\simeq {\Bbb U}^{\oplus 2}\oplus {\Bbb E}_{8}(2)$, so that these $(-2)$-vectors $l'$ are all ${\rm O}^{+}(\Lambda )$-equivalent.

With the irreducibility of $\bar {\mathcal {H}}_{\Lambda }$ verified, the proof of Theorem 6.1 is reduced to showing the non-emptiness of $\bar {\mu }_{\Lambda }^{-1}({\frak M}_{g}')$ and the inclusion

(6.1)\begin{equation} \bar{\mu}_{\Lambda}^{-1}({\frak M}_{g}') \subset \bar{\mathcal{H}}_{\Lambda}. \end{equation}

We only need to verify (6.1) outside a locus of codimension at least $2$ of $\mathcal {M}_{\Lambda }^{0}$. Our approach will be based on the following geometric observation.

Proof. Let $H_{\pm }$ denote the $\iota$-(anti-)invariant lattices of $(X, \iota )$. The cycle $D_{\pm }:=E\pm \iota (E)$ is contained in $H_{\pm }$, respectively. We will show that $D_{-}$ satisfies

(6.2)\begin{equation} (D_-, D_-)=2r-20, \quad D_-/2\in H_-^{\vee}, \quad [D_-/2] = {\boldsymbol 1}_{H_-} \in A_{H_-}. \end{equation}

The presence of such an anti-invariant cycle in the Picard lattice means that the period of $(X, \iota )$ lies in $\bar {\mathcal {H}}_{\Lambda }$.

Since $E$ is an elliptic curve and hence its class in $H^{2}(X,\textbf {Z})$ is isotropic, the first equality of (6.2) follows from

(6.3)\begin{equation} 2(E, \iota(E)) = (D_+, D_+) = (X^{\iota}, X^{\iota}) = 20-2r. \end{equation}

The second property in (6.2) holds because

\[ (D_-, H_-) = (D_-+D_+, H_-) =(2E, H_-) \subset 2\textbf{Z}. \]

To see the last equality of (6.2), we note that the anti-isometry $\lambda :A_{H_+}\to A_{H_-}$ induced from the relation $H_+=(H_-)^{\perp }\cap H^2(X,\textbf {Z})$ maps $[D_+/2]$ to $[D_-/2]$ because $D_+/2+D_-/2=E$ is contained in $H^2(X,\textbf {Z})$. Since $[D_+/2]$ is the characteristic element of $A_{H_+}$ by Lemma 6.5 below, so is $[D_-/2]$ in $A_{H_-}$.

Proof. Let $f\colon X\to Y$ be the quotient map by $\iota$, and let $B\subset Y$ be the branch curve. Every element of the dual lattice $H_+^{\vee }$ can be written as $f^{\ast }L/2$ for some $L\in {\rm Pic}(Y)$. Then

\[ (X^{\iota}/2, f^{\ast}(L/2)) = (B, L)/4 = (-K_Y, L)/2. \]

We can see that $(L+K_Y, L)\in 2\textbf {Z}$ from the Riemann–Roch formula. Therefore

\[ (X^{\iota}/2, f^{\ast}(L/2)) \equiv (L, L)/2 = (f^{\ast}(L/2), f^{\ast}(L/2)) \mod \,\textbf{Z}, \]

which means that $X^{\iota }/2 \in H_+^{\vee }$ and that $[X^{\iota }/2]$ is characteristic.

In §§ 6.2 and 6.3, we will construct an elliptic curve $E$ as in Proposition 6.4. The non-emptiness of $\bar {\mu }_{\Lambda }^{-1}({\frak M}_{g}')$ will be seen in the course of proof.

6.2 Proof of Theorem 6.1: the case $g=3$

We begin with the case $g=3$. Recall that a smooth curve of genus $3$ has an effective even theta characteristic precisely when it is hyperelliptic. So let $(X, \iota )$ be a 2-elementary $K3$ surface with $(g, k)=(3, 0)$ such that $X^{\iota }$ is hyperelliptic. Consider the degree $4$ line bundle $L=\mathcal {O}_{X}(X^{\iota })$, which is hyperelliptic in the sense of § 3.3. By Saint-Donat's classification, $L$ defines a degree $2$ morphism

\[ \phi : X \to Q \]

onto a quadric $Q\subset \mathbf{P}^3$. We may assume that $L$ is ample, because the locus where $X^{\iota }$ is hyperelliptic and $\mathcal {O}_{X}(X^{\iota })$ is non-ample has codimension at least $2$ in $\mathcal {M}_{\Lambda }^0$. In that case $Q$ is smooth.

We choose a ruling line $l$ on $Q$ and put $E = \phi ^{\ast }l$. Then $E$ is a smooth elliptic curve on $X$ for a general choice of $l$ and satisfies the linear equivalence

\[ E+\iota(E) \sim \phi^{\ast}\mathcal{O}_Q(1, 1) \sim X^{\iota}. \]

By Proposition 6.4, we get the inclusion (6.1).

The non-emptiness of $\bar {\mu }_{\Lambda }^{-1}({\frak M}_{3}')$ can be seen by reversing this construction: choose a bidegree $(4, 4)$ curve $B\subset \mathbf{P}^1\times \mathbf{P}^1$ preserved by the switch involution of $\mathbf{P}^1\times \mathbf{P}^1$ and take the double cover $X\to \mathbf{P}^1\times \mathbf{P}^1$ branched over $B$. The switch involution can be lifted to $X$ so that its fixed curve is the preimage of the diagonal of $\mathbf{P}^1\times \mathbf{P}^1$, which is hyperelliptic of genus $3$. Thus Theorem 6.1 is proved in the case $g=3$.

6.3 Proof of Theorem 6.1: the case $4\leq g\leq 10$

We next treat the case $4\leq g\leq 10$ in Theorem 6.1. Let $(X, \iota )$ be a 2-elementary $K3$ surface with $4\leq g\leq 10$, $k=0$ and $\delta =1$. Let $Y=X/\iota$ be the quotient surface and $C\subset Y$ be the branch ($-2K_Y$)-curve. The quotient map $X\to Y$ gives the canonical identification

\[ C\simeq X^{\iota}. \]

The anti-canonical model $\bar {Y}\subset \mathbf{P}^{g-1}$ of $Y$ is a Gorenstein del Pezzo surface of degree $g-1$, and $Y$ is its minimal resolution. Note that if $X\to \bar {X}$ is the contraction of $(-2)$-curves disjoint from $X^{\iota }$, we naturally have $\bar {Y}\simeq \bar {X}/\iota$. We may view $C$ also as lying on the smooth locus of $\bar {Y}$. By the adjunction formula we have $-K_Y|_C\simeq K_C$, and the restriction map $|{-}K_{Y}|\to |K_{C}|$ is isomorphic because $h^{0}(K_{Y})=h^{1}(K_{Y})=0$. Therefore the composition of inclusions

\[ C \subset \bar{Y} \subset \textbf{P}^{g-1} \]

gives the canonical embedding of $C$, and we can identify $\mathbf{P}^{g-1}$ with $|K_C|^{\vee }$. This also shows that $C$ is non-hyperelliptic.

Proof. If $h^0(L)\ne 2$, then $h^0(L)\geq 4$. By Clifford's theorem we have

\[ 6\leq 2 {\dim}|L| < \textrm{deg}(L) =g-1. \]

This holds only when $g\geq 8$ and $h^0(L)=4$. Actually the case $g=8$ can be excluded because a blow-down of $Y$ presents $C$ as a plane sextic with two double points and hence $C$ has Clifford index $2$. The case $g=10$, where $C$ is a smooth plane sextic, is treated in [Reference Arbarello, Cornalba, Griffiths and HarrisACGH85, Exercise VI. B-3]. In the case $g=9$, presenting $C$ as plane sextic with a node or cusp, we can argue similarly.