Proof. Set $\Omega =[X:M]$ . Let N be a minimal normal subgroup of X. Since X is solvable, $N\cong {\mathbb {Z}}_p^k$ for some prime p and integer k. Since X is 2-transitive, it is primitive, which implies that N is transitive on $\Omega $ and so is regular on $\Omega $ . Therefore, $X=N\rtimes X_\alpha \le {\mathrm {AGL}}(k,p)$ for some $\alpha \in \Omega $ . Since X is 2-transitive and $|\Omega |=p^k$ , we know $|X_\alpha |\ge p^k-1$ for any $\alpha \in \Omega $ .

1. (i) Suppose that X contains an element of order $p^k$ . By Proposition 2.10, we get $(k,p)=(2,2)$ so that $X=S_4$ , reminding us that $|\Omega |$ is not a prime.

2. (ii) Let $X=X(D)$ and $M=C=\langle c\rangle $ with the order of m. Then, $|X(D)|=2nm=p^km$ . By Proposition 2.3, we get $p^km\leq p^k(p^k-1)$ . Therefore, $m=p^k-1$ as $m=|M|=|X_{\alpha }|\geq p^k-1$ , which implies that $\langle c\rangle $ is a Singer subgroup of ${\mathrm {GL}}(k,p)$ and $X(D)=N\rtimes \langle c\rangle $ . Then, both D and N are regular subgroups, that is, $|D|=2n=|\Omega |=p^k$ , which implies $p=2$ . Now, we have $|X(D)|=2^{k}(2^k-1)$ . Since both N and D are Sylow 2-subgroups of $X(D)$ and $N\lhd X(D)$ , we get $D=N$ , so that $D\cong {\mathbb {Z}}_2^2$ and $X(D)=A_4$ .

Proof. Let $G=D$ and $X=X(D)$ . If $m=1$ , then $X=D$ and $M=\langle a\rangle $ , as desired. So in what follows, we assume $m\ge 2$ . Recall that $m, n\ge 2$ , $|X|$ is even and more than 7. The lemma is proved by induction on the order of X. Up to isomorphism, all cases of $|X|\le 24$ are listed in Table 2, showing the conclusion.

Assume $|X|\gneqq 24$ . Then we carry out the proof by the following three steps.

Step 1: Case of $M_X\ne 1$ . Suppose that $M_X\ne 1$ . Set $M=\langle a^i\rangle \langle c\rangle $ for some i. Since

$$ \begin{align*}a^i\in \cap_{l_2, l_3} M^{a^{l_2}b^{l_3}}=\cap_{l_1, l_2, l_3} M^{c^{l_1}a^{l_2}b^{l_3}}= M_X,\end{align*} $$

we get that $M_X=M_X\cap (\langle a^i\rangle \langle c\rangle )=\langle a^i\rangle \langle c^r\rangle $ for some r. Set ${\overline X}:=X/{M_X}={\overline G}\langle {\overline c}\rangle $ . Then we claim that ${\overline G}\cap \langle {\overline c}\rangle =1$ . In fact, for any ${\overline g}={\overline c}'\in {\overline G}\cap \langle {\overline c}\rangle $ , for some $g\in G$ and $c'\in \langle {\overline c}\rangle $ , we have $gc^{\prime -1}\in M_X$ , that is, $g\in \langle a^i\rangle $ and $c'\in \langle c^r\rangle $ , which implies ${\overline g}={\overline c}'=1$ . Therefore, ${\overline G}\cap \langle {\overline c}\rangle =1$ . Let $M_0/M_X=\langle {\overline a}^j\rangle \langle {\overline c}\rangle $ be the largest subgroup of ${\overline X}$ containing $\langle {\overline c}\rangle $ and contained in the subset $\langle {\overline a}\rangle \langle {\overline c}\rangle $ . Then, $\langle {\overline a}^j\rangle \langle {\overline c}\rangle =\langle {\overline c}\rangle \langle {\overline a}^j\rangle $ . Since

$$ \begin{align*}\langle a^j\rangle \langle c\rangle M_X=\langle a^j\rangle M_X\langle c\rangle =\langle a^j\rangle \langle a^i\rangle \langle c\rangle \quad \mathrm{and}\quad \langle c\rangle \langle a^j\rangle M_X=\langle c\rangle M_X\langle a^j\rangle =\langle c\rangle \langle a^i\rangle \langle a^j\rangle , \end{align*} $$

we get $\langle a^i, a^j\rangle \langle c\rangle \le X$ . By the maximality of M, we have $\langle a^i, a^j\rangle =\langle a^i\rangle $ so that $M_0=M$ .

Table 2

All cases of $|X|\le 24$ up to isomorphism.

Using the induction hypothesis on ${\overline X}={\overline G}\langle {\overline c}\rangle $ , noting that $M_0/M_X=M/M_X$ is core-free in ${\overline X}$ , we get that ${\overline X}$ is isomorphic to ${\mathbb {Z}}_2,\,D_8,\,A_4$ or $S_4$ , and correspondingly, $o({\overline a})=k$ , where $k\in \{1, 2, 3, 4\}$ , and so $a^k\in M_X$ . Since $M=\langle a^i\rangle \langle c\rangle $ and $M_X=\langle a^i\rangle \langle c^r\rangle $ , we know that $\langle a^i\rangle =\langle a^k\rangle $ , which implies that $i\in \{1, 2, 3, 4\}$ . Clearly, if ${\overline X}={\mathbb {Z}}_2$ , then $M_X=M$ ; if ${\overline X}=D_8$ and ${\mathrm {o}}({\overline c})=2$ , then $M_X=\langle a^2\rangle \langle c^2\rangle $ ; if ${\overline X} =A_4$ and ${\mathrm {o}}({\overline c})=3$ , then $M_X=\langle a^2\rangle \langle c^3\rangle $ ; if ${\overline X} =S_4$ and ${\mathrm {o}}({\overline c})=4$ , then $M_X=\langle a^3\rangle \langle c^4\rangle $ ; and if ${\overline X}=S_4$ and ${\mathrm {o}}({\overline c})=3$ , then $M_X=\langle a^4\rangle \langle c^3\rangle $ .

Step 2: Show that if $M_X=1$ , then $G\in \{D_{2kp} | k=2,3,4$ and p is a prime $\}$ . Suppose that $M_X=1$ . Since both $\langle a\rangle _X$ and $\langle c\rangle _X$ are contained in $M_X$ , we get $\langle a\rangle _X=\langle c\rangle _X=1$ . Arguing by contradiction, assume that $G_X\ne 1$ . If $|G_X|\gneqq 4$ , then by $G=\langle a, b\rangle \cong D_{2n}$ , we get $\langle a\rangle _X\ne 1$ , which is a contradiction. So $|G_X|\le 4$ . Since $G_X\lhd G\cong D_{2n}$ , we know that $|G:G_X|\le 2$ , which implies $|G|\le 8$ , that is, $G\cong D_4$ or $D_8$ . A direct check shows that X is $D_8$ , $A_4$ or $S_4$ , which contradicts $|X|\gneqq 24$ . Therefore, $G_X=1$ .

Next, we consider the faithful (right multiplication) action of X on the set of right cosets $\Omega :=[X:\langle c\rangle ]$ . By Proposition 2.9, every dihedral group is a Burnside group, which implies that X is either 2-transitive or imprimitive. If X is primitive, then noting that X has a cyclic point-stabilizer $\langle c\rangle $ , by Lemma 2.13(2), we get $G=D_4$ and $X=A_4$ , which contradicts $|X|\gvertneqq 24$ . So in what follows, we assume that X is imprimitive. Pick a maximal subgroup H of X that contains $\langle c\rangle $ properly. Then, $H=H\cap X=(H\cap G)\langle c\rangle =\langle a^s, b_1\rangle \langle c\rangle <X$ for some $b_1\in G\setminus \langle a\rangle $ and some s. Using the same argument as that in Step 1, one has $a^s\in H_X$ . Reset ${\overline X}:=X/H_X$ . Consider the faithful primitive action of ${\overline X}$ on $\Omega _1:=[{\overline X}:{\overline H}]$ , with a cyclic regular subgroup of $\langle {\overline a}\rangle $ , where $|\Omega _1|=s$ . By Proposition 2.9, we know that either s is a prime p such that ${\overline X}\le {\mathrm {AGL}}(1,p)$ or s is a composite such that ${\overline X}$ is 2-transitive on $\Omega _1$ . In what follows, we consider these two cases when $a^s=1$ or $a^s\ne 1$ , separately.

Case (1): $a^s=1$ . In this case, $H=\langle c\rangle \rtimes \langle b\rangle $ and $X=\langle c, b\rangle .\langle a\rangle $ . Then two cases when s is composite or prime are considered, separately.

Suppose that s is a composite such that ${\overline X} $ is 2-transitive on $\Omega _1$ . By Lemma 2.13, we get ${\overline X}\le {\mathrm {AGL}}(l,q)$ for some prime q, which contains a cyclic regular subgroup $\langle {\overline a}\rangle $ of order $q^l$ . By Lemma 2.13(1), ${\overline X}\cong S_4$ and ${\mathrm {o}}({\overline a})=4$ so that ${\mathrm {o}}(a)=4$ (as $H_X\le \langle b, c\rangle $ ), which in turn implies $G=D_8$ . In this case, checking by Magma, we have that either ${\mathrm {o}}(c)=2, 3$ and $|X|\le 24$ ; or ${\mathrm {o}}(c)=4$ , $|X|=32$ but $G_X\ne 1$ , which is a contradiction.

Suppose that s is a prime p such that ${\overline X}\le {\mathrm {AGL}}(1,p)$ . Then, ${\mathrm {o}}(a)=p$ such that $G\cong D_{2p}$ , where $p\ge 5$ , as $|X|\gvertneqq 24$ . Clearly, $\langle {\overline a}\rangle \lhd {\overline X}$ . Consider the action of X on the set of blocks of length 2 on $\Omega =[X:\langle c\rangle ]$ , that is, the orbital of $\Omega $ under H, with the kernel $K:=H_X$ . If $K=1$ , then we get ${\overline X}=X$ and $\langle a\rangle \lhd X$ , which is a contradiction. Therefore, $K\neq 1$ . Then, $K\nleqq \langle c\rangle $ (as $\langle c\rangle _X=1$ ) so that K interchanges two points $\langle c\rangle $ and $\langle c\rangle b$ , which implies $|K/K\cap \langle c\rangle |=2$ . Since $K\cap \langle c\rangle $ is cyclic and $K\cap \langle c\rangle $ fixes setwise each block of length 2, we get $|K\cap \langle c\rangle |\leq 2$ . Therefore, $|K|\le 4$ . Since $K\rtimes \langle a\rangle \lhd X$ and $p\ge 5$ , we have $K\rtimes \langle a\rangle =K\times \langle a\rangle $ so that $\langle a\rangle { \, \mathrm {char}\,} (K\times \langle a\rangle )\lhd X$ , which contradicts $G_X=1$ .

Case (2): $a^s\neq 1$ . First, show $s=p$ , a prime. Arguing by a contradiction, assume that s is composite. To do that, recall ${\overline X}=X/H_X=\langle {\overline a}\rangle {\overline H}$ and $\Omega _1:=[{\overline X}:{\overline H}]$ . Then, ${\overline X}$ is 2-transitive on $\Omega _1$ , with a cyclic regular subgroup of $\langle {\overline a}\rangle $ . Since $a^s\in H_X$ , we get $H_X\ne 1$ and of course $H_X\nleqq \langle c\rangle $ . Suppose that $\langle a^j\rangle \langle c\rangle \le H$ . Then $a^j\in M$ . Using the same arguments as in the first line of Step 1 again, we get $a^j\in M_X=1$ . Therefore, there exists an l such that $bc^l\in H_X$ , which implies ${\overline H}=\langle {\overline c}\rangle $ . Then, ${\overline X}=\langle {\overline a}\rangle \langle {\overline c}\rangle $ , a product of two cyclic subgroups, cannot be isomorphic to $S_4$ , which contradicts Lemma 2.13(1). Therefore, $s=p$ , a prime, and $X/H_X\le {\mathrm {AGL}}(1, p)$ .

Second, consider the quotient group ${\overline H}:=H/\langle c\rangle _H=\langle {\overline a}^p,{\overline b}\rangle \langle {\overline c}\rangle $ , taking into account $s=p$ , a prime. Clearly, we have $\langle {\overline c} \rangle _{{\overline H}}=1$ and $o({\overline a}^p)=o(a^p)$ . Let $H_0/\langle c\rangle _H=\langle {\overline a}^{pj}\rangle \langle {\overline c}\rangle $ be the subgroup of ${\overline H}$ with the largest order such that $\langle {\overline c}\rangle \le H_0/\langle c\rangle _H\subseteqq \langle {\overline a}^p\rangle \langle {\overline c}\rangle $ . Since $|{\overline H}|<|X|$ , by the induction hypothesis on ${\overline H}$ , we know that $H_0/\langle c\rangle _H=\langle {\overline a}^{pk}\rangle \langle {\overline c}\rangle $ for $k\in \{1, 2, 3, 4\}$ , which implies $\langle a^{pk}\rangle \langle c\rangle \langle c\rangle _H=\langle c\rangle \langle a^{pk}\rangle \langle c\rangle _H=\langle c\rangle \langle a^{pk}\rangle $ , giving $\langle a^{pk} \rangle \langle c\rangle \le H\le X$ . Therefore, we get $a^{pk}\in M_X$ . Since $M_X=1$ and $a^p=a^s\ne 1$ , we get that the order of a is $kp$ , where $ k\in \{2, 3, 4\}.$ Therefore, only the following three groups remain: $G=D_{2kp}$ , where $k\in \{2, 3,4\}$ and p is a prime.

Step 3: Show that G cannot be $D_{2kp}$ , where $k\in \{2, 3, 4\}$ and p is a prime, provided $M_X=1$ . Suppose that $G\cong D_{2kp}$ , $k\in \{2, 3, 4\}$ , recalling that $H=\langle a^p, b\rangle \langle c\rangle $ , $M_X=1$ , $\Omega =[X:\langle c\rangle ]$ and X has blocks of length $2k$ acting on $\Omega $ . Moreover, $\langle a^p\rangle _X=1$ and there exists no nontrivial element $a^j\in H$ such that $\langle a^j \rangle \langle c\rangle \le H$ . Here, we only give the proof for the case $k=4$ , that is, $G\cong D_{8p}$ . The proof for other cases is similar but easier.

Let $G=D_{8p}$ . If $p=2$ or $3$ , then $G\cong D_{16}$ or $D_{24}$ , which can be directly excluded by Magma. So assume $p\ge 5$ . Set $a_1=a^p$ and $a_2=a^4$ so that $H=\langle a_1, b\rangle \langle c\rangle $ , where ${\mathrm {o}}(a_1)=4$ . Set $C_0=\langle c\rangle _H$ and $K=H_X$ . Then, H contains an element $a_1$ of order 4 having two orbits of length 4 on each block of length 8, where $a_1\le K$ . Consider the action of ${\overline H}:=H/C_0=\langle {\overline a}_1, {\overline b}\rangle \langle {\overline c}\rangle $ on the block containing the point $\langle c \rangle $ , noting that $\langle {\overline c}\rangle $ is core-free. Recall that $\langle a_1\rangle _X=1$ . So $\langle {\overline a}_1,{\overline b}\rangle \cong D_8$ and ${\overline H}\cong S_4$ . Moreover, we have $N_{{\overline H}}(\langle {\overline c} \rangle )=\langle {\overline c}\rangle \rtimes \langle {\overline b}\rangle \cong D_6$ , by rechoosing b in $\langle a_1, b\rangle $ . Therefore, $L:=\langle c\rangle \langle b\rangle \le X$ .

Now we turn to considering the imprimitive action of X on $\Omega _2:=[X:L]$ , which is of degree $4p$ . Let $K\cap \langle c\rangle =\langle c_1\rangle $ . Then every orbit of $\langle c_1\rangle $ on $\Omega _2$ is of length no more than 4. Observing the cycle decomposition of $c_1L_X\in X/L_X$ on $\Omega _2$ , we know that $k_1:=|\langle c_1\rangle L_X/L_X| | 12$ . Therefore, $c_1^{k_1}$ fixes all points in $\Omega _2$ , which implies $c_1^{2k_1}$ fixes all points in $\Omega $ . Therefore, $c_1^{2k_1}=1$ (as $\langle c\rangle _X=1$ ), that is, $|K\cap \langle c\rangle | | 2k_1$ and, in particular, $|K\cap C_0| | 2k_1$ . Moreover, since $\langle a_2K\rangle \lhd X/K\cong {\mathbb {Z}}_p\rtimes {\mathbb {Z}}_r$ for some $r | (p-1)$ , we know that $K\rtimes \langle a_2\rangle \lhd X$ . Then, $|K\cap C_0| | 24$ , as $k_1 | 12$ . Also, $K/(K\cap C_0)\cong KC_0/C_0\lhd H/C_0\cong S_4$ . Since ${\overline a}_1\in KC_0/C_0$ , where ${\mathrm {o}}({\overline a}_1)=4$ , and every normal subgroup of $S_4$ containing an element of order 4 must contain $A_4$ , we know that $K/(K\cap C_0)$ contains a characteristic subgroup $K_1/(K\cap C_0)\cong A_4$ .

We claim that $C_{K_1}(K\cap C_0)=K_1$ . Arguing by contradiction, assume that $K\cap C_0\nleqslant Z(K_1)$ . Then, as a quotient of $A_4$ , we have $3| |K_1/C_{K_1}(K\cap C_0)|.$ However, since $K_1/C_{K_1}(K\cap C_0)\le {\mathrm {Aut\,}}(K\cap C_0)$ , $K\cap C_0\le {\mathbb {Z}}_3\times {\mathbb {Z}}_8$ and using the cycle decomposition of the generator of $K\cap C_0$ , we get that $K_1/C_{K_1}(K\cap C_0)$ contains no element of order 3, which implies a contradiction. Therefore, $C_{K_1}(K\cap C_0)=K_1$ , that is, $(K\cap C_0)\le Z(K_1)$ .

Since $K_1/(K\cap C_0)\cong A_4$ and $Z(A_4)=1$ , we get that $K\cap C_0=Z(K_1) { \, \mathrm {char}\,} K_1\lhd X$ . Therefore, $K\cap C_0\le 1$ (as $\langle c\rangle _X=1$ ) so that $K\cong A_4$ or $S_4$ , which implies

$$ \begin{align*}\langle a_2\rangle { \, \mathrm {char}\,} (K\times \langle a_2\rangle )\lhd X,\end{align*} $$

which contradicts $G_X=1$ again.

Proof. Since $\langle c\rangle _X=1$ , by Proposition 2.3, we have $m<|G|$ . So $S:=G\cap G^{c}\ne 1$ , otherwise $|X|\ge |G|^2>|X|$ . Take a subgroup T of order a prime p of S. Since $o(a^jb)=4$ for any integer j, we know $T\le \langle a\rangle $ . Since S has a unique subgroup of order p, we get $T^c=T$ , giving $T\lhd X$ and so $\langle a\rangle _X\ne 1$ , as desired.

Proof. Let X be a minimal counter-example. First, we claim that $\langle c\rangle $ is core-free. Arguing by contradiction, assume that $\langle c\rangle _X\ne 1$ . Consider ${\overline X}=X/\langle c\rangle _X$ . The subgroup $\overline {M_1}$ of ${\overline X}$ is chosen to have the largest order such that $\langle {\overline c}\rangle \le \overline {M_1}\subseteqq \langle {\overline a}\rangle \langle {\overline c}\rangle $ . Since $|{\overline X}|<|X|$ and ${\overline G}$ is a generalized quaternion group, by the minimality of ${\overline X}$ , we get $\overline {M_1}=\langle {\overline a}^i\rangle \langle {\overline c}\rangle $ , where $i\in \{1, 2,3,4\}$ . This gives $M=\langle a^k\rangle \langle c\rangle $ , where $k\in \{1, 2, 3, 4\}$ , which is a contradiction. Therefore, $\langle c\rangle $ is core-free, that is, $\langle c\rangle _X=1$ .

By Lemma 3.2, $\langle a\rangle _X\ne 1$ . If $\langle a\rangle _X=\langle a\rangle $ , then $X=(\langle a\rangle \rtimes \langle c\rangle ).\langle b\rangle $ , which implies $M=\langle a\rangle \langle c\rangle $ , and that is a contradiction. So $\langle a\rangle _X<\langle a\rangle $ . Set ${\overline X}:=X/\langle a\rangle _X={\overline G} \langle {\overline c}\rangle $ and let the subgroup $\overline {M_2}/\langle a\rangle _X$ of ${\overline X}$ be of the largest order such that $\langle {\overline c}\rangle \le \overline {M_2}\subseteqq \langle {\overline a}\rangle \langle {\overline c}\rangle $ . If $a^n\not \in \langle a\rangle _X$ , then ${\overline G}$ is a generalized quaternion group; if $a^n\in \langle a\rangle _X$ , then ${\overline G}$ is a dihedral group. For the first case, by the minimality of ${\overline X}$ , and for the second case, by Lemma 3.1, we get $\overline {M_2}=\langle {\overline a}^i\rangle \langle {\overline c}\rangle $ , where $i\in \{1, 2, 3, 4\}$ . Then, $\langle a^i\rangle \langle c\rangle \langle a\rangle _X=\langle c\rangle \langle a^i\rangle \langle a\rangle _X$ . This gives $(\langle a^i\rangle \langle a\rangle _X)\langle c\rangle \le X$ , which implies $M=\langle a^k\rangle \langle c\rangle $ , where $k\in \{1, 2, 3, 4\}$ , which is a contradiction.

Proof. Suppose that X is a minimal counter-example. Let $a_0$ be the involution of $\langle a\rangle $ . Since $\langle c\rangle _X=1$ , by Lemma 3.2, we get $\langle a\rangle _X\ne 1$ , which implies $\langle a_0\rangle \lhd X$ , as G is a 2-group. Consider ${\overline X}=X/\langle a_0\rangle ={\overline G}\langle {\overline c}\rangle $ . Note that ${\overline G}$ is a dihedral group. Set $\langle {\overline c} \rangle _X=(\langle a_0\rangle \times \langle c_0\rangle )/\langle a_0\rangle $ . Then, $\langle c_0^2\rangle \lhd X$ , which implies $c_0^2=1$ . If $c_0=1$ , by Proposition 2.11, we get $\langle {\overline a}^2\rangle \lhd {\overline X}$ . Then, $\langle a^2\rangle \lhd X$ is a contradiction, noting X is a minimal counter-example. Therefore, $o(c_0)=2$ . By using Proposition 2.11 on $X/\langle a_0,c_0\rangle $ , we get $(\langle a^2\rangle (\langle a_0\rangle \langle c_0\rangle ))/\langle a_0,c_0\rangle \lhd X/\langle a_0,c_0\rangle $ , that is, $H:=\langle a^2\rangle \rtimes \langle c_0\rangle \lhd X$ . Then we continue the proof by the following two steps.

Step 1: Show that X is a 2-group. In fact, noting that $\langle a^4\rangle =\mho _1(H){ \, \mathrm {char}\,} H\lhd X$ , relabel ${\overline X}=X/\langle a^4\rangle $ , where we write $\langle {\overline c}\rangle _{{\overline X}}=\langle {\overline c}^i\rangle $ . Then, $\langle a^4\rangle \rtimes \langle c^i\rangle \lhd X$ . Let Q be the $2'$ -Hall subgroup of $\langle c^i\rangle $ . Since ${\mathrm {Aut\,}}(\langle a^4\rangle )$ is a $2$ -group, we know that $[Q, a^4]=1$ and so $Q\lhd X$ , which contradicts $\langle c\rangle _X=1$ . Therefore, $\langle c^i\rangle $ is also a $2$ -group. Reset ${\overline X}=X/\langle a^4\rangle \langle c^i\rangle ={\overline G} \langle {\overline c}\rangle $ . Now, $|{\overline G}|=8$ and so ${\overline G}\cong D_8$ (clearly, ${\overline G}$ cannot be $Q_8$ ). Since $\langle {\overline c}\rangle _{{\overline X}}=1$ , we have ${\mathrm {o}}({\overline c})| 4$ . Therefore, ${\overline X}$ is a $2$ -group and so is X.

Step 2: Get a contradiction. Set $K:=\langle a_0\rangle \times \langle c_0\rangle \cong {\mathbb {Z}}_2^2$ . Consider the conjugacy of G on K. Since $\langle c_0\rangle \ntrianglelefteq X$ , we get $C_X(K)\cap G< G$ . Since G may be generated by some elements of the form $a^jb$ , there exists an element $a^ib\in G\setminus C_G(K)$ , exchanging $c_0$ and $c_0a_0$ (as $(a^ib)^2=a_0$ ). To make it easier to write, we write b instead of $a^ib$ . Since $X=GC=(\langle a\rangle \langle c\rangle ).\langle b\rangle $ , first we write $c^{b}=a^{s}c^{t}$ , where $t\neq 0$ . Note that

$$ \begin{align*}c=c^{b^2}=(a^sc^t)^{b}=a^{-s}(a^sc^t)^t=c^t(a^sc^t)^{t-1},\end{align*} $$

which implies

$$ \begin{align*}(a^sc^t)^{t-1}=c^{1-t}.\end{align*} $$

Then we have

$$ \begin{align*}(c^{t-1})^{b}=(c^{b})^{t-1}=(a^sc^t)^{t-1}=c^{1-t}.\end{align*} $$

If $t\neq 1$ , then $c_1^b\in \langle c^{t-1}\rangle ^b=\langle c^{t-1}\rangle $ , which contradicts $c_1^b=a_1c_1$ . So $t=1$ , that is, $c^b=a^sc$ . Second, we write $c^b=c^{t_1}a^{s_1}$ . With the same arguments as the above, we can get $t_1=1$ and $c^b=ca^{s_1}$ . Therefore, we have $a^{s}c=c^{b}=ca^{s_1},$ that is, $(a^{s})^{c}=a^{s_1}$ . Clearly, $\langle a^s\rangle =\langle a^{s_1}\rangle $ , which implies that c normalizes $\langle a^{s}, b\rangle $ . Note that

$$ \begin{align*}\langle a^{s}, b\rangle \leq \cap_{c^i\in \langle c\rangle } G^{c^i}=\cap_{x\in X} G^x=G_X,\end{align*} $$

which implies $b^a=ba^{-2}\in G_X$ . Thus, $a^2\in G_X$ , which implies $\langle a^2\rangle \lhd X$ . This contradicts the minimality of X.

Proof. Take a minimal counter-example X and set ${\mathrm {o}}(c)=m$ , ${\mathrm {o}}(a)=2n$ and $a_1:=a^n$ . By Lemma 4.1, we know that G is not a 2-group. We carry out the proof by the following three steps.

Step 1: Show that the possible groups for G are $Q_{4p^k}$ , where p is an odd prime. By Lemma 3.2, let p be the maximal prime divisor of $|\langle a\rangle _X|$ and set $a_0=a^{{2n}/p}$ . Clearly, $a_0\in \langle a^2\rangle $ if p is odd. Consider ${\overline X}=X/\langle a_0\rangle ={\overline G}\langle {\overline c}\rangle $ and set $\langle {\overline c}\rangle _{{\overline X}}=\langle \overline {c_0}\rangle $ . Then depending on whether p is 2, ${\overline G}$ is either a dihedral group or a generalized quaternion group. We claim that $1\ne \langle {\overline c}\rangle _{{\overline X}}=\langle \overline {c_0}\rangle \lneqq \langle {\overline c}\rangle $ . Arguing by contradiction, assume that $\langle {\overline c}\rangle _{{\overline X}}$ is either 1 or $\langle {\overline c}\rangle $ . Suppose that $\langle {\overline c} \rangle _{{\overline X}}=1$ . Then by the minimality of X or Proposition 2.11, we get $\langle {\overline a}^2\rangle \lhd {\overline X}$ , which implies $\langle a^2\rangle \langle a_0\rangle \lhd X$ . Since $\langle a^2\rangle \langle a_0\rangle $ is either $\langle a^2\rangle $ or $\langle a\rangle $ , we get $\langle a^2\rangle \lhd X$ , which is a contradiction. Suppose that $\langle {\overline c}\rangle \lhd {\overline X}$ . Then, ${\overline X}/C_{{\overline X}}(\langle {\overline c}\rangle )\le {\mathrm {Aut\,}}(\langle {\overline c}\rangle )$ which is abelian and so ${\overline X}'\le C_{{\overline X}}(\langle {\overline c}\rangle )$ . Then, ${\overline a}^2\in {\overline G}'\leq {\overline X}'\leq C_{{\overline X}}(\langle {\overline c}\rangle )$ , that is, $[a^2, c]\in \langle a_0\rangle $ , which implies $\langle a^2, a_0\rangle \lhd X$ , and again we get $\langle a^2\rangle \lhd X$ , which is a contradiction. Therefore, we have $1\ne \langle {\overline c}\rangle _{{\overline X}}=\langle \overline {c_0}\rangle \lneqq \langle {\overline c}\rangle $ .

Consider $X_1=G\langle c_0\rangle <X$ . By the minimality of X, we get $\langle a^2\rangle \lhd X_1$ , which implies that $\langle c_0\rangle $ normalizes $\langle a^2\rangle $ . Reset

$$ \begin{align*}K=\langle a_0\rangle \rtimes\langle c_0\rangle ,\,{\overline X}=X/K={\overline G}\langle {\overline c}\rangle \quad\mathrm{{and}}\quad H=\langle a^2\rangle \rtimes\langle c_0\rangle .\end{align*} $$

If ${\mathrm {o}}(a_0)<{\mathrm {o}}(c_0)$ , then $1\neq \langle c_0^j\rangle =Z(K)\lhd X$ is, for some j, a contradiction. Therefore, $1< {\mathrm {o}}(c_0)\le {\mathrm {o}}(a_0)$ . Note that K is either a Frobenius group or ${\mathbb {Z}}_p^2$ . Thus, we have the following two cases.

Case (1): $K=\langle a_0\rangle \rtimes \langle c_0\rangle \cong {\mathbb {Z}}_p\rtimes {\mathbb {Z}}_r$ , a Frobenius group, where $r\ge 2$ and $r\mid (p-1)$ . In this case, p is odd, which implies $a_0\in \langle a^2\rangle $ . Set ${\overline X}=X/K $ . By the minimality of X, we have $H/K=\langle {\overline a}^2\rangle \lhd {\overline X}$ , that is, $H:=\langle a^2\rangle \rtimes \langle c_0\rangle \lhd X$ . Since $K\lhd X$ , we know that $\langle a^2\rangle /\langle a_0\rangle $ and $\langle c_0\rangle \langle a_0\rangle /\langle a_0\rangle $ are normal in $H/\langle a_0\rangle $ . Then, $[a^2, c_0]\le \langle a_0\rangle $ . So

$$ \begin{align*}H=\langle a^2,c_0\mid a^n=c_0^r=1,\,(a^2)^{c_0}=a^2a_0^j\rangle .\end{align*} $$

Let $P\in {\mathrm {Syl}}_p(H)$ . Actually, $P\leq \langle a^2\rangle $ . Then, $P{ \, \mathrm {char}\,} H\lhd X$ so that $P\leq \langle a\rangle _X$ . Clearly, $Z(H)=\langle a^{2p}\rangle $ . Then, $\langle a^{2p}\rangle \leq \langle a\rangle _X$ . Noting that $\langle a^{2p},P\rangle =\langle a^{2p}, a^{n/p^k}\rangle =\langle a^2\rangle $ , where $p^k|| n$ , we get $a^2\in \langle a \rangle _X$ , which is a contradiction.

Case (2): $K=\langle a_0\rangle \times \langle c_0\rangle \cong {\mathbb {Z}}_p^2$ . In this case, we claim that $a_0\in \langle a^2\rangle $ . Arguing by contradiction, assume that $a_0\notin \langle a^2\rangle $ . Then, $p=2$ and n is odd. In ${\overline X}$ , we know that ${\overline H}\lhd {\overline X}$ , which implies $\langle a_0\rangle H\lhd X$ . Noting $\langle a^2\rangle $ is a $2'$ -Hall subgroup of $\langle a_0\rangle H$ , we have $\langle a^2\rangle { \, \mathrm {char}\,} \langle a_0\rangle H\lhd X,$ which implies $a^2\in \langle a\rangle _X$ , and that is a contradiction. Therefore, $a_0\in \langle a^2\rangle $ , which implies that p is an odd prime. With the same reason as that in Case (1), we have $H\lhd X$ . Let $H_1$ be the $p'-$ Hall subgroup of H. We get that $H_1$ is also the $p'$ -Hall subgroup of $\langle a^2\rangle $ as ${\mathrm {o}}(c_0)=p$ . Then, $H_1{ \, \mathrm {char}\,} H\lhd X$ , which implies $H_1\leq \langle a\rangle _X$ . We claim that $H_1=1$ . Arguing by contradiction, assume that $H_1\neq 1$ . Then there exists an element $a_1$ of prime order q in $H_1$ , where $q < p$ , as p is maximal. Consider ${\overline X}:=X/\langle a_1\rangle ={\overline G}\langle c\rangle $ . Similarly, we have $1\ne \langle {\overline c}\rangle _{{\overline X}}:=\langle \overline {c_1}\rangle \lneqq {\overline C}$ and $H_0:=\langle a^2\rangle \rtimes \langle c_1\rangle \lhd X$ . Let $P\in {\mathrm {Syl}}_p(H_0)$ . Then, $P{ \, \mathrm {char}\,} H$ and so $P\lhd X$ , which implies $P\leq \langle a\rangle _X$ . Noting $\langle H_1,P\rangle =\langle a^2\rangle $ , we therefore get $a^2\in \langle a\rangle _X$ , which is a contradiction. So $H_1=1$ , which means that G is $Q_{4p^k}$ where p is an odd prime as G is not a 2-group.

Step 2: Show that the possible values of m are $pq^e$ for a prime q (may be equal to p) and an integer e. Arguing by contradiction, assume that $m=pq^em_1$ , where $m_1\ne 1$ and $q\nmid m_1$ . Reset $a_1=a^n$ , the involution of $\langle a\rangle $ . Suppose that $a_1\in \langle a\rangle _X$ . Let $\langle a_1\rangle \rtimes \langle c_1\rangle $ be the core of $\langle a_1,c\rangle $ in X, noting $\langle a_1\rangle \lhd X$ . Since $\langle c_1^2\rangle \lhd X$ and $\langle c\rangle _X=1$ , we get $c_1^2=1$ . Consider ${\overline X}=X/\langle a_1\rangle \langle c_1\rangle $ . Since ${\overline G}\cong D_{2p^k}$ is a dihedral group, by Proposition 2.11, we get $\langle {\overline a}^2\rangle =\langle {\overline a}\rangle \lhd {\overline X}$ , which implies $\langle a\rangle \rtimes \langle c_1\rangle \lhd X$ . Then, $\langle a^2\rangle \lhd X$ is a contradiction. So in what follows (including Step 3), we assume that $a_1\notin \langle a\rangle _X$ , which implies that $\langle a\rangle _X$ is a p-group.

Recall $H=\langle a^2\rangle \rtimes \langle c_0\rangle $ . Since $H\lhd X$ and $b^2=a_1$ , we get ${\overline X}=X/H=\langle {\overline b}\rangle \langle {\overline c}\rangle $ and $\langle {\overline b}\rangle \cong {\mathbb {Z}}_4$ . By considering the permutation representation of ${\overline X}$ on the cosets $[{\overline X}:\langle {\overline c}\rangle ]$ of size 4, we know that $\langle {\overline c}^2\rangle \lhd {\overline X}$ . So $\langle b, c^2, H\rangle \le X$ , that is, $X_1:=\langle b, c^2, H\rangle =\langle a,b\rangle \langle c^2\rangle =G\langle c^2\rangle \le X$ .

First, suppose that m (= $o(c)$ ) is even. Then, $[X:X_1]=2$ . Let $\langle c_2\rangle $ be the Sylow 2-subgroup of $\langle c\rangle $ . By induction on $X_1$ , $\langle a^2\rangle \lhd X_1$ and, in particular, $\langle c^2\rangle $ normalizes $\langle a^2\rangle $ . By Proposition 2.6, we know that $\langle a^ib\rangle \langle c_2\rangle $ is a Sylow 2-subgroup of X for some i. Then we get $X_2:=H(\langle a^ib\rangle \langle c_2\rangle )=\langle a,b\rangle \langle c_0, c_2\rangle <X.$ By the minimality of X again, $\langle a^2\rangle \lhd X_2$ , which implies $\langle c_2\rangle $ normalizes $\langle a^2\rangle $ . Since both $\langle c_2\rangle $ and $\langle c^2\rangle $ normalize $\langle a^2\rangle $ and $\langle c_2, c^2\rangle =\langle c\rangle $ , we get $\langle a^2\rangle \lhd X$ , which is a contradiction.

Second, suppose that m is odd. Then both q and $m_1$ are odd, so that $X=X_1=((\langle a^2\rangle \rtimes \langle c_0\rangle ).\langle c\rangle )\rtimes \langle b\rangle $ . By induction on $X_3:=\langle H, c^{{m}/{m_1}}\rangle =\langle a,b\rangle \langle c_0, c^{{m}/{m_1}}\rangle <X$ and $X_4:=\langle H, c^{{m}/{pq^e}}\rangle =\langle a,b\rangle \langle c^{{m}/{pq^e}}\rangle <X$ , we respectively get both $\langle c^{{m}/{m_1}}\rangle $ and $\langle c^{{m}/{pq^e}}\rangle $ normalize $\langle a^2\rangle $ . Noting $\langle c^{{m}/{m_1}},c^{{m}/{pq^e}}\rangle =\langle c\rangle $ , we get $\langle a^2\rangle \lhd X$ , which is a contradiction again.

Step 3: Exclude the case $m=pq^e$ for a prime q and an integer e. Recall that $a_1=a^n$ is the unique involution in G; $\langle a_0\rangle $ is a normal subgroup of order p in X; $K=\langle a_0\rangle \times \langle c_0\rangle =(\langle a_0\rangle \langle c\rangle )_X\cong {\mathbb {Z}}_p^2$ ; $H=\langle a^2\rangle \rtimes \langle c_0\rangle \lhd X$ (by the induction hypothesis) and $X=((H.\langle c\rangle ).\langle a_1\rangle ).\langle b\rangle $ . Suppose that $e=0$ . Then, consider $S:=G\cap G^c$ . Since $a_1\notin \langle a\rangle _X\leq S$ , we get that S is also a p-group, $S\leq \langle a\rangle _X$ and $|S|\leq p^{k-1}$ . However, noting $16p^{k+1}\leq {|G||G^c|}/{|S|} < |X|=4p^{k+1}$ , we get a contradiction. Suppose that $q=2$ and $e=1,2$ . Then using the same argument as above, we get a contradiction again. So in what follows, we assume that either q is odd and $e\ge 1$ ; or $q=2$ and $e\geq 3$ .

Since H is a p-group and $\langle a^{2p}\rangle =\mho _1(H){ \, \mathrm {char}\,} H\lhd X$ , we get $\langle a^{2p}\rangle \lhd X$ . Set $X_5:=(H.\langle c^q\rangle )\rtimes \langle b\rangle =\langle a,b\rangle \langle c^q\rangle < X$ . By the induction hypothesis on $X_5$ , we get $\langle a^2\rangle \lhd X_5$ , that is, $X_5=(\langle a^2\rangle \rtimes \langle c^q\rangle )\rtimes \langle b\rangle $ . Clearly, $\langle a^2\rangle =G'\le X_5'\le \langle a^2, c^q\rangle $ . So set $X_5'=\langle a^2, c_3\rangle $ for some $c_3\in \langle c^q\rangle $ . By Proposition 2.2, both $X/C_{X}(\langle a^{2p}\rangle )$ and $X_5/C_{X_5}(\langle a^2\rangle )$ are abelian, which implies that $X'\leq C_X(\langle a^{2p}\rangle )$ and $X_5'\leq C_{X_5}(\langle a^2\rangle )$ . Then, $X_5'$ is abelian as $\langle a^2\rangle \leq X_5'=\langle a^2, c_3\rangle $ . The $p'$ -Hall subgroup of $X_5'$ is normal, which contradicts $\langle c^q\rangle _{X_5}=1$ , meaning that $X_5'$ is an abelian p-group. Set $L:=H\rtimes \langle a_1\rangle =\langle a\rangle \langle c_0\rangle < X_5$ .

We claim that $L\ntrianglelefteq X$ . Arguing by contradiction, assume that $L\lhd X$ . If H is abelian, then we get that either $\langle a^2\rangle =Z(L){ \, \mathrm {char}\,} L\lhd X$ , which is a contradiction; or L is abelian, forcing $\langle a_1\rangle { \, \mathrm {char}\,} L\lhd X$ , which is a contradiction again. Therefore, H is nonabelian. Note that $X_5'=\langle a^2, c_3\rangle $ for $c_3\in \langle c^q\rangle $ . If $c_3\ne 1$ , then $c_0\in \langle c_3\rangle \leq X_5'$ as ${\mathrm {o}}(c_0)=p$ , which implies that $H=\langle a^2,c_0\rangle $ is abelian, which is a contradiction. Therefore, $X_5'=\langle a^2\rangle $ , which implies $L=\langle a\rangle \rtimes \langle c_0\rangle $ . Note that $\langle a_1\rangle { \, \mathrm {char}\,} L\lhd X$ . Thus, we get $\langle a_1\rangle \lhd X$ , which contradicts $a_1\notin \langle a\rangle _X$ . Therefore, $L\ntrianglelefteq X$ , which implies that $\langle {\overline c}\rangle $ does not normalize $\langle \overline {a_1}\rangle $ in ${\overline X}=X/H$ .

In ${\overline X}=X/H=(\langle {\overline c}\rangle \rtimes \langle \overline {a_1}\rangle ).\langle {\overline b}\rangle $ , we get that either ${\overline c}^{\overline {a_1}}={\overline c}^{-1}$ if q is odd; or ${\overline c}^{\overline {a_1}}$ is either ${\overline c}^{-1}$ or ${\overline c}^{\pm 1+2^{e-1}}$ if $q=2$ . Then we divide the proof into the following two cases.

Case (1): q is an odd prime. In this case, q is odd. Since ${\overline c}^{\overline {a_1}}={\overline c}^{-1}$ in ${\overline X}=X/H$ , we get $\langle a^2, c^{qp}\rangle \leq X_5'\leq \langle a^2\rangle \langle c^q\rangle $ . Note that $X_5'$ is the abelian p-group. Thus, either $q\ne p$ and $e=1$ ; or $q=p$ . Suppose that $q\ne p$ and $e=1$ , that is, ${\mathrm {o}}(c)=pq$ . Consider $M=\langle a\rangle \langle c\rangle \lhd X$ . Then, by Proportion 2.4, $M'$ is abelian. Note that $\langle c\rangle _X=1$ and $G_X$ is the p-group. Thus, $M'$ is an abelian p-group with the same argument as the case of $X_5'$ . Noting that $\langle a_1\rangle \langle c^p\rangle $ is the $p'$ -Hall subgroup of M, we get $[a_1,c^p]\in \langle a_1\rangle \langle c^p\rangle \cap M'=1$ , which implies ${\overline c}^{\overline {a_1}}={\overline c}$ in ${\overline X}=X/H$ , which is a contradiction. So in what follows, we assume that $q=p$ , that is, ${\mathrm {o}}(c)=p^{e+1}$ . Note that ${\overline c}^{\overline {a_1}}={\overline c}^{-1}$ in ${\overline X}=X/H$ and $\langle a^2\rangle \leq X_5'$ . Thus, $X_5'$ is either $\langle a^2\rangle \langle c^p\rangle $ or $\langle a^2\rangle $ , noting $X_5'=\langle a^2\rangle $ only happens when $e=1$ .

Suppose that $X_5'=\langle a^2\rangle \langle c^p\rangle $ . Since $H=\langle a^2\rangle \rtimes \langle c_0\rangle \leq X_5'\leq C_{X_5}(\langle a^2\rangle )$ , we get that $H=\langle a^2\rangle \times \langle c_0\rangle $ is abelian. Note that both $\langle a^2\rangle $ and $\langle c\rangle $ are p-groups and $X=(H.\langle c\rangle )\rtimes \langle b\rangle $ . Set $(a^2)^{c}=a^{2s}c_0^{t}$ and $c^b=a^{2u}c^v$ , where $s\equiv 1\pmod {\ p}$ and $p\nmid v$ . Then for an integer w, we get $(a^2)^{c^w}=a^{2x_1}c_0^{wt}$ and $c_0^b=a^{x_2}c_0^{v}$ for some integers $x_1$ and $x_2$ . Since $((a^2)^{c})^b=(a^{2s}c_0^{t})^b$ , there exist some integers x and y such that

$$ \begin{align*}((a^2)^{c})^b=(a^{-2})^{c^v}=a^{x}c_0^{-vt}\quad\mathrm{{and}}\quad ((a^2)^sc_0^{t})^b=a^yc_0^{vt},\end{align*} $$

which gives $t\equiv 0\,({\mathrm {mod }}\ p)$ . Then, $\langle a^2\rangle \lhd X$ , which is a contradiction.

Suppose that $X_5'=\langle a^2\rangle $ . Then, ${\mathrm {o}}(c)=p^2$ , $c_0=c^p$ and $X_5=\langle a,b\rangle \langle c_0\rangle $ . Since $X_5'\leq G\leq X_5$ , we get that $\langle a_1\rangle { \, \mathrm {char}\,} G\lhd X$ , which implies that $\langle a_1\rangle \leq Z(X_5)$ as $a_1$ is an involution. Thus, $[c_0,a_1]=1$ . Set $c^{a_1}=a^xc^{-1+yp}$ as ${\overline c}^{\overline {a_1}}={\overline c}^{-1}$ in ${\overline X}=X/H$ . Then,

$$ \begin{align*}c=c^{a_1^2}=(a^xc^{-1}c_0^y)^{a_1}=a^x(a^xc^{-1}c_0^y)^{-1}c_0^y =a^xc_0^{-y}ca^{-x}c_0^y=a^xc^{1-yp}a^{-x}c^{yp},\end{align*} $$

which implies $(a^x)^{c^{1-yp}}=a^x$ . Then, $[a^x,c]=1$ . Note that $c_0=c^p$ and $c_0=c_0^{a_1}=(c^{a_1})^p=a^{x}c^{-p}$ for some x. Thus, we get $c_0^2=1$ , which contradicts ${\mathrm {o}}(c_0)=p$ .

Case (2): $q=2$ . In this case, we know that ${\mathrm {o}}(c)\ge 8p$ , $X_5=(\langle a^2\rangle \langle c^2\rangle )\rtimes \langle b\rangle \lhd X$ and ${\overline c}^{\overline {a_1}}$ is either ${\overline c}^{-1}$ or ${\overline c}^{\pm 1+2^{e-1}}$ in $X/H$ .

We show ${\overline c}^{\overline {a_1}}={\overline c}^{1+2^{e-1}}$ in $X/H$ . Since $\langle a^2\rangle \leq X_5'{ \, \mathrm {char}\,} X_5\lhd X$ and $X_5'$ is the abelian p-group, we get $X_5'=H$ , which implies that H is abelian. By Proposition 2.6, we get that both $\langle a^{i_1}b\rangle \langle c^p\rangle $ and $\langle a^{i_2}b\rangle \langle c^{2p}\rangle $ are Sylow 2-subgroups of X and $X_5$ for some $i_1$ and $i_2$ , respectively. Then, $[c^{2p},a^{i_2}b]\in X_5'\cap \langle c^{2p}\rangle \langle a^{i_2}b\rangle =1$ , which implies that $\langle c^{2p}\rangle \langle a^{i_2}b\rangle $ is abelian. So, $[c^{2p},a_1]=1$ . Since ${\overline c}^{\overline {a_1}}$ is either ${\overline c}^{-1}$ or ${\overline c}^{\pm 1+2^{e-1}}$ in $X/H$ , we get ${\overline c}^{\overline {a_1}}={\overline c}^{1+2^{e-1}}$ in $X/H$ . Since $\langle a^{i_1}b\rangle \langle c^p\rangle =(\langle c^p\rangle \rtimes \langle a_1\rangle ).\langle a^{i_1}b\rangle $ , we know $(c^p)^{a_1}=c^{p+2^{e-1}p}$ , which implies $c^{2^{e-1}p}\in M'$ .

Noting that $\langle c^p\rangle \rtimes \langle a_1\rangle =\langle a_1,c^p|a_1^2=c^{2^ep}=1,\,(c^p)^{a_1}=c^{(1+2^{e-1})p}\rangle $ , there are only three involutions in $\langle c^p\rangle \rtimes \langle a_1\rangle $ : $a_1,c^{2^{e-1}p}$ and $a_1c^{2^{e-1}p}$ . By Proposition 2.5, we know that $\langle c^{2p}\rangle \lhd \langle a^{i_1}b\rangle \langle c^q\rangle $ . Recall that $M=\langle a\rangle \langle c\rangle $ . By Proposition 2.4, $M'$ is abelian. Let $M_2$ be the Sylow 2-subgroup of $M'$ . Note that $M_2{ \, \mathrm {char}\,} M'{ \, \mathrm {char}\,} M\lhd X$ , $c^{2^{e-1}p}\in M'$ , $\langle c\rangle _X=1$ and $a_1$ is an involution. Thus, we get $M_2\cong {\mathbb {Z}}_2^2$ , that is, $M_2=\langle c^{2^{e-1}p}\rangle \times \langle a_1\rangle $ . Consider $HM_2\leq X$ . Since $M_2\lhd X$ , $H\lhd X$ , $H\cap M_2=1$ and p is odd prime, we get $HM_2=H\times M_2\lhd X$ , which implies that a normalizes $\langle c^{2^{e-1}p}\rangle $ . Recall that $\langle c^{2p}\rangle \langle a^{i_2}b\rangle $ is abelian. Then, we get $[c^{2p},a^{i_2}b]=1$ . Since $\langle c^{2^{e-1}p}\rangle \leq \langle c^{2p}\rangle $ , we get that b also normalizes $\langle c^{2^{e-1}p}\rangle $ . Since $X=\langle a,b,c\rangle $ , we get $\langle c^{2^{e-1}p}\rangle \lhd X$ , which is a contradiction.

Proof. (1) Suppose that $\langle c\rangle _X=1$ . For the five cases of Theorem 1.1, we know that $M_X$ is $M,\,\langle a^2\rangle \langle c^2\rangle ,\,\langle a^2\rangle \langle c^3\rangle ,\,\langle a^3\rangle \langle c^4\rangle $ or $\langle a^4\rangle \langle c^3\rangle $ . Set $M_X=\langle a^i\rangle \langle c^j\rangle $ , which is one of the above five cases, and $X_1=GM_X=\langle a,b\rangle \langle c^j\rangle $ , where $G\in \{ Q,\,D\}$ . Then, $\langle c^j\rangle _{X_1}=1$ . By Proposition 2.11 and Lemma 4.2, we get $\langle a^2\rangle \lhd X_1$ , that is, $c^j$ normalizes $\langle a^2\rangle $ , and so $M_X\cap \langle a^2\rangle \lhd M_X$ .

(2) Suppose $\langle c_1\rangle :=\langle c\rangle _X\ne 1$ . Then, by Proposition 2.2, we get $\langle c\rangle \leq C_X(\langle c_1\rangle )\lhd X$ and ${\overline X}=X/C_X(\langle c_1\rangle )=\langle {\overline a},{\overline b}\rangle $ is abelian. This implies $\langle {\overline a},{\overline b}\rangle \lessapprox {\mathbb {Z}}_2\times {\mathbb {Z}}_2$ or ${\mathbb {Z}}_4$ . Therefore, $\langle a^2, c\rangle \leq C_X(\langle c_1\rangle )$ . Therefore, $|X:C_X(\langle c_1\rangle )|\le 4$ .

Proof. $X=(\langle a\rangle \rtimes \langle c\rangle ).\langle b\rangle ,$ and so we may write $a^c=a^i$ and $c^b=a^kc^j$ . If $j=1$ , then $G\lhd X$ . So assume $j\ne 1$ . Since $b^2=a^n$ ( $o(a^n)=1$ or 2 if $G=D$ or $G=Q$ , respectively) and $\langle a^n\rangle \lhd X$ , we get $a^n\in Z(X)$ , which implies $c=c^{b^2}$ . Then,

$$ \begin{align*}c=(c^b)^b=(a^kc^j)^b=a^{-k}(a^kc^j)^j=c^j(a^kc^j)^{j-1},\end{align*} $$

that is, $c^{1-j}=(a^kc^j)^{j-1}=(c^{j-1})^b,$ so that b normalizes $\langle c^{1-j}\rangle $ . Since ${\overline X}=X/C_X(\langle a\rangle )\le {\mathrm {Aut\,}}(\langle a\rangle )$ , which is abelian, we get ${\overline c} ={\overline c}^{{\overline b}}={\overline c}^j$ , that is, $c^{1-j}\le C_X(\langle a\rangle )$ so that $[c^{1-j}, a]=1$ . Thus, we get $\langle c^{1-j}\rangle \lhd X$ . It follows from $\langle c\rangle _X=1$ that $j=1$ , which is a contradiction.