1. Introduction
1.1. Background
Consider the following Schrödinger system
\begin{align}
\begin{cases}
\imath \partial_t \Psi_i +\Delta \Psi_i-a_i(x)\Psi_i + \sum_{j=1}^{d}\beta_{ij}|\Psi_j|^{p}|\Psi_i|^{p-2}\Psi_i=0, \\
\Psi_i=\Psi_i(x,t), ~i=1,\ldots,d,~x\in \mathbb{R}^N, ~t \gt 0,\\
\Psi_i(x,t)\rightarrow 0, \text{as}~ |x|\rightarrow +\infty, ~i=1,\ldots,d,
\end{cases}
\end{align} where
$\imath$ is the imaginary unit,
$a_i(x)$ are potential functions,
$\beta_{ii} \gt 0, i=1,\ldots,d$,
$\beta_{ij}=\beta_{ji}$ for i ≠ j. System (1.1) comes from many physical models; for example, it can be used to explain the coexistence of several different Bose–Einstein condensates (see [Reference Timmermans26]). In quantum mechanics, the solutions
$\Psi_i(i=1,\ldots,3)$ are the corresponding wave amplitudes, βii represent self-interactions within the same component, while βij
$(i\neq j)$ describe the strength and type of interactions between different components
$\Psi_i$ and
$\Psi_j$. Furthermore,
$\beta_{ij} \gt 0$ represents a cooperative interaction, while, when
$\beta_{ij} \lt 0$, the interaction is competitive.
To obtain standing wave solutions of system (1.1), set
$\Psi_i(x,t)=e^{\imath \lambda_{i}t}u_i(x)$. System (1.1) is reduced to the following elliptic system
\begin{align}
\begin{cases}
-\Delta u_i+V_{i}(x)u_i=\sum\limits_{j=1}^{d} \beta_{ij}|u_j|^{p}|u_i|^{p-2}u_i, ~i=1,\ldots,d, ~x\in \mathbb{R}^N,\\
u_i(x)\rightarrow 0, \text{as}~ |x|\rightarrow +\infty, ~i=1,\ldots,d,
\end{cases}
\end{align} where
$V_i(x)=a_i(x)+\lambda_i$,
$d\geq 2$. For the subcritical case
$1 \lt p \lt 2^*/2$,
$2^*=\frac{2N}{N-2}$,
$N\geq 3$, and constant positive potentials
$V_{i}(x)=\lambda_i \gt 0$ for any
$i=1,\ldots, d$, the existence of solutions to (1.2) has been studied intensively in the last two decades with d = 2, see [Reference Abdellaoui, Felli and Peral1–Reference Bartsch, Dancer and Wang3], [Reference Maia, Montefusco and Pellacci14, Reference Mandel15, Reference Sirakov19, Reference Terracini and Verzini25, Reference Wei and Weth27] and references therein. In this situation, there is only one interaction constant
$\beta=\beta_{12}=\beta_{21}$. For an arbitrary number of equations
$d\geq 3$, the problem becomes more complicated due to the possible occurrence of simultaneous cooperation and competition, that is, the existence of at least two pairs,
$(i_1,j_1)$ and
$(i_2,j_2)$, such that
$i_1\neq j_1, i_2\neq j_2$,
$\beta_{i_1 j_1} \gt 0$ and
$\beta_{i_2 j_2} \lt 0$; the existence of solutions to (1.2) has attracted a lot of attention, see for example [Reference Byeon, Lee and Wang4, Reference Byeon, Sato and Wang5, Reference Correia, Oliveira and Tavares9], [Reference Lin and Wei12, Reference Peng, Wang and Wang17, Reference Sato and Wang18, Reference Soave20, Reference Soave and Tavares21] and references therein.
1.2. Motivation
For the critical case
$2p=2^*$ and
$N\geq 3$, if the potentials are constant
$V_{i}(x)=\lambda_i$ for any
$i=1,\ldots, d$, by letting
$\mathbf{u}=(u_1,\ldots,u_d)$ be any solution of (1.2), ones deduce from Pohozaev identity that
$\int_{\mathbb{R}^N}\sum_{i=1}^d \lambda_i u_i^2=0$, which yields that, if
$\lambda_1,\ldots, \lambda_d\neq 0$, then
$u_1,\ldots, u_d\equiv 0$ for any
$i=1,\ldots, d$. Next, we consider the case where
$V_{i}(x)$ is a nonconstant function for any
$i=1,\ldots, d$; in particular, consider the case when
$V_i(x)=-\frac{\lambda_{i}}{|x|^2}$ for any
$i=1,\ldots, d$. When
$V_i(x)=-\frac{\lambda_{i}}{|x|^2}$, d = 2,
$2p=2^*$ and
$N\geq 3$, in [Reference Abdellaoui, Felli and Peral1], the authors proved the existence and nonexistence of ground state solution to (1.2) with more general powers by using the concentration-compactness principle. Subsequently, Chen and Zou [Reference Chen and Zou8] obtained the existence of least energy positive solutions to (1.2) with
$V_i(x)=-\frac{\lambda_{i}}{|x|^2}$ and more general powers by complicated analysis.
To the best of our knowledge, less considered for system (1.2) is the critical case (
$2p=2^*$) with
$d\geq3$ and
$V_i(x)=-\frac{\lambda_{i}}{|x|^2}$,
$i=1,\ldots, d$. In this aspect, we refer to [Reference Guo, Luo and Zou10] for ground state solutions of (1.2). A natural question is
whether system (1.2) with
$V_i(x)=-\frac{\lambda_{i}}{|x|^2}$ admits a least energy positive solution when
$d\geq3$ (the high energy solution).
Here we refer to (1.11) for the definition of least energy positive solutions and point out that ground state solutions can be semi-trivial. The main purpose of the present paper is to make a contribution in this direction. Precisely, in what follows, we consider the coupled elliptic systems as follows
\begin{align}
\begin{cases}
-\Delta u_{i}-\frac{\lambda_{i}}{|x|^2}u_{i}=|u_i|^{2^*-2}u_i+\sum_{j\neq i}^{3}\beta_{ij}|u_{j}|^{\frac{2^*}{2}}|u_i|^{\frac{2^*}{2}-2}u_i, ~x\in \mathbb{R}^N, ~i=1,2,3,\\
u_i\in D^{1,2}(\mathbb{R}^N), i=1,2,3,
\end{cases}
\end{align} where
$2^*=\frac{2N}{N-2}$,
$N\geq3$,
$\lambda_i\in (0,\Lambda_N), \Lambda_N:= \frac{(N-2)^2}{4}$,
$\beta_{ii}=1, i=1,2,3$,
$\beta_{ij}=\beta_{ji}$ for i ≠ j, and
$D^{1,2}(\mathbb{R}^N):= \{u\in L^{2^*}(\mathbb{R}^N): |\nabla u|\in L^2(\mathbb{R}^N)\}$ with norm
\begin{align*}
\|u\|^2:=\int_{\mathbb{R}^N}|\nabla u|^2.
\end{align*}By Hardy’s inequality,
\begin{align*}
\Lambda_N \int_{\mathbb{R}^N}\frac{v^{2}}{|x|^2} \leq \int_{\mathbb{R}^N}|\nabla v|^2, \quad \forall v\in D^{1,2}(\mathbb{R}^N),
\end{align*} we have that, for
$\lambda_i\in (0,\Lambda_N)$,
\begin{align*}
\|u\|_i^2:=\int_{\mathbb{R}^N}\left(|\nabla u|^2-\frac{\lambda_{i}}{|x|^2} u^{2}\right)
\end{align*} are equivalent norms to
$\|\cdot\|$, for
$i=1,2,3$. By [Reference Terracini24], the following problem
\begin{align}
\begin{cases}
-\Delta v-\frac{\lambda_{i}}{|x|^2}v=v^{2^*-1}, ~ x\in \mathbb{R}^N,\\
v\in D^{1,2}(\mathbb{R}^N), ~ v \gt 0 \text{ in}~ \mathbb{R}^N\backslash \{0\}
\end{cases}
\end{align}has exactly one-dimensional C 2 positive solutions given by
\begin{align}
\mathcal{Z}_i=\left\{z^i_\mu(x)=\mu^{-\frac{N-2}{2}} z^i_1(\frac{x}{\mu}), \mu \gt 0 \right\},
~\text{where}~
z^i_1(x)=\frac{C}{|x|^a\left( 1+ |x|^{2-\frac{4a}{N-2}}\right)^{\frac{N-2}{2}}},
\end{align}
$a=\frac{N-2}{2}-\sqrt{\frac{(N-2)^2}{4}-\lambda_i}$ and C > 0 is a constant. Moreover, all positive solutions of (1.4) satisfy
\begin{align}
S_i:= \inf_{v\in D^{1,2}(\mathbb{R}^N)\backslash \{0\}} \frac{\|v\|_i^2}{|v|^2_{2^*}}=\frac{\|\omega_i\|_i^2}{|\omega_i|^2_{2^*}}=\left(1-\frac{4\lambda_i}{(N-2)^2}
\right)^{\frac{N-1}{N}}\mathcal{S},
\end{align}and
\begin{align}
A_i=I_i(\omega_i):=\frac{1}{2}\|\omega_i\|_i^2-\frac{1}{2^*}|\omega_i|_{2^*}^{2^*}
=\frac{1}{N}S_i^{\frac{N}{2}} \lt \frac{1}{N}\mathcal{S}^{\frac{N}{2}},
\end{align} where
$\mathcal{S}$ is the sharp constant of the embedding
$D^{1,2}(\mathbb{R}^N)\hookrightarrow L^{2^*}(\mathbb{R}^N)$,
\begin{align*}
\mathcal{S}\left( \int_{\mathbb{R}^N} |v|^{2^*}\right)^{\frac{2}{2^*}}\leq \int_{\mathbb{R}^N} |\nabla v|^2.
\end{align*}Moreover, we have
\begin{align}
\int_{\mathbb{R}^N}\left(|\nabla u|^{2}-\frac{\lambda_{i}}{|x|^2} u^{2}\right)\geq
(NA_i)^{\frac{2}{N}}\left(\int_{\mathbb{R}^N}|u|^{2^*}\right)^{\frac{2}{2^*}}, ~\forall u\in D^{1,2}(\mathbb{R}^N).
\end{align} Let
$\mathbf{D}=D^{1,2}(\mathbb{R}^N;\mathbb{R}^3)$. It is well known that any solution of (1.3) corresponds to a critical point of the C 1 functional
$I:\mathbf{D} \rightarrow \mathbb{R}$ given by
\begin{align}
I(\mathbf{u}) & =\frac{1}{2}\sum_{i=1}^3\int_{\mathbb{R}^N}\left(|\nabla u|^{2}-\frac{\lambda_{i}}{|x|^2} u_i^{2}\right)-\frac{1}{2^*}\sum_{i=1}^3\int_{\mathbb{R}^N}|u_i|^{2^*}
-\frac{1}{2^*}\sum_{i\neq j}^3\int_{\mathbb{R}^N}\beta_{ij} |u_i|^{\frac{2^*}{2}}|u_j|^{\frac{2^*}{2}}.
\end{align}Definition 1.1.
A solution
$\mathbf{u}=(u_1,u_2,u_3)$ of (1.3) is called trivial if all its components are zero, i.e.,
$u_1,u_2,u_3\equiv 0$. A solution is called semi-trivial if there exists at least one (but not all) vanishing component. A solution u is called fully nontrivial if all of its components
$u_1,u_2,u_3$ are nontrivial. A solution u is positive (resp. nonnegative), if
$u_i \gt 0$ (resp.
$u_i\geq 0$) for every
$i=1,2,3$.
In this paper, we focus on the existence of ground state solutions, achieving the following ground state level
and the existence of least energy positive solutions, which attain the least energy positive level
Define
and
Let
\begin{align}
A:= \inf_{\mathbf{u}\in \mathcal{M}}I(\mathbf{u})=\inf _{(u_1,u_2,u_3)\in \mathcal{M}}\frac{1}{N}\sum_{i=1}^3\|u_i\|_i^2, \quad
~\mathcal{A}:= \inf_{\mathbf{u}\in \mathcal{N}}I(\mathbf{u}),
\end{align} it is standard to show that
$0 \lt \mathcal{A}\leq A$. Next, we introduce the main results of this paper.
1.3. Main results
Our first result is related to the existence of ground state solutions to (1.3). Before proceeding, we introduce the following condition about the coupled coefficient βij
\begin{align}
\sum_{i,j=1}^{3}\int_{\Omega}\beta_{ij}|\varphi_i|^{\frac{2^*}{2}}|\varphi_j|^{\frac{2^*}{2}} \gt 0,\ ~~ \forall (\varphi_{1},\varphi_{2}, \varphi_{3})\in \mathbf{D}\setminus \{\mathbf{0}\},
\end{align} where we assume that
$\beta_{ii}=1, ~i=1,2,3$. Note that the condition (1.13) is true if
$\beta_{ij}\geq 0$ for any i ≠ j. In fact, the condition (1.13) holds if the matrix
$\mathcal{B}:=(\beta_{ij})_{1\leq i,j\leq d}$ is positively definite.
(1) Assume that
$N\geq 3$,
$\lambda_i\in (0, \Lambda_N)$ and the condition (1.13) holds, then
$\mathcal{A}$ is attained by a nonnegative solution u. Moreover, u is a ground state solution of (1.3).(2) Assume that N = 4,
$\lambda_i\in (0, \Lambda_4)$ and
\begin{align*}
0 \lt \beta_{ij} \lt \frac{\sqrt{2}}{2}, \quad \text{for}~\text{any}~ i\neq j,
\end{align*}then all ground state solutions of system (1.3) are semi-trivial, i.e.,
${\mathcal{A}} \lt A$.(3) Assume that N = 3,
$\lambda_i\in (0, \Lambda_3)$ and
\begin{align*}
0 \lt \beta_{ij} \lt 1, \quad \text{for}~\text{any}~ i\neq j,
\end{align*}then all ground state solutions of system (1.3) are semi-trivial, i.e.,
${\mathcal{A}} \lt A$.(4) Assume that
$N\geq 5$,
$\lambda_i\in (0, \Lambda_N)$ and
(1.14)
\begin{align}
\beta_{ij}\geq 0, ~\text{for}~\text{any}~ i\neq j,
\end{align}then (1.3) has a positive ground state solution u. Moreover, u is also a least energy positive solution in this case, and
$\mathcal{A}=A$.
Remark 1.2.
(1) In [Reference Guo, Luo and Zou10], a similar result to theorem 1.1-(1) was obtained only for the purely cooperative case (
$\beta_{ij}\geq 0$, for any i ≠ j). Here, theorem 1.1-(1) holds under the condition (1.13), which includes not only the purely cooperative case but also the case of simultaneous cooperation and competition.(2) When
$N=3,4$, theorem 1.1-(2) and (3) show that all ground state solutions of system (1.3) are semi-trivial for
$\beta_{ij}\geq 0$ small. However, when
$N\geq 5$, theorem 1.1-(4) states that system (1.3) has a positive ground state solution for any
$\beta_{ij}\geq 0$ (i ≠ j). Therefore, the structure of ground state solution to system (1.3) differs significantly between
$N=3, 4$ and higher dimensions
$N\geq 5$.(3) Although we only consider the case d = 3, theorem 1.1 is true for any number of equations
$d\geq 2$.
The proof of theorem 1.1-(2) and (3) is inspired from [Reference Correia, Oliveira and Tavares9, Theorem 1.7]; however, the assumption
$\beta_{ij}=b$ required therein is not needed in our proof. We highlight that there are technical differences between N = 3 and N = 4, please see the proof of theorem 1.1-(2) and (3) for more details.
The above theorem reveals that all ground state solutions of system (1.3) are semi-trivial for weakly cooperative case (
$\beta_{ij} \gt 0$ small, for any i ≠ j) when
$N=3,4$. A natural question is whether system (1.3) admits higher energy positive solutions for the weakly cooperative case when
$N=3,4$ and whether the level A is attained, where the level A is defined in (1.12). Here, we have the following results.
(1) Assume that N = 4,
$\lambda_i\in (0, \Lambda_4)$ and there exists
$\widetilde{\beta} \gt 0$ such that
(1.15)
\begin{align}
0\leq \beta_{ij} \lt \widetilde{\beta}, ~\text{for}~\text{any}~ i\neq j,
\end{align}then
$A=I(\mathbf{u})$ and system (1.3) has a least energy positive solution u.(2) Assume that N = 3,
$\lambda_i\in (0, \Lambda_3)$ and there exists
$\widehat{\beta} \gt 0$ such that
(1.16)
\begin{align}
0\leq \beta_{ij} \lt \widehat{\beta}, ~\text{for}~\text{any}~ i\neq j,
\end{align}then
$A=I(\mathbf{u})$ and system (1.3) has a least energy positive solution.
Remark 1.4.
The constant
$\widetilde{\beta}$ only depends on
$\mathcal{S}, \lambda_{i}, i=1,2, 3$, and the explicit expression for
$\widetilde{\beta}$ can be found in (3.69);
$\widehat{\beta}$ only depends on
$\mathcal{S}, \lambda_{i}, i=1,2, 3$, and the explicit expression can be found in (3.132).
Remark 1.5.
We mention that the Morse index of the ground state solution obtained in [Reference Guo, Luo and Zou10] is 1 when
$N=3,4$, while the Morse index of the least energy positive solution acquired in theorem 1.3 is 3. Therefore, the solution acquired in theorem 1.3 is different from the solution obtained in [Reference Guo, Luo and Zou10].
Remark 1.6.
For the two-component case, Chen and Zou [Reference Chen and Zou8] studied the following general system
\begin{align}
\begin{cases}
-\Delta u -\frac{\lambda_{1}}{|x|^2}u=|u|^{2^*-2}u+
\nu \alpha u^{\alpha-1}v^{\beta}, x \in \mathbb{R}^N,\\
-\Delta v -\frac{\lambda_{2}}{|x|^2}v=|v|^{2^*-2}u+\nu \beta u^{\alpha}v^{\beta-1}, ~x\in \mathbb{R}^N,\\
u, v \in D^{1,2} \, (\mathbb{R}^N), u, v \gt 0 \, \text{in}~ \mathbb{R}^N \backslash \{0\},
\end{cases}
\end{align} where
$N\geq 3$,
$\alpha+\beta=2^*$,
$2^*=2N/(N-2)$. In that paper, when N = 3 the authors proved that there exists a positive constant β 1 such that (1.17) has a least energy positive solution if
$\beta\in (0,\beta_1)$. The accurate definition of β 1 is not given in [Reference Chen and Zou8]. Here, we show that there exists
$\widehat{\beta} \gt 0$ such that (1.17) with
$\alpha=\beta=3$ has a least energy positive solution if
$\nu\in (0,\widehat{\beta})$ (see corollary 3.8) and give a precise definition of
$\widehat{\beta}$ by a different method.
The proof of theorem 1.3 (1) is inspired by the one of [Reference Chen and Zou8, Theorem 1.2]. However, the method in [Reference Chen and Zou8] cannot be used here directly due to the presence of multi-components, and we need some crucial modifications for our proof. Firstly, we need to establish new estimates (see lemma 3.4 and lemma 3.5). Secondly, unlike [Reference Chen and Zou8], our method does not require a comparison between the level A and the ground state level of the limiting system (
$\lambda_i=0$). Thirdly, we mention that dealing with many components does not allow to perform some explicit computations as in the two equation case. For instance, when projecting in the Nehari manifold, one cannot, in general, obtain the explicit expression of the coefficients; so, we depend on the notion of strictly diagonally dominant matrices as in [Reference Soave and Tavares21–Reference Tavares, You and Zou23] to show positive definiteness of the coupled matrices, see lemma 3.6.
We employ the idea in proving theorem 1.3-(1) to finish the proof of theorem 1.3-(2). However, there are technical differences between N = 3 and N = 4. For example, when projecting on the Nehari manifold, the equations the coefficients comply to are a linear system for N = 4 (see (3.45) and (3.51)) and the Cramer’s rule can be used. While the corresponding equations are a nonlinear system for N = 3 (see (3.127)) and the Cramer’s rule is not available. Thus, we introduced a new technique to deal with the case N = 3 (see the proof of lemma 3.13 and lemma 3.14).
Theorem 1.3 shows that system (1.3) has a least energy positive solution for weakly cooperative case (
$\beta_{ij} \gt 0$ small, for any i ≠ j) when
$N=3,4$. While [Reference Guo, Luo and Zou10, Theorem 1.1] states that system (1.3) with
$N\geq 3$ has no least energy positive solutions for the purely competitive case (
$\beta_{ij} \lt 0$ for any i ≠ j). A natural question is whether system (1.3) has a least energy positive solution for the mixed case. For the case
$\beta_{12} \gt 0$,
$\beta_{13}, \beta_{23} \lt 0$, we have the following result.
(1) Assume that N = 4,
$\lambda_i\in (0, \Lambda_4)$,
$\lambda_1=\lambda_2$ and
\begin{align*}
0 \lt \beta_{12} \lt \widetilde{\beta}, \quad \beta_{13} \lt 0, \quad \beta_{23} \lt 0,
\end{align*}then A is not attained, which implies that (1.3) has no least energy positive solutions.
(2) Assume that N = 3,
$\lambda_i\in (0, \Lambda_3)$,
$\lambda_1=\lambda_2$ and
\begin{align*}
0 \lt \beta_{12} \lt \widehat{\beta}, \quad \beta_{13} \lt 0, \quad \beta_{23} \lt 0,
\end{align*}then A is not achieved, which means that (1.3) has no least energy positive solutions.
The proof of theorem 1.7-(1) and (2) is inspired by [Reference Lin and Wei12, Theorem 3], where the authors proved the nonexistence of least energy positive solutions to Schrödinger systems for the subcritical case in
$\mathbb{R}^N$. But in [Reference Lin and Wei12] the matrix
$\mathcal{B}:=(\beta_{ij})_{1\leq i,j\leq d}$ is assumed to be positively definite. Here, the positive definiteness of the matrix
$\mathcal{B}$ can be removed. Thus, the method in [Reference Lin and Wei12] cannot be used here and we need some additional new tricks to deal with this problem. We mainly depend on the notion of strictly diagonally dominant matrices to show positive definiteness of the coupled matrices and control the growth of the energy functional I. Compared with the proof of theorem 1.7-(1), the proof of theorem 1.7-(2) is more complicated and we use different techniques to deal with the case N = 3 (see lemma 4.9 and lemma 4.10).
When
$N\geq5$, the Nehari Manifold
$\mathcal{M}$ is not a closed set and so is not a natural constraint. Then in turn we consider a sub-manifold of
$\mathcal{M}$
\begin{align*}
\mathcal{N}_{12,3}:&=\Bigg\{(u_1,u_2)\neq \mathbf{0}, u_3\neq 0: \|u_3\|_3^2=\sum_{i=1}^3\int_{\mathbb{R}^N}\beta_{i3}|u_i|^{\frac{2^*}{2}}|u_3|^{\frac{2^*}{2}}\\
&\,\,\,\,\,\,\, \,\,\,\,\,\,\, \sum_{i=1}^2\|u_i\|_i^2=\sum_{i,j=1}^2\int_{\mathbb{R}^N}\beta_{ij}|u_i|^{\frac{2^*}{2}}|u_j|^{\frac{2^*}{2}}
+\sum_{i=1}^2\int_{\mathbb{R}^N}\beta_{i3}|u_i|^{\frac{2^*}{2}}|u_3|^{\frac{2^*}{2}} \Bigg \}
\end{align*}and
\begin{align}
\widetilde{A}:=\inf_{\mathbf{u}\in \mathcal{N}_{12,3}}I(u_1,u_2,u_3).
\end{align} Parallel with theorem 1.7, for the case
$N\geq 5$ we have
Theorem 1.8 Assume that
$N\geq 5$ and
$\lambda_i\in (0, \Lambda_N)$ and
then the level
$\widetilde{A}$ is not attained and (1.3) has no least energy solutions
$\mathbf{u}=(u_1,u_2,u_3)$ with
$(u_1,u_2)\neq \mathbf{0}, u_3\neq 0$.
Remark 1.9.
At the end of this section, we would like to give the following remark. In the papers [Reference Liu, You and Zou13, Reference Tavares and You22, Reference Tavares, You and Zou23], the authors consider the following critical Schrödinger system
\begin{align}
\begin{cases}
-\Delta u_{i}-\lambda_iu_{i}=|u_i|^{2^*-2}u_i+\sum_{j\neq i}^{d}\beta_{ij}|u_{j}|^{\frac{2^*}{2}}|u_i|^{\frac{2^*}{2}-2}u_i \,\,\text{in}~ \Omega,\\
u_i\geq 0 \,\,\text{in}~ \Omega, u_i=0 \,\,\text{on}~ \partial \Omega, i=1,\cdots,d,
\end{cases}
\end{align} where Ω is a bounded domain in
$\mathbb{R}^N$,
$2^*=\frac{2N}{N-2}$,
$N\geq3$,
$\beta_{ii} \gt 0, ~i=1,\cdots,d$,
$\beta_{ij}=\beta_{ji}$ for any i ≠ j (the cases N = 3, N = 4 and
$N\ge5$ were considered respectively in [Reference Liu, You and Zou13], [Reference Tavares and You22], and [Reference Tavares, You and Zou23]). However, problem (1.3) is considerably different from problem (1.19).
(i) On the one hand, as implicitly pointed out by Chen and Zou in [Reference Chen and Zou8], the Palais–Smale condition of problem (1.3) does not hold for any energy level, so that it makes problem (1.3) very complicated. In [Reference Liu, You and Zou13, Reference Tavares and You22, Reference Tavares, You and Zou23], it shows that the Palais–Smale condition for problem (1.19) holds for some special ranges of the energy level, thanks to some accurate energy estimates.
(ii) On the other hand, the structure of least energy solutions to system (1.3) is much different from that of problem (1.19). For example, when N = 4, least energy positive solutions of system (1.19) are obtained in [Reference Tavares and You22] if
(1.20)
\begin{align}
\beta_{13} \lt 0, \beta_{23} \lt 0, \beta_{12} \gt 0\,\,\mbox{small},
\end{align}but in this paper, we show that system (1.3) with N = 4 has no least energy positive solutions under the assumption (1.20)(see theorem 1.7-(1)). When
$N\geq 5$, it is shown in [Reference Tavares, You and Zou23, Theorem 1.1](
$d=3, m=2$) that the level
$\mathcal{A}$ is attained if
(1.21)
\begin{align}
\beta_{13} \lt 0, \beta_{23} \lt 0, \beta_{12} \gt 0,
\end{align}whereas, in this paper, we show that the level
$\mathcal{A}$ is not attained under the condition (1.21) (see theorem 1.8). The reason underlying this the fact that system (1.3) is invariant in
$\mathbb{R}^N$ under the transformation
\begin{align*}
(u_1, u_2, u_3)\mapsto (a^{\frac{N-2}{2}}u_1(a\cdot),a^{\frac{N-2}{2}}u_2(a\cdot),a^{\frac{N-2}{2}}u_3(a\cdot)),\,\,a \gt 0.
\end{align*}
1.4. Further notations
• Denote by
$|\cdot|_p$ the norm of the
$L^p(\mathbb{R}^N)$,
$1\leq p \leq \infty$.•
$\mathbf{D}= D^{1,2}(\mathbb{R}^N; \mathbb{R}^3)$, with norm
$\|\mathbf{u}\|^{2}=\sum_{i=1}^{3}\|u_{i}\|_{i}^{2}$, where
\begin{align*}
\|u\|_i^2:=\int_{\mathbb{R}^N}\left(|\nabla u|^2-\frac{\lambda_{i}}{|x|^2} u^{2}\right).
\end{align*}• Set
(1.22)
\begin{align}
S:=\min\{S_1, S_2, S_3\},
\end{align}where Si is defined in (1.6).
• We always assume that
$\beta_{ii}=1$, for any
$i=1,2,3.$• Given
$\Gamma\subseteq\{1,2,3\}$ with
$|\Gamma|=2,3$, consider the following subsystem
(1.23)
\begin{align}
\begin{cases}
-\Delta u_{i}-\frac{\lambda_{i}}{|x|^2}u_{i}=|u_i|^{2^*-2}u_i+\sum_{j\neq i}^{3}\beta_{ij}|u_{j}|^
{\frac{2^*}{2}}|u_i|^{\frac{2^*}{2}-2}u_i, ~x\in \mathbb{R}^N, ~i\in \Gamma,\\
u_i\in D^{1,2}(\mathbb{R}^N), i\in \Gamma,
\end{cases}
\end{align}and define
(1.24)
\begin{align}
I_\Gamma(\mathbf{u}_\Gamma):=\frac{1}{2}\sum_{i\in \Gamma} \left\|u_i \right\|_i^2-\frac{1}{2^*}\sum_{i,j\in \Gamma} \int_{\mathbb{R}^N}\beta_{ij} |u_i|^
{\frac{2^*}{2}}|u_j|^{\frac{2^*}{2}},
\end{align}(1.25)
\begin{align}
\mathcal{M}_{\Gamma}=
\left \{\mathbf{u}_\Gamma\in D^{1,2}(\mathbb{R}^N; \mathbb{R}^{|\Gamma|}): u_i\neq 0, \|u_{i}\|_i^2=\sum_{j\in \Gamma} \int_{\mathbb{R}^N}\beta_{ij}|u_{i}|^{\frac{2^*}{2}}|u_{j}|^{\frac{2^*}{2}}, \ i\in \Gamma\right \}.
\end{align}Denote
(1.26)
\begin{align}
A_{\Gamma}:= \inf_{\mathbf{u}\in \mathcal{M}_{\Gamma}}I_\Gamma(\mathbf{u}_\Gamma)=\inf _{\mathbf{u}_\Gamma\in \mathcal{M}_{\Gamma}}\frac{1}{N}\sum_{i\in \Gamma}\|u_i\|_i^2,
\end{align}and
$A_{\Gamma}=A, \mathcal{M}_{\Gamma}=\mathcal{M}$ when
$\Gamma=\{1,2,3\}$.• Given
$\Gamma\subseteq\{1,2,3\}$ with
$|\Gamma|=2,3$,
$\mathbf{u}=(u_1,\cdots,u_{|\Gamma|})\in D^{1,2}(\mathbb{R}^N; \mathbb{R}^{|\Gamma|})$, define the
$|\Gamma|\times |\Gamma|$ matrix
(1.27)
\begin{align}
M_{B}[u_1,\cdots,u_{|\Gamma|}]:=\left( M_{B}[u_1,\cdots,u_{|\Gamma|}]_{hk}\right)_{h,k\in \Gamma}:= \left( \int_{\mathbb{R}^N}\beta_{hk}|u_{h}|^{\frac{2^*}{2}}|u_{k}|^{\frac{2^*}{2}}\right)_{h,k\in \Gamma}.
\end{align}
2. Ground state solutions
2.1. Existence of ground state solutions
Denote
\begin{align*}
E(u_1,u_2,u_3):=\sum_{i=1}^3\left(|\nabla u_i|^{2}-\frac{\lambda_{i}}{|x|^2} u_i^{2}\right), \quad
F(u_1,u_2,u_3):=\sum_{i,j=1}^3 \beta_{ij}|u_{i}|^{\frac{2^*}{2}}|u_{j}|^{\frac{2^*}{2}}.
\end{align*} For any
$\mathbf{u}\in \mathbf{D}\setminus \{\mathbf{0}\}$, by the condition (1.13), we know that
\begin{align*}
\int_{\mathbb{R}^N}F(u_1,u_2,u_3) \gt 0.
\end{align*} Then, thanks to the definition of
$\mathcal{A}$ in (1.12),
\begin{align*}
\mathcal{A}=\inf_{\mathbf{u}\in \mathbf{D}\setminus \{\mathbf{0}\}}\frac{1}{N}
\left[\frac{\int_{\mathbb{R}^N}E(u_1,u_2,u_3)}{\int_{\mathbb{R}^N}F(u_1,u_2,u_3)}\right]^{\frac{N}{2}}
\end{align*}and then
\begin{align}
\int_{\mathbb{R}^N}E(u_1,u_2,u_3)\geq (N\mathcal{A})^{\frac{2}{N}}\left(\int_{\mathbb{R}^N}F(u_1,u_2,u_3)\right)^{\frac{2}{2^*}}, ~~\forall ~\mathbf{u}\in \mathbf{D}.
\end{align}The proof of theorem 1.1-(1)
Based on the inequality (2.1), following the proof of [Reference Guo, Luo and Zou10, Lemma 3.3] we can obtain that
$\mathcal{A}$ is attained by a nonnegative solution u, which is a ground state solution.
2.2. Nonexistence of fully nontrivial ground state solutions for
$N=3,4$
In this subsection, we prove (2) and (3) of theorem 1.1.
The proof of theorem 1.1-(2)
The following proof is inspired by [Reference Correia, Oliveira and Tavares9, Theorem 1.7], and we sketch it here for completeness. Based on theorem 1.1-(1), let
$\mathbf{u}=(u_1,u_2,u_3)\in \mathcal{N}$ be a ground state solution of system (1.3). Assume now that the ground state solution u is fully nontrivial, that is
$u_i\neq 0$,
$i=1,2,3$. Note that
\begin{align}
(tu_1,0,0) \in \mathcal{N}\ \Leftrightarrow \ t^2=\cfrac{\left\| u_1\right\|_1^2 }{|u_1|_4^4}.
\end{align} Since
$\mathbf{u}=(u_1,u_2,u_3)$ is a ground state solution of system (1.3), then
$I(u_1,u_2,u_3)\leq I(tu_1,0,0)$, which yields that
\begin{align*}
\sum_{i=1}^3|u_i|_4^4+2\sum_{i \lt j}\beta_{ij}|u_iu_j|_2^2\leq t^4|u_1|_4^4.
\end{align*}Then
\begin{align*}
|u_1|_4^4\left(\sum_{i=1}^3|u_i|_4^4+2\sum_{i \lt j}\beta_{ij}|u_iu_j|_2^2\right)\leq
\left(|u_1|_4^4+\sum_{i=2}^3\beta_{1j}|u_1u_j|_2^2\right)^2,
\end{align*}which implies that
\begin{align*}
\sum_{i=2}^3|u_i|_4^4 \leq \left(\sum_{i=2}^3\beta_{1j}|u_j|_4^2\right)^2\leq 2\left(\max_{i\neq j}\{\beta_{ij}\}\right)^2 \sum_{i=2}^3|u_i|_4^4,
\end{align*}and so
\begin{align*}
\max_{i\neq j}\{\beta_{ij}\}\geq \frac{\sqrt{2}}{2}.
\end{align*}Therefore, if
\begin{align*}
0 \lt \beta_{ij} \lt \frac{\sqrt{2}}{2},
\end{align*}all ground state solutions of system (1.3) are semi-trivial.
Conclusion of the proof of theorem 1.1-(3)
Based on theorem 1.1-(1), let
$\mathbf{u}=(u_1,u_2,u_3)\in \mathcal{N}$ be a ground state solution of system (1.3). Assume now that the ground state solution u is fully nontrivial, that is
$u_i\neq 0$,
$i=1,2,3$. Note that
\begin{align}
(tu_1,0,0) \in \mathcal{N}\ \Leftrightarrow \ t^4=\cfrac{\left\| u_1\right\|_1^2 }{|u_1|_6^6}.
\end{align} Since
$\mathbf{u}=(u_1,u_2,u_3)$ is a ground state solution of system (1.3), then
$I(u_1,u_2,u_3)\leq I(tu_1,0,0)$, that is
\begin{align*}
\sum_{i=1}^3|u_i|_6^6+2\sum_{i \lt j}\beta_{ij}|u_iu_j|_3^3\leq t^2\|u_1\|_1^2.
\end{align*}Therefore,
\begin{align*}
|u_1|_6^6\left(\sum_{i=1}^3|u_i|_6^6+2\sum_{i \lt j}\beta_{ij}|u_iu_j|_3^3\right)^2\leq
\left(|u_1|_6^6+ \sum_{j=2}^3\beta_{1j}|u_1u_j|_3^3\right)^3,
\end{align*}which implies that
\begin{align*}
|u_1|_6^6\Big( \sum_{i=1}^3|u_i|_6^6 &+ 2\sum_{j=2}^3\beta_{1j}|u_1u_j|_3^3 +2\beta_{23}|u_2u_3|_3^3 \Big)^2 \leq |u_1|_6^6+\left(\sum_{j=2}^3\beta_{1j}|u_1u_j|_3^3\right)^3\\
& + 3|u_1|_6^{12}\sum_{j=2}^3\beta_{1j}|u_1u_j|_3^3
+3|u_1|_6^{6}\left(\sum_{j=2}^3\beta_{1j}|u_1u_j|_3^3\right)^2,
\end{align*}that is,
\begin{align}
|u_1|_6^6 \Big( \sum_{i=2}^3|u_i|_6^6\Big)^2+2|u_1|_6^{12}\sum_{i=2}^3|u_i|_6^6
& \leq \left(\sum_{j=2}^3\beta_{1j}|u_1u_j|_3^3\right)^3\leq \left(\sum_{j=2}^3\beta_{1j}|u_1|_6^3|u_j|_6^3\right)^3\nonumber\\
& \leq |u_1|_6^9 \left[\left(\sum_{j=2}^3\beta_{1j}|u_j|_6^3\right)^2\right]^{\frac{3}{2}}\\
& \leq 2^{\frac{3}{2}}|u_1|_6^9 \left(\max_{i\neq j}\{\beta_{ij}\}\right)^3\Big( \sum_{i=2}^3|u_i|_6^6\Big)^{\frac{3}{2}}.\nonumber
\end{align}On the other hand,
\begin{align*}
|u_1|_6^6 \Big( \sum_{i=2}^3|u_i|_6^6\Big)^2+2|u_1|_6^{12}\sum_{i=2}^3|u_i|_6^6\geq
2^{\frac{3}{2}}|u_1|_6^9\left( \sum_{i=2}^3|u_i|_6^6\right)^{\frac{3}{2}}.
\end{align*}Thus, combining this with (2.4), we see that
\begin{align*}
\max_{i\neq j}\{\beta_{ij}\}\geq 1.
\end{align*}Therefore, if
all ground state solutions of system (1.3) are semi-trivial.
The proof of theorem 1.1-(4)
Note that
$\|u\|_i^2=\int_{\mathbb{R}^N}\left(|\nabla u|^2-\frac{\lambda_{i}}{|x|^2} u^{2}\right)$ is a norm and the exponent
$2^*/2\in (1,2)$ when
$N\geq 5$, then the proof is the same as that of [Reference Oliveira and Tavares16, Theorem 1.2], and so we omit it.
3. Existence of least energy positive solutions
3.1. The case for N = 4
3.1.1. Preliminary results
Lemma 3.1. Set
\begin{align}
\overline{C}=\frac{3}{N} \mathcal{S}^{\frac{N}{2}},
\end{align}then we have
Proof. Taking
$v_1,v_2,v_3 \not\equiv 0$ such that
$v_i \cdot v_j \equiv 0$ when i ≠ j and setting
$\widehat{v}_i=t_i v_i$, where
\begin{align}
t_i^{2^*-2}=\frac{\|v_i\|_i^2}{|v_i|_{2^*}^{2^*}}.
\end{align} we have
$(\widehat{v}_1,\widehat{v}_2,\widehat{v}_3)\in \mathcal{M}$. By (3.2),
\begin{align}
A & \leq I(\widehat{v}_1,\widehat{v}_2,\widehat{v}_3)=\frac{1}{N}\sum_{i=1}^3 \|\widehat{v}_i\|_i^2= \frac{1}{N}\sum_{i=1}^3 t_i^2\|v_i\|_i^2\nonumber\\
& = \frac{1}{N}\sum_{i=1}^3\left(\frac{\|v_i\|_i^2}{|v_i|_{2^*}^{2^*}}\right)^{\frac{2}{2^*-2}}\|v_i\|_i^2
\leq\frac{1}{N}\sum_{i=1}^3\left(\frac{\|v_i\|^2}{|v_i|_{2^*}^{2}}\right)^{\frac{N}{2}}.
\end{align}Denote
\begin{align*}
\widehat{S}(\Omega):=\inf_{u\in H_{0}^{1}(\Omega)\setminus \{0\}}\frac{\|u\|^2}{|u|_{2^*}^{2}}.
\end{align*} By [Reference Terracini24], we know that
$\widehat{S}(\Omega)=\mathcal{S}$. Combining this with (3.3),
\begin{align}
A & \leq \frac{1}{N}\sum_{i=1}^3\inf_{\Omega \supset\Omega_1,\Omega_2,\Omega_3\neq \emptyset \atop \Omega_{i}\cap\Omega_{j}=\emptyset, i\neq j}\widehat{S}^{\frac{N}{2}}(\Omega_i)=\frac{3}{N} \mathcal{S}^{\frac{N}{2}}=\overline{C},
\end{align} where
$\overline{C}$ is defined in (3.1).
Denote
\begin{align}
\beta_1:=\frac{S^2}{16\overline{C}}.
\end{align}Lemma 3.2. Assume that N = 4 and
$0\leq\beta_{ij} \lt \beta_1$, for any i ≠ j, then there exist
$C_1,C_2 \gt 0$ dependent on
$\lambda_i, S, \overline{C}$, such that for any
$\mathbf{u} \in \mathcal{M}$ with
$I(\mathbf{u})\leq 2\overline{C}$,
Proof. Note that
$\mathbf{u} \in \mathcal{M}$ and
$I(\mathbf{u})\leq 2\overline{C}$, then we have
\begin{align*}
S|u_i|_4^2\leq \|u_i\|_i^2\leq \sum_{i=1}^3\|u_i\|_i^2\leq 8\overline{C},
\end{align*}which implies that
On the other hand, since
\begin{align*}
\|u_i\|_i^2 = \sum_{j=1}^3 \int_{\mathbb{R}^4}\beta_{ij}|u_i|^2|u_j|^2,
\end{align*}we have
\begin{align*}
S|u_i|_4^2 & \leq \sum_{j=1}^3 \int_{\mathbb{R}^4}\beta_{ij}|u_i|^2|u_j|^2
\leq |u_i|_4^4 + \sum_{j\neq i}\beta_{ij}|u_i|_4^2|u_j|_4^2\\
& \leq |u_i|_4^4 +\beta_1\sum_{j\neq i}S^{-1}\|u_j\|_j^2|u_i|_4^2\\
& \leq |u_i|_4^4+\beta_1 S^{-1}8\overline{C}|u_i|_4^2=|u_i|_4^4+\frac{S}{2}|u_i|_4^2.
\end{align*}Thus, we know that
\begin{align*}
|u_i|_4^2\geq C_2:=\frac{S}{2}.
\end{align*}Denote
\begin{align}
\beta_2:=\min\left\{\beta_1, \frac{SC_1^2}{16\overline{C}C_2}\right\}.
\end{align}Lemma 3.3. Suppose that N = 4 and
$0\leq\beta_{ij} \lt \beta_2$, for any i ≠ j. Let
$\widehat{\mathbf{u}}_n=(\widehat{u}_1^n,\widehat{u}_2^n,\widehat{u}_3^n)
\in\mathcal{M}$ be a minimizing sequence of A, then there exists a sequence
$\{\mathbf{u}_n\}\in\mathcal{M}$ such that
\begin{align*}
\lim_{n\rightarrow \infty} I(\mathbf{u}_n)=A, \quad \lim_{n\rightarrow \infty} I'(\mathbf{u}_n)=0.
\end{align*}Proof. We assume that
$I(\mathbf{u}_n)\leq 2\overline{C}$ for n large enough. Next, we claim that the matrix
$M_B[\widehat{u}_1^n,\widehat{u}_2^n,\widehat{u}_3^n]$ is positively definite, where
$M_B[\widehat{u}_1^n,\widehat{u}_2^n,\widehat{u}_3^n]$ is defined in (1.27). In fact, for any
$i=1,2,3$
\begin{align*}
&\int_{\mathbb{R}^4}\beta_{ii}|\widehat{u}_i^n|^4-\sum_{j\neq i}\left|\int_{\mathbb{R}^4}\beta_{ij}|\widehat{u}_i^n|^2|\widehat{u}_j^n|^2\right|\\
& \geq C_1^2- \beta_2\sum_{j\neq i}|\widehat{u}_i^n|_4^2|\widehat{u}_j^n|_4^2\geq C_1^2-\beta_2S \sum_{j\neq i}|\widehat{u}_i^n|_4^2\|\widehat{u}_j^n\|_j^2\\
& \geq C_1^2- \beta_28\overline{C} S^{-1}|\widehat{u}_i^n|_4^2\geq C_1^2-\beta_28\overline{C}S^{-1}C_2\geq \frac{1}{2}C_1^2,
\end{align*} which yields that the matrix
$M_B[\widehat{u}_1^n,\widehat{u}_2^n,\widehat{u}_3^n]$ is strictly diagonally dominant. Thus, by the Gershgorin circle theorem, we know that the matrix
$M_B[\widehat{u}_1^n,\widehat{u}_2^n,\widehat{u}_3^n]$ is positive definite. Based on these arguments, following the proof of [Reference Tavares and You22, Proposition 3.13], there exists a sequence
$\{\mathbf{u}_n\}\in\mathcal{M}$ such that
\begin{align*}
\lim_{n\rightarrow \infty} I(\mathbf{u}_n)=A, \quad \lim_{n\rightarrow \infty} I'(\mathbf{u}_n)=0.
\end{align*}3.1.2. Energy estimates
In this subsection, we establish energy estimates. Denote
\begin{align}
\beta'_3:= \frac{1}{2}\min_{i\neq j}\left\{\frac{1-\lambda_i}{1-\lambda_j}, \frac{(1-\lambda_i)^{\frac{3}{4}}(1-\lambda_j)^{\frac{3}{4}}}
{(1-\lambda_i)^{\frac{3}{2}}+(1-\lambda_j)^{\frac{3}{2}}}\right\}.
\end{align}It follows from [Reference Chen and Zou8, Lemma 3.1] that
Consider
\begin{align*}
f_{ij}(t_i,t_j):=\frac{1}{2}\sum_{i=1}^2t_i\|\omega^{ij}_i\|_i^2-\frac{1}{4}\sum_{i=1}^2t_i^{2}
\int_{\mathbb{R}^4}|\omega^{ij}_i|^{4}-\frac{1}{2}t_it_j\int_{\mathbb{R}^4}\beta_{ij}|\omega^{ij}_i|^2|\omega^{ij}_j|^2.
\end{align*}Set
\begin{align*}
\beta_3:=\min\left\{1, \beta'_3,\min_{i\neq j}\left\{\frac{S^4}{16(A_i+A_j)} \right\} \right\}.
\end{align*}Lemma 3.4. Assume that N = 4 and
$\beta_{ij}\in (0, \beta_3),i\neq j$. Then we have
\begin{align}
\max_{t_{i},t_{j}\geq0}f_{ij}(t_{i},t_{j})=f_{ij}(1,1)=A_{ij}, ~\text{for}~\text{any}~ i\neq j,
\end{align}and
Proof. Without loss of generality, we will present that
$
\max_{t_{1},t_{2}\geq 0}f_{12}(t_{1},t_{2})=f_{12}(1,1)=A_{12}
$ and
$A_{12} \gt \max\{A_1, A_2\}$. Assume that
$I_{12}(\omega^{12}_1,\omega^{12}_2)=A_{12}$. By (3.8), we know that
$A_{12}=\frac{1}{4}(\|\omega^{12}_1\|_1^2+\|\omega^{12}_2\|_2^2)\leq A_1+A_2$, combining this with
$\beta_{ij}\in (0, \beta_3)$, and following the proof of lemma 3.2, we obtain that
\begin{align}
|\omega^{12}_i|_4^2\geq \frac{S}{2}, ~~i=1,2.
\end{align} Note that
$\beta_{ij} \gt 0$ for any i ≠ j, then by (3.11) we have
\begin{align}
\max_{t_{1},t_{2}\geq0}f_{12}(t_{1},t_{2}) & \leq \frac{1}{2}t_1\|\omega^{12}_1\|_1^2-\frac{1}{4}t_1^{2}
\int_{\mathbb{R}^4}|\omega^{12}_1|^{4}+\frac{1}{2}t_2\|\omega^{12}_2\|_2^2-\frac{1}{4}t_2^{2}
\int_{\mathbb{R}^4}|\omega^{12}_2|^{4}\nonumber\\
& \leq \frac{1}{4}\sum_{i=1}^2 \frac{\|\omega^{12}_i\|_i^2}{|\omega^{12}_i|_4^{4}}\|\omega^{12}_i\|_i^2
\leq \frac{4(A_1+A_2)^2}{S^2}.
\end{align} Thus, f 12 has a global maximum point
$(\widetilde{t}_1, \widetilde{t}_2)$ in
$\overline{\mathbb{R}_{+}^2}$. Assume that the global maximum point
$(\widetilde{t}_1, \widetilde{t}_2)\in \partial \overline{\mathbb{R}_{+}^2}$, without loss of generality, we assume that
$\widetilde{t}_1=0$. Thus,
$\partial_2f_{12}(0, \widetilde{t}_2)=0$, that is
It follows from (3.12) and (3.13) that
\begin{align}
\frac{1}{4}\widetilde{t}_2\|\omega^{12}_2\|_2^2=I_{12}\left(0,\sqrt{\widetilde{t}_2}\omega^{12}_2\right)\leq \max_{t_{1},t_{2}\geq0}f_{12}(t_{1},t_{2})\leq
\frac{4(A_1+A_2)^2}{S^2}.
\end{align} Note that
$\beta_{ij}\in (0, \beta_3)$, then by (3.14), we obtain that
\begin{align}
\widetilde{t}_2\beta_{12}|\omega^{12}_1\omega^{12}_2|_2^2 & \leq \beta_{12}\widetilde{t}_2\frac{\|\omega^{12}_1\|_1^2\|\omega^{12}_2\|_2^2}{S^2}\nonumber\\
& \leq4\beta_{12}\frac{(A_1+A_2)^2}{S^2}
\|\omega^{12}_1\|_1^2\leq \frac{1}{4}\|\omega^{12}_1\|_1^2.
\end{align}It follows from (3.15) that
\begin{align*}
f_{12}(s, \widetilde{t}_2) - f_{12}(0, \widetilde{t}_2) & = \frac{1}{2}s\|\omega^{12}_1\|_1^2-\frac{1}{4}s^2|\omega^{12}_1|_4^4- \frac{1}{2}s\widetilde{t}_2\beta_{12}|\omega^{12}_1\omega^{12}_2|_2^2 \\
& \geq \frac{1}{4}s\|\omega^{12}_1\|_1^2-\frac{1}{4}s^2|\omega^{12}_1|_4^4 \gt 0 ~\text{for}~ s \gt 0 ~\text{small}~\text{enough,}
\end{align*} a contradiction. Therefore, the global maximum point
$(\widetilde{t}_1, \widetilde{t}_2)$ belongs to
$\mathbb{R}_{+}^2$.
On the other hand, we deduce from
$\beta_{ij} \lt 1$ that
\begin{align*}
\int_{\mathbb{R}^4}|\omega^{12}_1|^4 \int_{\mathbb{R}^4}|\omega^{12}_2|^4-\left(\beta_{12}\int_{\mathbb{R}^4}|\omega^{12}_1|^2
|\omega^{12}_2|^2\right)^2 \gt 0,
\end{align*} which implies that the matrix
$M_B[\omega_1,\omega_2]$ is positive definite, where
$M_B[\omega_1,\omega_2]$ is defined in (1.27). Thus, f 12 is strictly concave, and the critical point of f 12 is unique. Note that
$(\omega_1,\omega_2)$ is a solution of the corresponding subsystem, and so the point
$(1,1)$ is a critical point of f 12. Therefore, we know that
$(1,1)$ is the unique critical point of f 12 and is the global maximum point, which yields that
$\max_{t_{1},t_{2}\geq 0}f_{12}(t_{1},t_{2})=f_{12}(1,1)=A_{12}$.
Based on the above arguments, we also know that
\begin{align}
A_{12}=\max_{t_{1},t_{2}\geq0}f_{12}(t_{1},t_{2}) \gt \max\left\{\max_{t_{1} \gt 0}f_{12}(t_{1},0), \max_{t_{2} \gt 0}f_{12}(0,t_{2})\right\}=\max\{A_1, A_2\}.
\end{align}Denote
\begin{align}
\beta_4:=\min\left\{\beta_2, ~\beta_3, ~\frac{S^2}{16(A_1+A_2+A_3)} \right\},
\end{align}where S is defined in (1.22), and Ai is defined in (1.7).
Lemma 3.5. Assume that N = 4 and
$\beta_{ij}\in (0, \beta_4),i\neq j$. Then we have
Proof. Without loss of generality, we only show that
$A \lt A_{12}+A_3$. Assume that
$I_{12}(\omega^{12}_1,\omega^{12}_2)=A_{12}$ and consider
\begin{align*}
g(t_1,t_2,t_3):&=I(\sqrt{t_1}\omega^{12}_1,\sqrt{t_2}\omega^{12}_2,\sqrt{t_3}\omega_3)\\
& =\frac{1}{2}\sum_{i=1}^2t_i\|\omega^{12}_i\|_i^2+\frac{1}{2}t_3\|\omega_3\|_3^2-\frac{1}{4}\sum_{i=1}^2t_i^{2}
\int_{\mathbb{R}^4}|\omega^{12}_i|^{4}-\frac{1}{4}t_3^{2}\int_{\mathbb{R}^4}|\omega_3|^{4}\\
&\quad-\frac{1}{2}t_1t_2\int_{\mathbb{R}^4}\beta_{12}|\omega^{12}_1|^2|\omega^{12}_2|^2-\frac{1}{2}t_1t_3
\int_{\mathbb{R}^4}\beta_{13}|\omega^{12}_1|^2|\omega_3|^2\\
& \quad -\frac{1}{2}t_2t_3\int_{\mathbb{R}^4}\beta_{23}|\omega^{12}_2|^2|\omega_3|^2 \\
& \leq f_{12}(t_1,t_2)+f_3(t_3),
\end{align*}where
\begin{align}
f_3(t_3):=\frac{1}{2}t_3\|\omega_3\|_3^2-\frac{1}{4}t_3^{2}\int_{\mathbb{R}^4}|\omega_3|^{4}.
\end{align} It is easy to see that
$\max_{t_{3} \gt 0}f_{3}(t_{3})=A_3$. By lemma 3.4 we have
$\max_{t_{1},t_{2}\geq 0}f_{12}(t_{1},t_{2})=f_{12}(1,1)=A_{12}$. Therefore,
\begin{align}
\max_{t_{1},t_{2},t_3\geq0}I(\sqrt{t_1}\omega^{12}_1,\sqrt{t_2}\omega^{12}_2,\sqrt{t_3}\omega_3)=
\max_{t_{1},t_{2},t_3\geq0}g(t_{1},t_{2},t_3) \lt A_{12}+A_3.
\end{align} By (3.8) we know that
$A_{12} \lt A_1+A_2$, then
\begin{align}
\max_{t_{1},t_{2},t_3\geq 0}I(\sqrt{t_1}\omega^{12}_1,\sqrt{t_2}\omega^{12}_2,\sqrt{t_3}\omega_3) \lt A_{12}+A_3 \lt A_1+A_2+A_3.
\end{align} To obtain the desired result, it suffices to show that there exist
$\overline{t}_1,\overline{t}_2,\overline{t}_3 \gt 0$ such that
\begin{align*}(\sqrt{\overline{t}_1}\omega^{12}_1,\sqrt{\overline{t}_2}\omega^{12}_2,\sqrt{\overline{t}_3}\omega_3)\in\mathcal{M}.\end{align*}If so, it follows from (3.19) that
\begin{align}
A\leq \left(\sqrt{\overline{t}_1}\omega^{12}_1,\sqrt{\overline{t}_2}\omega^{12}_2,\sqrt{\overline{t}_3}\omega_3\right)\leq
\max_{t_{1},t_{2},t_3\geq0}I\left(\sqrt{t_1}\omega^{12}_1,\sqrt{t_2}\omega^{12}_2,\sqrt{t_3}\omega_3\right) \lt A_{12}+A_3.
\end{align} Thanks to (3.8),
$A_{12} \lt A_1+A_2$ and then
$A \lt A_1+A_2+A_3$. The following proof is similar to that of lemma 3.4.
Step 1. Similarly as above, g has a global maximum point
$(\overline{t}_1,\overline{t}_2,\overline{t}_3)$ in
$\overline{\mathbb{R}^{3}_{+}}$. Note that
$\partial_{i}g(\overline{t}_1,\overline{t}_2,\overline{t}_3)\leq 0$ if
$\overline{t}_i=0$ and
$\partial_{i}g(\overline{t}_1,\overline{t}_2,\overline{t}_3)= 0$ if
$\overline{t}_i \gt 0$. Thus,
\begin{align}
\overline{t}_i\|\omega^{12}_i\|_i^2=\sum_{k=1}^3M_B[\omega^{12}_1,\omega^{12}_2,\omega_3]_{ik}\overline{t}_i\overline{t}_k, ~i=1,2,
\end{align}
\begin{align}
\overline{t}_3\|\omega_3\|_3^2=\sum_{k=1}^3M_B[\omega^{12}_1,\omega^{12}_2,\omega_3]_{3k}\overline{t}_3\overline{t}_k.
\end{align} It is easy to see that the above two equations are still true when
$\overline{t}_i=0$. Then (3.22) and (3.23) imply that
\begin{align}
I\left(\sqrt{\overline{t}_1}\omega^{12}_1,\sqrt{\overline{t}_2}\omega^{12}_2,\sqrt{\overline{t}_3}\omega_3\right)=
\frac{1}{4}\left(\overline{t}_1\|\omega^{12}_1\|_1^2+\overline{t}_2\|\omega^{12}_2\|_2^2+\overline{t}_3\|\omega_3\|_3^2\right)
\end{align}Combining this with (3.20) we have
Step 2.
$\overline{t}_i \gt 0, i=1,2,3$. By contradiction, we assume that
$\overline{t}_1=0$, and
$(0, \overline{t}_2, \overline{t}_3)$ is the maximum point. Consider the following function
\begin{align}
h(s):&=I(\sqrt{s}\omega^{12}_1,\sqrt{\overline{t}_2}\omega^{12}_2,\sqrt{\overline{t}_3}\omega_3) \nonumber\\
&=\frac{1}{2}s\|\omega^{12}_1\|_1^2-\frac{1}{2}\sum_{i=2}^{3}M_B[\omega^{12}_1,\omega^{12}_2,\omega_3]_{1i}s\overline{t}_i
-\frac{1}{4}s^2\int_{\mathbb{R}^4}|\omega^{12}_1|^{4}\nonumber\\
&+ \frac{1}{2}\overline{t}_2\|\omega^{12}_2\|_2^2+\frac{1}{2}\overline{t}_3\|\omega_3\|_3^2-\frac{1}{4}\sum_{i,j=2}^3
M_B[\omega^{12}_1,\omega^{12}_2,\omega_3]_{ij}\overline{t}_i\overline{t}_j.
\end{align}On the other hand, by (3.25) we see that
\begin{align}
\sum_{i=2}^{3}M_B[\omega^{12}_1,\omega^{12}_2,\omega_3]_{1i}\overline{t}_i & =\overline{t}_2\int_{\mathbb{R}^4}
\beta_{12}|\omega^{12}_1|^{2}|\omega^{12}_2|^{2}+\overline{t}_3\int_{\mathbb{R}^4}
\beta_{13}|\omega^{12}_1|^{2}|\omega_3|^{3}\nonumber\\
& \leq \frac{\beta_3}{S^2}\overline{t}_2\|\omega^{12}_1\|_1^2\|\omega^{12}_2\|_2^2
+\frac{\beta_3}{S^2}\overline{t}_3\|\omega^{12}_1\|_1^2\|\omega_3\|_3^2\nonumber\\
& \leq 4\frac{\beta_3}{S^2}\|\omega^{12}_1\|_1^2(A_1+A_2+A_3)\leq \frac{1}{4}\|\omega^{12}_1\|_1^2.
\end{align}Thus,
\begin{align}
\frac{1}{2}s&\|\omega^{12}_1\|_1^2-\frac{1}{2}\sum_{i=2}^{3}M_B[\omega^{12}_1,\omega^{12}_2,\omega_3]_{1i}s\overline{t}_i
-\frac{1}{4}s^2\int_{\mathbb{R}^4}|\omega^{12}_1|^{4}\nonumber\\
&=\frac{1}{2}s\left(\|\omega^{12}_1\|_1^2-\sum_{i=2}^{3}M_B[\omega^{12}_1,\omega^{12}_2,\omega_3]_{1i}\overline{t}_i
-\frac{1}{4}s\int_{\mathbb{R}^4}|\omega^{12}_1|^{4} \right)\nonumber\\
& \geq \frac{1}{2}s\left(\frac{3}{4}\|\omega^{12}_1\|_1^2-\frac{1}{4}s\int_{\mathbb{R}^4}|\omega^{12}_1|^{4} \right) \gt 0,
\end{align}when s > 0 small enough. Therefore, by (3.26) and (3.28), we have
\begin{align}
I\left(\sqrt{s}\omega^{12}_1,\sqrt{\overline{t}_2}\omega^{12}_2,\sqrt{\overline{t}_3}\omega_3\right) \gt I\left(0,\sqrt{\overline{t}_2}\omega^{12}_2,\sqrt{\overline{t}_3}\omega_3\right),
\end{align}which is a contradiction.
Lemma 3.6. Suppose that
$\beta_{ij}\in (0, \beta_4)$ and let
$(u_1^n,u_2^n,u_3^n)\in \mathcal{M}$ be a minimizing sequence of A with
$I(u_1^n,u_2^n,u_3^n)\leq 2\overline{C}$, and
$(u_1^n,u_2^n,u_3^n)\rightharpoonup (0,0,0)$ weakly in
$\mathbb{D}$. Then for any σ > 0 and
$\delta\in (-\sigma,0)\cup (0,\sigma)$, there exists
$\rho\in (-\delta,0)\cup (0,\delta)$ such that, up to a subsequence,
\begin{align}
\text{either} ~\sum_{i=1}^3\int_{B_{\sigma+\rho}}|\nabla u_i^n|^2\rightarrow 0 ~\text{or}
~\sum_{i=1}^3\int_{\mathbb{R}^N \backslash B_{\sigma+\rho}}|\nabla u_i^n|^2\rightarrow 0.
\end{align}Proof. Without loss of generality, we only consider the case
$\delta\in (0,\sigma)$. Since
$(u_1^n,u_2^n,u_3^n)\in \mathcal{M}$ is a minimizing sequence of A, then
$(u_1^n,u_2^n,u_3^n)$ is uniformly bounded in
$\mathbb{D}$. By lemma 3.3,
$I'(u_1^n,u_2^n,u_3^n)\rightarrow 0$ as
$n\rightarrow \infty$. In the following, we adopt some idea in [Reference Chen and Zou8, Reference Terracini24] to prove (3.30).
Let
$\mathbb{S}$ be the unit sphere of
$\mathbb{R}^N$. Since
\begin{align}
\int_{\sigma}^{\sigma+\delta}\mathrm{d}\rho\int_{\rho\mathbb{S}}\sum_{i=1}^3|\nabla u_i^n|^2=\int_{\sigma\leq |x|\leq\sigma+\delta}\sum_{i=1}^3|\nabla u_i^n|^2,
\end{align} there exists
$\rho\in (0,\delta)$ such that
\begin{align}
\int_{(\sigma+\rho)\mathbb{S}}\sum_{i=1}^3|\nabla u_i^n|^2 \leq \frac{3}{\delta}\int_{\sigma\leq |x|\leq\sigma+\delta}\sum_{i=1}^3|\nabla u_i^n|^2
\end{align} holds for infinitely many n. Therefore,
$u_i^n$ is uniformly bounded in
$H^1((\sigma+\rho)\mathbb{S})$,
$i=1,2,3$. Note that
$H^1((\sigma+\rho)\mathbb{S})$ is compactly embedded into
$H^{\frac{1}{2}}((\sigma+\rho)\mathbb{S})$. Thus, we can assume that
$u_i^n\rightarrow u_i$ strongly in
$H^{\frac{1}{2}}((\sigma+\rho)\mathbb{S})$,
$i=1,2,3$. On the other hand, by the continuity of the embedding
$H^1(B_{\sigma+\rho})\hookrightarrow H^{\frac{1}{2}}((\sigma+\rho)\mathbb{S})$, we know that
$u_i=0$,
$i=1,2,3$, which means that
\begin{align}
u_i^n\rightarrow 0 ~\text{strongly}~\text{in}~ H^{\frac{1}{2}}((\sigma+\rho)\mathbb{S}), ~i=1,2,3.
\end{align} Let
$\omega^n_{i,j}, i=1,2,3,j=1,2$, be the solutions to the following problems
\begin{align}
\begin{cases}
\Delta \omega^n_{i,1}=0 ~~ \text{in}~~ B_{\sigma+\delta}\setminus B_{\sigma+\rho},\\
\omega^n_{i,1}=0 ~~ \text{on}~~ (\sigma+\delta)\mathbb{S},\\
\omega^n_{i,1}=u^n_1 ~~ \text{on}~~ (\sigma+\rho)\mathbb{S},
\end{cases}
~~
\begin{cases}
\Delta \omega^n_{i,2}=0 ~~ \text{in}~~ B_{\sigma+\rho}\setminus B_{\sigma-\delta},\\
\omega^n_{i,2}=0 ~~ \text{on}~~ (\sigma-\delta)\mathbb{S},\\
\omega^n_{i,2}=u^n_1 ~~ \text{on}~~ (\sigma+\rho)\mathbb{S}.
\end{cases}
\end{align} Combining (3.33) with continuity of the inverse Laplace operator from
$H^{\frac{1}{2}}(\partial\Omega)$ to
$H^1(\Omega)$, we see that
$\omega^n_{i,1}\rightarrow 0$ strongly in
$H^1(B_{\sigma+\delta}\backslash B_{\sigma+\rho})$ and
$\omega^n_{i,2}\rightarrow 0$ strongly in
$H^1(B_{\sigma+\rho}\backslash B_{\sigma-\delta})$, i=1, 2, 3. For any
$i=1, 2, 3$, define
\begin{align}
u_{i,1}^n(x)=
\begin{cases}
u_i^n(x) ~~ \text{if}~~ x\in B_{\sigma+\rho},\\
\omega_{i,1}^n(x) ~~ \text{if}~~ x\in B_{\sigma+\delta}\setminus B_{\sigma+\rho},\\
0 ~~\text{elsewhere},
\end{cases}
~~u_{i,2}^n(x)=
\begin{cases}
0 ~~ \text{if}~~ x\in B_{\sigma-\delta},\\
\omega_{i,2}^n(x) ~~ \text{if}~~ x\in B_{\sigma+\rho}\setminus B_{\sigma-\delta},\\
u_i^n(x) ~~\text{elsewhere}.
\end{cases}
\end{align}Obviously,
\begin{align}
\|u_i^n\|_i^2= \|u_{i,1}^n\|_i^2+\|u_{i,2}^n\|_i^2+o(1).
\end{align}Moreover,
where
Next, we claim that
\begin{equation}\text{either}~\lim_{n\rightarrow\infty}\sum_{i=1}^3\Arrowvert u_{i,1}^n\Arrowvert^2=0\;\text{or}\;\lim_{n\rightarrow\infty}\sum_{i=1}^3\Arrowvert u_{i,2}^n\Arrowvert^2=0.\end{equation}By contradiction, if (3.40) does not hold, then up to a subsequence,
\begin{align}
\text{both}\; \sum_{i=1}^3\lim_{n\rightarrow\infty}\|u_{i,1}^n\|^2 \gt 0 \text{ and}\;
\sum_{i=1}^3\lim_{n\rightarrow\infty}\|u_{i,2}^n\|^2 \gt 0.
\end{align}Set
\begin{align}
a_i:=\lim_{n\rightarrow\infty}\|u_{i,1}^n\|^2, \quad \mathbf{a}:=(a_1, a_2, a_3).
\end{align} It follows from (3.41) that
$\sum_{i=1}a_i \gt 0$, which implies that there exists at least one component of a is positive. Then we consider the following several cases.
Case 1. Three components of a are positive, that is,
$a_i=\lim_{n\rightarrow \infty}\|u_{i,1}^n\|^2 \gt 0$ for any
$i=1, 2, 3$.
Note that norms
$\|\cdot\|_{i}, i=1,2,3$ are equivalent to
$\|\cdot\|$. By (3.37), for any
$i=1,2,3$ we have
\begin{align}
\|u_{i,1}^n\|_i^2=\sum_{j=1}^3\beta_{ij}\int_{\mathbb{R}^4}|u_{i,1}^n|^{2}|u_{j,1}^n|^{2}+o(1),
\end{align}which yields that
\begin{align*}
\liminf_{n\rightarrow \infty}|u_{i,1}^n|_{4}^{4} \gt 0, \text{for}~\text{any}\; i=1,2,3.
\end{align*} We claim that there exists
$s_i^n \gt 0$,
$i=1,2,3$ such that
\begin{equation}\left(s_1^nu_{1,1}^n,s_2^nu_{2,1}^n,s_3^nu_{3,1}^n\right)\in\mathcal M\;\text{and}\; \lim_{n\rightarrow\infty}s_i^n=1.\end{equation}To prove this claim, we consider the following system
\begin{align}
\sum_{j=1}^3M_B[u_{1,1}^n,u_{2,1}^n,u_{3,1}^n]_{ij}s_j=\|u_{i,1}^n\|_i^2, ~i=1,2,3.
\end{align}By the proof of lemma 3.3, we see that
\begin{align}
M_B[u_{1,1}^n,u_{2,1}^n,u_{3,1}^n]_{ii}-\sum_{j\neq i}\left|M_B[u_{1,1}^n,u_{2,1}^n,u_{3,1}^n]_{ij}\right|\geq C, ~i=1,2,3,
\end{align} which implies that the matrix
$M_B[u_{1,1}^n,u_{2,1}^n,u_{3,1}^n]$ is strictly diagonally dominant. Thus, by the Gershgorin circle theorem, we know that the matrix
$M_B[u_{1,1}^n,u_{2,1}^n,u_{3,1}^n]$ is positive definite, and
where the constant C is independent on n. Therefore, system (3.45) has a unique solution
$(s_1^n, s_2^n, s_3^n)$, and so
$\left(s_1^nu_{1,1}^n,s_2^nu_{2,1}^n,s_3^nu_{3,1}^n\right)\in \mathcal{M}$. Moreover, by Cramer’s rule, we know that
$s_i^n$ are uniformly bounded. Set
$\overline{s}_i:=\lim_{n\rightarrow\infty}s_i^n$, and
\begin{align}
b_{ij}:=\lim_{n\rightarrow\infty}M_B[u_{1,1}^n,u_{2,1}^n,u_{3,1}^n]_{ij}.
\end{align}It follows from (3.46) that
\begin{align}
b_{ii}-\sum_{j\neq i}\left|b_{ij} \right|\geq C.
\end{align} Hence, by the Gershgorin circle theorem, we see that the matrix B is positive definite and
$\det(B)\neq 0$, where
\begin{align}
\sum_{j=1}^3b_{ij}(s_j-1)=0, ~i=1,2,3.
\end{align} We deduce form
$\det(B)\neq 0$ that
$\overline{s}_i=1, i=1,2,3$. So, (3.44) is true. Hence,
\begin{align*}
A & = \lim_{n\rightarrow \infty} I(\mathbf{u}_n)=\frac{1}{4}\lim_{n\rightarrow \infty}(\|u_1^n\|_1^2+\|u_2^n\|_2^2+\|u_3^n\|_3^2) \\
& = \frac{1}{4}\lim_{n\rightarrow \infty}\sum_{j=i}^3|s_i^n|^2\|u_{i,1}^n\|_i^2+\frac{1}{4}\lim_{n\rightarrow \infty}\sum_{j=i}^3\|u_{i,2}^n\|_i^2\\
& \gt \frac{1}{4}\lim_{n\rightarrow \infty}\sum_{j=i}^3|s_i^n|^2\|u_{i,1}^n\|_i^2\geq A,
\end{align*}which is a contradiction. Therefore, case 1 is impossible.
Case 2. Only two components of a are positive. Without loss of generality, we assume that
$a_1 \gt 0, a_2 \gt 0, a_3=0$, that is
$\lim_{n\rightarrow \infty}\|u_{1,1}^n\|^2 \gt 0$,
$\lim_{n\rightarrow \infty}\|u_{2,1}^n\|^2 \gt 0$,
$\lim_{n\rightarrow \infty}\|u_{3,1}^n\|^2=0$.
Based on lemma 3.2, it is standard to see that
$\lim_{n\rightarrow \infty}\|u_3^n\|^2 \gt 0$. Then we have
\begin{align}
\lim_{n\rightarrow \infty}\|u_{3,2}^n\|^2=\left(\lim_{n\rightarrow \infty}\|u_3^n\|^2-\lim_{n\rightarrow \infty}\|u_{3,1}^n\|^2\right) \gt 0.
\end{align} Case 2.1
$\lim_{n\rightarrow \infty}\|u_{1,2}^n\|^2=0$ and
$\lim_{n\rightarrow \infty}\|u_{2,2}^n\|^2=0$.
By (3.37), we obtain that
\begin{align}
\|u_{3,2}^n\|_3^2= \int_{\mathbb{R}^4}|u_{3,2}^n|^{4}+\beta_{31}\int_{\mathbb{R}^4}|u_{3,2}^n|^{2}|u_{1,2}^n|^{2}
+\beta_{32}\int_{\mathbb{R}^4}|u_{3,2}^n|^{2}|u_{2,2}^n|^{2} +o(1),
\end{align}which implies that
\begin{align}
\lim_{n\rightarrow \infty}\|u_{3,2}^n\|_3^2\geq S_3^{2}.
\end{align} Note that
$\lim_{n\rightarrow \infty}\|u_{3,1}^n\|^2=0$, then we deduce from (3.37) that
\begin{align}
\|u_{1,1}^n\|_1^2=\int_{\mathbb{R}^4}|u_{1,1}^n|^{4}+\beta_{21}\int_{\mathbb{R}^4}|u_{1,1}^n|^{2}|u_{2,1}^n|^{2}+o(1),
\end{align}
\begin{align}
\|u_{2,1}^n\|_2^2=\int_{\mathbb{R}^4}|u_{2,1}^n|^{4}+\beta_{21}\int_{\mathbb{R}^4}|u_{1,1}^n|^{2}|u_{2,1}^n|^{2}+o(1).
\end{align} Since
$\beta_{ij} \lt 1$ for any i ≠ j, then it is easy to see that there exist
$a_1^n,a_2^n$ such tat
\begin{align}
(a_1^nu_{1,1}^n, a_2^nu_{2,1}^n)\in \mathcal{M}_{12} ~\text{and}~ \lim_{n\rightarrow \infty}a_i^n=1,
\end{align} where
$\mathcal{M}_{12}$ is defined in (1.25). Thus,
\begin{align}
A_{12} & \leq \lim_{n\rightarrow \infty}I_{12}(a_1^nu_{1,1}^n, a_2^nu_{2,1}^n)=\frac{1}{4}\lim_{n\rightarrow \infty}
(|a_1^n|^2\|u_{1,1}^n\|_1^2+|a_2^n|^2\|u_{2,1}^n\|_2^2) \nonumber\\
& = \frac{1}{4}\lim_{n\rightarrow \infty}
(\|u_{1,1}^n\|_1^2+\|u_{2,1}^n\|_2^2)
\end{align}It follows from (3.54) and (3.58) that
\begin{align}
A & = \lim_{n\rightarrow \infty} I(\mathbf{u}_n)=\frac{1}{4}\lim_{n\rightarrow \infty}(\|u_1^n\|_1^2+\|u_2^n\|_2^2+\|u_3^n\|_3^2) \nonumber\\
&\geq \frac{1}{4}\lim_{n\rightarrow \infty}
(|a_1^n|^2\|u_{1,1}^n\|_1^2+|a_2^n|^2\|u_{2,1}^n\|_2^2+\|u_{3,2}^n\|_3^2) \nonumber\\
&\geq A_{12}+A_3,
\end{align}which contradicts with lemma 3.5. Therefore, case 2.1 is impossible.
Case 2.2 Either
$\lim_{n\rightarrow \infty}\|u_{1,2}^n\|^2=0$ or
$\lim_{n\rightarrow \infty}\|u_{2,2}^n\|^2=0$. Without loss of generality, we assume that
$\lim_{n\rightarrow \infty}\|u_{1,2}^n\|^2=0$ and
$\lim_{n\rightarrow \infty}\|u_{2,2}^n\|^2 \gt 0$.
Note that
$\lim_{n\rightarrow \infty}\|u_{1,2}^n\|^2=0$, combining this with (3.37) we know that there exist
$\widehat{s}_2^n,\widehat{s}_3^n$ such that
\begin{align}
(\widehat{s}_2^nu_{2,2}^n, \widehat{s}_3^nu_{3,2}^n)\in \mathcal{M}_{23} ~\text{and}~ \lim_{n\rightarrow \infty}\widehat{s}_i^n=1.
\end{align}Thus,
\begin{align}
A_{23} \leq \lim_{n\rightarrow \infty}I_{23}(\widehat{s}_2^nu_{2,2}^n, \widehat{s}_3^nu_{3,2}^n)
= \frac{1}{4}\lim_{n\rightarrow \infty}
(\|u_{2,2}^n\|_2^2+\|u_{3,2}^n\|_3^2).
\end{align}Combining this with lemma 3.4 and (3.58), we obtain that
\begin{align}
A & = \lim_{n\rightarrow \infty} I(\mathbf{u}_n)=\frac{1}{4}\lim_{n\rightarrow \infty}(\|u_1^n\|_1^2+\|u_2^n\|_2^2+\|u_3^n\|_3^2) \nonumber\\
&\geq \frac{1}{4}\lim_{n\rightarrow \infty}
\left(a_1^n\|u_{1,1}^n\|_1^2+a_2^n\|u_{2,1}^n\|_2^2\right)+\frac{1}{4}\lim_{n\rightarrow \infty}\left(\widehat{s}_2^n\|u_{2,2}^n\|_2^2+\widehat{s}_3^n\|u_{3,2}^n\|_3^2\right) \nonumber\\
&\geq A_{12}+A_{23} \gt A_{12}+A_3,
\end{align}which contradicts with lemma 3.5. Therefore, case 2.2 is impossible.
Case 3. Only one component of a is positive. Without loss of generality, we assume that
$a_1=0, a_2=0, a_3 \gt 0$, that is
$\lim_{n\rightarrow \infty}\|u_{1,1}^n\|^2=0$,
$\lim_{n\rightarrow \infty}\|u_{2,1}^n\|^2=0$,
$\lim_{n\rightarrow \infty}\|u_{3,1}^n\|^2 \gt 0$.
It is easy to see that
\begin{align}
\lim_{n\rightarrow \infty}\|u_{1,2}^n\|^2 \gt 0 ~\text{and} \lim_{n\rightarrow \infty}\|u_{2,2}^n\|^2 \gt 0.
\end{align}Similarly to (3.54) we have
\begin{align}
\lim_{n\rightarrow \infty}\|u_{3,1}^n\|_3^2\geq S_3^{2}.
\end{align} If
$\lim_{n\rightarrow \infty}\|u_{3,2}^n\|^2 \gt 0$, then similarly to case 1 we can obtain a contradiction. If
$\lim_{n\rightarrow \infty}\|u_{3,2}^n\|^2=0$, it is easy to get that there exist
$\widehat{a}_1^n and \widehat{a}_2^n$ such that
\begin{align}
(\widehat{a}_1^nu_{1,2}^n, \widehat{a}_2^nu_{2,2}^n)\in \mathcal{M}_{12} ~\text{and}~ \lim_{n\rightarrow \infty}\widehat{a}_i^n=1,
\end{align}and so
\begin{align}
\frac{1}{4}\lim_{n\rightarrow \infty}
(\|u_{1,1}^n\|_1^2+\|u_{2,1}^n\|_2^2)\geq A_{12}.
\end{align}Combining this with (3.54) we have
\begin{align}
A = \lim_{n\rightarrow \infty} I(\mathbf{u}_n)=\frac{1}{4}\lim_{n\rightarrow \infty}(\|u_1^n\|_1^2+\|u_2^n\|_2^2+\|u_3^n\|_3^2)\geq A_{12}+A_3,
\end{align}which contradicts with lemma 3.5. Therefore, case 3 is impossible.
From the above arguments, cases 1–3 are impossible, then (3.41) is not true. Therefore, (3.40) holds true and so does for (3.30) thanks to the definition of
$u_{i,j}$.
3.1.3. Proof of theorem 1.3-(1)
Set
\begin{align}
\kappa:=\frac{1}{2}\min_{1\leq i\leq3}\{\mathcal{S}^2-4A_i\}.
\end{align} It is easy to see that
$\mathcal{S}^2-\kappa \gt 4A_i$, for any
$i=1,2,3$. Denote
\begin{align}
\beta_{5}:=\frac{\kappa S}{6\mathcal{S}^3}, \quad
\widetilde{\beta}:=\min\{\beta_4, \beta_{5}\},
\end{align} where β 4 is defined in (3.17). From now on, we assume that
$\beta_{ij}\in (0, \widetilde{\beta})$ and fix such an βij.
The proof of theorem 1.3-(1)
Let
$\widetilde{\mathbf{u}}^n=(\widetilde{u}_1^n, \widetilde{u}_2^n,\widetilde{u}_3^n)$ be a minimizing sequence for the level A. Denote
\begin{align}
G(u_1,u_2,u_3):=\sum_{i=1}^3\left(|\nabla u_i|^2-\frac{\lambda_i}{|x|^2}\right).
\end{align} It is easy to see that there exists
$a_n \gt 0$ such that
\begin{align}
\int_{B_{a_n}}G(\widetilde{\mathbf{u}}^n)=\int_{\mathbb{R}^4\backslash B_{a_n}}G(\widetilde{\mathbf{u}}^n)=
\frac{1}{2}\sum_{i=1}^3\|\widetilde{u}_i^n\|_i^2.
\end{align}Define
\begin{align}
\widehat{u}_i^n:=a_n^{\frac{N-2}{2}}\widetilde{u}_i^n(a_nx), i=1,2,3, \text{and}~
\widehat{\mathbf{u}}^n:=(\widehat{u}_1^n,\widehat{u}_2^n,\widehat{u}_3^n).
\end{align} Then we have
$\widehat{\mathbf{u}}^n\in \mathcal{M}$ and
$I(\widehat{\mathbf{u}}^n)\rightarrow A$. Moreover, by (3.71) we know that
\begin{align}
\int_{B_1}G(\widehat{\mathbf{u}}^n)=\int_{\mathbb{R}^4\backslash B_1}G(\widehat{\mathbf{u}}^n)=
\frac{1}{2}\sum_{i=1}^3\|\widehat{u}_i^n\|_i^2\rightarrow 2A.
\end{align}By lemma 3.3, we see that
\begin{align}
\lim_{n\rightarrow \infty}I(\mathbf{u}^n)=A \text{and}~ \lim_{n\rightarrow \infty}I'(\mathbf{u}^n)=0.
\end{align}We deduce from (3.73) that
\begin{align}
\lim_{n\rightarrow \infty}\int_{B_1}G(\mathbf{u}^n)=\lim_{n\rightarrow \infty}\int_{B_1}G(\widehat{\mathbf{u}}^n)=2A,
\quad
\lim_{n\rightarrow \infty}\int_{\mathbb{R}^4\backslash B_1}G(\mathbf{u}^n)=\int_{\mathbb{R}^N\backslash B_1}G(\widehat{\mathbf{u}}^n)=2A.
\end{align} Observe that
$\{(u_1^n,u_2^n,u_3^n)\}$ are uniformly bounded in D, then up to a subsequence, we suppose that
$(u_1^n,u_2^n,u_3^n)\rightharpoonup (u_1,u_2,u_3)$ weakly in D. Hence,
$I'(u_1,u_2,u_3)=0$. Next, we will present that
$u_i\neq 0$ for any
$i=1,2,3$ and
$\mathbf{u}=(u_1,u_2,u_3)\in \mathcal{M}$ by contradiction.
Case 1.
$(u_1,u_2,u_3)=(0,0,0)$.
Based upon lemma 3.6, following the arguments in the proof of [Reference Chen and Zou7, Theorem 1.2], we can obtain that
\begin{align}
\int_{\mathbb{R}^4}\frac{|u_i^n|^2}{|x|^2}=o(1), ~\text{for}~\text{any}~ i=1,2,3.
\end{align} Combining this with
$(u_1^n,u_2^n,u_3^n)\in \mathcal{M}$, we have
\begin{align}
\int_{\mathbb{R}^4}|\nabla u_i^n|^2 & =\int_{\mathbb{R}^4}|u_i^n|^4+ \sum_{j\neq i}\int_{\mathbb{R}^4}\beta_{ij}|u_i^n|^2|u_j^n|^2+o(1) \qquad \qquad\qquad\quad\;
\end{align}
\begin{align}
\phantom{\int_{\mathbb{R}^4}|\nabla u_i^n|^2 }& \leq \mathcal{S}^{-2}\left(\int_{\mathbb{R}^4}|\nabla u_i^n|^2\right)^2+\sum_{j\neq i}\frac{\beta}{\mathcal{S}S}\|u_j^n\|_j^2\int_{\mathbb{R}^4}|\nabla u_i^n|^2+o(1)
\end{align}
\begin{align}
\phantom{\int_{\mathbb{R}^4}|\nabla u_i^n|^2 }& \leq \mathcal{S}^{-2}\left(\int_{\mathbb{R}^4}|\nabla u_i^n|^2\right)^2+ \beta\frac{6\mathcal{S}}{S}\int_{\mathbb{R}^4}|\nabla u_i^n|^2+o(1),\qquad \quad\;
\end{align}which yields that
\begin{align*}
\lim_{n\rightarrow \infty}\int_{\mathbb{R}^4}|\nabla u_i^n|^2\geq \mathcal{S}^2-\beta\frac{6\mathcal{S}^3}{S}\geq\mathcal{S}^2-\kappa \gt 4A_i, ~\text{for}~\text{any}~ i=1,2,3.
\end{align*}Therefore,
\begin{align*}
A = \lim_{n\rightarrow \infty}\frac{1}{4}\sum_{i=1}^3\|u_i^n\|_i^2
= \lim_{n\rightarrow \infty}\frac{1}{4}\sum_{i=1}^3\int_{\mathbb{R}^4}|\nabla u_i^n|^2
\geq \sum_{i=1}^3A_i,
\end{align*}which contradicts with lemma 3.5.
Case 2. Only one component of u is not zero.
Without loss of generality, we may assume that
$u_1\neq 0, u_2=0, u_3=0$. Since
$I'(u_1,u_2,u_3)(u_1,0,0)=0$, then
\begin{align}
\|u_1\|_1^2=\int_{\mathbb{R}^4}|u_1|^4\leq S_1^{-2}\|u_1\|_1^4,
\end{align}and so
Case 2.1.
$u_1^n\rightarrow u_1$ strongly in
$D^{1,2}(\mathbb{R}^4)$.
Note that
$u_1^n\rightarrow u_1$ strongly in
$L^4(\mathbb{R}^4)$. Note that
$u_i^n\rightharpoonup 0$ weakly in
$D^{1,2}(\mathbb{R}^4)$,
$i=2,3$, then we see that
$(u_i^n)^2\rightharpoonup 0$ weakly in
$L^2(\mathbb{R}^4)$. Hence,
\begin{align}
\int_{\mathbb{R}^4}|u_1^n|^2|u_i^n|^2=o(1), ~\text{for}~\text{any}~ i=2,3.
\end{align} Similarly to the proof of lemma 3.6, we know that there exist
$\overline{s}_2^n,\overline{s}_3^n$ such that
\begin{align}
(\sqrt{\overline{s}_2^n}u_{2}^n, \sqrt{\overline{s}_3^n}u_{3}^n)\in \mathcal{M}_{23} ~\text{and}~ \lim_{n\rightarrow \infty}\overline{s}_i^n=1.
\end{align}Therefore,
\begin{align}
A & = \lim_{n\rightarrow \infty} I(\mathbf{u}_n) \geq \frac{1}{4}\|u_1\|_1^2+\frac{1}{4}\lim_{n\rightarrow \infty}\left(\|u_{2}^n\|_2^2+\|u_{3}^n\|_3^2\right) \nonumber\\
&\geq \frac{1}{4}S_1^2+\frac{1}{4}\lim_{n\rightarrow \infty}\left(\overline{s}_2^n|\|u_{2}^n\|_2^2+\overline{s}_3^n\|u_{3}^n\|_3^2\right) \nonumber\\
&\geq A_{1}+A_{23},
\end{align} Case 2.2.
$\lim_{n\rightarrow \infty}\|u_1^n -u_1\|_1^2 \gt 0$.
Set
$\omega_1^n=u_1^n -u_1$. Note that
$(u_1^n,u_2^n,u_3^n)\in \mathcal{M}$, then by the Brezis–Lieb Lemma, we get that
\begin{align}
\|\omega_1^n\|_1^2=\sum_{j=1}M_B[\omega_{1}^n,u_{2}^n,u_{3}^n]_{1j}+o(1), ~\quad \|u_i^n\|_i^2=\sum_{j=1}M_B[\omega_{1}^n,u_{2}^n,u_{3}^n]_{ij}+o(1), i=2,3.
\end{align} Similarly to the proof of lemma 3.6, there exist
$\widetilde{s}_i^n \gt 0$,
$i=1,2,3$ such that
\begin{equation}\left(\sqrt{\widetilde s_1^n}\omega_1^n,\sqrt{\widetilde s_2^n}u_2^n,\sqrt{\widetilde s_3^n}u_3^n\right)\in\mathcal M\;\text{and}~\lim_{n\rightarrow\infty}\widetilde s_i^n=1.\end{equation}Therefore,
\begin{align*}
A & = \lim_{n\rightarrow \infty} I(\mathbf{u}_n)= \frac{1}{4}\|u_1\|_1^2+\frac{1}{4}\lim_{n\rightarrow \infty}(\|\omega_{1}^n\|_1^2+\|u_{2}^n\|_2^2+\|u_{3}^n\|_3^2)\\
& \gt \frac{1}{4}\lim_{n\rightarrow \infty}(\widetilde{s}_1^n\|\omega_{1}^n\|_1^2+\widetilde{s}_2^n\|u_{2}^n\|_2^2+\widetilde{s}_3^n\|u_{3}^n\|_3^2)\\
& \geq A,
\end{align*}which contradicts with lemma 3.5.
Case 3. Only two components of u are nontrivial.
Without loss of generality, we may assume that
$u_1\neq 0, u_2\neq0, u_3=0$. Since
$I'(u_1,u_2,0)=0$, then we have
Case 3.1.
$u_1^n\rightarrow u_1$ strongly in
$D^{1,2}(\mathbb{R}^4)$ and
$u_2^n\rightarrow u_2$ strongly in
$D^{1,2}(\mathbb{R}^4)$.
Since
$u_3^n\rightarrow 0$ weakly in
$D^{1,2}(\mathbb{R}^4)$, then we see that
\begin{align}
\int_{\mathbb{R}^4}|u_1^n|^2|u_3^n|^2=o(1), \quad \int_{\mathbb{R}^4}|u_2^n|^2|u_3^n|^2=o(1).
\end{align} Combining this with
$(u_1^n,u_2^n,u_3^n)\in \mathcal{M}$ we know that
\begin{align}
\|u_3^n\|_3^2=\int_{\mathbb{R}^4}|u_{3}^n|^{4}+o(1)\leq S_3^{-2}\|u_3^n\|_3^4+o(1),
\end{align}which implies that
\begin{align}
\lim_{n\rightarrow \infty}\|u_3^n\|_3^2\geq S_3^{2}.
\end{align}It follows from (3.87) and (3.90) that
\begin{align*}
A = \lim_{n\rightarrow \infty} I(\mathbf{u}_n) \geq \frac{1}{4}(\|u_1\|_1^2+\|u_2\|_2^2)+\frac{1}{4}\lim_{n\rightarrow \infty}\|u_{3}^n\|_3^2
\geq A_{12}+A_3,
\end{align*}which contradicts with lemma 3.5.
Case 3.2.
$\lim_{n\rightarrow \infty}\|u_1^n -u_1\|_1^2 \gt 0$ and
$u_2^n\rightarrow u_2$ strongly in
$D^{1,2}(\mathbb{R}^4)$.
Set
$\omega_1^n=u_1^n -u_1$. Since
$u_3^n\rightarrow 0$ weakly in
$D^{1,2}(\mathbb{R}^4)$, then we see that
\begin{align}
\int_{\mathbb{R}^4}|u_2^n|^2|u_3^n|^2=o(1), \quad \int_{\mathbb{R}^4}|u_1^n|^2|u_2^n|^2=\int_{\mathbb{R}^4}|u_1|^2|u_2|^2.
\end{align} Therefore, by Brezis–Lieb Lemma and (3.91) and
$I'(u_1,u_2,0)(u_1,0,0)=0$ we have
\begin{align}
\|\omega_1^n\|_1^2 & =\int_{\mathbb{R}^4}|\omega_1^n|^{4}+\beta_{13}\int_{\mathbb{R}^4}|\omega_1^n|^{2}|u_3^n|^{2}+o(1),\nonumber\\
\|u_3^n\|_1^2 & =\int_{\mathbb{R}^4}|u_3^n|^{4}+\beta_{13}\int_{\mathbb{R}^4}|\omega_1^n|^{2}|u_3^n|^{2}+o(1).
\end{align} Similarly to the proof of lemma 3.6, we know that there exist
$\pi_1^n,\pi_3^n \gt 0$ such that
\begin{align}
\left(\sqrt{\pi_1^n}\omega_1^n, \sqrt{\pi_3^n}u_{3}^n\right)\in \mathcal{M}_{13} ~\text{and}~ \lim_{n\rightarrow \infty}\pi_i^n=1.
\end{align}It follows from (3.10), (3.87), (3.93) and lemma 3.4 that
\begin{align*}
A & = \lim_{n\rightarrow \infty} I(\mathbf{u}_n)\geq \frac{1}{4}(\|u_1\|_1^2+\|u_2\|_2^2)+\frac{1}{4}\lim_{n\rightarrow \infty}(\|u_{3}^n\|_3^2+\|\omega_1^n\|_1^2)\\
& \geq A_{12}+\frac{1}{4}\lim_{n\rightarrow \infty}(\pi_3^n\|u_{3}^n\|_3^2+\pi_1^n\|\omega_1^n\|_1^2),\\
& \geq A_{12}+ A_{13} \gt A_2+A_{13},
\end{align*}which contradicts with lemma 3.5.
3.2. The case for N = 3
In this subsection, we present the proof of theorem 1.3-(2). Given
$\Gamma\subseteq\{1,2,3\}$ with
$|\Gamma|=2,3$, consider the following system
\begin{align}
\begin{cases}
-\Delta u_{i}-\frac{\lambda_{i}}{|x|^2}u_{i}=|u_i|^{4}u_i+\sum_{j\neq i}\beta_{ij}|u_{j}|^{3}|u_i|u_i, ~x\in \mathbb{R}^3, \\
u_i\in D^{1,2}(\mathbb{R}^3), i \in \Gamma.
\end{cases}
\end{align} Recalling
$I_\Gamma, A_\Gamma, \mathcal{M}_\Gamma$ are defined in (1.24), (1.25), (1.26), then we have the following proposition.
Proposition 3.7. Assume that
$\lambda_i\in (0, \Lambda_3)$. There exists
$\widehat{\beta} \gt 0$ such that, if
\begin{align}
0\leq \beta_{ij} \lt \widehat{\beta}, ~\text{for}~\text{any}~ i\neq j,
\end{align} then
$A_\Gamma=I_\Gamma(\mathbf{u})$ and (3.94) has a least energy positive solution.
We mention that
$\widehat{\beta}$ is defined in (3.132). When
$|\Gamma|=2$, by proposition 3.7 we have
Corollary 3.8. When N = 3,
$\Gamma=\{i,j\}$ and
$\lambda_i\in (0, \Lambda_3)$. There exists
$\widehat{\beta} \gt 0$ such that, if
\begin{align}
0\leq \beta_{ij} \lt \widehat{\beta}, ~\text{for}~\text{any}~ i\neq j,
\end{align} then
$A_\Gamma=I_\Gamma(\mathbf{u})$ and (3.94) has a least energy positive solution.
Noting that theorem 1.3-(2) is the consequence of proposition 3.7, in the following, we turn to prove that proposition 3.7 holds.
3.2.1. Preliminary results
Denote
\begin{align}
\widehat{\beta}_1:=\frac{7S^3}{12(6\overline{C})^2},
\end{align} where S is defined in (1.22) and
$\overline{C}$ is defined in (3.1).
Lemma 3.9. Assume that N = 3 and
$0\leq\beta_{ij} \lt \widehat{\beta}_1$, for any i ≠ j, then there exist
$C_3,C_4 \gt 0$ dependent on
$\lambda_i, S, \overline{C}$, such that for any
$\mathbf{u}_\Gamma \in \mathcal{M}_\Gamma$ with
$I_\Gamma(\mathbf{u}_\Gamma)\leq 2\overline{C}$ we have
Proof. The proof is similar to that of [Reference Liu, You and Zou13, Lemma 2.2], so we omit it.
Before proceeding, we define the matrix
$G_\Gamma(\mathbf{u})=(a_{ij}(\mathbf{u}))_{i,j\in \Gamma}$ by
\begin{align*}
a_{ii}(\mathbf{u})=4\int_{\mathbb{R}^3} |u_i|^6 +\sum_{\substack{j\neq i}} \int_{\mathbb{R}^3} \beta_{ij}|u_i|^3|u_j|^3,\quad
a_{ij}(\mathbf{u})=3\int_{\mathbb{R}^3}\beta_{ij}|u_i|^3|u_j|^3, ~i\neq j.
\end{align*}Set
Based on lemma 3.9 and definition of
$\widehat{\beta}_1$, following the proof of Lemma 2.3, Lemma 2.4, Lemma 2.5 in [Reference Liu, You and Zou13], we can get the following three lemmas, and we omit the proofs.
Lemma 3.10. Assume that
\begin{align*}
0 \lt \beta_{ij} \lt \widehat{\beta}_1 \quad \forall i,j = 1,2,3, i \neq j,
\end{align*} then the set
$\mathcal{M}_\Gamma \cap\Lambda_\Gamma$ is a smooth manifold. Moreover, the constrained critical points of I on
$\mathcal{M} \cap\Lambda$ are free critical points of I. In other words,
$\mathcal{M}_\Gamma \cap\Lambda_\Gamma$ is a natural constraint.
Lemma 3.11. Assume that
\begin{align*} 0 \lt \beta_{ij} \lt \widehat{\beta}_1 \quad \forall i,j = 1,2,3, i \neq j ,
\end{align*}then we have
Moreover, the constrained critical points of
$I_\Gamma$ on
$\mathcal{M}_\Gamma$ satisfying
$I_\Gamma(\mathbf{u})\leq 2 \overline{C}$ are free critical points of
$I_\Gamma$.
Lemma 3.12. Suppose that N = 3 and
$0\leq\beta_{ij} \lt \widehat{\beta}_1$, for any i ≠ j. Let
$\widehat{\mathbf{u}}_{n}
\in\mathcal{M}_\Gamma$ be a minimizing sequence of
$A_\Gamma$. Then there exists a sequence
$\{\mathbf{u}_n\}\in\mathcal{M}_\Gamma$ such that
\begin{align}
\lim_{n\rightarrow \infty} I_\Gamma(\mathbf{u}_n)=A_\Gamma, \quad \lim_{n\rightarrow \infty} I_\Gamma'(\mathbf{u}_n)=0.
\end{align}3.2.2. Energy estimates
In this subsection, we will establish the energy estimates.
Lemma 3.13. Assume that N = 3 and
$\beta_{ij}\in (0, 1),i\neq j$. Given
$\Gamma \subseteq \{1,2, 3\}$, suppose that AQ is achieved by uQ for every
$Q\subsetneq \Gamma$, then
\begin{align}
A_\Gamma \leq \min \{A_Q+\sum_{\substack{i\in I\backslash Q}} A_i:Q\subsetneq \Gamma\},
\end{align}and
\begin{align}
A_\Gamma \leq \sum_{i\in \Gamma }A_i.
\end{align}Moreover,
where we set
$A_\Gamma=A_{ij}$ when
$\Gamma=\{i,j\}$,
$A_{\Gamma}$ is defined in (1.26) and Ai is defined in (1.7).
Proof. Firstly, we show that (3.100) is true. Note that
$|\Gamma|=2$ or 3, without loss of generality, we only show that
$A\leq A_{12}+A_3$.
Step 1. Assume that
$I_{12}(\varphi^{12}_1,\varphi^{12}_2)=A_{12}$. Consider the function
$h(t_1,t_2,t_3):=I(t_1\varphi^{12}_1,t_2\varphi^{12}_2,t_3\omega_3)$. Note that
$\beta_{ij} \gt 0$ for any i ≠ j, then we see that
$h(t_1,t_2,t_3) \leq g_{12}(t_1,t_2)+g_3(t_3)$, where
\begin{align}
g_{12}(t_1,t_2):=\frac{1}{2}\sum_{i=1}^2t_i^2\|\varphi^{12}_i\|_i^2-\frac{1}{6}\sum_{i=1}^2t_i^{6}
\int_{\mathbb{R}^3}|\varphi^{12}_i|^{6}-\frac{1}{3}t_1^3t_2^3\int_{\mathbb{R}^3}\beta_{12}|\varphi^{12}_1|^3|\omega^{12}_2|^3,
\end{align}
\begin{align}
g_3(t_3):=\frac{1}{2}t_3^2\|\omega_3\|_3^2-\frac{1}{6}t_3^{6}\int_{\mathbb{R}^3}|\omega_3|^{6}.
\end{align} It is easy to see that
$\max_{t_{3} \gt 0}g_{3}(t_{3})=A_3$.
Step 2. We claim that
\begin{align}
\max_{t_{1},t_{2}\geq0}g_{12}(t_{1},t_{2})=g_{12}(1,1)=A_{12}.
\end{align} Firstly, note that
$\beta_{ij}\geq0$ for any i ≠ j, then we see that
\begin{align}
g_{12}(t_1,t_2)\leq \frac{1}{2}\sum_{i=1}^{2} t_i^2\left\|\varphi^{12}_i \right\|_i^2-\sum_{i=1}^{2} t_i^6\int_{\mathbb{R}^3}|\varphi^{12}_i|^{6} \to -\infty ,\quad \text{as}~ |\mathbf{t}|\to +\infty,
\end{align} where
$\mathbf{t}:=(t_1,t_2)$. Thus, by (3.106), we see that
$g_{12}(t_1,t_2)$ has a global maximum point in
$\overline{(\mathbb{R}^+)^2}$. Assume the global maximum point
$\mathbf{t}=(s_1,s_2)$ belongs to
$\partial \overline{(\mathbb{R}^+)^2}$. Without loss of generality, we assume that
$s_1=0$ and
$s_2 \gt 0$, then
\begin{align*}
g_{12}(0,s_2)=\frac{1}{2} s_2^2\left\|\varphi^{12}_2 \right\|_2^2-\frac{1}{6}s_2^6\int_{\mathbb{R}^3}\beta_{22} |\varphi^{12}_2|^6.
\end{align*}For s > 0 small enough, we have
\begin{align*}
g_{12}(s,s_2)-g_{12}(0,s_2)=\frac{1}{2}s^2\left\|\omega_1 \right\|_1^2-\frac{1}{6}s^6\int_{\mathbb{R}^3}\beta_{11}|\omega_1|^6-\frac{1}{3}s^3 s_2^3\int_{\mathbb{R}^3}\beta_{12} |\omega_1|^3|\varphi^{12}_2|^3 \gt 0,
\end{align*} which contradicts to the fact that
$(0,s_2)$ is a global maximum of
$\overline{(\mathbb{R}^+)^2}$. Thus, the global maximum point
$(s_1,s_2)$ is a interior point in
$ (\mathbb{R}^+)^2$, and so
$(s_1,s_2)$ is an critical point.
Based on the above arguments, we also know that
\begin{align*}
\max_{t_{1},t_{2}\geq0}g_{12}(t_{1},t_{2}) \gt \max\left\{\max_{t_{1} \gt 0}g_{12}(t_{1},0), \max_{t_{2} \gt 0}f_{12}(0,t_{2})\right\}=\max\{A_1, A_2\}.
\end{align*} Secondly, we prove that the critical point of
$g_{12}(t_{1},t_{2})$ is unique in
$(\mathbb{R}^+)^2$. Consider
\begin{align}
\tilde{g}_{12}(t_1,t_2)=\frac{1}{2}\sum_{i=1}^{2} t_i^{\frac{2}{3}}\left\|\varphi^{12}_i \right\|_i^2-\frac{1}{6}\sum_{i,j=1}^{2} t_it_j\int_{\mathbb{R}^3}\beta_{ij} |\varphi^{12}_i|^3|\varphi^{12}_j|^3 .
\end{align}By a direct calculation,
\begin{align}
\begin{aligned}
&\frac{\partial \tilde{g}_{12}}{\partial t_i}(t_1,t_2)=\frac{1}{3}t_i^{-\frac{1}{3}}\left\|\varphi^{12}_i \right\|_i^2-\frac{1}{3}\sum_{j=1}^2 t_j\int_{\mathbb{R}^3}\beta_{ij} |\varphi^{12}_i|^3|\varphi^{12}_j|^3 ,\quad 1\leq i \leq 2,\\
&\frac{\partial^2 \tilde{g}_{12}}{\partial t_i^2}(t_1,t_2)=-\frac{1}{9}t_i^{-\frac{4}{3}}\left\|\varphi^{12}_i \right\|_i^2-\frac{1}{3}\int_{\mathbb{R}^3}\beta_{ii} |\varphi^{12}_i|^6 , \quad 1\leq i \leq 2,\\
&\frac{\partial^2 \tilde{g}_{12}}{\partial t_i \partial t_j}(t_1,t_2)=-\frac{1}{3}\int_{\mathbb{R}^3}\beta_{12} |\varphi^{12}_1|^3|\varphi^{12}_2|^3.
\end{aligned}
\end{align} Thus, the Hessian matrix of
$\tilde{g}_{12}$ is
\begin{align}
\begin{aligned}
H(\tilde{g}_{12})& =-\frac{1}{9}\left(\begin{matrix}
t_1^{-\frac{4}{3}}\|\varphi^{12}_1 \|_1^2 & 0 \\
0 & t_2^{-\frac{4}{3}}\|\varphi^{12}_2 \|_2^2
\end{matrix}\right)
-\frac{1}{3}\left(\begin{matrix}
\int_{\mathbb{R}^3}\beta_{11} |\varphi^{12}_1|^6 & \int_{\mathbb{R}^3}\beta_{12} |\varphi^{12}_1|^3|\varphi^{12}_2|^3\\
\int_{\mathbb{R}^3}\beta_{12} |\varphi^{12}_1|^3|\varphi^{12}_2|^3 & \int_{\mathbb{R}^3}\beta_{22} |\varphi^{12}_2|^6
\end{matrix}\right)\\
&=: -\frac{1}{9}B(\mathbf{t})-\frac{1}{3}M_B[\varphi^{12}_1,\varphi^{12}_2],
\end{aligned}
\end{align} where
$ \mathbf{t}=(t_1,t_2)\in (\mathbb{R}^+)^2 $. Note that the matrix
$B(\mathbf{t})$ is positive definite. Next, we claim that the matrix
$M_B[\varphi^{12}_1,\varphi^{12}_2]$ is also positive definite. In fact, since
$\beta_{ij} \lt 1$ for any i ≠ j, then we have
\begin{align*}
\int_{\mathbb{R}^3}|\varphi^{12}_1|^6 \int_{\mathbb{R}^3}|\varphi^{12}_2|^6-\left(\beta_{12}\int_{\mathbb{R}^3}|\varphi^{12}_1|^3|\varphi^{12}_2|^3\right)^2 \gt 0,
\end{align*} which yields that the matrix
$M_B[\varphi^{12}_1,\varphi^{12}_2]$ is positive definite. Therefore, the Hessian matrix of
$\tilde{g}_{12}$ is negative definite, which implies that
$\tilde{g}_{12}$ has a unique critical point. Therefore, the critical point must be the global maximum point. Notice that
$ \tilde{g}_{12}(t_1^3,t_2^3) = g_{12}(t_1,t_2)$, thus g 12 has a unique critical point, which must be the global maximum point. Since
$(1,1)$ is a critical point of
$g_{12}(t_{1},t_{2})$, then
$(1,1)$ is a maximum point of g 12. Therefore,
\begin{align}
\max_{t_{1},t_{2} \gt 0}g_{12}(t_{1},t_{2})=g_{12}(1,1)=A_{12}.
\end{align}Hence, based on the above arguments, we obtain that
\begin{align}
\max_{t_{1},t_{2},t_3 \gt 0}h(t_{1},t_{2},t_3)\leq A_{12}+A_3.
\end{align} Step 3. We show that there exist
$\widetilde{t}_1,\widetilde{t}_2,\widetilde{t}_3 \gt 0$ such that
$(\widetilde{t}_1\varphi^{12}_1,\widetilde{t}_2\varphi^{12}_2,
\widetilde{t}_3\omega_3)\in\mathcal{M}$. It is easy to see that
$h(t_1,t_2,t_3)$ has a global maximum in
$\overline{(\mathbb{R}^+)^3}$. Following the proof of [Reference Liu, You and Zou13, Proposition 2.1], any global maximum point of
$h(t_1,t_2,t_3) $ does not belong to
$\partial \overline{(\mathbb{R}^+)^3}$, and any global maximum point
$(\widetilde{t}_1,\widetilde{t}_2,\widetilde{t}_3)$ is an interior point in
$(\mathbb{R}^+)^3$. It is standard to see that
If
$|Q|=1$, then by (3.100) we get that
\begin{align*}
A_\Gamma \leq \sum_{i\in \Gamma }A_i.
\end{align*}Set
\begin{align}
\widehat{\beta}_2:=\min\left\{\frac{C_3S^{\frac{3}{2}}}{2C_4^{\frac{1}{2}}(6\overline{C})^{\frac{3}{2}}},
\frac{C_3S^3}{12(6\overline{C})^2}\right\}
\end{align} where
$C_3, C_4$ are defined in lemma 3.9, S is defined in (1.22) and
$\overline{C}$ is defined in (3.1).
Lemma 3.14. Suppose that
$\beta_{ij}\in (0, \widehat{\beta}_2)$ and let
$\mathbf{v}_{n}\in \mathcal{M}_\Gamma$ be a minimizing sequence of
$A_\Gamma$ with
$I(\mathbf{v}_{n})\leq 2\overline{C}$, and
$\mathbf{v}_{n}\rightharpoonup \mathbf{0}$. Then for any
$\widetilde{\sigma} \gt 0$ and for every
$\widetilde{\delta}\in (-\widetilde{\sigma},0)\cup (0,\widetilde{\sigma})$, there exists
$\widetilde{\rho}\in (-\widetilde{\delta},0)\cup (0,\widetilde{\delta})$ such that, up to a subsequence,
\begin{align}
\text{either} ~\sum_{i\in \Gamma}\int_{B_{\widetilde{\sigma}+\widetilde{\rho}}}|\nabla v_i^n|^2\rightarrow 0 ~\text{or}
~\sum_{i\in \Gamma}\int_{\mathbb{R}^3 \backslash B_{\widetilde{\sigma}+\widetilde{\rho}}}|\nabla v_i^n|^2\rightarrow 0.
\end{align}Proof. Without loss of generality, we present the proof for the case of
$\Gamma=\{1, 2, 3\}$. The idea of the following proof is similar to that of lemma 3.6. By lemma 3.12,
$I'(v_1^n,v_2^n,v_3^n)\rightarrow 0$ as
$n\rightarrow \infty$. As in lemma 3.6, for any
$i=1, 2, 3$, we consider
\begin{align}
v_{i,1}^n(x)=
\begin{cases}
v_i^n(x) ~~ \text{if}~ x\in B_{\widetilde{\sigma}+\widetilde{\rho}},\\
\widetilde{\omega}_{i,1}^n(x) ~~ \text{if}~ x\in B_{\widetilde{\sigma}+\widetilde{\delta}}\setminus B_{\widetilde{\sigma}+\widetilde{\rho}},\\
0 ~~\text{elsewhere},
\end{cases}
~~v_{i,2}^n(x)=
\begin{cases}
0 ~~ \text{if}~ x\in B_{\widetilde{\sigma}-\widetilde{\delta}},\\
\widetilde{\omega}_{i,2}^n(x) ~~ \text{if}~~ x\in B_{\widetilde{\sigma}+\widetilde{\rho}}\setminus B_{\widetilde{\sigma}-\widetilde{\delta}},\\
v_i^n(x) ~~\text{elsewhere},
\end{cases}
\end{align} where
$\widetilde{\omega}^n_{i,1}\rightarrow 0$ strongly in
$H^1(B_{\widetilde{\sigma}+\widetilde{\delta}}\backslash B_{\widetilde{\sigma}+\widetilde{\rho}})$ and
$\widetilde{\omega}^n_{i,2}\rightarrow 0$ strongly in
$H^1(B_{\widetilde{\sigma}+\widetilde{\rho}}\backslash B_{\widetilde{\sigma}-\widetilde{\delta}})$. Obviously,
\begin{align}
\|v_i^n\|_i^2= \|v_{i,1}^n\|_i^2+\|v_{i,2}^n\|_i^2+o(1).
\end{align}Next, we claim that
\begin{align}
\text{either}~ \lim_{n\rightarrow\infty}\sum_{i=1}^3\|v_{i,1}^n\|^2=0 \,\,\text{or}~
\lim_{n\rightarrow\infty}\sum_{i=1}^3\|v_{i,2}^n\|^2=0.
\end{align}By contradiction, if (3.117) does not hold, then up to a subsequence,
\begin{align}
\text{both}~ \sum_{i=1}^3\lim_{n\rightarrow\infty}\|v_{i,1}^n\|^2 \gt 0 \,\,\text{and}~
\sum_{i=1}^3\lim_{n\rightarrow\infty}\|v_{i,2}^n\|^2 \gt 0.
\end{align}Set
\begin{align}
b_i:=\lim_{n\rightarrow\infty}\|v_{i,1}^n\|^2, \quad \mathbf{b}:=(b_1, b_2, b_3).
\end{align} It follows from (3.118) that
$\sum_{i=1}^3b_i \gt 0$, which implies that there exists at least one component of b is positive.
Assume that any component of b is positive, that is,
$b_i=\lim_{n\rightarrow \infty}\|v_{i,1}^n\|^2 \gt 0$ for any
$i=1,2,3$. Note that the norm
$\|\cdot\|_{i}$ is equivalent to
$\|\cdot\|$ for any
$i=1,2,3$ and
\begin{align}
\|v_{i,1}^n\|_i^2=\sum_{j=1}^3\beta_{ij}\int_{\mathbb{R}^3}|v_{i,1}^n|^{3}|v_{j,1}^n|^{3}+o(1), \quad i=1,2,3,
\end{align}which yields that
\begin{align}
\liminf_{n\rightarrow \infty}|v_{i,1}^n|_{6}^{6} \gt 0.
\end{align} We claim that there exist
$t_i^n \gt 0$,
$i=1,2,3$ such that
\begin{align}
\left(t_1^nv_{1,1}^n,t_2^nv_{2,1}^n,t_3^nv_{3,1}^n\right)\in \mathcal{M} \,\,\text{and}~ \lim_{n\rightarrow \infty}t_i^n=1.
\end{align}To prove this claim, we consider the following function
\begin{align*}
f(t_1,t_2,t_3) & := I(t_1v_{1,1}^n,t_2v_{2,1}^n,t_3v_{3,1}^n) \\
& = \frac{1}{2}\sum_{i=1}^{3}t_i^2\|v_{i,1}^n\|_i^2-\frac{1}{6}
\sum_{i,j=1}^{3}t_i^3t_j^3\beta_{ij}\int_{\mathbb{R}^3}|v_{i,1}^n|^3|v_{j,1}^n|^3.
\end{align*} By (3.116) we get that
$\|v_{i,1}^n\|_i^2\leq 7\overline{C}$ for n large enough. Combining this with lemma 3.9 and
$\beta_{ij}\geq 0$ for any i ≠ j we have
\begin{align}
f(t_1,t_2,t_3) & \leq \frac{1}{2}\sum_{i=1}^{3}t_i^2\|v_{i,1}^n\|_i^2-\frac{1}{6}
\sum_{i=1}^{3}t_i^6\int_{\mathbb{R}^3}|v_{i,1}^n|^6
\end{align}
\begin{align}
& \leq \sum_{i=1}^{3}\left(\frac{7\overline{C}}{2}t_i^2-\frac{1}{6}C_3
t_i^6\right),
\end{align}where C 3 is defined in lemma 3.9. Hence,
\begin{align}
f(t_1,t_2,t_3) \lt 0 \,\,\text{when}~ t_i\in (\overline{t}, +\infty), \text{where}~ \overline{t}= \sqrt[4]{\frac{7\overline{C}}{C_3}}.
\end{align} Therefore,
$f(t_1,t_2,t_3)$ has a global maximum in
$\overline{(\mathbb{R}^+)^3}$. Similarly to the proof of lemma 3.13, the global maximum point
$(t_1^n,t_2^n,t_3^n)$ is an interior point in
$(\mathbb{R}^+)^3$. It follows that
where
$\overline{t}$ is defined in (3.125). Denote
$\widehat{v}_{i}^n=t_i^nv_{i,1}^n, i=1,2,3$, then
$(\widehat{v}_{1}^n, \widehat{v}_{2}^n, \widehat{v}_{3}^n)\in \mathcal{M}$ and
$I(\widehat{v}_{1}^n, \widehat{v}_{2}^n, \widehat{v}_{3}^n)\leq 2\overline{t}^2\overline{C}$. Following the proof of lemma 3.9, there exists a constant
$\widetilde{C}_3 \gt 0$ dependent on
$\lambda_i, S, \overline{C}$ such that
\begin{align}
|\widehat{v}_{i}^n|_6^6\geq \widetilde{C}_3 ~\text{if} ~0 \lt \beta_{ij} \lt \widehat{\beta}_2.
\end{align}Set
\begin{align*}
B_{ij}^n:= \int_{\mathbb{R}^3}\beta_{ij}|v_{i,1}^n|^3|v_{j,1}^n|^3.
\end{align*} Note that
$(t_1^nv_{1,1}^n,t_2^nv_{2,1}^n,t_3^nv_{3,1}^n)\in \mathcal{M}$, then we have
\begin{align}
|t_i^n|^2\|v_{i,1}^n\|_i^2=\sum_{j=1}^3|t_i^n|^{3}|t_j^n|^{3}B_{ij}^n, ~i=1,2,3.
\end{align}Up to a subsequence, we can assume that
\begin{align*}
\overline{B}_{ij}:=\lim_{n\rightarrow\infty} B_{ij}^n= \lim_{n\rightarrow\infty}\int_{\mathbb{R}^3}\beta_{ij}|v_{i,1}^n|^3|v_{j,1}^n|^3,\quad
a_i:= \lim_{n\rightarrow\infty} \|v_{i,1}^n\|_i^2,\quad
\overline{t}_i:=\lim_{n\rightarrow\infty} t_i^n.
\end{align*}It follows from (3.126) that
\begin{align}
\lim_{n\rightarrow\infty}|\widehat{v}_{i}^n|_6^6= \overline{t}_i^6 \lim_{n\rightarrow\infty}|v_{i,1}^n|_6^6\geq \widetilde{C}_3 \gt 0,
\end{align}which yields that
Next, we prove that
$\overline{t}_i=1, i=1,2,3.$ Consider
\begin{align*}
\overline{f}(s_1,s_2,s_3):=\frac{1}{2}\sum_{i=1}^3a_is_i^2-\frac{1}{6}\sum_{i,j=1}^3\overline{B}_{ij}s_i^3s_j^3.
\end{align*} By (3.120) and (3.127) we know that both
$(1,1,1)$ and
$(\overline{t}_1, \overline{t}_2, \overline{t}_3)$ are critical points of
$\overline{f}(s_1,s_2,s_3)$. We claim that the critical point of
$\overline{f}(s_1,s_2,s_3)$ is unique in
$(\mathbb{R}^+)^3$. In fact, for any
$i=1,2,3$ we have
\begin{align*}
\int_{\mathbb{R}^3}|v_{i,1}^n|^6- \sum_{j\neq i}\left|\int_{\mathbb{R}^3}\beta_{ij}|v_{i,1}^n|^3|v_{j,1}^n|^3\right|
& \geq C_3-\widehat{\beta}_2 S^{-\frac{3}{2}} |v_{i,1}^n|_6^3\|v_{j,1}^n\|_j^3\\
& \geq C_3-\widehat{\beta}_2 S^{-\frac{3}{2}}C_4^{\frac{1}{2}}(6\overline{C})^{\frac{3}{2}}\\
& \geq \frac{1}{2}C_3,
\end{align*} where C 3 and C 4 are defined in lemma 3.9. Thus, for any
$i=1,2,3$ we have
\begin{align*}
\overline{B}_{ii}-\sum_{j\neq i}\overline{B}_{ij}\geq \frac{1}{2}C_3.
\end{align*} By Gershgorin circle theorem, we know that the matrix
$\overline{B}=(\overline{B}_{ij})_{1\leq i,j\leq3}$ is positive definite. Then following the arguments of lemma 3.13, we can get that the critical point of
$\overline{f}(s_1,s_2,s_3)$ is unique in
$(\mathbb{R}^+)^3$. By (3.129) we get that
$\overline{t}_i \gt 0, i=1,2,3$, thus
$(\overline{t}_1, \overline{t}_2, \overline{t}_3)=(1,1,1)$. Therefore,
\begin{align*}
A & = \lim_{n\rightarrow \infty} I(\mathbf{v}_n)= \frac{1}{3}\lim_{n\rightarrow \infty}\sum_{j=i}^3|t_i^n|^2\|v_{i,1}^n\|_i^2+\frac{1}{3}\lim_{n\rightarrow \infty}\sum_{j=i}^3\|v_{i,2}^n\|_i^2\\
& \gt \frac{1}{3}\lim_{n\rightarrow \infty}\sum_{j=i}^3|t_i^n|^2\|v_{i,1}^n\|_i^2\geq A,
\end{align*}which contradicts with lemma 3.13. Therefore, at least one component of b is negative. Similarly as above, by virtue of lemmas 3.13 and 3.6, ones can show that any component of b should be negative. This yields that (3.118) does not hold and then (3.114) and (3.117) are true.
3.2.3. Proof of proposition 3.7
Recall that
$ A_i \lt \frac{1}{3}\mathcal{S}^{\frac{3}{2}}$ (see (1.7)) for every
$1 \leq i \leq 3.$ Set
\begin{align}
\pi:=\frac{1}{2}\min_{1\leq i \leq 3}\{\mathcal{S}^3- (3A_i)^2\} \gt 0,
\end{align}then we have
Set
\begin{align}
\widehat{\beta}_3:=\frac{\pi S^3}{2\mathcal{S}^3(6\overline{C})^2} \quad
\text{and}~
\widehat{\beta}:=\min\{\widehat{\beta}_1, \widehat{\beta}_2, \widehat{\beta}_3, 1\},
\end{align} where
$\widehat{\beta}_1$ is defined in (3.97),
$\widehat{\beta}_2$ is defined in (3.113). From now on, we assume that
$\beta_{ij}\in (0, \widehat{\beta})$ and fix such an βij.
The proof of proposition 3.7
We will proceed by mathematical induction on the number of the equations in the subsystem. Set
$|\Gamma|=q$, that is q the number of the equations in the subsystem, and
$q=1,\ldots, 3$.
When q = 1, the system reduces to the following problem
\begin{align}
-\Delta v-\frac{\lambda_{i}}{|x|^2}v=v^{2^*-1}, ~ x\in \mathbb{R}^N,
\end{align}and we know proposition 3.7 is true.
Suppose by induction hypothesis that proposition 3.7 holds true for every level
$\mathcal{C}_\Gamma$ with
$|\Gamma|\leq q$ for some
$1\leq q\leq 2$, we need prove proposition 3.7 for
$\mathcal{C}_\Gamma$ with
$|\Gamma|=q+1$. Without loss of generality, we only present the proof for
$\Gamma=\{1,\ldots,q+1\}$. By induction hypothesis, lemma 3.13 is true for
$\mathcal{C}_\Gamma$.
Let
$\widetilde{\mathbf{u}}_\Gamma^n=(\widetilde{u}_1^n, \cdots,\widetilde{u}_{q+1}^n)$ be a minimizing sequence for the level
$A_\Gamma$. Recall that
\begin{align}
G(\widetilde{\mathbf{u}}_\Gamma^n)=\sum_{i \in \Gamma}\left(|\nabla u_i|^2-\frac{\lambda_i}{|x|^2}\right).
\end{align} Similarly to the proof of theorem 1.3-(1), we see that there exists a sequence
$\{(u_1^n,\cdots,u_{q+1}^n)\}\in \mathcal{M}_\Gamma$ such that
\begin{align}
\lim_{n\rightarrow \infty}I(\mathbf{u}^n)=A_\Gamma \,\,\text{and}~ \lim_{n\rightarrow \infty}I'(\mathbf{u}^n)=0.
\end{align} Observe that
$\{(u_1^n,\ldots,u_{q+1}^n)\}$ are uniformly bounded in D, then up to a subsequence we suppose that
$(u_1^n,\ldots,u_{q+1}^n)\rightharpoonup (u_1,\ldots,u_{q+1})$ weakly in D. Hence,
$I'(u_1,\ldots,u_{q+1})=0$. Next, we will present that
$u_i\neq 0$ for any
$i=1,\ldots,q+1$ and
$\mathbf{u}=(u_1,\ldots,u_{q+1})\in \mathcal{M}_\Gamma$ by contradiction.
Case 1.
$(u_1,\ldots,u_{q+1})=(0,\ldots,0)$.
Based upon lemma 3.14, following the arguments in the proof of [Reference Chen and Zou7, Theorem 1.2], we can get that
\begin{align}
\int_{\mathbb{R}^3}\frac{|u_i^n|^2}{|x|^2}=o(1), ~\text{for}~\text{any}~ i\in \Gamma.
\end{align}Set
\begin{align}
d_i:= \lim_{n\rightarrow\infty} \int_{\mathbb{R}^3}|\nabla u_i^n|^2.
\end{align} Combining this with
$(u_1^n,\ldots,u_{q+1}^n)\in \mathcal{M}_\Gamma$, we have
\begin{align}
\int_{\mathbb{R}^3}|\nabla u_i^n|^2 & =\int_{\mathbb{R}^3}|u_i^n|^6+ \sum_{j\neq i,j\in \Gamma}\int_{\mathbb{R}^3}\beta_{ij}|u_i^n|^3|u_j^n|^3+o(1) \nonumber\\
& \leq \mathcal{S}^{-2}\left(\int_{\mathbb{R}^3}|\nabla u_i^n|^2\right)^3+ \beta S^{-3}(6\overline{C})^{\frac{3}{2}}\left(\int_{\mathbb{R}^3}|\nabla u_i^n|^2+o(1)\right)^{\frac{3}{2}},
\end{align}which yields that
\begin{align}
d_i\leq \mathcal{S}^{-3}d_i^3+\beta S^{-3}(6\overline{C})^{\frac{3}{2}}d_i^{\frac{3}{2}}\leq \mathcal{S}^{-3}d_i^3+\beta S^{-3}(6\overline{C})^2d_i, ~\text{for}~\text{any}~ i\in \Gamma.
\end{align}Thus, we get that
and so
Therefore,
\begin{align*}
A_\Gamma & =\lim_{n\rightarrow \infty}\frac{1}{3}\sum_{i\in \Gamma}\|u_i^n\|_i^2
= \lim_{n\rightarrow \infty}\frac{1}{3}\sum_{i\in \Gamma}\int_{\mathbb{R}^3}|\nabla u_i^n|^2\\
&= \frac{1}{3}\sum_{i\in \Gamma} d_i \gt \sum_{i\in \Gamma}A_i,
\end{align*}which contradicts with lemma 3.13.
Case 2. Only one component of u is nontrivial.
Without loss of generality, we may assume that
$u_1\neq 0, u_2=0=\ldots=u_{q+1}=0$. Since
$I'(u_1,\ldots,u_{q+1})(u_1,\cdots,0)=0$, then
\begin{align*}
\|u_1\|_1^2=\int_{\mathbb{R}^3}|u_1|^6\leq S_1^{-3}\|u_1\|_1^6,
\end{align*} and so
$\|u_1\|_1^2\geq S_1^{\frac{3}{2}}.$
Case 2.1.
$u_1^n\rightarrow u_1$ strongly in
$D^{1,2}(\mathbb{R}^3)$.
Note that
$u_1^n\rightarrow u_1$ strongly in
$L^6(\mathbb{R}^3)$ and
$u_i^n\rightharpoonup 0$ weakly in
$D^{1,2}(\mathbb{R}^3)$,
$i=2,\ldots,q+1$, then we see that
\begin{align}
\int_{\mathbb{R}^3}|u_1^n|^3|u_i^n|^3=o(1), ~\text{for}~\text{any}~ i\in \Gamma.
\end{align} Note that q = 1 or 2. Then by (3.141) and
$\mathbf{u}_\Gamma^n \in \mathcal{M}_\Gamma$ we have
\begin{align}
\|u_{i}^n\|_i^2=\int_{\mathbb{R}^3}|u_{i}^n|^{6}+\beta_{ij}\int_{\mathbb{R}^3}|u_{i}^n|^{3}|u_{j}^n|^{3}+o(1), ~i\neq 1, i\in \Gamma.
\end{align} Similarly to the proof of lemma 3.6, we know that there exist
$\tau_2^n,\ldots,\tau_{q+1}^n$ such that
\begin{align}
(\tau_2^nu_{2}^n,\cdots, \tau_{q+1}^nu_{q+1}^n)\in \mathcal{M}_{\{2,\cdots, q+1\}} ~\text{and}~ \lim_{n\rightarrow \infty}\tau_i^n=1.
\end{align}Therefore,
\begin{align}
A_\Gamma & \geq \frac{1}{3}\|u_1\|_1^2+\frac{1}{3}\lim_{n\rightarrow \infty}\sum_{i=2}^{q+1}\|u_{i}^n\|_i^2\nonumber\\
&\geq \frac{1}{3}S_1^{\frac{3}{2}}+\frac{1}{3}\lim_{n\rightarrow \infty}\sum_{i=2}^{q+1}\tau_i^n|\|u_{i}^n\|_i^2 \nonumber\\
&\geq A_{1}+A_{\{2,\ldots, q+1\}},
\end{align} Case 2.2.
$\lim_{n\rightarrow \infty}\|u_1^n -u_1\|_1^2 \gt 0$.
Set
$\omega_1^n=u_1^n -u_1$. Similarly to the proof of lemma 3.14 there exist
$\alpha_i^n \gt 0$,
$i\in \Gamma$ such that
\begin{align}
\left(\alpha_1^n\omega_{1}^n,\ldots,\alpha_{q+1}^nu_{q+1}^n\right)\in \mathcal{M}_\Gamma \,\,\text{and}~ \lim_{n\rightarrow \infty}\alpha_i^n=1.
\end{align}Therefore,
\begin{align*}
A_\Gamma &= \frac{1}{3}\|u_1\|_1^2+\frac{1}{3}\lim_{n\rightarrow \infty}(\|\omega_{1}^n\|_1^2+\sum_{i=2}^{q+1}\|u_{i}^n\|_i^2)\\
& \gt \frac{1}{3}\lim_{n\rightarrow \infty}(\alpha_1^n\|\omega_{1}^n\|_1^2+\sum_{i=2}^{q+1}\alpha_i^n\|u_{i}^n\|_i^2)\\
& \geq A_\Gamma,
\end{align*}which contradicts with lemma 3.13.
Case 3. Only two component of u are nontrivial. Without loss of generality, we may assume that
$u_1\neq 0, u_2\neq0, u_{q+1}=u_3=0$. Similarly to the proof of theorem 1.3-(1), we see that Case 3 is impossible.
The proof of theorem 1.3-(2)
Taking
$\Gamma=\{1, 2, 3\}$, by proposition 3.7 we see that theorem 1.3-(2) is true.
4. Nonexistence of least energy positive solutions for the mixed case
In this section, we focus on the nonexistence of least energy solutions for the mixed case (i.e.
$\beta_{12} \gt 0, \beta_{13} \lt 0, \beta_{23} \lt 0$), and prove theorem 1.7-1.8.
4.1. The case N = 4
In this subsection, we assume that
$0 \lt \beta_{12} \lt \widetilde{\beta}, \beta_{13} \lt 0, \beta_{23} \lt 0$, and show the proof of theorem 1.7-(1). Define
\begin{align}
\widetilde{\mathcal{M}}_{12}:=\left\{u_1\neq 0, u_2\neq 0, \int_{\mathbb{R}^4}\left(|\nabla u_i|^{2}-\frac{\lambda_{i}}{|x|^2} u_i^{2}\right)\leq\sum_{j=1}^2\int_{\mathbb{R}^4}\beta_{ij}|u_i|^{2}|u_j|^{2}, i=1,2\right\}
\end{align}and
\begin{align}
\widetilde{A}_{12}:=\inf_{(u_1,u_2)\in \widetilde{\mathcal{M}}_{12}}I_{12}(u_1,u_2).
\end{align}Lemma 4.1. Assume that
$\lambda_1=\lambda_2$ and
$0 \lt \beta_{12} \lt \widetilde{\beta}$. Then we have
where A 12 is defined in (1.26).
Proof. Based on inequality (4.1), employing the proof of [Reference Chen and Zou6, Theorem 1.1], we see that there exist
$k,l \gt 0$ such that
$A_{12}=(\sqrt{k}\omega_1, \sqrt{l}\omega_1)$, where ω 1 is a least energy positive solution of (1.4). Similarly, we also can get that
$\widetilde{A}_{12}=(\sqrt{k}\omega_1, \sqrt{l}\omega_1)$. Therefore,
$\widetilde{A}_{12}=A_{12}$.
Consider
\begin{align*}
\widetilde{\mathcal{M}}_i:=\left\{u\neq 0, ~\int_{\mathbb{R}^N}\left(|\nabla u|^{2}-\frac{\lambda_{i}}{|x|^2} u^{2}\right)\leq\int_{\mathbb{R}^N}|u|^{2^*}\right\},
\end{align*} and
$\widetilde{A}_i:=\inf_{u\in \widetilde{\mathcal{M}}} \frac{1}{N}\int_{\mathbb{R}^N}\left(|\nabla u|^{2}-\frac{\lambda_{i}}{|x|^2} u^{2}\right)$.
Lemma 4.2. Assume that
$N\geq3$, then
$\widetilde{A}_i=A_i$, where Ai is defined in (1.26).
Proof. Note that
$\widetilde{A}_i\leq A_i$. If
$v\in \widetilde{\mathcal{M}}_i$ then take
$t^{2^*-2}:=\int_{\mathbb{R}^N}\left(|\nabla v|^{2}-\frac{\lambda_{i}}{|x|^2} v^{2}\right)/(\int_{\mathbb{R}^N}|v|^{2^*})\leq 1 $, so that
$tv\in \mathcal{M}_i$, where
$\mathcal{M}_i$ is defined in (1.25). Then
\begin{align*}
A_i \leq \frac{t^{\frac{2}{2^*-2}}}{N}\int_{\mathbb{R}^N}\left(|\nabla v|^{2}-\frac{\lambda_{i}}{|x|^2} v^{2}\right),
\end{align*} and by taking the infimum for
$v\in \widetilde{\mathcal{M}}$ we get
$A_i\leq \widetilde{A}_i$ and so
$\widetilde{A}_i=A_i$.
Lemma 4.3. Assume that
$0 \lt \beta_{12} \lt \widetilde{\beta}, \beta_{13} \lt 0, \beta_{23} \lt 0$, then
$A\geq A_3+A_{12}$.
Proof. This was proved in [Reference Lin and Wei12, Theorem 1] for subcritical case, but we sketch it here for completeness. Since
$\beta_{13} \lt 0, \beta_{23} \lt 0$, then for any
$\mathbf{u}=(u_1,u_2,u_3)\in \mathcal{M}$ we have,
$
I(u_1,u_2,u_3)\geq I_{12}(u_1,u_2)+I_3(u_3),
$ where
\begin{align*}
I_{12}(u_1,u_2)=\sum_{i=1}^2\frac{1}{2}\int_{\mathbb{R}^4}\left(|\nabla u_i|^{2}-\frac{\lambda_{i}}{|x|^2} u_i^{2}\right)-\frac{1}{4}\sum_{i,j=1}^2\int_{\mathbb{R}^4}\beta_{ij}|u_i|^{2}|u_j|^{2},
\end{align*}and
\begin{align*}
I_3(u_3)=\frac{1}{2}\int_{\mathbb{R}^4}\left(|\nabla u_3|^{2}-\frac{\lambda_{3}}{|x|^2} u_3^{2}\right)-\frac{1}{4}\int_{\mathbb{R}^4}|u_3|^{4}.
\end{align*}Thus, lemma 4.1 and lemma 4.2 yields that
Therefore,
$A \geq A_3+A_{12}$.
Lemma 4.4. Assume that
$0 \lt \beta_{12} \lt \widetilde{\beta}, \beta_{13} \lt 0, \beta_{23} \lt 0$, then
$A\leq A_3+A_{12}$.
Proof. Note that
$0 \lt \beta_{12} \lt \widetilde{\beta}$, then A 12 is attained and we assume that
$A_{12}=I_{12}(\omega^{12}_1,\omega^{12}_2)$. Recall that
$z^3_\mu$ is defined in (1.5). Since
$z^3_\mu\rightharpoonup 0$ in
$D^{1,2}(\mathbb{R}^4)$ as
$\mu\rightarrow \infty$, then
$(z^3_\mu)^2\rightharpoonup 0$ in
$L^{2}(\mathbb{R}^4)$ as
$\mu\rightarrow \infty$. Thus
\begin{align}
\int_{\mathbb{R}^4}|z^3_\mu|^{2}|\omega^{12}_i|^{2}\rightarrow 0 \,\,\text{as}~ \mu\rightarrow \infty, ~\text{for}~\text{any}~ i=1,2.
\end{align} Set
$v_1:= \omega^{12}_1, v_2:= \omega^{12}_2, v_3:= z^3_\mu$. Then we have
\begin{align}
\int_{\mathbb{R}^4}\left(|\nabla v_3|^{2}-\frac{\lambda_{3}}{|x|^2} v_3^{2}\right)=\int_{\mathbb{R}^4}|v_3|^{4}=4A_3.
\end{align}Consider the following system
\begin{align}
\int_{\mathbb{R}^4}\left(|\nabla v_i|^{2}-\frac{\lambda_{i}}{|x|^2} v_i^{2}\right)=\sum_{j=1}^3t_j\int_{\mathbb{R}^4}\beta_{ij}|v_i|^{2}|v_j|^{2}, \quad i=1,2,3.
\end{align} Note that
$0 \lt \beta_{12} \lt \widetilde{\beta}\leq 1$, then the matrix
$M_B[v_1, v_2]$ is positively definite, and so
$det(M_B[v_1, v_2]) \gt 0$, which is independent on µ. Define
\begin{align*}
M_B^\infty:=
\left(
\begin{matrix}
M_B[v_1, v_2] & \mathbf{0} \\
\mathbf{0} & \int_{\mathbb{R}^4}|v_3|^{4}
\end{matrix}
\right).
\end{align*} Then
$det(M_B^\infty) \gt 0$, which is independent on µ. Note that
$det(M_B[v_1, v_2, v_3])$ is a continuous function about the variables
$\int_{\mathbb{R}^4}\beta_{ij}|v_i|^{2}|v_j|^{2}$ with
$1\leq i,j\leq 3$. It follows form (4.4) that
\begin{align}
det(M_B[v_1, v_2, v_3]) \gt \frac{1}{2}det(M_B^\infty) \gt 0 \,\,\text{for}~\text{large}~\text{enough}~ \mu.
\end{align} Therefore, the system (4.6) has a unique solution
$(t_1^\mu, t_2^\mu, t_3^\mu)$ for large enough µ. Note that every element of
$M_B[v_1, v_2, v_3]$ is uniformly bounded, combining this with (4.7) and Cramer’s rule we know that
$t_i^\mu$ is uniformly bounded,
$i=1,2,3$. Then we can assume that
\begin{align*}
\widetilde{t}_i:=\lim_{\mu\rightarrow \infty}t_i^\mu.
\end{align*} Letting
$\mu\rightarrow \infty$ in (4.6) we have
\begin{align}
\begin{cases}
\int_{\mathbb{R}^4}\left(|\nabla v_1|^{2}-\frac{\lambda_{1}}{|x|^2} v_1^{2}\right)=\sum_{j=1}^2\widetilde{t}_1\int_{\mathbb{R}^4}\beta_{12}|v_1|^{2}|v_2|^{2}\\
\int_{\mathbb{R}^4}\left(|\nabla v_2|^{2}-\frac{\lambda_{2}}{|x|^2} v_2^{2}\right)=\sum_{j=1}^2\widetilde{t}_2\int_{\mathbb{R}^4}\beta_{12}|v_1|^{2}|v_2|^{2}\\
\int_{\mathbb{R}^4}\left(|\nabla v_3|^{2}-\frac{\lambda_{3}}{|x|^2} v_3^{2}\right)=\widetilde{t}_3\int_{\mathbb{R}^4}|v_3|^{4}.
\end{cases}
\end{align} Note that
$det(M_B^\infty) \gt 0$, then the system (4.8) has a unique solution. It is easy to see that
$(1,1,1)$ is a solution of system (4.8). Thus,
$\widetilde{t}_i=1, i=1,2,3$, that is,
\begin{align}
\lim_{\mu\rightarrow \infty}t_i^\mu=1, ~\text{for}~\text{any}~ i=1,2,3.
\end{align} We deduce from (4.6) that
$(\sqrt{t_1^\mu}v_1, \sqrt{t_2^\mu}v_2, \sqrt{t_3^\mu}v_3)\in \mathcal{M}$. Therefore,
\begin{align}
A & \leq I\left(\sqrt{t_1^\mu}v_1, \sqrt{t_2^\mu}v_2, \sqrt{t_3^\mu}v_3\right) \nonumber\\
& =I_{12}(t_1^\mu v_1, t_2^\mu v_2)+\frac{1}{2}t_3^\mu\int_{\mathbb{R}^4}\left(|\nabla v_3|^{2}-\frac{\lambda_{3}}{|x|^2} v_3^{2}\right)-\frac{1}{4}(t_3^\mu)^2\int_{\mathbb{R}^4}|v_3|^{4} \nonumber\\
& \quad -\frac{1}{2}\sum_{i=1}^2t_i^\mu t_3^\mu\int_{\mathbb{R}^4}\beta_{i3}|v_i|^{2}|v_3|^{2}.
\end{align} Letting
$\mu\rightarrow \infty$ in (4.10), then by (4.9), (4.4), and (4.5) we have
The proof of theorem 1.7-(1)
By contradiction, we assume that A is achieved by a
$(\overline{u}_1, \overline{u}_2, \overline{u}_3) \in \mathcal{M}$ with
$\overline{u}_i\neq0, i=1,2,3$. It follows from lemma 4.3 and lemma 4.4 that
$A=A_3+A_{12}$. Note that
$\beta_{13} \lt 0, \beta_{23} \lt 0$, then similarly to the proof of lemma 4.3 we have
\begin{align*}
A & =I(\overline{u}_1, \overline{u}_2, \overline{u}_3) =I_{12}(\overline{u}_1, \overline{u}_2)+I_3(\overline{u}_3)-\frac{1}{2}\sum_{i=1}^2
\int_{\mathbb{R}^4}\beta_{i3}|\overline{u}_i|^{3}|\overline{u}_3|^{3} \\
& \gt I_{12}(\overline{u}_1, \overline{u}_2)+I_3(\overline{u}_3)\geq A_{12}+A_3,
\end{align*}which is a contradiction.
4.2. The case for N = 3
In this subsection, we assume that
$0 \lt \beta_{12} \lt \widehat{\beta}, \beta_{13} \lt 0, \beta_{23} \lt 0$ and give the proof of theorem 1.7-(2). The idea of the following proof is similar to that of theorem 1.7-(1). But there are technical differences between N = 3 and N = 4. For example, when projecting on the Nehari manifold, the equations the coefficients comply to are a linear system for N = 4 (see (4.8)) and the Cramer’s rule can be used. While the corresponding equations are a nonlinear system for N = 3 and the Cramer’s rule is not available. Thus, we introduced a different technique to deal with the case N = 3.
Define
\begin{align}
\widehat{\mathcal{M}}_{12}:=\left\{u_1\neq 0, u_2\neq 0, \int_{\mathbb{R}^4}\left(|\nabla u_i|^{2}-\frac{\lambda_{i}}{|x|^2} u_i^{2}\right)\leq\sum_{j=1}^2\int_{\mathbb{R}^4}\beta_{ij}|u_i|^{3}|u_j|^{3}, i=1,2\right\}
\end{align}and
\begin{align}
\widehat{A}_{12}:=\inf_{(u_1,u_2)\in \widehat{\mathcal{M}}_{12}}I_{12}(u_1,u_2).
\end{align}Lemma 4.5. Assume that
$\lambda_1=\lambda_2$ and
$0 \lt \beta_{12} \lt \widehat{\beta}$. Then we have
\begin{align*}
\widehat{A}_{12}=A_{12},
\end{align*}where A 12 is defined in (1.26).
Proof. Based on inequality (4.1), employing the proof of [Reference Guo and Zou11, Theorme 1.1] we know that there exist
$k_0,l_0 \gt 0$ such that
$A_{12}=(\sqrt{k_0}\omega_1, \sqrt{l_0}\omega_1)$, where ω 1 is a least energy positive solution of (1.4). Similarly, we also can get that
$\widehat{A}_{12}=(\sqrt{k_0}\omega_1, \sqrt{l_0}\omega_1)$. Thus,
$\widehat{A}_{12}=A_{12}$.
Lemma 4.6. Assume that
$0 \lt \beta_{12} \lt \widehat{\beta}, \beta_{13} \lt 0, \beta_{23} \lt 0$, then
$A\geq A_3+A_{12}$.
Proof. The proof is similar to that of lemma 4.3, and so we omit it.
Proposition 4.7. Assume that
$0 \lt \beta_{12} \lt \widehat{\beta}, \beta_{13} \lt 0, \beta_{23} \lt 0$, then
$A\leq A_3+A_{12}$.
Next, we show the proof of proposition 4.7. Note that
$0 \lt \beta_{12} \lt \widehat{\beta}$, then by proposition 3.7 we know that A 12 is attained and we assume that
$A_{12}=I_{12}(\omega^{12}_1,\omega^{12}_2)$. Recall that
$z^3_\mu$ is defined in (1.5). Note that
$z^3_\mu\rightharpoonup 0$ in
$D^{12}(\mathbb{R}^3)$ as
$\mu\rightarrow \infty$, then
$(z^3_\mu)^3\rightharpoonup 0$ in
$L^{2}(\mathbb{R}^3)$ as
$\mu\rightarrow \infty$. Set
$e_1:= \omega^{12}_1, e_2:= \omega^{12}_2, e_3:= z^3_\mu$. Then we have
\begin{align}
\int_{\mathbb{R}^3}\left(|\nabla e_3|^{2}-\frac{\lambda_{3}}{|x|^2} e_3^{2}\right)=\int_{\mathbb{R}^3}|e_3|^{6}=3A_3,
\end{align}and
\begin{equation}\int_{\mathbb{R}^3}\vert e_i\vert^3\vert e_3\vert^3\rightarrow0\;\text{as}~\mu\rightarrow\infty,\;\text{for}~\text{any}~i=1,2.\end{equation}Similarly to lemma 3.9 we have
Lemma 4.8. Assume that
$0 \lt \beta_{12} \lt \widehat{\beta}$. Then there exists
$C_3,C_4 \gt 0$ such that
\begin{align*}
C_3 \leq \int_{\mathbb{R}^3}|e_1|^{6} \leq C_4, \quad C_3 \leq \int_{\mathbb{R}^3}|e_2|^{6}\leq C_4,
\end{align*}C 3 is independent on β 12.
Denote
\begin{align}
\gamma:=\min\left\{\frac{1}{4}C_3, \frac{3}{2}A_3\right\}
\end{align}Lemma 4.9. Assume that
$0 \lt \beta_{12} \lt \widehat{\beta}, \beta_{13} \lt 0, \beta_{23} \lt 0$. Then the matrix
$M_B[e_1, e_2, e_3]$ is strictly diagonally dominant and the matrix
$M_B[e_1, e_2, e_3]$ is positively definite for µ large enough. Moreover,
where θmin is the minimum eigenvalues of
$M_B[e_1, e_2, e_3]$ and γ is defined in (4.15), independently on µ.
Proof. By (4.14), (4.13), and lemma 4.8, we have
\begin{align}
&\int_{\mathbb{R}^3}|e_1|^{6} - \sum_{j=2}^3\left|\int_{\mathbb{R}^3}\beta_{1j}|e_1|^{3}|e_j|^{3}\right| \nonumber \\
& \geq C_3-\beta_{12}\|e_1\|^3_6 \|e_3\|^3_6+|\beta_{13}|\int_{\mathbb{R}^3}|e_1|^{3}|e_3|^{3}\nonumber\\
& \geq C_3-\beta_{12}C_4^{\frac{1}{2}}(3A_3)^{\frac{1}{2}}+|\beta_{13}|\int_{\mathbb{R}^3}|e_1|^{3}|e_3|^{3}\nonumber\\
& \geq \frac{1}{4}C_3, \text{for}~ \mu \,\,\text{large}~\text{enough.}
\end{align}Similarly, we see that
\begin{align}
\int_{\mathbb{R}^3}|e_2|^{6} - \sum_{j\neq 2}\left|\int_{\mathbb{R}^3}\beta_{2j}|e_2|^{3}|e_j|^{3}\right|\geq \frac{1}{4}C_3, \text{for}~ \mu \,\,\text{large}~\text{enough.}
\end{align}It follows from (4.14) and (4.13) that
\begin{align}
\int_{\mathbb{R}^3}|e_3|^{6} &- \sum_{j=1}^2\left|\int_{\mathbb{R}^3}\beta_{j3}|e_3|^{3}|e_j|^{3}\right| \nonumber \\
& = \int_{\mathbb{R}^3}|e_3|^{6}+|\beta_{13}|\int_{\mathbb{R}^3}|e_1|^{3}|e_3|^{3}+
|\beta_{23}|\int_{\mathbb{R}^3}|e_2|^{3}|e_3|^{3}\\
& \geq \frac{3}{2}A_3, \text{for}~ \mu \,\,\text{large}~\text{enough.}\nonumber
\end{align} It follows from (4.17), (4.18), and (4.19) that the matrix
$M_B[e_1, e_2, e_3]$ is strictly diagonally dominant, and so the matrix
$M_B[e_1, e_2, e_3]$ is positively definite. For any eigenvalue θ of
$M_B[e_1, e_2, e_3]$, by (4.17), (4.18), and (4.19), and the Gershgorin circle theorem, we know that
$\theta\geq \gamma$ and so
$\theta_{min}\geq \gamma$.
Denote
\begin{align}
\widehat{t}^4=\frac{\max\left\{\sum_{i=1}^2\|e_i\|_i^2, 3A_3\right\}}{\gamma}.
\end{align}Then we have the following lemma
Lemma 4.10. Suppose that the assumptions in lemma 4.9 are satisfied, then there exists
$(t_{1,\mu},t_{2,\mu},t_{3,\mu})\in (\mathbb{R}^+)^3$ such that
$(t_{1,\mu}e_1,t_{2,\mu}e_2,t_{3,\mu}e_3)\in \mathcal{M}$. Moreover,
\begin{align}
0 \lt t_{i,\mu} \lt \widehat{t}, \quad i=1,2,3,
\end{align} where
$\widehat{t}$ is defined in (4.20) and independent on µ.
Proof. For any
$t_1,t_2,t_3 \gt 0$, consider
\begin{align*}
f(t_1,t_2,t_3)&:= I(t_1e_1,t_2e_2,t_3e_3) \\
& =\frac{1}{2}\sum_{i=1}^3t_i^2\|e_i\|_i^2-\frac{1}{6}\sum_{i=1,j}^3t_i^3t_j^3
\int_{\mathbb{R}^3}\beta_{ij}|e_i|^{3}|e_j|^{3}.
\end{align*}By lemma 4.9 we get that
\begin{align*}
f(t_1,t_2,t_3)& \leq \frac{1}{2}\sum_{i=1}^2t_i^2\|e_i\|_i^2+\frac{3}{2}A_3t_3^2-\frac{1}{6}\gamma\sum_{i=1}^3t_i^6\\
& \leq \sum_{i=1}^3\left(\frac{1}{2}\max\left\{\sum_{i=1}^2\|e_i\|_i^2, 3A_3\right\}t_i^2-\frac{1}{6}\gamma t_i^6\right),
\end{align*} which implies that
$f(t_1,t_2,t_3) \lt 0$ when
$t_1,t_2,t_3 \gt \widehat{t}$, where
$\widehat{t}$ is defined in (4.20). Thus,
\begin{align*}
\max_{t_1,t_2,t_3 \gt 0}f(t_1,t_2,t_3)=\max_{0 \lt t_1,t_2,t_3\leq\widehat{t}}f(t_1,t_2,t_3).
\end{align*} Based on the above arguments, we know that the function
$f(t_1,t_2,t_3)$ has a global maximum point in
$\overline{(\mathbb{R}^+)^3}$, and the global maximum point
$t_{i,\mu}\leq \widehat{t}, i=1,2,3$. Then following the proof of lemma 3.13, we get that the global maximum point
$t_{i,\mu} \gt 0, i=1,2,3$. Thus, the global maximum point
$(t_{1,\mu}, t_{2,\mu}, t_{3,\mu})$ is an interior point in
$ (\mathbb{R}^+)^3$, and so
$(t_{1,\mu}, t_{2,\mu}, t_{3,\mu})$ is a critical point of
$f(t_1,t_2,t_3)$. Therefore, we get that
$(t_{1,\mu}e_1,t_{2,\mu}e_2,t_{3,\mu}e_3)\in \mathcal{M}$.
Proof of proposition 4.7
By lemma 4.10, we get that
$(t_{1,\mu}e_1,t_{2,\mu}e_2,t_{3,\mu}e_3)\in \mathcal{M}$. Note that
$\beta_{13},\beta_{23} \lt 0$. Then we see that
\begin{align}
A & \leq I(t_{1,\mu}e_1,t_{2,\mu}e_2,t_{3,\mu}e_3)=I_{12}(t_{1,\mu}e_1,t_{2,\mu}e_2)+I_3(t_{3,\mu}e_3) \nonumber\\
& \quad -\frac{1}{3}\sum_{i=1}^2t_{i,\mu}^3t_{3,\mu}^3\int_{\mathbb{R}^3}\beta_{i3}|e_i|^{3}|e_3|^{3}\nonumber\\
& \leq \max_{t_1,t_2 \gt 0}I_{12}(t_{1}e_1,t_{2}e_2)+\max_{t_3 \gt 0}I_3(t_{3}e_3)
+\frac{1}{3}\sum_{i=1}^2\widehat{t}^6|\beta_{i3}|\int_{\mathbb{R}^3}|e_i|^{3}|e_3|^{3},
\end{align} where
$\widehat{t}$ is defined in (4.20). Letting
$\mu\rightarrow \infty$ in (4.22), by (4.14) we have
\begin{align}
A \leq \max_{t_1,t_2 \gt 0}I_{12}(t_{1}e_1,t_{2}e_2)+\max_{t_3 \gt 0}I_3(t_{3}e_3).
\end{align} It is easy to see that
$\max_{t_3 \gt 0}I_3(t_{3}e_3)=A_3$. By (3.105) we have
\begin{align*}
\max_{t_1,t_2 \gt 0}I_{12}(t_{1}e_1,t_{2}e_2)=A_{12}.
\end{align*}Combining this with (4.23) that we know that
The proof of theorem 1.7-(2)
Based on lemma 4.6 and proposition 4.7, the proof is same as proof of theorem 1.7-(1), and so we omit it.
4.3. The case for
$N\geq 5$
In this subsection, we assume that
$\beta_{12} \gt 0, \beta_{13} \lt 0, \beta_{23} \lt 0$ and give the proof of theorem 1.8. Set
$p=\frac{N}{N-2}$. Note that
$N\geq 5$,
$1 \lt p \lt 2$. Define
\begin{align*}
\mathcal{N}_{12}:=\Bigg\{(u_1,u_2)\neq \mathbf{0}:
\sum_{i=1}^2\|u_i\|_i^2=\sum_{i,j=1}^2\int_{\mathbb{R}^N}\beta_{ij}|u_i|^{p}|u_j|^{p} \Bigg \},
\end{align*}and
\begin{align*}
\widehat{A}_{12}:=\inf_{\mathbf{u}\in \mathcal{N}_{12}}I_{12}(u_1,u_2),
\end{align*} where
$\beta_{ii}=1$ and I 12 is defined in (1.24). By [Reference Chen and Zou8, Theorem 1.1] we get that
$\widehat{A}_{12}$ is attained and
$\widehat{A}_{12}=I_{12}(\omega^{12}_1,\omega^{12}_2)$. Consider
\begin{align*}
\widetilde{\mathcal{N}}_{12}:=\Bigg\{(u_1,u_2)\neq \mathbf{0}:
\sum_{i=1}^2\|u_i\|_i^2\leq\sum_{i,j=1}^2\int_{\mathbb{R}^N}\beta_{ij}|u_i|^{p}|u_j|^{p} \Bigg \},
\end{align*}and
\begin{align*}
\widetilde{A}_{12}:=\inf_{(u_1,u_2)\in \widetilde{\mathcal{N}}_{12}}\frac{1}{N}\sum_{i=1}^2\|u_i\|_i^2.
\end{align*}Then we have the following lemma
Lemma 4.11. Assume that
$N\geq 5$, then
$\widetilde{A}_{12}=\widehat{A}_{12}$.
Proof. The proof is similar to that of lemma 4.2, and so we omit it.
Lemma 4.12. Assume that
$\beta_{12} \gt 0, \beta_{13} \lt 0, \beta_{23} \lt 0$, then
$\widetilde{A}\geq A_3+\widehat{A}_{12}$, where
$\widetilde{A}$ is defined in (1.18).
Proof. The proof is the same as lemma 4.3, and so we omit it.
Proposition 4.13. Assume that
$\beta_{12} \gt 0, \beta_{13} \lt 0, \beta_{23} \lt 0$, then
$ \widetilde{A}\leq A_3+\widehat{A}_{12}$, where
$\widetilde{A}$ is defined in (1.18).
Next, we present the proof of proposition 4.13. Note that
$\widehat{A}_{12}=I_{12}(\omega^{12}_1,\omega^{12}_2)$. Since
$z^3_\mu\rightharpoonup 0$ in
$D^{12}(\mathbb{R}^N)$ as
$\mu\rightarrow \infty$, then
$(z^3_\mu)^p\rightharpoonup 0$ in
$L^{2}(\mathbb{R}^N)$ as
$\mu\rightarrow \infty$. Thus
\begin{align}
\int_{\mathbb{R}^N}|z^3_\mu|^{p}|\omega^{12}_i|^{p}\rightarrow 0 \,\,\text{as}~ \mu\rightarrow \infty, ~\text{for}~\text{any}~ i=1,2.
\end{align} Set
$z_1:= \omega^{12}_1, z_2:= \omega^{12}_2, z_3:= z^3_\mu$. Then we have
\begin{align}
\int_{\mathbb{R}^N}\left(|\nabla z_3|^{2}-\frac{\lambda_{3}}{|x|^2} z_3^{2}\right)=\int_{\mathbb{R}^N}|z_3|^{2p}=NA_3,
\end{align}and
\begin{equation}\int_{\mathbb{R}^N}\vert z_i\vert^p\vert z_3\vert^p\rightarrow0\;\text{as}~\mu\rightarrow\infty,\;\text{for}~\text{any}~i=1,2.\end{equation}Lemma 4.14. Suppose that
$\beta_{12} \gt 0, \beta_{13} \lt 0, \beta_{23} \lt 0$, then there exists
$(t_{\mu},s_{\mu})\in (\mathbb{R}^+)^2$ such that
$(t_{\mu}z_1,t_{\mu}z_2,s_{\mu}z_3)\in \mathcal{N}_{12,3}$ when µ is large enough. Moreover,
\begin{align}
\lim_{\mu\rightarrow\infty}t_{\mu}=\lim_{\mu\rightarrow\infty}s_{\mu}=1.
\end{align}Proof. For simplicity, we denote
\begin{align*}
B_{\mu}:=|\beta_{13}|\int_{\mathbb{R}^N}|z_1|^{p}|z_3|^{p}+|\beta_{23}|\int_{\mathbb{R}^N}|z_2|^{p}|z_3|^{p}.
\end{align*} It is easy to see that
$B_{\mu} \gt 0$. By (4.26) we have
\begin{align}
\lim_{\mu\rightarrow\infty}B_{\mu}=0.
\end{align} It is standard to see that
$(tz_1, tz_2, sz_3) \in \mathcal{N}_{12,3}$ for
$t,s \gt 0$ is equivalent to
$t,s \gt 0$ satisfying
\begin{align}
NA_{12}t^2=NA_{12}t^{2p}-B_{\mu}t^ps^p, \quad NA_{3}s^2=NA_{3}s^{2p}-B_{\mu}t^ps^p,
\end{align}that is,
\begin{align}
NA_{12}t^{2-p}=NA_{12}t^{p}-B_{\mu}s^p, \quad NA_{3}s^{2-p}=NA_{3}s^{p}-B_{\mu}t^p.
\end{align}Note that the first equation in (4.30) is equivalent to
\begin{align}
s=h(t):=\left(\frac{NA_{12}t^{p}-NA_{12}t^{2-p}}{B_{\mu}}\right)^{\frac{1}{p}} \gt 0, \quad
t \gt 1.
\end{align}Thus, it just needs to prove that
\begin{align*}
g(t):= \left(\frac{NA_{12}-NA_{12}t^{2-2^*}}{B_{\mu}}\right)^{\frac{N-4}{N}}+
t^{2^*-2}\left(\frac{B_{\mu}^2-N^2A_3A_{12}+N^2A_3A_{12}t^{2-2^*}}{B_{\mu}}\right),
\end{align*} has a solution t > 1. It is easy to see that
$\lim_{t\searrow 1} g(t)=B_{\mu} \gt 0$. We deduce from (4.28) that
$B_{\mu}^2-N^2A_3A_{12} \lt 0$ for µ large enough, then we get that
$\lim_{t\rightarrow+\infty} g(t)=-\infty$. Hence, there exists
$t_\mu \gt 1$ such that
$g(t_\mu)=0$. Set
$s_\mu=h(t_\mu)$, then we have
$s_\mu \gt 0$. Thus,
$(t_{\mu}z_1,t_{\mu}z_2,s_{\mu}z_3)\in \mathcal{N}_{12,3}$ for µ large enough.
Next, we claim that
$t_{\mu}, s_{\mu}$ are uniformly bounded. By contradiction, up to a subsequence, we assume that
$t_{\mu}\rightarrow +\infty$ as
$\mu\rightarrow +\infty$. By (4.29), we have
\begin{align*}
NA_{12}t_\mu^{2p}-NA_{12}t_\mu^2=NA_{3}s_\mu^{2p}-NA_{3}s_\mu^2,
\end{align*} which yields that
$s_{\mu}\rightarrow +\infty$ as
$\mu\rightarrow +\infty$. It follows from (4.29) that
\begin{align*}
\lim_{\mu\rightarrow\infty}B_{\mu}^2=\lim_{\mu\rightarrow\infty}\frac{N^2A_{12}A_{3}(t_\mu^{2p}-t_\mu^2)(s_\mu^{2p}-s_\mu^2)}{t_\mu^{2p}s_\mu^{2p}}
=N^2A_{12}A_{3} \gt 0,
\end{align*} which contradicts with (4.28). Thus,
$t_{\mu}, s_{\mu}$ are uniformly bounded. Passing to a subsequence, by (4.28) and (4.29) we have
$\lim_{\mu\rightarrow\infty}t_{\mu}=\lim_{\mu\rightarrow\infty}s_{\mu}=1$.
Proof of proposition 4.13
By lemma 4.14 we know that
$(t_{\mu}z_1,t_{\mu}z_2,s_{\mu}z_3)\in \mathcal{N}_{12,3}$, thus
\begin{align}
\widetilde{A} & \leq I(t_{\mu}z_1,t_{\mu}z_2,s_{\mu}z_3)=I_{12}(t_{\mu}z_1,t_{\mu}z_2)+I_3(s_{\mu}z_3)\nonumber\\
& \quad -\frac{1}{p}\sum_{i=1}^2t_{\mu}^ps_{\mu}^p\int_{\mathbb{R}^N}\beta_{i3}|z_i|^{p}|z_3|^{p}.
\end{align} Letting
$\mu\rightarrow\infty$ in (4.32), by (4.26) and lemma 4.14 we have
$\widetilde{A} \leq A_3+\widehat{A}_{12}$.
The proof of theorem 1.8
Based on lemma 4.12 and proposition 4.13, the proof is the same as proof of theorem 1.7-(1), and so we omit it.
Acknowledgements
The authors would like to express their sincere gratitude to the anonymous referee for his/her valuable suggestions and comments. Also the authors join to thank Hugo Tavares for several fruitful discussions and valuable comments. Song You was supported by the Scientific Research Foundation of Chongqing Normal University (Grant No. 22XLB031), the Natural Science Foundation of Chongqing (Grant No. CSTB2023NSCQ-MSX0467), and the Science and Technology Research Program of Chongqing Municipal Education Commission (Grant No. KJQN202300520) and NSFC (No.12301130). Jianjun Zhang was supported by NSFC (Nos. 12371109, 11871123).
Data availability statement
Data are contained within the article.
Conflicts of interest
On behalf of all authors, the corresponding author states that there is no conflict of interest.























