1. Introduction

The classical Hardy–Littlewood–Sobolev inequality that was obtained by Hardy and Littlewood [Reference Hardy and Littlewood36] for $n = 1$ and by Sobolev [Reference Sobolev50] for general $n$ states that

(1.1)\begin{equation} \int_{\mathbb{R}^{n}}\int_{\mathbb{R}^{n}}|x-y|^{-(n-\alpha)} f(x)g(y){\rm d}x{\rm d}y\leq C_{\alpha,n,p}\|f\|_{L^{p}(\mathbb{R}^{n})}\|g\|_{L^{q'}(\mathbb{R}^{n})} \end{equation}

with $1< p,\,q'<\infty,\, 0<\alpha < n$ and $\frac {1}{p}+\frac {1}{q'}+\frac {n-\alpha }{n}=2$.

Lieb [Reference Lieb39] employed the rearrangement inequalities to obtain the existence of the extremal functions of inequality (1.1). Furthermore, they also classified extremals of the inequality (1.1) and computed the sharp constant $C_{\alpha,n,p}$ only when one of $p$ and $q'$ is equal to $2$ or $p=q'$.

Through the inequality (1.1), we can deduce many important geometrical inequalities such as the Gross logarithmic Sobolev inequality [Reference Gross31] and the Moser–Onofri–Beckner inequality [Reference Beckner1]. It is also well-known that if we pick $\alpha =2,\, p = q' = {2n}/{(n+2)}$, then the Hardy–Littlewood– Sobolev inequality is in fact equivalent to the Sobolev inequality by Green's representation formula. By using the competing symmetry method, Carlen and Loss [Reference Carleman8] provided a different proof from Lieb's of the sharp constants and extremal functions in the diagonal case $p = q' = {2n}/{(n+\alpha )}$ and Frank and Lieb [Reference Frank and Lieb25] offered a new proof using the reflection positivity of inversions in spheres in the special diagonal case. Frank and Lieb [Reference Frank and Lieb26] further employed a rearrangement-free technique developed in [Reference Frank and Lieb27] to recapture the best constant of inequality (1.1). Folland and Stein [Reference Folland and Stein24] extended the inequality (1.1) to the Heisenberg group and established the Hardy–Littlewood–Sobolev inequality on Heisenberg group. Frank and Lieb [Reference Frank and Lieb27] classify the extremals of this inequality in the diagonal case. This extends the earlier work of Jerison and Lee [Reference Jerison and Lee38] for sharp constants and extremals for the Sobolev inequality on the Heisenberg group in the conformal case in their study of CR Yamabe problem. Furthermore, Han et al. [Reference Han, Lu and Zhu34] established the double-weighted Hardy–Littlewood–Sobolev inequality (namely, Stein–Weiss inequality) on the Heisenberg group and discussed the regularity and asymptotic behaviour of the extremal functions. Recently, Chen et al. [Reference Chen, Lu and Tao13] used the concentration-compactness principle to obtain existence of extremals of the Stein–Weiss inequality on the Heisenberg group for all indices. We also mention that when $p = q' = {2n}/{(n+\alpha )}$, Euler–Lagrange equation of the extremals to the Hardy–Littlewood–Sobolev inequality in the Euclidean space is a conformal invariant integral equation. The inequality (1.1) and its extensions have many applications in partial differential equations. Some remarkable extensions have already been obtained on the upper half space by Dou and Zhu [Reference Dou and Zhu22], on compact Riemannian manifolds by Han and Zhu [Reference Han and Zhu35] and the reversed (weighted) Hardy–Littlewood–Sobolev inequality in [Reference Chen, Liu, Lu and Tao10, Reference Dou and Zhu23, Reference Ngô and Nguyen48, Reference Ngô and Nguyen49]. For more results about the (weighted) Hardy–Littlewood–Sobolev inequality, the general weighted inequalities and their corresponding Euler–Lagrange equations, refer to e.g. [Reference Beckner2, Reference Brascamp and Lieb3, Reference Chen, Liu and Lu9, Reference Chen and Li15–Reference Dai and Liu20, Reference Gao, Liu, Moroz and Yang28, Reference Han and Lin32, Reference Hu and Liu37, Reference Lieb and Loss42–Reference Lu and Zhu45, Reference Moroz and Van Schaftingen47, Reference Stein and Weiss51] and the references therein.

Recently, Gluck [Reference Gluck30] proved the following sharp Hardy–Littlewood–Sobolev inequality with extended kernel in the conformal invariant case ($p=\tfrac {2(n-1)}{n+\alpha -2}$, $q'=\tfrac {2n}{n+\alpha +2\beta }$)

(1.2)\begin{equation} \Big|\int_{\mathbb{R}_+^{n}}\int_{\partial\mathbb{R}^{n}_+} K(x'-y,x_n) f(y)g(x) {\rm d}y{\rm d}x\Big|\leq C_{n,\alpha,\beta,p} \|f\|_{L^{p}(\partial\mathbb{R}_+^{n})} \|g\|_{L^{q'}(\mathbb{R}_+^{n})}. \end{equation}

where $K$ is a kernel of the form

\[ K(x',x_n)=K_{\alpha,\beta}(x',x_n)=\frac{x_n^{\beta}}{(|x'|^{2}+x_n^{2})^{{(n-\alpha)}/{2}}}, \quad x=(x',x_n)\in \mathbb{R}^{n-1}\times (0,\infty), \]

and $\alpha$, $\beta$ satisfy $\beta \geq 0$, $0<\alpha +\beta < n-\beta$,

(1.3)\begin{equation} \frac{n-\alpha-2\beta}{2n}+\frac{n-\alpha}{2(n-1)}<1. \end{equation}

In fact, for $\alpha =0$, $\beta =1$, the kernel $K_{\alpha,\beta }$ is the classical Poisson kernel. Hang et al. [Reference Hang, Wang and Yan33] derived the Hardy–Littlewood–Sobolev inequality with the Poisson kernel and proved the existence of extremals for this inequality by the concentration-compactness principle [Reference Lions40, Reference Lions41]. For the conformal invariant case, they classified the extremal functions of the inequality, and computed the sharp constant. Integral inequality with the Poisson kernel is highly related to Carleman's proof of isoperimetric inequality in the plane (see [Reference Carleman7]). For $\alpha \in (0,\,1)$, $\beta =1-\alpha$, the kernel $K_{\alpha,\beta }$ is related to the divergence form operator $u\mapsto {\rm div}(x_n^{\alpha } \nabla u)$ (the poly-harmonic extension operator) on the half space. Chen [Reference Chen14] established sharp Hardy–Littlewood–Sobolev inequality (1.2). He also generalized Carleman's inequality for harmonic functions in the plane to poly-harmonic functions in higher dimensions. Dou and Zhu [Reference Dou and Zhu22] studied the sharp Hardy–Littlewood–Sobolev inequality on the upper half space and the existences of extremal functions for $\beta =0$. Dou et al. [Reference Dou, Guo and Zhu21] investigated the integral inequality (1.2) in the special index through the methods based on conformal transformation for $\beta =1$. Different from Dou et al. [Reference Dou, Guo and Zhu21], Chen et al. [Reference Chen, Lu and Tao12] derived the Hardy–Littlewood–Sobolev inequality to all critical index for $\beta =1$. Furthermore, Chen et al. [Reference Chen, Liu, Lu and Tao11] extended it to the weighted Hardy–Littlewood–Sobolev inequality.

In this paper, we extended the Hardy–Littlewood–Sobolev inequality with extended kernel in the conformal invariant case to all critical index. That is,

Theorem 1.1 Let $n\geq 2,$ $1< p,\, q'<\infty,$ $\beta \geq 0,$ $\alpha +\beta >1$ and suppose that $\alpha,$ $\beta,$ $p,$ $q'$ satisfy

\[ \frac{n-1}{n}\frac{1}{p}+\frac{1}{q'}-\frac{\alpha+\beta-1}{n}=1. \]

Then there is a constant $C_{n,\alpha,\beta,p}>0$ such that for any nonnegative functions $f\in L^{p}(\partial \mathbb {R}_+^{n}),$ $g\in L^{q'}(\mathbb {R}_+^{n}),$

(1.4)\begin{equation} \int_{\mathbb{R}_+^{n}}\int_{\partial\mathbb{R}^{n}_+} \frac{x_n^{\beta} }{|x-y|^{n-\alpha}} f(y)g(x) {\rm d}y{\rm d}x\leq C_{n,\alpha,\beta,p}\|f\|_{L^{p}(\partial\mathbb{R}_+^{n})} \|g\|_{L^{q'}(\mathbb{R}_+^{n})}. \end{equation}

We remark that the constant $C_{n,\alpha,\beta,p}$ above can be considered as the least one such that the above inequality holds for all nonnegative functions $f\in L^{p}(\partial \mathbb {R}^{n}_+)$, $g\in L^{q'}(\mathbb {R}^{n}_+)$. This constant $C_{n,\alpha,\beta,p}$ is often referred as the best constant for the Hardy–Littlewood–Sobolev inequality with extended kernel.

Define

\[ Tf(x)=\int_{\mathbb{R}^{n}_+}\frac{x_n^{\beta}}{|x-y|^{n-\alpha}}f(y){\rm d}y,\quad T'g(y)=\int_{\partial\mathbb{R}^{n}_+}\frac{x_n^{\beta}}{|x-y|^{n-\alpha}}g(x){\rm d}x. \]

Throughout this paper, we always assume that $q$ and $q'$ are conjugate numbers. That is, $q$ and $q'$ satisfy $\frac {1}{q}+\frac {1}{q'}=1$. By duality, it is easy to verify that the inequality (1.4) is equivalent to the following two corollaries.

Corollary 1.2 Assume that $n\geq 2,$ $\beta \geq 0,$ $\alpha +\beta >1,$ $1< p<\frac {n-1}{\alpha +\beta -1},$ and

\[ \frac{1}{q}=\frac{n-1}{n}\left(\frac{1}{p}-\frac{\alpha+\beta-1}{n-1}\right). \]

Then there is a constant $C_{n,\alpha,\beta,p}>0$ such that

(1.5)\begin{equation} \|Tf\|_{L^{q}(\mathbb{R}^{n}_+)}\leq C_{n,\alpha,\beta,p}\|f\|_{L^{p}(\partial\mathbb{R}^{n}_+)}. \end{equation}

Corollary 1.3 Assume that $n\geq 2,$ $\beta \geq 0,$ $\alpha +\beta >1,$ $1< q'<\frac {n}{\alpha +\beta },$ and

\[ \frac{1}{p'}=\frac{n}{n-1}\left(\frac{1}{q'}-\frac{\alpha+\beta}{n}\right). \]

Then there is a constant $C_{n,\alpha,\beta,q'}>0$ such that

(1.6)\begin{equation} \|T'g\|_{L^{p'}(\partial\mathbb{R}^{n}_+)}\leq C_{n,\alpha,\beta,p}\|g\|_{L^{q'}(\mathbb{R}^{n}_+)}. \end{equation}

Once we establish the Hardy–Littlewood–Sobolev inequality with extended kernel, it is natural to ask whether the extremal functions for inequality (1.4) actually exist. To answer this question, we turn to consider the following maximizing problem

(1.7)\begin{equation} C_{n,\alpha,\beta,p}:= \sup \{\|Tf\|_{L^{q}(\mathbb{R}^{n}_+)} \mid \|f\|_{L^{p}(\partial\mathbb{R}^{n}_+)}=1, f\geq 0\}, \end{equation}

where $p$, $q$ satisfy

\[ \frac{1}{q}=\frac{n-1}{n}\left(\frac{1}{p}-\frac{\alpha+\beta-1}{n-1}\right). \]

It is not hard to verify that the extremals of inequality (1.5) are those solving the maximizing problem (1.7). We use the rearrangement inequality to prove the attainability of maximizers for the maximizing problem (1.7).

Theorem 1.4 Let $n\geq 2,$ $1< p,\, q<\infty,$ $\beta \geq 0,$ $\alpha +\beta >1,$ and suppose that $\alpha,$ $\beta,$ $p,$ $q$ satisfy

\[ \frac{1}{q}=\frac{n-1}{n}\left(\frac{1}{p}-\frac{\alpha+\beta-1}{n-1}\right). \]

Then there exists some function $f\in L^{p}(\partial \mathbb {R}^{n}_+)$ such that $f\geq 0,$ $\|f\|_{L^{p}(\partial \mathbb {R}^{n}_+)}=1,$ and $\|Tf\|_{L^{q}( \mathbb {R}^{n}_+)}=C_{n,\alpha,\beta,p}$. Moreover, all extremal functions are radially symmetric and strictly decreasing about some point $y_0\in \partial \mathbb {R}^{n}_+$.

We now turn our attention to study the regularity of the extremal functions for inequality (1.5), the Euler–Lagrange equation for extremal functions, up to a constant multiplier, is given by

(1.8)\begin{equation} f^{p-1}(y)=\int_{\mathbb{R}^{n}_+}\frac{x_n^{\beta}}{|x-y|^{n-\alpha}}(Tf(x))^{q-1}{\rm d}x. \end{equation}

We prove

Assume that

\[ u(y)=f^{p-1}(y), \quad v(x)=Tf(x). \]

Denote

\[ \theta=\frac{1}{p-1},\quad \kappa=q-1. \]

Euler–Lagrange equation (1.8) can be rewritten as the following integral system

(1.9)\begin{equation} \begin{cases} u(y)=\displaystyle\int_{\mathbb{R}^{n}_+}\frac{x_n^{\beta} }{|x-y|^{n-\alpha}} v^{\kappa}(x){\rm d}x, & y\in\partial\mathbb{R}^{n}_+,\\ v(x)=\displaystyle\int_{\partial\mathbb{R}^{n}_+}\frac{x_n^{\beta} }{|x-y|^{n-\alpha}}u^{\theta}(y) {\rm d}y, & x\in\mathbb{R}^{n}_+. \end{cases} \end{equation}

We use the Pohozaev identity to prove the following theorem.

Theorem 1.6 For $n\geq 2,$ $\beta \geq 0,$ $\alpha +\beta > 1,$ $\theta >0,$ $\kappa >0,$ assume that $(u,\,v)\in L^{\theta +1}(\partial \mathbb {R}^{n}_+)\times L^{\kappa +1}(\mathbb {R}^{n}_+)$ is a pair of nonnegative nontrivial $C^{1}$ solutions of (1.9), then a necessary condition for $\theta$ and $\kappa$ is

\[ \frac{n-1}{\theta+1}+\frac{n}{\kappa+1}=n-\alpha-\beta. \]

Obviously, extremals $(f,\,g)$ of inequality (1.4) satisfies the integral system (1.9). In light of theorems Reference Chen, Lu and Tao3.1, Reference Chen4.1 and Reference Christ, Liu and Zhang5.1, we obtain the sufficient and necessary condition for existence of positive solutions to the integral system (1.9).

Theorem 1.7 For $\theta >0,$ $\kappa >0,$ let $n,$ $\alpha,$ $\beta,$ $p,$ $q$ satisfy all the hypotheses of theorems Reference Chen, Lu and Tao3.1, Reference Chen4.1 and Reference Christ, Liu and Zhang5.1, then the sufficient and necessary condition for the existence of a pair of nonnegative nontrivial solutions $(u,\,v)\in L^{\theta +1}(\partial \mathbb {R}^{n}_+)\times L^{\kappa +1}(\mathbb {R}^{n}_+)$ to system (1.9) is

\[ \frac{n-1}{\theta+1}+\frac{n}{\kappa+1}=n-\alpha-\beta. \]

The following Liouville type theorem was proved by Gluck.

Theorem 1.8 (see [Reference Gluck30])

Let $n\geq 2$ and suppose $\alpha,$ $\beta$ satisfy $\beta \geq 0,$ $0<\alpha +\beta < n-\beta$ and (1.3). If $u\in L^{\theta +1}(\partial \mathbb {R}^{n}_+)$ and $v\in L^{\kappa +1}(\mathbb {R}^{n}_+)$ are positive solutions of (1.9) with $\theta =\frac {n+\alpha -2}{n-\alpha }$ and $\kappa =\frac {n+\alpha +2\beta }{n-\alpha -2\beta }$. Then there exists $c_1>0,$ $d>0$ and $y_0\in \partial \mathbb {R}^{n}_+$ such that

\[ u(y)=\frac{c_1}{\big(d^{2}+|y-y_0|^{2}\big)^{{(n-\alpha)}/{2}}} \ {\rm for\ all\ } \ y\in \partial\mathbb{R}^{n}_+. \]

With the help of theorem 1.7, we use weaker assumption (1.10) to obtain theorem 1.9 instead of the conformal invariant case.

Theorem 1.9 Let $n\geq 2$ and suppose $\alpha,$ $\beta$ satisfy $\beta \geq 0,$ $0<\alpha +\beta < n-\beta$. If $u\in L^{\theta +1}(\partial \mathbb {R}^{n}_+)$ and $v\in L^{\kappa +1}(\mathbb {R}^{n}_+)$ are nonnegative nontrivial solutions of (1.9) with

(1.10)\begin{equation} 0<\theta\leq\frac{n+\alpha-2}{n-\alpha},\quad 0<\kappa\leq\frac{n+\alpha+2\beta}{n-\alpha-2\beta}. \end{equation}

Then

\[ \theta=\frac{n+\alpha-2}{n-\alpha},\quad \kappa=\frac{n+\alpha+2\beta}{n-\alpha-2\beta}. \]

Moreover, there exists $c_1>0$, $d>0$ and $y_0\in \partial \mathbb {R}^{n}_+$ such that

\[ u(y)=\frac{c_1}{\big(d^{2}+|y-y_0|^{2}\big)^{{(n-\alpha)}/{2}}} \ {\rm for\ all\ } \ y\in \partial\mathbb{R}^{n}_+. \]

From theorem 1.7, we must have $\theta =\frac {n+\alpha -2}{n-\alpha }$ and $\kappa =\frac {n+\alpha +2\beta }{n-\alpha -2\beta }$. Then, the proof is completely similar to the proof by Gluck in [Reference Gluck30], so we omit the details.

This paper is organized as follows. In § 2, we prove the Hardy–Littlewood–Sobolev inequality with the extended kernel. In § 3, by the rearrangement inequality, we obtain the existence of extremals of the inequality. Section 4 is devoted to the regularity estimate of the extremal functions of the Hardy–Littlewood–Sobolev inequality with the extended kernel. In § 5, using the Pohozaev identity in integral forms, we give sufficient and necessary conditions for the existence of nonnegative nontrivial solutions.