1.1. Weighted Sobolev trace embeddings

A fundamental element of the analysis is the Sobolev trace theorem for the upper half-space

\begin{equation*} \mathbb{R}^N_+ = \{x = (x', x_N) \in \mathbb{R}^N : x' \in \mathbb{R}^{N-1},\ x_N \gt 0\} \end{equation*}

whose boundary is $\partial \mathbb{R}^N_+ = \mathbb{R}^{N-1}$. This theorem guarantees the existence of a bounded linear operator, known as the trace operator, which in the present context asserts that for any smooth function $u$ with compact support in $\mathbb{R}^{N}$, the following inequality holds:

(1)\begin{equation} K(N,p) \left( \int_{\mathbb{R}^{N-1}} |u|^{p_{\ast}}\, \mathrm{d}x' \right)^{p/p_{\ast}} \leq \int_{\mathbb{R}^N_+} |\nabla u|^p\, \mathrm{d}x, \end{equation}

where $N \geq 3$ and $1 \lt p \lt N$. Here, $p_{\ast} := \frac{(N-1)p}{N - p}$ denotes the critical exponent associated with the Sobolev trace theorem, which plays a central role in the analysis of trace embeddings and related inequalities. The best constant $K(N,p) \gt 0$ in inequality (1) is attained by the family of functions

(2)\begin{equation} u_\varepsilon(x',x_N) = \left( \frac{\varepsilon^{1/p}}{|(x', x_N + \varepsilon)|} \right)^{\frac{N - p}{p - 1}}, \end{equation}

where $\varepsilon \gt 0$; see [Reference Beckner7, Reference Escobar16, Reference Nazaret25] for details. Inequality (1) illustrates that the trace of a Sobolev function on the boundary $\mathbb{R}^{N-1}$ is effectively controlled by the $L^p$-norm of its gradient in the domain $\mathbb{R}^N_+$, providing a crucial tool for the analysis of elliptic problems with boundary conditions.

Let $C_\delta^\infty(\mathbb{R}^N_+)$ be the space of $C_0^\infty(\mathbb{R}^N)$-functions restricted to $\mathbb{R}^N_+$ and for $0 \lt p-1 \lt \gamma$ consider the weighted Sobolev space $E$ defined as the completion of $C_\delta^\infty(\mathbb{R}^N_+)$ with respect to the norm

(3)\begin{equation} \|u\|=\left(\int_{\mathbb{R}_+^N}(1+x_N)^{\gamma}|\nabla u|^p\, \mathrm{d} x+\int_{\mathbb{R}^{N-1}}|u|^p \, \mathrm{d} x'\right)^{1/p}. \end{equation}

The space $ E $ depends on $ p, N, \gamma $; however, for the sake of simplicity, this dependence is omitted in the notation.

To derive trace embeddings from $ E $ into $ L^q(\mathbb{R}^{N-1}) $ spaces, the following Hardy–Sobolev type inequality is first presented:

Theorem 1 (Hardy type)

If $0 \lt p-1 \lt \gamma$, for every $u \in C_0^{\infty}(\mathbb{R}^N)$, the following inequality holds:

(4)\begin{equation} C_{p,\gamma}^p \int_{\mathbb{R}_{+}^N} \frac{|u|^p}{(1 + x_N)^{p - \gamma}} \, \mathrm{d}x + C_{p,\gamma}^{p-1} \int_{\mathbb{R}^{N-1}} |u|^p \, \mathrm{d}x' \leq \int_{\mathbb{R}_{+}^N} (1 + x_N)^\gamma |\nabla u|^p \, \mathrm{d}x, \end{equation}

where

(5)\begin{equation} C_{p,\gamma} := \frac{\gamma - p + 1}{p}. \end{equation}

Before proceeding, the following remarks are in order.

Remark 2. Inequality (4) is fundamental in the derivation of the main results. From the definition of $ E $, combined with (4) and a density argument, it follows that

(6)\begin{equation} C_{p,\gamma}^p \int_{\mathbb{R}_{+}^N} \frac{|u|^p}{(1 + x_N)^{p - \gamma}} \, \mathrm{d}x \leq \int_{\mathbb{R}_{+}^N} (1 + x_N)^\gamma |\nabla u|^p \, \mathrm{d}x, \quad \forall u \in E \end{equation}

and

(7)\begin{equation} C_{p,\gamma}^{p-1} \int_{\mathbb{R}^{N-1}} |u|^p \, \mathrm{d}x' \leq \int_{\mathbb{R}_{+}^N} (1 + x_N)^\gamma |\nabla u|^p \, \mathrm{d}x, \quad \forall u \in E . \end{equation}

It was proved in [Reference Abreu, Furtado and Medeiros1] that if $ \gamma \gt p-1 \gt 0$ it holds

\begin{equation*} S_{\gamma,p}:= \inf_{u \in E\setminus\{0\}} \frac{\int_{\mathbb{R}_{+}^N} (1 + x_N)^\gamma |\nabla u|^p \, \mathrm{d}x}{\int_{\mathbb{R}_{+}^N} (1+x_N)^{\gamma-p}|u|^p \, \mathrm{d}x } = C_{p,\gamma}^{p} \gt 0. \end{equation*}

Thus, $C_{p,\gamma}^{p}$ is the best constant for inequality (6). It is worth noting that obtaining the best constant for inequality (7) remains an interesting open question. More precisely, define

(8)\begin{equation} T_{\gamma,p}:= \inf_{u \in E\setminus\{0\}} \frac{\int_{\mathbb{R}_{+}^N} (1 + x_N)^\gamma |\nabla u|^p \, \mathrm{d}x}{\int_{\mathbb{R}^{N-1}} |u|^p \, \mathrm{d}x^\prime }. \end{equation}

It follows from (7) that $ C_{p,\gamma}^{p-1} \leq T_{\gamma,p} $ whenever $ \gamma \gt p - 1 $. The question of whether equality holds remains open.

The following result is also a direct consequence of (6):

Corollary 1 (Equivalence of Norms)

In the space $E$, the functional

\begin{equation*} \|u\|_{\gamma} \;:=\;\biggl(\int_{\mathbb{R}^N_{+}}(1+x_{N})^{\gamma}\,\lvert\nabla u\rvert^{p}\,\mathrm{d}x\biggr)^{1/p} \end{equation*}

defines a norm equivalent to the original norm $\|\cdot\|$ given in (3).

Based on the Hardy type inequality presented in Theorem 1, we are able to establish the Sobolev trace embedding in the context of the space $E$, given by:

Theorem 2 (Sobolev Trace-type Embedding)

The Sobolev trace embedding

\begin{equation*} E \hookrightarrow L^q(\mathbb{R}^{N-1}), \quad \forall q \in [p, p_\ast] \end{equation*}

is continuous. In other words, there exists a constant $ C = C(N, p, q, \gamma) \gt 0 $ such that

(9)\begin{equation} \left(\int_{\mathbb{R}^{N-1}} |u|^q \, \mathrm{d}x'\right)^{p/q} \leq C \int_{\mathbb{R}_+^N} (1 + x_N)^\gamma |\nabla u|^p \, \mathrm{d}x, \quad \forall u \in E. \end{equation}

Moreover, it holds:

1. (1) If $q=p_{\ast}$ and $\gamma \gt p-1$ the result is

   \begin{equation*} T_{\gamma,p_\ast}= \inf_{u \in E\setminus\{0\}} \frac{\int_{\mathbb{R}_{+}^N} (1 + x_N)^\gamma |\nabla u|^p \, \mathrm{d}x}{\left(\int_{\mathbb{R}^{N-1}} |u|^{p_\ast} \, \mathrm{d}x^\prime\right)^{p/p_\ast} }=K(N,p), \end{equation*}
   where $K(N,p)$ is the best constant for the trace inequality (1).

2. (2) If $ \gamma \lt p - 1 $, then $ T_{\gamma, p} = 0 $.

1.2. Quasilinear equations involving nonlinear boundary conditions

As an application of Theorem 2, the following class of problems is considered:

\begin{equation*} \left\lbrace \begin{aligned} -\mathrm{div}(\rho(x)|\nabla u|^{p-2}\nabla u ) &=0\quad&\mbox{in }&\,\mathbb{R}^N_+, \\ \rho(x',0)|\nabla u|^{p-2}\nabla u\cdot\nu&=h(x')|u|^{q-2}u-m(x')|u|^{s-2}u,\quad &\mbox{on }&\,\mathbb{R}^{N-1}, \end{aligned} \right. \quad (\mathcal{P}) \end{equation*}

where $\nu$ is the unit outer normal to the boundary $\partial\mathbb{R}^N_+:=\mathbb{R}^{N-1}$. Assume that $ h, m \in L^1_{\mathrm{loc}}(\mathbb{R}^{N-1}) $ are positive functions and that $ \rho \in L^1_{\mathrm{loc}}(\mathbb{R}^N_+) $ satisfies

(H)\begin{equation} \exists \, \rho_0 \gt 0\ \text{and } \gamma \gt p-1 \; \text{ s.t. } \; \rho(x)\geq \rho_0(1+x_N)^\gamma \quad \text{a.e.}\quad x \in \mathbb{R}^N_+. \quad \end{equation}

To establish the variational framework, define the weighted space $ E_{\rho} $ as the closure of $ C_\delta^\infty(\mathbb{R}^N_+) $ with respect to the norm

\begin{equation*} \|u\|_{\rho}:=\left(\int_{\mathbb{R}^{N}_+}\rho(x)|\nabla u|^p\, \mathrm{d} x\right)^{1/p}. \end{equation*}

We stress that assumption (H) plays a purely technical role and is introduced to ensure the continuous embedding $ E_{\rho} \hookrightarrow E.$ Although the analysis was initially carried out for the specific weight $ \rho(x) = (1 + x_N)^{\gamma}, $ a closer examination of the proof reveals that no essential property of this particular choice is used. In fact, the argument remains entirely unchanged for any admissible weight that satisfies the assumption (H).

A weak solution of problem ( $\mathcal{P}$) is a function $ u \in E_{\rho} $ such that, for every smooth function $ \varphi $ with compact support in $ \mathbb{R}^{N} $, the following identity is satisfied:

(10)\begin{equation} \int_{\mathbb{R}^{N}_+}\rho(x)|\nabla u|^{p-2}\nabla u \cdot \nabla \varphi \, \mathrm{d} x=\int_{\mathbb{R}^{N-1}}(h(x')|u|^{q-2}u -m(x')|u|^{s-2}u) \varphi \, \mathrm{d} x'. \end{equation}

In the following sections, we establish existence and nonexistence results for problem ( $\mathcal{P}$). As observed in [Reference Alama and Tarantello5], the presence of a sign-changing nonlinearity makes the solvability of the problem highly sensitive to the interaction between the competing terms $ h\,|u|^{q-2}u \quad \text{and} \quad m\,|u|^{s-2}u. $ The balance between these terms is determined by qualitative properties of the ratio of the weight functions $h$ and $m$. In particular, this interplay is captured by conditions (12), (13), and (17) below, which play a decisive role in distinguishing regimes of existence from nonexistence.

1.3. Liouville-type results

Next, inspired by some ideas from [Reference Lyberopoulos23], we establish nonexistence results for problem ( $\mathcal{P}$). To this end, the following notation is adopted:

(11)\begin{equation} \eta(r, q, t) := \frac{(t-r)^{t-r}}{(q-r)^{q-r}(t-q)^{t-q}} \quad \text{for} \quad r \lt q \lt t. \end{equation}

First, the case $1 \lt s \lt q \lt p \lt N$ is analysed.

Theorem 3 (p-sublinear case)

Suppose that assumption (H) holds, and let $ m \in L^1(\mathbb{R}^{N-1}) \cap L^\infty(\mathbb{R}^{N-1}) $. If $ 1 \lt s \lt q \lt p \lt N $ and $ h/m \in L^\infty(\mathbb{R}^{N-1}) $, with

(12)\begin{equation} \left\|\frac{h}{m}\right\|_{\infty}^{p-s} \left(\frac{\|m\|_{\infty} C_{p,\gamma}^{1-p}}{\rho_0}\right)^{q-s} \lt \eta(s, q, p), \end{equation}

then problem ( $\mathcal{P}$) has only the trivial weak solution.

A straightforward computation shows that for sufficiently small $\lambda \gt 0$, the functions

\begin{equation*} h(x') = \frac{\lambda}{(1 + |x'|)^{\theta_1}} \quad \text{and} \quad m(x') = \frac{1}{(1 + |x'|)^{\theta_2}} \end{equation*}

satisfy the assumptions of Theorem 3 provided that $0 \lt N-1 \lt \theta_2 \leq \theta_1$.

In the second nonexistence result, the case $p \lt q \lt s$ is considered, and the implications of this ordering of exponents on the solvability of the problem are explored.

Theorem 4 (p-superlinear case)

Assume that (H) holds, and let $m \in L^\infty(\mathbb{R}^{N-1})$. If $1 \lt p \lt q \lt s \leq p_{\ast}$ and $h / m \in L^\infty(\mathbb{R}^{N-1})$, with

(13)\begin{equation} \left\|\frac{h}{m}\right\|_{\infty}^{s-p} \left(\frac{\|m\|_\infty C_{p,\gamma}^{1-p}}{\rho_0}\right)^{s-q} \lt \eta(p, q, s), \end{equation}

then ( $\mathcal{P}$) admits only the trivial weak solution.

A direct computation reveals that for sufficiently small $\lambda \gt 0$, the functions

\begin{equation*} h(x') = \frac{\lambda}{(1 + |x'|)^{\theta_1}} \quad \text{and} \quad m(x') = \frac{1}{(1 + |x'|)^{\theta_2}} \end{equation*}

satisfy the hypotheses of Theorem 4 whenever $0 \leq \theta_2 \leq \theta_1$.

1.4. Existence results

The existence results are established using the fibering method. Theorem 2, combined with the additional conditions on the potentials, more precisely, $h,m \in L^1(\mathbb{R}^{N-1}) \cap L^\infty(\mathbb{R}^{N-1})$, ensure that the embeddings

\begin{equation*} E_\rho \hookrightarrow L^q(\mathbb{R}^{N-1},h(x'))\quad\text{and}\quad E_\rho\hookrightarrow L^s(\mathbb{R}^{N-1},m(x')) \end{equation*}

are both valid and compact. Consequently, this enables the application of the fibering method, as illustrated in [Reference Kandilakis and Lyberopoulos19, Reference Lyberopoulos23].

In the first existence result, two specific combinations of the exponents are examined. More precisely, the cases $1 \lt s \lt p \lt q \lt p_{\ast}$ and $1 \lt p \lt s \lt q \lt p_{\ast}$ are analysed. Particular attention is given to ensuring that the interplay among the exponents aligns with the structural requirements of the problem.

For the second existence result, a different set of exponent combinations is considered: $1 \lt q \lt s \lt p$ and $1 \lt q \lt p \lt s \lt p_{\ast}$. These cases reflect distinct configurations of the exponents, each leading to unique mathematical challenges and results.

We note that Figures 1 and 2 illustrate that the results established in Theorems 3 and 4, together with Theorems 5 and 6, exhibit a complementary character, as they apply to mutually complementary regions.

Figure 1.

Nonexistence of solutions. The diagonal-striped region corresponds to Theorem 3, and the crosshatched region corresponds to Theorem 4.

Figure 1 Long description

A single plot with a horizontal axis labeled q and a vertical axis labeled s, each ending with an arrowhead. Dashed guide lines form a grid. On the q axis, labeled positions include 1, p and p asterisk. On the s axis, labeled positions include 1, p and p asterisk. A solid diagonal line rises from the point at q equals 1 and s equals 1 to the point at q equals p and s equals p and continues upward to the right toward the point at q equals p asterisk and s equals p asterisk. Two shaded regions are shown. One region uses diagonal stripe shading and forms a right triangle with vertices at (q equals 1, s equals 1), (q equals p, s equals 1) and (q equals p, s equals p). The second region uses crosshatch shading and forms a larger right triangle with vertices at (q equals p, s equals p), (q equals p, s equals p asterisk) and (q equals p asterisk, s equals p asterisk). Below the plot, two legend swatches appear: a diagonal stripe swatch labeled Theorem 3 and a crosshatch swatch labeled Theorem 4.

Figure 2.

Existence of nontrivial solutions. The diagonal-striped region corresponds to Theorem 6, and the dotted region corresponds to Theorem 5.

Figure 2 Long description

Existence of nontrivial solutions. A two dimensional parameter plot with horizontal axis label q and vertical axis label s. No units are shown. The q axis has labeled ticks at 1, p and p subscript asterisk. The s axis has labeled ticks at 1, p and p subscript asterisk. Dashed guide lines mark q equals 1, q equals p, q equals p subscript asterisk and s equals 1, s equals p, s equals p subscript asterisk. A diagonal dashed line runs upward from the point at q equals 1 and s equals 1, passes through the point at q equals p and s equals p and continues toward the point at q equals p subscript asterisk and s equals p subscript asterisk. Two shaded regions are shown and a legend at the bottom maps the fill patterns to theorem labels. The dotted fill corresponds to Theorem 5. The diagonal hatched fill corresponds to Theorem 6. Theorem 6 region: a diagonally hatched quadrilateral bounded by q equals 1 and q equals p and by s equals p subscript asterisk at the top and the diagonal line from (q equals 1, s equals 1) to (q equals p, s equals p) at the bottom. Theorem 5 region: a dotted triangular region bounded by q equals p and q equals p subscript asterisk, by s equals 1 at the bottom and by the diagonal line from (q equals p, s equals p) to (q equals p subscript asterisk, s equals p subscript asterisk) on the slanted side.

To establish the third existence result, consider the functionals defined on $ E_\rho $ by

(14)\begin{equation} H(u)=\int_{\mathbb{R}^{N-1}}h|u|^q \, \mathrm{d} x' \quad\text{and}\quad M(u)=\int_{\mathbb{R}^{N-1}}m|u|^s \, \mathrm{d} x'. \end{equation}

Note that under the assumptions of Theorem 4, for $u \in E_\rho$, the following key inequality holds (see Lemma 11):

(15)\begin{equation} H(u)^{s-p} \lt \eta(p,q,s) M(u)^{q-p} \|u\|_\rho^{p(s-q)} \lt \eta(p,q,s) \left(\frac{q}{p}\right)^{s-q} M(u)^{q-p} \|u\|_\rho^{p(s-q)}. \end{equation}

Therefore, the existence of solutions for ( $\mathcal{P}$) can be considered in a subset of the complementary situation to the inequality (15), precisely:

(16)\begin{equation} \mathcal{D}:= \left\{u \in E_\rho: H(u)^{s-p} \gt \left(\frac{q}{p}\right)^{s-q}\eta(p,q,s)M(u)^{q-p}\|u\|^{p(s-q)}_\rho\right\}. \end{equation}

Theorem 7. Assume (H), $h, m \in L^1(\mathbb{R}^{N-1}) \cap L^\infty(\mathbb{R}^{N-1})$, $p \lt q \lt s \lt p_\ast$, $\mathcal{D} \neq \emptyset$, and

(17)\begin{equation} \frac{h^{1/q}}{m^{1/s}} \in L^{\frac{sq}{s-q}}(\mathbb{R}^{N-1}). \end{equation}

Then, ( $\mathcal{P}$) admits a nontrivial and nonnegative weak solution. Moreover, $u$ is positive in $\mathbb{R}^N_+$ provided that $\rho \in L^\infty_{\mathrm{loc}}(\mathbb{R}^N_+)$. We note that Figure 3 illustrates the result established in Theorem 7.

Figure 3.

The diagonal dotted region corresponds to Theorem 7.

Figure 3 Long description

The vertical axis is labeled s. The horizontal axis is labeled q. Dashed guide lines form a grid. On the horizontal axis, the marked labels are 1, p and p subscript asterisk. On the vertical axis, the marked labels are 1, p and p subscript asterisk. A dashed diagonal line runs upward from the point at q equals 1 and s equals 1, passing through the point at q equals p and s equals p and reaching the point at q equals p subscript asterisk and s equals p subscript asterisk. A dotted, right-triangle region is shown with vertices at q equals p and s equals p; q equals p and s equals p subscript asterisk; and q equals p subscript asterisk and s equals p subscript asterisk. The left edge of the region is a vertical segment at q equals p from s equals p to s equals p subscript asterisk. The top edge is a horizontal segment at s equals p subscript asterisk from q equals p to q equals p subscript asterisk. The slanted edge follows the dashed diagonal line from q equals p and s equals p to q equals p subscript asterisk and s equals p subscript asterisk.

Remark 3. The functions $h=m=\lambda w$, with $w$ given by

\begin{equation*} w(x')=\frac{1}{(1+|x'|)^{\theta}}, \end{equation*}

satisfy the assumptions of Theorem 7 for $\theta \gt N-1$ and $\lambda$ sufficiently large. First, observe that

\begin{equation*} \left[\frac{h(x')^{1/q}}{m(x')^{1/s}}\right]^{\frac{sq}{s-q}}=h(x')=\lambda (1+|x'|)^{-\theta}\in L^1(\mathbb{R}^{N-1}), \end{equation*}

whenever $\theta \gt N-1$. For $u \in E_\rho\setminus\{0\}$ fixed, one has

\begin{equation*} \frac{H(u)^{s-p}}{M(u)^{q-p}}=\lambda^{s-q}\|u\|_{L^q(\mathbb{R}^{N-1},w)}^{q(s-p)}\|u\|_{L^s(\mathbb{R}^{N-1},w)}^{s(p-q)}. \end{equation*}

Since $s \gt q$, for $\lambda$ sufficiently large it holds

\begin{equation*} H(u)^{s-p} \gt \left(\frac{q}{p}\right)^{s-q}\eta(p,q,s)M(u)^{q-p}\|u\|^{p(s-q)}_\rho, \end{equation*}

hence $\mathcal{D}\neq \emptyset$. Conversely, it is easy to verify that for $\lambda$ small potentials, they satisfy the condition for nonexistence (13).

As stated in [Reference Lyberopoulos23], the condition $\mathcal{D} \neq \emptyset$ can be interpreted as an indication that $h$ is sufficiently ‘large’ with respect to $m$.

The next existence result is derived from the consideration of the set

(18)\begin{equation} \mathcal{C}:= \left\{u \in E_\rho : H(u)^{p-s} \gt \left(\frac{q}{s}\right)^{p-s} \eta(s,q,p) M(u)^{p-q} \|u\|^{p(q-s)}_\rho \right\}, \end{equation}

where, analogously to the earlier case, the hypothesis is that $ h $ is sufficiently large in comparison to $ m $, as indicated by the condition $ \mathcal{C} \neq \emptyset $. The fourth existence result for problem ( $\mathcal{P}$) is stated below:

Figure 4.

The diagonal dotted region corresponds to Theorem 8.

Figure 4 Long description

The vertical axis label is s. The horizontal axis label is q. The horizontal axis shows the labels 1, p and p subscript asterisk. The vertical axis shows the labels 1, p and p subscript asterisk. Dashed horizontal lines are drawn at s equals 1, s equals p and s equals p subscript asterisk. Dashed vertical lines are drawn at q equals 1, q equals p and q equals p subscript asterisk. A dashed diagonal line runs upward from near the point where q equals p and s equals p and continues toward the upper right, passing through the point where q equals p subscript asterisk and s equals p subscript asterisk. A shaded right triangle is drawn with vertices at (q equals 1, s equals 1), (q equals p, s equals 1) and (q equals p, s equals p). The triangle spans q from 1 to p and spans s from 1 to p at q equals p.

We observe that the result established in Theorem 7 does not contradict Theorem 4, since the hypothesis $\mathcal{D} \neq \emptyset$ is complementary to the assumption (13). Similarly, Theorem 8 does not contradict Theorem 3, since the hypothesis $\mathcal{C} \neq \emptyset$ is complementary to the assumption (12).

1.5. Background and related works

A substantial body of research underscores the importance of weighted Sobolev trace embeddings and their applications in studying quasilinear elliptic problems with nonlinear boundary conditions. These embeddings play a crucial role in examining the existence and nonexistence of solutions to these problems, serving as a fundamental tool for their analytical investigation. Without claiming exhaustiveness, the following selection highlights foundational studies closely related to the present research on this class of Sobolev embeddings.

The existence of extremals for the inequality (1) and a complete classification for the case $p=2$ were established in [Reference Beckner7, Reference Escobar16]. In [Reference Nazaret25], the author extended the analysis of (1) to the general case of $ 1 \lt p \lt N $ using the mass transportation method, thereby resolving the conjecture posed by Escobar in [Reference Escobar16] regarding the minimizers of the inequality. It is well known that the Euler–Lagrange equation associated with (1) is the following quasilinear equation with a nonlinear Neumann boundary:

\begin{equation*} \left\lbrace \begin{aligned} -\mathrm{div}(|\nabla u|^{p-2}\nabla u ) &=0\quad&\mbox{in }&\,\mathbb{R}^N_+, \\ |\nabla u|^{p-2}\nabla u\cdot\nu&= C|u|^{p_{\ast}-2}u,\quad &\mbox{on }&\,\mathbb{R}^{N-1}, \end{aligned} \right. \end{equation*}

where $\nu$ is the unit outer normal to the boundary $\partial\mathbb{R}^N_+:=\mathbb{R}^{N-1}$.

It is worth noting that the Hardy type inequality in the upper half-space, for the particular case $ p = 2 $, was addressed by Maz’ya in [Reference Maz’ya24], where the inequality was established for every smooth function $ u $ with compact support in $ \mathbb{R}^N_+ $:

\begin{equation*} \left( \frac{p-1}{p} \right)^p \int_{\mathbb{R}^N_+} \frac{|u|^p}{x_N^p} \, \mathrm{d}x + \alpha(p, \tau) \int_{\mathbb{R}^N_+} \frac{|u|^p}{x_N^{p-\tau}(x_{N-1}^2 + x_N^2)^{\tau/2}} \, \mathrm{d}x \leq \int_{\mathbb{R}^N_+} |\nabla u|^p \, \mathrm{d}x, \end{equation*}

for any smooth function $u$ with compact support on $\mathbb{R}^{N}_+$ for the specific case $p = 2$. For the general case $1 \lt p \lt N$, this inequality was later proved in [Reference Tidblom27], thereby extending Maz’ya’s result and providing an affirmative answer to a question he posed about the validity of such inequalities in the $L^p$ framework.

For $\alpha \in \mathbb{R}$ and $\rho(x):=d\left(x, \partial \mathbb{R}_{+}^N\right)$ be the distance from the boundary of $\mathbb{R}_{+}^N$ it was proved in [Reference D’Ambrosio and Dipierro14] the following inequality

\begin{equation*} \left(\frac{|p-1-\alpha|}{p}\right)^p \int_{\mathbb{R}_{+}^N} \rho^\alpha \frac{|u|^p}{\rho^p} d x \leqslant \int_{\mathbb{R}_{+}^N} \rho^\alpha|\nabla u|^p d x, \; u \in \mathrm{C}_0^{\infty}\left(\mathbb{R}_{+}^N\right). \end{equation*}

It is worth noting that the authors in [Reference D’Ambrosio and Dipierro14, Reference Maz’ya24, Reference Tidblom27] considered test functions $ u \in C_0^{\infty}(\mathbb{R}_+^N) $, thereby working under homogeneous Dirichlet boundary conditions. In contrast, the objective here is to investigate this class of inequalities in the upper half-space for arbitrary smooth functions $ u $ with compact support in $\mathbb{R}^N$, a framework that plays a fundamental role in the analysis of nonlinear elliptic problems with nonlinear boundary conditions.

Let $ \Omega \subset \mathbb{R}^N$ be a smooth domain and denote by $C_\delta^\infty(\Omega)$ the space of $C_0^\infty(\mathbb{R}^N)$-functions restricted to $\Omega$. For $ w_{\theta}(x) := (1+|x|)^\theta$ with $\theta \geq 0, $ consider the weighted Lebesgue space $L^r(\Omega,w_{r})$ and the weighted Sobolev space $ W$ is defined as the completion of $C_{\delta}^\infty(\Omega)$ under the norm $ ||u||_W := \left( \int_\Omega |\nabla u|^p \, dx + \int_\Omega w_p |u|^p \, dx \right)^{1/p}. $ In [Reference Janßen18, Reference Pflüger26], it was proved the embedding result $ W \hookrightarrow L^\sigma( \partial\Omega; w_\theta) $ is continuous if $p\leq \sigma\leq p_\ast$ and $(N-1)\left(1-\frac{\sigma}{p_\ast}\right)\leq \theta \lt N$. In particular, if the upper bound for $\sigma$ and the lower bound for $\theta$ are strict, then the embedding is compact.

The result above consolidates all the embedding properties relevant to the purposes of Lyberopoulos in [Reference Lyberopoulos23], where the author studied the existence and Liouville-type theorems for indefinite quasilinear elliptic problems with potentials vanishing at infinity of the form:

\begin{equation*} \left\lbrace \begin{aligned} -\mathrm{div}(\rho(x)|\nabla u|^{p-2}\nabla u ) &= a(x)|u|^{q-2}u - b(x)|u|^{s-2}u, & \text{in } \Omega, \\ \rho(x)|\nabla u|^{p-2} \, \frac{\partial u}{\partial \nu} &+ h(x)|u|^{p-2}u = 0, & \text{on } \partial\Omega, \end{aligned} \right. (\mathcal{PL}) \end{equation*}

where $\Omega$ is an arbitrary unbounded domain in $\mathbb{R}^N$ with a noncompact smooth boundary $\partial\Omega$, whose outward unit normal is denoted by $\nu$, which includes the upper half-space.

In [Reference Lyberopoulos23], a quasilinear elliptic problem with Robin boundary conditions was investigated under the assumption that $ h(x) \geq 1 $ almost everywhere on $ \partial \Omega $. Reference is also made to [Reference Abreu, Felix and Medeiros3], which addresses a related problem in the upper half-space and relies on a fundamental inequality established in [Reference Abreu, Felix and Medeiros2], analogous to the result derived in [Reference Pflüger26]. The present contribution extends the analysis in [Reference Lyberopoulos23] by considering a broader class of weight functions and incorporating Neumann boundary conditions – features not treated in the references listed above

For elliptic equations with Neumann boundary conditions in a half-space, see [Reference Berestycki, Caffarelli and Nirenberg8, Reference Cabré and Solà-Morales10–Reference Chipot, Shafrir and Fila12, Reference Filippucci, Pucci and Radulescu17, Reference Li and Zhang21]. These works highlight the intricate interplay between boundary conditions and solution behaviour, offering valuable insights into the qualitative properties of solutions in diverse settings. The continued investigation of these connections holds promise for a deeper understanding of the underlying mathematical structures.

This paper is organized as follows. Section 2 presents the proofs of Theorems 1 and 2, establishing fundamental inequalities for the subsequent analysis. Section 3 contains the Liouville-type nonexistence results (Theorems 3 and 4). The existence results appear in Section 4, with proofs of Theorems 5, 6, and 7. The final section, Section 5, discusses open problems and potential directions for future research.

1.6. Notations

• • $ p_\ast := \frac{(N-1)p}{N-p} $ denotes the critical exponent associated with the Sobolev trace theorem.

• • $ \mathbb{R}^N_+ := \{x = (x', x_N) \in \mathbb{R}^N : x' \in \mathbb{R}^{N-1},\, x_N \gt 0\} $ denotes the upper half-space, whose boundary is given by $ \partial \mathbb{R}^N_+ = \mathbb{R}^{N-1} $.

• • Let $ \Omega \subset \mathbb{R}^N $ be a smooth domain. The space $ C_0^\infty(\Omega) $ consists of smooth functions with compact support in $ \Omega $.

• • The space $ C_\delta^\infty(\Omega) $ consists of restrictions to $ \Omega $ of functions in $ C_0^\infty(\mathbb{R}^N) $.