1 Introduction

We consider modified quasilinear Schrödinger equations in $\mathbb {R}^N$ of the form

(1.1) $$ \begin{align} \alpha\Delta^2u-\Delta u+ V(x)u-u\Delta(u^2)=g(u), \end{align} $$

where $\alpha \ge 0$ , $V\in C(\mathbb {R}^N)$ is the potential, $g\in C(\mathbb {R}^N\times \mathbb {R})$ is the nonlinearity and $\Delta ^2=\Delta (\Delta )$ is the biharmonic operator.

Equation (1.1) with $\alpha =0$ is a quasilinear Schrödinger equation in $\mathbb {R}^N$ , that is,

(1.2) $$ \begin{align} -\Delta u+ V(x)u-u\Delta(u^2)=g(u). \end{align} $$

These equations arise from seeking standing waves $\psi (t,x)=e^{-i\omega t}u(x)$ and $\rho (s)=s$ for the time-dependent quasilinear Schrödinger equation

(1.3) $$ \begin{align} i\partial_ t\psi=\alpha\Delta^2\psi-\Delta\psi+U(x)\psi-{\rho}^{\prime}(|\psi|^2)\psi[\Delta\rho(|\psi|^2)]-\tilde{g}(|\psi|)\psi, \quad(t,x)\in\mathbb{R}^+\times\mathbb{R}^N, \end{align} $$

where $\psi :\mathbb {R}^N\times \mathbb {R}\rightarrow \mathbb {C}$ is the time-dependent wave function, $U:\mathbb {R}^N\rightarrow \mathbb {R}$ is an external potential, $\rho :\mathbb {R}^N\rightarrow \mathbb {R}^+$ is an appropriate function and $\tilde {g}(|\psi |)\psi $ is the interaction effect among particles. Equation (1.3) models self-trapped electrons in quadratic or hexagonal lattices [Reference Brizhik, Eremko, Piette and Zakrzewski2] and the superfluid film equation in plasma physics [Reference Kurihara8]. Note that in (1.2),

$$ \begin{align*} V(x)=U(x)-\omega, \quad g(t)=\tilde{g}(|t|). \end{align*} $$

To the best of our knowledge, the first studies of (1.2) involving variational methods were by Poppenberg et al. [Reference Poppenberg, Schmitt and Wang18] for $N=1$ and Liu and Wang [Reference Liu and Wang10] for higher dimensions with $V(x)$ radially symmetric. The existence results are obtained by constrained minimisation arguments. For higher dimensions and $V(x)$ nonradially symmetric, formally, the solutions of (1.2) should be critical points of the functional

$$ \begin{align*} J(u)=\frac{1}{2}\int(1+2u^2)|\nabla u|^2+\frac{1}{2}\int V(x)u^2-\int G(u) \quad\text{where}~ G(t)=\int_{0}^{t}g(\tau)\,d\tau. \end{align*} $$

However, the functional J may not be defined for every $u\in H^1(\mathbb {R}^N)$ . Liu et al. [Reference Liu, Wang and Wang11] and Colin and Jeanjean [Reference Colin and Jeanjean5] introduced a nonlinear transformation f so that if $v\in H^1(\mathbb {R}^N)$ is a critical point of $\phi : H^1(\mathbb {R}^N)\rightarrow \mathbb {R}$ , where

$$ \begin{align*} \phi(v)=\frac{1}{2}\int|\nabla v|^2+\frac{1}{2}\int V(x)f^2(v)-\int G(f(v)), \end{align*} $$

then $u=f(v)$ is a solution of (1.2). Using this transformation, many results on (1.2) have been obtained (see, for example, [Reference Fang and Szulkin6, Reference Liu and Zhou16, Reference Wu21, Reference Yang22]).

In all these papers and many others on the quasilinear Schrödinger equation (1.2), the nonlinearity g is superlinear at zero, that is,

(1.4) $$ \begin{align} \lim_{t\rightarrow0}\frac{g(t)}{t}=0, \end{align} $$

and satisfies other conditions such as the classical Ambrosetti–Rabinowitz condition. These conditions are needed to ensure the boundedness of Palais–Smale (PS) sequences. When the nonlinearity g is sublinear at zero (that is, the limit in (1.4) is $+\infty $ ), Zhang et al. [Reference Zhang, Tang and Chen24] considered the case $V(x)=0$ with the nonlinearity

$$ \begin{align*} g(x,u)=f(x,u)+\theta h(x,u), \end{align*} $$

where $f(x,u)$ is sublinear near the origin with respect to u. Bao and Cheng [Reference Bao and Cheng1] studied (1.2) when the nonlinearity g is sublinear and the potential V is allowed to be sign-changing. They obtained a sequence of solutions $\{v_k\}$ for the truncated problem with an odd and subcritical $g_h$ in place of $\tilde {g}:=g+V_0u$ , where $g_h(x,f(v))= \tilde {g}(x,f(v))h(f(v))$ for $|v|$ small and h is an even function. They then applied an $L^{\infty }$ -estimate to show that $|v_k|_{\infty }\rightarrow 0$ and concluded that the $u_k=f(v_k)$ for k large are solutions of the original problem. For other recent results where the nonlinearity is sublinear at zero, we refer to [Reference Liu and Zhao13] for semilinear Schrödinger equations and to [Reference Liu12, Reference Liu and Zhao14] for $p(x)$ -Laplacian problems.

In this paper, we assume the nonlinearity g is sublinear at zero (assumption $(g_2)$ below) and do not require any assumption at infinity except for subcritical growth. The potential V is allowed to be sign-changing, which makes it difficult to prove the boundedness of (PS) sequences. Motivated by these observations and a recent result of He and Wu [Reference He and Wu7] on the semilinear elliptic boundary value problem

$$ \begin{align*} \begin{cases} -\Delta u+ V(x)u=f(x,u) &\text{in } \Omega, \\ u=0 &\text{on }\partial\Omega \end{cases} \end{align*} $$

for a bounded domain $\Omega \subset \mathbb {R}^N$ , we study (1.2) by using the truncation method and Liu and Wang’s variant of Clark’s theorem [Reference Liu and Wang17, Theorem 1.1]. It is worth noting that our work does not require the $L^\infty $ -estimate used in [Reference Bao and Cheng1].

We make the following assumptions on V and g:

1. (V) $V\in C(\mathbb {R}^{N})$ and for some $q\in (1,2)$ and some $r>0$ ,

   $$ \begin{align*} V>0\ \text{in}~\mathbb{R}^N\setminus B_r, \quad V^{-q/(2-q)}\in L^1(\mathbb{R}^N\setminus B_r); \end{align*} $$

2. (g ₁) $g\in C(\mathbb {R})$ is odd and sublinear at zero, that is,

   $$ \begin{align*} \lim_{t\rightarrow0}\frac{g(t)t}{t^q}=a \quad\text{for some}~a>0; \end{align*} $$

3. (g ₂) $g\in C(\mathbb {R})$ is subcritical, that is,

   $$ \begin{align*} \lim_{|t|\rightarrow\infty}\frac{g(t)t}{|t|^{2\cdot2^*}}=0, \end{align*} $$
   where the critical Sobolev exponent $2^*=2N/(N-2)$ for $N\ge 3$ and $2^*=\infty $ for $N=2$ .

Before stating our results, we introduce a suitable function space in which we will find critical points of $\phi $ . Since V can be negative inside the big ball $B_r$ , as in [Reference Liu and Zhao13, page 1872], we choose $m\ge 1$ such that $\tilde {V}(x)=V(x)+m\ge 1$ for all $x\in \mathbb {R}^N$ . We equip the subspace

$$ \begin{align*} X=\bigg\{u\in H^1(\mathbb{R}^N) \ \bigg|\int{V}(x)u^2<\infty\bigg\} \end{align*} $$

with the inner product

$$ \begin{align*} (u, w)_X=\int(\nabla u\cdot\nabla w+\tilde{V}(x)uw) \end{align*} $$

and the corresponding norm $\Vert u\Vert _X=(u, u)^{1/2}_X$ . Then, X is a Hilbert space and, by Liu and Zhao [Reference Liu and Zhao13, Lemma 2.1], there is a compact embedding $X\hookrightarrow L^s(\mathbb {R}^N)$ for $s\in [q,2^*)$ . We are now ready to state our main result.

Remark 1.2. The assumption $(V)$ allows the potential V to be sign-changing. However, $\tilde {V}$ and $\tilde {g}(t)=g(t)+mt$ satisfy the assumptions $(V)$ , $(g_1)$ and $(g_2)$ . Thus, (1.2) is equivalent to the equation

$$ \begin{align*} -\Delta u+ \tilde{V}(x)u-u\Delta(u^2)=\tilde{g}(u) \quad\text{in}~\mathbb{R}^N. \end{align*} $$

Using the same idea, we obtain a similar result for the fourth-order quasilinear Schrödinger equations of the form (taking $\alpha =1$ , $3\leq N\leq 5$ in (1.1))

(1.5) $$ \begin{align} \Delta^2u-\Delta u+ V(x)u-u\Delta(u^2)=g(u) \quad\text{in}~\mathbb{R}^N, \end{align} $$

which was used to model travelling waves in suspension bridges by Lazer and McKennna [Reference Lazer and McKenna9]. Since the first work of Chen et al. [Reference Chen, Liu and Wu3], the superlinear fourth-order quasilinear Schrödinger equation (1.5) has been an active field of research (see, for example, [Reference Liu and Zhao15, Reference Shen and Yang19, Reference Yin and Jiang23]). However, it seems there are few results involving a nonlinearity that is sublinear at zero.

At first glance, (1.5) seems more complicated than (1.2) because of the biharmonic term $\Delta ^2u$ . Actually, (1.5) has a variational structure and does not require a nonlinear transformation like the functional in (1.2). However, we will be seeking the critical points of a functional I in $H^2(\mathbb {R}^N)$ , so that we must require g to have subcritical growth:

1. (g ₃) $g\in C(\mathbb {R})$ is subcritical, that is,

   $$ \begin{align*} \lim_{|t|\rightarrow\infty}\frac{g(t)t}{|t|^{2_*}}=0\text{,} \end{align*} $$
   where $2_*$ is the critical Sobolev exponent for $H^2(\mathbb {R}^N)$ , namely $2_*=+\infty $ for $N\leq 4$ and $2_*=2N/(N-4)$ for $N>4$ .

Under the assumptions $(V)$ , $(g_1)$ and $(g_3)$ , we can show that the functional I, where $I: H^2(\mathbb {R}^N)\rightarrow \mathbb {R}$ is given by

$$ \begin{align*} I(u)=\frac{1}{2}\int(|\Delta u|^2+|\nabla u|^2+V(x)u^2)+\int u^2|\nabla u|^2-\int g(u), \end{align*} $$

is well defined and is of class $C^1$ , and that u solves (1.5) if and only if it is a critical point of I.

Since the potential V is allowed to be sign-changing, as for (1.2), we introduce the appropriate function space for (1.5). There exists a constant $m\ge 1$ such that $\tilde {V}(x)=V(x)+m\ge 1$ and $\tilde {g}(t)=g(t)+mt$ . We equip the subspace

(1.6) $$ \begin{align} E=\bigg\{u\in H^2(\mathbb{R}^N) \ \bigg|\int {V}(x)u^2<\infty\bigg\} \end{align} $$

with the inner product

$$ \begin{align*} (u, w)_E=\int(\Delta u\Delta w+\nabla u\cdot\nabla w+\tilde{V}(x)uw) \end{align*} $$

and the corresponding norm $\Vert u\Vert _E=(u, u)^{1/2}_E$ . Then, E is a Hilbert space.

Remark 1.4. For (1.5), we make a remark similar to Remark 1.2. Equation (1.5) is equivalent to the equation

$$ \begin{align*} \Delta^2u-\Delta u+ \tilde{V}(x)u-u\Delta(u^2)=\tilde{g}(u)\quad\text{in}~\mathbb{R}^N\text{.} \end{align*} $$

To prove Theorems 1.1 and 1.3, it suffices to find a sequence of critical points of $\phi $ and I, respectively. To this end, we need the following version of Clark’s theorem due to Liu and Wang [Reference Liu and Wang17].

Proposition 1.5 [Reference Liu and Wang17, Theorem 1.1]

Let X be a Banach space and $\Phi \in C^1(X,\mathbb {R})$ be an even coercive functional satisfying the $(\mbox {PS})_c$ condition for $c\leq 0$ and $\Phi (0)=0$ . If for any $k\in \mathbb {N}$ there is a k-dimensional subspace $X_k$ and $\delta _k>0$ such that

$$ \begin{align*} \sup_{X_k\cap S_{\delta_k}}\Phi<0\text{,} \end{align*} $$

where $S_r=\{u\in X\mid \Vert u\Vert =r\}$ for $r>0$ , then $\Phi $ has a sequence of critical points $u_k\neq 0$ such that $\Phi (u_k)\leq 0$ , $u_k\rightarrow 0$ .