2. Moment selection in forecast rationality

This section explores the critical role of moment selection in testing forecast rationality within the EKT (2005) framework, highlighting both its importance and challenges, and motivating the proposed estimation procedure. Forecast rationality in the EKT (2005) framework is assessed through an asymmetric loss function, capturing a forecaster’s preferences via an attitude parameter $\alpha$ that reflects asymmetric penalties for over- and under-predictions. Effective estimation of $\alpha$ requires that we identify the optimal forecast model, along with selecting moments that capture all relevant and valid information.

Consider a stochastic process $ X =\{X_{t} \, : \, \Omega \rightarrow \mathbb{R}^{m+1}, m \in \mathbb{N}, t=1, \dots , T\}$ on a probability space $(\Omega , \mathcal{F}, P)$ , where $\mathcal{F}=\left \{\mathcal{F}_{t}, t=1,\dots ,T \right \}$ and $\mathcal{F}_{t}$ is the $\sigma$ -field generated by $\mathcal{F}_{t}=\sigma \left \{X_{s}, s \leqslant t\right \}$ . Let $Y_{t}$ be the continuous variable of interest in the observed vector $X_t$ . The $h$ -step-ahead forecast $f_{t+h}$ of $Y_{t+h}$ depends on an information set $\mathcal{F}_{t}$ and is modeled as $f_{t+h} = \theta ^{\prime } W_{t}$ , where $\theta$ is an unknown vector of parameters and $W_{t}$ contains $\mathcal{F}_{t}$ -measurable variables. EKT (2005) propose a generalized loss function, $L(p, \alpha , \theta )$ , that accommodates asymmetry in the penalization of forecast errors

(2.1) \begin{equation} L(p, \alpha , \theta ) \equiv \left [\alpha +(1-2 \alpha ) \mathbb{I}\left (Y_{t+h}-f_{t+h}\lt 0\right )\right ]\lvert Y_{t+h}-f_{t+h}\rvert ^{p}, \end{equation}

where $Y_{t+h}-f_{t+h}=Y_{t+h}-\theta ^{\prime } W_{t}$ represents the forecast error $e_{t+h}$ . For analytical simplicity, EKT (2005) assume that the true value of the loss shape parameter $p_0$ is known and deliberate on the estimation of forecaster’s attitude parameter $\alpha$ . Given $p_{0}$ and the true value $\alpha _{0}$ , forecast rationality implies that the forecast $f_{t+h}^{*}=\theta ^{* \prime } W_{t}$ is optimal when it minimizes expected loss. Formally, this optimality condition is represented by $\theta ^{*}=\arg \min _{\theta \in \Theta }$ $\mathbb{E}\left \{L\left (p_{0}, \alpha _{0}, \theta \right )\right \}$ , leading to the moment condition

(2.2) \begin{equation} \mathbb{E} \left (W_{t} \left [\mathbb{I}\left (Y_{t+h}-f_{t+h}^{*}\lt 0\right )-\alpha _{0}\right ]\lvert Y_{t+h}-f_{t+h}^{*}\rvert ^{p_{0}-1}\right )=0. \end{equation}

Note that (2.2) can be considered as a moment condition $ \mathbb{E}[W_{t}(\mathbb{I}(e_{t+h}^{*}\lt 0)-\alpha _{0})\lvert e_{t+h}^{*}\rvert ^{p_{0}-1}]=0$ , where $e_{t+h}^{*} = Y_{t+h}-f_{t+h}^{*}=Y_{t+h}-\theta ^{* \prime } W_{t}$ is the forecast error given $f^*_{t+h}$ . It suggests that if the forecasts are optimal, then any information $W_t$ must be correctly included in $f_{t+h}^{*}$ , so that $W_{t}(\mathbb{I}(e_{t+h}^{*}\lt 0)-\alpha _{0})$ is orthogonal to $\lvert Y_{t+h}-\theta ^{* \prime } W_{t} \rvert ^{p_0-1}$ . For accurate estimation of $\alpha$ , this condition highlights the importance of selecting the right moments that capture all relevant information in $W_t$ .

In practice, not all variables in $W_t$ may be observable or known to the forecast user. Forecast users typically rely on an observed $d$ -vector $V_t \subset W_t$ to approximate the information set, leading to a modified moment condition

(2.3) \begin{equation} \mathbb{E} (V_{t} [\mathbb{I}(Y_{t+h}-f_{t+h}^{*}\lt 0)-\alpha _{0}]\lvert Y_{t+h}-f_{t+h}^{*}\rvert ^{p_{0}-1})=\mathbb{E}(C_t-\alpha _0 B_t)=0, \end{equation}

where $ C_t=V_{t} \mathbb{I}(Y_{t+h}-f_{t+h}^{*}\lt 0) \lvert Y_{t+h}-f_{t+h}^{*}\rvert ^{p_{0}-1}$ and $B_t=V_{t} \lvert Y_{t+h}-f_{t+h}^{*}\rvert ^{p_{0}-1}$ . Thenceforth, $\alpha _0$ can be solved by minimizing the quadratic norm of (2.3) such that $\min _{\alpha } Q(\alpha )=\mathbb{E}(C_t-\alpha B_t)' S^{-1} \mathbb{E}(C_t-\alpha B_t)$ , in which $S=\mathbb{E}((C_t-\alpha B_t)(C_t-\alpha B_t)')=\mathbb{E}[V_{t} V_{t}^{\prime }\big(\mathbb{I}\big(Y_{t+h}-f_{t+h}^{*}\lt 0\big)-\alpha _{0}\big)^{2}\lvert Y_{t+h}-f_{t+h}^{*}\rvert ^{2 p_{0}-2}]$ is a positive definite weighting matrix. Solving for $\alpha _0$ under this condition is non-trivial because the inclusion of irrelevant or invalid instruments in $V_t$ can distort the estimation, introducing inefficiencies and bias. Therefore, moment selection is crucial to isolating only those instruments that are both valid and relevant for consistent estimation of $\alpha$ .

Given a sample of instrument vectors $\left (V_{1}^{\prime }, \ldots , V_{T}^{\prime }\right )^{\prime }$ with $T$ observations, an estimator for $\alpha _0$ is obtained as

(2.4) \begin{equation} \hat {\alpha }_{T}=(\hat {B}_{T}^{\prime } \hat {S}_{T}^{-1} \hat {B}_{T})^{-1}(\hat {B}_{T}^{\prime } \hat {S}_{T}^{-1} \hat {C}_{T}), \end{equation}

where $\hat {B}_{T}=1/T \sum _{t=1}^{T} V_{t}\lvert Y_{t+h}-{f}^*_{t+h}\rvert ^{p_{0}-1}$ , $\hat {C}_{T}={1}/{T} \sum _{t=1}^{T} V_{t} \mathbb{I}(Y_{t+h}-{f}^*_{t+h}\lt 0)\lvert Y_{t+h}-{f}^*_{t+h}\rvert ^{p_{0}-1}$ , and $\hat {S}_{T} \equiv {1}/{T} \sum _{t=1}^{T} [V_{t} V_{t}^{\prime }(\mathbb{I}(Y_{t+h}-f_{t+h}^{*}\lt 0)-\hat {\alpha }_{T})^{2}\lvert Y_{t+h}-f_{t+h}^{*}\rvert ^{2 p_{0}-2}]$ is a consistent estimate of $S$ . The accuracy of $\hat {\alpha }_{T}$ depends heavily on the selection of valid instruments in $V_t$ , as each set of instruments produces different estimates and asymptotic variances. In practice, however, forecast users may not know which variables in $V_t$ are valid, underscoring the need for an effective estimation procedure to assess instrument validity and avoid bias.

Following the estimator construction, we have asymptotic normality (EKT (2005) $Proposition$ 4) for $\hat {\alpha }_{T}$ , i.e., $T^{\frac {1}{2}}\left (\hat {\alpha }_{T}-\alpha _{0}\right ) \stackrel {d}{\rightarrow } \mathcal{N}(0,(B^{\prime } S^{-1} B)^{-1})$ with $B=\mathbb{E}(B_t)$ , which can be utilized to test whether $\hat {\alpha }_{T}$ differs significantly from $\alpha _{0}$ . For rationality testing (as outlined in EKT (2005) $Corollary$ 5), we can conduct a joint test of forecast rationality and flexible loss specification when $d \gt 1$ instruments are available, using the overidentification test statistic

(2.5) \begin{equation} J_{T}\left (\hat {\alpha }_{T}\right )=T \times \hat {Q}_{T}\left (\hat {\alpha }_{T}\right )=T \times \hat {A}_{T}\left (\hat {\alpha }_{T}\right )^{\prime } \hat {S}_{T}^{-1}\left (\hat {\alpha }_{T}\right ) \hat {A}_{T}\left (\hat {\alpha }_{T}\right ) \stackrel {d}{\rightarrow } \chi _{d-1}^{2}, \end{equation}

where $\hat {A}_{T}\left (\hat {\alpha }_{T}\right )=\hat {C}_{T}-\hat {\alpha }_{T} \hat {B}_{T}$ and a large $J_{T}(\hat {\alpha }_T)$ suggests rejection of forecast rationality. In the presence of a single instrument, overidentification testing is not feasible, yielding a unique closed-form solution for $\hat {\alpha }_T$ . For testing loss symmetry, we can fix $\hat {\alpha }_T=0.5$ , resulting in

(2.6) \begin{equation} J_{T}\left (0.5 \right )=T \times \hat {Q}_{T}\left (0.5\right )=T \times \hat {A}_{T}\left (0.5\right )^{\prime } \hat {S}_{T}^{-1}\left (\hat {\alpha }_{T}\right ) \hat {A}_{T}\left (0.5\right )\stackrel {d}{\rightarrow } \chi _{d}^{2} \end{equation}

with $d$ degrees of freedom as no parameter estimation is involved. As suggested in EKT (2005), the use of $\hat {S}_{T}^{-1}\left (\hat {\alpha }_{T}\right )$ can improve the finite sample power. The rejection of this test indicates asymmetry in loss preferences, provided the rationality hypothesis (2.5) is not rejected. While the conditional distribution of (2.6) is difficult to derive, following Wang and Lee (Reference Wang and Lee2014), we interpret $J_T(0.5)$ as a symmetry test following the selection of all valid and relevant moment conditions. These considerations underscore that the statistical properties of the GMM estimator $\hat {\alpha }_T$ depend heavily on the reliability of the moment conditions. Thus, incorporating information on both moment validity and relevance is advantageous for conducting robust rationality and symmetry tests.

Remark 2.1. EKT (2005) assume that the forecasts are generated using a linear model $f_{t+h}=\theta ' W_t$ . In the case of a nonlinear model where $f_{t+h}=f(\theta , W_t)$ and $f(\cdot )$ is continuously differentiable, there exists a parameter $\theta ^*$ that satisfies

\begin{equation*} \mathbb{E} \big (f^{*'}_{t+h} \big [\mathbb{I}\big (Y_{t+h}-f_{t+h}^{*}\lt 0\big )-\alpha _{0}\big ]\lvert Y_{t+h}-f_{t+h}^{*}\rvert ^{p_{0}-1}\big )=0, \end{equation*}

where $f^{*'}_{t+h}=\partial f(\theta ^*, W_t)/ \partial \theta ^*$ . This allows for the use of $V_{v,t}$ , a subvector of $f^{*'}(\theta ^*, W_t)$ , to conduct the rationality test. In both linear and nonlinear cases, the selection of valid and relevant moments is essential for accurate estimation and testing, as the choice of moments directly influences the efficiency and consistency of the parameter estimates; see simulations in Section 4 . However, the nonlinear case poses an additional challenge, as $f^*(\theta ^*, W_t)$ depends on both $\theta ^*$ and $W_t$ , while the true $\theta ^*$ used by the forecast user is typically unknown. Consequently, this paper focuses exclusively on the linear case.

Rationality tests under asymmetric loss functions have received extensive attention in forecasting literature, as traditional reliance on the squared error loss function may not fully capture the diverse preferences of forecasters; see Christoffersen and Diebold (Reference Christoffersen and Diebold1997), Christoffersen (Reference Christoffersen1998), Patton and Timmermann (Reference Patton and Timmermann2007), Elliott et al. (Reference Elliott, Komunjer and Timmermann2005, Reference Elliott, Komunjer and Timmermann2008), Patton (Reference Patton2020), Binder and Sekkel (Reference Binder and Sekkel2024), among others. The EKT (2005) framework is particularly notable for its flexibility in accommodating various asymmetric loss functions, enabling more tailored tests of forecast optimality across diverse forecasting contexts. However, a critical underlying assumption in EKT (2005) is that the forecast user has access to a comprehensive set of information, as captured by the moment condition in (2.2). In practice, forecast users often have incomplete information compared to forecasters, making (2.3) particularly useful, as it reflects a more realistic scenario where users cannot access all forecaster information.

Figure 2.

Moment selection in EKT (2005) framework.

Note : The plot highlights the importance of selecting valid and relevant moments in the EKT (2005) framework to ensure efficient estimation of the forecaster’s attitude parameter $\alpha$ , leading to more reliable and accurate forecast rationality tests. By carefully choosing moments, we can enhance the robustness of the model and improve the accuracy of subsequent analyses.

Despite having access to an observed set of instruments $V_t$ , it remains challenging to determine how forecasting institutions incorporate these variables into their models and which instruments will yield the most efficient estimate of the forecaster’s attitude parameter $\alpha$ . Data-driven selection methods, such as those proposed by Donald and Newey (Reference Donald and Newey2001), which focus on isolating valid instruments within a predefined subset, provide partial solutions but remain limited. When many potential instruments are available, and their validity is uncertain, selecting irrelevant or invalid instruments can lead to biased estimates and unreliable rationality testing outcomes. This highlights the importance of rigorously assessing instrument validity and relevance, which is the primary focus of this paper (see Figure 2). To address these challenges, we in the following section employ a P-GMM estimation approach within the EKT (2005) framework, enabling systematic selection of all valid and relevant instruments, thus ensuring consistent and efficient estimation and testing under flexible loss preferences.

3. Penalized moment selection: validity and relevance

The estimation and testing of forecast rationality under flexible loss functions are of significant interest for forecast users, as these processes are critical both theoretically and empirically. A key concern arises when forecasts fail to efficiently incorporate the information in $V_t$ ; in such cases, the estimator of $\alpha$ would differ substantially across moment conditions, leading to a rejection in the overidentification test. Despite the practical importance of accurate moment selection, empirical applications currently lack standardized guidelines for choosing appropriate instruments. Determining the suitable (valid and relevant) instruments to reveal the forecaster’s attitude parameter $\alpha$ and to improve estimation efficiency remains an open question in the context of EKT (2005). To illustrate the motivation behind the methodology proposed in this paper, consider a set of potentially misspecified moment conditions

(3.1) \begin{equation} \mathbb{E} \left (V_{t} \left [\mathbb{I}\left (Y_{t+h}-f_{t+h}^{*}\lt 0\right )-\alpha _{0}\right ]\lvert Y_{t+h}-f_{t+h}^{*}\rvert ^{p_{0}-1}\right ) \overset {?}{=} 0, \end{equation}

where the instruments $V_t$ satisfy equality only for certain moments, indicated by the notation “ $\overset {?}{=}$ ”. Including these misspecified moments in estimation risks producing inconsistent results for $\alpha$ . To address this, we employ the P-GMM estimation method (Liao, Reference Liao2013; Cheng and Liao, Reference Cheng and Liao2015), which can select valid and relevant moment conditions simultaneously to obtain not only an unbiased but also a more efficient estimator of $\alpha$ (Figure 3).

Figure 3.

Roadmap of instrument selection.

Note : The instruments could be divided into four types: valid and invalid, relevant and irrelevant. To reduce bias, valid instruments must be selected. For obtaining efficiency, relevant instruments should be chosen. To get the unbiased and efficient estimation of the forecaster’s attitude parameter $\alpha$ , we need to choose valid and relevant instruments, which is close to “Oracle Estimator.”

Practically, (2.3) serves as the fundamental criterion for selecting valid moment conditions. Specifically, if an instrument fails to satisfy (2.3), it cannot be used for estimation purposes. Once the set of valid instruments is determined, the next step is to identify the relevant instruments by examining the variance of the resulting estimators. A set of moment conditions that produces a smaller variance, indicating that it incorporates more valuable information, will be considered relevant, with $(B'S^{-1}B)^{-1}$ serving as the primary metric for this relevance. These two selection steps, focused on validity and relevance, are critical and widely emphasized in the literature on instrument or moment selection; see Lai et al. (Reference Lai, Small and Liu2010). P-GMM estimation streamlines these two steps by incorporating both validity and relevance assessments through a penalized approach, attaching an appropriate penalty to the standard GMM criterion. This method offers a systematic approach to refine moment selection, balancing bias and efficiency. Before presenting the specifics of P-GMM estimation within the EKT (2005) framework, we first define the concepts of valid and relevant instruments as applied in this paper.

To approach the moment selection process systematically, we categorize the instrument set $V_t$ , with dimension $\dim (V_t)=d$ , into two primary groups: “Good” and “Doubt” instruments. This categorization, represented as $V_t=(V_{G,t}, V_{D,t})'$ , distinguishes the instruments $V_{G,t} \in \mathbb{R}^{k_0}$ (the “Good” instruments) from $V_{D,t} \in \mathbb{R}^{d-k_0}$ (the “Doubt” instruments) with $k_0\lt d$ . For the “Good” group, we assume that the corresponding moment function, $g_G(V_{G,t}, \alpha )=C_{G,t}-\alpha B_{G,t} \in \mathbb{R}^{k_{0}}$ , is directly usable for the identification of parameter $\alpha$ without further testing for validity or relevance, where $ C_{G,t}=V_{G,t}\mathbb{I}(Y_{t+h}-f_{t+h}^{*}\lt 0) \lvert Y_{t+h}-f_{t+h}^{*}\rvert ^{p_{0}-1}$ and $B_{G,t}=V_{G,t} \lvert Y_{t+h}-f_{t+h}^{*}\rvert ^{p_{0}-1}$ . For instance, setting $k_0=2$ allows us to define $V_{G,t}=(1, Y_{t-1})'$ , where the constant and lagged dependent variable are acknowledged as established valid and relevant instruments. To construct an optimal collection of moment conditions, we apply the framework by Liao (Reference Liao2013) and Cheng and Liao (Reference Cheng and Liao2015), where the “Doubt” moment conditions are represented by $g_D(V_{D,t}, \alpha )=C_{D,t}-\alpha B_{D,t} \in \mathbb{R}^{d-k_{0}}$ and are subject to tests for both validity and relevance. Here, $ C_{D,t}=V_{D,t}\mathbb{I}(Y_{t+h}-f_{t+h}^{*}\lt 0) \lvert Y_{t+h}-f_{t+h}^{*}\rvert ^{p_{0}-1}$ and $B_{D,t}=V_{D,t}$ $\lvert Y_{t+h}-f_{t+h}^{*}\rvert ^{p_{0}-1}$ . The combined moment function, $g(V_t, \alpha )=C_t-\alpha B_t=[g_G(V_{G,t}, \alpha ), g_D(V_{D,t}, \alpha )]' \in \mathbb{R}^{d}$ , thus encompasses both the “Good” and “Doubt” sets of all available moments.

To guide the selection of appropriate moments, we introduce a slackness parameter $\beta$ defined as $\beta =\mathbb{E}[g_D(V_{D,t}, \alpha )]$ . This parameter allows us to refine the estimation within the P-GMM framework by adjusting for potential misspecifications in the “Doubt” instruments. Integrating this slackness parameter into (2.3), we redefine the expectation condition as follows

(3.2) \begin{equation} \mathbb{E}\left (V_{t}\left [\mathbb{I}\left (Y_{t+h}-f_{t+h}^{*}\lt 0\right )-\alpha _{0}\right ]\lvert Y_{t+h}-f_{t+h}^{*}\rvert ^{p_{0}-1}-\begin{bmatrix} 0\\ \beta _0 \end{bmatrix} \right )\equiv \mathbb{E} \begin{bmatrix} g_G(V_{G,t}, \alpha _0)\\ g_D(V_{D,t}, \alpha _0)-\beta _0 \end{bmatrix} =0, \end{equation}

where $\beta _0=\mathbb{E}[g_D(V_{D,t}, \alpha _0)]$ . Subsequently, the instrument selection process is conducted within the set ${D}=\mathcal{A} \cup \mathcal{B}_{1} \cup \mathcal{B}_{0}$ with the cardinality $\lvert {D} \rvert =d-k_0$ , in which $\mathcal{A}$ represents the set of valid and relevant moments, $\mathcal{B}_0$ indicates the set of valid but irrelevant ones, and $\mathcal{B}_1$ is the set of invalid moments. While both the moments in set $\mathcal{A}$ and $\mathcal{B}_0$ are valid, only those within $\mathcal{A}$ are considered relevant for efficient estimation. Leveraging this structure, the simultaneous estimation of $\alpha$ and $\beta$ is achieved through (3.2), allowing us to derive robust estimates that incorporate the complete set of validated instruments.

The P-GMM estimation approach achieves efficient estimation and selective moment selection by minimizing a penalized objective function designed to balance fit with the complexity of the model, thereby optimizing both parameter estimation and the relevance of the instruments employed. Specifically, this is expressed as

(3.3) \begin{equation} {\hat {\alpha }}_{0}=\underset {\alpha }{\arg \min }\bigg [{\overline {g}_T}(\alpha , \beta )^{\prime } S^{-1}_T {\overline {g}_T}(\alpha , \beta )+\lambda _T \sum _{\ell \in \mathcal{A} \cup \mathcal{B}_{1} \cup \mathcal{B}_{0}} \omega _{T,\ell }\lvert \beta _{\ell }\rvert \bigg ], \end{equation}

where $\overline {g}_{T}(\alpha , \beta )=\frac {1}{T} \sum _{t=1}^{T} [ g_G(V_{G,t}, \alpha ), g_D(V_{D,t}, \alpha )-\beta ]'$ represents the sample average of the moment conditions for both “Good” and “Doubt” instruments, $S_T$ denotes a symmetric weighting matrix, and $\lambda _T$ indicates a tuning parameter that calibrates the penalty applied to each slackness parameter $\beta _{\ell }$ , encouraging sparsity in the selection of moment conditions. The penalty term $\sum _{\ell \in \mathcal{A} \cup \mathcal{B}_{1} \cup \mathcal{B}_{0}} \omega _{T,\ell }\lvert \beta _{\ell }\rvert$ plays a crucial role in determining which moments are retained in the final model, where $\omega _{T,\ell }$ acts as an adaptive weight for each slackness parameter. This adaptive adjustment term, defined by $\omega _{T,\ell }=\dot {\mu }_{T,\ell }^{r_{1}}\lvert \dot {\beta }_{T,\ell }\rvert ^{-r_{2}}$ , incorporates both empirical information and preliminary estimates to refine moment selection. The parameter $\dot {\mu }_{T,\ell }$ , as further discussed below in Remark3.2, quantifies the information content of moment $\ell \in {D}$ , thereby emphasizing moments that contribute substantial information to the model. The term $\dot {\beta }_{T,\ell }$ represents a preliminary consistent estimator of ${\beta }_{0,\ell }=\mathbb{E}[g_D(V_{D,t,\ell }, \alpha _0)]$ . By setting positive constants $r_1$ and $r_2$ ( $r_2 \leq r_1$ ), we can control the sensitivity of the penalty, ensuring that the weight $\omega _{T,\ell }$ diminishes for moments with negligible relevance, thereby reducing the likelihood of including irrelevant or redundant instruments in the model. By minimizing the objective function (3.3), P-GMM achieves an optimal balance between bias and variance, leveraging the penalty to enforce a parsimonious model that reduces estimation error while retaining only the relevant moments. This approach is particularly beneficial in large-dimensional settings where the selection of valid and relevant moments is computationally intensive.

The parameter $\dot {\mu }_{T,\ell }$ in $\omega _{T,\ell }$ is determined based on a relevance criterion given by

(3.4) \begin{equation} \dot {\mu }_{T, \ell }=\rho _{\max }\big (\hat {B}'_{G+\ell }\hat {S}^{-1}_{G+\ell } \hat {B}_{G+\ell }-\hat {B}'_{G} \hat {S}^{-1}_{G} \hat {B}_{G}\big ), \ \ell \in {D}=\mathcal{A} \cup \mathcal{B}_{1} \cup \mathcal{B}_{0}, \end{equation}

in which $\hat {B}_{G}$ and $\hat {S}_{G}$ correspond to estimators derived analogously to $\hat {B}_{T}$ and $\hat {S}_{T}$ as defined in Section 2. The term $\rho _{\max }$ captures the maximum increase in information content gained by adding moment condition $\ell$ , highlighting the additional contribution of the moment $\ell$ to estimator efficiency. Specifically, $(\hat {B}'_{G} \hat {S}^{-1}_{G} \hat {B}_{G})^{-1}$ represents the asymptotic variance of the optimal estimator for $\alpha _0$ prior to adding the moment, while $(\hat {B}'_{G+\ell }\hat {S}^{-1}_{G+\ell } \hat {B}_{G+\ell })^{-1}$ reflects the asymptotic variance once the moment condition $\ell$ is included. The criterion thus assesses the relevance of the moment $\ell$ in terms of its informational gain relative to previously selected moments in $G$ , incorporating only those conditions that significantly enhance estimation efficiency. We then have the following lemma demonstrating the efficiency gain.

Lemma1 establishes that incorporating a valid moment $\ell$ does not diminish the efficiency of the P-GMM post-selection estimator. This property ensures that the P-GMM procedure is capable of retaining only those moment conditions that substantively contribute to estimator precision, systematically refining the estimator by integrating the most informative moments. This mechanism is crucial in high-dimensional settings, where only a subset of moments enhances estimator accuracy and efficiency, thus optimizing the asymptotic behavior of the estimator in large samples.

The initial estimates of the parameters $\dot {\alpha }_{T}$ and $\dot {\beta }_{T,\ell }$ can be obtained by setting $\lambda _T=0$ in (3.3), essentially applying the unpenalized P-GMM criterion. Thereafter, the value of $\dot {\mu }_{T,\ell }$ could be obtained from (3.4). Following Cheng and Liao (Reference Cheng and Liao2015), the tuning parameter $\lambda _T$ for the $\ell$ th moment condition is calculated as

(3.5) \begin{equation} \hat {\lambda }_{T, \ell }=2\big \| \hat {S}_{T}^{-\frac {1}{2}}(\ell ) \widehat {\Pi }_{T}\big \| d^{\frac {r_{2}}{4}} T^{-\frac {1}{2}-\frac {r_{2}}{4}}, \end{equation}

where $\| \cdot \|$ represents the Euclidean norm, $\hat {S}_{T}^{-1}(\ell )$ denotes the $\ell$ th row of the matrix $\hat {S}_{T}^{-1}$ , $\widehat {\Pi }_{T}$ is the estimator of $\Pi _{T}=I_{d}-S_{T}^{-\frac {1}{2}} \Gamma _{\alpha }\left (\Gamma _{\alpha }^{\prime } S^{-1}_{T} \Gamma _{\alpha }\right )^{-1} \Gamma _{\alpha }^{\prime } S_{T}^{-\frac {1}{2}}$ , and $\Gamma _{\alpha }$ indicates the first derivative of $\overline {g}_T(\alpha ,\beta )$ . The introduction of the $L_1$ penalty in (3.3) enables the P-GMM estimator to achieve selection consistency. This consistency is achieved by correctly distinguishing between valid, relevant, and irrelevant moment conditions. Specifically, in large samples, P-GMM asymptotically identifies non-zero parameters ${\beta }_{\ell }$ in sets $\mathcal{B}_0$ and $\mathcal{B}_1$ as non-zero, indicating their irrelevance or invalidity. In contrast, it consistently estimates parameters within $\mathcal{A}$ as zero, effectively excluding any invalid or irrelevant moments. Thus, the P-GMM approach achieves consistent moment selection, systematically incorporating all valid and relevant moment conditions while excluding those that do not contribute meaningfully to the estimation of $\alpha$ within the EKT (2005) framework.

Theorem 1. Let $\hat {\beta }_{\ell }$ denote the estimator of the slackness parameter indexed by $\ell$ from (3.3). Under the model specifications and conditions established in previous sections, the following selection consistency results hold asymptotically

\begin{equation*} \operatorname {P}\left (\cup _{\ell \in \mathcal{B}_{1}}\left \{\hat {\beta }_{\ell }=0\right \}\right ) \rightarrow 0 \ \text{ as } T \rightarrow \infty \quad \text{(Invalid)}, \end{equation*}

\begin{equation*} \operatorname {P}\left (\cup _{\ell \in \mathcal{B}_{0}}\left \{\hat {\beta }_{\ell }=0\right \}\right ) \rightarrow 0 \ \text{ as } T \rightarrow \infty \quad \text{ (Irrelevant)}, \end{equation*}

\begin{equation*} \operatorname {P}\left (\cap _{\ell \in \mathcal{A}}\left \{\hat {\beta }_{\ell }=0\right \}\right ) \rightarrow 1 \ \text{ as } T \rightarrow \infty \quad \text{(Valid and Relevant)}. \end{equation*}

Theorem1 establishes the asymptotic behavior of the moment selection mechanism within the P-GMM estimator, demonstrating that invalid or irrelevant moments are asymptotically excluded while valid and relevant moments are retained to enhance estimator performance within the EKT (2005) framework. These results demonstrate that, with probability approaching one, the P-GMM procedure can reliably distinguish between valid, relevant moments and those that are either invalid or irrelevant. Consistent with established GMM theory (Hansen, Reference Hansen1982; Hall, Reference Hall2005), incorporating additional valid instruments leads to efficiency gains, as shown in Lemma1. As a result, the P-GMM approach not only identifies and selects all valid and relevant moment conditions but also optimizes the estimation of $\alpha$ through lower standard errors, thus improving estimator precision and robustness.

From an asymptotic perspective, selecting only highly informative moments enhances the efficiency of the estimator $\hat {\alpha }_0$ by focusing the information in $(B'S^{-1}B)^{-1}$ on the moments contributing most significantly. We can then achieve the following asymptotic normality with a reduced variance, aligning with EKT (2005) $Proposition$ 4 and leveraging the asymptotic properties of GMM.

Theorem 2. Under consistent moment selection in Theorem1, the asymptotic distribution of the P-GMM post-selection estimator $\hat {\alpha }_0$ is given by

\begin{equation*} \sqrt {T}\left ( \hat {\alpha }_0-\alpha _0 \right ) \stackrel {d}{\rightarrow } \mathcal{N}\left (0,(B_{\mathcal{A}}^{\prime } S_{\mathcal{A}}^{-1} B_{\mathcal{A}})^{-1}\right ), \end{equation*}

where $B_{\mathcal{A}}=\mathbb{E}(\partial \bar {g}_T(\alpha _0,\beta _0)/\partial \alpha )$ for moments in $\mathcal{A}$ and $S_{\mathcal{A}}$ denotes the corresponding symmetric weighting matrix, ensuring only valid and relevant moments contribute to the limiting distribution.

Theorem2 demonstrates that the P-GMM post-selection estimator achieves efficiency comparable to an oracle GMM estimator based on all valid and relevant moments, as supported by the simulation results below. This oracle property highlights the potential of P-GMM estimation to enhance the power of rationality and symmetry tests by reducing the impact of uninformative instruments. Consequently, by systematically excluding moments that are either irrelevant or invalid, P-GMM effectively optimizes estimator precision, thereby bolstering the robustness and efficiency of the estimator in practical applications where instrument selection poses a critical challenge.

4. Monte Carlo simulations

We conduct simulation experiments to evaluate the finite sample properties of the proposed P-GMM estimation within the EKT (2005) framework, analyzing cases of both linear and nonlinear dependencies between the forecast user’s information set and forecast error. For both scenarios, we treat the constant instrument as part of the valid and relevant set $V_{G,t}$ defined in Section 3. Throughout this section, we employ the identity matrix as the weighting matrix in the GMM optimization to avoid additional parameter estimation, with the forecast horizon set to $h=1$ . We explore the behavior of the estimator of $\alpha$ and investigate the characteristics of the $J$ -test statistics to underscore the impact of instrumental variable selection on estimator performance.

DGP 1: linear case

To investigate the efficacy of the proposed P-GMM approach in a high-dimensional setting, we consider a scenario with $d=30$ instruments in $V_t$ . We generate the random samples from the following data generating process (DGP)

(4.1) \begin{equation} Y_{t+1}=\theta _0+W_{1,t}\theta _1+W_{2,t}\theta _2+U_t, \end{equation}

where $Y_{t+1}$ represents the dependent variable, $W_{1,t}$ and $W_{2,t}$ are covariates, $U_t$ denotes the error term, and the parameter values are set to $\theta _0=1$ , $\theta _1=0.5$ (strong identification) or 0.07 (weak identification), and $ \theta _2=0.4$ , capturing a baseline configuration with significant contributions from each predictor. In this simulation, the goal is to differentiate the impact of valid, invalid, and irrelevant instruments on the estimation and the moment selection procedure. To achieve this, the simulated instrument set includes both a valid but irrelevant instrument vector $W_{v,t}$ and an invalid instrument vector $W_{inv,t}$ . The samples are generated through the following multivariate normal distribution

(4.2) \begin{equation} \begin{aligned} & \quad \quad \quad \quad \quad \left ( \begin{array}{c}{W_ {1,t}} \\ {W_{2,t}} \\{W_{v,t}}\\ {W^*_{v,t}} \\ {U_{t}}\end{array}\right ) \sim N \left ( 0, \left ( \begin{array}{ccccccccc} {1} & \quad {0.2} & \quad {0} & \quad {0} & {0} \\ {0.2} & \quad {1} & \quad {0} & \quad {0} & \quad {0} \\ {0} & \quad {0} & \quad {I_{14}} & \quad {0} & \quad {0} \\ {0} & \quad {0} & \quad {0} & \quad {I_{14}} & \quad {0} \\ {0} & \quad {0} & \quad {0} & \quad {0} & \quad {1} \end{array} \right ) \right ), \\[4pt] & W_{inv,t}(\ell )=W^*_{v,t}(\ell )+c_\ell \times U_t, \ \text{and} \ c_{\ell }=c_{0}+\frac {(\ell -1)\left (\overline {c}-c_{0}\right )}{d / 2-1}, \ \ell =1, \dots , d-k_0 \end{aligned} \end{equation}

with $k_0=3$ for $V_{G,t}=(1,W_{1,t},W_{2,t})'$ , where $ W_{inv,t}(\ell )$ and $W^*_{v,t}(\ell )$ are the $\ell$ th instruments in the simulated data. The formulation of $W_{inv,t}(\ell )$ incorporates a linear dependence on the error term $U_t$ through coefficients $c_{\ell }$ that vary across the set, enabling us to systematically examine the impact of invalid instruments of differing strengths on the estimator. Since $W_{inv,t}(\ell )$ is constructed to be correlated with $U_t$ , it does not meet the criteria for validity within the set of instruments used for estimating and testing forecast rationality.

DGP 2: nonlinear case

In this simulation setting, we explore a nonlinear DGP with a relatively high dimensionality, specifying $d=31$ instruments in $V_t$ . The model is specified as follows

(4.3) \begin{equation} Y_{t+1}=\theta _0+W^2_{1,t}\theta _1+(W_{1,t}\times W_{2,t})\theta _2+\exp (W_{3,t})\theta _3+U_t, \end{equation}

where $Y_{t+1}$ represents the dependent variable, and $W_{1,t}$ , $W_{2,t}$ , and $W_{3,t}$ are predictor variables. Each predictor is transformed through nonlinear functions to introduce diverse structural relationships within the model. The parameter values are set to $\theta _0=1$ , $\theta _1=0.5$ or 0.07, $ \theta _2=0.5$ , and $\theta _3=0.4$ , reflecting a scenario with varying weights on each component. Similar to DGP 1, the instrument set is augmented to include both a valid but irrelevant instrument vector $W_{v,t}$ and an invalid vector $W_{inv,t}$ . The samples are generated according to the following multivariate normal distribution

(4.4) \begin{equation} \begin{aligned} & \quad \quad \quad \quad \left ( \begin{array}{c}{W_ {1,t}}\\ {W_{2,t}} \\ {W_{3,t}}\\ {W_{v,t}}\\ {W^*_{v,t}} \\ {U_{t}}\end{array}\right ) \sim N \left ( 0, \left ( \begin{array}{ccccccccc} {4} & {0.2} & {0.04}& {0} & {0} & {0} \\ {0.2} & \quad {10} & \quad {0.2} & \quad {0} & \quad {0} & \quad {0} \\ {0.04} & \quad {0.2} & \quad {8} & \quad {0} & \quad {0} & \quad {0} \\ {0} & \quad {0} & \quad {0} & \quad {I_{14}} & \quad {0} & \quad {0} \\ {0} & \quad {0} & \quad {0} & \quad {0} & \quad {I_{14}} & \quad {0} \\ {0} & \quad {0} & \quad {0} & \quad {0} & \quad {0} & \quad {1} \end{array} \right ) \right ), \\[4pt] & W_{inv,t}(\ell )=W^*_{v,t}(\ell )+c_\ell \times U_t, \ \text{and} \ c_{\ell }=c_{0}+\frac {(\ell -1)\left (\overline {c}-c_{0}\right )}{d / 2-1}, \ \ell =1, \dots , d-k_0 \end{aligned} \end{equation}

with $k_0=4$ for the set of valid instruments $V_{G,t}=(1,W^2_{1,t},W_{1,t}W_{2,t},\exp (W_{3,t}))'$ . By introducing correlations between $W_{inv,t}(\ell )$ and $U_t$ through varying coefficients $c_{\ell }$ , we systematically incorporate degrees of invalidity within the instrument set.

In both DGPs, the initial parameters are set to $c_0=0.5$ and an upper bound $\overline {c}=2.4$ , with user-selected constants $r_1=3$ and $r_2=2$ chosen in line with Cheng and Liao (Reference Cheng and Liao2015). We investigate two sample sizes, $T=200$ and $T=2000$ , each evaluated over 5000 simulation repetitions to ensure robustness. In all simulation experiments, each dataset is split such that 50% of the observations are used by the forecast user to estimate the forecasting model parameters $ \theta$ . The estimated coefficients $ \hat {\theta }$ are then used to generate one-step-ahead forecasts for the remaining 50% of the sample. This fixed-window design preserves parameter estimation error in the forecasts, allowing us to assess the finite sample performance of the proposed P-GMM estimator under realistic conditions. As illustrated in Remark3.3, the induced estimation uncertainty is not incorporated in the theoretical analysis, which treats forecasts as primitives, but is accounted for in the simulations to evaluate the robustness of the inference procedures. The forecasts are generated by estimating $\hat {\theta }$ for a fixed loss shape parameter $p_0=2$ , given a selected subset of variables from the forecast user’s information set and the forecaster’s known asymmetric preference parameter $\alpha _0$ , where $\hat {\theta }$ represents the estimated parameter vector within each respective DGP framework. Note that it is reasonable to fix the loss shape parameter $p$ since Komunjer and Owyang (Reference Komunjer and Owyang2012) have demonstrated that different values of $p$ yield consistent estimates for the asymmetric parameter $\alpha$ . The asymmetric preference parameter is varied across the values $\alpha _0 \in \{0.2,0.4,0.5,0.6,$ $0.8\}$ to encompass a spectrum of forecasting behaviors, from highly asymmetric preferences through values approaching symmetry to the fully symmetric case. Subsequently, we estimate different values of $\alpha$ based on the resultant one-step-ahead forecast errors using the P-GMM estimation approach, enabling us to assess the consistency and efficiency of $\alpha$ estimates across differing levels of asymmetry in user preferences.

Table 1.

Performance of P-GMM estimation (DGP 1)

Note : In 5000 simulation repetitions, “P(VR)” represents the probability that P-GMM estimation selects all valid and relevant moments; “P(VR+)” indicates the probability that P-GMM estimation selects all valid and relevant moments along with some irrelevant ones; and “P(INV)” denotes the probability that P-GMM estimation selects invalid moment conditions.

The performance of P-GMM estimation for DGP 1 with various values of the asymmetric parameter is presented in Table 1. For the strong identification scenario, we observe that for each value of $\alpha$ , the probability of selecting invalid moment conditions is consistently 0, indicating that P-GMM estimation effectively selects only the valid moment conditions. With a smaller sample size of $T=200$ , the probability of P-GMM estimation capturing all valid and relevant moments (specifically, $W_1$ and $W_2$ ) in the context of EKT (2005) is approximately 0.65, while the probability of selecting all valid and relevant moments plus a few irrelevant ones (from $W_{inv}$ ) is around 0.15. Consequently, in the EKT (2005) framework, P-GMM estimation identifies the complete set of valid and relevant moments with a probability close to 0.8. As the sample size increases to 2000, the probability of correctly identifying all valid and relevant moment conditions improves to approximately 0.95, and the likelihood of incorporating irrelevant moments remains close to 0. These results suggest strong finite sample performance of P-GMM estimation. In the context of EKT (2005), we further assess P-GMM estimation’s efficacy under weak identification conditions. For smaller sample sizes, the probability of selecting only valid and relevant moments is reduced, yet the probability of capturing all valid and relevant moments approaches 0.85. With a larger sample size of 2000, the probability of P-GMM estimation mistakenly including invalid moment conditions remains exactly 0.

Table 2.

Performance of P-GMM estimation (DGP 2)

The results for DGP 2 in Table 2 reveal similar performance trends in P-GMM estimation as seen in DGP 1, with slight differences in probabilities due to the nonlinear structure of DGP 2. In the strong identification case with a smaller sample size of $T=200$ , the probability of selecting all valid and relevant moments is slightly lower than in DGP 1, ranging from 0.617 to 0.685 across different values of $\alpha$ , reflecting the added complexity from nonlinear dependence. The probability of selecting all valid and relevant moments plus some irrelevant ones remains similar to DGP 1, around 0.17 to 0.19, while the probability of selecting invalid moments is zero, confirming P-GMM’s effectiveness in excluding invalid moments. When the sample size increases to $T=2000$ , the probability of selecting only valid and relevant moments increases to approximately 0.95, with a slight reduction in selecting additional irrelevant moments. In the weak identification case, P-GMM’s performance parallels DGP 1, though with marginally higher probabilities of selecting irrelevant moments due to the nonlinear effects. Overall, P-GMM estimation demonstrates reliable finite sample performance, particularly in excluding invalid moments while retaining all valid and relevant moments. The nonlinear structure slightly impacts precision in smaller samples, yet the estimator remains robust and effective as the sample size increases.

In Tables 3 and 4, we present the finite sample properties of the estimator $\alpha$ for DGP 1 and DGP 2, comparing five different scenarios: (a) P-GMM estimation; (b) estimation using all valid and relevant moment conditions; (c) estimation using all moment conditions; (d) estimation with a constant instrument; and (e) estimation with a constant instrument plus instrument $W_1$ . These varying instrument sets include both omitted and irrelevant information, providing a comprehensive comparison across scenarios with different levels of instrument relevance and validity.

All estimates are calculated based on 5000 simulation repetitions to ensure statistical reliability, with values averaged across these runs. For each repetition, we compute the $J(0.5)$ -statistic and $J$ -statistic based on (2.5) and (2.6). We also report empirical rejection frequencies of the $J(0.5)$ -statistic and $J$ -statistic at the 5% significance level across 5000 simulation repetitions. These frequencies reflect the proportion of simulations in which the null hypothesis is rejected, thereby serving as an informative functional summary of the $p$ -value distribution under the alternative hypothesis.

Table 3.

Estimation and testing results—DGP 1

Note : The first row related to $\alpha$ reports the average estimates over 5000 repetitions, with “se” representing the average standard error calculated from these 5000 estimates. “VR” signifies estimation using all valid and relevant moments, while “All” indicates estimation based on all moments. “CON” means estimation using only a constant as an instrument, and “CON+ $W_1$ ” refers to estimation using a constant plus the instrument $W_1$ . For each $\alpha$ value, the first $RF$ (rejection frequency) corresponds to the $J$ (0.5)-statistic and the second $RF$ corresponds to the $J$ -statistic, with both values averaged over 5000 repetitions.

Table 4.

Estimation and testing results—DGP 2

Note : The notations are consistent with those in Table 3, except that “CON+ $W^2_1$ ” refers to estimation using a constant instrument combined with $W^2_1$ .

Table 3 provides a detailed comparison of the finite sample properties of the P-GMM estimator, showcasing its consistency and reduced standard error relative to other estimation approaches, with the exception of scenario (b), which serves as the oracle estimator, representing an ideal and theoretically attainable model. The precision of the P-GMM estimator for the asymmetry parameter $\alpha$ improves substantially with an increasing sample size, reinforcing its robustness under larger samples. In scenario (c), where estimation includes all available moment conditions, the estimator exhibits bias, shifting away from the true parameter value due to the presence of invalid instruments. This bias remains as the sample size grows, indicating that simply adding observations cannot mitigate the influence of invalid moments. Such biases can inflate type I error, raising the probability of erroneously rejecting valid null hypotheses in hypothesis testing. Given these limitations, we exclude the $J$ -test statistics for scenario (c) in subsequent analyses of forecast rationality, as the inherent biases compromise the reliability of the test. This outcome illustrates the necessity of selective instrument inclusion when estimating and testing forecast rationality under flexible loss functions, as arbitrary instrument selection can introduce substantial biases. These findings therefore emphasize the critical role of rigorous moment selection within the EKT (2005) framework. The results for scenario (d), where only a constant instrument is included, show that the estimator remains consistent despite limited information, in line with theoretical expectations given the validity and relevance of a constant instrument. However, this approach lacks efficiency, as indicated by larger standard errors and reduced statistical power in comparison to scenarios (a) and (b). Notably, the P-GMM post-selection estimator in scenario (a) performs comparably to the oracle estimator in scenario (b). This alignment is theoretically anticipated, as P-GMM post-selection is designed to retain only the most informative and relevant moments, thereby improving estimator efficiency. This outcome highlights the effectiveness of P-GMM in refining estimator precision by prioritizing relevance and validity within the selected instrument set.

In evaluating the tests conducted after P-GMM moment selection, we find that the rejection frequencies from the rationality and symmetry tests outperform all scenarios except for the oracle case (b). Across all values of $\alpha$ , the rejection frequencies of the $J$ -test statistics using P-GMM estimation exceed those of other estimation approaches, apart from the oracle estimator. For instance, at $\alpha = 0.2$ , the P-GMM approach yields a rejection frequency of 0.8122 for the $J$ -test statistic in a sample of size $T=200$ , rising to 0.8544 as $T$ increases to 2000. This outcome demonstrates P-GMM’s effectiveness in enhancing the power of the rationality test by selectively incorporating only valid and relevant moments. Specifically, for testing loss symmetry at $\alpha = 0.5$ , the rejection frequency of the $J(0.5)$ -test under P-GMM is 0.7182 at $T=200$ and improves to 0.7946 with $T=2000$ , closely aligning with the oracle’s superior results. In contrast, scenarios such as estimation using a constant instrument yield lower rejection frequencies (0.7134 at $T=200$ and 0.7204 at $T=2000$ ), reflecting lower sensitivity and effectiveness in capturing symmetry deviations. This result underscores that P-GMM can bolster the power of the symmetry test by judiciously choosing moments that best align with the structure of the model. For other asymmetry parameter values ( $\alpha = 0.2$ , 0.4, 0.6, and 0.8), the rejection frequencies of the $J(0.5)$ -test statistics using P-GMM estimation are slightly higher than the oracle scenario but still outperform other cases. For example, at $\alpha = 0.8$ , the rejection frequency for the $J(0.5)$ -statistic is 0.0106 with $T=200$ and 0.0074 with $T=2000$ , again outperforming scenarios involving all moments or less selective instrument sets. This outcome suggests that by reducing the impact of uninformative instruments, P-GMM strengthens the test’s sensitivity to symmetry deviations, effectively capturing more precise estimates of the asymmetry parameter. Additionally, as the sample size increases, the power of the $J$ -test statistics shows further improvement, validating P-GMM’s capacity to achieve both theoretical rigor and practical efficacy in managing instrument selection. All of these highlight P-GMM’s potential to deliver the theoretical advantages and computational efficiency needed to optimally filter instrumental variables for robust forecast rationality estimation and testing.

Table 4 represents the finite sample properties of the asymmetry parameter $\alpha$ for DGP 2, where we observe trends and patterns similar to those noted for DGP 1. As shown, P-GMM estimation demonstrates consistently robust performance across all values of $\alpha$ , with estimates closely aligning to the true parameter values and lower standard errors than other scenarios. This finding highlights the effectiveness of P-GMM in selecting relevant instruments, even under nonlinear dependencies present in DGP 2. In the scenario utilizing only a constant as an instrument, the estimates of $\alpha$ show reasonable consistency, though the standard errors are relatively higher, reflecting a trade-off in precision due to limited instrument information. When the instrument set includes both the constant and $W^2_1$ , the precision of $\alpha$ estimates improves slightly, yet remains inferior to P-GMM estimation. For cases incorporating all available moment conditions, we observe higher bias and standard errors, particularly in smaller samples. This trend, consistent with DGP 1, underscores the detrimental effect of including invalid instruments, which induces bias that cannot be mitigated by sample size alone. Such biases also inflate type I error, as indicated by the lower rejection frequencies for the $J(0.5)$ - and $J$ -statistics in these cases. Thus, including all moments risks over-rejecting the null hypothesis of forecast rationality and symmetry. Analyzing the rejection frequencies from rationality and symmetry tests, P-GMM estimation yields the highest rejection frequencies across most $\alpha$ values compared to other estimation methods, except for the oracle scenario. This pattern continues across all tested values of $\alpha$ , indicating greater test power by focusing on the most informative instruments. The improved power in rationality and symmetry testing aligns with the theoretical advantages of P-GMM, which prioritize moment relevance and validity, thus enhancing estimator efficiency and hypothesis testing robustness. Finally, as the sample size increases to $T=2000$ , the performance of P-GMM estimation improves further, with standard errors decreasing and the power of the $J$ -tests increasing. This trend validates P-GMM’s capability to maintain theoretical rigor and practical efficacy in managing instrument selection, optimizing performance in both small and large samples. These findings affirm that P-GMM estimation not only delivers strong finite sample properties in terms of bias and precision but also optimally balances test power and efficiency, even within complex data structures as represented by DGP 2.

5. Evaluating forecast rationality with SPF data

The rational expectations hypothesis assumes that agents incorporate all available information when generating forecasts. Macroeconomic forecasts from established sources, particularly those by professional forecasters, play a critical role in shaping economic outlooks and decision-making processes (Carroll, Reference Carroll2003). This importance underscores the need to empirically evaluate the rationality of forecasts, particularly those from the SPF, given that professional forecasters’ incentives and expertise distinguish them in economic modeling. The SPF is particularly well-suited for rationality testing because its participants, unlike other economic agents such as households, are incentivized to reveal accurate beliefs and are typically more informed (Keane and Runkle, Reference Keane and Runkle1990). In this section, we employ the proposed P-GMM estimation within a forecast rationality context, using SPF data for empirical illustration. To streamline the GMM optimization, we utilize the identity matrix as a weighting matrix and limit the forecast horizon to a one-quarter horizon ( $h=1$ ). The analysis draws on data from the Federal Reserve Bank of Philadelphia’s SPF, the oldest quarterly survey of macroeconomic forecasts in the United States, which commenced in the fourth quarter of 1968 (denoted as 1968:Q4). Survey participants in the SPF provide point forecasts on various macroeconomic indicators, some of which have been included since the survey’s inception, while others were added beginning in the third quarter of 1981. There is a substantial body of literature on the analysis of data from this survey; see Elliott et al. (Reference Elliott, Komunjer and Timmermann2008), Ghysels and Wright (Reference Ghysels and Wright2009), Wang and Lee (Reference Wang and Lee2014), Rossi and Sekhposyan (Reference Rossi and Sekhposyan2016), Demetrescu et al. (Reference Demetrescu, Hanck and Kruse-Becher2022), among others. In our analysis, we assume a linear forecasting model, consistent with the SPF’s structured data collection. Additionally, we assume that forecasters operate under a quadratic loss function ( $p_0=2$ ), reflecting the common assumption of symmetry in forecast error preferences. This structure enables a rigorous examination of P-GMM’s application within the rationality framework, allowing for robust assessment of the relevance and validity of selected instruments.

We focus on business cycle forecasts and the following series are chosen consequently as the target variable $(Y_{t+1})$ : the quarterly growth rates for nominal GNP/GDP (1968 : Q4–2019 : Q4), the quarterly growth rates for the price index for GNP/GDP (1968 : Q4–2019 : Q4), the quarterly growth rates for consumption (1981 : Q3–2019 : Q4), and the quarterly growth rates for real residential fixed investment (1981 : Q3–2019 : Q4). All values are obtained from the most recently revised data vintage (2024 : Q3), which should represent past economic conditions as accurately as possible from the current perspective. We restrict our analysis to data until 2019 : Q4. This exclusion prevents distortions from the extreme fluctuations induced by the COVID-19 pandemic, which caused substantial disruptions to economic activity and introduced atypical variations in many macroeconomic indicators. Given that the SPF encompasses predictions from a broad array of professional forecasters, the forecasts inherently exhibit a degree of dispersion across individual predictions; see Capistran and Timmermann (Reference Capistran and Timmermann2009), Patton and Timmermann (Reference Patton and Timmermann2010), Wang and Lee (Reference Wang and Lee2014), and Carabotta and Claeys (Reference Carabotta and Claeys2024). In this paper, we adopt the mean response from each survey as the consensus forecast, thereby mitigating individual forecasting biases and providing a collective, representative forecast for each target variable. The growth rates are calculated as the difference between natural logarithms of subsequent values. The periods $t$ and $t+1$ refer, respectively, to the quarters in which forecasts are released and the subsequent quarters to which these forecasts apply.

Table 5 summarizes key descriptive statistics for each variable, illustrating notable differences in variance and distributional properties. For instance, the GNP/GDP growth rate displays the highest mean (1.5465) and median (1.4001), reflecting more stable growth compared to the other series. The investment growth rate exhibits the largest variance and kurtosis among the selected series, suggesting that it is inherently more volatile and susceptible to economic fluctuations than other variables. This finding aligns with existing literature on investment cycles, which are typically more sensitive to economic conditions. Before undertaking the estimation and testing of forecast rationality, we conduct a stationarity assessment using the unit root test developed by Elliott et al. (Reference Elliott, Rothenberg and Stock1996). Our results indicate that all growth rate series reject the null hypothesis of a unit root, confirming their stationarity. Forecast errors are subsequently calculated as the difference between actual values and one-step-ahead forecasts (Figure 4), where the forecast represents the average prediction across all SPF participants. Figure 4 visually captures forecast errors over time, highlighting significant deviations during the 2008 financial crisis. This period of heightened economic uncertainty and unexpected market dynamics led to substantial forecast errors, as forecasters struggled to account for rapidly changing conditions. However, in the quarters immediately following the crisis, forecast errors visibly decrease, suggesting a swift adaptation by forecasters who recalibrated their models to better accommodate the post-crisis economic landscape. Notably, the forecast errors for investment growth are especially volatile, which aligns with the high variance and kurtosis values observed in Table 5. This preliminary statistical and graphical examination underscores the utility of the SPF data for analyzing forecast rationality under the proposed P-GMM framework. By capturing both the dispersion among individual forecasts and the adjustment process following major economic events, our approach leverages the strengths of SPF data to rigorously evaluate forecast accuracy and the relevance of instrumental variables within a structured rationality testing framework.

Table 5.

SPF sample statistics

Figure 4.

SPF forecast errors for different variables.

In assessing the instruments used for estimation and forecast rationality testing, we construct a high-dimensional instrument set that includes a constant, absolute lagged forecast errors, three lags of forecast errors, lagged changes in the actual values of all macroeconomic variables (real GDP, price index, consumption, and investment), as well as lagged changes in forecasts themselves. To facilitate comparison, we present the results for three distinct scenarios: (i) using all available instruments, denoted as “All”; (ii) using only a constant and the lagged change in actual values, labeled as “CON + IV”; and (iii) implementing the proposed P-GMM estimation approach. The resulting estimates of the asymmetry parameter $\alpha$ , along with their standard errors and the corresponding rationality test outcomes, are summarized in Table 6, providing a comprehensive view of the impact of instrument selection on forecast rationality and symmetry testing.

Table 6.

Estimation and testing for symmetry and rationality

Note : In the table, “se” denotes standard error, “CON” indicates a constant instrument, “IV” represents lagged change in actual values, “ $t$ -Stat” is the $t$ -statistic used to assess whether the estimated asymmetry parameter $\alpha$ deviates significantly from 0.5, and “ $J$ -Stat” is the $J$ -statistic employed to evaluate forecast rationality. In the method column, “All” signifies estimation using all available moment conditions without selection, “CON + IV” refers to estimation based on a constant and the lagged instrument, and “P-GMM” denotes estimation using the proposed approach.

To determine whether the asymmetry parameter $\alpha$ significantly deviates from 0.5 and to quantify the uncertainty around this estimate, we conduct a $t$ -test for each forecasting scenario, enabling us to evaluate whether any observed deviations are statistically significant or merely reflect sampling variability. For GNP/GDP growth rate, the “All” and “CON + IV” methods produce estimates of $\alpha$ that are slightly below 0.5, suggesting greater sensitivity to under-predictions; however, these estimates are not significantly different from 0.5, indicating a roughly symmetric cost structure for forecast errors. The P-GMM estimation yields an $\alpha$ value marginally above 0.5, which also supports the hypothesis of symmetry in loss. This consistency across methods implies that for GNP/GDP growth rates, forecasters may assign similar costs to both over- and under-predictions, thus indicating a balanced approach in their cost functions. For price index growth rate forecasts, all three estimation methods yield $t$ -statistics that confirm symmetry in the loss function, reinforcing the notion that forecasters view errors on both sides equally costly. This outcome suggests that when predicting price levels, forecasters likely incorporate a symmetric consideration of costs associated with potential over- or under-estimations, reflecting a stable and balanced view on inflation-related measures. In the case of consumption growth rate forecasts, results reveal a markedly different pattern. Across all estimation methods, the asymmetry parameter $\alpha$ deviates considerably from 0.5, clearly indicating an asymmetric loss function that places greater weight on negative forecast errors. This asymmetry highlights a cautious approach in consumption forecasting, where forecasters may avoid overestimating growth due to the potential high economic costs of underestimating consumer demand, which could have adverse effects on business planning and economic stability. For investment growth rate forecasts, the P-GMM results reveal a marked difference from those based on the “All” and “CON + IV” estimations. While the latter two methods yield values of $\alpha$ close to 0.5, indicating symmetric preferences, the P-GMM approach suggests a distinctly asymmetric loss function, with $\alpha$ significantly below 0.5. This discrepancy highlights the potential for P-GMM to capture nuances in forecast behavior that may be obscured when all instruments are included without selection. In addition, the P-GMM estimate aligns with practical concerns in investment forecasting, i.e., overestimating growth could lead to costly excesses in inventory or capital, while underestimation might lead to missed investment opportunities. This disparity in costs likely causes forecasters to apply differential weights to positive and negative forecast errors, resulting in an asymmetric loss function that P-GMM is uniquely suited to reveal by optimizing the selection of relevant and valid instruments.

Furthermore, the reduced standard errors in P-GMM estimation for the considered series underscore the method’s efficiency in parameter estimation. The P-GMM approach benefits from its capacity to select moments that satisfy the conditions of validity and relevance (Definitions1 and 2), effectively reducing the asymptotic variance of the estimator. This technical advantage is consistent with the theoretical properties of P-GMM, where irrelevant instruments—those that do not meaningfully contribute to the identification of the parameter of interest—are systematically excluded, thereby diminishing the potential for overfitting and bias in finite samples. This selective mechanism leads to a more efficient estimation of the asymmetry parameter $\alpha$ , with lower standard errors enhancing the precision of $\alpha$ in capturing forecasters’ asymmetric loss preferences. By focusing on the most informative instruments, P-GMM not only improves statistical efficiency but also enhances the robustness of the rationality and symmetry tests. This robustness is particularly critical in the context of forecast rationality, as it ensures that the evaluation of forecasters’ preferences and biases is reliable across various economic contexts. The P-GMM method thus enables a more granular understanding of forecast behavior by delivering refined estimates that closely reflect forecasters’ economic incentives and risk sensitivities, offering a rigorous basis for testing the rationality hypothesis in professional economic forecasting.

Turning to the forecast rationality test using the $J$ -statistic, we observe consistent findings across P-GMM and the alternative methods (“All” and “CON + IV”) for GNP/GDP growth rate, price index growth rate, and investment growth rate forecasts. All methods suggest that these forecasts satisfy rational expectations at the 5% significance level, indicating that forecasters use the available information efficiently when predicting these economic indicators. However, a notable divergence occurs with the consumption growth rate. While the “All” and “CON + IV” methods do not reject rationality, the P-GMM $J$ -test does reject rationality at the 5% significance level. This discrepancy may stem from the presence of unselected instruments in the other two methods, which could introduce noise and reduce the test’s power. In the P-GMM estimation, by contrast, the $p$ -value associated with the $J$ -statistic is significantly lower, suggesting that the more targeted selection of valid and relevant moments enhances the sensitivity of the rationality test. This efficiency gain aligns with the theoretical expectation that P-GMM estimation improves model performance by excluding uninformative instruments.

Note that the empirical analysis suggests that SPF data remains valuable for consumption forecasts but underscores the importance of carefully selecting relevant instruments to enhance forecast rationality and precision. The P-GMM approach shows that traditional methods might either include irrelevant instruments or miss crucial information, potentially obscuring true biases in consumption forecasts. Specifically, for consumption, where forecast errors can carry substantial economic costs, the targeted selection of instruments helps capture any asymmetries in forecasters’ loss functions, leading to more reliable forecasts. Rather than discouraging the use of SPF data, the findings recommend refining model specifications to leverage the most informative instruments, which can ultimately improve both the accuracy of consumption forecasts and the robustness of rationality assessments.