A. Sequence of Intercourt Interaction

Suppose a two-stage interaction sequence between a referral and a referring court, as in figure 1. In stage 1, the referring court must decide whether to submit a high-quality reference, a low-quality reference, or none at all. If it submits a case, the referral court either accepts or rejects the reference. The referring court observes the referral court’s decision and accrues the corresponding payoff. In stage 2, the referring court must decide again whether to submit a high-quality reference, a low-quality reference, or none at all. In this informal framework, we further suppose that the referring court has incomplete information over the referral court’s acceptance criteria. Inexperienced judges may even ignore the possibility that a submission might be rejected.Footnote ⁷ After observing the referral court’s decision in stage 1, the referring court revises its prior beliefs about the referral court’s true docket policy. As we shall see, the assumption of incomplete information is important here to understand when and why chilling and learning might occur.

Figure 1.

Sequence of interaction between referring and referral court over two stages

B. Case Selection Assumptions

We assume that the referral court has a preference for high-quality submissions. This reflects the intuition that referral courts treat the choice of docket policy as an optimization problem given the overall caseload and aggregate response of referring courts. Similar to a high-ranking academic journal, the referral court can afford to be more demanding when submitters are numerous and, thanks to the journal’s prestige, unlikely to be put off by a rejection. The analogy with the process by which academic journals select papers for publication further suggests that to the degree that strategizing occurs, it is likely to be restricted to prominent authors. In a legal context like the EU legal order, this implies that strategic behavior is more plausible when domestic apex courts—the legal equivalent of star academics—are involved. Our empirical design controls for this possibility by conditioning on peak courts or via the inclusion of court fixed effects.

C. Referring Court’s Preferences

Whenever the referring court submits a reference and the referral court accepts it, the referring court accrues the benefit of cooperation. We assume that the referring court always prefers an acceptance resulting in a ruling over a rejection yielding no ruling. In the real world, referral court rulings may occasionally fail to meet the referring courts’ expectations to the point that they would have preferred no ruling at all. However, what matters for the decision to submit is the expected gain from a ruling, not the actual gain.

What benefit the referring court expects from cooperation determines the intensity of its team motive. We do not seek to investigate the specific nature of the gain referring judges may expect from cooperation. Instead, we restrict ourselves to evaluating its intensity. We note that team models of adjudication are compatible with various understandings of the nature of this motive (Kornhauser Reference Kornhauser1994). Referrals address the law-creation and law-finding dimensions of adjudication, and referring judges may draw satisfaction from securing a legally sound outcome. Alternatively, though, they may seek one closer to their policy preferences or one fairer to litigants than the status quo. In other words, the benefit of cooperation can be understood in terms of the legal model, the attitudinal model, or richer, hybrid variants of these theories of judicial decision making (Stephenson Reference Stephenson2009; Epstein, Landes, and Posner Reference Epstein2013; Ash and MacLeod Reference Ash2015).

Preparing, writing, and submitting a reference takes time and intellectual effort. Writing a high-quality reference, though, requires more work than writing a low-quality one. A high-quality reference requires a referring judge to research the case law and to write her reference carefully in accordance with referral court’s rules of procedure.Footnote ⁸ As we shall see, both learning and learning effects presuppose that referring judges attach some nontrivial costs to writing high-quality references. This is why, ignoring or underestimating the possibility of rejection, they initially decide to submit low-quality work.

Another consideration that may weigh on the referring court’s behavior is reputation. Because referral procedures are transparent and their outcome is public, a social stigma may be attached to the referring courts that have experienced a procedural dismissal.Footnote ⁹ Depending on its perceived magnitude, this reputational risk may affect the referring court’s decision-making calculus both in stage 1 and in stage 2 when contemplating a new submission.

D. Chilling and Learning

Of central interest to the present study is the referring court’s response to the referral court’s decision in stage 2. Assuming that the referral court always accepts high-quality references and accepts at least some low-quality ones, it is easy to see that if the referring court submitted a high-quality reference and this was accepted, its beliefs about the referral court’s docket policy should remain unchanged. After observing a rejection, by contrast, the referring court should update its beliefs, leading it to revise the utility it expects from referring a low-quality reference. In so doing, belief revision can thus induce a change of behavior in stage 2. Chilling occurs when the referring court, after referring a low-quality reference in stage 1, observes a rejection and then refrains from referring in stage 2 because the costs associated with writing a high-quality reference outweigh the benefits of cooperation. A chilling effect also presupposes that the higher expected risk of experiencing another rejection when submitting another low-quality reference will discourage this option, either because the expected benefit of cooperation has been revised down or the reputational risk has been revised up.

Learning occurs when the referring court submits a low-quality reference in stage 1 but decides, upon observing a rejection, to submit a high-quality reference in stage 2. Learning presupposes that the expected benefit of cooperation compensates the effort cost attached to writing a high-quality reference. It also presupposes that the expected payoff of submitting a high-quality reference is larger than going for another low-quality reference in stage 2.

As mentioned previously, a rejection reveals shirking on the part of the referring court. What explains the difference between learning and chilling is that the learning judge sees the benefit of interjudicial cooperation as large enough to justify extra work whereas the chilled judge does not. With regard to learning, though, it is important to take into account the time elapsed between stage 1 and stage 2. Indeed, if a long period of time elapses between the exposure to negative feedback in stage 1 and resubmission in stage 2, the referring court’s knowledge may become obsolete, either because the referral court has readjusted its docket policy in reaction to the evolution of submission numbers or because of skills atrophy and receding institutional memory—for example, as a consequence of a change in judicial personnel. As with academic journals, a paper that would have been recognized as a high-quality submission 10 years ago may no longer meet the evolving standards of editors and reviewers. So, to the degree that learning occurs, we should expect time to reduce its impact.

In sum, theory suggests that, depending on the relative strength of the referring court’s team motive, negative feedback may induce either a chilling or a learning effect. In the following sections we test the chilling and learning hypotheses using data on referral activity and formal dismissals in the EU legal system.

A. Empirical Test

We conduct three tests to assess the existence of a chilling effect. Our first test operationalizes the dependent variable as the number of preliminary references submitted by court i of member state j in year t.Footnote ¹⁰ If their team motive is weak, we expect that the courts that have experienced a procedural dismissal in year $t-1$ will be less likely to submit a reference in year t than if their reference had been accepted.

For this first test, our primary independent variable is a lag ( $t-1$ ) measuring the number of dismissals experienced in the previous year. Here the set of observations ( $N=\mathrm{22,831}$ ) includes all courts that have submitted at least once.

Second, our theoretical framework implies that, if judges were exposed to negative feedback in year $t-1$ , the choice between referring and not referring in the following period will depend, in part, on the postfeedback reassessment of the reputational risk. Thus, as a second test for the chilling effect, we operationalize our independent variable as an ordinal measure taking the value 4 if the reference is dismissed as manifestly inadmissible, 3 when dismissed on the ground that the question has already been settled, 2 when dismissed for lack of jurisdiction, 1 when rejected for other reasons, and 0 if ended by a preliminary ruling. This ordering reflects the degree of stigma legal scholars associate with each of these outcomes (Kornezov Reference Kornezov2015). Consistent with our theoretical framework, we hypothesize that if chilling is caused by reputational concerns, courts that have experienced a more negative evaluation will be more reluctant to resubmit than courts that have experienced a less negative one.

Finally, our third test restricts the analysis to first submitters. Because first submitters, by definition, do not have a history of interactions with the ECJ, they have less information about the ECJ’s docket policy. To the extent that inexperienced submitters are initially less confident about the ECJ’s true docket policy, far-reaching belief revision after experiencing a dismissal may make them more prone to chilling.

B. Covariates

We control for covariates of referral activity considered in previous studies: familiarity with EU law (Hornuf and Voigt Reference Hornuf2015; Lampach and Dyevre Reference Lampach2020), monism (Carrubba and Murrah Reference Carrubba2005), support for EU membership (Carrubba and Murrah Reference Carrubba2005), intra-EU trade (Stone Sweet and Brunell Reference Stone Sweet1998; Carrubba and Murrah Reference Carrubba2005), GDP (Vink et al. Reference Vink2009), and enlargement (Wind, Martinsen, and Rotger Reference Wind2009; Lampach and Dyevre Reference Lampach2020). We also control for peak court. Compared to courts lower down the domestic judicial hierarchy, peak-court judges typically enjoy greater resources to process a comparatively smaller caseload. Moreover, as previously mentioned, last-instance courts do not only have a duty to raise a preliminary reference when EU law is material to the dispute at hand (Dyevre, Glavina, and Atanasova Reference Dyevre2020), but the ECJ is also more likely to behave strategically when deciding whether to accept a reference submitted by such a court. Our dummy for peak court takes the value 1 if the court is identified as a peak court and 0 otherwise. Information on the definition, operationalization, and data sources for our controls is provided in the appendix.

C. Modeling Resubmission Behavior

Our modeling exercise presents several challenges. First, court-level resubmission rates, our dependent variable for the chilling hypothesis, can only take nonnegative integer values. Second, because lower courts are very sporadic in their use of the preliminary ruling mechanism and many never have a second go at the Article 267 procedure after their first submission, our dependent dependent variable includes many zeros. Its distribution is best approximated by a zero-inflated negative binomial distribution. Third, while one may be tempted to apply a frequentist, fixed-effects approach, this estimation strategy runs into intractable convergence problems.

To address these issues, we estimate the number of references Y submitted by court i in country j in year t conditional on lagged ECJ dismissal decision $d_{\mathrm{ijt}-1}$ and covariates x _ijt using Bayesian multilevel modeling. Multilevel models offer less protection against omitted-variable bias. Yet because they discard less variation and are able to detect thresholds effects via the random intercepts, they often have greater power than fixed-effects models. In that sense, if our multilevel model does not detect a chilling effect, we can be more confident that this effect is not present. The model can be expressed as

(1) $\begin{array}{}\text{(1)}& p(Y_{\mathrm{ijt}}=y_{\mathrm{ijt}}|d_{\mathrm{ijt}-1},x_{\mathrm{ijt}},{\beta }_t,{\psi }_t)=\{\begin{array}{cc}{p}_{\mathrm{ijt}}+(1-p_{\mathrm{ijt}}){\left(\frac{{\psi }_t}{{\mu }_{\mathrm{ijt}}{\psi }_t}\right)}^{{\psi }_t}& \text{ if }{y}_{\mathrm{ijt}}=0;\\ (1-p_{\mathrm{ijt}})\frac{\Gamma \left(y_{\mathrm{ijt}+{\psi }_t}\right)}{y_{\mathrm{ijt}}!\Gamma \left({\psi }_j\right)}{\left[\frac{{\mu }_{\mathrm{ijt}}}{{\mu }_{\mathrm{ijt}+{\psi }_t}}\right]}^{y_{\mathrm{ijt}}}{\left[\frac{{\psi }_t}{{\mu }_{\mathrm{ijt}}+{\psi }_t}\right]}^{{\psi }_t}& \text{ if }{y}_{\mathrm{ijt}}=1,2,\dots \end{array}\text{,}\end{array}$

where ψ⁻¹ is a dispersion parameter with $\psi >0$ , Γ(·) is the gamma function, $E\left(Y_{\mathrm{ijt}}\right|x_{ijt},{\beta }_t,{\psi }_t)={\mu }_{\mathrm{ijt}}$ , $\mathrm{Var}\left(Y_{\mathrm{ijt}}\right|x_{\mathrm{ijt}},{\beta }_t,{\psi }_t)={\mu }_{\mathrm{ijt}}(1+{\mu }_{\mathrm{ijt}}/{\psi }_t)$ , and $0\le p_{ijt}\le 1$ . Note that the distribution has two components: one for $y_{\mathrm{ijt}}=0$ and one for $y_{\mathrm{ijt}}=1$ , 2, … . The log-likelihood function for the vector of parameters $\theta ={(\psi ,{\beta }^T,z^T)}^T$ has the form

(2) $\begin{array}{}\text{(2)}& l\left(\theta \right)=\sum _{i:y_{\mathrm{ijt}}=0}{l}_1({\psi }_t,d_{\mathrm{ijt}-1}^T\beta ,x_{\mathrm{ijt}}^T\beta ,z_{\mathrm{ijt}}^T\gamma )+\sum _{i:y_{\mathrm{ijt}}>0}{l}_2({\psi }_t,d_{\mathrm{ijt}-1}^T\beta ,x_{\mathrm{ijt}}^T\beta ,z_{\mathrm{ijt}}^T\gamma )\text{.}\end{array}$

As with the distribution, l(θ) has two components. We estimate equation (2) via Markov Chain Monte Carlo (MCMC) to obtain the posterior inference π(θ) for the parameters β, γ, and ψ. We put conservative priors—which assign lower probability to implausibly large values—on our model parameters.Footnote ¹¹ Accordingly, we assume that

${\beta }_{\mathrm{lk}}=\underset{1\times k}{\underbrace{G_{\mathrm{kl}}}}\underset{k\times 1}{\underbrace{{\eta }_k}}+{\alpha }_{\mathrm{ijk}}$

and

${\gamma }_{\mathrm{lq}}=\underset{1\times q}{\underbrace{S_{\mathrm{ql}}}}\underset{q\times 1}{\underbrace{{\eta }_q}}+{\alpha }_{\mathrm{ijq}}$

where we specify multivariate normal priors on η _k and η _q , and α _ijk as well as α _ijq follow half-student prior ( ${\alpha }_{\mathrm{ijk}}=t(\nu ,0,s)$ , ${\alpha }_{ijq}=t(\nu ,0,s)$ ) with $\nu =3$ and $s=10$ . The dispersion parameter ψ follows a gamma distribution satisfying $\psi ~\mathrm{Gamma}(a,b)$ , where $a=0.01$ and $b=0.01$ ; the zero-inflation probability of referrals follows a beta prior with shape parameters $\alpha =1$ and $\beta =1$ .

A. Testing Approach

Our analytical framework entails that if the team motive is strong and writing costs are neither trivial nor too high, the referring court will respond to the negative feedback by submitting a high-quality reference. Establishing a learning effect necessitates an evaluation of the quality of the referring court’s submission. Instead of developing an external measure of quality, we assume that the ECJ’s decision at the acceptance stage reflects the reference’s underlying quality. In other words, we assume that the ECJ is more likely to reject a poor-quality reference regardless of the domestic court’s referral history.Footnote ¹²

Consistent with our theoretical framework, if negative feedback induces a learning effect, we expect courts that have experienced a procedural dismissal to improve the quality of their references, making it less likely that they will experience a procedural dismissal at the next attempt. Accordingly, we define our dependent variable as a dichotomous measure taking the value 1 if the reference submitted in year t is disposed of by an order and 0 if disposed by a judgment. For this test, we consider the impact of an ECJ admissibility decision over all years following feedback rather than just the year after the ECJ decision. This is because we assume that the effect of negative feedback may last beyond the year following exposure (Schwartz Reference Schwartz2000; Huang, Dominici, and Bell Reference Huang2005; Gasparrini Reference Gasparrini2011). However, because knowledge and skills tend to atrophy when unused and institutional memory is likely to weaken over time, we expect the exposure effect to be larger when the exposure is more recent and weaker when the exposure is more distant in time. The universe of observations for this test is all the courts that have submitted at least two preliminary references ( $N=\mathrm{1,824}$ ).

As a robustness test, we restrict the analysis to those procedural dismissals that solely state formal shortcomings as reasons for rejecting the submission, that is, those held manifestly inadmissible ( $N=\mathrm{1,768}$ ). Two considerations motivate this additional test. First, because they refer to the substance rather than to the form of the submission, the two other grounds invoked by the ECJ to reject cases, “settled case law” and “no jurisdiction,” may in some cases be interpreted as signaling the type of questions ECJ judges would like to see domestic courts referring. When so, procedural dismissals may serve a docket construction strategy which may potentially confound the learning mechanism our analysis aims to uncover. Second, some courts—particularly in jurisdictions where the authority of precedents is less established—may deliberately submit questions known to be considered settled case law simply to secure an authoritative statement that the case law applies to the case at hand.

B. Covariates

We control for four factors that we believe may affect the ECJ’s decision to dismiss a reference. The first is the hierarchical position of the referring court. For reasons pointed to above, a higher resource/caseload ratio implies that courts higher up in the judicial hierarchy have a greater capacity to produce high-quality references, which may make them less prone to formal dismissals. Our second control is familiarity with EU law, for which length of membership in the EU is a proxy (Lampach and Dyevre Reference Lampach2020). Judges in member states that have been in the EU for a longer period are more likely to have received EU law training, to have presided over disputes involving EU law arguments, and to know the ECJ’s case law. The third factor is the ECJ’s own workload. A larger caseload may induce the ECJ to apply its docket policy with more severity. We measure caseload as the number of pending cases aggregating all procedures through which cases can be brought before the ECJ (Articles 267, 263, 259, and 258 of the TFEU).Footnote ¹³

C. Modeling the Delayed Effect of Negative Feedback on Odds of Acceptance

We assess the learning hypothesis using distributed lag modeling. Distributed lag models (DLMs) are designed to capture delayed effects (Schwartz Reference Schwartz2000; Huang et al. Reference Huang2005). The linear form of our basic DLM can be expressed as

$\begin{array}{}\text{(3)}& \log \left(\frac{{\pi }_{\mathrm{ijt}}}{1-{\pi }_{\mathrm{ijt}}}\right)={\beta }_0+\sum _{s=0}^q{\gamma }_s{x}_{\mathrm{ijt}-s}+\sum _{r=1}^R{\nu }_r{\alpha }_{\mathrm{rijt}}\text{,}\end{array}$

where γ _s is the lagged weight. Lagged weights collectively define the lag distribution. Here the lagged weights specify that the exposure linearly affects the outcome over time. The term α _rijt includes other predictors with linear effects defined by the related parameters ν _r .

As explained above, owing to knowledge and skills atrophy or receding institutional memory, we expect the exposure effect to be larger when the exposure is more recent and weaker when the exposure is more distant in time. The literature on DLMs has proposed ways to relax the assumption of linearity (Roberts and Martin Reference Roberts2007; Gasparrini Reference Gasparrini2011). Delayed effects can be modeled by a polynomial distributed lag restricting the lagged coefficients to fit a cubic function. Relaxing the linearity assumption, equation (3) can be rewritten as

$\begin{array}{}\text{(4)}& \log \left(\frac{{\pi }_{\mathrm{ijt}}}{1-{\pi }_{\mathrm{ijt}}}\right)={\beta }_0+\sum _{s=0}^q({\kappa }_0+{\kappa }_{1}s+{\kappa }_2{s}^2+{\kappa }_3{s}^3)\left(x_{\mathrm{ijt}-s}\right)+\sum _{r=1}^R{\nu }_r{\alpha }_{\mathrm{rijt}}\text{,}\end{array}$

where κ₀, κ₁, κ₂, and κ₃ are the parameters of the cubic function describing the lagged weights. The number of parameters is chosen on the basis of a visual inspection of the data (see figs. 5 and 6).

Figure 5.

Proportion of rejected submissions by years elapsed since experiencing last negative feedback, EU-15 (EU member states that joined before Eastern enlargement). Plot shows the fraction of dismissed references as a function of the year in which the last formal dismissal was experienced. Courts in member states that joined after 1995 are excluded. Smoothed trend line.

Figure 6.

Proportion of rejected submissions and date of previous rejection, EU-28 (all EU member states, including the UK and countries that joined following Eastern enlargement). Plot shows the proportion of dismissed references as a function of the year in which the last formal dismissal was experienced. Smoothed trend line.

In addition to conditional logit models with court fixed effects, we also report the results of multilevel variants of these models in the appendix. As Bayesian DLM is still in its infancy and technical hurdles remain,Footnote ¹⁴ we estimate the parameters of these models via frequentist EM-ML (expectation maximization maximum likelihood).