I. INTRODUCTION: THE DAUBERT DOCTRINE
The Federal Rules of Evidence (FRE) govern the admission of evidence in US federal courts. When evidence consists of expert testimony, admission is governed by FRE 702—Testimony by Expert Witness. Unlike other witnesses, expert testimony is admissible without direct foundation in the evidence, i.e., without preliminary evidence (say, personal knowledge of the facts) showing the testimony’s authenticity and relevance. To justify this deference to expert knowledge, courts need a clear definition of who is an expert. FRE 702 states that expert testimony is admissible if given by a witness who is “qualified as an expert by knowledge, skill, experience, training, or education,” and if it is both “reliable” and “relevant” to the facts of the case.Footnote 1 The rule has been revised in 2000 to reflect the standard articulated by the Supreme Court in a series of decisions taken between 1993 and 1999, starting with Daubert v. Merrell Dow Pharm. Inc., 509 US 579 (1993).
Prior to Daubert, many American courts followed the general acceptance standard—also called the Frye standardFootnote 2—according to which expert testimony was considered relevant, and therefore admissible, if the expert’s ideas were “generally accepted” in the specific scientific field. Not all courts followed that standard, though, and the first 1975 edition of the Federal Rules of Evidence made no express reference to it. The Daubert Court settled the issue by establishing that the Frye standard had been superseded by the FRE; accordingly, it rejected the consensus or ipse dixit—and, therefore, deliberately conservative—notion of “general acceptability.”Footnote 3 To be admitted in court, the Supreme Court averred, an expert testimony must be “relevant” and “reliable.” In particular, expert testimony may be considered “reliable”Footnote 4 only if it is scientifically valid. As Justice Harry Blackmun, writing for the Court, put it: “the requirement that an expert’s testimony pertain to ‘scientific knowledge’ establishes a standard of evidentiary reliability” (Daubert, p. 590).
By equating “reliable” to “scientific,” the Daubert decision directed courts’ attention to whether expert testimony is, or is not, science. Since 1993, it is therefore up to judges to ensure not only that expert testimony is sufficiently tied to the facts of the case (relevance) but also that the methodology underlying the testimony is valid science (reliability). In legal jargon, Daubert and the ensuing FRE 702 have assigned courts the role of gatekeepers for the admission of scientific experts—a role they must perform by identifying what is relevant evidence and, above all, what is scientifically valid, and thus reliable, testimony.
While expressly avoiding to establish any “definitive check-list or test,” the Daubert opinion pointed at some key factors for judges to consider in their gatekeeping function. The four so-called Daubert criteria to be checked are: (1) whether the expert's methodology can or has been tested; (2) whether the methodology has been subject to peer review and publication; (3) whether the employed scientific technique has a known or potential error rate or whether it admits of standards controlling its operation; and, in partial recognition of the Frye standard, (4) whether the methodology has attracted widespread acceptance within the relevant scientific community (Daubert, pp. 593–594). These criteria, and the gatekeeping function itself, have triggered intense academic debate and, more importantly, changed the way expert testimony is proffered in courts.
With Daubert the Supreme Court wanted to make sure juries will be presented only with serious scientific evidence, rather than with so-called junk science (which could easily, though falsely, impress untrained jurors). The legal grounds for, as well as the practical advisability of, the new doctrine is thus beyond question: baseless testimony by unqualified witnesses should never be presented to a jury. Never forget that the judge’s gatekeeping role should be just to ensure that expert testimony has the proper methodological grounding and fit with the facts to be admissible; the determination of which of the (possibly multiple and contrasting) admitted expert testimony is correct—that is to say, the evaluation of the merit of the case—still belongs to the jury. As a matter of fact, however, Daubert has added an important weapon to litigation strategies. Raising a so-called Daubert challenge has become an oft-used option, with momentous consequences. A party may ask the court to assess the admissibility of the opposing party’s expert witnesses, with the goal of excluding them at a preliminary stage. If successful, the challenge may strike a decisive blow to a party’s chances, especially when the outcome of the case depends on technical or scientific matters.
The following pages focus on the impact of the Daubert decision on the admissibility of expert economic testimony in antitrust cases. The reason is that, in a word, the economists’ record vis-à-vis Daubert challenges has been poor. More often than in any other field of the law where they may be called to provide their expertise, economists’ testimonies in antitrust litigations have failed to satisfy the Daubert standard and thus have been rejected by courts as either irrelevant or unreliable, or both. Is there anything special about antitrust economics that makes it especially weak against the Daubert criteria of valid science?Footnote 5
While some literature already exists on this issue,Footnote 6 the paper innovates under three aspects. First, it relates the intense—if probably fruitless—methodological debate on the meaning of “valid science” under Daubert (section II) to the specific methodology adopted by economics in general (section III) and by so-called Post-Chicago antitrust economics in particular (section VII). Second, the paper provides new, updated numbers on the actual impact of Daubert challenges on the admissibility of expert economic testimony in antitrust cases (section IV). Third, it offers some tentative explanations for the special difficulty encountered by antitrust economists to meet those challenges (sections V–VI). As it turns out, these explanations stem from the peculiar historical trajectory followed by the application of economic ideas to antitrust law. The conclusion hints at what the intersection of that trajectory with the Daubert doctrine may actually entail, methodology-wise.
II. METHODOLOGY MATTERS—DOES IT?
It is because of the judges’ post-Daubert gatekeeping role that scientific methodology has gained attention in US legal debates of the last two decades: defining what is “valid science” has become a hot issue for legal scholars, both academically and jurisprudentially. The Supreme Court itself sparked this controversy, by way of the above-mentioned four Daubert criteria—each of which is of apparent methodological import.
Writing shortly after Daubert, Shana Solomon and Edward Hackett depicted the issue as a matter of border control: “How might judges determine whether certain scientific assertions are valid, and who should assist and guide their inquiry? To answer such questions judges must draw boundaries between science and non-science, settled science and unsettled science, valid science and dubious science. Even the standards of proof are different in law and science, giving rise to another boundary-setting debate” (Solomon and Hackett Reference Solomon and Hackett1996, pp. 148–149). While Frye had left scientific communities in control of those boundaries, Daubert partially shifted such control into judges’ hands. The irony was that, while “the decision recognizes that science may speak to issues of causation with many voices, not the unison voice of general acceptance,” when it comes to “the methods and behaviors that produce scientific knowledge, the decision assumes that there is a single valid scientific method that judges can inspect to determine the soundness of evidence or expert testimony” (p. 152).
Should, then, judges become, if not scientists themselves, at least expert in scientific methodology? Not at all, claims British philosopher and Daubert leading expert Susan Haack, according to whom the preoccupation with methodology triggered by that ruling has been “fruitless, and sometimes laughable” (Haack Reference Haack2015, p. 65). Such misplaced concerns hinged on the Daubert Court’s mistaken equation between reliability of expert testimony and scientific validity.
Daubert seemingly endorsed a standard conception of scientific method. The first, and most important, of the four criteria suggested by the Supreme Court—testability—envisioned science as proceeding by the generation of new hypotheses, the development of experiments that generate data with which to test them, and the validation of results through repetition. In the words of Justice Blackmun: “Ordinarily, a key question to be answered in determining whether a theory or technique is scientific knowledge that will assist the trier of fact will be whether it can be (and has been) tested” (Daubert, p. 593). Blackmun quoted in support two giants of twentieth-century philosophy of science: Karl Popper and Carl Hempel. The Court subscribed to Popper’s statement in Conjectures and Refutations, according to which “the criterion of the scientific status of a theory is its falsifiability, or refutability, or testability” (Daubert, p. 593, quoting from Popper Reference Popper1962, p. 37).Footnote 7 Under the two premises that, in order to be reliable, expert testimony must be grounded on “valid science” and that only falsifiable theories are valid science, it follows that expert testimony can be deemed reliable only if based on a falsifiable theory. Since Daubert, the falsification standard has thus become the judicially approved yardstick to identify “valid science.”
Both premises are questionable, Haack argues. On the very same page where he quoted Popper, Justice Blackmun also added a reference to Hempel’s Philosophy of Natural Science: “the statements constituting a scientific explanation must be capable of empirical test” (Hempel Reference Hempel1966, p. 49). Apparently, the Court mixed up Popper’s and Hempel’s approaches, under the false belief that being amenable to testing is equivalent to falsifiability.Footnote 8 The point is not merely philosophical, but—as noted above by Solomon and Hackett—eminently practical. Neither Popper's nor Hempel's views about scientific validity are “suitable to the task at hand” (Haack Reference Haack2005, p. 67): namely, establishing the reliability of expert testimony.
Theories that have been tested but not yet falsified are, for Popper, “corroborated,” the degree of corroboration depending on the number and severity of the tests passed. However, Haack notes, Popper’s corroboration is inappropriate to meet the Supreme Court’s concern with reliability.Footnote 9 The fact that a theory has been corroborated, to whatever high a degree, “does not show that it is true, or even probable; [it] does not mean it is rational to believe it” (ibid.). Indeed, Popper insisted that corroboration must not be confused with confirmation. The degree of corroboration of a theory is only an indicator of its past performance, and, therefore, in Popper’s own words, “it says nothing whatever about future performance, or about the ‘reliability’ of a theory” (Popper Reference Popper1972, p. 18; original emphasis). Even the best-tested theory—while being the rational pick when one must be chosen—is not reliable in the fullest sense of the word (ibid., p. 22). Hempel’s approach is not really helpful, too.Footnote 10 While aiming at a “logic of confirmation,” Hempel focused on the supportiveness of available evidence with respect to a given hypothesis, i.e., only on “relative confirmation.” This, by Hempel’s express acknowledgment, fell short of “absolute confirmation,” the true warrant of a scientific claim. Yet, to be able to discriminate reliable from unreliable expert testimony—as required by Daubert—a judge would need exactly that, i.e., an absolute notion of confirmation.
Behind Daubert’s methodological confusion lies a deeper misunderstanding, revealed by the first premise: the Supreme Court's equation of “reliability” with “valid science.” According to Haack (Reference Haack2005, p. 66), “there can be no criterion discriminating scientific, and hence reliable, testimony from the unscientific and unreliable; for not all, and not only, scientific evidence is reliable.” What really matters is not, as Justice Blackmun averred, whether expert testimony is scientifically valid but whether it provides the court with real knowledge, as different from mere opinion. This, by the way, is also how the Court itself came to view the issue just a few years after Daubert.
The Daubert decision was only partially unanimous. While all Justices undersigned the part of Blackmun’s opinion that superseded Frye and read the 1975 FRE as requiring judges to screen expert testimony not only for relevance but also for reliability, two Justices dissented from the other part—which established that to determine whether proffered testimony was valid science, and, therefore, reliable knowledge, courts had to look not at an expert’s conclusions but rather at her “methodology.” Two issues were controversial: first, the emphasis on methodology. As Chief Justice William Rehnquist put it in his dissent: “I do not doubt that Rule 702 confides to the judge some gatekeeping responsibility in deciding questions of the admissibility of proffered expert testimony. But I do not think it imposes on them either the obligation or the authority to become amateur scientists in order to perform that role” (Daubert, pp. 600–601). The second issue is whether Blackmun’s “general considerations” on relevance and reliability applied only to scientific knowledge or also to “technical or other specialized knowledge”—the other types of expert knowledge policed by FRE 702 (ibid., p. 600). This second objection is of special import here. The Daubert case was about the effects of a drug, so it entailed expert testimony in natural sciences—namely, those so-called hard sciences where the four criteria and the Popperian/Hempelian discourse, imperfect as it might be, find direct application. Hence, it was not just that, as Rehnquist lamented, Blackmun’s opinion said nothing about “technical or other specialized knowledge.” One could legitimately wonder whether Daubert also policed expert testimony in non-natural sciences, such as economics.
It took little time for both issues to be solved. Writing for the majority in General Electric v. Joiner (522 U.S. 136, 1997), Rehnquist largely deflated the importance of an expert’s methodology—and, with it, of much of the current and future epistemological controversies raised by Daubert. Joiner established the principle that, when there is “too great an analytical gap between the data and the opinions offered,” a court may legitimately reject an expert’s testimony on the basis of its conclusions rather than of its methodology. This is because, the Chief Justice added as a warning against judges’ temptation to indulge in methodological quibbles, “conclusions and methodology are not entirely distinct from one another” (Joiner, p. 146). Two years later, the Supreme Court stepped in again to finalize what has since then been dubbed the “Daubert trilogy.” In Kumho Tire Co. v. Carmichael (526 U.S. 137, 1999, p. 141)—a case involving the testimony of a car tires expert—it affirmed that the Daubert standard of relevance and reliability covers any kind of “scientific, technical or other specialized knowledge” (so FRE 702 now recites) proffered in expert testimony. What really matters, the Court explained, is the testimony’s relevance and reliability, not its being “scientific,” let alone “validly” so. Accordingly, Kumho Tire also clarified that the four Daubert criteria, which evoked a narrow notion of science, “neither necessarily nor exclusively appl[y] to all experts or in every case” (ibid.).
Alas, while sparing them the obligation to become amateur philosophers of science, the Daubert trilogy did not really help American judges to perform their gatekeeping duty. Since 1999, the most aware observers have recognized that both the notion of scientific validity and the methodology/conclusions dichotomy are, at most, distractions from the truly crucial issue: whether it is possible for a judge to identify “operationally effective indicia of reliability applicable to any and all of the host of potentially legally relevant fields of expertise” (Haack Reference Haack2015, p. 66). The answer is negative. As Haack puts it, “federal courts are left with large responsibility and broad discretion in screening expert testimony in all its nearly limitless variety, but little substantive guidance about how to do this” (ibid., p. 55). What can be drawn from the Daubert trilogy is just that, in order to assess the reliability of every kind of specialized knowledge or technique, judges “should use any, all, or none of the four Daubert factors, and/or such other factors as they deem appropriate. In short, that they should use their judgment, and do the right thing” (ibid.).Footnote 11 The rest of this paper shows how American judges have actually fared, while trying to do “the right thing” in the specific context of expert economic testimony in antitrust litigations.
III. THE DAUBERT CRITERIA AND ECONOMICS
It is in the above-mentioned vacuum of truly specific guidelines that judges are called upon to evaluate the relevance and reliability of expert economic testimony. Should experts in a social science like economics be held to the same standards as experts in the natural sciences or should the bar be set differently? For better or for worse, the four Daubert criteria frame reliability assessments when expert testimony in the hard sciences is involved. But what about “soft” sciences? The Kumho Tire Court averred that the criteria are neither necessary nor exhaustive but did not explain in what cases—i.e., for what typology of expert knowledge—they can be considered more or less binding. While Justice Blackmun’s naive conception of science, as epitomized by those very criteria, may somehow work for laboratory physics or chemistry, its applicability to social science in general, and to economics in particular, is more debatable.
On the one side, the extension sounds straightforward. Economists, too, claim to proceed by the development of new theories and the generation of testable hypotheses. On the other side, the kind of empirical testing of those very hypotheses that is typical of economics does not fully match Daubert’s first criterion. Save for the still circumscribed area of experimental economics, testing in economics means comparing a theory’s predictions against real-world observations, rather than against experimental data generated under controlled conditions.Footnote 12 So-called natural experiments may work nicely. Still, they are not what the Daubert Court required (just think of the vexed issues of replicability and external validity); ditto for applied techniques based on data mining and constructing a story from it. The point is, again, not only methodological but practical. In the highly competitive realm of court litigation, the difference between the falsification standard envisioned by the Supreme Court and the actual methods of applied economics may be large enough to warrant exclusion of an expert economist’s testimony.
The mismatch extends to the other Daubert criteria. As Peter Bronsteen and Asim Varma (Reference Bronsteen and Varma2001, p. 14) put it: “Some sound economic analyses may not be empirically testable or have a known or measurable error; some sound economic analyses may not be of sufficient interest to merit publication or peer review; and some sound economic analyses may be too esoteric to have attracted widespread acceptance within the relevant scientific community.” In all those circumstances, those very sound economic analyses would not be admitted in court under a strict reading of the Daubert criteria. The fact that testimony considered “professional” by academic economists may not be appropriate for litigation entails that the standard set by Daubert is higher than—or at least different from—that required for publication in professional journals.Footnote 13
A solution might be reducing the court’s scrutiny of expert economists to a mere formality. But while it is true that “[j]udges are not qualified to second-guess the procedures and methodologies employed by experts in technically demanding areas such as economics [or] statistics” (Hovenkamp Reference Hovenkamp2005, p. 80), shortcuts are not acceptable. Daubert does not allow judges to look at economic methodologies at “too high a level of generality,” satisfying themselves with vague remarks, such as “the expert relied on ‘statistics’ or ‘regression analysis’” or “statistical methodologies are generally accepted in the academic community, subjected to peer review, and have a known error rate.” Virtually any methodology would receive a green light, and the Daubert check would be too superficial (ibid., pp. 80–81). This is definitely not what the Daubert Court meant by assigning judges a gatekeeping task. Another solution would be to admit that the Daubert criteria, with their explicit, if confused, reference to natural science methodology, are not applicable to a social science like economics—to admit, in short, that economists are more like the car tire experts in Kumho Tire than the biologists in Daubert. Hence, an economist’s testimony should never be rejected under the falsifiability standard, for the simple reason that economics partakes more of “technical or specialized” knowledge, where such a standard does not apply, than of truly “scientific” knowledge. Alas, this way out would clash with the self-constructed image of “true scientists” that economists have strived for decades to build in the eyes of public opinion.
The troubled relation between Daubert and economics culminates in antitrust cases. This is somehow ironic, given that industrial organization (IO henceforth) is where real experiments in economics were actually bornFootnote 14 and that a specific subfield exists called “forensic IO,” allegedly supporting, both theoretically and empirically, all stages of competition law enforcement.Footnote 15 Yet, data show that neither tradition nor self-esteem may save antitrust economists from falling more often than most other experts under the Daubert ax. For reasons examined in the following pages, the economic argument underlying antitrust testimonies frequently contains features that make it especially weak against a Daubert challenge, especially when the challenge comes from the defendant. As we argue below, those challenges strike at the methodological core of the economic theories commonly used in support of antitrust complaints.
The gap between the Daubert criteria and the practice of antitrust economics has made a Daubert challenge a regular in American antitrust cases. The outcome has often been shocking. Several economists, including some big names of the discipline, and even a few Nobel prizewinners, have seen their expert testimony rejected—in full or in part—by federal courts, following their failure to meet the Daubert standard of relevance and reliability. More systematic analysis (see next section) reveals that antitrust economists testifying for the plaintiff have approximately a one-in-two chance that at least part of their opinion will be excluded—an abnormal percentage in comparison with other disciplines, including other areas within economics.
We already know that the bar is quite high, in that to satisfy the Daubert standard, “[e]xpert economic testimony must identify the specific models and methods of economics supporting each opinion expressed, the training and experience that qualifies the witness to apply those models and methods, the rationale for selecting those models and methods, and the economic logic that leads to the ultimate conclusions reached” (Werden Reference Werden2008, pp. 215–216). But what is it in antitrust cases that makes the bar rise further, often to unreachable levels? Why is antitrust economics so exposed to Daubert challenges? The brief answer is that, as remarked two decades ago by Andrew Gavil, Daubert and its jurisprudential progeny “arrived on the scene and crossed paths with antitrust at a critical juncture in its history” (Gavil Reference Gavil2000, p. 843), when the Chicago approach—which placed economic theory at center stage of antitrust law—had just received formal endorsement by the Supreme Court but, at the same time, was being replaced at the core of IO by game-theoretic analysis, which meant even more reliance in antitrust litigation on sophisticated economic testimony. It is this answer the following pages elaborate upon.
IV. THE TROUBLED RELATIONSHIP BETWEEN DAUBERT AND ANTITRUST ECONOMICS
The frequent use of expert economic opinion, fears of unreliable expert testimony, and lawyers’ strategic incentives have combined to make Daubert challenges a common feature of antitrust litigations.
The first element is obvious. Economics expertise is indispensable in modern antitrust enforcement. Since the late 1970s, progress in IO and, specifically, the Supreme Court’s endorsement of the Chicago approach have shifted antitrust analysis away from per se prohibitions and towards an effects-based rule-of-reason approach.Footnote 16 Courts are called upon to determine which business behaviors are anti-competitive and which are not. This assessment entails complex factual issues: What is the relevant market? Does the defendant possess market power? Are the defendant’s actions legitimate profit seeking or an anti-competitive exercise of such power? The plaintiff has first to establish a coherent theory of anti-competitive effects and then assemble sufficient evidence to support the theory and overcome the defendant’s efforts to defeat it. Hence, both parties make recourse to economic analysis and, consequently, to expert economic testimony.
When applied to economics, the Daubert doctrine seems straightforward. Expert economic testimony is admissible if and only if (1) the witness is an expert in relevant aspects of economics, (2) the testimony is well-grounded in those aspects of economics, and (3) the testimony applies the tools of economics to the facts of the case. In the general spirit of Daubert and FRE 702, these principles should allow courts to get rid of “junk economics.” Like other scientists, economists should thus welcome Daubert.
Rules with commendable goals may have unintended consequences, though. When applied to antitrust cases, Daubert’s impact has gone beyond its expected deterrent effect on irrelevant or unreliable testimony. It is indeed questionable whether its application has really served to protect antitrust juries from “junk economics.” On the one side, Daubert rulings have de facto replaced the jury with the judge as the finder of facts who weighs opposing expert evidence. On the other side, and precisely because of the key role expert economic testimony plays in antitrust litigations, Daubert challenges have become a customary defense strategy, whose success is, alas, independent of the merit of the case, i.e., the alleged antitrust violation,Footnote 17 and sometimes even of the proffered testimony’s intrinsic quality, as certified by the “general acceptance” that the given theory of the violation may have in top-notch IO literature.
Available data validate the latter claim. Although Daubert applies equally to plaintiffs’ and defendants’ experts, an analysis by James Langenfeld and Christopher Alexander of more than 400 Daubert motions lodged against economic experts (Langenfeld and Alexander Reference Langenfeld and Alexander2011; L&A hereafter) revealed that Daubert has created an additional barrier to plaintiffs’ successful assertion of antitrust claims.Footnote 18 A few regularities emerged.
First, economists appeared to be challenged more frequently in antitrust cases than in all other legal areas (e.g., intellectual property, product liability, or securities) where they are summoned as expert witnesses.Footnote 19 Second, while Daubert challenges of economic experts had a fair chance of succeeding in all legal areas, L&A’s data suggested that economists were much more likely to be excluded when testifying on antitrust matters. Third, confirming that Daubert motions had become a routine litigation tool for antitrust defendants, plaintiffs’ experts were more likely to be challenged than defendants’ experts: more than 80% of such motions attacked the former. Fourth, a greater chance existed in antitrust cases of excluding plaintiffs’ economic experts as compared with defendants’ experts. L&A showed that almost 50% of challenges against plaintiffs’ economic experts succeeded, in all or in part. Even notable economists could be excluded when acting as plaintiff’s witnesses, such as leading industrial economists Franklin Fisher and Richard Schmalensee, 2010 AEA President Robert Hall, and Nobel prizewinner Robert Lucas. In contrast, antitrust plaintiffs had been less effective in excluding economists supporting the defendant. Those numbers thus confirmed that, by 2011, Daubert had created a new, disproportionate burden on plaintiffs bringing antitrust claims.
Courtesy of Daubert Tracker, an online paid service providing litigants with the broadest database of Daubert challenges, divided by subjects, we can update L&A’s numbers.Footnote 20 The outcomes are the same—possibly worse. Between 2012 and 2018, 4,550 Daubert challenges have been raised, in US Federal Courts and all areas of the law, against expert testimony on the subjects of economics and finance. For comparison, only 2,109 such challenges have been raised against expert testimony on DNA analysis and forensic science in the same period. Overall, since 1993, economics and finance together rank fourth (behind only medicine, engineering, and psychology) among the areas of expertise most often challenged under Daubert.
As to antitrust law, 971 cases have been litigated in US Federal Courts between 2012 and 2018. Within them, an astounding 886 Daubert-related challenges have been raised against all areas of expertise. Of these, 584 challenges attacked economics expert witnesses. Even considering that almost always more than one expert is called to testify and that multiple Daubert challenges may be raised in a single case, the numbers are impressive. Again for comparison, consider that, within a universe of more than 3,000 cases for each area of the law, economists have been challenged only 254 times when providing testimony on securities and finance law and 371 times as experts on patent and copyright law. Under the safe assumption that parties in litigation are more likely to raise a Daubert challenge against testimonies whose relevance and reliability look a priori questionable, the picture emerging from these data is as dismal for expert economic witnesses in antitrust cases as that depicted by L&A in 2011.
Again in line with L&A’s remark about the higher frequency of challenges against economic experts for the plaintiff, roughly two-thirds of the 584 challenges raised in antitrust cases between 2012 and 2018 attacked plaintiff experts (391 out of 584). Of the remaining, 192 targeted experts for the defendant and 1 a (rare instance of) court-appointed expert. About half of those challenges were successful in either partially or fully excluding expert economic testimony from being presented to the court: 269 out of 584 (46%). Of these, 155 (40%) succeeded against plaintiff experts and 114 (59%) against defendant experts.Footnote 21
The list of big names in economics or, more specifically, IO struck down under Daubert has also lengthened since 2012. According to Daubert Tracker, expert testimony by the likes of Douglas Bernheim, Timothy Bresnahan, Dennis Carlton, James Heckman, Janusz Ordover, Daniel Rubinfield, Robert Willig, Orley Ashenfelter, Daniel McFadden, Carl Shapiro, and Joe Stiglitz has been either partially or totally excluded in an antitrust case, following a successful Daubert challenge. Courts have partially or totally dismissed those testimonies, as well as many others by scores of professional economists, as either irrelevant, or unreliable, or both. While leaving to future research the analysis of the specific reasons for those rebuttals,Footnote 22 the reality is that antitrust economic testimony is still especially likely to fall under the Daubert ax. Why?
Part of the evidence, like the greater propensity of antitrust defendants to challenge the admissibility of economic experts, has a straightforward explanation. Plaintiffs have the burden of proof in antitrust cases and, as we know, economics plays an essential part in it. “In order to win,” L&A (2011, p. 21) explained, “plaintiffs may need to wade into complicated economic waters to establish the necessary elements of their claims, such as relevant market, market power, and antitrust injury.” An antitrust complaint may thus founder even before the beginning of the trial if deprived of the expert economist’s testimony supporting it. The converse is not true for antitrust defendants, whose task is often just to demolish the plaintiff’s argument—a task that may in principle be carried on even without specific economic expertise. In short, the incentives for raising a Daubert challenge are much stronger for antitrust defendants than for plaintiffs.
Some other parts, however, are harder to explain. The commonest reason for exclusion of economic testimony is lack of reliability, which in turn stems in the courts’ daily practiceFootnote 23 from the expert’s inadequate methodology or the insufficiency of her data, given the methodology. As we know (section II), the Kumho Tire Court only qualified, but not dismissed, the four Daubert criteria and their implied characterization of “valid science.” While it is not legally compulsory anymore, the consensus among practitioners is that expert economic testimony should partake of such a characterization because, of course, economics is “valid science” (whatever this may mean). The latter belief is a founding premise of the image most economists—above all, applied ones like those providing testimony in antitrust litigations—have self-constructed of their own profession: “Forensic science is the application of scientific disciplines to the legal process. … Economics is a science that has methods and tools to offer for application in legal processes” (Schinkel Reference Schinkel2008, pp. 1–2). Alas, differently from other sciences, the methodological blueprint of antitrust economics shows a poor fit with the notion of “valid science” embodied by the Daubert criteria. Antitrust cases thus become a trap where even the best economic minds may fall.
If the Daubert characterization of “valid science” is (at least seemingly) clear and if economics is indeed “valid science,” why have successful challenges to economic experts been so frequent? Why didn’t well-trained (and often very well-paid) attorneys merely follow the Daubert criteria and properly instruct their expert economists to provide a testimony with unassailable “scientifically valid” content? Alternatively, why didn’t the threat of a successful Daubert challenge just deter irrelevant and/or unreliable economic testimony? As it turns out, there is something in the economists’ methodology in general, and in the antitrust economists’ methodology in particular, that exposes them to devastating attacks under the Daubert doctrine. Maybe it has to do with the different epistemological requirements of acceptable economic analysis and valid legal arguments. The rest of the paper elaborates on that.
V. THE DAUBERT SYNDROME: PRELIMINARY ANSWERS
The simplest answer is that economists are right and courts are wrong—in other words, courts fall short of the task assigned them by Daubert for want of economic knowledge. How can judges assess the validity of an expert’s methodology? Worse, how can they assess whether the expert’s opinion fits the facts of the case? Yet, this is what they are called to do when, as it often happens in antitrust cases, the appraisal of broad methodological principles blurs into the even more difficult evaluation of how the expert’s methodology handles the available evidence.Footnote 24 Moreover, no court should be called to “resolve disputes among respected, well credentialed scientists about matters squarely within their expertise, in areas where there is no scientific consensus.”Footnote 25 If asked to do so, especially in highly technical matters like antitrust complaints, courts will likely make bad calls and fail to properly grasp economic science, possibly dismissing valid expert testimonies by “well credentialed” economists.
This answer contains more than a grain of truth, yet the true meaning of Daubert and FRE 702 should not be misunderstood. No court is compelled to resolve scientific disputes among experts or replace the jury as fact finder. The four Daubert criteria aim precisely at avoiding such difficult calls by allowing all kinds of expert testimony that may be deemed scientific, to the sole exclusion of unscientific ones. As leading antitrust scholar Gregory Werden (Reference Werden2008, p. 817) underlines, the spirit and the letter of Daubert are that the deficiencies that should lead a court to exclude an expert testimony on, say, economic issues should be identifiable without any specific expertise in economics. This optimistic view must, however, be weighed vis-à-vis the reality of modern antitrust litigation, where complex economic (and statistical) analysis is the bread and butter of every expert testimony. It defies credulity to believe that such complexity may have no impact on judicial decision making. Indeed, antitrust decisions are often criticized because of the judges’ misunderstanding or misapplication of the relevant economics, their failure to recognize the critical economic issue, or their reliance on the opinions and analysis of the wrong expert.Footnote 26
The theme of the federal judiciary’s economic literacy has come under the spotlights in the last half-century, following the establishment in 1976 of Henry G. Manne’s summer program in economics for legal scholars—the so-called Pareto in the Pines program.Footnote 27 The program, which ended in 1999, consisted in two weeks of intensive training in basic economic notions. Its success was huge. According to some estimates, by 1990 around 40% of the US federal judiciary—including some would-be Supreme Court Justices and dozens of members of federal courts of appeals—had attended Manne’s program. While that figure is evidence of American judges’ thirst for economic instruction, an important issue, on which we will return below, concerns the kind of economics the program taught them.
Save for the indirect evidence provided by the success of Manne’s program, specific claims that the federal judiciary is not equipped to competently evaluate complex economic evidence in antitrust cases have been seldom subjected to empirical validation. The best such effort so farFootnote 28 has been undertaken some years ago by the Antitrust Section of the American Bar Association (ABA), which assembled a task force of leading economists and lawyers, plus a federal judge, to study the role of economic evidence in federal courts. The 2006 ABA report reached a consensus “regarding the importance of economics in modern antitrust law” and recognized that “it is critical that judges and juries understand economic issues and economic testimony in order to reach sound decisions.” It also admitted that lack of proper economic understanding “can seriously affect the adversarial process by skewing judicial outcomes, by leading decision makers to ignore conflicting economic testimony or come to ‘wrong’ conclusions, and can increase litigation costs” (Baker and Morse Reference Baker and Morse2006, p. 2). Indeed, less than a quarter of the antitrust economists interviewed by the task force believed that judges “usually” understand the economic issues in an antitrust case (ibid., app. II, p. 2).
An alternative, and somehow complementary, answer explains the poor record of antitrust economists against Daubert with the workings of the legal system. Ideally, antitrust complaints should be drafted by the plaintiff’s attorneys after consultation with an economist. What happens in reality, note John Solow and Daniel Fletcher (Reference Solow and Fletcher2006, p. 495), is that the expert economist comes into play only at a second stage, after the complaint has been presented. Hence, the economist is constrained to work within the theory offered in the complaint and with the data the plaintiff has been able to obtain through discovery. Upon taking on the case, the economist cannot turn to whatever market provides data from which to draw conclusions, as she would do in her academic activity, but has to work only with the evidence available in the case at hand. Life is easier for the defendant’s expert, who by legal rules has the advantage of responding to the claims made by the plaintiff’s economist, and so may limit himself to criticizing the opposing expert’s opinion.
This asymmetry imposed by legal rules mixes up with the confused epistemology underlying the Daubert doctrine to generate a tricky situation. It has been notedFootnote 29 that the Supreme Court’s reading of the crucial “Is it reliable?” question fits nicely with the requirements of legal analysis but much less so with those of economic theory. The original Daubert criteria raised an absolute question—Can the alleged scientific testimony be falsified?—which admits only a dichotomic, yes-or-no answer. The Court’s purely instrumental reading of falsificationism as a tool allowing scientists to discard theories was tailor-made for courtroom use, because it reduced the analysis of an expert testimony to a specific question, which abstracted from reality and considered explanation unnecessary. Indeed, this is what courts normally look for: straightforward decision-making tools, more than realistic explanations or elegant deductive chains.
What makes the mix of skewed procedural rules and muddled epistemology dangerous is then a simple matter of incentives. As early as 1999, in a short essay tellingly titled “Going for the Gold,” Michael Mandel complained about the undesirable effects of what he dubbed the economists’ “booming expert business.” “What we are seeing here,” he wrote, “is the legalization of much of the economics profession, the shift to an adversarial model from one seeking truth. Economists would be the first to agree that large-scale monetary incentives can change behavior … the importance of consulting income encourages economists who serve as expert witnesses to become locked into their opinions, thus impeding intellectual debate” (Mandel Reference Mandel1999, pp. 113–114). When serving as expert witness, the economist becomes part of a legal process, whose logic is “adversarial, rather than scientific,” and where “lawyers are calling the shots” (ibid., p. 117). The combination of hefty fees and lawyers’ dominance changes the nature of the expert’s economics, whose aim becomes to win the case: only legal success matters, not the best economic understanding.Footnote 30 But in an adversarial game the rules are not those of academic publication. While it may still be true that cross-examination is “the greatest legal engine ever invented for the discovery of truth” (Wigmore Reference Wigmore1904, p. 1697), the yes-or-no question implicit in any Daubert challenge, when combined with the economist’s urge to make the case for the party paying her and with the de jure comparative weakness of the plaintiff’s position, may expose a testimony’s methodological frailties to the point of making it unacceptable by the Daubert standard.
One last immediate reason for the high rate of succesful Daubert challenges in antitrust jurisprudence calls into play the specificity of antitrust cases. No economist—not even a Nobel Laureate—is qualified to testify about every economic issue that may arise in antitrust litigation. It is not just that even the best economic minds may not have enough experience on the particular issues and specific analytical methods of a given case. IO specialists are obviously qualified to testify on most issues arising in antitrust cases; still, IO expertise is neither necessary nor sufficient to save an economist from a Daubert challenge. Even a highly respected IO economist might be unqualified to testify on basic issues in a specific case.
As we know, FRE 702 requires that expert testimony be “based upon sufficient facts or data.” In antitrust the proviso is often interpreted as requiring that the testimony be grounded in the facts of the given industry relevant for the given case. This interpretation led the trial court in re Brand Name Prescription Drugs Antitrust Litig. (1999-1 Trade Cas. [CCH] 72, 446, N.D. Ill.), a conspiracy case, to exclude most of the testimony for the plaintiff of Nobelist Robert Lucas. According to the court, Lucas had inadequate knowledge of the specific nature of the pharmaceutical industry. “Dr. Lucas’ opinions about the pharmaceutical drug industry failed every test for admissibility,” the decision read. “[A] review of his testimony shows quite clearly how vacuous the Class Plaintiffs’ evidence of conspiracy really is” (p. 128). Generally speaking, an economist with no research record in the particular industry, or whose testimony incorporates few case-, or industry-, specific facts but rather offers a “one size fits all” account, is more likely to suffer rejection under Daubert.
This conclusion is shared by the best economic knowledge. “The application of reasonableness standards in antitrust decision-making and the modern empirical literature in industrial organization economics have an important commonality,” write leading IO scholars Jonathan Baker and Timothy Bresnahan (Reference Baker, Bresnahan and Buccirossi2008, p. 27). “[B]oth treat the industry as the appropriate unit of observation. Both fields recognize that a one-size-fits-all approach to analyzing business conduct won’t do, since so much variation in outcomes arises from factors specific to each individual industry.” Again, an important asymmetry between plaintiff and defendant is also at play here: “little or no factual foundation may be required for economic testimony that merely critiques the analysis of another economic expert, because errors or gaps in economic logic may be clear without any knowledge of the particular facts of a case” (Werden Reference Werden2008, p. 810). How many of the casualties of a Daubert challenge were economists providing a generic, non-industry-based testimony on behalf of the plaintiff?
VI. WHY IS ANTITRUST ECONOMICS ESPECIALLY WEAK AGAINST DAUBERT?
The previous section offered a few reasons for the unusual frequency and success rate of Daubert challenges in antitrust cases. The judges’ insufficient familiarity with economic methods, the procedural peculiarities of the legal system, and the less-than-encyclopedic factual experience of even the smartest economists combine to produce a not-so-favorable environment for economic experts. Still, these explanations seem hardly specific to antitrust and could apply to economic testimony in other areas of the law, such as, say, product liability. Our data on post-Daubert challenges suggest otherwise. Something else seems to affect economic expertise when summoned and scrutinized in antitrust cases. This section and the next examine some specific features that justify the higher exposure to Daubert-based attacks of economic testimony for antitrust plaintiffs.
Antitrust cases are always theory-driven: the economic theory endorsed either explicity or implicitly by the court governs the legality of the business practice at issue. This is because antitrust laws, such as the Sherman and Clayton Acts, have no self-evident meaning. What these statutes mean requires the intermediation of a theoretical construct that interprets their general language in terms of the specific situation.Footnote 31 Any allegation of anti-competitive behavior must therefore rely upon a properly specified economic theory, as well as on a set of data supporting it. In other words, stand-alone data are of very little use to a court. The well-known distinction between correlation and causation reminds us that no empirical relationship between economic magnitudes can be interpreted without theoretical guidance.Footnote 32
Presenting the court with the theory upholding an antitrust allegation is the expert economist’s task. In the post-Daubert world, the court is then asked to check the relevance and reliability of that economic argument. A provisional conclusion may thus be drawn. The sheer number and high success rate of Daubert challenges against economic experts for the plaintiff are evidence of the questionable status of the economics underlying so many antitrust allegations—questionable, that is to say, to the legally trained eyes of the defendant’s attorneys and the court.
The problem is indeed familiar to antitrust economists. Like most work in empirical social science, antitrust analysis makes inferences from evidence without the benefit of performing fully fledged experiments.Footnote 33 A distinctive issue thus arises, called the “identification problem.” Economists must “look carefully for settings in which nature has created an experiment for them” and must “explain why it is reasonable to interpret the data as having been created by an implicit experiment, and describe the nature of that experiment” (Baker and Bresnahan Reference Baker, Bresnahan and Buccirossi2008, p. 13). This explanation—called “identification”—is a crucial part of empirical economic analysis, in that it should clarify on which basis a given theory can be preferred to another for interpreting the available evidence. The identification problem has obvious relevance in antitrust, when juries—in their role of finder of facts—must select between alternative interpretations of industry behavior. Yet, the problem may also affect litigation at a preliminary stage, that of a Daubert challenge raised against the specific theory identified by the expert for the plaintiff to support the allegation.
It is at this junction that a major difference emerges between economics and most other “scientifically valid” disciplines. As Werden admits, “economics has no well-established standards governing the selection and application of particular models and methods” (Werden Reference Werden2008, p. 815), which amounts to saying that a Daubert challenge necessarily brings an antitrust court to transcend the four criteria of “valid science.” This is where the identification problem sets in, directing attention on the correspondence between the evidence and the competing economic theories, and thus on whether the evidence can be used to distinguish between competing theories of the case.
The point is not so much that economic experts present conflicting models of the same antitrust issue. These analyses may all rest on solid methodological grounds and, as we know, Daubert neither requires nor empowers courts to determine which of multiple competing scientific theories is the best, A divergence may also exist about the facts themselves, contrasting versions of which may be offered by the parties. Again, FRE 702 is not intended to authorize the exclusion of an expert’s testimony on the ground that the judge believes one version of the facts or another. The true point is how to verify the reliability of the economic (and often also statistical) theory supporting the interpretation of the facts. It is, in short, a matter of model calibration—or lack thereof.
“Calibration” is the term used in economic jargon when an economic model is made to fit the specific quantitative facts of a given market or industry. For example, it is customary in modern antitrust to apply basic oligopoly models (like Cournot or Bertrand) to predict the effects of a merger. Yet, no such model is of any use if it is not calibrated on the basis of parameters such as pre-merger market shares and demand elasticities. Hence, no testimony based on an economic model would be admissible if the expert fails to properly calibrate her model.
A noteworthy application of this principle in an antitrust case came in Concord Boat Corp v Brunswick (207 F.3d 1039, 8th Cir., 2000), in which the Eighth Circuit excluded expert testimony by Stanford economist Robert Hall.Footnote 34 Plaintiffs were a group of boat manufacturers who alleged that defendant Brunswick, whose market share was almost 80%, had monopolized the North American market for inboard and stern-drive marine engines through a variety of anti-competitive pricing schemes and acquisitions. Faced with the problem of constructing a counter-factual version of the market without the anti-competitive conduct (the so-called but-for world), Hall had employed a standard approach in IO, building his argument on a basic Cournot model that he took as a proper representation of the interaction between Brunswick and its rivals. Hall had argued that, absent Brunswick’s anti-competitive actions, the market would be a symmetric Cournot duopoly, consisting of two firms, each with a 50% market share. At the equilibrium of that but-for market, the price of engines would be determined by marginal cost plus a markup that depended on demand elasticity.
Defendants challenged Hall’s testimony under Daubert, complaining that his economic model “was unreliable and did not have the capacity to distinguish between lawful and unlawful market behavior.” In their view, Hall was merely assuming that, absent anti-competitive behavior, the but-for market would result in equal shares, which implied that every extra point of market share that Brunswick actually possessed was evidence of anti-competitive behavior. IO economists often assume that markets are symmetric Cournot duopolies for theoretical purposes, but of course there is no a priori justification for taking that structure to hold for any real-world industry. In this specific case, though, Hall’s thesis that the but-for market would be a roughly symmetric Cournot duopoly was not merely hypothetical, as he had evidenced structural features (largely homogeneous products and a market so small that it could accommodate only a few producers of minimum efficient scale) that pointed to that outcome.
The district court declined to exclude the challenged testimony, which thus became the primary source for the jury to find for the plaintiff and award antitrust damages. Defendants challenged again the testimony on appeal before the Eighth Circuit. In spite of the deference normally accorded to trial courts on Daubert-related questions, the Court of Appeals agreed with defendants and reversed. According to the Eighth Circuit, the testimony had to be excluded because Hall had failed to adjust his model to account for the acknowledged facts of the marine engines industry. The court recalled that, under Daubert, an expert’s economic model “should not be admitted if it does not apply to the specific facts of the case,” and found that the oligopoly model used by the plaintiffs’ expert was “not grounded in the economic reality” of the industry (Concord Boat, p. 1056). Among various deficiencies, the appellate judges underlined that Hall’s model attributed to the challenged conduct all sales made by Brunswick in excess of half of the total—against the evidence that the defendant already had a 75% market share even before undertaking the challenged conduct (ibid.). The conclusion was drastic: “Dr. Hall's opinion should not have been admitted because it did not incorporate all aspects of the economic reality of the stern drive engine market and because it did not separate lawful from unlawful conduct. Because of the deficiencies in the foundation of the opinion, the expert’s resulting conclusions were mere speculation” (ibid., p. 1057).
The episode is telling of the difficulties even the best economic expertise may encounter when facing a Daubert challenge. The first lesson to be drawn is that proposing an economic model that may be “scientifically valid” only under unique circumstances is not sufficient to overcome the challenge.Footnote 35 As the Eighth Circuit remarked, a model used to make antitrust predictions must fit the facts of the industry to which it is applied. “The critical test of a model,” Werden (Reference Werden2008, p. 812) writes, “is whether it explains reasonably well those aspects of past industry performance the model is being used to predict.” This is where Hall’s model failed, because—at least in the appellate court’s evaluation—it just assumed, rather than demonstrated, the unlawful origin of Brunswick’s higher market share.
Concord Boat brings to the fore another key issue in Daubert challenges: causation. Legal rules require that plaintiffs meet the burden of showing that a given event (usually, a defendant’s behavior) caused the alleged injury. Hence, the question “Did the challenged behavior cause the anti-competitive effect?” is at the core of every antitrust litigation. To survive a Daubert challenge, the plaintiff’s economic model cannot simply answer that “it may have caused it.” It is not enough that the expert economist shows that a certain behavior may give rise to a competitive concern, not even if a high degree of probability is appended to that “may.”Footnote 36 What the model is asked to do in order to overcome the challenge is to also control for alternative explanations—i.e., different causes—of the same effect. A price hike may, for example, be due to background inflation or exogenous shifts in demand, rather than anti-competitive behavior. Specifically, defendants in Concord Boat lamented that Hall’s model did not mention the circumstance that Brunswick’s chief competitor had been forced to recall its engines because of a technical defect—an event that had automatically boosted Brunswick’s market share.
Models that cannot account for alternative explanations do not really help the court and therefore —in the spirit of FRE 702—must be rejected, together with the expert proposing them. Indeed, Daubert jurisprudence is replete with economic testimonies that have been excluded as either irrelevant or unreliable because the suggested models failed to account for important causal factors (“obvious and significant alternative explanations,” as a New York district court called them) and thus were unable to isolate the impact of the cause of interest.Footnote 37 Parsimony of variables and hypotheses, while preserving a good predictive power, is usually a plus in models. However, as Herbert Hovenkamp (Reference Hovenkamp2005, p. 83) underlines, frugal modeling may be bad law. And the law has the final word, because Daubert and FRE 702 assign to the (usually untrained, economically and statistically speaking) judge the task of deciding whether the model’s neglect of some market circumstances undermines the expert economist’s testimony.
VII. GAME-THEORETIC ANTITRUST FACES DAUBERT’S THREAT
Concord Boat also provides a valuable lesson about the use of game theory in post-Daubert antitrust litigations. Since the late 1980s, Bayesian game theory has replaced price theory as the analytical backbone of IO and, consequently, antitrust economics. Much like Chicago-style antitrust, also the so-called Post-Chicago approach prioritizes efficiency but distinguishes itself for an ample use of game-theoretic reasoning.Footnote 38
The investigation of coordinated interaction (viz., collusion, tacit or explicit) was an obvious beneficiary of the game-theoretic revolution. Still, the most significant improvements came in the area of so-called unilateral effects. Strategic reasoning has greatly expanded, well beyond the traditional dominant firm theory, the antitrust enforcers’ insight into those structural features and business practices that may allow a single firm to negatively affect competition. However, antitrust economic experts have soon realized that an important characteristic of game theory, usually listed among its greatest analytical advantages, is also a major practical drawback when the analysis is exported from classroom to courtroom. Inferences from game-theoretical models are always highly assumption-sensitive: even minor changes in the underlying assumptions may generate wholly different results. Because of this peculiarity, the use of game theory in antitrust testimony is particularly exposed to Daubert challenges.
To better understand the problem, let us consider the distinction between general and possibility theories.Footnote 39 In any science, general theories address broad relationships covering a wide range of facts; they give rise to generic hypotheses that can be tested and possibly falsified independently of the specific factual conditions. These theories are, in a sense, “scientifically valid” (see section II) by construction, in that they are falsifiable and capable of generating meaningful predictions. On the contrary, possibility theories merely show that a particular result may and will occur when some specific assumptions are exactly met. These theories may thus be considered “valid science” only when they formally include a testable hypothesis linked to the occurrence of the facts specific to the theory itself. Once these theories have survived falsification, they can also be used to generate predictions.
In a 1989 paper MIT economist Franklin Fisher applied this distinction to distinguish between generalizing and exemplifying theories in economics. The former proceed from wide assumptions to inevitable consequences and speak in terms of what must happen, given the background circumstances, the best example being Chicago-style price theory. The latter focus on determining what can happen and are highly sensitive to the assumptions used. According to Fisher, oligopoly theory has always belonged to the exemplifying category: a collection of “a large number of stories, each one an anecdote describing what might happen in some particular situation” (Fisher Reference Fisher1989, p. 118). The rise of game theory, he added, had only made things worse, because the only generalizing result had been a negative one, the so-called Folk Theorem, which “tells us that we cannot hope for a general oligopoly theory based only on cost and demand functions and free of the context in which oligopolists operate” (ibid.). Three decades later, Fisher’s paper is still crucial to grasp the difficulties of expert economic witnesses facing a Daubert challenge. To gain hearing in court, economists’ reasoning must at the same time be generalizing enough to hold true in a broad range of concrete situations—like, indeed, most price-theoretic propositions do—and sufficiently operational to grant practical application in the case at hand. The latter requirement relates to the above-mentioned exigency of courts to make yes-or-no decisions (see above, section V). The former deserves further scrutiny.
As a matter of fact, the game-theoretic approach to antitrust is a poor fit with both requirements. Post-Chicago models are so dependent on their underlying assumptions that they are almost impossible to test (Coate and Fischer Reference Coate and Fischer2001, pp. 823–824). And even admitting that they make for “reliable science” (viz., that they are falsifiable), it is even more debatable that they can be considered “relevant” in the Daubert sense, i.e., that they may generate fact-based outcomes (ibid., p. 828). Accordingly, courts have often applied Daubert by rejecting, as in Concord Boat, testimonies grounded on game-theoretic claims when no further evidence was provided relating the challenged conduct to actual market facts or when those facts were indistinguishable from the results of pro-competitive behavior.
It may be argued that the poor record of the Post-Chicago approach in American courts is due to external (a.k.a. ideological) factors. A popular explanation calls into play the specific kind of economics taught in the above-mentioned summer program for judges organized by Henry Manne (see section V). A founder of modern law and economics, Manne was among the most active popularizers of the Chicago approach to the discipline. While his summer program featured as instructors economists of all political inclinations, and while Manne himself took special care on guaranteeing that it consisted of an allegedly neutral training in price theory, with applications to “everything except legal issues” (Butler Reference Butler1999, p. 352), the fact itself of prioritizing basic price theory—the epitome of Chicago-style microeconomics—rather than, say, game theory, may well have had momentous consequences for the subsequent application of antitrust law in general, and of the Daubert doctrine in particular, by American judges.Footnote 40
While this explanation may contain more than a grain of truth,Footnote 41 the true difficulty of the Post-Chicago approach vis-à-vis Daubert is intrinsic to the way game-theoretic models are built. These models’ empirical value is usually nil because their solutions can be determined only when the underlying assumptions are exactly met. In other words, the array of outcomes that in a game-theoretic model, depending on the model’s specific assumptions, may possibly arise as equilibria make the model itself, and any expert testimony based on it, inevitably ad hoc. While this is a boon for academic investigation, because it grants modelers freedom and flexibility, it is an indictment for courtroom applicability (Giocoli Reference Giocoli2013, p. 326). Thus, Post-Chicago models, and the antitrust complaints based on them, can hardly achieve the status of “relevant and reliable science” that could guarantee a pass under Daubert.
A game-theoretic model that points to a possible anti-competitive outcome of a given business behavior is by itself insufficient to survive a Daubert challenge. No rigorous theorizing demonstrating the possible anti-competitive rationale behind some unilateral strategies may alter the legal response to specific instances of these practices. On the one side, the virtual, stylized facts upon which Post-Chicago models are founded cannot be verified in the courtroom; on the other side, the real, often opaque facts in the trial record are of no use in assessing the models themselves (Coate and Fischer Reference Coate and Fischer2001, p. 832). The verdict, as Hall learned in Concord Boat (where, remember, the kind of game-theoretic analysis he used was as basic as Cournot duopoly), seems inevitable: Daubert would require antitrust courts to reject as “mere speculation” any expert testimony based only on this kind of reasoning. Claiming that something may theoretically happen, without offering a concrete, facts-based analysis of how it is to happen, is unscientific under Daubert and cannot be admitted in court. “The mathematical elegance of the model’s deductive logic does not address [the Daubert] problem,” write Malcolm Coate and Jeffrey Fischer (Reference Coate and Fischer2012, p. 128): “Without a test, the expert has only an opinion, possibly a complicated opinion, or even a publishable opinion, but still just an opinion.”Footnote 42
So dire a picture can be lightened a little bit, though. Once again, a reference to the methodology of science may help.Footnote 43 Possibility theories perform a key scientific function. Many real-world processes are too complex to be represented with generalizing models. A proper scientific response is then to generate possibility theories addressing special instances of these processes. It may well be argued that the kind of competition going on in real-world oligopolistic markets is one such complex process and that, for this reason, economists must make recourse to possibility models to predict behavior in any given oligopolistic market. With adequate parameterization, even game-theoretic models are therefore capable of accounting for real-world competitive interactions.
This observation provides a rationale for the antitrust use of Post-Chicago exemplifying theories, though seldom a sufficient one. Whenever one such theory is proposed in the courtroom, it is essential to conduct a case-specific check before its actual application. The problem is still the same: possibility theories, standing alone, are useless for an antitrust court, because they cannot exclude other possible accounts of market evidence that would reach a different conclusion.Footnote 44 While it is relatively easy for economists to parameterize a game-theoretic model, parameterization is not enough. Without evidence that the model has been put to a fact-based test (in Popperian terms, without an attempt at falsification), “the expert has only a parameterized set of equations” (Coate and Fischer Reference Coate and Fischer2012, p. 128). It does not even suffice that the model’s explicit assumptions be valid for the market under scrutiny, because every model also contains many implicit assumptions—usually, to justify the exclusion of other factors. If these implicit assumptions are wrong (like, say, when a firm’s rising market share depends on factors different from that firm’s behavior: think again of Concord Boat), the expert’s theory may well be deemed irrelevant or unreliable.
The bottom line is therefore that, while generalizing theories may abstract from a wide range of considerations and still generate valid predictions, exemplifying theories need not just be parameterized, but adequately parameterized, i.e., their parameterization must be robust against case-specific factual evidence. The bar for Post-Chicago theories ends up being just too high—surely higher than that for Chicago-style generalizing arguments. Having acknowledged that the expert testimony does not rely on a generalizing theory but merely presents a possibility account, the Daubert judge must verify whether the expert has parameterized her exemplifying model correctly (in the above-specified sense): this is the necessary condition for the testimony itself to be deemed relevant and reliable. Alas, the latter scrutiny is almost impossible to overcome, given how game-theoretic models are built in order to be determinate. It will not be difficult for the defendant’s expert to point at some case-specific evidence the plaintiff’s expert model did not—indeed, could not—account for. If so, the latter’s testimony would be doomed under Daubert.
VIII. CONCLUSION: A SAFE METHODOLOGICAL HAVEN?
No question exists as to which theoretical approach has won the battle of ideas in modern IO. Since the late 1980s, the application of Bayesian game theory has revolutionized the discipline, replacing price theory as the theoretical workhorse from which most descriptive and prescriptive implications are drawn for being subjected to empirical validation. As we said, in the realm of antitrust economics, the modern game-theoretic approach is dubbed Post-Chicago. The name has been chosen not only for chronological reasons but above all to emphasize that many of the antitrust claims typical of the Chicago school—claims that almost invariably affirm the lawfulness of many allegedly anti-competitive business behaviors—have been proved wrong by the new approach. Indeed, worse than wrong: inefficient. Several business behaviors that received a green light under Chicago price theory are actually anti-competitive when read through the Post-Chicago lenses of Bayesian game theory.
While triumphant in the marketplace of ideas, the Post-Chicago approach has struggled so far in antitrust courts. Only in very few cases have US courts affirmed antitrust complaints whose economics relied on game-theoretic arguments. It is as if from the viewpoint of the economic analysis underlying their antitrust jurisprudence, US courts are still stuck to the 1980s and the pro-market, pro-business claims of the Chicago School. Several reasons may explain this surprising outcome. The Daubert doctrine is one of them.
Among the most remarkable unplanned consequences of Daubert is that it has helped antitrust defendants to dismiss complaints based on Post-Chicago analysis. Expert testimony relying on modern IO in general, and on game-theoretic reasoning in particular, is especially likely to succumb under Daubert challenges. This is due to intrinsic reasons peculiar to the Post-Chicago methodology itself—reasons, that is to say, from which every game-theoretic argument is going to suffer, even when satisfying latu sensu the four criteria of scientific reliability. These reasons, this paper has argued, depend on the different needs and standards of legal procedures with respect to economists’ practices. When applied to economic testimony, the Daubert doctrine has allowed smart defendants to win antitrust battles in the courtroom that they would probably lose if fought on pure academic terms in scientific meetings or economic journals.
If we think of the alleged rationale of the Daubert doctrine itself—to help courts separate true science from junk science, while at the same time allowing new scientific ideas to enter courtrooms without being pre-empted by ipse dixit views and established methods—the outcome is shocking. How to reconcile that rationale with the reality of the application of Daubert to antitrust litigations? One might provocatively conclude that for American judges, Post-Chicago antitrust economics—which is tantamount to saying most of contemporary IO—is itself junk science, or ipse dixit science, or, in short, invalid science.
While so drastic an inference would be too far-fetched vis-à-vis the widespread courtroom application of modern economics, a (slightly) less provocative conclusion stems from reflecting on the methodological message of game theory’s weakness against Daubert challenges. Modern economic methodologists and historians of economics offer several possible accounts of the success of a given approach or technique that go well beyond the falsification story. Falsificationism, they argue, is too naive an explanation of how science in general, and economic science in particular, proceeds. The history of economics shows that only rarely has a theory’s empirical content been the reason for its success, or lack thereof. Other features, such as formal elegance, political usefulness, business patronage, or the internal dynamics of the economists’ profession have often been more decisive. Methodologists then conclude that, because of the complicated and contingent mix of factors affecting a theory’s success, it is impossible to claim that a given approach or school is “superior” to another. Surely no evidence of superiority stems from the fact that a theory has been published in a leading journal or is taught by star economists. The notion itself of “superiority” is indeed meaningless to modern methodologists’ eyes.
But not, we can argue, to the eyes of an antitrust court performing a Daubert scrutiny. By endorsing an—admittedly outdated and defective—notion of science, Daubert has created a sort of reservoir, a protected environment where economic theories can be evaluated according to their pure empirical content. No external factors should enter the reasoning of a Daubert judge assessing the scientific validity of an economic testimony. Devoid of all its political or sociological paraphernalia, modern IO—the approach that has won the battle of ideas and that is taught in top academic departments—is naked in front of those sitting at the bench, empowered to decide whether it is valid science or not. Whatever the decision, it is made by someone who, while herself influenced by multiple factors,Footnote 45 is free of the economists’ biases and concerns; someone who has the professional duty to apply the impartial, if debatable, yardstick of Popperian/Hempelian science. Like football teams playing in neutral ground, the Daubert match between opposing interpretations of a given antitrust claim will not be affected by home court advantage. The chosen neutral ground may not provide the greenest pitch, but at least it is supposed to be a level one.
