In Left Turn, Tim Groseclose makes a compelling case that journalists should be as scrupulous about disclosing their ideological biases as they are in disclosing other conflicts of interest. Although I am not a journalist, I feel compelled to oblige. First, the conflict of interest: This author has collaborated on a published journal article with Groseclose. I have been fortunate to have many terrific co-authors but the experience with Groseclose stands out for being intellectually rewarding and a lot of fun. Groseclose does have all the ideological biases that he readily admits, but none of those are as strong as his bias in favor of rigorous social science research. Now, the ideological biases: On taking the Groseclose's PQ examination I scored a 50. This was mainly because my modal answer was “it depends.” Of course, this doesn't make me unbiased. Indeed, I am a very biased moderate. I hope that these revelations help the reader to evaluate my comments on Left Turn.

Left Turn is but one of the latest examples of social science books written for a popular audience. Some of the works in this genre are apolitical. But given the nature of the questions social scientists ask, many are explicitly political with a point of view. What contributions to the genre have in common is a combination of rigorous social science research but a lot of speculation when actual research is sparse.Footnote ¹

The core of Left Turn is based on Groseclose's work with Jeff Milyo.Footnote ² In a now-influential paper, Groseclose and Milyo compared the distribution of citations to think tanks and other policy organizations made by the media to those made by members of Congress. Their key finding was that most media organizations have citation patterns that look much more like those of Democratic legislators than those of Republican legislators. Indeed, only two media outlets had citation patterns that were reflective of legislators that were more conservative than the median, “Fox News Special Report” and the Washington Times. Alternatively, CBS News and the New York Times had citations close to the average Democrat.

Since publication, the study has received a large amount of criticism, both academic and blogospheric. Some have leveled criticisms about methodology and data choice. The most systematic of these critiques is that of John Gasper.Footnote ³ Gasper argues that Groseclose and Milyo's results are not robust in that the estimated media slants change over time, moving in a conservative direction. Moreover, he argues that the exclusion of the National Taxpayers Union (NTU) significantly reduces the liberal bias. These are important caveats, but ultimately not damning. The matrix of citations to think tanks by legislators and media outlets is sparse. So it takes a large amount of data over a long period of time to estimate the effects. Rolling averages such as those estimated by Gasper are undoubtedly subject to much error. As for the NTU, it is a clear ideological bellwether so citations to it ought to be a very good indicator of slant. I'd be more worried if the results were not robust to the inclusion of the National Association to Promote Space Tourism.

While Groseclose and Milyo have yet to formally respond to Gasper, Left Turn contains a fair amount of material corroborating the results of the original study. First, Groseclose notes that Matthew Gentzkow and Jesse Shapiro's analysis of the correspondence between congressional speech and newspaper coverage produces similar results. Second, Groseclose reports on an innovative new analysis of the coverage of George W. Bush's tax cut. Here he analyzes the propensity of media outlets to report certain objective facts about the distributional consequences of the Bush tax cut. He finds that outlets which he and Milyo estimated as liberal were much more likely to report “liberal” facts than those they estimated as conservative.

So while I remain fairly confident in the main results of the Groseclose and Milyo study, I do share some of the concerns that have been raised about their interpretation. Why should we consider variation in the citation patterns of the news media a form of bias? In the social sciences and statistics, bias is usually taken to imply some deviation from an objective truth. Without that objective benchmark, how can one label some outlets as more or less biased? Groseclose spends much time explaining that the issue is simply one of semantics, that he does not mean bias in that sense. Rather, he means it in the narrower sense that some media behave like liberals, some like conservatives, and some like moderates. Those that behave as moderates are less biased because they are closer to the behavior of the typical legislator. This seems fine as a definition, but it is not what we usually mean by bias. So if it is simply semantics, he should have used a different word.

While Groseclose has often been criticized for over-interpreting the extent of bias or slant in the original results, I am inclined to believe that his interpretation may understate the ideological variation across media outlets. Groseclose assumes that the media and legislators have an identical decision-making process over citations. So when the New York Times uses the same citation pattern as Joe Lieberman, Groseclose infers that the Times has the same ADA score as Lieberman. But in reality, Lieberman and the Times have somewhat different objectives. When Lieberman gives a speech, he is trying to persuade as part of an adversarial deliberation. He is under no professional or other obligation to present both sides of the issue. If he leaves out some fact from a Heritage Foundation report, it's the job of his opponents to point it out. The media operates under very different norms. They are supposed to be unbiased, balanced, and present both sides. So if the Times left out the Heritage fact, it could be embarrassing and the public editor might even have to write an apology. So all things being equal, the Times will tend to cite the other side. If the media do succumb to these professional norms, Groseclose and Milyo's procedure will estimate them as much more moderate than they actually are. Of course, one could object to this reasoning by pointing out that all that matters is that the Times reported the Heritage fact. That they did include it means that the coverage was less biased than the Lieberman speech. But if the Heritage report was mentioned in a perfunctory manner or a counter argument was presented, there may be no difference in the impact on a reader.

As I mentioned earlier, the pop social science genre often contains a fair amount of speculation and extrapolation beyond rigorous social science research, and Left Turn is no exception. This is especially true of Groseclose's argument about the large effect of media bias on elections. His analysis is based on the estimate of quantity that he labels the media lambda or λ. Formally, the media lambda is defined as the solution to

where V is the voter's views after consuming media, M is the media slant, and N is the voter's “natural” views. Thus, λ is the weight that the voter places on the media's coverage.

If one rearranges these terms, we can see that λ = (V − N)/(M − N). While the Groseclose-Milyo study provides the estimates of M, estimates of V and N are required to estimate the lambda. Because N is an inherently counterfactual quantity (what would the voter believe if she had not been exposed), Groseclose rightly insists on using only experimental or quasi-experimental estimates for purposes of calibration. He identifies three such studies:

• • Stefano DellaVigna and Ethan Kaplan's study on the effects of the introduction of Fox News on presidential voting behavior.Footnote ⁴

• • Alan S. Gerber, Dean Karlan and Daniel Bergan's study of the effect of randomized free subscriptions to the Washington Post and Washington Times on voting in the 2005 Virginia governor's race.Footnote ⁵

• • Hongbin Cai and Joseph Wang's laboratory study of strategic communication games.Footnote ⁶

So Groseclose's analysis of the effects of media bias is indeed based on a clear framework and high-quality social science research. Yet it is the combination of these elements that is ultimately speculative and not entirely convincing.

Let me begin with the framework. The theoretical basis of the lambda is questionable. That a voter would have beliefs that are the weighted average of her own views and the slant of the media she consumes is rather ad hoc. The biggest problem is that the model does not allow for citizens to discount biased sources of information. A conservative voter should place less weight on information from a liberal media source than a moderate voter who should place even less than a liberal voter.Footnote ⁷ But under his assumptions, the net effect of liberal media slant is greatest for conservatives.Footnote ⁸ Groseclose anticipates such criticisms and defends his estimates by noting that he calibrates the lambda based on average or moderate voters. Yet in many of his counterfactuals on the 2008 elections he extrapolates these effects to much more conservative voters.

As a second defense, Groseclose appeals to the findings of Cai and Wang that show that the response bias in a signaling game increases with preference divergence. But this argument is not persuasive either.Footnote ⁹ Cai and Wang conduct four experiments where N = 0 and the media's true preferences take on values of .5, 1.2, 2, or 4. Across these experiments, the media players chose average slants (M) of .153, 1.161, .894, and 2.186 (I have done some renormalization for explication purposes). The average values of V for each experiment are .078, .377, .282, and .592. Groseclose interprets these results as four different values of λ (.510, .324. .315, and .270) and then focuses on the median. But I believe the correct interpretation is that these data are not consistent with Groseclose's media lambda model. Importantly, the weight that voters put on the media declines as the media's slant increases.

My concern is not simply a theoretical one. It has substantial consequences for his analysis of the 2008 elections. If voters actually discounted biased sources so that the net effect of liberal media slant was the lowest for conservatives, Groseclose's method would vastly overstate the effects of the media. This issue would also distort Groseclose's estimate of what he calls “clarity”: the counterfactual views of the voters in a world without media bias. His calculations suggest that media bias has moved the average voter from a moderately conservative Republican (such as Tom Ridge) into a conservative Democrat (such as John Breaux). But if the Ridge clones discounted liberal media appropriately, it is hard to see how the media would turn them into Breaux clones. But instead, Groseclose argues that the true location of clarity is actually more conservative than Ridge. But this argument assumes that the media lambda is independent of and unaffected by the media slant. This is also inconsistent with the biased information.

Beyond the theory of the media lambda, I have concerns about how the experimental studies were used to calibrate it. The use of the Gerber, Karlan, and Berger results are especially problematic. They report a difference of 3.8 percentage points in support for the Democratic candidate between those who were offered free subscriptions to the Washington Post versus those offered the Washington Times. This large treatment effect implies a large value of V − N which in turn implies a large value of the media lambda. It is revealed, however, (n. 4, p. 275) that voters in both the Post and Times treatments were more likely to vote for the Democratic candidate than the control group. The estimated pro-Democratic effects are 11.2 and 7.4 percentage points for the Post and Times, respectively. These estimates are hard to square with Groseclose's premises. While Groseclose's estimate of the Times PQ is only moderately conservative, it is still to the right of the average voter and it seems implausible that its slant would increase a Democratic vote share by an astounding 7.4 percent. A second concern is that the differences in the Post and Times treatments are not statistically significant. Of course, there may be true differences, but the experiment was not powerful enough to estimate them precisely. But Groseclose fails to point out that a media lambda of zero cannot be rejected by the Gerber, Karlan, and Bergan data.

I am also wary of using a field experiment to produce estimates of real world counterfactuals. The power of the field experimental approach derives precisely from the fact that the researcher can randomize in a way that self-selection does not contaminate the causal inferences. But in the real world, media exposure is not randomized; there are large amounts of self-selection into media outlets based on ideology and slant. So any attempt to evaluate the real-world effects, therefore, has to account for this selection.Footnote ¹⁰

In Left Turn, Tim Groseclose has produced a highly accessible and often entertaining polemic on media bias. Much of it is based on state-of-the-art social scientific findings that I generally find credible. But there is still much that we don't know about the ideological leanings of the media and the effects of that slant on political behavior. As a polemicist, Groseclose is entitled to speculate based on the evidence at hand. As a social scientist, he has a little bit more work to do if he seeks to get to the bottom of the media's influence on politics.