No CrossRef data available.
Published online by Cambridge University Press: 16 June 2005
Rethinking Social Inquiry: Diverse Tools, Shared Standards. Edited by Henry E. Brady and David Collier. Lanham, MD: Rowman & Littlefield, 2004. 384p. $72.00 cloth, $27.95 paper.
This essay collection is an exceptionally important book that establishes a pluralistic vision of social science methodology. Until a few years ago, the discipline equated methods with quantitative methods and formal modeling; qualitative research was often seen as lacking precision and rigor and therefore as undeserving of the “methods” label. How things have changed—and for the better! The American Political Science Association now has a flourishing section on Qualitative Methods, which has attracted a rapidly increasing membership.
This essay collection is an exceptionally important book that establishes a pluralistic vision of social science methodology. Until a few years ago, the discipline equated methods with quantitative methods and formal modeling; qualitative research was often seen as lacking precision and rigor and therefore as undeserving of the “methods” label. How things have changed—and for the better! The American Political Science Association now has a flourishing section on Qualitative Methods, which has attracted a rapidly increasing membership.
The volume edited by Henry Brady and David Collier provides a comprehensive, sophisticated, and evenhanded methodological justification for this striking reassertion of qualitative approaches. It demonstrates convincingly that case studies and small-n comparisons are not inherently inferior to statistical analyses; they have great strengths of their own and suffer from less severe weaknesses than a number of quantitatively oriented scholars have claimed. At the same time, quantitative methods themselves stand on shakier ground than is often assumed. Since qualitative and quantitative approaches both have significant advantages and disadvantages, they should be seen as equally legitimate procedures for social scientific inquiry. Rather than continuing to engage in a battle for superiority, advocates of these “diverse tools” should converge on “shared standards,” as the volume proclaims in its programmatic subtitle.
With this crucial message, Brady and Collier take aim at the influential volume by Gary King, Robert Keohane, and Sidney Verba, Designing Social Inquiry (1994). In that book—aptly summarized and systematized in Chapter 2 of the present collection—a leading quantitative methodologist and his coauthors offered to help out their inferentially challenged qualitative brethren, who were crippled by such problems as the difficulty of sorting out a large number of variables with a small number of cases. The advice of King and his colleagues, explicitly drawn from a statistical template, was to mainstream qualitative research by making it as similar as possible to quantitative studies; this is evident in the frequently repeated recommendation to increase the number of observations.
Yet Designing Social Inquiry, which initially increased the pressure in the discipline on qualitative scholars, soon triggered a revival of qualitative methodology, which demonstrated the inherent value of this approach and revealed quantitative claims to superiority as methodologically unjustified. Brady and Collier reprint more or less updated versions of several important essays that evaluated those claims (Chapters 5–6, 9–11) and add a number of original chapters. The introduction and the two concluding chapters are especially wide-ranging and useful; they successfully pull together the discussion. An extensive glossary makes the volume accessible to nonspecialists. While somewhat disparate, the book is impressive in the intellectual caliber of its contributors and the diversity of their methodological orientation; the cooperation among qualitatively oriented scholars and outstanding statistical researchers Henry Brady and Larry Bartels (the author of Chapter 3) gives the collection's pluralistic message special legitimacy. This is not a flaming manifesto of qualitative die-hards, but a balanced assessment of different yet complementary methodological approaches, guided by a quest for shared standards.
In this spirit, the volume demonstrates that some of King, Keohane, and Verba's claims to quantitative superiority rest on problematic assumptions. For instance, they suggested that large-n studies were similar to experiments in their randomization strategies, which allowed researchers to isolate the impact of causal variables; by contrast, small-n research was plagued by severe selection problems and the impossibility of sorting out causal effects with few cases. Yet Brady and Collier insist on the fundamental difference between experiments, which allow for the true randomization of causal treatments, and observational studies that analyze given cases; the latter category includes both large-n and small-n analyses. Qualitative and quantitative research therefore confronts similar problems of inference, such as the danger of endogeneity; large-n analysis is not by nature superior. Several authors in the collection, especially in Chapter 6, also show that warnings by King et al. about selection bias in qualitative research were exaggerated. “Selection on the dependent variable,” often depicted as a cardinal sin, does not invalidate all inferences. In fact, causal heterogeneity—that is, the divergent operation of causal factors in variegated contexts—can justify the truncation of samples.
The Brady and Collier volume also demonstrates the serious trade-offs that following the advice of King et al. can entail. For instance, the constant exhortation to increase the number of observations may push researchers to trespass the bounds of causal heterogeneity and to engage in conceptual stretching. By concentrating primarily on problems of causal inference, the previous authors overlooked other essential issues, such as how to attain conceptual validity. Since qualitative research is much more attentive to these conceptual and theoretical questions, it has important advantages over statistical methods in some respects. In fact, the Brady and Collier volume diverges from the correlational approach to causal inference that underlies the earlier book; they introduce the notion of causal-process observations to argue that much of the information on causal mechanisms and their operation that qualitative researchers unearth does not fit into quantitative data sets, but nevertheless sheds decisive light on cause-and-effect relations (pp. 252–71). Thus, qualitative research makes crucial contributions that go far beyond the confines of quantitative analysis.
Most of the Brady and Collier volume is fully convincing. Its emphasis on the complementary strengths and weaknesses of qualitative and quantitative approaches and its rejection of claims of inherent superiority hold great promise for overcoming the methodological wars in our conflictual discipline and for creating agreement on methodological pluralism.
But pluralism does not mean uniformity. Perhaps these editors go too far in their search for common ground. Is it really feasible to find “shared standards” (subtitle)? By confronting the methodological trade-offs aptly analyzed in Chapter 12, qualitative and quantitative researchers have always prioritized divergent standards—generality versus accuracy, parsimony versus depth of understanding, operationalization versus conceptual validity, and so forth. While some attention to competing standards could improve both strands of research, they are interested in different aspects of this hypercomplex reality called politics. In fact, divergent standards may derive from different views on “how the world works.” In the statistical worldview, variables affect given cases in regular, fairly uniform, objectively observable ways; by contrast, qualitative scholars see shifting combinations of causal factors operating differently in different contexts. Quantitative and qualitative methods reflect these divergent worldviews and are especially appropriate for analyzing those aspects of politics that each worldview highlights. The demand for shared standards may therefore make it difficult for both sides to use their special strengths to the greatest advantage. Instead, quantitative and qualitative approaches may best be seen as complementary: They both add crucial—yet different—facets to a complex, multidimensional picture.
Notwithstanding this question, Rethinking Social Inquiry is one of the most important methodological contributions published during the last decade. It should be required reading for all political scientists. In fact, one hopes that the authors will follow up on this abstract analysis of methodological principles, flesh out the specific advice offered especially in Chapters 7 and 8, and soon produce a pluralistic equivalent to the King, Keohane, and Verba volume—a comprehensive, “practical, down to earth, simple, and above all, reliable” (Brady, p. 53) guide on how to do good qualitative research.