Reading Skarbek's paper may come as a surprise for most researchers in social sciences that there is the need to defend methodological pluralism. For most economists, it would rather be the other way around: they would wonder (at best) or more likely express deep skepticism about benefits to expect from looking at the other side of the road, particularly at case studies! The obsession of my colleagues with quantification, most of the time boiled down to econometrics (recently complemented by lab experiments), leaves almost no room for other approaches, particularly qualitative ones. Quotes from Rodrick, who is not the more narrow-minded in the profession, are illustrative.
In that respect, this defense and illustration of what can be expected from qualitative evidence is most welcome. Although Skarbek focuses attention mostly on case studies and their relevance when it comes to analyzing institutional issues, and particularly establishing causal relations, the points he makes have more general relevance. Actually, some of us may even consider that institutions are ultimately what economics is about. What can we say of the allocation of resources in education, health care systems, labor markets, and supporting organizations without a look at the institutional dimension? In that respect, the distinction he borrows from Coppedge (Reference Coppedge1999) between ‘thin’ and ‘thick’ theories and concepts suggests that the later covers very specific issues, of limited interest if not connected or providing access to more general ones.
I agree with almost all the points made in the paper, articulated around the following statement: ‘Institutional research, in particular, regularly asks questions that would benefit a great deal from qualitative evidence.’ However, there is a certain paradox in the examples Skarbek uses to substantiate this statement. Indeed, he focuses almost exclusively on the micro-level of the organization of police and prisons in relation to crimes. Of course, this is his turf, and it gives consistency to the paper by illustrating different facets of what can be expected from case studies. But what about the other institutional dimensions involved if we hope a ‘thick’ theory of the sources of criminality and the role of institutions like police and prisons to deal with that? For example, what is the role of the different meso-institutional layers (city, regional, states, national) ruling and monitoring police and prisons? And what about macro-institutional factors that may play a key role in shaping ‘corrections culture’ and various modalities of ‘governance’ of prisons, as he alludes when referring to differences between California and Norway? More specifically, what is, for example, the impact of gun-control regulations and their variations across states (or countries) on the actual behavior of criminals as well as policemen and on how crimes are monitored? What kind of insights can we expect from qualitative evidence on such issues? Do ‘case studies’ offer a value added over quantitative methods when it comes to capturing the role of these different institutional layers? And if so, what are the gains to be expected? Say differently: can case studies and other qualitative methods help moving from the small picture to bigger ones and from ‘thin’ theories to ‘thick’ ones?
Let me make it clear that I do share with Skarbek the conviction that case studies can make very positive contributions to economics, notwithstanding the reluctance of mainstream economists to engage in that direction. Indeed, what would be a modern economic theory without the cases of pin factory (Adam Smith's division of labor), auctions (Walras' general equilibrium), Boeing's organization (Arrow's learning curve), or the regulation of radio broadcasting (Coase's transaction costs)? On the other hand, the important limitations of ‘external validity’ that mainstream economists are so happy to emphasize cannot be ignored, and Skarbek does not elude the question. Ostrom riding with police officers or Moskos working as a city police officer for a year might help better understanding how the organization of tasks affects police performance in deterring crime. It also makes an inference, whether descriptive or causal, particularly challenging.
The variety of criteria regarding ‘informational quality’ and in assessing the robustness of comparative case studies illustrates. How can we overcome these difficulties? Can we find ways to make quantitative analysis complementing and substantiating qualitative evidence so as to make our case studies more persuasive? For example, why does the paper neglect the role of quantitative studies in discarding erroneous generalization derived from specific cases? Making steps in this direction would not only improve the credibility of ‘case studies’; it could also support major progress in the analysis of institutions. And what about the potential role of other methods to reach that goal (see Bates et al., Reference Bates, Greif, Levi, Rosenthal and Weingast1998)? Skarbek is short on that, notwithstanding the rich discussion he offers regarding case studies. The other example he briefly introduces, ‘process tracing,’ lacks in his own word ‘a standard definition’ and apparently a clear methodology. Would more extensive use of simulations be relevant? Could scenarios elaborated by game theorists help focusing on appropriate questions to explore through case studies (see Aoki, Reference Aoki2001; or Greif, Reference Greif2006)?
Skarbek rightly emphasizes the many questions that quantitative data and their processing through available econometric and statistical methods cannot adequately explore, not to speak of providing relevant answers. How could econometrics or lab experiments explain why and where major events like the industrial revolution or economic development happened? And can we satisfy ourselves, as economists, just discarding such issues? Sharbek's paper provides clear arguments about the potential contribution of alternative, qualitative methods barely considered legitimate by most economists. The questions raised above only intend to emphasize the need to go further in our reflection on methodological issues in economics.
Let us hope that some leading economists and, even more importantly, numerous young ones will take note of Skarbek’ paper and pay more attention to methods that have now acquired robustness in other social sciences. There is an issue of persuasion here, something that Siggelkow (Reference Siggelkow2007) finely analyzed with respect to the usage of case studies, a paper that would have deserved reference. Indeed, one thing is to be right, the other is to convince!
You have
Access
Reading Skarbek's paper may come as a surprise for most researchers in social sciences that there is the need to defend methodological pluralism. For most economists, it would rather be the other way around: they would wonder (at best) or more likely express deep skepticism about benefits to expect from looking at the other side of the road, particularly at case studies! The obsession of my colleagues with quantification, most of the time boiled down to econometrics (recently complemented by lab experiments), leaves almost no room for other approaches, particularly qualitative ones. Quotes from Rodrick, who is not the more narrow-minded in the profession, are illustrative.
In that respect, this defense and illustration of what can be expected from qualitative evidence is most welcome. Although Skarbek focuses attention mostly on case studies and their relevance when it comes to analyzing institutional issues, and particularly establishing causal relations, the points he makes have more general relevance. Actually, some of us may even consider that institutions are ultimately what economics is about. What can we say of the allocation of resources in education, health care systems, labor markets, and supporting organizations without a look at the institutional dimension? In that respect, the distinction he borrows from Coppedge (Reference Coppedge1999) between ‘thin’ and ‘thick’ theories and concepts suggests that the later covers very specific issues, of limited interest if not connected or providing access to more general ones.
I agree with almost all the points made in the paper, articulated around the following statement: ‘Institutional research, in particular, regularly asks questions that would benefit a great deal from qualitative evidence.’ However, there is a certain paradox in the examples Skarbek uses to substantiate this statement. Indeed, he focuses almost exclusively on the micro-level of the organization of police and prisons in relation to crimes. Of course, this is his turf, and it gives consistency to the paper by illustrating different facets of what can be expected from case studies. But what about the other institutional dimensions involved if we hope a ‘thick’ theory of the sources of criminality and the role of institutions like police and prisons to deal with that? For example, what is the role of the different meso-institutional layers (city, regional, states, national) ruling and monitoring police and prisons? And what about macro-institutional factors that may play a key role in shaping ‘corrections culture’ and various modalities of ‘governance’ of prisons, as he alludes when referring to differences between California and Norway? More specifically, what is, for example, the impact of gun-control regulations and their variations across states (or countries) on the actual behavior of criminals as well as policemen and on how crimes are monitored? What kind of insights can we expect from qualitative evidence on such issues? Do ‘case studies’ offer a value added over quantitative methods when it comes to capturing the role of these different institutional layers? And if so, what are the gains to be expected? Say differently: can case studies and other qualitative methods help moving from the small picture to bigger ones and from ‘thin’ theories to ‘thick’ ones?
Let me make it clear that I do share with Skarbek the conviction that case studies can make very positive contributions to economics, notwithstanding the reluctance of mainstream economists to engage in that direction. Indeed, what would be a modern economic theory without the cases of pin factory (Adam Smith's division of labor), auctions (Walras' general equilibrium), Boeing's organization (Arrow's learning curve), or the regulation of radio broadcasting (Coase's transaction costs)? On the other hand, the important limitations of ‘external validity’ that mainstream economists are so happy to emphasize cannot be ignored, and Skarbek does not elude the question. Ostrom riding with police officers or Moskos working as a city police officer for a year might help better understanding how the organization of tasks affects police performance in deterring crime. It also makes an inference, whether descriptive or causal, particularly challenging.
The variety of criteria regarding ‘informational quality’ and in assessing the robustness of comparative case studies illustrates. How can we overcome these difficulties? Can we find ways to make quantitative analysis complementing and substantiating qualitative evidence so as to make our case studies more persuasive? For example, why does the paper neglect the role of quantitative studies in discarding erroneous generalization derived from specific cases? Making steps in this direction would not only improve the credibility of ‘case studies’; it could also support major progress in the analysis of institutions. And what about the potential role of other methods to reach that goal (see Bates et al., Reference Bates, Greif, Levi, Rosenthal and Weingast1998)? Skarbek is short on that, notwithstanding the rich discussion he offers regarding case studies. The other example he briefly introduces, ‘process tracing,’ lacks in his own word ‘a standard definition’ and apparently a clear methodology. Would more extensive use of simulations be relevant? Could scenarios elaborated by game theorists help focusing on appropriate questions to explore through case studies (see Aoki, Reference Aoki2001; or Greif, Reference Greif2006)?
Skarbek rightly emphasizes the many questions that quantitative data and their processing through available econometric and statistical methods cannot adequately explore, not to speak of providing relevant answers. How could econometrics or lab experiments explain why and where major events like the industrial revolution or economic development happened? And can we satisfy ourselves, as economists, just discarding such issues? Sharbek's paper provides clear arguments about the potential contribution of alternative, qualitative methods barely considered legitimate by most economists. The questions raised above only intend to emphasize the need to go further in our reflection on methodological issues in economics.
Let us hope that some leading economists and, even more importantly, numerous young ones will take note of Skarbek’ paper and pay more attention to methods that have now acquired robustness in other social sciences. There is an issue of persuasion here, something that Siggelkow (Reference Siggelkow2007) finely analyzed with respect to the usage of case studies, a paper that would have deserved reference. Indeed, one thing is to be right, the other is to convince!