CHALLENGES IN STUDYING NORMS

Institutions are systems of rules that work together to coordinate and regulate social behavior (Raymond et al. Reference Raymond, Weldon, Kelly, Arriaga and Clark2014). All institutions, no matter how well established, and no matter whether they are formal or informal, rely to some degree on norms (March and Olsen Reference March and Olsen1989). Norms comprise an essential part of the creation and operation of both formal and informal institutions. Norms can be rules that constitute formal institutions (Posner Reference Posner2000), or norms can support or obstruct compliance with formal rules (Chappell Reference Chappell2014; Helmke and Levitsky Reference Helmke and Levitsky2006; Tyler Reference Tyler2006). The more that norms support formal institutions, the more embedded they are (Raymond et al. Reference Raymond, Weldon, Kelly, Arriaga and Clark2014).Footnote ¹ Norms about identity and social hierarchy undergird operations in most institutions without ever being formally articulated (March and Olsen Reference March and Olsen1989; Young Reference Young1990). Informal institutions also rely on norms for their creation and operation, and norms may constitute some or all of the informal rules that comprise informal institutions (Finnemore and Sikkink Reference Finnemore and Sikkink1998).

In order to meaningfully study institutions, then, we must get at these mushy, amorphous, shifting aspects of social practice—that is, norms—that are not written down anywhere or formally articulated in any official documents. Many researchers have preferred qualitative methods used in fieldwork, such as interviews, participant observation, and other types of deeply qualitative, case-oriented work in order to study institutions. Many excellent studies of informal institutions and norms, both feminist and mainstream, employ such qualitative methods (Chappell Reference Chappell2014; Helmke and Levitsky Reference Helmke and Levitsky2006; McLean Reference Maclean2010). This excellent work is critical to our understanding of these phenomena. But even for these difficult-to-pin-down phenomena, large-scale cross-national research has a role to play and can add additional insight to that generated by these deep qualitative studies. Below I outline the challenges and elaborate on why these techniques can nevertheless provide new understanding.

Formal rules are comparatively straightforward as objects of study: they tend to be written down and codified in official documents. Policy documents and written pieces of legislation, for example, express these formal rules. Formal rules often have a set spatial and temporal scope so that jurisdictions are officially determined. There may even be formal procedures for changing rules or offering new interpretations so that changes in formal rules or their applications may be demarcated in space and time.

In contrast, informal rules, or norms, are not written down explicitly in official documents. The scope and application of the norm may be unclear: To whom does the rule apply? In what temporal or spatial domain? For example, in Lafayette, Indiana, where I live, it is safe to say, “Boys around here, for the most part, do not wear dresses or skirts.” However, in Chicago, only a few hours away, there are many more spaces in which such attire would go unquestioned. Even in Lafayette there are occasions (drag queen competitions; the Robbie Burns Dinner) for which the rule is suspended or inapplicable or identity groups (visiting groups of males from other nations with different standards of dress; transgender people) for whom the norm does not hold. Moreover, it is also clear that the consequences of violating the rules are shifting a little: in the past, it might have been more certain that boys or men wearing dresses would provoke violence or overt censure, but in contemporary Lafayette, Indiana, such a response is less certain, particularly in some contexts.

In other words, for these informal rules, there is no formal articulation, no hard and fast line delineating the scope of the rule's application or the consequences of its violation. If one wanted to study how these informal rules varied across the state of Indiana, for example, tracking the considerable variation between urban and rural areas, or if one wished to compare these informal practices across states, say, comparing Indiana to California, one would have to do more than obtain copies of legislation or judicial decisions. One would need to devise a way to determine which informal rules are being followed, where they are being followed, and by whom and to what degree. Because they are not written down, articulated, or even consciously followed by those who obey them, such rules must be inferred from observations of behavior, through skillful questioning, or by parsing policy statements to disinter the assumptions implicit in formal statements or that are manifest in their application or interpretation.

To complicate matters, rules may shift from formal to informal status and back again over time. Even in one time period, such informal rules often bleed into or undergird formal practices. Indiana law does not require male residents to abstain from wearing dresses or skirts, but many school districts have dress codes about “distracting” dress that they use to send home boys in skirts, girls in skirts deemed to be “too short,” and so on.

The processes by which formal rules are made (such as legislative processes) may in some sense be directly observed. One can track proposals and votes and official discussions. In contrast, norms are never directly observed; like gravity, we can only observe norms' effects and infer the nature of the implicit rule being followed.Footnote ² We cannot even necessarily ask people straightforwardly about the effect of norms, as they often operate—and perhaps are most powerful—on an unconscious level. The shifting and informal nature of norms means that the scope and application of the norms follow fuzzy boundaries and that exact points or degree of change may be difficult to pinpoint.

These challenges are of particular importance to gender scholars because gender is fundamentally concerned with norms of appropriate behavior for particular identity groups. Gender is a constellation of institutions, and norms comprise a significant element of those institutions (Young Reference Young1990; Reference Young1994). Overcoming the research challenges presented by norms, then, is critical for both feminists and institutionalists.

WHY CROSS-NATIONAL STATISTICAL ANALYSIS?

The difficulty in pinpointing informal institutions, in comparison to studying formal ones, has led many feminist and institutionalist scholars to prefer local, qualitative studies, employing techniques such as interviews and participant observation, as well as a sort of “soak and poke” technique of immersion in the local context. This approach enables the researcher to see things from the perspective of a local resident and to use this local knowledge to unearth norms. These studies have been useful in uncovering norms, to be sure. But perhaps counterintuitively, large-scale cross-national statistical analysis also has a role to play in advancing our understanding of norms. The key advantages statistical techniques offer are (1) the ability to summarize large quantities of information that are difficult to eyeball or summarize using traditional qualitative tools, (2) the ability to estimate the degree to which observed relationships can be attributed to chance, and (3) the ability to parse the degree to which different factors shape outcomes of interest. Cross-national use of these statistical techniques permits examination of the importance of macrolevel institutional and societal factors and their interrelationships. Cross-national comparisons also offer a greater ability to denaturalize local social practices (particularly useful to feminists seeking to critique male dominance). I explain each of these advantages further below.

First, statistical techniques are effective tools for summarizing large quantities (and many different kinds) of information. When we do qualitative case studies of institutions, we make inferences about social practices and historical tradition from various types of data, typically interviews, transcripts, newspapers, other documents, examination of artifacts, observed behavior of participants in certain events, among others. Many scholars no doubt have returned from fieldwork swimming in data or have felt overwhelmed at the complexity of a set of relationships. How do we start to see if our theoretical ideas can make sense of this morass? Statistical analysis offers a basic set of tools for uncovering relationships, including associations, interactions, and linear and nonlinear relationships, to name a few. A standard set of tests can reveal the plausibility of our ideas about whether a set of relationships holds in a sea of data. Multiple indicators can be used serially or combined into an index to create robust measures of difficult-to-get-at phenomena. These tools can help us relate our theoretical ideas, our ideas about how the world works, to a very complex set of observations. Employing conventional and preestablished decision rules may also help to dispel concerns about being perceived as cherry-picking cases, examples, or anecdotes.

Not only can we uncover relationships, but we can also get a sense of how likely it is that they are mere accidents, unlikely to occur again. Scholars who confront politics and institutions at the most granular level are keenly aware of the seemingly idiosyncratic and particular nature of the causal chains observed. Are these relationships in fact due to chance? Which elements of the story are idiosyncratic, and which can be expected to recur or be visible in other locales? Statistical techniques can provide an assessment of how likely it is that a relationship observed in a given set of observations is due to chance. This is very difficult to tell when one is relying on a single, local, qualitative account.

Third, we can obtain valuable empirical assessment about the relative importance of a series of factors in determining a particular outcome. So often in our qualitative studies, we are presented with long lists of complex combinations of factors that seem to lead to a particular outcome. But which combinations are most powerful? Which matter most? Does the sequence change the effect? Traditionally, qualitative scholars seek to leverage a series of comparative case studies to establish the answers to these questions, but this approach tends to produce comparisons that are far from definitive because of the many differences across cases. They are particularly crude when it comes to assessments of “how much” various factors matter and which of a set of important conditions is most (or least) critical. Statistical modeling provides tools to answer these counterfactual questions by providing techniques to sort out how much variation is in fact associated with certain factors across a set of cases and which factors or dimensions have the strongest associations with the outcomes. They can also provide support for different accounts of the nature of the relationship (e.g., Linear or nonlinear? Monotonic? Exponential? Interactive?). These sorts of assessments can rarely be supported by comparative case studies because there is an insufficient range and/or degree of variation in the limited number of cases examined (Collier and Mahoney Reference Collier and Mahoney1996).

Some scholars criticize these types of analyses for being so far removed from local contexts that they study politics everywhere and nowhere, that data become so far removed from the practical context of political action that they become meaningless (e.g., Peters Reference Peters1998; Sartori Reference Sartori1991). While there is some truth to the claim that generalization sacrifices attention to specific contexts and details, it is important to remember that all studies of norms, both qualitative and quantitative, rely on a similar process of inference. It is important to remember that even intensively local qualitative research on norms relies on inference, and even in these very detailed studies, norms themselves are observed through their effects and never “observed” directly. They remain theoretical constructs that help us explain patterns in social and political practices at the local level.

A similar inferential process can be used on a wider scale, particularly where inferential goals are more modest or precise. For example, we can infer from the fact that the majority of children are now born to unmarried parents and by tracking the dramatic change in the proportion of children born to unmarried parents, that something has changed in the institution of the family. We can infer from the increases in married women and working mothers that new norms must have emerged relating to women and work. This does not tell us what the new norms are, but it does tell us that something has changed. If we think that such practices are related to changes in formal institutions or laws, we can examine the relationship between formal and informal practices over time. Joseph Henrich, for example, examines patterns of monogamy and polygamy over time, showing how monogamy comes to replace polygamy, which once characterized roughly 85% of known human societies as the most common form of family structure (Henrich Reference Henrich2012).

These wider inferences cost us nuance and introduce new challenges related to the cross-national or over-time comparability of data sources. Nevertheless, where comparable data sets can be constructed, these broader comparisons are worth the effort, as they offer insights not available in local studies. These wider-scale inferences establish the generalizability of theses based on local studies and help establish their boundary conditions: if we find that industrialization destroys social norms that work to mitigate inequality, or dependency among the elderly, in one locale, we can examine whether that effect is one that is visible more generally or only in some kinds of societies. We can accumulate the learning from local studies more effectively when we can examine the plausibility of the generalizability of the claim about norms.

Similarly, we can examine rates of violence over time to understand how social practices and norms related to violence against women are changing (especially as data on rates of violence against women improve). For example, Italy saw increased numbers of femicides (murders of women by men) from 1992 to 2008 (Codini Reference Codini2011; Poggioli Reference Poggioli2012). This should prompt inquiry: there is something to explain, a pattern to interpret.

Such statistical analysis can be useful when focused on a single national context, as the example about Italy above demonstrates. Similarly, much study of the U.S. Congress constitutes statistical analysis of a single national context, and this research approach can contribute important insights about norms. For example, the filibuster is an informal rule that enables individual legislators to disrupt the ability of the members of the U.S. Senate to vote on a given provision. It is a very important phenomenon to understand if one wishes to understand the operation of U.S. political institutions such as Congress. We can fruitfully study the use of this informal rule in this single national context by charting changes over time and examining the correlates of those changes (Binder, Lawrence, and Smith Reference Binder, Lawrence and Smith2002). This systematic analysis provided insights counter to the conventional wisdom about the use of the filibuster examined in a shorter time period, based on examinations of only a few cases. So statistical analysis of a single national context can provide insights into informal institutions that are not otherwise available.

Employing such techniques in larger scale cross-national studies provides an additional analytic edge. Cross-national studies can examine factors more likely to vary across nations than over time (such as electoral systems) and can explore the interplay of differences in institutional structures with other time-variant and time-invariant factors and conditions. The degree of institutional variation, whether formal or informal, may not be as great inside nations, or over time inside individual nations, as it is across them. So large-scale cross-national studies allow us to examine the conditions under which these institutions change or stay the same and to chart their effects.

For example, we can ask what difference it makes to switch from a PR system to a SMP system. Which system best promotes the representation of women (Matland Reference Matland1993; Matland and Studlar Reference Matland and Studlar1996)? What difference does it make to adopt either system when neither existed previously (Moser Reference Moser2001)? We may want to know the likelihood of effects of particular institutional changes. Statistical analysis, at its best, can give us a sense of how likely it is that desirable changes are actually due to systematic transformations in institutions and how much is probably due to chance. It can tell us which factors are most strongly associated with particular outcomes and even the degree to which a desired outcome is associated with a particular institutional change, on its own or in interaction with various contextual elements or with other factors altogether.

These methods offer additional insights to, and have been used by, institutionalist scholars who take a wide range of theoretical approaches, including feminist, sociological, rational choice, and even historical. Sometimes, historical institutionalists suggest that deep qualitative casework is necessitated by taking an historical approach, but historians include statistical methods in their tool kits (Jarausch Reference Jarausch1991). Indeed, some of the great works of institutionalism are sweeping and cross-national in scope.

Rich qualitative studies contain large amounts of data and can illuminate complex connections. Such studies are persuasive in delineating complex causal relationships and telling compelling stories of how a combination of conditions or events produced the subsequent chain of events leading to institutional stability or change. These studies have considerable internal validity. But confidence about causality can be greater when we also have a sense of the scope of the causal argument outlined and the limits and likelihood of its application to different contexts. Statistical analysis of relationships can demonstrate effectively how the relationship observed in one qualitative case is evidenced in other instances.

Even at its best, statistical analysis produces only plausible links between causes and effects at a theoretical or predictive level, so qualitative studies are essential to understanding causality, as they are able to illuminate in detail the processes that link cause and effect. But the very richness and labor-intensive nature of case study data gathering limit the number of cases that can meaningfully be considered in a single study. Which of the myriad conditions and factors employed here are the ones that made the difference? Theoretical lenses help us to sort through the complex reality we inevitably discover in these case studies, but they do not persuade the skeptic of the general applicability of the argument as effectively as empirical evidence that supports the general nature of the causal relationships posited.

When we use statistical analysis to examine a broader set of cases (as opposed to just change over time), we can see if claims made about these combinations in one situation apply to a more general set of observations. This is important for establishing causal relations more generally, as any causal claim necessarily implies that a similar constellation of conditions or events, if they were to occur again in substantially similar situations, would lead to the same outcome. If we find that this is not the case, then, it even calls into question the causal relationships we think we have identified in the local case.

For example, many careful case studies by human rights scholars demonstrate the role that international law on human rights plays in particular cases. Nonetheless, significant skepticism persists about whether international legal commitments actually have improved practices related to human rights. By complementing case study work with cross-national statistical analysis, Beth Simmons' important work demonstrates that human rights laws do have an effect on government actions. She also shows the sort of cases in which human rights laws can be expected to have the greatest effect and the sort of cases in which these impacts will be less keenly felt (Simmons Reference Simmons2009). Larger scale cross-national comparisons can move us beyond discussions of whether such laws matter to an exploration of when they do or do not, or where the effects are likely to be the greatest.Footnote ³

Similarly, cross-national comparisons can reveal variability in relationships that may seem fixed or inevitable in a single context where institutional, social, or cultural contexts condition outcomes in ways that are invisible to observers looking at a single national context. For example, sex differences in public opinion on “rape myths” in the United States might seem to suggest that such differences stem from a felt vulnerability to sexual assault, or from other essentialist accounts of gender differences. But cross-national studies of public opinion show that differences between men and women vary across national contexts, even disappearing in some (Nayak et al. Reference Nayak, Byrne, Martin and Abraham2003). Sex differences in public opinion similarly vary on other issues. Explanations attributing public opinion to supposedly universal features of the female conditions (motherhood, victimization, or whatever) need to be modified to take into account the things that vary cross-nationally, such as the degree of women's political and social mobilization, proportion of women in the workforce, and so on. This is very important from a feminist perspective, as it reveals and denaturalizes male domination.

For example, in interviews I conducted with public officials about violence against women in Oslo, Norway, in 1995, I was told repeatedly that Norway was a less violent society than the U.S. context from which I came and that, consequently, domestic violence and other forms of violence against women were less common than in the U.S. This was the reason given for the failure to adopt government policies to address such violence. It is cross-national statistical analysis that allows us to show that in the 1990s, rates of violence in the woman-friendly welfare states of Northern Europe were about the same as rates of violence in Canada and the United States (see the discussion and Appendix A in Weldon Reference Weldon2002; see also Elman Reference Elman1996; Elman and Eduards Reference Elman and Eduards1991; Heise, Pitanguy, and Germain Reference Heise, Pitanguy and Germain1994). First-hand impressions of the comparative salience of this phenomenon would have been quite mistaken.

Such large-scale research need not focus only on formal institutions, and it need not ignore social practice or questions of implementation, as some may worry: we can examine statistics about family structure to examine divergences and convergences between legal rhetoric and practice. We can examine take-up rates (the rate at which eligible populations actually make use of benefits available to them) to see if social programs exist primarily on paper or actually reach the intended population. Not only can we observe the divergence between rhetoric and practice, but we can also get a sense of how great the gap is and examine potential explanations for that gap. Such research may also employ and compare local perceptions and attitudes: cross-national survey researchers, like those involved in the World Values Survey or the Eurobarometer, offer important insights for political scientists in general and gender scholars in particular (Inglehart and Norris Reference Inglehart and Norris2003). We can learn from these surveys about attitudes toward women. We can learn about variation in attitudes about whether domestic violence should be viewed as a crime, for example, and we can ask how these attitudes are related to rates of violence and laws themselves (Eurobarometer 2010). Thus, cross-national statistical analysis offers tools for the systematic analysis of the relationships between and changes in formal and informal institutions (such as norms).

Cross-national statistical analyses offer unique insights into institutional stability and change because they offer a perspective on patterns at a macro, cross-national level, revealing associations and preconditions of change that may not be visible from examination of a single case. They also offer us techniques for estimating the relative impact of different factors on institutional change, as well as for estimating the degree to which such change is due to chance. Last, having a bigger picture of institutional change helps us to identify which features of our individual cases are truly remarkable, unique, or worth investigating. They help denaturalize features of the polity otherwise seen as immutable features of “political culture,” or impossible to change for other reasons (such as the underrepresentation of women, or rates of violence against women). Statistical tools provide a way to summarize large sets of observations, identifying patterns for scholars to interpret when qualitative tools would prove unwieldy or inadequate to the task. Such analyses comprise a vital element in the tool kit not only for feminist social scientists, but also for social scientists more generally.