Integrating Item-Response Theory and International Norms Research

Item-response theory is well suited to the task of measuring the adoption of international norms. Although we cannot directly observe the position of a subject along a latent dimension, IRT models provide a framework for assessing how a subject’s position is related to the observed responses among a set of indicators.Footnote ² Our understanding of norm adoption naturally operates in a similar fashion; the degree of norm adoption varies and cannot be directly observed, yet we can observe the behaviors associated with the norm. To be clear, we may be interested in a specific behavior, but the study of norms often requires us to consider several observable implications of the norm.

I focus specifically on norm adoption by states to assess the plausibility of the strategy. The state is the “natural referent” and relevant target for many types of international norms (Cloward Reference Cloward2016, 10). Further, norm adoption by the state (i.e., institutionalization) is productively distinguished from other aspects, such as the effectiveness of implementation or broad attitudinal shifts. While each are important, there is theoretical value in considering the components separately to facilitate the study of the relationship between them.Footnote ³

The application of IRT models to norm adoption requires the identification of indicators (akin to survey questions in the ideology example). Using international legal agreements, resolutions, and treaties related to a single norm offers one potential avenue, but such an approach is of limited use and generalizability. International institutionalization is not a necessary condition of international norms (Finnemore and Sikkink Reference Finnemore and Sikkink1998, 900; see also Clapp and Swanston Reference Clapp and Swanston2009; Orchard Reference Orchard2016). Such indicators may also be of limited use in assessing norm adoption over time, as a one-off vote on issue x in year t provides no insight on the norm in other years. As noted by Simmons and Jo (Reference Simmons and Jo2019), the literature has converged towards measuring norms through a combination of state practices or behavior and social responses to violations of the norm. Moreover, the distinction between domestic and international norms is often blurred in practice, as the creation of either domestic or international norms may precede and shape the other (Clapp and Swanston Reference Clapp and Swanston2009; Gest et al. Reference Gest, Armstrong, Carolan, Fox, Holzer, McLellan, Mogan and Talib2013).

Using a theoretically informed set of state policies/laws thus offers a useful way to evaluate norm adoption over time, as states can adopt or repeal policies/laws in any year. Such an approach also allows for the substance of the norm to shift, as one manifestation of norm evolution is found in the adoption of new state behaviors. As illustrated in Figure 1, the distribution of countries along the latent dimension (representing degree of norm adoption) provides insight into dynamic models of norm evolution. Beyond norm “rejection” or “adoption,” we may observe shifts towards “polarization” (division into two distinct groups of actors) or “rejection/backlash” (distancing from the norm, as observed through a shift over time from adoption towards rejection) (Nuñez-Mietz and García Iommi Reference Nuñez-Mietz and Iommi2017; Symons and Altman Reference Symons and Altman2015).

Figure 1.

Stylized Distributions of Latent Estimates

Note: Each panel represents the stylized distribution of latent scores across countries in a single year, corresponding with potential theoretical expectations of norm evolution.

The IRT model developed here builds on previous scholarship using Bayesian dynamic models to measure changing standards of accountability in human rights practices (Fariss Reference Fariss2014). Simpler additive approaches model the latent dimension (norm adoption) as the sum of policies/laws in a given year, assuming equally weighted indicators and no error (Fariss Reference Fariss2018, 250). Instead, dynamic Bayesian approaches easily allow us to investigate indicator weights, temporal dependence in latent estimates, and measurement uncertainty. In other words, this approach allows for policies/laws that are not equally “difficult” for countries to adopt while also recognizing that our estimate of the degree of norm adoption is uncertain and conditioned by our beliefs about a country’s norm adoption in previous years. Importantly, the policies/laws are expected to be related to each other through their shared relationship with the latent dimension (norm adoption), but they are otherwise assumed to be independent.

The data thus consists of country-year observations where i = 1, ..., N indexes countries, t = 1, ..., T indexes year, and j = 1, ... J indexes policies/laws. The probability distribution, where $ F\left(\cdot \right) $ identifies the logistic cumulative distribution function, is

(1) $$ P\left[{y}_{it j}=1\right]=F\left({\alpha}_{jt}+{\beta}_{jt}{\theta}_{it}\right). $$

For ease of interpretation, $ \alpha $ represents the “difficulty” of a positive response for a policy/law given a country’s position on the latent dimension; $ \beta $ represents the ability of a policy/law to discriminate between different positions on the latent dimension; and $ \theta $ represents the latent dimension (degree of norm adoption). The likelihood function is

(2) $$ L\left(\alpha, \beta, \theta |y\right)=\prod \limits_{i=1}^N\prod \limits_{t=1}^T\prod \limits_{j=1}^J\left[F{\left({\alpha}_{jt}+{\beta}_{jt}{\theta}_{it}\right)}^{y_{it j}}\ast {\left(1-F\left({\alpha}_{jt}+{\beta}_{jt}{\theta}_{it}\right)\right)}^{\left(1-{y}_{it j}\right)}\right]. $$

Following previous work (Fariss Reference Fariss2014), the prior for each difficulty parameter is modeled as $ {\alpha}_{jt}\sim N\left(0,1\right) $ and for each discrimination parameter as $ {\beta}_{jt}\sim Gamma\left(4,3\right) $ .Footnote ⁴ But this model differs from others in two important ways. First, the $ \alpha $ and $ \beta $ parameters are estimated in each year, which allows the difficulty of the policies/laws to change over time. Second, the dynamic prior for the latent estimate is tied to domestic regime changes. Using data from the Archigos project (Goemans, Gleditsch, and Chiozza Reference Goemans, Gleditsch and Chiozza2009) to identify country-years with political leadership changes, I specify priors of $ {\theta}_{i1}\sim N\left(0,{\tau}_1\right) $ for the first year in each country’s regime and for years t = {2, ..., T} it is $ {\theta}_{it}\sim N\left({0}_{i,t-1},{\tau}_2\right) $ . The prior for each variance is $ \tau \sim Gamma\left(1,.1\right) $ . These changes improve the connection between the model specification and our theoretical understanding of norms. We expect some state behaviors to appear less difficult as the norm is more widely adopted. It is also possible for countries to rapidly change position on the latent dimension, notwithstanding their dependence on previous years. Further, linking the dynamic latent prior to domestic regime change operationalizes the expected temporal patterns of change and stability that correspond to the political dynamics of regime change. This imposes a structure that is between fully unstructured independent priors and fully smoothed dynamic priors.Footnote ⁵

Lesbian, Gay, and Bisexual Equality Norm