3.1. Deficiency

Here I apply the first two criteria to the case. The first criterion holds that existing technologies are deficient for some purpose. In the case of discrimination scales, the deficiency is in part based on common assumptions about the relevant dimensions of the theoretical construct. Existing scales do not measure certain kinds of racially discriminatory experiences, namely, those that are institutional but noninterpersonal factors (such as policies) and those that cannot be clearly attributed to race. Reducing dimensionality of complex social constructs is often a necessary part of social science. It does not by itself demonstrate deficiency. It must be shown that, for some particular purpose, it is deficient to merely employ the existing measures. In other words, do we have purposes for which it would be useful to measure other dimensions?

Here I want to claim that existing scales are deficient for two purposes: (1) understanding the causes of social alienation and (2) the integration of evidence about interpersonal and institutional discrimination. I address each in turn.

Some experiences of discrimination lack clear actors or cannot be clearly attributable to one’s social identity; these experiences may be more associated with feelings of social alienation than cases that are clearly caused by, for example, racial hatred. Here are some suggestive reasons in support of this claim. Researchers have found that high self-identification with one’s social group can prevent some negative effects of racial discrimination (e.g., Mossakowski Reference Mossakowski2003; Sellers et al. Reference Sellers, Caldwell, Schmeelk-Cone and Zimmerman2003). This evidence is consistent with high self-identification playing a role in the appraisal of the experience. Perhaps individuals with high self-identification with a particular racial group are more likely to interpret cases of discrimination as clearly attributable to their race, but also to dismiss the negative evaluations of their racial group that accompany this treatment. Individuals with low self-identification may be less likely to report discrimination because their experiences of discrimination will be less clearly attributable to race. This could be one cause of social alienation from social groups in general. Measures of ambiguously attributable discrimination are needed to test the potential mediation of the relationship between self-identification with a racial group and social alienation.

The second purpose for subjective measures of discrimination could be to integrate evidence concerning interpersonal and institutional discrimination. Currently there are different methods for measuring discrimination that broadly track either interpersonal discrimination or institutional discrimination. Subjective measures tend to measure interpersonal interactions and their effects (excluding scales of Major Discrimination). Institutional discrimination is often measured using large-scale population studies or field experiments. Large-scale population studies assess the impact of policy changes or other environmental factors on different racial groups. Field experiments investigate when individuals are treated differently due to race by examining response rates to nearly identical individuals (except for race) in various decision procedures (e.g., housing, employment). However, some institutional discrimination is mediated by interpersonal interactions. Some agent acts as a representative and enforcer of institutional policies, such as police officers who escalate interactions with the public on the basis of probable cause that disproportionately impacts Black and Hispanic populations. Subjective measures concerning institutional-interpersonal discrimination could help researchers integrate evidence about these dimensions of discrimination. While Williams’s Major Discrimination scale does this on some items, other items are left open to interpretation (e.g., denied a bank loan by whom?). More intentional design of the scale to identify experiences with both interpersonal and institutional dimensions would better serve integrative goals.

3.2. Superior alternatives

The second criterion for path dependence is that alternative technologies are superior (compared to existing technologies) with respect to some particular goal. Here we can again appeal to the goals of explaining social alienation and integrating evidence about institutional discrimination that occurs interpersonally. Alternative subjective measures of ambiguous discrimination and of institutional discrimination that occurs using representatives of the institution would better serve these goals than existing measures. Creating measures of ambiguous discrimination will likely need a different approach than directly asking individuals to report experiences of discrimination, which will favor reports of clear cases. Instead, a subjective measure may offer vignettes of putative cases of ambiguous discrimination and measure interpretations of the case along with other relevant variables (i.e., self-identification with group). However, these alternative subjective measures, to my knowledge, do not yet exist. Thus, the second criterion is only partially met. Were such alternative measures to exist, then they would better serve some research purposes.

3.3. Disadvantaged alternatives

The third criterion for path dependence requires that the alternative technologies are disadvantaged relative to existing technologies. It is this criterion that demonstrates how path dependence is problematic for research. Despite having particular goals for which alternative measures would be better, these measures are not developed or ignored. For example, disadvantage could be due to institutional factors (such as the prevalence of training programs for employees using QWERTY layouts), social factors (such as devaluing the tools created based on testimony from marginalized groups; Wu Reference Wu2023), or factors concerning philosophy of measurement. In the next two sections, I argue that adopting a coherentist view of measurement (section 4) along with current validation practices in psychometrics (section 5) can produce this disadvantage. I conclude that the contemporary coherentists approach is incomplete, not that it is incorrect.

5.1. Content validity

Bastos et al. (Reference Bastos, Celeste, Faerstein and Barros2010) is the only meta-analysis of validation practices concerning subjective racial discrimination scales. They identified twenty-four scales, twenty-one of which assessed content validity and/or convergent validity.

Bastos and colleagues’ (Reference Bastos, Celeste, Faerstein and Barros2010) found that the majority of the items on the scales were developed through literature review (15/24 studies). That is, on the basis of theoretical and empirical literature on subjective experiences of racial discrimination, the relevant dimensions to be measured were identified. Fewer studies developed items based on the population’s understanding of the construct or through empirical investigation (focus group: 6 of 24; interview: 5 of 24; empirical investigation: 5 of 24), which would allow for dimensions not already published in the literature to be included. Once the scale was developed and used with a study population, its content validity can be evaluated. According to Bastos et al. (Reference Bastos, Celeste, Faerstein and Barros2010), only six of the twenty-four scales evaluated content validity using focus groups, pretests, pilot studies, or interviews. These evaluations are more open to including alternative dimensions of the construct. Four of twenty-four scales assessed content validity using literature reviews or expert panels, which is much more likely to evaluate scales relative to the dimensions of the revised construct.

Together the development of items and assessment of content validity means that decisions about which construct dimensions to measure in the first place are propagated in later scales. An early scale is developed using a construct that emphasizes certain dimensions and neglects others. This common for social science concepts that lack precise definitions (Bradburn et al. Reference Bradburn, Cartwright and Fuller2017), but it is also a feature of the coherentist picture that some dimensions of constructs get removed as the construct is refined. By relying on previous research to develop items, this privileges the dimensions chosen early over others regardless of the strength of reasons to focus on those dimensions. In combination with assessing content validity using the dimensions of the revised construct, there is little opportunity to include new dimensions of the construct.

There are a few reasons why some revised construct may be too narrow. First, the revised construct and the scale may ignore dimensions that are relevant for some purposes. Second, the revised construct may differ too significantly from the original theoretical construct, especially in cases in which the construct we seek to measure involves subjective experiences. However, if content validity is not assessed (through interviews, focus groups, and testimony by participants), then researchers may not recognize when the revised construct is too narrow (Cupples Reference Cupples and McClimans2017; McClimans Reference McClimans and McClimans2017) or when there is a mismatch between participants’ and researchers’ understandings of a construct (Cupples Reference Cupples and McClimans2017, 109).

5.2. Convergent validity

Convergent validity is often assessed by examining correlations among the indications of the scales. Sixteen of the twenty-four scales assessed convergent validity, which means they assessed the extent to which scores on the new scale were positively correlated with scores on existing scales (Bastos et al. Reference Bastos, Celeste, Faerstein and Barros2010).

This indicates that researchers take these scales to measure the same phenomenon. By itself, this does not amount to path dependence. However, it may contribute to the extent that construct validity is taken to be an important source of information about scale quality. If high convergent validity is needed for the community to agree that a new scale is valid, then researchers will seek new scales that produce scores similar to existing measures. Thus, this provides another avenue for path dependence.

5.3. Same construct?

Here I have treated discrimination as a single construct with multiple dimensions. This is in line with researchers’ conceptions of discrimination as being measured in different ways (e.g., subjective measures, statistical analyses, audit studies). Could it also be that these different kinds of measures get at distinct constructs? First, we should avoid reifying the measurement process as features of the construct. This principle was important in the debates about indirect measures and implicit attitudes. Subjective measures might be used to measure interpersonal dimensions or institutional dimensions of discrimination. Second, we might ask whether interpersonal and institutional dimensions warrant treatment as different constructs. Some hold that dissociative evidence (concerning some construct dimensions) warrants that a construct be split into distinct constructs (Feest Reference Feest2011). However, there are some reasons to be cautious of this response. First, there is not (yet) evidence of dissociation between the dimensions because there is not (yet) a diversity of measures. Second, dissociation among dimensions can sometimes be useful in construct development that later results in a new unified construct (Bloch-Mullins Reference Bloch-Mullins2022). Proponents of this objection should still accept my arguments that a diversity of measures is needed.