A new tool
DeVylder et al. (Reference DeVylder, Oh, Nam, Sharpe, Lehmann and Link2016) develop and implement a quantitative survey of police–public encounters, which is useful in assessing the exposure of a population to victimisation by police. They validate their survey in a sample taken from four large metropolitan cities (Baltimore, New York, Philadelphia and Washington DC). Their survey is especially valuable in that it extends research on the impact of police behaviour away from a focus strictly on the use of lethal force and/or physical violence (e.g. Ross, Reference Ross2015; Fryer, Reference Fryer2016; Miller et al. Reference Miller, Lawrence, Carlson, Hendrie, Randall, Rockett and Spicer2017) and into other domains – psychological and emotional outcomes – with relevance to public health and well-being. Specifically, they investigate the extent to which the public report positive encounters with police and the extent to which they report being the victim of: physical violence with and without a weapon, sexual victimisation (i.e., inappropriate contact and public strip searches) or psychological victimisation (i.e., threats, inappropriate stops and/or use of slurs) during encounters with police, or neglect by police (i.e., failure to respond when called). Further, they investigate if there are correlations between these measures and psychological and emotional outcomes. The survey gains credibility through its balance, in that it can allow researchers interested in locally resolved analysis of police behaviour (Ross, Reference Ross2016) to study where communities feel that police are engaging in appropriate behaviour and where community reports suggest otherwise.
Research tools like this one, if applied more broadly across the American geography, may help to shift public and academic dialogue away from overly-generalised narratives about police behaviour, and towards more targeted explorations of the specific areas with high absolute levels of reported police victimisation and/or high levels of racial disparities in reported police victimisation. It is possible that many police departments and police unions might engage in less defensive responses to criticism (i.e., like the Santa Clara police union threatening to stop providing security at the stadium where the San Francisco 49ers play, in response to the constitutionally protected speech of Colin Kaepernick (Lopez, Reference Lopez2016)), if criticism of police practices is directed at the specific locations where finely-resolved quantitative data can clearly demonstrate problems. As the data provided by DeVylder et al. (Reference DeVylder, Oh, Nam, Sharpe, Lehmann and Link2016, Table 2) show, the majority of in-sample respondents (even within each race/ethnic group) report having positive encounters with police relative to each of the other response categories. This fact can be acknowledged and celebrated while still noting, as do the authors, that when police victimisation does occur, it disproportionately impacts people of colour and queer people (DeVylder et al. Reference DeVylder, Oh, Nam, Sharpe, Lehmann and Link2016). Better diagnostic tools for measuring exposure of communities and sub-communities to police victimisation should be welcomed by all sides, as they will help to identify cases in which there are lessons to be learned about good policing practices, as well as those cases where interventions are needed – we need the localised ability to identify cases to emulate as well as those needing correction.
Limitations and extensions
While DeVylder et al. (Reference DeVylder, Oh, Nam, Sharpe, Lehmann and Link2016) develop a new and useful tool for studying police victimisation, their study has some key limitations that the authors already acknowledge thoroughly; minor modifications to their methodology, however, might enhance the insights to be gained from future implementation of the survey. I detail some proposed changes below.
-
(1) Their findings concerning population-level racial disparities in police victimisation add another brick in the wall of a growing literature documenting such effects, but their findings are affected by many of the same limitations as previous studies. For instance, the extent to which racial disparities in population-level outcomes are due to racially motivated malfeasance on the part of a sub-set of police officers or simply geographic variation in crime rates or other factors is not immediately clear from their results. There is a public health issue here regardless of cause, but knowing how to attenuate the problem depends on better identifying the causes.
A simple extension of their methodology might be able to address issues that few prior studies of police victimisation have been able to. Specifically, their survey methodology could be modified, such that if respondents report unusually positive or unusually negative encounters with police, then those respondents would be flagged for more detailed qualitative interviews by the research team. These follow-up interviews would, of course, only provide the civilian's perspective on the incident, but even so, they would likely provide a richer understanding of the causes of racial differences in exposure to police victimisation than the quantitative data alone.
-
(2) The direction of causality between exposure to police victimisation and psychological distress is unclear. Again, the authors correctly point out that they demonstrate a public health problem, regardless of the direction of causality. Nonetheless, their survey might be better able to speak to causality if they add one or two basic questions to their distress survey. For example, ‘If you indicated experiencing positive levels of psychological distress, what do you feel is the primary cause of this distress?’ or, more directly, ‘Would you say that a negative encounter with police was a primary cause of the psychological distress symptoms that you have reported?’ (Note that these particular examples might be poorly thought-out, but give some idea as to what the questions might accomplish – survey design experts will surely have better formulations).
-
(3) Their study currently has a limited geographic reach, and previous work has shown that levels of police victimisation (or excellence) can vary strongly by county. This, of course, is not a flaw in the original study, as validation of the methodology in a limited sample is an essential first step to this line of research. A wider, equally rigorous (in terms of sample size per county and representativeness of the sample) implementation of this study across US counties, however, would be a much welcomed piece of research. Studies like Ross (Reference Ross2015) have identified counties with disproportionately high levels of lethal force against unarmed black relative to unarmed white civilians. It would be very interesting to see the DeVylder et al. (Reference DeVylder, Oh, Nam, Sharpe, Lehmann and Link2016) methodology, modified as suggested above, applied in these counties and in counties with little evidence of racial disparities in police shootings.
While use of a weapon or physical force is normally justified by police officials on the grounds of officer self-defence (sometimes rightly and sometimes wrongly), use of racial slurs or sexual humiliation by police has no credible justification on the grounds of officer self-defence. This raises a question: do small geographic areas with disproportionately high rates of racial disparities in police shootings of unarmed civilians also show disproportionately high rates of racially motivated psychological or sexual victimisation that serves no purpose in terms of community protection or officer self-defence? If so, such data might provide reason to believe that racial animus rather than response to crime might be an important driver of racial disparities in police use of force in that area. The methods introduced by DeVylder et al. (Reference DeVylder, Oh, Nam, Sharpe, Lehmann and Link2016) will allow researchers to conduct such tests.
-
(4) Finally, DeVylder et al. (Reference DeVylder, Oh, Nam, Sharpe, Lehmann and Link2016) present only fully pooled analyses of the data, and do not seem to have publicly released the raw data, statistical methodology, or code used in their analysis. Open statistical methods and data are essential to the scientific process – others in the field should be able to replicate and review analyses to check for errors and robustness to alternative model specifications. Additionally, if data are open, it facilitates new analyses by others that the authors themselves may not have initially considered. The authors do state that: ‘The complete data set is available on request from the principal investigator and corresponding author.’ But the Center for Open Science states that the standard for open data should be that: ‘Digitally-shareable data are publicly available on an open-access repository. The data must have a persistent identifier and be provided in a format that is time-stamped, immutable and permanent.’
Although this study carries much value as presented, there are greater analytical and policy advantages to a sharper focus on fine-scale variation in police victimisation. For example, we would learn much more from analysing how police victimisation is structured by geography, than by analysing the central tendency across a few large cities. The potential of the ecological inference fallacy looms large over most published analyses of police victimisation – including my own. Part of the value in this work by DeVylder et al. (Reference DeVylder, Oh, Nam, Sharpe, Lehmann and Link2016) is that their methods produce individual-level data; while variables like ‘crime rate’ will always be aggregate-level variables, the data produced by DeVylder et al. (Reference DeVylder, Oh, Nam, Sharpe, Lehmann and Link2016) should allow analysis at a much finer scale than the county-level data that previous research has focused on in the past. If zip-code level indicator variables are available, the literature would benefit from a multi-level reanalysis of these data.
Because of the value of this work and the importance of the data collected herein, I think it would be a benefit to the field to see this data set and all of the survey tools placed in a GitHub repository. Then, anybody who feels so inclined could easily reanalyse the data, extend this initial analysis, or implement the same study in new cities.
A new tool
DeVylder et al. (Reference DeVylder, Oh, Nam, Sharpe, Lehmann and Link2016) develop and implement a quantitative survey of police–public encounters, which is useful in assessing the exposure of a population to victimisation by police. They validate their survey in a sample taken from four large metropolitan cities (Baltimore, New York, Philadelphia and Washington DC). Their survey is especially valuable in that it extends research on the impact of police behaviour away from a focus strictly on the use of lethal force and/or physical violence (e.g. Ross, Reference Ross2015; Fryer, Reference Fryer2016; Miller et al. Reference Miller, Lawrence, Carlson, Hendrie, Randall, Rockett and Spicer2017) and into other domains – psychological and emotional outcomes – with relevance to public health and well-being. Specifically, they investigate the extent to which the public report positive encounters with police and the extent to which they report being the victim of: physical violence with and without a weapon, sexual victimisation (i.e., inappropriate contact and public strip searches) or psychological victimisation (i.e., threats, inappropriate stops and/or use of slurs) during encounters with police, or neglect by police (i.e., failure to respond when called). Further, they investigate if there are correlations between these measures and psychological and emotional outcomes. The survey gains credibility through its balance, in that it can allow researchers interested in locally resolved analysis of police behaviour (Ross, Reference Ross2016) to study where communities feel that police are engaging in appropriate behaviour and where community reports suggest otherwise.
Research tools like this one, if applied more broadly across the American geography, may help to shift public and academic dialogue away from overly-generalised narratives about police behaviour, and towards more targeted explorations of the specific areas with high absolute levels of reported police victimisation and/or high levels of racial disparities in reported police victimisation. It is possible that many police departments and police unions might engage in less defensive responses to criticism (i.e., like the Santa Clara police union threatening to stop providing security at the stadium where the San Francisco 49ers play, in response to the constitutionally protected speech of Colin Kaepernick (Lopez, Reference Lopez2016)), if criticism of police practices is directed at the specific locations where finely-resolved quantitative data can clearly demonstrate problems. As the data provided by DeVylder et al. (Reference DeVylder, Oh, Nam, Sharpe, Lehmann and Link2016, Table 2) show, the majority of in-sample respondents (even within each race/ethnic group) report having positive encounters with police relative to each of the other response categories. This fact can be acknowledged and celebrated while still noting, as do the authors, that when police victimisation does occur, it disproportionately impacts people of colour and queer people (DeVylder et al. Reference DeVylder, Oh, Nam, Sharpe, Lehmann and Link2016). Better diagnostic tools for measuring exposure of communities and sub-communities to police victimisation should be welcomed by all sides, as they will help to identify cases in which there are lessons to be learned about good policing practices, as well as those cases where interventions are needed – we need the localised ability to identify cases to emulate as well as those needing correction.
Limitations and extensions
While DeVylder et al. (Reference DeVylder, Oh, Nam, Sharpe, Lehmann and Link2016) develop a new and useful tool for studying police victimisation, their study has some key limitations that the authors already acknowledge thoroughly; minor modifications to their methodology, however, might enhance the insights to be gained from future implementation of the survey. I detail some proposed changes below.
(1) Their findings concerning population-level racial disparities in police victimisation add another brick in the wall of a growing literature documenting such effects, but their findings are affected by many of the same limitations as previous studies. For instance, the extent to which racial disparities in population-level outcomes are due to racially motivated malfeasance on the part of a sub-set of police officers or simply geographic variation in crime rates or other factors is not immediately clear from their results. There is a public health issue here regardless of cause, but knowing how to attenuate the problem depends on better identifying the causes.
A simple extension of their methodology might be able to address issues that few prior studies of police victimisation have been able to. Specifically, their survey methodology could be modified, such that if respondents report unusually positive or unusually negative encounters with police, then those respondents would be flagged for more detailed qualitative interviews by the research team. These follow-up interviews would, of course, only provide the civilian's perspective on the incident, but even so, they would likely provide a richer understanding of the causes of racial differences in exposure to police victimisation than the quantitative data alone.
(2) The direction of causality between exposure to police victimisation and psychological distress is unclear. Again, the authors correctly point out that they demonstrate a public health problem, regardless of the direction of causality. Nonetheless, their survey might be better able to speak to causality if they add one or two basic questions to their distress survey. For example, ‘If you indicated experiencing positive levels of psychological distress, what do you feel is the primary cause of this distress?’ or, more directly, ‘Would you say that a negative encounter with police was a primary cause of the psychological distress symptoms that you have reported?’ (Note that these particular examples might be poorly thought-out, but give some idea as to what the questions might accomplish – survey design experts will surely have better formulations).
(3) Their study currently has a limited geographic reach, and previous work has shown that levels of police victimisation (or excellence) can vary strongly by county. This, of course, is not a flaw in the original study, as validation of the methodology in a limited sample is an essential first step to this line of research. A wider, equally rigorous (in terms of sample size per county and representativeness of the sample) implementation of this study across US counties, however, would be a much welcomed piece of research. Studies like Ross (Reference Ross2015) have identified counties with disproportionately high levels of lethal force against unarmed black relative to unarmed white civilians. It would be very interesting to see the DeVylder et al. (Reference DeVylder, Oh, Nam, Sharpe, Lehmann and Link2016) methodology, modified as suggested above, applied in these counties and in counties with little evidence of racial disparities in police shootings.
While use of a weapon or physical force is normally justified by police officials on the grounds of officer self-defence (sometimes rightly and sometimes wrongly), use of racial slurs or sexual humiliation by police has no credible justification on the grounds of officer self-defence. This raises a question: do small geographic areas with disproportionately high rates of racial disparities in police shootings of unarmed civilians also show disproportionately high rates of racially motivated psychological or sexual victimisation that serves no purpose in terms of community protection or officer self-defence? If so, such data might provide reason to believe that racial animus rather than response to crime might be an important driver of racial disparities in police use of force in that area. The methods introduced by DeVylder et al. (Reference DeVylder, Oh, Nam, Sharpe, Lehmann and Link2016) will allow researchers to conduct such tests.
(4) Finally, DeVylder et al. (Reference DeVylder, Oh, Nam, Sharpe, Lehmann and Link2016) present only fully pooled analyses of the data, and do not seem to have publicly released the raw data, statistical methodology, or code used in their analysis. Open statistical methods and data are essential to the scientific process – others in the field should be able to replicate and review analyses to check for errors and robustness to alternative model specifications. Additionally, if data are open, it facilitates new analyses by others that the authors themselves may not have initially considered. The authors do state that: ‘The complete data set is available on request from the principal investigator and corresponding author.’ But the Center for Open Science states that the standard for open data should be that: ‘Digitally-shareable data are publicly available on an open-access repository. The data must have a persistent identifier and be provided in a format that is time-stamped, immutable and permanent.’
Although this study carries much value as presented, there are greater analytical and policy advantages to a sharper focus on fine-scale variation in police victimisation. For example, we would learn much more from analysing how police victimisation is structured by geography, than by analysing the central tendency across a few large cities. The potential of the ecological inference fallacy looms large over most published analyses of police victimisation – including my own. Part of the value in this work by DeVylder et al. (Reference DeVylder, Oh, Nam, Sharpe, Lehmann and Link2016) is that their methods produce individual-level data; while variables like ‘crime rate’ will always be aggregate-level variables, the data produced by DeVylder et al. (Reference DeVylder, Oh, Nam, Sharpe, Lehmann and Link2016) should allow analysis at a much finer scale than the county-level data that previous research has focused on in the past. If zip-code level indicator variables are available, the literature would benefit from a multi-level reanalysis of these data.
Because of the value of this work and the importance of the data collected herein, I think it would be a benefit to the field to see this data set and all of the survey tools placed in a GitHub repository. Then, anybody who feels so inclined could easily reanalyse the data, extend this initial analysis, or implement the same study in new cities.