Research

A series of controversial police-involved killings and nationwide protests have recently reinvigorated the study of racial bias in policing. But a fractured interdisciplinary literature presents contradictory claims, and scholars have struggled to reconcile a dizzying array of seemingly incompatible analytic approaches that often rely on implausible and/or unstated assumptions. This confusion arose in part because data constraints have prompted researchers to examine only isolated aspects of the police–civilian encounters they seek to understand — focusing only on traffic stops in one study, or fatal shootings in another — while neglecting the complex, multi-stage nature of these interactions. The result is a conflicting and at times misleading body of evidence. To move toward a scientific consensus, scholars should converge on a common empirical framework that unites these disparate approaches under a shared conceptual umbrella, acknowledges the causal nature of the study of racial bias, accounts for the fundamental limitations of policing data, and yields substantively interpretable results that are useful to policymakers. We present such a framework and demonstrate its capacity to adjudicate conflicting claims, accumulate knowledge, and characterize the severity of one of the most pressing problems of institutional performance of our time.

A recent PNAS study, Johnson et al., investigates the role of race in fatal police shootings. Unlike previous studies which focused on victim race alone, the paper features original data about the race of officers who use deadly force and offers a rare accounting of other shooting attributes that contextualize fatal encounters. Johnson et al. discuss possible “discrimination by White officers,” but conclude racial diversity in police agencies brings limited benefits— a claim cited by major news outlets and in US Congressional testimony, inflaming an already contentious policy debate. Despite the value of this much-needed research, its approach is mathematically incapable of supporting its central claims. In this letter, we clarify the gap between what Johnson et al.’s study asserts and what it actually estimates, as well as the implications of that difference for policymaking and future scholarship on race and policing.

Researchers often lack the necessary data to credibly estimate racial discrimination in policing. In particular, police administrative records lack information on civilians police observe but do not investigate. In this article, we show that if police racially discriminate when choosing whom to investigate, analyses using administrative records to estimate racial discrimination in police behavior are statistically biased, and many quantities of interest are unidentified—even among investigated individuals—absent strong and untestable assumptions. Using principal stratification in a causal mediation framework, we derive the exact form of the statistical bias that results from traditional estimation. We develop a bias-correction procedure and nonparametric sharp bounds for race effects, replicate published findings, and show the traditional estimator can severely underestimate levels of racially biased policing or mask discrimination entirely. We conclude by outlining a general and feasible design for future studies that is robust to this inferential snare.