Matt Franchi
Wendy Ju
ABSTRACT
Large-scale policing data is vital for detecting inequity in police behavior and policing algorithms. However, one important type of policing data remains largely unavailable within the United States: aggregated police deployment data capturing which neighborhoods have the heaviest police presences. Here we show that disparities in police deployment levels can be quantified by detecting police vehicles in dashcam images of public street scenes. Using a dataset of 24,803,854 dashcam images from rideshare drivers in New York City, we find that police vehicles can be detected with high accuracy (average precision 0.82, AUC 0.99) and identify 233,596 images which contain police vehicles. There is substantial inequality across neighborhoods in police vehicle deployment levels. The neighborhood with the highest deployment levels has almost 20 times higher levels than the neighborhood with the lowest. Two strikingly different types of areas experience high police vehicle deployments — 1) dense, higher-income, commercial areas and 2) lower-income neighborhoods with higher proportions of Black and Hispanic residents. We discuss the implications of these disparities for policing equity and for algorithms trained on policing data.
Supplemental Material
Available for Download
- Desmond Ang. 2021. The effects of police violence on inner-city students. The Quarterly Journal of Economics 136, 1 (2021), 115–168.Google Scholar
Cross Ref
- Shamena Anwar and Hanming Fang. 2006. An alternative test of racial prejudice in motor vehicle searches: Theory and evidence. American Economic Review 96, 1 (2006), 127–151.Google ScholarCross Ref
- American Psychological Association 2002. Ethical principles of psychologists and code of conduct. American psychologist 57, 12 (2002), 1060–1073.Google Scholar
- Peter C Austin and Elizabeth A Stuart. 2015. Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies. Statistics in medicine 34, 28 (2015), 3661–3679.Google Scholar
- Brenden Beck. 2020. Policing gentrification: Stops and low–level arrests during demographic change and real estate reinvestment. City & Community 19, 1 (2020), 245–272.Google ScholarCross Ref
- Geoff Boeing. 2017. OSMnx: New Methods for Acquiring, Constructing, Analyzing, and Visualizing Complex Street Networks. Computers, Environment and Urban Systems 65 (Sept. 2017), 126–139. https://doi.org/10.1016/j.compenvurbsys.2017.05.004 arXiv:1611.01890 [physics].Google Scholar
- Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency. PMLR, 77–91.Google Scholar
- Andrew Campbell, Alan Both, and Qian Chayn Sun. 2019. Detecting and mapping traffic signs from Google Street View images using deep learning and GIS. Computers, Environment and Urban Systems 77 (2019), 101350.Google ScholarCross Ref
- Cindy Chang and Ben Poston. 2019. LAPD will drastically cut back on pulling over random vehicles over racial bias concerns. Los Angeles Times (2019).Google Scholar
- Cindy Chang and Ben Poston. 2019. Stop-and-frisk in a car’: Elite LAPD unit disproportionately stopped black drivers, data show. Los Angeles Times (2019).Google Scholar
- Irene Y Chen, Emma Pierson, Sherri Rose, Shalmali Joshi, Kadija Ferryman, and Marzyeh Ghassemi. 2021. Ethical machine learning in healthcare. Annual review of biomedical data science 4 (2021), 123–144.Google Scholar
- Tahiya Chowdhury, Ansh Bhatti, Ilan Mandel, Taqiya Ehsan, Wendy Ju, and Jorge Ortiz. 2021. Towards sensing urban-scale COVID-19 policy compliance in New York City. In Proceedings of the 8th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation. 353–356.Google ScholarDigital Library
- Tahiya Chowdhury, Qizhen Ding, Ilan Mandel, Wendy Ju, and Jorge Ortiz. 2021. Tracking urban heartbeat and policy compliance through vision and language-based sensing. In Proceedings of the 8th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation. 302–306.Google ScholarDigital Library
- Sam Corbett-Davies and Sharad Goel. 2018. The measure and mismeasure of fairness: A critical review of fair machine learning. arXiv preprint arXiv:1808.00023 (2018).Google Scholar
- Sam Corbett-Davies, Emma Pierson, Avi Feller, and Sharad Goel. 2016. A computer program used for bail and sentencing decisions was labeled biased against blacks. It’s actually not that clear. Washington Post 17 (2016).Google Scholar
- Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. 2017. Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd acm sigkdd international conference on knowledge discovery and data mining. 797–806.Google ScholarDigital Library
- Bahar Dadashova, Chiara Silvestri Dobrovolny, and Mahmood Tabesh. 2021. Detecting Pavement Distresses Using Crowdsourced Dashcam Camera Images. (2021).Google Scholar
- Justin Davidson. 2019. Hudson Yards is a billionaire’s fantasy city and you never have to leave — provided you can pay for it.New York Magazine (2019).Google Scholar
- Elizabeth Davis, Anthony Whyde, and Lynn Langton. 2018. Contacts between police and the public, 2015. US Department of Justice Office of Justice Programs Bureau of Justice Statistics Special Report (2018), 1–33.Google Scholar
- Angel Diaz. 2021. Data-driven policing’s threat to our constitutional rights. Brookings Institution (2021).Google Scholar
- Frank Edwards, Hedwig Lee, and Michael Esposito. 2019. Risk of being killed by police use of force in the United States by age, race–ethnicity, and sex. Proceedings of the national academy of sciences 116, 34 (2019), 16793–16798.Google ScholarCross Ref
- Charles R Epp, Steven Maynard-Moody, and Donald P Haider-Markel. 2014. Pulled over: How police stops define race and citizenship. University of Chicago Press.Google Scholar
- Alan Feuer and Azi Paybarah. 2020. Thousands Protest in N.Y.C., Clashing With Police Across All 5 Boroughs. The New York Times (2020).Google Scholar
- David Fleet, Tomas Pajdla, Bernt Schiele, and Tinne Tuytelaars (Eds.). 2014. Computer Vision – ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V. Lecture Notes in Computer Science, Vol. 8693. Springer International Publishing, Cham. https://doi.org/10.1007/978-3-319-10602-1Google Scholar
- Roland G Fryer Jr. 2019. An empirical analysis of racial differences in police use of force. Journal of Political Economy 127, 3 (2019), 1210–1261.Google ScholarCross Ref
- Timnit Gebru, Jonathan Krause, Yilun Wang, Duyun Chen, Jia Deng, Erez Lieberman Aiden, and Li Fei-Fei. 2017. Using deep learning and Google Street View to estimate the demographic makeup of neighborhoods across the United States. Proceedings of the National Academy of Sciences 114, 50 (2017), 13108–13113.Google ScholarCross Ref
- Andrew Gelman, Jeffrey Fagan, and Alex Kiss. 2007. An analysis of the New York City police department’s “stop-and-frisk” policy in the context of claims of racial bias. Journal of the American statistical association 102, 479 (2007), 813–823.Google ScholarCross Ref
- Sharad Goel, Justin M Rao, and Ravi Shroff. 2016. Precinct or prejudice? Understanding racial disparities in New York City’s stop-and-frisk policy. The Annals of Applied Statistics 10, 1 (2016), 365–394.Google ScholarCross Ref
- Felipe Goncalves and Steven Mello. 2021. A few bad apples? Racial bias in policing. American Economic Review 111, 5 (2021), 1406–41.Google ScholarCross Ref
- Jeffrey Grogger and Greg Ridgeway. 2006. Testing for racial profiling in traffic stops from behind a veil of darkness. J. Amer. Statist. Assoc. 101, 475 (2006), 878–887.Google ScholarCross Ref
- Andrea Marie Headley, Stewart J D’Alessio, and Lisa Stolzenberg. 2020. The effect of a complainant’s race and ethnicity on dispositional outcome in police misconduct cases in Chicago. Race and justice 10, 1 (2020), 43–61.Google Scholar
- Elizabeth M Hill and Mary N Monahan. 2011. Naturalistic Observation in Public Settings: Applying for Institutional Review Board Approval. Human Ethology Bulletin (2011).Google Scholar
- Nathalie Japkowicz and Shaju Stephen. 2002. The class imbalance problem: A systematic study. Intelligent data analysis 6, 5 (2002), 429–449.Google ScholarDigital Library
- Jennifer Jenkins, Monika Mathur, John Muyskens, Razzan Nakhlawi, Steven Rich, and Andrew Ba Tran. 2023. Police shootings database 2015-202s3. The Washington Post (2023). https://www.washingtonpost.com/graphics/investigations/police-shootings-database/Google Scholar
- Wendy Ju, Sharon Yavo-Ayalon, Ilan Mandel, Federico Saldarini, Natalie Friedman, Srinath Sibi, JD Zamfirescu-Pereira, and Jorge Ortiz. 2020. Tracking urban mobility and occupancy under social distancing policy. Digital Government: Research and Practice 1, 4 (2020), 1–12.Google ScholarDigital Library
- Jongbin Jung, Sam Corbett-Davies, Ravi Shroff, and Sharad Goel. 2018. Omitted and included variable bias in tests for disparate impact. arXiv preprint arXiv:1809.05651 (2018).Google Scholar
- Sandhya Kajeepeta, Emilie Bruzelius, Jessica Z. Ho, and Seth J. Prins. 2022. Policing the pandemic: estimating spatial and racialized inequities in New York City police enforcement of COVID-19 mandates. Critical Public Health 32, 1 (Jan. 2022), 56–67. https://doi.org/10.1080/09581596.2021.1987387Google ScholarCross Ref
- Allison Koenecke, Andrew Nam, Emily Lake, Joe Nudell, Minnie Quartey, Zion Mengesha, Connor Toups, John R Rickford, Dan Jurafsky, and Sharad Goel. 2020. Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences 117, 14 (2020), 7684–7689.Google ScholarCross Ref
- Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, 2021. Wilds: A benchmark of in-the-wild distribution shifts. In International Conference on Machine Learning. PMLR, 5637–5664.Google Scholar
- Vladimir A Krylov, Eamonn Kenny, and Rozenn Dahyot. 2018. Automatic discovery and geotagging of objects from street view imagery. Remote Sensing 10, 5 (2018), 661.Google ScholarCross Ref
- Benjamin Laufer, Emma Pierson, and Nikhil Garg. Working paper, 2023; ICML Workshop on Responsible Decision Making in Dynamic Environments, 2022. Detecting Disparities in Capacity-Constrained Service Allocations. (Working paper, 2023; ICML Workshop on Responsible Decision Making in Dynamic Environments, 2022).Google Scholar
- Laura and John Arnold Foundation. 2016. Public safety assessment: Risk factors and formula.Google Scholar
- Xiaojiang Li. 2021. Examining the spatial distribution and temporal change of the green view index in New York City using Google Street View images and deep learning. Environment and Planning B: Urban Analytics and City Science 48, 7 (Sept. 2021), 2039–2054. https://doi.org/10.1177/2399808320962511 Publisher: SAGE Publications Ltd STM.Google ScholarCross Ref
- Xiaojiang Li, Chuanrong Zhang, Weidong Li, Robert Ricard, Qingyan Meng, and Weixing Zhang. 2015. Assessing street-level urban greenery using Google Street View and a modified green view index. Urban Forestry & Urban Greening 14, 3 (2015), 675–685. https://doi.org/10.1016/j.ufug.2015.06.006Google ScholarCross Ref
- Kristian Lum and William Isaac. 2016. To predict and serve?Significance 13, 5 (2016), 14–19.Google Scholar
- Eddy Maddalena, Luis-Daniel Ibáñez, and Elena Simperl. 2020. Mapping points of interest through street view imagery and paid crowdsourcing. ACM Transactions on Intelligent Systems and Technology (TIST) 11, 5 (2020), 1–28.Google ScholarDigital Library
- Vashisht Madhavan and Trevor Darrell. 2017. The BDD-Nexar Collective: A Large-Scale, Crowsourced, Dataset of Driving Scenes. Master’s thesis. EECS Department, University of California, Berkeley. http://www2.eecs.berkeley.edu/Pubs/TechRpts/2017/EECS-2017-113.htmlGoogle Scholar
- Benjamin Mueller. 2017. As Troopers Are Diverted, Deployment Data Remains Restricted. (08 2017). https://www.nytimes.com/2017/08/17/nyregion/new-york-state-police-deployment-data.htmlGoogle Scholar
- Benjamin Mueller and Al Baker. 2016. Rift Between Officers and Residents as Killings Persist in South Bronx. The New York Times (2016).Google Scholar
- Benjamin Mueller and Al Baker. 2017. New York Police Urged to Fix Inequities in Deployment of Investigators. The New York Times (2017).Google Scholar
- Xingchao Peng, Ben Usman, Neela Kaushik, Dequan Wang, Judy Hoffman, and Kate Saenko. 2018. VisDA: A Synthetic-to-Real Benchmark for Visual Domain Adaptation. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE, Salt Lake City, UT, USA, 2102–21025. https://doi.org/10.1109/CVPRW.2018.00271Google Scholar
- Emma Pierson, Sam Corbett-Davies, and Sharad Goel. 2018. Fast threshold tests for detecting discrimination. In International conference on artificial intelligence and statistics. PMLR, 96–105.Google Scholar
- Emma Pierson, Camelia Simoiu, Jan Overgoor, Sam Corbett-Davies, Daniel Jenson, Amy Shoemaker, Vignesh Ramachandran, Phoebe Barghouty, Cheryl Phillips, Ravi Shroff, 2020. A large-scale analysis of racial disparities in police stops across the United States. Nature human behaviour 4, 7 (2020), 736–745.Google Scholar
- Michael Pinard. 2018. Predicting more biased policing in Baltimore. The Baltimore Sun (2018).Google Scholar
- PoliceCarWebsite.net. 2023. NYPD Police Cars. http://www.policecarwebsite.net/fc/ny/nypd/nypddiv.htmlGoogle Scholar
- Ben Poston and Cindy Chang. 2019. LAPD searches blacks and Latinos more. But they’re less likely to have contraband than whites. The Los Angeles Times (2019).Google Scholar
- Rajeev Ramchand, Rosalie Liccardo Pacula, and Martin Y Iguchi. 2006. Racial differences in marijuana-users’ risk of arrest in the United States. Drug and alcohol dependence 84, 3 (2006), 264–272.Google Scholar
- Rashida Richardson, Jason M Schultz, and Kate Crawford. 2019. Dirty data, bad predictions: How civil rights violations impact police data, predictive policing systems, and justice. NYUL Rev. Online 94 (2019), 15.Google Scholar
- Greg Ridgeway and John M MacDonald. 2009. Doubly robust internal benchmarking and false discovery rates for detecting racial bias in police stops. J. Amer. Statist. Assoc. 104, 486 (2009), 661–668.Google ScholarCross Ref
- Konstantin Riedl, Sebastian Huber, Maximilian Bömer, Julian Kreibich, Felix Nobis, and Johannes Betz. 2020. Importance of Contextual Information for the Detection of Road Damages. In 2020 Fifteenth International Conference on Ecological Vehicles and Renewable Energies (EVER). IEEE, 1–7.Google Scholar
- Scale.AI. 2023. Scale.AI. https://scale.comGoogle Scholar
- Andrew D Selbst. 2017. Disparate impact in big data policing. Ga. L. Rev. 52 (2017), 109.Google Scholar
- Hao Sheng, Keniel Yao, and Sharad Goel. 2021. Surveilling Surveillance: Estimating the Prevalence of Surveillance Cameras with Street View Data. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. ACM, Virtual Event USA, 221–230. https://doi.org/10.1145/3461702.3462525Google ScholarDigital Library
- Camelia Simoiu, Sam Corbett-Davies, and Sharad Goel. 2017. The problem of infra-marginality in outcome tests for discrimination. (2017).Google Scholar
- Samuel Sinyangwe 2022. The Police Scorecard. (2022). https://policescorecard.org/Google Scholar
- Visual Investigations Team. 2023. Visual investigations of police misconduct. The New York Times (2023). https://www.nytimes.com/spotlight/visual-investigations-police-misconductGoogle Scholar
- Floyd v. City of New York. 2013. 959 F. Supp. 2d 540 - Dist. Court, SD New York. (2013).Google Scholar
- Andrew Thirlwell and Ognjen Arandjelović. 2020. Big data driven detection of trees in suburban scenes using visual spectrum eye level photography. Sensors 20, 11 (2020), 3051.Google ScholarCross Ref
- Victor J. D. Tsai, Jyun-Han Chen, Pei-Shan Tsai, Hsun-Sheng Huang, and Ke-Jhong Chen. 2017. Exploring Traffic Features on Google Street View Images. In Proceedings of the 2017 International Conference on Wireless Communications, Networking and Applications (Shenzhen, China) (WCNA 2017). Association for Computing Machinery, New York, NY, USA, 230–234. https://doi.org/10.1145/3180496.3180638Google ScholarDigital Library
- UltraLytics. 2021. YOLOv5. https://github.com/ultralytics/yolov5/releasesGoogle Scholar
- United States Department of Justice Civil Rights Division and United States Attorney’s Office for Northern District of Illinois. 2017. Investigation of the Chicago Police Department. (2017).Google Scholar
- Vinay Vishnani, Anikait Adhya, Chinmay Bajpai, Priya Chimurkar, and Kumar Khandagle. 2020. Manhole detection using image processing on google street view imagery. In 2020 Third International Conference on Smart Systems and Inventive Technology (ICSSIT). IEEE, 684–688.Google ScholarCross Ref
- Rob Voigt, Nicholas P Camp, Vinodkumar Prabhakaran, William L Hamilton, Rebecca C Hetey, Camilla M Griffiths, David Jurgens, Dan Jurafsky, and Jennifer L Eberhardt. 2017. Language from police body camera footage shows racial disparities in officer respect. Proceedings of the National Academy of Sciences 114, 25 (2017), 6521–6526.Google ScholarCross Ref
- Chien-Yao Wang, Alexey Bochkovskiy, and Hong-Yuan Mark Liao. 2022. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv preprint arXiv:2207.02696 (2022).Google Scholar
- Patricia Warren, Donald Tomaskovic-Devey, William Smith, Matthew Zingraff, and Marcinda Mason. 2006. Driving while black: Bias processes and racial disparity in police stops. Criminology 44, 3 (2006), 709–738.Google ScholarCross Ref
- Benjamin Wilson, Judy Hoffman, and Jamie Morgenstern. 2019. Predictive inequity in object detection. arXiv preprint arXiv:1902.11097 (2019).Google Scholar
- Qian Xie, Dawei Li, Zhenghao Yu, Jun Zhou, and Jun Wang. 2020. Detecting Trees in Street Images via Deep Learning With Attention Module. IEEE Transactions on Instrumentation and Measurement 69, 8 (2020), 5395–5406. https://doi.org/10.1109/TIM.2019.2958580Google ScholarCross Ref
- Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madhavan, and Trevor Darrell. 2020. Bdd100k: A diverse driving dataset for heterogeneous multitask learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2636–2645.Google ScholarCross Ref
- JD Zamfirescu-Pereira, Jerry Chen, Emily Wen, Allison Koenecke, Nikhil Garg, and Emma Pierson. 2022. Trucks Don’t Mean Trump: Diagnosing Human Error in Image Analysis. In 2022 ACM Conference on Fairness, Accountability, and Transparency. 799–813.Google Scholar
- Hui Zhou, Han Wang, Handuo Zhang, and Karunasekera Hasith. 2020. LaCNet: Real-time end-to-end arbitrary-shaped lane and curb detection with instance segmentation network. In 2020 16th International Conference on Control, Automation, Robotics and Vision (ICARCV). IEEE, 184–189.Google ScholarCross Ref
Recommendations
UK-Based Police Officers' Perceptions of, and Role in Investigating, Cyber-Harassment as a Crime
Within the UK, cyber-harassment is a criminal act and perpetrators can be prosecuted if the crime is reported to the police. Cyber-harassment has a detrimental impact and causes psychological distress to victims but due to its online nature, complaints ...
A rigorous approach to formalising the technical investigation stages of cybercrime and criminality within a UK law enforcement environment
As the Internet evolves and continues to become a compelling part of our everyday lives, individuals, communities and nations alike are becoming increasingly exposed to the growing threat of the cybercriminal. The aim of this paper is to widen the ...
Improving Citizen-initiated Police Reform Efforts through Interactive Design: A Case Study in Allegheny County
EAAMO '22: Proceedings of the 2nd ACM Conference on Equity and Access in Algorithms, Mechanisms, and OptimizationIncreasing police accountability in the US has been an issue for decades, but citizen-initiated reform efforts have been less than effective at moving the needle on the issue. Our project aims to provide citizens and community organizers with a better ...
Comments