Beyond the Black Box: Understanding Juvenile Justice Risk and Needs Assessments

This project used a mixed-methods approach to study how juvenile court judges use juvenile risk and need assessments (JRNAs) in their disposition decisions. By analyzing administrative data from a state juvenile justice system, conducting interviews with juvenile justice researchers, and surveying judges, we studied judicial decision making, the contextual factors shaping judicial decisions, and their impacts on youth outcomes.

Context

Research has consistently shown that youth who are involved in the juvenile justice system are less likely to reoffend when the intensity of the intervention that they receive is aligned with their relative risk for reoffending. Accordingly, juvenile justice systems throughout the nation have adopted research-based JRNA tools that can inform youths’ case plans. These tools—most of which are algorithmically generated—are used by decision makers (including judges and probation officers) to estimate youths’ risk for reoffending, identify factors that may make re-offense more (or less) likely, and identify youth needs.

JRNAs are intended to make decision making fairer and more transparent by relying on data and a consistent set of criteria for evaluation. However, some studies suggest that JRNA tools can also perpetuate and mask racial disparities and other inequities because of improper use by decision makers.

JRNAs do not operate in isolation; their effectiveness depends on how decision makers interpret and apply them. Most existing research on JRNAs primarily concentrates on evaluating the accuracy and fairness of the algorithms these tools employ, but this body of research often overlooks a crucial aspect: how decision makers use JRNAs. In practice, JRNAs are tools utilized by justice system officials, like juvenile court judges, to guide their decision-making processes. The main goal of our study was to bridge this knowledge gap in current JRNA research by providing essential insights into how judges use JRNAs to make decisions.

Approach

We aimed to better understand when and why juvenile court judges use JRNAs to guide their disposition decisions and how the use (or non-use) of this research-based tool affects youth outcomes. To do so, we collected and analyzed a variety of data sources:

• Administrative data from a large southern state, including information on juvenile court intakes, risk scores from the Youth Assessment and Screening Instrument (YASI), and disposition outcomes
• Geographic indicators from the County Health Ranking and Roadmaps (CHR&R) datasets
• Surveys from juvenile court judges across multiple states
• Key informant interviews with justice researchers

We used these sources of data to answer four core questions:

• How often do judges select dispositions that align with youths’ JRNA risk levels?
• How do judges’ use of JRNAs vary by youth characteristics, youth offending history, and community characteristics?
• What are judges’ perceptions of JRNAs?

Funder

National Science Foundation: Award #2244705

Core Team Members

• Principal investigator: Sarah Kelley
• Project director: Jennifer Widstrand
• Staff: Claire Kelley, Valerie Martinez, Kristine Chan, Ja’Chelle Ball

External Consultants

• Amanda Heitkamp, Troy University
• Trent Buskirk, Old Dominion University

Acknowledgements

• Kelly Murphy, Edwin Crockett