A couple weeks ago, we posted an introduction to the Risk Assessment Instrument (RAI) used by probation officers to make detention determinations. We were left with a question: does the RAI accurately predict risk to miss court or reoffend before the first court appearance, in a way that will help officers make the right decisions at a crisis moment? Juvenile Probation asked the same question, and contracted with a neutral evaluator to assess the tool. Today, we’re going to dive into that evaluation, completed by the University of Nebraska at Omaha’s Juvenile Justice Institute (JJI).
The JJI researchers looked at all intake screening decisions between September 2013 and August 2014. They asked:
- When a child is released after being scored on the RAI, does she appear for her next court date?
- When a child is released after being scored on the RAI, does she reoffend prior to her next court date?
- Does the detention decision made by the officer match the recommendation of the RAI tool, based on the child’s score?
- How often are officers “overriding” the tool, and what are the reasons listed for the override?
The hope for the study was to validate the tool: essentially, prove that if officers use it correctly, they are more likely to make the right decisions about whether to detain, implement some other safety plan, or release a child home without any restrictions. Instead, the researchers found what we have roughly divided into three categories: the Good, the Bad, and the Unknown.
The Good:
The first positive takeaway from the study is simply that it happened, and we know about it. A recent report by the Inspector General for Child Welfare highlighted concerns about transparency in the Probation Administration, but noted this evaluation and its public release as a counter-example of Probation committing and investing in data-driven accountability to achieve better results for kids.
Even better, the answers to the study’s first questions — did released youth attend their next court date, without committing further offenses? — were extremely promising. JJI found that 93.3% of youth who were released did, in fact, show up for their next court appearance. 91.1% of those released had no new law violation prior to the next scheduled hearing — and that’s counting fairly minor, adolescent behaviors like running away. Nearly all youth screened away from detention are coming to court, without further offense.
That’s the good news. Now for…
The Bad:
It doesn’t seem like intake officers trust the tool when it comes to release, unless the child is scoring very low indeed. If you recall from our introductory post, the tool recommends scores of 0-5 be released without any restriction on liberty, 6-9 be released with some kind of intervention (home with a safety plan or an ankle bracelet, or to an emergency shelter instead of the detention center), 10-11 be sent to staff-secure detention if available, and 12+ be securely detained.
The average RAI score during the period studied was 7.74 — suggesting most youth should have been released to the community with some intervention in place. However, officers “overrode” the tool 45% of the time — almost always upward to detention. The average score of all youth released (with or without interventions) was 4.94, a score that the tool recommends go home without any restriction. The average score of all youth detained was 9.19, a score that the tool recommends be screened away from detention with some kind of intervention.Thus, of 1,221 youth who scored for release, only 621 were released. Based on their RAI scores, 578 youth could have been released but were detained instead. The total number of youth detained was 1,191, meaning nearly half (48%) of the youth who were detained between September 2013 and August 2014 actually scored to be released.
The study tracked officers’ reasons for overriding the RAI tool. The reason most juveniles (65,8%) were detained on an override was “other”: in this category, officers most often noted concerns that the child might be a flight risk. The second largest category (11.7%) was “parent not willing to take the juvenile home.”
The Unknown:
So where does this leave us?
We have some good numbers showing that children who are released instead of detained are coming to court without reoffending. However, because of the high override rate, those youth represent the very lowest-risk category. Ultimately, JJI couldn’t formally validate the tool, because there wasn’t enough consistency in application. We don’t know what would have happened with those youth who scored for release but were detained instead.
We also don’t have RAI scores for a large population of youth who are detained not for new law violations, but for technical violations of their probation.
The Takeaway:
It is good public policy for an agency to take on external evaluation and ensure its public release. This kind of data-driven thinking is what will make ultimately change for our youth, because we need to know where we are in order to know where we need to go. We’ve known for some time that our numbers of detained youth are too high. This analysis, disheartening though some of the findings may be, lends to our knowledge of what is systemically happening to inflate those numbers. Based on what we see from the RAI evaluation, we can confidently propose:
-Even without validation, Probation should grow internal confidence in the tool, encouraging officers to trust the scores, particularly for medium risk youth. The high rate of released youth attending court and refraining from reoffense are terrific results that can be built on with greater trust in the tool. Moreover, we hope Probation will consider expanding use of the tool to (or doing a similar analysis of) the population of youth entering detention on probation violations.
-Probation should consider an expanded runaway analysis for those children who are being overridden as a “flight risk.” Not all runners look alike, and there should be a way to capture the difference between a youth who takes off to California on his own, and a youth who gets bored and heads down the street to his grandma’s house without explicit permission.
-On this note, if “flight risk” is contributing to a high number of overrides into detention, this likely reflects a gap in our service array. We shouldn’t have to lock children up to keep them safe.
Post a comment