As a clinical psychologist in the 1980s, David Kraus wanted to provide treatment that helped his patients. But how could he know if the treatment was truly effective?
“We had accountability and quality measurements in other areas, but not so much in health care,” he says. “On a national level, people were calling for ways to demonstrate effectiveness in medicine – to show that what we did on a daily basis was actually helping.”
Kraus began looking for a tool that he could use to assess his patients’ well-being over time. When he couldn’t find one that he liked, he created one.
In 1990, Kraus founded what is now Outcome Referrals to help improve the quality of behavioral health care through outcomes management. His Treatment Outcome Package – TOP – has been used across the country to help patients by directing them to appropriate services and effective therapists. Over the years, he has amassed the world’s largest set of psychiatric outcome data.
“From the beginning, I had the belief that we needed an in-depth questionnaire that would help clinicians get a deeper picture of a patient’s real issues early on,” Kraus says. “And if they filled it out again after a month, it would help us know what’s working, or to refine our treatment. Instead of using hunches, clinicians would be able to use aggregate data to make good predictions about where things were heading.”
With the Annie E. Casey Foundation, The Duke Endowment is supporting efforts to see if Kraus’ assessment tool could be used effectively in child welfare. Annie E. Casey launched a pilot in Ohio; in 2013, the Endowment awarded a $750,000 grant to begin pilots in two North Carolina counties.
Rhett Mabry, president of the Endowment, says the assessment tool will also be introduced for potential use in Mecklenburg County – and South Carolina has expressed interest as well.
“For the Endowment’s Child Care program area, child well-being is our overarching focus as we work in the areas of prevention and early intervention and out-of-home care,” he says. “In child welfare, we believe that an effective assessment is critical to helping vulnerable children receive the best care.”
Kraus, who lives in Massachusetts, talks about his work in the following interview.
Before now, TOP has been used in behavioral health. How does the assessment tool relate to child welfare?
The same kinds of questions resonate in child welfare. Finding out whether a child is resilient, or if they are doing well in school, or whether they have conduct disorder issues – all of those things are highly meaningful in child welfare as well as in behavioral health.
How did you get involved with Annie E. Casey?
In 2011, we published the largest study to date in behavioral health that looks at the strengths and weaknesses of clinicians across the country. We dispelled the myth that a good clinician is a good clinician regardless of who you throw at them. We found that if a patient is randomly assigned to a clinician, he or she has a 50 percent chance of getting good care. Virtually everybody has strengths and weaknesses and you need to pay attention to them when looking for the right person to treat a patient.
About 2 and a half years ago, the Annie E. Casey Foundation heard us talking about that study. They said, this is really interesting, but that’s not how child welfare contracts with its providers. Kids are placed in specific programs or with specific providers, not referred to a specific therapist.
They asked us to mine our database, which had more than 1 million patients in it, to see if what we had discovered in behavioral health relates to child welfare placement.
And then the question was, could you collect this type of data in child welfare. That’s how the first pilot began?
We started the Ohio pilot just over a year ago. We spent more than a year working to adapt the implementation and data collection and reporting processes from behavioral health to make sure it worked for child welfare. We wanted to make sure that caseworkers found that what we were doing was valuable.
What adaptations did you make?
The assessment tool questions haven’t changed at all. It’s the technology, the reporting, who we collect the data from, and how we report it back with a caseworker’s perspective in mind.
In behavioral health, children complete the tool themselves if they are old enough, and that’s it. In child welfare, we’re trying to get multiple perspectives. Most of these kids have been traumatized and, since trauma is very complex, every adult in that kid’s life is going to have a slightly different perspective.
A clinician might have a better understanding of what kind of flashbacks a child is experiencing. A foster parent might have a better idea of how the child’s sleep is being affected by nightmares.
What is happening with the Ohio pilot now?
We’ve trained all of the workers and registered almost all of the kids in the system and we are focusing on getting enough data so that we can build algorithms to make better placement decisions.
Within about 6 months, I think we are going to have a clear picture of what types of kids in Cleveland are being well served. Within a year, I think we are going to be able to start placing kids based on provider strengths, rather than just ‘Where’s the closest open bed.’
What will that look like?
It started with wanting to make better placement decisions based on performance-based contracting. Now, in addition to changing the way child welfare is practiced, the assessment tool’s report will become a cornerstone of most every important decision related to the child. Who should he or she be placed with? What level of care should they receive? Should we be thinking about stepping down from a highly-restrictive level of care to going home?
Tell us about the two North Carolina pilots.
We are now going through the same process in both Wake and Cumberland counties, just at a more rapid pace. We’re working to see if the adaptions we made to make the assessment tool caseworker-friendly in Cleveland will work for the Carolinas. We’re collecting enough data to start building the database.
What needs to happen for the pilot in North Carolina to be successful?
Our goal is to make sure that the tool and reports are refined so that each of the major stakeholders has a reliable understanding of a child’s needs and is able to direct the child’s treatment to the most appropriate services. We hope caseworkers wonder how they got their job done before the tool existed.
Tamika D. Williams
Associate Director, Child Care