
Thoughts for the Journey — Data with Soul
There’s a moment in many human-service programs when everything appears to have gone right.
The assessment is complete.
The services were delivered.
The goals were documented.
The participant met the requirements.
The case is closed.
Successful completion.
And in many ways, it is a success. People worked hard. Resources were invested. Requirements were met.
But there’s a question we don’t always stay around long enough to answer.
What actually lasts after the program ends?
When Completion Becomes the Outcome
Public systems need accountability.
They need to know how many people were served, whether services were delivered, how resources were spent, and whether program requirements were met.
Those are legitimate responsibilities.
But somewhere along the way, completing the process can become confused with achieving the purpose.
A young person exits foster care.
Someone graduates from a treatment program.
A youth completes diversion requirements.
The records reflect progress.
But what happens when the organization is no longer involved?
That’s where another kind of accountability begins.
What the Research Tells Us
Research across human-service systems raises questions about what happens after formal services end.
Consider young people transitioning out of foster care.
A study following young people aging out of foster care in three Midwestern states found that an estimated 31% to 46% experienced homelessness at least once by age 26.
These were young people who had reached the point of leaving the foster care system.
But leaving care and having the conditions for a stable adult life are not necessarily the same thing.
What happens to housing, relationships, education, and support when the structure surrounding a young person changes?
And how do we know whether preparation for adulthood has become something the young person can actually carry forward?
Now consider drug treatment courts.
A 2018 study examining 534 participants found that 37.6% of program graduates experienced an arrest, conviction, or incarceration within two years after completing the program.
That finding deserves attention.
But so does another: among participants who did not successfully complete the program, the figure was 95.3%.
Completion was associated with substantially better outcomes.
That is progress worth recognizing.
Yet it also leaves us with a question.
What might have made the difference for graduates who did not return to the justice system?
And what additional conditions might have supported those who did?
These are not questions we can answer from completion rates alone.
There Is Good News Worth Examining
Research into juvenile diversion offers another perspective.
A study of 933 young people participating in a community-based diversion program found that the vast majority successfully completed their requirements and avoided rearrest within three years.
That matters.
It reminds us that programs can accomplish more than procedural compliance. Some approaches are associated with meaningful outcomes that extend beyond participation.
But shouldn’t that make us even more curious?
What contributed to those outcomes?
What did young people gain during their participation?
What relationships, resources, understanding, or opportunities continued after the program ended?
And just as importantly, how could we learn from those results without assuming that the same approach will work for everyone?
The purpose of examining outcomes isn’t simply to identify where programs fall short. It’s also to discover what makes progress possible—and what allows it to continue.
The Difference Between Services Delivered and Progress Sustained
I don’t believe most people enter human services simply wanting to complete paperwork.
They want to make a meaningful difference.
And I don’t believe most organizations deliberately overlook what happens to people after their involvement ends.
But our systems are often structured around what can be documented during a defined period of service.
A requirement is completed.
A service is delivered.
A goal is marked achieved.
And those activities become the evidence of success.
Yet the person who participated may still be trying to understand how everything they were required to do connects to the life they want for themselves.
Did they gain greater clarity about their choices?
Can they recognize the resources available to them and understand how to use them?
Do they have relationships they can turn to when circumstances change?
Were they given an opportunity to imagine a future beyond the immediate requirements of the program?
Or did they simply learn how to successfully navigate the program while they were enrolled?
There is a difference between learning how to complete a program and developing the understanding and ownership needed to navigate what comes next.
That distinction deserves our attention.
A Different Kind of Accountability
This isn’t about expecting human-service organizations to control every outcome.
They can’t.
Circumstances change. Resources disappear. Housing becomes unaffordable. Relationships are disrupted. New challenges emerge.
And returning for services doesn’t automatically mean someone—or a program—has failed.
Sometimes returning is exactly the right decision.
But those realities shouldn’t prevent us from examining whether the conditions we create support progress beyond the period we’re funded to provide services.
What if we looked beyond the day of discharge?
What if we followed up six months later? A year later?
Not simply to ask whether someone returned to the system, but to understand what continued.
What worked?
What didn’t?
What did people carry forward?
What obstacles emerged that our programs hadn’t anticipated?
And perhaps most importantly:
How do the people who participated define the progress they’ve made?
Their answers might tell us things our existing performance measures don’t capture.
They might also reveal what our programs are doing exceptionally well.
The Part Worth Reconsidering
Accountability is not simply demonstrating that we did what we said we would do.
It is also examining whether what we did contributed to the purpose we intended to achieve.
That requires looking at both organizational performance and the lived outcomes of the people our systems serve.
Neither should replace the other.
We need both.
Because understanding what lasts may require us to reconsider what we count, when we count it, and whose definition of progress informs our conclusions.
A completed assessment tells us something.
A successful discharge tells us something.
A participant meeting every requirement tells us something.
But none of those measures, by itself, tells us what will remain after the program ends.
Try This
If you are responsible for designing, funding, delivering, or evaluating programs, consider three questions:
- What do we currently measure to determine whether someone has successfully completed our program?
- What do we know about their circumstances, choices, and progress after they leave?
- What might we discover if we asked participants what they were actually able to carry forward?
These aren’t questions intended to diminish what organizations accomplish.
They’re an invitation to understand those accomplishments more fully.
Closing Reflection
Perhaps the question isn’t whether people successfully completed our programs.
Perhaps it’s whether our programs equipped them to continue moving toward the future they want after they left.
Because in the end, completing every requirement is not the same as creating lasting results.
When every box is checked, what actually lasts?
Research and Practice
Foster care transitions
Dworsky, A., Napolitano, L., & Courtney, M. (2013). Homelessness During the Transition From Foster Care to Adulthood. American Journal of Public Health.
Research following young people aging out of foster care in three Midwestern states estimated that 31%–46% experienced homelessness at least once by age 26.
Extended foster care in California
Courtney, M. E., Okpych, N. J., & Park, S. (2021). Impacts of Extended Care on Youth Outcomes Two Years after Foster Care Has Ended. Chapin Hall.
California research found that additional time in extended foster care was associated with better outcomes in areas including education, employment, housing stability, and social support. This provides an important example of continued services contributing to better outcomes after care ends.
Drug treatment court outcomes
Shannon, L. M., Jackson Jones, A., Newell, J., & Payne, C. (2018). Examining Individual Characteristics and Program Performance to Understand Two-Year Recidivism Rates Among Drug Court Participants: Comparing Graduates and Terminators. International Journal of Offender Therapy and Comparative Criminology.
The study examined post-program recidivism among 534 drug court participants and found substantially lower rates among graduates than noncompleters, although some graduates still experienced later justice-system involvement.
Juvenile diversion outcomes
NeMoyer, A., Gale-Bentz, E., Goldstein, N. E. S., & Harvey, L. P. (2020). Factors Associated With Successful Completion of a Community-Based, Postarrest Juvenile Diversion Program and Subsequent Rearrest. Crime & Delinquency.
The researchers examined 933 participants. Most successfully completed the program and avoided rearrest within the three-year follow-up period.
Donna Ewing Marto
LUV Solutions
Thoughts for the Journey
Part of the Data with Soul series
If this message resonated with you, please share it with someone who may need it today.