National FFCMH Evaluation Discussion

Congress, representing the population of the country, is responsible for the federal budget and therefore, for selecting which of the thousands of proposed programs (such as the SAMHSA administered Children’s Mental Health Initiative) get funded. For many programs, they mandate (order/direct) that a portion of the overall program budget be used to conduct program evaluation. Those program evaluations are then designed to be implemented across all of the grants (such as the system of care communities funded by SAMHSA’s Children’s Mental Health Initiative). The kinds of questions those evaluations are designed to answer are:

• Does the program create the outcomes it was intended to create?
• Is the program implemented the way it was suppose to be implemented (put into practice)?
• Is this a cost effective way to use the federal budget?

Those served by the program (such as youth and families in the system of care communities) may have additional questions to be answered. Such as:

• Will this work for me?
• How do I know it will work for me?
• If it doesn’t work for me, how will the necessary changes be made so it will work for me?

The Cluetrain Manifesto states: “The questions we ask drive the future.”

What questions would you want answered within a national program evaluation?

Disparity means the condition or fact of being unequal according to http://www.Dictionary.com. For example, a group of people or a population is considered to experience disparity when their need for mental health services is disproportionately unequal. This does not mean they have more people in need of services. It means that a higher percentage of their total population is in need of services than other total populations. Consider a group of Xs compared to a group of Os.

There are 100 Xs and 9 of them need services. That is 9/100, or 9%, of Xs are in need.

There 10 Os and 9 of them need services. That is 9/10, or 90%, of Os are in need.

There is a disparity between the group with 90% of their members in need and the group with only 9% of their members in need.
Families of the populations experiencing that inequality – or disparity -- need to be at the center of the conversation as we enter into an evidence based culture, a culture where the standard of practice is to offer those treatments that have research findings attached that indicate some assurance that the treatments work. Of course, on the face of it, who could disagree with a culture that requires some assurances that what is being done will help?

The issue around whether a particular practice works with all people is a complicated one. Testing it on everyone, in differing communities, with differing social economic status, different ethnic back grounds, circumstances and community norms would involve an extremely high level of burden. So how do we have some degree of confidence that a particular practice will work for our family if we are a family that is “different” than those that the evidence was gathered from?

My family has participated in practices and treatments that were provided because they were the only ones available. They were the only ones available because there was empirical evidence that they worked. That empirical evidence had been gathered from families that were culturally different than my own. Some of those treatments worked, some of them had very harmful long term consequences.
In order to reduce the current disparities we need to be part of the process in the collection of “experience” in our respective communities to demonstrate what works and what doesn’t.