Integrating Data Instruments into Clinical Practice

One issue that MHCD has looked at recently in improving care of mental health consumers, is around the use of data instruments that Evaluation and Research Department uses to collect information on consumer recovery, and how to translate this back to the consumers, resulting in more insight about their treatment progress?  The Evaluation and Research Department uses an instrument called the Consumer Recovery Marker (CRM) and the Recovery Marker Index (RMI).  The CRM is a 15 item survey, filled out by clients and measures “active growth/orientation, hope, symptom interference, sense of safety and social networks” and is described as “consumer’s perception of their mental health recovery” (Deroche, Olmos, Hester, McKinney 2007: 1).  The RMI is a 7 item survey filled out by clinicians that includes such items as “employment, education/learning, active/growth orientation, symptom interference, engagement, housing” and substance use (added since inception), and is described as measuring “indicators usually associated with individual’s recovery, but are not necessary for recovery” (Deroche, et al. 2007: 1).  In combination, these two instruments allow for multiple perspectives on the consumer’s recovery, as well as collect information to make evaluation possible, as well as translating progress back to MHCD’s various shareholders.  The Evaluation and Research Department also utilizes a tool, the Recovery Profile (RP), which combines all the data collected from the CRM and RMI and puts them in an interpretable form, through use of line and bar graphs, showing averages of all scores, etc.  The question that MHCD faces is how to provide this information to clinicians, as well as consumers, so that it can be used to make clinical decisions around consumer treatment, as well as offering insight to the consumer about their recovery process?  In considering this issue, literature review has shown a number of factors that come into play.

Dilemma for Clinician and Organization

Another perspective is to consider the dilemma that clinicians feel, as a result of data instruments being introduced into clinical practice, which represents a switch to more standardized, or evidence based practices.  An article by Broom, Adams and Tovey (2009) looks at this issue within the healthcare and in particular, oncology practice.  This article describes using evidence based practices within medicine and states the challenge is to adopt these principles, while still maintaining “professional autonomy, clinical judgment, and therapeutic integrity” (Broom et al. 2009: 192).  I think some of clinician fears around using instruments, is that it would minimize some of their intuitive experiences with the consumers, as well as doesn’t allow for their unique talents as clinicians to shine through.  In other words, with the mental health field, a lot of a consumer’s story is shared through narrative forms, treatment plans, intakes, histories, etc. and so there might be some hesitance initially in how that can be translated into more quantitative forms, or using data instruments.   From this study, it was found that executive management was more likely to be in support of using evidence based medicine as it minimizes clinician error and is more science, objective based, and clinicians were more in opposition as they felt it takes away from some of their expertise, uniqueness as individuals (Broom et al. 2009).  The challenge is to try to find a balance, where the clinician’s unique talents can be represented and acknowledged, as well as having more information at the clinician’s disposal, in making decisions around the consumer’s treatment.

Another aspect of this dilemma is to consider the strain placed on an organization in trying to satisfy reporting needs of its stakeholders, as well as using that data in a way that improves its programs.  Carman (as cited by Hoole and Patterson 2008) reports that 65% of nonprofits engage in formal program evaluation, 95% report to their boards and 90% experience site visits from their funders (3).  The problem brought up by Hoole and Patterson (2008) in regards to this is that most of data is just collected, not used to actually improve programs.  This is most likely due to conflicting shareholder demands, as well as having support and funding for data collection, being able to translate this importance to managers, as well as staff across the organization (Hoole and Patterson 2008).  With nonprofit organizations, depending on funders, their outcome goals can often reflect interests of the shareholders, and thus are not intertwined with mission of the organization (Hoole and Patterson 2008).  Basically, the answer to this for Hoole and Patterson (2008) are for the nonprofit to work more with the shareholders on integrating outcome requirements with that of their own internal mission or goal statements.  This holds true for MHCD, as well in that they currently are making efforts to integrate data already being collected to satisfy Medicaid funding, or other state, private needs, in a way that can be translated to clinicians and used in clinical practice and making decisions around client care.

Structure and Funding

An article by Carman and Fredericks (2009) describes that how successful evaluation is implemented depends on “autonomy, internal structures and external relations, leadership styles and maturity” (Carman and Fredericks 2009: 3).  Carman and Fredericks also identify the Executive Director as the key into how research and evaluation is implemented into non-profits (Carman and Fredericks 2009).  This study uses three different clusters of non-profits and finds that those that are primarily funded by government, Medicaid, public funds have fewer problems of support and funding for research and evaluation (Carman and Fredericks 2009).  The following is a breakdown of funding for MHCD and was taken from the “MHCD 2009: Report to the Community” and covers the fiscal year concluded June 30, 2009.

Source                                                            Amount                                  Percentage

Medicaid                                             $23,925,630                            44.4

State of Colorado                               $12,945,194                            24.0

Client, Third Party, Pharmacy            $8,826,030                              16.4

Contracts and Grants                          5,632,041                                10.5

Interest, Rent,             Other                           $1,604,777                              3

Medicare                                             $498,286                                 0.9

Public Support                                                $444,236                                 0.8

Some other findings were the larger the organization, more likely to identify staff resistance as a problem in evaluation, data collection (Carman and Fredericks 2009).  Younger organizations were found to have more technical assistance issues with evaluation and organization; with connections to housing or community development, there are fewer problems with implementation and design (Carman and Fredericks 2009).  With MHCD being a larger organization and having a lot of support for evaluation from executive management, as well as having primary funding from Medicaid and the State of Colorado, there is already a lot of familiarity with data collection and evaluation, at least at the administrative level.  However, with it being a large organization and with multiple layers and treatment teams, residential and employment facilities, translating the information that is collected by Evaluation and Research Department to the clinicians and consumers, offers chance for more resistance.

Philosophy

            The mental health field has some differences in philosophy in comparison to medical field as well, that have made it more difficult for evidence based practices to be implemented successfully.  Rishel (2007) points out that the mental health field focuses more on clinical outcome trials or best models of treatment, rather than on prevention itself.  Rishel (2007) goes on to state two of possible reasons for this are prevention coming from a public health perspective and looking at population as a whole, which varies from a clinical approach aimed at best methods to treat those already diagnosed with mental illness.  Also, prevention methods are usually thought of as requiring larger samples and for participants to be followed for a long period of time, compared with clinical trials which are shorter and require smaller sample size (Rishel 2007).  The mental health field is also seen as being hard to evaluate in terms of outcomes, as there are really no standardized outcomes (Rishel 2007).  This becomes even more difficult with less definition attributed to non-profit organizations.  This proves true with MHCD in that most of the focus to this point has been best treatment models to integrate into clinical practice.  However, with integration of data instruments, this allows for more longitudinal data and more of a focus to prevention side. 

Clinician Feeling Towards Instruments     

A study done looking at mental health field and how clinicians feel towards data instruments was conducted by Garland, Kruse, Aarons (2003).   This study was done on mental health system in California and found that 92% of clinicians reported never using scores from data instruments in their clinical practice (Garland et al. 2003: 400).   Further, 90% identified a collecting the data as a “significant time burden” in terms of fitting into their daily work tasks (Garland et al. 2003: 400).  This data shows clinicians not putting much value into data collection, as well as thinking of it more as a burden, versus something that could be useful for them in their practice.  The article also went on to show that 55% of clinicians in reference to measures used by the instrument’s, felt they were “not appropriate, nor valid, for their particular patient population” (Garland et al. 2003: 400).  It would be a hard sell to get clinicians to buy into using some of these instruments, if they don’t believe they are valid or useful for their population.  When asked what changes the clinicians wanted to see with how the data was reported, answers were “briefer administration” or “simpler language”, as well as wishing these were presented in “narrative, as opposed to quantitative form” (Garland et al. 2003: 400).  As we can see, there seems to be a lot of doubt from clinicians around trusting the data is appropriate for their client populations, as well as being able to interpret the data, and doubting whether the data reflects anything that can be used in clinical practice.

           

            As we can see from the literature, there are multiple interests to consider when implementing data instruments into clinical practice.  MHCD is unique from a lot of non-profits in having its own internal Evaluation and Research Department, and this creates a lot of opportunity to progress forward, in terms of how clinical information is relayed to clinicians, as well as consumers receiving mental health services.  MHCD has already made a lot of steps towards helping ease this transition.  The MHCD “Recovery Committee”, which is already a committee that was in place, but is now working on how to help with this transition.  There were focus groups held with clinicians around what concerns they had with the instruments, and this information was used to make some changes to the instruments, as well as in developing trainings.  The trainings were designed to show how the data collection instruments can be interpreted, as well as how to use that information in discussions with consumers to give them incentive in participating in data collection.  MHCD also has a team made up of MHCD consumers who have taken the responsibility of going to each site and sharing with other consumers their experiences with looking at the RP, and how beneficial it is to view this data and learn more about their treatment progress.  MHCD will continue to evaluate how this integration has gone, but it has created a unique opportunity as an organization, to bring various clinical teams and consumers together, to work on getting the most out of this data that is already collected.  In the medical health field, I think we have seen a lot of growth in terms of how we can see lab work, communication with our doctors through e-mail, other electronic means, etc.  I think through this example, MHCD has shown how the mental health field can benefit from data instruments as well, resulting in more client informed, as well as clinician informed care. 

Submitted By Jim Linderman- Evaluation Specialist with Evaluation and Research Department at MHCD, as well as Sociology M.A. student at University of Colorado Denver Sociology Program. 

References

Broom, A., J. Adams, P. Tovey (2009). “Evidence-based healthcare in practice: A study of clinician resistance, professional de-skilling, and inter-specialty differentiation in oncology” Social Science & Medicine 68(192-200).

Carman, J.G., K.A. Fredericks (2009). “Evaluation Capacity and Nonprofit Organizations: Is Glass Half-Empty or Half-Full?” American Journal of Evaluation 31:84.

DeRoche, K., Hester, M., Olmos, P.A., McKinney, C.J. (October, 2007). Evaluation of Mental Health Recovery: Using Data to Inform System Change. Poster presented at the 'Culture of Data' Conference. Denver, CO.

Garland, Ann F., M. Kruse, G. A. Aarons (2003). “Clinicians and Outcome Measurement: What’s the Use?”.  Clinicians and Outcome Measurement 30(4):393-405.

Hoole, E. and T.E. Patterson (2008). “Voices from the Field: Evaluation as part of a Learning Culture”. Nonprofits and Evaluation. New Directions for Evaluation 119:93-113.

MHCD 2009: Report to the Community (can be found at www.mhcd.org)

Rishel, C. (2007). “Evidence Based Prevention Practice in Mental Health: What is it and how do we get there?” American Journal of Orthopsychiatry 77(1): 153-164.          

Denver’s Homeless Street Youth – characteristics and treatment challenges

In 2004 Denver’s homeless youth population was estimated to include approximately 850 young adults between the ages of 12 and 24[1] with at least 1500 homeless youth in the state of Colorado[2]. I believe these figures are a serious underestimate of the actual number of homeless youth in Denver and the state. This population is notoriously hard to count due to their itinerant nature, mistrust of authority figures, unwillingness to participate in surveys or be counted, and because many simply don’t want to be found. For these reasons it can be extremely difficult to get a true idea of how many homeless youth are living on the streets using traditional survey methods administered in schools, shelters and drop-in centers.

Street youth, or youth who live primarily on the streets, are distinct from other youth experiencing homelessness who utilize shelters or transitional housing programs. Their basic needs are not consistently met; they are exposed to greater levels of stress and trauma and are more likely to engage in high risk behaviors[3].  For the purpose of this post I will limit the discussion to homeless street youth.

                There are many possible reasons a teen might choose to live on the streets rather than with relatives or in a foster home. The environment at home could be unsafe due to domestic violence, neglect, substance and/or alcohol abuse, or a combination of these issues. If the teen is having trouble following rules, exhibits features of Conduct Disorder or Oppositional Defiant Disorder, uses drugs or alcohol,  or identifies as gay, lesbian, bi-sexual, questioning, or transgender (GLBQT), this can create conflict within the home which may lead to homelessness.

Sometimes it is a matter of finances; if the family can no longer afford to feed and care for all of their children, the older children may be forced to leave the house to reduce the financial burden and provide for themselves. There is also a disturbingly high rate of homelessness among kids who “age-out” of the foster care system, or who are released from the juvenile justice system. The estimated amount of homeless youth who have aged out of foster care or out-of-home placement ranges from 21%[4] to 53%[5]. There are other situations which could result in a teen leaving home and turning to the streets, these are some of the more common explanations.

                Once on the streets youth are left to navigate a very dangerous and adult world, with little experience and limited physical/mental/emotional development.  In this environment they are exposed to extreme violence, such as muggings, physical and sexual assault, shootings, gang violence, emotional abuse, etc. They have a greater chance than their housed peers of being the victims of this violence[6] and of being re-victimized in the future[7].

Often they will participate in illegal activities, such as theft, battery, breaking and entering, etc., to obtain food, shelter, money or drugs. Survival sex is another strategy used by some street youth; it is defined as the exchange of sex for shelter, food, drugs, or money. Teens who reported having used survival sex  to meet their basic needs also reported higher rates of substance use, suicide attempts, days away from home, STD’s, pregnancy, and victimization[8].

To combat some of the risks and trauma associated with life on the streets, youth will often seek to re-create the family they left behind, or never had. Typically older kids or adults who have been on the streets longer will take the newcomers under their wings. They form a large “street family,” which consists of a “street Dad” and “street Mom,” etc. Depending on the make-up of the group and their activities, street families can be a protective factor for new homeless teens, or they can expose the youth to more harm than they would otherwise experience.

If youth who live on the streets didn’t have a mental health or substance abuse problem when they left home, their chances of acquiring one while living on the streets are very high.  Due to the increased exposure to violence and trauma, they are more vulnerable to PTSD, mood disorders, substance and/or alcohol abuse, conduct disorder, oppositional defiant disorder, and suicidal ideation/attempts.  The relationship between homelessness and mental illness goes both ways; if a teen was already experiencing symptoms of mental illness before leaving home, this may contribute to familial conflict which can in turn lead to homelessness.  The incidence of mental illness and substance abuse are significantly higher for homeless street youth, as compared to homeless youth who live in shelters, or the general youth population[9]. 

                The barriers to treatment for this population are numerous and difficult, but not impossible, to overcome. To begin, they are a hard population to reach physically, as they move around the city frequently, changing camp sites, squat locations, or staying with friends in different areas. This can make planning and delivering services extremely difficult. There is also a culture of mistrust with regards to authority figures and service providers, which can make building a therapeutic relationship challenging. In addition, there seems to be a lag in the mental health field when it comes to applying the principles of recovery to the homeless youth population, specifically regarding consumer directed therapy and strengths based services. It can be difficult for some providers to recognize the autonomy of these youth and their right to determine what direction their lives will take; this is due in part to their young age, their involvement in high risk behaviors, and the parental instincts of some of the providers. It’s important to remember that these kids are by now solely responsible for their own lives, where they sleep, how they eat, and how they spend their time; to treat them otherwise is counterproductive.

I believe the foundations for treatment can be established by providers who are a consistent, caring, attentive and non-judgmental presence in the lives of our homeless youth. On the next post I’d like to discuss some of the various treatments that are currently being used with homeless youth, and which have been found to be the most successful. 

Written by Felice Seigneur

Felice Seigneur is and Evaluation Specialist with the Evaluation and Research Department

at the Mental Health Center of Denver

The content of this blog is based on current and past research on the subject of homeless and street youth, and in part from my own experience as an outreach counselor working with homeless street youth in Denver. If you have any questions about what has been written, or would like to add to the conversation, please feel free to leave a comment below.

               

---

[1] Metropolitan Denver Homeless Initiative, Final Report, www.cdhs.state.co.us/shhp/PDFs/2004_PIT_Final_Report.pdf

[2] Colorado Dept. of Public Health, www.cde.state.co.us/cdeprevention/download/pdf/Office%20of%20Homeless%20Youth%20Report.pdf

[3] Treatment Outcome for Street-Living, Homeless Youth. Natasha Slesnick, Ph.D,Jillian L. Prestopnik, Ph.D., Robert J. Meyers, Ph.D., and Michael Glassman, Ph.D. Addict Behav. 2007 June ; 32(6): 1237–1251.

[4] Cauce, A. M., Paradise, M., Embry, L., Morgan, C., Theofelis, J., Heger, J., & Wagner, V. (1998).

Homeless youth in Seattle: Youth characteristics, mental health needs, and intensive case

management. In M. Epstein, K. Kutash, & A. Duchnowski (Eds.), Outcomes for children and

youth with emotional and behavioral disorders and their families: Programs and evaluation best

practices. Austin, TX: PRO-ED.

[5] Toro, P. A., & Goldstein, M. S. (2000, August). Outcomes among homeless and matched housed

adolescents: A longitudinal comparison. Presented at the 108th Annual Convention of the

American Psychological Association, Washington, DC.

[6] - Homeless Youth in the United States: Recent Research Findings and Intervention Approaches. Paul A. Toro, PhD, Wayne State University, Detroit, MI, Amy Dworsky, PhD, University of Chicago, Chicago, IL, Patrick J. Fowler, MA, Wayne State University, Detroit, MI. The 2007 National Symposium on Homelessness Research

[7] Whitbeck, L. B., Hoyt, D. R., & Ackley, K. A. (1997). Abusive family backgrounds and victimization among runaway and homeless adolescents. Journal of Research on Adolescence, 7, 375–392.

[8] Prevalence and Correlates of Survival Sex Among Runaway and Homeless Youth. Jody M. Greene, MS, Susan T Ennett, PhD, and Christopher L. Ringwalt, DrPH. American Journal of Public Health, September 1999, Vol. 89, No. 9

[9] Toro, et al., 2007

Future Blogs

We would like to let you know it takes time to compile Research on different subjects.  

Because of this reason we will only be posting on this blog monthly.

Hopefully the subjects will be of interest to you and we hope you will continue to visit our blog in the future.

Thank you

Resillence

The term “resilience” is a word and concept that often gets thrown around in a variety of contexts within the mental health field. Despite the prevalence of the terminology, it is frequently unclear as to what professionals are trying to capture through the use of this construct. The multitude of definitions and interchangeability of resilience with other constructs (such as recovery) make it difficult to establish a common language among mental health providers, particularly with regard to interventions and research designed to facilitate resiliency. In Ungar’s 2004 article on resilience, he points out the “definitional ambiguity in the resilience construct.” Through this article I hope to provide a brief overview of the etiology and evolution of resilience while highlighting some of the past and recent research. Hopefully this information will help to inform our future application and efforts to foster resiliency in our own lives and those around us.
Historical accounts date the origin of resilience from between 1620-30 C.E. with the Latin root “resiliens,” meaning to “spring back” or “rebound” (Friesen, 2005; Luthans, Vogelgesand, & Lester 2006; Online Etymology Dictionary, 2008). The resilience that we now associate with mental health became a prominent construct in the 1970s when researchers began to examine individuals who were able to follow a positive developmental trajectory despite the presence of high-risk conditions or adversity (Luthar & Zigler, 1991). Since that time, there have been three recognized waves of research involving resiliency, “resilient qualities,” “the resiliency process,” and “innate resiliency” (Richardson, 2002).
Resilient qualities research has sought to identify particular traits or characteristics that have helped them survive some form of adversity. Various studies have identified these protective factors to include items such as gender, tolerance, achievement orientation, good communicator, altruism, self-efficacy, future orientation, high expectations, good self-esteem, happiness, faith, creativity, and morality, among others (Baumeister & Exline 2000; Buss, 2000; Myers, 2000; Simonton, 2000; Werner, 1982; 2005; Werner & Smith 1992). These specific developmental assets remain of interest to resiliency researchers while an emphasis on the process involved in fostering resilient responses has gained even greater attention.
The resiliency process research has sought to view resiliency as more of a dynamic developmental process between person and environment while reflecting some positive adjustment despite some form of adversity (Friesen, 2005; Edeschi & Kilmer, 2005). This movement within the field of resilience has sought to transform the construct from a trait to be expressed into a state that is either developed or elicited within particular context (Lussier, Derevensky, Gupta, Bergevin, & Ellenbogen, 2007). The exploration of the interactional and environmental nature of resiliency welcomed another wave of research into how resilience might be fostered, developed, and learned.
Innate resiliency research drew into question many of the assumptions that had previously been made about the resilience construct. Resilience was beginning to be viewed as no longer an either yes or no condition that individuals were predetermined to have (or not), but a construct that falls along a continuum and may be continually enhanced (Cairns-Descoteaux, 2005). This further development also began to draw into question the necessity that there be the presence of some stressor or adversity (to overcome) in order for their truly to be a resiliency process in effect.
Many current explorations of resiliency have begun to view resiliency as something innate to us all. In Bonnie Benard’s The Foundations of the Resiliency Framework emphasizes the “process of connectedness” within resiliency and the importance of the how we do what we do, trying to move our focus in mental health from our fixation on the content of what we do and instead on the context. This concept is further elucidated (within an educational context) by Dr. Truebridge’s in her blog Resilience, Research, and Educational Reform resilience-research-and-educational-reform/) in which she discusses the importance of change in the person delivering a particular service and the way it is delivered (and not necessarily the service itself) in terms of facilitating resilience in those with whom we come in contact. These recent examinations have helped to highlight the role of our own beliefs (and those within the broader social context) as a crucial element in creating resilience.
As can be seen by the previous review of resiliency literature, the construct remains somewhat of an enigma. The many various interpretations and understandings of resilience has led to much of the ambiguity in the term and has led some researchers to draw into question the utility of the construct itself in meaningfully contributing to the research and literature. Through my own research of resilience I tried to address this issue through the process of a meta-synthesis of other resiliency studies in the hopes of identifying common themes and creating a more meaningful understanding of the construct. The results of the study suggested the presence of eight core processes within resiliency of internal locus of control, reconstruction of the narrative, altruism, acceptance, flexibility, optimistic outlook, interpersonal effectiveness, and social support (Nebel, 2008). Resilience remains a prominent issue of debate within the clinical and research fields of psychology. Hopefully this blog was able to provide a brief overview of some of the current views and applications of the resiliency construct in mental health while highlighting the ongoing need for continued dialogue and research.

By Scott Nebel, Psy.D.

• Scott is a Psychologist on MHCD’s Intensive In-Home Treatment Team and collaborates with the MHCD Research Institute.

Baumeister, R., Exline, J. (2000). Self-Control, Morality, and Human Strength, Journal of Social and Clinical Psychology, 19, 29-42.

Buss, D. (2000). The Evolution of Happiness. American Psychologist, 55, 15-23.

Cairns-Descoteaux, B. (2005). The Journey to Resiliency: An Integrative Framework for Treatment for Victims and Survivors of Family Violence. Social Work & Christianity, 32(4), 305-320.

Friesen, B. (2005). The Concept of Recovery: “Value Added” for the Children’s Mental Health Field?. Focal Point, 19(1), 5-8.


Lussier, I., Derevensky, J., Gupta, R., Bergevin, T., Ellenbogen, S. (2007). Youth Gambling Behaviors: An Examination of the Role of Resilience. Psychology of Addictive Behaviors, 21(2), 165-173.

Luthans, F., Vogelgesang, G., Lester, P. (2006). Developing the Psychological Capital of Resiliency. Human Resource Development Review, 5(1), 25-44.

Luthar, S., Zigler, E. (1991). Vulnerability and Competence: A Review of Research on Resilience in Childhood. American Journal of Orthopsychiatry, 61(1), 6-22.

Myers, D. (2000). The Funds, Friends, and Faith of Happy People. American Psychologist, 55, 56-67.

Online Etymology Dictionary, 2008

Richardson, G. (2002). The Metatheory of Resilience and Resiliency. Journal of Clinical Psychology, 58(3), 307-321.

Simonton, D. (2000). Creativity. American Psychologist, 55, 151-158.
Ungar, M. (2004). A Constructionist Discourse On Resilience. Youth & Society, 35(3), 341-365.

Werner, E. (2005). Resilience and Recovery: Findings From the Kauai Longitudinal Study. Focal Point, 19(1), 11-14.

Werner, E., Smith, R. (1992). Overcoming the Odds: High Risk Children from Birth to Adulthood. Ithaca, NY: Cornell University Press.