In behavioral healthcare (mental health and substance abuse treatment) routine health outcomes measurement

Routine health outcomes measurement

Evidence-based practice describes a healthcare system in which evidence from published studies, often mediated by systematic reviews or processed into medical guidelines is incorporated into clinical practice. The flow of information is one way; from research to practice...

 has expanded beyond aggregating measurements for quality improvement studies and has placed equal emphasis on the therapeutic gain delivered by real-time patient-level outcome feedback. With the $200 billion behavioral health marketplace in a state of public health crisis as defined by the US Surgeon General with most patients receiving substandard care, outcome management is helping to provide quality controls, data and structure for a large subsection (more than 27%) of the US population.

Almost by definition, psychotherapy is a rather unstructured process, leaving many people who are going through the process of guided self-discovery and behavioral change to wonder whether therapy is helping. Tracking progress with repeated administrations of a self-report questionnaire allows both therapist and client to know what is getting better from the perspective that matters most—the patient's.

Definition

Behavioral health outcome management (BHOM) involves the use of behavioral health outcome measurement data to help guide and inform the treatment of each individual patient. Like blood pressure, cholesterol and other routine lab work that helps to guide and inform general medical practice, the use of routine measurement in behavioral health is proving to be invaluable in assisting therapists to deliver better quality care.

Uses of behavioral health outcome management data

The data collected through formal (typically self-report) measurement (like the PHQ-9 for depression) has been used to enhance the accuracy of clinical assessments, provide a basis for treatment planning, deliver an objective methodology for tracking treatment progress, alert therapists with clinically proven guidelines to get refractory cases back on track, help prevent hospitalizations with warning guidance, and provide primary care physician

Primary care physician

A primary care physician, or PCP, is a physician/medical doctor who provides both the first contact for a person with an undiagnosed health concern as well as continuing care of varied medical conditions, not limited by cause, organ system, or diagnosis....

s and other referral sources with outcome-based referrals linking new patients to therapists with a proven track record of delivering exceptional care to patients with similar behavioral health needs.

More accurate clinical assessments

Behavioral healthcare rarely uses genetic markers or blood tests to assist in diagnosing major depression or other behavioral health disorders like schizophrenia and substance abuse. Instead, the field relies on the careful assessment of symptoms, like changes in mood and behavior, to make a formal diagnosis. However, with more than half of behavioral healthcare delivered by primary care physicians where there is rarely sufficient time or expertise to conduct a formal interview, a standardized assessment and screening process using formal questionnaires administered in the waiting room or over the internet are invaluable. Even for behavioral health specialists like psychologists, psychiatrists and social workers who are trained to interview and diagnose behavioral health problems, their efforts are hampered by patient's willingness, or lack thereof, to honestly report relevant symptoms. For example, in an initial interview, only half of patients honestly disclose previous suicide attempts, greatly hampering therapists' efforts to keep patients safe. By contrast, six controlled studies document that patients not only like completing paper-and-pencil or web-based questionnaires, patients are more accurate when they do so, giving therapists and primary care physicians more accurate assessments of client's symptoms and issues, and a new and potentially different perspective than seen in face-to-face interviews.

Treatment planning

With formal assessments of patient needs, the clinician can devise an individualized treatment plan that incorporates these needs and evidence-based practices and principals. Some health plans now require the integration of BHOM into routine care and treatment planning and a well designed BHOM system provides the clinician feedback and guidance to keep up with these evidence-based principals.

Tracking treatment progress

Obviously, the most fundamental use of BHOM is to use the data to track treatment progress on a patient-by-patient basis. Especially with session-by-session assessments, real-time scoring and report generation the data provides clinicians and patients with excellent feedback about the course of treatment and whether adjustments need to be made to the treatment plan. Some of the first and second generation BHOM tools have been hampered by their ability to document reliable patient improvement with only 20-30% of cases showing reliable improvement, while some of the more advanced tools can document reliable improvement on more than 50% of cases on a single domain, and more than 90% with multi-dimensional analyses. If documenting patient improvement to payers and purchasers is important, these statistics may be vitally important to evaluate before building and deploying a BHOM system (see below).

Refractory case alerts

Researchers at Brigham Young and other universities have clinically proven that BHOM helps therapists to deliver better care by identifying cases that are not likely to improve with the current treatment plan. These refractory cases, or negative responders, account for approximately 15% of all treatment cases when a BHOM system is not used. A well designed BHOM system can reduce the population of negative responders to less than 5%, increasing the effectiveness of psychotherapy by at least 10%. Since clinicians, on their own, are abysmal at predicating what patients will negatively respond to treatment, a BHOM system is the only known way to facilitate this level of effectiveness.

Hospitalization and other warnings

Other uses of BHOM are also emerging as the systems evolve from simple, single-domain alert systems like the OQ-45 to more complex, multi-dimensional systems. Single-domain alert systems, while effective at identifying some negative responders with a single, global measure of distress, do not capture the full complexity of issues seen in behavioral healthcare (e.g. depression, psychosis, mania, anxiety, substance abuse, etc.). With more elaborate measurement, BHOMs have been shown to facilitate the identification of high-risk cases (clients that are at high-risk for behavioral health hospitalization due to suicide, violence, etc., and other high-cost/high-risk services) allowing the health plan and clinician to collaborate in flexing treatment benefits to prevent the loss of life and increased healthcare costs.

Outcome-based referrals

Not all therapists are equal; some have more experience and are better at treating alcohol disorders while others have more experience and effectiveness at treating depression or psychosis. Some researchers have effectively argued that most of the differences seen in controlled studies comparing different treatments (cognitive behavioral therapy, interpersonal/psychodynamic therapy, medication management, etc.) are really documenting the relative differences between the therapists involved in the studies. In fact, it has been shown that an excellent psychiatrist using a placebo can get better results than a poor psychiatrist using a well-documented anti-depressant.

A patented application of a BHOM system helps large organizations (health plans, community mental health centers and hospitals) identify which therapists are getting excellent outcomes with specific diagnostic or domain-specific problems, and refer incoming patients screened with an initial evaluation of the BHOM to the most effective clinicians.

How to create or evaluate a BHOM

Behavioral health outcomes management requires a host of interconnected components in order to create the most benefit for patients. These are reviewed below.

Real-time data processing and reporting

Unless clinicians have the time to score, record and chart each client administration, they need to create an infrastructure to process this data in real time. Patients typically complete questionnaires before each session and these questionnaires are most useful if the results are available before the session starts.

Processing methods

Computer-based applications have great appeal. They can process data quickly and reduce staff involvement, but not all patients feel comfortable interacting with a computer or a PDA and may require a paper-based backup system. Paper questionnaires can be scanned or faxed into a central processing system where dedicated staff can verify and process the data. Hybrid systems are also available that allow users the flexibility of using both types of processing approaches in one application with print-on-demand paper forms bar coded with necessary patient and for-office-use-only information, eliminating the need for hand-writing recognition.

Interactive voice recognition (IVR) systems have been attempted, but with little success. Remembering to press one for "all the time", two for "some of the time", etc., is rather cumbersome, and complex question wording may require the system to repeat both the questions and answers several times before a confused or psychotic patient can effectively answer a single question.

Outcome domains

The American Psychological Association

American Psychological Association

The American Psychological Association is the largest scientific and professional organization of psychologists in the United States. It is the world's largest association of psychologists with around 154,000 members including scientists, educators, clinicians, consultants and students. The APA...

 and the largest international society of psychotherapy researchers (The Society for Psychotherapy Research

Society for Psychotherapy Research

The Society for Psychotherapy Research is a learned society founded in 1970. It is multidisciplinary, international association for research into psychotherapy...

—cofounded by Ken Howard, the grandfather of behavioral health outcomes management research) held a Core Battery Conference (CBC) in 1994 to develop the minimum requirements for an outcome battery. According to the CBC, across diagnostic groups, a core battery should assess three distinct domains:

• General distress or quality of life
• Symptom clusters (like depression, anxiety, psychosis, mania, etc.), and
• Functional domains (like work and social functioning)

Typically, a well-designed BHOM system will need to integrate multiple questionnaires to measure all of these areas as there is only one, currently identified outcome battery in the literature that meets all of the CBC defined criteria with one, short questionnaire.

For each questionnaire's domain, key psychometric qualities especially important for BHOM applications include:

• Meeting current confirmatory factor analytic standards including reported statistics on CFI, GFI, TLI, and RMSEA.
• Percent of patient documenting reliable improvement. As stated earlier, first- and second-generation tools typically had values in the 20-30% range. More modern tools have single domain values in the 50% range and total-tool, multi-dimensional statistics as high as 90%. If the goal is to document client improvement (for the patient, therapist, payer, purchaser, accrediting bodies, etc.) this becomes a most important statistic and benchmark for comparing measures.

Case-mix and risk-adjustment

Just measuring symptom reduction and functional improvement is not enough. Some clinical cases are more difficult to treat than others. For example, patients who have had multiple hospitalizations, co-morbid medical conditions, and extensive life stress are more challenging to treat than those who have none of these complicating issues. In order to evaluate the relative effectiveness of each clinician's treatments, these variables need to be identified and measured within the BHOM system in order to make accurate comparisons.

Benchmarking database

The first generation computers were awe-inspiring, taking up several floors of academic buildings. However their power, memory and speed is dwarfed by everyday laptops, meaning that nothing has relative value standing on its own. BHOM data must be taken in context and in comparison to something. Understanding a therapist's strengths and weaknesses can only be done by comparing his or her work to others', and here the size of the comparison database matters, allowing researchers and users to find matching comparative samples through disaggregation and statistical techniques. There are currently two sources for these massive comparative datasets: managed care, and provider-based systems.

Managed care systems: Responding to purchaser and regulatory demands for outcome and accountability data, managed care companies have begun to collect data on their network therapists. PacifiCare's development and use of the Life Status Questionnaire (LSQ) is the largest known effort to date, with data reported on 99,004 patients in at least one study. These datasets, while massive, have significant limitations, including:

• they are isolated to a small segment of each provider's practice
• they are proprietary to the health plan and typically not accessible, even to participating providers
• they are potentially replete with measurement error as most health plans cannot resist the temptation of using the data to manage patient benefits, authorizing inpatient care or number of out-patient sessions based on how, or if, questions are answered. This use of outcome data can be jarring to most patients and significantly different from the Hippocratic-Oath standard of "do no harm". As a result, many patients bias their responses in order to receive the benefit they are looking for or hide important information about psychosis or substance abuse from insurance companies and their employer.

Provider-based systems: The other source of massive datasets comes from provider-centric BHOM firms that help providers measure all of their patients and confidentially benchmark their results to their peers with some datasets already exceeding a million cases.

Limitations of these approaches include:

• a self-regulated user base that is in constant flux as payers and other requirements force providers to use other systems for specific patient populations.
• a self-selection bias where those providers that opt into such systems may be of a higher quality, committed to using data to improve care.

Cost

Cost is a critical factor when considering integrating an already existing BHOM system or building one from scratch. Building a system can take more than ten years to design studies, develop and refine outcome tools, program necessary hardware and software to process the data, and seed the benchmarking database with sufficient volume to provide comparison data. However, wanting to own and control the process is a powerful motivator. Purchasing licensing rights to existing systems can also be expensive and need to be carefully evaluated for the hidden costs associated with each of the critical components listed above. Fortunately competition is driving prices down and some of these well developed BHOM system are now free and can include questionnaires, data processing systems, and real-time client reports.

Conclusion

BHOM has been clinically proven to improve treatment in a mental health system that is in a state of crisis. Patients typically welcome these efforts to integrate their feelings into treatment plans and to make the process of therapy more objective and progress measurable. Whether mandated or adopted willingly, BHOM should be a part of standard practice and good quality care.