Advocacy

Principles of Outcome Evaluation in the Treatment of Substance-Related Disorders

Adoption Date:
January 1, 2001

Principles for Outcome Evaluation in the Treatment of Substance-Related Disorders: a Joint AMBHA-ASAM Statement

PDF

Introduction

The American Managed Behavioral Healthcare Association (AMBHA) and the American Society of Addiction Medicine (ASAM) share an interest in the treatment of addiction and of other substance-related disorders. Both organizations seek to promote treatment interventions that have high-quality outcomes, and encourage studies of the outcome of each step of the full continuum of services for substance-related disorders. Determining the most salient dimensions of treatment effectiveness is challenging. Relevant derivative questions include “what kind of treatment?” and “for what kind of condition?” as well as “what aspect of effectiveness?” and “quality as defined by whom?” AMBHA and ASAM have previously collaborated in the development of conjoint statements on Effective Treatment of Addictive Disorders, Parity in Benefit Coverage, Practice Guidelines, and A Guideline for Credentialing and Privileging of Clinical Professionals for Care of Substance-Related Disorders. This paper is a conjoint effort of the two national organizations to describe the salient aspects of treatment outcome for substance-related disorders and the principles that should guide any critique of treatment outcome studies. In an era in which demands for accountability are mounting throughout health care, addiction treatment outcome studies are valuable in generating data to inform system improvements, quality assurance monitoring, and resource allocation within the health care delivery system.

Standardized terminology and methodology should be basic features of treatment outcome studies so that results from different studies can be compared. In other areas of medical research, consensus has been reached on such standardization. One example is in oncology research where studies utilize agreed-upon measures to describe the kind of tumor, the severity/extent of illness, the modality of treatment, the dosage and duration of treatment, the dimensions of outcome, and the time frames over which evaluation occurs. Thus, when a treatment is evaluated, two-year and five-year survival can be compared with the survival rates for other treatments and for the untreated condition. Addiction treatment outcome research should be satisfied with no less precision than is seen in other areas of medicine.

Substance use also has multiple and broad impacts, and therefore treatment can have multiple effects. Disorders related to substance use also vary by the extent of substance use, the type of substance used, and severity of substance-related disorder. Moreover, it is important to recognize that the addictive disorders population is not uniform: it varies by gender, age, ethnicity, and comorbidity. Hence, treatment interventions can have different outcomes when applied to different subgroups of the population, and outcome measures must account for these variations.

For individuals with substance-related disorders, a wide range of aspects of functional status can be measured. The following are some examples:

• Substance use pre-treatment, during continuing treatment, and post-treatment can be measured, and by different methods (self-report, collateral-report, body fluid analysis, etc.).
• Quantity of use, frequency of use, number of using days per hundred days, duration of periods of total abstinence, and periods of low-volume use or low-problem use can be measured.
• School or work attendance, school or work performance, and job safety can be relevant variables to measure in outcome studies of treatment of substance-related disorders.
• The effects of treatment on criminality are often measured in addiction treatment research (a kind of variable not usually measured in treatment services research for other health care conditions).

• Due to the chronic nature of substance use disorders, outcome measurements can monitor rates of compliance with ongoing monitoring and disease-management strategies, and the effect on outcomes of participation in such strategies.
• Outcome evaluations can be structured to not only examine functional status and remission/relapse rates after treatment has ceased, but also evaluate proximal outcomes, such as attendance at treatment sessions, treatment drop-out rates, involvement of patients/families in the treatment planning process, and measures of the patient’s clinical status during the process of ongoing treatment.

Other dimensions of outcome can be measured; for example:
• Utilization of health care services (emergency room visits, medical/surgical admissions, psychiatric admissions, detoxification admissions, other addiction admissions, office visits) can be measured before and after an index episode of addiction treatment.
• Utilization of social services (corrections encounters, child welfare encounters, homeless shelter encounters) can be measured before and after an index episode of addiction treatment.
• Effects of substance use on third parties in the population can also be measured, e.g., the prevalence rates of infectious disease in a population or the prevalence rates of auto crashes in a population.

It is also important to recognize that an index episode of treatment for a substance-related disorder can range from a pharmacotherapy trial to a provision of brief advice in a primary care clinic, to a course of psychosocial rehabilitation counseling for addiction, to a psychoeducational lecture series, to an inpatient hospitalization.

Beyond clinical aspects of treatment that warrant measurement, a systems issue that interacts with treatment outcome is the effect of the organization of the treatment system and the method of payment for services within that system on referral, treatment seeking and outcome. Programs need to assess outcome with this potential interaction in mind, given the unique problems of stigma, impaired cognition and the counter-motivational effects of the substances of abuse.

CONCEPTS:

Process Evaluation:
This examines the steps of the clinical process that are engaged in by the health care organization on the patient’s behalf. It includes concurrent and retrospective review of events occurring during the treatment episode. Process evaluation can employ performance indicators, which are examinations of critical process steps to see if services are being carried out as expected or in a standardized manner, or to see if interventions are delivering the expected interim results at particular intervals in the course of the treatment encounter. There is a common assumption that reduction in unnecessary variation in the treatment process can improve the outcomes of that process. Process evaluations can determine whether services are being delivered in accordance with specified practice guidelines or according to specified care management criteria. They can also measure an individual’s satisfaction with the treatment process.

Outcome Evaluation:
This examines the status of the patient after (and presumably as a result of) involvement in clinical services. The status can be the patient’s functional status, the patient’s degree of symptoms/distress, or the patient’s level of satisfaction after treatment. Outcome evaluation can also measure the satisfaction of collateral parties (family members, employers, and referents) or even the satisfaction of treatment providers with the clinical process. Patient behaviors (e.g., substance use, job/school attendance,
involvement with health care providers, criminal behaviors) can also be evaluated post-treatment and compared to pre-treatment measures.

Prognostic Indices:
These are variables which influence treatment outcomes. They are generally pre-treatment dimensions, and may include static features of the patient, such as gender and ethnicity; or dynamic features which can change over time, including attitudinal or motivational measures such as the patient’s stage of change or the patient’s resistance to specific aspects of treatment. The patient’s gender, age, ethnicity, psychiatric comorbidity, severity of substance use and the severity of substance-related disorder, are all pre-treatment variables which can affect post-treatment status as much as can the treatment experience itself. Hence, treatment interventions can have different outcomes when applied to different subgroups of the population, and outcome measures must be tailored to reflect these variables.

PRINCIPLES:

With these issues in mind, and in an attempt to bring clarity to an area that is challenging and at times confusing for the field, AMBHA and ASAM have identified the following principles which they believe should guide treatment outcome measurement efforts:

1. The multiple, often overlapping reasons for conducting outcome evaluation should be recognized, as should the often differing motivations of various parties who have interest in outcomes data. For example:

• A provider may be interested in measures of continuity of care, time in treatment versus treatment dropout, patient satisfaction, and the thoroughness and accuracy of patient screening, patient placement, and the processes of treatment.
• A managed behavioral health care organization (MBHO) may be interested in measures of access to care, performance indicators, quantifiable symptom reduction, quality of consultations, and referral to ongoing services.
• An employer may be more interested in measures of continued employment, productivity, safety, absenteeism, and utilization of benefits.
• Public policy makers may be interested in measures that reflect changes in the social and economic costs of addiction, such as public health, social welfare, and criminal justice expenditures. They may also be interested in the influence on treatment outcome of access to private and/or publicly funded health insurance benefits.

2. Outcome evaluation should be meaningful. It should address areas of genuine concern and interest to a variety of groups and stakeholders. All significant stakeholders in the measurement of treatment outcome should participate in the development of those measures, to assure that the proper domains of data are collected.

3. Evaluation studies of the processes of treatment or of the outcomes of treatment should specify a number of different domains:

• the clinical condition being studied for pre-treatment and post-treatment prevalence, severity, and intensity; e.g., substance use, substance dependence, substance abuse, or substance withdrawal
• the treatment modality which is being evaluated; e.g., chronic disease management, addiction rehabilitation, brief interventions, or detoxification services
• the effects of reimbursement processes on outcomes
• the time frames of measurement; e.g., immediate outcomes, short-term outcomes, or long-term outcomes
• the populations of individuals for whom treatment impacts are being evaluated; (e.g., patients, family members, employers, or communities), recognizing the importance of both outcome measures for individuals and outcome measures evaluation for populations
• the specific ongoing and post-treatment variables being measured, e.g., the biomedical effects of treatment; emotional, cognitive, and behavioral variables; effects on relationships and social networks; or post-treatment utilization of services by individuals with a substance-related disorder or by their family members.

4. Parameters being evaluated should be measurable and comparable. Outcome evaluations should employ state-of-the-art statistical methodologies and the process of benchmarking, as well as conform to the tenets of quality improvement. Outcome evaluation should also employ standardized terminology and methodology to facilitate comparisons by different constituencies. Thus, providers who work with various insurers or MBHOs would not face differing specifications for data, and MBHOs which work with various providers would not receive data in different formats that would preclude comparability.

5. Addiction treatment outcome measures should be as consistent as possible with outcome measures utilized in generally accepted public health research and health care delivery research.

6. Abstinence should not be the only variable considered in evaluating the effectiveness of a treatment intervention for a substance-related disorder. However, addiction treatment outcome studies can and should measure abstinence from substance abuse as an important variable.

7. Treatment outcome studies should take into account that persons with addictive disorders are not a uniform population. Hence, outcome studies should be severity-adjusted and adapted to the subpopulations involved (and their pre-treatment prognostic indices) when appropriate.

8. Outcome evaluations should creatively apply the available capabilities and recognize the limitations of currently existing data sets and methodologies. They should generate useful and usable information and thereby add value, without adding undue complexity, rework, and expense to clinical operations. They thus should build on existing knowledge about health services research and be fiscally expedient rather than fiscally burdensome for providers, for MBHOs, or for employers, governments or other purchasers of health care services.

9. Outcome studies in addiction treatment must comply with ethical standards for treatment services research. Studies must be designed so that they avoid harm or indignity to the subject, assure the patient’s right to privacy, and preserve the right of individuals to decline participation without any adverse impact upon themselves or their access to further treatment services.

2 comments

Leave a comment
  1. Amelia Loomis Apr 28, 2014 - 12:43 PM

    I am a hypnotherapist doing addiction recovery support.  Is there any reference to hypnotherapy in this model?  

  2. Jeffrey Hermes May 30, 2012 - 10:01 PM
    HTML

    Leave a comment