Public Policy Statement on Recommendations for Design of Treatment Efficacy Research with Emphasis on Outcome Measures
These recommendations are directed to those interested in treatment efficacy in the field of chemical dependency; that is, comparing one type of treatment with another to assess how effective one is when compared with the other. These recommendations apply to any type of program or treatment for chemical dependency. Efficacy of treatment must be distinguished from outcome, per se, which is the person's condition at a particular time after starting treatment. Outcome will depend not only on treatment efficacy (if any), but also on all the other factors influencing the person's condition at that time.
These recommendations represent the consensus of over 70 experts polled by the Society.
Section One describes nine essential elements of the design of a study. Section Two describes the outcome indicators (or variables) to be assessed at follow-up in research on the efficacy of chemical dependency treatment:
• Those which look at the patient's alcohol or drug dependence or use.
• Those which look at the patient's physical and psychological health and subjective sense of well-being;
• Those which look at the patient's function or role in relationship to society.
Structure of the Research Project
There are nine essential elements of the design of a study. These elements should always be integrated into the communication of study results to the professional community.
1. The Starting Number of Patients. The number of patients who meet the entry criteria at the time of initiation of treatment should be noted. Sample size should be large enough to ensure adequate statistical power for the study. The number of patients who did not meet the criteria for entry into the treatment program should be specified.
2. Initial Patient Characterization. Patients should be adequately characterized at the time they enter the study to give a baseline on all variables to be used as outcome measures and all variables known to influence prognosis (e.g., psychiatric diagnoses, demographics, significant relationships, addict's environment, and payment source). The mechanism for characterizing patients and, when appropriate, the validity and inter-rater reliability of the method should be described. Collateral data are recommended.
3. Comparison of Two or More Groups. Efficacy studies require comparison of two or more groups. The process of group assignment should be described, and should follow recognized research standards (random assignment, when possible) and should control for, or measure, factors affecting the selection of treatment and prognostic indicators in order to minimize bias as much as possible. Inclusion and exclusion criteria should be described and reasons should be given to show why certain patients are excluded.
4. Description of the Treatment Program. The program of treatment should be described in a way that makes clear the important structure and process components, as well as the program philosophy. These elements should be described:
A. The theoretical or conceptual rationale guiding treatment.
B. Short- and long-term goals.
D. Level of care: the number of contact hours per day in group and/or individual contact with treatment staff.
E. Staff and their level of training.
F. Which assessment tools were used.
G. Main treatment elements and the duration of each.
H. Treatment individualization, the matching of treatment to patient characteristics and patient needs.
5. Continuing Care Compliance, Frequency, and Duration. The study should describe the continuing care activities, including percent compliance with recommendations, and actual frequency and duration of the patients' and significant others' participation in out¬patient/aftercare/maintenance services. Participation in self-help activities (e.g., Alcoholics Anonymous, Narcotics Anonymous, a church, etc.) should be described.
6. Discharge Category. Each patient who entered the study should be accounted for. For all patients who met the entry criteria, the report of the study should specify the duration of treatment and the patients' condition at the termination of participation in the study, whether planned or unplanned.
7. The Number of Patients Followed Up. The number of patients for whom outcome data are known at the time of follow-up should be specified.
8. The Follow-Up Time. The length of time from initiation of treatment to follow-up should be specified. Multiple follow-up points are recommended. Three important time points are six months, 12 months, and 24 months.
9. Costs. The cost per day of treatment and the method of payment for care should be specified.
Variables To Be Measured
We recommend measurement of the following six variables, listed here in order of priority. We also recommend validation of information received from patient self-report.
1. Substance Use.
A. Report the status of use of primary drug(s) of dependence.
B. Report the status of use of other drugs of dependence, including nicotine.
For A and B above, report the following:
i. Total abstinence since start of treatment.
ii. Time to first use from start of treatment.
iii. Time to first use from end of treatment.
iv. Number of days between last use and follow-up time.
v. Percent days abstinent since start of treatment.
vi. Days of substance use since start of treatment.
C. Report any use, since the start of treatment, of medications prescribed for their psychotropic action or which have psychotropic side effects (including detoxification medications).
Quantity/frequency measures such as average drinks per drinking day may be useful for some studies, but may be harder to measure and validate than abstinence measures.
Confirmation of a patient's self-report of substance use or non-use is recommended, by either biochemical measure or corroborative report.
2. Readmission for Chemical Dependency Treatment Due to Re-Use or Threatened Re-Use. Admission may be to chemical dependence or psychiatric inpatient or outpatient facility.
3. Health Status.
A. Health service utilization:
ii. Hospitalizations since start of treatment.
iii. Medical/psychiatric/dental outpatient visits since start of treatment.
B. Confirmation of health status is recommended.
i. Appropriate biochemical markers associated with illness.
ii. Standardized interviews, psychological tests and rating scales.
4. Employment Function Status.
A. Employment status at follow-up.
B. Number (or percent) of days worked compared to total days eligible to work (e.g., after release from inpatient or full-time day treatment).
Confirmation by the employer is recommended.
5. Legal Problems (During Periods When Not Confined).
Confirmation by checking the public records of arrests is recommended.
6. Evaluation of the Patient's Relationship With Close Family and Significant Others.
A. Marital status at follow-up (includes same-sex and opposite-sex partners).
B. Relationship status assessments may include:
i. Satisfaction with relationship at follow-up.
ii. Quality of family interactions at follow-up.
C. If a parent of minors, custody status of children.
Confirmation from family members/significant others is recommended.
THESE AREAS NEED FURTHER STUDY:
1. Global Function Assessments. Before we recommend global function as a variable to be measured, we suggest these questions for further study:
A. Have the global function assessment instruments been reported in the chemical dependency literature?
B. Have the instruments been validated appropriately?
2. Assessment of Psychological (Emotional) and Social Functioning. One of the goals of some major treatment modalities for substance abuse disorders is to improve the patient's psychological and social functioning. Standardized psychological/psychometric instruments are available to measure the patient's subjective sense of well-being and to assess aspects of the patient's psychological and social functioning.
Questions for further study:
A. What valid instruments or methods assess the patient's ability to cope with stressors, the subjective sense of well-being, or the adequacy of psychological and social functioning?
B. What valid instruments assess the relatively stable aspects of psychological and social functioning, yet still accurately measured changes in these areas over time?
C. Which of the above have been used in assessing the outcome of chemical dependency treatment?
3. Assessments of Major Life Stressors In The Follow-Up Period. Measuring major stressors occurring in the follow-up period would help determine whether two groups were affected by different prognostic variables occurring following treatment. This may help assess the efficacy of specific stress management treatment interventions. There are instruments to assess both stressors and family and work environmental scales, and they should be studied further to identify their validity and their ability to contribute to measurements of the effectiveness of treatment.
These recommendations were developed by the California Chapter (the California Society of Addiction Medicine) and the ASAM Treatment Outcome Research Committee.