Guest Editorial – Using Human-in-the-Loop AI to Understand Stigma During the Deployment of Addiction Evidence-Based Practices in the HEALing Communities Study

Medications for opioid use disorder (MOUD), and overdose education and naloxone distribution (OEND), are among the most supported evidence-based practices (EBPs) in addiction medicine.^1–3 Yet the real-world deployment of those EBPs at scale in diverse communities remains limited.^1,2 Research shows that stigma toward these EBPs, toward people who use drugs or people receiving MOUD, plays a significant role in limiting the deployment and use of these EBPs in communities impacted by the opioid crisis.^2,4

The HEALing Communities Study (HCS) created a rare opportunity to increase the prioritization and deployment of these EBPs to reduce opioid overdose deaths. HCS was a multisite, community-level, cluster-randomized, wait-list controlled trial that tested the Communities That HEAL (CTH) intervention across 67 communities in 4 states (Kentucky, Massachusetts, New York, and Ohio).^1,5 CTH is a coalition-driven, data-guided approach designed to increase adoption of EBPs across settings, paired with communication campaigns intended to reduce stigma and ultimately reduce opioid overdose deaths.^1,5–7 The community coalitions implemented hundreds of EBPs to reduce fatal overdose.¹ Our research focused on New York, where each coalition included service providers; county and state government officials; addiction, health, and mental health stakeholders; law enforcement; people with lived experience; family members of people who died from overdose; coroners; medical examiners; and others.^5,8

In New York, 16 HCS counties prioritized and deployed 213 EBPs and locally tailored communication campaigns.¹ We collected meeting minutes in 13 of the 16 counties, with meetings commonly lasting 2–4 hours, and most minutes were audio recorded.^1,9 Minutes captured how coalitions learn about EBPs, interpret local data, and identify local partners to implement EBPs.^8,10 In our paper, we examined the following questions: (1) In which EBP did stigma discussion cluster (MOUD vs OEND vs safer prescribing)? (2) Was stigma, framed as a myth about MOUD, a clinical workflow barrier (eg, pharmacies, health care settings), a public messaging challenge, or an equity concern?⁹ To answer these questions, minutes from the coalition meetings were analyzed to determine how coalitions learn, deliberate, and use data-driven decision making on the prioritization and deployment of EBPs in New York.¹⁰

Power of the Minutes in Community Coalition Meetings

Coalition meeting minutes in community-based research represent a rich and valuable source of data for understanding meeting dynamics and the issues discussed. However, these materials have historically been underutilized in community-based research because manual qualitative coding at this scale is both time intensive and costly.⁹ In addition, documentation practices for ongoing community coalition meetings can vary considerably and may include audio recordings, handwritten notes, summary notes, Zoom recordings, or AI-assisted documentation tools. Among these approaches, audio recording has been the most commonly used method because it provides the most complete capture of meeting deliberations and discussions. At the New York site of the HCS study, 127 meetings were audio-recorded and transcribed for analysis. In three counties, written summaries were used in place of recordings, and in the remaining counties, some meetings were not recorded because of technical difficulties or scheduling conflicts. Overall, we analyzed 127 coalition meeting transcripts drawn from 250 meetings conducted during the CTH intervention in the HCS.⁹

Human-in-the-Loop AI Changes What Is Feasible

We tested a credible workflow post hoc, for turning minutes into analyzable, implementation-relevant data while preserving human judgment where it matters most.⁹ This workflow used Columbia University's ChatGPT Enterprise secure platform to support coding and structured review while maintaining data security and privacy safeguards aligned with GDPR and HIPAA expectations. Data were housed on secure Columbia University IT servers for efficient processing, and topic segmentation was employed to manage summaries.⁹

We used Insanely Fast Whisper (IFW) model transcription (a fast implementation of Whisper) with transcription review assisted by ChatGPT to correct repeated sentences or nonsensical fragments.^9,11 Sentences were used as the unit of analysis, deidentified with spaCy's named entity recognition pipeline, cleaned, and stored with metadata (county, date, wave, sentence length).^9,12 After comparing LDA and BERTopic for topic modeling, BERTopic was selected because it generated topics that were more coherent, semantically meaningful, and easier to interpret. BERTopic combines contextual language embeddings with clustering algorithms and TF-IDF–based topic representation methods, allowing it to better capture semantic relationships between sentences and identify nuanced topic structures within the data. In addition, BERTopic provides the ability to detect and exclude outlier sentences that do not meaningfully align with the primary topic clusters. We selected BERTopic because it leveraged contextual word embeddings and was able to identify and exclude outlier sentences.¹³ From the 127 minutes, we extracted 67,969 sentences for analysis.⁹

Using a human-AI collaboration (HAIC) workflow encouraged the inclusion of human-in-the-loop, in which AI tools supported transcription review, data processing, and topic modeling, while investigators repeatedly reviewed outputs (eg, verifying deidentification and completeness) and provided human inputs to refine topics and ensure relevance for analysis. This preserves speed and scale while ensuring validity, equity sensitivity, and alignment with community meaning-making.⁹ Additionally, the PRISM-Capabilities Model served as a conceptual model to guide this human-in-the-loop approach.¹⁴ The PRISM-Capabilities Model provided an overall framework for research in which AI supports detection and synthesis, while humans and communities govern interpretation and decisions about action using an AI–human-in-the-loop approach through an ethically grounded process that prioritizes confidentiality, equity, and community governance in data collection and analysis.

Stigma, Equity, and Local Context Included in the Coalitions Conversation

One of the most consequential findings for addiction medicine to come from our research is that stigma discussions in the coalition meetings were linked to discussions of race/ethnicity.⁹ Specifically, stigma-related sentences were about 57% more likely to occur when racial or ethnic disparities were mentioned (OR=1.57; 95% CI, 1.22–2.03). Additionally, as the percentage of racial/ethnic minoritized populations increased in a community, so did the strength of the associations between EBP discussions and stigma. It is also important to focus on stigma in communities that tend to have a higher percentage of people of color who face more discrimination than counties that have fewer people of color, as found in our paper.

Using large language model (LLM) summaries to compare counties with statistically significant differences, we found that one county frequently linked stigma to barriers accessing MOUD within health care settings in the context of race/ethnicity and systemic discrimination, while another emphasized stigma reduction through community-based campaigns. This is a practical point for clinicians, implementation researchers, and policymakers: stigma is not a single target. It may require health system interventions in one community and community narrative strategies in another, or often both.

Addiction medicine practitioners, clinicians, and researchers have spent decades building the evidence base for MOUD and naloxone distribution,^2,3 yet coalition minutes from the HCS show that in the real work of community implementation of EBPs stigma continues to shape the deployment of EBPs, especially for MOUD.

Expanding Human-in-the-Loop AI for Implementation Research

Beyond the HCS study, this human-in-the-loop AI workflow should be used not only as a retrospective analytic strategy, but also as a broader implementation research method with potential application across addiction medicine and other community-engaged research. In this approach, AI can efficiently identify patterns across large volumes of routine narrative data, including team meetings, supervision notes, case conferences, implementation logs, and participant feedback, while human interpretation remains essential for determining the meaning, relevance, and appropriate use of those findings.

When applied to ongoing quality improvement efforts in health systems, community organizations, and public agencies, this model can help identify recurring workflow challenges, stigma-related dynamics, equity concerns, and implementation barriers that are often difficult to synthesize in real time through conventional methods alone. The same framework can also support monitoring of intervention fidelity and adaptation in large multisite clinical trials by flagging possible drift or site-level variation in delivery. However, this must be interpreted in context where community partners, frontline staff, and implementation teams are often best positioned to determine whether observed variation reflects poor fidelity, appropriate local adaptation, mistrust, stigmatizing communication, or structural barriers to participation. As such, the human-in-the-loop model used in this study is critical not only for analytic efficiency but also for ensuring that AI-supported implementation assessment remains contextually grounded, ethically informed, and responsive to community realities.