Evaluation of real‐world outcomes associated with use of a prescription digital therapeutic to treat substance use disorders

Abstract Background and Objectives Digital therapeutics can expand the reach and fidelity of behavioral treatment for substance use disorders (SUDs). This analysis evaluated real‐world engagement and clinical outcomes in patients diagnosed with SUD who were prescribed reSET®, an FDA‐authorized prescription digital therapeutic (PDT). Methods Patients were prescribed a 12‐week PDT comprising 61 therapy lessons (31 “core” and 30 “keep learning” lessons) and contingency management rewards (positive reinforcement message or monetary gift cards) based on lesson completion and negative urine drug screens. Engagement (defined as any activity in the PDT), retention (any activity in Weeks 9–12), and substance use data were collected automatically by the PDT and analyzed descriptively. Associations between early lesson completion and end‐of‐treatment outcomes were assessed. Results Six hundred and fifty‐eight patients filled their prescription. Evaluated were 602 patients who were exposed to therapeutic content by completing at least one lesson (median age 37 years, 33% female, 41% male, 26% unreported sex). Median lessons completed was 33 (out of 61 possible), and 52% of patients completed all core modules. Retention in treatment during the last 4 weeks of treatment was 74%, and 62% were abstinent (missing data considered positive). [Correction added on 13 December 2022, after first online publication: In the preceding sentence, the treatment percentage values were revised from 74.6% to 74%.] Discussion and Conclusions Patients with SUD exhibited robust engagement with a PDT, high rates of retention through 12 weeks, and substantial rates of abstinence at end of treatment when the therapeutic was used in a real‐world setting. PDT's hold promise as a new way to access effective SUD treatment. Scientific Significance This study is the first to report real‐world PDT engagement and clinical outcomes data from a large, geographically diverse population of patients with SUDs.


INTRODUCTION
Substance use disorder (SUD), which includes dependence on alcohol, cannabis, stimulants, sedatives, or opioids, affects up to 41 million people in the United States, and yet recent data show that only about 6.5% of these people ever receive SUD treatment. 1 Evidence-based behavioral treatments for SUD exist, but even among patients who access SUD treatment, most do not receive these therapies due to a lack of specialty facilities and/or clinicians trained in evidence-based psychological or behavioral therapies. 2 Other barriers to widespread use of evidence-based behavioral approaches include inconsistent delivery, quality, and fidelity across healthcare providers, and high turnover among providers. 3 These issues are more acute in rural communities, where substance use treatment centers or addiction specialists may not exist or be prohibitively distant. 3 Means of digitally delivering evidence-based treatments remotely with fidelity and with minimal clinician involvement may help overcome limitations of existing treatment approaches while improving outcomes.
Patient attrition from SUD treatment is also a common barrier to successful recovery. Roughly a third of those starting an SUD treatment program quit within the first month, 4 and attrition rates of 50% or higher have been reported within the first 3 months of treatment. 5 Research has shown a positive association between active engagement in treatment, improved retention, and successful recovery. 6 Further, there are examples of this association in studies involving digital therapeutics. One randomized trial evaluating a smartphone application for alcohol dependence found that the number of days the application was used and number of pages viewed were significantly associated with a reduction in the number of risky drinking days. 7 Another study of a computer-based behavioral intervention for SUD found that the number of sessions attended and number of homework assignments completed were positively associated with abstinence. 8 Prescription digital therapeutics (PDTs) are software-based disease treatments that adhere to standards of Good Manufacturing Practices, have been evaluated for safety and efficacy in randomized controlled trials (RCTs), and are authorized by the US Food and Drug Administration (FDA). Prescribed by treating clinicians, and delivered on mobile devices, PDTs may expand access to evidence-based therapies including those used in the treatment of SUDs. This is especially useful when the SUD involves cocaine, cannabis, and stimulants such as methamphetamines, for which no FDA-approved pharmacotherapies exist.
The reSET ® PDT delivers a 12-week therapeutic program combining cognitive behavioral therapy (CBT) based on the Community Reinforcement Approach (CRA), 9 a contingency management (CM) system providing motivational incentives (positive reinforcement messages or monetary gift cards) for lesson completion and abstinence, and fluency training to reinforce concept mastery. Studies have consistently demonstrated that CM interventions, particularly abstinence-based incentives, can support treatment and recovery in individuals with a wide range of SUDs. 10 After a clinician prescribes reSET, patients download the therapeutic, enter an access code, and set a password, which allows them to begin working and learning in the PDT. The cost of the PDT is either covered by some form of health insurance or, rarely, by patients themselves. The effectiveness of the treatment program available in the PDT was evaluated in several RCTs involving approximately 1500 patients with SUDs. These studies used the Therapeutic Education System, which was the computer-based precursor product to reSET. [12][13][14][15][16] The trials demonstrated improved rates of abstinence and treatment retention among patients receiving the computer-delivered therapy as an adjunct to standard care compared to those who received treatment-as-usual alone. This study is the first to report RWD in a large, geographically diverse population of patients with SUDs who were treated with a PDT. Engagement and other therapeutic use data were analysed on a population level with descriptive statistics. Patient engagement was assesed two ways. An "active day" was defined as a day in which a patient used any feature of reSET (i.e., opening the app, viewing a lesson, completing assessments, recording triggers or cravings, or reporting abstinence status). Engagement was also assessed by counting completion of therapeutic lessons during the 12 weeks of treatment.

METHODS
Substance use was evaluated as a composite of patient selfreports recorded via the PDT as well as urine drug screens (UDS) performed and recorded by clinicians via the clinician dashboard. As a real-world evaluation, the data available for these analyses were either entered by the clinicians via the dashboard or by patients' use and data entry into the PDT.
Abstinence was evaluated during the last 4 weeks (Weeks 9-12) of the prescription, an assessment window that is based on the Clinical Trials Network Treatment Effect and Assessment Measures Task Force recommendations to evaluate the effect of a treatment based on abstinence during the last month of treatment 20 and that has been used in previous trials of this therapeutic. 14,15 Abstinence during the assessment window was considered a binary outcome, based on available data for each week. An "abstinent week" was defined as a week with no positive self-reported use or UDS. Consistent with prior real-world and observational studies, 21 missing abstinence data for any given week were imputed in two ways. The first considered an individual positive if either any self-report or UDS was positive in the last 4 weeks or if no data were available ("missing data positive"). This analysis included the entire population of individuals who engaged with the PDT (n = 602). The second approach was "missing data excluded" in which patients without any self-reports or UDS results during any of the last 4 weeks were excluded from the analysis and only patients with either positive or negative reports were included (n = 434). To access lessons, patients were required to report drug or alcohol use every 4 days (yes/no; with no requirement to identify specific drugs used) via the PDT. Clinician-entered negative UDSs were rewarded by giving the patient a chance to earn CM rewards.
Alternatively, to look at substance use data across the entire prescription as a secondary measure, patients were also classified as "responders" defined as having ≥80% of the 12 weeks with negative UDS or self-reports of no drug use (weeks without data assumed positive for use).
Retention in treatment was defined as any patient activity in the PDT during the last 4 weeks of the prescription. Associations were explored between early engagement with the PDT (i.e., number of lessons completed in Weeks 1-4) and abstinence or treatment retention outcomes in Weeks 9-12.

RESULTS
This analysis evaluated 602 patients with SUD from 28 states who received their first prescription for reSET, consented to use, filled their prescription, and were exposed to therapeutic content by completing at least one lesson (

Abstinence
Four hundred and thirty-four patients (72%) provided at least one substance use self-report during Weeks 9-12. A total of 92 patients (15%) had at least one clinician-entered UDS report during Weeks 9-12.
In an analysis combining self-report and UDS data, abstinence in Weeks 9-12 was 62% using the "missing data positive" analysis (N = 602) and 86% were abstinent using the "missing data excluded" (N = 434) analysis ( Figure 3). Use of the responder criteria identified 40.9% of patients who had an overall abstinence rate ≥80% across the duration of the prescription when missing weeks were assumed to be positive. F I G U R E 3 Abstinence in Weeks 9-12 of treatment. Percentage of patients who were abstinent in the last 4 weeks of treatment in an analysis combining self-report and UDS data using two methods of missing data imputation. "Missing data positive" means that patients who had no abstinence data available in the last 4 weeks were considered nonabstinent (green bar). "Missing data excluded" means that patients who had no abstinence data available in the last 4 weeks (n = 168) were excluded from the analysis (black bar).

Associations with early engagement
F I G U R E 4 Association between early engagement and end of treatment outcomes. Positive association between early engagement (lessons completed) and end-of-treatment outcomes. The average number of modules completed in each of the first 4 weeks of treatment (total lessons/ 4 and rounded) and the percentage of patients who were either abstinent (green bars) or who were retained in treatment (black bars) in the last 4 weeks of treatment.
was 81% among the patients who completed an average 4 or more lessons in Weeks 1-4, but only 26% among the 65 patients who completed an average of fewer than 1 lesson per week in Weeks 1-4.

DISCUSSION
Patient attrition from SUD treatment is a common barrier to successful recovery, thus treatments or programs that increase engagement with, and retention in treatment may support recovery from SUDs. The RWD from this evaluation strongly suggest that adults of all ages who fill a prescription for a PDT to treat SUD readily engage with the PDT, Retention rates in SUD treatment are typically low and highly variable and depend on many circumstances including characteristics of the treatment population, philosophy, content, and location of the treatment program. The 74% retention rate (any activity in the PDT in Weeks 9-12) we observed is notable, especially as this evaluation was not a clinical trial, but rather a report of RWD from a broad sample of SUD patients who were assigned the intervention by a clinician and began using it after filling their prescription. These patients were diverse in terms of age, geographic location, and the setting in which they were treated.
The engagement and retention data we observed are consistent with, or higher than, those observed in studies of some other interventions for SUD, including a patient-centered behavioral intervention (49%), 22 outpatient drug-free programs (25%), 6 and residential SUD treatment programs (48%). 6 With only 6.5% of patients with SUD receiving any treatment, and high levels of attrition among patients who do receive treatment, a therapeutic that can be conveniently used throughout the day (i.e., outside of typical clinic hours) and is associated with high rates of engagement and retention, offers the potential for overcoming existing limitations in addiction treatment programs.
The abstinence rate of 62% we observed (using a conservative "patients with no data positive" analysis) is encouraging. This rate is higher than has been observed in RCTs such as in the pivotal trials on which FDA approval of reSET 15  does not collect information related to the structure and format of treatment programs in which the patient is enrolled, nor about the nature of concurrent pharmaceutical and/or behavioral treatments being received. This limits the ability to examine demographic subgroups or to discern any potential interactions associated with treatment context or receipt of other therapies.
Although data captured by the PDT regarding app utilization and lesson completion is a strength, drug use data input was not systematic and relied on patient self-reports and clinician entry of urinalysis data.
Interpretation is limited by variable and low rates of self-report and low rates of urinalysis reporting, resulting in heavy reliance on selfreport data. The use in these analyses of a composite measure of abstinence (self-report and/or UDS), however, is consistent with other RWE and observational studies. Two studies used self-report/UDS data, 24,25 and two have used self-reported abstinence data only. 26,27 Unlike a clinical trial where UDS and/or self-report are collected systematically to evaluate effectiveness, decisions to collect UDS in real-world clinical practice settings are determined by the care model of a particular addiction facility, by clinicians based on the specific needs and status of individual patients, or by attempts to minimize unnecessary testing. For these reasons, the amount and quality of UDS data available for analysis are limited. Missing data is also an issue; data are reported both for samples with and without data available at end of treatment. The responder analysis represents a conservative, worst-case scenario, imputing missing data as positive, thus is heavily impacted by the decline in available self-report and UDS data. Interpretation of abstinence rates at end of treatment is limited by lack of pretreatment drug use information and may underestimate or overestimate real rates depending on the case-for example, patients who begin therapy already in remission from drug use have a good prognosis for long-term abstinence. 28