Adherence to sleep restriction therapy – An evaluation of existing measures

Sleep restriction, a key element of cognitive behavioural therapy for insomnia, involves considerable behavioural changes in patients' lives, leading to side‐effects like increased daytime sleepiness. Studies on sleep restriction rarely report adherence, and when assessed it is often limited to the average number of therapy sessions attended. This study aims to systematically evaluate different measures of adherence to cognitive behavioural therapy for insomnia and their relationship with treatment outcome. This is a secondary analysis of data from a randomized controlled trial investigating cognitive behavioural therapy for insomnia (Johann et al. (2020) Journal of Sleep Research, 29, e13102). The sample included 23 patients diagnosed with insomnia according to DSM‐5 criteria who underwent 8 weeks of cognitive behavioural therapy for insomnia. The following adherence measures based on sleep diary data were used: number of sessions completed; deviations from agreed time in bed; average percentage of patients deviating from bedtime by 15, 30 or 60 min; variability of bedtime and wake‐up time; change in time in bed from pre‐ to post‐assessment. Treatment outcome was assessed using the Insomnia Severity Index. Multiple regression models were employed, and insomnia severity was controlled for. Results showed that none of the adherence measures predict insomnia severity. Baseline insomnia severity, dysfunctional thoughts and attitudes about sleep, depression or perfectionism did not predict adherence. The limited variance in the outcome parameter due to most patients benefiting from treatment and the small sample size may explain these findings. Additionally, using objective measures like actigraphy could provide a better understanding of adherence behaviour. Lastly, the presence of perfectionism in patients with insomnia may have mitigated adherence problems in this study.


| INTRODUCTION
Empirical research concerning adherence to cognitive behavioural therapy for insomnia (CBT-I) has increased substantially over the last decades, with a particular focus on sleep restriction and stimulus control therapies (Agnew et al., 2021).These two behavioural components can present a significant burden for some patients, increasing the likelihood of adverse effects that impair adherence.For example, during sleep restriction therapy (SRT), it is recommended to temporarily reduce the time spent in bed.Patients are asked to adjust their sleep window, that is, to spend less time in bed (TIB), to increase homeostatic sleep pressure and sleep efficiency.Here, the emergence of adverse side-effects like excessive daytime sleepiness or physical symptoms such as headaches may accentuate pre-existing impairments in daytime functioning (Kyle et al., 2011).Throughout the treatment period, the implementation of certain lifestyle changes associated with CBT-I can pose significant challenges, particularly when addressing long-standing habits.That said, even minimal improvements to adherence are associated with enhanced CBT-I treatment outcomes (Trockel et al., 2014).Unfortunately, adherence is often a secondary research question and not the primary research focus of clinical studies.
In the current paper, we summarize these systematic reviews with a particular focus on SRT, alongside a secondary analysis of adherence data from a randomized controlled trial of CBT-I (Johann et al., 2020) systemically examining different predictors and several measures of adherence to SRT.As such, we provide an overview of advantages and disadvantages of the measures, provide an evaluation of existing adherence measures, and present an outlook for future research on CBT-I adherence.
Formerly, the term compliance has been used in the literature instead of the term adherence.However, by definition, compliance implies a degree of medical authority and passiveness of the patient.
Because this is at odds with the interpersonal behaviour of most clinicians/therapists and psychotherapy clients, the term adherence is currently favoured in the literature emphasizing the active engagement of patients.Three meanings of the term adherence can be distinguished (Mellor et al., 2022), as follows.
1. Therapy adherence: the patient persistently enacts the given treatment recommendations.
2. Study adherence: the patient fills in all questionnaires and takes all study measures asked for.
3. Therapist adherence: the therapist follows the treatment manual of the study (this is more often called treatment fidelity in the literature).
The focus of this work is on therapy adherence.While adherence to medication is a quite simple concept ("Does the patient take the prescribed pill as prescribed?"), adherence to CBT-I is more complex because it is a multicomponent intervention involving the implementation of multiple behaviours as well as an active engagement of the patients in questioning dysfunctional attitudes and beliefs.Additionally, there is a large variability in the therapists' implementation of CBT-I components (see an example of the variation in SRT as reported by Kyle et al., 2015).Thus, a measure of adherence that may be appropriate in one study may not be meaningful in another one.
Prior to 2013, when the first of the above-mentioned systematic reviews was published, studies on CBT-I rarely examined behavioural indicators of adherence.With that in mind, Matthews et al. (2013) summarized that most studies only reported session attendance and/or overall study attrition as indicators for disengagement with treatment.The problem with this metric is that disengagement with the treatment does not necessarily imply non-adherence to the treatment recommendations.While non-attrition, as a prerequisite for adherence, is easy to assess and analyse, an in-depth understanding of process-outcome relations may be more helpful to refine CBT-I with the aim of increasing its effectiveness.In fact, the field has clearly developed in this direction, as can be seen from the two more recent systematic reviews (Agnew et al., 2021;Mellor et al., 2022).
All three systematic reviews on CBT-I adherence report a large variability in adherence measures across clinical trials as well as large variability in the processing and reporting of these measures.
CBT-I adherence has been measured with single-or multiple-item questionnaires, sleep diaries, structured or semi-structured interviews, and actigraphy.For questionnaires, little consistency of phrasing questions was observed and only a few validated questionnaires measuring global adherence exist to date, for example, Patient Adherence Form (Koffel et al., 2018;Trockel et al., 2014), Treatment Adherence Rating Scale (Dolsen et al., 2017;Dong et al., 2018) and Therapist-Rated Adherence Questionnaire (Vincent & Hameed, 2003).
In the literature, adherence was either assessed globally or with respect to specific CBT-I components.Components that were assessed are sleep hygiene education (SHE), SRT, stimulus control therapy (SCT), SCT/SRT combined, relaxation therapy (RT), and cognitive therapy (CT) using questionnaires, sleep diary data and actigraphy.Most of the studies focused on adherence to SRT, SCT or the combination of both.
Adherence measures in the literature were retrieved either directly from patients, the clinician/therapist, or a person closely related to the patient (e.g.spouse).Direct measures ask questions such as "Did you adhere to treatment recommendation xy?".When adherence measures were derived indirectly from the sleep diary, they were mostly related to the question whether patients adhered to their prescribed bedtimes.In some studies, other parameters like daytime naps or alcohol use were also examined indirectly via sleep diary.
The reviews report that calculations derived from sleep diaries varied substantially, and limit the ability to quantify the average deviations from prescribed bedtimes or variability in bedtimes across studies (Mellor et al., 2022).Another issue is that there are no generally accepted standards for thresholds of adherent or non-adherent behaviour (Agnew et al., 2021).Studies differ regarding the timepoint of assessing adherence, that is, some studies assessed adherence directly after each session, others at the end of treatment.On average, studies reported about two adherence measures (Mellor et al., 2022).
Of all behaviours, adherence to SRT is easiest to capture (although not easiest to operationalize) because the behaviour of going to bed and getting up at a certain time is captured by the sleep diary.So, adherence to SRT is often operationalized as the actual times corresponding to the prescribed times.However, inconsistencies occur because times could mean total TIB, risetime or bedtime, and because of different thresholds for the deviation from the prescribed time to be considered non-adherence.The following measures of SRT have been reported in the literature (Agnew et al., 2021).
7. Mean proportion of TIB reduction that was adhered to (n = 1).8. Scores ranging from 0 to 49 measuring adherence to individual components (n = 1).Agnew et al. (2021) and Mellor et al. (2022) found that many variables have been tested as predictors of adherence to CBT-I.According to that, better sleep before and after treatment sessions predicts better adherence (global, SRT and SCT), fewer dysfunctional beliefs at baseline predict better adherence (global, SRT and SCT/SRT combined), greater social support predicts adherence to SRT and SCT, greater self-efficacy predicts better adherence (global, SHE and SCT), and higher motivation predicts better adherence to SRT.Some of these predictors have been only investigated in one or two studies.
For example, self-efficacy has been tested in two studies and motivation only in one.No consistent evidence has been found for demographics, general health variables, comorbidities (somatic and psychiatric), insomnia severity and other baseline insomnia variables, sleep questionnaires and other sleep-related variables, features of the intervention and other psychological parameters.Baseline insomnia severity is the variable that has been most often investigated as a predictor of adherence, but results do not suggest a reliable association (Agnew et al., 2021;Mellor et al., 2022).
Previous studies have found that lower adherence (global, therapist-rated adherence to SCT and SHE) predicts lower posttreatment insomnia severity.Clinician-rated adherence studies found that adherence is related to higher Insomnia Severity Index (ISI) score reduction and insomnia remission.Additionally, clinician/therapistrated adherence predicted reduction of dysfunctional beliefs, less sleep-related impairment and better sleep quality.These results need to be judged with caution because there are also studies that do not show these relations (Agnew et al., 2021;Mellor et al., 2022).Furthermore, clinician/therapist-rated adherence may be biased by treatment response.
All three systematic reviews called for an increased and systematic investigation of adherence to CBT-I components, and Agnew et al. (2021) recommended focusing on individual components.In the current paper, we report on a secondary analysis of data from a randomized controlled trial (Johann et al., 2020), and we focus on the analysis of adherence to SRT measured indirectly via sleep diaries.We analyse if baseline insomnia severity, dysfunctional beliefs and attitudes about sleep, baseline depression severity, and perfectionism predict adherence.Additionally, we test if adherence predicts insomnia severity.TIB was 4 hr.On a weekly basis, TIB was increased by 30 min when sleep efficiency was ≥ 90%, decreased by 30 min when sleep efficiency was < 80%, and not changed when sleep efficiency was ≥ 80 and < 90%.SCT was conducted following Bootzin (1972).CT included cognitive restructuring, constructive worry, and paradoxical intention.RT included progressive muscle relaxation and autogenic training.

| Adherence and outcome measures
We calculated different measures of adherence based on sleep diary data.Sleep diaries were filled in for 1 week before treatment, during treatment, and for 1 week following treatment.
First, deviation of TIB was calculated as the mean of all differences between prescribed TIB and actual TIB (

| Statistical analysis
All analyses were conducted with the statistical software R (Beck et al., 1961;R Core Team, 2022).For the analyses reported in this manuscript, the subset of 23 patients treated with CBT-I was used.
The analysis focuses on adherence to SRT, which was assessed indirectly by comparing sleep diary-derived TIB with prescribed TIB as extracted from the video-recordings.
To assess the relationship between adherence and outcomes, multiple linear regression models were used.The number of attended sessions, TIB deviation, absolute TIB deviation, percentages of days patients adhering to their prescribed TIB within 15, 30 and 60 min of deviation, bedtime and risetime variability, and pre-to post-treatment TIB change were entered as independent variables in these models.
Post-treatment ISI scores were used as dependent variables in the regression models.
Additionally, linear regression models were used to investigate whether baseline insomnia (ISI), dysfunctional belief and attitudes about sleep (DBAS-16), depression (BDI) and perfectionism (FMPS) predict adherence (measures: TIB deviation, the absolute TIB deviation, the percentages of days patients adhering to their prescribed TIB within 15, 30 and 60 min of deviation and the bedtime and risetime variability, pre-to post-treatment TIB change).

| Sample characteristics
Three patients refused to be video-recorded.Thus, prescribed TIB were available for 20 of 23 patients of the CBT-I group and these were included in the final analysis.Patients in the CBT-I group attended 7.6 ± 0.8 treatment sessions in 66.2 ± 37.8 days.None of the patients worsened, 12 of 23 patients responded (ISI change ≥ 8), and 14 out of 23 patients remitted (ISI post ≤ 7).

| Descriptive statistics for adherence measures
Study participants provided 20.1 ± 13.7 days of sleep diary data, which could be matched with prescribed TIB from video-recordings.
On average, actual TIB was 8.5 ± 14.9 min longer than prescribed TIB, the average of the absolute values of these deviations was 23.8 ± 15.3 min.Patients spent 27.7 ± 20.4% of days 15 min or longer in bed than prescribed, 14.4 ± 14.4% of days at least 30 min longer in bed than prescribed, and 6.7 ± 9.9% of days at least 60 min longer in bed than prescribed.The average within-week variance for bedtime was 50.2 ± 38.7 min and 49.9 ± 40.0 min for risetime.TIB was reduced by 60.6 ± 80.5 min from pre-to post-treatment.

| Prediction of treatment outcome by adherence measures
When controlling for pre-treatment insomnia severity, none of the investigated adherence measures significantly predicted posttreatment insomnia severity, the primary outcome of the trial: number

| DISCUSSION
In the current study, none of the examined adherence measures predicted treatment outcome, and none of the examined baseline measures predicted treatment adherence.Several factors might play a role for this pattern of null findings.First, a link between adherence and outcome is more likely to be found when a certain amount of variance is present in the variables of interest.While this appears to be true for a number of adherence measures of the current work, the vast majority of patients responded quite well to the treatment reducing the variance of the outcome variable.In particular, none of the patients worsened and 14 out of 23 patients remitted.Second, the current study is based on a small sample size for this kind of analysis.While the sample size of the trial was sufficient to detect CBT-I efficacy (Johann et al., 2020), the analysis of treatment adherence does not include data of control groups and is correlational in nature.Thus, even in properly powered clinical trials, the power of post hoc adherence analyses is usually low, which is also true for this study.To get a better overall picture of the relationship between adherence and CBT-I outcome, future research should pool data from several clinical studies and create a common database like in the field of depression (see www.metapsy.org).This would require some work to homogenize existing datasets with regard to calculation and reporting of adherence measures (Agnew et al., 2021;Mellor et al., 2022).Third, the current study focused on adherence to SRT and on adherence measures that were derived from sleep diary data.While we believe that this was a reasonable decision from a clinical point of view and in light of the available CBT-I literature on the adherence-outcome link, a more complete evaluation may include direct measures of adherence such as those derived from actigraphy or mattress sensors.Lastly, the link between adherence and treatment outcome might not be as strong as one would expect from a clinical point of view.One potential reason for this is that adherence may be specifically related to maladaptive perfectionism, which in turn is related to increased sleep effort, presleep arousal, and dysfunctional beliefs and attitudes about sleep (Johann et al., 2022).Thus, the presumed positive effects of high levels of adherence on CBT-I outcomes may be dampened by the potential negative effects of perfectionism on sleep (Johann et al., 2017(Johann et al., , 2018;;Stricker et al., 2023).We agree with this recommendation and demonstrate again with the present study that adherence can be evaluated even if it was not the primary outcome measure of the parent study.Like in many CBT-I studies, sleep diaries were used in our trial to guide SRT and to evaluate treatment effects.Thus, adherence to the behavioural components of CBT-I can be examined using existing data and it would be a missed opportunity if these data were not made publicly available.
The emergence of open science practices may also help to re-analyse other previous clinical trials with the aim of creating a large and homogenous database before drawing firm conclusions about CBT-I adherence.
We would like to add to this recommendation that the field may benefit most if researchers are able to assess several different adherence measures and not just one.Ideally, the empirical evidence will then help clinicians to reach a consensus about what constitutes optimal adherence.Additionally, it should be borne in mind that adherence measures may differ between different CBT-I delivery formats, for example, digital interventions versus face-to-face psychotherapy.
To understand the process-outcome relation between adherence and treatment efficacy, it could also be useful to include continuous assessments of outcomes in CBT-I trials, for example, weekly assessments of insomnia severity.For example, it may be that strict adherence is important in the first weeks of the intervention and, vice versa, it may even be a consequence of a sufficient treatment response when patients adhere less rigidly to prescribed TIB toward the end of treatment.For an additional understanding, it seems to be valuable to measure adherence at follow-up timepoints.
Further, we agree that studies on CBT-I adherence should also cover adherence to the single components of the treatment.For a modular intervention like CBT-I, a global approach to assessing adherence may fail to capture the complexity of the issue.For example, the Treatment Adherence Rating Scale is a standardized measure to assess global adherence, with three items representing treatment understanding, treatment agreeableness and homework completion.
In our clinical experience, this instrument is sometimes difficult to use as a therapist because patients may adhere differently to different components of the treatment.To solve this problem, a questionnaire specifically designed for assessing CBT-I global and componentrelated adherence may be developed in the future (see also Bouchard et al., 2003;Koffel et al., 2018;Trockel et al., 2014  We think the most informative measure to report on is raw number deviations.In our example, many patients had days with shorter and days with longer TIB in comparison to the prescribed TIB.Bedtimes that are shorter than the prescribed ones are not necessarily an indicator of non-adherence during SRT.For example, when SRT is combined with SCT, shorter than prescribed bedtimes can even be in line with recommendations given by the therapist (e.g."go to bed only when sleepy").Thus, bedtimes that are shorter than the prescribed bedtimes could either be ignored (set to 0) when calculating average deviations or set to negative values potentially compensating for bedtimes that are longer than prescribed bedtimes in other nights.Surprisingly, the precise formula for calculating bedtime deviations is not clearly outlined in many CBT-I adherence studies.
To date, there appears to be insufficient empirically based evidence to establish significant cut-offs.The most mentioned cut-offs in the literature are the ones provided in the recommendation above, but no definitive recommendations for optimal cut-offs to guide clinicians can be derived from this.In the absence of such recommendations, it may be worthwhile to reach a theory-driven consensus on reasonable cut-offs.Nonetheless, a data-driven approach (e.g.dose-response studies examining the relationship between the dose of adherence and outcome) is preferable in the long run.
Third recommendation: Report variance of bedtime, TIB and risetime in minutes, including clear descriptions of how variance is calculated (Agnew et al., 2021).
The variability of bed-and risetimes and TIB is an interesting measure to consider, but it may also be misleading in some instances.For example, part of SRT is the recommendation to go to bed only if sleepy at the prescribed time and to delay bedtime until feeling sleepy.Accordingly, variability of bedtime may be a result of patients following this recommendation accurately.Thus, the importance of adherence to SRT in its truest sense (which may encourage variability in bedtimes) or consistency in bed-and risetimes needs to be further investigated.

Fourth recommendation:
Explore other objective measurements of adherence in clinical trials of CBT-I and to establish a gold-standard that is (most) unaffected by bias (Agnew et al., 2021).
All three reviews highlighted considerable heterogeneity within subjective measures of adherence.Objective measures are rarely used.
It remains an open question whether objective measures can provide better adherence predictions.We believe that utilizing objective measures such as actigraphy or mattress sensors has the potential to reduce 2 | METHODS 2.1 | Participants The study sample consisted of 46 patients meeting DSM-5 criteria for insomnia disorder.Half of the sample was randomized into an intervention group receiving CBT-I (n = 23), and half of the sample was randomized into a waitlist control group (n = 23).The sample consisted of outpatient sleep clinic patients aged 18-65 years.Medication affecting sleep was not allowed 2 weeks prior and during study participation.Patients with another comorbid somatic, psychiatric or sleep disorder, night shift work, suicidality and previous treatment with CBT-I were also excluded.For excluding other sleep disorders, all patients spent two consecutive nights in the sleep laboratory of the Department of Psychiatry and Psychotherapy at the Medical Center -University of Freiburg, Germany.For more details of the inclusion and exclusion criteria, see Johann et al. (2020).The study protocol received ethical approval by the Institutional Review Board of the University Medical Centre Freiburg.All patients signed a written informed consent form before participating in the study.

2. 2 |
Cognitive behavioural therapy for insomnia The CBT-I was provided in an individual face-to-face format of eight weekly sessions (duration: 50 min each) by authors AJ and KS.The treatment included SHE, SRT, SCT, CT and RT.Sleep hygiene included general information about sleep, insomnia and behavioural or environmental factors that might interfere with sleep, like exercise, caffeine, alcohol use, light, noise and temperature in the bedroom.During SRT, the initial sleep window was set based on the average sleep duration of 1 week according to sleep diary data.Timing of the sleep window was chosen in accordance with patient's preference, and minimum days of the participant; pTIB = prescribed TIB; aTIB = actual TIB).Second, absolute deviation of TIB was calculated as the mean of the absolute values of all differences between prescribed TIB and actual TIB ( of days during which patients adhered to prescribed TIB with a deviation of 15, 30 and 60 min, respectively.Because longer but not shorter TIB compared with prescribed TIB are an indicator of non-adherence, we used positive values only and not the absolute values to calculate these deviations.Fourth, variability of bedtime and risetime as a measure of regularity was calculated using the variance of bed-and risetimes calculated separately for each week of each patient.Subsequently, these variances were averaged for each patient across weeks.Finally, TIB change from pre-to post-treatment was assessed.Most treatment sessions were video-recorded for quality assurance.These recordings were used to determine prescribed TIB.The following questionnaires were used: ISI(Bastien et al., 2001), brief version of the Dysfunctional Beliefs and Attitudes about Sleep Scale (DBAS-16;Morin et al., 2007), Beck Depression Inventory (BDI;Beck et al., 1961) and Frost Multidimensional Perfectionism Scale   (FMPS;Frost et al., 1990).
Agnew et al. (2021) made five recommendations for designing future research on adherence to CBT-I.We would like to reflect on four of these recommendations and complement them with some further ideas.First recommendation:Clinical trials with CBT-I need to include a measure of adherence.There needs to be a focus on the individual components, and clinicians need to work toward a consensus of what constitutes optimal adherence to CBT-I(Agnew et al., 2021).
Raw minute deviations from TIB, bedtime and risetime; (2) number of days participants were within 15 min of bedtime, 15 min within their TIBs, and 30 min within their risetimes; and (3) number of participants who, on average, were within 15 min of bedtime, 15 min within their TIBs, and 30 min within their risetimes.If possible, the authors encourage investigators to report adherence to other cut-offs in supplementary material (e.g. 30 and 60 min of TIB; Agnew et al., 2021).
heterogeneity and to provide a more comprehensive understanding of adherence to CBT-I.Additionally, as CBT-I-based digital interventions become more and more available, different measures of adherence that are specific to this format can be evaluated.These measures can be automatically assessed like the amount of time spent on the treatment program and provide a more objective way of measuring adherence.Based on the findings of this study, it appears that different CBT-I adherence measures do not have a significant impact on treatment outcome.Further, we could not identify predictors for adherence.Despite this, it is important to acknowledge the value of negative results in scientific research as they can contribute to an enhanced understanding of the topic.One promising direction for future research in this field is the adoption of open science practices and data pooling.By openly sharing data and methodology across multiple studies, more statistical power and, in turn, a more comprehensive view of the factors that contribute to the effectiveness of CBT-I might be achieved.This approach could be used to guide the development of a more personalized treatment and to enhance the quality of care for individuals struggling with insomnia.