In the past three decades, at least 160 controlled and comparative studies have examined the effects of psychological treatments compared to control conditions and to other treatments.1 This large number of studies has clearly shown that most psychological treatments studied in a trial have large effects on depression. Psychological treatments that have proved effective include cognitive-behaviour therapy,2 behavioural activation treatments,3 interpersonal psychotherapy,4 problem-solving treatment5, 6, and life review for older adults.7 These treatments are not only effective in adults with depression, but also in older adults,8 in women with postpartum depression,9 and in patients with both depression and general medical disorders, including multiple sclerosis,10, stroke patients,11 and cancer patients.12 Psychological treatments can be delivered effectively in group format, individual format, or guided self-help format.13, 14 Probably, the latest contribution to this list is the provision of treatment via the Internet.15
One group of studies has focused on the comparative effects of psychological and pharmacological treatments. This research has shown that the effects of both types of treatment are comparable.16 It is also clear that a combined treatment is somewhat more effective than treatment with pharmacotherapy alone.17, 18 It is not very clear, however, whether combined treatment is more effective than a psychological treatment alone. Clinicians often have different views on this, in particular when it comes to more severe depression when combined treatments often are recommended
Although several studies have examined the effects of combined treatment, the results are not entirely conclusive. Some studies find no significant difference.19–21 whereas others have found that combined treatment is significantly more effective than psychological treatment alone.23, 24There are even a few studies found that psychological treatment alone is more effective than combined treatment.25, 26
Because it can be expected that the difference between psychological and combined treatments are small, large sample sizes are required to find significant differences. When small effect sizes are expected in individual studies, meta-analytical techniques can be used to integrate the results of individual studies and to increase the statistical power. Although several systematic reviews have examined the difference between psychological and combined treatments,27, 28 this question has not been examined in recent statistical meta-analyses using up-to-date methods. For example, one meta-analysis conducted in this area has included only a small portion of the available studies (8 of the 18 studies we have identified, see below.17 This meta-analysis also focused on several research questions, did not pool the results of all studies together, and did not for example conduct subgroup analyses to examine the characteristics of the studies, which were related to the differential effects of the psychological and combined treatments. It did, however, find some indications that combined treatment was somewhat more effective than psychological treatment alone. Another small meta-analysis did examine the comparative effects of psychological versus combined treatment,29 but focused only on the seven studies that examined outpatients. This study also found that combined treatment was superior to psychological treatment alone.
We decided to conduct a new, comprehensive meta-analysis in which all available studies are included. We wanted to examine whether the small benefit of combined treatment, which was found in the earlier meta-analyses,17, 29 would also be found when the number of examined studies was increased. Finally, we wanted to examine whether study characteristics were related to the relative effects of psychological and combined treatments.
IDENTIFICATION AND SELECTION OF STUDIES
First, we used a large database of studies on the psychological treatment of depression in general. This database has been described in detail elsewhere.3, 5, 6, 8 It was developed through a comprehensive literature search (from 1966 to December 2007) in which we examined 7,911 abstracts in Pubmed (1,403 abstracts), Psycinfo (2,097), Embase (2,207), and the Cochrane Central Register of Controlled Trials (2,204). We identified these abstracts by combining terms indicative of psychological treatment (psychotherapy, psychological treatment, cognitive therapy, behaviour therapy, interpersonal therapy, counselling, family therapy, marital therapy, problem-solving therapy, psychodynamic therapy, psychoanalysis, relaxation, reminiscence, life review) and depression (both MeSH-terms and textwords). For this database, we also collected the primary studies from 22 meta-analyses of psychological treatment of depression.1 We retrieved a total of 857 papers for further study. These papers were studied, and we selected the ones which examined the relative effects of psychological and combined treatments of depression.
Second, references of earlier reviews and meta-analyses were examined.1, 16–18, 27–28 Finally, the references of retrieved papers were checked.
We included studies in which (1) a psychological intervention (2) was compared to a combined treatment (3) consisting of the same psychological intervention plus an antidepressant, (4) in a randomized trial, (5) aimed at subjects with depression (as diagnosed through a clinical interview and/or a self-report questionnaire). Psychological interventions were defined as treatments in which verbal communication between a therapist and a client was the core element and in which the communication of the therapist is based on a specific theoretical framework about the causes of depression and how the communication can reduce it. Studies were included when all randomized participants had a depressive disorder (or scored above a cut-off point on a self-report instrument); comorbid general medical or psychiatric disorders were not used as an exclusion criterion. No language restrictions or age limits were applied. We excluded studies on maintenance treatments and studies in which a psychological treatment plus placebo were compared to a psychological treatment plus active antidepressant, because it cannot be ruled out that the placebo has an effect in itself.
At least 25 scales have been used to assess the validity and quality of RCTs,30 but evidence of their reliability and validity is lacking. We adopted the Cochrane Handbook's 4 criteria,30 to assess study validity: (1) randomization to conditions conducted by an independent (third) party; (2) concealment of randomization to conditions; (3) blinding of assessors of outcome; (4) completeness of follow-up data.
We directly compared the psychological and combined treatments on depressive symptomatology (continuous outcomes), recovery (dichotomous outcomes), and drop-out.
Effects on depressive symptomatology (continuous outcomes)
We calculated effect sizes indicating the difference between psychological and combined treatments. The effect sizes (d) were calculated by subtracting (at post-test) the average score of the psychological treatment group (Mp) from the average score of the combined treatment group (Mc) and dividing the result by the pooled standard deviations of the experimental and control groups (SDpc). An effect size of .5 thus indicates that the mean of the psychological treatment group is half a standard deviation larger than the mean of the combined treatment group. Effect sizes of .8 can be assumed to be large, whereas effect sizes of .5 are moderate, and effect sizes of .2 are small.31 In the calculations of effect sizes, only those instruments were used that explicitly measure depression (Table 1). If more than one depression measure was used, the mean of the effect sizes was calculated, so that each study (or contrast group) had only one effect size. When means and standard deviations were not reported, we used other statistics (t-value, p-value) to calculate effect sizes.
Table 1. Selected studies of randomized studies comparing the effects of psychological treatment to the effects of combined treatment
1]Abbreviations (alphabetical): Activ sched: activity scheduling; AD: antidepressant medication; AUS: Australia; CAN: Canada; CBASP: cognitive behavioural-analysis system of psychotherapy; CBT: cognitive-behaviour therapy; Clin: clinical recruitment; Comm: community recruitment; DO: drop-out; GER: Germany; GP: general practitioner; IPT: interpersonal psychotherapy; Mar ther: marital therapy; MDD: major depressive disorder; MEX: Mexico; Mn: months; NL: The Netherlands; NR: not reported; Nrsp: number of respondents; Nses: number of sessions; PLA: placebo; PST: problem-solving therapy; Psychodyn: psychodynamic; Psychother: psychotherapy; Recr: recruitment; Soc. skills tr: social skills training; Supp: support; Ther: therapy; UK: United Kingdom; US: United States of America.
To calculate pooled mean effect sizes, we used the computer program Comprehensive Meta-analysis (version 2.2.021), developed for support in meta-analysis. We conducted all analyses using both the fixed effects model and the random effects model.25 As an indicator of homogeneity, we calculated the Q-statistic. We also calculated the I2-statistic, which is an indicator of heterogeneity in percentages. A value of 0% indicates no observed heterogeneity, and larger values show increasing heterogeneity, with 25% as low, 50% as moderate, and 75% as high heterogeneity.
We examined whether specific characteristics of the studies were related to the effect sizes by using the procedures for subgroup analyses as implemented in Comprehensive Meta-analysis. In these analyses we used the mixed effects model. This means that the random effects model is used to calculate the effect size for each subgroup, whereas the fixed effects model tests the difference between the subgroups of studies.
We conducted subgroup analyses on the selected characteristics of the three core elements of the studies: participants, interventions, and study characteristics. With regard to the participants we examined the recruitment method (clinical samples versus other samples) and target group (adults in general versus specific population). We also examined the type of psychological intervention (CBT versus other types of psychological treatments) and the type of pharmacological treatment (tricyclics, SSRIs, or other). General characteristics of the studies we reviewed in subgroup analyses were: drop out rate (less than 20% drop-out versus 20% or more drop-out), type of analyses (intention-to-treat analyses versus completers-only analyses), and the period in which the study was conducted (before 1995 versus 1995 or later).
Effects on recovery (dichotomous outcomes)
We calculated the relative risk (RR) of recovery in subjects receiving psychological treatment compared to subjects receiving combined treatment. Again, we conducted all meta-analyses both with the fixed effects model and with the random effects model, using the Comprehensive Meta-analysis (version 2.2.021) computer program, and we calculated the Q-statistic and the I2-statistic to estimate heterogeneity between study outcomes.
Publication bias was tested by inspecting the funnel plot on primary outcome measures, and by Duval and Tweedie's trim and fill procedure, which yields an estimate of the effect size after the publication bias has been taken into account (as implemented in Comprehensive Meta-analysis, version 2.2.021).
We also calculated “Orwin's fail safe N.” This number indicates how many studies with an effect size of zero should be found to reduce the effect size that is found to a lower value (for example, .10). A larger N indicates that the effect size found can be further generalized.
DESCRIPTION OF INCLUDED STUDIES
All inclusion criteria were met by 19 studies, with a total of 1,838 subjects (934 in the psychological treatment groups, and 904 in the combined treatment groups). Selected characteristics of these studies are described in Table 1.
Fourteen studies were aimed at adults in general, and five at specific populations (two at adults with HIV; one focused on multiple sclerosis patients; one on adults with chronic depression; and one on older adults). Eleven studies were aimed at clinical populations, six at subjects recruited from the community, and two did not report the recruitment method or they used systematic screening to recruit patients. In all but one study only subjects who met diagnostic criteria for a depressive disorder (including two studies specifically aimed at patients with dysthymia) were included. In eight studies, CBT was used as psychological treatment. In three studies, amitryptiline was used as the antidepressant medication, in three other studies imipramine was used, a further three had desipramine, two had fluoxetine, and the remaining 12 used another medication or used more than one type of medication. Six studies were conducted before 1990, five between 1990 and 1999, and the remaining eight after 2000. Seventeen studies were published in English, one in Spanish, and one in German.
QUALITY OF THE INCLUDED STUDIES
The quality of the studies varied. Only four studies reported that allocation to conditions was conducted by an independent party. Concealment of random allocation to respondents was not possible or not reported in any of the studies, whereas blinding of assessors was reported in 11 studies. Drop-out numbers ranged from 6 to 55%.
CONTINUOUS OUTCOMES AT POST-TEST
In 17 studies (19 comparisons), effect sizes could be calculated which indicated the difference between psychological and combined treatments at post-test. The mean effect size indicating the difference between psychological and combined treatments was .35 (95% CI: .24∼.45; P<.001) both in the fixed and the random effects model. Heterogeneity was very low (Q=15.85 n.s.; I2=0). The results of these analyses are summarized in Table 2 and Figure 1.
Table 2. Meta-analyses of studies comparing the effects of psychological treatments on depression compared to combined treatments at post-test: continuous outcomes
Because the weight of one study was very high21 (28.50%), we also conducted a meta-analysis in which this study was removed. The results were almost the same (fixed effects and random effects model: d=.27; 95% CI: .15∼.39, P<.001; I2=0).
Two studies contained more than one comparison of a psychological versus a combined treatment.32, 33 We included the four comparisons from these studies in our meta-analysis. Because these comparisons were not independent of each other, we examined whether this influenced the outcomes. We conducted a new meta-analysis in which only the smallest effect size from each of these two studies was included, but found that this scarcely influenced the results (fixed effects and random effects model: D=0.36; 95% CI: .25∼.46; Z=6.56, P<.001; Q=15.39 n.s.; I2=0).
In 15 studies (17 comparisons), the Hamilton Rating Scale for Depression (HRSD) was used. When only this scale was used for the calculation of the effect sizes, a mean effect size of .35 (95% CI: .24∼.46) was found with the fixed effects model, and .31 (95% CI: .18∼.45) with the random effects model (with low heterogeneity: Q=19.73 n.s.; I2=18.91). The fifteen comparisons using the Beck Depression Inventory (BDI) resulted in a mean effect size of 0.28 (95% CI: 0.12∼0.43), both with the fixed and the random effects model (Q=9.86, n.s.; I2=0).
We conducted several subgroup analyses (Table 2). No significant difference was found between the effect size of studies in which clinically referred patients were examined and those in which subjects were recruited in other ways (mostly through community recruitment). We also found no difference between older (<1995) and newer (>1995) studies; between studies which used intention-to-treat analyses and those using completers-only analyses; and between studies with a drop-out rate below 20% and those with a drop-out rate of 20% or higher.
We found a significant difference (P<.05) between studies which used cognitive behavioural intervention and those in which other types of psychological treatments were found (Table 2). For that comparison the advantage of combination was less when medication was added to CBT. We also found a trend showing that studies using tricyclic medications or SSRIs were used had smaller effect sizes than studies using another medication (nefazodone)24 or more than one type of medication.22, 41
We found a trend (P<.1) indicating a significant difference between studies aimed at adults with depression and the effect size of studies targeting specific populations (two studies on HIV-patients; one study on adults with chronic depression, one on patients with multiple sclerosis, and one study on older adults).
DICHOTOMOUS OUTCOMES AT POST-TEST
The RR indicating the difference between psychological and combined treatments in recovery was .78 (95% CI: .71∼0.860; 17 comparisons; Z=−4.91; P<.001), both in the fixed and the random effects model. Heterogeneity was low (Q=12.90; n.s.; I2=0). The results of these analyses are summarized in Table 3 and Figure 2.
Table 3. Meta-analyses of studies comparing the effects of psychological treatments on depression compared to combined treatments at post-test: Dichotomous outcomes
We conducted the same subgroup analyses as with the continuous outcomes (Table 3). However, none of the variables we examined in these analyses resulted in a significant difference between subgroups (including the three variables that were found to be significant in the analyses with the continuous outcomes).
Neither the funnel plots nor Duval and Tweedie's trim and fill procedure pointed at a significant publication bias. And this was true both in the analyses with the continuous outcomes and in the analyses with the dichotomous outcomes. The effect size indicating the difference in depressive symptomatology between psychological treatment and combined treatment conditions did not change after adjustment for possible publication bias (the observed and adjusted d were identical), and this was also true for the RR indicating the difference between psychological and combined treatments.
We then calculated Orwin's fail-safe N, which is the number of studies with an effect size of zero that should be found to reduce the mean effect size (indicating the difference between psychological and combined treatments) to .10; this was found to be 44. The number of studies (with an RR of 1) needed to reduce the RR from 0.78 to 0.90 was found to be 24.
EFFECTS AT FOLLOW-UP
In seven studies (10 comparisons) the difference between psychological and combined treatments at follow-up were presented. The results of these analyses are presented in Table 2. The follow-up periods ranged from 1 month to 2 years. As there was only one study which reported the results at 2-year follow-up, this study was not included in the analyses. Nor did we include the study in which a follow-up of one month was used.
None of the analyses indicated a significant difference between psychological and combined treatments at 3–6 months follow-up, and at 12 months follow-up. Because of the small number of comparisons and the differences in follow-up periods, we did not analyse the results at follow-up any further.
We found clear evidence that a combined psychological and pharmacological treatment of depression is superior to psychological treatment alone in the short term. This was true when we pooled different effect sizes, but also when we examined specific outcome measures. Heterogeneity was low in these analyses, indicating that the studies, target populations, and interventions are probably highly comparable to each other. We found no indication of publication bias, and fail-safe analyses showed that a large number of studies with no difference would have to be found to reduce the overall effects on a nonsignificant outcome. Therefore, there should be little doubt that there is indeed a difference in effects between psychological and combined treatments. Because the effect size indicating the difference between psychological and combined treatments was small to moderate, several individual studies failed to find a significant difference, and it is a clear advantage of a meta-analytic approach that makes it possible to combine the results of multiple studies and validate such small effect sizes.
However, the effect size indicating the difference between psychological and combined treatments is small to moderate. Effect sizes lower than 0.30 are usually considered to be small, and our effect size was only just above that threshold (d=0.35). Although our finding was statistically significant, it is not clear whether this has any clinical relevance. To obtain an idea of the clinical relevance of these effect sizes, the use of the binomial effect size display is recommended.46, 47 This displays the difference in success rate between the two treatments. When we compare psychological treatment to combined treatment, the binomial effect size display (based on d=.35) indicates that the success rate increases from .41 to .59. This suggests that combined treatment is superior for some patients and that combined treatment should be offered to patients before psychological treatment alone is considered.
However, these results should be considered with caution. In our study, we did not examine the comparative effects of pharmacotherapy and combined treatment. Several studies have found that combined treatment is more effective than pharmacotherapy alone.21, 33, 48, 49 If this is indeed the case, then combined treatment should be considered as a first line treatment, as it is superior to both psychological treatment alone and pharmacological treatment alone. However, if pharmacotherapy appears to be equally effective as combined treatment, it may be better to consider pharmacotherapy first and only offer combined treatment when pharmacotherapy alone does not work. Because many patients prefer psychological treatments and are not willing to take antidepressants,50 psychological treatments (without additional pharmacotherapy) will remain an important part of routine care, even when combined treatments are clearly superior. More (meta-analytic) research is needed to examine these questions.
One reason why the suggested superiority of combined over psychological treatment alone should be considered with caution is that no difference was found in the longer term. This is probably at least in part related to the small number of available effect sizes at follow-up. It does mean that there is currently no evidence that combined treatment is superior to psychological treatment alone in the longer term.
Our findings should also be considered in the light of the study's limitations. First, we found that the quality of a number of the examined studies was not optimal. Although it is clearly inherent in studies of research on psychological treatments that it is not possible to conceal to subjects to which condition they are assigned, many studies did not meet other major quality criteria, such as assignment to conditions by an independent person, and blinding of assessors. Another important limitation of this meta-analysis is that we were only able to include a relatively small number of studies. Therefore, the results of our analyses should be considered very cautiously.
This meta-analysis only reports about average effects. However, the average patient does not exist, and it is entirely possible that different types of patients are best served by different treatment strategies. One finding of this study is research on specific populations found significantly larger differences between psychological and combined treatments than studies in adult outpatients. It is possible that psychological treatments are less effective in some of these populations, which increases the effects of the added pharmacological treatments. But it also possible that the relative effects of psychological and combined treatments are related to severity, chronicity, and comorbid psychiatric or general medical disorder. More research is needed to examine these issues.
We found a trend indicating that the difference between psychological and combined treatments is smaller when cognitive behavioral therapy is used compared to other psychological treatments. It could be possible that cognitive behavioral treatments have stronger effects than other psychological treatments. In that case, the additional value of the pharmacological treatment may be limited. Although there is some evidence that cognitive behavioral treatments are more effective than other treatments,2, 51 the evidence is certainly not conclusive.52
Although we did find that combined treatment is more effective than psychological treatment alone, more research is needed to examine this difference. Further research is called for on the relative efficacy of psychological and combined treatments in the longer term. More basic research is needed required to explore the mechanisms through which both treatments work. And more clinical research is necessary to examine which treatment or combination of treatments is effective at in the longer term.