Contradictions, methodological flaws, and potential for misinterpretations in ranking treatments of depression

the NICE guidelines as prioritizing some treatments over others, as appears to be the case by Malhi et al. 1 For example, for less severe depression, Malhi et al. state 1, p. 468 : “Notably, antidepressants are ranked below CBT, BA, and IPT but are recommended ahead of STPP.” For more severe depression, Malhi et al. concluded from the NICE treatment ranking 1, p. 468 : “This suggests that individuals with severe acute depression should be offered CBT, BA, antidepressant, individual problem- solving, and counseling prior to considering STPP…”. According to the NICE recommendations, however, treatment ranking is secondary to patient preferences and other factors when it comes to deciding which first- line treatment to pro-vide. 2, p. 13, 45 For example, NICE emphasizes 2, p. 44– 45 : “Discuss treatment options with people who have a new episode of more severe depression, and match their choice of treatment to their clinical needs and preferences… use table 2 and the visual summary to guide and inform the conversation [and] take into account that all treatments

In this journal, Malhi et al. recently argue 1 that the UK National Institute for Health and Care Excellence (NICE) guidelines for depression 2 rank short-term psychodynamic therapy (STPP) as last of 11 therapies recommended for less severe depression and 7 out of 10 treatments recommended for more severe depression.They stress that STPP was ranked by NICE below counseling and that individual cognitive-behavior therapy (CBT) has the highest ranking. 1 However, we argue that the NICE guidelines for depression are ambiguous in that they recommend multiple treatments as equal first-line treatments for depression on the one hand, but then go on to rank order them in terms of effectiveness and cost-effectiveness based on the NICE guideline's committee interpretation of the available evidence.We are concerned that this ambiguity leads to possible misinterpretations of the evidence as evidenced by Malhi and colleagues.Furthermore, we briefly summarize several methodological flaws in the NICE guidelines proposed ranking of treatments, which further questions the prioritization of one treatment over another in the treatment of depression.Indeed, in their main document, the NICE guidelines recommend several treatments as first-line treatments for less severe depression, emphasizing that patient preferences and other factors such as experiences with previous treatments are important in deciding the type of treatment that is offered to a given patient 2, p. 13, 45 : "….take into account that all treatments in table 1 can be used as first-line treatments" (NICE, 2022, p. 28).Similarly, for more severe depression NICE clearly states that "all treatments in table 2 can be used as first-line treatments" (p.45).As we will discuss below this is in line with the head-to-head comparisons conducted by NICE themselves as well as independent meta-analytic evidence.Yet, contradicting these recommendations, NICE 2, p. 13, 45 also rank orders for these treatments based "on the committee's interpretation of their clinical and cost effectiveness and consideration of implementation factors".This contradiction creates considerable ambiguity and allows for (mis-)interpretation of the NICE guidelines as prioritizing some treatments over others, as appears to be the case by Malhi et al. 1 For example, for less severe depression, Malhi et al. state 1, p. 468 : "Notably, antidepressants are ranked below CBT, BA, and IPT but are recommended ahead of STPP."For more severe depression, Malhi et al. concluded from the NICE treatment ranking 1, p. 468 : "This suggests that individuals with severe acute depression should be offered CBT, BA, antidepressant, individual problem-solving, and counseling prior to considering STPP…".p. 13, 45 For example, NICE emphasizes 2, p. 44-45 : "Discuss treatment options with people who have a new episode of more severe depression, and match their choice of treatment to their clinical needs and preferences… use table 2 and the visual summary to guide and inform the conversation [and] take into account that all treatments COMMENTARY in table 2 can be used as first-line treatments".This is consistent with the fact that few statistically or clinically significant differences were found in head-to-head comparisons of the treatments listed by NICE as first-line as discussed below.In their reading of the NICE guidelines, Malhi and colleagues do not mention the emphasis in the NICE guidelines on patient preference and other factors and that NICE stresses that all the listed treatments can be offered as a first-line treatment.
Furthermore, as described in more detail elsewhere, 3 the validity of the NICE treatment ranking itself is questionable because of the methodological procedures applied by the NICE committee and the committee's interpretation of results.Our concerns about the NICE treatment ranking can be summarized as follows: 1. Based on effect sizes between each treatment and placebo or treatment as usual (TAU), the NICE committee concluded that some treatments appeared to be more effective than others. 2wever, assuming differences between two treatments if one shows a descriptively larger effect size compared to a control condition than another treatment without comparing the effect sizes directly by a statistical test depicts a statistical flaw. 3,4Even following the NICE committee's approach, the differences between the treatment-control effect sizes of STPP compared to CBT or SSRIs (0.16-0.25) are below the minimal clinically significant difference defined by the NICE committee itself (SMD, standardized mean difference = 0.50).3 With differences in effect sizes below the threshold of clinical significance defined by the NICE committee itself, treatment ranking carries considerable uncertainty.3

Malhi et al. mistakenly assert that the NICE treatment ranking
p. 467 In fact, however, this was not the case, they were based on indirect comparisons of the various treatments with TAU or placebo as described above (see 1).
4. The NICE committee did report head-to-head comparisons of active treatments but only in a supplement and did not take these results as a basis for deriving treatment rankings.The reasons for this omission are not clear.In less severe depression the NICE direct comparisons found a few clinically significant differences including only one for STPP, which showed a statistically and clinically significant superiority of STPP over counseling (SMD = − 0.61, 95% CI −1.05, −0.17). 2,3NICE, however, ranked STPP below counseling. 2 In more severe depression the differences between individual behavioral therapy, individual CBT, individual interpersonal therapy (IPT), and individual STPP were neither statistically nor clinically significant (SMD < 0.50). 3Consistent with these results a comprehensive meta-analysis available to both the NICE committee and Mahli et al. did not find CBT to be superior to other psychotherapies in depressive disorders.5 6.As described in the previous article, 3 the cost-effectiveness analyses additionally used by NICE to establish a hierarchy of treatment recommendations 2 may not be valid as well, since they were based on the questionable indirect comparisons described above.2 7. Finally, an explicit link between evidence and recommendations is missing.The NICE committee found it "difficult… to link the recommendations directly to the NMA results".6, B, p. 48, 66 Therefore the committee based their recommendations ultimately on "their clinical experience".6, B, p. 66 However, it is unclear whether clinical experience can offer any solid guidance when treatment differences are modest, uncertainty is high and bias is substantial.
In summary, neither the indirect nor direct comparisons carried out by the NICE committee nor the cost-effectiveness analyses or independent research 5  In conclusion, we argue that the NICE guidelines for depression are ambiguous and even contradictory.This ambiguity may easily lead to misinterpretations as done by Malhi and colleagues resulting in a misrepresentation of the evidence for psychodynamic psychotherapy and other types of psychotherapy.Furthermore, we highlighted several methodological flaws in the NICE treatment ranking.Presently it is not clear which patients benefit from which empirically-supported treatment.Thus, we continue to discourage the devaluing of efficacious treatments so that as many patients as possible may benefit from them.

CO N FLI C T O F I NTER E S T S TATEM ENT
The following authors have been trained in PDT: FL, AA, PL, and CS.
support Malhi et al.'s claim of superiority of counseling over STPP, prioritizing CBT over other treatments, and ranking STPP among the least effective treatments. 1 SR has been trained in CBT but has mainly done research on psychodynamic therapy.NH is presently in training of PDT.PL received royalties from Guilford Press, Wiley, Routledge, and Cambridge University Press.AA received royalties from Seven Leaves Press.FL received royalties from Hogrefe Publisher.