Criteria for considering studies for this review
Types of studies
Randomised, double-blind, controlled trials using a parallel group design that compare antidepressants with placebo as monotherapy.
We will include cross-over trials, randomised placebo-controlled trials with more than two arms, and cluster randomised placebo-controlled trials.
We will exclude quasi-randomised trials, such as those allocated by using alternate days of the week.
Types of participants
Participants aged 18 years or older with a primary diagnosis of panic disorder, with or without agoraphobia, diagnosed according to any of the following criteria: Feighner criteria, Research Diagnostic Criteria, DSM-III, DSM-III-R, DSM-IV or ICD-10. In case study eligibility focused on agoraphobia, rather than panic disorder, studies will be included if operationally diagnosed according to the above-named criteria and when it can be safely assumed that at least 30% of the participants were suffering from panic disorder as defined by the above criteria. There is evidence that over 95% of patients with agoraphobia seen clinically suffer from panic disorder as well (Goisman 1995). However, the effect of the inclusion of these studies will be examined in a sensitivity analysis.
We will exclude participants with serious comorbid physical disorders (e.g. myocardial infarction, chronic obstructive pulmonary disorder, uncontrolled diabetes, electrolyte disturbances) as they may confound treatment effectiveness and tolerability.
We will include participants with comorbid mental disorders, but the effect of including these participants will be examined in sensitivity analyses.
Types of interventions
Any trial comparing antidepressants as monotherapy with placebo in the treatment of panic disorder, with or without agoraphobia.
Only acute treatment studies treating participants for less than six months will be included. We will exclude relapse prevention studies.
We will include the following antidepressants.
Tricyclic antidepressants (TCAs): amitriptyline, amoxapine, clomipramine, desipramine, dosulepin/dothiepin, doxepin, impiramine, lofepramine, maprotiline, nortriptyline, proptriptyline, trimipramine.
Selective Serotonin Reuptake Inhibitors (SSRIs) : fluoxetine, fluvoxamine, sertraline, citalopram, paroxetine, escitalopram.
Monomine oxidase inhibitors (MAOIs): phenelzine, isocarboxazide, tranylcypromine, moclobemide, brofaromine.
Serotonin-Noradrenaline Reuptake Inhibitors (SNRIs): venlafaxine, desvenlafaxine, duloxetine, milnacipran.
Noradrenergic and Specific Serotonergic Antidepressants (NaSSAs): mirtazapine.
Noradrenergic and Dopaminergic Reuptake Inhibitors (NDRIs): bupropion.
Noradrenergic Reuptake Inhibitors (NRIs): reboxetine.
Others: agomelatine, trazodone, nefazodone, mianserin, maprotiline, non-conventional herbal products (e.g Hypericum).
We will include studies in which irregular (i.e. not daily) use of benzodiazepines took place. Excluding such studies would be meaningless because it has been documented that a minority of patients take benzodiazepines surreptitiously when they are prohibited by the protocol (Clark 1990). We will exclude studies in which benzodiazepines were regularly administered at a constant dosage for a long time or as part of the study medication. Possible differences in co-interventions (such as differential usage of benzodiazepines in antidepressant trials) will be noted and their influence will be examined in sensitivity analyses.
No restriction on dose, frequency, intensity or duration will be applied.
We will exclude studies administering psychosocial therapies targeted at panic disorder concurrently.
Types of outcome measures
1. Rate of 'response', i.e. substantial improvement from baseline as defined by the original investigators. Examples would be “very much or much improved” according to the Clinical Global Impression Change Scale, more than 40% reduction in the Panic Disorder Severity Scale score, and more than 50% reduction in the Fear Questionnaire Agoraphobia Subscale.
2. Total number of dropouts for any reason as a proxy measure of treatment acceptability.
3. 'Remission', i.e. satisfactory end-state as defined by global judgment of the original investigators. Examples would be “panic free” and “no or minimal symptoms” according to the Clinical Global Impression Severity Scale.
4. Panic symptom scales and global judgment on a continuous scale. Examples include Panic Disorder Severity Scale total score (0 to 28), Clinical Global Impression Severity Scale (1 to 7), and Clinical Global Impression Change Scale (1 to 7). When multiple measures are used, preference will be given in the order as above, with preference given to panic symptom scales. The actual measure entered into meta-analyses will be indicated at the top of the listings in the Table of Included Studies.
5. Frequency of panic attacks, as recorded, for example, by a panic diary.
6. Agoraphobia, as measured, for example, by the Fear Questionnaire, Mobility Inventory, or behavioural avoidance test.
7. General anxiety, as measured, for example, by the Hamilton Rating Scale for Anxiety, Beck Anxiety Inventory, State-Trait Anxiety Index, Sheehan Patient-Rated Anxiety Scale, or Anxiety Subscale of SCL-90-R.
8. Depression, as measured, for example, by the Hamilton Rating Scale for Depression, Beck Depression Inventory, or Depression Subscale of SCL-90-R.
9. Social functioning, as measured, for example, by the Sheehan Disability Scale, Global Assessment Scale, or Social Adjustment Scale-Self Report.
10. Quality of life, as measured for example by SF-36 or SF-12.
11. Patient satisfaction with treatment.
12. Economic costs.
13. Number of dropouts due to adverse effects.
14. Number of patients experiencing at least one adverse effect.
Timing of outcome assessment
All outcomes are short term which we define as acute phase treatment which normally would last two to six months.
When studies report response rates at different time points within two to six months, the time point closest to 12 weeks will be given preference.
Search methods for identification of studies
1. The Cochrane Depression, Anxiety and Neurosis Review Group's Specialised Register (CCDANCTR)
The Cochrane Depression, Anxiety and Neurosis Group (CCDAN) maintain two clinical trials registers at their editorial base in Bristol, UK: a references register and a studies-based register. The CCDANCTR-References Register contains over 31,500 reports of RCTs in depression, anxiety and neurosis. Approximately 65% of these references have been tagged to individual, coded trials. The coded trials are held in the CCDANCTR-Studies Register and records are linked between the two registers through the use of unique Study ID tags. Coding of trials is based on the EU-Psi coding manual, using a controlled vocabulary; please contact the CCDAN Trials Search Coordinator for further details. Reports of trials for inclusion in the Group's registers are collated from routine (weekly), generic searches of MEDLINE (1950-), EMBASE (1974-) and PsycINFO (1967-); quarterly searches of the Cochrane Central Register of Controlled Trials (CENTRAL) and review-specific searches of additional databases. Reports of trials are also sourced from international trials registers c/o the World Health Organization's trials portal (the International Clinical Trials Registry Platform (ICTRP)), pharmaceutical companies, the handsearching of key journals, conference proceedings and other (non-Cochrane) systematic reviews and meta-analyses.
Details of CCDAN's generic search strategies (used to identify RCTs) can be found on the Group's website.
The CCDAN registers will be searched using the following terms:
Diagnosis = "panic disorder" and Intervention = placebo*
Records will be screened for placebo-controlled antidepressant trials.
The References Register will be searched using the free-text term 'panic*' to identify additional untagged references. Abstracts will be screened for antidepressant trials and full-text articles will be retrieved, where necessary, to check for placebo controls.
2. National and international trials registers
Complementary searches will also be conducted on the WHO International Clinical Trials Registry Platform (ICTRP) and ClinicalTrials.gov.
Searching other resources
Review authors will check the reference lists of all included studies, non-Cochrane systematic reviews and major textbooks of affective disorders (written in English), for published reports and citations of unpublished research. A citation search will also be conducted via the Web of Science (included studies only) to identify additional works. We will also contact experts in the field.
Data collection and analysis
Selection of studies
The selection of trials for inclusion in this systematic review will be done independently by two of the authors: GG (clinical expertise) and MK (methodological expertise).
GG and MK will inspect the search hits by reading the titles and abstracts to see if they meet the inclusion criteria. Doubts will be resolved by consultation with the other co-authors. Each potentially-relevant study located in the search will be obtained as a full article and independently assessed for inclusion by two review authors and, in the case of discordance, resolution will be sought by discussion between the review authors. The discordance in the selection of studies will be calculated using Cohen's Kappa (k) (Cohen 1960), a more robust measure than a simple per cent agreement calculation since it takes into account the agreement between review authors that occurs by chance. Where it will not be possible to evaluate the study because of language problems or missing information, the study will be classified as 'study awaiting assessment' until a translation or further information can be obtained. The reasons for the exclusion of potentially-relevant trials will be reported in the 'Characteristics of excluded studies' table.
All decisions made during selection process, with numbers of studies and references, will be recorded and presented in a PRISMA flow diagram (Moher 2009) at the end of the review.
Data extraction and management
Two review authors will use a data extraction form to independently extract the data from included studies concerning participant characteristics (age, sex, severity of panic disorder, study setting), intervention details (dosage, duration of study, sponsorship), study characteristics (blinding, allocation etc) and outcome measures of interest. The extraction sheet will be piloted on a sample of 10% of the included studies. Again, any disagreement will be resolved by consensus or by the third member of the review team. If necessary, we will contact authors of studies to obtain clarification.
Antidepressants as a whole versus placebo.
TCAs versus placebo.
SSRIs versus placebo.
MAOIs versus placebo.
SNRIs versus placebo.
NaSSAs versus placebo.
NDRIs versus placebo.
NRIs versus placebo.
Other antidepressants versus placebo.
Assessment of risk of bias in included studies
Two authors will independently assess the risk of bias in included studies using the tool described in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011). This tool encourages consideration of how the sequence was generated, how allocation was concealed, the integrity of blinding at outcome assessment, the completeness of outcome data, selective reporting and other biases. We will also consider sponsorship bias.
The risk of bias, in each domain and overall, will be assessed and categorised into:
low risk of bias, plausible bias unlikely to seriously alter the results;
high risk of bias, plausible bias that seriously weakens confidence in the results;
unclear risk of bias, plausible bias that raises some doubt about the results.
If the assessors disagree, the final rating will be made by consensus or with the involvement of another member of the review group. Where inadequate details of randomisation and other characteristics of trials are provided, we will contact authors of the studies in order to obtain further information. Non-concurrence in quality assessment will also be reported.
Measures of treatment effect
The main outcome result will be reduction of severity of panic and agoraphobia symptoms. The improvement will usually be presented as a change in a panic disorder scale(s) (mean and standard deviation) or as a dichotomous outcome (responder or non-responder, remitted or not-remitted), or both.
Binary or dichotomous data
For binary outcomes we will calculate a standard estimation of the random-effects model risk ratio (RR) and its 95% confidence interval (CI). It has been shown that a random-effects model has a good generalisability (Furukawa 2002) and that RR is more intuitive (Boissel 1999) than odds ratio. Furthermore, odds ratios tend to be interpreted as RR by clinicians (Deeks 2000). This may lead to an overestimation of the impression of the effect (Higgins 2011). For all primary outcomes we will calculate the number needed to treat to benefit or harm statistic (NNTb or NNTh) and its 95% CI using Visual Rx (http://www.nntonline.net/), taking account of the event rate in the control group.
(a) Summary statistics
It is likely that different studies have used varied panic rating scales; therefore we will use standardised mean difference (SMD). If all included studies have used the same instrument, we will use mean difference (MD).
(b) Endpoint versus change data
Trials usually report results either using endpoint means and standard deviation of scales or using change in mean values from baseline of assessment rating scales. We prefer to use scale endpoint data, which typically cannot have negative values and are easier to interpret from a clinical point of view. If endpoint data are unavailable, we will use the change data in separate analyses. In case we use MD, we will pool results based on change data and endpoint data in the same analysis.
Unit of analysis issues
Crossover trials are trials in which all participants receive both the control and intervention treatment but in a different order. The major problem is a carryover effect from the first phase to the second phase of the study, especially if the condition of interest is unstable (Elbourne 2002). As this is the case with panic disorder, randomised crossover studies will be included but only data up to the point of first crossover will be used.
Studies with multiple treatment groups
Where a study involves more than two treatment arms, especially two appropriate dose groups of the same drug, the different dose arms will be pooled and considered to be one. If the arms involve one placebo arm and two or more arms of different classes of antidepressants, we will compare each arm with placebo separately. In this case, a possibility of unit-of-analysis error can occur, due to the unaddressed between the estimated intervention effects from multiple comparisons (Higgins 2011), resulting in double counting. In order to avoid that, we will include each pair-wise comparison separately, according to the recommendations in the Cochrane Handbook for Systematic Reviews of Interventions, section 16.5.4 (Higgins 2011). If the variable is dichotomous, we will divide the shared intervention group evenly among the comparisons. If the variable is continuous, only the total number of participants will be divided up, and the means and standard deviations will be left unchanged.
Cluster randomised trials
In cluster randomised trials groups of individuals rather than individuals are randomised to different interventions. If we identify cluster placebo-controlled randomised trials, we plan to use the generic inverse variance technique, if such trials have been appropriately analysed taking into account intraclass correlation coefficients to adjust for cluster effects. Where trialists have not adjusted for the effects of clustering, we will attempt to do this by obtaining an intracluster correlation coefficient and then following the guidance given in chapter 16.3.4 of the Cochrane Handbook (Higgins 2011).
Dealing with missing data
We will try to contact the study authors for all relevant missing data.
(1) Dichotomous outcomes
Response, or remission on treatment, will be calculated using an intention-to-treat analysis (ITT). We will follow the principle 'once randomised always analysed'. Where participants left the study before the intended endpoint, it will be assumed that they would have experienced the negative outcome. The validity of the above assumption will be tested by sensitivity analysis, applying worst and best case scenarios. When dichotomous outcomes are not reported but the baseline mean and standard deviation on a panic disorder scale are reported, we will calculate the number of responding or remitted participants according to a validated imputation method (Furukawa 2005). The validity of the above approach will be analysed by sensitivity analysis. If necessary, authors of studies will be contacted to obtain data and/or clarification.
(2) Continuous outcomes
Concerning continuous data, the Cochrane Handbook recommends avoiding imputation of continuous data and suggests using the data as presented by the original authors. Where ITT data are available they will be preferred to 'per-protocol analysis'. If necessary, authors of studies will be contacted to obtain data and/or clarification.
(3) Skewed or qualitative data
Skewed and qualitative data will be presented descriptively.
Several strategies will be considered for skewed data. If papers report a mean and standard deviation and there is also an absolute minimum possible value for the outcome, we will divide the mean by the standard deviation. If this is less than two then we will conclude that there is some indication of skewness. If it is less than one (that is the standard deviation is bigger than the mean) then there is almost certainly skewness. If papers have not reported the skewness and simply report means, standard deviations and sample sizes, these numbers will be used. Because there is a possibility that these data may not have been properly analysed, and can also be misleading, analyses will be conducted with and without these studies. If the data have been log-transformed for analysis, and the geometric means are reported, skewness will be reduced. This is the recommended method of analysis of skewed data (Higgins 2011). If papers use non-parametric tests and describe averages using medians, they cannot be formally pooled in the analysis. We will follow the recommendation made in the Cochrane Handbook that results of these studies be reported in a table in our review, along with all other papers. This means that the data will not be lost from the review and the results can be considered when drawing conclusions, even if they cannot be formally pooled in the analyses.
(4) Missing statistics
When only P or standard error (SE) values are reported, we will calculate standard deviations (SDs) (Altman 1996). In the absence of supplementary data after requests to the authors, the SDs will be calculated according to a validated imputation method (Furukawa 2006). We will examine the validity of these imputations in the sensitivity analyses.
Assessment of heterogeneity
Following the Cochrane Handbook recommendations, we will quantify heterogeneity using the I2 statistic. The Cochrane Handbook recommends overlapping intervals for I2 interpretation (section 9.5.2, Higgins 2011) , as follows:
0% to 40%: might not be important;
30% to 60%: may represent moderate heterogeneity;
50% to 90%: may represent substantial heterogeneity; and
75% to 100%: considerable heterogeneity.
We will also use the Chi2 test and its P value to determine the direction and magnitude of the treatment effects. In a meta-analysis of few trials Chi2 will be underpowered to detect heterogeneity, if it exists. P = 0.10 will be used as a threshold of statistical significance.
Assessment of reporting biases
Reporting biases arise when the dissemination of research findings is influenced by the nature and direction of results. These are described in section 10 of the Cochrane Handbook (Higgins 2011). A funnel plot is usually used to investigate publication bias. However, it has a limited role when there are only few studies of similar size. Secondly, asymmetry of a funnel plot does not always reflect publication bias. Visual inspection of funnel plots will be used to assess publication bias as well as the statistical test for funnel plot asymmetry proposed by Eggers or Rücker (Higgins 2011). We will not use funnel plots for outcomes if there are 10 or fewer studies, or if all studies are of similar size.
We will use a random-effects model to calculate the treatment effects. We prefer the random-effects model as it takes into account differences between studies even when there is no evidence of statistical heterogeneity. It gives a more conservative estimate than the fixed-effect model. We note that the random-effects model gives added weight to small studies, which can either increase or decrease the effect size. We will apply a fixed-effect model, on primary outcomes only, to see whether it markedly changes the effect size.
Subgroup analysis and investigation of heterogeneity
Subgroup analyses are often exploratory in nature and should be interpreted cautiously: firstly, because they often involve multiple analyses leading to false positive results; and secondly, these analyses lack power and are more likely to result in false positive results. Keeping in mind these reservations, we will perform the following subgroup analyses.
For classes of antidepressants, such as TCAs, SSRIs, and others.
For participants with agoraphobia and for participants without agoraphobia, because the same treatment may have differential effectiveness with regard to panic and agoraphobia.
If groups within any of the subgroups are found to be significantly different from one another, we will run meta-regression for exploratory analyses of additive or multiplicative influences of the variables in question.
We will compare acute phase treatment studies that last for less than four months versus acute phase treatment studies that last for four months or more.
The following sensitivity analyses are planned. We will examine if the results change and check for the robustness of the observed findings by:
Excluding trials with high risk of bias (i.e. trials with inadequate allocation concealment and blinding, with incomplete data reporting and/or with high probability of selective reporting);
Excluding trials with dropout rates greater than 20%;
Excluding studies funded by the pharmaceutical company marketing each antidepressant. This sensitivity analysis is particularly important in view of the repeated findings that funding strongly affects outcomes of research studies (Als-Nielsen 2003; Lexchin 2003; Bhandari 2004) and because industry sponsorship and authorship of clinical trial reports have increased over the last 20 years (Buchkowsky 2004);
Excluding studies whose protocols do not explicitly prohibit concomitant use of BDZ. According to Clark et al (Clark 1990), 10% to 20% of those assigned to placebo or imipramine arms in a randomised controlled trial took explicitly-prohibited anxiolytic medication; and
Excluding studies whose participants clearly have significant psychiatric co-morbidities including primary or secondary depressive disorders.
Our routine application of random-effects and fixed-effect models as well as our secondary outcomes of remission rates and continuous severity measures might be considered additional forms of sensitivity analyses.
'Summary of findings' table
We will summarise the findings using a 'Summary of findings' table, applying the GRADE approach (Higgins 2011).