Criteria for considering studies for this review
Types of studies
Randomised, double-blind, controlled trials using a parallel-group design that compare azapirones with placebo as monotherapy.
We will include cross-over trials, randomised placebo-controlled trials with more than two arms and cluster-randomised placebo-controlled trials.
We will exclude quasi-randomised trials, such as those in which allocation is performed by using alternate days of the week.
Types of participants
Participants 18 years of age or older with a primary diagnosis of panic disorder, with or without agoraphobia, diagnosed according to any of the following criteria: Feighner criteria, Research Diagnostic Criteria, DSM-III, DSM-III-R, DSM-IV or International Classification of Diseases, 10th Edition (ICD-10). In case study eligibility focused on agoraphobia, rather than panic disorder, studies are included if operationally diagnosed according to the above-named criteria, and when it can be safely assumed that at least 30% of participants were suffering from panic disorder as defined by the above criteria. Evidence suggests that more than 95% of participants with agoraphobia seen clinically suffer from panic disorder as well (Goisman 1995). However, the effect of the inclusion of these studies will be examined in a sensitivity analysis.
We will exclude participants with serious comorbid physical disorders (e.g. myocardial infarction, chronic obstructive pulmonary disorder, uncontrolled diabetes, electrolyte disturbances), as they may confound treatment effectiveness and tolerability.
We will include participants with comorbid mental disorders, but the effect of including these participants will be examined in sensitivity analyses.
Types of interventions
Any trial comparing azapirones (buspirone, gepirone, tandospirone, ipsapirone or lesopitron) as monotherapy with placebo in the treatment of panic disorder, with or without agoraphobia. Only acute treatment studies treating participants for less than six months will be included. Relapse prevention studies will be excluded.
No restriction on dose, frequency, intensity, route of administration or duration will be applied.
Studies administering psychosocial therapies targeted at panic disorder concurrently will also be excluded.
Types of outcome measures
1. Rate of 'response' (i.e. substantial improvement from baseline as defined by the original investigators). Examples include “very much or much improved” according to the Clinical Global Impression Change Scale, more than 40% reduction on the Panic Disorder Severity Scale and more than 50% reduction on the Fear Questionnaire Agoraphobia Subscale.
2. Total number of dropouts for any reason as a proxy measure of treatment acceptability.
3. 'Remission' (i.e. satisfactory end-state as defined by global judgement of the original investigators). Examples include “panic free” and “no or minimal symptom” according to the Clinical Global Impression Severity Scale.
4. Panic symptom scales and global judgement on a continuous scale. Examples include Panic Disorder Severity Scale total score (0 to 28), Clinical Global Impression Severity Scale (1 to 7), Clinical Global Impression Change Scale (1 to 7), etc. When multiple measures were used, steps will be followed in the order as above, with preference given to panic symptoms scales. The actual measure entered into meta-analysis will be indicated at the top of the listings in the 'Table of included studies'.
5. Frequency of panic attacks, as recorded, for example, by a panic diary.
6. Agoraphobia, as measured, for example, by Fear Questionnaire, Mobility Inventory, behavioural avoidance test, etc.
7. General anxiety, as measured, for example, by Hamilton Rating Scale for Anxiety, Beck Anxiety Inventory, State-Trait Anxiety Index, Sheehan Patient-Rated Anxiety Scale, Anxiety Subscale of Symptom Checklist (SCL)-90-R, etc.
8. Depression, as measured, for example, by Hamilton Rating Scale for Depression, Beck Depression Inventory, Depression Subscale of SCL-90-R, etc.
9. Social functioning, as measured, for example, by Sheehan Disability Scale, Global Assessment Scale, Social Adjustment Scale-Self Report, etc.
10.Quality of life, as measured, for example, by Short Form (SF)-36, SF-12, etc.
11. Patient satisfaction with treatment.
12. Economic costs.
13. Number of dropouts due to adverse effects.
14. Number of participants experiencing at least one adverse effect.
Timing of outcome assessment
All outcomes are short-term, which we define as acute phase treatment that normally would last two to six months.
When studies report response rates at different time points within two to six months, the time point closest to 12 weeks will be given preference.
Search methods for identification of studies
The Cochrane Depression, Anxiety and Neurosis Review Group's Specialised Register (CCDANCTR)
The Cochrane Depression, Anxiety and Neurosis Group (CCDAN) maintains two clinical trials registers at its editorial base in Bristol, UK: a references register and a studies based register. The CCDANCTR-References Register contains over 31,500 reports of RCTs in depression, anxiety and neurosis. Approximately 65% of these references have been tagged to individual, coded trials. The coded trials are held in the CCDANCTR-Studies Register and records are linked between the two registers through the use of unique Study ID tags. Coding of trials is based on the EU-Psi coding manual, using a controlled vocabulary, please contact the CCDAN Trials Search Coordinator for further details. Reports of trials for inclusion in the Group's registers are collated from routine (weekly), generic searches of MEDLINE (1950-), EMBASE (1974-) and PsycINFO (1967-); quarterly searches of the Cochrane Central Register of Controlled Trials (CENTRAL) and review specific searches of additional databases. Reports of trials are also sourced from international trials registers c/o the World Health Organization's trials portal (the International Clinical Trials Registry Platform (ICTRP)), pharmaceutical companies, the handsearching of key journals, conference proceedings and other (non-Cochrane) systematic reviews and meta-analyses.
Details of CCDAN's generic search strategies can be found on the Group's website.
The CCDAN registers will be searched using the following terms.
Search 1: Azapirones and Panic
Diagnosis = (panic) and Intervention = ((azapirone or alnespirone or binospirone or buspirone or enilospirone or eptapirone or gepirone or ipsapirone or revospirone or tandospirone or zalospirone or *piron*) and placebo*)
The References Register will be searched using the following free-text terms to identify additional untagged references:
(panic or agoraphobi*) and (azapirone or alnespirone or binospirone or buspirone or enilospirone or eptapirone or gepirone or ipsapirone or revospirone or tandospirone or zalospirone or *piron*)
Abstracts will be screened for azapirone trials and full-text articles will be retrieved, where necessary, to check for placebo controls.
Search 2: Azapirones and Anxiety Disorders not otherwise specified (ADNOS)
A further search of the CCDANCTR will be carried out to identify reports of azapirone trials for ‘Anxiety Disorders Not Otherwise Specified’ (ADNOS).
Condition = (anxiety or anxious) and Intervention = ((azapirone or alnespirone or binospirone or buspirone or enilospirone or eptapirone or gepirone or ipsapirone or revospirone or tandospirone or zalospirone or *piron*) and placebo*)
The References Register will be searched using the following free-text terms to identify additional untagged references: ((anxiety or anxious or ADNOS) and (azapirone or alnespirone or binospirone or buspirone or enilospirone or eptapirone or gepirone or ipsapirone or revospirone or tandospirone or zalospirone or *piron*)) and not (agoraphobi* or panic or (social NEAR (anxi* or phobi*)) or generalised or generalized or obsessive or compulsive or OCD or PTSD or post-trauma* or “post trauma*” or posttrauma*)
Abstracts will be screened and full-text articles will be retrieved, where necessary, to check for appropriate placebo controlled trials.
No restrictions will be placed on date, language or publication status.
National and International Trials Registers
Complementary searches will be conducted on the WHO International Clinical Trials Registry Platform (ICTRP) and ClinicalTrials.gov.
Searching other resources
Reference lists of all included studies, non-Cochrane systematic reviews and major textbooks of affective disorders (written in English) will be checked by review authors, for published reports and citations of unpublished research. A citation search will also be conduct via the Web of Science (included studies only) to identify additional works. Experts in the field will be contacted.
Data collection and analysis
Selection of studies
The selection of trials for inclusion in this systematic review will be done independently by two of the authors: GG (clinical expertise) and MK (methodological expertise).
GG and MK will inspect the search hits by reading the titles and the abstracts to see whether they meet the criteria. Possible doubts will be resolved by consultation with the co–review authors. Each potentially relevant study located in the search will be obtained as a full article and independently assessed for inclusion by two review authors, and, in the case of discordance, resolution will be sought by discussion between the review authors. The discordance in the selection of studies will be calculated using Cohen’s kappa (k) (Cohen 1960), a more robust measure than a simple per cent agreement calculation because it takes into account the agreement between review authors that occurs by chance. When it will not be possible to evaluate the study because of language problems or missing information, the study will be classified as 'study awaiting assessment' until a translation or further information can be obtained. Reasons for the exclusions of trials will be reported in the 'Characteristics of excluded studies' table.
All decisions made during the selection process, along with numbers of studies and references, will be recorded and presented in a PRISMA flow diagram (Moher 2009) at the end of the review.
Data extraction and management
Two review authors will use a data extraction form to independently extract the data from included studies concerning participant characteristics (age, sex, severity of panic disorder, study setting), intervention details (dosage, duration of study, sponsorship), study characteristics (blinding, allocation, etc.) and outcome measures of interest. The extraction sheet will be piloted by a sample of 10% of the included studies. Again, any disagreement will be resolved by consensus or by the third member of the review team. If necessary, authors of studies will be contacted to obtain clarification.
Azapirones as a whole versus placebo.
Assessment of risk of bias in included studies
Two review authors will independently assess risk of bias using the tool described in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011). This tool encourages consideration of how the sequence was generated, how allocation was concealed, the integrity of blinding at outcome assessment, the completeness of outcome data, selective reporting and other biases. Sponsorship bias will also be considered.
The risk of bias, in each domain and overall, is assessed and categorised as:
low risk of bias: plausible bias unlikely to seriously alter the results;
high risk of bias: plausible bias that seriously weakens confidence in the results; or
unclear risk of bias: plausible bias that raises some doubt about the results.
If the assessors disagree, the final rating will be made by consensus or with involvement of another member of the review group. When inadequate details of randomisation and other characteristics of trials are provided, authors of the studies will be contacted to obtain further information. Non-concurrence in quality assessment will also be reported.
Measures of treatment effect
The main outcome result will be reduction of severity of panic and agoraphobia symptoms. Improvement will usually be presented as a change on a panic disorder scale(s) (mean and standard deviation), as dichotomous outcomes (responder or non-responder, remitted or not-remitted) or as both.
Binary or dichotomous data
For binary outcomes, we will calculate a standard estimation of the random-effects model risk ratio (RR) and its 95% confidence interval (CI). It has been shown that a random-effects model has good generalisability (Furukawa 2002) and that RR is more intuitive (Boissel 1999) than odds ratio. Furthermore, odds ratios tend to be interpreted as RRs by clinicians (Deeks 2000). This may lead to an overestimation of the impression of the effect (Higgins 2011). For all primary outcomes, we will calculate the number needed to treat for an additional beneficial outcome or the number needed to treat for an additional harmful outcome statistic (NNTB or NNTH) and its 95% CI using Visual Rx (http://www.nntonline.net/), while taking account of the event rate in the control group.
1. Summary statistics
It is likely that different studies have used varied panic rating scales; therefore we will use standardised mean differences (SMDs). If all included studies have used the same instrument, we will use mean differences (MDs).
2. Endpoint versus change data
Trials usually report results using endpoint means and standard deviations of scales or using change in mean values from baseline of assessment rating scales. We prefer to use scale endpoint data, which typically cannot have negative values and are easier to interpret from a clinical point of view. If endpoint data are unavailable, we will use the change data in separate analyses. In case we use MDs, we will pool results based on change data and endpoint data in the same analysis.
Unit of analysis issues
Cross-over trials are trials in which all participants receive both the control and the intervention treatment but in a different order. The major problem is a carryover effect from the first phase to the second phase of the study, especially if the condition of interest is unstable (Elbourne 2002). As this is the case with panic disorder, randomised cross-over studies will be eligible, but only data up to the point of first cross-over will be used.
Studies with multiple treatment groups
When a study involves more than two treatment arms, especially two appropriate dose groups of the same drug, the different dose arms will be pooled and considered to be one. If the arms involve one placebo arm and two or more arms of antidepressants of different classes, we will compare each arm with placebo separately. In this case, a unit of analysis error can occur because of the unaddressed differences between the estimated intervention effects from multiple comparisons (Higgins 2011), resulting in double counting. To avoid this, we will include each pair-wise comparison separately, according to the recommendations of the Cochrane Handbook for Systematic Reviews of Interventions, Section 16.5.4 (Higgins 2011). If the variable is dichotomous, we will divide the shared interventions group evenly among the comparisons. If the variable is continuous, only the total number of participants will be divided up, and means and standard deviations will be left unchanged.
In cluster-randomised trials, groups of individuals rather than separate individuals are randomly assigned to different interventions. In case we identify cluster placebo-controlled randomised trials, we plan to use the generic inverse variance technique if such trials have been appropriately analysed, while taking into account intraclass correlation coefficients to adjust for cluster effects. When trialists have not adjusted for the effects of clustering, we will attempt to do this by obtaining an intracluster correlation coefficient and then following the guidance given in Chapter 16.3.4 of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011).
Dealing with missing data
We will try to contact the study authors to obtain all relevant missing data.
1. Dichotomous outcomes
Response, or remission on treatment, will be calculated using an intention-to-treat (ITT) analysis. We will follow the principle 'once randomised always analysed'. When participants left the study before the intended endpoint, it will be assumed that they would have experienced the negative outcome. The validity of the above assumption will be tested by sensitivity analysis, with application of worst and best case scenarios. When dichotomous outcomes are not reported but the baseline mean and standard deviation on a panic disorder scale are reported, we will calculate the number of responding or remitted participants according to a validated imputation method (Furukawa 2005). The validity of the above approach will be analysed by sensitivity analysis. If necessary, authors of studies will be contacted to obtain data and/or clarification.
2. Continuous outcomes
Concerning continuous data, the Cochrane Handbook for Systematic Reviews of Interventions recommends avoiding imputations of continuous data and suggests that data should be used as presented by the original authors. When ITT data are available, they will be preferred to 'per-protocol analysis'. If necessary, authors of studies will be contacted to obtain data and/or clarification.
3. Skewed or qualitative data
Skewed or qualitative data will be presented descriptively.
Several strategies will be considered for skewed data. If papers report a mean and a standard deviation, and an absolute minimum possible value is also available for the outcome, we will divide the mean by the standard deviation. If the value obtained is less than two, we will conclude that some skewness is indicated. If the value obtained is less than one (i.e. the standard deviation is larger than the mean), skewness is almost certain. If papers have not reported the skewness and simply report means, standard deviations and sample sizes, these numbers will be used. Because these data may not have been properly analysed and can be misleading, analyses will be conducted with and without these studies. If the data have been log-transformed for analysis, and geometric means are reported, skewness will be reduced. This is the recommended method of analysis of skewed data (Higgins 2011). If papers use non-parametric tests and describe averages using medians, they cannot be formally pooled in the analysis. We will follow the recommendation made by Cochrane that results of these studies be reported in a table in our review, along with all other papers. This means that the data will not be lost from the review, and the results can be considered when conclusions are drawn, even if they cannot be formally pooled in the analyses.
4. Missing statistics
When only P or standard error (SE) values are reported, we will calculate standard deviations (SDs) (Altman 1996). In the absence of supplementary data after requests have been made to the authors, the SDs will be calculated according to a validated imputation method (Furukawa 2006). We will examine the validity of these imputations in the sensitivity analyses.
Assessment of heterogeneity
In accordance with the recommendations of the Cochrane Handbook for Systematic Reviews of Interventions, heterogeneity will be quantified by the I2 statistic. The Cochrane Handbook for Systematic Reviews of Interventions recommends overlapping intervals for I2 interpretation (Section 9.5.2, Higgins 2011) as follows:
0% to 40%: might not be important;
30% to 60%: may represent moderate heterogeneity;
50% to 90%: may represent substantial heterogeneity; and
75% to 100%: may represent considerable heterogeneity.
We will also use the Chi2 test and its P value to determine the direction and magnitude of the treatment effects. In a meta-analysis of few trials, Chi2 will be underpowered to detect heterogeneity, if it exists. P = 0.10 will be used as a threshold of statistical significance.
Assessment of reporting biases
Reporting biases arise when dissemination of research findings is influenced by the nature and direction of the results. These are described in Section 10 of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011). A funnel plot is usually used to investigate publication bias. However, it has a limited role when only a few studies of similar size are identified. Also, asymmetry of a funnel plot does not always reflect publication bias. Visual inspection of funnel plots will be used to assess publication bias, as will a statistical test for funnel plot asymmetry, as proposed by Eggers or Rücker (Higgins 2011). We will not use funnel plots for outcomes if 10 or fewer studies are identified, or if all studies are of similar size.
We will use a random-effects model to calculate treatment effects. We prefer the random-effects model, as it takes into account differences between studies, even when no evidence of statistical heterogeneity is found. It gives a more conservative estimate than the fixed-effect model. We note that the random-effects model gives added weight to the findings of small studies, which can either increase or decrease the effect size. We will apply a fixed-effect model to primary outcomes only to see whether this markedly changes the effect size.
Subgroup analysis and investigation of heterogeneity
Subgroup analyses are often exploratory in nature and should be interpreted cautiously, first, because they often involve multiple analyses and lead to false-positive results; and second, because these analyses lack power and are more likely to result in false-positive results. While keeping in mind the above reservations, we would perform the following subgroup analyses.
For participants with agoraphobia and for participants without agoraphobia, we would perform subgroup analyses because the same treatment may have differential effectiveness with regard to panic and agoraphobia.
If groups within any of the subgroups are found to be significantly different from one another, we will run meta-regression for exploratory analyses of additive or multiplicative influences of the variables in question.
We will compare acute phase treatment studies that last less than four months with acute phase treatment studies that last four months or longer.
The following sensitivity analyses will be planned a priori. We will examine whether the results change and will check for the robustness of observed findings by:
excluding trials with high risk of bias (i.e. trials with inadequate allocation concealment and blinding, with incomplete data reporting and/or with high probability of selective reporting);
excluding trials with dropout rates greater than 20%;
excluding studies funded by the pharmaceutical company marketing each azapirone. This sensitivity analysis is particularly important (a) because repeated findings indicate that funding strongly affects outcomes of research studies (Als-Nielsen 2003; Lexchin 2003; Bhandari 2004), and (b) because industry sponsorship and authorship of clinical trial reports have increased over the past 20 years (Buchkowsky 2004); and
excluding studies whose participants clearly have significant psychiatric comorbidities, including primary or secondary depressive disorders.
Our routine application of random-effects and fixed-effect models, as well as our secondary outcomes of remission rates and continuous severity measures, might be considered additional forms of sensitivity analyses.
Summary of findings table
We will summarise the findings using a 'Summary of findings' table, according to the GRADE approach (Higgins 2011).