## 1 Introduction

Performing a meta-analysis involves making a number of important decisions. The Cochrane Handbook [1], Section 9.7, describes several of these and advocates sensitivity analysis as a suitable approach for evaluating their implications. In particular, under the heading of ‘Analysis methods’, the Cochrane Handbook asks ‘For dichotomous outcomes, should odds ratios, risk ratios or risk differences be used?’ Deeks and Altman [2] and Sinclair and Bracken [3] provide good accounts of the issues involved in choosing a suitable measure and, in particular, distinguish between relative (e.g. the odds ratio) and absolute (e.g. the risk difference) measures of treatment effect.

Other outcome measures for comparative binary data are also available, including the recently proposed arcsine difference[4]. This measure was proposed for the analysis of trials with zero or small counts, and we regard this as an important measure to incorporate into our methods. For example, odds ratios or risk differences may have been identified, *a priori*, as the outcome measure to be used in analysis. If, however, some or many studies are subsequently found with zero counts, then the implications of using the arcsine difference may be of particular interest. The choice between a relative or an absolute measure may be especially crucial when the event is rare, so we suggest that a sensitivity analysis may be particularly important in such instances.

The implications of the choice of outcome measure was also investigated by Deeks [5], who examines which type of measures for comparative binary data appears to be the most consistent. We agree with Deeks that the choice of outcome measure should not be determined by the ‘best fitting’ model but rather that this should be guided by empirical evidence and clinical debate. Deeks finds that the risk difference is a less consistent measure than the relative measures he investigated, and interest in quantifying the impact of heterogeneity has subsequently increased. We therefore consider the now very popular *I*^{2} statistic[6] resulting from the choice of outcome measure to be an important quantity to explore.

We assume that the data are from randomised controlled trials, so that all the usual measures of treatment effect are appropriate and may easily be calculated. We also assume that the meta-analytic data are in the common form where in each study there are two treatment groups and we have counts for the number of participants who experience and do not experience the event of interest. These data can be presented in the form of a series of two by two tables.

Although sensitivity analyses may be criticised on the grounds that they do not provide a single answer, in addition to the Cochrane Handbook's recommendation, several authors have suggested using sensitivity analyses in the context of meta-analysis. For example, Copas and Shi [7] and Bowden *et al*. [8] suggest using them when assessing publication bias. The term ‘sensitivity analysis’ covers a wide range of strategies, but the approach adopted here is to introduce a sensitivity parameter that describes the type of outcome measure used.

The rest of the paper is set out as follows. In Section 2, we introduce our generalised outcome measure, and we show how the standard measures are special cases of this. In Section 3, we describe our proposed procedure for a sensitivity analysis, and in Section 4, we apply this to some examples. We conclude with a discussion in Section 5.