2.1. Testing the hypothesis of equal proportions
Another choice for wj is which is proportional to the harmonic mean of n1j and n2j. This weighting strategy is generally referred to as the Cochran–Mantel–Haenszel (CMH) weighting in the literature because the test statistics, proposed by Cochran and Mantel–Haenszel, respectively, in (3) use such weights . A continuity correction could be added to both test statistics. In (3), is the observed proportion in the jth trial, that is, , and is the total number of subjects in the jth study
A third approach to combine results in a stratified analysis of a binary endpoint is the minimum risk approach proposed by Mehrotra and Railkar . This approach, designed to increase the power in hypothesis testing, lets data decide the weights used to combine results from individual studies. According to Mehrotra and Railkar, the minimum risk approach is not recommended if the primary objective of the analysis is on estimation in a meta analysis where the strata are defined by studies. Because we are interested in estimation in this paper, we will not pursue this approach further for the remaining of the paper.
2.2. Reporting cumulative proportions
Assuming a fixed-effect model and dj = d for all j, a meta analyst can use in (1) with IV weighting as the basis for constructing a confidence interval for d. Similarly, the CMH approach will lead to a point estimate and a confidence interval for d. Because the IV weighting produces the minimum variance estimate in this case, the CI confidence interval produced by the IV approach is narrower than that produced under the CMH approach.
We will now focus on methods to report the cumulative proportion of subjects reporting the target AE in multiple studies. Because pij's are likely to differ among studies, an important question concerns the desirable properties of the resulting cumulative proportion. In our opinion, at a minimum, an acceptable strategy should lead to comparable cumulative proportions between treatment groups if the proportions are comparable within each study.
The above motivated us to first look at the cumulative proportions in (4) where proportions in studies are combined using either the IV or the CMH weights. To differentiate the proportions in (4) from that obtained from the pooled data (i.e. , we will call cumulative proportions in (4) ‘adjusted’ cumulative proportions
Applying the above concept to Table II, we obtained the results in Table III. Even though both studies have 400 subjects, they do not have the same weight in forming the adjusted cumulative proportions under either approach. Study 2, with equal number of subjects on the two treatments, enjoys a heavier weight than Study 1. This reflects the fact that given a fixed total sample size, equal allocation generally produces greater information value than most other randomization ratios. Compared with the CMH weighting, the IV weighting gives even less weight to study 1. This is because sampling variability associated with a 60% proportion is higher than that associated with a 30% proportion. By design, the adjusted cumulative proportions are the same for the two treatment groups under either weighting scheme. Their values are 42% (IV) and 43% (CMH).
Table III. (a) Weights applied to Studies 1–2 in Table II to construct adjusted cumulative proportions under the inverse variance (IV) and the CMH methods and (b) Weights applied to Studies 1–6 in Table I to construct adjusted cumulative proportions under the IV, CMH, and SS-based methods.
| ||Study||IV||CMH|| |
| ||2||0.60||0.57|| |
|Adjusted cumulative proportion||New treatment||42%||43%|| |
| ||Control||42%||43%|| |
| ||Study||IV||CMH||Study size|
|Adjusted cumulative proportion||New treatment||4.1%||11.4%||11.9%|
For studies in Table I, the weights given to the six studies as well as the adjusted cumulative proportions under the IV and CMH approaches are given in Table III. Table III exposes the greatest drawback of the IV method in combining proportions. Recall that the IV weights are inversely proportional to the variance of the differences in the observed proportions. For a fixed sample size, studies that have smaller proportions will weigh more heavily than studies with higher proportions as long as they are all less than 50%. The opposite is true if the proportions are greater than 50%. As a result, Study 3 and (to a lesser extent) Study 4 in Table I have high weights under the IV approach, resulting in much lower adjusted cumulative proportions. A second disadvantage of the IV method is the inclusion of observed proportions associated with the other treatment in constructing the adjusted cumulative proportions. For these reasons, we do not recommend using IV weights to obtain adjusted cumulative proportions.
Another sensible approach is to weigh the observed proportion in a study by the percentage of subjects in that study among the pooled population. In other words, wj will be set to as in (5).
For convenience, we will call this the SS-based method (SS for study size). The adjusted cumulative proportion in (5) has the flavor of the cumulative proportions if patients in the pooled population all received treatment i. In other words, the cumulative proportion for each treatment group is ‘normalized’ to the composition of the pooled population. In doing so, we avoid the potential impact of Simpson's Paradox.
In Table III, the SS-based method produces higher adjusted cumulative rates for both groups than those under the CMH method. In general, these two approaches produce very similar adjusted cumulative proportions; but one approach does not always produce values higher than the other. A simple example is illustrated in Table IV where data from three studies were used to calculate the adjusted cumulative proportion for the new treatment under three hypothetical scenarios. Results in the table show that depending on the randomization ratio and the observed proportions, the CMH approach produces adjusted cumulative proportions that could be higher or lower than their counterparts under the SS-based method.
Table IV. Adjusted cumulative proportions for the new treatment based on data from three studies with 1:1, 2:1 and 3:1 randomization ratios.
|Sample size (New treatment, control)||Observed proportion for new treatment|
|(100, 100)|| ||5%||10%||15%|
|(200, 100)|| ||10%||15%||5%|
|(300, 100)|| ||15%||5%||10%|
|CMH approach||Adjusted cumulative proportion (Standard deviation)||0.107(0.012)||0.098(0.013)||0.096(0.013)|
|SS-based approach||Adjusted cumulative proportion (Standard deviation)||0.111(0.013)||0.094(0.012)||0.094(0.012)|