Sequential change detection and monitoring of temporal trends in random‐effects meta‐analysis

Temporal changes in magnitude of effect sizes reported in many areas of research are a threat to the credibility of the results and conclusions of meta‐analysis. Numerous sequential methods for meta‐analysis have been proposed to detect changes and monitor trends in effect sizes so that meta‐analysis can be updated when necessary and interpreted based on the time it was conducted. The difficulties of sequential meta‐analysis under the random‐effects model are caused by dependencies in increments introduced by the estimation of the heterogeneity parameter τ 2. In this paper, we propose the use of a retrospective cumulative sum (CUSUM)‐type test with bootstrap critical values. This method allows retrospective analysis of the past trajectory of cumulative effects in random‐effects meta‐analysis and its visualization on a chart similar to CUSUM chart. Simulation results show that the new method demonstrates good control of Type I error regardless of the number or size of the studies and the amount of heterogeneity. Application of the new method is illustrated on two examples of medical meta‐analyses. © 2016 The Authors. Research Synthesis Methods published by John Wiley & Sons Ltd.

as given by (16). The efficient score statistic V k (θ 0 , Substituting these values in equation (14), the Gombay test statistic is given by A.2 When is the analytical approximation of the sequence T k by Wiener process valid?, end Section 3.2 Consider a sequence of statistics T k given by equation (17), for k ≥ 1. For simplicity, assume known weights w i = σ 2 i /n i + τ 2 and let the study sample sizes n i all be of the same order, n i = O(n). Then the weights w i = τ −2 (1 + O(1/n)), and the sums of weights W k = k 1 w i = τ −2 k(1 + O(1/n)). For a sequence T k to be a Wiener process, we require E(T k ) = 0, var(T k ) = k (both true, at least approximately), and also independent increments when comparing T k to T k+u , u ≥ 1. This is equivalent to requiring cov(T k , T k+u ) = k. Let us consider when this condition is approximately true. and Therefore cov(T k , T k+u ) = (k+u)W k kW k+u k and we require (k+u)W k kW k+u k 2 → k 2 . Since W k /k = τ −2 (1 + O(1/n)), this condition is equivalent to k 2 /n → 0 for any k ≤ K.
should be large in comparison to squared truncation point K 2 . For instance, if K = 50, the sample sizes should be large in comparison to 2500, i.e. too large to be practical.
A.3 Section 4: Illustration of a possible bias inτ 2 DL due to a change in location parameter θ Consider a shift in location θ i = θ 0 for i = 1, · · · , r and θ i = θ 0 + δ for i ≥ r + 1 for a case of τ 2 = 0.
Using the inverse variance weights w i = n i /σ i for the ith study, the weighted meanθ w = θ 0 + p∆ The DerSimonian estimatorτ 2 DL is obtained aŝ Now let us investigate the order of the terms in the above equation for the case of fixed r and fixed sample sizes n i and increasing K. In the numerator, p → 1 and 1 − p → 0 in the limit when K → ∞; also W = Kw is of order K, so (K − 1)/W → c for a constant c. w 2 i /W 2 is of order 1/K, so the denominator converges to 1 in the limit K → ∞. Therefore,τ 2 DL will take on a negative value, and will be truncated at zero. Thus, for a large enough K, an estimatedτ 2 K is unbiased.
B Tables  Table 1: Data and results of the meta-analysis of 23 studies on magnesium for myocardial infarction by Li et al. (2009). The subscripts T and C refer to the treatment and control arms of the studies. The columns headed n T and n C are the sample sizes, and x T and x C are the numbers of events in each study. The columns headed ϕ, v and ϕ cum are the log-odds ratios, their variances and the cumulative effects, respectively. The next two columns (z            Table 7: Data and results of the meta-analysis of 53 studies on nicotine replacement therapy for smoking cessation by Stead et al. (2008) data cont'd S/N GH   Paule and Mandel (1982) and REML estimators of τ 2 (GDL, GH, GMP and GREML, respectively). K is the number of studies; n is the average sample size; ∆ is the effect parameter, τ 2 is the between-study variance. The black straight line is at zero; the yellow, green, purple and red lines corespond to GDL, GH, GMP and GREML, respectively.  Paule and Mandel (1982) and REML estimators of τ 2 (GDL, GH, GMP and GREML) against θ. K is the number of studies; n is the average sample size; ρ is the power while ∆ is the effect parameter, τ 2 is the between-study variance. The yellow, green, purple and the red lines represent GDL, GH, GMP and GREML, respectively.  Paule and Mandel (1982) and REML estimators of τ 2 (GDL, GH, GMP and GREML) when θ = 0.05. K is the number of studies; n is the average sample size; ρ is the deviations in powers from the average power of the four test while ∆ is the effect parameter, τ 2 is the between-study variance. The black straight line is the nominal value of 5% for the test while the yellow, green, purple and the red lines represent GDL, GH, GMP and GREML, respectively.  Paule and Mandel (1982) and REML estimators of τ 2 (GDL, GH, GMP and GREML) when τ 2 = 0.05. K is the number of studies; n is the average sample size; ρ is the deviations in powers from the average power of the four test while ∆ is the effect parameter, τ 2 is the between-study variance. The black straight line is the nominal value of 5% for the test while the yellow, green, purple and the red lines represent GDL, GH, GMP and GREML, respectively.   Paule and Mandel (1982) and the REML estimators of τ 2 (GDL, GH, GMP and GREML).
The target value is set at 0 and the red dashed lines in GDL, GH, GMP and GREML tests plots are the upper boundary values for one-sided tests.