Re‐randomization tests as sensitivity analyses to confirm immunological noninferiority of an investigational vaccine: Case study

Here we present as case study how re‐randomization tests were performed in two randomized, controlled clinical trials as sensitivity analyses, as recommended by the United States Food and Drug Administration in the context of adaptive randomization. This was done to confirm primary conclusions on immunological noninferiority of an investigational new fully liquid presentation of a quadrivalent cross‐reacting material conjugate meningococcal vaccine (MenACWY‐CRM), over its licensed lyophilized/liquid presentation. In two phase 2b studies (Study #1: NCT03652610; Study #2: NCT03433482), noninferiority of the fully liquid presentation of MenACWY‐CRM to the licensed presentation was assessed and demonstrated for immune responses against meningococcal serogroup A (MenA), the only vaccine component modified from lyophilized to liquid in the new presentation. The original vaccine assignment algorithm, with a minimization procedure accounting for center or center within age strata, was used to re‐randomize participants belonging to the fully liquid and licensed vaccine groups while keeping antibody responses, covariates and entry order as observed. Test statistics under re‐randomization were generated according to the ANCOVA model used in the primary analysis. To confirm immunological noninferiority following re‐randomization, the corresponding p‐values had to be <0.025. For both studies and all primary objective evaluations, the re‐randomization p‐values were well below 0.025 (0.0004 for Study #1; 0.0001 for the two co‐primary endpoints in Study #2). Re‐randomization tests performed to comply with a regulatory request confirmed the primary conclusions of immunological noninferiority for the MenA of the fully liquid compared to the licensed vaccine presentation.


| INTRODUCTION
MenACWY-CRM (Menveo, GSK) is a quadrivalent meningococcal ACWY-nontoxic diphtheria cross-reacting material (CRM) conjugate vaccine licensed in more than 60 countries for active immunization to prevent invasive meningococcal disease caused by Neisseria meningitidis serogroups A, C, W and Y. 1 MenACWY-CRM is supplied in two vials, one containing the liquid MenCWY conjugate components and the other containing the lyophilized meningococcal serogroup A (MenA) conjugate component, to be reconstituted before vaccine injection. 2,3 To simplify vaccine administration, a fully liquid single-vial presentation of MenACWY-CRM has been developed. Two phase 2b, multicenter, randomized, controlled, observer-blind clinical trials were conducted to demonstrate immunological noninferiority of the fully liquid vaccine presentation, over the licensed one, for the MenA component, the only one modified from lyophilized to liquid. 4,5 In both clinical studies, the original randomization (1:1 to either the investigational fully liquid MenACWY-CRM or the licensed MenACWY-CRM presentation) was performed via a central randomization system using a minimization procedure accounting for center (Study #1) 4 or center within age strata (Study #2). 5 The need for re-randomization tests as sensitivity analyses in the context of adaptive randomization, such as minimization, is supported by statistical studies investigating operating characteristics of conventional asymptotic procedures. Asymptotic tests, although they may be valid, can be influenced by model misspecification to be either conservative or liberal. [6][7][8] On the other hand, simulation studies have shown that re-randomization tests under adaptive randomization control type I error and can be comparable in terms of power to asymptotic tests if the model is correctly designed. 9,10 Recently, the role and use of conventional asymptotic procedures in clinical studies employing covariateadaptive randomization methods were disputed by regulatory authorities, and the European Medicines Agency and the United States Food and Drug Administration have developed regulatory guidelines on adaptive study design and analysis 11,12 to address concerns that argue the lack of adequate control of the type I error rate. 13 In this paper we present the results of re-randomization tests recommended by the United States Food and Drug Administration in the context of adaptive randomization, 12 which were performed for these two vaccine trials as sensitivity analyses to confirm fulfillment of the primary immunogenicity objectives.

| Case study
The two phase 2 studies for which we performed the re-randomization were very similar in design. Both included two clinical visits (the first for baseline blood draw and vaccination and the second to test immune responses approximately 1 month postvaccination) and three phone calls at 2 weeks, 3 months and 6 months, to follow up for safety.
In both studies, the investigational fully liquid MenACWY-CRM vaccine was tested under stress conditions induced by either an artificially accelerated controlled aging process, with vaccine storage at around 22.5 C (±2 C) for approximately 2 months during the usual 2-8 C storage period (Study #1 [ClinicalTrials.gov NCT03652610]; individuals aged 18 to 40 years), 4 or after storage up to the intended shelf-life, approximately 24 or 30 months, at 2-8 C in normal conditions (Study #2 [ClinicalTrials.gov NCT03433482]; individuals aged 10 to 40 years). 5 The aim was to assess if, even under these unfavorable conditions, the MenA component, more labile in its liquid presentation, was still able to elicit noninferior immune responses to those induced by the licensed vaccine presentation, administered in both clinical trials to participants randomized to the control groups.
Study summaries are available at www.gsk-studyregister.com (identifiers 205343 and 207467). The studies were conducted in accordance with the Declaration of Helsinki and Good Clinical Practice and approved by the appropriate ethics committees. Before enrollment in the two studies, all individual participants aged at least 18 years provided written informed consent and parents/legally acceptable representatives provided written informed consent for individual participants younger than 18 years.
Considering that the only component modified in the new fully liquid presentation was MenA, from lyophilized to liquid, the primary immunogenicity objective of the two studies was to demonstrate noninferiority of the investigational fully liquid MenACWY-CRM vaccine to the currently licensed presentation in terms of human serum bactericidal assay (hSBA) geometric mean titers (GMTs) directed against MenA at 1 month after a single dose. Noninferiority was considered met if the lower limit of the two-sided 95% confidence interval (CI) for the ratio of hSBA GMTs against MenA between the fully liquid and the licensed presentation was >0.5. The choice of 0.5 (i.e., 0.301 on the Àlog 10 scale) as a noninferiority margin is subject to regulatory scrutiny and is accepted in the context of specific vaccine immunobridging studies (e.g., World Health Organization Guidelines on clinical evaluation of vaccines). 14 GMTs were adjusted using an analysis of covariance (ANCOVA) model with pre-vaccination titer as covariate and group and country as factors. To adjust for multiplicity in Study #2, the co-primary objectives for the fully liquid presentations aged 24 and 30 months were evaluated sequentially. In both clinical trials, the primary analysis demonstrated immunological noninferiority of the fully liquid MenACWY-CRM vaccine for MenA. 4,5

| Re-randomization tests
Based on the original covariate-adaptive vaccine assignment algorithm (Table 1 and Figure 1), the re-randomization test [15][16][17] was performed as a sensitivity analysis on the primary objective of the two studies based on the immunogenicity per-protocol set. This set comprised participants who received the study vaccination correctly, had no major protocol deviations, and had assay results available for at least one serogroup at the concerned time points.
If X = (X 1 , …, X N ) are the logarithmically-transformed hSBA titers in the investigational fully liquid vaccine group, and Y = (Y 1 , …, Y N ) are the logarithmically-transformed hSBA titers in the licensed vaccine group for serogroup A at 1 month after vaccination, a noninferiority re-randomization test can be obtained from a superiority re-randomization test on Ẋ = (X 1 + δ, …, X N + δ) against Ẏ = (Y 1 , …, Y N ) with δ = Àlog 10 (0.5) = 0.301. Of note, using the same N in both groups can be assumed without loss of generality, and the procedure would hold also with different values for N.
The original vaccine assignment algorithm (Table 1 and Figure 1) was used to re-randomize participants belonging to the investigational fully liquid vaccine group and currently licensed vaccine group (see step 2A in Figure 2), while T A B L E 1 Minimization algorithm of GSK's internet randomization system (i.e., SBIR) 20

in Study #1 and Study #2
Study #1 4 Study #2  keeping hSBA titers, covariates and entry order as observed. Test statistics for the superiority of Ẋ against Ẏ under rerandomization were generated according to the ANCOVA model used in the primary analysis. To demonstrate immunological noninferiority by re-randomization test, the proportion of test statistics under re-randomization (T) greater than or equal to the test statistic for the superiority of Ẋ against Ẏ under original assignment (T*) ("re-randomization p-value") had to be <0.025. The complete procedure is described in Figure 2.

| RESULTS
In the original Study #1 analyses, the ratio of adjusted hSBA GMTs against MenA at 1 month postvaccination (investigational fully liquid over the currently licensed vaccine group) was 0.88. The lower limit of the 95% CI was 0.64, greater than the noninferiority margin of 0.5, thus meeting the primary immunogenicity objective of the study. 4 By applying the sensitivity analysis to Study #1 primary immunogenicity data, the re-randomization test p-value was 0.0004 (Figure 3), thus confirming study conclusions. According to the planned Study #2 analyses, the ratio of adjusted hSBA GMTs against MenA at 1 month postvaccination (investigational fully liquid vaccine group over the currently licensed vaccine group) was 1.21 (95% CI: 0.94-1.57) for the investigational vaccine aged 24 months and 1.11 (95% CI: 0.87-1.42) for the fully liquid vaccine aged 30 months. These results allowed to conclude on the fulfillment of the two co-primary study objectives. According to the sensitivity analyses for the co-primary objectives of Study #2, the re-randomization test p-value was 0.0001 (Figure 3), lower than the predefined value of 0.025. More precisely, no re-randomizations resulted in T ≥ T*, indicating an actual p-value <0.0001. F I G U R E 2 Diagram of re-randomization procedure. M is the number of repetitions such that T ≥ T*. R is the number of rerandomizations.
As such, the re-randomization tests confirmed the original conclusions of the two studies, namely the immunological noninferiority for MenA of the fully liquid MenACWY-CRM vaccine compared to the licensed lyophilized/liquid presentation, according to the prespecified protocol criteria.

| DISCUSSION
Re-randomization tests in clinical trials guarantee statistical validity, 18 even in the context of adaptive randomizations. 12 They can be employed as ancillary analyses to evaluate the robustness of the inferences that are made from the main estimator 19 when the assumptions underlying other statistical testing procedures might be considered questionable.
F I G U R E 3 Adjusted hSBA GMTs against N. meningitidis serogroup A and between group ratios at 1 month postvaccination, with pvalues for the re-randomization test (per-protocol immunogenicity set). CI, confidence interval; GMT, geometric mean titer; hSBA, human serum bactericidal assay; MenA, N. meningitidis serogroup A; N, number of participants with pre-and postvaccination results available. *Primary results published in Vandermeulen et al. 4 **Primary results published in Díez-Domingo et al. 5 ; the two orange bars correspond to the fully liquid presentations aged 24 and 30 months, respectively. ***investigational fully liquid MenACWY-CRM group over the licensed MenACWY-CRM group. The dashed line represents the success threshold for the 95% confidence interval lower limit of the between groups GMTs ratio in the primary analysis; to demonstrate noninferiority by re-randomization test, the corresponding p-values had to be <0.025.
Here we presented as case study the experience acquired during the clinical development of an investigational vaccine and the need to fulfill a regulatory request. In both clinical studies evaluating the new fully liquid presentation of the MenACWY-CRM conjugate vaccine, the primary analysis demonstrated noninferiority of immune responses directed against the only component modified (MenA) compared to currently licensed presentation. 4,5 To comply with a regulatory request, the evidence generated using re-randomization tests confirmed the conclusions of the two studies and the statistical evidence generated in support of the improved vaccine presentation. For both studies considered for sensitivity analyses, re-randomization tests confirmed the primary conclusions, with p-values well below the predefined threshold of 0.025.
A limiting factor for the re-randomization method applied is related to the utilization of computational resources. Due to the high number of re-randomizations performed to estimate the distribution of test statistics, the computational complexity of a re-randomization test might be substantial and hence take time to complete. Among factors that may influence the required computational resources are the number of study participants, the complexity of the original randomization algorithm (e.g., number of minimization covariates), and the complexity of the model used to generate the test statistics. These elements should be factored in when considering inclusion of re-randomization tests in a statistical analysis plan. It would be advisable, according to our experience, to make simulations in order to assess upfront the computational complexity, which would allow to estimate the time and computational resources required for rerandomization tests and to fine-tune the test parameters.
In conclusion, noninferiority of the new fully liquid to the licensed MenACWY-CRM presentation was confirmed by re-randomization tests. In the specific case of the two considered clinical trials, the robustness of the study results is further confirmed by the fact that the investigational fully liquid vaccine was used either after artificial aging (Study #1) or at the end of its intended shelf-life (after 24 or 30 months under usual storage conditions; Study #2). The evaluation of an aged investigational vaccine was done to help clinical validation of the release specifications and the shelf-life of the new fully liquid presentation of the vaccine.