Tutorial in Biostatistics
Traditional multiplicity adjustment methods in clinical trials
Abstract
This tutorial discusses important statistical problems arising in clinical trials with multiple clinical objectives based on different clinical variables, evaluation of several doses or regiments of a new treatment, analysis of multiple patient subgroups, etc. Simultaneous assessment of several objectives in a single trial gives rise to multiplicity. If unaddressed, problems of multiplicity can undermine integrity of statistical inferences. The tutorial reviews key concepts in multiple hypothesis testing and introduces main classes of methods for addressing multiplicity in a clinical trial setting. General guidelines for the development of relevant and efficient multiple testing procedures are presented on the basis of application‐specific clinical and statistical information. Case studies with common multiplicity problems are used to motivate and illustrate the statistical methods presented in the tutorial, and software implementation of the multiplicity adjustment methods is discussed. Copyright © 2013 John Wiley & Sons, Ltd.
Citing Literature
Number of times cited according to CrossRef: 76
- Alex Dmitrienko, Ilya Lipkovich, Aaron Dane, Christoph Muysers, Data-Driven and Confirmatory Subgroup Analysis in Clinical Trials, Design and Analysis of Subgroups with Biopharmaceutical Applications, 10.1007/978-3-030-40105-4_3, (33-91), (2020).
- Hui Wang, Ilana Belitskaya-Lévy, Ying Lu, Biomarker-Targeted Confirmatory Trials, Design and Analysis of Subgroups with Biopharmaceutical Applications, 10.1007/978-3-030-40105-4_2, (19-31), (2020).
- Catherine M. Meyers, Qilu Yu, Trials in Complementary and Integrative Health Interventions, Principles and Practice of Clinical Trials, 10.1007/978-3-319-52677-5, (1-26), (2020).
- Olga V. Marchenko, Lisa M. LaVange, Natallia V. Katenka, Biostatistics in Clinical Trials, Quantitative Methods in Pharmaceutical Research and Development, 10.1007/978-3-030-48555-9, (1-70), (2020).
- S. V. Larsen, K. Köhler‐Forsberg, V. H. Dam, A. S. Poulsen, C. Svarer, P. S. Jensen, G. M. Knudsen, P. M. Fisher, B. Ozenne, V. G. Frokjaer, Oral contraceptives and the serotonin 4 receptor: a molecular brain imaging study in healthy women, Acta Psychiatrica Scandinavica, 10.1111/acps.13211, 142, 4, (294-306), (2020).
- Liv V. Hjordt, Brice Ozenne, Sophia Armand, Vibeke H. Dam, Christian G. Jensen, Kristin Köhler-Forsberg, Gitte M. Knudsen, Dea S. Stenbæk, Psychometric Properties of the Verbal Affective Memory Test-26 and Evaluation of Affective Biases in Major Depressive Disorder, Frontiers in Psychology, 10.3389/fpsyg.2020.00961, 11, (2020).
- Sara Marie Ulv Larsen, Hans-Peter Landolt, Wolfgang Berger, Maiken Nedergaard, Gitte Moos Knudsen, Sebastian Camillo Holst, Haplotype of the astrocytic water channel AQP4 is associated with slow wave energy regulation in human NREM sleep, PLOS Biology, 10.1371/journal.pbio.3000623, 18, 5, (e3000623), (2020).
- Ayodele Odutayo, Dmitry Gryaznov, Bethan Copsey, Paul Monk, Benjamin Speich, Corran Roberts, Karan Vadher, Peter Dutton, Matthias Briel, Sally Hopewell, Douglas G Altman, Lars G Hemkens, Stefan Schandelmaier, Benjamin Kasenda, Kimberly Alba Mc Cord, Alain Nordmann, Alain Amstutz, Ramon Saccilotto, Belinda von Niederhäusern, Christiane Pauli-Magnus, Erik von Elm, Elena Ojeda Ruiz, Yuki Tomonaga, Matthias Schwenkglenks, Jörg Meerpohl, Anette Blümle, Karin Bischoff, Katharina Wollmann, Laura Rehner, Jason W Busse, Dominik Mertz, Jacqueline Wong, Ngai Chow, Patrick Ji Ho Hong, Design, analysis and reporting of multi-arm trials and strategies to address multiple testing, International Journal of Epidemiology, 10.1093/ije/dyaa026, (2020).
- Tianyu Zhan, Alan Hartford, Jian Kang, Walter Offen, Optimizing Graphical Procedures for Multiplicity Control in a Confirmatory Clinical Trial via Deep Learning, Statistics in Biopharmaceutical Research, 10.1080/19466315.2020.1799855, (1-11), (2020).
- Shogo Nomura, Yu Miyauchi, Yoshikazu Ajisawa, Kento Isogaya, Takashi Sozu, Study Designs in Multi-arm Trials for Breast Cancer: A Systematic Literature Review of Major Journals, Therapeutic Innovation & Regulatory Science, 10.1007/s43441-020-00141-3, (2020).
- Heather M. Johnson, Lisa Sullivan-Vedder, KyungMann Kim, Patrick E. McBride, Maureen A. Smith, Jamie N. LaMantia, Jennifer T. Fink, Megan R. Knutson Sinaise, Laura M. Zeller, Diane R. Lauver, Rationale and study design of the MyHEART study: A young adult hypertension self-management randomized controlled trial, Contemporary Clinical Trials, 10.1016/j.cct.2019.01.010, 78, (88-100), (2019).
- Amber Salter, J. Philip Miller, Controlling for Multiplicity, Eligibility, and Exclusions, Principles and Practice of Clinical Trials, 10.1007/978-3-319-52677-5, (1-9), (2019).
- Kirsten Fischer, Othman Al-Sawaf, Jasmin Bahlo, Anna-Maria Fink, Maneesh Tandon, Mark Dixon, Sandra Robrecht, Simon Warburton, Kathryn Humphrey, Olga Samoylova, Anna M. Liberati, Javier Pinilla-Ibarz, Stephen Opat, Liliya Sivcheva, Katell Le Dû, Laura M. Fogliatto, Carsten U. Niemann, Robert Weinkove, Sue Robinson, Thomas J. Kipps, Sebastian Boettcher, Eugen Tausch, Rod Humerickhouse, Barbara Eichhorst, Clemens-Martin Wendtner, Anton W. Langerak, Karl-Anton Kreuzer, Matthias Ritgen, Valentin Goede, Stephan Stilgenbauer, Mehrdad Mobasher, Michael Hallek, Venetoclax and Obinutuzumab in Patients with CLL and Coexisting Conditions, New England Journal of Medicine, 10.1056/NEJMoa1815281, 380, 23, (2225-2236), (2019).
- S. E. Ebert, P. Jensen, B. Ozenne, S. Armand, C. Svarer, D. S. Stenbaek, K. Moeller, A. Dyssegaard, G. Thomsen, J. Steinmetz, B. H. Forchhammer, G. M. Knudsen, L. H. Pinborg, Molecular imaging of neuroinflammation in patients after mild traumatic brain injury: a longitudinal 123I‐CLINDE single photon emission computed tomography study, European Journal of Neurology, 10.1111/ene.13971, 26, 12, (1426-1432), (2019).
- Brice Ozenne, Tae‐Hee Cho, Irene Klærke Mikkelsen, Marc Hermier, Götz Thomalla, Salvador Pedraza, Pascal Roy, Yves Berthezène, Norbert Nighoghossian, Leif Østergaard, Jean‐Claude Baron, Delphine Maucort‐Boulch, Individualized quantification of the benefit from reperfusion therapy using stroke predictive models, European Journal of Neuroscience, 10.1111/ejn.14505, 50, 8, (3251-3260), (2019).
- Victoria Vickerstaff, Rumana Z. Omar, Gareth Ambler, Methods to adjust for multiple comparisons in the analysis and sample size calculation of randomised controlled trials with multiple primary outcomes, BMC Medical Research Methodology, 10.1186/s12874-019-0754-4, 19, 1, (2019).
- Sigurd Evensen, Ingvild Saltvedt, Stian Lydersen, Torgeir Bruun Wyller, Kristin Taraldsen, Olav Sletvold, Delirium motor subtypes and prognosis in hospitalized geriatric patients – A prospective observational study, Journal of Psychosomatic Research, 10.1016/j.jpsychores.2019.04.020, 122, (24-28), (2019).
- Lisa M. LaVange, Statistics at FDA: Reflections on the Past Six Years, Statistics in Biopharmaceutical Research, 10.1080/19466315.2019.1571322, 11, 1, (1-12), (2019).
- Mei-Yin C. Polley, Edward L. Korn, Boris Freidlin, Phase III Precision Medicine Clinical Trial Designs That Integrate Treatment and Biomarker Evaluation, JCO Precision Oncology, 10.1200/PO.18.00416, 3, (1-9), (2019).
- Francisco Javier Ariza-López, José Rodríguez-Avi, Juan Francisco Reinoso-Gordo, Íñigo Antonio Ariza-López, Quality Control of “As Built” BIM Datasets Using the ISO 19157 Framework and a Multiple Hypothesis Testing Method Based on Proportions, ISPRS International Journal of Geo-Information, 10.3390/ijgi8120569, 8, 12, (569), (2019).
- Mark A. Socinski, Robert M. Jotte, Federico Cappuzzo, Francisco Orlandi, Daniil Stroyakovskiy, Naoyuki Nogami, Delvys Rodríguez-Abreu, Denis Moro-Sibilot, Christian A. Thomas, Fabrice Barlesi, Gene Finley, Claudia Kelsch, Anthony Lee, Shelley Coleman, Yu Deng, Yijing Shen, Marcin Kowanetz, Ariel Lopez-Chavez, Alan Sandler, Martin Reck, Atezolizumab for First-Line Treatment of Metastatic Nonsquamous NSCLC, New England Journal of Medicine, 10.1056/NEJMoa1716948, 378, 24, (2288-2301), (2018).
- Brookes Folmli, Bulent Turman, Peter Johnson, Allan Abbott, Dose response of somatosensory cortex repeated anodal transcranial direct current stimulation on vibrotactile detection: a randomized sham-controlled trial, Journal of Neurophysiology, 10.1152/jn.00926.2017, 120, 2, (610-616), (2018).
- Hideaki Uehara, Evaluation of Intrasubject Parallelism in Balanced Ex Vivo Bioassay Using One-sided Efficient Score Tests of Slope Difference Quantile, Japanese Journal of Biometrics, 10.5691/jjb.39.9, 39, 1, (9-22), (2018).
- B. Dobenecker, S. Reese, W. Jahn, M. Schunck, J. Hugenberg, H. Louton, S. Oesser, Specific bioactive collagen peptides (PETAGILE®) as supplement for horses with osteoarthritis: A two‐centred study, Journal of Animal Physiology and Animal Nutrition, 10.1111/jpn.12863, 102, S1, (16-23), (2018).
- Alex Dmitrienko, Ralph B. D’Agostino, Multiplicity Considerations in Clinical Trials, New England Journal of Medicine, 10.1056/NEJMra1709701, 378, 22, (2115-2122), (2018).
- George Patounakis, Micah J. Hill, Complexities and potential pitfalls of clinical study design and data analysis in assisted reproduction, Current Opinion in Obstetrics and Gynecology, 10.1097/GCO.0000000000000454, (1), (2018).
- F.J. Ariza-Lopez, J. Rodriguez-Avi, M.V. Alba-Fernandez, undefined, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium, 10.1109/IGARSS.2018.8517540, (1222-1225), (2018).
- Scott R Evans, Dean Follmann, Ying Liu, Thomas Holland, Sarah B Doernberg, Nadine Rouphael, Toshimitsu Hamasaki, Yunyun Jiang, Judith J Lok, Thuy Tien T Tran, Anthony D Harris, Vance G Fowler, Helen Boucher, Barry N Kreiswirth, Robert A Bonomo, David Van Duin, David L Paterson, Henry Chambers, Sequential, Multiple-Assignment, Randomized Trials for COMparing Personalized Antibiotic StrategieS (SMART-COMPASS), Clinical Infectious Diseases, 10.1093/cid/ciy912, (2018).
- George Kordzakhia, Alex Dmitrienko, Eiji Ishida, Mixture-based gatekeeping procedures in adaptive clinical trials, Journal of Biopharmaceutical Statistics, 10.1080/10543406.2017.1399901, 28, 1, (129-145), (2017).
- George Kordzakhia, Thomas Brechenmacher, Eiji Ishida, Alex Dmitrienko, Winston Wenxiang Zheng, David Fuyuan Li, An enhanced mixture method for constructing gatekeeping procedures in clinical trials, Journal of Biopharmaceutical Statistics, 10.1080/10543406.2017.1399900, 28, 1, (113-128), (2017).
- Alex Dmitrienko, Ralph B. D'Agostino, Editorial: Multiplicity issues in clinical trials, Statistics in Medicine, 10.1002/sim.7506, 36, 28, (4423-4426), (2017).
- Alex Dmitrienko, Brian Millen, Ilya Lipkovich, Multiplicity considerations in subgroup analysis, Statistics in Medicine, 10.1002/sim.7416, 36, 28, (4446-4454), (2017).
- Abdul J. Sankoh, Haihong Li, Ralph B. D'Agostino, Composite and multicomponent end points in clinical trials, Statistics in Medicine, 10.1002/sim.7386, 36, 28, (4437-4440), (2017).
- Naitee Ting, Ding-Geng Chen, Shuyen Ho, Joseph C. Cappelleri, Naitee Ting, Ding-Geng Chen, Shuyen Ho, Joseph C. Cappelleri, Overview of Phase III Clinical Trials, Phase II Clinical Development of New Drugs, 10.1007/978-981-10-4194-5_12, (225-241), (2017).
- Dylan S. Mackay, Stephanie Jew, Peter J.H. Jones, Best practices for design and implementation of human clinical trials studying dietary oils, Progress in Lipid Research, 10.1016/j.plipres.2016.10.003, 65, (1-11), (2017).
- Hui Wang, Ilana Belitskaya-Lévy, Mei-Chiung Shih, Ying Lu, Advanced Statistical Methods, Clinical Trials Design in Operative and Non Operative Invasive Procedures, 10.1007/978-3-319-53877-8, (163-179), (2017).
- Gautier Paux, Alex Dmitrienko, Penalty-based approaches to evaluating multiplicity adjustments in clinical trials: Traditional multiplicity problems, Journal of Biopharmaceutical Statistics, 10.1080/10543406.2017.1397010, (1-23), (2017).
- Brian L. Wiens, Multiple comparisons in non-inferiority trials: Reaction to recent regulatory guidance on multiple endpoints in clinical trials, Journal of Biopharmaceutical Statistics, 10.1080/10543406.2017.1378667, (1-11), (2017).
- Gerald Hlavin, Lisa V. Hampson, Franz Koenig, Many-to-one comparisons after safety selection in multi-arm clinical trials, PLOS ONE, 10.1371/journal.pone.0180131, 12, 6, (e0180131), (2017).
- Magnus B. Arnli, Thomas Backer-Grøndahl, Borgny Ytterhus, Unn S. Granli, Stian Lydersen, Sasha Gulati, Sverre H. Torp, Expression and clinical value of EGFR in human meningiomas, PeerJ, 10.7717/peerj.3140, 5, (e3140), (2017).
- Toshimitsu Hamasaki, Scott R. Evans, Koko Asakura, Design, data monitoring, and analysis of clinical trials with co-primary endpoints: A review, Journal of Biopharmaceutical Statistics, 10.1080/10543406.2017.1378668, (1-24), (2017).
- Ilya Lipkovich, Alex Dmitrienko, Christoph Muysers, Bohdana Ratitch, Multiplicity issues in exploratory subgroup analysis, Journal of Biopharmaceutical Statistics, 10.1080/10543406.2017.1397009, (1-19), (2017).
- Hengrui Sun, Ellen Snyder, Gary G. Koch, Statistical planning in confirmatory clinical trials with multiple treatment groups, multiple visits, and multiple endpoints, Journal of Biopharmaceutical Statistics, 10.1080/10543406.2017.1378664, (1-23), (2017).
- Ajit C. Tamhane, Jiangtao Gou, Advances in p-Value Based Multiple Test Procedures, Journal of Biopharmaceutical Statistics, 10.1080/10543406.2017.1378666, (1-18), (2017).
- Mohammad Huque, Thamban Valappil, Mohamed Alosh, Consistency ensured test strategies for supportive evidence of treatment efficacy in noninferiority clinical trials, Journal of Biopharmaceutical Statistics, 10.1080/10543406.2017.1399899, (1-14), (2017).
- Ilya Lipkovich, Alex Dmitrienko, Ralph B., Tutorial in biostatistics: data‐driven subgroup identification and analysis in clinical trials, Statistics in Medicine, 10.1002/sim.7064, 36, 1, (136-196), (2016).
- Anh Nguyen Duc, Marcel Wolbers, Weighted analysis of composite endpoints with simultaneous inference for flexible weight constraints, Statistics in Medicine, 10.1002/sim.7147, 36, 3, (442-454), (2016).
- C. I. VAHL, Q. KANG, Statistical strategies for multiple testing in the safety evaluation of a genetically modified crop, The Journal of Agricultural Science, 10.1017/S0021859616000861, 155, 05, (812-831), (2016).
- Phillipe Delorme, Pierre Lafaye Micheaux, Benoit Liquet, Jérémie Riou, Type‐II generalized family‐wise error rate formulas with application to sample size determination, Statistics in Medicine, 10.1002/sim.6909, 35, 16, (2687-2714), (2016).
- Mark A. Healy, Huiying Yin, Sandra L. Wong, Multimodal cancer care in poor prognosis cancers: Resection drives long‐term outcomes, Journal of Surgical Oncology, 10.1002/jso.24217, 113, 6, (599-604), (2016).
- Mark A. Healy, Huiying Yin, Rishindra M. Reddy, Sandra L. Wong, Use of Positron Emission Tomography to Detect Recurrence and Associations With Survival in Patients With Lung and Esophageal Cancers, JNCI: Journal of the National Cancer Institute, 10.1093/jnci/djv429, 108, 7, (2016).
- R. Talebi, A. Ahmadi, F. Afraz, R. Abdoli, Parkinson's disease and lactoferrin: Analysis of dependent protein networks, Gene Reports, 10.1016/j.genrep.2016.05.006, 4, (177-183), (2016).
- Four Artemisinin-Based Treatments in African Pregnant Women with Malaria, New England Journal of Medicine, 10.1056/NEJMoa1508606, 374, 10, (913-927), (2016).
- Matthew L. Stevens, Daniel Steffens, Manuela L. Ferreira, Jane Latimer, Qiang Li, Fiona Blyth, Chris G. Maher, Patients’ and Physiotherapists’ Views on Triggers for Low Back Pain, SPINE, 10.1097/BRS.0000000000001193, 41, 4, (E218-E224), (2016).
- Paul Talsma, How much sensory panel data do we need?, Food Quality and Preference, 10.1016/j.foodqual.2016.12.005, (2016).
- Hideaki Uehara, Kazuko Satoh, Nagisa Komokata, Yohei Tokita, Mitsue Nishiyama, Maiko Iida, Junko Ishikawa, Kazuo Ogawa, Masahiro Yamamoto, Assessment of Intrasub ject Parallelism in Ex Vivo Bioassay Using Two One-Sided Tolerance Limits, Japanese Journal of Biometrics, 10.5691/jjb.37.101, 37, 2, (101-118), (2016).
- Ilana Belitskaya-Lévy, Hui Wang, Mei-Chiung Shih, Lu Tian, Gheorghe Doros, Robert A. Lew, Ying Lu, A New Overall-Subgroup Simultaneous Test for Optimal Inference in Biomarker-Targeted Confirmatory Trials, Statistics in Biosciences, 10.1007/s12561-016-9174-8, (2016).
- Alex Dmitrienko, Gautier Paux, Erik Pulkstenis, Jianliang Zhang, Tradeoff-based optimization criteria in clinical trials with multiple objectives and adaptive designs, Journal of Biopharmaceutical Statistics, 10.1080/10543406.2015.1092032, 26, 1, (120-140), (2015).
- Alex Dmitrienko, Christoph Muysers, Arno Fritsch, Ilya Lipkovich, General guidance on exploratory and confirmatory subgroup analysis in late-stage clinical trials, Journal of Biopharmaceutical Statistics, 10.1080/10543406.2015.1092033, 26, 1, (71-98), (2015).
- Thomas Ondra, Alex Dmitrienko, Tim Friede, Alexandra Graf, Frank Miller, Nigel Stallard, Martin Posch, Methods for identification and confirmation of targeted subgroups in clinical trials: A systematic review, Journal of Biopharmaceutical Statistics, 10.1080/10543406.2015.1092034, 26, 1, (99-119), (2015).
- Alex Dmitrienko, George Kordzakhia, Thomas Brechenmacher, Mixture-based gatekeeping procedures for multiplicity problems with multiple sequences of hypotheses, Journal of Biopharmaceutical Statistics, 10.1080/10543406.2015.1074917, 26, 4, (758-780), (2015).
- Alex Dmitrienko, Ilya Lipkovich, Alan Hopkins, Yu-Ping Li, Whedy Wang, Biomarker Evaluation and Subgroup Identification in a Pneumonia Development Program Using SIDES, Applied Statistics in Biomedicine and Clinical Trials Design, 10.1007/978-3-319-12694-4_26, (427-466), (2015).
- Sue‐Jane Wang, Frank Bretz, Alex Dmitrienko, Jason Hsu, H. M. James Hung, Gary Koch, Willi Maurer, Walt Offen, Robert O'Neill, Multiplicity in confirmatory clinical trials: a case study with discussion from a JSM panel, Statistics in Medicine, 10.1002/sim.6561, 34, 26, (3461-3480), (2015).
- Zhiwei Zhang, Chenguang Wang, James F. Troendle, Optimizing the order of hypotheses in serial testing of multiple endpoints in clinical trials, Statistics in Medicine, 10.1002/sim.6425, 34, 9, (1467-1482), (2015).
- Johannes Foss Sigurdson, A. M. Undheim, J. L. Wallander, S. Lydersen, A. M. Sund, The long-term effects of being bullied or a bully in adolescence on externalizing and internalizing mental health problems in adulthood, Child and Adolescent Psychiatry and Mental Health, 10.1186/s13034-015-0075-2, 9, 1, (2015).
- Victoria Vickerstaff, Gareth Ambler, Michael King, Irwin Nazareth, Rumana Z. Omar, Are multiple primary outcomes analysed appropriately in randomised controlled trials? A review, Contemporary Clinical Trials, 10.1016/j.cct.2015.07.016, 45, (8-12), (2015).
- Alex Dmitrienko, Gautier Paux, Thomas Brechenmacher, POWER CALCULATIONS IN CLINICAL TRIALS WITH COMPLEX CLINICAL OBJECTIVES, Journal of the Japanese Society of Computational Statistics, 10.5183/jjscs.1411001_213, 28, 1, (15-50), (2015).
- Seitaro Yoshida, Kentaro Sakamaki, Gatekeeping Procedure in Confirmatory Clinical Trials, Japanese Journal of Biometrics, 10.5691/jjb.36.S165, 36, Special_Issue_2, (S165-S171), (2015).
- Takashi Sozu, Multiplicity Issues in Clinical Trials, Japanese Journal of Biometrics, 10.5691/jjb.36.S87, 36, Special_Issue_2, (S87-S98), (2015).
- Takahiro Hasegawa, Multiplicity Adjustment Methods for Non-pre-specified Hypothesis Ordering, Japanese Journal of Biometrics, 10.5691/jjb.36.S123, 36, Special_Issue_2, (S123-S145), (2015).
- Steven Y. Hua, Siyan Xu, Ralph B. D'Agostino, Multiplicity adjustments in testing for bioequivalence, Statistics in Medicine, 10.1002/sim.6247, 34, 2, (215-231), (2014).
- Abdul J. Sankoh, Haihong Li, Ralph B. D'Agostino, Use of composite endpoints in clinical trials, Statistics in Medicine, 10.1002/sim.6205, 33, 27, (4709-4714), (2014).
- Jelle J. Goeman, Aldo Solari, Multiple hypothesis testing in genomics, Statistics in Medicine, 10.1002/sim.6082, 33, 11, (1946-1978), (2014).
- Jan Siewe, Jan Bredow, Johannes Oppermann, Timmo Koy, Stefan Delank, Peter Knoell, Peer Eysel, Rolf Sobottke, Kourosh Zarghooni, Marc Röllinghoff, Evaluation of efficacy of a new hybrid fusion device: a randomized, two-centre controlled trial, BMC Musculoskeletal Disorders, 10.1186/1471-2474-15-294, 15, 1, (2014).
- Olivier Guilbaud, Sharper Confidence Intervals for Hochberg- and Hommel-Related Multiple Tests Based On an Extended Simes Inequality, Statistics in Biopharmaceutical Research, 10.1080/19466315.2013.872999, 6, 2, (123-136), (2014).
- Mohamed Alosh, Frank Bretz, Mohammad Huque, Advanced multiplicity adjustment methods in clinical trials, Statistics in Medicine, 10.1002/sim.5974, 33, 4, (693-713), (2013).
