Research Article
Comparing syndromic surveillance detection methods: EARS' versus a CUSUM‐based methodology†
†
This article is a U.S. Government work and is in the public domain in the U.S.A.
Abstract
This paper compares the performance of three detection methods, entitled C1, C2, and C3, that are implemented in the early aberration reporting system (EARS) and other syndromic surveillance systems versus the CUSUM applied to model‐based prediction errors. The cumulative sum (CUSUM) performed significantly better than the EARS' methods across all of the scenarios we evaluated. These scenarios consisted of various combinations of large and small background disease incidence rates, seasonal cycles from large to small (as well as no cycle), daily effects, and various types and levels of random daily variation. This leads us to recommend replacing the C1, C2, and C3 methods in existing syndromic surveillance systems with an appropriately implemented CUSUM method. Published in 2008 by John Wiley & Sons, Ltd.
Citing Literature
Number of times cited according to CrossRef: 68
- Emily Alsentzer, Sarah-Blythe Ballard, Joan Neyra, Delphis M. Vera, Victor B. Osorio, Jose Quispe, David L. Blazes, Luis Loayza, Assessing 3 Outbreak Detection Algorithms in an Electronic Syndromic Surveillance System in a Resource-Limited Setting, Emerging Infectious Diseases, 10.3201/eid09.191315, 26, 9, (2020).
- Emily Alsentzer, Sarah-Blythe Ballard, Joan Neyra, Delphis M. Vera, Victor B. Osorio, Jose Quispe, David L. Blazes, Luis Loayza, Assessing 3 Outbreak Detection Algorithms in an Electronic Syndromic Surveillance System in a Resource-Limited Setting, Emerging Infectious Diseases, 10.3201/eid2609.191315, 26, 9, (2196), (2020).
- Gideon Mensah Engmann, Dong Han, Multichart Schemes for Detecting Changes in Disease Incidence, Computational and Mathematical Methods in Medicine, 10.1155/2020/7267801, 2020, (1-14), (2020).
- Christin Schröder, Luis Alberto Peña Diaz, Anna Maria Rohde, Brar Piening, Seven Johannes Sam Aghdassi, Georg Pilarski, Norbert Thoma, Petra Gastmeier, Rasmus Leistner, Michael Behnke, Lean back and wait for the alarm? Testing an automated alarm system for nosocomial outbreaks to provide support for infection control professionals, PLOS ONE, 10.1371/journal.pone.0227955, 15, 1, (e0227955), (2020).
- Yousef Alimohamadi, Seyed Mohsen Zahraei, Manoochehr Karami, Mehdi Yaseri, Mojtaba Lotfizad, Kourosh Holakouie‐Naieni, The comparative performance of wavelet‐based outbreak detector, exponential weighted moving average, and Poisson regression‐based methods in detection of pertussis outbreaks in Iranian infants: A simulation‐based study, Pediatric Pulmonology, 10.1002/ppul.25036, 0, 0, (2020).
- Ünal Cengiz, Mehmet Karahasan, A new statistical early outbreak detection method for biosurveillance and performance comparisons, Statistics in Medicine, 10.1002/sim.8368, 38, 27, (5236-5258), (2019).
- Mengru Yuan, Nikita Boston-Fisher, Yu Luo, Aman Verma, David L. Buckeridge, A systematic review of aberration detection algorithms used in public health surveillance, Journal of Biomedical Informatics, 10.1016/j.jbi.2019.103181, (103181), (2019).
- Ronald D. Fricker, Howard S. Burkom, Data aggregation in disease surveillance, Journal of Quality Technology, 10.1080/00224065.2019.1611360, (1-6), (2019).
- Angela Noufaily, Roger A Morbey, Felipe J Colón-González, Alex J Elliot, Gillian E Smith, Iain R Lake, Noel McCarthy, Comparison of statistical algorithms for daily syndromic surveillance aberration detection, Bioinformatics, 10.1093/bioinformatics/bty997, (2019).
- Di Wang, Xi Zhang, Dynamic Field Monitoring Based on Multitask Learning in Sensor Networks, Sensors, 10.3390/s19071533, 19, 7, (1533), (2019).
- Felipe J. Colón-González, Iain R. Lake, Roger A. Morbey, Alex J. Elliot, Richard Pebody, Gillian E. Smith, A methodological framework for the evaluation of syndromic surveillance systems: a case study of England, BMC Public Health, 10.1186/s12889-018-5422-9, 18, 1, (2018).
- Sarah Valentin, Renaud Lancelot, Mathieu Roche, How to combine spatio-temporal and thematic features in online news for enhanced animal disease surveillance?, Procedia Computer Science, 10.1016/j.procs.2018.07.283, 126, (490-497), (2018).
- Hong Yan, Zhongqiang Zhang, Jian Zou, Dynamic Space-Time Model for Syndromic Surveillance with Particle Filters and Dirichlet Process, Handbook of Dynamic Data Driven Applications Systems, 10.1007/978-3-319-95504-9, (139-152), (2018).
- Jian Zou, Zhongqiang Zhang, Hong Yan, A hybrid hierarchical Bayesian model for spatiotemporal surveillance data, Statistics in Medicine, 10.1002/sim.7909, 37, 28, (4216-4233), (2018).
- Meng J. Zhao, Anne R. Driscoll, Srijan Sengupta, Ronald D. Fricker, Dan J. Spitzner, William H. Woodall, Performance evaluation of social network anomaly detection using a moving window–based scan method, Quality and Reliability Engineering International, 10.1002/qre.2364, 34, 8, (1699-1716), (2018).
- A. C. M. Fong, B. Fong, P. Davey, H. Gholamhosseini, C. K. Li, undefined, 2018 IEEE International Conference on Consumer Electronics (ICCE), 10.1109/ICCE.2018.8326260, (1-2), (2018).
- Céline Faverjon, John Berezowski, Choosing the best algorithm for event detection based on the intended application: A conceptual framework for syndromic surveillance, Journal of Biomedical Informatics, 10.1016/j.jbi.2018.08.001, 85, (126-135), (2018).
- Eunjoo Yang, Hyun Park, Yeon Choi, Jusim Kim, Lkhagvadorj Munkhdalai, Ibrahim Musa, Keun Ryu, A Simulation-Based Study on the Comparison of Statistical and Time Series Forecasting Methods for Early Detection of Infectious Disease Outbreaks, International Journal of Environmental Research and Public Health, 10.3390/ijerph15050966, 15, 5, (966), (2018).
- Honglong Zhang, Liping Wang, Shengjie Lai, Zhongjie Li, Qiao Sun, Peng Zhang, Surveillance and early warning systems of infectious disease in China: From 2012 to 2014, The International Journal of Health Planning and Management, 10.1002/hpm.2434, 32, 3, (329-338), (2017).
- Weizhong Yang, Zhongjie Li, Yajia Lan, Jiaqi Ma, Lianmei Jin, Shengjie Lai, Yilan Liao, Wei Lv, Qiao Sun, Jinfeng Wang, China Infectious Diseases Automated-Alert and Response System (CIDARS), Early Warning for Infectious Disease Outbreak, 10.1016/B978-0-12-812343-0.00007-2, (133-161), (2017).
- Nancy VanStone, Adam van Dijk, Timothy Chisamore, Brian Mosley, Geoffrey Hall, Paul Belanger, Kieran Michael Moore, Characterizing the Effects of Extreme Cold Using Real-time Syndromic Surveillance, Ontario, Canada, 2010-2016, Public Health Reports, 10.1177/0033354917708354, 132, 1_suppl, (48S-52S), (2017).
- Luis Enrique Jeronimo-Martinez, Raul E. Menendez-Mora, Holman Bolivar, undefined, 2017 Congreso Internacional de Innovacion y Tendencias en Ingenieria (CONIITI), 10.1109/CONIITI.2017.8273326, (1-6), (2017).
- Geert H. Groeneveld, Anton Dalhuijsen, Chakib Kara-Zaïtri, Bob Hamilton, Margot W. de Waal, Jaap T. van Dissel, Jim E. van Steenbergen, ICARES: a real-time automated detection tool for clusters of infectious diseases in the Netherlands, BMC Infectious Diseases, 10.1186/s12879-017-2300-5, 17, 1, (2017).
- Michael Huart, Gabriel Bedubourg, Cédric Abat, Philippe Colson, Jean Marc Rolain, Hervé Chaudet, Pierre Edouard Fournier, Didier Raoult, Xavier Deparis, Implementation and Initial Analysis of a Laboratory-Based Weekly Biosurveillance System, Provence-Alpes-Côte d’Azur, France, Emerging Infectious Diseases, 10.3201/eid2304.161399, 23, 4, (582-589), (2017).
- Nina Fefferman, Elena Naumova, Innovation in observation: a vision for early outbreak detection, Emerging Health Threats Journal, 10.3402/ehtj.v3i0.7103, 3, 1, (7103), (2017).
- Robert W. Mathes, Ramona Lall, Alison Levin-Rector, Jessica Sell, Marc Paladini, Kevin J. Konty, Don Olson, Don Weiss, Evaluating and implementing temporal, spatial, and spatio-temporal methods for outbreak detection in a local syndromic surveillance system, PLOS ONE, 10.1371/journal.pone.0184419, 12, 9, (e0184419), (2017).
- Gabriel Bédubourg, Yann Le Strat, Evaluation and comparison of statistical methods for early temporal detection of outbreaks: A simulation-based study, PLOS ONE, 10.1371/journal.pone.0181227, 12, 7, (e0181227), (2017).
- Chulwoo Rhee, Howard Burkom, Chang-gyo Yoon, Miles Stewart, Yevgeniy Elbert, Aaron Katz, Sangwoo Tak, Syndromic Surveillance System for Korea–US Joint Biosurveillance Portal: Design and Lessons Learned, Health Security, 10.1089/hs.2015.0067, 14, 3, (152-160), (2016).
- Chuchu Ye, Zhongjie Li, Yifei Fu, Yajia Lan, Weiping Zhu, Dinglun Zhou, Honglong Zhang, Shengjie Lai, David L. Buckeridge, Qiao Sun, Weizhong Yang, SCM: a practical tool to implement hospital-based syndromic surveillance, BMC Research Notes, 10.1186/s13104-016-2098-z, 9, 1, (2016).
- Ye Li, Michael Whelan, Leigh Hobbs, Wen Qi Fan, Cecilia Fung, Kenny Wong, Alex Marchand-Austin, Tina Badiani, Ian Johnson, Data-driven approach of CUSUM algorithm in temporal aberrant event detection using interactive web applications, Canadian Journal of Public Health, 10.17269/cjph.107.5228, 107, 1, (e9-e15), (2016).
- Isabelle Behaeghel, Anouk Veldhuis, Libo Ren, Estelle Méroc, Frank Koenen, Pierre Kerkhofs, Yves Van der Stede, Jacques Barnouin, Marc Dispas, Evaluation of a hierarchical ascendant clustering process implemented in a veterinary syndromic surveillance system, Preventive Veterinary Medicine, 10.1016/j.prevetmed.2015.03.002, 120, 2, (141-151), (2015).
- Mi Lim Lee, Comparison of Multivariate CUSUM Charts Based on Identification Accuracy for Spatio-temporal Surveillance, Journal of the Korean society for quality management, 10.7469/JKSQM.2015.43.4.521, 43, 4, (521-532), (2015).
- Ta-Chien Chan, Yung-Chu Teng, Jing-Shiang Hwang, Detection of influenza-like illness aberrations by directly monitoring Pearson residuals of fitted negative binomial regression models, BMC Public Health, 10.1186/s12889-015-1500-4, 15, 1, (2015).
- L. Faisandier, A. Fouillet, D.J. Bicout, F. Golliot, I. Ahmed, S. Bringay, D. Eilstein, Surveillance et détection des événements inhabituels en toxicovigilance : revue des méthodes pertinentes, Revue d'Épidémiologie et de Santé Publique, 10.1016/j.respe.2015.01.072, 63, 2, (119-131), (2015).
- Hong Zhou, Howard Burkom, Carla A. Winston, Achintya Dey, Umed Ajani, Practical comparison of aberration detection algorithms for biosurveillance systems, Journal of Biomedical Informatics, 10.1016/j.jbi.2015.08.023, 57, (446-455), (2015).
- Jian Zou, Alan F Karr, Gauri Datta, James Lynch, Shaun Grannis, A Bayesian spatio–temporal approach for real–time detection of disease outbreaks: a case study, BMC Medical Informatics and Decision Making, 10.1186/s12911-014-0108-4, 14, 1, (2014).
- Mi Lim Lee, David Goldsman, Seong-Hee Kim, Kwok-Leung Tsui, Spatiotemporal biosurveillance with spatial clusters: control limit approximation and impact of spatial correlation, IIE Transactions, 10.1080/0740817X.2013.785296, 46, 8, (813-827), (2014).
- Zhongjie Li, Shengjie Lai, Honglong Zhang, Liping Wang, Dinglun Zhou, Jizeng Liu, Yajia Lan, Jiaqi Ma, Hongjie Yu, David L Buckeridge, Chakrarat Pittayawonganan, Archie CA Clements, Wenbiao Hu, Weizhong Yang, Hand, foot and mouth disease in China: evaluating an automated system for the detection of outbreaks, Bulletin of the World Health Organization, 10.2471/BLT.13.130666, 92, 9, (656-663), (2014).
- Huifen Chen, Chaosian Huang, The use of a CUSUM residual chart to monitor respiratory syndromic data, IIE Transactions, 10.1080/0740817X.2012.761369, 46, 8, (790-797), (2014).
- Preetika Rani, Vaskar Raychoudhury, Sandeep Singh Sandha, Dhaval Patel, undefined, 2014 IEEE 16th International Conference on e-Health Networking, Applications and Services (Healthcom), 10.1109/HealthCom.2014.7001890, (483-488), (2014).
- Yunzhou Fan, Mei Yang, Hongbo Jiang, Ying Wang, Wenwen Yang, Zhixia Zhang, Weirong Yan, Vinod K. Diwan, Biao Xu, Hengjin Dong, Lars Palm, Li Liu, Shaofa Nie, Estimating the Effectiveness of Early Control Measures through School Absenteeism Surveillance in Observed Outbreaks at Rural Schools in Hubei, China, PLoS ONE, 10.1371/journal.pone.0106856, 9, 9, (e106856), (2014).
- Huifen Chen, Yu Chen, CUSUM Residual Charts for Monitoring Enterovirus Infections, Proceedings of the Institute of Industrial Engineers Asian Conference 2013, 10.1007/978-981-4451-98-7, (871-878), (2013).
- Yasin Şahin, A Tool for Comparing Outbreak Detection Algorithms, Advances in Computational Science, Engineering and Information Technology, 10.1007/978-3-319-00951-3_6, (59-72), (2013).
- Anna Chu, Rachel Savage, Michael Whelan, Laura C. Rosella, Natasha S. Crowcroft, Don Willison, Anne-Luise Winter, Richard Davies, Ian Gemmill, Pia K. Mucchal, Ian Johnson, Assessing the Relative Timeliness of Ontario’s Syndromic Surveillance Systems for Early Detection of the 2009 Influenza H1N1 Pandemic Waves, Canadian Journal of Public Health, 10.17269/cjph.104.3721, 104, 4, (e340-e347), (2013).
- Carlos Castillo‐Salgado, ROLE AND STRATEGIES OF SURVEILLANCE NETWORKS IN HANDLING EMERGING AND REEMERGING VIRAL INFECTIONS, Viral Infections and Global Change, 10.1002/9781118297469, (213-231), (2013).
- Rashaad Jones, Haydee Cuevas, Cheryl Bolstad, Diana Horn, Mica Endsley, Tonya Smith-Jackson, Marc Resnick, Kayenda Johnson, Disaster Management Systems, Cultural Ergonomics, 10.1201/b15563, (163-186), (2013).
- Fernanda C. Dórea, Beverly J. McEwen, W. Bruce McNab, Javier Sanchez, Crawford W. Revie, Syndromic Surveillance Using Veterinary Laboratory Data: Algorithm Combination and Customization of Alerts, PLoS ONE, 10.1371/journal.pone.0082183, 8, 12, (e82183), (2013).
- Matthew J. Heaton, David L. Banks, Jian Zou, Alan F. Karr, Gauri Datta, James Lynch, Francisco Vera, A spatio‐temporal absorbing state model for disease and syndromic surveillance, Statistics in Medicine, 10.1002/sim.5350, 31, 19, (2123-2136), (2012).
- Jian Zou, Alan F. Karr, David Banks, Matthew J. Heaton, Gauri Datta, James Lynch, Francisco Vera, Bayesian methodology for the analysis of spatial–temporal surveillance data, Statistical Analysis and Data Mining: The ASA Data Science Journal, 10.1002/sam.10142, 5, 3, (194-204), (2012).
- Ronald D. Fricker, David Banschbach, Optimizing biosurveillance systems that use threshold-based event detection methods, Information Fusion, 10.1016/j.inffus.2009.12.002, 13, 2, (117-128), (2012).
- Fahad Pervaiz, Mansoor Pervaiz, Nabeel Abdur Rehman, Umar Saif, FluBreaks: Early Epidemic Detection from Google Flu Trends, Journal of Medical Internet Research, 10.2196/jmir.2102, 14, 5, (e125), (2012).
- Justin R Boyle, Ross S Sparks, Gerben B Keijzers, Julia L Crilly, James F Lind, Louise M Ryan, Prediction and surveillance of influenza epidemics, Medical Journal of Australia, 10.5694/j.1326-5377.2011.tb02940.x, 194, S4, (S28-S33), (2011).
- John L. Szarka, Linmin Gan, William H. Woodall, Comparison of the early aberration reporting system (EARS) W2 methods to an adaptive threshold method, Statistics in Medicine, 10.1002/sim.3913, 30, 5, (489-504), (2011).
- Ronald D. Fricker, Some methodological issues in biosurveillance, Statistics in Medicine, 10.1002/sim.3880, 30, 5, (403-415), (2011).
- Rebecca Katz, Larissa May, Julia Baker, Elisa Test, Redefining syndromic surveillance, Journal of Epidemiology and Global Health, 10.1016/j.jegh.2011.06.003, 1, 1, (21-31), (2011).
- C. Läubrich, N. Bocter, H. Fickenscher, G. Selck, P. Rautenberg, Integriertes Bulletin zur automatisierten Surveillance meldepflichtiger Infektionserkrankungen in Schleswig-Holstein (IBISSH)Integrated bulletin for the automated surveillance of notifiable communicable diseases in Schleswig-Holstein (IBISSH), Bundesgesundheitsblatt - Gesundheitsforschung - Gesundheitsschutz, 10.1007/s00103-011-1299-9, 54, 7, (875-885), (2011).
- Sarah K. Rhea, Seth W. Glickman, Anna Waller, Amy Ising, Jean-Marie Maillard, Elizabeth M. Lund, Larry T. Glickman, Evaluation of Routinely Collected Veterinary and Human Health Data for Surveillance of Human Tick-Borne Diseases in North Carolina, Vector-Borne and Zoonotic Diseases, 10.1089/vbz.2009.0255, 11, 1, (9-14), (2011).
- David J. Biau, Raphaël Porcher, A method for monitoring a process from an out of control to an in control state: Application to the learning curve, Statistics in Medicine, 10.1002/sim.3947, 29, 18, (1900-1909), (2010).
- Cheryl Bolstad, Haydee Cuevas, Jingjing Wang-Costello, Mica Endsley, Walton Page, Taha Kass-Hout, Integrating Human Capabilities into Biosurveillance Systems, Biosurveillance, 10.1201/b10315, (2010).
- Xiaoli Wang, Daniel Zeng, Holly Seale, Su Li, He Cheng, Rongsheng Luan, Xiong He, Xinghuo Pang, Xiangfeng Dou, Quanyi Wang, Comparing early outbreak detection algorithms based on their optimized parameter values, Journal of Biomedical Informatics, 10.1016/j.jbi.2009.08.003, 43, 1, (97-103), (2010).
- C. Campbell, S. Blanchard, S. Chin, C. Henderson, M. Holland, K. Jennings, P. Kuehl, D. Lucey, M. McCoy, J. McCracken, B. Pecheux, M. Pietrzak, D. Roberts, M. Sanders, K. Taylor, D. Whitford, R. P. DiMassimo, undefined, 2010 IEEE Symposium on Visual Analytics Science and Technology, 10.1109/VAST.2010.5653004, (265-266), (2010).
- Anita M Pelecanos, Peter A Ryan, Michelle L Gatton, Outbreak detection algorithms for seasonal disease data: a case study using ross river virus disease, BMC Medical Informatics and Decision Making, 10.1186/1472-6947-10-74, 10, 1, (2010).
- Thomas Lotze, Galit Shmueli, Inbal Yahav, Simulating and Evaluating Biosurveillance Datasets, Biosurveillance, 10.1201/b10315, (2010).
- William H. Woodall, Olivia A. Grigg, Howard S. Burkom, Research Issues and Ideas on Health-Related Surveillance, Frontiers in Statistical Quality Control 9, 10.1007/978-3-7908-2380-6, (145-155), (2010).
- Rochelle E Watkins, Serryn Eagleson, Bert Veenendaal, Graeme Wright, Aileen J Plant, Disease surveillance using a hidden Markov model, BMC Medical Informatics and Decision Making, 10.1186/1472-6947-9-39, 9, 1, (2009).
- Thomas H. Lotze, Galit Shmueli, How does improved forecasting benefit detection? An application to biosurveillance, International Journal of Forecasting, 10.1016/j.ijforecast.2008.11.012, 25, 3, (467-483), (2009).
- Rochelle E Watkins, Serryn Eagleson, Bert Veenendaal, Graeme Wright, Aileen J Plant, Applying cusum-based methods for the detection of outbreaks of Ross River virus disease in Western Australia, BMC Medical Informatics and Decision Making, 10.1186/1472-6947-8-37, 8, 1, (2008).
- Ronald D. Fricker, Matthew C. Knitt, Cecilia X. Hu, Comparing Directionally Sensitive MCUSUM and MEWMA Procedures with Application to Biosurveillance, Quality Engineering, 10.1080/08982110802334104, 20, 4, (478-494), (2008).
