Replicating and breaking models: good for you and good for ecology
Abstract
There are two major limitations to the potential of computational models in ecology for producing general insights: their design is path‐dependent, reflecting different underlying questions, assumptions, and data, and there is too little robustness analysis exploring where the model mechanisms explaining certain observations break down. We here argue that both limitations could be overcome if modellers in ecology would more often replicate existing models, try to break the models, and explore modifications. Replication comprises the re‐implementation of an existing model and the replication of its results. Breaking models means to identify under what conditions the mechanisms represented in a model can no longer explain observed phenomena. The benefits of replication include less effort being spent to enter the iterative stage of model development and having more time for systematic robustness analysis. A culture of replication would lead to increased credibility, coherence and efficiency of computational modelling and thereby facilitate theory development.
Citing Literature
Number of times cited according to CrossRef: 22
- Fredy Andrey López González, Rodrigo Ignacio Vargas Allende, Juan Manuel Soares de Lima, Maria Eugênia Andrighetto Canozzi, Amir Gil Sessim, Júlio Otávio Jardim Barcellos, Intensification of cow-calf production: How does the system respond biologically to energy inputs in a long-term horizon?, Livestock Science, 10.1016/j.livsci.2020.104058, (104058), (2020).
- Wenwu Tang, Volker Grimm, Leigh Tesfatsion, Eric Shook, David Bennett, Li An, Zhaoya Gong, Xinyue Ye, Code Reusability and Transparency of Agent-Based Modeling: A Review from a Cyberinfrastructure Perspective, High Performance Computing for Geospatial Applications, 10.1007/978-3-030-47998-5_7, (115-134), (2020).
- M. À. Colomer, P. Oliva‐Vidal, J. Jiménez, J. M. Martínez, A. Margalida, Prioritizing among removal scenarios for the reintroduction of endangered species: insights from bearded vulture simulation modeling, Animal Conservation, 10.1111/acv.12549, 23, 4, (396-406), (2019).
- Iris Lorscheid, Uta Berger, Volker Grimm, Matthias Meyer, From cases to general principles: A call for theory development through agent-based modeling, Ecological Modelling, 10.1016/j.ecolmodel.2018.10.006, 393, (153-156), (2019).
- References, Geographical Modeling, 10.1002/9781119687290, (193-220), (2019).
- Jean Marchal, Steven G. Cumming, Eliot J. B. McIntire, Turning Down the Heat: Vegetation Feedbacks Limit Fire Regime Responses to Global Warming, Ecosystems, 10.1007/s10021-019-00398-2, (2019).
- Antoni Margalida, Pilar Oliva-Vidal, Alfonso Llamas, Mª Àngels Colomer, Bioinspired models for assessing the importance of transhumance and transboundary management in the conservation of European avian scavengers, Biological Conservation, 10.1016/j.biocon.2018.11.004, 228, (321-330), (2018).
- Frederik De Laender, Community‐ and ecosystem‐level effects of multiple environmental change drivers: Beyond null model testing, Global Change Biology, 10.1111/gcb.14382, 24, 11, (5021-5030), (2018).
- Volker Grimm, Gary Polhill, Julia Touza, Documenting Social Simulation Models: The ODD Protocol as a Standard, Simulating Social Complexity, 10.1007/978-3-319-66948-9_15, (349-365), (2017).
- Devin Rose, Brandon P. M. Edwards, Ross Kett, Michael Yodzis, Justin Angevaare, Daniel Gillis, undefined, 2017 IEEE Canada International Humanitarian Technology Conference (IHTC), 10.1109/IHTC.2017.8058206, (85-89), (2017).
- Nuno Fachada, Vitor V. Lopes, Rui C. Martins, Agostinho C. Rosa, Model-independent comparison of simulation output, Simulation Modelling Practice and Theory, 10.1016/j.simpat.2016.12.013, 72, (131-149), (2017).
- Denise Pumain, Romain Reuillon, Denise Pumain, Romain Reuillon, An Incremental Multi-Modelling Method to Simulate Systems of Cities’ Evolution, Urban Dynamics and Simulation Models, 10.1007/978-3-319-46497-8_4, (57-80), (2017).
- Jesse T. Rieb, Rebecca Chaplin-Kramer, Gretchen C. Daily, Paul R. Armsworth, Katrin Böhning-Gaese, Aletta Bonn, Graeme S. Cumming, Felix Eigenbrod, Volker Grimm, Bethanna M. Jackson, Alexandra Marques, Subhrendu K. Pattanayak, Henrique M. Pereira, Garry D. Peterson, Taylor H. Ricketts, Brian E. Robinson, Matthias Schröter, Lisa A. Schulte, Ralf Seppelt, Monica G. Turner, Elena M. Bennett, When, Where, and How Nature Matters for Ecosystem Services: Challenges for the Next Generation of Ecosystem Service Models, BioScience, 10.1093/biosci/bix075, 67, 9, (820-833), (2017).
- Elizabeth Donkin, Peter Dennis, Andrey Ustalakov, John Warren, Amanda Clare, Replicating complex agent based models, a formidable task, Environmental Modelling & Software, 10.1016/j.envsoft.2017.01.020, 92, (142-151), (2017).
- Nuno David, Nuno Fachada, Agostinho C. Rosa, Verifying and Validating Simulations, Simulating Social Complexity, 10.1007/978-3-319-66948-9_9, (173-204), (2017).
- Design of a Model Base Framework for Model Environment Construction in a Virtual Geographic Environment (VGE), ISPRS International Journal of Geo-Information, 10.3390/ijgi6050145, 6, 5, (145), (2017).
- Viktoriia Radchuk, Steffen Oppel, Jürgen Groeneveld, Volker Grimm, Nicolas Schtickzelle, Simple or complex: Relative impact of data availability and model purpose on the choice of model types for population viability analyses, Ecological Modelling, 10.1016/j.ecolmodel.2015.11.022, 323, (87-95), (2016).
- Iris Lorscheid, Matthias Meyer, Divide and conquer: Configuring submodels for valid and efficient analyses of complex simulation models, Ecological Modelling, 10.1016/j.ecolmodel.2015.11.013, 326, (152-161), (2016).
- Greta Bocedi, Justin M. J. Travis, Models of Dispersal Evolution Highlight Several Important Issues in Evolutionary and Ecological Modeling, The American Naturalist, 10.1086/684191, 187, 1, (143-150), (2016).
- Volker Grimm, Uta Berger, Robustness analysis: Deconstructing computational models for ecological theory and applications, Ecological Modelling, 10.1016/j.ecolmodel.2015.07.018, 326, (162-167), (2016).
- James Millington, John Wainwright, Comparative Approaches for Innovation in Agent-Based Modelling of Landscape Change, Land, 10.3390/land5020013, 5, 2, (13), (2016).
- Clémentine Cottineau, Romain Reuillon, Paul Chapron, Sébastien Rey-Coyrehourcq, Denise Pumain, A Modular Modelling Framework for Hypotheses Testing in the Simulation of Urbanisation, Systems, 10.3390/systems3040348, 3, 4, (348-377), (2015).




