Volume 1, Issue 2
Article

The accuracy of extrapolation (time series) methods: Results of a forecasting competition

S. Makridakis

INSEAD, 77305 Fontainebleau, France

A. Andersen, Department of Economic Statistics, The University of Sydney, New South Wales 2006, Australia

R. Carbone, Faculté des Sciences de l'Administration, Université Laval, Quebec, Canada G1K 7P4

R. Fildes, Manchester Business School, University of Manchester, Manchester M15 6PB, England

R. Lewandowski, Marketing Systems GMBH, Postfach 230109, Hunsruckstraße 9a, D‐4300 ESSEN 1 (Bredeney), West Germany

J. Newton and E. Parzen, Institute of Statistics, Texas A&M University, College Station, Texas 77843, U.S.A.

R. Winkler, Indiana University, Graduate School of Business, Bloomington, IN 47405, U.S.A.

Spyros Makridakis is Professor of Management Science at INSEAD, France. He received his degree from the School of Industrial Studies in Greece and his M.B.A. and Ph.D. from New York University. He has published extensively in the areas of general systems and forecasting and has co‐authored Computer‐Aided Modeling for Managers (Addison‐Wesley, 1972), Forecasting Methods for Management, Third Edition (Wiley, 1979), Interactive Forecasting, Second Edition (Holden‐Day, 1978), and Forecasting: Methods and Applications (Wiley‐Hamilton. 1978). He has also been the co‐editor of TIMS Studies, Vol. 12, in Management Science. After his first degree in Statistics. Allan Andersen completed a Ph.D. in the Faculty of Economics at the University of Queensland. At present, he is a lecturer at the University of Sydney in the Department of Statistics. Allan's research interests lie in the general field of forecasting; in particular the time‐series approaches to the subject. He has published more than ten articles, and has co‐authored a book on non‐linear time‐series methods.

Search for more papers by this author
A. Andersen

INSEAD, 77305 Fontainebleau, France

A. Andersen, Department of Economic Statistics, The University of Sydney, New South Wales 2006, Australia

Search for more papers by this author
R. Carbone

INSEAD, 77305 Fontainebleau, France

Robert Carbone is the chairman of the Department of Management Sciences at La Faculte des Sciences de l'Administration de l'Université Laval. His Ph.D. is in urban and public affairs from Carnegie‐Mellon University. He is a member of AIDS, TIMS, and ORSA. His papers have appeared in The Journal of Environmental Systems, Management Science, IN FOR, Atmospheric Environment, and other journals. His principal current research interest is in data analysis and forecasting.

Search for more papers by this author
R. Fildes

INSEAD, 77305 Fontainebleau, France

R. Carbone, Faculté des Sciences de l'Administration, Université Laval, Quebec, Canada G1K 7P4

Robert Fildes is a lecturer in business forecasting at the Manchester Business School, University of Manchester. He received a Bachelor's degree (mathematics) from Oxford and a Ph.D. (statistics) from the University of California. He is co‐author of Forecasting for Business (Longmans, 1976) and an editor of Forecasting and Planning (Teakfield, 1978). He has also authored several articles on forecasting and applied statistics and served as a consultant in these fields. During 1978 he taught at the University of British Columbia and the University of California Berkeley.

Search for more papers by this author
M. Hibon

INSEAD, 77305 Fontainebleau, France

Michèle Hibon is currently a research assistant at INSEAD. For the last several years she has been working on various studies dealing with forecasting accuracy of time‐series methods. Before INSEAD, she was associated with the Ecole Polytechnique (Laboratoire de Recherche de Mecanique des Solides) as a computer specialist. She holds a degree in science and a diploma in advanced studies in physics.

Search for more papers by this author
R. Lewandowski

INSEAD, 77305 Fontainebleau, France

Rudolf Lewandowski, born 1938 in Valencia/Spain, studied mathematics and economics at the Sorbonne in Paris, and the universities of Bochum and Bonn and obtained degrees in both subjects. He came to the Federal Republic of Germany in 1962 as a mathematical adviser for a big French company. In 1966 he became the manager of the operations research, economics and marketing department in a leading software firm in West Germany, and founded in 1973‘MARKETING SYSTEMS’, and has been its general manager since. In 1973, he obtained a doctoral degree in economics at the Sorbonne. He has published papers on Markov processes, forecasting methodology and forecast systems, and is author of Prognose und Informationssysteme und ihre Anwendungen, 2 vols. (De Giuyter‐Verlag, Berlin 1974 and 1980), and La Prévision à Court Terme (Dunod, Paris 1979). He has lectured at various European universities, and is a member of several European Marketing Associations.

Search for more papers by this author
J. Newton

INSEAD, 77305 Fontainebleau, France

Joseph Newton is an authority on the theory and algorithms of time series analysis particularly in the area of multivariate time series. His dissertation was written in 1975 at SUNY at Buffalo in the area of estimating the parameters of vector valued autoregressive moving average time series. His publications range from work on the asymptotic distribution of maximum likelihood estimators in multiple time series to the development of efficient algorithms for the prediction of time series. He has been a research assistant professor at SUNY at Buffalo and is currently an assistant professor in the Institute of Statistics at Texas A&M University.

Search for more papers by this author
E. Parzen

INSEAD, 77305 Fontainebleau, France

Emanuel Parzen is Distinguished Professor of Statistics at Texas A&M University. He has made research contributions to probability limit theorems, statistical spectral analysis by kernel methods, statistical communication theory, time series analysis by reproducing kernel Hilberg space methods, probability density estimation, multiple time series analysis, statistical spectral analysis by autoregressive methods, order determination criteria for autoregressive schemes, forecasting, and non‐parametric statistical data modelling using quantile and density‐quantile functions. He is the author of two widely used books: Modern Probability Theory and Its Applications (1960) and Stochastic Processes (1962).

Search for more papers by this author
R. Winkler

INSEAD, 77305 Fontainebleau, France

Robert Winkler is a distinguished professor of Quantitative Business Analysis in the Graduate School of Business, Indiana University, Bloomington, Indiana. For the 1980‐81 academic year he was a Visiting Professor at INSEAD, Fontainebleau, France. He received a B.S. from the University of Illinois and a Ph.D. from the University of Chicago, and he has held visiting positions at the University of Washington, the International Institute for Applied Systems Analysis, and Stanford University. He is the author of two books and numerous journal articles. His primary research interests involve probability forecasting, Bayesian inference, and statistical decision theory. He is a member of Tl MS and is currently serving as a Departmental Editor for Management Science.

Search for more papers by this author
First published: April/June 1982
Citations: 812

Abstract

In the last few decades many methods have become available for forecasting. As always, when alternatives exist, choices need to be made so that an appropriate forecasting method can be selected and used for the specific situation being considered. This paper reports the results of a forecasting competition that provides information to facilitate such choice. Seven experts in each of the 24 methods forecasted up to 1001 series for six up to eighteen time horizons. The results of the competition are presented in this paper whose purpose is to provide empirical evidence about differences found to exist among the various extrapolative (time series) methods used in the competition.

Number of times cited according to CrossRef: 812

  • Behavioral Analysis of Human-Machine Interaction in the Context of Demand Planning Decisions, Advances in Artificial Intelligence, Software and Systems Engineering, 10.1007/978-3-030-20454-9_13, (130-141), (2020).
  • Spatiotemporal Precipitation Modeling by AI Based Ensemble Approach, 10th International Conference on Theory and Application of Soft Computing, Computing with Words and Perceptions - ICSCCW-2019, 10.1007/978-3-030-35249-3_16, (127-136), (2020).
  • Generalizing the Theta method for automatic forecasting, European Journal of Operational Research, 10.1016/j.ejor.2020.01.007, (2020).
  • Encountered Problems of Time Series with Neural Networks: Models and Architectures, Recent Trends in Artificial Neural Networks - from Training to Prediction, 10.5772/intechopen.77409, (2020).
  • Forecasting Model Selection Using Intermediate Classification: Application to MonarchFx Corporation, Expert Systems with Applications, 10.1016/j.eswa.2020.113371, (113371), (2020).
  • CNN-based image recognition for topology optimization, Knowledge-Based Systems, 10.1016/j.knosys.2020.105887, (105887), (2020).
  • Forecasting Method of Product Shipment, Scientific and Technical Revolution: Yesterday, Today and Tomorrow, 10.1007/978-3-030-47945-9_63, (581-592), (2020).
  • Forecasting PM10 concentrations using time series models: a case of the most polluted cities in Turkey, Environmental Science and Pollution Research, 10.1007/s11356-020-08164-x, 27, 20, (25612-25624), (2020).
  • Spatial–temporal variations and forecasting analysis of municipal solid waste in the mountainous city of north-western Himalayas, SN Applied Sciences, 10.1007/s42452-020-2975-x, 2, 7, (2020).
  • Treating and Pruning: New approaches to forecasting model selection and combination using prediction intervals, International Journal of Forecasting, 10.1016/j.ijforecast.2020.07.005, (2020).
  • Demand forecasting in the presence of systematic events: Cases in capturing sales promotions, International Journal of Production Economics, 10.1016/j.ijpe.2020.107892, (107892), (2020).
  • A new iterative method for solving the joint dynamic storage location assignment, order batching and picker routing problem in manual picker-to-parts warehouses, Computers & Industrial Engineering, 10.1016/j.cie.2020.106645, 147, (106645), (2020).
  • Automatic robust estimation for exponential smoothing: perspectives from statistics and machine learning, Expert Systems with Applications, 10.1016/j.eswa.2020.113637, (113637), (2020).
  • Cross-Validation Approach to Evaluate Clustering Algorithms: An Experimental Study Using Multi-Label Datasets, SN Computer Science, 10.1007/s42979-020-00283-z, 1, 5, (2020).
  • Designing Causal Inference Systems for Value-Based Spare Parts Pricing, Perspectives in Business Informatics Research, 10.1007/978-3-030-61140-8_13, (191-204), (2020).
  • Environmental pressure of the European agricultural system: Anticipating the biophysical consequences of internalization, Ecosystem Services, 10.1016/j.ecoser.2020.101195, 46, (101195), (2020).
  • How oviform is the chicken egg? New mathematical insight into the old oomorphological problem, Food Control, 10.1016/j.foodcont.2020.107484, (107484), (2020).
  • Can a Neural Network Property Portfolio Selection Process Outperform the Property Market?, Journal of Real Estate Portfolio Management, 10.1080/10835547.2005.12089721, 11, 2, (105-121), (2020).
  • A machine learning approach to univariate time series forecasting of quarterly earnings, Review of Quantitative Finance and Accounting, 10.1007/s11156-020-00871-3, (2020).
  • Mobility aware autonomic approach for the migration of application modules in fog computing environment, Journal of Ambient Intelligence and Humanized Computing, 10.1007/s12652-020-01854-x, (2020).
  • Forecasting the novel coronavirus COVID-19, PLOS ONE, 10.1371/journal.pone.0231236, 15, 3, (e0231236), (2020).
  • Comparative analysis of forecasting for air cargo volume: Statistical techniques vs. machine learning, Journal of Data, Information and Management, 10.1007/s42488-020-00031-1, (2020).
  • Decision Tree to Predict the Color Quality of Refined Bleached Deodorized Palm Oil (RBPO), IOP Conference Series: Materials Science and Engineering, 10.1088/1757-899X/851/1/012008, 851, (012008), (2020).
  • Assessment and prediction of surface ozone in Northwest Indo-Gangetic Plains using ensemble approach, Environment, Development and Sustainability, 10.1007/s10668-020-00841-8, (2020).
  • Are Combined Tourism Forecasts Better at Minimizing Forecasting Errors?, Forecasting, 10.3390/forecast2030012, 2, 3, (211-229), (2020).
  • Comparison of forecast accuracy of Ata and exponential smoothing , Journal of Applied Statistics, 10.1080/02664763.2020.1803813, (1-11), (2020).
  • Evaluating time series forecasting models: an empirical study on performance estimation methods, Machine Learning, 10.1007/s10994-020-05910-7, (2020).
  • Spatiotemporal precipitation modeling by artificial intelligence-based ensemble approach, Environmental Earth Sciences, 10.1007/s12665-019-8755-5, 79, 1, (2019).
  • The inventory performance of forecasting methods: Evidence from the M3 competition data, International Journal of Forecasting, 10.1016/j.ijforecast.2018.01.004, 35, 1, (251-265), (2019).
  • Perspectives on supply chain forecasting, International Journal of Forecasting, 10.1016/j.ijforecast.2018.11.002, 35, 1, (121-127), (2019).
  • Integrating human judgement into quantitative forecasting methods: A review, Omega, 10.1016/j.omega.2018.07.012, 86, (237-252), (2019).
  • Looking for Accurate Forecasting of Copper TC/RC Benchmark Levels, Complexity, 10.1155/2019/8523748, 2019, (1-16), (2019).
  • Introductory overview: Error metrics for hydrologic modelling – A review of common practices and an open source library to facilitate use and adoption, Environmental Modelling & Software, 10.1016/j.envsoft.2019.05.001, (2019).
  • Predictive analytics of the copper spot price by utilizing complex network and artificial neural network techniques, Resources Policy, 10.1016/j.resourpol.2019.101414, 63, (101414), (2019).
  • Weighted sequential hybrid approaches for time series forecasting, Physica A: Statistical Mechanics and its Applications, 10.1016/j.physa.2019.121717, (121717), (2019).
  • Demand Forecasting: A Case Study in the Food Industry, Computational Science and Its Applications – ICCSA 2019, 10.1007/978-3-030-24302-9_5, (50-63), (2019).
  • undefined, 2019 IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC), 10.1109/COMPSAC.2019.10228, (330-335), (2019).
  • A Study on the Efficiency of Hybrid Models in Forecasting Precipitations and Water Inflow Albania Case Study, Advances in Science, Technology and Engineering Systems Journal, 10.25046/aj040129, 4, 1, (2019).
  • Demand and order‐fulfillment planning: The impact of point‐of‐sale data, retailer orders and distribution center orders on forecast accuracy, Journal of Operations Management, 10.1002/joom.1026, 65, 5, (468-486), (2019).
  • Current Midyear Municipal Budget Forecast Accuracy, The Palgrave Handbook of Government Budget Forecasting, 10.1007/978-3-030-18195-6_13, (257-272), (2019).
  • Forecasting Across Time Series Databases using Recurrent Neural Networks on Groups of Similar Series: A Clustering Approach, Expert Systems with Applications, 10.1016/j.eswa.2019.112896, (112896), (2019).
  • Why does forecast combination work so well?, International Journal of Forecasting, 10.1016/j.ijforecast.2019.03.010, (2019).
  • Are forecasting competitions data representative of the reality?, International Journal of Forecasting, 10.1016/j.ijforecast.2018.12.007, (2019).
  • Responses to discussions and commentaries, International Journal of Forecasting, 10.1016/j.ijforecast.2019.05.002, (2019).
  • The M4 competition: Bigger. Stronger. Better, International Journal of Forecasting, 10.1016/j.ijforecast.2019.05.005, (2019).
  • A brief history of forecasting competitions, International Journal of Forecasting, 10.1016/j.ijforecast.2019.03.015, (2019).
  • The M4 competition: Conclusions, International Journal of Forecasting, 10.1016/j.ijforecast.2019.05.006, (2019).
  • On the M4.0 forecasting competition: Can you tell a 4.0 earthquake from a 3.0?, International Journal of Forecasting, 10.1016/j.ijforecast.2019.03.023, (2019).
  • The M4 Competition: 100,000 time series and 61 forecasting methods, International Journal of Forecasting, 10.1016/j.ijforecast.2019.04.014, (2019).
  • A combination-based forecasting method for the M4-competition, International Journal of Forecasting, 10.1016/j.ijforecast.2019.03.030, (2019).
  • Criteria for classifying forecasting methods, International Journal of Forecasting, 10.1016/j.ijforecast.2019.05.008, (2019).
  • Forecasting in social settings: The state of the art, International Journal of Forecasting, 10.1016/j.ijforecast.2019.05.011, (2019).
  • Analysis of Drought Potential in Sumba Island until 2040 Caused by Climate Change, Journal of Physics: Conference Series, 10.1088/1742-6596/1373/1/012004, 1373, (012004), (2019).
  • An artificial neural network-differential evolution approach for optimization of bidirectional functionally graded beams, Composite Structures, 10.1016/j.compstruct.2019.111517, (111517), (2019).
  • Multi-station artificial intelligence based ensemble modeling of reference evapotranspiration using pan evaporation measurements, Journal of Hydrology, 10.1016/j.jhydrol.2019.123958, (123958), (2019).
  • Multi-step ahead modeling of reference evapotranspiration using a multi-model approach, Journal of Hydrology, 10.1016/j.jhydrol.2019.124434, (124434), (2019).
  • Forecasting container throughput with long short-term memory networks, Industrial Management & Data Systems, 10.1108/IMDS-07-2019-0370, ahead-of-print, ahead-of-print, (2019).
  • Tuning Forecasting Algorithms for Black Swans, IFAC-PapersOnLine, 10.1016/j.ifacol.2019.11.411, 52, 13, (1496-1501), (2019).
  • Numerical Solution for the Extrapolation Problem of Analytic Functions, Research, 10.34133/2019/3903187, 2019, (1-10), (2019).
  • Crystal Balls and Black Boxes: What Makes a Good Forecast?, Journal of Planning Literature, 10.1177/0885412219838495, (088541221983849), (2019).
  • A Data-Weighted Prior Estimator for Forecast Combination, Entropy, 10.3390/e21040429, 21, 4, (429), (2019).
  • Automatic SARIMA modeling and forecast accuracy, Communications in Statistics - Simulation and Computation, 10.1080/03610918.2019.1618471, (1-22), (2019).
  • Demand Forecasting with Five Parameter Exponential Smoothing, IOP Conference Series: Materials Science and Engineering, 10.1088/1757-899X/495/1/012014, 495, (012014), (2019).
  • Rainfall Scenario of West Nusa Tenggara in 2040 Based on CCAM RCP 4.5, IOP Conference Series: Earth and Environmental Science, 10.1088/1755-1315/303/1/012033, 303, (012033), (2019).
  • Economic and mathematical modeling as an effective tool of the analysis of economic processes in industry, Russian Journal of Industrial Economics, 10.17073/2072-1633-2019-3-316-322, 12, 3, (316-322), (2019).
  • Improving the forecasting performance of temporal hierarchies, PLOS ONE, 10.1371/journal.pone.0223422, 14, 10, (e0223422), (2019).
  • Selection of Prediction Method of Basic Statistical Work Parameters of N.V. Sklifosovsky Research Institute for Emergency Medicine of the Moscow Healthcare Department, Russian Sklifosovsky Journal "Emergency Medical Care", 10.23934/2223-9022-2019-8-3-246-256, 8, 3, (246-256), (2019).
  • Sales forecasting in financial distribution: a comparison of quantitative forecasting methods, Journal of Financial Services Marketing, 10.1057/s41264-019-00068-3, (2019).
  • ATA Method, Hacettepe Journal of Mathematics and Statistics, 10.15672/hujms.461032, (1-7), (2019).
  • Multi-region Modeling of Daily Global Solar Radiation with Artificial Intelligence Ensemble, Natural Resources Research, 10.1007/s11053-018-09450-9, (2019).
  • Kripto Para Fiyatlarının Klasik ve Yapay Sinir Ağı Modelleri ile Tahmini, Kafkas Üniversitesi İktisadi ve İdari Bilimler Fakültesi Dergisi, 10.36543/kauiibfd.2019.026, 10, 20, (608-640), (2019).
  • Integrated innovative product design and supply chain tactical planning within a blockchain platform, International Journal of Production Research, 10.1080/00207543.2019.1651947, (1-21), (2019).
  • Temperature prediction based on a space–time regression-kriging model, Journal of Applied Statistics, 10.1080/02664763.2019.1671962, (1-23), (2019).
  • State-space ARIMA for supply-chain forecasting, International Journal of Production Research, 10.1080/00207543.2019.1600764, (1-10), (2019).
  • A novel error-output recurrent neural network model for time series forecasting, Neural Computing and Applications, 10.1007/s00521-019-04474-5, (2019).
  • Tourism forecast combination using the stochastic frontier analysis technique, Tourism Economics, 10.1177/1354816619868089, (135481661986808), (2019).
  • The influence of graphical format on judgmental forecasting accuracy: Lines versus points, FUTURES & FORESIGHT SCIENCE, 10.1002/ffo2.7, 1, 1, (2018).
  • OR in spare parts management: A review, European Journal of Operational Research, 10.1016/j.ejor.2017.07.058, 266, 2, (395-414), (2018).
  • Forecasting for big data: Does suboptimality matter?, Computers & Operations Research, 10.1016/j.cor.2017.05.007, 98, (322-329), (2018).
  • Exploring the sources of uncertainty: Why does bagging for time series forecasting work?, European Journal of Operational Research, 10.1016/j.ejor.2018.01.045, 268, 2, (545-554), (2018).
  • Objectivity, reproducibility and replicability in forecasting research, International Journal of Forecasting, 10.1016/j.ijforecast.2018.05.001, 34, 4, (835-838), (2018).
  • The M4 Competition: Results, findings, conclusion and way forward, International Journal of Forecasting, 10.1016/j.ijforecast.2018.06.001, 34, 4, (802-808), (2018).
  • Managing electricity price modeling risk via ensemble forecasting: The case of Turkey, Energy Policy, 10.1016/j.enpol.2018.08.053, 123, (390-403), (2018).
  • Material optimization of functionally graded plates using deep neural network and modified symbiotic organisms search for eigenvalue problems, Composites Part B: Engineering, 10.1016/j.compositesb.2018.09.087, (2018).
  • Predictive analytics of crude oil prices by utilizing the intelligent model search engine, Applied Energy, 10.1016/j.apenergy.2018.07.071, 228, (2387-2397), (2018).
  • A least squares-based parallel hybridization of statistical and intelligent models for time series forecasting, Computers & Industrial Engineering, 10.1016/j.cie.2018.02.023, 118, (44-53), (2018).
  • Prediction of Static and Dynamic Behavior of Truss Structures Using Deep Learning, Journal of The korean Association For Spatial Structures, 10.9712/KASS.2018.18.4.69, 18, 4, (69-80), (2018).
  • Railroad transportation of crude oil in Canada: Developing long-term forecasts, and evaluating the impact of proposed pipeline projects, Journal of Transport Geography, 10.1016/j.jtrangeo.2018.04.019, 69, (98-111), (2018).
  • Introducere n Analiza Trendului, Partea a Patra (Introduction to Trend Analysis, Part 4), SSRN Electronic Journal, 10.2139/ssrn.3148768, (2018).
  • Revisiting the value of information sharing in two-stage supply chains, European Journal of Operational Research, 10.1016/j.ejor.2018.04.040, 270, 3, (1044-1052), (2018).
  • dynXcube – Categorizing dynamic data analysis, Information Sciences, 10.1016/j.ins.2018.06.026, 463-464, (21-32), (2018).
  • Capacity restrictions and supply chain performance: Modelling and analysing load-dependent lead times, International Journal of Production Economics, 10.1016/j.ijpe.2018.08.008, 204, (264-277), (2018).
  • Hybrid Neural Networks for Time Series Forecasting, Artificial Intelligence, 10.1007/978-3-030-00617-4_21, (230-239), (2018).
  • undefined, 2018 IEEE 32nd International Conference on Advanced Information Networking and Applications (AINA), 10.1109/AINA.2018.00038, (181-188), (2018).
  • Times Series Analysis, Multiscale Forecasting Models, 10.1007/978-3-319-94992-5, (1-29), (2018).
  • Autocorrelated process control: Geometric Brownian Motion approach versus Box-Jenkins approach, Journal of Physics: Conference Series, 10.1088/1742-6596/995/1/012039, 995, (012039), (2018).
  • A PSO-Based ANN Model for Short-Term Electricity Price Forecasting, Ambient Communications and Computer Systems, 10.1007/978-981-10-7386-1_47, (553-563), (2018).
  • Model confidence sets and forecast combination: an application to age-specific mortality, Genus, 10.1186/s41118-018-0043-9, 74, 1, (2018).
  • Demand of automotive fuels in Brazil: Underlying energy demand trend and asymmetric price response, Energy Economics, 10.1016/j.eneco.2018.07.005, 74, (644-655), (2018).
  • Non-stationarity and ARIMA(p,d,q) Processes, Erste Hilfe bei Brustkrebs, 10.1007/978-3-319-98282-3_5, (101-122), (2018).
  • See more

The full text of this article hosted at iucr.org is unavailable due to technical difficulties.