Volume 70, Issue 3

Stochastic Optimization: a Review

Dimitris Fouskakis

Department of Mathematical Sciences, University of Bath, Claverton Down, Bath BA2 7AY, UK. Email: df@maths.bath.ac.uk

Search for more papers by this author
David Draper

Department of Applied Mathematics and Statistics, Baskin School of Engineering, University of California, 1156 High Street, Santa Cruz CA 95064, USA. Email: wdraper@ams.ucsc.edu; web http://www.ams.ucsc.edu/~draper.

Search for more papers by this author
First published: 15 January 2007
Citations: 84

Summary

en

We review three leading stochastic optimization methods—simulated annealing, genetic algorithms, and tabu search. In each case we analyze the method, give the exact algorithm, detail advantages and disadvantages, and summarize the literature on optimal values of the inputs. As a motivating example we describe the solution—using Bayesian decision theory, via maximization of expected utility—of a variable selection problem in generalized linear models, which arises in the cost‐effective construction of a patient sickness‐at‐admission scale as part of an effort to measure quality of hospital care.

Résumé

fr

Nous examinons trois méthodes principales d'optimisation stochastique. Dans chaque cas nous analysons la méthode, donnons l'algorithme exact, détaillons les avantages et inconvénients la et résumons la littérature sur les valeurs optimales des facteurs. Comme example significatif nous décrivons la solution—utilisant la théorie de décision Bayésienne, via la maximisation de l'utilité attendue—d'un problème de sélection de variable dans les modéles linéaires généralisés, qui se pose dans la construction coût‐efficacité de l'échelle de maladie à l'admission d'un patient comme partie d'un effort pour mesurer la qualité du service hospitalier.

Number of times cited according to CrossRef: 84

  • An LS-SVM classifier based methodology for avoiding unwanted responses in processes under uncertainties., Computers & Chemical Engineering, 10.1016/j.compchemeng.2020.106860, (106860), (2020).
  • A parallel multi-scenario learning method for near-real-time power dispatch optimization, Energy, 10.1016/j.energy.2020.117708, (117708), (2020).
  • An Online Control Approach for Forging Machine Using Reinforcement Learning and Taboo Search, IEEE Access, 10.1109/ACCESS.2020.3020550, 8, (158666-158678), (2020).
  • A Bayesian decision‐theoretic approach to incorporate preclinical information into phase I oncology trials, Biometrical Journal, 10.1002/bimj.201900161, 62, 6, (1408-1427), (2020).
  • A Hessian-Free Gradient Flow (HFGF) Method for the Optimisation of Deep Learning Neural Networks, Computers & Chemical Engineering, 10.1016/j.compchemeng.2020.107008, (107008), (2020).
  • Quantifying the impacts of climate change and extreme climate events on energy systems, Nature Energy, 10.1038/s41560-020-0558-0, (2020).
  • A flexible sequential Monte Carlo algorithm for parametric constrained regression, Computational Statistics & Data Analysis, 10.1016/j.csda.2019.03.011, (2019).
  • undefined, 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA), 10.1109/ICMLA.2019.00222, (1365-1370), (2019).
  • Multiobjective Simulated Annealing: Principles and Algorithm Variants, Advances in Operations Research, 10.1155/2019/8134674, 2019, (1-13), (2019).
  • Metamodel-Based Optimization of a Lithium-Ion Battery Cell for Maximization of Energy Density with Evolutionary Algorithm, Journal of The Electrochemical Society, 10.1149/2.0611902jes, 166, 2, (A211-A216), (2019).
  • Robust Optimization of Energy Hubs Operation Based on Extended Affine Arithmetic, Energies, 10.3390/en12122420, 12, 12, (2420), (2019).
  • Design of Flotation Circuits Using Tabu-Search Algorithms: Multispecies, Equipment Design, and Profitability Parameters, Minerals, 10.3390/min9030181, 9, 3, (181), (2019).
  • On‐chip evolution of combinational logic circuits using an improved genetic‐simulated annealing algorithm, Concurrency and Computation: Practice and Experience, 10.1002/cpe.5486, 0, 0, (2019).
  • Optimization of a Lithium-Ion Battery for Maximization of Energy Density with Design of Experiments and Micro-genetic Algorithm, International Journal of Precision Engineering and Manufacturing-Green Technology, 10.1007/s40684-019-00106-4, (2019).
  • Heat exchanger network cleaning scheduling: From optimal control to mixed-Integer decision making, Computers & Chemical Engineering, 10.1016/j.compchemeng.2017.12.004, 111, (1-15), (2018).
  • Adaptive decision-making of breast cancer mammography screening: A heuristic-based regression model, Omega, 10.1016/j.omega.2017.05.001, 76, (70-84), (2018).
  • Overlapping clustering of gene expression data using penalized weighted normalized cut, Genetic Epidemiology, 10.1002/gepi.22164, 42, 8, (796-811), (2018).
  • Gradient Surfing: A New Deterministic Approach for Low-Dimensional Global Optimization, Journal of Optimization Theory and Applications, 10.1007/s10957-018-1397-z, (2018).
  • Método de combinación de pronósticos usando modelos Bayesianos y una metaheurística, caso de estudio, DYNA, 10.15446/dyna.v85n207.68424, 85, 207, (337-345), (2018).
  • A new hybrid denoising model based on PDEs, Multimedia Tools and Applications, 10.1007/s11042-017-4858-8, 77, 10, (12057-12072), (2017).
  • undefined, 2017 International Artificial Intelligence and Data Processing Symposium (IDAP), 10.1109/IDAP.2017.8090197, (1-4), (2017).
  • Categorical Colormap Optimization with Visualization Case Studies, IEEE Transactions on Visualization and Computer Graphics, 10.1109/TVCG.2016.2599214, 23, 1, (871-880), (2017).
  • Probabilistic optimal scheduling of networked microgrids considering time-based demand response programs under uncertainty, Applied Energy, 10.1016/j.apenergy.2017.04.071, 198, (267-279), (2017).
  • Generalized Simulated Annealing, Computational Optimization in Engineering - Paradigms and Applications, 10.5772/62604, (2017).
  • Scheduling interconnected micro energy grids with multiple fuel options, Smart Energy Grid Engineering, 10.1016/B978-0-12-805343-0.00004-8, (83-99), (2017).
  • Procuring load curtailment from local customers under uncertainty, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 10.1098/rsta.2016.0311, 375, 2100, (20160311), (2017).
  • Spatial Clusters in EU Productivity Growth, Growth and Change, 10.1111/grow.12165, 48, 1, (40-60), (2016).
  • Configuration Robustness Analysis of the Optimal Design of Cable-Driven Manipulators, Journal of Mechanisms and Robotics, 10.1115/1.4033695, 8, 6, (061006), (2016).
  • Minimizing sum of the due date assignment costs, maximum tardiness and distribution costs in a supply chain scheduling problem, Applied Soft Computing, 10.1016/j.asoc.2016.06.005, 47, (343-356), (2016).
  • Improving Genetic Algorithm with Fine-Tuned Crossover and Scaled Architecture, Journal of Mathematics, 10.1155/2016/4015845, 2016, (1-10), (2016).
  • Integration of fault diagnosis and control based on a trade-off between fault detectability and closed loop performance, Journal of Process Control, 10.1016/j.jprocont.2015.12.007, 38, (42-53), (2016).
  • undefined, 2016 IEEE Innovative Smart Grid Technologies - Asia (ISGT-Asia), 10.1109/ISGT-Asia.2016.7796394, (250-257), (2016).
  • ICRS-Filter: A randomized direct search algorithm for constrained nonconvex optimization problems, Chemical Engineering Research and Design, 10.1016/j.cherd.2015.12.001, 106, (178-190), (2016).
  • Exploiting antenna directivity in wireless NoC architectures, Microprocessors and Microsystems, 10.1016/j.micpro.2016.01.019, 43, (59-66), (2016).
  • undefined, Geo-Chicago 2016, 10.1061/9780784480144.014, (131-140), (2016).
  • Optimal Design of Cable-Driven Manipulators Using Particle Swarm Optimization, Journal of Mechanisms and Robotics, 10.1115/1.4032103, 8, 4, (041003), (2016).
  • High fidelity multidisciplinary design optimisation of an electromagnetic device, Structural and Multidisciplinary Optimization, 10.1007/s00158-015-1375-0, 53, 5, (1113-1127), (2015).
  • Fault detection and diagnosis with parametric uncertainty using generalized polynomial chaos, Computers & Chemical Engineering, 10.1016/j.compchemeng.2015.02.009, 76, (63-75), (2015).
  • Stochastic optimization models for energy management in carbonization process of carbon fiber production, Applied Energy, 10.1016/j.apenergy.2015.08.008, 158, (643-655), (2015).
  • undefined, Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD '15, 10.1145/2783258.2788617, (2247-2256), (2015).
  • Computational optimization techniques applied to microgrids planning: A review, Renewable and Sustainable Energy Reviews, 10.1016/j.rser.2015.04.025, 48, (413-424), (2015).
  • Electrical Motor Parameters Estimator Improved by a Computational Algorithm, Handbook of Research on Advanced Intelligent Control Engineering and Automation, 10.4018/978-1-4666-7248-2.ch021, (567-600), (2015).
  • Non cascaded short-term hydro-thermal scheduling using fully-informed particle swarm optimization, International Journal of Electrical Power & Energy Systems, 10.1016/j.ijepes.2015.06.030, 73, (983-990), (2015).
  • Economic growth in Italian NUTS 3 provinces, The Annals of Regional Science, 10.1007/s00168-014-0628-y, 53, 1, (273-293), (2014).
  • Asynchronous optimisation with the use of a cascade search algorithm, Computers & Chemical Engineering, 10.1016/j.compchemeng.2014.02.009, 66, (276-289), (2014).
  • Complete hardware evolution based SoPC for evolvable hardware, Applied Soft Computing, 10.1016/j.asoc.2013.12.014, 18, (314-322), (2014).
  • Optimization-based resource allocation in communication networks, Computer Networks, 10.1016/j.comnet.2014.03.013, 66, (32-45), (2014).
  • A novel Hybrid RBF Neural Networks model as a forecaster, Statistics and Computing, 10.1007/s11222-013-9375-7, 24, 3, (365-375), (2013).
  • Bayesian Inference, Essential Statistical Inference, 10.1007/978-1-4614-4818-1_4, (163-203), (2013).
  • M-Estimation (Estimating Equations), Essential Statistical Inference, 10.1007/978-1-4614-4818-1_7, (297-337), (2013).
  • Monte Carlo Simulation Studies, Essential Statistical Inference, 10.1007/978-1-4614-4818-1_9, (363-383), (2013).
  • Likelihood Construction and Estimation, Essential Statistical Inference, 10.1007/978-1-4614-4818-1_2, (27-124), (2013).
  • Bootstrap, Essential Statistical Inference, 10.1007/978-1-4614-4818-1_11, (413-448), (2013).
  • Jackknife, Essential Statistical Inference, 10.1007/978-1-4614-4818-1_10, (385-411), (2013).
  • Large Sample Results for Likelihood-Based Methods, Essential Statistical Inference, 10.1007/978-1-4614-4818-1_6, (275-293), (2013).
  • Hypothesis Tests under Misspecification and Relaxed Assumptions, Essential Statistical Inference, 10.1007/978-1-4614-4818-1_8, (339-359), (2013).
  • Roles of Modeling in Statistical Inference, Essential Statistical Inference, 10.1007/978-1-4614-4818-1_1, (3-23), (2013).
  • Likelihood-Based Tests and Confidence Regions, Essential Statistical Inference, 10.1007/978-1-4614-4818-1_3, (125-161), (2013).
  • A methodology for the synthetic generation of hourly wind speed time series based on some known aggregate input data, Applied Energy, 10.1016/j.apenergy.2012.06.044, 101, (541-550), (2013).
  • Exploring isospectral cantilever beams using electromagnetism inspired optimization technique, Swarm and Evolutionary Computation, 10.1016/j.swevo.2012.09.005, 9, (37-46), (2013).
  • undefined, 2013 10th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE), 10.1109/ICEEE.2013.6676090, (256-261), (2013).
  • Online Geometric Calibration of Cone-Beam Computed Tomography for Arbitrary Imaging Objects, IEEE Transactions on Medical Imaging, 10.1109/TMI.2012.2224360, 32, 2, (278-288), (2013).
  • Permutation and Rank Tests, Essential Statistical Inference, 10.1007/978-1-4614-4818-1_12, (449-530), (2013).
  • Large Sample Theory: The Basics, Essential Statistical Inference, 10.1007/978-1-4614-4818-1_5, (207-274), (2013).
  • Domain-Knowledge Optimized Simulated Annealing for Network-on-Chip Application Mapping, Advances in Intelligent Control Systems and Computer Science, 10.1007/978-3-642-32548-9_34, (473-487), (2013).
  • A Review on Property Estimation Methods and Computational Schemes for Rational Solvent Design: A Focus on Pharmaceuticals, Industrial & Engineering Chemistry Research, 10.1021/ie301329y, 52, 21, (6869-6893), (2013).
  • Using Constrained Optimization for the Identification of Convergence Clubs, Computational Economics, 10.1007/s10614-012-9325-z, 42, 2, (151-174), (2012).
  • Local stationarity in small area estimation models, Statistical Methods & Applications, 10.1007/s10260-012-0208-1, 22, 1, (81-95), (2012).
  • Stochastic Optimization, Handbook of Computational Statistics, 10.1007/978-3-642-21551-3, (173-201), (2012).
  • Bayesian variable selection in generalized linear models using a combination of stochastic optimization methods, European Journal of Operational Research, 10.1016/j.ejor.2012.01.040, 220, 2, (414-422), (2012).
  • Modeling and optimization of an energy generation island based on renewable technologies and hydrogen storage systems, International Journal of Hydrogen Energy, 10.1016/j.ijhydene.2011.10.073, 37, 3, (2081-2093), (2012).
  • The Cascade Optimization Algorithm: A New Distributed Approach for the Stochastic Optimization of Engineering Applications, Industrial & Engineering Chemistry Research, 10.1021/ie1014603, 50, 9, (5266-5278), (2011).
  • undefined, 2011 International Conference on Computer and Management (CAMAN), 10.1109/CAMAN.2011.5778732, (1-5), (2011).
  • Evolutionäre Algorithmen im MarketingEvolutionary algorithms in marketing research: foundations, bibliometric review and outlook, Zeitschrift für Management, 10.1007/s12354-011-0136-2, 6, 1, (29-51), (2011).
  • Exploring isospectral spring-mass systems with firefly algorithm, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 10.1098/rspa.2011.0119, 467, 2135, (3222), (2011).
  • Product portfolio planning: a metaheuristic-based simulated annealing algorithm, International Journal of Production Research, 10.1080/00207540903329338, 49, 8, (2327-2350), (2010).
  • Bayesian kernel projections for classification of high dimensional data, Statistics and Computing, 10.1007/s11222-009-9161-8, 21, 2, (203-216), (2009).
  • Guided wave damage characterisation in beams utilising probabilistic optimisation, Engineering Structures, 10.1016/j.engstruct.2009.07.009, 31, 12, (2842-2850), (2009).
  • Genetic Algorithms, a Nature-Inspired Tool: Survey of Applications in Materials Science and Related Fields, Materials and Manufacturing Processes, 10.1080/10426910802612270, 24, 2, (174-197), (2009).
  • A new parameter extraction method for accurate modeling of PEM fuel cells, International Journal of Energy Research, 10.1002/er.1525, 33, 11, (978-988), (2009).
  • Engineering design optimization based on intelligent response surface methodology, Journal of Shanghai Jiaotong University (Science), 10.1007/s12204-008-0285-3, 13, 3, (285-290), (2008).
  • Maximizing the profitability of forestry projects under the Clean Development Mechanism using a forest management optimization model, Forest Ecology and Management, 10.1016/j.foreco.2006.02.002, 226, 1-3, (341-350), (2006).
  • A review of robust optimal design and its application in dynamics, Computers & Structures, 10.1016/j.compstruc.2004.10.007, 83, 4-5, (315-326), (2005).
  • Optimal Bayesian Design by Inhomogeneous Markov Chain Simulation, Journal of the American Statistical Association, 10.1198/016214504000001123, 99, 467, (788-798), (2004).

The full text of this article hosted at iucr.org is unavailable due to technical difficulties.