Stochastic Optimization: a Review
Summary
enWe review three leading stochastic optimization methods—simulated annealing, genetic algorithms, and tabu search. In each case we analyze the method, give the exact algorithm, detail advantages and disadvantages, and summarize the literature on optimal values of the inputs. As a motivating example we describe the solution—using Bayesian decision theory, via maximization of expected utility—of a variable selection problem in generalized linear models, which arises in the cost‐effective construction of a patient sickness‐at‐admission scale as part of an effort to measure quality of hospital care.
Résumé
frNous examinons trois méthodes principales d'optimisation stochastique. Dans chaque cas nous analysons la méthode, donnons l'algorithme exact, détaillons les avantages et inconvénients la et résumons la littérature sur les valeurs optimales des facteurs. Comme example significatif nous décrivons la solution—utilisant la théorie de décision Bayésienne, via la maximisation de l'utilité attendue—d'un problème de sélection de variable dans les modéles linéaires généralisés, qui se pose dans la construction coût‐efficacité de l'échelle de maladie à l'admission d'un patient comme partie d'un effort pour mesurer la qualité du service hospitalier.
Citing Literature
Number of times cited according to CrossRef: 84
- Freddy A. Lucay, Luis A. Cisternas, Edelmira D. Gálvez, An LS-SVM classifier based methodology for avoiding unwanted responses in processes under uncertainties., Computers & Chemical Engineering, 10.1016/j.compchemeng.2020.106860, (106860), (2020).
- Jinyu Guan, Hao Tang, Ke Wang, Jianguo Yao, Shengchun Yang, A parallel multi-scenario learning method for near-real-time power dispatch optimization, Energy, 10.1016/j.energy.2020.117708, (117708), (2020).
- Dapeng Zhang, Zhiwei Gao, Zhiling Lin, An Online Control Approach for Forging Machine Using Reinforcement Learning and Taboo Search, IEEE Access, 10.1109/ACCESS.2020.3020550, 8, (158666-158678), (2020).
- Haiyan Zheng, Lisa V. Hampson, A Bayesian decision‐theoretic approach to incorporate preclinical information into phase I oncology trials, Biometrical Journal, 10.1002/bimj.201900161, 62, 6, (1408-1427), (2020).
- Sushen Zhang, Ruijuan Chen, Wenyu Du, Ye Yuan, Vassilios S. Vassiliadis, A Hessian-Free Gradient Flow (HFGF) Method for the Optimisation of Deep Learning Neural Networks, Computers & Chemical Engineering, 10.1016/j.compchemeng.2020.107008, (107008), (2020).
- A. T. D. Perera, Vahid M. Nik, Deliang Chen, Jean-Louis Scartezzini, Tianzhen Hong, Quantifying the impacts of climate change and extreme climate events on energy systems, Nature Energy, 10.1038/s41560-020-0558-0, (2020).
- Kenyon Ng, Berwin A. Turlach, Kevin Murray, A flexible sequential Monte Carlo algorithm for parametric constrained regression, Computational Statistics & Data Analysis, 10.1016/j.csda.2019.03.011, (2019).
- Ajay Shrestha, Ausif Mahmood, undefined, 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA), 10.1109/ICMLA.2019.00222, (1365-1370), (2019).
- Khalil Amine, Multiobjective Simulated Annealing: Principles and Algorithm Variants, Advances in Operations Research, 10.1155/2019/8134674, 2019, (1-13), (2019).
- Chang-Wan Kim, Hyun-Ik Yang, Kyu-Jin Lee, Dong-Chan Lee, Metamodel-Based Optimization of a Lithium-Ion Battery Cell for Maximization of Energy Density with Evolutionary Algorithm, Journal of The Electrochemical Society, 10.1149/2.0611902jes, 166, 2, (A211-A216), (2019).
- Antonio Pepiciello, Alfredo Vaccaro, Mario Mañana, Robust Optimization of Energy Hubs Operation Based on Extended Affine Arithmetic, Energies, 10.3390/en12122420, 12, 12, (2420), (2019).
- Freddy Lucay, Edelmira Gálvez, Luis Cisternas, Design of Flotation Circuits Using Tabu-Search Algorithms: Multispecies, Equipment Design, and Profitability Parameters, Minerals, 10.3390/min9030181, 9, 3, (181), (2019).
- Qianyi Shang, Lijun Chen, Peng Peng, On‐chip evolution of combinational logic circuits using an improved genetic‐simulated annealing algorithm, Concurrency and Computation: Practice and Experience, 10.1002/cpe.5486, 0, 0, (2019).
- Dong-Chan Lee, Kyu-Jin Lee, Chang-Wan Kim, Optimization of a Lithium-Ion Battery for Maximization of Energy Density with Design of Experiments and Micro-genetic Algorithm, International Journal of Precision Engineering and Manufacturing-Green Technology, 10.1007/s40684-019-00106-4, (2019).
- Riham Al Ismaili, Min Woo Lee, D. Ian Wilson, Vassilios S. Vassiliadis, Heat exchanger network cleaning scheduling: From optimal control to mixed-Integer decision making, Computers & Chemical Engineering, 10.1016/j.compchemeng.2017.12.004, 111, (1-15), (2018).
- Fan Wang, Shengfan Zhang, Louise M. Henderson, Adaptive decision-making of breast cancer mammography screening: A heuristic-based regression model, Omega, 10.1016/j.omega.2017.05.001, 76, (70-84), (2018).
- Sebastian J. Teran Hidalgo, Tingyu Zhu, Mengyun Wu, Shuangge Ma, Overlapping clustering of gene expression data using penalized weighted normalized cut, Genetic Epidemiology, 10.1002/gepi.22164, 42, 8, (796-811), (2018).
- Efrat Taig, Ohad Ben-Shahar, Gradient Surfing: A New Deterministic Approach for Low-Dimensional Global Optimization, Journal of Optimization Theory and Applications, 10.1007/s10957-018-1397-z, (2018).
- David Higuita-Alzate, Marisol Valencia-Cárdenas, Juan Carlos Correa-Morales, Método de combinación de pronósticos usando modelos Bayesianos y una metaheurística, caso de estudio, DYNA, 10.15446/dyna.v85n207.68424, 85, 207, (337-345), (2018).
- Neda Mohamadi, Ali R. Soheili, Faezeh Toutounian, A new hybrid denoising model based on PDEs, Multimedia Tools and Applications, 10.1007/s11042-017-4858-8, 77, 10, (12057-12072), (2017).
- Selahaddin Batuhan Akben, Osmaniye Korkut Ata, undefined, 2017 International Artificial Intelligence and Data Processing Symposium (IDAP), 10.1109/IDAP.2017.8090197, (1-4), (2017).
- H. Fang, S. Walton, E. Delahaye, J. Harris, D. A. Storchak, M. Chen, Categorical Colormap Optimization with Visualization Case Studies, IEEE Transactions on Visualization and Computer Graphics, 10.1109/TVCG.2016.2599214, 23, 1, (871-880), (2017).
- Nima Nikmehr, Sajad Najafi-Ravadanegh, Amin Khodaei, Probabilistic optimal scheduling of networked microgrids considering time-based demand response programs under uncertainty, Applied Energy, 10.1016/j.apenergy.2017.04.071, 198, (267-279), (2017).
- Yang Xiang, Sylvain Gubian, Florian Martin, Generalized Simulated Annealing, Computational Optimization in Engineering - Paradigms and Applications, 10.5772/62604, (2017).
- A. Zidan, H.A. Gabbar, Scheduling interconnected micro energy grids with multiple fuel options, Smart Energy Grid Engineering, 10.1016/B978-0-12-805343-0.00004-8, (83-99), (2017).
- Aleksandar Mijatović, John Moriarty, Jure Vogrinc, Procuring load curtailment from local customers under uncertainty, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 10.1098/rsta.2016.0311, 375, 2100, (20160311), (2017).
- Paolo Postiglione, Maria Simona Andreano, Roberto Benedetti, Spatial Clusters in EU Productivity Growth, Growth and Change, 10.1111/grow.12165, 48, 1, (40-60), (2016).
- Joshua T. Bryson, Xin Jin, Sunil K. Agrawal, Configuration Robustness Analysis of the Optimal Design of Cable-Driven Manipulators, Journal of Mechanisms and Robotics, 10.1115/1.4033695, 8, 6, (061006), (2016).
- Parisa Assarzadegan, Morteza Rasti-Barzoki, Minimizing sum of the due date assignment costs, maximum tardiness and distribution costs in a supply chain scheduling problem, Applied Soft Computing, 10.1016/j.asoc.2016.06.005, 47, (343-356), (2016).
- Ajay Shrestha, Ausif Mahmood, Improving Genetic Algorithm with Fine-Tuned Crossover and Scaled Architecture, Journal of Mathematics, 10.1155/2016/4015845, 2016, (1-10), (2016).
- Yuncheng Du, Hector Budman, Thomas A. Duever, Integration of fault diagnosis and control based on a trade-off between fault detectability and closed loop performance, Journal of Process Control, 10.1016/j.jprocont.2015.12.007, 38, (42-53), (2016).
- Oktoviano Gandhi, Carlos D. Rodriguez-Gallegos, Dipti Srinivasan, undefined, 2016 IEEE Innovative Smart Grid Technologies - Asia (ISGT-Asia), 10.1109/ISGT-Asia.2016.7796394, (250-257), (2016).
- Biyu Li, Viet H. Nguyen, Chieh. L. Ng, E.A. del Rio-Chanona, Vassilios S. Vassiliadis, Harvey Arellano-Garcia, ICRS-Filter: A randomized direct search algorithm for constrained nonconvex optimization problems, Chemical Engineering Research and Design, 10.1016/j.cherd.2015.12.001, 106, (178-190), (2016).
- Andrea Mineo, Maurizio Palesi, Giuseppe Ascia, Vincenzo Catania, Exploiting antenna directivity in wireless NoC architectures, Microprocessors and Microsystems, 10.1016/j.micpro.2016.01.019, 43, (59-66), (2016).
- Tarek Kormi, Nizar Bel Hadj Ali, Tarek Abichou, Roger Green, undefined, Geo-Chicago 2016, 10.1061/9780784480144.014, (131-140), (2016).
- Joshua T. Bryson, Xin Jin, Sunil K. Agrawal, Optimal Design of Cable-Driven Manipulators Using Particle Swarm Optimization, Journal of Mechanisms and Robotics, 10.1115/1.4032103, 8, 4, (041003), (2016).
- J. N. Stander, G. Venter, M. J. Kamper, High fidelity multidisciplinary design optimisation of an electromagnetic device, Structural and Multidisciplinary Optimization, 10.1007/s00158-015-1375-0, 53, 5, (1113-1127), (2015).
- Yuncheng Du, Thomas A. Duever, Hector Budman, Fault detection and diagnosis with parametric uncertainty using generalized polynomial chaos, Computers & Chemical Engineering, 10.1016/j.compchemeng.2015.02.009, 76, (63-75), (2015).
- Hamid Khayyam, Minoo Naebe, Alireza Bab-Hadiashar, Farshid Jamshidi, Quanxiang Li, Stephen Atkiss, Derek Buckmaster, Bronwyn Fox, Stochastic optimization models for energy management in carbonization process of carbon fiber production, Applied Energy, 10.1016/j.apenergy.2015.08.008, 158, (643-655), (2015).
- undefined, Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD '15, 10.1145/2783258.2788617, (2247-2256), (2015).
- Carlos Gamarra, Josep M. Guerrero, Computational optimization techniques applied to microgrids planning: A review, Renewable and Sustainable Energy Reviews, 10.1016/j.rser.2015.04.025, 48, (413-424), (2015).
- Flah. Aymen, Habib Kraiem, Sbita. Lassaâd, Electrical Motor Parameters Estimator Improved by a Computational Algorithm, Handbook of Research on Advanced Intelligent Control Engineering and Automation, 10.4018/978-1-4666-7248-2.ch021, (567-600), (2015).
- Muhammad Salman Fakhar, Syed Abdul Rahman Kashif, Muhammad Asghar Saqib, Tehzeeb ul Hassan, Non cascaded short-term hydro-thermal scheduling using fully-informed particle swarm optimization, International Journal of Electrical Power & Energy Systems, 10.1016/j.ijepes.2015.06.030, 73, (983-990), (2015).
- Domenica Panzera, Paolo Postiglione, Economic growth in Italian NUTS 3 provinces, The Annals of Regional Science, 10.1007/s00168-014-0628-y, 53, 1, (273-293), (2014).
- F. Cecelja, A. Kokossis, D. Du, S. Yang, Asynchronous optimisation with the use of a cascade search algorithm, Computers & Chemical Engineering, 10.1016/j.compchemeng.2014.02.009, 66, (276-289), (2014).
- A. Swarnalatha, A.P. Shanthi, Complete hardware evolution based SoPC for evolvable hardware, Applied Soft Computing, 10.1016/j.asoc.2013.12.014, 18, (314-322), (2014).
- G. Tychogiorgos, K.K. Leung, Optimization-based resource allocation in communication networks, Computer Networks, 10.1016/j.comnet.2014.03.013, 66, (32-45), (2014).
- Oguz Akbilgic, Hamparsum Bozdogan, M. Erdal Balaban, A novel Hybrid RBF Neural Networks model as a forecaster, Statistics and Computing, 10.1007/s11222-013-9375-7, 24, 3, (365-375), (2013).
- Denni D Boos, L A Stefanski, Dennis D Boos, L. A Stefanski, Bayesian Inference, Essential Statistical Inference, 10.1007/978-1-4614-4818-1_4, (163-203), (2013).
- Denni D Boos, L A Stefanski, Dennis D Boos, L. A Stefanski, M-Estimation (Estimating Equations), Essential Statistical Inference, 10.1007/978-1-4614-4818-1_7, (297-337), (2013).
- Denni D Boos, L A Stefanski, Dennis D Boos, L. A Stefanski, Monte Carlo Simulation Studies, Essential Statistical Inference, 10.1007/978-1-4614-4818-1_9, (363-383), (2013).
- Denni D Boos, L A Stefanski, Dennis D Boos, L. A Stefanski, Likelihood Construction and Estimation, Essential Statistical Inference, 10.1007/978-1-4614-4818-1_2, (27-124), (2013).
- Denni D Boos, L A Stefanski, Dennis D Boos, L. A Stefanski, Bootstrap, Essential Statistical Inference, 10.1007/978-1-4614-4818-1_11, (413-448), (2013).
- Denni D Boos, L A Stefanski, Dennis D Boos, L. A Stefanski, Jackknife, Essential Statistical Inference, 10.1007/978-1-4614-4818-1_10, (385-411), (2013).
- Denni D Boos, L A Stefanski, Dennis D Boos, L. A Stefanski, Large Sample Results for Likelihood-Based Methods, Essential Statistical Inference, 10.1007/978-1-4614-4818-1_6, (275-293), (2013).
- Denni D Boos, L A Stefanski, Dennis D Boos, L. A Stefanski, Hypothesis Tests under Misspecification and Relaxed Assumptions, Essential Statistical Inference, 10.1007/978-1-4614-4818-1_8, (339-359), (2013).
- Denni D Boos, L A Stefanski, Dennis D Boos, L. A Stefanski, Roles of Modeling in Statistical Inference, Essential Statistical Inference, 10.1007/978-1-4614-4818-1_1, (3-23), (2013).
- Denni D Boos, L A Stefanski, Dennis D Boos, L. A Stefanski, Likelihood-Based Tests and Confidence Regions, Essential Statistical Inference, 10.1007/978-1-4614-4818-1_3, (125-161), (2013).
- Roberto Carapellucci, Lorena Giordano, A methodology for the synthetic generation of hourly wind speed time series based on some known aggregate input data, Applied Energy, 10.1016/j.apenergy.2012.06.044, 101, (541-550), (2013).
- Rajdeep Dutta, Ranjan Ganguli, V. Mani, Exploring isospectral cantilever beams using electromagnetism inspired optimization technique, Swarm and Evolutionary Computation, 10.1016/j.swevo.2012.09.005, 9, (37-46), (2013).
- Jose Antonio Medina Hernandez, Felipe Gomez Castaneda, Jose Antonio Moreno Cadenas, undefined, 2013 10th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE), 10.1109/ICEEE.2013.6676090, (256-261), (2013).
- Yuanzheng Meng, Hui Gong, Xiaoquan Yang, Online Geometric Calibration of Cone-Beam Computed Tomography for Arbitrary Imaging Objects, IEEE Transactions on Medical Imaging, 10.1109/TMI.2012.2224360, 32, 2, (278-288), (2013).
- Denni D Boos, L A Stefanski, Dennis D Boos, L. A Stefanski, Permutation and Rank Tests, Essential Statistical Inference, 10.1007/978-1-4614-4818-1_12, (449-530), (2013).
- Denni D Boos, L A Stefanski, Dennis D Boos, L. A Stefanski, Large Sample Theory: The Basics, Essential Statistical Inference, 10.1007/978-1-4614-4818-1_5, (207-274), (2013).
- Ciprian Radu, Lucian Vinţan, Domain-Knowledge Optimized Simulated Annealing for Network-on-Chip Application Mapping, Advances in Intelligent Control Systems and Computer Science, 10.1007/978-3-642-32548-9_34, (473-487), (2013).
- M. Harini, Jhumpa Adhikari, K. Yamuna Rani, A Review on Property Estimation Methods and Computational Schemes for Rational Solvent Design: A Focus on Pharmaceuticals, Industrial & Engineering Chemistry Research, 10.1021/ie301329y, 52, 21, (6869-6893), (2013).
- Paolo Postiglione, M. Simona Andreano, Roberto Benedetti, Using Constrained Optimization for the Identification of Convergence Clubs, Computational Economics, 10.1007/s10614-012-9325-z, 42, 2, (151-174), (2012).
- Roberto Benedetti, Monica Pratesi, Nicola Salvati, Local stationarity in small area estimation models, Statistical Methods & Applications, 10.1007/s10260-012-0208-1, 22, 1, (81-95), (2012).
- James C. Spall, Stochastic Optimization, Handbook of Computational Statistics, 10.1007/978-3-642-21551-3, (173-201), (2012).
- D. Fouskakis, Bayesian variable selection in generalized linear models using a combination of stochastic optimization methods, European Journal of Operational Research, 10.1016/j.ejor.2012.01.040, 220, 2, (414-422), (2012).
- Roberto Carapellucci, Lorena Giordano, Modeling and optimization of an energy generation island based on renewable technologies and hydrogen storage systems, International Journal of Hydrogen Energy, 10.1016/j.ijhydene.2011.10.073, 37, 3, (2081-2093), (2012).
- Antonis C. Kokossis, Patrick Linke, Siyu Yang, The Cascade Optimization Algorithm: A New Distributed Approach for the Stochastic Optimization of Engineering Applications, Industrial & Engineering Chemistry Research, 10.1021/ie1014603, 50, 9, (5266-5278), (2011).
- Siyu Yang, Antonis Kokossis, Franjo Cecelja, undefined, 2011 International Conference on Computer and Management (CAMAN), 10.1109/CAMAN.2011.5778732, (1-5), (2011).
- Edlira Shehu, Rick Vogel, Evolutionäre Algorithmen im MarketingEvolutionary algorithms in marketing research: foundations, bibliometric review and outlook, Zeitschrift für Management, 10.1007/s12354-011-0136-2, 6, 1, (29-51), (2011).
- R. Dutta, R. Ganguli, V. Mani, Exploring isospectral spring-mass systems with firefly algorithm, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 10.1098/rspa.2011.0119, 467, 2135, (3222), (2011).
- A. Sadeghi, A. Alem-Tabriz, M. Zandieh, Product portfolio planning: a metaheuristic-based simulated annealing algorithm, International Journal of Production Research, 10.1080/00207540903329338, 49, 8, (2327-2350), (2010).
- Katarina Domijan, Simon P. Wilson, Bayesian kernel projections for classification of high dimensional data, Statistics and Computing, 10.1007/s11222-009-9161-8, 21, 2, (203-216), (2009).
- C.T. Ng, M. Veidt, H.F. Lam, Guided wave damage characterisation in beams utilising probabilistic optimisation, Engineering Structures, 10.1016/j.engstruct.2009.07.009, 31, 12, (2842-2850), (2009).
- Wojciech Paszkowicz, Genetic Algorithms, a Nature-Inspired Tool: Survey of Applications in Materials Science and Related Fields, Materials and Manufacturing Processes, 10.1080/10426910802612270, 24, 2, (174-197), (2009).
- M. T. Outeiro, R. Chibante, A. S. Carvalho, A. T. de Almeida, A new parameter extraction method for accurate modeling of PEM fuel cells, International Journal of Energy Research, 10.1002/er.1525, 33, 11, (978-988), (2009).
- Guo-hui Song, Yu Wu, Cong-xin Li, Engineering design optimization based on intelligent response surface methodology, Journal of Shanghai Jiaotong University (Science), 10.1007/s12204-008-0285-3, 13, 3, (285-290), (2008).
- Víctor Hugo Gutiérrez, Mauricio Zapata, Carlos Sierra, William Laguado, Alí Santacruz, Maximizing the profitability of forestry projects under the Clean Development Mechanism using a forest management optimization model, Forest Ecology and Management, 10.1016/j.foreco.2006.02.002, 226, 1-3, (341-350), (2006).
- C. Zang, M.I. Friswell, J.E. Mottershead, A review of robust optimal design and its application in dynamics, Computers & Structures, 10.1016/j.compstruc.2004.10.007, 83, 4-5, (315-326), (2005).
- Peter Müller, Bruno Sansó, Maria De Iorio, Optimal Bayesian Design by Inhomogeneous Markov Chain Simulation, Journal of the American Statistical Association, 10.1198/016214504000001123, 99, 467, (788-798), (2004).




