Improving quality of prediction in highly dynamic environments using approximate dynamic programming
Article first published online: 8 SEP 2010
Copyright © 2010 John Wiley & Sons, Ltd.
Quality and Reliability Engineering International
Special Issue: Recent Advancements in Quality and Reliability
Volume 26, Issue 7, pages 717–732, November 2010
How to Cite
Ganesan,, R., Balakrishna, P. and Sherry, L. (2010), Improving quality of prediction in highly dynamic environments using approximate dynamic programming. Qual. Reliab. Engng. Int., 26: 717–732. doi: 10.1002/qre.1127
- Issue published online: 28 OCT 2010
- Article first published online: 8 SEP 2010
- air transportation;
- approximate dynamic programming;
- quality of prediction;
- reinforcement learning;
- taxi-out time
In many applications, decision making under uncertainty often involves two steps—prediction of a certain quality parameter or indicator of the system under study and the subsequent use of the prediction in choosing actions. The prediction process is severely challenged by highly dynamic environments that particularly involve sequential decision making, such as air traffic control at airports in which congestion prediction is critical for smooth departure operations. Taxi-out time of a flight is an excellent indicator of surface congestion and is a quality parameter used in the assessment of airport delays. The regression, queueing, and moving average models have been shown to perform poorly in predicting taxi-out times because they are slow in adapting to the changing airport dynamics. This paper presents an approximate dynamic programming approach (reinforcement learning, RL) to taxi-out time prediction. The taxi-out prediction performance was tested on flight data obtained from the Federal Aviation Administration's (FAA) Aviation System Performance Metrics (ASPM) database on Detroit International (DTW), Washington Reagan National (DCA), Boston (BOS), New York John F. Kennedy (JFK) and Tampa International (TPA) airports. For example, at the Boston airport (presented in detail) the prediction accuracy by RL model was 14running-average model. In general, the RL model was 35–50% more accurate than the regression model for all of the above airports. Copyright © 2010 John Wiley & Sons, Ltd.