Resource performance prediction is becoming more and more important in cloud environment, and CPU load prediction is helpful for system maintenance and application schedule. However, the best predictor often varies from one resource to another. At the same time, CPU load in cloud environment has a wide range of dynamics so that the best predictor for any particular CPU load time series may change over time. Ensemble method, which can use multiple models to obtain better performances, is investigated to support CPU load prediction in this paper. The novel ensemble model proposed consists of two layers. The predictor optimization layer can continuously incorporate new predictor instances and remove those ones with a poor performance. The ensemble layer is responsible for producing the final prediction based on the results of multiple predictor instances. In addition, the ensemble layer can also provide feedbacks to the predictor optimization layer, which helps it to adopt appropriate optimization strategies. Extensive experiments have been performed on CPU load traces collected from an in-production private cloud environment, and it shows this ensemble model has a better performance than any individual model. Copyright © 2013 John Wiley & Sons, Ltd.