Long-term phenology monitoring has documented numerous examples of changing flowering dates during the last century. A pivotal question is whether these phenological responses are adaptive or not under directionally changing climatic conditions. We use a classic dynamic growth model for annual plants, based on optimal control theory, to find the fitness-maximizing flowering time, defined as the switching time from vegetative to reproductive growth. In a typical scenario of global warming, with advanced growing season and increased productivity, optimal flowering time advances less than the start of the growing season. Interestingly, increased temporal spread in production over the season may either advance or delay the optimal flowering time depending on overall productivity or season length. We identify situations where large phenological changes are necessary for flowering time to remain optimal. Such changes also indicate changed selection pressures. In other situations, the model predicts advanced phenology on a calendar scale, but no selection for early flowering in relation to the start of the season. We also show that the optimum is more sensitive to increased productivity when productivity is low than when productivity is high. All our results are derived using a general, graphical method to calculate the optimal flowering time applicable for a large range of shapes of the seasonal production curve. The model can thus explain apparent maladaptation in phenological responses in a multitude of scenarios of climate change. We conclude that taking energy allocation trade-offs and appropriate time scales into account is critical when interpreting phenological patterns.