One challenge of evolutionary ecology is to predict the rate and mechanisms of population adaptation to environmental variations. The variations in most life history traits are shaped both by individual genotypic and by environmental variation. Forest trees exhibit high levels of genetic diversity, large population sizes, and gene flow, and they also show a high level of plasticity for life history traits. We developed a new Physio-Demo-Genetics model (denoted PDG) coupling (i) a physiological module simulating individual tree responses to the environment; (ii) a demographic module simulating tree survival, reproduction, and pollen and seed dispersal; and (iii) a quantitative genetics module controlling the heritability of key life history traits. We used this model to investigate the plastic and genetic components of the variations in the timing of budburst (TBB) along an elevational gradient of Fagus sylvatica (the European beech). We used a repeated 5 years climatic sequence to show that five generations of natural selection were sufficient to develop nonmonotonic genetic differentiation in the TBB along the local climatic gradient but also that plastic variation among different elevations and years was higher than genetic variation. PDG complements theoretical models and provides testable predictions to understand the adaptive potential of tree populations.