Forest responses to climate change will depend on demographic impacts in the context of competition. Current models used to predict species responses, termed climate envelope models (CEMs), are controversial, because (i) calibration and prediction are based on correlations in space (CIS) between species abundance and climate, rather than responses to climate change over time (COT), and (ii) they omit competition. To determine the relative importance of COT, CIS, and competition for light, we applied a longitudinal analysis of 27 000 individual trees over 6–18 years subjected to experimental and natural variation in risk factors. Sensitivities and climate and resource tracking identify which species are vulnerable to these risk factors and in what ways. Results show that responses to COT differ from those predicted based on CIS. The most important impact is the effect of spring temperature on fecundity, rather than any input variable on growth or survival. Of secondary importance is growing season moisture. Species in the genera Pinus, Ulmus, Magnolia, and Fagus are particularly vulnerable to climate variation. However, the effect of competition on growth and mortality risk exceeds the effects of climate variation in space or time for most species. Because sensitivities to COT and competition are larger than CIS, current models miss the most important effects. By directly comparing sensitivity to climate in time and space, together with competition, the approach identifies which species are sensitive to climate change and why, including the heretofore overlooked impact on fecundity.