Rising temperatures are predicted to cause temperate tree species to expand north into currently boreal dominated forests. Other factors, such as overabundant deer, may hinder temperate expansion. We examined how interactions among temperature, browse pressure, light availability, and initial size impact height and radial growth of naturally regenerated, competing temperate and boreal saplings across their overlapping range limits in central North America. In 9 of 10 growth model comparisons, the inclusion of mean summer temperature and browse damage as explanatory variables strongly improved model performance over the base model with only initial size and light availability as parameters. Potential growth reductions due to browse damage and temperature limitation were similar in magnitude (up to ~50%). Temperate sapling growth increased and boreal growth decreased with temperature across a regional summer temperature gradient (2.3 °C), causing a rank reversal in growth rates, and suggesting that temperature is a key driver of sapling performance and range boundaries. However, under high browse pressure positive temperate responses to temperature were eliminated, essentially pushing the crossover point in growth between temperate and boreal species further south. These results highlight the importance of interactions among global change agents and potential impediments for tree species to track a rapidly changing climate.