Climate change is altering phenology and distributions of many species and further changes are projected. Can species physiologically adapt to climate warming? We analyse thermal tolerances of a large number of terrestrial ectotherm (n = 697), endotherm (n = 227) and plant (n = 1816) species worldwide, and show that tolerance to heat is largely conserved across lineages, while tolerance to cold varies between and within species. This pattern, previously documented for ectotherms, is apparent for this group and for endotherms and plants, challenging the longstanding view that physiological tolerances of species change continuously across climatic gradients. An alternative view is proposed in which the thermal component of climatic niches would overlap across species more than expected. We argue that hard physiological boundaries exist that constrain evolution of tolerances of terrestrial organisms to high temperatures. In contrast, evolution of tolerances to cold should be more frequent. One consequence of conservatism of upper thermal tolerances is that estimated niches for cold-adapted species will tend to underestimate their upper thermal limits, thereby potentially inflating assessments of risk from climate change. In contrast, species whose climatic preferences are close to their upper thermal limits will unlikely evolve physiological tolerances to increased heat, thereby being predictably more affected by warming.