Changes in animal body size have been widely reported as a correlate of contemporary climate change. Body size affects metabolism and fitness, so changing size has implications for resilience, yet the climatic factors that drive size variation remain poorly understood. We test the role of mean and extreme temperature, rainfall, and remotely sensed primary productivity (NDVI) as drivers of body size in a sedentary, semi-arid Australian passerine, Ptilotula (Lichenostomus) penicillatus, over 23 years. To distinguish effects due to differential growth from changes in population composition, we analysed first-year birds and adults separately and considered climatic variation at three temporal scales (current, previous, and preceding 5 years). The strongest effects related to temperature: in both age classes, larger size was associated with warmer mean temperatures in the previous year, contrary to Bergmann's Rule. Moreover, adults were larger in warmer breeding seasons, while first years was larger after heatwaves; these effects are more likely to be mediated through size-dependent mortality, highlighting the role of body size in determining vulnerability to extinction. In addition to temperature, larger adult size was associated with lower primary productivity, which may reflect a trade-off between vegetative growth and nectar production, on which adults rely. Finally, lower rainfall was associated with decreasing size in first year and adults, most likely related to decreased food availability. Overall, body size increased over 23 years, strongly in first-year birds (2.7%) compared with adults (1%), with size outcomes a balance between competing drivers. As rainfall declined over time and productivity remained fairly stable, the temporal increase in body size appears largely driven by rising mean temperature and temperature extremes. Body size responses to environmental change are thus complex and dynamic, driven by effects on growth as well as mortality.