A central focus of ecology and biogeography is to determine the factors that govern spatial variation in biodiversity. Here, we examined patterns of ant diversity along climatic gradients in three temperate montane systems: Great Smoky Mountains National Park (USA), Chiricahua Mountains (USA), and Vorarlberg (Austria). To identify the factors which potentially shape these elevational diversity gradients, we analyzed patterns of community phylogenetic structure (i.e. the evolutionary relationships among species coexisting in local communities). We found that species at low-elevation sites tended to be evenly dispersed across phylogeny, suggesting that these communities are structured by interspecific competition. In contrast, species occurring at high-elevation sites tended to be more closely related than expected by chance, implying that these communities are structured primarily by environmental filtering caused by low temperatures. Taken together, the results of our study highlight the potential role of niche constraints, environmental temperature, and competition in shaping broad-scale diversity gradients. We conclude that phylogenetic structure indeed accounts for some variation in species density, yet it does not entirely explain why temperature and species density are correlated.