A key task in ecology is to understand the drivers of animal distributions. In arid and semi-arid environments, this is challenging because animal populations show considerable spatial and temporal variation. An effective approach in such systems is to examine both broad-scale and long-term data. We used this approach to investigate the distribution of small mammal species in semi-arid ‘mallee’ vegetation in south-eastern Australia. First, we examined broad-scale data collected at 280 sites across the Murray Mallee region. We used generalized additive mixed models (GAMMs) to examine four hypotheses concerning factors that influence the distribution of individual mammal species at this scale: vegetation structure, floristic diversity, topography and recent rainfall. Second, we used long-term data from a single conservation reserve (surveyed from 1997 to 2012) to examine small mammal responses to rainfall over a period spanning a broad range of climatic conditions, including record high rainfall in 2011. Small mammal distributions were strongly associated with vegetation structure and rainfall patterns, but the relative importance of these drivers was species-specific. The distribution of the mallee ningaui Ningaui yvonneae, for example, was largely determined by the cover of hummock grass; whereas the occurrence of the western pygmy possum Cercartetus concinnus was most strongly associated with above-average rainfall. Further, the combination of both broad-scale and long-term data provided valuable insights. Bolam's mouse Pseudomys bolami was uncommon during the broad-scale survey, but long-term surveys showed that it responds positively to above-average rainfall. Conceptual models developed for small mammals in temperate and central arid Australia, respectively, were not, on their own, adequate to account for the distributional patterns of species in this semi-arid ecosystem. Species-specific variation in the relative importance of different drivers was more effectively explained by qualitative differences in life-history attributes among species.