Understanding the evolution of body size and sexual size dimorphism has been a longstanding goal in evolutionary biology. Previous work has shown that environmental stress can constrain male-biased sexual size dimorphism at the population level, but we know little about how this might translate to geographical patterns of body size and sexual size dimorphism at the species level. Environmental constraints due to a highly seasonal, resource-poor and/or variable environment have often been cited to explain the unusual lack of sexual size dimorphism among Madagascar's diverse and numerous primate taxa; however, empirical tests of this hypothesis are lacking. Using a phylogenetic approach and a geographical information system platform, we explored the role of seasonality, interannual variability and annual measures of temperature and rainfall, and net primary productivity on patterns of body size and sexual size dimorphism across 130 species of primates. Phylogenetically controlled comparisons showed no support for a role of environmental constraints in moderating sexual size dimorphism at the interspecific level, despite significant associations of environmental variables with body mass. Results suggest that the focus of discussions that have dominated in the last two decades regarding the role of environmental constraints in driving patterns of monomorphism of Madagascar's lemurs should be reconsidered; however, the conundrum remains.