We examined assemblages of trees and two major groups of vertebrate seed dispersers, birds and primates, in Ugandan protected areas to evaluate the roles of dispersal limitation and species sorting in community assembly. We conducted partial Mantel tests to investigate relationships between community similarity, environmental distance and geographic distance. Results showed that environmental factors, specifically temperature and rainfall, significantly and more strongly structured tree assemblages than geographic distance. Analysis of tree dispersal modes revealed wind-dispersed tree guilds were significantly dispersal limited but trees dispersed by animals were not. For assemblages of vertebrate seed dispersers, dispersal limitation significantly and more strongly structured assemblages of primates than species sorting whereas environmental factors significantly and more strongly structured assemblages of birds than dispersal limitation. We therefore examined whether trees dispersed by primates were more dispersal limited than trees dispersed by birds. We found consistent trends that primate fruit trees were more dispersal limited than bird fruit trees using three definitions of dispersal syndromes based on fruit color. Our results suggest that the dispersal abilities of primary consumers may affect the distribution of primary producers at large spatial scales.