Disturbance caused by large herbivores can affect the relative importance of ecological processes in determining community assembly and may cause a systematic loss of biodiversity across scales. To examine changes in the community assembly pattern caused by an overabundance of large herbivores in Japan, we analyzed community composition data from before and after the overabundance occurred. The community assembly pattern becomes more random after the deer overabundance. In addition, result of variation partitioning revealed decrease in importance of environmental processes and increase in importance of spatial processes. However, response of turnover rate, niche breadth, and niche overlap was heterogeneous, according to scale of each environmental gradient. Our results emphasize the importance of conserving habitat specialists that represent the local environment (habitat type and topography) at various altitudinal ranges to maintain biodiversity at regional scales under the increasing pressure of large herbivores.