Large mammalian herbivores manifest a strong top-down control on ecosystems that can transform entire landscapes, but their impacts have not been reviewed in the context of terrestrial carbon storage. Here, we evaluate the effects of plant biomass consumption by large mammalian herbivores (>10 kg adult biomass), and the responses of ecosystems to these herbivores, on carbon stocks in temperate and tropical regions, and the Arctic. We calculate the difference in carbon stocks resulting from herbivore exclusion using the results of 108 studies from 52 vegetation types. Our estimates suggest that herbivores can reduce terrestrial above- and below-ground carbon stocks across vegetation types but reductions in carbon stocks may approach zero given sufficient periods of time for systems to respond to herbivory (i.e. decades). We estimate that if all large herbivores were removed from the vegetation types sampled in our review, increases in terrestrial carbon stocks would be up to three orders of magnitude less than many of the natural and human-influenced sources of carbon emissions. However, we lack estimates for the effects of herbivores on below-ground biomass and soil carbon levels in many regions, including those with high herbivore densities, and upwards revisions of our estimates may be necessary. Our results provide a starting point for a discussion on the magnitude of the effects of herbivory on the global carbon cycle, particularly given that large herbivores are common in many ecosystems. We suggest that herbivore removal might represent an important strategy towards increasing terrestrial carbon stocks at local and regional scales within specific vegetation types, since humans influence populations of most large mammals.