The feedbacks between changes in atmospheric CO2 levels, climate, and CO2 drawdown into rocks are incompletely understood. In particular, the role of the upper oceanic crust in this long-term carbon cycling is debated. Here, a simple model for the precipitation of calcite in the upper oceanic crust is developed with the aim of understanding why Late Mesozoic upper oceanic crust contains several times higher CO2 concentrations (~2.5 wt%) than Cenozoic upper oceanic crust (~0.5 wt%). The modeling shows that neither heating of seawater, nor leaching of Ca from the rock with charge balance maintained by Mg uptake by the rock, can lead to >0.2 wt% CO2 uptake by the oceanic crust. Alkalinity production during fluid-rock reaction in the crust allows substantially more CO2 to be taken up by the crust in calcite, and is consistent with changes in the major element composition of Late Mesozoic upper oceanic crust due to hydrothermal alteration. The higher CO2 content of Late Mesozoic than Cenozoic upper oceanic crust thus requires greater alkalinity production by fluid-rock reactions in the Late Mesozoic. This may have been due to higher bottom water temperature and/or seawater having a different composition leading to different secondary minerals forming in the Late Mesozoic. Irrespective of the mechanism, the negative feedback on atmospheric CO2 levels provided by enhanced hydrothermal CO2 consumption in the Late Mesozoic was of similar magnitude to that from continental weathering.