Nitrogen (N) leaching to surface waters from grain farms in the Mississippi River Basin (MRB), USA, is the primary cause of hypoxia in the Gulf of Mexico. Regional-scale N mass balances indicate that a small, intensively cropped area of the upper MRB contributes disproportionately to nitrate loading. These aggregate balances miss small-scale variability, especially that caused by differences in farm management. We constructed N mass balances for a gradient of farm types, from corn–soybean monocultures to diversified grain farms that rely on biological N fixation (BNF) as a primary N source, to compare the relative efficiency of diverse farming systems in the MRB. Five-year N balances were calculated for a most and least productive field on each farm using data collected from interviews with 95 grain farmers in Iowa, Ohio, Minnesota, and Wisconsin; from legume biomass and corn grain samples collected from a subset of farms; and published values from the literature. Nitrogen balances ranged from high average annual surpluses (149 kg N·ha−1·yr−1) to large deficits (80 kg N·ha−1·yr−1), and differed based on N source and crop rotation. Fields with >50% of total N additions from legume N sources and fields with complex crop rotations that included both annual and perennial species were approximately in balance (3.7 kg N·ha−1·yr−1 and 5.7 kg N·ha−1·yr−1, respectively) compared to fertilizer-based practices in corn–soybean rotations with average annual surpluses near 35 kg N·ha−1·yr−1. Surplus N was also inversely related to the proportion of total N inputs from BNF for medium (80–160 kg N·ha−1·yr−1) to high (>160 kg N·ha−1·yr−1) N rates. Diversified farmers were more likely to adjust their management practices in response to environmental variability compared to fertilizer-based farmers. Taken together, results from this study suggest that significantly reducing surplus N in agroecosystems will require reducing N inputs and increasing C availability to support the internal biological mechanisms for storing N in farm fields.