Lowland rice paddy soils may accumulate significant amounts of organic matter. Our aim was to investigate the role of prolonged paddy management on the nitrogen (N) status of the soils, and to elucidate the contribution of bacteria and fungi to long-term N accumulation processes. For this purpose, we sampled a chronosequence of 0–2000 years of rice cropping with adjacent non-paddy systems in the Bay of Hangzhou, China. The samples were analyzed for bulk density, total, mineral and microbial N (Nmic), and amino sugars as markers for microbial residues. The results showed that during the first 100 years of land embankment, both paddy and non-paddy soils accumulated N at a rate of up to 61 and 77 kg ha−1 per annum, reaching steady-state conditions after 110–172 years, respectively. Final N stocks in paddy fields exceeded those of the non-paddies by a factor of 1.3. The contribution of amino sugars to total N increased to a maximum of 34 g N kg−1 N in both land-use systems, highlighting a significant accumulation of N in microbial residues of the surface soils. Correspondingly, the ratio of Nmic to microbial residue-N decreased to a constant value. In the paddy subsoils, we found that bacterial residues particularly contributed to the pool of microbial residue-N. Nevertheless, the absolute contents of amino sugars in paddy subsoils decreased during the last 1700 years of the chronosequence. We conclude that under paddy cultivation, soil microorganisms may accumulate parts of this N in their residues despite low overall N availability. However, this N accumulation is limited to initial stages of paddy soil development and restricted to the surface horizons, thus challenging its sustainability with future land-use changes.