The introduction of nitrite and nitrate to the relatively reduced environment of the early Earth provided impetus for a tremendous diversification of microbial pathways. However, little is known about the first organisms to produce these valuable resources. In this review, the latest microbial discoveries are integrated in the evolution of the nitrogen cycle according to the great ‘NO-ON’ time debate, as we call it. This debate hypothesizes the first oxidation of nitrogen as abiotic and anoxic (‘NO’) versus biological and aerobic (‘ON’). Confronting ancient biogeochemical niches with extant prokaryotic phylogenetics, physiology and morphology, pointed out that the well-described ammonia and nitrite oxidizing Proteobacteria likely did not play a pioneering role in microbial nitrogen oxidation. Instead, we hypothesize ancestral and primordial roles of methanotrophic NC10 bacteria and ammonia oxidizing archaea, respectively, for early nitrite production, and of anammox performing Planctomycetes followed by Nitrospira for early nitrate production. Additional genomic and structural information on the prokaryotic protagonists but also on their phages, together with the continued search for novel key players and processes, should further elucidate nitrogen cycle evolution. Through the ramifications between the biogeochemical cycles, this will improve our understanding on the evolution of terrestrial and perhaps extraterrestrial life.