The temporal sequence of microbial establishment in the rumen of the neonatal ruminant has important ecological and pathophysiological implications. In this study, we characterized the rumen microbiota of pre-ruminant calves fed milk replacer using two approaches, pyrosequencing of hypervariable V3–V5 regions of the 16S rRNA gene and whole-genome shotgun approach. Fifteen bacterial phyla were identified in the microbiota of pre-ruminant calves. Bacteroidetes was the predominant phylum in the rumen microbiota of 42-day-old calves, representing 74.8% of the 16S sequences, followed by Firmicutes (12.0%), Proteobacteria (10.4%), Verrucomicrobia (1.2%) and Synergistetes (1.1%). However, the phylum-level composition of 14-day-old calves was distinctly different. A total of 170 bacterial genera were identified while the core microbiome of pre-ruminant calves included 45 genera. Rumen development seemingly had a significant impact on microbial diversity. The dazzling functional diversity of the rumen microbiota was reflected by identification of 8298 Pfam and 3670 COG protein families. The rumen microbiota of pre-ruminant calves displayed a considerable compositional heterogeneity during early development. This is evidenced by a profound difference in rumen microbial composition between the two age groups. However, all functional classes between the two age groups had a remarkably similar assignment, suggesting that rumen microbial communities of pre-ruminant calves maintained a stable function and metabolic potentials while their phylogenetic composition fluctuated greatly. The presence of all major types of rumen microorganisms suggests that the rumen of pre-ruminant calves may not be rudimentary. Our results provide insight into rumen microbiota dynamics and will facilitate efforts in formulating optimal early-weaning strategies.