Mycorrhizal association is a common characteristic in a majority of land plants, and the survival and distribution of a species can depend on the distribution of suitable fungi in its habitat. Orchidaceae is one of the most species-rich angiosperm families, and all orchids are fully dependent on fungi for their seed germination and some also for subsequent growth and survival. Given this obligate dependence, at least in the early growth stages, elucidating the patterns of orchid–mycorrhizal relationships is critical to orchid biology, ecology and conservation. To assess whether rarity of an orchid is determined by its specificity towards its fungal hosts, we studied the spatial and temporal variability in the host fungi associated with one of the rarest North American terrestrial orchids, Piperia yadonii. The fungal internal transcribed spacer region was amplified and sequenced by sampling roots from eight populations of P. yadonii distributed across two habitats, Pinus radiata forest and maritime chaparral, in California. Across populations and sampling years, 26 operational taxonomic units representing three fungal families, the Ceratobasidiaceae, Sebacinaceae and Tulasnellaceae, were identified. Fungi belonging to the Sebacinaceae were documented in orchid roots only at P. radiata forest sites, while those from the Ceratobasidiaceae and Tulasnellaceae occurred in both habitats. Our results indicate that orchid rarity can be unrelated to the breadth of mycorrhizal associations. Our data also show that the dominance of various fungal families in mycorrhizal plants can be influenced by habitat preferences of mycorrhizal partners.