The degree to which organisms retain their environmental preferences is of utmost importance in predicting their fate in a world of rapid climate change. Notably, marine invertebrates frequently show strong affinities for either carbonate or terrigenous clastic environments. This affinity is due to characteristics of the sediments as well as correlated environmental factors. We assessed the conservatism of substrate affinities of marine invertebrates over geological timescales, and found that niche conservatism is prevalent in the oceans, and largely determined by the strength of initial habitat preference. There is substantial variation in niche conservatism among major clades with corals and sponges being among the most conservative. Time-series analysis suggests that niche conservatism is enhanced during times of elevated nutrient flux, whereas niche evolution tends to occur after mass extinctions. Niche evolution is not necessarily elevated in genera exhibiting higher turnover in species composition.