Chemical and microbial characterisations of particle-size fractions (PSFs) from a rice paddy soil subjected to long-term heavy metal pollution (P) and nonpolluted (NP) soil were performed to investigate whether the distribution of heavy metals (Cd, Cu, Pb and Zn) regulates microbial community activity, abundance and diversity at the microenvironment scale. The soils were physically fractionated into coarse sand, fine sand, silt and clay fractions. Long-term heavy metal pollution notably decreased soil basal respiration (a measurement of the total activity of the soil microbial community) and microbial biomass carbon (MBC) across the fractions by 3–45% and 21–53%, respectively. The coarse sand fraction was more affected by pollution than the clay fraction and displayed a significantly lower MBC content and respiration and dehydrogenase activity compared with the nonpolluted soils. The abundances and diversities of bacteria were less affected within the PSFs under pollution. However, significant decreases in the abundances and diversities of fungi were noted, which may have strongly contributed to the decrease in MBC. Sequencing of denaturing gradient gel electrophoresis bands revealed that the groups Acidobacteria, Ascomycota and Chytridiomycota were clearly inhibited under pollution. Our findings suggest that long-term heavy metal pollution decreased the microbial biomass, activity and diversity in PSFs, particularly in the large-size fractions.