The identification of societal vulnerable counties and regions and the factors contributing to social vulnerability are crucial for effective disaster risk management. Significant advances have been made in the study of social vulnerability over the past two decades, but we still know little regarding China's societal vulnerability profiles, especially at the county level. This study investigates the county-level spatial and temporal patterns in social vulnerability in China from 1980 to 2010. Based on China's four most recent population censuses of 2,361 counties and their corresponding socioeconomic data, a social vulnerability index for each county was created using factor analysis. Exploratory spatial data analysis, including global and local autocorrelations, was applied to reveal the spatial patterns of county-level social vulnerability. The results demonstrate that the dynamic characteristics of China's county-level social vulnerability are notably distinct, and the dominant contributors to societal vulnerability for all of the years studied were rural character, development (urbanization), and economic status. The spatial clustering patterns of social vulnerability to natural disasters in China exhibited a gathering–scattering–gathering pattern over time. Further investigations indicate that many counties in the eastern coastal area of China are experiencing a detectable increase in social vulnerability, whereas the societal vulnerability of many counties in the western and northern areas of China has significantly decreased over the past three decades. These findings will provide policymakers with a sound scientific basis for disaster prevention and mitigation decisions.