Advanced stage at diagnosis contributes to low breast cancer survival rates in sub-Saharan Africa. Living far from health services is known to delay presentation, but the effect of residential distance to hospital, the radius at which this effect sets in and the women most affected have not been quantified. In a periurban South African setting, we examined the effect of a geographic information system (GIS)-measured straight-line distance, from a patient's residence to diagnostic hospital, on stage at diagnosis in 1,071 public-sector breast cancer patients diagnosed during 2006–2012. Generalized linear models were used to estimate risk ratios for late stage (stage III/IV vs. stage I/II) associated with distance, adjusting for year of diagnosis, age, race and socioeconomic indicators. Mean age of patients was 55 years, 90% were black African and diagnoses were at stages I (5%), II (41%), III (46%) and IV (8%). Sixty-two percent of patients with distances >20 km (n = 338) had a late stage at diagnosis compared to 50% with distances <20 km (n = 713, p = 0.02). Risk of late stage at diagnosis was 1.25-fold higher (95% CI: 1.09, 1.42) per 30 km. Effects were pronounced in an underrepresented group of patients over age 70. This positive stage–distance association held to 40 km, and plateaued or slightly reversed in patients (9%) living beyond this distance. Studies of woman and the societal and healthcare-level influences on these delays and on the late stage at diagnosis distribution are needed to inform interventions to improve diagnostic stage and breast cancer survival in this and similar settings.