A time series of aerial censuses of Cape fur seal colonies, spanning four decades (1972–2009) and three countries (South Africa, Namibia, and Angola), was analyzed to assess spatio-temporal changes in population numbers. A weighted quantile regression approach was used to estimate trends in pup counts that were used as proxies for numbers of older animals at breeding colonies. There was a 74% increase in the number of breeding colonies over the study period, from 23 in 1973 to 40 in 2009. There was also a significant northward shift in the distribution of the breeding population. This was largely attributable to events in the northern part of the population's range coinciding with Namibia, where seal numbers declined at most colonies in the south of Namibia while several new breeding colonies developed in the northern part of Namibia and one in southern Angola. Despite range expansion and the development of new colonies, the overall size of the population in 2009 was similar to that of the early 1990s, according to the pup count models. Potential mechanisms for the observed changes, and their management implications, are discussed.