Strong barriers to genetic exchange can exist at divergently selected loci, whereas alleles at neutral loci flow more readily between populations, thus impeding divergence and speciation in the face of gene flow. However, ‘divergence hitchhiking’ theory posits that divergent selection can generate large regions of differentiation around selected loci. ‘Genome hitchhiking’ theory suggests that selection can also cause reductions in average genome-wide rates of gene flow, resulting in widespread genomic divergence (rather than divergence only around specific selected loci). Spatial heterogeneity is ubiquitous in nature, yet previous models of genetic barriers to gene flow have explored limited combinations of spatial and selective scenarios. Using simulations of secondary contact of populations, we explore barriers to gene flow in various selective and spatial contexts in continuous, two-dimensional, spatially explicit environments. In general, the effects of hitchhiking are strongest in environments with regular spatial patterning of starkly divergent habitat types. When divergent selection is very strong, the absence of intermediate habitat types increases the effects of hitchhiking. However, when selection is moderate or weak, regular (vs. random) spatial arrangement of habitat types becomes more important than the presence of intermediate habitats per se. We also document counterintuitive processes arising from the stochastic interplay between selection, gene flow and drift. Our results indicate that generalization of results from two-deme models requires caution and increase understanding of the genomic and geographic basis of population divergence.