We investigate the effects of halo shape and its alignment with larger scale structure on the galaxy correlation function. We base our analysis on the galaxy formation models of Guo et al., run on the Millennium Simulations. We quantify the importance of these effects by randomizing the angular positions of satellite galaxies within haloes, either coherently or individually, while keeping the distance to their respective central galaxies fixed. We find that the effect of disrupting the alignment with larger scale structure is a ∼2 per cent decrease in the galaxy correlation function around r ≈ 1.8 h−1 Mpc. We find that sphericalizing the ellipsoidal distributions of galaxies within haloes decreases the correlation function by up to 20 per cent for and increases it slightly at somewhat larger radii. Similar results apply to power spectra and redshift-space correlation functions. Models based on the halo occupation distribution, which places galaxies spherically within haloes according to a mean radial profile, will therefore significantly underestimate the clustering on sub-Mpc scales. In addition, we find that halo assembly bias, in particular the dependence of clustering on halo shape, propagates to the clustering of galaxies. We predict that this aspect of assembly bias should be observable through the use of extensive group catalogues.