We study predictions for dark matter (DM) phase-space structure near the Sun based on high-resolution simulations of six galaxy haloes taken from the Aquarius project. The local DM density distribution is predicted to be remarkably smooth; the density at the Sun differs from the mean over a best-fitting ellipsoidal equidensity contour by less than 15 per cent at the 99.9 per cent confidence level. The local velocity distribution is also very smooth, but it differs systematically from a (multivariate) Gaussian distribution. This is not due to the presence of individual clumps or streams, but to broad features in the velocity modulus and energy distributions that are stable in both space and time and reflect the detailed assembly history of each halo. These features have a significant impact on the signals predicted for weakly interacting massive particle and axion searches. For example, weakly interacting massive particles recoil rates can deviate by ∼10 per cent from those expected from the best-fitting multivariate Gaussian models. The axion spectra in our simulations typically peak at lower frequencies than in the case of multivariate Gaussian velocity distributions. Also in this case, the spectra show significant imprints of the formation of the halo. This implies that once direct DM detection has become routine, features in the detector signal will allow us to study the DM assembly history of the Milky Way. A new field, ‘DM astronomy’, will then emerge.