Semiconductor billiards are often considered as ideal systems for studying dynamical chaos in the quantum mechanical limit. In the traditional picture, once the electron's mean free path, as determined by the mobility, becomes larger than the device, disorder is negligible and electron trajectories are shaped by specular reflection from the billiard walls alone. Experimental insight into the electron dynamics is normally obtained by magnetoconductance measurements. A number of recent experimental studies have shown these measurements to be largely independent of the billiard's exact shape, and highly dependent on sample-to-sample variations in disorder. In this paper, we discuss these more recent findings within the full historical context of work on semiconductor billiards, and offer strong evidence that small-angle scattering at the sub-100 nm length-scale dominates transport in these devices. This has important implications for the role these devices can play for experimental tests of ideas in quantum chaos.