With properly timed sequences of ultrafast electron pulses, it is now possible to image complex molecular structures in the four dimensions of space and time with resolutions of 0.01 Å and 1 ps, respectively. The new limits of ultrafast electron diffraction (UED) provide the means for the determination of transient molecular structures, including reactive intermediates and non-equilibrium structures of complex energy landscapes. By freezing structures on the ultrafast timescale, we are able to develop concepts that correlate structure with dynamics. Examples include structure-driven radiationless processes, dynamics-driven reaction stereochemistry, pseudorotary transition-state structures, and non-equilibrium structures exhibiting negative temperature, bifurcation, or selective energy localization in bonds. These successes in the studies of complex molecular systems, even without heavy atoms, and the recent development of a new machine devoted to structures in the condensed phase, establish UED as a powerful method for mapping out temporally changing molecular structures in chemistry, and potentially, in biology. This review highlights the advances made at Caltech, with emphasis on the principles of UED, its evolution through four generations of instrumentation (UED-1 to UED-4) and its diverse applications.