Most galaxies follow well-defined scaling relations of metallicity (O/H), star formation rate (SFR) and stellar mass (Mstar). However, low-metallicity starbursts, rare in the Local Universe but more common at high redshift, deviate significantly from these scaling relations. On the ‘main sequence’ of star formation, these galaxies have high SFR for a given Mstar; and on the mass–metallicity relation, they have excess Mstar for their low metallicity. In this paper, we characterize O/H, Mstar and SFR for these deviant ‘low-metallicity starbursts’, selected from a sample of ∼1100 galaxies, spanning almost two orders of magnitude in metal abundance, a factor of ∼106 in SFR, and of ∼105 in stellar mass. Our sample includes quiescent star-forming galaxies and blue compact dwarfs at redshift 0, luminous compact galaxies at redshift 0.3, and Lyman break galaxies at redshifts 1–3.4. Applying a principal component analysis (PCA) to the galaxies in our sample with Mstar ≤ 3 × 1010 M⊙ gives a Fundamental Plane (FP) of scaling relations; SFR and stellar mass define the plane itself, and O/H its thickness. The dispersion for our sample in the edge-on view of the plane is 0.17 dex, independently of redshift and including the metal-poor starbursts. The same FP is followed by 55 100 galaxies selected from the Sloan Digital Sky Survey, with a dispersion of 0.06 dex. In a companion paper, we develop multi-phase chemical evolution models that successfully predict the observed scaling relations and the FP; the deviations from the main scaling relations are caused by a different (starburst or ‘active’) mode of star formation. These scaling relations do not truly evolve, but rather are defined by the different galaxy populations dominant at different cosmological epochs.