We study relations between stellar mass, star formation and gas-phase metallicity in a sample of 177 071 unique emission line galaxies from the Sloan Digital Sky Survey Data Release 7, as well as in a sample of 43 767 star-forming galaxies at z= 0 from the cosmological semi-analytic model l-galaxies. We demonstrate that metallicity is dependent on star formation rate at fixed mass, but that the trend is opposite for low and for high stellar mass galaxies. Low-mass galaxies that are actively forming stars are more metal poor than quiescent low-mass galaxies. High-mass galaxies, on the other hand, have lower gas-phase metallicities if their star formation rates are small. Remarkably, the same trends are found for our sample of model galaxies. By examining the evolution of the stellar component, gas and metals as a function of time in these galaxies, we gain some insight into the physical processes that may be responsible for these trends. We find that massive galaxies with low gas-phase metallicities have undergone a gas-rich merger in the past, inducing a starburst which exhausted their cold gas reservoirs and shutdown star formation. Thereafter, these galaxies were able to accrete metal-poor gas, but this gas remained at too low a density to form stars efficiently. This led to a gradual dilution in the gas-phase metallicities of these systems over time. These model galaxies are predicted to have lower-than-average gas-to-stellar mass ratios and higher-than-average central black hole masses. We use our observational sample to confirm that real massive galaxies with low gas-phase metallicities also have very massive black holes. We propose that accretion may therefore play a significant role in regulating the gas-phase metallicities of present-day massive galaxies.