Dust particles, ubiquitous throughout the Universe, continuously evolve in processes closely entangled with the stellar life cycle. Dust nucleates in outflows of dying stars and is heavily processed in the journey through the interstellar medium, until it is finally subsumed in a next-generation star or its surrounding planetary system. Although the formation of silicates has been studied experimentally and theoretically for decades, the stardust nucleation process in the condensation zone of oxygen-rich stellar outflows still remains mysterious. These silicates are mostly ternary oxides consisting of O, Mg and Si, which cannot nucleate directly from gaseous monomers. Previous work has suggested that silicates form on nucleation seeds consisting of low-abundant elements or from addition of metals to SiO-nuclei. However, our extensive computational study of the thermodynamic properties of a large number of clusters shows that pure SiO nucleation is unfeasible, while heteromolecular nucleation of Mg, SiO and H2O is a plausible mechanism to form magnesium silicates under stellar outflow conditions.