Deposition of a massive (104–105 M⊙) giant molecular cloud (GMC) into the inner parsec of the Galaxy is widely believed to explain the origin of over a hundred unusually massive young stars born there ∼6 Myr ago. An unknown fraction of that gas could have been accreted by Sgr A*, the supermassive black hole of the Milky Way. It has been recently suggested that two observed γ-ray-emitting bubbles emanating from the very centre of our Galaxy were inflated by this putative activity of Sgr A*. We run a suite of numerical simulations to test whether the observed morphology of the bubbles could be due to the collimation of a wide-angle outflow from Sgr A* by the disc-like Central Molecular Zone (CMZ), a well-known massive repository of molecular gas in the central ∼200 pc. We find that an Eddington-limited outburst of Sgr A* lasting ≃1 Myr is required to reproduce the morphology of the Fermi bubbles, suggesting that the GMC mass was ∼105 M⊙ and it was mainly accreted by Sgr A* rather than used to make stars. We also find that the outflow from Sgr A* enforces strong angular momentum mixing in the CMZ disc, robustly sculpting it into a much narrower structure – a ring – perhaps synonymous with the recently reported ‘Herschel ring’. In addition, we find that Sgr A* outflow is likely to have induced formation of massive star-forming GMCs in the CMZ. In this scenario, the Arches and Quintuplet clusters, the two observed young star clusters in the central tens of parsecs of the Galaxy, and also GMCs such as Sgr B2, owe their existence to the recent Sgr A* activity.