We model the interaction between the wind from a newly formed rapidly rotating magnetar and the surrounding supernova shock and host star. The dynamics is modelled using the two-dimensional, axisymmetric thin-shell equations. In the first ∼10–100 s after core-collapse the magnetar inflates a bubble of plasma and magnetic fields behind the supernova shock. The bubble expands asymmetrically because of the pinching effect of the toroidal magnetic field, even if the host star is spherically symmetric, just as in the analogous problem of the evolution of pulsar wind nebulae. The degree of asymmetry depends on Emag/Etot, the ratio of the magnetic energy to the total energy in the bubble. The correct value of Emag/Etot is uncertain because of uncertainties in the conversion of magnetic energy into kinetic energy at large radii in relativistic winds; we argue, however, that bubbles inflated by newly formed magnetars are likely to be significantly more magnetized than their pulsar counterparts. We show that for a ratio of magnetic to total power supplied by the central magnetar the bubble expands relatively spherically. For , however, most of the pressure in the bubble is exerted close to the rotation axis, driving a collimated outflow out through the host star. This can account for the collimation inferred from observations of long-duration gamma-ray bursts (GRBs). Outflows from magnetars become increasingly magnetically dominated at late times, due to the decrease in neutrino-driven mass loss as the young neutron star cools. We thus suggest that the magnetar-driven bubble initially expands relatively spherically, enhancing the energy of the associated supernova, while at late times it becomes progressively more collimated, producing the GRB. The same physical processes may operate in more modestly rotating neutron stars to produce asymmetric supernovae and lower energy transients such as X-ray flashes.