Our understanding of mantle convection and the motion of plates depends intimately on our understanding of the viscosity structure of the mantle. While geoid and gravity observations have provided fundamental constraints on the radial viscosity structure of the mantle, the influence of short-wavelength variations in viscosity is still poorly understood. We present 2-D and 3-D finite-element models of mantle flow, including strong lateral viscosity variations and local sources of buoyancy, owing to both thermal and compositional effects. We first use generic 2-D models of a subduction zone to investigate how different observations depend on various aspects of the viscosity structure, in particular, the slab and lower-mantle viscosity and the presence of a low-viscosity region in the mantle wedge above the slab. We find that: (1) the strain rate provides a strong constraint on the absolute viscosity of the slab (1023 Pa s); (2) stress orientation within the slab is sensitive to the relative viscosity of the slab, lower mantle and the wedge; and (3) the stress state and topography of the overriding plate depend on the wedge viscosity and local sources of buoyancy. In particular, the state of stress in the overriding plate changes from compression to extension with the addition of a low-viscosity wedge. We then use observations of strain rate, stress orientation, dynamic topography and the geoid for the Tonga–Kermadec subduction zone as simultaneous constraints on the viscosity and buoyancy in a 3-D regional dynamic model. Together these observations are used to develop a self-consistent model of the viscosity and buoyancy by taking advantage of the sensitivity of each observation to different aspects of the dynamics, over a broad range of length-scales. The presence of a low-viscosity wedge makes it possible to match observations of shallow dynamic topography and horizontal extension within the backarc, and down-dip compression in the shallow portion of the slab. These results suggest that a low-viscosity wedge plays an important role in controlling the presence of backarc spreading. However, for a model with a low-viscosity and low-density region that provides a good fit to the observed topography, we find that a reduction of the slab density by a factor of 1.3 relative to the reference density model, is required to match the observed geoid. These results suggest that compensation of the slab by dynamic topography may be a much smaller effect at short to intermediate wavelengths than predicted by long-wavelength modelling of the geoid.