Accretion rates on to active galactic nuclei (AGNs) are likely to be extremely variable on short time-scales; much shorter than the typical cooling time of X-ray emitting gas in elliptical galaxies and galaxy clusters. Using the Langevin approach it is shown that, for a simple feedback system, this can induce variability in the AGN power output that is of much larger amplitude, and persists for longer time-scales, than the initial fluctuations. An implication of this is that rich galaxy clusters are expected to show the largest and longest-lived fluctuations. Stochastic variations in the accretion rate also mean that the AGN injects energy across a wide range of time-scales. This allows the AGN to maintain a much closer balance with its surroundings than if it was periodically activated. The possible non-linear correlation between Bondi accretion rate and jet power, found by Allen et al., can be explained if the instantaneous accretion rate, scaled by jet power, varies log-normally. This explanation also implies that the duty cycle of AGN activity increases with the radiative losses of the surroundings, in qualitative agreement with Best et al.