where *p*(*x*) denotes the probability that a patch is inhabited by exactly *x*(*x* = 0, 1, 2, ... , ) prey and *p*(*y* |* x*) the conditional probability that patches with *x* prey are occupied by *y*(*y* = 0, 1, 2, ... , ) predators. *f*(*x*,* y*) is the instantaneous predation rate of a predator individual staying in a patch occupied by *x* prey and *y* predators. *f*(*x, y*) depends on the size of a patch (which for simplicity is assumed to be the same for all patches), the number of prey (the behavioural functional response), the number of predators (the aggregative response), and the interaction between predators (mutual interference). The challenge is to find mathematical expressions to substitute *p*(*x*), *p*(*y* |* x*) and *f*(*x, y*) in eqn 2. These expressions should be simple enough to provide explicit convergent solutions to the infinite summations and at the same time be reasonably realistic.

#### aggregative response of the predator

The expected number of predators inhabiting a patch with *x* prey is assumed to increase with *x* according to the general expression

- ( eqn 3)

Five main types of aggregative response (see, e.g. Van der Meer & Ens 1997) can be produced by eqn 3 depending on its parameter values: (i) *c* = 0: the predators do not show any aggregative response; (ii) *c* > 0, *m* = 0, µ *=* 0: the aggregative response increases linearly with prey density; (iii) *c* > 0, *m* = 0, µ > 0: the response accelerates with prey density; (iv) *c* > 0, *m* = 0, µ < 0, the response increases with decelerating slope and approaches an upper asymptote: (v) *c* > 0, *m* = 1, µ < 0: the response is sigmoid. Type (ii), (iv) and (v) correspond to what Gascoigne & Lipcius (2004) classify as type I, II and III aggregative response, respectively. General solutions for ȳ_{x} are found by integration of eqn 3 (Appendix 1). The higher the value of *c*, the more will the predators tend to aggregate in patches with abundant prey, whereas ȳ_{x} will decrease in patches with few prey. As ȳ_{x} cannot be negative, it sets an upper limit to how large *c* can be (see Fig. 1).

To obtain explicit solutions to the aggregative response function, it is necessary to specify whether the prey distribution *p*(*x*) is clumped, random or even (Appendix 2). Furthermore, as it is most realistic that the aggregative response levels off at high prey densities, only cases in which µ is negative will be considered, encompassing convex (*m* = 0) and sigmoid aggregative responses (*m* = 1).

#### functional response to prey density

The behavioural functional response of a predator individual staying in a patch with *x* prey is assumed to be convex (type II). A model that describes such response is given by Ivlev (1961) as

- ( eqn 4)

where *f*_{m} is the maximal predation rate per individual, ψ a positive constant expressing the efficiency of the predators to find and attack prey and *A* the patch area. For simplicity, *A* is assumed to be the same for all patches (and equal to the mean patch area Ā).

#### combining the various predator responses

- ( eqn 6)

- ( eqn 6a)

if *p*(*y* | *x*) follows a NBD with parameters (¥_{x},κ), and to

- ( eqn 6b)

if *p*(*y* | *x*) follows a Poisson distribution with mean ¥_{x}. Finally, if the aggregative response has no prey density-independent component (i.e. = 0), all patches with *x* prey are expected to be inhabited by ¥_{x} predators, so and eqn 6 therefore becomes

- ( eqn 6c)

The three special cases of eqn 6 can be solved numerically, because the infinite sums converge as *x* . However, approximate analytical solutions are derived by making some simplifying assumptions. The terms (1 + (ȳ_{x}/κ)(1 −*e*^{−ɛ/A}))^{−(κ+1)} in eqn 6a, in eqn 6b, and *e*^{−ɛȳ}_{x}^{/A} in eqn 6c will be equal to 1 when ɛ = 0 and less than 1 when ɛ > 0. Hence, these terms represent the inhibiting effect of mutual interference on the predation rate. As long as ɛ/*A* is small, all three terms will be close to unity, but decrease with an increase in ȳ_{x}, which in turn increases with ȳ. Therefore, as an approximation, the overall inhibiting effect of mutual interference is assumed to depend on the overall density of predators (ȳ) rather than on ȳ_{x}. Equation 6 can therefore be replaced by

- ( eqn 7)

where *I* is a factor accounting for mutual interference (0 < *I* 1), calculated as *I*_{clumped} = (1 + (ȳ/κ)(1 −*e*^{−ɛ/A}))^{−(κ+1)}, and *I*_{even} = *e*^{−ɛȳ/A}, depending on whether the predators for a given value of *x* are clumped, randomly or evenly distributed, respectively. It appears that for ɛ > 0, *I*_{clumped} < *I*_{random} < *I*_{even} and that *I*_{clumped} declines with decreasing κ. This implies that scatter (prey density-independent variation) in the aggregative response will increase mutual interference and thereby reduce the overall predation rate.

ȳ_{x} in eqn 7 depends on prey distribution and on the shape of the aggregative response (Appendix 2). If the prey is either clumped or randomly distributed and the predators show a convex aggregative response, substitution of ȳ_{x} in eqn 7 by eqn A6 gives

- ( eqn 8a)

where the functions *Q*_{0}(·) and *Q*_{1}(·) depend on the density and distribution of the prey (Appendix 5).

If the predators show a sigmoid aggregative response, and the prey is either clumped or randomly distributed, ȳ_{x} is replaced by eqn A7, yielding

- ( eqn 8b)

If the predators show no aggregative response and prey distribution is either clumped or random, ȳ_{x} will be equal to ȳ for all *x*. Setting *c* = 0 in eqn 8b yields

- ( eqn 8c)

Finally, if the prey is evenly distributed, so that prey density is x̄/*A* in all patches, eqn 7 reduces to

- ( eqn 8d)