Additive genetic variance (*V*_{A}) and heritability (*h*^{2}, the ratio of genetic to phenotypic variance) are fundamental parameters in our understanding of the evolutionary potential and dynamics of traits in nature (Lande 1982; Houle 1992). Quantitative genetic models rely on the phenotypic similarities between relatives to estimate them (Falconer and Mackay 1996; Lynch and Walsh 1998). The application of “animal models,” a form of mixed-effects model in which *V*_{A} is estimated using a genetic relatedness matrix derived from a multi-generational pedigree (Lynch and Walsh 1998), in wild populations has advanced our understanding of evolutionary genetics in nature (Ellegren and Sheldon 2008; Kruuk et al. 2008). However, wild populations are characterized by high levels of environmental heterogeneity and relatives often share environments. It has been argued that the multi-generational approach of the “animal model” to estimating heritability reduces bias from environmental similarities because the model uses both phenotypic resemblance between close relatives and more distant relatives, who are less likely to live under similar environmental conditions (Postma and Charmantier 2007). Nonetheless, failing to properly account for such shared environmental effects is known to bias estimates of parameters derived from “animal models” (Kruuk and Hadfield 2007), and it has become common practice to account for certain kinds of shared environmental effects (e.g., parental identity, nest, group, or region of study area) by incorporating these into models as fixed or random effects (e.g., Kruuk et al. 2001; MacColl and Hatchwell 2003; Charmantier et al. 2004; Wilson et al. 2005; Kruuk and Hadfield 2007).

However, beyond these shared environment effects, social structure and natal philopatry—both of which are ubiquitous in wild vertebrates—are likely to result in spatial associations among relatives throughout individuals’ lives. Where relatives are associated in space throughout their lives, and the environment is spatially heterogeneous, it follows that relatives are more likely to experience similar fine-scale environmental effects than nonrelatives. Relatives will therefore show greater resemblance to one another. If related individuals share both genes and space, the potential exists for a positive correlation between genetic relatedness and similarity resulting from spatial effects. Although more challenging to incorporate within “animal models” than most shared environments currently considered in the wild animal literature, like all nongenetic causes of phenotypic similarity between relatives spatial similarities have clear potential to bias estimates of *V*_{A} and *h*^{2}, as well as other components of variance (Falconer and Mackay 1996). To date, the importance of spatial similarity in quantitative genetic studies of wild vertebrates has been largely dismissed. Here, we examine the effects of spatial autocorrelation (SAC) and home range overlap on phenotypic variation and their potential to bias estimates of *V*_{A} and *h*^{2} in a wild red deer population.

SAC is the dependence of a given variable's value on the values of the same variable measured at nearby locations (Cliff and Ord 1981; Fortin and Dale 2005). It has long been recognized as a source of bias in quantitative genetic analyses of plant agriculture and forestry studies (Cullis and Gleeson 1989, 1991; Burgueno et al. 2000; Costa e Silva et al. 2001), as well as more generally in ecology, both as a source of bias but also in identification of relevant and interesting spatial processes (Legendre 1993; Kissling and Carl 2008; Fortin and Dale 2009). In quantitative genetic analyses of agricultural and forestry trials, SAC can be accounted for to some extent by experimental design and appropriate fitting of block effects. However, particularly in forestry trials, substantial heterogeneity may exist within sites that can be further modeled by the inclusion of particular SAC functions (Dutkowski et al. 2002). Simulation studies have shown that variance component estimates in mixed-effects models were upwardly biased when positive SAC was not accounted for (Magnussen 1993), although other forestry studies have found that accounting for SAC can have differing effects on estimates of additive genetic variation (Costa e Silva et al. 2001; Dutkowski et al. 2002).

In studies of wild animals, the effect of SAC on estimates of quantitative genetic parameters has received little attention. The notable exception is a study of laying date and clutch size in a wild great tit population, which used parent–offspring regression to estimate *V*_{A} and *h*^{2} (van der Jeugd and McCleery 2002). Here, it was found that failure to account for SAC resulted in substantial overestimation (more than 60%) of heritability in laying date, but not in clutch size. Although suggesting that SAC can in some cases represent an important source of both phenotypic variation and bias in quantitative genetic analyses, this study did not apply particularly powerful or informative statistical techniques. Parent–offspring regression conflates parental environment and genetic effects; the “animal model” provides a much more powerful tool for accurately estimating *V*_{A} and separating environmental and genetic sources of variance (Lynch and Walsh 1998; Kruuk 2004). Furthermore, the study examined SAC effects by simply comparing parent–offspring regressions among groups of parents and offspring breeding at three different distances apart (van der Jeugd and McCleery 2002). In fact, as the forestry studies discussed above illustrate, autocorrelation functions can be simultaneously estimated and accounted for directly within mixed-effects models that also estimate *V*_{A} and from which *h*^{2} can therefore be calculated. To our knowledge, such an approach has yet to be applied to test the importance or nature of SAC underlying phenotypic variation, or its effects on parameter estimates from “animal models,” in any wild vertebrate system.

Implementation of SAC functions within mixed models requires individuals to be assigned specific spatial locations (e.g., average lifetime locations, locations of nest). However, most animals are mobile and home range sizes and shapes are likely to vary markedly between individuals. Methods for specifically assessing the importance of home range overlap effects on phenotypic variation and in estimating quantitative genetic parameters are therefore also desirable. In an “animal model,” a matrix of pairwise genetic relatedness coefficients (the “A matrix”) among individuals in a population is fitted within a mixed-effects model to estimate *V*_{A} as the phenotypic similarity among relatives (Henderson 1953, 1976). In animal breeding, it is relatively common practice to fit additional matrices to estimate dominance or epistatic genetic variance (e.g., Smith and Maki-Tanila 1990; Palucci et al. 2007). Multi-matrix approaches (fitting additional vectors of random effects with their associated covariance matrices) have recently been advocated to estimate and account for shared environment effects alongside genetic effects (Danchin et al. 2011), but have yet to be implemented empirically. Coefficients measuring the degree of home range overlap among individuals, or indeed any measure of social or spatial association, can readily be calculated if sufficient spatial or social interaction data are available. Dyadic home range overlap coefficients, assuming they scale similarly to the relatedness coefficients in the A matrix, can be built into a matrix incorporating all pairwise comparisons among individuals (which we term the “S matrix”) and fitted as a random effect alongside the A matrix (see methods). Such a “double matrix” approach would yield estimates of both *V*_{A} and the variance attributable to home range overlap among individuals and could provide important insight into shared environmental effects and reduce bias in estimates of heritability.