Suppose that researchers have conducted a longitudinal study following two (or more) groups of subjects and measuring a continuous response (or dependent or outcome) variable over time. Their typical primary aims involve testing whether the mean of the response changes over time for each of the groups, as well as whether such change occurs differentially within groups. One common example of this occurs in the context of an intervention study, in which response variable trajectories over time for the intervention and control groups are compared.

The classical approach for addressing this issue is to conduct a repeated measures analysis of variance (ANOVA) fitting a full factorial fixed effects model containing terms for the main effects of group and time as well as their interaction. Testing for a differential change over time within groups can be addressed through the group-by-time interaction effect. This model may seem like a reasonable choice, but it is important for researchers to understand that this choice involves assumptions that may be inappropriate for the data. Specifically, it assumes that variances are constant (or homogeneous) over time, when they might change over time instead, and that correlations between two different response measurements are consistently the same, no matter how far apart they occur in time. This scenario is called compound symmetry (CS).

For longitudinal data, especially when collected over long time periods, it seems more natural for correlations to weaken as measurements occur further apart, which is reflected through autoregressive (AR) structures. Moreover, there are many other choices besides CS and AR for modeling correlations, together with either constant or heterogeneous variances. Furthermore, researchers should understand that the different choices for how the variances and correlations (or, equivalently, the covariance structure) are modeled can lead to critically different significance levels for hypothesis tests about means (e.g., Park, Park, & Davis, 2001 and the example given below) addressing their research aims. Linear mixed models (LMMs) can be used to generate these alternatives for the covariance structure, along with alternatives for the fixed effects (for modeling the mean of the response).

Choosing a standard covariance model like CS or AR subjectively can lead to erroneous conclusions. We provide below an example where that is precisely the case. We describe a more objective approach, in which model selection criteria are used to make such decisions. As an added benefit, this approach to model selection is separate from tests of the fixed effects, as used to address the specific aims of a study, and so we can avoid bias for those results by keeping them masked until the final choice of the covariance structure is made.

Use of model selection criteria to identify an appropriate covariance structure should be standard practice. In this article, we review alternative model selection procedures. We describe strategies for choosing the most appropriate model and then demonstrate their use with an example.