If utterances are the observational unit of analysis and there are no sequential patterns to the interaction, two alternative statistical models may be applied. A hierarchical design in which utterances are nested within subjects and subjects are nested within treatment condition is considered. One method of analysis pools utterances within treatment condition; the other method collapses across utterances to obtain subject means. Inappropriate application of the pooling model instead of the subject means model can lead to Type I errors, decreased generalizability, and inflated variance estimates that attenuate univariate and multivariate correlations. The effect of utterance reliability on the apparent unidimensionality and parallelism of multiple indicators is also discussed.