Pain and symptoms of depression and anxiety have been observed to co-exist in the community-dwelling elderly. While depression and pain have been suggested to be predictive of one another temporally, the longitudinal associations between anxiety and pain remain undefined. The aim of this study was to investigate the reciprocal longitudinal associations of self-reported pain interference and affective symptoms, as measured by the Hospital Anxiety and Depression Scale, in community-dwelling older adults and report the potentially modifying effect of co-morbid anxiety or depression on these relationships. The study population were adults aged over 50-years, recruited previously to the North Staffordshire Osteoarthritis project (NorStOP), who had returned a health survey at both baseline and 3-year follow-up (n = 4234). Logistic regression was used to evaluate the pain–affect associations, with associations expressed as odds ratios with 95% confidence intervals (CI). Probable depression (odds ratio = 2.42; 95% CI 1.24, 4.69) and anxiety (2.30; 1.67, 3.17) at baseline predicted new-onset pain interference at 3-year follow-up. Conversely, pain interference at baseline was a risk factor for developing possible or probable depression (2.47; 1.96, 3.11) and anxiety (2.02; 1.60, 2.55) at 3-year follow-up. Adjusting for age, gender and co-morbid anxiety or depression slightly reduced the strength of the relationships, though most remained statistically significant. In conclusion, we have found evidence for both pain–depression and pain–anxiety relationships longitudinally, and in a reciprocal manner. Such findings have important implications for the future management of primary care patients presenting with symptoms of pain, anxiety or depression.