Spousal concordance and reliability of the ‘Prudence Score’ as a summary of diet and lifestyle
Dr Sanjoti Parekh, School of Population Health, The University of Queensland, Herston, Queensland 4006. Fax: (07) 3365 5442; e-mail: email@example.com
Objectives: This paper describes a composite ‘Prudence Score’ summarising self-reported behavioural risk factors for non-communicable diseases. If proved robust, the ‘Prudence score’ might be used widely to encourage large numbers of individuals to adopt and maintain simple, healthy changes in their lifestyle.
Methods: We calculated the ‘Prudence Score’ based on responses collected in late 2006 to a postal questionnaire sent to 225 adult patients aged 25 to 75 years identified from the records of two general medical practices in Brisbane, Australia. Participants completed the behavioural, dietary and lifestyle items in relation to their spouse as well as themselves. The spouse or partner of each addressee completed their own copy of the study questionnaire.
Results: Kappa scores for spousal concordance with probands' reports (n = 45 pairs) on diet-related items varied between 0.35 (for vegetable intake) to 0.77 (for usual type of milk consumed). Spousal concordance values for other behaviours were 0.67 (physical activity), 0.82 (alcohol intake) and 1.0 (smoking habits). Kappa scores for test-retest reliability (n = 53) varied between 0.47 (vegetable intake) and 0.98 (smoking habits).
Conclusion: The veracity of self-reported data is a challenge for studies of behavioural change. Our results indicate moderate to substantial agreement from life partners regarding individuals' self-reports for most of the behavioural risk items included in the ‘Prudence Score’. This increases confidence that key aspects of diet and lifestyle can be assessed by self-report.
Implications: The ‘Prudence Score’ potentially has wide application as a simple and robust tool for health promotion programs.
Due to a combination of falling incidence and better survival from acute illnesses, as well as the rapid ageing of populations, many countries face a growing burden of chronic, non-communicable diseases (NCDs), at least in terms of absolute caseload if not prevalence and incidence.1 There is considerable overlap between the risk factors for the common NCDs and many are associated with particular aspects of diet and lifestyle.2 Working with a cohort of 12,203 elderly men in Perth, Western Australia, Spencer et al. showed that a simple additive score summarising eight aspects of habitual personal behaviour had significant predictive ability for mortality from all causes over five years.3 Interestingly, the relationship between total score and mortality was very close to linear, with each additional ‘prudent’ behaviour being associated with a reduction in mortality of 0.62% over the period of follow-up. Spencer et al. also observed that the initial distribution of Prudence Scores were close to Gaussian, indicating that most individuals already habitually followed some lifestyle behaviours that mitigated their risk of developing NCDs.
We are seeking to extend this work by determining whether advice from a general practitioner (GP) can prompt individuals to adopt additional healthy habits of living and increase their Prudence Score. Ultimately, it will be important also to demonstrate that such increases are sustained and that they are followed by reductions in morbidity and mortality from NCDs. Meanwhile, however, we are faced with the challenge of demonstrating that self-reports, such as claims that an individual rarely or never adds salt to food after it is cooked, are valid and reliable. A potential strength of using the Prudence Score to promote health is that it is based on actual behaviours that tend to be habitual and easily described, rather than on biomedical risk factors such as blood cholesterol, which require technical assessment. On the other hand, objective measures for many aspects of diet and lifestyle either do not exist or are too complex to apply to large samples of the population.
This paper reports the findings of a preliminary study designed to assess validity of self-reports of components of the ‘Prudence Score’ by reference to corroborating evidence provided by each participating individual's spouse or life-partner. We also assessed test-retest reliability of the Prudence Score questionnaire.
We identified 227 potentially eligible adult patients aged 25 to 75 years from the consultation records of two general practices in Brisbane, Australia, during the period January-June 2006. The treating general practitioner (GP) was asked to vet the list of their patients for exclusion criteria – active cancer, major mental illness, cognitive impairment or recent bereavement – resulting in two subjects being excluded.
In August 2006, we posted reply-paid baseline questionnaires (described below) to participants, each accompanied by a covering letter on the relevant practice letterhead and signed electronically by their GP. Collection of baseline data closed in October 2006, after two cycles of reply-paid reminder packages had been sent to non-respondents at intervals of three weeks. We did not provide any feedback on completed questionnaires or attempt to prompt changes in lifestyle or preventive activities.
The survey protocol was repeated in February/March 2007 to assess reliability of participants' responses. Brisbane has a sub-tropical climate with limited seasonal variation in temperature or the availability of particular foods, and no daylight saving. In the absence of an intervention, there is likely to be very little change in dietary behaviour or patterns of physical activity over an interval of five months.
The baseline questionnaire carried an identification number only and consisted of 28 items based on guidelines for healthy behaviour and lifestyle promulgated by leading health authorities in Australia such as the National Health and Medical Research Council (NHMRC), the National Heart Foundation and the Queensland Cancer Fund.4–10 As may be seen from Table 1, the questionnaire sought information on the usual frequency of consumption of meat, fish, fruit, vegetables and salt, on the usual type of milk used and usual spread used on bread. Other items addressed smoking status and history of smoking, usual pattern of alcohol consumption (number of ‘standard’ (10g alcohol) drinks consumed on each day of the week), frequency of moderate and vigorous exercise, and average number of hours spent sitting each day. Respondents were asked to report their height and weight (from which we calculated body mass index (BMI = weight in kg/square of height in metres), and their adherence to recommendations regarding tetanus immunisation, protection against the sun (hat, shirt, sunscreen) and, for women, screening for breast and cervical cancer.11–14 All participants were asked to indicate the extent of any smoke-free policies in their private homes (see Table 1).
Table 1. Frequency of prudent behaviours and other protective behaviours for 102 participants from Brisbane, January to June 2006.
|Fish||At least 2 serves/week||NHF b4||60||58.8|
|Milk||Low Fat or Fat Free||NHF4||65||63.7|
|Salt||No added salt||NHF5||53||52.0|
|Vegetables||At least 5 serves/day||NHMRC6||15||14.7|
|Fruit||At least 2 serves/day||NHMRC6||44||43.1|
|Meat||Maximum 4 serves/week||NHMRCa6||69||67.6|
|Physical Activity||150 min per week||ACSM/AHAc7||45||44.6|
|Alcohol||Women<20 g, men<40 g per day with one alcohol free day||NHMRC8||72||70.6|
|Body weight||BMI <25 kg/m2||NIH10||41||40.2|
|Tetanus Immunisation||Booster every 10 years||Australian Immunisation Handbook1||52||51.0|
|Use of Sunscreen||Always between 10am and 3pm for >10 min in the sun||Cancer Council Australia11||41||41.2|
|Use of Hat||Always between 10am and 3pm for >10 min in the sun||Cancer Council Australia11||48||47.1|
|Use of a Shirt||Always between 10am and 3pm for >10 min in the sun||Cancer Council Australia11||20||19.6|
|Cervical Cytology (n = 79)||Every two years after becoming sexually active||Australian Department of Health and Aging12||58||73.0|
|Mammography (n = 38)||Every two years after the age of 50 years||Australian Department of Health and Aging13||29||76.0|
|Passive Smoking||No smoking inside the house||The Cancer Council Australia14||80||78.4|
In both the baseline and reliability phases of the preliminary study, each questionnaire also asked the individual to provide all of the information listed above for their spouse or partner. Furthermore, each study package included a separate covering letter, questionnaire and reply-paid envelope for the partner to report his or her own profile directly and independently. The partner also provided data on the lifestyle and behaviour of the original addressee. Completed questionnaires for spouse- or partner-pairs could be linked by reference to the numerical code.
After double-entry into an electronic database, we used SPSS version 15 (SPSS Inc, 2006) and Stata version 9 (Stata Corporation, 2005) software to analyse the data, beginning with frequency tables for each item in the questionnaire. Having summarised patterns of physical activity as numbers of minutes of exercise each week, counting minutes of vigorous exercise as double, we calculated the Prudence Score for each participant with data on all of the 10 components. The Prudence Score is the simple sum of the number of items in the first part of Table 1 for which the individual apparently met the relevant guideline. Intake of fruit and vegetables were scored as two separate items. BMI was not included in the Prudence Score. Summary statistics for the distribution of Prudence Scores were calculated using data only from the 102 original addressees who responded because within-household correlations could reduce the evident degree of variation within the population.
We assessed spousal-concordance using the kappa statistic and corresponding 95% confidence intervals to summarise the extent of agreement between reports from each individual and how the relevant spouse or partner described that person's behaviour (see Table 2). We used the same statistical approach to assess reliability, that is, the extent of agreement between reports from a given individual in the baseline and second surveys. Table 3 shows the standard qualitative descriptors assigned to particular ranges of kappa scores.15 We have summarised concordance of Prudence Scores between spouse-pairs and between test-retest responses using intraclass correlation co-efficient.
Table 2. Spousal-concordance and reliability of self-reports of prudent behaviours and other protective behaviours for study participants from Brisbane, 2006.
|Use of Sunscreen||0.56||0.42-0.70||0.65||0.50-0.77|
|Use of Hat||0.46||0.26-0.61||0.69||0.66-0.71|
|Use of a Shirt||0.37||0.18-0.57||0.60||0.44-0.66|
|Cervical Cytology (n = 79)||0.72||0.52-0.92||0.76||0.71-0.84|
|Mammography (n = 38)||0.86||0.60-1.0||0.90||0.90-0.96|
Table 3. Standard descriptors for ranges of the kappa statistic.a
The Human Research Ethics Committee of the University of Queensland approved the protocol for the study. All covering letters accompanying questionnaires included contact details for the research team so participants in the study could raise specific questions or exercise their right to withdraw.
Figure 1 summarises the study. Of 225 baseline questionnaires distributed, responses were received from 113 (52%) of the 219 patients for whom apparently valid addresses were available. The mean age of respondents was 49 years (standard deviation = 13.2 years) and 37% of participants were male. The baseline ‘Prudence Score’ was calculated for 102 individuals who were original invitees and between-partner concordance for 45 couples. The repeat survey yielded 53 questionnaires for assessment of reliability.
The Prudence Scores at baseline varied from 1 to 10, with a median of 6 (inter-quartile range (IQR): 4-7). The proportions of participants complying with individual guidelines are provided in Table 1 and are generally comparable with population-wide data published by a number of Australian agencies.16–20 For example, the National Health Survey conducted in 2004/0519 found the proportion of non-smokers (never- and ex-smokers combined) to be 77%, similar to the crude proportion of 78.5% in this study. Participation in mammography by 76% of women is also typical of the population-wide figure.16 The mean BMI was 27.2 kg/m2, with 55.4% of participants in the ‘overweight’ or ‘obese’ ranges, which is again consistent with 54% from the National Health Survey 2004/05.19 Both the individual prevalences and the overall Prudence Scores indicate that there is ample scope to improve habitual behaviours.
Table 2 provides kappa statistics for assessments of agreement between self and spouse or partner reports for 18 different aspects of lifestyle or preventive behaviours. Applying the standard descriptors for kappa (see Table 3), the majority (n=16) were at the level of ‘moderate’ or better, with six of these indicating ‘substantial’ agreement and four ‘almost perfect’ agreement. Only two comparisons (for consumption of vegetables and use of a shirt while in the sun) were at the level of ‘fair’ agreement, and none indicated ‘slight’ or ‘poor’ agreement. We also calculated separate series of kappa statistics for female participants reporting for male partners and for male participants reporting for female partners. The confidence intervals for all of the sex-specific analyses overlapped, pointing to an absence of systematic differences in sex-specific agreement (data not shown). The intraclass correlation coefficient for the Prudence Score calculated for the self-report and the spousal report was 0.67 (95% confidence interval (CI): 0.40-0.96). The median Prudence Scores for individuals with a corresponding spouse report as well as individuals without a paired response was 6 (IQR 4-7 for both subsets), suggesting very limited, if any, selection bias affecting the subset of cases included in the assessment of concordance.
Of the 102 original addressees who completed and returned a questionnaire at baseline, 53 also responded at the three-month follow-up. For 11 of the 18 items, kappa statistics for test-retest reliability indicated ‘substantial’ or higher agreement, including ‘almost perfect’ agreement for smoking status and participation in mammography, while seven comparisons indicated ‘moderate’ agreement only. The intraclass correlation coefficient for initial and repeat Prudence Scores was 0.72 (95% CI: 0.45-0.97).
This study has demonstrated that it is possible to collect information on multiple aspects of lifestyle and preventive behaviours via an unannounced postal survey within Australian general practices. Importantly, as judged by formal tests of agreement between relevant paired responses, self-reports regarding these risk factors and participation in preventive programs were generally both valid and reliable. We have also demonstrated that a simple summary score of behavioural risk factors for NCDs that was originally developed in a cohort consisting exclusively of older men3 has similar statistical properties when calculated on the basis of responses from a middle-aged sample of the population in which women outnumbered men by almost 2:1. The profile of scores indicates that most of the population is following at least some habits that the epidemiological evidence would deem ‘prudent’, but that the majority also have a number of ways in which they might reduce their risk of NCDs still further.
There is a substantial literature in the psychosocial field in which information from spouses or partners has been used to validate self-reports of behaviour. However, the instances in which this approach has been applied to self-reports of smoking status, physical activity, sun protection behaviours, height and weight and drinking and dietary habits are far fewer.21–27 For studies of large-scale interventions directed towards whole communities, measurement of blood pressure, let alone urinary electrolytes, are effectively impractical as core methods of evaluation. They are also subject to significant issues of quality control, especially when changes within individuals are likely to be quite modest. On a population basis, however, small changes distributed widely are likely to exert substantial influences on the incidence of NCDs.28–31 For individuals, messages regarding primary prevention must focus on behaviour. Since much of any change in relevant behaviours is likely to take place independently of formal health services, self-reports of behaviour will remain key items of data. This study increases the confidence in using self-reports to assess impacts of community-based risk reduction interventions.
While our study involved an unselected sample of adults who had consulted one of the participating general practices earlier in the same year, it does have some limitations. At 52%, the response to the first wave of questionnaires was modest, men were systematically under-represented, even for a consulting population, and paired data were available for fewer than half of the participants at baseline. However, it seems implausible that response/non-response would be confounded by level of agreement. Our covering letters asked for the spouse or partner to complete his or her questionnaire independently of the addressee, and responses were returned in separate envelopes, sometimes several weeks apart. One cannot see pairs of potential participants comparing their answers on independently-completed questionnaires and deciding to send them back to us only if the level of agreement looked reasonable. That the overall patterns of risk factors appear similar to those revealed by other national surveys also suggests that selective response has not been a large problem. On the other hand, since the kappa statistic is strongly affected by the prevalence of the attribute, the number of categories and the distribution of exposure in the population, our kappa statistics for different traits cannot be directly compared.32
Effective prevention of NCDs requires attention to multiple behavioural risk factors. The Robert Wood Johnson Foundation has sponsored a review of the efficacy of behaviour change programs for disease prevention and management, which highlighted the need for research on simultaneous interventions on multiple behaviours in patient populations.33 Although the literature includes studies addressing a wide range of behaviours, the review noted that almost all these studies targeted single risk behaviours,34 with very limited evidence available as to the impact of simultaneous interventions for multiple behaviours.35
General practice is potentially an important setting for implementing such strategies because a large proportion of the population sees a primary care physician at least once a year,36 and doctors are widely regarded as credible sources of preventive health information.37,38 For some risk factors objective validation could conceivably be undertaken only for small samples and would be prone to a significant Hawthorne effect. For other relevant factors, such as preferred form of milk consumed, direct observation of the individual is the only possible way to validate self-reports of behaviour, yet it is the long-term, habitual aspect of lifestyles that are particularly important. Apart from the individual, only his or her spouse or partner is likely to be able to report on this. Our study suggests that self-reports of lifestyle behaviours are sufficiently valid and reliable to forego the need to collect the confirmatory evidence from a second informant as a matter of routine.
The work is supported by a grant from the MBF Foundation, Australia. The National Health and Medical Research Council provided a scholarship for SP. We wish to thank the general practitioners, all staff associated with the project and all the patients who participated in this study.