The validity of a depression screening tool modified for use with Aboriginal and Torres Strait Islander people
This article is corrected by:
- Errata: Correction Volume 32, Issue 5, 496, Article first published online: 8 October 2008
Dr Fay Johnston, Menzies Research Institute, Private Bag 23, Hobart, Tasmania 7001. Fax: (03) 6226 7755; e-mail: email@example.com
Objectives: To assess the reliability and validity of a depression screening tool — the PHQ-9©Pfizer Inc. modified for use with Aboriginal and Torres Strait Islander people. We also sought to determine the prevalence of depression in a sample of Indigenous people with ischaemic heart disease (IHD).
Methods: The modified PHQ-9 was administered to a sample of Indigenous people with IHD by an Aboriginal Health Worker (AHW). Tool results were then compared with the results of a psychiatric diagnostic interview conducted by a medical practitioner. Thirty four IHD patients attending an Aboriginal Community Controlled Health Service (ACCHS) in Darwin in 2006 and 2007 participated in the study. The modified PHQ-9's sensitivity, specificity, positive and negative predictive value were calculated for major and minor depression. Chronbach's alpha of the screening test was calculated to measure internal consistency. The prevalence of depression in the study group was also determined.
Results: The prevalence of major depression in the sample was 15.4% (95% CI 7.2%-29.7%). When assessing for major depression the modified PHQ-9 was 80% sensitive (95% CI 66.4-93.6%) and 71.4% (95% CI 56.0-86.8%) specific. A ‘mini’ version of the modified PHQ-9 demonstrated 100% sensitivity (95% CI 100%-100%) and 12.5% specificity (95% CI 7.0% -25.7%) Chronbach's alpha was 0.8.
Conclusion: The modified PHQ-9 and the mini-tool, showed promise in this setting. Further investigation with a larger number of Aboriginal and Torres Strait Islander participants is warranted.
Implications: This study has implications both for the Medicare funded Aboriginal Adult Health Checks and for program planning for Aboriginal IHD patients.
The high burden of mental illness weighs heavily on the Aboriginal and Torres Strait Islander community.1,2 Despite this, little has been published concerning the Aboriginal and Torres Strait Islander prevalence of depression.3
While some elements of depression are a common human experience, other manifestations can be culturally determined.4,5 Recognition of this has led to the concept of cultural equivalence which can be defined as a psychological assessment tool's performance with people who belong to a different cultural group from those with whom the tool was originally assessed.
The importance of cultural equivalence was demonstrated in the recent Western Australian Aboriginal Child Health Survey which included development of a research tool for assessing the psychological wellbeing of Indigenous children.6 Similarly, the National Indigenous Health Survey incorporated an Emotional Social Wellbeing module7 which is appropriate for detecting general psychological distress. However, the ability of psychological assessment tools to detect mental illness in adult Aboriginal and Torres Strait Islander adults remains untested. This is congruent with Sanson-Fisher's observation concerning the paucity of research assessing culturally appropriate measures.8 The scarcity of published work concerning culturally appropriate psychological screening has implications in the current policy context of Medicare-funded Indigenous Adult Health Checks, which include an “Emotional Social Wellbeing” component.
In a previous paper we have discussed the modification of a widely used depression screening tool, the PHQ-9, for use with Aboriginal and Torres Strait Islander people9 (Table 1). The modified PHQ-9 was deemed, if administered according to specific guidelines, to be acceptable to a sample of Aboriginal and Torres Strait Islander people. Integral to this modification was the inclusion of a question pertaining to what may represent a culturally-specific symptom of depression, anger.
Table 1. The Modified PHQ-9.
|(1)||Not enjoying things like you used to.|
| ||Qualifier: The administrator may ask “What do you normally like doing?”“How often do you enjoy doing ______ in the last 2 weeks”|
|(2)||Feeling down, depressed or hopeless.|
| ||Qualifier: The word “sad” may be substituted for “depressed” if the patient doesn't understand it.|
|(3)||Trouble falling or staying asleep, or sleeping too much.|
|(4)||Feeling tired or having little energy.|
|(5)||Eating more or less than you used to.|
|(6)||Feeling bad about yourself. Feeling shamed or that you have let yourself or others down.|
|(7)||Trouble paying attention to what is going on around you.|
| ||Qualifier: If the person doesn't understand this you can ask them “What do you normally do?”“How often have you been able to pay attention when doing _____ in the last 2 weeks?”|
|(8)||Moving or speaking so slowly that other people could have noticed?|
| ||Or the opposite — Being so nervous or restless that you have been moving around a lot more than usual.|
|(9)||Thoughts that you would be better off dead or of hurting yourself or others in some way.|
|(10)||Have you felt angry?|
Although a significant cause of morbidity and mortality of itself, depression has recently been described as a risk factor for IHD. This risk extends to both the development of clinical IHD, and its prognosis.10 Depression as a risk factor is independent of traditional risks for IHD such as smoking, body mass index and high blood lipids with studies carried out across a range of countries and cultures supporting this association. The association is a strong one. Major depression has been found to be associated with a 3-5 fold increase in rates of IHD, while minor depression is associated with a 1-2 fold increase.10 Major depression is therefore of a similar risk magnitude to the traditional risk of smoking and hypercholesterolemia.10
Ischaemic heart disease (IHD) is the largest contributor to morbidity and mortality in Australia, with Aboriginal and Torres Strait Islander people suffering under the burden of this disease, both younger, and more frequently than non-Indigenous Australians. Death attributable to ischaemic heart disease in Indigenous Australians was found to be almost twice the rate in the non-Indigenous population.11 Indigenous Australians are hospitalised for IHD at significantly higher rates than non-Indigenous Australians.12
Because of this high burden of IHD, and the role of depression as a risk factor we decided to assess the modified PHQ-9's validity with Indigenous IHD patients. The study also provided the opportunity to determine depression prevalence in this group.
The study was conducted at Danila Dilba, an Aboriginal Community Controlled Health Service (ACCHS) for six months in Darwin in 2005-06. It provides services to Darwin's urban Indigenous population, people from remote localities, and a number of Aboriginal camps (both town camps and those in the Darwin Rural area). As a result clients come from a wide variety of social settings.
This study was approved by the joint Human Research Ethics Committee of the Menzies School of Health Research and the Northern Territory Department of Health and Community Services and by the ACCHS's management and community-elected committee.
All English-speaking patients with IHD presenting to the Health Service within the funded study period were approached to participate by an Aboriginal Health Worker (AHW). A person was deemed to be of Aboriginal or Torres Strait Islander descent if they identified as such. People were considered to have IHD if they fulfilled the following criteria:
- (a) They had a history of myocardial infarction documented by a cardiac enzyme rise.
- (b) They had a positive angiogram or
- (c) They had a positive cardiac stress test.
Clinical symptoms alone suggestive of IHD were not sufficient for inclusion in the study.
The modified PHQ-9 was administered by an Aboriginal Health Worker. A semi-structured diagnostic clinical interview was conducted by a GP from the health service who was registered to provide psychiatric services in the Northern Territory (NT). Each subject was diagnosed with major depression if they met the DSM IV criteria for this diagnosis.5 A diagnosis of minor depression was given to participants who had between two and four of these criteria including at least one of lowered mood or anhedonia. All interviews occurred within two days of screening tool administration. The GP was blinded to the screening tool result. Clinic organisational changes coincided with the study, resulting in less people presenting to the clinic than anticipated. Even after extending the duration of the study, recruitment had to be stopped before reaching the planned sample size of 95.
The prevalence of depression was calculated based on the results of the diagnostic psychiatric interview. The results of the modified PHQ-9 were compared with those of the diagnostic interview. The ‘gold standard’ in diagnosis of depression is a diagnostic psychiatric interview. Ideally, such interviews rely on following a rigorous protocol with the use of pre-formulated language. The use of the structured diagnostic interview has itself not been assessed with Aboriginal and Torres Strait Islanders. Rather than follow a structured diagnostic interview, we therefore employed a semi-structured psychiatric interview which took into account both the DSM-IV criteria for depression and culturally appropriate clinical communication. The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) of the modified PHQ9 were calculated. Internal consistency was calculated using Cronbach's alpha, a standard approach for describing the reliability of multi-point formatted questionnaires or scales. The Alpha coefficient ranges in value from 0-1 with values closest to 1 indicating the greatest internal consistency. A value of 0.7 is considered to be acceptable.13 Receiver-operator characteristic (ROC) curves were constructed to determine the most appropriate cut-off score for the modified PHQ9, by maximising sensitivity without unacceptable loss of specificity.
We also examined the validity of a ‘mini-tool’ using only questions 1 and 2. We examined the performance of this mini-tool variously interpreting a positive result as being:
- 1) Reflected by any affirmative answer to either question 1 or question 2.
- 2) Reflected by a score of 2 or more on either question 1 or 2.
- 3) Reflected by a score of 3 or more on either question 1 or 2.
Thirty-four people participated in the screening and psychiatric interview, five people in the psychiatric interview only. Table 2 describes participant demographics. There were two refusals to participate. Two clients from remote localities agreed to participate when they were next in Darwin, however, they had not represented at the time of study closure.
Table 2. Demographic characteristics of participants who completed both the screening and diagnostic interviews (n=34).
|Torres Strait Islanders||2 (6%)|
|Live in Darwin||33 (97%)|
|Born in a capital city||12 (35.3%)|
|Born in a remote community||13 (38%)|
|Born in a regional centre||9 (26.7%)|
|Average age||57.7 years|
In this sample the prevalence of minor depression, as measured by diagnostic psychiatric interview, was 12.8% (95% CI 5.6%-26.7%), the prevalence of major depression was 15.4% (95% CI 7.2%-29.7%) and the prevalence of either minor or major depression was 28.2% (95 % CI 16.6-43.8).
Chronbach's alpha for the modified PHQ-9 was 0.8. The ROC curves identified nine as the best cut-off score. This cut off score was used to calculate the attributes of the test.
The sensitivity, specificity, PPV and NPV of the modified PHQ9 and mini tool are shown in Table 3. The proportion of participants that were diagnosed with depression are shown in Table 4 along with the proportion of participants who were identified by each of the screening tests. The concordance of each screening test with the outcome of the diagnostic interview is shown in Table 5.
Table 3. Sensitivity, specificity, PPV and NPV of the modified PHQ9 using a cut-off score of 9a and the three versions of the mini tool.
|Minor or major depression using the modified PHQ9||70%||78.3%||58.3%||85.7%|
|Major depression only using the modified PHQ9||80%||71.4%||33.3%||95.2%|
|Minor or major depression using the mini-tool with any affirmative answer to questions 1 or 2||100%||12.5%||32.3%||100%|
| || ||(7.0-25.7)||(15.8-48.7)|| |
|Minor or major depression using the mini-tool scoring 2 or more on either question 1 or 2||90%||20.8%||32.1%||83.%|
|Minor or major depression using the mini-tool scoring 3 or more on either question 1 or 2||30% (16-58.4)||70.8% (52.6-89)||30% (16-58.4)||70.8% (52.6-89)|
Table 4. Percentage (95%CI) of participants diagnosed with depression compared with the percentage identified by each screening test.
|Psychiatric diagnostic interview||28.2% (16.6-43.8)|
|Modified PHQ-9||32.4% (6.7-48.1)|
|Mini-tool with any affirmative answer to questions 1 or 2||91.2 % (81.7-100.0)|
|Mini-tool scoring 2 or more on either question 1 or 2||85.3% (73.4-97.2)|
|Mini-tool scoring 3 or more on either question 1 or 2||29.4% (14.1-44.7)|
Table 5. Concordance of the results of the screening tools with the results of the diagnostic psychiatric interview.
|Modified PHQ-9||73.6% (58.7-88.3)|
|Mini-tool with any affirmative answer to questions 1 or 2||38.2% (21.9-54.6)|
|Mini-tool scoring 2 or more on either question 1 or 2||44.1% (27.4-60.8)|
|Mini-tool scoring a 3 on either question 1 or 2||55.9% (39.2-72.6)|
The modified PHQ-9 performed well in this sample. The study also demonstrated a prevalence of depression which is higher than in the general population (thought to be 6%).14 The high prevalence of depression in this sample highlights the potential role it may play in the epidemic of IHD experienced by Indigenous Australians. Evidence suggests that traditional cardiovascular risk factors do not account for the magnitude of the cardiovascular disease burden placed on the Indigenous community.15 Placing more emphasis on depression case finding among Indigenous IHD sufferers may therefore be prudent. Unfortunately, a reliable IHD register was not in place at the service during the study period, hence we do not know the extent to which our sample was representative. Some reassurance may be gained from the low number of qualifying patients refusing to participate. The inclusion criteria did mean that those who have silent ischaemia (e.g. due to diabetes) or those who had not engaged with Western medicine in order to reach a diagnosis were excluded. Whether or not depression prevalence or tool validity would have differed in these groups remains unknown.
Although it is tempting to view the mini-tool as better than the entire measure due to its improved sensitivity and negative predictive value, this should be done with caution. Either depressed mood or anhedonia are compulsory if a diagnosis of depression is to be reached according to the DSM IV.14 A collection of other symptoms may also be present. It is not surprising then that an abridged tool, looking at only these compulsory symptoms, and ignoring the others would appear more sensitive. It may be because the mini-tool is more closely linked to the ‘gold standard’. In addition, the mini tool, in its three forms, exhibited disappointing specificity and positive predictive value. The complete modified tool exhibited both the highest positive predictive value and the highest overall concordance with the results of the gold standard. However, a limitation to our study may be the ‘cultural equivalence’ of the ‘gold standard’ itself.
It is important to note that the recommended cut-off score for the original PHQ-9 is 10. In our sample, the use of the test with this cut-off score had lower sensitivity and specificity, highlighting the importance of validating psychological assessment instruments in different social and cultural contexts.
Time and funding constraints led to some limitations of their own. Ideally, data would have been collected from a larger sample size. However, in the context of minimal validation studies comparing assessment tool use with diagnostic criteria even a small study provides important information. As well as the demonstrated high level of internal consistency (Chronbach's alpha = 0.8), other measures of reliability such as test/retest would be useful. They were not calculated as they require two administrations of the instrument.
For screening purposes, a general measure of psychological distress may be more appropriate than a diagnosis specific tool. Use of a general measure acknowledges that mental illness diagnoses are a Western medical construct and recognises the need for a broader understanding of ‘Social Emotional Wellbeing’. Nonetheless, the preference of clinicians to work within diagnostic paradigms aligned with the current evidence base lends itself to the use tools for which diagnostic sensitivity is known.
It must be noted that anger may have an independent association with cardiovascular risk.16 This could mean that the validity of the tool in the sample population does not correspond with validity in the general population. There is the possibility that IHD patients may tend to score higher on the modified PHQ-9 due to the inclusion of a question relating to anger which may account for the lower cut-off score than is used with original PHQ-9 in the general population. Ultimately, a variety of clinical situations exist, with the mini-tool more appropriate in one setting and the entire tool in another. For example the high negative predictive value of mini-tool may make it appropriate as part of Emotional Social Wellbeing screening during an Adult Health Check. The current National Aboriginal Community Controlled Health Organisations (NACCHO) guidelines for the Emotional Social Component of the Aboriginal Adult Health Check involves the practitioner asking two questions, one relating to depressed mood and the other to anhedonia.17“Over the past two weeks have you felt little interest or pleasure doing things”, and, “Over the post two weeks have you felt down depressed or hopeless?”
Asking about depressed mood and anhedonia as they are worded in this modified PHQ-9 is therefore concurrent with the NACCHO recommendations, although possibly with more appropriate wording which has been validated for depression. This study suggests that a negative result to the mini tool could be used to reassure that further Social Emotional Wellbeing assessment is not necessary as part of the Adult Health Check. Either using any affirmative answer to questions one and two, or a score of at least two on either of these questions gives an adequate NPV for this purpose. Those who receive a positive screen with the mini tool could then have the entire tool administered. The poor specificity, PPV and overall concordance with the gold standard of the mini-tool preclude its use as an independent depression screen. The entire tool may be suitable for depression screening of an IHD patient for whom recognition and treatment of their depression is an integral part of their cardiovascular risk management. The high prevalence of depression in our sample supports the suggestion that such screening is warranted. The study must certainly act as a complement for clinicians and policy makers to the currently available evidence concerning the use of such measures in the Indigenous context.
The diversity in Aboriginal and Torres Strait Islander culture means that caution must be taken before generalising these results to the wider Aboriginal and Torres Strait Islander population such as those without IHD, or those living in remote communities. This study may be seen as a promising pilot justifying future research in other Indigenous contexts.
This study has demonstrated the validity of the modified PHQ-9 for screening for depression in a group of Indigenous patients with IHD. It has also demonstrated that such screening is feasible and that the tool is acceptable as part of a clinical assessment. Validation of the modified tool with other Indigenous groups in Australia is warranted. The research also found a much higher prevalence of depression in this sample of Indigenous people with ischaemic heart disease than that reported for the general Australian population. It highlights the importance of depression as risk factor for ischaemic heart disease in Indigenous patients and the need for reliable and validated tools for screening for this condition in Indigenous populations.
The authors wish to thank Danila Dilba Health Service and Ms Leisa McCarthy, Dr Christine Connors, Dr Wendy Gunthorpe, Mr Stephen Halpin and Dr Trish Nagel. Aspects of ‘Sadness and Heart Disease’ were funded by an NHMRC General Practice Clinical Research Program Grant, a GPET Registrar Research Grant and a Northern Territory PHCRED Research Bursary.