• Open Access

Smoking, nutrition, alcohol and physical activity interventions targeting Indigenous Australians: rigorous evaluations and new directions needed

Authors


Correspondence to:
Dr Anton Clifford, National Drug and Alcohol Research Centre, University of New South Wales, 2052; e-mail: a.clifford@unsw.edu.au

Abstract

Objective: To describe and critique methodological aspects of interventions targeting reductions in smoking, poor nutrition, alcohol misuse and physical inactivity (SNAP risk factors) among Indigenous Australians.

Methods: An electronic search of eight databases and a manual search of reference lists of literature reviews and reference libraries for Indigenous-specific intervention studies published in peer-reviewed journals (January 1990 to August 2007) were undertaken. Alcohol, smoking, nutrition or physical activity needed to be the primary focus of the study and the intervention needed to specifically target Indigenous Australians.

Results: Twenty studies were selected for inclusion in the review. Methodologically, few studies employed randomisation or a control group, most omitted important details (e.g. costs), some did not report process measures (e.g. attrition rates), and some did not use validated measures. Two-thirds of interventions were implemented at the community level and employed multiple strategies.

Conclusion: There is a need for more rigorous evaluations of interventions targeting reductions in SNAP risk factors among Indigenous Australians, and to establish the reliability and validity of measures to quantify their effect.

Implications: It may be beneficial for future Indigenous-specific intervention research to focus on the evaluation of secondary prevention to complement the current concentration of effort targeting primary prevention. Community-wide interventions, combining strategies of greater intensity for high risk individuals with those of less intensity targeting lower risk individuals, might also offer considerable promise.

Smoking, poor Nutrition, Alcohol misuse and Physical inactivity (SNAP risk factors) are universally recognised as key behavioural risk factors for chronic disease.1,2 Indigenous Australians experience a disproportionately greater burden of harm from SNAP risk factors than the general Australian population.3 Compared to the general Australian population, Indigenous Australians are between two and seven times more likely to die from a tobacco-related disease;4 be hospitalised for an alcohol-related condition,5,6 and develop an obesity-related disease.7

Despite the disproportionately high burden of SNAP-related harm borne by Indigenous Australians, the number of Indigenous-specific intervention programs,8 including associated resource materials,9 to address this harm are less than optimal, and evaluations of Indigenous-specific SNAP interventions implemented to date, appear to be inadequate, both in terms of their quantity10 and their quality.11,12 With regard to quantity, a review of Indigenous health research found that approximately 10% of original research publications for specific time periods between 1987 and 2003 (inclusive) were intervention studies, while approximately 81% were descriptive studies.10 With regard to quality, a review of alcohol interventions targeting Indigenous Australians in 2000 found that less than 25% were published in peer-reviewed journals, leading the authors to conclude that, while a broader range of interventions ought to be implemented, these needed to be more rigorously evaluated in collaboration with Indigenous communities.11

A critical methodological review of evaluations of Indigenous-specific SNAP intervention studies published in the peer review literature is timely for at least two reasons. First, there have been no systematic critical reviews to date of nutrition or physical activity interventions targeting Indigenous Australians. Second, existing reviews of the alcohol and smoking literature focus on identifying and describing the types of Indigenous-specific intervention research, rather than examining their methodological quality.11,12 Therefore, this paper has two aims: to critique the methodological and contextual aspects of evaluations of Indigenous-specific SNAP intervention studies; and to examine the effect of these studies on reducing SNAP-related harm in Indigenous Australian communities.

Method

Search Strategy

Search 1: A simultaneous search of electronic databases, MEDLINE, EMBASE, CCTR (Cochrane Controlled Trials Register), CDSR (Cochrane Database of Systematic Reviews, ACP (American College of Physicians) Journal Club and DARE (Database of Abstracts of Reviews of Effects) was conducted to locate articles on smoking, nutrition, alcohol or physical activity and Indigenous Australians, published between January 1990 and August (week 1) 2007.

Search 2: The electronic database PsychINFO was searched separately as it was not possible to include the database in Search One.

Search 1 and 2 were conducted using the terms “Indigenous or Aborigin$ or Torres Strait Islander” and “nutriti$ or diet$ or physical or exercis$ or alcohol$ or grog or tobacco or smok$ or nicotine;” and “evaluation or intervention or prevention or outcome or dissemination.”

Search 1 resulted in 431 articles, after electronic removal of duplicates. Search 2 resulted in 181 articles, after electronic removal of duplicates.

Search 3: The National Indigenous Australian Alcohol and Other Drugs Bibliographic Database13 was searched using a search term listed in the database. The term Intervention was selected as it identified the greatest number of publications (n=540) relative to other search terms.

In addition, the reference lists of identified literature reviews and the Australian Indigenous HealthInfoNet Bibliography14 were examined. Four additional articles were identified. Combining searches resulted in 1,156 publications.

Classification of studies

The abstracts of the 1,156 identified publications were classified in two steps.

Step 1. Applying exclusion criteria: Articles were excluded if: (a) the study sample was not predominantly Indigenous Australian (n=377); (b) smoking, nutrition, alcohol or physical activity was not the primary focus of the study or a primary outcome measure (n= 214); (c) publications were duplicates or not journal articles (n=404).

A total of 995 articles were excluded, leaving 161 articles.

Step 2. Identifying intervention studies: The abstracts of the remaining 161 articles were examined by the first author to identify intervention studies. A study was defined as intervention if it assessed the effectiveness of a clinical or public health intervention among Indigenous Australians, or an intervention to improve healthcare delivery to Indigenous Australians.10 All articles were examined a second time by an individual blinded to the results of the first author. The level of agreement between individuals was 78%. Discrepancies were discussed and resolved. Thirty-seven intervention studies were identified, of which 11 were further classified as dissemination studies on the basis they evaluated an intervention to improve healthcare delivery. The remaining 26 articles were classified as intervention. Of these (n=26), six were excluded for the following reasons: four reported follow-up results for the same intervention study published in different journals; one full paper was inaccessible; and one study was published in two different journals. In total, 20 intervention studies were included for review. The process used to identify and extract studies is summarised in Figure 1.

Figure 1.

Process to identify and extract studies.

Extraction of information from intervention studies

Three main types of information were extracted from intervention studies.

(i) Methodological aspects: Criteria to examine methodological aspects of studies were adapted from a schema for evaluating public health interventions.15 The schema was selected as it includes established study appraisal criteria16–18 and was developed in consultation with Indigenous communities.15 Priority was given to criteria that addressed issues of internal rather than external validity,16 since the focus of this review was to examine the methodological rigour of evaluations. Review criteria, shown in Table 1, summarises study design, sample size and characteristics, attrition rates, use of validated measures, intervention exposure and outcome measures.

Table 1.  Summary of intervention studies in the methodological review.
First author and year publishedTargeted SNAP factorRegion of studyStudy designControl groupSample sizeSample characteristics reported (i.e. age, gender)>80 per cent completed the studyValidated measuresExposure to interventionOutcome measures
          BehaviourKnowledge/attitudesHarmHealth outcomes
  1. Notes:

  2. a) n=1 for intervention targeting one whole community.

  3. b) Original sample only — authors recruited additional subjects but exact number is unclear.

  4. (n) reported number of community members participating in x-sectional surveys.

Chang 2006NutritionRuralRCT187
Fraser 1996NutritionRemotePre/post271
Kruske 1999NutritionRemoteRCT51
Kukuruzovic 2002NutritionRemoteRCT180
Lee 1996NutritionRemotePre/posta1PartlyNANA
McDermott 2000 Scrimgeour 1994NutritionRuralPre/posta1 (335, 331, 304)Not reported
Chan 2007Nutrition/Physical ActivityUrbanCohort101Not reported
Rowley 2000Nutrition/Physical activityRemotePre/posta1 (437, 424) F/up=267Not reported
Rowley 2003 Lee 1995 Lee 1994Nutrition/Physical activityRemoteTime Seriesa1 (68, 45, 50, 29, 46)PartlyNot reported
Rowley 2000 Rowley 2001Nutrition/Physical activityRemoteCohort✓ self-selected49 (high risk cohort) a1 (200, 185, 132)
Smith 2000NutritionRemoteCohortNot reported
Egger 1999Physical activityRemotePre/post57Not reported
Ivers 2003SmokingRemotePre/post111
Ivers 2005SmokingRemotePre/post643
Johnston 1998SmokingRemotePre/post220
Mark 2004SmokingUrbanPre/post115Not reported
Crundall 1997AlcoholRemote/RuralPre/post52
Douglas 1998AlcoholRemoteTime Seriesa1PartlyNANA
Gray 2000AlcoholRemoteTime Seriesa1 (270)PartlyNANA
Gray 1998Alcohol/SmokingRuralPre/postb27Not reportedNot reported

(ii) Intervention characteristics: Information on the characteristics of the intervention and context of its implementation were extracted as both have been shown to influence intervention effectiveness.19 As summarised in Table 2, information on the intervention setting, the type of intervention strategies employed, intervention costs and the type of Indigenous involvement were extracted.

Table 2.  Summary of intervention characteristics.
First author and year publishedTargeted SNAP factorSetting(s)Primary intervention strategiesCostsType of Indigenous involvement
     DevelopmentImplementationEvaluation
Chang 2005 2006NutritionHospitalPharmacotherapyAboriginal research officer involved in participant consent process
Fraser 1996NutritionCommunity outstationsPharmacotherapyAboriginal council requested intervention
Kruske 1999NutritionHealth Service HomePharmacotherapy
Kukuruzovic 2002NutritionHospitalPharmacotherapy
Lee 1996NutritionFood retailImproved food supplyAboriginal board of directors of Arnhem land progress association
McDermott 2000 Scrimgeour 1994NutritionCommunity health Food retailEducation Health promotion Improved food supply FeedbackCommunity members surveyed. Consultation with local community and health councilAboriginal project officer assisted with implementation
Chan 2007Nutrition/Physical ActivityCommunity healthEducationAboriginal elders and AHWs involved in development of community education program
Rowley 2000Nutrition/Physical ActivityCommunity venues Community healthEducation FeedbackAboriginal community surveyedAboriginal community members decide process for screening in each community
Rowley 2003 Lee 1995 Lee 1994Nutrition/Physical activityFood retail Community venuesEducation Feedback Improved food supply65,000AHWs and Aboriginal community members identified need for intervention as part of community consultation and surveyAboriginal elders involved in implementationAboriginal research assistant provided community feedback
Rowley 2000 Rowley 2001Nutrition/Physical activityFood retail Community venuesEducation Health promotion Improved food supplyCommunity members initiated interventionAHWs, Aboriginal community council and community members delivered intervention components
Smith 2000Nutrition/Physical ActivityCommunity healthClinical assessments Counselling Education SupportAboriginal women and Aboriginal community council consultedAboriginal women coordinated education program
Egger 1999Physical activityCommunity venuesEducation Health promotionAboriginal male community members and elders consultedAHW assist in delivery of health education
Ivers 2003SmokingACCHS Community healthPharmacotherapy Brief interventionSurvey of health professionalsAboriginal research assistants administer questionnaire
Ivers 2005SmokingCommunity venuesMediaAboriginal research assistants
Johnston 1998SmokingEducation Community venuesEducation Media Health promotionAboriginal community members and health professionals assist in intervention developmentAboriginal sporting personalities deliver health educationAboriginal research assistants administer questionnaire
Mark 2004SmokingCommunity venuesPharmacotherapy Support groups Brief interventionAHWs work with health promotion co-ordinator to develop programAHWs promote program Aboriginal Project Officer facilitates quit smoking groupsAboriginal Project Officer administered telephone survey
Crundall 1997AlcoholPrisonEducationAboriginal men involved in tailoring of interventionAboriginal corrections officers follow-up participants
Douglas 1998AlcoholAlcohol retail EducationAlcohol restrictionsAboriginal community members and representatives from Aboriginal organisations members of committee that consulted with communityCommunity action to complement restrictions
Gray 2000AlcoholAlcohol retailAlcohol restrictionsAboriginal organisations lobby for interventionAboriginal interviewers
Gray 1998Alcohol/SmokingEducationEducation Health promotion$32,000Aboriginal education centre initiates interventionCommunity membersCommunity members and staff collect data

(iii) Intervention effectiveness: Information on the effectiveness of the intervention at reducing SNAP-related harm was obtained.

Results

SNAP risk factors targeted

SNAP risk factors targeted by the 20 intervention studies comprised: nutrition (n=7);20–26 nutrition and physical activity (n=4);27–34 physical activity (n=1);35 smoking (n=4);36–39 alcohol (n=3);40–42 and alcohol and smoking (n=1).43

Study design

Of the 20 intervention studies, 17 were evaluated using a non-randomised experimental design: three a time series;29–31,41,42 three a cohort;27,32–34 eight a pre-test/post-test with no control group;21,24,28,35,37,39,42,43 and three a pre-test/post-test with control group.36,38,40 Three interventions were evaluated using a Randomised Controlled Trial (RCT).20,22,23

Sample characteristics

Sample sizes of interventions targeting individuals (n=12) ranged from 27 to 271, with a mean of 125 and a median of 106.

Of the 17 studies that recruited study participants, nine reported the mean age of participants, which ranged from 14 months to 56.5 years with a mean age of 26.5 years.21–23,25,27,32,34,35,39,43 Eleven studies reported the percentage of study participants in various age groups.20,21,23,25,31,36,37,38,40,43 Sixteen studies reported the sex of participants, with the proportion of males ranging from 26% to 100%, with a mean of 53%.20–36,39,40

Fourteen of the 17 studies recruiting study participants reported the percentage of the eligible study population recruited, which ranged from 10% to 100%, with a mean of 75%. Thirteen studies reported attrition rates. Of these, six reported that >80% of study participants completed the intervention and were followed up successfully.22,23,34–36,40

Measures

Measurement Instruments

Seventeen studies used validated measurement instruments, including biochemical and/or clinical measures (n=11)20–23,27,28,32–36 and community level data (n=six).24–26,29–33.41,42 Seven studies used questionnaires (35%),32,36–40,43 of which two used existing or pre-tested instruments.37,40 One study conducted interviews.42

Process measures – measures of intervention implementation and participants’ exposure to the intervention

Of the 17 studies measuring individual level outcomes, nine measured study participants’ level of exposure to intervention components using checklists or self-report.20–23,32,36–38,40

Outcome measures

a) Smoking (five studies)

Four studies measured knowledge and attitudes,36–38,43 four smoking status,36–39 one self-efficacy39 and one readiness to quit.36

b) Nutrition and/or Physical Activity (12 studies)

Eleven studies measured nutrition-related indicators: biological markers (eight studies)21–24,27–33 and anthropometry (11 studies).21–35 Four studies measured dietary intake24,29–31,32,33 and two physical activity levels.32,35

c) Alcohol (four studies)

Three studies measured alcohol consumption: per capita41,42 and individual consumption.40,43 Three studies measured alcohol harms.40–42 One study measured attitudes and knowledge,43 and one community perceptions.42

Data collection methods

a) Smoking (five studies)

Four smoking studies used self-report questionnaires.36–38,43 Three smoking studies measured point prevalence abstinence,36,38,39 of which one validated self-report cessation with a biochemical measure.36 One study employed interviews and observation.37

b) Nutrition and/or Physical Activity (12 studies)

Nutrition and physical activity studies used clinical assessment (11 studies),20–23,25–35 survey (three studies),32,33 food store turnover (three studies)24–26,30–31 and recorded observation (one study).26

c) Alcohol (four studies)

Alcohol studies used self-report questionnaires (three studies),40,42,43 alcohol sales audit (two studies),41,42 semi-structured interviews (two studies)42 and audit of health, crime and welfare data (two studies).41,42

Data collection points

Eighteen interventions (90%) identified the timeframe for data collection, as post intervention (10 studies) and at intervals during the intervention (eight studies).

Intervention characteristics

Intervention setting

Interventions were implemented across eight types of settings: food retail (n=4);24–26,29–33 community health (n=6);21,22,25–36,39 community venues (n=5)31,32,35,38,39 education (n=3);38,41,43 hospital (n=2);20,23 alcohol retail (n=2);38,41 prison (n=1)40 and Aboriginal Health Service (n=1).36

Intervention strategies

Ten principal intervention strategies were identified from key words authors of studies used to describe the intervention. Education (10 studies)25–35,38,40,43 strategies were primarily employed, followed by pharmacotherapy (six studies),20–23,36,39 health promotion (five studies)25,26,32,33,35,38,43 improved food supply (four studies),24–26,29,32 feedback (three studies),25,26,28,29–31 alcohol restrictions (two studies),41,42 media (two studies); support (two studies);34,39 counselling (one study)34 and clinical assessment (one study). Four studies employed three strategies29–33,38,39 and two each employed two35,43 and four strategies.25,26,34

Costs

Only two studies reported costs associated with the intervention.31,43

Indigenous involvement

Eighteen of the 20 studies reported Indigenous involvement in intervention development, (n=16 studies),21,24–36,38–43 implementation (n=11 studies)20,25,26,29–35,37–39,41,43 and/or evaluation (n=7 studies).29–33,36,38–40,42,43 Four studies reported Indigenous involvement in intervention development, implementation and evaluation.29–33,38,39,43

Intervention effectiveness

Due to the methodological deficiencies of included studies, and the variability in outcomes reported, effect sizes were not summarised for comparison. Notwithstanding this, some observations are made.

For smoking interventions, smoking cessation rates, were 15% at six months;36 6% at three months;38 and 6% at 12 months.37 Smoking interventions reported few positive changes in knowledge and attitudes.36,38,43

For nutrition, pharmacotherapy interventions (n=4)20–23 were more likely to demonstrate a positive effect than education or health promotion interventions (n=7).25–34 For alcohol, community-level restrictions (n=2)41,42 appeared more effective than individual-level education (n=2),40,43 with reductions in alcohol consumption41,42 and alcohol-related hospital admissions and crime reported.

Discussion

Consistent with previous reviews, few Indigenous-specific interventions were identified in the peer-reviewed literature,10–12 and there was evidence of weak intervention effects and methodologically deficient evaluations. While some interventions yielded results suggestive of a positive effect, non-randomised study designs, a lack of a comparison group and poor attrition rates yielded less than optimal evidence.

Intervention types and strategies

Two-thirds of intervention studies were implemented at the community level, and included components such as media, alcohol and food retail outlets and schools. Such components have been known to increase the impact of community-based interventions,44 so their inclusion is encouraging. This predominance of community level strategies possibly reflects that the effects of SNAP risk factors in Indigenous communities are cumulative, extending beyond the individual to the family and community.45 There would, therefore, likely be value in evaluating an intervention combining a mix of individual, family and community-based strategies targeting SNAP risk factors in the one coordinated approach.

The lack of studies evaluating secondary preventive interventions, such as Brief Intervention (BI), is less than optimal: brief intervention is cost-effective46 and Indigenous-specific BI resources have been developed.8 The lack of Indigenous-specific evidence for BI is likely to be one reason for its low uptake in Indigenous primary healthcare.47

Study designs

Only half of the intervention studies in this review were evaluated using study designs with random allocation or a control group. This offers the potential to improve the methodological quality of evaluations of Indigenous SNAP interventions through the application of more rigorous study designs. While the RCT is the strongest design to assess the effectiveness of healthcare interventions,16 its implementation in Indigenous communities can pose significant ethical and practical challenges.48 Guidelines for evaluating complex interventions underline the importance of researchers employing rigorous and practical study designs to improve the quality of intervention evaluations.49 Methodologically rigorous study designs, such as interrupted time series with a control group and comparative studies with concurrent controls should, therefore, be considered. These designs can be implemented at relatively low cost, maintain the advantages of randomisation and present fewer practical challenges.50

Measurement instruments and measures

Encouragingly, 17 of the 20 studies reported using validated instruments. Less encouraging, only half of the studies administering a questionnaire reported it to be validated or adapted from a validated instrument. Without standardised instruments the accuracy of research findings is questionable and comparison across studies is problematic.51 Few measurement studies have been conducted in Indigenous communities, resulting in limited knowledge of which measurement instruments are methodologically sound and culturally appropriate.10 Different approaches are often needed to effectively measure Indigenous-specific health outcomes.52 Until the validity of measurement instruments in Indigenous communities is established, researchers should examine the reliability and validity of outcome measures through pilot studies or test/re-test before using them to measure the effect of an intervention.53

Outcome measures were generally collected at reasonable follow-up periods, however, the length of follow-up was not always optimal. For example, one smoking study measured cessation rates at three months follow-up,9 despite evidence that smoking cessation is not reliably established until six months.54 Less than optimal follow-up periods may reflect shortcomings in intervention funding, which may have been short-term and insufficient to undertake repeat evaluations of outcomes at multiple time points.

Standard of reporting

The standard of reporting of key methodological issues was variable. Poor reporting of an intervention evaluation makes it difficult for the intervention to be replicated or adapted for other Indigenous communities, or for wider implementation.55 For example, only two intervention studies reported quantitative measures of exposure to the intervention. The remaining 18 studies did not report a measure of exposure or only reported the method used to measure exposure, making it difficult to determine the extent to which poor outcomes were attributable to poor exposure or weak intervention effect.

Indigenous Involvement

Indigenous involvement in Indigenous health research is recommended ethical practice,56 offers practical advantages57 and can improve understanding of how change is achieved.58 Although 18 of the 20 studies in this review reported Indigenous involvement, only seven reported Indigenous involvement in the evaluation. Intervention evaluation research is complex, requiring resources, expertise and skills unlikely to be available in local Indigenous communities. Research designs that utilise methodological approaches shown to build local Indigenous research capacity, forge effective partnerships between Indigenous health care services and research institutions, and engage Indigenous peoples as equal partners in the research process are, therefore, crucial.58

Effectiveness

Heterogenic outcome measures meant that reliable estimates and comparisons of the effectiveness of interventions in a meta-analysis was not possible.

Limitations of the review

The search method may not have located all relevant intervention studies published in the peer-review literature; eight electronic databases were searched simultaneously and MESH terms were not included. A search of two electronic databases specific to Indigenous health and examination of reference lists of published reviews only identified four additional studies, making it unlikely that the search publication bias is another potential limitation, with evidence that preference is given to publishing studies demonstrating a statistically significant effect.59 The potential outcomes of this scenario would be underestimation of the actual number of studies conducted, under-representation of studies demonstrating a weak effect and, therefore, overestimation of program effectiveness. As such, the studies here most likely represent a best-case scenario of intervention studies in Indigenous Australian communities. Finally, the criteria used to assess the methodological adequacy of the identified intervention studies might be inappropriate. That these studies were judged to be of generally poor methodological quality is, however, consistent with findings of previous reviews,13–15 which suggests that it is more likely that the studies, rather than the review criteria, are problematic.

Conclusion

This review shows methodologically rigorous evaluations are relatively rare, which is consistent with previous reviews showing that intervention studies are undertaken infrequently in Indigenous health and that their effects tend to be small. Therefore, a number of clear recommendations appear warranted. First, more intervention studies utilising more methodologically rigorous evaluation designs are needed in Indigenous-specific settings in Australia. Second, the strong evidence base for the cost-effectiveness of secondary prevention and the disproportionately high burden of SNAP-related harm borne by Indigenous Australians, suggests an increase in evaluations of secondary preventive interventions, such as brief intervention, are needed to complement the current concentration of effort targeting primary prevention. Community-wide interventions, co-coordinating a series of strategies across whole communities, also offer considerable promise, particularly if strategies of greater intensity targeting high risk individuals can be cost-effectively implemented to complement those of less intensity targeting lower risk individuals. Third, establishing the acceptability, reliability and validity of measures, for evaluating the effect of Indigenous-specific interventions is urgently required. Finally, building the capacity of local Indigenous communities and their healthcare services to engage as equal partners in the research process, should inform the development and implementation of evaluation designs.

Ancillary