A review of community engagement in cancer control studies among Indigenous people of Australia, New Zealand, Canada and the USA

Authors

  • J. MILLER B PSYCH (HONS), EVALUATION OFFICER,

    1. Cancer Council SA, Eastwood, SA
    Search for more papers by this author
  • V.E. KNOTT PHD, ASSISTANT PROFESSOR,

    Corresponding author
    1. Centre for Applied Psychology, Faculty of Health, University of Canberra, Bruce, ACT
      Vikki Knott, Faculty of Health, University of Canberra, Bruce, ACT 2601, Australia (e-mail: vikki.knott@canberra.edu.au).
    Search for more papers by this author
  • C. WILSON PHD, CANCER COUNCIL SA CHAIR IN CANCER PREVENTION,

    1. Cancer Council SA, Eastwood, SA, and Flinders Centre for Cancer Prevention and Control, School of Medicine, Flinders University, Bedford Park, SA
    Search for more papers by this author
  • D. RODER PHD, CHAIR OF CANCER EPIDEMIOLOGY

    1. Division of Health Sciences, Division Office Research HSC, University of South Australia, Adelaide, SA, Australia
    Search for more papers by this author

Vikki Knott, Faculty of Health, University of Canberra, Bruce, ACT 2601, Australia (e-mail: vikki.knott@canberra.edu.au).

Abstract

MILLER J., KNOTT V.E., WILSON C. & RODER D. (2012) European Journal of Cancer Care21, 283–295

A review of community engagement in cancer control studies among Indigenous people of Australia, New Zealand, Canada and the USA

This review aimed to address studies of cancer control in Indigenous populations, with a focus on: (1) the nature and extent of community engagement; and (2) the extent to which community engagement has facilitated successful outcomes. Articles addressing Indigenous cancer control using some degree of community engagement were identified by a search of the following electronic databases: MEDLINE (via Ovid and Pubmed), psycINFO, CINAHL and Google Scholar. Relevant studies were scored and analysed according to Green et al.'s guidelines for participatory research. Studies often engaged the community only minimally. Where studies resulted in successful outcomes, they tended to have included Indigenous community members in genuine research roles, from planning, to implementation, to presentation of results at conferences. Studies with positive health outcomes were often initiated by a combination of academic researchers and community members or organisations. This narrative review highlighted significant scope for improvement in community-based studies addressing Indigenous cancer control. Increased attention to the philosophical underpinnings of community engagement is required to ensure that the benefits of this approach are translated to achieve improved cancer control outcomes. An increased awareness of the benefits of community engagement may prove effective in conducting cancer control research that leads to improved outcomes in Indigenous communities.

INTRODUCTION

Significant disparities in cancer outcomes exist between Indigenous and non-Indigenous peoples in Australia, New Zealand, Canada and the USA. Indigenous peoples in these countries experience disproportionately high mortality rates for a number of cancers, including breast [New Zealand (New Zealand Ministry of Health 2005), USA (Goggins & Wong 2007)], cervix [Australia (Cunningham et al. 2008), New Zealand (New Zealand Ministry of Health 2005), Canada (Marrett et al. 2004)], gallbladder [Canada (Marrett et al. 2004)], lip and oropharynx [Australia (Cunningham et al. 2008)], liver [Australia (Cunningham et al. 2008), New Zealand (New Zealand Ministry of Health 2005)], lung [Australia (Cunningham et al. 2008), New Zealand (New Zealand Ministry of Health 2005), USA (Goggins & Wong 2007)], prostate [New Zealand (New Zealand Ministry of Health 2005), USA (Goggins & Wong 2007)] and stomach [New Zealand (New Zealand Ministry of Health 2005), USA (Lanier et al. 2006)]. Attempts to greatly reduce these disparities have so far proven unsuccessful. One reason for this may be a lack of willingness on the part of Indigenous peoples to engage with mainstream health services (Morgan et al. 1997), as well as a distrust (Holmes et al. 2002) and rejection (Humphery 2001) of western research programmes, some of which have been harmful to various Indigenous groups.

Historically, Aboriginal and Torres Strait Islander people have often been treated as the ‘passive subjects’ of research (Holmes et al. 2002). However, governments, policy makers and concerned community members have increasingly advocated for a paradigm shift, whereby issues are prioritised by the communities concerned and research results are translated into health policy and practice (Viswanathan et al. 2004). In Australia, the National Health and Medical Research Council (NHMRC) has highlighted the necessity of ‘community involvement in the development, conduct and communication of the research’[The Aboriginal and Torres Strait Islander Research Agenda Working Group (RAWG) of the NHMRC 2004], with consideration for the interests of Aboriginal and Torres Strait Islander people involved.

A growing number of researchers claim to incorporate new research models that achieve these goals through active community engagement in the research process. Community engagement, which is a crucial component within a range of related research paradigms (e.g. community-based participatory research, participatory action research and collaborative community action research), involves a close liaison between the researcher and the community in which the research will occur. ‘Community’ here refers to a ‘socially-constructed unit of identity’ (Israel et al. 1998). In the context of community-based research, decisions and responsibilities are shared by researchers, community organisations (e.g. community health services or tribal councils) and members, with results disseminated to all partners (Israel et al. 2005) and benefits to the involved communities (Strickland 2006). Ideally, there is a balance between action and research with the aim of reducing health disparities (Minkler & Wallerstein 2008).

Community engagement aims to create the possibility for a range of benefits, both to the communities involved and to the relevance of the research. Projects that engage the community hold the potential to provide communities with additional resources (Altman 1995), such as funding from research grants to support health initiatives, and the creation of new employment positions for community members. Other benefits include the potential to examine and reduce marginalisation, to increase community capacity through involving community partners in the identification of health priorities and solutions and to provide training and health education (Israel et al. 1998). There is an inevitable degree of subjectivity in the designation of outcomes as ‘benefits’, and evaluation of community benefits is likely to be more relevant if involvement of community partners enables due account to be taken of their values and priorities. For researchers, community engagement can provide the opportunity to develop theories incorporating the ‘lived experience’ of those involved (Israel et al. 1998). Oliver et al. (2004) have suggested that the inclusion of consumers in research can serve to remove barriers to participation, improve the profile of research and improve the uptake of research findings.

The theoretical and potential practical benefits of community-based studies among Indigenous populations appear significant, but there remains little research on the quality and effectiveness of Indigenous cancer studies that engage the community as part of research and programme evaluation. A 2010 review of cancer control among Aboriginal and Torres Strait Islander peoples recommended a review of factors behind the success or failure of community-based studies (Miller et al. 2010). The extent of community involvement in Indigenous projects is often limited, with community members tending to be included in some aspects (e.g. focus group participation) but not others (e.g. data analysis) (Teufel-Shone et al. 2006; Daley et al. 2010), and some health researchers express anxiety about the possibility of tokenism overshadowing genuine consumer engagement (Ward et al. 2009). Even where researchers are intent on involving Indigenous communities in projects, insufficient funding (Shahid et al. 2009a) and short deadlines (Burhansstipanov et al. 2005) can render this a difficult goal to achieve. Green et al. (1995) have outlined criteria for the evaluation of participatory research according to the degree of community engagement, the potential of research to empower the community and the extent of benefits to community members. The present review aims to assess studies that use a community engagement approach to address cancer among Indigenous populations, with an emphasis on: (1) the extent of community involvement, ownership, benefit and empowerment that has been facilitated; and (2) the overall effectiveness of each programme, with the aim of examining relationships between the design and implementation of research programmes and their effectiveness.

METHOD

The search was undertaken using Medline (via Ovid and Pubmed), psycINFO, CINAHL and Google Scholar. In order to identify references to Indigenous cancer control research that incorporated principles of community engagement, the following search terms were used: (‘Indigenous’ OR ‘Aboriginal’ OR ‘Torres Strait Islander’) AND (‘cancer’ OR ‘nicotine’ OR ‘tobacco’ OR ‘smoking’ OR ‘tobacco control’ OR ‘smoking cessation’) AND (‘cancer control’ OR ‘prevention’ OR ‘screening’ OR ‘support’ OR ‘palliative care’ OR ‘survivorship’ OR ‘treatment’ OR ‘culturally appropriate’ OR ‘participatory action research’ OR ‘community engagement’ OR ‘action-research’ OR ‘consumer engagement’ OR ‘translational research’ OR ‘community-based participatory research’ OR ‘collaborative community action research’ OR ‘interventions’ OR ‘translational research’ OR ‘equity’ OR ‘programs’ OR ‘evaluations’ OR ‘beliefs’ OR ‘perceptions’ OR ‘cultural appropriateness’ OR ‘service models’ OR ‘cancer coordination’).

Studies that met the following criteria were included:

  • • Original, peer-reviewed articles published since 2004.
  • • Studies aiming to improve Indigenous health through programme evaluation or research in the area of cancer control [i.e. primary prevention, early detection and/or therapy (Australian Institute of Health and Welfare 2011), including tobacco control and smoking cessation]. Programme evaluation is taken to mean assessment of specific programmes, whereas research involves the pursuit of findings that are generalisable outside the studied group to other groups, times and contexts (Fain 2005).
  • • Studies that engaged the community in at least some sense, e.g. through the involvement of community members in the research process, through consultation with community-based health organisations or in the dissemination of results to the relevant community.

Green et al. (1995) have outlined a list of six main criteria comprising 25 sub-criteria (each scored from 0 to 4) for assessing participatory research funding proposals, and these form a broad standard against which to assess existing studies (see Table 1).

Table 1.  Guidelines for participatory research in health promotion (Green et al. 1995)
1. Participants and the nature of their involvement
 (a) Is the community of interest clearly described or defined?
 (b) Do members of the defined community participating in the research have concern or experience with the issue?
 (c) Are interested members of the defined community provided opportunities to participate in the research process?
 (d) Is attention given to barriers to participation, with consideration of those who have been under-represented in the past?
 (e) Has attention been given to establishing within the community an understanding of the researchers’ commitment to the issue?
 (f) Are community participants enabled to contribute their physical and/or intellectual resources to the research process?
2. Origin of the research question
 (a) Did the impetus for the research come from the defined community?
 (b) Is an effort to research the issue supported by members of the defined community?
3. Purpose of the research
 (a) Can the research facilitate learning among community participants about individual and collective resources for self-determination?
 (b) Can the research facilitate collaboration between community participants and resources external to the community?
 (c) Is the purpose of the research to empower the community to address determinants of health?
 (d) Does the scope of the research encompass some combination of political, social and economic determinants of health?
4. Process and context-methodological considerations
 (a) Does the research process apply the knowledge of community participants in the phases of planning, implementation and evaluation?
 (b) For community participants, does the process allow for learning about research methods?
 (c) For researchers, does the process allow for learning about the community health issue?
 (d) Does the process allow for flexibility or change in research methods and focus, as necessary?
 (e) Are procedures in place for appraising experiences during implementation of the research?
 (f) Are community participants involved in analytical issues: interpretation, synthesis and the verification of conclusions?
5. Opportunities to address the issue of interest
 (a) Is the potential of the defined community for individual and collective learning reflected by the research process?
 (b) Is the potential of the defined community for action reflected by the research process?
 (c) Does the process reflect a commitment by researchers and community participants to social, individual or cultural actions consequent to the learning acquired through research?
6. Nature of the research outcomes
 (a) Do community participants benefit from the research outcomes?
 (b) Is there attention to or an explicit agreement for acknowledging and resolving in a fair and open way any differences between researchers and community participants in the interpretation of the results?
 (c) Is there attention to or an explicit agreement between researchers and community participants with respect to ownership of the research data?
 (d) Is there attention to or an explicit agreement between researchers and community participants with respect to the dissemination of the research results?

Studies identified in the search were checked against Green et al.'s (1995) criteria by J. M. in order to assess the nature of community engagement, and the relationship of this engagement to community benefits. The total score for each article was divided by the number of sub-criteria that were assessable from information provided in the article. This resulted in a maximum possible score of 4 for each article. Authors V. K. and C. W. each assessed a random selection of studies (n= 5) against Green et al.'s (1995) criteria, and inter-rater reliability ranged from r= 0.84 (P= 0.08) to r= 0.96 (P < 0.05).

RESULTS

Searches yielded a total of 4891 initial results, which were assessed by J. M. for inclusion based on relevance to inclusion criteria. Thirty-nine articles reporting on 34 studies were selected for inclusion (see Fig. 1). These were checked by a second author (C. W.) and it was agreed by consensus that two articles should be removed, leaving 37 articles reporting on 32 studies (Table 2). Green et al.'s (1995) criteria were used to assess extent of community-based participation, as described above. Final scores ranged from 0.58 to 3.32 (M = 1.96; SD = 0.65).

Figure 1.

Flow chart of identified studies.

Table 2.  Review of community-engaging Indigenous cancer control studies
PaperGroup; locationMain purpose/research questionnFindingsResearch methodsScore (0–4)
  1. HPV, human papillomavirus.

Begay et al. (2007)Hopi Tribe; Arizona, USATo gain understanding regarding Hopi tradition, culture, health knowledge and protective behaviours in relation to breast and cervical cancer, and to describe the research process33Some insight into the relationship between Hopi culture and cancer, cancer screening and treatmentFocus groups2.08
Bottorff et al. (2010)Six small reserve communities identifying as Gitxsan or Wet'suwet’en; British Columbia, CanadaExploration of factors behind smoking in homes and attempts of First Nations women to minimise exposure for themselves and their children26Findings centred on the social factors behind smokingSemi-structured interviews; focus groups1.13
Boyle et al. (2010)Aboriginal people; Perth, Kalgoorlie, and Broome, Western AustraliaEvaluation of awareness of a mass media advertising campaign among Aboriginal smokers in Western Australia198Majority of participants had seen and/or heard of the campaignStructured interviews1.42
Cerigo et al. (2011)Inuit women; Nunavik, Quebec, CanadaTo investigate awareness and knowledge regarding Human Papillomavirus175Low level of awareness and knowledge (31% of the sample had heard of HPV)Surveys1
Christopher et al. (2008a,b)Women on the Apsáalooke Reservation; Montana, USA(1) To assess changes in a community following the introduction of a health education programme, and (2) to describe the research methodology83(1) Significant improvements from pre- to post-intervention in knowledge, comfort in discussing cancer issues and cancer awareness levels among Apsáalooke women; (2) subjective report of the benefits and difficulties of adopting the authors’ particular approach to community-based participatory researchIntervention trial3.07
Croager et al. (2010)Aboriginal health professionals; Western AustraliaDelivery and evaluation of a cancer education course for Indigenous health professionals35Self-rated cancer knowledge and confidence increased among participantsWorkshop evaluation2.36
Daley et al. (2010)Native Americans; Kansas and Missouri; USADiscussion of issues involved in carrying out focus groups in community-based participatory research519Subjective report of the benefits and difficulties of adopting the authors’ particular approach to community-based researchFocus groups2.78
English et al. (2008)Ramah Navajo women; New Mexico, USADiscussion of issues involved in carrying out community-based participatory research39Satisfaction ratings from participants, and subjective report of the benefits and difficulties of adopting this approach to community-based researchFocus groups; intervention3
Fernandez and Wilson (2008)Māori women; New ZealandQualitative assessment of effectiveness of smoking cessation initiatives5Participants focused on the importance of whānau values in relation to smoking behaviour, as well as perceived factors in smoking cessationFocus groups1.6
Gilligan et al. (2009)Aboriginal and Torres Strait Islander women attending a Far North Queensland health service; AustraliaTo identify factors underlying smoking during pregnancy, and to investigate smoking behaviour among pregnant Aboriginal and Torres Strait Islanders145Having a smoking partner, having an Aboriginal partner, and high levels of stress were related to increased risk for antenatal smokingStructured interview0.58
Grigg et al. (2008)Māori smokers and their friends, families or workmates; New ZealandTo assess effects of a Māori quit smoking campaign1128Seventy-eight per cent of Māori smokers and 73% of their whānau had seen and recalled the campaign 1 year after its launchTelephone surveys2.71
Haozous et al. (2010)Community health representatives serving Comanche women; Oklahoma, USAInvestigation into health–illness beliefs and barriers to breast health for a group of Comanche women7Identification of systemic and cultural barriers to breast screening'Talking circle’ (intertribal communication method similar to a focus group)1.73
Jacoby et al. (2008)Aboriginal and non-Aboriginal children; Kalgoorlie-Boulder region of Western AustraliaTo assess risk of otitis media associated with passive smoking in young children239Children exposed to passive smoking and not attending childcare had an increased risk of otitis mediaProspective cohort study; structured interviews and clinical examinations2
Johnston and Thomas (2008)Members of three Indigenous communities, and health staff; Northern Territory, AustraliaTo explore the reasons Indigenous people commence and continue smoking, and the barriers and facilitators to quitting38Insight into the historical, social, cultural, psychological and physiological factors underlying Indigenous smoking behaviourSemi-structured interviews; participant observation1.38
Kaholokula et al. (2008)Native Hawaiians; a rural community in HawaiiTo investigate barriers and facilitators to smoking cessation among Native Hawaiian former and current smokers52Identification of barriers and facilitators to smoking cessation, including some relating to family values, health awareness and religiosity/spiritualityFocus groups2.17
Ka’Opua (2008)Native Hawaiian women from four predominantly Hawaiian communities; HawaiiTo identify cultural factors behind breast screening participation60Identification of barriers to participation, including lack of knowledge, fear and concern over healthcare insurance'Talk story’ (described as a discussion with a more leisurely pace than a conventional focus group); interviews; surveys1
Kulukulualani et al. (2008)Native Hawaiians; HawaiiTo describe the methodology behind development of culturally targeted cancer education brochures for Native Hawaiians193Culturally targeted brochures were perceived to be more attractive than existing materials, with good post-test comprehension of materialsFocus groups; structured interviews2.17
MacLaren et al. (2010)Three Indigenous communities; Arnhem Land, AustraliaTo assess sensitivity and specificity of breath carbon monoxide (BCO) tests for validating self-reported tobacco smoking in Aboriginal Australians in Arnhem Land, NT400BCO cut-off of >5 parts per million provided good agreement between self-report and BCOStructured interviews1
Mishra et al. (2009)Samoan women; American Samoa, USATo assess efficacy of a cervical cancer education programme for Samoan women in American Samoa398Intervention group was significantly more likely to self-report Pap smear use at post-testIntervention trial3.32
Mokuau and Braun (2007)Native Hawaiian women with cancer and their families; HawaiiTo investigate types of family support provided, family role dynamics and recommendations for healthcare providers25Participants considered the family the most important source of support for women with cancer, and reported a need for more informational supportFocus groups2
Mokuau et al. (2008)Native Hawaiian women with cancer; HawaiiTo investigate feasibility of a culturally appropriate intervention for Native Hawaiian women and their families dealing with cancer10The authors reported that incorporation of Hawaiian values and practices was a strength. However, there was great difficulty recruiting Native HawaiiansRandomised control trial (pilot)2.25
O’Brien et al. (2009)First Nation Cree women; CanadaTo investigate attitudes and beliefs of First Nation Cree women regarding cervical screening8Participants felt they did not have adequate information on cervical screening. Discussion of cultural barriers to screeningParticipant observation; interviews2.5
Poudrier and Mac-Lean (2009)Aboriginal women; Saskatchewan, CanadaInvestigation of Canadian Aboriginal women's experiences with breast cancer12Exploration of Canadian Aboriginal women's experiences‘Photovoice’, a qualitative research technique2
Prior (2009)Aboriginal women in two communities in Queensland; AustraliaTo investigate reasons for Aboriginal women participating in screening but being reluctant to follow-up or accept medical advice48Rural Aboriginal women reacted fearfully and fatalistically towards cancer, and blamed the disease on colonisationParticipant observation; unstructured interviews; ethnographic fieldwork2.3
Puaina et al. (2008)Samoan men; Carson, California, and Pago Pago, American Samoa, USATo explore possible influence of the ‘Samoan way of life’ on cancer prevention practices60While participants did not perceive disease prevention to be a central part of the Samoan way of life, they reported that seeking input from leaders could improve prevention effortsFocus groups1.5
Read and Bateson (2009)Aboriginal and non-Aboriginal women; Dubbo, NSW, AustraliaTo obtain HPV status of Aboriginal and non-Aboriginal women43Forty-three women were recruited to screeningQuantitative data collection1.67
Reath and Carey (2008)Aboriginal women; three Northern Territory communities, AustraliaTo run and evaluate strategies to improve GP early detection of breast and cervical cancer in Aboriginal and Torres Strait Islander women23List of factors reported to contribute to project successQuantitative and qualitative data analysis1.57
Santos et al. (2008)Native Hawaiians; HawaiiTo describe the methodology behind a tobacco cessation protocol for Native HawaiiansN/ASubjective identification of necessary factors behind successful protocol, e.g. the need for long-term commitment to the protocolCase study2.23
Shahid et al. (2009a,b,c, 2010)Aboriginal people; Western AustraliaTo explore Aboriginal perspectives and experiences of cancer and cancer services in Western Australia, as well as documenting the study's methodology in detail37Issues identified included need for relationship-based and culturally appropriate research. Findings on the role of bush medicine in cancer treatment for Aboriginal peopleSemi-structured interviews, observations, field notes2
Varcoe et al. (2010)Six First Nations rural reserve villages; Gitxsan territory, BC, CanadaTo investigate influences on smoking behaviour among First Nations people66Older people expressed interest in being more involved in their communities. Others saw older people as important in addressing tobacco exposureFocus groups; interviews; participant observation2.36
Wardman et al. (2007)Aboriginal and non-Aboriginal smokers or recent ex-smokers; 12 First Nations communities, CanadaTo explore factors behind underutilisation of drug therapy among Aboriginal and non-Aboriginal smokers and recent ex-smokers509Many Aboriginal smokers interested in and/or attempting cessation. Underutilisation of physician services and lack of willingness to use drug therapySurveys1.6
Wood et al. (2008)Indigenous women; Perth, Western AustraliaTo investigate experience, knowledge, and attitudes regarding smoking during pregnancy among Aboriginal women50Greater understanding of factors behind smoking during pregnancy among Aboriginal womenFocus groups and semi-structured interviews2.29

Reporting of study findings was often highly subjective, making it difficult to establish the extent to which the research had achieved its stated goals. This was unsurprising given the often highly qualitative and exploratory nature of the included studies, but created significant problems in evaluating the quality of the research.

Evaluation of studies according to Green et al.'s (1995) criteria for evaluating community participation

Participants and the nature of their environment [1 of Green et al.'s (1995) criteria]

In 30 of the 32 studies, the community was defined clearly, although in 12 of these cases the definition was broad (e.g. ‘Canadian Aboriginal people’). Seven studies engaged community members who had concern or experience with the relevant issues, but despite this level of community involvement, there were no studies in which it was apparent that interested members were given opportunities to participate in the research process. Where community members were engaged, they were often established members of pre-existing community organisations, or drafted by the authors as research assistants, with no apparent possibility for others to participate if they wished. None of the studies addressed barriers to community participation in the research process. Thirteen studies attempted to establish an understanding of the researchers’ commitment to the issue within the community, through either promotional efforts including posters (Steuart 1993) and local media (Bottorff et al. 2010), presentations at community groups (Daley et al. 2010), or the attendance of researchers at community gatherings (Christopher et al. 2008b). In 17 studies, at least one community participant was able to contribute physical and/or intellectual resources to the research process, and in 15 cases Indigenous people were listed as authors. Sometimes, this involved significant contributions in areas of expertise; for example, in Daley et al. (2010) study, community members worked as paid research associates, paid community advisory board members and unpaid advisors from partner organisations. In other studies, community contribution was minimal and involved little more than researchers hiring a community member to carry out recruitment.

Origin of the research question [2 of Green et al.'s (1995) criteria]

For the majority of studies that engaged with community members, the impetus for research came from researchers (n= 24). One study was initiated by a community organisation (a tribal health programme), three were initiated by community members and six studies were initiated in collaborations between researchers and community members, organisations or both. Ten studies reported that their programmes were supported by the community.

Purpose of the research [3 of Green et al.'s (1995) criteria]

In eight studies, some degree of learning occurred among community participants regarding individual and collective resources for self-determination, for example, where participants were provided with opportunities to learn about research methodology (Daley et al. 2010) or behavioural skills training (Mokuau et al. 2008). Six studies facilitated collaboration between community participants and resources external to the community. There were nine studies in which it was apparent that the research was intended to empower the community to address determinants of health in some way, either as part of an intervention trial [as in English et al. (2008) trial of a mammography intervention], or through the participation of community members in the research process (e.g. Mishra et al. 2009). Seventeen studies encompassed some combination of political, social and/or economic determinants of health, although this was often simply a case of asking focus groups about the social influences on cancer-related behaviours.

Process and context-methodological considerations [4 of Green et al.'s (1995) criteria]

Of those studies that attempted some form of community engagement, 21 applied the knowledge of community participants in the phases of planning, implementation and/or evaluation. Application of community knowledge ranged from carrying out community focus groups to inform culturally appropriate curriculum development (Braun et al. 2008) to requesting research planning assistance from a community reference group (Johnston & Thomas 2008). Seven studies allowed for some learning related to research methods on the part of community participants. Seven studies clearly provided some opportunity for flexibility or change in research methods and focus. Procedures for appraising experiences during research implementation were apparently adopted in only one study. Finally, there were 11 studies in which community participants were involved in at least one aspect of the analytic process, either in interpretation, synthesis or verification. The extent of involvement ranged from member checking (Puaina et al. 2008) to consulting local elders in order to refine emerging research themes (Johnston & Thomas 2008).

Opportunities to address the issue of interest [5 of Green et al.'s (1995) criteria]

The potential of the defined community for individual and collective learning was apparent in 11 studies. The Daley et al. (2010) study was notable in terms of community learning, providing community members with human subjects training and the opportunity to write research grants, a skill which they were later able to use within their own organisations. Six studies reflected the potential of the defined community for action. In one example (Shahid et al. 2009a), the researchers supported a community woman to form an Indigenous Women's Cancer Support Group after the study's conclusion. There were seven projects in which the research process reflected a commitment by research and community participants to social, individual or cultural actions consequent to the learning acquired through research. These actions included presenting findings to a tribal council to assist in future grant writing (Begay et al. 2007), development of health education materials (English et al. 2008) and conducting a cervical cancer education intervention (Mishra et al. 2009).

Nature of the research outcomes [6 of Green et al.'s (1995) criteria]

In 11 of the 32 studies, community participants appeared to benefit from research outcomes. The nature of this benefit varied and included dissemination of results to the community (O’Brien et al. 2009), development of cancer education brochures (Kulukulualani et al. 2008), increased awareness of cervical cancer (Christopher et al. 2008a) and increased screening participation rates (Read & Bateson 2009). Such apparent benefits were categorised broadly by the reviewers as health benefits, interventions and capacity building/empowerment; these categories are referred to with more detail in the following paragraph. Attention to or explicit protocols addressing differences between researchers and community participants in regards to interpretation of results were apparent in five studies. In one example (Mishra et al. 2009), a conflict resolution mechanism was adopted to address a wide range of problems including those relating to data collection and data management. Only two studies explicitly addressed ownership of research data. Six studies outlined planned and actual dissemination of results to community members. There was no mention of explicit agreements between researchers and community participants with respect to dissemination of results, although this seemed to be implied in a small number of studies, for example, where participants took responsibility for aspects of dissemination (Daley et al. 2010).

Potential factors behind positive outcomes

The studies in this review pursued various goals through differing research designs, reported heterogeneously and generally with no assessment of statistical significance. There is therefore no single criterion of success against which each can be measured, and the authors of this review have tended to take qualitative reports of success at face value. The authors considered a range of positive outcomes, including increased community capacity and empowerment, enhanced screening participation rates and awareness of cancer-related services, and attempted to identify the factors with which these were associated. Of the 10 studies that resulted in increased community capacity and/or empowerment, six had engaged community members in basic aspects of the research process (e.g. facilitating focus groups or translation), and seven involved community members in more detailed aspects (e.g. planning and authorship). Indigenous steering committees were consulted in four cases. Studies resulting in increased community capacity were sometimes initiated by researchers alone (n= 4), and other times by a combination of researchers and community members or organisations (n= 5). In some cases the study incorporated an intervention (n= 5) and/or involved the provision of some health benefit to the community during the process of the research (n= 9), for example, by increasing screening participation (Read & Bateson 2009) or providing screening promotion (Reath & Carey 2008). Of the nine studies linked to positive health outcomes in the community, six were initiated by combinations of academic researchers and community members or organisations, while only three were initiated by researchers alone. Seven studies resulting in positive health outcomes involved the engagement of community members in a basic component of the research process (e.g. planning and authorship), and six engaged community members in detailed aspects of the research.

DISCUSSION

This evaluation of Indigenous community-based studies of cancer control has led to two main findings: a widespread willingness to draw on individual aspects of community engagement without adopting the approach comprehensively, and a possible link between certain aspects of community engagement and health benefits or capacity building.

First, each of the studies in this review engaged the community under study in at least some sense, but there were few that attempted to involve communities in the whole research process. This may be due to a range of factors. As Shahid et al. (2009a) have noted, many of the goals behind such studies are costly to pursue, and where researchers are hampered by a lack of funding, their capacity for prolonged community-based research is likely to be constrained. Researchers may also have experienced difficulty in securing community engagement, with Indigenous people potentially reluctant to be involved in research which has been historically harmful and inequitable. A further possibility is that some researchers lack awareness of community engagement philosophy, instead continuing to adopt more traditional research values that may be inappropriate to studies involving Indigenous communities. It is also important to consider the variety of contexts in which research programmes were carried out; different communities are already empowered to varying extents at the commencement of a research programme and the degree to which Green et al.'s (1995) capacity-building guidelines are relevant will vary accordingly.

Second, there was a possible link between aspects of community engagement and positive capacity-building or health outcomes. Studies resulting in increased community capacity or positive health outcomes often involved community members working in a research capacity, both in central aspects like planning and authorship, and in more basic aspects such as facilitating focus groups and translation. In terms of capacity building, part of the reason for this finding is obvious – through participation in the research process community members develop or improve skills (e.g. grant-writing) that may be of future benefit to their communities. The apparent relationship between community involvement and positive health benefits corresponds with Israel et al.'s (1998, 2005) philosophy in which community-based research ideally informs health-promoting activity. Community members may also have helped to culturally tailor research to participants, potentially improving participation rates, intelligibility of research goals and perceived relevance.

The impetus for the current projects (whether they were initiated by researchers, community members or both) did not appear to influence capacity-building outcomes. However, projects resulting in positive health outcomes were more often initiated by combinations of academic researchers and community members/organisations than by researchers alone. The reason for this is not entirely clear. In terms of the association between jointly initiated research projects and positive health outcomes, it is possible that health issues identified by community members or organisations are more likely to be perceived as priorities by research participants than issues decided by external research bodies, thus increasing participation rates and/or the relevance of concepts under study. Further investigation into this area could serve to identify the precise factors underlying the apparent relationship between joint project initiation and positive health outcomes, allowing researchers to prioritise these factors in future research.

Limitations

It is important to recognise that, although Green et al.'s (1995) criteria may constitute a gold standard, they are not universally appropriate to all community-based research. The implementation of community-based studies varies according to contextual factors, including the community's willingness to engage with research, restrictive funding timeframes, previous experience with research projects and existing levels of health knowledge.

Very few articles explicitly described the way in which communities were engaged or the precise benefits of studies to those communities. For example, Indigenous research assistants were engaged in several studies, but it was often unclear whether they were members of the relevant community or whether they had any experience with the issue. The lack of detailed reporting on these issues need not imply negligence; factors such as journal word limits may have prevented authors from addressing such aspects in detail. Nevertheless, without explicit attention on behalf of journals to in-depth reporting of the engagement process and health benefits, evaluation of the benefits of community-based research is likely to remain inadequate.

The breadth in definitions of ‘community’ is also worth noting. In some studies, the community was a clearly defined unit held together through social interaction and common values, whereas other studies described much broader units of membership. The vagueness of such broad units can be problematic. For example, where researchers define ‘Aboriginal and Torres Strait Islander people’ as the community of interest, while only recruiting participants from a specific community, consultation with a panel of Aboriginal researchers from outside that specific community is unlikely to constitute meaningful community engagement.

The designation of any study result as a ‘benefit’ is an inherently subjective exercise on the part of the researcher. The present review has been carried out by researchers from outside the communities in question, relying on evidence often presented from the perspective of non-community members. As such, our conclusions on the benefits of these studies may fail to represent the values of appropriate community members.

Furthermore, the extent to which community-based engagement leads to positive community outcomes may be overestimated in the current study as a result of the ‘file-drawer problem’ (Rosenthal 1979) (also known as publication bias) whereby journals neglect to publish papers with negative findings. This could lead to an inflated impression of the effectiveness of community engagement as a research approach. Paradoxically, it is only relatively recently that journals have emerged that are interested in publishing outcomes from evaluations or action research, tending in the past to see the threats to internal validity of such approaches as outweighing the concerns about external validity. It is impossible to be conclusive on the extent to which publication bias of either sort has taken place.

There are two conflicting views on the issue of evaluating quality in qualitative research. A number of research groups have composed criteria for the assessment of qualitative studies, with over 100 qualitative evaluation tools reported in the literature (Dixon-Woods et al. 2004). These address various aspects of quality including policy implications (Long 2004), description of data analysis processes (Boulton et al. 1996) and conventions for taking field notes (Spencer et al. 2003). Given the diversity of requirements for adjudicating quality, others have called the entire process of evaluation to question, arguing that qualitative research cannot be treated as a homogenous whole (Dixon-Woods et al. 2004). The argument is that structured evaluations of qualitative research are inappropriate and are not necessarily more useful than unstructured evaluations; inter-rater agreement is possible using the unstructured process without use of standardised checklists (Dixon-Woods et al. 2007). In an attempt to dispel the notion of a ‘quantitative-qualitative dichotomy’, Rolfe (2006) has argued for the evaluation of individual studies on their own terms.

While undertaking the present review, we examined the potential usability of a number of qualitative study assessment tools (Cesario et al. 2001; Spencer et al. 2003; Long 2004; Tong et al. 2007). Even the least prescriptive tool (Cesario et al. 2001) included a number of items that were impossible to assess in the majority of studies. For example, ‘heuristic relevance’ and ‘theoretical connectedness’ (Cesario et al. 2001) were rarely addressed by the studies in question.

The space constraints of most refereed journals ensure that most of the information required to judge quality of the research process is unavailable or is inferred, possibly inaccurately, from other things stated. Until space constraints are addressed, perhaps by the increasing utilisation of additional information provided via online appendices, assessing qualitative community-based participatory research for quality, particularly translation quality, will remain a difficult task.

Implications

At present, there is much room for improvement in community-based cancer control research. As a small number of these studies have shown, there is scope for community involvement at every stage of the research process. The challenge for further projects is not so much to ‘tick all the boxes’ as to strive for the highest possible level of engagement within a given context. By working through a series of criteria such as those of Green et al. (1995), researchers will be reminded to address the multifaceted nature of engagement, which can be so easily eclipsed by the everyday concerns of research. Such engagement may be beneficial to both researchers and communities, serving to increase participation rates, as well as facilitating positive outcomes in areas such as community capacity, health education, screening and prevention. Studies that stop at engaging an Indigenous translator or initially consulting with Indigenous health organisations fail to achieve the degree of mutual advantage that is possible within a design that genuinely engages the community.

ACKNOWLEDGEMENT

This paper reflects work conducted for Cancer Australia as part of its Aboriginal and Torres Strait Islander Cancer Control Research Project, an initiative aimed at reducing disparities in cancer outcomes for Aboriginal people. Further information can be found at: http://www.canceraustralia.gov.au

Ancillary