SEARCH

SEARCH BY CITATION

In this issue of JAN, Sandelowski’s (2008) paper (pp. 104–110) discusses the reading and writing practices that define systematic review. Sandelowski has been at the forefront of critical thinking of systematic review methodology and no doubt anticipates that her paper will ruffle some ‘methodological and organizational feathers’ and encourage important debate.

This paper and its propositions indicate just how far literature review methodology has moved on in the last 30 years. A far cry from my early experiences of tending to gravitate towards articles that supported the argument and resisting those that did not! Sandelowski suggests that resistance at a number of levels still exists but, once hidden within a systematized review process, we do not consider our thoughts or actions as resisting and, in taking an argument to its logical conclusion, we cannot see biases that are inherent in our methods, reading and decisions.

This paper is particularly relevant to me as JAN’s Reviews Editor and as lead convenor of the Cochrane Qualitative Research Methods Group (CQRMG). The group is constantly debating the issues and tensions highlighted in Sandelowski’s paper; and, within the Cochrane Collaboration, we are working vigorously to gain increased recognition of the value of more flexible and inclusive approaches to evidence synthesis.

All of the propositions put forward by Sandelowski are worthy of wider discussion but, for now, I have selected one to develop some debate, namely: ‘if what makes a review systematic is adherence to a protocol, what makes a review unsystematic is simply that it does not adhere to a protocol’ (p. 105).

This proposition fits well with the stance taken by organizations such as the Cochrane Collaboration, which undertakes systematic reviews of quantitative evidence to establish effects. The Cochrane Handbook describes a linear approach to protocol development. However, in defining what makes a review systematic – a comprehensive and planned search strategy that endeavours to locate all relevant research – is considered a key component, not just adherence to a protocol (Green & Higgins 2008).

The CQRMG have long argued that protocol development for a synthesis of qualitative evidence can be iterative and not linear, and that a review question should be ‘a compass and not an anchor’ (Eakin & Mykhalovskiy 2003). Likewise, many of the methods of analysis used to synthesize qualitative evidence have their origins in methods of primary data analysis. These techniques (such as thematic analysis) usually ascribe activities that move backward and forwards through various stages and cycles in a non-linear way. Thus, one could argue that a qualitative evidence synthesis protocol can if appropriate be iterative and flexible – and still presumably can be ‘systematic’ as the process is underpinned by a method.

Many Cochrane reviews would arguably have greater utility if a synthesis of qualitative evidence could be incorporated to explore questions beyond effect size, such as experience of the intervention by the person receiving it. Exemplars have been published in JAN (Noyes & Popay 2007).

The Cochrane Collaboration supports synthesizing qualitative evidence that is associated with included trials, such as process evaluations, but the arguments for moving towards inclusion of supplemental qualitative evidence syntheses and flexible iterative protocols go well beyond methodological ideology and include branding, organizational focus, software issues and resource constraints. Thus, maybe the concept of ‘resisting organizations’ (even if ‘resisting’ by adhering to organizational objectives) could be added to the debate alongside ‘resisting’ readers and texts? I look forward to debating the issues raised by Sandelowski and invite contributions to JAN Forum.

Acknowledgement

  1. Top of page
  2. Acknowledgement
  3. References

I thank Sally Green and Andrew Booth for their comments on drafts of this Editor’s Choice.

References

  1. Top of page
  2. Acknowledgement
  3. References