Evidence-based policy: a realist perspective
Article first published online: 3 MAY 2007
Australian and New Zealand Journal of Public Health
Volume 31, Issue 2, pages 193–196, April 2007
How to Cite
(2007), Evidence-based policy: a realist perspective. Australian and New Zealand Journal of Public Health, 31: 193–196. doi: 10.1111/j.1753-6405.2007.00043.x
- Issue published online: 3 MAY 2007
- Article first published online: 3 MAY 2007
By RayPawson . Published by Sage Publications , London , 2006 . Paperback , 196 pages with index . RRP $66 . ISBN 1 4129 1060 9 .
Reviewed by Libby Kalucy, Department of General Practice, Flinders University, South Australia
I read this book with considerable anticipation and I was not disappointed – it is a challenging, stimulating, and enlightening volume that focuses on a different way of making sense of information about complex issues. I was receptive partly because of frustrating recent experience with standard syntheses of complex material in primary health care and partly because of my experience with an earlier book by Ray Pawson and Nick Tiller (Realistic Evaluation, Sage, 1997). I found the question ‘What works for whom in what circumstances and why’ very useful to make sense of the complicated evaluation I was then undertaking, although realistic evaluation methods were difficult to apply in my situation in 1998.
Evidence-based policy: a realist perspective outlines in considerable detail realist synthesis of evidence for policy. Ray Pawson is a distinguished sociologist with a background in criminology who is now reader in social research methodology at the University of Leeds. He contributed to one of the three groups working on different methods of synthesising evidence for policy, jointly funded by the NHS Service Delivery Organisation and the Canadian Health Service Research Foundation. (For comments by policy makers on the three forms of synthesis, see the special supplement to the Journal of Health Services Research, July 2005.)
The practical challenges of achieving evidence-based policy are closely allied with the challenge of demonstrating research impact, which will occupy much thought and effort as Australian researchers tackle the Research Quality Framework in the next two years. To incorporate the findings from a single research project into ‘the evidence’ requires a systematic, transparent process. The existence of the Cochrane Collaboration demonstrates an approach that is well developed for pooling results from clinical treatments, but works much less well for synthesising the results of active interventions to active subjects, with long chains dependent on cumulative effects of many interactions between mechanism and context.
This book offers an alternative approach to the standard systematic review for social interventions where, in the author's words, complex systems are thrust amidst complex systems. The realist approach works by policy abstraction and theory building, not data extraction and number crunching. It seeks not to judge but to explain “what works for whom in what circumstances and in what respects”. It produces explanations rather than evidence. Realist synthesis should eliminate the expectation that synthesis will provide the solution to the problem. Pawson considers that the realist approach supplements the breakthrough quality improvement approach to organisational learning (of which there are several examples in Australia, such as the National Primary Care Collaborative Program) by thinking through the configurations of context and mechanism that need to be attended to in finetuning a program.
The book starts with the role of systematic reviews in evidence-based policy, then introduces realist methodology as a philosophy of social inquiry and a tool to dissect how social interventions work. The title of the third chapter indicates the author's opinion of systematic reviews: ‘Systematic obfuscation: a critical analysis of the meta-analytic approach’. In his words, this dominant model is charged with providing mistaken answers to misplaced questions. Pawson then introduces the principles and practice of the new realist synthesis model, followed by three chapters illustrating realist synthesis in the United States sex offender registration and community notification program, youth mentoring programs and ‘naming and shaming’ interventions (a chapter quite relevant to Australia's proposals for introducing hospital league tables). The final chapter is a very useful summary of what realist synthesis may be able to achieve in infiltrating the evidence base.
Given the need for systematic reviews to reduce the vast amounts of black and grey literature for use and application in public health, and the limited value of so many of these reviews despite the best efforts of talented people, this approach is much needed. I would strongly recommend the book to anyone undertaking or commissioning a review of the evidence, especially about complex social interventions, and to those who use such reviews to inform their decisions. The book requires concentrated reading, but the style is appropriate to the ideas conveyed, it is written very well and illustrated effectively with tables and models, as well as with examples from reviews. Pawson is modest in his claims for this approach, which is still developing, and takes pains to avoid suggesting that realist synthesis is simple. The book has its own website with supplementary readings at http://www.leeds.ac.uk/realistsynthesis/