• Open Access

Research methods – most common reason for rejection


  • The Editors

Journal contact details
Mail: Australian and New Zealand Journal of Public Health, PO Box 351, North Melbourne, Victoria 3051.
Street deliveries: c/- SUBStitution Pty Ltd, 1st Floor, 484 William Street, Melbourne, Victoria 3003.
Phone: (03) 9329 3535 Fax: (03) 9329 3550

In 2010, this Journal faces a new dilemma. Manuscript submission rates to the Journal increased markedly over the past year. The standard of submitted papers has also improved. The fortunate aspect of this is that we can continue to increase the academic quality of the Journal, but the flip-side is that our rejection rate must rise. In the past few years the rejection rate has increased from 30 per cent to 50 per cent and it will need to be higher still. In order not to overburden the administrative staff, the Editors and our long-suffering panel of reviewers, we anticipate more stringent criteria for acceptance. More papers will be rejected without review, after review, and, very importantly, following an unsatisfactory response to reviewers' comments in revised papers. With such changes, it is important to be clear about our decision making.

One obvious reason for rejecting a paper is that it reports findings that are not particularly novel, and do not advance our understanding of public health issues or how they might be better addressed. Apart from this, the most common reason for rejecting a paper is because of research method. Here we provide a bit more detail about the methodological considerations for accepting or rejecting papers.

Public health is a multidisciplinary activity: we draw on the skills and insights of people working in a broad range of increasingly diverse sub-specialities. Our editorial process and decision making has to be cognisant of the highest current standards in each of the component disciplines. However, in addition, we serve a broad public health practice community, and many of our readers are interested in whether the results of a research study can be applied with confidence, for example, in the patients in a general practice, in communities in a suburb, with other vulnerable groups, to the whole Australian or New Zealand population, or internationally. Can health policy makers here or elsewhere be confident that they can base policy decisions on its conclusions? In order to contribute to the public health knowledge base and, at the same time to satisfy the needs of practitioners, we need to know that the design and conduct of any study justifies the conclusions reached in the paper. We also need to know to what extent the findings apply to groups or settings other than those in the study. Authors need to make these arguments, and make them accessible both to readers who are familiar with an area and research style but also (and arguably more importantly) to readers for whom this is unfamiliar territory.

In the ideal case this is easy. In studies with a large population, careful sampling, well-developed research methods and a good response rate, we may be confident that the results are trustworthy and that they are likely to apply not only to the sample but to the whole population, and sometimes even to a national population. Moreover, if it is a well-conducted study and it tackles an important question, a negative finding is no barrier to publication in this Journal.

This ideal may be unattainable. In public health research there are often methodological, practical and ethical limitations to the way in which is study is conducted. If authors can demonstrate that the study addresses an important issue that is under-researched, or difficult to research, a paper with less than ideal methods and outcome is still eminently publishable. However, we do need authors to acknowledge in the methods section of the paper that they faced methodological problems and to show how they addressed them. In the discussion section, authors should acknowledge limitations and conclusions should be explicit about the extent to which the results are generalisable to population groups other than those in the study.1

Given the range of skills of the team of Editors, we are well able to judge articles that depart from the methodological norm but that still make an important contribution to public health knowledge. We have published single case studies, for example, including one that alerted us to the risk that a strain of Legionella bacteria usually associated with cooling towers may also be found in the soil in a plant nursery. The point here is that the single case was an exception to a well-established knowledge base and the paper argued that we need to be alert to this exception.2

Methods for systematically reviewing the literature have improved greatly and systematic reviews are, of course, recognised as research projects in their own right. The results of individual studies can now be assessed in the light of the systematic review of similar studies. It is even possible to combine quantitative studies, as a block or by temporal accrual, regardless of individual effect size, for more robust and statistically significant results. Systematic reviews of the literature are also useful for demonstrating that there is a great deal, a little, or nothing in the way of good evidence in a particular field, thus justifying a new study, ideally designed to overcome the limitations of previous research.

Central to the methods of systematic review is the rule that researchers should state the methods used in their serach of the literature and specify the criteria for assessing the quality of studies included in the review. Defining criteria for judging the quality of a study is not contentious in most of the health sciences but, in the social sciences, there is a vigorous debate about whether this restricts our knowledge base — and many of our authors share these views. Norman Denzin, an influential US-based qualitative researcher, has argued persuasively that specific criteria for judging the quality of qualitative research are to be resisted as part of a restrictive ‘global audit culture’. He emphasises the need for ‘a narrative of passion, and commitment, a narrative which teaches others that ways of knowing are always already partial, moral and political.’2 We are sympathetic to Denzin's views. But where does that leave editors, reviewers and readers who are perhaps less versed in the sometimes esoteric narratives that underpin Denzin's approach to research? Our view is that the practical intent of public health research means that we need to know to what extent the findings from qualitative research are valid and apply to similar groups elsewhere or to other social groups. We are aware of the desire of researchers to be true to the voices of their participants, but, at the least, we need an account of the experiences of the full set of participants and not just those who express their views in a moving narrative.4

And so to return to the future of the Journal. We need to see plainly the methods our authors used in their research — regardless of the kind of papers they are writing, conclusions need to be underpinned by valid, solid, ethical research. In addition, we are pleased to consider provocative but well-substantiated commentaries that challenge us to think about contemporary practice of public health in Australia, New Zealand, and internationally. Given the pressure on journal space, smaller word counts are preferred unless a large word count is justified because of the topic or the method. Tables with superfluous data will come under particular scrutiny. Our necessarily tightened criteria will lead to some unhappy authors, but will also bring you a richer, tighter, more scientific journal.

In this issue

This month we summarise the contents of the journal under three theme headings, bringing you papers about Policy issues, Indigenous issues, and Lifestyles choices.

The Policy section begins with an observation study from William McBride and colleagues about the best way to comprehensively screen arriving international passengers and to follow up those with fever, showing that symptomatic people generally do not follow up advice to consult a doctor and that other public health measures are needed. The second paper, from Wendy Hoy and colleagues, addresses the need for screening program delivery for Indigenous people in remote settings. In reporting the short term program outcomes, they note the need for a whole-of-life approach as well as some of the many challenges this produces in the long-term. Next, it being summer, Jill Sherriff and colleagues have turned to the cricket and analysed food and alcohol sponsorship by quantifying the length of time each main sponsor's logo was identifiable during telecasts (answer: 44-74% of the broadcast), vastly more than during paid advertising time. The last paper in this section draws from Robin Burns and colleagues on the results of a series of focus groups designed to identify effective ways to communicate with the public during emergencies. Undertaken after the 2004 Victorian bushfires, the study shows some of the issues identified by the public then were repeated in 2009.

Indigenous issues make a regular appearance in this journal. In Auckland, John Sluyter and colleagues undertook a dietary intake study by means of a food frequency questionnaire among teenagers and young adult high school students, showing some interesting differences in the various ethnic groups with implications for dietary advice, especially in Māori and Pacific Islander adolescents. Katie Panaretto and colleagues have examined the ability of ATSI medical services to deliver brief interventions within the SNAP (smoking, nutrition and physical activity) framework concluding that staff time is already overcommitted and training not flexible enough, making important barriers to service delivery. An interesting and robust multi-method of measuring tobacco consumption is evaluated by a team led by David Thomas. Lixin Ou and colleagues compared health service use and the health status of Indigenous and non-Indigenous infants, with lower SES and fewer clinic visits being associated once again with poorer health outcomes. Delvina Gorton's team surveyed adult shoppers to find out how they interpret the nutrition labels on food items, finding that misinterpretation is common, and is an especially difficult issue for indigenous New Zealanders.

In Lifestyle issues, Bridget Kool and colleagues have undertaken a study of the prevalence of risk factors for falls in a random sample of young and middle aged Auckland residents, suggesting that lower SES groups have more structural risks for falls than their wealthier neighbours. A group headed by Marcus Chen has considered how to estimate the number of unlicensed brothels in Melbourne by undertaking a systematic advertisement scan, and considered the implications for the mainly women workers in this section of the sex industry. Lifestyle is a predictor of various chronic diseases, and some of these are the subject of our last section. Ben Cowie has lead a study of markers for hepatitis B, a disease that receives little attention since the introduction of a universal immunisation program, in population blood samples. There are important differences by age, sex, and area of residence, and the introduction of HBV vaccine has made a lower than expected difference to chronic carriage. Neil Parker and colleagues have also used blood samples, this time to study Q-fever markers, and show that this underrated infection is relatively common in rural South West Queensland, again with age and sex differences. Physical activity, or lack of it, is the subject of Michael Rosenberg and Lisa Wood's paper. They conducted a natural experiment studying the effect of the introduction of daylight saving in WA on exercise patterns.

A previously published paper on indigenous views of cancer (2009;33:109-118) has generated a letter about and response from the authors. Another five letters address various issues of importance and include a number of snippets of interest. Lastly, three reviews are about books which further address our Policy and Lifestyle sections this month.