Critical appraisal of qualitative research in clinical journals challenged
Version of Record online: 7 APR 2005
Copyright © 2005 by the American College of Rheumatology
Arthritis Care & Research
Volume 53, Issue 2, pages 314–316, 15 April 2005
How to Cite
Sale, J. E. M. and Hawker, G. A. (2005), Critical appraisal of qualitative research in clinical journals challenged. Arthritis & Rheumatism, 53: 314–316. doi: 10.1002/art.21088
- Issue online: 7 APR 2005
- Version of Record online: 7 APR 2005
- CIHR Post-Doctoral Fellowship
- CIHR Scientist Award
To the Editor:
The appearance of qualitative research in clinical journals has been increasing over the last 20 years as evidenced by the number of citations in online databases. A quick search using the keyword “qualitative research” in Medline (the word only became a MESH term in 2003) yielded 5 hits in 1983 and 792 hits in 2003. Unfortunately, based on one author's (JS) experience with several qualitative manuscripts in circulation, editorial review boards appear to be lagging in their ability to appropriately critically appraise articles of a qualitative nature. (These manuscripts were submitted to journals with impact factors of ≤2 that demonstrated receptivity to qualitative research in their author guidelines.) We would like to respond to 3 criticisms not generally addressed by critical appraisal criteria guidelines for qualitative researchers and to clarify the misconceptions surrounding these criticisms.
It is acknowledged that the need for critical appraisal criteria in qualitative research is controversial partly because there are no universal criteria; furthermore, there is much debate about criteria proposed between, and within, various qualitative traditions. What is not disputed is that critical appraisal criteria differ for qualitative and quantitative research, and that these criteria have little to do with reliability and validity (1–5). While we are not arguing for, or against, specific critical appraisal criteria applied to qualitative research, there are common elements to these criteria that need to be presented for the integrity and status of qualitative research to be promoted in the clinical realm.
The term “generalizability” as it is typically understood does not exist in qualitative research. Qualitative research ers refer to terms such as “transferability,” (the ability to transfer findings from one context to another ), or “theoretical generalization,” (the ability to produce theoretical explanations about a phenomenon or the ability to contribute to a theory ). In qualitative research, decisions around sample size and sampling procedures concern the data collected, not the participants involved in the study; there is little emphasis on generalizability based on the demographics of the participants. One recruits as many participants (or cases or observations) as is considered necessary to illuminate the phenomenon of interest, not to generalize to a particular population of interest (8, 9). For example, a sample size of 10 may be adequate for critical case sampling but too large for narrative analysis and too small to develop a theory (10). Therefore, it is inappropriate to criticize a study on the basis of sample size and its inability to generalize to a particular population.
The role of the literature review varies in qualitative research. In some traditions, such as phenomenology, it is acceptable to conduct the literature review before and/or after data are collected and analyzed (11). Renown grounded theorists Glaser, Strauss, and Corbin (12, 13) actually discourage extensive consultation with the literature, believing that the researcher can become constrained or stifled by it. This does not mean that qualitative researchers should not conduct a literature search to ensure that their study is novel and will make a contribution to the field. However, often, much of the literature review is conducted after the study findings have been developed. At this point, the purpose of the review is to situate, contextualize, or link the findings to what others have found (14). In other words, the findings in qualitative research may determine the direction and content of the literature review; therefore, the placement of the literature review depends on the strategy used by the author. While some journals (e.g., Journal of Advanced Nursing) state in their author guidelines that the literature review can be placed in the introduction, the findings, and/or the discussion of the manuscript, this practice does not suit the conventional formatting of most clinical journals. This can be a difficult challenge for qualitative researchers to overcome because it encourages 2 discussions of the literature (making it difficult to not exceed word limits that are already stringent from a qualitative perspective) or it forces them to focus on an introductory literature review over a discussion of the literature based on study findings.
Qualitative researchers are not viewed as detached, objective entities but as individuals who actively participate in, and shape, the research findings (1). As a result, the use of the first person in presenting qualitative research is conventional (1, 15). This practice, a requirement of the American Psychological Association 5th Edition Publication Manual guidelines (the writing style used by many qualitative researchers), translates to the use of “I” for single authors. Some qualitative studies such as phenomenologic studies are commonly undertaken by single researchers and thus are reported by only one author. The writing style of these manuscripts may appear to be informal but with many clinical journals requiring the active voice in their writing styles, the use of “I” is difficult to avoid.
The above criticisms present additional challenges for qualitative researchers who wish to publish in clinical journals. Although clinical journals pay “lip service” to being receptive to qualitative papers, review boards of these journals appear to be lacking in qualitative membership and reviewers continue to apply inappropriate critical appraisal to these studies. Unfortunately for qualitative researchers, the impact factor of journals dedicated to qualitative research (e.g., Qualitative Report, Qualitative Inquiry) is low or nonexistent, which means that qualitative research is still considered a second-class citizen in the clinical realm. Ironically, we have had little difficulty receiving ethical approval for qualitative studies or acquiring the funds to conduct them. However, the publication challenges mentioned above have prompted us to question the ease with which ethical review boards grant approval, the utility of receiving funding for these studies, as well as the burden to individuals who participate in qualitative research. It is time for qualitative researchers to be more proactive about addressing these, and other, critical appraisal challenges so that they are more successful in their submissions to clinical journals.
Dr. Sale is supported by a CIHR Post-Doctoral Fellowship; Dr. Hawker is supported by a CIHR Scientist Award.
- 4Evaluating qualitative research. Qual Health Res 1991; 1: 283–6..
- 6Naturalistic inquiry. Newbury Park (CA): Sage Publications; 1985., .
- 7Dictionary of qualitative inquiry. 2nd ed. Thousand Oaks (CA): Sage Publications, Inc.; 2001..
- 11Qualitative inquiry and research design: choosing among five traditions. Thousand Oaks (CA): Sage Publications; 1998..
- 12Discovery of grounded theory: strategies for qualitative research. Chicago: Aldine; 1967., .
- 13Basics of qualitative research: techniques and procedures for developing grounded theory. 2nd ed. Thousand Oaks (CA): Sage Publications; 1998., .
Joanna E. M. Sale PhD*, Gillian A. Hawker MD, MSc, FRCP(C)*, * Sunnybrook & Women's College Health Sciences Centre and University of Toronto, Toronto, Ontario, Canada.