SEARCH

SEARCH BY CITATION

Abstract

  1. Top of page
  2. Abstract
  3. Priority Areas for Education Research
  4. Preferred Study Designs
  5. Myths of Education Research
  6. Conclusions
  7. References

In this commentary, common misperceptions about education research, and specifically for emergency medicine education research, are addressed. Recommendations for designing and publishing high-quality projects are also provided.

Academic Emergency Medicine (AEM) encourages the submission of high-quality educational advances and education research manuscripts. Since the 2012 AEM consensus conference “Education Research in Emergency Medicine: Opportunities, Challenges, and Strategies for Success,”[1, 2] which galvanized the educator community to work on multiple research agendas, 12 education manuscripts have been accepted for publication in the journal. While we anticipate even more output in the coming months, we present this commentary to further clarify priority areas of education research and introduce our new instructions for authors for education submissions. We also address several myths in education research.

Priority Areas for Education Research

  1. Top of page
  2. Abstract
  3. Priority Areas for Education Research
  4. Preferred Study Designs
  5. Myths of Education Research
  6. Conclusions
  7. References

Acdemic Emergency Medicine is devoted to advancing the science of medical education by publishing rigorously developed and piloted innovations and well-done education research studies. To be considered for publication, both submissions must add to the current knowledge by addressing clearly defined needs or questions and the results must have the potential to affect the educational practice of AEM readers. We are particularly interested in advancing the agenda put forth in the 2012 consensus conference issue (http://onlinelibrary.wiley.com/doi/10.1111/acem.2012.19.issue-12/issuetoc), which describes the most pressing research needs in eight different areas involving educational methods, assessment, training, and infrastructure. We will be prioritizing publication of work in these areas.

Preferred Study Designs

  1. Top of page
  2. Abstract
  3. Priority Areas for Education Research
  4. Preferred Study Designs
  5. Myths of Education Research
  6. Conclusions
  7. References

Academic Emergency Medicine aims to report rigorously conducted scholarship. Most often, educational scholarship falls into one of the following four areas: discovery, teaching, integration, or application.[3] We will consider submissions that address any of these four broad areas of scholarship. All submissions should adhere to the “author guidelines” as listed on the AEM website.[4] Studies must add to the current knowledge and meet the FINER acronym criteria: Feasible, Interesting, Novel, Ethical, and Relevant.[5] The categories of educational submissions that will be considered by AEM include:

Educational Innovations

While AEM does not in general publish curricula without outcomes data, we will consider educational innovations that are timely or meet pressing needs in emergency medicine (EM) education. To be considered, innovations must be developed using an established method of instructional systems design and report at least feasibility and acceptability data. Well-done needs assessments will also be considered for publication.

Systematic Reviews and Meta-analyses

Systematic reviews and meta-analyses that demonstrate the scholarship of integration and use established methods will be considered.[6]

Validity and Reliability Studies

Studies that collect evidence to determine validity and reliability relating to scores of psychometric instruments (such as questionnaires, assessment instruments, or observer rating scales) used in education will be considered for publication. All submission should use a framework for validity that considers evidence from five sources: content, response process, internal structure, relations to other variables, and consequences.[7-9]

Descriptive Studies

While descriptive studies are rarely accepted for publication, a study that aims to describe a phenomenon in education will be considered for publication if the question is relevant and the methods are strong, and a descriptive study is the best tool for measuring the outcome of interest. We are particularly cautious about survey designs, which are well suited to measuring perceptions and attitudes, but can be subject to numerous sources of error and bias if not developed, piloted, and implemented in a rigorous fashion. The survey instrument would need to be rigorously developed, piloted, and ideally validated if it were to meet the criteria for publication in AEM.[10-12]

Experimental Studies

Studies that aim to test the effect of an educational intervention using quasi-experimental or experimental designs are encouraged by AEM. Studies with higher-order outcomes on the modified Kirkpatrick hierarchy are most favorably considered, such as long-term effects on learner behavior and practice, as opposed to learner satisfaction and short-term gains in knowledge.[13] Studies demonstrating the highest level of outcome, that of benefit to patients, are strongly sought-after as well.

Qualitative Research

Studies that aim to explore little-understood phenomena or generate hypotheses using qualitative methods will be considered for publication. The methods should adhere to best practices for qualitative research in medical education.[14-16]

Authors must adhere to good scientific writing guidelines. One editorial that can serve as a guide is cited here.[17]

Myths of Education Research

  1. Top of page
  2. Abstract
  3. Priority Areas for Education Research
  4. Preferred Study Designs
  5. Myths of Education Research
  6. Conclusions
  7. References

Misperceptions regarding education research abound. We have compiled and address these below. References are included in each area for further review, with the intended audience being those who have not had formal training in research methods, yet may be clinician-educators motivated to publish scholarly work. Noneducation EM researchers may appreciate learning about these challenges, which can differ from those in more clinical research fields. We believe the top 10 myths of education research in EM to be:

  1. “You should study that.” Reality: Not all educational interventions need to be studied in a research format. The mentality that every educational intervention must be studied can create unintended consequences—such as the sense that preparing to study the intervention, and gathering preintervention data, would take too much effort, so the intended curriculum never materializes. At times, to “study” a curriculum, one would need to look at an outcome that is not educationally relevant. Well-meaning researchers may give this advice, not realizing that the result is already known. Finally, educators are continually implementing so many curricula that one cannot, and should not, study them all.[3, 18]
  2. “You should publish that curriculum in a research journal.” Reality: Some steps of curriculum development can be well suited to publication, such as needs assessments or instrument validity studies. However, entire curricula are best-suited to online publication in venues such as MedEdPortal. MedEdPortal can also be a venue for smaller curriculum interventional studies, such as single-site pilots of new instructional methods. AEM has a dedicated editor at MedEdPortal who can assist EM educators in getting their work published through this avenue: https://www.mededportal.org/partner/currentpartners/saem.
  3. “Educators need to publish investigative research to be promoted.” Reality: Many institutions' promotion committees now recognize the Boyer categorization of four types of scholarship, only one of which is investigative research. Educators can adequately demonstrate scholarship by producing new knowledge through teaching, integration, and application, and research should not be seen as the only required path to scholarship or promotion.[3]
  4. “You can't be a serious education researcher unless you are grant-funded.” Reality: Although funding can lead to higher-quality studies, grant funding for education research is rare.[19-21]
  5. “Every study should aim for patient-related outcomes.” Reality: This approach is often appropriate, and a good goal to aim for, but sometimes inappropriate and can actually weaken the study with potential confounders. While a change in outcome may be demonstrated, if the reason for the change is not clearly identified (due to dilution, multifaceted interventions, etc.) then the educational science is not actually advanced.[22]
  6. “A randomized controlled trial is the criterion standard study design.” Reality: This study design is optimal for noneducation research, but often inappropriate for education research designs. Quasi-experimental variations can be a better fit, and some studies that are extremely important in education research, like instrument validity studies, use different methods altogether.[22, 23]
  7. “Qualitative research is good if you don't have a lot of research resources, but journals are biased against it.” Reality: Conducting well-done qualitative research is just as rigorous and resource-intensive as using a quantitative study design; a “short-cut” or sloppy qualitative study has just as little chance of publication as a quantitative study of similar rigor. AEM welcomes high-quality, methodologically sound qualitative studies. The study design should be prompted by the research question. If the question is exploring a little-understood phenomenon, or generating hypotheses, qualitative methods are more likely to answer the question. If the aim is to test a hypothesis or establish statistical correlation or causality, quantitative methods should be used.[14-16]
  8. “Survey studies are a quick and easy way to perform research.” Reality: Survey studies can be labor-intensive and require considerable expertise to design an instrument that minimizes sources of error. Survey studies are not the best design for certain educational outcomes; while survey studies can be used for quantifying perceptions, they are less ideal for assessing knowledge and are poor at documenting behaviors. As a result, we rarely publish surveys. These designs are also inferior to qualitative research for generating hypotheses as to why perceptions occur. Surveys also require a committed study population that provides a good overall response rate (ideally over 70%) and complete, well-thought-out answers. Semiqualitative methods incorporating standardized questions plus open-ended discussion with learners may give more robust results.[10-12]
  9. “The majority of the introduction should be a literature search about the educational intervention.” Reality: The introduction of an educational manuscript should focus on the conceptual framework for education research studies, cite the most pertinent literature, and set up the research problem, question, or hypothesis.[24]
  10. “There is no way to measure quality in education research.” Reality: There is no one measure of quality, but several instruments exist to guide the educator in planning and executing quality scholarship. The medical education research study quality instrument (MERSQI),[19] which has some validity evidence, and the Newcastle-Ottawa Scale (NOS),[25] are some tools that can guide authors in their study preparations.[26] Researchers are encouraged to score their planned studies using these scales to identify in advance any potential areas of methodologic weakness that may need to be redesigned.

Conclusions

  1. Top of page
  2. Abstract
  3. Priority Areas for Education Research
  4. Preferred Study Designs
  5. Myths of Education Research
  6. Conclusions
  7. References

This article aims to guide both the education researcher who may not have had formal training in methods and the noneducation researcher who may not be aware of the specific issues related to education studies. Researchers should consider these points in the study design phase of their work to ensure the highest-quality product possible. Potential authors are also always welcome to contact the editors with any queries about submitting education research papers to Academic Emergency Medicine. AEM welcomes the continued submission of high-quality education research.

References

  1. Top of page
  2. Abstract
  3. Priority Areas for Education Research
  4. Preferred Study Designs
  5. Myths of Education Research
  6. Conclusions
  7. References