Measuring knowledge of natural selection: A comparison of the CINS, an open-response instrument, and an oral interview

Authors

  • Ross H. Nehm,

    Corresponding author
    1. College of Education and Human Ecology, Department of Evolution, Ecology, and Organismal Biology, The Ohio State University, 333 Arps Hall, Columbus, Ohio 43210
    • College of Education and Human Ecology, Department of Evolution, Ecology, and Organismal Biology, The Ohio State University, 333 Arps Hall, Columbus, Ohio 43210.
    Search for more papers by this author
  • Irvin Sam Schonfeld

    1. School of Education, Department of Psychology, The City College, City University of New York, New York, New York
    Search for more papers by this author

Abstract

Growing recognition of the central importance of fostering an in-depth understanding of natural selection has, surprisingly, failed to stimulate work on the development and rigorous evaluation of instruments that measure knowledge of it. We used three different methodological tools, the Conceptual Inventory of Natural Selection (CINS), a modified version of Bishop and Anderson's (Bishop and Anderson [1990] Journal of Research in Science Teaching 27: 415–427) open-response test that we call the Open Response Instrument (ORI), and an oral interview derived from both instruments, to measure biology majors' understanding of and alternative conceptions about natural selection. We explored how these instruments differentially inform science educators about the knowledge and alternative conceptions their students harbor. Overall, both the CINS and ORI provided excellent replacements for the time-consuming process of oral interviews and produced comparable measures of key concept diversity and, to a lesser extent, key concept frequency. In contrast, the ORI and CINS produced significantly different measures of both alternative conception diversity and frequency, with the ORI results completely concordant with oral interview results. Our study indicated that revisions of both the CINS and ORI are necessary because of numerous instrument items characterized by low discriminability, high and/or overlapping difficulty, and mismatches with the sample. While our results revealed that both instruments are valid and generally reliable measures of knowledge and alternative conceptions about natural selection, a test combining particular components of both instruments—a modified version of the CINS to test for key concepts, and a modified version of the ORI to assess student alternative conceptions—should be used until a more approprite instrument is developed and rigorously evaluated. © 2008 Wiley Periodicals, Inc. J Res Sci Teach 45: 1131–1160, 2008

Ancillary