Evaluating technology-, information literacy-, and content-related learning outcomes among undergraduate students in face-to-face and social networking environments



Recruitment and retention of women and students of color in science, technology, mathematics, and engineering (STEM) disciplines remain challenging for secondary and post-secondary educational institutions. Research into classroom experiences of primary and secondary school students suggest that ethnic and gendered communication styles impact student learning in instances in which instructors have misaligned expectations of student participation and performance (Gay, 2000). Cultural differences in identity construction among African American, Latino, American Indian, and Asian Pacific American students also have profound effects on the ways in which students from various racial and ethnic groups form attitudes about academic subjects in this environment (Tatum, 2003). Changes in the ways that STEM topics are taught have the potential not only to increase recruitment and retention among these historically underrepresented groups but to generate increased interest in scientific topics and processes among students and citizens more broadly.

One ‘traditional’ model of education in STEM disciplines'a lecture/exam model’ often emphasizes memorization. Students may become successful test-takers, able to memorize and recite facts, although research indicates they do not necessarily develop a deeper understanding of the discipline being studied (Norman, Eva, Schmidt, 2005). Bloom and Krathwohl (1956) highlighted this limitation by noting that most instructional methods among their contemporaries focused on fact-transfer and information-recall rather than facilitating meaningful intellectual development. In one of the central studies of why undergraduate students begin and then leave majors in the science, mathematics, and engineering (SME) disciplines (Seymour Hewitt, 1997), the five most frequently reported factors in these switching decisions were: lack of/loss of interest in SME [‘turned off science’] (43%); non-SME major offers better education/more interest (40%); poor teaching by SME faculty (36%); curriculum overload, fast pace overwhelming (35%); SME career options/rewards felt not worth effort to get degree (31%) (p. 33).

Further complicating the issues of recruitment and retention are the ways in which computer- and network-based technologies are integrated into the learning environment. From a later study by Seymour and colleagues (Seymour, Melton, Wiese, and Pedersen-Gallegos, 2005), teaching assistants in an interactive astronomy class for undergraduate students responded that:

“Learning new computer skills to do the work of class can be a bonus to students. However, it may take too much time – both the students' time in learning how to use a program and the [instructors'] time in troubleshooting software so that students can do assigned work.” (p. 90)

The ways in which students (particularly from underrepresented groups: women and students of color) develop and sustain interest in the sciences (particularly in technologically-mediated learning environments) is an area of research which may require collaboration between information science, education, and STEM researchers.

Bot 2.0 Project

The NSF-funded Bot 2.0 project is an innovative technological approach to enhance student interest in the biological and botanical sciences and address the lack of diversity among the population of students who ultimately pursue advanced education and careers in these fields. The project involves introducing students from local universities and community colleges to a three-phase curriculum involving reading and field exercises that incorporate inquiry-based learning, communal learning, and reflection.

The initial Bot 2.0 project goals included:

  • Using social software to connect students to the natural world and botanical science;

  • Studying how students organize information;

  • Examining the role and potential use of the memex framework to facilitate learning of science, technology, engineering, and mathematics (STEM) topics; and

  • Recruiting students from diverse and underrepresented populations to the field of botany.

BotCamp 2008

The primary activity associated with the Bot 2.0 project is an educational initiative called BotCamp. In the summer of 2008, a cohort of undergraduate students from a range of institutions in the Piedmont region of North Carolina were recruited to take part in an online and face-to-face curriculum. Roughly half of these students had not yet declared a major. The curriculum incorporated Web 2.0 technologies such as social software and cell phones to facilitate student engagement with the disciplinary content of the course (Daniel, Greenberg, Kramer-Duffield, Seiberling, Shoffner, Taylor, Weakley, Woodbury, 2008; Shoffner, Greenberg, Kramer-Duffield, Woodbury, 2008). To evaluate this iteration of the project, the project team conducted a baseline survey of experiences with botany, metadata concepts, and technological tools; a plant identification experiment; small group interviews; focus groups; and a summative evaluation in the form of a survey.

BotCamp 2009

The second iteration of BotCamp is scheduled for the summer of 2009. Based on the results of the research and evaluation of BotCamp 2008, the espoused student learning outcomes, the structure of the curriculum, the technologies being using, and the associated methods of evaluation have been modified to better align with student needs and with the goals of the project. In particular, the questions from the small group interviews and focus groups have been modified; for example, students will be asked to respond to prompts such as:

  • What are some of the challenges you've encountered when studying science and/or botany?

  • How important is science, especially of plants, in your daily life?

  • On a typical day, what do you think scientists do?

  • What do you see yourself doing in your career or working life?

  • What technologies have been useful to you in your social life (e.g., when you're out with your friends, when you're spending time with your family)?

  • What technologies have been useful to you in your academic life?

  • How would you describe the relationship between academic and social technologies?

  • Do you think of yourself any differently than you did when you began BotCamp? If ‘yes', How?

This poster discusses the process through which curricular and technological changes were derived and assessed” discussing lessons learned from the BotCamp 2008 curriculum; connecting these conclusions to the rationale for subsequent modifications of learning outcomes, curriculum, technologies, and evaluation methods; and presenting the preliminary results of the BotCamp 2009 evaluation.


This project is funded under award #0737466 from the National Science Foundation; Course, Curriculum, and Laboratory Improvement (CCLI)-Phase 1 (Exploratory), Scholarships in Science, Technology, Engineering, and Mathematics (S-STEM).