Technological advances in recent years have resulted in DNA sequence data being generated much faster than it can be fully analyzed. Genome annotation involves taking DNA sequence information from an organism and putting it into a biological context by predicting features such as locations of coding sequences and functions of gene products. The genomic databases are full of sequence information that has been passed through automated genome annotation algorithms. Although this automated approach certainly has its utility, it is estimated that anywhere from 5 to 40% of automated annotations in databases are incorrect [1, 2]. An additional issue is that these errors can become perpetuated, as incorrectly annotated genes become references for future annotations. Thus, there is an important need for curators to analyze and interpret genome annotations. Engaging undergraduates in genome annotation helps to fulfill this important need while at the same time providing a mechanism for students to build on key concepts in biology and biochemistry such as evolutionary relationships, organismal diversity, gene/protein structure and function, and cellular metabolism. It also provides an avenue to engage students in course-related research experiences.
In recent years, science education reports have emphasized the need for biology and chemistry curricula to engage students in research experiences as part of an undergraduate education [3, 4]. However, many institutions do not have adequate resources to offer individualized research experiences for all their students. Several studies have shown that course-related research experiences can result in student learning gains similar to apprentice style research experiences [5-7]. To further reduce costs, course-related research projects can be conducted by student teams. This also provides an opportunity to emphasize the importance of teamwork in science, as scientific research is becoming a more collaborative process than ever before .
In 2008, the Department of Energy JGI launched a program to support microbial genome annotation in undergraduate courses. The JGI hosts the online IMG-ACT  that guides students through various bioinformatics analyses and provides an online laboratory notebook where individual students or student teams can post their results. There are many Genomic Encyclopedia of Bacteria and Archaea genomes available to annotate, and many of the organisms are very safe to work with (Biosafety Level 1) and relatively easy to grow in the laboratory. In addition, some of the organisms, such as Mucilaginibacter paludis , have not yet been very well studied, and therefore, IMG-ACT serves as a framework around which research projects in a single course or multiple courses might be built at an institution .
Here, we describe the integration and assessment of a project in which students in microbiology and biochemistry analyzed amino acid biosynthetic pathways from M.paludis (Fig. 1). Using Escherichia coli K-12 amino acid biosynthetic pathways as models, student teams in microbiology collected and interpreted bioinformatics evidence to predict if M. paludis was able to synthesize each amino acid. The students then tested their predictions in the laboratory by designing and conducting growth experiments. A pathway annotated by a microbiology team from a previous semester was passed to the biochemistry class so that their student teams could work to functionally characterize the steps in the pathway via gene cloning and complementation analysis of E. coli mutants. Although the two classes did most of their work independently of each other, several joint sessions were held during the semester to initially introduce the project, to engage students in an exercise to make them more aware of interpersonal communication and team dynamics, and to discuss the qualities of a good presentation. At the end of the semester, each team gave a 12- to 13-minute oral presentation, which was evaluated by students from both classes.
Demographics of courses
Student demographics for the two courses for spring 2012 are as follows. Microbiology is a 200-level course primarily populated by biology and biochemistry majors with a starting enrollment of 20 students, of which 14 were sophomores, six were juniors, and one was a senior. It is an elective course for both biology and biochemistry majors. Biochemical Metabolism is a 300-level course with the prerequisite of Introduction to Biochemistry with an enrollment of 12 students. The class is normally only taken by biochemistry majors with an even distribution between juniors and seniors. Seven of the 12 students had previously taken Microbiology. One-third of the microbiology students and three-fourth of the biochemistry students had previously participated in research either in another course or in apprentice style experience.
Pathway Reconstruction in Microbiology
The activities associated with metabolic pathway reconstruction was a semester-long project that used eight laboratory sessions as well as some out of class work by the students. Teams containing two to three students were each assigned an amino acid biosynthetic pathway to focus on throughout the semester. The project was divided into three main activities (Fig. 1).
Write a Mini-Review of the E. coli Pathway
The first part of the project involved each team gathering information pertaining to their pathway in our chosen model organism, E. coli K-12. The purpose of this assignment was for students not only to become familiar with the pathway but also to give them some experience in locating, reading, and summarizing information from primary literature. One laboratory session was spent describing strategies for locating and reading primary literature and giving students some time to work as a group to locate sources pertaining to their pathway. The first assignment required each team to submit an annotated bibliography for their pathway. Each member of the team was required to locate and write annotations for three sources, and no one in the group was allowed to annotate the same article. Students were encouraged to work together to locate relevant sources; however, the students were required to write their own annotations. Each team then wrote a summary of their pathway in E. coli with particular emphasis on the number of steps in the pathway, identity, and functions of the gene products necessary for each step, and the relative locations of the pathway genes in the genome (i.e. looking for operons).
Reconstruction Pathway in M. paludis
Four laboratory sessions were devoted to learning to navigate through IMG-ACT and use various bioinformatics tools to address specific questions (Table 1). Student teams were created through the IMG-ACT interface that allowed all students in a team to upload and edit information relating to their pathway. The students divided the pathway steps among the team members, and each team member collected the evidence to answer the questions for his/her particular steps, and then the results were uploaded into the IMG-ACT laboratory notebook. Then, each team worked to integrate all their results into a single report. Because there were some differences between the gene products found in E. coli when compared with M. paludis, students were encouraged to highlight those differences. For example, the different domains of the E. coli TrpC and TrpD were split into separate gene products in M. paludis (Fig. 2a).
Table 1. Microbiology Laboratory Sessions for Bioinformatics Reconstruction and Analysis of Biochemical Pathways in M. paludis
- Open-reading frames, genetic code - Translational initiation in bacteria - Gene structure and operons
Where are the pathway proteins predicted to be localized in M. paludis?
- TMHMM - SignalP - PSORTb - Phobius
- Bacterial cell structure - Membrane versus soluble proteins - Protein secretion
Were any of the pathway genes likely acquired via lateral gene transfer?
- TCOFFEE - Phylogeny.fr
- Mechanisms of gene transfer - Global versus local alignment - Multiple sequence alignment - Phylogenetic trees
Test for Pathway Presence in M. paludis
Based on the evidence obtained from the bioinformatics pathway reconstruction, teams were asked to hypothesize whether or not they thought M. paludis would be able to synthesize the particular amino acid. Then each group was asked to design an experiment to test the hypothesis. This exercise built on previous laboratories in the course where they learned about different types of growth media. Students were told to consider controls that were needed in the experiment. M. paludis TPT56 was obtained from the American Type Culture Collection (#BAA-1394). Students had access to the 25× stock of minimal medium (1.25% glucose, 49 mM MgSO4·7H2O, 0.4 mM CaCl2·2H2O), agar that had been washed extensively with distilled water to remove excess nutrients and then dried, and 10 mg/mL stocks of all 20 amino acids that were used at a final concentration of 10 μg/mL. Students prepared their own media to test their hypotheses. As illustrated in Fig. 2b, the team researching valine biosynthesis included multiple growth controls that allowed them to conclude that although M. paludis seemed to be able to synthesize valine, it needed some amino acids other than valine to grow on minimal medium. Two laboratory periods were devoted to this experiment. A few teams had problems making media and had to come in at other times to complete the experiment. The final laboratory session prior to the oral presentations was set aside for the teams to work on their presentations.
Student teams were required to prepare both a written report and an oral presentation that integrated all three aspects of the project. The students were told that these reports would serve as a starting point for a future biochemistry class. For most of the students, this was the first time that they had to pull together so many pieces of evidence into a coherent story. Each team submitted a draft of the written report 2 weeks before the end of the semester and received comments so that they could revise the report. The final report was due at the beginning of finals week.
Functional Genomics in Biochemistry
Historically, this biochemistry laboratory was used for independent projects; however, as enrollment increased, the feasibility of this became impractical. The class then moved into a more scripted protein purification and characterization scenario. Students found this approach less interesting because it emphasized a known outcome, which they were trying to replicate. Although the experience was new to the students, the fact that it had been done before limited the degree of engagement in the outcome of the project.
The functional characterization of an amino acid biosynthesis pathway in M. paludis gave the students an opportunity to engage in a truly novel research experience. As a starting point, the biochemistry class received a list of the predicted gene products and supporting evidence for the serine biosynthesis pathway that a team in the fall 2011 microbiology class had produced. Each biochemistry team consisting of two students focused on one of the gene products and worked to define and refine methods to determine if the M. paludis gene could functionally complement a particular E. coli ser mutant (Fig. 1). In general, students adapted the approach developed by the GENI to study functional complementation of E. coli K-12 mutants with Agrobacterium genes . After being introduced to the project, students learned to navigate the IMG-EDU database  and were provided general protocols for PCR, ligation-independent cloning, and bacterial transformation from Current Protocols in Molecular Biology  and GENI, which they would adapt to the specifics of their project (see Methods_Biochemistry document in Supporting Information). Each member of the team would have to contribute to decide which protocol to use and how to adapt it to fit the desired outcome for each experiment. One of the biggest impediments to student engagement is the fear of making a mistake because of an overriding concern for how it will impact their grade. In the research laboratory, even our best efforts do not guarantee that mistakes will not happen and that these can be learning opportunities. With this in mind, it should also be noted that although comments were made to help the students at every step of the process, students were free to make mistakes without fear of penalty if they decided not to heed the advice given to them. Each failed experiment was met with the questions: What went wrong and what do I need to do differently next attempt? For example, two of the most common PCR problems were either no product or too many products. The problem of no product was usually a problem of not adding all the components to the reaction mixture. Repeating the procedure more carefully fixed this problem. The problem of too many products was a function of an overly permissive annealing temperature and was fixed by moving to more stringent conditions.
Because this course focused on getting students take ownership of their project and think like scientists rather than having the instructor anticipate and/or troubleshoot problems, none of the groups progressed beyond cloning their gene into the expression vector. Despite this, the work done by this group of students serves as a starting point for the next group of students in the class. This emphasizes the ongoing nature of scientific research.
Weekly Action Plan
As a key emphasis of this laboratory was to give students practice planning and refining their research direction as they gathered and interpreted their data, the students were asked to provide a weekly action plan. Each plan included a summary/analysis of previous data, justification for the next experiment, list of procedures and materials needed, expected results, and a list of waste generated (Fig. 3). Each plan was submitted two class sessions prior to their laboratory session and was returned the following class session with specific comments and recommendations to consider. Students then had time until their laboratory period to consider the feedback and decide whether or not to modify procedures for that week's laboratory session. The purpose of this strategy was to move the students into a mode of following the data to decide what to do next instead of the more traditional retrospective laboratory report format.
Coordinated Class Sessions
The two courses were scheduled with concurrent lecture sections. This allowed us to meet as a group several times during the semester (Fig. 1).
During the first week, we presented some background information about M. paludis and explained what each class would be working on during the semester.
Teamwork Activity and Discussion
As the project had an emphasis on teamwork, we invited Peter DeLisle, Director of the College's Posey Leadership Institute, to lead a discussion on communication and accountability in a team during the fifth week of the semester. Prof. DeLisle gave a short lecture on the relationship of complexity and discontinuity . In this context, complexity is defined as how difficult a task is perceived to be, and discontinuity is an event where a seemingly simple task becomes daunting or unmanageable. To reinforce the lecture, students were paired up for an exercise (see Team_Activity document in Supporting Information). Teams of two were given the instruction to think of a very simple phrase and communicate it to their teammate. After allowing the team a few moments to formulate a plan, they were given a discontinuity; one team member was “mute” and the other was “blind.” At the end of the exercise, students discussed as a group the message they received and how they dealt with their discontinuity. Following the lecture, students were given the opportunity to answer a few open-ended questions about their experiences.
“This provides a great model for the current lab set up with the different groups. Each group member is responsible for an equal share of the work and communicating with the other members. It's really important, therefore, to recognize the need to be aware and competent in situations.”
“Yes, commonly students will hit a wall of difficulty and retreat from the daunting task however in order to progress forward, this challenge must be met. This has applications in every class, but especially the sciences.”
Oral Presentation Discussion
During the ninth week, students discussed good and bad attributes of oral and poster presentations in small groups and then shared the highlights of their discussions with the entire group.
Student Team Oral Presentations
The teams were given the option of presenting their work as a poster or as a PowerPoint presentation. All the teams elected to give a PowerPoint presentation. Each team was allotted 12–13 min for the presentation and 2 min for questions. Presentations were peer evaluated using a rubric that had a scale of 1–4 for each item being assessed (see Presentation_Rubric document in Supporting Information). Each team was given the evaluation rubric ahead of time so that they knew the criteria for evaluation.
The student evaluation scores were collected, and the mean score for each category on the oral presentation rubric was determined for each team (Fig. 4). The students had an overall favorable impression of the work presented by their peers; the aggregate mean was within the range of 3–4. In the dataset, the item with the lowest mean score and greatest variation was “answering questions” posed by audience members at the end of the presentations. The lower score here is attributed mostly to a single group who went over their allotted time and thus forfeited the chance to answer questions. Therefore, with exception of “answering questions,” there was little variation in the presentation scores among the groups. This is consistent with the study by Yankulov and Couto , which showed that undergraduates had a difficult time discerning differences between good and excellent research proposals during peer review. Although we do not use peer review to assign grades, we believe that there is still value in having students engage in peer assessment as this practice is such an important part of the scientific enterprise.
The project was assessed in three different ways. First, students were given a quiz at the beginning and end of the project that was designed by the instructors. The quiz contained 10 multiple-choice questions that assessed major concepts specifically addressed in the project (see Assessment_Quiz document in Supporting Information). Students also provided open-ended feedback of what they felt they learned through their participation in the project. Finally, students took the RISC survey , developed by Lopatto, at the beginning and the end of the semester. The Austin College Institutional Review Board approved all assessment tools.
Assessment Quiz Results
Comparison of paired prequiz and postquiz results showed that overall, students exhibited significant learning gains through the project (Fig. 5a). When the students were separated by course, only students in microbiology showed significant learning gains. This might be explained by the fact that students in microbiology had on average lower scores on the pretest than students in biochemistry. The questions on the quiz related primarily to either molecular/evolutionary concepts or bioinformatics concepts. At the beginning of the semester, students answered about twice as many questions correctly relating to molecular/evolutionary concepts as bioinformatics concepts. Students are exposed to these molecular/evolutionary concepts in many courses in the curriculum. When the prescores and postscores were examined based on these two categories, there were no significant learning gains relating to the molecular/evolutionary concepts (Fig. 5b); however, there were significant gains relating to the bioinformatics concepts for the students in microbiology (Fig. 5c). This was not too surprising as the microbiology course project focused on building these skills. Seven of the 12 biochemistry students had previously taken microbiology, which may explain why that class started out with higher scores relating to the bioinformatics concepts than the microbiology class. The fact that the biochemistry students did not exhibit bioinformatics-related learning gains indicates that these concepts may need to be reinforced multiple times in the curriculum.
Research on Integrated Science Curriculum
This survey is designed to evaluate the perceived learning gains made by students as part of a research experience in a class setting. It is composed of three components. First is an instructor-completed course rubric to describe the activities and goals of the class. Second is a precourse survey for the students, which asks them to report on their experience with the same components that appear on the faculty survey and information regarding learning style and attitudes toward science. The third part is a postcourse evaluation, which asks the students to rank their learning gains on the same list of activities. Overall, Austin College students started the semester with similar levels of experience as other students participating in the RISC survey (Fig. 6a). In the presurvey, our students not only reported the highest level of experience with working in small groups or teams but also reported the highest experiential gain with this item at the end of the course. This might be attributed to the special class session focused on team dynamics. Similar to other students participating in the presurvey, our students reported the least experience in a course where they focused on one or a few problems, problems with no known outcome, a project of their design, and problems with simultaneous and interactive multiple causes. At the end of the semester, our students reported similar or higher gains in experience for these items when compared with other students participating in the survey. In particular, our students reported high experiential gains relative to all the students working on one or few problems in which no one knows the outcome.
One set of items included in the RISC survey is also included in the Summer Undergraduate Research Experience survey  so that learning gains reported by our students could be compared with a summer research experience (Fig. 6b). Again, the learning gains reported by our students were similar to students participating in apprentice-like research experiences. Austin College students reported particularly high learning gains for giving oral presentations, understanding science, and understanding that scientific assertions need to be supported with evidence. These are certainly skills that were emphasized in both courses.
Open-Ended Student Comments
Students in both courses were asked what they felt they gained from a learning perspective through their participation in this project. This provided the students an opportunity to respond to what they learned in an unstructured context. Thirty-one of the 32 students who finished the course provided a comment. Student statements were coded by learning gains based on the articulated comments, and the percent responses for each category was evaluated by course (Fig. 7a) or whether or not the student had previously participated in a research project either in another course or in an apprentice-like experience (Fig. 7b). Forty-five percent of the students included a comment relating to a gain in technical experience typically relating to laboratory techniques or the use of bioinformatics tools. Nearly 40% of the students commented on how the project required them to think critically and/or troubleshoot problems. In both cases, the majority of students making these comments had prior research experience (Fig. 7b).
The students in microbiology were the only ones to make comments relating to organizing and presenting evidence, making connections to content in the course, and appreciation for the research process (Fig. 7a). Seven of the eight students commenting about organizing and presenting evidence were sophomores, and the comments indicated that this was the first time they had to collect, interpret, and present so much information. Some of the students found this difficult and frustrating at first, but in the end, they felt that they learned a lot from the experience:
“I learned a lot about the background work that goes into understanding a gene pathway. For most of the semester I had no clue what was going on but by writing the paper and creating the PowerPoint presentation most things came together in my head. I see the significance of our work now and think what we did was cool. So the project helped me learn.”
“The way the project was implemented by the instructor was somewhat a hands off approach which in turn I think forced me to learn a little more on my own and forced me to think a little more critically. Having to do the presentation also helped me really pull together all I was learning throughout the semester.”
Interestingly, comments relating to making connections to content in the course and appreciation for the research process were made primarily by students without any previous research experience (Fig. 7b). For example, a few comments relating to this were as follows:
“It helped me to see how much and what kind of work goes into annotating genes. We learn about so many different types of proteins and genes in all our classes but I never realized how long it must have taken to put that information together.”
“How much work actually goes into doing a research project. I also learned how to interpret different forms of data and put all of the pieces together to create an overall picture for my pathway in M. paludis.”
This seems to indicate that novice researchers may experience a newfound appreciation for the research process and how research connects to what they are learning in the classroom. It is not to say that students who have engaged in research previously do not appreciate the research process, but rather for novice researchers, this represents an “aha moment” that impacts them enough to comment on it.
In this study, we integrated a joint research project to annotate and functionally characterize amino acid biosynthetic pathways of M. paludis in microbiology and biochemistry. Similar to other studies examining the impact of course-related research experiences [5, 6], our students experienced perceived learning gains similar to students participating in a summer undergraduate research experience. In an apprentice-like research experience, students will typically encounter a single yet multifaceted project. Here, we intended to provide a similar opportunity to the students by constructing a semester-long project that had several complementary parts. Our assessment results indicate that our intentions were in line with student perceptions that they worked on a coherent-unified project. The learning gains were as follows: “How scientists work on real problems” and “readiness for more demanding research” were both higher than the national mean, suggesting that students consider themselves as having taken on a real research project and are confident in their abilities to do so again.
Although students in both courses perceived that this project enhanced their learning, the nature of learning experienced in microbiology and biochemistry was somewhat different. The students in microbiology showed significant knowledge gains relating to bioinformatics concepts and also reported a greater understanding of the importance of evidence necessary to support assertions. The nature of the project was such that they were collecting a lot of different pieces of bioinformatics evidence. While each team worked on a particular amino acid biosynthetic pathway, individual team members collected and recorded evidence for individual gene products in the wiki laboratory notebook. Some students commented about being lost for much of the semester until their team had to synthesize and integrate all the team evidence for their presentation, emphasizing the value of a final written and/or oral presentation in promoting their ability to see how the individual parts fit together in a broader context. In addition, these primarily novice researchers reported an epiphany-like experience in terms of their view of research in relation to the production of scientific knowledge. In biochemistry, less emphasis was placed on learning bioinformatics concepts, and more emphasis was placed on the research process. As such, students did not show significant learning gains in their knowledge of bioinformatics concepts. Instead, their learning related more to building their skills to think and work like scientists, similar to benefits that were commonly reported by senior-level undergraduates in interviews following apprenticeship-type research experiences .
One of the main focuses in this project was to get students to see how science is a collaborative effort. Often students in a laboratory section are paired into groups who will follow the same set of instructions to come to a common conclusion and then go off to write individual laboratory reports. Although this approach does lend itself to development of technique, it does not necessarily promote teamwork. Here, we created an environment where both classes worked together on a common project and where student teams were cooperatively responsible for part of the project. Furthermore, work conducted in microbiology was handed off to students in biochemistry, simulating the ongoing nature of scientific research. At the beginning of the semester, students perceived a level of experience for working in small groups or teams that was consistent with the RISC aggregate mean (Fig. 6a), whereas the postcourse perceptions were slightly higher than the national mean. What is most interesting is that students reported gains higher than their initial perceived level of experience, indicating that they perceived that teamwork was indeed an integral part of the experience. However, our assessment also indicates that we could do a better job encouraging collaboration between the courses. In future years, we plan to form teams composed of both microbiology and biochemistry students. The teams will meet every few weeks to update each other on their progress. We hope that this will give the students more practice articulating their research to others earlier in the semester and also promote more cross-course interaction throughout the semester.
In summary, we used IMG-ACT as a foundation to incorporate cooperative microbial genome research into the curriculum. This approach not only served as a vehicle for introducing the use and interpretation of bioinformatics evidence but also promoted the development and strengthening of research skills that are transferable to other courses. Because IMG-ACT is easily accessible via the Internet, similar projects are certainly feasible at other institutions. Additionally, the amount of genomic data in need of annotation and characterization is significantly enough to provide novel research opportunities to other institutions interested in adopting this model.
The authors thank Cheryl Kerfeld, head of the Joint Genome Institute Education Program, for the opportunity to take part in the Interpret a Genome project. The Research on the Integrated Science Curriculum survey was created by Prof. David Lopatto of Grinnell College with the support of the Howard Hughes Medical Institute. The authors thank Prof. Lopatto and Ms. Leslie Jaworski for their help with data analysis. They also thank Kathryn Houmiel, GENI Program Manager, for the expression vector pKT-1 and the protocol for ligation-independent cloning.