Redesigning a course based undergraduate research experience for online delivery

The COVID‐19 pandemic forced educators to teach in an online environment. This was particularly challenging for those teaching courses that are intended to support bench science research. This practitioner article tells the story of how an instructor transformed their Course‐based Undergraduate Research Experience (CURE) using the Backwards Design Method into a synchronous online course. Research objectives in this transformed course included: conducting a literature review, identifying research questions and hypotheses based on literature, and developing practical and appropriate research methodologies to test these hypotheses. We provide details on how assignments were created to walk students through the process of research study design and conclude with recommendations for the implementation of an online CURE. Recommendations made by the instructor include scaffolding the design, building opportunities for collaboration, and allowing students to fail in order to teach the value of iteration. The Backwards Design framework naturally lends itself to a scaffolded instructional approach. By identifying the learning objectives and final assessment, the learning activities can be designed to help students overcome difficult concepts by filling in the gaps with purposeful instruction and collaborative opportunities. This present course also practiced iteration through the extensive feedback offered by the instructor and opportunities for students to revise their work as their understanding deepened. Anecdotally, based on end of course reviews, students overall had a positive experience with this course. Future work will examine the efficacy of student learning in this online environment and is forthcoming.

the sciences, including gains in content knowledge and technical skills. 4UREs promote student confidence in their research abilities as well as the development of important critical thinking skills, often termed "thinking like a scientist". 5,6These research experiences often involve the student working in a faculty member's laboratory as an extracurricular outside of their normal course load. 7owever, these experiences are often difficult for a student to gain access to because the student is either unaware of the research opportunities available, or there is limited availability in faculty members' research laboratories for undergraduates, therefore making the research opportunities have competitive application processes. 7To overcome these limitations, a model for providing research experience within a semester-long course was developed and is known as a Course-based Undergraduate Research Experience (CURE). 8he development and implementation of a CURE takes a considerable amount of time and preparation.So, what happens when there is global pandemic, and educators must now pivot from the classroom to online learning to comply with new social distancing mandates?Online learning and social distancing do not mean a stop to educating our future scientists.Many would argue that the continued education of our future scientists is more crucial now than ever.Faced with this dilemma, an instructor and their graduate teaching assistants decided to revise a previous in person CURE experience for online learning, largely by substituting new learning objectives associated with the process of scientific research that could be taught online in place of those that could not.This practitioner article tells the story of how they redesigned and implemented their course for use online in response to the COVID-19 pandemic using the Backwards Design Method. 9It concludes with advice for others interested in providing CURE experiences online.

| CURE design recommendations
When an instructor designs a CURE, they must make careful considerations to their design and planning of the curriculum.Some considerations a faculty member should make before even designing a CURE include identifying the overall purpose of investing time into a CURE.The purpose of the CURE might include collecting data for research, recruiting students to work in the lab, piloting research with a grant, engaging more deeply with students, or connecting more with teaching. 10All these documented reasons for investing time into a CURE are valid, but still require careful consideration to ensure the overall goal of the experience is successful for both faculty and students.Once the purpose is identified, overall goals or outcomes that the faculty proposes for the student can be determined.A popular method amongst educators for designing instruction once outcomes have been identified is the "Backwards Design" method. 9n fact, when designing a CURE, it is recommended that the Backwards Design method be utilized because it can help instructors create a research experience for students that is based on a solid foundation of instructional practices that will help students achieve the learning goals identified by the instructor and make the research experience more meaningful for the student. 11According to the Backwards Design model, instructors must first determine what outcomes or goals they want students to achieve within the course.Goals of a CURE course will often involve both learning goals as well as a gain in technical skills and will be centered on the research question or objective that guides students through the research experience.The next phase of the Backwards Design model involves instructors determining how students will be assessed and what assessment measures are most appropriate.This may involve making decisions regarding how students' progress is assessed throughout the duration of the CURE, which may take the form of classroom observations, monitoring the organization of a student's lab notebook, or how a student may present final work.In the final phase of the Backwards Design process, the instructor must determine what experiences will take place in the course to achieve those learning goals.In a CURE setting, this final phase will take the form of planning daily and weekly activities, assessing whether those activities are helping students gain the necessary cognitive and technical skills needed to achieve the learning goals, and finally assessing to assure there is proper alignment between learning goals and the assessments of those goals throughout the duration of the course.

| Pivot to online
Recently, practitioner papers discussing how faculty made that pivot from face-to-face instruction to online/ remote delivery during the initial March 2020 shutdown have been published.Many of these practitioner papers outline how the faculty members utilized different resources to help students learn concepts that would typically be taught in the laboratory setting.3][14] One article detailed how a group of faculty members who teach an upper-level capstone laboratory made the pivot to online delivery and helped their students still gain important skills needed for scientific research. 15The practitioners describe how in the original capstone course students gain important research skills through designing, executing, and reporting on a novel research project.However, because of the social distancing mandates and the requirement that all in-person instruction halt, wetbench laboratory research was no longer an option.In response to the shift to complete remote/online delivery, the practitioners detailed how in lieu of being able to collect and analyze data, students instead developed research protocols based on primary literature, predicted results and potential analysis, and troubleshooted their protocols simulating the real-life application of troubleshooting skills that would occur at the bench.The practitioners found that, overall, the remote/online format worked well; however, students struggled in writing their experimental design and in making connections between the literature and practical methods used in the lab setting.Ultimately, the authors concluded that the pivot to the online/remote delivery was successful.However, the authors noted that for students to make larger gains in the application of research skills, more careful instructional planning is required to help walk students through the experimental design process.
It has been noted that because of the pandemic online learning perceptions should be reevaluated.Courses believed to not be feasible to the online delivery method were in many instances successful.Therefore, it stands to reason that other courses like the capstone course discussed above could be adapted in similar ways to the online/remote setting.But as noted, a CURE should undergo careful planning using a framework, such as Backwards Design, to design an appropriate course experience.In fact, these researchers have indicated that there is little evidence that instructors are using the Backwards Design framework to design these types of courses.This insight led to the inspiration of this paper to document how the fifth author and their TAs were able to redesign and implement a CURE for a synchronous online setting using Backwards Design.This present manuscript outlines the learning objectives and how the assessments and learning activities were designed with reference to the Backwards Design model.

| CURE ADAPTATION
A molecular biology lab was initially designed as a CURE.The purpose of this CURE was to provide students with a unique opportunity to gain important and necessary skills required to conduct scientific research.
This course ran as a traditional CURE, meaning inperson, during the summer of 2019 and enrolled a total of 16 students (8 females and 8 males) with 60% of the students beginning their senior year.However, because of the COVID-19 pandemic and the state-wide social distancing and closure mandates, this course was no longer available to students as a face-to-face experience.Nonetheless, it was decided by the instructor of record that this course experience should still be offered to students through distance education.The intent of the new iteration of this course is still the same: to provide students with an opportunity to gain the important and necessary skills needed to participate in scientific research.
This course was designed as an 8-week summer course.The course content and delivery were conducted by the instructor of record and two TAs via the Learning Management System (LMS) of the institution.This summer course enrolled a total of 12 students (6 females and 6 males) and 60% of the students would be starting their senior year.The course content consisted of instructor made videos, handouts, homework assignments, reflection questions pertaining to the nature of science, scientific articles, and quizzes.Each week, the instructor would post a weekly agenda, which was termed a "Weekly Workflow."The purpose of this workflow was to help students stay on task throughout the week.The workflow detailed what videos students would be viewing, handouts they should review, worksheet assignments and when they were due, if they had reflection questions to answer, and by what date they would need to complete their quiz and submit homework.In addition to this content, students were tasked with a large group project known as the "Virtual Project."In this assignment, students were tasked with developing a research question based on their review of primary literature, creating a hypothesis, and developing an experimental design protocol to test that hypothesis.
Students also participated in weekly Discussions and Project groups through Cisco Webex.These weekly remote sessions would meet for approximately 1 hr.The purpose of the Discussion group was to have students meet on a weekly basis to discuss homework assignments, nature of science reflection questions, and have an opportunity to ask questions and connect with other classmates and instructors.Students were placed in groups of four.Each week for the Discussion group a different instructor, either the instructor of record or one of two TAs, would lead that students' group.This rotation of instructors for the Discussion group was done to help introduce students to all instructors of the course.Likewise, the Project group met on a weekly basis.This group also consisted of no more than four students.However, the Project group had the same instructor lead the group each time it met.The purpose of the Project group was to have students meet via Webex to collaboratively progress their Virtual Project.

| The virtual project and assignments
The course learning objectives aimed to help students develop critical thinking skills related to scientific research.Since learning had to occur at a distance, hands-on laboratory bench research was impossible.However, students were still able to gain important experience in the form of learning how to review literature, develop a research question and hypothesis based on the literature they reviewed, and then develop an experimental protocol to test the hypothesis and troubleshoot that protocol.These aspects are all important to scientific research, but ones that students often struggle to successfully master. 15he goal of this course was to have students develop a plausible experimental design that was informed by relevant primary literature.This goal takes the form of the final assessment, the Virtual Project.This project was done as a collaborative group project.Using the Backwards Design Framework, the learning objectives were established first.The instructor and TAs were then able to work backwards to determine what evidence would show students had met the objective.The Virtual Project was determined to be the ultimate assessment to determine if students had met the learning objectives.To help students meet this goal, 12 worksheets were created by the instructor and TAs to guide the students through the process of experimental design (see Table S1; worksheets will be made available upon request).Worksheets 1-5 were designed to help students develop foundational skills related to experimental design which include the following: (1) how to identify essential information in a primary literature article; (2) how to identify a hypothesis; (3) how to critically read a primary literature article; (4) how to write a plausible and testable hypothesis; and (5) how to revise a hypothesis based on feedback.These worksheets were designed for students to gain these skills on their own.After laying the foundation needed, worksheets 6-10 were designed to help students develop their experimental design in their Project groups, which would ultimately cumulate into the final Virtual Project.Students collaborated in their Project groups to complete these worksheets (both on their own and with the instructor), which helped students learn the following: (6) generating a research question based on a literature review; (7) using the literature review to develop a testable hypothesis; (8) creating an experimental design protocol; (9) writing up the final Virtual Project by combining previous worksheets (worksheets 6-8); (10) practicing troubleshooting through instructor made scenarios based on individual Project group experimental designs; and (11) reviewing other student projects and making predictions of results based on their hypotheses.
After instructors created each worksheet assignment, the content was then created to impart essential information to help students be successful in completing the worksheets.Content took the form of short instructor made videos that covered assorted topics including conceptual content related to molecular biology and various techniques used in molecular research (such as immunofluorescence).In addition, the instructor created handouts providing details pertaining to hypotheses, how to collect and analyze data, and how to troubleshoot when an experiment goes wrong.By using the Backwards Design model, the instructors were able to develop all content and assignments necessary to help students meet the overall course learning objectives.

| Scaffolded design
As explained in the Course Design section above before any course design occurred, the instructors first had to determine what they wanted students to gain from this course.The first step in this design process was to identify the main course objectives.Once those course objectives were solidified, the instructor and TAs were then able to determine what the final assessment would be that would indicate that students had met those objectives.In this course, the final assessment was the Virtual Project.From this final assessment, the instructors worked backwards to identify key components of that assessment and what students would need to do to work up to that final project.This backwards process informed the creation of the worksheets.Once each worksheet was created, the instructors determined what content students would need to successfully complete the work.Once the content was identified, the instructors determined the best way to deliver that content which took the form of videos, readings, and instructor-led remote class discussions (i.e. the Discussion and Project Groups).
This Backwards Design process led to a natural scaffolding of the assignments, which built up to the final Virtual Project.Through the framework of Backwards Design, the instructor was able to chunk out all the necessary pieces of a research proposal, which include a literature review, research questions, hypotheses, and experimental design.For a student to be able to conduct a literature review, they first had to learn how to read a research article (Worksheets 1 and 3).For a student to be able to generate a hypothesis based on a literature review, they first had to understand what a hypothesis is and then practice generating their own (Worksheets 2, 4, and 5).By chunking out the pieces that made up the Virtual Project, the instructor was able to ensure students could accomplish each necessary part knowing it would become gradually more difficult for an inexperienced student.

| Making room for failure and feedback
Iteration is a crucial part of the scientific process and a hallmark feature of a CURE.Iteration is more than having multiple experiments to ensure the validity of results.Designing a hypothesis is a challenging task, and then designing the experiment to test that hypothesis is just as difficult if not more so.The iterative process helps refine our understanding as we collect data through experimentation and deepen our understanding by reading literature.Iteration is done to troubleshoot unexpected or troubling or our experimental design.In letting students be more active in the iterative process, they may develop deeper senses of ownership in their learning and projects because they have been afforded autonomy over them.
However, in an online course, students are not going to have the same opportunities to try things out at the bench as students in a traditional CURE, which is where they would learn immediately whether something is going to work in the experiment.Therefore, feedback and revision will act as an iteration and will play a crucial role in helping the student develop the critical and technical thinking skills in a synchronous online CURE.Feedback can and should occur in a variety of ways.As mentioned above, students in this course received feedback on assignments as well as during the group meetings.Students also received feedback through the instructors modeling how to think through a problem.This took place through group discussions where instructors would walk students through how they would approach the assignments.
For example, in this CURE course, it was important for students to know how to generate a testable hypothesis.The concept of the hypothesis is not new for an undergraduate science major.However, generating a plausible hypothesis based on scientific literature can be challenging.To help students learn and improve this skill, the instructor provided videos, handouts, worksheet assignments, group discussions, and feedback with opportunities for revision to ensure students could accomplish this learning objective.Students first practiced identifying hypotheses in simple instructor made scenarios and were then provided with opportunities to identify hypotheses in the primary literature.This activity was expanded through remote discussion groups where instructors were able to help further guide students in how to identify the hypotheses in the assigned research article.Prior to these remote discussions, students were asked to turn in their worksheets where they practiced identifying and writing their own hypotheses.Instructors would provide feedback before the remote session, expand further on that feedback during the class discussion, and then allow students to revise their worksheets and resubmit.
In addition to instructor feedback, this course was structured in a way where students also had numerous opportunities to receive feedback from their peers.During the stages of creating the hypothesis and developing the experimental design, students were afforded opportunities to have their work reviewed by their peers as well as provide feedback to others.The instructor acknowledged during the interviews that feedback from multiple sources would be an added benefit to the students because it would help teach them how to be critical and constructive regarding how to review others work and more importantly their own.
When teaching critical thinking skills, especially in a CURE, it is important to remember that not all students start at the same level.In fact, not all students will end up on the same level either.In this course, students were graded less on correct answers and more on overall improvement.Improvement was viewed as the student being able to take the feedback they were given and apply it to the revision and the next task.

| Collaboration
Scientific research does not happen in a vacuum.Research is often a collaborative endeavor.An online CURE does not mean that students cannot collaborate with one another.However, if the intention is to have the students collaborate, that needs to be thought through before the course begins.Allowing students to determine how they communicate and work together will be essential.However, it is the role of the instructor to determine what types of methods are acceptable.Therefore, it will be important to set the guidelines and expectations.Student collaboration was accomplished in this CURE through what the instructor termed a "Groupwork Agreement."This was a form that students in the group would fill out at the beginning of each assignment and submit before they began working together.On the form, students would indicate how they would communicate with one another, what days and times they would meet, and who was responsible for each part of the assignment.The purpose of this agreement was to help students establish rapport and boundaries with one another to complete the assigned tasks.
However, keep in mind that if you allow students to collaborate in one way, asking them to change midway through the course and communicate in another way might not work.For example, in this CURE, students indicated that they communicated via Facetime on their smartphones.However, the instructor wanted the communication to occur on a discussion board so that they could evaluate the groups' thought processes.Because the students had already met via Facetime, the reenactment of the group meeting on the discussion board was ineffective.The instructor indicated that next time they will have the students post bullet points of the discussion and key takeaways.This way, the instructor can still see the thought processes and pose new questions to the group to help them dig deeper in their understanding.
In addition to student-student collaboration, a major feature of a CURE is the collaboration between instructor and student.An online setting can still accomplish this instructor-student collaboration through building rapport with students.In this CURE, the instructors created a rapport with their students through consistent feedback on assignments, and in the Discussion and Project group meetings.To help students become more comfortable discussing research ideas, the instructors rotated leading the different Discussion groups.This was done to help students find an instructor they could connect with and feel comfortable with asking for help when it was needed.In addition, the instructors held weekly team meetings where they discussed teaching strategies as well as identified struggling students.These meetings were used to ensure all instructors were providing similar instruction and to determine how best to help struggling students meet the next learning goal.

| CONCLUSIONS
The student experience needs to be meaningful and useful regardless of the instructional modality.Online teaching involves more purposeful planning because you are not able to gauge student response in real-time.Therefore, there is more integration and scaffolding that must take place.Through using the Backwards Design framework the instructor and their TAs were able to create a scaffolded course.During the interviews, the instructor commented that the scaffolding seemed to naturally occur through using the Backwards Design method.As mentioned above in the section entitled, "Scaffolded Design," the scaffolding of this course happened by parsing out the final Virtual Project through the creation of the worksheets.Much of this scaffolding seemed to naturally occur because the instructor and TAs strived to make content and instruction as explicit as possible.During team meetings, the instructor and TAs would discuss what pieces of the worksheets would need to be made more explicit through the content (handouts and videos).It was through this process of working as a team and thinking aloud how to make as much instruction as explicit as possible that the natural scaffolding of the course became clear.Furthermore, the instructor stated that this natural scaffolding would not have been possible without using the Backwards Design method to guide this course design process.The course objectives are complex in nature, and the instructor indicated that the Backwards Design framework provided a roadmap in how to help students accomplish these learning goals and deepen their critical thinking skills.
It is also important to keep in mind that expectations of student knowledge levels must be reasonable and related to their level of experience.Even upper-level undergraduates are not going to understand the complex science that a PhD level does unless they are brought up to speed with the jargon.The purpose of this synchronous online CURE was to help students have a shift in attitude towards science and begin to learn how to think like a scientist.This "thinking like a scientist" involves standard critical thinking skills, but also involves understanding the iterative nature that is scientific research.Iteration is more than just running multiple experiments to ensure the validity of data.Iteration is how scientists cope with failure.3][14] In this course, iteration was simulated through multiple revisions of all of the worksheets leading up to the experimental design.In fact, it was how this course was designed through scaffolding that led to numerous opportunities for students to receive feedback from both instructors and peers.Similarly, the tailored troubleshooting assignment (Worksheet 12) also provided students with iteration in a more traditional context where students had to troubleshoot their experimental design.The aim of troubleshooting a student designed product is to create a more meaningful experience for the student. 12This aspect of this course design could lend itself to future research to explore its efficacy in student learning and motivation.
Moreover, when designing a CURE, whether online or in-person, it is important to not try to do too much.
Students cannot do it all.Not initially at least.So, the first question you need to ask is what is the goal of the course?Is the goal practical firsthand technical experience, or is the goal to be able to better think through problems and reason like a scientist?What are the goals of your student group?Are they wanting to enter the job market when they graduate?Then a more practical firsthand experience might be better where they can learn and perfect techniques used in the lab.Are they wanting to enter graduate school?The techniques will be important, but it will be more important that they can begin to think like a scientist.They need to learn how to analyze a research study, ask follow-up questions and pose new hypotheses, and be able to pull in relevant background information to design an experiment to study that question properly.
Lastly, during the interviews, the instructor of record reiterated the importance of having the TAs be involved in not only the facilitation of this course, but also in the design process.The instructor indicated that this course would not have been possible without the TAs involvement.The instructor commented that the collaboration with the TAs led to a more successful course design and overall implementation and noted that this type of experience for the TAs to be involved in course design to a greater extent, as was the case for this course, is something that does not normally occur.Whereas this course has potential to be a growth opportunity for students to develop critical thinking skills related to research, the instructor also noted the potential growth opportunities for TAs to better understand their own craft of research and to learn intricate course design processes necessary for careers within academia.Additionally, the instructor indicated that this collaborative process where the TAs were given power to direct the design, which is not typical, allowed for a more meaningful student experience than would have been produced solely by the instructor alone.Moreover, the instructor indicated that this collaboration was mutually beneficial in that it benefited the TAs by providing practical teaching and designing experience, and it benefited the instructor by helping create a richer course experience for the student.Ultimately, the design and implementation of this synchronous online CURE afforded opportunities for both student and instructor alike.

| FUTURE WORK
Preliminary data regarding student content knowledge was collected during this course.The results of the content knowledge pre/post survey indicates that on average the students did improve in their understanding.Anecdotally, students largely had positive sentiments towards the course design based on their end of course review.This is promising and leads to future work to examine the efficacy of this type of online course experience.Currently, data regarding the student experience has been collected and is being analyzed with the intention of submitting for future review.