Same Program Different Delivery: Adapting the Woods Hole Partnership Education Program for a Virtual Era

The Woods Hole Partnership Education Program (PEP) has provided a 10‐week residential summer internship program for underrepresented minorities since 2009. Six weeks prior to the planned 2020 residential program, the gravity of the COVID‐19 pandemic prompted the unanimous decision of the PEP leadership and staff to deliver a virtual program. However, this decision engendered a new set of considerations for a successful undergraduate research program. In this article, we discuss how PEP and its partner institutions galvanized around PEP's overarching goals and guiding principles to rapidly adapt its course, research projects, and supplementary activities for a virtual environment. Based on the insights and feedback from multiple sources, we offer insights and lessons learned for other summer internship programs adapting their programs for a virtual delivery.


Introduction
Multiple studies conclude undergraduate research experiences that integrate hands-on learning are especially effective in retaining undergraduates in STEM through their graduate degrees and careers. Specifically, summer internship programs can positively affect underrepresented minorities' (URM) commitment to STEM careers (Pender et al. 2010). Additionally, scientists acting as mentors for undergraduate research experiences note the inclusion of undergraduates into their laboratories and research helped expand their research significantly (Russell et al., 2007).
Unfortunately, in 2020, the COVID-19 pandemic canceled numerous undergraduate research internships across the U.S. Valuable career-advancing research experiences disappeared, and many students lost their summer income and housing in the midst of the worst economic recession since the 2008 financial crisis (Sloan et al. 2020). Knowing the longstanding lack of participation of URM students in the sciences, in all likelihood these changes disproportionately affected URM students. The diversity of the geoscience workforce has not changed meaningfully in the past 40 yr, with 8% of doctoral degrees in geosciences awarded to URM (Bernard and Cooperdock 2018).

Partnership Education Program model
The Woods Hole Partnership Education Program (PEP) established by the Woods Hole Diversity Initiative, a consortium of six Woods Hole-based scientific institutions, and the University of Maryland Eastern Shore to address the underrepresentation of African American/Black, Hispanic, Indigenous, and Asian American students in the marine and ocean sciences. PEP's two overarching goals are: (1) recruit talented undergraduates from underrepresented backgrounds to pursue research and ultimately research careers in Woods Hole and (2) help the Woods Hole community become a more welcoming, diverse, and inclusive scientific community (Liles 2019). Concerted participation and resources from all member institutions have been vital in the program's creation and long-term sustainability. Holistic 1 selection criteria, attaining a critical mass 2 of interns, financial support for program operations and full student participation, management and administrative personnel from diverse backgrounds, informal and formal evaluations, and diversity training for research mentors are key program elements that undergird the PEP model. PEP's three primary components are a credit-bearing course, an experiential research internship, and supplemental activities.
The 4-week course, accredited by the University of Maryland Eastern Shore introduces interns to key ocean and environmental science content, builds research skills, and prepares interns for success in their research project. Beginning in 2018 the addition of a 5-d voyage aboard the Sea Education Association's SSV Corwith Cramer provides an "opportunity to devote significant attention to understanding some aspect of the complex marine environment, and to gain first-hand 1 Holistic review refers to the consideration of a broad range of candidate qualities, including "non cognitive" or personal attributes (Kent and McCarthy 2016). 2 Critical mass ensures that underrepresented students are not isolated, with isolation being one of the greatest challenges that underrepresented students face in STEM fields (Allen-Ramdial and Campbell 2014). experience conducting scientific research" (Bower et al. 2004).
Through mini-research projects undertaken at sea, interns gain research skills including literature searches, developing hypotheses, designing experiments, analyzing data, and presenting findings. Mini-projects, in conjunction with exposure to additional ocean and environmental topics, build interns' base knowledge and skills necessary for conducting a successful research project. Additionally, coding and computational knowledge has shown to be necessary for all science careers, thus R (a coding language) has been added to the course content.
During the course, interns begin to transition to their independent research project which becomes the main focus of the internship for the program's final six weeks. Each intern is paired with a research mentor who participated in diversity training and has demonstrated a clear understanding of PEP's overarching goals and guiding principles. The research project serves as an experiential learning opportunity where interns gain skills, experience, and confidence in pursuing geoscience, marine, ocean, and environmental science careers. The research component concludes with a culminating event-a scientific symposium where interns give a 15-min oral presentation of their research project.
Supplemental activities include a variety of career, personal, and professional development that build and sustain a sense of community and belonging, an important and sometimes overlooked component of the undergraduate experiences. Topics such as scientific ethics, writing, and résumé development are included. Field trips to New England museums and sites with rich whaling, historic, and scientific history that further advance and attract underrepresented students in the marine and ocean sciences round out these activities. Additionally, throughout the summer, interns engage with their PEP cohort, staff, and PEP alum participating in other Woods Hole programs. Such informal engagements provide mentorship and networking opportunities. These relationships, whether professional or nearpeer mentoring, are critical for professional growth and developing a sense of belonging in the Woods Hole scientific community.
PEP's pre-COVID-19 success can be measured multiple ways, though most compelling is the statistical data on the PEP alumni. Through surveys and informal, personal communications, we have followed the academic/career progress of 98 of 122 (80%) alumni who have been out of the program long enough to have finished their undergraduate work. Of the 98 alumni, 71% The success of PEP cannot be attributed to any single component, but rather the sum of the components working together. For example, the diversity training for research mentors alone would likely not have the same success without incorporating a holistic intern recruitment process or concepts of critical mass.
As the potential interruptive influence of COVID-19 became clear, the necessity to offer PEP and to continue to serve its local and national communities was an imperative. Thus, PEP became one of few programs that provided a virtual undergraduate research program, by adapting traditional fieldand lab-based research experiences to a virtual environment. As the COVID-19 pandemic continues to interrupt all educational and research experiences, virtual experiences may continue. The virtual PEP experience serves as an example from which other programs can borrow while strengthening and adapting their undergraduate research programs in these uncertain times. In this article, we highlight the decision-making process, the adaptive strategies undertaken, the challenges expected and faced, lessons learned, and broader implications for summer research internships in the shift to a virtual delivery modality.

Adaptive strategies and approaches
Six weeks prior to the start of PEP's planned residential program the reality that an inperson 2020 internship would not be feasible became abundantly apparent. However, the PEP staff recognized the importance of providing an enriching experience for underrepresented students, no matter the uncertainty and challenges posed by COVID-19. We decided unanimously (without serious reservations) that it was important to offer a virtual experience for two reasons: (1) the students we serve do not have the luxury of taking a summer off at what is, for them, a critical point in their burgeoning science careers; and (2) the Woods Hole community cannot afford to take a year off in its effort to become a more inclusive and more diverse community. With the decision made, the staff moved swiftly into the adaptive mode to create an infrastructure that could support the virtual delivery of PEP's three components.
The 2020 PEP staff included African-American/Black, Hispanic, and Indigenous individuals, some of whom were long-time PEP staff and some of whom were PEP alumni returning as new hires. In addition to the PEP director, course director, and coordinator, a course assistant to support the course director, a computational advisor to assist interns in leveraging necessary resources for completing their research project, and a career building coordinator to plan and host career building, professional development, and networking sessions were hired (Table 1).
We were able to hire additional staff with funds unused for in-person costs, such as student travel and lodging, field trip admissions, and so on. Hiring diverse individuals with a strong connection to PEP, who were willing to adapt and be flexible in this uncharted experience, shored up the foundation necessary to make the virtual transition.
PEP interns had been selected and notified of their acceptance prior to the onset of the COVID-19 pandemic. Once the decision had been made to provide a completely virtual experience, we offered the 2020 interns a description of the proposed virtual program (very different in some ways from the program to which they had applied months earlier); we acknowledged that the program is an experiment and that interns would need to commit to learning how to make a virtual program work. We also offered assurances that we were confident we could devise a program that would accomplish the PEP goal of offering the interns a career boosting experience.
Redesigning a 10-week experience across four time zones required foresight, understanding, and addressing the needs of 16 interns. Some interns lived at home with their families, rather than on their own or in a dormitory as they were accustomed. The sharing of spaces, limited internet bandwidth, lack of access to school and office equipment, varying household and family expectations, and additional personal challenges naturally arose. Proactive actions were necessary, and we disseminated preprogram surveys to better understand the technology and resources available to them at home, and to understand where we could alleviate potential challenges. Interns lacking necessary equipment, such as computers or software, to successfully complete the internship were provided with what they needed. PEP sent laptops, a monitor, various cables, and accessories to multiple interns, while PEP's computational advisor helped interns install and operate new software. In addition, the PEP Director and Coordinator communicated with each intern weekly; accountability was critical for foreseeing and addressing their needs.
In addition to the support provided by the PEP staff and research mentor, each PEP intern was assigned a PEP Advisor with whom they met individually on a regular basis (weekly or biweekly). The PEP Advisor served as another connection to the Woods Hole community and provided personal and professional mentorship. At the end of the summer, PEP Advisors felt confident in writing strong letters of recommendation or support for each of their mentees, having spent significant time getting to know them, understanding their interests and goals, and providing connections and support.
Previous studies indicate that success in STEM relies on a sense of belonging. It is crucial that URM feel a sense of community within STEM and perceive themselves as belonging (Marín-Spiotta et al. 2020). To create a connection between the interns and Woods Hole, a care package of Woods Hole and Cape Cod items was sent at the beginning of the summer. Their box included classic mementos of the area such as candy from a local shop, hot sauce made from chilies grown in Woods Hole, a bag of sand and shells from a local beach, stickers, booklets and magnets from the science institutions, and so on (Fig. 1).
Additionally, local PEP staff members filmed scenic areas and places of interest to provide the interns a visual experience of the area.

Implementation factors
Adaptation of the course Offered online, the course's purpose was unchanged: prepare students for success in their research projects by building understanding of key ocean and environmental science topics and helping them to advance their research skills.
To help develop research skills, the structure of a central mini-project and additional lecturing was maintained. Although we could not offer an at-sea experience, interns completed projects using data collected by previous PEP interns and built capacity in conducting a literature review, creating research questions, analyzing data, working collaboratively, and presenting their findings. Given the data-heavy nature of these projects, and that of their upcoming PEP research project, we also invested more time than previous years to train interns in data analysis and visualization using R software.
To expand student understanding of oceanography and environmental sciences, we usually rely on field trips, lab visits, and guest speakers. This has the additional benefit of showing interns around Woods Hole and to learn about the range of research being undertaken in the Woods Hole community. Although unable to visit sites in person, we arranged a series of Advanced Topic seminars featuring prominent scientists from all of the Woods Hole science institutions. Advanced Topics included marine policy, coastal climate change, deforestation, and ecosystems-based fisheries management. This exposed interns to some of the research underway and provided access to leading scientists in their respective fields.
The online format presented many new logistical challenges such as creating a simple, engaging, supportive learning environment, and facilitating group work. To provide a simple, central source of information, we created a class website (https://pepclass2020. netlify.app/) which included the syllabus, information on the course logistics, and daily student instruction. We also simplified communications with interns by using Slack, a communication platform, for all courserelated updates.
For simplicity and ease of engagement, each day was similarly structured. The day began with an asynchronous component, which included reading (a combination of textbooks, research papers, videos, and podcasts) followed by a list of questions to reflect upon or an exercise. Students were required to complete this work by 12:00 h EDT. At noon, we randomly assigned students a peer partner to meet with for 30 min to discuss their answers to the questions posed. These peer sessions also facilitated cohort building. Students then had to post comments or questions to a dedicated #Peer_Discussion Slack channel and the  (1) Coordinator (1) Coordinator (1)  course director and assistant would respond to questions and comments. The students also weighed-in on others' ideas creating an organic discussion environment. At 14:00 h EDT, we held a synchronous 1-h class period on Zoom, where any unresolved questions from the Slack discussion were answered, additional insight was provided, and students discussed topics in break-out rooms. We also used this time for student presentations and for our guest speakers. Additional support was provided throughout the day through virtual office hours hosted by the course assistant and computational advisor.
For the mini-projects, we helped facilitate group work by providing assignments to guide the research process, requiring students to meet with their groups in designated windows each day, encouraging the use of Slack for communication, and providing Google Drive storage space and tools for collaborative document creation. Each group also developed a teamwork action plan which we reviewed to ensure that everyone understood their role. In addition, the online format provided us with an unforeseen benefit: we recruited seven PEP Alumni Mentors to be on Slack and attend group working sessions to guide student groups through their mini-projects. The class culminated in a poster presentation session which we hosted as a series of Google Meets wherein guests could visit individual posters and ask questions of the presenting group.
We asked interns to evaluate and provide feedback on the course (Fig. 2).
One student responded, "This was the most interactive and discussion based online course I have ever taken, which I appreciated. Online classes often make me feel as if I am on my own and this class was the opposite of that." Another noted: "I liked that each topic introduced to us built on the last and by the end of class we could use all of what we learned to complete our miniresearch project successfully."

Adaptation of the research project
The PEP interns and the mentors were selected and matched prior to the COVID-19 pandemic. Once COVID-19 closures rendered PEP a completely virtual experience, we communicated with each mentor to determine if their planned project could be converted to an online-only experience, or if there was another project available for the intern. A few mentors were unable to adapt their projects, but other mentors, including new mentors, were able to take on an additional student. The mentor pool included five first-time mentors and seven returning mentors.
Due to the selection and matching prior to the COVID-19 pandemic, interns were not expecting a completely online, computational research experience and few interns had any coding or computational experience. Some coding training was integrated into the course; however, additional program staff was critical to facilitating the research component, specifically a Computational Advisor (CA). The CA hired was particularly skilled in R, which most students used for their research internship. The CA hosted daily office hours, an open Zoom connection that interns could freely log onto to seek help, or they could schedule individual meeting times. During these office hours and meetings, the CA would review, discuss, and "debug" the intern's code. The CA also hosted informal coding "bootcamps" and posted resources in a designated #Computational Slack channel. In addition to the CA, other staff members who were more familiar with other coding languages, specifically MATLAB, also hosted open and individual "office hours" to assist interns.
During an in-person experience, interns would likely participate in data collection, work with other lab members on their projects, and literally see how their research was an integral part of the lab group. Without having seen any data gathering or being able to assist others in the lab, the interns initially felt disconnected from their research project. To help the interns connect with their research, mentors spent a significant amount of time discussing and reiterating the importance of the research and how the intern's project fit into their group's broader research goals. Additionally, unlike an in-person experience, where mentors and interns communicate throughout the day and organically form professional mentor-mentee relationships, mentors and interns were simultaneously learning how to build virtual mentor-mentee relationships and networks while pursuing research.
Each project and mentor-mentee relationship was different and it is not possible to describe the exact adaptations and changes required to create and implement each research project. However, some general themes arose. Mentors collectively expressed the need for almost daily virtual meetings (via Zoom, Google Hangouts, etc.) with their mentees, and constant communication via Slack, Microsoft Teams, or similar messaging platforms. The mentors who hosted multiple interns, even if working on different projects or from different research programs, expressed that pairing was beneficial to the interns and the mentor. Interns were able to collaborate and produce a productive feedback loop, where interns could communicate the challenges they faced to the other interns, developing their ability to crystallize vague computational concepts, and other students could help troubleshoot, further cementing their growing coding experience.
PEP staff provided critical support to mentors throughout the summer, beginning with a virtual diversity training and ending with a mentor debriefing session. The mentors were provided a #Mentor Slack channel where they could communicate among themselves and with PEP Staff, and access to the Slack channel allowed for direct messaging between the mentors and students, and mentors and PEP Staff. The PEP Coordinator and Director emailed or sent direct messages to the mentors on a regular basis, at least twice per week, and more often in other cases. Halfway through the research internship we hosted a Zoom meeting with all the PEP mentors to discuss current challenges and successes for virtual mentoring, opportunities for adaptation, and shared ideas for enhancing the intern's virtual experience. At the conclusion of the program, a debriefing session was held to discuss the challenges and successes of the research internships, and specifically ideas for what to incorporate into the PEP model for future years, regardless of virtual or in-person activities.
The research project concluded with the public research symposium. The symposium is usually hosted in the Madden Center of Sea Education Association, yet this year it was hosted via Zoom Webinar. This provided an opportunity for the interns to present their research in a scientific symposium and reach an audience wider than the Woods Hole community. Following the symposium, a private graduation was held (Fig. 3).

Adaptation of supplementary activities
During the in-person program, PEP interns usually live on the campus of Sea Education Association, where they share intimate living spaces and depend on each other for grocery shopping, cooking, cleaning, and so on. Strong bonds and feelings of community arise organically, but in this remote setting, events and opportunities meant to foster and facilitate community building needed to be intentionally built into the 2020 program. The supplementary activities became a part of the strategy to facilitate community building. (Table 2).
The career development activities were bolstered, with additional sessions and topics than usually presented in person. Topics usually include an ethics workshop, writing workshop, résumé building, public speaking, and a panel on pathways back to Woods Hole (via the scientific institutions' internships or programs). New, virtual additions by the Career Development Coordinator included: an individual development plan activity, a networking panel, how to craft a personal statement, how to apply to federal positions (specifically navigating the USAJobs website), and managing personal finances. Fifteen PEP alumni participated as panelists for the various activities, sharing insights into their careers and even sharing their successful NSF GRFP and other fellowship application materials. Additionally, interns received a tailored, personal presentation from the Fulbright Program and an invitation to keep in contact with program officers.
For each activity or panel, two or three interns were selected as "moderators." Moderators were charged with moderating the activity or panel, including the Q&A, preparing one to two questions for the panelists, and writing "thank you" emails to the panelist at the event's conclusion. This requirement instilled a sense of ownership over the activities and experience moderating a session. These activities also encouraged community building within the 2020 cohort and between the 2020 cohort and the PEP community.
On alternate Saturdays, a "PEP Rally" was hosted. The first PEP Rally was a virtual "House Party" where interns were placed into different breakout rooms; each with a different theme, game, or activity to promote community building. Other PEP Rally activities included playing "JackBox TV" games over Zoom, a cooking party, where interns cooked the same meal over Zoom with directions from an actual chef (interns were reimbursed the cost of their meal ingredients, to remove barriers of participation) and a "real talk" session, where the men and women had their own "talking circles," led by Dr. Ambrose Jearld, Jr. and Dr. Anna Martin-Jearld. In the "real talk" session, the interns were provided a space to express and process their feelings around being an URM in geosciences, the current COVID-19 pandemic, and the impact of current racial tensions on their lives.
During the research component, the PEP Coordinator hosted weekly "coffee breaks", where interns could drop in as they wanted to chat with each other and staff members. Recreating an actual visit to a local coffee shop, interns were able to "pop" in and out of breakout rooms. Each year PEP interns design their own t-shirts, and 2020 was no exception. The interns' shirts were sent to them individually, with a 2021 Woods Hole calendar, to help reinforce their Woods Hole connection. Together, these intentional community building events were a critical step toward fostering a sense of belonging to the geosciences community.

Lessons learned
While the guiding principles remained the same, significant program management and delivery adaptations were crucial to PEP's virtual implementation. We share some of our learnings below as we reflect on the broader implications of PEP's decision to Communication was critical, with interns and among staff, beginning with frank discussions about the virtual program and continuing throughout the summer. Clear communications is, of course, always critical to the success of a program, but it was especially challenging in a setting where almost all communications were via electronic media. The Program Coordinator is responsible for, among other things, maintaining open and clear lines of communication between the PEP staff, mentors, and interns at all times. In this virtual setting that required significantly more time and proved more challenging than in-person; the coordinator was not able to speak to interns at home (on the SEA campus) or stop by the laboratory to check in on interns and mentors together. Instead, the Coordinator relied on Slack, Zoom, calls, and texts for all communications (Table 3).
This foresight, into the time commitment required to maintain communications, led to the development of the Career Development Coordinator and Course Assistant, relieving the Coordinator of those specific duties.
Three new staff positions (Career Development Coordinator; Course Assistant; Computational Advisor) were necessary to meet the need for increased individual  Assistant played a key role in facilitating the course, specifically by promoting classroom discussion, managing the mini-projects, and integrating the Advanced Topics into the course. The Computational Advisor was key in the success of the computationally rigorous internship. Overall, the most critical element in PEP is the commitment of the staff to the program's principles. This commitment is the bedrock of the in-person, residential program and was the driving force in the building of an online program.
Adapting an immersive course to completely online was challenging, but not impossible. By inviting guest speakers as part of the Advanced Topic series and using PEP alumni mentors and data for the miniresearch projects, we encouraged a connection between the interns, the Woods Hole community, and the PEP alumni. Requiring group and teamwork as components of the course work, and thus their course grade, encouraged community building and a sense of belonging.
The research internship required flexibility on the part of staff and mentors, most of whom had never before mentored a student remotely. The staff and mentors had to be willing to design projects that focused on data analysis rather than data gathering, to use new methods of communication, and to be sensitive to the conditions under which the interns worked. The interns likewise had to be flexible-a requirement we emphasized at the beginning when it was announced the program was being offered virtually.
Mentors found the communication and availability of the PEP staff was a key resource and appreciated the addition of a midsummer check-in. They requested additional check-ins with staff and other mentors in future summers, whether hosted in-person or virtually. Mentors noted that most challenges they faced through the summer were predictable yet difficult to overcome, such as home/family obligations interrupting scheduled meetings (for both mentors and interns), pressures of the social-political unrest, and the reallife immediate implications of the COVID-19 pandemic. Despite these challenges, mentors felt overwhelmingly that the program went well and they were proud of the research completed and the final presentations presented in the virtual symposium. Ultimately, the mentors who were able to accommodate an online research experience adapted not only their project but also their mentoring style, means of communication, lab meetings, and expectations.
Ongoing program evaluation, with debriefing of staff and mentors, and intern feedback have been an integral part of PEP activities. Through our external evaluation, we learned the interns had an overwhelmingly positive experience. Students ranked the components/activities that were most useful to their education and career goals: research and career development supplemental activities, presentations and community building supplemental activities, course and community lectures, and computational/tech support. One student said, "I plan on using the research experience and knowledge I gained from PEP to help me get into and thrive in future career opportunities and internships. I will do so through the use of my newfound coding and networking skills." Another stated, "PEP has helped me better understand my identity as a scientist and researcher, and I will use that to push myself in my career as far as I can."

Conclusion
The physical lack of the PEP interns in Woods Hole could be considered a primary disadvantage of the virtual modality. Specifically, to be present and experience the laboratory, participate in fieldwork firsthand, explore the community, and meet local scientists and other interns are removed in a virtual program. However, the virtual delivery of PEP offers potential adaptive strategies to address the challenge that may also be encountered by other internships in STEM as they consider alternative delivery modalities in the face of external exigences.
The emergence of PEP graduates as significant forces in the program was a major factor in the 2020 virtual program, demonstrating that PEP is no longer just a program; it has become a community with underpinnings strong enough to engender its survivability and even prosperity in changing times. More than a dozen PEP graduates joined the staff and/or served as volunteer mentors and panelists, playing key roles in building and delivering the virtual program-an unexpected benefit.
The decision to deliver PEP virtually was in a real sense an experiment. Given that the program seeks in part to support careers in science, a complete assessment of program effectiveness cannot be accomplished until the interns finish their undergraduate degrees and are able to launch careers in science. However, the immediate evaluation of the 2020 program reveals initial successes, and we confidently enumerate some critical elements in delivering a virtual program to support undergraduates, with a focus on undergraduates from communities underrepresented in science.