Rebooting nuclear medicine specialist education under the COVID‐19 pandemic: From plenary lectures to active e‐learning

Despite major reforms of specialist training in the Nordic countries towards concrete learning outcomes and promoting active learning, most specialist courses continue to be based on lectures. We redesigned our mandatory 5‐day course in clinical nuclear medicine (NM) that was last held in 2016 towards active learning. Thirty 1‐h lectures were replaced with 10 thematic blocks of 3 h each. Each block was taught by a single teacher in a blend of short introductory lectures alternating with small groups of residents reading NM cases from our newly established national case library in diagnostic format. Due to COVID‐19, the entire course in 2021 needed to be run on a videoconferencing system rather than in a computer laboratory as had been originally planned. At the end of the course, we conducted the same anonymized survey as in 2016. All 19 course participants responded. 74% fully agreed that the e‐course format had been ‘good’. One hundred per cent fully agreed that the practical exercises were ‘useful’ versus 50% in 2016 (p < 0.001). In their free text answers on the merits or downsides of e‐learning, 12/12 respondents only mentioned advantages. Our newly established library of anonymized teaching cases within our national health network is an effective tool for organising courses based on active learning. Despite the change towards distance learning enforced by the pandemic, course participants reported the same high levels of satisfaction with active learning in small groups as in the earlier traditional lecture‐based course format.

A key element in the recent reforms of medical specialist education in the Nordic countries was a shift towards outcome-based education (Norwegian Directorate of Health, 2014): Detailed catalogues of attainable and verifiable learning outcomes were defined for each specialty (Norwegian Health Department, 2021).While most European countries conclude medical specialist education with a final oral and/or written examination, Nordic countries instead have a focus on mandatory course education (Hellwig et al., 2012).

Despite efforts from the Norwegian Ministry of Education and
Research to move university teaching from a traditional lecturebased model to more active learning strategies and measurable learning outcomes that are in line with modern pedagogical research and practice (Biggs et al., 2022;Dent & Harden, 2021), the medical specialist education courses being offered in Norway have remained largely unchanged, consisting almost entirely in a lecture-based based format.This applies to medical imaging-both nuclear medicine and radiology-as well as to most clinical specialties such as internal medicine.
When we arranged the mandatory 30-h course in 'Clinical Nuclear Medicine' (NM3) for nuclear medicine residents in 2016, we were still bound to the original format of 30 1-h lectures covering most topics in nuclear medicine under the prevailing system; however, we tested out supplementing the lectures with active e-learning assignments on diagnostic workstations.In their responses in an anonymous user survey at the end of the course, course participants criticised that the active learning elements were only offered as an add-on beyond regular course hours, not as an integral part of the course (Haslerud et al., 2017).
To meet this criticism, we redesigned the course from the ground up with a focus on active learning and student engagement (Barkley, 2010;Fink, 2013): We reduced the number of topics from 30 to 10 and planned a blend of short introductory lectures alternating with group work on diagnostic workstations in our hospital's computer laboratory attached to a single local teaching database.However, due to the COVID-19 pandemic, all face-to-face (F2F) teaching suddenly came to a full stop, and the course needed to be arranged remotely on a videoconferencing system.As in our NM course for medical students at the onset of the pandemic (Biermann et al., 2021), we were faced with a choice: Should we return to traditional lecturebased teaching with video-streaming?Or should we implement the new course design, but combine it with distance learning at the same time?The latter scenario entailed the added technical challenge that our teaching cases needed to be available at all participating institutions.We prioritised pedagogy over expedience and chose to stick with our new design but adapt it to the pandemic conditions.
In the following, we describe the new redesigned course, 'Clinical Nuclear Medicine', which we based on active learning and conducted on a videoconferencing system between 31 May to 4 June 2021.We then compare user feedback with the same end-of-course survey as administered in 2016, when we offered the original lecture-based version of the same course (Haslerud et al., 2017).

| Course structure
The course 'Clinical Nuclear Medicine' was held as a fully digital course between 31 May to 4 June 2021.Each of 10 topics was allotted 3 h.Each topic was taught by one of nine NM specialists in a blend of short introductory lectures interspersed with reading of relevant clinical cases in small groups followed by plenary discussions.Groups met in separate break-out rooms via the videoconferencing system, and the group 'leader' loaded the image data sets in the diagnostic software on her workstation and shared the screen with the other group members.Topics no longer covered by the redesigned course were integrated into our newly created weekly nuclear medicine courses on organ and cancer imaging NM4 and NM5 (File S1).

| IT infrastructure
The course was based on the newly established national teaching architecture for nuclear medicine, which consisted of the following: (i) video conferencing system, (ii) learning management system (LMS), (iii) central teaching database containing anonymized cases in diagnostic format with matching diagnostic image viewing software installed on the connected hospital workstations.Each of the three components can be replaced by another technical platform independent of the other components if need arises such as student lock-out under the pandemic (Biermann et al., 2021).
Ad i: The video conferencing platform (https://whereby.com/)was run on regular hospital workstations at all participating hospitals.
Ad ii: Moodle LMS (https://moodle.org)running on a dedicated server connected to the Internet (https://nukit.ihelse.net) was used to deliver all teaching materials, including multiple choice questions (MCQ) for the practical exercises as previously published (Gulati et al., 2019); other platforms such as Canvas LMS (https://instructure.com) are similarly capable (Biermann et al., 2021).
Ad iii: The teaching database containing anonymized teaching cases in native digital imaging and communications in medicine (DICOM) format on a dedicated server attached to the Norwegian Health Network, separate from the Internet.To review the imaging data in diagnostic format, course participants need to run dedicated client software (Segami Oasis; https://segamicorp.com).In the beginning, the teaching database was accessible only at Haukeland University Hospital (Gulati et al., 2019).After 2019, access was established for all Norwegian teaching hospitals with nuclear medicine departments.As the largest teaching hospital still lacked access at the time of the course, we transferred the teaching datasets to the nuclear medicine client-server system at that hospital as an alternative image viewing platform (syngo.via;Siemens Healthineers, Hoffman Estates/IL).
Further implementation details of our e-teaching architecture are given in File S1.

| Course organisation
All teachers received personalised support by the course committee (Andreas Tulipan, Ankush Gulati, Torjan Haslerud and Martin Biermann).Teachers were encouraged to split their existing 45-min lectures on their topic into smaller thematic units and asked to provide matching clinical cases from their own institutions.The original DICOM data of the teaching cases were anonymized and uploaded to the national server.Clinical information on each of the cases was distilled into case vignettes for matching MCQ on the LMS.
Each of the thematic units was to be taught in two or three cycles of the following sequence: short introductory lecture, group exercises, and review of the exercises in a short plenary session, intermission.
Since it was difficult to predict how quickly groups would progress through the cases under the course, we encouraged teachers to prepare extra cases so that all students could be kept active in the allotted time frame.To instruct course participants in how to use the digital tools, we prepared a quiz that walked the participants through all necessary practical tasks, including loading an anonymized patient study from the database and sharing a diagnostic screen with a colleague at a remote hospital.In addition, we recorded a 6-min screencast demonstrating the use of the diagnostic image viewing software (https://www.uib.no/en/radioweb/nuclear/teaching).Since teachers already were fully committed to generating new course content in relatively short time, the course committee decided to eliminate the MCQ examination that had been given at the end of the previous courses.Fisher's exact tests were applied at a significance level of 5% (twosided), respectively.Free text answers were translated from Norwegian.Plots were generated with the R library ggplot2 (Wickham, 2009).Overall satisfaction with the course was high: 14/19 (74%) participants fully agreed that the e-course format was 'good' (Question 6), and 85%, 81%, and 76% participants fully agreed that the lectures, exercises, and discussions had been 'interesting and relevant' (Q2-4), respectively.

| RESULTS
F I G U R E 1 Comparison of the surveys at the end of our courses in nuclear medicine specialist education in 2021 and 2016.Statistically significant differences are indicated by asterisks (*).Abbreviations: Suppl, additional materials (such as guidelines); lecturers, contact/get acquainted with the instructors; peers, get acquainted with other course participants; cases, practical exercises; remote, remote access.
In the statistical comparison of the survey responses after the courses in 2021 and 2016, only the following survey items were significantly different: 100% of respondents in 2021 fully agreed that the practical cases had been 'useful' versus 50% in 2016 (p < 0.001; Q10).78% of course participants expressed a wish for more contact with their peers versus 0% in 2016 (p < 0.001).Thirty-eight per cent in 2021 requested more lectures versus 8% in 2016 (p < 0.01).Sixtythree per cent asked for more imaging cases versus 25% in 2016 (p < 0.01; all Q16 and Figure 1).
The free text question on the perceived utility or downsides of e-learning (Q13) was answered by 12 participants.One hundred per cent of the answers emphasised only benefits of e-learning.When specifically asked for the weaknesses in the course, six out of nine free answers named technical problems (Q21).When asked to make suggestions for future iterations of the course, six of nine respondents recommended maintaining the new course format.
Three asked for a physical meeting, and one requested more training for the image processing software to be used during the course (Q22).Fortunately, half of the institutions already were able to connect to our teaching database through the national health network, and the remaining institutions could get access to the cases via the clinical system run at the country's largest teaching hospital.Since January 2023, our national teaching database has become fully operational with seamless access to cases in diagnostic format from all hospitals teaching nuclear medicine residents and is used throughout the year in our weekly courses for nuclear medicine residents (http://nukit.

ihelse.net/moodle).
Given the double challenge of changing the course structure from traditional lectures to active learning at the same time as the mode of delivery from F2F to videoconferencing under pandemic conditions, we were surprised that course participants in 2021 were no less satisfied than in 2016.Not all studies on the transition from lectures to active learning report increased levels of student satisfaction and engagement; in fact, student resistance is often more common (Tharayil et al., 2018).In their survey among 60 radiology residents on a distributed e-learning resource for radiology in the United Kingdom, residents criticised that e-courses did not prepare them for the detailed knowledge required in written examinations even though they provided for deeper learning (Upadhyay & Wadkin, 2021), which in our view relates to lack of constructive alignment between teaching and final examination (Biggs, 2014).In our recent student course in nuclear medicine, which was based on a similar active learning architecture and also taught under pandemic conditions, only 51% of medical students in their 5th year approved of the course on the two highest levels of a 5-point Likert scale (Biermann et al., 2021).A recent study on teaching nuclear cardiology to 244 medical students in a classroom versus a distance learning setting revealed lower satisfaction scores for distance learning (Bell et al., 2022) While teaching manual skills as in surgery or endoscopy requires simulation rooms and/or equipment that will usually be restricted to a few central locations (Kneifel et al., 2023), all that is needed for active learning in the medical imaging specialties is network access to anonymized digital datasets.For radiology, a vast number of proprietary as well as open-source tools are already available.At the one end of the continuum, one could provide a national picture archival and communications system (PACS) containing only anonymized cases with software clients at all teaching hospitals.On the other end of the spectrum are purely Internet-based viewers that rely on a modern web browser as the only client-side component.An example is the teaching website https://radiopaedia.org that hosts a variety of courses on all radiological modalities, including magnetic resonance imaging (MRI).Image viewers for nuclear medicine are special in that alpha blending (colour overlays) is needed for the optimal rendering of hybrid datasets such as PET/CT (Biermann et al., 2021).We chose a commercial solution for our national teaching database that is similar in design to a full scale PACS system with our LMS serving as the radiology information system (RIS) component.This solution has been fully operational in the Norwegian health network with seamless client access from all teaching hospitals in nuclear medicine since 2023.An Internet-based viewer with support for alpha-blending, zooming and windowing is under active development by Kanoun and coworkers (Biermann et al., 2021), which can be directly incorporated into Moodle LMS for nuclear medicine and/or radiology courses on the Internet.Gulati et al., 2019).Results from a follow-up study in 2021 in the same cohort suggest that PET/CT reading skills are retained over up to 2 years in the majority of students (Biermann et al., 2021).

| Limitations
In conclusion, blended learning based on the combination of classroom instruction and online teaching is an important emerging mode of instruction also in specialist education.When combined with meaningful online activities such as reporting on imaging studies in diagnostic format, it can enhance learning while freeing classroom time for discussion and critical reflection.
Course participants were asked to complete the same anonymous survey as in 2016 with only minimal adaptations to the new course format via the Moodle 'Feedback' activity comprising 11 MCQ and 12 free text questions (Figure 1; File S2).None of the course participants made use of the opportunity to reserve themselves from participation in the anonymous survey and subsequent publication of the survey results.The survey was approved by the institutional data protection officer.For data analysis, the anonymous survey data of the courses in 2016 and 2021 were downloaded from the Moodle Feedback module and imported into R (R Core Team, 2021).For comparison of Likert scores and binary responses in 2021 versus 2016, Mann-Whitney-Wilcoxon (de Winter & Dodou, 2021) and All 19 course participants (100%) responded to the anonymous survey at the end of the course versus 12 out of 15 course participants (80%) in 2016.Seventy-four per cent of course participants were female versus 64% in 2016, while 37% were employed at the country's largest institution versus 29% in 2016.The detailed results are presented in File S2.
To our knowledge, this was the first course in Norwegian medical specialist education that combined videoconferencing-as necessitated by the COVID-19 pandemic-with active learning.Originally the course had been planned as a plenary course held in the hospital's computer laboratory equipped with 15 diagnostic workstations attached to a single local image database.Scaling up the course to electronic format distributed over nine participating institutions required access to the teaching cases from all sites.
. To our surprise, the nuclear medicine residents in our 2022 nuclear medicine course reported similarly high levels of approval as in our original lecture-based course in 2016 despite the transition to distance learning.While the satisfaction scores from the present survey indicate that our new course design worked, they cannot be used to infer which teaching approach-lectures versus active learning-is superior.The major change in the responses between 2021 and 2016 concerns the attitude to active learning: 100% of the course participants in 2021, but only 50% of the respondents in 2016 affirmed the value of imaging cases in their training.We interpret this as confirmation of our pedagogical intentions.The 2016 nuclear medicine course was the first to introduce reading anonymized cases in diagnostic format, if only as voluntary add-on activity after regular course hours (Haslerud et al., 2017), whereas in 2021, reading cases was integral part of the course work.That more course participants in 2021 than in 2016 would have preferred more lectures (38% vs. 8%) implies that the lectures in the present course were perceived as useful and relevant whereas regular course hours in 2016 were restricted to lectures.The desire of 78% of participants of the 2021 course for more meetings with their peers was specific to the videoconferencing format forced by the pandemic; this wish had been met in the lecture-based course in 2016, which had been F2F.However, free-text answers from 50% of the course participants in 2021 illustrate how videoconferencing in combination with group work in small breakout-rooms provided a platform for getting to know colleagues across institutions even under pandemic conditions.Most of the technical issues mentioned in the free text responses have been solved, and since January 2023, all teaching hospitals have seamless access to the central image database in the Norwegian health network.For the next iteration of the course in November 2023, we are planning a blended design with 2 ½ days of F2F teaching in a computer laboratory on our campus to provide opportunity of personal interaction across institutions, and 2 ½ days of videoconferencing to reduce costs for teaching infrastructure and accommodation.The 2021 course on 'Clinical Nuclear Medicine' based on active learning principles opens up new modes for specialist training for teaching medical imaging-not just nuclear medicine-to residents and specialists.Based on the same framework of video conferencing/ streaming, LMS and image database, we established two extra mandatory courses for nuclear medicine residents on imaging organ function (NM4) and oncological imaging (NM5) since 2019.Each of the 30 course lessons is introduced by a 30-min streamed live lecture.The course participants are then given tasks on the LMS that include an MCQ quiz that covers material from the day's lecture, supporting open-access articles on the LMS and one or two cases from the national teaching database (File S1).Course NM5 has been arranged three times, and NM4 is at present in its 3rd iteration.We also created four electronic courses for continuing medical education (CME) on positron emission tomography (PET) for prostate cancer, neuroendocrine tumours, thyroid ultrasound, and scintigraphy in the endocrine neck counting five CME points each under the auspices of the Norwegian Physician's Association.
(i) Our study is limited to a specialist course in nuclear medicine.While the same course architecture could be used for teaching radiology, the number of course participants in comparable Norwegian radiology courses is usually in excess of 120 participants.(ii) While the manuscript describes a particular technical implementation for the interactive presentation of medical imaging data in the context of the Norwegian Health Network, a wide range of alternative networked solutions is available.(iii) We lack documentation of learning outcomes as we chose to eliminate the examination at the end of the course.However, we have previously tested learning outcomes at the end of an e-learning course with similar practical assignments in third year medical students in the form of an objective structured clinical examination (OSCE).Ninety-five per cent of the students passed the OSCE against predetermined performance criteria (