Making each other smarter: assessing peer mentoring groups as a way to support learning system dynamics

Between 2015 and 2018, the Student Chapter of the System Dynamics (SD) Society sponsored the creation of nine peer mentoring groups (PMGs). PMGs are widely recognized as an important way to support learning and professional development, especially by novices (Ashwin, 2003; Terrion and Leonard, 2007; Huizing, 2012; Dhirasasna et al., 2021). This note reports on the experiences of members of these groups, leaders of the nine groups, and senior faculty membersi who volunteered to work with some of the groups. Many PMG members were PhD students, while others worked in academia but not as PhD students or in industry and consultancy.

• Presentations usually about participants' current research, but other SD topics also welcome. • Feedback and questions throughout the meeting or at the end of the presentation at the presenter's request.
Some groups had fixed meeting days and times, which were determined at the beginning of the year during a kickoff meeting to accommodate as many group members as possible. Others preferred to be flexible and set a time at the end of each meeting by considering personal schedules. For this research note we sent a survey to all current and former members of the groups. Next, we used a semi-structured interview protocol to interview the leaders of the groups as well as senior faculty members who were embedded in some of the groups. Results of this research are presented below after a brief discussion of how this work relates more broadly to novice learning in system dynamics.
Peer groups and mentoring in the system dynamics community One purpose of this note is to provide some empirical evidence to support ongoing discussions within the SD community of how best to enable learning about system dynamics. Figure 1 presents a rough taxonomy of how we see our work, presented in this note, as fitting within a broader framework.
The heavy dashed line in Figure 1 illustrates the boundary of our work. Our pilot project fits broadly within a cluster of experiments and ongoing activities in the area of mentoring and tutoring.
Of course, how mentors and mentees get connected can and does vary. Commonly, academic programs, especially PhD programs that feature system dynamics, connect faculty and students in expert-novice mentor-mentee relationships. These mentoring relationships also contain a substantial component of tutoring. We distinguish tutoring from mentoring by the efforts that tutors make to teach technical details of the system dynamics craft. We believe that the new one-on-one mentoring (as opposed to peer mentoring) program recently created by the SD Society is a good blend of mentoring with tutoring activities.
Beside tutoring within PhD programs and as indicated in Figure 1, other important tutoring pathways also exist in the field of system dynamicsimportant complements to the work we are describing here. The grayed-out areas of Figure 1 serve to remind us all that tutoring and mentoring must exist within a broad field of teaching and learning activities. Increasingly, face-to-face classes are being matched with fully online and blended class offerings ii that may ultimately culminate with fully developed MOOCs (Massive Open Online Course).
Of course, online instructional materials extend beyond those aimed specifically to support online classes. Short videos modeled after the Khan Academy (Khan Academy, 2020) come up repeatedly as a missing instructional capability in our field. Online cases (Meadows, 2007;Sterman, 2014;Ku et al., 2016;McFarland et al., 2016) are examples of online materials that are filling an important gap. Importantly, there have been several attempts to define the goals of well-formed learning programs for system dynamics (Martinez-Moyano and Richardson, 2013;Schaffernicht and Groesser, 2016).
The largest gap in our existing inventory of learning material appears to be in the broad area of evaluation, accreditation, and certification. As learning modules become more and more detached from university course-based learning, the need for some form of recognition of student learning and achievement becomes more acute. Various regimes ranging from simple "badge" systems recognizing specific and limited proficiencies to a fullfledged competency-based professional certification process have been under As examples see (Worchester Polytechnic Institute, 2020) or the "Roadmap" series (www.systemdynamics. org/road-maps). discussion at various times. iii This topic easily turns into a disputed conversation that is beyond the scope of this work.

Surveying PMG members to characterize PMGs
To characterize existing and no longer existing PMGs, an online survey was distributed among past and present members of nine PMGs (five still existing, four no longer existing). The questionnaire consisted of four sections and was designed according to the principles laid out by Iarossi (2006) and Horvat et al. (2019). The questionnaire is presented in the supplementary material SI-1.
In total, 24 responses were collected. Nine respondents belonged to PMGs that no longer existed, while 15 were from existing PMGs. Respondents lived in 11 different countries on four continents (Africa (n = 1), Australia and Oceania (n = 3), Europe (n = 16), North America (n = 4)). The most represented countries were Germany (n = 6), the Netherlands (n = 4), and the United States of America (n = 4). The final survey sample is further described in Figure 2. Additional demographic data on PMGs are presented in the supplementary material SI-8. Summary of the results of the survey distributed among PMG members iv Respondents' membership in online PMGs and relationship with their peers Figure 3 shows that the number of people joining a PMG slowly increased from 2015 to 2018. The majority (n = 18) of respondents joined their groups by attending a peer mentoring meeting organized by the Student Chapter at international conferences of the System Dynamics Society. Table 1 shows that respondents from both existing and no longer existing groups found it important to be a member. However, more people from existing groups found it extremely important (mode = 10). Respondents mainly joined PMGs to connect with people practicing SD to discuss and exchange SD-related thoughts. Respondents from no longer existing groups left their PMGs because the group became inactive (e.g. because nobody took the lead to schedule meetings or the group ran out of topics to discuss). There was a substantial difference between how respondents from existing and no longer existing groups saw their peers. Some of the former saw their peers as friends (20 percent of respondents replied that they saw all of their peers as friends, 60 percent saw some of their peers as friends and some as colleagues,  and 20 percent saw all of their peers as colleagues only). The latter thought of their peers mainly as their colleagues (78 percent thought of their peers as colleagues only, while 22 percent saw some of their peers as friends and some as colleagues).
Respondents' presenting in PMG meetings and the usefulness of peers' feedback Figure 4A shows a clear distinction between the frequency of presentations by respondents from existing and no longer existing groups. The majority of respondents from no longer existing groups presented only once before the group became inactive. Respondents from existing groups mainly presented more than once. As shown in Figure 4B, work in progress was the main content of respondents' presentations in both groups. Figures 5A and 5B show that respondents from existing groups mainly thought of their peers' feedback as helpful as well as useful in solving their SD problems. However, within no longer existing groups, there was a bifurcation between respondents who thought the feedback was helpful and useful versus indecisive persons (those responding in the middle of the Likert scale).
Respondents' assessment of the presence of a faculty adviser Eight respondents from existing groups stated that they currently have a faculty adviser and two respondents used to have one. Within no longer existing groups, six respondents stated that their group had a faculty adviser. Figure 6 shows that all respondents from no longer existing PMGs and more than half of the respondents from existing groups thought that the presence of a faculty adviser positively affected the quality of feedback. However, a mode of 5 indicates that some respondents from existing groups were indecisive.
Within the existing and no longer existing PMGs, respondents stated that faculty advisers positively contributed with their unique sets of skills and knowledge, by giving feedback when other peers could not, and by providing learning resources. Three respondents thought that senior members had a potential to retain more members and were a driving force of the meetings. Some respondents from existing PMGs critiqued that some senior members did not play the role of a peer but more of a teacher. Respondents from no longer existing PMGs mentioned senior members' lack of time to attend every meeting as a negative factor.
Respondents' perception of positive and negative aspects of joining a PMG Respondents from existing and no longer existing PMGs most frequently mentioned learning SD, networking within the field, and getting diverse feedback as positive aspects of joining a PMG. Some respondents stated that due to PMGs they gained confidence in their own skills. For some, PMGs have been critical in staying motivated and finishing their studies or in continuing practicing SD. Many respondents from existing PMGs stated that this activity is time consuming but some of them do not mind that. Furthermore, respondents mentioned problems related to peers' skill levels, i.e. there are fewer learning opportunities if peers' skill levels are on a low or similar level, and it can be problematic to keep more advanced peers interested. Respondents from no longer existing PMGs mentioned the lack of dedication and persistence of peers, and the lack of structure and organization of meetings as the most negative traits of PMGs.
Interviewing PMG leaders and faculty advisers to gain a deeper understanding of peer mentoring experiences Following the survey, which aimed at characterizing the PMG groups, data collection consisting of semi-structured in-depth interviews of PMG group leaders was performed to gain a deeper understanding of the topics that were addressed in the survey. Overall, nine group leaders were interviewed, with five of them representing still existing PMGs, to increase understanding of their role in PMGs and to discuss how to move peer mentoring forward. The protocols for interviewing group leaders and faculty advisers are presented in the supplementary materials SI-2 and SI-3, respectively.
We condensed the findings of the interviews with group leaders and faculty advisers on the following pages of this note. The full results from these interviews have been written up in the supplementary materials SI-5 and SI-6. The extended material will be of special interest to persons wishing to learn more about online peer mentoring.

Summary of the results of the group leader interviews
Existing PMGs: what made them survive?
The following factors were mentioned as potential survival factors: • Having a strong group leader who fulfills a manager role and plays "more or less of the glue" to hold the group together. In fact, it does not have to be only one person. Group leaders might rotate. • Creating a schedule for presentations within one semester or even 1 year.
• Treating each other with respect and trying to help each other with comments and feedback. • Forming a group with people who could be potential friends.
• Being highly motivated. Peer mentoring is a task everyone should see the value of themselves but not be forced into it. • Celebrating each other's achievements within the group, such as conference paper or journal paper publications, presentations, or finalization of a PhD project. • Having cameras on. Human relationships keep such groups going.
No longer existing PMGs: why did they not "survive"?
Four out of nine formed PMGs fell apart not long after their formation. The reasons for the groups collapsing differed, but all group leaders said it was a shame they stopped meeting and wished they could somehow continue with this activity. Some reasons for those groups not surviving were: • Lack of leadership or being inflexible in terms of leadership change.
• Lack of dedication and persistence of all group members to continue. • Members' availability and busy schedules. In fact, a time commitment was also mentioned by group leaders of still existing groups as being a significant downside of forming a PMG. However, as a few interviewees pointed out, "it is time well spent." Why is peer mentoring important to the PMG members?
Even though some of the formed PMGs did not survive, all respondents agreed on the importance of such groups. The following benefits of being a part of PMGs were mentioned: • There is always so much to be learned from others doing similar work, whether that is a topic, domain, or process, as this type of knowledge can be useful in the future. • Regular feedback and exchange on work in progress help to spark new ideas and fill knowledge gaps. • Practice presenting SD work is very useful as it is a pilot test before the work is presented to a more formal audience. • Given that many people are self-taught, it is helpful to get recommendations for further SD reading. • Having regular meetings for exchange on SD applications and research is especially valuable for PhD students and practitioners who are the only person working with SD in their daily working environment. • Forming friendly relations and creating personal ties.
• PMGs being a "game changer" in building a career in SD and finishing an SD-related PhD. • Being more connected to the wider SD community and expanding professional networks.
Overall, all respondents agreed that PMGs are vital as they advance the development of professional skills and help meet the need to quickly activate a critical reinforcing loop important to the growth of the field. Within this loop, enhanced professional skills reinforce the retention of new members, increasing the base of resources that can be applied to create beneficial relations among members (such as PMGs) that further enhance professional skills.
What needs to be done to support and promote PMGs?
As one respondent remarked, "the SD Society looks like a typical 'aging population' model. We need newcomers who do not drop out but become professionals. Peer mentorship is a crucial activity for producing knowledgeable system dynamicists." To support this belief, group leaders offered some valuable insights into what could be done to better support peer mentoring to spread SD knowledge.
• There needs to be more effort by the SD Society to promote the utility of PMGs through organizing a work group that would study and develop principles on how to form and manage PMGs. • Some team-building activities could help people connect better-either online or right at the SD conference. • Groups should be instructed what to do when there is nothing to present, how they can continue being active so that the meetings continue being useful for everyone. • If a group does not have a strong leader (and it falls apart), somebody from the outside could check who is motivated to continue and help them by pushing them to new group. If only some are motivated, they could be connected with people from other no longer existing groups and form a new group. • Teaching people how to spot mistakes in models and how to give constructive feedback could be helpful. It would be useful to have a moment where all groups could join some webinars where they could learn something that is useful for the survival and functioning of PMGs. • For many groups, faculty advisers who had been part of a group for many years were powerful motivators. Having experienced people involved helps. • Group formation process needs to be restructured to become more formal.
Many people might not know others in the Society so fostering those relationships would be important. The process of forming the groups should be a main point at the annual SD conference, and not just a side meeting. For example, having a plenary presentation and/or a panel discussion in front of the whole conference to talk about how and why people should be part of peer mentorship would be extremely helpful. At the very least it could take place during the SD Colloquium, assuming enough senior modelers are present.

Summary of the results of the faculty advisers' interviews
What was the role of faculty advisers in PMGs?
This question was the main focus of the interview protocol and took 50 percent to 80 percent of the time in each interview. In their interviews, the faculty respondents reflected on three questions: (i) What did faculty participants actually do while interacting with their groups, (ii) What factors seemed to move the group forward, and (iii) What factors inhibited the peer mentoring process.Most of the respondents moved beyond these logistical questions to comment on the issue of role definition. Respondents contrasted the sometimes competing and sometimes cooperating roles of classroom instructor, individual tutor, and mentor (both individual and peer group mentors). All the respondents were experienced in all three of these roles and many of them reported creative tensions around maintaining a balance between these three supporting roles.
Why have peer mentoring?
Faculty respondents crafted thoughtful responses to the question of why have peer mentoring at all. One respondent characterized many of the students in system dynamics PMGs as "locally lonely," possessing a passionate interest in system dynamics coupled with an inability to connect with working partners, colleagues, and mentors in their immediate work environment. This, she argued, was a fate not only of PhD students but also of a large fraction of those seeking to pursue academic and other professional work in our field. Another respondent continued, "But we really need to 'do it right.' We need to train a cadre of facilitators and equip them with the tools that have been proven to really work in supporting distributed collaborative work. We need to have guidelines and rules that all the participants understand and are prepared to follow. This is important work and we need to discover what is known to be working out there and make use of the best co-facilitation tools that have been proven to be effective." What needs to be done to promote peer mentoring?
The respondents were quick to suggest activities that could support the creation of more widespread peer mentoring. Four typical suggestions included: • Make this a long-term program. The SD Society should put in time, effort, and other resources to help manage lists, recruit persons, manage times, and maybe create supporting platforms. • Create a screening process to ensure that group members meet some minimum level of competency before entering into engagement with a group. • Find some way to use the modeling assistance workshops as an "on ramp" to these PMGs. • Create some sort of an evaluative process-the kind of evaluation that aims to be supportive and helps all to improve what they are doing.
What other types of supporting resources need to be created?
Finally, all the respondents believed that effective peer mentoring needs to be embedded within a wider context of supporting resources. Five typical suggestions were: • Create open access to books and other online learning materials. Out-ofprint books such as the Richardson and Pugh III (1994) book or Nancy Roberts et al.'s book (1994) should be placed in open access. • Follow up on the idea of a "Forrester Academy" focused on the production of Khan-Academy-like video materials. The Khan Academy is a collection of short instructional videos organized around some technical topic-see Thompson (2011). Then, when a question comes up, mentors would have materials to introduce. • Create "check your understanding" evaluative-type material with adaptive feedback so that students who are learning can check their mastery. • Create complete MOOC-like material modeled on the work that Davidsen et al. (2014) and Pruyt (2013) have done along these lines. • Put together a "training guide" for mentors, creating tips and pointers for how mentoring should go. A guide for the groups themselves with tips and pointers for how best to make use of embedded advisers would also be extremely beneficial.

Conclusions
In this note, we examine existing and no longer existing online SD PMGs by looking into peer mentorship experiences of their members, leaders, and senior faculty advisers, using a blend of survey and interview methods. Overall, respondents mainly mentioned their positive experiences of being PMG members. Participation was particularly beneficial for gaining SD knowledge and for feeling connected to other people who use SD. Importance of participating in PMG was scored high among respondents from both existing and no longer existing groups. Multiple respondents characterized their PMG membership as being crucial for advancing in their careers or for maintaining interest in SD.
However, we noticed some differences between the responses of members from existing and no longer existing groups. Results indicated that in existing groups there might be a larger number of highly motivated members to whom participation is extremely important (Likert score 10). Results also indicated that people in existing groups developed friendships with some of their peers. Those could be the forces that helped existing PMGs to maintain their momentum. Furthermore, members of existing groups presented more diverse topics, focusing also on general SD topics, and not explicitly on their own work. Interestingly, some existing groups had even worked on scientific publications together. Last, no longer existing groups more frequently reported problems related to scheduling and participation. They indicated a need for both increased responsibility from an individual who could act as a leader in planning the meetings, and also for group responsibility, i.e. showing up and participating in the meetings.
Challenges that need to be resolved in the future involved sustaining a diverse competence level within a group to avoid all group members being beginners and to increase learning opportunities, and to assure the existence of strong leaders within each group to overcome the loss of momentum that can lead to PMG dissolution. Similarly, senior faculty members also mentioned strong leadership and personal motivation and commitment as factors important for a group's survival. Although the presence of a senior member did not have a strong effect on group survival, it was generally assessed as very beneficial for individual members and the whole group. Furthermore, senior faculty advisers agreed that improving the PMG program should be a priority for a healthy SD field.
While this note illustrates the positive influence of the PMG program on professional life of "locally lonely" members, there is still much left to learn about this initiative. If we want to increase the chances of PMG survival, there is a need to unravel the complexity of the social-psychological, cognitive, and emotional factors that contribute to the existence or dissolution of PMGs. Further, even if we knew all that we needed to know, we would have to marshal significant resources to successfully scale-up a program in peer mentoring.
Finally, throughout the production of this note, group members, group leaders, and interviewed faculty members have called for the Society to take notice of these online peer mentoring efforts and find ways to support them. We believe that these voices speak for themselves. We urge our Society's leadership to take seriously this chorus calling for change. In the online supplementary material SI-7, we present 14 "Kernels of wisdom" aimed to help those who want to start their own peer mentoring group.

Biographies
Emiliya Suprun is a Postdoctoral Research Fellow and a Sessional Lecturer at the School of Engineering and Built Environment, Griffith University, Queensland, Australia. Her research interests include integrated decision support systems using system dynamics modeling, systems thinking, structural analysis, and operational research methods. Her PhD research was focused on modeling an innovation system in the context of the construction industry.
Andrijana Horvat is a Postdoctoral Researcher at the Food Quality and Design Group (FQD) of Wageningen University and Research, the Netherlands. Her research project is related to developing a Code of Practice for organic food processing. She completed her PhD program related to system dynamics. The results of her PhD research are presented in the thesis entitled: Systems thinking in managing the success of food products -Exploring the potential of a system dynamics approach.
David Andersen is the O'Leary Distinguished Service Professor, Emeritus, of Public Administration, Public Policy, and Information Science at the Rockefeller College, University at Albany. His work centers on applying system dynamics, systems thinking, and information technology approaches to problems in the public, not-for-profit, and private sectors. He has served as a technical consultant to organizations at the federal, state, and local levels. Professor Andersen is co-author of Introduction to Computer Simulation: The System Dynamics Modeling Approach (winner of the Forrester Award in 1983) and Government Information Management as well as a number of journal articles, book chapters, monographs, and edited volumes. He holds a Ph.D. in Management from MIT's Sloan School (1977) with a specialization in System Dynamics.