SEARCH

SEARCH BY CITATION

Keywords:

  • problem-based learning;
  • group functionality;
  • student engagement

Abstract

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Acknowledgements
  9. Ethical approval
  10. References

Introduction:  In a new enquiry-based learning dental curriculum, problem-based learning (PBL) was chosen as a central methodology because it promotes a collaborative and constructive approach to learning. However, inevitably, some groups function worse than others. This study explores the relationship between group functionality and individuals’ results on knowledge-based assessment. It also sought to establish whether using the Belbin team role theory could improve group functionality.

Methods:  Students completed the Belbin team role inventory that assigns individuals to a team role type and were allocated to either an ideal Belbin group or a control group. To evaluate the functionality of the groups, Macgowan’s group engagement measure was completed after 18 and 31 weeks for each student by their group facilitator. The scores were summed and averaged giving a group engagement score for each group. Relationships between group engagement, individual performance in assessment in weeks 18 and 31 and Belbin and non-Belbin teams were investigated.

Results:  Individual group engagement scores and performance in the knowledge tests had a statistically significant positive relationship despite the small number of students involved (62). However, no correlation was shown between Belbin groups and group engagement scores.

Conclusions:  Those students who engaged most with the PBL process performed markedly better in assessments of knowledge. Using Belbin’s team role theory to place students in PBL groups in an effort to increase group functionality had no effect when compared with non-Belbin control groups.


Introduction

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Acknowledgements
  9. Ethical approval
  10. References

Problem-based learning (PBL) (1–11) is a method and philosophy of learning through which knowledge is both gained and structured using a series of problem scenarios. PBL is a collaborative learning process in which students learn in facilitated small groups as well as by their individual self-directed work. There is much evidence that students working in small groups not only learn more but retain what they learn for longer (11–13), and it has been perceived by students generally as a more enjoyable way to learn than traditional didactic courses (3, 4). Constructing learning in teams would also appear to be very desirable for students whose future working lives will be team based. For these reasons, as a new dental school, we decided to use problem-based learning (PBL) in our enquiry-based learning curriculum, as have other dental schools worldwide (14–17).

Over the 40 years PBL has been practised, different definitions have been put forth and versions of the process have evolved (18). At this school, PBL is central to the curriculum in years 1 and 2. All curricular elements of the programme are set in the context of the PBL case in which they occur. In year 1, students meet up six times during the 2-week case in their groups of eight with their content expert facilitator (dental professional). The process follows the seven steps (19). In the first session, the groups read and understand the case, identify prior knowledge, then brainstorm leading to the construction of the learning objectives, which they go away to research and learn. All students address all the learning issues and share what they have learnt at subsequent sessions.

The facilitators were given 2-days intensive training on PBL, the school curriculum and facilitation skills, in the first instance. In addition, they meet up weekly to discuss their groups and fortnightly with the first author for problem solving. These sessions are invaluable for sharing group difficulties such as student over- or under-participation or loss of focus by the group. In talking together, ideas for solutions to problems are formed and then taken away and tried, before reporting back at the next meeting. It is excellent ongoing staff development. The facilitators sit in on each others’ PBL groups twice a year for peer review.

As has been reported in the literature (2,20–24), there are always some PBL groups that are less functional than others due to poor group dynamic. This is because of the mix and balance of characteristics of group members. Personality clashes between students, a vocal or competitive student dominating discussion, a shy student not contributing or lateness and absenteeism are all possible causes of dysfunctionality. Good facilitation can improve the dynamic but not always cure it. However, in the 1970s, Meredith Belbin demonstrated that group function could be improved by deliberately manipulating group membership structure.

Observing groups completing an executive management exercise at a management college in the United Kingdom, Belbin noticed that some groups performed well whilst others failed to achieve (25). He therefore attempted to elucidate why this should be by giving volunteer group members a battery of psychometric tests. He then devised teams that consisted of people with the same characteristics, for example teams of the cleverest people or teams of stable extroverts, and studied their behaviour and achievement in business games. Teams formed by people displaying these different characteristics performed better than teams made up of people displaying the same characteristics as each other. Members bring different skills and roles to the group which enable it to achieve its goals. The roles with their current names are shown in Table 1. Belbin team role theory has been used worldwide and has held up to 25 years of scrutiny, although independent research is limited (26).

Table 1.   Belbin team roles
Completer-finisher (CF)Ensures team is protected from mistakes of commission and omission; pays attention to detail; has a sense of urgency
Co-ordinator (CO)Controls way in which team moves towards group objectives by making best use of team resources; recognises team’s strengths, weaknesses and member’s potential
Implementer (IMP)Turns concepts and plans into practical working procedures; is systematic and efficient
Monitor-evaluator (ME)Analyses problems; evaluates ideas and suggestions enabling team to take balanced decisions
Plant (PL)Advances new ideas and strategies with special attention to major issues; looks for breaks in approach to problems
Resource-investigator (RI)Explores and reports on ideas, developments and resources outside team; makes external contacts; conducts any negotiations
Shaper (SH)Shapes way team effort is applied; directs setting of objectives; seeks to impose shape on team discussion and outcomes
Teamworker (TW)Supports members in strengths and weaknesses; improves communications; fosters team spirit

This study investigated whether it would be possible to select group members in a way which enhanced the overall functionality of the PBL group by optimising the group dynamic using Belbin teams. If this proved to be the case, we could then select members for all of our groups and eliminate dysfunctionality to a large extent. Furthermore, we sought to explore the relationship between students’ engagement with the PBL group learning process using a group engagement measure and individuals’ results on knowledge-based assessments.

Methods

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Acknowledgements
  9. Ethical approval
  10. References

Prior to commencing the programme, students took the online Belbin self-perception inventory (2007 version) that gives a profile of the eight roles for each individual. The highest scoring role is assigned for each individual. The cohort of 63 first-year students had an average age on entry of 27 years ranging from 22 to 41 years. They were divided into seven groups of eight students and one group of seven. The four Belbin groups comprised of students with each of the different Belbin team roles, whilst students were randomly assigned to the other four control groups irrespective of their Belbin team roles. The control groups were checked to ensure there was not an accidental Belbin group. Where students had dual profiles (the self-perception inventory scored them highest on two roles), the role which best fitted the making of the four ideal Belbin group was used.

In year 1, each facilitator works with two of the eight groups. Neither the group facilitators nor the authors knew which were Belbin groups, and neither did they know they each had a Belbin and a control group. Students did not know whether they were assigned to a Belbin group.

The functionality of the groups in terms of engagement of individuals in the group was measured using an evaluated tool, Macgowan’s Group Engagement Measure (GEM). It was designed to rate the engagement of individuals in small, closed groups (three to 12 members; no new members after the first session) and has been shown to have a high reliability and modest to high validity dependent on the construct or criterion used (27). It comprises a series of 37 questions each with a Likert scale of 1–5 on which a facilitator makes judgements about a student’s behaviour within the group. The questions are grouped into five sections which describe attendance and punctuality, relating to others in the group, contracting with others, working on own problems and working on others’ problems. After both 18 and 31 weeks of the programme, the PBL facilitators completed the 37 questions on the GEM for individual members of each group but did not summate the scores. The scores for each student were summed by the first author. The individual student scores for each group were then summed and averaged to give a mean group engagement score for comparison of one group with another, and these data were not disclosed to the facilitators. Five facilitators worked with the same groups throughout the 31 weeks. Three facilitators had two groups each, and the other two facilitators job-shared the facilitation of the remaining two groups. The job-sharers both scored every student in their groups and the two facilitators’ GEM scores were averaged for each student.

At 18 and 31 weeks, the facilitators also completed a written questionnaire comprised of seven questions assessing group function: how well the group got on together, how much it monitored its own performance and how much collaboration, competition and affection was in the group. The questionnaire required Likert scale and free-text answers.

In week 18, the students completed a formative applied dental knowledge progress test (PT1) and in week 31 a similar formative test (PT2) and a summative assessment of knowledge of dental (DS) and life sciences (LS). All of these tests comprised of best fit of five choices multiple choice questions. The PT questions were general questions relating to all aspects of dentistry. The two summative tests were specifically about the science integrated into the PBL cases in year 1.

Independent samples t-tests and Pearson’s correlations were carried out to investigate the relationship between the group mean GEM scores and Belbin or non-Belbin groups, and the individual GEM scores and performance on assessments, using SPSS v15.0 for Windows.

Seven students did not complete the self-perception inventory. These students were placed in the non-Belbin groups. One student left the programme during the 31-week duration of the experiment leaving a cohort of 62 for analysis.

Results

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Acknowledgements
  9. Ethical approval
  10. References

Findings relating to Belbin groups, performance and functionality

There was no correlation between Belbin groups and mean GEM scores. T-tests showed that there was no relationship between Belbin grouped individuals and performance on any of the assessments. Figure 1 shows box plots for performance on PT1 for Belbin and non-Belbin groups and is representative of all the performance data. There were no statistically significant relationships between Belbin grouped individuals and any of the other variables [t(60) = 0.92(PT1), 0.08 (PT2), 1.74 (LS), 1.83 (DS), P > 0.05, two-tailed]. Where data were recorded on more than one occasion, statistically insignificant relationships between data sets did not develop into significant trends.

image

Figure 1.  Box plot showing range and distribution of Progress Test (February 2008) scores in the Belbin and non-Belbin group students.

Download figure to PowerPoint

There was a direct match between the mean GEM scores for each group and the written questionnaire judgements of the facilitators. Groups which facilitators felt were most functional had the highest mean GEM scores. Two of the facilitators reported that the Belbin group (unknown to the facilitator) was the more functional of their two groups and two facilitators reported that the controls were more functional although for three out of four there was very little difference between their groups. No groups were considered dysfunctional in that learning was not stifled.

Findings relating to individual student’s engagement and performance

There was a direct statistically significant relationship between individual students GEM scores and results in the first formative PT1 in February and in the summative LS assessments in June and for an average of all test results (Table 2). Figure 2 shows the scattergram showing the relationship between GEM scores and LS assessment. A crude comparison of the performance of the top and bottom GEM scorers in each assessment at various cuts, top and bottom 10 students, top and bottom third of students and top and bottom halves of students (Table 3) shows just how strong this finding is.

Table 2.   Pearson’s correlations of group engagement measure score mean with percentage scores on four assessments
 Gem score mean
  1. N = 62

  2. *Correlation is significant at the 0.05 level (2-tailed).

  3. **Correlation is significant at the 0.01 level (2-tailed).

Progress test 10.340**
Progress test 20.214
Life sciences0.314*
Dental sciences0.096
Mean overall for 4 assessments0.296*
image

Figure 2.  Scattergram showing positive correlation between average group engagement measure scores for all 62 students and results on the Life Sciences assessment.

Download figure to PowerPoint

Table 3.   Difference in raw scores and percentage on assessments between the top and bottom group engagement measure (GEM) scorers at various cuts a) top and bottom 10, b) top and bottom thirds and c) top and bottom halves
GEM scorerAssessments
Progress test 1Progress test 2Life scienceDental science
a) Top1037.8533.9363.7064.85
 Bottom1027.6825.5350.2362.80
 Difference (%)36.7532.9126.833.26
b) Top 1/335.2532.2760.6163.86
 Bottom 1/329.0625.8848.6958.86
 Difference (%)21.3124.7024.478.50
c) Top 1/233.7231.0156.9162.06
 Bottom 1/230.3629.2550.4158.40
 Difference (%)11.066.0112.896.27

Discussion

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Acknowledgements
  9. Ethical approval
  10. References

Using the Belbin model made no difference to the PBL groups’ performance as assessed by both the GEM mean scores and the facilitators’ qualitative written assessments. An explanation for this finding may be that in business management, the team is working together towards a common defined goal of selling the most products or making the most money or providing the best service. To do this, they must collaborate as a team and they all share in the one achievement together. It is more apparent to members of a business team that competition between members could be a hindrance to best achievement of their goals. Although the facilitators in this study could not always identify the reasons for lesser functionality of a group, competition between members was a large factor in some cases. An anecdotal example of a less functional group is one with a vocal, competitive student and two or three average contributors, one of whom is very able and knowledgeable. The rest of the group are below average contributors with one or two being very shy. In this kind of mix, competition between the competitive and able student can dominate and intimidate the quieter members into contributing little and feeling unhappy. In the UK education system, high-achieving students naturally compete against each other and it is difficult for some individuals to make the change to a more collaborative style. Students have their own learning goals rather than a group goal, and in this dental programme, students are taking assessments as individuals and each has their own individual result. If the assessments were designed to take into account students’ performance in group work, then this effect might be mitigated.

It is noteworthy that there should be such a strong, statistically significant correlation (r = 0.296, n = 62, P < 0.05) between how much individual students engage with the process of PBL and how well they fair in knowledge-based assessments. Reasons for success in assessment are complex and multiple; however, this evidence supports the thesis that collaborative working results in higher knowledge achievement. We believe that this is an important finding, especially so in the light of the following factors. Experience of higher education knowledge-based assessment does not lead us to believe that it is the norm that high achievers are the most outward going or the most altruistic in sharing their knowledge. Indeed, there is evidence and opinion that high achievers in secondary and tertiary education prefer to work in isolation (28–30). Also, the diversity of this cohort has not skewed this result: the students behind these statistics are individuals from varied backgrounds with an age range of 19 years. They range from shy, diligent black and minority ethnic females, who have lower GEM scores but work very hard, to extremely community-spirited health care professionals returning to education with high GEM scores who find assessments difficult.

Limitation of the study

There are certainly limitations to this study. We used Macgowan’s GEM to assess the functionality of the PBL groups. The GEM was designed to measure the level of engagement of students in a learning group. Summing and averaging of individual GEM scores has not been used before as a proxy measure for group function but we decided that it was the best quantitative measure we could find to give another dimension to the qualitative assessments the facilitators made about group functionality. The fact that the facilitators did not know the summative score of the 37-item GEM for each student, let alone the group scores, which correlated with their written opinions we feel justifies the use of the GEM in this way. Another limitation is that the judgements by each facilitator for the GEM and their qualitative assessments were subjective and facilitators only assessed their own groups. Therefore, there was no standardisation between facilitators as to how they assessed students. However, the mean GEM scores for each group were in a small range with only 19% variation between the highest and lowest. There is face validity in the facilitators’ written assessments of the group functionality because of the continual discussion between facilitators and the peer review process. The factor that compensates most for this limitation is that, unknowingly, each facilitator (or job-sharing facilitators) had a Belbin and a non-Belbin group, whose performance they obviously compared. In our fortnightly discussions throughout the 31 weeks, facilitators reported differences in group dynamic between their two groups which affected functionality. All of the data give a consistent picture.

Conclusion

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Acknowledgements
  9. Ethical approval
  10. References

Using Belbin’s team role theory to place students in PBL groups in an effort to increase group functionality had no effect when compared with non-Belbin control groups in this study. This is possibly because the context of a business management team and a PBL team is different. Unlike a functional business team, students retain individual learning goals, rather than a group learning goal. Although using the Belbin instrument for improving functionality in PBL groups was not successful, we would argue that finding other instruments for this purpose is a worthy exercise.

All of the groups in this study showed collaborative behaviour but some form of group assessment may improve collaboration and also diminish competition amongst students within groups. Although it is recognised that students have their own learning goals, most students engaged well with the PBL process. Those who engaged most performed markedly better in assessments of knowledge, which adds further evidence for the hypothesis that collaborative learning results in deeper understanding and broader learning.

Acknowledgements

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Acknowledgements
  9. Ethical approval
  10. References

The facilitators Carrie Bradburn, Gerry Butler, Cathy Coelho, Paula Fernley and Pam Ward are thanked for their participation in this study and their enduring dedication to the process without which this work would not have been possible. Jo Duffin is also thanked for the organisational input into designing the Belbin and non-Belbin groups. There are no conflicts of interest.

Ethical approval

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Acknowledgements
  9. Ethical approval
  10. References

The Chair of Peninsula College of Medicine and Dentistry advised that ethical approval was not required for this data collection. No plausible harm would result from publication. However, all students gave written consent for their participation in the project.

References

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Acknowledgements
  9. Ethical approval
  10. References
  • 1
    Schmidt HG. Problem-based learning; rationale and description. Med Educ 1983: 17: 1116.
  • 2
    Norman GR, Schmidt HG. The psychological basis of PBL. A review of the evidence. Acad Med 1992: 67: 557565.
  • 3
    Albanese MA, Mitchell S. Problem-based learning: a review of literature on its outcomes and implementation issues. Acad Med 1993: 68: 5281.
  • 4
    Vernon DTA, Blake RI. Does problem based learning work? A meta analysis of evaluative research. Acad Med 1993: 68: 550563.
  • 5
    Savery JR, Duffy TM. Problem-based learning: an instructional model and its constructivist framework. Educ Technol 1995: 135: 3138.
  • 6
    Schmidt HG, van der Molen HT. Self-reported competency ratings of graduates of a problem-based medical curriculum. Acad Med 2001: 76: 466468.
  • 7
    Abrandt Dahlgren M, Dahlgren LO. Portraits of PBL: students’ experiences of the characteristics of problem-based learning in physiotherapy, computer engineering and psychology. Instr Sci 2002: 30: 111127.
  • 8
    Prosser M. A student learning perspective on teaching and learning with implications for problem-based learning. Eur J Dent Educ 2004: 8: 5158.
  • 9
    Hmelo-Silver CE. Problem-based learning: what and how do students learn? Educ Psychol Rev 2004: 16: 235266.
  • 10
    Dolmans DHJM, de Grave WS, Wolfhagen IHAP, van der Vleuten CPM. Problem-based learning: future challenges for educational practice and research. Med Educ 2005: 39: 732741.
  • 11
    Dolmans DHJM, Schmidt HG. What do we know about cognitive and motivational effects of small group tutorials in problem-based learning? Adv Health Sci Educ 2006: 11: 321336.
  • 12
    Gross Davis B. Tools for teaching. Jossey-Bass Publishers: San Francisco, 1993.
  • 13
    Slavin R. Cooperative learning. In: Gall JP, Gall MD, Borg WR eds Applying educational research: a practical guide. edn. New York: Longman, 1999: 4.
  • 14
    Fincham AG, Baehner R, Chai Y, et al. Problem-based learning at the University of Southern California School of Dentistry. J Dent Educ 1997: 61: 417425.
  • 15
    Rohlin M, Petersson K, Svensäter G. The Malmö model: a problem-based learning curriculum in undergraduate dental education. Eur J Dent Educ 1998: 2: 103114.
  • 16
    Shuler CF, Fincham AG. Comparative achievement on National Dental Board Examination Part 1 between dental students in problem-based learning and traditional educational tracks. J Dent Educ 1998: 62: 666670.
  • 17
    Haghparast N, Sedghizadeh PP, Shuler CF, Ferati D, Christersson C. Evaluation of student and faculty perceptions of the PBL at two dental schools from a student perspective: a cross-sectional survey. Eur J Dent Educ 2007: 11: 1422.
  • 18
    Maudsley G. Do we all mean the same thing by ‘Problem-based learning’? A review of the concepts and a formulation of the ground rules Acad Med 1999: 74: 178185.
  • 19
    Wood DF. ABC of learning and teaching in medicine. Br Med J 2003: 326: 328330.
  • 20
    De Grave WS, Dolmans DHJM, van der Vleuten CPM. Student perceptions about the occurrence of critical incidents in the tutorial group. Med Teach 2001: 23: 4954.
  • 21
    De Grave WS, Dolmans DHJM, van der Vleuten CPM. Student perceptions on critical incidents in the tutorial group. Adv Health Sci Educ 2002: 7: 201209.
  • 22
    Hendry GD, Ryan G, Harris J. Group problems in problem-based learning. Med Teach 2003: 25: 609615.
  • 23
    Hitchcock MA, Anderson AS. Dealing with dysfunctional tutorial groups. Teach Learn Med 1997: 9: 1924.
  • 24
    Houlden RL, Collier CP, Frid PJ, John SL, Pross H. Problems identified by tutors in a hybrid problem-based learning curriculum. Acad Med 2001: 76: 81.
  • 25
    Belbin RM. Management teams – why they succeed or fail, 2nd edn. Oxford: Elsevier, 2004.
  • 26
    Pritchard JS, Stanton NA. Testing Belbin’s team role theory of effective groups. J Manage Dev 1999: 18: 652665.
  • 27
    Macgowan MJ. Evaluation of a measure of engagement for group work. Res soc work practice 2000: 10: 348361.
  • 28
    Stockdale S, Williams RL. Cooperative learning groups at the college level: differential effects on high, average and low exam performers. J Behav Educ 2004: 13: 3750.
  • 29
    Murray-Harvey R. Metacognition makes a difference: identifying characteristics of successful tertiary students using path analysis. Flinders University of South Australia. http://www.aare.edu.au/93pap/murrr93159.txt [accessed 20 November, 2010].
  • 30
    Uzuntiryaki E. Learning styles and high school students’ chemistry achievement. Sci Educ Int 2007: 18: 2537.