Online discussion boards in dental education: potential and challenges

Authors


A. Damien Walmsley
School of Dentistry
The University of Birmingham
St. Chad’s Queensway
Birmingham B4 6NN, UK
Tel: +44(0)121 237 2824
Fax: +44(0)121 237 2932
e-mail: a.d.walmsley@bham.ac.uk

Abstract

Background:  Online discussion boards may enhance critical analysis and reflection, and promote the acquisition of knowledge.

Aims:  To assess the effectiveness of online discussion board as a pedagogical tool in augmenting face-to-face teaching in dental education.

Method:  Data were collected from a discussion archive offered through the E-course website of the School of Dentistry, University of Birmingham, UK in 2008. A multi-component metric included; participation, social learning, cognitive processing, role of instructors, and quality of discussion. Messages were coded for 14 variables to evaluate these dimensions. Data were analyzed using content analysis methodology and a complete message was uses as the unit of analysis.

Results:  There were no significant difference in participation between students and instructors (< 0.05). Social interaction with peers appeared only through students posting messages with open questions (27/135 messages). The discussion board was mainly used by students to understand concepts (27/102 messages) and apply procedural knowledge (17/102 messages). Instructors were mainly replying to students’ messages with (49/120 messages) or without (54/120 messages) proposing another action.

Conclusions:  Online discussion boards were found to be successful pedagogical tools in dental education. Further development of instructor-led discussion approach is needed to promote higher level learning and collaborative thinking.

Introduction and aim

General dental practitioners are facing many professional challenges to meet the oral health needs of the public throughout the 21st century (1). In response, the American Dental Education Association’s Commission on Change and Innovation in Dental Education (ADEA CCI) proposed changes that should be made to the dental educational strategies, based on best practices in the literature. The teaching of critical thinking skills is considered to be an important educational principle that helps dental students in developing life-long learning (2).

The incorporation of online elements in education has been reported to bring many added benefits to traditional face-to-face teaching (3). Researchers have recommended the use of online discussion boards for its pedagogical strength. It supports online virtual communities, which engage groups of students allowing them to collaborate and learn from each other in a social learning network. Such innovations remove time constraints and are claimed to enhance in-depth critical analysis and reflection (4). Despite their potential, developing critical thinking skills in these virtual text-based environments remains a major challenge for educators. It requires the construction of an inquiry-based environment that encourages students to challenge assumptions as well as reflect on their own experiences (5).

The potential of online discussion boards to support learning in the health professional fields has been recognised as a successful educational strategy, particularly in supporting collaborative learning in distance education (6). However, reported evaluation of such technology and learning methods in a blended approach in dental education is sparse and requires more research.

Several elements and tools have been proposed in the literature for evaluating the design and components of online discussion boards. Garrison et al., in their Community of Inquiry Framework, identified three prerequisites for the successful performance of such communities. These elements are social, cognitive and teacher presence (7). Kay (8) identified a further 12 dimensions that were considered to be important when designing online communities and developed a comprehensive multi-component metric. These dimensions are social learning, cognitive processing, quality of discussion, initial question, role of educator, navigation issues, challenges for students, types of users, attitudes towards discussion, response time, learning outside of school and learning performance.

Different methodologies were also used to assess and translate the structure and successful functioning of online discussion boards. Content analysis was found to be a potentially rewarding methodology as it can provide important insights into why a session on the discussion forum is successful. However, the process of analysing discussions on such boards can prove to be time consuming (9).

The purpose of this study was to explore the dynamics of using online discussion boards and investigate methods of maximising their success in dental education.

Material and method

Sample

The E-course website of the School of Dentistry, University of Birmingham, UK, was developed using the software Bespoke (Bespoke Microsoft Interdev 6. Microsoft Certified Partner, UK). Examples may be found online at http://www.dentistry.bham.ac.uk/ecourse. Its main objective was to augment and support the traditional teaching in the school. Discussion boards were incorporated as part of the e-course website and were accessible for all members of the school. Each year the discussion boards were archived for future reference. The records for the Prosthetics course for the year 2008 were taken as the sample for this study.

Procedure

Participation in the online discussion board on Prosthetics is voluntary. It was used by both undergraduate dental students and teachers and did not attract any grading. Three teachers moderated the board and all were confident in IT skills and have been operating the educational and dental components of the forum for 3 years. The online discussion board provided group interaction where students could share ideas and experiences, with the view to promote high-level, in-depth interaction amongst students. It also facilitated the communication and feedback processes between the students and their teachers. Students were advised to title their messages with their year of study. Posting names was left to the students’ preferences. Thus, messages from students were grouped according to their year of study. Because of the anonymous nature of posting messages, it was not possible to measure the proportion of the full student cohort who used the discussion board.

Following the assessment metric tool (8) and the Community of Inquiry framework (7), a multi-component metric, comprising of five dimensions, was created for this study. These dimensions were participation, quality of discussion, social learning, cognitive learning and teacher presence.

The overall participation in the online discussion board on Prosthetics in 2008 was assessed using the following six variables:

  •  Total number of threads and messages,
  •  Number of posted threads and messages per term,
  •  Mean length of discussion threads,
  •  Mean number of words per message,
  •  Types of users
  •  Posting time (learning location).

The actual quality of discussion in individual threads was measured as follows: message clarity, content type, author of initial question, external resources used, response time and resolution of discussion threads.

The aim of the social learning dimension was to assess the interaction with peers (student-to-student interaction and reflection). The criteria for this dimension included messages from students in threads, which included four or more messages. Two variables were used to assess social learning in these threads: primary purpose of posted messages and interaction level.

Cognitive learning was assessed as a measure of the level of interaction with the content. The criteria for this dimension included messages from students with course-related information only. Three variables were used to assess this dimension: knowledge type, processing level and the primary purpose of posted messages.

Teacher presence was assessed as a measure of the role of teachers in promoting higher level discussion. The primary purpose of messages posted by teachers was used as a key variable to assess their presence. The latter was compared between two types of threads: (i) threads with four or more messages and (ii) threads with <4 messages.

To make the coding scheme as transparent as possible, a detailed rubric for the key variables used in this study is provided in Table 1.

Table 1.   Detailed rubric for the variables used to analyse discussion board messages
VariablesRatingCriteria
AuthorInstructorThe authority of the person posting the message
Student
Author level (designed for this study)InstructorThe level of the person posting the message
First-year undergraduate dental student (BDS1)
Second-year undergraduate dental student (BDS2)
Third-year undergraduate dental student (BDS3)
Fourth-year undergraduate dental student (BDS4)
Fifth-year undergraduate dental student (BDS5)
Response time (8)In daysDifference between the date a message is posted and the date the following message is posted
Same day responseA message is followed by another message in a thread on the same day
EndThe last message in a thread
No replyA message that is not followed by another message or reply (e.g. one message in a thread)
Posting time (learning location) (designed for this study)Weekend/holidaysIf message was posted in the weekend or holiday time
WeekdaysIf message was posted during the week
Number of words (8)NumberTotal number of words in a message (by word count)
Message clarity (8)UnclearMessage is unclear or confusing – it is typically followed by a message asking for clarification
Somewhat clearMessage is somewhat clear, but there are still confusing or vague points that need clarification
ClearThe message is clear and appears to be understood by the participants in the discussion thread
Primary purpose (8)Open questionOpen question or information directed to all students and instructors (no names are included)
Specific questionSpecific question or information directed to a specific student or teacher
Reply onlyReply to a question, including ‘Yes’ and ‘That’s right’
Reply followed by an actionReply to a question, including ‘Yes’ and ‘That’s right’, followed by another action (question, propose readings, asking for further clarification or checking with the supervisor)
Independent commentIndependent comment, question or answer including ‘Thank you’, ‘Asking for clarification’ or requesting for handouts, lectures, articles….etc.
Non-academicA comment, question or answer to a non-academic condition. This includes administrative issues, clinical arrangements, dates and marking issues, and technical support issues
External resources (8)None/unknownNo clear resources are noted or evident
Teacher/course informationReference is made to a teacher or course information in a message
Another messageReference is made to information in another posted message
WebReference is made to a website
BookReference is made to a book
ArticleReference is made to an article
E-courseReference is made to the e-course or a page in the e-course
Past examsReference is made to past exam papers
More than one resourceMore than one resource is mentioned in a message
CourseworkReference is made to an essay, coursework or homework
Student interaction level (14)Independent thinkingStudents present their own thoughts in the posted message. Including ‘Thank you’
Interactive thinkingStudents reflect on other’s thoughts and answer others questions or propose an action to others (e.g. open questions and seeking advice from anyone)
N/AFor staff messages
Content type (8)Social commentNo knowledge is provided (e.g. social comment –‘thank you’– asking for clarification – requesting articles, handouts or lectures)
Course unrelatedKnowledge is provided that is unrelated to the course (e.g. technical support)
AdministrativeAdministrative knowledge (e.g. due dates, the requirements for final project, or clinical arrangement)
Course relatedKnowledge is provided that supports the course curriculum, including ‘Yes’ and ‘That’s right’
Knowledge type (8)Non-academicFor non-academic/technical support/clinical arrangements/administrative issues/request of handouts, articles and lectures
FactStudent offers an isolated fact
ConceptStudent presents two or more connected facts (e.g. connecting facts with conjunctive adverbs like because, consequently, therefore, otherwise)
ProcedureStudent provides information on how to achieve a specific task
Meta-cognitiveStudents is reflecting about a strategy to solve a problem task or emotional state whilst learning
N/AFor staff messages
Processing level (8)ClarificationStudent is asking what a question or comment means – often referring to a specific element or fact in a problem. Including ‘Thank you’, technical support, clinical arrangements, administrative issues, and requesting articles, handouts and lectures
RememberEvidence that student is recalling or trying to recall a fact, concept or procedure
UnderstandThe student understands or is trying to understand a concept or a procedure
ApplyA student is applying or trying knowledge which typically involves the use of a procedure
AnalyseA student is actively making connections between two or more concepts
EvaluateStudent provides comments about effectiveness of a procedure or approach to solving a problem
N/AFor staff messages
Resolution of discussion thread (8)UnresolvedInformation was not given to solve the question(s) raised in the thread
Partially resolvedInformation is offered that partially answers the question (s) being asked in the thread
ResolvedComplete and correct information is provided to resolve the questions being asked in the thread

Data collection and analysis

The messages posted on the Prosthetic’s discussion archive in 2008 were coded for the various variables using content analysis method. The content analysis technique can be defined as ‘a research methodology that builds on procedures to make valid inferences from text’ (7). A complete message was used as the unit of analysis in this study.

Intra-examiner reliability was then measured for the variables coded using Kappa statistics. These variables are message clarity, content type, external resources used, resolution of discussion threads, primary purpose of posted messages, students’ interaction level, knowledge type and processing level. Data were then analysed using SPSS for descriptive and inferential statistics with significant levels set at P < 0.05.

Results

Intra-examiner reliability test

After repeated measures, the final Kappa statistical value ranged from 0.9 to 1 for the coded variables, thus, indicating high agreement levels.

Participation

Both teachers and undergraduate dental students posted a total of 108 threads consisting of 330 messages with no significant participation difference (P < 0.05). However, when the latter group was further analysed, Kruskal–Wallis test (asymptotic significant value = 0.000) revealed that there was a significant difference (P < 0.05) between the number of posted messages by students from different years of the undergraduate course. The majority of messages (n = 146/176) were posted by fourth-year undergraduate dental students in a 5-year undergraduate programme (Fig. 1).

Figure 1.

 Percentages of messages posted by undergraduate dental students from first to fifth year (in a 5-year programme).

The discussion board on Prosthetics was used both during weekdays (190/330 messages, 58%) and weekends or holidays (140/330 messages, 42%). A majority of messages (255/330 messages, 77%; 79/108 threads, 73%) were posted during the period September to December of the academic year 2008. The mean length of a discussion thread consisted of three messages (SD = 2.3, range: 1—15 messages). The mean number of words per message was 54 words (SD = 56.9, range: 1—464 words).

Quality of discussion

Posted messages were mostly clear (315/330 messages, 96%), with course-related information (261/330 messages, 79%). All threads (100%) were student initiated, and discussion issues were mostly completely resolved (84/108 threads, 78%). A majority of messages (n = 266/330, 81%) had no reference to any external resources (Fig. 2).

Figure 2.

 Frequencies of posted messages using different types of external resources; none/unknown, teacher/course information, another message, web page, book, article, e-course page, past exam paper, coursework and more than one resource.

The mean response time was calculated after eliminating three types of messages: the end message, messages with no reply and messages with outliers in response time (e.g. response time >20 days, n = 2 messages). The mean response time was then found to be 1 day (SD = 2.4, range 0—19 days).

The discussion board in the period September to December of the year 2008 was characterised by having the greatest number of threads and messages compared to all other terms of the year. The Prosthetic discussion archive for that period was, thus, chosen for further analysis to study the effectiveness of discussion boards on the learning process. Three dimensions were then assessed: social learning, cognitive learning and teacher presence. A majority of messages were posted by fourth-year undergraduate students (Fig. 1). Thus, the data were analysed at two general authors’ level only, teachers and students, and the results are presented in the following sections.

Social learning

The number of threads containing four or more messages were 23/79 threads, 29%, and the number of messages posted by students in those threads were 71/135 messages, 53%. Almost half (41/71 messages, 58%) of those messages showed interactive thinking with peers. The majority of this interaction was in the form of open questions (27/41, 66% messages), for example:

Hi, could somebody tell me what impression material would you use for the primary impression for edentulous mouth with undercuts preset? Thanks.

Other types of interaction were in the form of reply to other student (3/41 messages, 7%), reply to other students followed by an action (7/41 messages, 17%), sharing independent comments (2/41 messages, 5%) and discussing non-academic issues with peers (2/41 messages, 5%) as shown in Table 2.

Table 2.   Pivot table showing the frequencies of students’ posted messages using the following variables; students’ interaction level, primary purpose of posted message and threads’ length
Primary purpose of posted messagesStudent’s interaction levelTotal
Independent thinkingInteractive thinking
Thread with <4 messagesThread with ≥4 messagesThread with <4 messagesThread with ≥ messages
Open question00382765
Specific question6100016
Reply091313
Reply followed by action031711
Independent comment688224
Non-academic issue20226
Total14305041135

Cognitive learning

The majority of students’ messages (102/135 messages, 76%) presented course-related information. Pearson Chi-Square test (asymptotic significant value = 0.000) revealed that there was a significant association (P < 0.05) between the knowledge type and the processing level of the content in these messages. Students were mainly trying to understand concepts (27/102 messages, 27%), followed by applying procedures (17/102 messages, 17%), remembering facts (11/102 messages, 11%), evaluating meta-cognitive knowledge (9/102 messages, 9%) and analysing procedural and meta-cognitive knowledge (7/102 messages, 7%) (Table 3).

Table 3.   Pivot table showing the frequencies of students’ messages posted at different knowledge type and processing level
Knowledge typeProcessing levelTotal
RememberUnderstandApplyAnalyseEvaluate
Fact11600017
Concept22714034
Procedure03177128
Meta-cognitive1247923
Total1438221810102

Data were further analysed according to the primary purpose of posted messages. Pearson Chi-Square test revealed that the significant association (P < 0.05) between knowledge type and processing level varied depending on the primary purpose of posted messages. When students were posting open questions (64/102 messages, 63%), they were mainly trying to understand concepts (22/64 messages, 34%), for example:

Are partial and complete dentures made in RCP? Thanks.

To a lesser extent, students were trying to apply (11/64 messages, 17%) and analyse procedures (6/64 messages, 9%), for example:

Hi, Am I right in thinking that the female component of the dolder bar… is flared. Hence it does not fit flush onto the male component of the bar… Thus when axial forces are transmitted on to the arch, … there is some degree of rotation. This then dissipates the forces without dangerous overloading of the abutment teeth?

When students were posting questions directed towards a specific teacher (16/102 messages, 16%), they were trying to understand concepts (5/16 messages, 31%) and to a lesser extent applying procedures (4/16 messages, 25%). In contrary, when students were posting reply messages (13/102 messages, 13%), they were basically trying to remember facts (6/13 messages, 46%). However, when they were posting messages with reply followed by an action (9/102 messages, 9%), such as asking another question or referring to an external resource, they were mainly evaluating meta-cognitive knowledge (3/9 messages, 33%). For example:

In response to (thread #3108) you explained that Buccal upper and lingual lower cusps relate to supporting cusps, maintaining OVD? This confused me because I thought that Upper palatal cusps and Lower buccal cusps are described as the supporting cusps, …and that once initial adjustment to these cusps had occured to correct initial ICP interference that they should be left else loss of OVD occurs and consequently increase in FWS? I would be grateful for any clarification thank you.

Students were also, to a lesser extent, trying to remember facts (2/9 messages, 22%) or applying procedures (2/9 messages, 22%).

Teacher presence

Teachers were mainly replying to students’ messages with (49/120 messages, 41%) or without proposing another action (54/120 messages, 45%). However, when the former group was further analysed, Pearson Chi-Square test (asymptotic significant value = 0.014) revealed that there was a significant association (P < 0.05) between the type of action taken by teachers in their posted messages and the length of discussion threads. The number of messages with reply followed by a question was significantly higher in threads with four or more messages (18/24 messages, 75%). The number of messages with reply followed by a referral to external resources was significantly higher in threads with less than four messages (14/21 messages, 67%). Thus, messages from teachers with reply followed by a question seemed to play a role in promoting discussion (Table 4).

Table 4.   Pivot table showing the frequencies of instructors’ messages posted using the following variables; types of action in reply messages and threads’ length
Primary purposeThreads’ lengthTotal
Threads <4 messagesThreads ≥4 messages
Reply followed by a question61824
Reply followed by a referral to an external resource14721
Reply followed by both a question and referral to an external resource134
Total212849

Discussion

The changing pace of Internet learning technology is creating new interactions for learners (10). This study shows that technology is now able to support online environments, which in turn enhances teaching and learning in dental education. The extensive use of the discussion board by students in the current study not only highlights their popularity but also their significance and pedagogical strength. It is also found to be an environment where students feel comfortable in using this form of interaction. However, our findings show that the learning benefits for the students are not inherent in the technology, but depend upon collaborative activities between themselves and with their teachers (7).

At the end of the period of study, i.e. December 2008, an end of speciality examination took place, which was part paper based and part oral. The presence of this examination explains the high use of the discussion board in the 4-month period before the end of the observation period. It was the fourth-year students who were being examined, which explains the high use of the discussion board by this group. The discussion board is open so junior years are able to view the responses of their senior colleagues. Such a learning activity was not monitored in this study but may be an area for further research into the interaction between year groups.

Varied degrees of social and cognitive presence were found to take place when students used the online discussion board. The pattern of student and teacher interaction showed a substantial alteration in roles with the learner adopting a centric approach (11). However, higher levels of critical analysis and collaborative learning were not always present. It was found that promoting and developing such skills was highly dependent on both the role and presence of the teacher in the online environment (12).

The results highlight two main challenges to the successful incorporation of online discussion boards in dental education. These are curriculum design and teacher development. Contemporary teaching approaches in dental education should focus more on developing critical thinking and reflective learning skills amongst students. This in turn requires that the learning environment should be designed in a way that learning is situated within the context of the curriculum and there should be planned pre-learning activities (13). Online learning strategies should then be merged within the main teaching strategies. This will require support for the teachers to help them redesign the curriculum so that these technologies are used effectively and that they themselves are able to participate and interact online.

The current study assessed an in-depth one discussion archive in one dental speciality. It provided a preliminary insight into the dynamics of such approaches and explored the challenges facing successful incorporation of such technology in teaching and learning in dentistry. These challenges were found to be clustered around the pedagogical skills of teachers in using and successfully integrating such approaches within their conventional teaching. Further studies are needed to assess the use of discussion boards across different specialities before generalising the results. Further work including focus group interviews with both teachers and students is also needed, which will allow the assessment of attitudes towards the use of such technologies.

Conclusion

Online discussion boards may offer a new pedagogical process that will promote teaching and learning in dentistry. The current preliminary results indicate that the educational philosophy underlying the design of an online asynchronous programme is crucial to the way in which it could support and augment teaching and learning. Further support in training teachers to effectively incorporate online elements in their curriculum to achieve their final goal of effective teaching and learning is necessary. The findings of this study are considered an initial step towards providing evidence-based research that highlights specific pedagogies in designing effective online components within the dental curriculum.

Acknowledgements

The authors thank Mr Giles Perryer, E-course developer, Mr Upen Patel and Dr David Attrill who assisted in answering questions on the discussion board.

Ancillary