Development and evaluation of e‐learning materials for dental hygiene students in six schools: Using smartphones to learn dental treatment procedures

Abstract Objective The purpose of this study was to develop and evaluate the effectiveness of using e‐learning on smartphones to provide dental hygiene education on dental treatment procedures. Methods This was a randomized controlled trial. Two‐hundred ninety‐three dental hygiene students in four universities, one junior college and one technical school took a preliminary examination, and based on the results, participants in each school were randomly divided into two groups, a test or control group. Both groups took pre‐ and post‐examinations at a 4‐week interval. The test groups learned dental treatment procedures and four‐handed techniques with interactive learning materials using smartphones. The learning materials allowed them to watch videos of dental treatments. The control groups were not provided any learning material. Results Results of all schools combined showed that the changes from pre‐ to post‐examination scores in the test groups were significantly higher than those of the control groups (p < .05). Post‐examination scores were significantly higher than pre‐examination scores in the test groups in all schools (p < .05). Also, post‐examination scores of the test groups were significantly higher than those of the control groups (p < .05). The changes from pre‐ to post‐examination scores in the test groups of two schools were significantly higher than those of control groups (p < .05). Post‐examination scores of the control groups in two schools were significantly higher than pre‐examination scores (p < .05). Conclusions Learning dental treatment procedures through e‐learning on a smartphone was effective in developing participants’ understanding of dental treatment procedures and four‐handed techniques.


| INTRODUC TI ON
In Japan, there are 123,831 employed dental hygienists and 90.6% of them work at private dental clinics. Japanese dental hygienists have three main areas of focus: prevention, patient education, and dental hygiene practice and dental assistance. 1,2 Therefore, all dental hygiene students have to learn treatments for prevention, techniques for patient education, and dental treatment procedures to complete a professional dental hygienist course. However, it is difficult for many students to understand the dental treatment procedures performed by dentists before observing actual dental treatments in clinical practice. Previous attempts administered questionnaires to supervisors of clinical practice reported that supervisors answered that most of dental hygiene students should learn dental assisting more before starting clinical practice. 3 Even though dental hygienists do not perform the same treatments as dentists, they must learn four-handed techniques to assist dentists and ensure smooth treatment. Nevertheless, classes on four-handed techniques on dental treatments are limited since dental hygiene students are required to learn other professional subjects, including dental hygiene practice and oral care for elderly people. [4][5][6] Learning materials that enable students to imagine dental treatment procedures and to develop their understanding without increasing the number of classes they must take are required.

Information and communication technology and electronic learn-
ing systems (e-learning) have been gradually introduced in dental education, and positive effects have been identified. 7 Most e-learning materials can only be accessed on a personal computer (PC). [8][9][10] However, some studies have shown that young people prefer to use their smartphones to access the Internet, 11 which means that they use smartphones as information devices more than PCs. In fact, 99.4% of Japanese adults in their 20s possess a smartphone, 12 and this rate is also increasing in other many countries. 13,14 One study found that dental students follow the same trend of smartphone use. 15 Moreover, previous research showed that learning materials that can be accessed on mobile devices were effective in increasing students' motivation for learning. 16,17 Therefore, development of e-learning materials that can be accessed on smartphones is needed.
Previous research showed that watching videos, which is considered one form of information and communication technology, is positively evaluated by dental hygiene students. 3,18 However, there have been few studies on the effectiveness of watching videos of dental treatments on dental hygiene students' learning and understanding of four-handed dental treatments. Therefore, the purposes of this study were to develop e-learning materials for smartphones that allow dental hygiene students to watch videos of dental treatments and to evaluate their effectiveness.

| S TUDY P OPUL ATI ON AND ME THODOLOGY
The study protocol was approved by the Ethical Committee of Tokyo Medical and Dental University (Approval No. D2018-082).

| Determination of sample size
This study was conducted in four undergraduate universities, one junior college and one technical school, all of which consented to participate in this study. All participants took a preliminary examination and the examination score was set as the primary outcome because it was an objective measure. The sample size of each school was determined by power analysis. Mean intra-group differences in examination scores of 5≧ with a standard deviation of 5 would require 17 participants per group to detect an effect with alpha = 0.05 and power = 0.8 (Statistical Discovery, SAS, Cary, NC, USA).

| Participants
This study focused on dental hygiene students who had not yet begun clinical practice. Third-year university students, second-year junior college and technical school students were targeted for participation since university students start clinical practice in their third year, and junior college and technical school students start in their second year. Eighteen dental hygiene schools including universities, junior colleges and technical schools, in which educators teach whom authors could get in touch, were asked to participate in the study. And then, 6 schools including 4 universities, 1 junior college and 1 technical school accepted. Educators of these schools agreed that it is difficult for their students to completely understand dental treatment procedures and dental assisting techniques before starting clinical practice. All of 6 schools did not utilize information and communication technology when they taught dental treatment procedures. The numbers of participants in each school, labelled A to F, were as follows: A = 38, B = 20, C = 68, D = 28, E = 61, and F = 78. The total number of participants in this study was 293 (mean age ± standard deviation: 20.3 ± 2.0 years old). The trial was registered in the UMIN clinical trials registry (ID: UMIN000035544). Informed consent was obtained from all participants.

| Learning materials
Learning materials to acquire knowledge of dental treatment procedures that could be accessed on any type of information device were created by five educators who teach at a dental hygiene school or university. The materials were composed of five subtopics: resin restoration, inlay restoration, crown restoration, root canal treatment and fillings, and tooth extraction, which dental hygiene students should learn first before starting clinical practice. 1. Tapping the play icon, the participants could watch a short video of dental treatment.

2.
A question was displayed at the end of the each short video.
3. Participants chose the correct answer(s) by checking the circle(s).

| Preliminary examination
The results of this examination were used for grouping of participants. The maximum examination score was 40. There were 20 multiple choice questions (2 points each). These questions asked about the procedures of each dental treatment, which were extracted from the past Dental Hygienist's National Examinations.

| Pre-and post-examinations
The maximum examination score was 50. There were 15 multi-  Figure 2).

| Randomization
The participants at each school were systematically and randomly divided into two groups (a test group and a control group) according to the results of the preliminary examination and their experience as a part-time dental assistant. Systematic randomization was conducted by listing the participants in descending order of preliminary examination scores in Microsoft Excel ® sheet. The each pair of participants from the top of the list were systematically and randomly allocated into either the test or control group. Randomization of each participant was generated by computer at the same time until the average preliminary examination scores and the number of students who with part-time job experience as dental assistant roughly equal. There was no significant difference in preliminary examination scores between the two groups. In addition, the number of students who with parttime job experience as a dental assistant was roughly equal between the group (see Table 1a and b).

| Experimental design
Both groups took pre-and post-examinations with a 4-week interval. The test group learned four-handed dental treatment procedures using a smartphone. The control group was not given any learning materials. After the post-examination, we provided the control group participants access to the learning materials to allow the same learning opportunity as was provided the test group participants. All participants had already finished the lectures related to the dental treatment procedures presented in the learning materials; therefore, the learning materials were used for review.
All participants completed a questionnaire related to the learning materials after finishing the post-examination. The questionnaire was composed of nine questions, including 7 Likert scale questions (see Figure 3) and the following 2 free response description questions: "Please describe the positive points of these learning materials", and "Please describe how the learning materials should be improved."

| Statistical analysis
A non-parametric test was used since pre-examination scores showed a non-normal distribution (Shapiro-Wilk test). The Mann-Whitney U test was used for inter-group analysis of changes between pre-and post-examination scores. The Wilcoxon signed rank test was used to evaluate changes between pre-and post-examination scores between groups. Statistical analyses were conducted using JMP version 11 for Windows (Statistical Discovery).

| RE SULTS
The total average learning activity time (hours) and the completion rate of test groups in each school were as follows: A = 1.7 ± 0.8, 89%; B = 2.0 ± 0.9, 93%; C = 2.8 ± 1.3, 96%; D = 2.3 ± 1.3, 95%; E = 1.4 ± 0.8, 75%; and F = 1.7 ± 0.9, 85%. Watching less than 95% of the duration of each video was treated as material that was not learned. As shown in Table 2, the discrimination index of question 2 was 0.04, and therefore, it was considered an inappropriate question and was removed. Finally, the maximum score for both the pre-and post-examinations was 49.

| Examination scores
The post-examination scores of the test groups were significantly higher than the pre-examination scores in all schools (A: p = .0315; B: p = .0234; C: p = .0001; D: p = .0029; E: p = .0456; F: p = .0006).
Post-examination scores were significantly higher than pre-examination scores in both groups (test group: p < .0001; control group: p < .0001). Combined data of all school showed that the changes from pre-to post-examination scores of the test groups were significantly higher than those of the control groups (p = .002). The changes from pre-to post-examination scores in the test groups   Tables 1 and 3).
There were no significant differences in the changes from pre-to post-examination scores by experience working as a part-time dental assistant.

| Questionnaire
More than 80% of the participants answered positively to the questions, "These learning materials were helpful for reviewing," and "I was interested in these learning materials." About 50% of the participants answered positively to "These learning materials were easy to use," and "These learning material content was easy." Seventy-three participants indicated that the "Videos were easy to understand," and 37 participants indicated that "Audio explanations were easy to understand" as positive points. Sixty participants indicated that the "Videos of these learning materials often froze" as a point to be improved (see Figure 4).

| D ISCUSS I ON
Japanese dental hygiene students are required to understand dental treatment procedures and four-handed techniques sufficiently to assist dentists and to ensure smooth dental treatment, even though they do not perform those dental treatments themselves. In this randomized controlled study, e-learning materials that could be accessed using smartphones were developed and their effectiveness was evaluated objectively. The results of this study showed that learning dental treatment procedures using the developed learning materials was effective in significantly enhancing participants' understanding of dental treatment procedures. We observed three main findings from our data.
First, e-learning materials that allowed students to watch videos of dental treatments were effective in presenting images of real dental Controlability of these learning materials was good.
These learning materials gave me more confidence to assist dentists.
I would like to study more these kinds of learning materials.
Studying these learning materials should be continued.
These learning materials were helpful for re-learning.
These learning materials were easy.
I was interested in these learning materials.
Strongly agree Agree Somewhat agee Somewhat disagree Disagree Strongly disagree treatment procedures. Watching videos enabled participants to understand how an operator performs the treatment and uses instruments even though they have never watched dental treatments in person.
Actually, some previous studies concluded that in addition to demonstrations, procedural videos can improve learners' professional skills. [19][20][21] Some dental hygiene students could have difficulty imagining the details of dental treatments from verbal explanations or written materials with photographs; however, the procedure could be easier to follow by watching videos, which also allows them to see the operator's actual motions while performing four-handed dental treatment procedures.
Also, students prefer watching videos for learning to traditional learning method. 22,23 Furthermore, the introduction of videos in dental hygiene education can lead to more uniform education and help lessen educators' workload since the time needed to perform demonstrations could be reduced. Demonstrations differ depending on the operator, which means that the contents and quality of lectures are not uniform.
If educators could reduce the time spent preparing for and conducting demonstrations, they could gain creative time to improve their teaching and coaching skills and enhance the quality of their lectures. In addition, e-learning materials that allow learners to watch videos will be useful tools to train not only dental hygiene students, but also dental hygienists who return to work after an absence. Japan currently has a shortage of working dental hygienists and Japan Dental Hygienists' Association work to support dental hygienists' reinstatement after absences. 24 Increasing the number of working dental hygienists would lead to better promotion of oral health among the general population.
More e-learning materials that utilize videos should be developed in dental hygiene education.
Second, more e-learning materials that can be accessed on smartphones should be developed and introduced in dental hygiene education. The results of the present questionnaire showed that most participants were interested in the learning materials since they could learn by using their own smartphone, which is the most user-friendly device for them. In fact, young people like watching videos on YouTube and prefer using smartphones to PCs. 25  participants. In a future study, both the test and control groups should be allocated materials on the same topics using different learning methods to more clearly identify the effectiveness of the learning materials.
For instance, the test group could learn about dental treatment procedures using e-learning materials, while the control group could learn using traditional handouts. Identifying the factors that highly impact students' understanding and motivation to learn will lead to the development of high-quality learning materials and heighten the quality of dental hygiene education. Also, it was another limitation that the gain of 8% in examination score of test group was not outstanding improvement. If the learning duration was prolonged or the contents of learning materials were improved, higher changes would be found. However, it is difficult to understand dental treatment procedures and acquire dental assisting techniques completely before starting clinical practice. Then, gain of 8% in examination score after learning using smartphones by themselves was not too low. Furthermore, not only acquirement of knowledge but also skills should be evaluated in the future study to evaluate effectiveness of learning materials developed in this study.

| CON CLUS ION
In this randomized controlled study, e-learning materials that could be accessed on smartphones were developed, and these materials allowed participants to learn dental treatment procedures over a 4-week study period. The results of this study showed that learning dental treatment procedures with e-learning materials was effective in significantly enhancing participants' understanding of the materials.

| CLINI C AL RELE VAN CE
Scientific rationale for study: Novel learning materials and methods in dental hygiene education should be developed since the demand for dental hygienists have recently increased.
Principal findings: Learning materials that could be accessed on smartphones and allowed participants to watch videos were effective in improving participants' understanding of dental treatment procedures.