A comparison of visual identification of dental radiographic and nonradiographic images using eye tracking technology

Abstract Objectives Eye tracking has been used in medical radiology to understand observers' gaze patterns during radiological diagnosis. This study examines the visual identification ability of junior hospital dental officers (JHDOs) and dental surgery assistants (DSAs) in radiographic and nonradiographic images using eye tracking technology and examines if there is a correlation. Material and methods Nine JHDOs and nine DSAs examined six radiographic images and 16 nonradiographic images using eye tracking. The areas of interest (AOIs) of the radiographic images were rated as easy, medium, and hard, and the nonradiographic images were categorized as pattern recognition, face recognition, and image comparison. The participants were required to identify and locate the AOIs. Data analysis of the two domains, entire slide and AOI, was conducted by evaluating the eye tracking metrics (ETM) and the performance outcomes. ETM consisted of six parameters, and performance outcomes consisted of four parameters. Results No significant differences were observed for ETMs for JHDOs and DSAs for both radiographic and nonradiographic images. The JHDOs showed significantly higher percentage in identifying AOIs than DSAs for all the radiographic images (72.7% vs. 36.4%, p = .004) and for the easy categorization of radiographic AOIs (85.7% vs. 42.9%, p = .012). JHDOs with higher correct identification percentage in face recognition had a shorter dwell time in AOIs. Conclusions Although no significant relation was observed between radiographic and nonradiographic images, there were some evidence that visual recognition skills may impact certain attributes of the visual search pattern in radiographic images.

Diagnostic errors in the accuracy and interpretation of medical images have been reported since the 1940s, and abnormalities have been either missed or over-read with various rates across numerous experimental studies (Cooper, Gale, Darker, Toms, & Saada, 2009).
The misinterpretation of chest radiograph from missing subtle early changes has led to delay in diagnosis and initiation of treatment (Lee, Nagy, Weaver, & Newman-Toker, 2013;Turkington, Kennan, & Greenstone, 2002).
In dentistry, among special investigations, radiological diagnosis is one of the key cornerstones for accurate diagnosis and subsequent patient care. The radiographic evaluation of the periapical area by dentists has been reported to be unpredictable and inconsistent regarding the diagnosis of pulpal and bone disease (Sherwood, 2012).
Errors in identification of abnormalities account for a major part of misdiagnosis in radiology and can result from cognitive biases or a faulty visual search (Van der Gijp et al., 2017). Variability in radiologists' performances may occur for several reasons, including differences in decision making and recognition abilities. In turn, these abilities may be influenced by variability in training and experience or preexisting individual differences in perceptual abilities (Sunday, Donnelly, & Gauthier, 2017).
In dental education until now, training on radiological diagnosis has been done by several conventional models, such as masterapprenticeship model, lectures, and case discussions in small groups. However, new technologies allow different methods for understanding the subject and enhancing the teaching and learning methods.
Despite the widespread use of eye tracking in other disciplines, there has been surprisingly little use in dental research and education.
Tracking of visual search parameters such as dwell time, gaze pattern, and gaze duration has been shown to help in understanding the reasons behind false positive and false negative radiological diagnosis (Brunyé, Drew, Weaver, & Elmore, 2019;Krupinski, Chung, Applegate, DeSimone, & Tridandapani, 2016). Despite extensive research on eye tracking of radiographic images from various medical disciplines, limited work has been done till date on eye tracking of dental radiographic images in identification of teeth and jaw bone-related pathologies. Eye tracking could be a valuable tool in dental education for training students to prevent false positive and false negative results by tracking their radiographic search pattern. Tracking visual search patterns of students on radiographic images can potentially generate large data sets, which can help in understanding the reasons for errors in misdiagnosis. This in turn can aid in improving teaching and learning methods on how to prevent such errors.
There is a common observation that students have different ability to observe and diagnose normal and abnormal clinical conditions despite similar training and experience. One possible hypothesis is that some students may have a greater inherent ability of pattern recognition for identification. The extent to which individuals in the normal population vary in perceptual ability is largely unknown, but recent studies have shown large individual differences in perceptual processing of faces, of various familiar object categories, and even of novel objects (Sunday et al., 2017). Research in psychology has shown that people differ in their ability in facial recognition or pattern identification, whereas some inherently have superior ability compared with others (Bobak, Bennetts, Parris, Jansari, & Bate, 2016). Russell, Duchaine, and Nakayama (2009) in his study provided evidence for the existence of people with exceptionally good face recognition ability by identifying a group of individuals who outperformed control participants on tests of face memory, face perception, and familiar face recognition. It is not known if such particular ability observed in face recognition may be also repeated for identifying radiographic abnormalities or anomalies.
The aim of the present study was to determine if the ability to visually identify particular targets or abnormalities in nonradiographic images affords greater skills in the identification of the dental radiographic abnormalities or anomalies using the eye tracking technology.
We also provide some recommendations/considerations for future eye tracking studies (Sunday et al., 2017).
The following hypotheses were tested in the study: 1 There would be no difference in identification and eye tracking parameters between dentists and assistants while assessing dental radiographic and nonradiographic images.
2 The enhanced ability to identify anomalies or targets on nonradiographic images will translate to the ability to identify abnormalities on dental radiographic images.

| MATERIAL AND METHODS
This study protocol was reviewed and approved by the Institutional Review Board of the University of Hong Kong/Hospital Authority Hong Kong West Cluster ). An informed written consent was obtained from each participant in the study. This pilot investigation was undertaken with two readily available cohorts of dental personnel, namely, dentists and dental surgery assistants (DSAs).

| Subjects
Junior residents, with the working title of junior hospital dental officers (JHDOs), and DSAs from the Faculty of Dentistry were invited to be participants in the present study on a voluntary basis. The JHDOs were in their first year of work and the DSAs had a minimum of 15 years of experience. These two groups were selected as convenient samples, and subsequently, nine DSAs and nine JHDOs were recruited.

| Images
Twenty digital panoramic radiographs were selected and anonymized from the patient database of Oral and Maxillofacial Radiology, Faculty of Dentistry, HKU. Inclusion criteria were good image quality as perceived by the expert panel and with one to three abnormalities. Exclusion criteria included presence of distractors that may affect the natural eye tracking pattern of participants. Therefore, radiographs with multiple missing teeth, amalgam fillings, crowns and bridges, and presence of obvious radiographic errors were not included. A panel of five experts consisting of specialists from Oral and Maxillofacial Radiology, Pediatric Dentistry and Prosthodontics selected a total of six panoramic images from the initial sample of 20. Five of these contained a total of 11 anomalies/abnormalities, known as areas of interest (AOIs ; Table 1), whereas one panoramic image was normal. Each AOI was categorized by the panel of experts over three meetings as easy, medium, and hard. All the images were shown using a software with no manipulation of the contrast, brightness, and magnification. Therefore, all the images were preprocessed for contrast and brightness.
To identify potential super pattern recognizers who are good at visual identification, we tested the subjects with 16 nonradiographic images, among which 11 were dealing with pattern recognition, three with face recognition, and the remaining two with image comparison.
Ultimately, this study consisted of six radiographic and 16 nonradiographic images. The radiographic and nonradiographic images were shown in the same random order to each participant.

| Eye tracking procedure
The RED-m (Sensomotoric Instruments, Teltow, Germany) system was used to track the eye movements. The operating distance between the device and an observer's eyes was between 50 and 75 cm. The system has a gaze position accuracy of 0.5 and a spatial resolution of 0.1 . A 9-point initial on-screen calibration was used for each participant that was followed by a 4-point calibration to confirm the preliminary calibration. The process was repeated until an accuracy value of 0.5 was obtained.
During the experiment, each participant was invited to sit inside a quiet seminar room with normal illumination. The aim of the experiment was explained to all participants, and a written consent form was obtained from each participant. The subject would face the wall with a laptop computer screen placed in front of him or her on the table. The operating distance between the device and the observer's eyes was between 50 and 75 cm. Instructions were displayed on the screen, indicating that there was no time limit for the experiment and the images could contain none, single, or multiple AOIs. Whenever a participant identified an AOI, he or she had to gaze at the AOI and left click on the mouse. The system then recorded the answer. Subjects were explicitly explained that their task was to identify the presence and location of the AOIs without the need to recognize the identity of the AOIs.
For data analysis, two main measurement domains were selected-the entire slide and AOI ( Figure 1). The AOI is defined as the area of the abnormality on the panoramic images, whereas the entire slide refers to the whole panoramic radiographic/nonradiographic image and the question template. To evaluate these two domains, two analytics were defined-the eye tracking metrics (ETM) and the performance outcomes ( Table 2). The ETM parameters are described and defined in Table 3

| Statistical analysis
Fleiss' kappa was calculated to evaluate the inter-observer agreement on the difficulty level of the AOIs in the radiographs among the five experts. The final decision of difficulty level was set as the mode (the 3 | RESULTS

| Types of AOI
The six panoramic images used in this study contained a total of 11 AOIs.

| Comparisons on the descriptive data of the parameters in JHDOs and DSAs
The ETMs (as defined in Table 3  that the JHDOs who were better in face recognition identified the first AOI quickly, though it was not statistically significant. Table 9 shows the performance outcomes for the nonradiographic images.

| Association between normal radiograph and the radiographs with abnormalities
For both the groups, the number of false positive responses in normal radiograph was higher than the radiographs with abnormalities, although there was no statistically significant difference between the groups (Table 10). Interestingly, both groups had more number of fixations, a longer path scanned, and more time spent on the normal radiograph than the radiographs with anomalies/abnormalities. T A B L E 6 Descriptive data of the parameters for the area of interest (AOI) in radiographic images-degree of difficulty (easy, medium, and hard) and participants (JHDO/DSA) size to explore this relation. A number of factors were identified during the capture and analysis of data from which we propose recommendations or guidelines so that others may conduct more robust eye tracking studies (Table 11). Suwa, Furukawa, Matsumoto, and Yosue (2001) reported a longer time spent and more number of fixations in the normal images on analyzing the eye movement of dentists during their reading of the CT images of head and neck region. This is supported by Turgeon who examined panoramic images reported with dental students and oral and maxillofacial radiologists who spent longer search times, covered greater distances, and had greater number of eye fixations for normal images than images of pathoses (Turgeon & Lam, 2016). This is similar to our study with participants spending more time in normal images with more number of fixations and a longer path scanned. This finding may reflect that more time is spent looking at normal radiographs to identify an AOI or because of the testing environment; participants spend longer searching for something to identify.
In the current study, the JHDOs were better at identification percentages in the overall and easy radiographs; this may be expected given the training afforded albeit with a shorter period of practice their recent graduation. However, the DSAs with 15 years of experience may have had acquired a particular skill set in pattern recognition in identifying radiographic abnormalities that they have acquired vicariously observing radiographic diagnosis in a teaching clinic. However, it is to be remembered that the diagnosis of the lesions was not part of the current study and therefore, a greater degree of difference would have been expected in the diagnosis differences.
There are many parameters that can be captured and analyzed using the eye tracking technology. These parameters are not universal; many of them are useful only for specific purposes. Depending on the task, one needs to choose proper ETM to reveal features related to the aims of the analysis. The most common eye tracking parameters are based on fixations and/or saccades. A fixation is the amount of time that the participant's gaze remains still, and the number of T A B L E 7 Spearman's correlation showing the relationship between correct identification percentage in the three categories of the nonradiographic images (pattern recognition, face recognition, and image comparison) and the parameters selected (only significant parameters shown) in radiographic images in JHDO and DSA fixations may reflect a more careful scrutiny of the image and attention to a particular area in the image. A saccade is defined as a small rapid change in gaze location from one fixation to another so that a fixation is regarded as being bordered by two saccades. However, in the current study, we did not analyze saccades as this is a kind of proxy for fixation as a saccade is bounded by two fixations. There are also ETM applying both saccades and fixations called scanpath. The scanpaths analysis provides insights into how individuals prioritize locations of semantic interest although the analysis of these paths is difficult. A longer scanpath may correspond to a more detailed image searching and a more methodical search pattern. In addition, time metrics can be added to these to determine the amount of time taken with particular metrics.
The present study also evaluates the time on task with no significant differences between the two groups. The time on task correlates to an attentive and a systematic methodological approach in searching the image. It has been observed that in general, the visual search time decreases with increasing levels of expertise (Giovinco et al., 2015;Rubin et al., 2014;Wood et al., 2013), although in some studies time on task did not significantly differ between different experience levels (Donovan & Litchfield, 2013;Mallett et al., 2014).
In this present study, the Fleiss' kappa value of .665 among the five experts implied substantial inter-observer agreement on the difficulty level of the AOIs in the radiographs (Cohen, 1960). Similarly, a previous study has evaluated the inter-observer agreement of assessing the developmental stage of third molars on panoramic T A B L E 9 Descriptive data for performance outcomes of the 16 nonradiographic images-types of nonradiographic images (pattern recognition, face recognition, and image comparison) and participants (JHDO/DSA) radiographs and reported kappa values of .52-.68, which can be considered as comparable with the kappa value of the present study (Dhanjal, Bhardwaj, & Liversidge, 2006).
It is difficult to source radiographs with only one AOI, and as such, there were between one and three AOIs on the radiographs that may be considered more authentic to the clinical cases. There are limitations in the present study and therefore termed a pilot study. The major limitation is a small cohort of participants in the study. Furthermore, these observers varied greatly in radiographic interpretation expertise. Also, the present study reported where participants' eyes gazed within the radiographs but did not demonstrate their cognitive interpretations.
Observers' interpretations may not necessarily be associated with their gaze fixations (Drew & Williams, 2017). Eye tracking research alone cannot fully explain radiographic interpretation; both perceptual and cognitive processes are necessary and can be considered with doing a talk aloud review after performing the eye tracking.

| CONCLUSION
The present pilot eye tracking study has presented the ability to identify AOIs in radiographic and nonradiographic images. Although no significant relation was observed, there was some evidence that face recognition may impact certain attributes of eye tracking on radiographic images. Further studies are needed to explore this phenomenon.