Does three‐dimensional anatomy improve student understanding?

We aim to provide an overview of the various digital three‐dimensional visualizations used for learning anatomy and to assess whether these improve medical students' understanding of anatomy compared to traditional learning methods. Furthermore, we evaluate the attitudes of the users of three‐dimensional visualizations. We included articles that compared advanced newer three‐dimensional anatomy visualization methods (i.e., virtual reality, augmented reality, and computer‐based three‐dimensional visualizations) to traditional methods that have been used for a long time (i.e., cadaver and textbooks) with regard to users' understanding of anatomy. Of the 1,148 articles identified, 21 articles reported data on the effectiveness of using three‐dimensional visualization methods compared to two‐dimensional methods. Twelve articles found that three‐dimensional visualization is a significantly more effective learning method compared to traditional methods, whereas nine articles did not find that three‐dimensional visualization was a significantly more effective method. In general, based on these articles, medical students prefer to use three‐dimensional visualizations to learn anatomy. In most of the articles, using three‐dimensional visualization was shown to be a more effective method to gain anatomical knowledge compared to traditional methods. Besides that, students are motivated and interested in using these new visualization methods for learning anatomical structures. Clin. Anat. 32:25–33, 2019. © 2019 Wiley Periodicals, Inc.


INTRODUCTION
Medical students often experience difficulties obtaining adequate spatial understanding of threedimensional (3D) anatomy from two-dimensional (2D) images, such as those in anatomy books and on the internet (Battulga et al., 2012;Berney et al., 2015). This may be due to the fact that it is difficult for *Correspondence to: Charlotte P. R. Triepels, Department of Obstetrics and Gynaecology, Maastricht University Medical Center + , P.O. Box 5800, 6202 AZ Maastricht, The Netherlands. E-mail: charlotte.triepels@mumc.nl students in general to mentally rotate static, 2D illustrations (Beermann et al., 2010;Venail et al., 2010). Moreover, due to the complexity of anatomy, medical students and clinicians in training alike experience difficulties in recognizing anatomy in the clinical setting (Smith et al., 2014).
Many different methods are used to learn anatomy, including numerous internet websites dedicated to anatomy images, applications on mobile phones, lectures, oral presentations, 2D pictures (atlases), and cadaveric dissection (Sugand et al., 2010;Estai and Bunt, 2016). Cadaver dissection is often considered to be the gold standard for learning anatomy (Parker, 2002). Currently, more and more 3D visualization methods for teaching anatomy are being developed, at least in part due to the limited availability of cadavers, the high costs associated with obtaining and maintaining them, and the ethical debate surrounding the use of cadavers (McLachlan et al., 2004;Bergman et al., 2011;Ghosh, 2017).
Despite the numerous teaching methods available, many undergraduate and graduate students rate their anatomical knowledge as insufficient (Fitzgerald et al., 2008;Triepels et al., 2018). Many students report being interested in using 3D images to learn anatomy (Bergman et al., 2011;Triepels et al., 2018). A recent study recommended combining 2D and new 3D teaching methods in order to achieve the desired level of anatomical knowledge (Bergman et al., 2011). Yammine and Violato (2015) conducted a meta-analysis of the effectiveness of using any kind of 3D technology to gain factual and spatial knowledge. They concluded that the use of a 3D tool resulted in higher factual anatomy knowledge and spatial anatomy knowledge compared to traditional methods (Yammine and Violato, 2015). Nevertheless, there is only limited information on the different types of 3D learning methods that are being developed and how effective these different visualization methods are when compared to more traditional anatomy learning methods.
The aim of this review is to provide a comprehensive overview of the various digital 3D visualization methods that have been developed for teaching anatomy and their effectiveness compared to more traditional methods. This review provides the medical educator with a better understanding of digital 3D resources that are available to use. Because of the many differences between the 3D visualization methods that are being developed, we did not aim to pool measures of effectiveness into a single inference. For a meta-analysis on the topic, see Yammine and Violato (2015).

MATERIALS AND METHODS
This systematic review is performed in accordance with the guidelines described in the PRISMA statement (Moher et al., 2010).

Information Sources and Search Strategy
A systematic literature search was performed through computerized databases including Medline, Embase, Cochrane, and Education Resource Information Center. This search was restricted to articles published between 2002 and mid-2017. The structured PubMed search can be reproduced using the following keywords: (anatomical knowledge or anatomy knowledge or clinical anatomy or anatomy education or anatomical education or "Anatomy/education"[Mesh]) and (3D or 3-dimensional or three-dimensional or "Virtual Reality" [MeSH] or digital model or augmented reality or "Imaging, Three-dimensional" [MeSH]) and (test result or achievement or knowledge or "Knowledge" [MeSH] or cognitive load or skill or effectiveness or opinion or survey or "Surveys and Questionnaires" [MeSH] or attitude or perspective or view or point of view or stance) and (book or textbook or control or traditional or 2D or twodimensional or 2-dimensional or atlas or cadaver or "Cadaver" [MeSH]).

Selection of Articles
In this study, we selected only original articles that evaluated digital 3D visualization methods for teaching anatomy and compared these to any traditional method based on test results or students' feedback. The 3D visualizations consisted of virtual models (virtual), digital rotatable structures, and augmented reality. The methods were allowed to be based on consecutive 2D images that were placed in three dimensions (such as viewing 3D magnetic resonance imaging [MRI] images based on numerous slices in a single plane). Only articles that had a post-test design, controlled quasiexperimental studies, or randomized clinical trials (RCTs) were included. Traditional methods were defined as any method involving anatomy books, training on cadavers, or 2D anatomical pictures. Two authors (C.P. R.T. and C.F.A.S.) independently assessed the title and abstract of each of the articles for eligibility. If at least one of the authors considered one reference eligible, the full text was obtained for complete assessment by a third author. We restricted the search to include articles published from 2002 onward, because the 3D technologies available before that time were considered to be of insufficient relevance because of advancements in digital technology. The selected articles were retrieved for full text review. We excluded articles if they were published in a language other than English, if they dealt with 3D techniques designed to practice surgical procedures, or if they described nondigital 3D visualizations (such as 3D printing and making use of pipe cleaners, glove, or clay models) and if they made use of gross anatomy or dental-related 3D technology. The references of the included articles were also reviewed for relevant articles that were not found in the search.
The 3D visualization methods were divided into three categories: (1) computer-based 3D methods, (2) 3D augmented reality methods, and (3) 3D virtual reality methods. According to the Merriam-Webster dictionary, virtual reality is defined as a technology used to create or access an artificial environment that is experienced through sensory stimuli (such as sights and sounds) provided by a computer and in which one's actions partially determine what happens in the environment. Augmented reality is a technology used to create an enhanced version of reality created by the use of technology to overlay digital information on an image of something being viewed through a device (such as a smartphone camera). The included articles are clustered in the computer-based 3D methods group when there is no augmented reality nor virtual reality. Computer-based 3D tools were characterized by electronic and especially computerized technology (Merriam-Webster's collegiate dictionary, 1999).

Risk of Bias
We assessed study-level risk of bias using the Cochrane Risk of Bias Tool (Green et al., 2008). The included articles were categorized as having low risk, high risk, or unclear risk of bias based on items of three domains (i.e., selection bias, detection bias, and attrition bias). Selection bias refers to systematic differences between baseline characteristics of the groups that are compared. Detection bias refers to systematic differences between groups in how outcomes are determined. Attrition bias refers to systematic differences between groups in withdrawals from a study. A judgment of "unclear risk of bias" was made in cases where insufficient information was reported to permit judgment of low or high risk.

Study Selection
The electronic search strategy identified 1,148 articles that were assessed for eligibility. Twenty-one articles were selected in which the effectiveness of using 3D visualization methods with using traditional methods was compared. A review of the references of the included articles did not produce any additional articles. The selection process is shown in Figure 1.
The main features of the articles included in this review are show in Table 1. Of the 21 included studies, none were published in the period between 2002 and 2005. Most of the articles were published in the last 10 years. The included studies emerged from different countries, but most often from the United States of America (33%). In regards to the number of participants, most of the articles included fewer than 100 participants except four articles (Venail et al., 2010;Hoyek et al., 2014;Saltarelli et al., 2014;Kockro et al., 2015). An overview of the selected articles including sample size, anatomical structure, study design, type of 3D tool, and study outcomes and differences between groups on the results on the anatomical tests are shown in Table 2. Table 3 shows the study-level risk of bias. An extensive assessment of the risk of bias can be found in Supporting Information Appendix 1. Of all studies, 12 described using randomization to allocate students to either a 3D visualization method or a traditional method. Naturally, blinding of the participants was not possible in any study because participants would always know to what method they were exposed.

Overview of the Different Three-Dimensional Methods Used for Learning Anatomy
Computer-based 3D methods. Fifteen articles investigated the effectiveness of using a computer- based 3D computer model to learn anatomy (Table 1). Four RCTs used a neuroanatomical model for the assessment of a computer-based 3D tool, of which three concluded that the use of a computer-based 3D methods could improve anatomy teaching. Ruisoto et al. (2012) constructed volumetric images with embedded 3D graphic models from functional positron emission tomography scans. The group exposed to the 3D models showed significantly more correctly identified anatomical structures than a group that studied subcortical structures in 2D cross sections (42.1% compared to 25.4%, scale 0%-100%, P < 0.01; Ruisoto et al., 2012). De Faria et al. (2016) assessed an interactive stereoscopic lecture (a computer-based virtual reality method) that is accessible from personal computers. They concluded that the 3D method resulted in greater improvement in students' anatomical knowledge as the 3D group had significantly higher test scores compared with a group that attended a conventional lecture with 2D images (respectively, 6.45 and 4.36; scale 0-10; p < 0.05; de Faria et al., 2016). Agbetoba et al. (2017) evaluated the effectiveness of a tool that permits the learner to draw 3D boxes on relevant anatomical structures and to highlight the frontal sinus outflow pathway. According to their results, most of the students (89.3%) concluded that 3D methods would help them understand spatial orientations (Agbetoba et al., 2017). Additionally, 89.7% of the participants in this study agreed or strongly agreed that they would continue to utilize the 3D software in their clinical practice if the software were available (Agbetoba et al., 2017). Only one study using the   neuroanatomical model did not find significantly higher scores for a 3D digital brain model (Brewer et al., 2012). They found that the mean test score after using that model was not significantly better than using 2D images in atlases (23.5% vs. 22.3% on a 0%-100% scale, P = 0.95; Brewer et al., 2012).
In addition to neuroanatomy, other anatomical structures have been used to compare the effectiveness of traditional and 3D methods, with conflicting results. Computer-based 3D models of the ear were proved very promising in two studies (Nicholson et al., 2006;Ng et al., 2015). Ng et al. (2015) used a model constructed with Google Sketchup, which could be used on an iPad. In this study, the 3D group that used anatomical textbooks and a journal article with detailed illustrations of the epitympanum scored significantly higher, achieving a mean score of 65.1% compared to the 2D group's mean score of 32.4% (scale 0%-100%; P < 0.001; Ng et al., 2015). Nicholson et al. (2006) used a model constructed from a high-resolution MRI scan of the middle and inner ear of a human cadaver. The intervention group's mean score on the quiz was significantly higher than the control group's score that used text and 2D images (respectively, 83% and 65%; scale 0%-100%; P < 0.001; Nicholson et al., 2006). Hoyek et al. (2014) used QuickTime Player (Apple) to show 3D structures of the upper limb and trunk that students could pause and rewind. The traditional group was taught with lectures and was presented 2D anatomical pictures. The 2D group outperformed the 3D group on both general knowledge questions (P < 0.001) and spatial understanding questions (P < 0.001; Hoyek et al., 2014). Donnelly et al. (2009) andHassinger et al. (2010) both focused on anatomical structures in the abdomen. Donnelly et al. (2009) used "Virtual Human Dissector" software and concluded that the differences in mean scores between intervention and control group were not significant. The study by Hassinger et al. (2010) of relatively low quality (Table 3) used an interactive virtual anorectal and pelvis model. This model was created from magnetic resonance and computed tomography images of a male patient. Structures were colored and labeled with clinically relevant descriptions. Most of the participants (90%) agreed that the simulator is a useful tool for learning anatomy (Hassinger et al., 2010).
In an RCT, Codd and Choudhury (2011) of relatively low quality (Table 3) focused on the anatomy of the liver and biliary system. They used an interactive environment created using Macromedia Flash, in which anatomical structures were labeled (Codd and Choudhury, 2011). The 3D group scored higher than the 2D group (taught using dissection and textbooks) with a mean score of 74% and 64% respectively, although the difference was not statistically significant (P = 0.33; Codd and Choudhury, 2011). Saltarelli et al. (2014) explored the effectiveness of a multimedia simulation software that uses high-resolution illustrations to construct a cadaver and provides animations showing the function of blood vessels in the brain. They concluded that human cadaver dissection offered a significant advantage over the multimedia simulation  Venail et al. (2010) of relative low quality (Table 3) and a multicenter quasi-experimental design study aimed to determine if 3D computer software enhanced users' knowledge of the anatomy of the temporal bone. The results of those who took the 3D reconstruction tutorial course (89.92 AE 1.84) were higher than the results of the traditional method group that gets a lecture without a 3D reconstruction (80.91 AE 2.18; scale 0-100; P < 0.001; Venail et al., 2010). Tan et al. (2012) conducted an RCT to investigate the effectiveness of a computer-generated 3D laryngeal model. This model was constructed using computed tomography and MRI, which were segmented into key structures (Tan et al., 2012). The model was subsequently imported into Microsoft PowerPoint software and enhanced with audio, color, video clips, and clinical vignettes. The traditional method group in this study had a mean score of 15.5 AE 2.3 compared to the 3D group, which had a mean score of 15.7 AE 2.0 (P = 0.7222; Tan et al., 2012). Using a quasiexperimental design study, Viswasom and Jobby (2017) investigated a video demonstration which included 3D views. This study had a relative high risk of bias (Table 3). Test results showed that the mean score of the traditional method group was 5.43, whereas for the 3D group, the mean score was 4.59 (scale unclear, P = 0.047; Viswasom and Jobby, 2017). Battulga et al. (2012) employed an RCT that focused on the opinion of medical students regarding the effectiveness of animated and interactive 3D computer graphics (3DCG). They concluded that there was a significant difference in mean scores between the 3DCG and the textbook-only group (4.26 and 3.85, respectively; five-point Likert scale; P = 0.001; Battulga et al., 2012).
Augmented reality. Two articles investigated the effectiveness of augmented reality as a learning tool. One study found that although students preferred traditional methods, using augmented reality produced better test results compared to traditional dissection and lecture learning (Peterson and Mlynarczyk, 2016). Additionally, the other study found that test scores of students who used mobile augmented reality were statistically significantly higher than those who used 2D pictures, graphs, and text (P < 0.05; Kucuk et al., 2016).
Virtual reality. Four articles investigated the effectiveness of using virtual reality to learn anatomy. Three out of four articles that investigated the educational effectiveness of virtual reality found that virtual reality methods were more successful than using books alone for studying (Solyar et al., 2008;Codd and Choudhury, 2011;Kockro et al., 2015), although not all statistically significant (P > 0.05) (Codd and Choudhury, 2011). Only one study that explored the educational effectiveness of using virtual reality found it offered no advantage over static presentations of 2D anatomical depictions (Khot et al., 2013). The mean test score of the traditional method group (that studied six photographed views of a plastic model) was almost equal to that of the virtual reality group (40% and 41%, respectively, scale 0%-100%; Khot et al., 2013).

Students' Opinion About Three-Dimensional Visualization
In 17 of the included articles, a questionnaire was used to measure the user's subjective evaluation of   Viswasom et al. (2017) the 3D tool. Most of the participants in the studies reported that the 3D methods were easier and more enjoyable to use. Sometimes, however, due to the complicated anatomical configuration, participants found the 3D methods disorienting and frustrating (Agbetoba et al., 2017). Three articles focused only on the participant's subjective opinion of the tool and did not test the effect of using the tool on the participants' anatomical knowledge (Hassinger et al., 2010;Battulga et al., 2012;Agbetoba et al., 2017); all three of these articles concluded that a 3D tool is useful for learning anatomy.

DISCUSSION
The relative effectiveness of the use of 3D visualizations was examined in 21 published articles. Twelve articles showed that, according to the users' test results, using a 3D visualization method was significantly more effective than using traditional methods. Although nine articles found no significant difference between the effectiveness of using a 3D visualization method compared to traditional methods, three articles found using the latter to be significantly more effective. Based on three articles which only observed the participants' subjective opinion concluded that 3D methods are useful for learning anatomy. In the computer-based 3D visualization group, we found 15 articles, whereas in the augmented realty and virtual reality group, we only found two and four articles, respectively. Augmented reality and virtual reality are two relatively new types of 3D visualization techniques and thus do not have a large pool of literature. As a result, it is difficult to provide an overall conclusion of the usefulness of these techniques.
Strengths of this systematic review were the thorough and systematic search and the independent selection of articles and data extraction by multiple authors. Although the aims of the included articles were very similar, there were many differences in the methods used and the primary outcomes that were reported. Because of the variability in the outcomes from the included articles, it was decided to assess the risk of bias of each article.
This systematic review has some limitations. As shown in Table 3, several studies included in this review have a high risk of bias. One frequent reason is the lack of randomization. Selection bias may have occurred in these nonrandomized studies, but it is unclear from the manuscripts to what extent this may have happened. Another limitation is that the included articles are almost all based on different parts of the body. The effectiveness of 3D methods for learning anatomy may be affected by difficulty (Fernandez et al., 2011;Nguyen et al., 2012). For one, learning the anatomy of the medical neurosciences is more difficult than learning musculoskeletal anatomy (Allen et al., 2016). The nervous system is one of the most spatially complex systems of the human body (Brewer et al., 2012) and the shoulder is considered to be one of the most difficult joints for medical students (Battulga et al., 2012). Another source of variation between studies is the fact that participants were included who are in a different phase of their education compared to other studies. For example, some studies' participants are first-year medical students (Nicholson et al., 2006;Solyar et al., 2008;Donnelly et al., 2009;Ng et al., 2015) and other studies' participants are fifth-year medical students (Venail et al., 2010). Obviously, the longer medical students study the more basic knowledge of anatomy they have. This makes comparing results between studies complicated. In addition, two of the studies had only a small number of participants, which may have affected their representativeness. For example, Solyar et al. (2008) had only 17 participants and Hassinger et al. (2010) had 10 participants. Other aspects that could have influenced the users scores are the difficulty of the anatomical test and the number of test questions. Some of the included articles did not assess the difficulty of the anatomical structures and/or the difficulty of the questions on the anatomical test.
In conclusion, the included studies demonstrate that computer-based, virtual reality and augmented reality 3D learning methods in general are more effective means of learning anatomy, based on users' test scores, compared to traditional methods. However, the techniques varied greatly, and more research into augmented and virtual reality should be performed as the number of studies on those techniques was low. In most studies, participants state that they prefer to learn anatomical structures using a 3D tool.