Surgical accuracy of CAD/CAM splints using virtual surgical planning in orthognathic surgery: policy implications for healthcare in Australia

This study examines post‐surgical outcomes of maxillary position using virtual surgical planning (VSP) with computer designed and manufactured surgical splints, without the use of costly patient specific implants (PSI), in the treatment of routine nonsyndromic orthognathic patients. The cost of these personalized medical devices and their impact in the setting of cranio‐maxillofacial surgery is currently under review by The Department of Health and Aged Care in Australia.


Introduction
][4] Early adoption of 3D-VSP technologies in orthognathic surgery has been primarily in teaching hospitals where issues relating to cost, access, and case selection, are readily addressed.[7] The benefits of 3D-VSP in orthognathic surgery are being increasingly reported.][10][11][12][13] The outcome of orthognathic surgery is generally accepted to be clinically accurate if landmark locations are within 2 mm and 2 of their intended position. 14lobally, the cost of PSI with customized cutting guides and plates is still prohibitive for most routine orthognathic cases outside teaching institutions.In 2022, The Department of Health and Aged Care in Australia initiated a review of the use of all craniomaxillofacial surgical guides (including cutting guides) and biomodels, in response to the growing use of these devices and the additional burden of cost to the local health care ecosystem.Their data showed that the total benefit amounts paid by private health insurers to hospitals for surgical guides and biomodels have increased nearly tenfold, from $1.9 million in 2016-2017 to $17.2 million in 2020-2021, and this was mostly related to craniomaxillofacial procedures. 15This continuous and exponential increase in the cost of these devices at its current rate is not sustainable.As such, the Department is currently undertaking a Stage 1 review, analysing the evidence and the role for surgical guides and biomodels in clinical practice.A subsequent Stage 2 review is intended to assess the cost-effectiveness of surgical guides and biomodels. 15he use of CAD/CAM surgical splints without PSI and cutting guides offers an economically viable solution in appropriate orthognathic cases.This approach affords the surgeon intra-operative flexibility with respect to the final vertical maxillary position and angulation.This is favoured by some surgeons, acknowledging it may introduce intraoperative errors in vertical maxillary position and maxillary plane angulation.The importance of accurately positioning these elements of the maxilla during orthognathic surgery has been well documented. 16n this retrospective study we aim to assess the post-surgical outcomes of maxillary position using 3D-VSP and CAD/CAM surgical splints without cutting guides and PSI, in the treatment of routine non syndromic orthognathic surgery patients by comparing their pre-operative planned position with their post-operative imaging.

Materials and methods
This is a single-centre retrospective analysis of 49 patients who underwent bimaxillary orthognathic surgery with a standard Le Fort I approach followed by bilateral sagittal split osteotomies by a single surgeon at Epworth Richmond Hospital (Victoria, Australia) over a period spanning 2016 to 2020.Patients were included in the study provided their surgery was planned with the use of 3D-VSP technique outlined below.
Patients were excluded from the study if they did not meet the criteria for 'routine' orthognathic surgery: • A history of prior orthognathic surgery; • Patients who underwent Le Fort II and Le Fort III procedures; • Patients who underwent segmental surgery; • The presence of cleft, craniofacial, or other congenital syndromes; • Acquired (e.g., post-traumatic) deformities; • Skeletal immaturity; • Patients who underwent surgery utilizing 3D-VSP PSI and cutting guides.The 3D-VSP workflow employed for all patients in this study was as follows: (1) All patients in the study underwent an in person standardized orthognathic planning appointment in preparation for their 3D-VSP, once their presurgical orthodontic phase was completed.followed by mandibular surgery.Based on this CASS, intermediate and final splints were fabricated using CAD/CAM.The splints were utilized during the procedure in the same sequence.All patients acquired a post-operative lateral cephalogram and OPG between 1 and 2 weeks following surgery.Five linear and angular measurements outlined in Table 1 were utilized to compare the planned pre-operative position of the maxilla based on the 3D-VSP, against the post-operative position of the maxilla on the post-operative lateral cephalogram.To avoid bias, the treating surgeon was excluded from all data collection.
To account for interobserver variability, all 2D radiographic datapoints for the study were collected twice, independently by two examiners, and the variability of this data was assessed.The data points and measurements obtained from the 3D imaging was done so through a single experienced biomedical engineer to maintain consistency.The variability of the data points and measurements obtained from the 2D images were compared against measurements obtained from the 3D images as an independent exercise to ensure accuracy.If any data points were found to be significantly different between 2D and 3D sources, or between observers, these were excluded from the subsequent analysis of actual surgical outcomes.
Thus, the details of the three independent pairs of data sets used in the methodology of this study to validate the outcome are outlined below.
(1) Accuracy of the two independent observers.This was assessed through an initial inter-observer variability study by comparing the pre-operative lateral cephalometric measurements (outlined in Table 1), obtained by each observer.An anatomical landmark-based approach was used to identify the datapoints along with standard digital tracing and measurement software (AudaxCeph, Audax 2020), calibrated against a universal radio-opaque ruler against the patient's face.

Statistical analysis
Statistical analysis was performed using Stata V17 (StataCorp.2021.Stata Statistical Software: Release 17. College Station, TX: StataCorp LLC).As multiple variables were being compared, the alpha level was adjusted to take into consideration the multiple tests, and statistical significance was set at P < 0.01 (0.05/7).

Interobserver reliability
Shapiro-Wilk tests for normality indicated all preoperative measurements by both observers were found to be normally distributed and as such interobserver reliability was compared using paired t-tests, where a null hypothesis of similarity between means was adopted (H 0 : μ = μ 0 ).There was a high degree of correlation between the two independent observers of the preoperative lateral cephalograms; no statistically significant differences were noted across the measurements.

Validation of comparing 2D and 3D datasets
To confirm the validity of directly comparing landmark-based measurements obtained from 2D lateral cephalograms against voxel-based measurements obtained from 3D CBCT, the mean preoperative 2D measurements of both observers were compared against the computer-measured cephalometric values of the preoperative 3D CBCT data (H 0 : μ = μ 0 ).All pre-measurement data were normally distributed, and thus compared using paired t-tests.Furthermore, equivalence testing using the 2 one-sided test was used to determine whether the measurements from the 2D and 3D data were within 2 or 2 mm of each other (H 0 :jμj ≥ 2 or 2 mm).In equivalence testing a P < 0.05 indicates the deviation is within 2 or 2 mm, whilst a P > 0.05 indicates the difference between the two measurements is greater than 2 or 2 mm.
There was no statistically significant difference between the two sources for measurements SN-PP (P = 0.37), UI (P = 0.40), and AUFH (P = 0.37).A statistically significant difference was noted for SNA (P < 0.05) and PUFH (P < 0.05), and for this reason these measurements were not used in determining the accuracy of VSP in this study (Section 3.3) as they were demonstrated to be unreliable when this technique was applied (Table 2).
When evaluating whether the 2D and 3D CBCT measurements were within 2 or 2 mm of each other, SN-PP, UI, and AUFH were all within the deviation (P < 0.05).

Accuracy of VSP planning in determining outcome
Finally, the mean planned surgical movements and the actual surgical changes of the maxilla were compared (H 0 : μ = μ 0 ).The paired t-test was used to identify the positional differences for normally distributed data, SN-PP and UI.The differences for AUFH were not normally distributed, and as such the Wilcoxon signed rank test was used for these comparisons.
No statistically significant differences were found in the variables SN-PP, UI and AUFH (P > 0.05) (Table 3).Equivalence testing was again used to determine whether the measurements from the planned and actual movements were within 2 or 2 mm of each other (H 0 :jμj ≥ 2 or 2 mm).A P < 0.05 in this instance would indicate that the deviation is within 2 or 2 mm.When evaluating whether the planned positions were within 2 or 2 mm of the actual outcome, SN-PP, UI, AUFH were all within the deviation (P < 0.05).

Discussion
Surgical success in orthognathic surgery is dependent upon accurate assessment and diagnosis of the patient deformity by the treating surgeon, creation of a treatment plan that considers both aesthetic and functional outcomes across soft and hard tissue domains, and the precise transfer of such a treatment plan from the planning stage to the patient intraoperatively. 17Virtual surgical planning represents a paradigm shift in the planning of orthognathic surgery.The use of 3D imaging combined with CAD-CAM technology has effectively obviated the need for analogue workflows and their associated limitations, including increased preparatory time, laboratory costs, and potential cumulative errors with the transfer of data from the laboratory to the patient. 18,191][22] In Australia, the increasing utilization of surgical guides and biomodels has resulted in exponential cost increases, prompting a Prostheses List Post Listing Review, with a focus on comparative clinical effectiveness and cost-effectiveness of these products.
This retrospective study has demonstrated predictable outcomes for maxillary position in routine orthognathic cases with a focus on maxillary incisor angulation and the position of the maxilla in relation to the palatal plane using CAD/CAM 3D-VSP surgical splints.These outcome measures were identified as being particularly sensitive to change intra-operatively when CAD/CAM 3D-VSP surgical splints are used without PSI and customized cutting guides and plates.Our findings may assist in clarifying the optimal role of surgical guides, biomodels and PSIs by the Department of Health and Aged Care.
In this patient population, maxillary surgery was performed first, followed by mandibular surgery.Whilst the sequencing of surgery remains an area of controversy in some circles, there is now sufficient evidence to support that with proper planning and execution either sequence can provide an accurate and acceptable outcome. 23urvey has stated that the order of the surgery should be a surgeon's decision, 24 whilst in 2014 Ritto et al. demonstrated that the maxilla could be accurately positioned irrespective of the order of surgery. 25A systematic review of the literature by Borba et al. further confirms this approach. 26he limitations of this study are not dissimilar to other studies in this area, including the retrospective nature of the study and the small sample size. 9,13,18,20,27In addition, a further limitation of this study is that the outcomes were measured comparing the 3D planned positions pre-operatively against the 2D post-operative radiographs.In our region, it is not commonly accepted practice for patients to undergo routine post-operative CBCTs along with the additional costs and radiation associated with CBCTs.In future we anticipate that post-operative CBCTs for orthognathic patients may become routine practice as the additional costs and radiation are addressed with advanced equipment and technology.We aimed to address this limitation in two ways.In the first instance we conducted an inter-observer reliability test for the selected linear and angular measurements between two independent observers of the  Surgical accuracy of CAD/CAM splints cohorts' pre-operative lateral cephalograms.We found no statistically significant difference between the two observers-a result similar to prior studies that have used a similar methodology. 28Following this, we validated the comparative accuracy of the linear and angular measurements taken from the patients' pre-operative 2D lateral cephalogram against the patients' pre-operative 3D CBCT data.Two datapoints were deemed statistically significant between these two datasets (SNA and PUFH), indicating that these data sets could not be reliably used in this study to determine the accuracy of VSP outcomes using our study design.The authors propose this to be due to measurement error in operator identification of the anterior and posterior nasal spines, and the nature of the narrow margin of error in data analysis.Therefore, these datapoints were removed from the final surgical outcome analysis.
A further limitation of this study relates to the use of the term 'routine' or 'simple' orthognathic surgery.It is imperative to note that this study contains a substantial list of exclusion criteria for those patients who were considered to have undergone complex orthognathic surgery.The exclusion criteria had to be listed in this manner as there is currently no other reliable method for distinguishing the complexity of a case based on existing coding systems such as the Medicare Benefits Schedule (MBS) numbers.For instance, the MBS items do not indicate if patients are undergoing surgery for correction of acquired or congenital birth defects, nor do they indicate if the patient is undergoing revision surgery.Furthermore, this study looked at the final position of single jaw surgery (maxilla) with respect to palatal plane and maxillary incisor angulation.

Conclusion
This retrospective study demonstrated reliable positioning of the maxilla using the maxillary incisor angulation and maxillary plane as an outcome measure in patients who underwent VSP with CAD/CAM 3D printed surgical splints without the use of Patient Specific Implants.The former is a more cost-effective way to utilize the existing technology to deliver this type of personalized medicine which will no doubt continue to shape the future of medicine and surgery in general.
Ongoing research with large randomized controlled trials is necessary in this rapidly growing field.This will continue to shape our understanding of the benefits of the technology and assist with more patient centred resource allocation.In the interim, any resource allocation policy should be agile enough in its approach to ensure future proofing of the benefits.

( 2 ) 3 )
Validating comparison of 2D vs. 3D data sources.The comparison of numerical measurements from different data sources (3D CBCT vs. 2D lateral cephalogram) was validated through the comparison of 'like-for-like' datapoints between the pre-operative lateral cephalograms and the preoperative CT data.The mean values of the two sets of preoperative measurements from the same two independent observers were then compared against the values of the preoperative 3D reconstructed CBCT data.(Analysis of surgical outcomes.The mean values for each measurement obtained by the two independent observers for the 2D post-operative measurements were compared against the surgical planned movements of the 3D-VSP.This additional measure was added to further enhance the accuracy of the comparison between the 2D post-operative lateral cephalogram and the pre-planned 3D-VSP plan, when comparing the outcomes of the surgery.The project was approved and authorized by the Epworth Research Development and Governance Unit (Epworth ID: EH2021-685).This retrospective analysis involved the use of nonidentifiable data, and thus deemed negligible risk as per paragraph 2.1.7 of the NHMRC National Statement of Ethical Conduct in Human Research (2007).

Table 2
Validation of comparing landmark-and voxel-based methods of linear and angular measurements

Table 3
Accuracy of VSP planning in determining final maxillary position No significant differences were noted for the variables SN-PP, UI and AUFH.© 2023 The Authors.ANZ Journal of Surgery published by John Wiley & Sons Australia, Ltd on behalf of Royal Australasian College of Surgeons.