Validation of a machine learning technique for segmentation and pose estimation in single plane fluoroscopy

Kinematics of total knee replacements (TKR) play an important role in assessing the success of a procedure and would be a valuable addition to clinical practice; however, measuring TKR kinematics is time consuming and labour intensive. Recently, an automatic single‐plane fluoroscopic method utilizing machine learning has been developed to facilitate a quick and simple process for measuring TKR kinematics. This study aimed to validate the new automatic single‐plane technique using biplanar radiostereometric analysis (RSA) as the gold standard. Twenty‐four knees were imaged at various angles of flexion in a dedicated RSA lab and 113 image pairs were obtained. Only the lateral RSA images were used for the automatic single‐plane technique to simulate single‐plane fluoroscopy. Two networks helped automate the kinematics measurement process, one segmented implant components and the other generated an initial pose estimate for the optimization algorithm. Kinematics obtained via the automatic single plane and manual biplane techniques were compared using root‐mean‐square error and Bland–Altman plots. Two observers measured the kinematics using the automated technique and results were compared with assess reproducibility. Root‐mean‐square errors were 0.8 mm for anterior–posterior translation, 0.5 mm for superior–inferior translation, 2.6 mm for medial–lateral translation, 1.0° for flexion–extension, 1.2° for abduction–adduction, and 1.7° for internal–external rotation. Reproducibility, reported as root‐mean‐square errors between operator measurements, was submillimeter for in‐plane translations and below 2° for all rotations. Clinical Significance: The advantages of the automated single plane technique should aid in the kinematic measurement process and help researchers and clinicians perform TKR kinematic analyses.


| INTRODUCTION
Kinematics are an important consideration when assessing total knee replacements (TKR), as the procedure often results in knee joint kinematics that deviate from the normal knee. [1][2][3][4][5] Reduced posterior femoral rollback, paradoxical anterior femoral translation, femoral condylar separation, and other deviations from normal knee kinematics that occur post-TKR can lead to multiple issues including increased polyethylene wear, reduced quadriceps efficiency, impingement, and component migration. 1,[6][7][8][9] Therefore, it would be highly desirable to obtain kinematic information for a patient in the clinic. While there are current methods for acquiring kinematics that are relatively simple to implement, such as motion capture using surface sensors, they lack the ability to accurately assess tibiofemoral mechanics. 10,11 TKR kinematics have been accurately measured in vivo using single-plane fluoroscopy for nearly three decades. 1,6,10,12 The singleplane approach is appealing because the equipment required to obtain appropriate images can be found in most hospitals. Early methods implemented shape libraries or precomputed distance maps to match a three-dimensional (3D) implant model's projection to the implant's projection on a radiographic image. [12][13][14] Modern singleplane fluoroscopy methods obtain implant model poses with the use of optimization techniques that iteratively fit the model projections to the projections on the radiograph. 15,16 These optimization techniques help to automate the process and can greatly reduce the amount of human error and time required to produce kinematic measurements. However, the process still requires a human operator to find the initial poses of the implant models in the first frame using an edge detection algorithm. As such, acquiring TKR kinematics is still a time consuming and labor-intensive task, especially for untrained operators.
Machine learning algorithms, specifically neural networks, have successfully performed multiple computer vision tasks including segmentation, pose estimation, and classification that might help to automate measurements of TKR kinematics. [17][18][19][20][21] Capitalizing on the advantages of these networks, Jensen et al. 22 have proposed a machine learning technique for further automating the TKR kinematic measurement process. Two networks were developed for use in a common, publicly available joint kinematic measurement software (JointTrack Auto). The first network segments each radiograph to identify the tibial or femoral component, and the second finds an initial pose that can be used as a starting point for the optimization algorithm. They reported that the networks were successful at segmenting the implant components and finding initial pose estimates that were sufficient starting points for the optimization algorithm. This new machine learning technique reduces the time and labor required to obtain TKR kinematics, which should ease the measurement process for researchers conducting kinematic studies and has the potential to allow surgeons to order kinematic exams in the clinic. However, it is important that the machine learning technique is accurate enough to provide clinically useful measurements. Therefore, it was the goal of this study to assess the accuracy of the machine learning technique in calculating the relative pose between the tibial and femoral components, using a biplanar radiostereometric analysis (RSA) technique as a gold standard. RSA is a highly accurate technique that can be used for measuring kinematics without the drop-off in out-of-plane accuracy seen in single-plane fluoroscopy, resulting in measurements reasonably close to ground truth. 12,16,23,24 As a secondary goal, the reproducibility of the machine learning technique was assessed by comparing kinematics measured with two different operators.
A tertiary goal was to investigate whether the machine learning technique sped up the measurement process, which was determined by timing how long it took to measure kinematics using the machine learning technique versus a standard version of the same software that does not implement the neural networks.

| METHODS
Twenty-three patients (24 knees) from a previous RSA study were included in the present study. 9 All patients received identical

| RESULTS
For the 113 images, the segmentation network was able to correctly identify 113/113 femoral components and 112/113 tibial components. Table 1  No correlation was found between the automated single-plane method and the manual biplane method for medial-lateral translation (r 2 = 0.029, p = 0.074) or abduction-adduction (r 2 = 0.009, p = 0.309).
The Bland-Altman plots are displayed in Figure 1, with the bias and 95% LoA also reported in Table 1 Reproducibility of the automated single plane method, defined as RMS errors between kinematics measured by the two operators, was 0.7 mm for anterior-posterior translation, 0.4 mm for superior-inferior T A B L E 1 Column one contains the differences between the automated single plane method and the gold standard manual biplane method, reported as root-mean-square (RMS) errors, for the six knee motions. Columns two and three report the bias and 95% limits of agreement for each of the six knee motions that were plotted on the Bland-Altman graphs in Figure 1.