Establishment and validation of systematic prognostic nomograms in patients over 60 years of age with osteosarcoma: A multicenter external verification study

Abstract Background The aim of this study was to develop and validate systematic nomograms to predict cancer specific survival (CSS) and overall survival (OS) in osteosarcoma patients aged over 60 years. Methods We used data from the Surveillance, Epidemiology, and End Results (SEER) database and identified 982 patients with osteosarcoma over 60 years of age diagnosed between 2004 and 2015. Overall, 306 patients met the requirements for the training group. Next, we enrolled 56 patients who met the study requirements from multiple medical centers as the external validation group to validate and analyze our model. We collected all available variables and finally selected eight that were statistically associated with CSS and OS through Cox regression analysis. Integrating the identified variables, we constructed 3‐ and 5‐year OS and CSS nomograms, respectively, which were further evaluated by calculating the C‐index. A calibration curve was used to evaluate the accuracy of the model. Receiver operating characteristic (ROC) curves measured the predictive capacity of the nomograms. The Kaplan–Meier analysis was used for all patient‐based variables to explore the influence of various factors on patient survival. Finally, a decision curve analysis (DCA) curve was used to analyze whether our model would be suitable for application in clinical practice. Results Cox regression analysis of clinical variables identified age, sex, marital status, tumor grade, tumor laterality, tumor size, M‐stage, and surgical treatment as prognostic factors. Nomograms showed good predictive capacity for OS and CSS. We calculated that the C‐index of the OS nomogram of the training population was 0.827 (95% CI 0.778–0.876), while that of the CSS nomogram was 0.722 (95% CI 0.665–0.779). The C‐index of the OS nomogram evaluated on the external validation population was 0.716 (95% CI 0.575–0.857), while that of the CSS nomogram was 0.642 (95% CI 0.50–0.788). Furthermore, the calibration curve of our prediction models indicated the nomograms could accurately predict patient outcome. Conclusions The constructed nomogram is a useful tool for accurately predicting OS and CSS at 3 and 5 years for patients over 60 years of age with osteosarcoma and can assist clinicians in making appropriate decisions in practice.


| INTRODUCTION
Osteosarcoma is a malignant tumor with a poor prognosis. Its histological composition involves the mesenchymal tissue or undifferentiated connective tissue, and it is the second most common tumor after plasma cell myeloma among primary bone cancers occurring predominantly in younger and older individuals. The prognostic significance of age in patients with osteosarcoma remains to be clarified and it is generally accepted that osteosarcoma occurs with a bimodal age dispersion, with the primary incidence occurring between 10 and 20 years of age and the second highest incidence occurring after age 60 years. 1,2 Therefore, osteosarcoma patients are of great significance at age 60 years, which is not only represents the age group with the highest incidence of morbidity but also the age group with higher mortality. Given the aging population, we must consider the prognosis of osteosarcoma patients in this age group. Osteosarcoma is usually localized in the lengthy bones of the extremities close to the epiphyseal increase plate, and it especially in the distal femur, proximal tibia, and possibly in the epiphyseal plate of the proximal humerus epiphysis, 3,4 and its prognosis is very poor. Recently, the development and utilization of highly efficient chemotherapeutic agents and extended local tumor control, the incorporation of these novel therapy modalities into treatment regimens has prolonged the overall survival (OS) of patients with osteosarcoma from <20% before the 1970s to approximately 70% currently. 5 In addition, treatment and pre/post analysis of osteosarcoma patients in recent years has revealed that the best strategy for the treatment of osteosarcoma is to introduce adjuvant chemotherapy, and then perform surgical resection of malignant areas. Adjuvant chemotherapy improves the survival rate to more than 70%, 6 while amputation provides the local control with a survival rate of only 20%-30%. 7 In contrast, more than half of cases treated with surgical excision of osteosarcoma alone without chemotherapy die within 12 months of diagnosis, and most pulmonary metastases occur at a median time of 10 months, providing a relatively rapid endpoint for surgery. [8][9][10][11] In addition, pulmonary and lymphatic metastases in patients with osteosarcoma have been the focus of attention in recent years. 12,13 Low-grade osteosarcoma can be treated by simple surgical resection. 7 The prognosis and survival of osteosarcoma are affected by many factors, including patient age, primary location of tumor and laterality, tumor size, and T, N, and M stage, surgery, radiotherapy, and other clinically factors. The chemotherapy regimen alone cannot be used as a treatment for osteosarcoma. Limb salvage surgery (LSS), with the addition of novel adjuvant chemotherapy is the first choice for many surgeons when treating osteosarcoma. 14 However, if LSS cannot achieve a good therapeutic effect or is not suitable for LSS, amputation is recommended. Amputation is a more dramatic and direct method, which can immediately eliminate all parts of the tumor. 15 The most recent techniques for the treatment of osteogenic sarcoma range from 70% to 80%, with encouraging 5-year OS survival rates. 6 However, previous research has identified several unbiased prognostic factors associated with survival, although no single feature can accurately predict survival of osteosarcoma patients. Consequently, there is an urgent need to establish an personalized multivariate model to accurately predict the survival and prognosis of osteosarcoma patients over 60 years of age. The Surveillance, Epidemiology, and End Results (SEER) program is a massive population-based database for research on cancer-related epidemiology and healthrelated therapy. It gathers statistics from 18 geographic populations based in cancer registries that cover almost 30% of the US population. The nomogram is an easy to use and accurate prediction tool and has been widely applied in medical studies, mainly to evaluate cancer survival and prognosis. Nomograms contain many associated factors. It can be used to calculate the survival prognosis of individual patients by rating of the associated elements in the nomogram. Thus, the nomogram has become a reliable tool for clinical decision making and for predicting the medical outcomes of many cancers. Even though few studies have analyzed trends in the incidence and prognosis of elderly patients with osteosarcoma, no study has ever examined the factors influencing survival in patients with osteosarcoma over 60 years of age, and no study has produced a nomogram of survival in patients with osteosarcoma over 60 years of age. We have not only analyzed the factors influencing overall survival and cancer-specific survival in patients with osteosarcoma over 60 years of age. We also created a nomogram to more visually demonstrate the risks affecting survival in patients with osteosarcoma over the age of 60.

Conclusions:
The constructed nomogram is a useful tool for accurately predicting OS and CSS at 3 and 5 years for patients over 60 years of age with osteosarcoma and can assist clinicians in making appropriate decisions in practice.

K E Y W O R D S
60 years, cancer, cancer-specific survival, nomogram, osteosarcoma, overall survival, SEER The aim of our study was to incorporate many risk factors in the construction of two nomogram prediction models able to predict 3-year and 5-year survival rates of older aged patients with osteosarcoma, particularly those over 60 years of age. To achieve this goal, we evaluated the consequences of osteosarcoma patients extracted from the SEER database aged over 60 years of age from 2004 to 2015 in the form of nomograms.

| Study design and patient selection
Based on the SEER database, we extracted data of patients 60 years or older diagnosed with osteosarcoma from 2004 to 2015 using the SEER*STAT software (version 8.3.9.2; National Cancer Institute).
A total of 306 osteosarcoma patients were included in this study. The specific flowchart including and excluding all patients is shown in Figure 1.
In addition, information on the multicenter, externally validated patient cohort was obtained from the Shanxi Cancer Hospital, the Second Hospital of Shanxi Medical University, and the Third Hospital of Shanxi Medical University.
The histological type was determined according to the International Classification of Diseases in Oncology codes. The tumor stage was established according to the seventh TNM classification of the American Joint Committee on Cancer.
This study involving human participants was reviewed and approved by the Medical Ethics Committees of the Second Hospital of Shanxi Medical University, the Third Hospital of Shanxi Medical University, and the Shanxi Cancer Hospital, respectively. In accordance with national legislation and institutional requirements, written informed consent from the legal guardians or next of kin of the participants was not required for participation in this study.

| Data collection
The clinical and pathological information of all patients was obtained from the SEER database and included age, race, sex, marital status, tumor grade, tumor location, tumor laterality, tumor size, and stage N or M tumor, and whether or not the patient received surgical treatment, chemotherapy, or radiation therapy. Patients had two endpoint events: cancer-specific survival (CSS) and OS.

| Nomograms construction and verification
We designated patients diagnosed between 2004 and 2015 as the model population and patients diagnosed with osteosarcoma collected by our multiple medical centers as the external validation population. Nomograms were constructed to predict OS and CSS at 3 and 5 years incorporating many variables. The discriminative power of the nomograms was measured by receiver operating characteristic (ROC) curves and area under curve (AUC) values of survival for 3-and 5-year-OS and CSS. After establishing nomograms, we also constructed calibration plots and calculated the C-index scores, to evaluate the predictive power of the nomograms.

| Statistical analysis
A total of 982 patients were initially screened at the beginning of the study. After applying the inclusion and exclusion criteria, 306 patients met the study requirements.
F I G U R E 1 Flow chart of the study process.
T A B L E 1 Baseline characteristics of the included patients.

Characteristic
Training cohort (n = 306)/n (%) Validation cohort (n = 56)/n (%) p value Information on externally validated patient data for our multicenter study was initially collected from three medical centers, the Shanxi Cancer Hospital, the Second Hospital of Shanxi Medical University and the Third Hospital of Shanxi Medical University, on a total of 77 patients aged 60 years or older with osteosarcoma. After careful screening, we removed 11 patients for whom grade was not available, seven patients for whom tumor size was unknown, and three patients for whom life and death were unknown, leaving a total of 56 patients who met the requirements. We constructed nomograms for the prognosis of OS and CSS in osteosarcoma patients over 60 years based on clinical or pathological factors. Unlike previous studies that only used a few variables, we collected many variables, and finally screened out eight variables that were more meaningful than other variables through Cox regression analysis in Table 2 and Table 3. After constructing nomograms, we evaluated the predictive ability of the nomograms using the ROC curve, the AUC area, and the C-index. An AUC > 0.9 indicated that the model has high prediction accuracy, while an AUC between 0.7 and 0.9 indicated the model had good accuracy. An AUC between 0.5 and 0.7 indicated that the model has medium and low accuracy, while an AUC < 0.5, indicated that the accuracy of the established model was very low, and the result was poor.
The C-index was used to evaluate the accuracy of the model. We also used a calibration curve to evaluate the nomogram prediction models constructed and to evaluate the accuracy of the model. The diagonal of the calibration curve was used as the reference. This diagonal represented the optimal situation of the model, which is generally impossible to achieve. For comparison, the closer the line to 45°, the better the result; thus, the better the accuracy of the model. Moreover, using the external verification cohort, we established the prediction ability and accuracy of the model. SPSS (version 25.0) and R software (version 4.0.5, https://www.r-proje ct.org/) were used for statistical analysis. R software was used to develop predictive models using the "rms" package and the "surv" package to build nomograms.

| Patient baseline characteristics
A total of 982 patients aged 60 years or older who were diagnosed with osteosarcoma between 2004 and 2015 were included in this study; all patients were drawn from the SEER database. Of these, 445 patients had unreported or unknown T-stage, 220 had unknown grade or tumor size, and after excluding 11 patients with unknown marital status, 306 complete data were obtained, as shown in the exclusion flow chart in Figure 1. We also found that the clinical characteristics of the modeled population and externally validated patients were summarized, as shown in Table 1. Of the modeled patients, 156 (51%) were male and 150 (49%) were female, compared to 60.7% male and 22 (39.3%) female in the validation population. In addition, in terms of tumor grade, poorly differentiated (grades III and IV) were the most common in the modeling population at 87.3%, compared to 92.8% in the validation population. T stage was predominantly T1 and T2 in both groups, and in addition the majority of patients underwent surgery, indicating that surgery remains readily accepted. The specific clinical information and pathological characteristics of all patients are shown in Table 1.

| Kaplan-Meier survival analysis
We used all clinical variables and pathological characteristics of osteosarcoma patients to calculate the outcomes using the Kaplan-Meier analysis, which was also used to analyze interesting variables not reported in the baseline data table. The specific survival curves are shown in Figure 2. The prognosis of patients aged 70-79 years was better than that of patients aged 60-69 and >80 years. The prognosis of the surgically treated patients was significantly stronger than that of those who received surgical treatment. Of these, the prognosis of grade I and II patients was better, while the prognosis of grade III and IV was inferior, which is also an important finding. Of interest, the prognosis of osteosarcoma patients who received radiotherapy or chemotherapy was similar to that of patients who do not receive therapy, and may correspond to one of the reasons for the low proportion of osteosarcoma patients willing to undergo chemotherapy or radiotherapy. Willingness to undergo therapy may be associated with the lack of obvious contribution to survival and prognosis. Kaplan-Meier findings, including the described variables, are shown in Figure 2.

| Construction of the OS nomogram
After a Cox regression analysis of the patient data, we found that the most statistically significant overall survival for osteosarcoma patients over 60 years of age was for eight variables, including age, gender, tumor grade, and size, as shown in Table 2. Therefore, we next chose these eight variables to construct a nomogram for OS. The nomograms were constructed by collecting all clinical variables and pathological characteristics of osteosarcoma patients over 60 years of age. Figure 3 shows the nomogram chart, which differs from previous studies because it is drawn using as clinical features and pathological characteristics of many variables. The objective was to achieve a personalized prediction of OS in patients over 60 years of age. Based on the nomogram, the basic, clinical and pathological characteristics of various patients on survival and prognosis are reflected by the line that indicates the total points and the corresponding single score for each factor can be obtained. Individualized scores can be determined for each patient. The individual scores corresponded to basic information, clinical, and pathological characteristics of the patient, and then the calculated total score corresponded to the percentage of line segments in the

| Construction of the CSS nomogram
After Cox regression analysis of the patient data, we found that the most statistically significant variables for cancer specific survival in osteosarcoma patients over 60 years of age were age, sex, grade, and size of the tumor, as shown in Table 3. Therefore, we next chose these eight variables to construct a nomogram for the CSS. A CSS nomogram was developed for osteosarcoma patients aged 60 years or older using all the variables collected ( Figure 4). It was drawn by combining multiple factors including general information, clinical and pathological characteristics. In the nomogram, the combination of numbers produced a score. The influence of each factors on outcome differed; therefore, each score reflected a different outcome.

CSS nomograms
ROC curves ( Figure 6) were used to evaluate the predictive performance of the 3-and 5-year OS for the training groups (AUC = 0.831 and AUC = 0.828, respectively) ( Figure 6A) and the for external verification group (AUC = 0.730 and AUC = 0.765, respectively) ( Figure 6C), the 3-and 5-year CSS for the training group (AUC = 0.829 and AUC = 0.827) ( Figure 6B) and for the external verification group of (AUC = 0.805 and AUC = 0.708) ( Figure 6B). Overall, these findings indicate that the models were effective predictive nomograms. Figure 5 shows the calibration diagram. The calibration curves of the training group and the validation group ( Figure 5) were used to verify the accuracy of the nomogram prediction results relative to the true incidence rate. When the calibration curve is diagonal, which is ideal case, the prediction probability of the calibration curve is equal to the actual probability. In general, this is not a straight diagonal line. However, the closer to the diagonal line, the better the predictive effect. Finally, we found that the calibration curve obtained from the study showed a good fit with its 45° diagonal line. We also determined that the OS C-index of the training population was 0.827 (95%CI 0.778-0.876), and the CSS C-index of the training population was 0.722 (95%CI 0.665-0.779), which indicated that the accuracy of the nomogram models was very good. Figure 7 shows decision curve analysis. We also used the DCA curve to show that the nomograms calculated from the training population are of great significance and effectiveness in clinical application.

| DISCUSSION
Cancer has become a global problem that cannot be ignored-is one of the most common causes of death among older aged adults, with a high mortality rate from osteosarcoma. Over 3600 new bone cancer diagnoses and 1720 deaths from bone cancer occur every year in the United States, 16 with osteosarcoma being the predominating type. Based on the SEER database, we constructed two new nomogram tools that are easy to use and exhibited high precision in verification cohorts, indicating that they can be used to predict survival probability in terms of 3-and 5-year OS and CSS of osteosarcoma patients over 60 years of age. This could be considered an improvement over a previous study that neglected to establish a predictive model for patients over 60 years of age, especially given that global aging is an increasing concern. In both the training and the validation populations, the nomograms showed very good and convincing accuracy, which indicated that the nomograms have potential clinical practical significance. No study in osteosarcoma patients has evaluated as many variables as those evaluated in the present study.
Nomograms have long been used to predict clinical diseases, especially the survival and prognosis of various cancers, mainly because nomograms can simplify the statistical prediction model into individual scores, which can be customized for individual patients, and the final score estimates the probability of the end point (such as death or metastasis). The easy-to-understand graphical interface of the nomogram facilitates their use when clinicians communicate with patients about treatment options or prognostic strategies and provides clinicians with more optional decision-making information. 17 Nomograms satisfy the need for biological and therapeutic integrated models, as well as the need for personalized treatment that can provide unique patient prognosis prediction. 18 Validation of the nomogram before clinical application is F I G U R E 4 Nomogram for the prediction of 3-and 5-years CSS of osteosarcom of over 60 years old. crucial for physicians. 19 However, the statistics used in the construction of these models must be carefully reviewed, and the uncertainty of point estimation must be verified and calibrated using different approaches. Many previous studies have developed prediction models that estimate the survival and prognosis of osteosarcoma patients. Among these, some studies established nomograms but did not distinguish between age groups. 20,21 Additionally, Zhao et al. constructed and validated a nomogram for patients with osteosarcoma in patients under 60 years of age. 22 However, in our study, osteosarcoma patients older than 60 years were used to establish our model, which reflects the importance of stratifying patients over the age of 60 years. Furthermore, research on predictors of prognosis and survival in patients with osteosarcoma remains controversial. Therefore, considering the above reasons, our study aim was to develop and validate an easy-tounderstand, high-precision nomogram tool. The age range was limited to patients 60 years or older. The most important difference of our model is that we used many variables to establish the tool. To better reflect the prediction ability of the nomogram, we evaluated the nomogram in the most comprehensive way. For many cancers, studies have long recognized age as an important factor that affects prognosis and survival 23,24 in patients with osteosarcoma. A retrospective study by Hagleitner et al. analyzed 102 osteosarcoma patients. The prognosis of older patients was relatively poor, while the survival prognosis of younger patients was much better. 25 Furthermore, Tsuchie et al. used age at 40-years as the cut-off value, to evaluate the prognosis of younger and older individuals with primary osteosarcoma and determined that older patients exhibited a poorer prognosis. 26 Our study showed that OS and CSS in patients with osteosarcoma over 60 years of age were better in those of patients 70-79 years of age.
The usefulness of chemotherapy for osteosarcoma patients is controversial. Our study also determined that OS and CSS in patients with osteosarcoma over 60 years of age receiving chemotherapy was slightly superior than those who did not receive chemotherapy, but this difference was not significant. Reasons may be related to delayed medical advice, resistance to chemotherapy, low tolerance to unstable surgery, and other metastases at diagnosis. A total of 736 patients with osteosarcoma in adolescents ineligible for chemotherapy were included in the latest study. 27 This is also similar to our findings, although the age groups differed.
Our study showed that marital status is also an important factor affecting the survival of osteosarcoma patients-married patients having a slightly better prognosis than single patients-which is similar to the results of a study by Qiu et al. 28 Those researchers concluded that marital status was related to survival, implying that married patients had a higher survival rate than widowed subjects, and worse prognoses of osteosarcoma. Several recent studies have confirmed a relationship between marital status and between survival performance, marital status being a predictor of OS and CSS. In most cases, the prognosis of widowed patients used to be worse than in different groups. 29 Another study also, confirmed that the risk of death was greater in unmarried, divorced/separated, and widowed patients in contrast to the married patients. 30 Our study further highlights how the impact of marital status on cancer is similar to that of other studies describing the close relationship between cancer and marital status. [31][32][33][34] The benefits of marriage for the disease may be due to the fact that both partners support each other, give each other confidence in overcoming the disease, reduce anxiety and negative thoughts, and are more willing to cooperate with treatment. Therefore, clinicians should improve awareness and decision-making approaches for single unmarried patients.
Surgical resection of the primary tumor site plays a key role in the management of patients with osteosarcoma.
In our investigation, we demonstrated favorable survival rates in older surgically resected patients. Similar to the results of other studies, surgical treatment enhanced the prognosis of the patient and survival. 35,36 The SEER database does not collect data throughout chemotherapy and surgical treatment, so we could not explore the impact of these two variables.
The improvement in patient prognosis of patients receiving chemotherapy was not significant. 27 Based on the SEER database, this study analyzed data limited to age, race, sex, marital status, tumor grade, location of tumor, tumor laterality, tumor size and T, N, M stage, surgical treatment, chemotherapy, and radiation therapy were predictors of survival in osteosarcoma patients 60 years and older. Although there have been many studies of bone tumors in recent years, 22,37,38 none have examined the survival rates of patients over 60 years of age with osteosarcoma as we have done in this study. It should be clear that nomograms cannot fully and accurately predict the prognosis and survival of all patients. Although we used ROC curve analysis and its AUC, as well as the DCA, Cindex, and calibration chart, nomograms can only be used as a tool for clinical work. Finally, a prediction model for patients with osteosarcoma warrants further research and development. In addition, SEER data does not contain variables including body mass index, alcohol, and tobacco consumption habits, which are also have a potential influence on patient prognosis and survival time, and thus, represents a study limitation. The insufficient number of patients collected is one of the limitations of our study because osteosarcoma is inherently a rare disease, and although patients over 60 years of age have the second highest incidence of osteosarcoma, the number is even rarer compared to adolescent patients. We collected multicenter validation, more using data from Chinese patients for validation. This is one of the limitations of our manuscript. But, we believe that our use of data from Chinese patients is generalizable in that we collected all consecutive patients meeting the criteria over a period of time, and although all are persons of the yellow race, there is no significant bias, and these patients are inherently highly randomized. More importantly, persons of the yellow race we used, if validated with fair results, is more indicative of better extrapolation of the model, as models are often built with blacks and whites in mind.

| CONCLUSIONS
OS and CSS at 3 and 5 years were predicted in osteosarcoma patients aged over 60 years using a nomogram including a sizeable number of patient characteristics. The nomogram was shown to be accurate and reliable, and would allow patients to receive better and more personalized care and clinicians to forecast patients' survival prognosis, make more scientific judgments, and design therapy and follow-up procedures.