Validation of a novel algorithm with a high specificity in ruling out MDS

A previously published web‐based App using Gradient‐boosted models (GBMs) of eight laboratory parameters was established by Oster et al. to facilitate diagnosis or exclusion of myelodysplastic syndromes (MDS) in patients.


| INTRODUCTION
Myelodysplastic syndromes (MDS) are a heterogeneous group of clonal myeloid disorders arising from hematopoietic stem cells.Dysplastic changes of bone marrow and peripheral blood cells are accompanied by ineffective hematopoiesis.Clonal expansion leads to acute myeloid leukemia (AML) in about 25% of patients. 1The incidence of MDS is about 4/100.000 and the prevalence about 7/100.000 in the European Union (EU). 2 Diagnosis of MDS requires bone marrow aspiration and/or core biopsy to analyze cytomorphology, histopathology and cytogenetics.Additionally, molecular genetic analyses are increasingly used as a diagnostic and prognostic tool.Since the median age at time of diagnosis is approximately 75 years, associated with an increased likelihood of comorbidities, most MDS patients are not eligible for intensive therapy.These patients are also likely to prefer non-invasive diagnostics over invasive bone marrow biopsy.Recently, Oster et al. published a predictive algorithm to aid in diagnosing or ruling-out MDS. 3 This algorithm is based on Gradient-boosted models (GBMs), a type of machine learning technique that produces a strong prediction model by combination of weak prediction models.Using the patients' gender and age as well as eight laboratory parameters it assesses the probability of having an MDS or not.These eight parameters are white blood cell count (WBC), hemoglobin level, platelet count, mean corpuscular volume (MCV), absolute neutrophil count (ANC), absolute monocyte count, as well as creatinine and blood glucose.The authors used data of 502 MDS patients from the European MDS registry and compared them to a control group of 502 patients who initially showed unexplained anemia but had MDS ruled out by bone marrow investigations.
The analyses based on that cohort resulted in the accurate prediction and exclusion of MDS in 86% of the patients.Depending on the calculated GBM score (ranging from 0 to 1), patients were categorized as "probably having an MDS" (pMDS), "probably not having an MDS" (pnMDS) or as "indeterminate."A GBM score of less than 0.68 yielded a negative predictive value (NPV) of 0.94, which means that 94% of patients with a score <0.68 were correctly categorized as pnMDS.A GBM ≥0.82 yielded a positive predictive value (PPV) of 0.88, indicating that 88% of patients with a score >0.82 were correctly categorized as pMDS.The remaining patients (0.68 ≥ GBM < 0.82) were classified as indeterminate. 3To facilitate and improve the diagnostic approach to patients with suspected MDS, we aimed to validate the Web-based calculator by applying it to an independent cohort of 175 patients with a diagnosis of MDS included in the German MDS Registry as well as a control cohort of 1378 patients without a diagnosis of MDS.

| PATIENTS AND METHODS
We used the aforementioned laboratory results of 175 anemic MDS patients from our German MDS Registry, which contains patients diagnosed with MDS in our department of hematology between 1982 and 2021, and calculated their GBM score using the web-based app.
We chose the laboratory values which have been closest to MDS diagnosis and excluded patients with missing data.MDS diagnoses were based on bone marrow examinations performed according to our standardized cytomorphologic analysis. 4The distribution of MDS subtypes in the cohort is shown in Table 1.The control group consisted of 1378 randomly selected patients of the university hospital in Dusseldorf whose laboratory tests were triggered by non-hematologic indications.The only requirement was that patients were anemic (according to WHO definition 5 ) and not known to have MDS as the underlying cause.Clinical characteristics and laboratory parameters of MDS patients and controls are shown in Table 2. Medical departments taking care of patients in the control group were mainly gastroenterology (31.6%), neurology (24.6%), and nephrology (23.7%), followed by cardiology (7.4%) and others except for hematology.Furthermore, we analyzed 29 patients from our MDS Registry with chronic myelomonocytic leukemia (CMML) to test how the calculator classifies this entity.We calculated sensitivity and specificity of all mentioned cohorts.For the calculation of positive predictive values

| RESULTS
Classification of patients after applying the algorithm is shown in Figure 1.Among 175 MDS patients from the Registry, the web-based app correctly identified 54 individuals as probably having MDS (pMDS), yielding a sensitivity of 30.9%.A total of 84 patients were incorrectly classified as probably not having MDS (48%), and the remaining 37 patients were rated as indeterminate (20.3%).Of all control patients, 65 (4.7%) were wrongly allocated to the pMDS group, resulting in a specificity of 90.9% (for our calculation of sensitivity and specificity, we handled "indeterminate" as wrongly allocated, too).Of these 65 individuals, 48 (73.8%) either had a diagnosis that explained the laboratory abnormalities or underwent treatment known to be hematotoxic.The diagnoses in the control group are shown in disease.Later on, though, a hematological disease was diagnosed in these patients: Myeloma (two patients), lymphoma (one patient), acute myeloid leukemia (one patient), essential thrombocythemia (one patient) and myelodysplastic syndrome with multilineage dysplasia after the treatment of prostate cancer (one patient).Thus, in this case, allocation into the subgroup "probable MDS" cannot be categorized as incorrect.Of note, Oster et al. also included control patients who underwent bone marrow biopsy as part of the diagnostic workup for a known lymphoproliferative disorder, which again may cause laboratory abnormalities.Interestingly, we could not reproduce the high positive predictive value of 88% or the even higher negative predictive value of 94%.One possible reason might be the huge difference regarding risk stratification between our cohort and the cohort analyzed by Oster et al.The proportion of high risk patients in the German MDS registry is much higher compared to the EU MDS Registry. 6When comparing the degree of cytopenia according to WHO and IPSS between their MDS patients and ours, our patients were more often severely cytopenic, possibly causing the calculator to conclude that these patients may already have transformed to AML and should therefore be classified as pnMDS.In our view, a sensitivity of 31% does not qualify the webbased app as a useful diagnostic aid for physicians assessing patients with a possible diagnosis of MDS.Of 175 MDS patients in our study, almost 50% were classified as probably not having MDS.
In contrast, a specificity of nearly 91% suggests that the calculator may be useful to exclude MDS in a patient with an incidental finding of anemia.We conclude that all patients classified as probable MDS should still undergo bone marrow biopsy.The same applies to patients who are classified as indeterminate and do not show a plausible explanation for being anemic.Referring to our included CMML cohort, only one patient was categorized as pMDS while more than 95% of patients were classified as indeterminate or pnMDS suggesting that highly proliferative bone marrow disorders (typically going along with higher values of leukocytes) are correctly excluded.In our opinion, the web-based app will be useful for physicians treating non-hematologic patients who have anemia as an additional problem.The app could be employed as a first diagnostic step before referring the patient to a hematologist.By rendering a diagnosis of MDS highly unlikely, the app may help to avoid unnecessary bone marrow biopsies.Since MCV, serum creatinine and neutrophils are the main influential variables in this model, 2 patients with iron deficiency, for example, or renal anemia are reliably excluded as MDS candidates.Fortunately, the options for diagnosing MDS from peripheral blood samples via cytogenetic and molecular analysis are expanding, 7 we can envisage the calculator to harness that progress as shown in our analyses illustrating a realworld situation.
T A B L E 3 Diagnoses of control group patients wrongly classified as pMDS and their age distribution.

(
PPVs) and negative predictive values (NPVs) we reduced the size of the control group to 175 patients by randomly choosing about every seventh patient.Patients classified as indeterminate were not included in the calculation of PPV and NPV following the same procedure of calculation like Oster et al in the underlying analyses.All analyses were performed using IBM SPSS Statistics version 27.

Table 3 .
Using 175 of 1378 controls to analyze the positive and negative predictive value yielded a PPV of 77.1% and a NPV of 56.8%.Further differentiation of PPV, NPV, specificity and sensitivity for MDS patients and controls and their type of cytopenia according to WHO and IPSS are shown in Table 4.Of our 29 patients diagnosed with CMML, only one patient (3.4%) was assigned to the pMDS group T A B L E 1 MDS subtypes by WHO 2017 classification.Baseline characteristics and laboratory parameters of MDS patients and control group.
while 62.1% (18 patients) were classified as pnMDS and the remaining patients (34.5%) as indeterminate.4|DISCUSSIONWeevaluated the calculator of Oster et al. published in 2021 with the aim of possibly sparing patients the invasive diagnostic procedure of a bone marrow biopsy.Therefore, we analyzed 175 anemic MDS patients from our German MDS Registry and compared them to 1378 and 175 control patients, respectively, who were anemic without having MDS.In contrast to the cohort analyzed by Oster et al., our control group did not consist of bone marrow-proven non-MDS patients but of randomly selected anemic patients who consulted medical departments other than hematology.In this conchemotherapy, which probably explained their anemia or general cytopenia.Furthermore, 33% of these wrongly classified patients had received immunosuppressive or antiviral drugs, known to be hematotoxic, too.In six other patients of the control group, initial laboratory tests were initiated by non-hematological departments.At that time point, there was no indication for a hematological T A B L E 2 F I G U R E 1 Algorithm results Düsseldorf cohort.512SCHULZ ET AL.
PPV, NPV, sensitivity, and specificity for MDS patients and control group.