The Outpatient Bleeding Risk Index (OBRI) and the Kuijer, RIETE and Kearon scores are clinical prognostic scores for bleeding in patients receiving oral anticoagulants for venous thromboembolism (VTE). We prospectively compared the performance of these scores in elderly patients with VTE.
In a prospective multicenter Swiss cohort study, we studied 663 patients aged ≥ 65 years with acute VTE. The outcome was a first major bleeding at 90 days. We classified patients into three categories of bleeding risk (low, intermediate and high) according to each score and dichotomized patients as high vs. low or intermediate risk. We calculated the area under the receiver-operating characteristic (ROC) curve, positive predictive values and likelihood ratios for each score.
Overall, 28 out of 663 patients (4.2%, 95% confidence interval [CI] 2.8–6.0%) had a first major bleeding within 90 days. According to different scores, the rate of major bleeding varied from 1.9% to 2.1% in low-risk, from 4.2% to 5.0% in intermediate-risk and from 3.1% to 6.6% in high-risk patients. The discriminative power of the scores was poor to moderate, with areas under the ROC curve ranging from 0.49 to 0.60 (P = 0.21). The positive predictive values and positive likelihood ratios were low and varied from 3.1% to 6.6% and from 0.72 to 1.59, respectively.
In elderly patients with VTE, existing bleeding risk scores do not have sufficient accuracy and power to discriminate between patients with VTE who are at a high risk of short-term major bleeding and those who are not.
The annual incidence of venous thromboembolism (VTE) rises exponentially with age, varying from less than one case per 1000 person-years in persons aged below 50 years to more than six cases per 1000 person-years in persons aged 80 years or older [1, 2]. Elderly patients with VTE who receive oral anticoagulation have a higher overall mortality and a higher risk of bleeding complications than younger patients [2, 3]. While the 3-month risk of major bleeding usually ranges from 2% to 3% among patients receiving oral anticoagulants, this risk doubles in patients aged ≥ 65 years . The risk of intracerebral bleeding doubles in patients aged ≥ 80 years . The increased bleeding risk in the elderly can be explained by several factors: a lower metabolic clearance of vitamin K antagonists, multiple comorbidities, polypharmacy, pathological changes in cerebral vessels (e.g. amyloid angiopathy) and an increased risk for falls .
Evidence suggests that physicians estimate the probability of major bleeding no better than expected by chance . Methods to identify patients with VTE who are at high risk of bleeding could be useful because such patients could potentially benefit from more intensive anticoagulation monitoring and the avoidance of concomitant use of platelet inhibitors. To aid physician decision-making, four clinical prognostic scores have been previously developed to estimate the risk of major bleeding in patients receiving oral anticoagulants for VTE: the Outpatient Bleeding Risk Index (OBRI), and the Kuijer, Kearon and RIETE score (Appendix) [6-9]. Based on predefined clinical and laboratory variables, these scores stratify patients under oral anticoagulation into three categories of increasing risk of major bleeding, with bleeding incidences varying from 0% to 2.1% among low-risk and from 7.3% to 23% among high-risk patients [6-9]. To our knowledge, these scores were never directly compared in elderly patients with VTE and their prognostic performance in such patients is uncertain. In a multicenter cohort of elderly patients with VTE, we sought to prospectively examine the prognostic performance of these bleeding risk scores and, specifically, whether these scores accurately identify individuals who are at a high risk of bleeding. Because the risk of bleeding is higher in the early phase of anticoagulation and two of the four scores were derived to predict the 3-month bleeding rate, we focused on the first 90 days after VTE [7, 9].
The study was conducted between September 2009 and June 2011 as part of a prospective, multicenter cohort study to assess medical outcomes of patients aged ≥ 65 years with acute, symptomatic VTE (deep vein thrombosis [DVT] and/or pulmonary embolism [PE]) from all five Swiss university and four high-volume non-university hospitals. Potential participants were consecutively identified in the inpatient and outpatient services of all participating study sites. We defined DVT as the acute onset of leg pain or swelling plus incomplete compressibility of a venous segment on ultrasonography or an intraluminal filling defect on contrast venography . Because the iliac vein and the inferior vena cava may be technically difficult to compress, an iliac/caval DVT was defined as abnormal duplex flow patterns compatible with thrombosis or an intraluminal filling defect on contrast computed tomography (CT) or magnetic resonance imaging venography . Given that ultrasonography has a reduced sensitivity and specificity for a distal DVT, patients with a distal DVT were included only if the incompressible distal vein transverse diameter was at least 5 mm . We defined PE as the acute onset of dyspnea, chest pain or syncope coupled with a new high-probability ventilation/perfusion lung scan; a new contrast filling defect on spiral CT or pulmonary angiography; or the new documentation of a proximal DVT either by venous ultrasound or contrast venography [13, 14]. Radiographic studies used to diagnose VTE were interpreted by on-site vascular specialists or radiologists. The type of anticoagulant treatment (i.e. parenteral anticoagulant followed by vitamin K antagonists or parenteral anticoagulation alone) was left to the discretion of the managing physicians.
We excluded patients with the following criteria from the cohort: (i) thrombosis at a different site than the lower limb (e.g. mesenteric or cerebral vein thrombosis); or catheter-related thrombosis because the natural history of these conditions might be different ; (ii) insufficient spoken language ability in German or French; (iii) inability to provide informed consent (e.g. due to severe dementia); (iv) follow-up not possible (e.g. terminally ill patients); and (v) prior enrollment in the cohort. We asked eligible patients to provide informed consent. The study was approved by the ethics committees at each participating center.
Baseline data collection
For all enrolled patients, trained study nurses prospectively collected baseline demographic information (age and gender), comorbid conditions (cancer, anemia, diabetes mellitus, chronic renal and liver disease, a recent myocardial infarction, and a history of a stroke and a history of bleeding), laboratory findings (hemoglobin, creatinine and platelets), VTE-related treatment before and after the event (low-molecular-weight heparin, unfractionated heparin, fondaparinux and vitamin K antagonists) and concomitant antiplatelet therapy using standardized data collection forms.
Based on patient demographics and baseline clinical data obtained by chart review, we determined the presence of the prognostic variables comprising the OBRI and the Kuijer, Kearon and RIETE score. Using the prognostic variables, we calculated the four prognostic scores and classified each patient in the low-, intermediate- or high-risk category (Appendix). Because our database did not distinguish between a stroke and transient ischemic attack, we used a history of a stroke or transient ischemic attack as a proxy variable for a history of a stroke [6, 8]. We calculated the hematocrit value in percent by multiplying the hemoglobin level in g dL−1 by three [6, 16].
For any of the variables constituting the four prognostic scores, missing values were assumed to be normal. This strategy is widely used in the clinical application of prognostic models [17, 18]. In a sensitivity analysis, all missing laboratory values were assumed to be abnormal.
Our primary outcome was the occurrence of a first major bleeding at 90 days after the index VTE. We defined major bleeding as a fatal bleeding, a symptomatic bleeding in a critical organ (intracranial, intraspinal, intraocular, retroperitoneal, intraarticular, pericardial, or intramuscular with compartment syndrome), a bleeding with a reduction of hemoglobin ≥ 20 g L−1 or a bleeding leading to the transfusion of ≥ 2 units of packed red blood cells . The secondary outcome was the occurrence of a first non-major bleeding requiring medical attention at 90 days after the index VTE. We assessed outcomes at a clinic visit 90 days after the index VTE event. Study nurses interviewed patients and their primary care physicians and reviewed medical charts to document any bleeding episodes. In patients who died during follow-up, study nurses obtained information about the cause of death from hospital discharge letters and autopsy reports if available.
Based on all available information, a committee of three blinded clinical experts adjudicated all bleeding events and determined the cause of death. Death was considered to be bleeding related if it followed an intracranial hemorrhage or a bleeding episode leading to hemodynamic deterioration . All other deaths were classified as not bleeding related. Final classification was based on the full consensus of this committee.
We compared baseline characteristics of patients with and without a first major bleeding using chi-square tests for categorical variables and non-parametric rank tests for continuous variables. We estimated the cumulative incidence of a first major bleeding using the Kaplan–Meier technique. We described the proportion of low-, intermediate- and high-risk patients and the 90-day major and non-major bleeding rate within each category for each score. To determine the accuracy of each score to predict a first major bleeding at 90 days, we estimated sensitivity, specificity, and positive and negative predictive values and likelihood ratios. Because we were specifically interested in individuals who are at a high risk of bleeding, we dichotomized patients as low and intermediate vs. high risk. We assessed the discriminative power of each score to predict a first major bleeding at 90 days by calculating the area under the receiver-operating characteristic (ROC) curve, performing a non-parametric test of the equality of the areas under the four ROC curves. We determined the goodness-of-fit of the score points for each score in a logistic regression model using Pearson's chi-square test. We also calculated the percentage of time spent in a given international normalized ratio (INR) range (< 2.0, 2.0–3.0 and > 3.0) . All analyses were done using stata 11 (Stata Corporation, College Station, TX, USA).
Our final study sample comprised 663 patients (Fig. 1). Excluded patients were older (median age 78 vs. 75 years, P < 0.001) and more likely to be women (59.2% vs. 45.4%, P < 0.001) than analyzed patients.
Patients who experienced a first major bleeding were significantly older and less likely to have received low-molecular-weight heparin for initial treatment than patients who did not (Table 1). The median bleeding risk score values were not significantly different across patients with or without a first major bleeding. Overall, 85.4% received treatment with vitamin K antagonists during the 90 days after the VTE event. Among these, the percentage of time spent in the therapeutic INR range (2.0–3.0) was 54.2%.
VTE, venous thromboembolism; PE, pulmonary embolism; DVT, deep vein thrombosis; GI, gastro-intestinal; VKA, vitamin K antagonist; INR, international normalized ratio. *Data were missing for hemoglobin (5.4%), serum creatinine (6.5%), platelet count (5.6%) and INR values (7.2%). †Defined as cancer that required therapy during the past 3 months. ‡Defined as a myocardial infarction occurring during the past 3 months. §Defined as history of ischemic or hemorrhagic stroke, or a transient ischemic attack. ¶Defined as liver cirrhosis, chronic hepatitis, chronic liver failure or hemochromatosis. **Defined as diabetic or hypertensive nephropathy, chronic glomerulonephritis, chronic interstitial nephritis, myeloma-related nephropathy, or cystic kidney disease.††Defined as any bleeding leading to a hospital admission or red blood cell transfusions during the last 3 months. ‡‡Defined as any previous gastrointestinal bleeding leading to hospital admission or transfusions. §§Defined as serum hemoglobin < 13 g dL−1 for men or < 12 g dL−1 for women.¶¶The hematocrit was obtained by multiplying the hemoglobin level (in g dL−1) by three .***Defined as a platelet count < 150 109 L−1. †††Defined as the use of aspirin, clopidogrel, prasugrel, or aspirin/dipyridamol. ‡‡‡Of the 97 patients who did not receive treatment with VKA, 87 received another anticoagulant (e.g. low-molecular-weight heparins or fondaparinux) and 10 did not receive any anticoagulant at all. §§§Defined as catheter-directed or systemic thrombolysis. ¶¶¶Indicates the percentage of time spent in a given INR range during the 90-day follow-up period. INR values during the first 7 days of VKA treatment were excluded.
Overall, 28 patients (4.2%) experienced a first major bleeding within 90 days of the index VTE event, resulting in a total of 32 major bleeding episodes. Bleeding was fatal in three cases (0.5%). Most first major bleedings occurred during the first 45 days of follow-up (Fig. 2). Thirty-four patients developed a first non-major bleeding, with a total of 45 non-major bleeds. Overall, 38 patients (5.7%) died within 90 days.
Given that age is a predictor variable in the OBRI (≥ 65 years) and the Kuijer score (≥ 60 years), these scores did not classify any patients as low risk in our study population aged 65 years or older (Table 2). The proportion of patients classified as high risk varied from 7.1% for the OBRI to 17.3% for the Kearon score. The proportion of a first major bleeding in the low-, intermediate- and high-risk category was 1.9–2.1%, 4.2–5.0% and 3.1–6.6%, respectively, whereas the proportion of a first non-major bleeding was 0–3.7%, 5.0–5.8% and 3.3–7.8% (Table 2).
Table 2. Risk classification of patients and first bleeding at 90 days
Patients with a first non-major bleeding* (N = 34)
% (95% CI)
% (95% CI)
% (95% CI)
CI, confidence interval; OBRI, Outpatient Bleeding Risk Index. *The denominator used was the number of patients per risk category. Bleeding rates could not be statistically compared across prediction rules because the denominators differed.
Comparison of predictive accuracy and discriminatory power
When dichotomized as high vs. intermediate and low risk, the four scores had low sensitivities (7.1–21.4%), positive predictive values (3.1–6.6%) and positive likelihood ratios (0.72–1.59) for predicting a first major bleeding at 90 days (Table 3). The Goodness-of-fit was adequate for each score. The areas under the ROC curve varied from a low 0.49 (95% CI 0.45–0.52) for the Kuijer score to a moderate 0.60 (95% CI 0.56–0.64) for the RIETE score, without any significant difference in the overall comparison (P = 0.21) (Fig. 3). When we assumed missing values to be abnormal or when we excluded patients who did not receive any vitamin K antagonists during follow-up in sensitivity analyzes, the results did not change markedly. The results remained also similar when we used a more restrictive definition of major bleeding, excluding bleeding episodes that were merely followed by a decline in hemoglobin level > 20 g L−1.
Table 3. Accuracy for high- vs. intermediate-/low-risk categories to predict a first major bleeding at 90 days
Our prospective comparison showed that the four prognostic scores did not have adequate accuracy and power to discriminate between elderly patients with VTE who are at a high risk of short-term major bleeding and those who are not. Although the RIETE score had a slightly better positive predictive value (6.6%) and positive likelihood ratio (1.59) than the other scores, neither of the four scores appeared particularly useful in identifying elderly patients with VTE who are at a high risk of major bleeding. The discriminative power of the four scores was poor to moderate at the best, with the areas under the ROC curve varying from 0.49 for the Kuijer to 0.60 for the RIETE score.
Our results are consistent with a recent prospective study of anticoagulated patients (25% had VTE), in which the areas under the ROC curve for the OBRI, Kuijer and RIETE score were 0.56, 0.54 and 0.57, respectively . Our results contrast with the more optimistic findings of the original score derivation studies. In these studies, the OBRI and the Kuijer scores had an area under the ROC curve of 0.72 and 0.82, respectively [6, 9]. These differences may be explained by several factors. First, with a median age of 75 years, our study population was older and therefore, probably sicker than the patients enrolled in the score derivation studies. As a consequence, the incidence of major bleeding (4.2%) was somewhat higher in our study than in the derivation studies of the Kuijer, Kearon and RIETE scores (2.3–3.7%) [7-9]. A notable exception is the OBRI that used data collected between 1977 and 1983 when more aggressive oral anticoagulant regimens were common. This may have resulted in the high 3-month major bleeding incidence of 5.2% . Second, the score derivation studies used a slightly different definition for major bleeding (Appendix), which could potentially explain differences in the prognostic performance. Third, patients were followed for a longer time in the OBRI (median 7 months) and Kearon (average 2.4 years) derivation studies (Appendix). Because bleeding risk is known to vary with time, these different follow-up times may have influenced the prognostic accuracy of these scores in our analysis . Finally, the relatively low proportion of patients with VTE (15%) in the derivation sample of the OBRI could explain the relatively modest prognostic performance of this score . Although the OBRI has been successfully externally validated in prospective studies comprising a substantially larger proportion of patients with VTE, these patients were substantially younger (mean age 58–60 years) than in our study [6, 24].
Our study has important clinical and research implications. Existing clinical bleeding risk scores have a limited accuracy and discriminative power in elderly patients with VTE and cannot be recommended for routine use in clinical practice for the elderly. Thus, novel clinical risk assessment models are needed that accurately and reliably predict the risk of major bleeding in such patients.
Our study has several potential limitations. First, our sample may not reflect the full prognostic spectrum of patients with VTE because enrolled patients were younger and more likely to be men than excluded patients. Thus, we cannot exclude the possibility that the four prognostic scores would have performed differently in more severely ill patients. However, the 54% enrollment rate that we achieved is laudable for a prospective cohort study focusing on elderly patients and compares well with the Cardiovascular Health Study, in which 39.6% of elderly patients with whom contact was made, were enrolled . Second, in our analysis we assumed missing values for any of the predictor variables to be normal, a strategy that is widely used in the clinical application of prognostic models [17, 18]. Our results remained similar when missing values were assumed to be abnormal in a sensitivity analysis, confirming the robustness of our findings. Finally, because peptic ulcer disease was not recorded in our database, we assumed this variable to be normal when calculating the Kearon score. Although the incidence of this condition is very low (0.37 per 1000 person-years), it is possible that our analysis underestimated the bleeding risk for the Kearon score [8, 26]. Similarly, we used a history of stroke or transient ischemic attack as a proxy variable for history of a stroke (OBRI, Kearon score). Thus, we cannot exclude that our analysis overestimated the bleeding risk for these scores.
In conclusion, our results indicate that existing bleeding risk scores do not have sufficient accuracy and power to discriminate between elderly patients with VTE who are at a high risk of short-term major bleeding and those who are not. Thus, these scores do not appear to be useful for targeting preventive interventions in elderly patients receiving anticoagulants for VTE. Novel prognostic scores should be developed that accurately predict the risk of bleeding in such patients.
N. Scherz, M. Méan, A. Limacher: planning of the study, data collection, statistical analyses and drafting of the manuscript. M. Righini, K. Jaeger, H. -J. Beer, B. Frauchiger, J. Osterwalder, N. Kucher, A. Angelillo-Scherrer, B. Lämmle, H. Bounameaux, J. Cornuz, N. Rodondi: data collection, intellectual review of the manuscript and obtaining funding from the Swiss National Science Foundation. C. M. Matter, M. Banyai, M. Husmann, M. Egloff, M. Aschwanden: data collection, intellectual review of the manuscript. D. Aujesky: principal investigator, planning of the study, data collection, drafting of the manuscript, obtaining funding from the Swiss National Science Foundation.
This work was supported by a grant of the Swiss National Science Foundation (no. 33CSCO-122659).
Disclosure of Conflicts of Interests
The authors state that they have no conflict of interest.
Appendix: Bleeding risk scores for venous thromboembolism
a. Study design b. Sample size c. Average patient age d. Length of follow-up e. Indication for anticoagulation
Overt bleeding with a decline in hemoglobin level ≥ 20 g L−1, if there was a need for transfusion ≥ 2 units of red blood cells, if it was retroperitoneal or intracranial, or if it warranted permanent discontinuation of treatment
Overt bleeding associated with a decrease in the hemoglobin level ≥ 20 g L−1 or a need for transfusion ≥ 2 units of red cells or if it involved a critical site (e.g. retroperitoneal or intracranial bleeding)