Predictive nomogram models for unfavorable prognosis after aneurysmal subarachnoid hemorrhage: Analysis from a prospective, observational cohort in China

Abstract Aim The aim of the study was to identify predictors for 3‐month poor functional outcome or death after aSAH and develop precise and easy‐to‐use nomogram models. Methods The study was performed at the department of neurology emergency in Beijing Tiantan Hospital. A total of 310 aSAH patients were enrolled between October 2020 and September 2021 as a derivation cohort, while a total of 208 patients were admitted from October 2021 to March 2022 as an external validation cohort. Clinical outcomes included poor functional outcome defined as modified Rankin Scale score (mRS) of 4–6 or all‐cause death at 3 months. Least absolute shrinkage and selection operator (LASSO) analysis, as well as multivariable regression analysis, were applied to select independent variables associated with poor functional outcome or death and then to construct two nomogram models. Model performance were evaluated through discrimination, calibration, and clinical usefulness in both derivation cohort and external validation cohort. Results The nomogram model to predict poor functional outcome included seven predictors: age, heart rate, Hunt‐Hess grade on admission, lymphocyte, C‐reactive protein (CRP), platelet, and direct bilirubin levels. It demonstrated high discrimination ability (AUC, 0.845; 95% CI: 0.787–0.903), satisfactory calibration curve, and good clinical usefulness. Similarly, the nomogram model combining age, neutrophil, lymphocyte, CRP, aspartate aminotransferase (AST) levels, and treatment methods to predict all‐cause death also revealed excellent discrimination ability (AUC, 0.944; 95% CI: 0.910–0.979), satisfactory calibration curve, and clinical effectiveness. Internal validation showed the bias‐corrected C‐index for poor functional outcome and death was 0.827 and 0.927, respectively. When applied to the external validation dataset, both two nomogram models exhibited high discrimination capacity [poor functional outcome: AUC = 0.795 (0.716–0.873); death: AUC = 0.811 (0.707–0.915)], good calibration ability, and clinical usefulness. Conclusions Nomogram models constructed for predicting 3‐month poor functional outcome or death after aSAH are precise and easily applicable, which can help physicians to identify patients at risk, guide decision‐making, and provide new directions for future studies to explore the novel treatment targets.


| INTRODUC TI ON
Aneurysmal spontaneous subarachnoid hemorrhage (aSAH) is a severe type of stroke, usually affecting young and healthy adults with a sudden dramatic onset. 1,2The mortality rate of aSAH is up to 35% and approximately one-third of survivors suffered from severe disability and functional dependence throughout their resting of life. 3Due to the high mortality and low functional independent rate of aSAH, it is crucial to predict the prognosis promptly and accurately instead of relying on clinical intuition. 4Clinical prediction models, with a combination of various characteristics of patients and disease, can serve as useful tools to evaluate individuals' outcomes, guide decision-making about targeted therapy, and facilitate effective communications with patients and their family members.
Several clinical prediction tools have been applied to evaluate the outcomes in patients with aSAH.6][7] However, these three grading systems do not incorporate other clinical parameters like age and comorbidities. 8Also, evaluating prognosis based on these systems is somewhat subjective because of the ambiguous description between adjacent scales. 9Therefore, researchers have devoted themselves to exploring models that combine various factors to better predict the prognosis.Recently, the Subarachnoid Hemorrhage International Trialists (SAHIT) group developed three models (a core, neuroimaging, and full model) based on pooled randomized controlled trial data to predict death or functional outcome after 3 months in patients with aSAH. 4 The core SAHIT models incorporate age, premorbid hypertension, and WFNS grade, which performed well and demonstrated good accuracy in an external validation cohort. 4Nonetheless, applying SAHIT models is difficult to some extent due to some poorly defined predictors like hypertension and their inconvenient use with an online calculator. 3,9Furthermore, predictors in these models just partially explained the variability in outcomes, indicating that other factors may have an impact on outcome prediction.Hence, it is crucial to incorporate other new, objective and readily available predictors to establish easy-to-use clinical prediction models in order to guide targeted treatment and provide inspirations for future studies.
Among the computational models to predict outcomes, the nomogram is a valuable and user-friendly prediction tool, since it can present a model graphically and have the advantage to accurately calculate the probability of a clinical event compared with the conventional odd ratios (ORs). 10erefore, based on this modern-day, observational cohort, we aimed to identify new predictors for 3-month poor functional outcome or all-cause death and develop two novel nomogram models.

| Patients
The study was a prospective, observational cohort study of aSAH

| Data collection
Clinical information including demographic data (sex and age), previous history (hypertension, diabetes mellitus, coronary heart disease, current smoking, and drinking), neurological status, radiological characteristics, and laboratory tests on admission were all collected.Neurological status on admission were assessed by Hunt-Hess grade and World Federation of Neurosurgical Societies (WFNS) grade.Radiological characteristics included modified Fisher grade, the number ("single" or "multiple") of aneurysms, the location ("anterior cerebral artery," "middle cerebral artery," "internal carotid artery," and "posterior circulation") of the aneurysms, 4 and the morphology of aneurysms ("single-sac aneurysms with smooth physicians to identify patients at risk, guide decision-making, and provide new directions for future studies to explore the novel treatment targets.
Treatment modalities, including surgical clipping, endovascular coiling, or conservative treatment, were determined by the professional and experienced endovascular specialists and neurosurgeons according to the condition of each patient under the guidance of the current guidelines. 12In addition, willing of the patients' family were also taken into fully consideration.All treatment methods were not at the discretion of our researchers, and patients in our study were not predefined to a certain group of an intervention or exposure.Also, there was no experimental therapy performed on enrolled patients and no biospecimens were additionally extracted or retained for further investigation.Therefore, our study was a purely observational study with no-trial designs.
According to the clinical registration requirements recommended by the International Committee Medical Journal Editors (ICMJE), a purely observational study does not require to register. 13,14Hence, the clinical registration information for the present study was waived.

| Outcomes
All the patients were followed up through telephone interviews at 3 months after symptom onset by trained interviewers who were blinded to baseline characteristics and prognostic factors.Clinical outcomes included unfavorable functional outcome defined as mRS score of 4-6 or all-cause death at 3 months. 4

| Sample size
Currently, the events per variable (EPV) criterion, especially an EPV of 10, is widely used as the lowest limit for logistics regression models to predict a binary outcome. 15A total of seven variables were finally included in the multivariable logistics regression analysis to predict poor functional outcome.Therefore, the effective sample size of the derivation cohort should be at least 70 patients.Additionally, considering the occurrence of poor functional outcome after aSAH is around 30% worldwide, 3 there should be at least 233 patients in the derivation cohort.

| Statistics analysis
Statistics analysis was performed using IBM SPSS Statistics for Windows, version 22.0 (IBM Corp, Armonk, NY, USA) and R software (https://www.r-project.org/, version 4.1.2).Shapiro-Wilk test was conducted to check whether the included variables were normally distributed.Continuous variables are expressed as means ± SD or medians with interquartile range according to the distribution of the data.Categorical variables are described as numbers (percentages).
Concerning the limited sample size of our study, LASSO regression, which has the advantage to analyze high-dimensional data, 16 was performed to select the potential prognostic factors for unfavorable functional outcome and death, respectively.Features with nonzero coefficients in the LASSO regression were then entered into multivariable logistics regression models.Backward Wald selection method was applied to obtain the independent prognostic factors.Finally, nomogram models in the two dimensions of poor functional outcome and all-cause death were built based on the independent factors in logistics regression models respectively.
Model performance was assessed in terms of discrimination capacity, calibration ability, and clinical effectiveness.The area under the receiver operating characteristics (ROC) curve (AUC) was applied to evaluate the discrimination capacity of the nomogram models.DeLong's test was performed to compare the discriminative capacity between the nomogram models and the core SAHIT model.Calibration curves were plotted to show the agreement between the nomogram models and the expected observation.A plot near the 45-degree line would implicate the perfect calibration ability of the model.Furthermore, decision curve analysis was performed to assess the clinical effectiveness of the models by calculating the net benefits at different threshold possibilities. 17Moreover, bootstrapping method (1000 bootstrap resamples) was applied to validate the models internally and a corrected C-index was also calculated.In addition, the model performance in terms of discrimination, calibration and clinical effectiveness was also assessed using the external validation cohort.We used the "Matrix" and "glmnet" package to generate the LASSO regression results.The "SparseM," "ggplot2," "Formula," "survival," "lattice," "Hmisc," and "rms" package were applied to build nomogram models, generate the C-index and calibration curves.In addition, the "pROC" and "rmda" package were performed to plot ROC curves and decision curves, respectively.All tests of significance were two-tailed, and p-values <0.05 were considered statistically significant.

| Characteristics of patients in derivation cohort
A total of 310 patients diagnosed with aSAH were included in derivation cohort.Among them, 64.5% (200) were females and the mean age was 56.1 ± 12.4 years old.Unfavorable functional outcome after 3 months was observed in 17.1% of the patients, while 3-month all-cause death was found in 9.0% of the patients.The baseline characteristics of patients in derivation cohort grouped by 3-month poor functional outcome or all-cause death are shown in Table 1 and Table S1, respectively.

| Feature selection and model development
Least absolute shrinkage and selection operator regression analysis was applied to select the potential predictors.In terms of 3-month poor functional outcome, an optimal λ of 0.026 was adopted based on the minimum error criterion (Figure 1A).Therefore, a total of 46 variables were reduced to 15 variables according to the optimal λ (Figure 1B).These predictors with nonzero coefficients were age, diastolic blood pressure, heart rate, Hunt-Hess grade, lymphocyte, neutrophil, platelet, CRP, D-dimer, glucose, BUN, ALT, ALB, DBIL, and WFNS grade.
Similarly, regarding 3-month death, ten predictors with non-zero coefficients were screened based on an optimal λ of 0.029 (Figure 1C,D).3).
Based on the result of multivariable logistics regression analysis, two nomogram models were constructed to better visualize the estimation of the individual risk for poor functional outcome or allcause death (Figure 2A,B).Each predictor was projected upward to an exact point and the total sum of the points in the nomogram model were converted into an individual risk of the 3-month poor functional outcome or all-cause death.

| Model performance and validation
The ROC curve analysis showed that the nomogram model to predict 3-month poor functional outcome yielded greater AUC values (0.845, 95% CI: 0.787-0.903)than that of the core SAHIT model (AUC: 0.756, 95% CI: 0.685-0.826,p = 0.002) (Figure 3A).In terms of 3-month all-cause death, similar results were also discovered for the superior discrimination power of nomogram model (AUC: 0.944, 95% CI: 0.910-0.979)than did the SAHIT model (AUC: 0.755, 95% CI: 0.658-0.852,p < 0.001) (Figure 3).Compared with the core SAHIT models, the calibration curves of both two nomogram models displayed a more satisfactory agreement between prediction and actual observation for 3-month poor functional outcome or all-cause death (Figure S1).Furthermore, the decision curve analysis was displayed to explore the clinical usefulness of the nomogram models.
The result showed that both two nomogram models revealed superior overall net benefits compared with the SAHIT model (Figure S2).
The two nomogram models were internally validated by 1000 bootstraps validation method.The bias-corrected C index for predicting 3-month poor functional outcome and all-cause death was 0.827 and 0.927, respectively, which all exhibited excellent discrimination capacities.
External validation of two nomogram models was also conducted.A total of 208 eligible aSAH patients were included in the external validation cohort.The majority of the patients were female (63%) with a mean age of 56.8 years old.Poor functional outcome after 3 months was observed in 19.7% of the patients, while the mortality rate was 10.6%.No significant difference was identified between the derivation cohort and external validation cohort (Table S2).The result of ROC analysis, calibration curves, and decision curve analysis based on the external validation cohort were demonstrated in Figure S3.The nomogram model predicting poor functional outcome showed an AUC of 0.795 (95% CI: 0.716-0.873).
A similar result was identified in the nomogram model to predict all-cause death, with an AUC of 0.811 (95% CI: 0.707-0.915).The calibration plots of the two nomogram models also presented an acceptable agreement between the outcome prediction and actual observation.In terms of decision curve analysis, nomogram models to predict poor functional outcome or all-cause death all provided good clinical utility.

| DISCUSS ION
In the present study, we developed and validated two novel nomogram models for predicting poor functional outcome or all-cause death at 3 months in patients with aSAH.Both two models demonstrated favorable performance and predictive accuracy in derivation and externally validation cohorts.(Continues) model to predict 3-month poor functional outcome.9][20] This could be explained by the fact that the elderly is more likely to have poor vascular conditions, 21 less ability to recover from the initial bleeding and prone to develop delayed cerebral ischemia, and other complications. 8,21Apart from the advanced age, neurological grade at admission such as Hunt-Hess grade, is usually applied in clinical settings to evaluate the severity and prognosis of aSAH patients. 22The Hunt-Hess grade categorizes the neurological status into five levels according to patients' symptoms and level of consciousness. 23though this grading system has showed superior performance for predicting poor functional outcome, it is not reliable enough due to its subjective nature of each parameter in this system, especially applying in intubated patients. 24Moreover, Hunt-Hess grade performs poorly at demarcating between adjacent grades due to the ambiguous defined scales of consciousness, 25 which makes it difficult to apply for further stratified management and targeted treatment.Hence, it's crucial to consider other relatively objective parameters that can better reflect the pathogenesis of aSAH for further predictions and treatment.Recently, the role of early brain injury and its associated peripheral-blood-based biomarkers to predict prognosis after aSAH is worthy of recognition. 26,27Higher levels of CRP could be a good reflection of acute neuroinflammation response after aSAH occurs, subsequently resulting in early brain injury and unfavorable functional outcomes. 28,29The mechanisms of this process might include blood-brain barrier disruption, neuronal death, and white matter injury. 30,31Also, bleeding in the subarachnoid space initiates platelet activation and aggregation, 32 which further triggers early brain injury owing to microthrombosis, cerebral tissue ischemia, and death. 33,34Therefore, elevated platelet counts associated with poor functional outcome adds to the evidence that thrombosis formation might participate in the mechanism of early brain injury. 35In addition, bilirubin is a hemoglobin degradation product after aSAH and also takes part in early brain injury. 36Blood clots in the subarachnoid space induce free radical formation, which can subsequently act on bilirubin and produce bilirubin oxidation products. 36These toxic products ultimately damage vascular smooth muscle cells and affect the blood flow of the brain tissue, 37 aggravating the brain injury.

| The Nomogram model to predict poor functional outcomes
On the contrary, lymphocytes play an important role in antiinflammatory response and immune surveillance.TA B L E 1 (Continued)

| The Nomogram model to predict all-cause death
Variables associated with all-cause death at 3 months after aSAH onset are partially overlapped with those associated with disability, such as lymphocyte and CRP, which explains the shared pathogenesis of death and disability after aSAH.Additional parameters important for death prediction include neutrophil, AST levels and treatment methods.Aneurysmal bleeding promotes neuroinflammatory responses, which is characterized by elevated neutrophil counts. 30,38creased levels of neutrophils take crucial part in early brain injury and poor prognosis through a variety of mechanisms, including activation of reactive oxygen species, disruption of endothelial cell conjunctions, and increased blood brain barrier permeability. 41,42I G U R E 1 Selection of potential factors to predict 3-month poor functional outcome (A, B) or all-cause death (C, D) by LASSO regression analysis.(A, C) Five-fold cross-validation and minimum error criteria were applied to identify the optimal penalization coefficient.The left vertical line represents the optimal λ values when applying the minimum error criteria, while the right vertical line represents the optimal λ values when applying the one standard error of the minimum criteria.(B, D) LASSO coefficient profile plots against the log (λ).Optimal λ based on the minimum error criteria was used to select variables with nonzero coefficients.

TA B L E 2
Multivariable logistic regression analysis for 3-month poor functional outcome.
In this study, we also found that elevated AST levels were one of the major risk factors for 3-month death in patients with aSAH.The underlying mechanisms of this association remain unclear and still need future studies to uncover.We speculate that increased AST levels may indicate concurrent subclinical liver disease, 43 which may be closely related to vascular inflammation, coagulation dysfunction, and impaired endothelial function. 44en it comes to treatment methods, our study has demonstrated that endovascular coiling was associated with patient survival.As it known to all, surgical treatment can significantly improve the prognosis of patients with aSAH compared with conservative treatment.However, how to choose between endovascular coiling and surgical clipping remains controversial.
The International Subarachnoid Aneurysm Trial (ISAT) compared the safety and effectiveness of endovascular coiling and surgical clipping in 2143 patients with ruptured aneurysms. 45Results have shown that endovascular coiling was more likely to be associated with independent survival at 1 year and 10 years compared with surgical clipping. 46However, the risk of late rebleeding was higher after endovascular coiling than after surgical clipping. 47ncerning the inclusion criteria of ISAT trials that requires patients suitable for both surgical treatments, patients with postcirculation aneurysms or larger wide-necked aneurysms may have been excluded from the ISAT cohorts.Therefore, other study still tried to explore the benefits of different surgical methods.A metaanalysis by Li et al. showed that there was no significant difference in terms of 1-year mortality between the two surgical methods. 48fferent results between literatures may be primarily explained by different characteristics of study populations (e.g., the severity of disease, the location of aneurysms).In ISAT trials, about 88% of patients were low-grade aSAH patients (WFNS grade 1-2), and 95% of aneurysms located in the anterior circulation. 45Similarly, in our study, low-grade aSAH patients accounted for 89%, and nearly 90% of the patients had aneurysms located in the anterior TA B L E 3 Multivariable logistic regression analysis for 3-month all-cause death.

F I G U R E 2
Nomogram models for predicting 3-month poor functional outcome (A) or all-cause death (B).AST, aspartate aminotransferase; CRP, C-reactive protein; DBIL, direct bilirubin.
circulation.However, in the meta-analysis mentioned above, 48 the percentage of high-grade aSAH patients and aneurysms located in the posterior circulation were higher than our study and ISAT trials, so the advantage of endovascular coiling for improving survival may be diminished.All in all, endovascular coiling may be a good choice not only for patients who are suitable for both the two surgical methods but also for patients with low preoperative grading scales and aneurysms located in the anterior circulation.

| Comparisons between the SAHIT model
Recently, Jaja et al. developed the SAHIT prediction models for poor functional outcome or death after 3 months in patients with aSAH. 4 The models were constructed based on a large population gathered from several clinical trials and have been validated rigorously. 49Thus, the SAHIT models are the most widely accepted model to predict outcomes after aSAH.However, the predictors just explained 23%-31% of the variability in outcomes, 4 implying other factors may take part in the outcome prediction and they were not fully captured from the SAHIT models.Furthermore, the predictor hypertension in the SAHIT models were not welldefined 3 and WFNS grade incorporated in the SAHIT models performed poorly in patients with adjacent grade scales. 25On the contrary, our nomogram models take other objective blood-based biomarkers that could reflect the pathogenesis of aSAH into account, which yielded a better performance compared with the core SAHIT models.Another weakness of the SAHIT models is that it developed on trial data.Trials may select specified study population, which can differ from the general population in real practice. 3,9And randomization to the studied intervention could alter the expectations of clinicians and patients, which may affect the outcomes eventually. 9In contrast, our current models were based on patient data derived from real-world practice in a modern-day cohort, showing an advantage compared with the core SAHIT models.In addition, the SAHIT models can only be utilized through an online calculator, making it difficult to be widely used to some extent.In the current study, nomogram plots were applied to present our models, which is easy to use in various clinical settings.

| Strengths and limitations
There are several strengths in the current study.First, the identified parameters not only can be readily measured and obtained from routine examinations but also convenient to use in acute aSAH phase without the need for much experience to evaluate imaging scales and consider the subsequent complications during treatment course.Second, the LASSO regression applied in our study has advantages to lower data dimensionality and reduce multicollinearity between variables, thus, it is performed well in selecting the potential significant parameters.Moreover, nomogram excels in simplicity over traditional analysis utilizing odd ratios and can help to better visualize the model to generate an individual risk of poor outcomes, making it beneficial for physicians to apply in clinical practice.
However, there are also some limitations in our study to be recognized.First, our study was conducted in a single tertiary center with a relatively small sample size, which might introduce selection bias.A multicenter and large study is required to validate our findings in the future.Second, we only evaluated outcomes at 3 months

| Clinical implications
The nomogram prediction models in the present study could assist in providing patient-centered information of poor prognosis and help clinicians to communicate effectively with patients and their relatives.More importantly, using the readily obtained variables on admission, our prediction models help meet the demand for identifying and screening patients at high risk of poor outcomes quickly at an early stage of aSAH.Therefore, clinicians can pay more attention to these high-risk patients and promptly conduct personalized therapy to prevent further deterioration and significantly improve prognosis of these patients.Furthermore, another potential clinical significance of our study is that the nomogram models contain predictors that can reflect the process of early brain injury.Hence, it provides meaningful insight into future studies to explore treatments that target neuroinflammatory response, microthrombosis formation, toxic effects of degradation products and so on.

| CON CLUS ION
In the present study, nomogram models constructed for predicting 3-month poor functional outcome or death after aSAH demonstrated good discrimination, accuracy, and clinical usefulness.The predictor items in these two nomogram models are not only easy to obtain upon admission but also can reflect the pathogenesis of aSAH to a certain extent.Therefore, these two models can help physicians to identify patients at risk, guide decision-making at an early stage, and provide new directions for future studies to explore the novel treatment targets.
patients admitted to the department of neurology emergency in Beijing Tiantan Hospital.The derivation cohort comprised eligible aSAH patients admitted between October 2020 and September 2021, while the external validation cohort was composed of aSAH patients enrolled from October 2021 to March 2022.The study was performed in compliance with the ethical guidelines from the Helsinki Declaration and was approved by the Institutional Review Board of Beijing Tiantan Hospital.Written informed consent was obtained from all the participants or their legal proxies.The inclusion criteria were (1) age of 18 years or older, (2) spontaneous SAH diagnosed via head CT scan, (3) presence of intracranial aneurysm confirmed by CT angiopathy (CTA), or digital subtraction angiopathy (DSA).The exclusion criteria encompassed: (1) SAH caused by other reasons, such as trauma, cerebral arteriovenous malformations, moyamoya disease, and intracranial tumors, (2) previous history of cerebral infarction, intracranial hemorrhage, vascular anomalies, and malformations, (3) acute kidney injury or chronic kidney disease, (4) concurrent systemic comorbidities including malignancy and liver cirrhosis, (5) premorbid modified Rankin Scale (mRS) >1.

F I G U R E 3
ROC curves of the two newly developed nomogram models and SAHIT models for predicting 3-month poor functional outcome (A) or all-cause death (B).ROC: receiver operating characteristic; AUC: area under the curve; SAHIT: Subarachnoid Hemorrhage International Trialists'.after symptom onset in patients with aSAH, future studies should focus on the long-term prognosis of aSAH patients.Moreover, external validation from a different cohort is still needed to further verify the generalizability of our models.
A total of seven pronounced risk factors, including age, Hunt-Hess grade, heart rate at presentation, lymphocyte, CRP, platelet, and DBIL levels, were finally incorporated into the nomogram Baseline characteristics of the patients in derivation cohort grouped by 3-month functional outcome.