Development of a nomogram for predicting clinical outcome in patients with angiogram‐negative subarachnoid hemorrhage

Abstract To the best of our knowledge, this is the largest clinical retrospective study in AN‐SAH patients, and is the first time to establish accurate predictive models paired with bleeding pattern. Background Angiogram‐negative subarachnoid hemorrhage (AN‐SAH) has a definite incidence of delayed cerebral ischemia (DCI) and poor clinical outcomes. The purpose is to screen independent factors and establish a nomogram to guide the clinical therapy and assess post‐discharge prognosis. Methods We identified 273 consecutive patients referred to our institute from 2013 to 2018 for AN‐SAH. A nomogram to predict poor outcomes was formulated based on the multivariable models of independent risk factors. The accuracy and discrimination of nomograms were determined in training and internal validation cohorts. Results The overall poor outcome rates of AN‐SAH were 14.3% and 8.7% at 3 months and 12 months, respectively. In addition, perimesencephalic AN‐SAH (PAN‐SAH) presented with a more unfavorable prognosis compared with non‐perimesencephalic AN‐SAH (NPAN‐SAH). The clinical prognosis was associated with the World Federation of Neurosurgical Societies scale (WFNS) (odds ratio, 3.82 [95% CI, 1.15‐12.67] for 3‐month outcome; and odds ratio, 31.69 [95% CI, 3.65‐275.43] for 12‐month outcome), Subarachnoid hemorrhage Early Brain Edema Score (SEBES) (odds ratio, 10.39 [95% CI, 1.98‐54.64] for 3‐month outcome; odds ratio, 10.01 [95% CI, 1.87‐53.73] for 12‐month outcome), and symptomatic vasospasm (odds ratio, 3.16 [95% CI, 1.03‐9.70] for 3‐month outcome; odds ratio, 5.15 [95% CI, 1.34‐19.85] for 12‐month outcome). The nomogram was constructed based on the above features, which represented great predictive value in clinical outcomes. Conclusions Symptomatic vasospasm, high WFNS, cerebral edema, and NPAN‐SAH after hemorrhage were associated with poor outcome of AN‐SAH. The nomogram with WFNS (3‐5), SEBES (3‐4), vasospasm, and NPAN‐SAH represented a practical approach to provide individualized risk assessment for AN‐SAH patients.


| INTRODUC TI ON
Subarachnoid hemorrhage (SAH) is an uncommon and severe subtype of stroke, accounting for 5% of the patients with stroke. The rupture of an intracranial aneurysm is the underlying cause in the majority of cases. 1 However, in approximately 15% of SAH cases, vascular abnormalities are not demonstrated through vascular imaging or digital subtraction angiogram (DSA), a condition referred to as AN-SAH. 2 AN-SAH is a unique variant of SAH, for which no structural aberration can be found as a primary cause for spontaneous hemorrhage. 2,3 AN-SAH can be categorized as PAN-SAH or NPAN-SAH according to the bleeding patterns. 4 Half of the patients with AN-SAH were presented with the perimesencephalic type, which is known to present better clinical outcomes and lower rates of vasospasm than those observed with aneurysmal SAH. 5 However, AN-SAH still accompanied definite incidence of DCI and poor clinical outcomes, especially in patients with the non-perimesencephalic bleeding pattern. 6 So far, more than 40 clinical grading scores have been proposed to assess the clinical severity after SAH, including the Hunt-Hess (HH), Subarachnoid hemorrhage Early Brain Edema Score (SEBES), Glasgow Coma Score (GCS), the World Federation of Neurosurgical Societies scale (WFNS), and the modified Fisher Scale (mFS). [7][8][9][10][11][12][13] These prognostic grading scores were not initially designed for AN-SAH patients, but were also commonly used in the clinical assessment and prognostic prediction of AN-SAH patients due to the lack of standardized evaluation indicators. In our previous study, we compared the prognostic value of several clinical scores in AN-SAH patients, and demonstrated their favorable adaptation to this type. 5 However, we did not distinguish the difference between PAN-SAH and NPAN-SAH, or did we evaluate other significant factors associated with outcomes. Until now, the effectiveness of clinical, radiological, or combined scores had rarely been analyzed and validated in AN-SAH patients. Furthermore, the small sample size of included patients and the lack of further assessment of bias due to individual interpretation of AN-SAH may skew the results. 6 An optimal selection of grading scales may assist clinicians in making informed decisions, and allow for improved care and effective treatment in AN-SAH patients. Herein, the aim of this study is to screen out independent associated factors, emphasize optimal grading scores, and establish a nomogram to guide the clinical therapy and assess post-discharge prognosis.

| Study population and follow-up
A retrospective study was conducted to identify all consecutive patients who were referred to our institute and diagnosed with AN-SAH between January 2013 and December 2018. This study included past clinical records and imaging data of 1,219 consecutive patients with spontaneous SAH. The demographic, clinical, and imaging data were reviewed. The inclusion criteria utilized are listed as follows: (1) patients were screened by computed tomography angiography (CTA) at admission, followed by emergent DSA within 72 hours, and diagnosed with AN-SAH; and (2) patients with a follow-up period > 12 months after discharge. Exclusion criteria encompassed: (1) patients with a history of trauma or previous brain injury; (2) patients who had serious comorbidities prior to SAH onset; and (3) patients with unavailable radiological images or missing follow-up data. A total of 35 patients were excluded, leaving 273 patients eligible for final analysis ( Figure S1). The Institutional Review Board approved all aspects of this study and waived informed consent.

| Patient management and grading
All patients were managed according to the Neurocritical Care Society and American Heart Association SAH guidelines. 14 Symptomatic vasospasm and delayed cerebral infarction were defined as delayed cerebral ischemia (DCI). [18][19][20] Clinical prognosis was assessed using the modified Rankin Scale (mRS) at 3 months and 12 months following discharge, which was recorded by outpatient records and telephone interview. According to the mRS, the prognosis was categorized as either poor outcome (mRS 2-6) or favorable outcome (mRS 0-1).

| Nomogram construction and validation
The patients were randomly divided into a training cohort and a validation cohort using a random number list generated by SPSS. In

K E Y W O R D S
angiogram-negative subarachnoid hemorrhage, nomograms, outcome research, prognosis our study, we selected one half of the patients as the training group, and the remaining patients as the validation group, so that the clinical characteristics except the four independent factors showed no significance between these two sets, and aimed to emphasize predictive value of nomogram model with more accuracy. We analyzed and compared the general clinicopathological data of the two cohorts. The factors with P value less than 0.05 in multivariated analysis were selected as predictors in nomogram. In addition, due to the significant difference of clinical progression and prognosis between PAN-SAH and NPAN-SAH, the factor of NPAN-SAH was also considered as a label in the nomogram. Based on these factors, the R software version 2.13.2 (https://www.r-proje ct.org) generated nomogram models to predict post-discharge outcomes for patients at 3 and 12 months. The performance evaluation of the nomogram includes discrimination, calibration curves, and ROC curve, which were verified by the calibration curve generated by the validation cohort.

| Statistical methods
The continuous variables were reported as Mean±SD. For the clinical and neuroradiological data, patients with favorable or unfavorable outcomes were compared using descriptive statistics, Student's t-test for continuous variables, and χ 2 or fisher exact test for categorical variables.
Both univariate and multivariable logistic regression were performed to estimate the risk of disability (mRS score ≥2 at treatmentfree follow-up). We used a multivariable model with a forward stepwise regression procedure to screen out the potential predictors for poor outcome. The performance and accuracy of the nomogram was evaluated with the calibration and ROC curves to measure internal calibration and discriminative ability.
All analyses and graphical representations were performed with IBM-SPSS V24.0 (SPSS Inc., Armonk, NY), Prism 8 (GraphPad Software, Inc.), and R Version 3.6.3 GUI 1.70 (The R Foundation for Statistical Computing). Statistical significance was set at P < 0.05.

| Demographics and clinical characteristics of the study population
The demographic features and clinicopathological characteristics of patients with PAN-SAH and NPAN-SAH are presented in women and 54 men, with a mean age of 57.4 ± 11.8 years. As depicted in Table 1, the demographic data showed no statistical significance between the PAN-SAH and NPAN-SAH groups, including BMI. However, hospital complications, including symptomatic vasospasm, delayed cerebral infarction, rebleeding, and encephaledema, occurred more frequently in the NPAN-SAH type (all P < 0.05), with the exception of seizure (P = 0.249). In addition, greater SAH severity (characterized by clinical data and radiological images) and poorer outcomes (assessed due to mRS) were observed in patients with NPAN-SAH (P < 0.05).
Of all the AN-SAH patients, the majority of patients (81.3%) with mRS greater than 2 points were observed at discharge, of which poor outcomes were improved after 3 and 12 months with 14.3% and 8.5%, respectively ( Figure 1A). In addition, there were 5 incidents of death, accounting for 1.8% of all cases. The distribution of patient outcome in PAN-SAH demonstrated more favorable outcomes, with 7.1% and 2.1% at 3 months and 12 months, respectively.
However, the proportion of poor outcome are 29.2% and 21.3% in the patients with NPAN-SAH, respectively. It should be noted that these five cases of death were all identified as NPAN-SAH, which accounted for 5.6% of the total NPAN-SAH patients ( Figure 1B and C).

| Factors associated with poor outcomes in PAN-SAH and NPAN-SAH
According to the significant difference found in outcomes between the two bleeding patterns, we screened the factors associated with favorable outcomes in PAN-SAH and NPAN-SAH. Regarding SAH severity, disturbance of consciousness (GCS<15), high WFNS, high HH, and high SEBES all correlated with unfavorable outcomes at 3 months in the NPAN-SAH type (all P < 0.0001), as well as the occurrence of complications, including symptomatic vasospasm, delayed cerebral infarction, rebleeding, and encephaledema (all P < 0.0001). However, only WFNS (P < 0.0001), symptomatic vasospasm (P = 0.004), and delayed cerebral infarction (P = 0.002) are related to the 3-month outcome in PAN-SAH patients. Besides, similar results were observed in 12-month outcomes. As depicted in Table 1, except for the above factors, the occurrence of seizure (P = 0.044) is also related to 12-month outcome of patients with NPAN-SAH, while disturbance of consciousness (P = 0.013) is also related to 12-month outcome in PAN-SAH.    Although the clinical presentation and prognosis present significant difference between two bleeding patterns, the NPN-SAH shows no such statistical significance in multivariate analysis, which may be as a result of the limited data size. In some cases, factors with important definition or clinical utility are also included, although the P value is more than 0.05, so that the nomogram may be a more reasonably available model. [21][22][23] Thus, eventually, the high WFNS, high SEBES, and symptomatic vasospasm were determined as predictors. And the factor of NPAN-SAH was included as a label, so that the nomogram can be reasonably applied in both types of AN-SAH and be more clinically practical.

| Multicollinearity test, nomograms, and validation of predictive accuracy for poor outcome
To establish a predictive model for clinical prognosis of AN-SAH, we first tested the potential collinearity between variants within two models in the overall analysis. Variance inflation factor (VIF) < 3.5, tolerance > 0.3, minimum eigenvalue > 0.16, and condition index (CI) < 5 revealed no statistical significance in collinearity between variants in both of the present models.
Then, the patients were randomly divided into training cohorts and validation cohorts. There were no significant differences between the two groups of patients in demographic information and clinical characteristics (Table S2). Based on the multivariable regression analysis, we used nomograms to show the risk stratification for 3-month and 12-month disability with factors of WFNS, SEBES, and  Then, draw a vertical line from the feature status towards the points axis to obtain respective points based on each feature. Finally, draw a vertical line through the total points axis, according to the sum of the total score, which will intersect the probability of poor outcomes axis at the predicted probability. (B) Calibration curve for training cohort (left) and validation cohort (right). The grey line represents performance of ideal nomogram where the predicted probability perfectly corresponds to observed probability. Nomogram of 12-month outcome (C) and its calibration curve (D) symptomatic vasospasm of the training cohort (Figure 2A and B). For an individual patient, risk factors contributed to respective points and the sum corresponded to the probability of 3-month and 12month disability. The calibration curves suggested that the nomogram was relatively well calibrated and corresponded to prediction and observation. In the validation cohort, the calibration curve also showed minor discrepancies between observed and predicted probabilities ( Figure 2C and D). The results showed that the AUC of the 3-month nomogram of the training and validation cohorts are 0.752 and 0.975, respectively, while the AUC of the 12-month nomogram are 0.931 and 0.986, respectively ( Figure S2).

| DISCUSS ION
SAH is caused by ruptured intracranial aneurysms. Despite advanced therapy and critical care management, SAH is still companied with significant morbidity, mortality, and socioeconomic impact. 24 This is not only caused by the hemorrhagic event, but further associated with the sequelae of the irritating blood by-products, which can result in cerebral vasospasm and delayed cerebral ischemia (DCI).
DCI has been reported to correlate with prognosis and neurological function after SAH. 25 Several studies even applied micro-CT, DSA to longitudinally monitor vasospasm and tried to reveal the progression after SAH. 26 Besides, early brain injury (EBI) encompasses the entire brain injury occurring within 72 h after SAH, which is related to oxidative stress, neuroinflammation, neuronal death, blood-brain barrier (BBB) damage, autophagy-apoptosis, and ferroptosis. [27][28][29][30] Although timely and effective treatment is implemented, the SAHinduced neurological impairments are permanent in individual. Thus, it is important to monitor and prevent the occurrence of DCI.
To the best of our knowledge, this is the largest clinical retrospec- An updated meta-analysis, performed in 2019, suggested that differences in diagnostic strategy for identification of future vasospasm events and clinical status at admission may limit the skew of interpretation, which emphasized further analysis based on individual patient data to elucidate the clinical outcomes according to the hemorrhagic pattern in patients with AN-SAH. 6 In our present study, we demonstrated that symptomatic vasospasm is the more effective independent predictor than delayed cerebral infarction and other clinical complications.
Consequently, patients with AN-SAH require prompt augmentation of blood pressure or endovascular intervention if DCI occurs, and they should be monitored for vasospasm in the intensive care unit. 32 In order to accelerate the turnover rate of patients, patients with AN-SAH are often discharged for rehabilitation as soon as possible.
Thus, the prognosis of patients at 3 months and 12 months varies widely, which emphasized the necessity of prognostic prediction in AN-SAH patients. The present study focused on the prognosis of AN-SAH patients from a larger sample size and presented a novel attempt to provide quantitative risk assessment of the poor outcome.
We observed that the overall disability rate is 14.3% and 8.6% after recovery periods of 3 months and 12 months, respectively.
Current radiographic rating scales, the most widely used being the mFS grading system, are mainly focused on quantifying the amount of blood for the purpose of predicting vasospasm and DCI. 10  There are several limitations in our study worth mentioning.
First, the nomogram was established using retrospective data obtained from a single institution. Second, the outcome data of patients were recorded from outpatient and telephone interview at 3 and 12 months, for which there was referral bias as some patients were unacceptable due to unfavorable neurological status. Thus, a potential underestimation of poor prognosis might have influenced the overall estimate. Additionally, because of the relatively benign progression and limited cases, there is potential for minor bias to skew the interpretation. Although more data from other institutions are required for further validation of our nomograms, the individualized quantitative risk assessment using the present nomograms would be a more practical approach for predicting prognosis and counseling patients.

| CON CLUS ION
Symptomatic vasospasm, WFNS, SEBES, and bleeding pattern are independent predictors for 3-month and 12-month prognosis in patients with AN-SAH. The nomograms could provide individualized prediction of post-discharge outcomes of AN-SAH patients, and represent a practical approach to disability assessment.

ACK N OWLED G M ENTS
We thank the nursing staff of the Department of Neurosurgery at the Second Affiliated Hospital of Zhejiang University. This study adhered to the TRIPOD checklist.

CO N FLI C T O F I NTE R E S T
None.

DATA AVA I L A B I L I T Y S TAT E M E N T
The supplementary material for this article can be found online. All processed data and R codes used in this study can be obtained from the corresponding author on reasonable request.