Generalizability of nociception level as a measure of intraoperative nociceptive stimulation: A retrospective analysis

Nociception‐guided intraoperative opioid administration might help reduce postoperative pain. A commonly used and validated nociception monitor system is nociception level (NOL), which provides the nociception index, ranging from 0 to 100, with 0 representing no nociception and 100 representing extreme nociception. We tested the hypothesis that NOL responses are similar in men and women given remifentanil and fentanyl, across various types of anesthesia, as a function of American Society of Anesthesiologists physical status designations, and over a range of ages and body morphologies.

across a number of characteristics such as age, sex, body mass index, and type of anesthesia.Not all events could, however, be classified.Readers can note competing interests for some coauthors.

| INTRODUCTION
Anesthesia allows patients to tolerate surgical procedures that would otherwise inflict unbearable pain, provoke extreme physiologic exacerbations, and result in unpleasant memories.Nociception is defined as the neural process of detection, transduction, and transmission of noxious stimuli. 1 There is a general consensus that noxious stimuli should be managed during general anesthesia to prevent excessive sympathetic nervous system activation.Excessive noxious stimulation may also activate pain pathways that might augment postoperative pain and potentiate intraoperative recall. 2 The challenge, of course, is that general anesthesia precludes facile evaluation of stimuli that would normally provoke pain, which by definition is a conscious response.4][5][6] A commonly used and well validated system is the PMD-200 monitor with Nociception Level (NOL) technology (Medasense Biometrics Ltd.Ramat Gan, Israel).The NOL index is based on a combination of nociception related physiological variables including heart rate, heart rate variability, photo-plethysmographic waveforms, skin conductance, and their time derivative. 7[9] The NOL index responds consistently to various levels of noxious stimulation [9][10][11] with higher sensitivity and specificity than other nociceptive measures. 8,10A reasonable question is whether NOL responses are consistent across various patient and anesthetic characteristics; that is, the extent to which NOL responses are generalizable.
To address this pertinent question, we re-evaluated eight studies in which NOL responses to non-noxious and various noxious stimuli were prospectively recorded.Specifically, we tested the hypothesis that NOL responses are similar in men and women, in patients given remifentanil and fentanyl, across various types of anesthesia, as a function of American Society of Anesthesiologists (ASA) physical status designations, and over a range of ages and body morphologies.

| METHODS
As of January 2021 when our analysis began, there were 12 studies of NOL monitoring with a total of 592 patients that were published or known to Medasense, of which 8 were included reflecting 522 patients or 88% of patients known to date.Three studies were excluded due to lack of data sharing agreement; one was excluded because it did not annotate surgical stimulation.
We analyzed NOL recordings from adults who had elective surgical procedures (general, gynecological, and urological) with permission of the investigators of eight prospective studies conducted between August 2016 and December 2020 (Table 1, Supplemental Table 1).All eight studies were individually approved by the relevant Institutional Review Boards, and all patients enrolled signed written informed consents (Supplemental Table 2).3][14][15] We considered all 522 patients who met enrollment criteria for the underlying studies.
As suggested in the manufacturer's user manual, we used a NOL threshold of 25 (on a 0-100 scale) to define positive responses to noxious or non-noxious stimulation.Specifically, we expected NOL values less than 25 in response to defined periods of non-noxious stimulation, and NOL values exceeding 25 in response to defined noxious stimulation.We considered five periods during surgery: a non-noxious period (e.g., before skin incision), minor stimuli (e.g., urethral catheterization), moderate stimuli (insufflation, small incision), moderate-to-severe stimuli (e.g., first incision, trocar insertion), and severe stimulation (laryngoscopy and intubation).The timepoints were analyzed per patient to quantify the intraoperative response to noxious and non-noxious stimuli during the surgical procedure.The post-stimulus reaction was defined as the average of the non-continuous 30 seconds with the highest NOL values within a 3-min window post stimulus.
Analgesic medications obviously blunt responses to nociceptive stimuli and should therefore reduce the NOL response to stimulation.
The definition of an appropriate characterization of stimuli must therefore consider stimulation intensity in context of analgesic level.
We used the Combined Index of Stimulus and Analgesia (CISA) score to grade the combination of the stimulus level and the analgesia level. 7The CISA score is a linear combination of the stimulus intensity and the effect of analgesic drugs.The stimulus intensity level was defined by experts as a discrete ordinal number between 0 and 10 that represents the intensity of the surgical event.The effect of analgesic drugs was defined according to effect-site concentrations of opioids (scaled by a normalization factor).The effect of the site concentration of opioids was continuously calculated based on annotated infusion rates, boluses, and the pharmacokinetics models of Minto et al 16 and Schafer et al 17 as appropriate for the opioid used in the study.The type and timing of stimuli, and administration of opioids were continuously annotated during surgeries and processed following the surgeries to compute the CISA score.The CISA score set the "ground truth" of expected nociceptive level for our analysis and was performed offline.NOL value, in contrast, represent instantaneous output of the measurement system.
We defined nociceptive events as having a CISA score exceeding 4.9 on the 0-10 scale (the threshold above which we consider CISA as definitely nociceptive): This is equivalent to a severe stimulus with analgesia effect-site concentration up to 3.3 ng/mL.Non-nociceptive events were defined by CISA scores <2.5 (the threshold below which we consider CISA as definitely non-nociceptive): This is equivalent to non-noxious stimulus, minor stimulus with an opioid concentration of at least 1.5 ng/mL, or moderate stimulus with opioid concentration of at least 4.5 ng/mL.

| Statistical analysis
Differences between noxious and non-noxious stimuli were assessed with box plots, histograms, and receiver operating characteristics T A B L E 3 Median and mean NOL value across subgroups in response of noxious versus non-noxious stimuli.
(ROC) curves of the NOL response based on NOL >25 for noxious stimuli and ≤25 for non-noxious and opioid-blunted stimuli.The whiskers for the boxplots were set at ±2.7 standard deviations representing 99.3% coverage for normal distribution, 95% CI for the means were estimated from the means and standard errors (SEs), using the formula: lower limit = mean -(z 1Àα Â SE) and the upper limit = mean + (z 1Àα Â SE).α is defined as 0.05 and z 1Àα is 1.96.The SE for the ROC was calculated using the method presented by DeLong et al. 18 Sensitivity and specificity were calculated using the Clopper-Pearson formula. 19e test significance level ( p-value) was calculated using AUC = 0.5 as the null hypothesis: ERFC is the complementary error function and SE is the standard error.
We performed an analysis of the non-noxious events in which the threshold of 25 was crossed and of the noxious events in which the NOL value was less than 25 to estimate the false positive and false negative rates.At least 30 noxious stimuli and 30 non-noxious stimuli were needed to achieve the target sensitivity and specificity of 0.75 for the lower CI, based on MedCalc software (version 19.8, Med-Calc Software Ltd., Ostend, Belgium).Since the actual numbers of events exceeded 30 by a factor of 3-16 our analysis was well powered.
Data were processed using MATLAB version R2018a (The MathWorks Inc.) and analyzed using MedCalc Statistical Software.

| RESULTS
A total of 522 eligible patients from 8 studies were considered and 447 were included in our analysis (Table 1).A total of 1271 events were collected from the NOL monitors.A total of 595 events were excluded because CISA scores were between 2.5 and 4.9, making it unclear whether stimuli should be considered noxious or non-noxious.
T A B L E 4 Sensitivity and specificity of the NOL index across the groups.There were distinct differences in the NOL response to noxious and non-noxious stimuli.The average NOL in response to 315 noxious stimuli was 47 ± 15 (95% CI = 45-49).The average NOL in response to 361 non-noxious stimuli was 10 ± 12 (95% CI = 9-11).Thus, NOL values during non-noxious stimuli were largely below 25, whereas noxious stimuli provoked NOL values exceeded 25 (Figure 1).
Nociception level values were similar across various populations, with no clinically meaningful or statistically significant differences across age, BMI, ASA physical status, analgesia, or anesthetic group for either noxious or non-noxious stimuli (Table 3).The area under the curve across the populations exceeded 0.9 in all groups, thus easily meeting our predefined target criteria of 0.75 (Table 4).
Only 5% of non-noxious stimuli provoked a NOL value more than 25, and 94% of these events lasted less than 1 min long which would not generally be considered a clinically meaningful event.The remaining false positive events represented just 0.3% of the events included in our analysis.For noxious events, the false negative rate (NOL <25 in the presence of a noxious stimulus) was under 8%.

| DISCUSSION
Previous studies have shown that NOL reliably distinguishes nonnociceptive and nociceptive events. 7,10,20,21Sensitivity and specificity are both high with a cutoff of 25 which is what we used in this analysis. 22Our summary results from eight prospective studies are consistent, showing that NOL was nearly always <25 without noxious stimulation or when stimulation was blunted by opioids.In contrast, NOL nearly always exceeded 25 shortly after noxious stimuli.
Specifically, NOL responded to more than 300 noxious stimuli with an average NOL score of 47 ± 15, well above the manufacturer's threshold of 25 for identifying noxious versus non-noxious stimuli.
Only about 8% of noxious stimulation events failed to provoke NOL scores exceeding 25.NOL also responded to 5% of non-noxious stim- A limitation of our analysis is that we were restricted to eight NOL studies where Medasense had access to data.For various reasons, we were unable to obtain necessary raw data from three other studies, but those studies represented only 12% of available data.Furthermore, 75 patients from the eight included studies were excluded from analysis because of poor data quality.Nonetheless, our analysis is based on nearly 450 patients and presumably reasonably characterizes noncardiac adult surgical patients.NOL performed well over the range of characteristics and anesthetic approaches we evaluated, but of course there are many other factors that we did not specifically evaluate.
Another possible limitation of this study may be related to inaccurate annotations that remained after data review.The data review process included both manual and automated data checks, and we are therefore confident, that the probability is low.Variability among anesthesia providers and study protocols may further add another source of bias.By nature, surgical stimuli varied in duration and intensity, and responses to nociceptive stimuli in NOL guided patients may have varied.
In summary, we evaluated NOL over six patient demographic and anesthetic approaches and found that nociception was assessed comparably well across all tested groups.NOL thus appears to provide accurate estimates of nociception over a broad range of patients and anesthetic conditions.
). Patients who participated in the eight included studies.
T A B L E 2 Events included in the analysis.
Our results thus indicate that NOL provides accurate estimates of nociception over a broad range of patients and anesthetic conditions.The threshold of 25 thus appears generalizable, which was not previously established.
achieved adequate sensitivity across all sub-groups and reliably responded to noxious stimuli across all pre-specified subgroups: under 65 versus above 65 years of age, intraoperative opioids management using remifentanil versus fentanyl, desflurane versus sevoflurane versus propofol for anesthesia maintenance, patient's physical status 1 versus 2 versus3, normal versus overweight versus obese (BMI), and female versus male.