Both the American Society of Anesthesiologists and the UK NAP4 project recommend that an unspecified pre-operative airway assessment be made. However, the choice of assessment is ultimately at the discretion of the individual anaesthesiologist. We retrieved a cohort of 188 064 cases from the Danish Anaesthesia Database, and investigated the diagnostic accuracy of the anaesthesiologists’ predictions of difficult tracheal intubation and difficult mask ventilation. Of 3391 difficult intubations, 3154 (93%) were unanticipated. When difficult intubation was anticipated, 229 of 929 (25%) had an actual difficult intubation. Likewise, difficult mask ventilation was unanticipated in 808 of 857 (94%) cases, and when anticipated (218 cases), difficult mask ventilation actually occurred in 49 (22%) cases. We present a previously unpublished estimate of the accuracy of anaesthesiologists’ prediction of airway management difficulties in daily routine practice. Prediction of airway difficulties remains a challenging task, and our results underline the importance of being constantly prepared for unexpected difficulties.
Unanticipated difficult mask ventilation and difficult intubation may cause serious complications [1-4]. Accurate prediction of difficult airway management may reduce potential complications by allowing the allocation of experienced personnel and the use of relevant equipment . No single predictor of difficult intubation is sufficiently reliable [6-11], and meta-analyses have found either none, or only sparse evidence, for a pre-operative assessment based on a single risk factor [6, 7, 11].
Difficult mask ventilation is associated with difficult intubation , and a situation with both difficult mask ventilation and difficult intubation is potentially life-threatening. Few studies have examined predictors for, as well as the proportions of, difficult mask ventilation [13, 14]. The American Society of Anesthesiologists (ASA) recommends a pre-operative assessment of the patient's airway, based on eleven anatomical variables [15, 16], but without any elaboration regarding which factors are mandatory for examination, nor on how they should be weighted in an overall airway assessment. The ASA argues that the decision to assess some, or all, risk factors depends on the clinical context. Consequently, it is left to the discretion of the individual anaesthesiologist [15, 16]. Likewise, in the UK, the recently published NAP4 project also recommends an unspecified pre-operative airway assessment .
Just as in the UK and USA, there are no specific national recommendations for pre-operative airway assessment in Denmark. Consequently, in daily routine practice, prediction of airway management difficulties is based on the individual anaesthesiologist's subjective answers to the following questions: (1) Do I anticipate difficult tracheal intubation? (2) Do I anticipate difficult mask ventilation? The answers to these questions may or may not be based on a diverse array of pre-operative airway examinations, depending on the individual anaesthesiologist and/or departmental recommendations.
Pre-operative airway assessment, in some form, is widely practised in Denmark, and it is likely that the subjective predictions are based on the examination of one or several known predictors of difficult airway management . Previously published studies on this topic have focused on the predictive value of a single risk factor, or the value of combining several known risk factors into a multivariable predictive model. This study allows a novel estimate of the diagnostic accuracy of anaesthesiologists’ subjective prediction of airway management difficulties, pragmatically reflecting daily clinical practice.
The Danish Anaesthesia Database (DAD), a national clinical quality assurance database, requires mandatory answers to questions (1) and (2) on anticipation of difficult tracheal intubation and mask ventilation, and is filled in pre-operatively by the anaesthesiologists. In addition, details regarding the patient's actual airway management conditions are registered, enabling comparisons between predictions and actual events. We hypothesised that the anaesthesiologists’ subjective predictions would confirm prior predictive tools and risk factors, showing poor to moderate sensitivity and positive predictive value, and high specificity and negative predictive value .
Using this pragmatic approach, which reflects the heterogeneous nature of everyday clinical practice, we aimed to estimate the accuracy with which subjective anticipation of airway difficulties predicts difficult tracheal intubation, and difficult mask ventilation.
This was an observational cohort study, and the Scientific Ethics Committee of Copenhagen County therefore waived the need for individual patient consent. Data extraction was approved by the Danish Data Protection Agency, and by the steering committee of DAD. In 2011, approximately 75% of all departments of anaesthesia in Denmark recorded data into the DAD. Prospective and consecutive data from all 37 Danish departments of anaesthesia recording in the DAD was extracted between 1 June 2008 and 1 June 2011.
The DAD is a national clinical quality assurance database containing selected quantifiable indicators, covering the anaesthetic process from the pre-operative assessment, through anaesthesia and surgery, until the end of the postoperative recovery period. Anaesthesiologists have to tick Y/N boxes to answer two mandatory questions regarding the anticipation of difficult tracheal intubation and difficult mask ventilation, following pre-operative airway assessment. In addition, a scheduled airway management plan is recorded. Immediately following airway management, an intubation score is registered, based on the actual conditions of the tracheal intubation. An analogue score for mask ventilation is registered for patients on whom mask ventilation was attempted (Fig. 1). The National Board of Health and the Data Protection Agency approved the registration of all patients for anaesthesia.
We included all patients undergoing attempted intubation of their tracheas, and all patients in whom mask ventilation was attempted, in the study. The primary analysis was undertaken on patients who underwent attempted tracheal intubation initially with direct laryngoscopy. We then undertook two sensitivity analyses. The first (sensitivity analysis 1) included patients defined in the primary analysis, plus patients predicted by the anaesthesiologist as having tracheas difficult to intubate, who were scheduled for and underwent attempted tracheal intubation by an advanced intubation technique (e.g. videolaryngoscopy). The second (sensitivity analysis 2) was of all patients undergoing attempted tracheal intubation, regardless of technique. These are expanded upon below. In order to prevent bias, all patients were included with only one entry . Patients having more than one episode of surgery in the recruitment period were included with their last data entry. We did not study children under the age of 15 years.
We investigated the following outcomes and analyses. Primary outcome measure: unanticipated difficult intubations [false negative]/all difficult intubations ([false negative] + [true positive]) = 1 − sensitivity. Secondary outcome measures included: (1) sensitivity = all correctly predicted difficult intubations [true positive]/all difficult intubations ([true positive] + [false negative]); (2) specificity = all correctly predicted easy intubations [true negative]/all easy intubations ([true negative] + [false positive]); (3) positive predictive value = all correctly predicted difficult intubations [true positive]/all intubations predicted difficult ([true positive] + [false positive]); (4) negative predictive value = all correctly predicted easy intubations [true negative]/all intubations predicted easy ([true negative] + [false negative]); (5) positive likelihood ratio = (sensitivity/(1 − specificity)); (6) negative likelihood ratio ((1 − sensitivity)/specificity); (7) odds ratio = ([true positive]/[false positive])/([false negative]/[true negative]).
We had two exploratory outcome measures: (i) the proportion of patients with anticipated difficult intubation scheduled for airway management by direct laryngoscopy; (ii) all-cause 30-day mortality in patients whose tracheas were intubated. Equivalent outcomes were measured for the mask ventilation population, and where appropriate, for the population of patients with both difficult tracheal intubation and difficult mask ventilation.
We used the definition of difficult intubation and difficult mask ventilation predefined in the DAD (Fig. 1). The intubation score was based on the number of intubation attempts, and the use of specialized airway equipment. The mask ventilation score was based on a simplified dichotomous version of Kheterpal and colleagues' definition of difficult mask ventilation [13, 21]. Patients whose tracheas were intubated by a more advanced technique than direct laryngoscopy were, according to the intubation score in DAD, categorised as having an airway that was difficult to intubate, regardless of the reason behind this choice (e.g. educational purposes). In the primary analysis we therefore excluded patients who were scheduled for intubation by advanced techniques before the operation. If a patient's trachea was subsequently intubated using advanced techniques, we assumed that the change of intubation equipment was due to difficulties with direct laryngoscopy, thus representing a difficult intubation (Fig. 2).
In order to avoid the erroneous exclusion of correctly identified difficult tracheal intubations, we conducted a first sensitivity analysis. We assumed that if the anaesthesiologist had predicted a difficult intubation, this was the reason for choosing an advanced technique. Consequently, we identified a group of patients who were predicted to have a difficult intubation, and who were scheduled for and underwent an advanced intubation technique. We assumed that these patients were correctly identified as difficult to intubate, and they were included as true positives, in addition to the patients in the primary analysis.
To explore if the accuracy of the predictions would improve when the severity of the intubation difficulties increased, we performed a second sensitivity analysis. We chose a more rigorous definition of difficult intubation than the predefined definition in DAD, thus defining difficult intubation as an intubation score ≥ 3. We included all patients who underwent attempted tracheal intubation.
As the study was an observational cohort study on a fixed available sample (DAD data 2008–2011), presenting frequencies of events, the number of patient cases entered into the DAD during the study period determined the sample size. The diagnostic accuracy of a subjective prediction of a difficult intubation, difficult mask ventilation, and the combination of both, was measured by: sensitivity; specificity; positive and negative predictive values; positive and negative likelihood ratios; and diagnostic odds ratio with 95% CI [22, 23]. A multivariable logistic regression analysis was used to adjust the odds ratios for potential confounders. The adjusted analysis included the following potential confounders: sex; age; body mass index; ASA physical status; use of neuromuscular blocking agents; and surgical priority (elective/emergency) [8, 9]. For all statistical analysis, we used SPSS v.22.0.0 (IBM Corp., Armonk, NY, USA). The variables used in this study were all mandatory for registration in the DAD, and thus we believe that the dataset was complete, with no missing variables. We reported the study according to the STROBE criteria .
We found a total of 3383 (1.86%) difficult tracheal intubations registered. The number of unanticipated difficult intubations was 3154 (1.73%), and the primary outcome measure, the fraction of difficult intubations that were not predicted pre-operatively, was 93% per cent (Table 1). In 25% of the cases where the anaesthesiologist anticipated difficult tracheal intubation, a difficult intubation actually occurred (the positive predictive value). The positive likelihood ratio was 17.28, reflecting how much the probability of difficult intubation increases if the patient tests positive (difficult intubation is anticipated). The adjusted odds ratio for a difficult intubation, when anticipated, was 17.04.
Table 1. Diagnostic accuracy of the anaesthesiologists’ prediction of difficult intubation for patients undergoing attempted intubation with direct laryngoscopy initially (primary analysis). Values are number (proportion and/or 95% CI)
In sensitivity analysis 1, we categorised a group of patients as true positive cases. As a consequence, the number of correctly identified difficult tracheal intubation increased to 1060, corresponding to 75% of all difficult intubations' being unanticipated. The positive predictive value was 60%, and the positive likelihood ratio was 65.19. The adjusted odds ratio was 85.85 (Table 2).
Table 2. Diagnostic accuracy of the anaesthesiologists’ prediction of difficult intubation for patients with anticipated difficult intubation, scheduled for and performed by an advanced method, categorised as true positives (sensitivity analysis 1). Values are number (proportion and/or 95% CI)
The 1060 patients include those in whom intubation was initially attempted by direct laryngoscopy and those in whom intubation was anticipated to be difficult, and planned for and intubated by an advanced method.
The subtotals vary slightly from those in the primary analysis after including additional patients and checking that each patient was included only once in the analysis.
In sensitivity analysis 2, tightening the definition of difficult intubation, there were 2657 difficult tracheal intubations. There were 2422 unanticipated difficult intubations, equalling 91% of all difficult intubations (Table 3).
The all-cause 30-day mortality for all patients whose tracheas were intubated was 3.1%.
Table 3. Diagnostic accuracy of the anaesthesiologists’ prediction of difficult intubation for all patients undergoing attempted tracheal intubation; difficulty defined as an intubation score ≥ 3. Values are number (proportion and/or 95% CI)
We found 857 (0.66%) cases of difficult mask ventilation (Table 4). The number of cases of unanticipated difficult mask ventilation was 808 (0.62%), equivalent to 94% of all cases of difficult mask ventilation. The positive predictive value was 22%, and the positive likelihood ratio was 43.68. The adjusted odds ratio was 32.91 (Table 4). The all-cause 30-day mortality for mask-ventilated patients was 1.9%.
Table 4. Diagnostic accuracy of the anaesthesiologists’ prediction of difficult mask ventilation. Values are number (proportion and/or 95% CI)
Of 857 patients with difficult mask ventilation, 424 (49.5%) patients were also registered with a difficult tracheal intubation. Thus, difficult mask ventilation was associated with difficult intubation, with an odds ratio of 45.77 (95% CI 39.83–52.61). In 29 of the 424 patients (6.8%), either difficult intubation or difficult mask ventilation was anticipated. In 16 cases (3.8%), both difficult intubation and difficult mask ventilation were correctly anticipated, illustrating that in 89.4% of the patients whose lungs proved difficult to ventilate, no airway management difficulties were predicted (Table 5). Failed tracheal intubation occurred in 3.7% of cases of difficult mask ventilation, compared with 0.1% where mask ventilation was not difficult. Thus, failed intubation was associated with difficult mask ventilation, with an OR of 38.48 (25.98–57.00).
Table 5. Predicted and actual outcomes in airway management. Values are number (95% CI)
Combined intubation and mask ventilation
All intubated and mask ventilated
129 962 (100%)
Difficult intubation and difficult mask ventilation
424 (0.33% (0.30–0.36%))
Failed intubation and difficult mask ventilation
32 (0.02% (0.02–0.03%))
Difficult intubation and difficult mask ventilation
Anticipated one or both of difficult intubation and difficult mask ventilation
45 (10.61% (7.68–13.54%))
Anticipated both difficult intubation and difficult mask ventilation
16 (3.77% (1.96–5.59%))
Failed intubation and difficult mask ventilation
Anticipated one or both of difficult intubation or difficult mask ventilation
6 (18.75% (5.23–32.27%))
Anticipated difficult airway management
Anticipated difficult intubation
Scheduled for intubation by an advanced method
835 (47.52% 45.19–49.86%))
Scheduled for intubation by direct laryngoscopy
850 (48.38% (46.04–50.71%))
Not scheduled for intubation, but subsequently intubated
72 (4.10% (3.17–5.02%))
Anticipated difficult intubation and difficult mask ventilation
Scheduled for intubation by an advanced method
200 (51.68% (46.70–56.66%))
Scheduled for intubation by direct laryngoscopy
162 (41.86% (36.95–46.78%))
Not scheduled for intubation, but subsequently intubated
25 (6.46% (4.01–8.91%))
We identified 1757 patients in whom difficult tracheal intubation was predicted. Of these patients, 47.5% were scheduled for tracheal intubation by an advanced method, and 48.4% were scheduled for intubation by direct laryngoscopy. A total of 4.1% were not scheduled for intubation, but subsequently had their tracheas intubated (Table 5).
This study presents a novel, and previously unpublished, estimate of the diagnostic accuracy of predictions of difficult airway management in daily clinical practice. The high positive likelihood ratio indicates that the anaesthesiologists’ predictions of anticipated difficult airways are a strong diagnostic test. However, the high proportion of unanticipated difficult tracheal intubation, and the low positive predictive values, considerably reduces its value as a reliable diagnostic test in a clinical context.
In the first sensitivity analysis, the diagnostic accuracy of the prediction increased noticeably, as a result of defining potentially true positive cases as such. When these cases were included, the number of difficult tracheal intubations that were not predicted pre-operatively reduced from 93% to approximately 75%. The ‘true’ accuracy of the anaesthesiologists’ predictions of intubation difficulties probably lies somewhere between the predictive values found in the two populations, with a tendency in the primary analysis to underestimate the predictive power, and a tendency in the sensitivity analysis to overestimate it. The predictive accuracy did not improve when we employed a more rigorous definition of difficult intubation, as in the second sensitivity analysis. In situations comprising both difficult intubation and difficult mask ventilation, the predictive accuracy rose. Nevertheless, 89.4% of combined difficult intubation and difficult mask ventilation were unanticipated in the primary population.
Remarkably, almost half of the patients with anticipated tracheal intubation difficulties were still scheduled for direct laryngoscopy. Furthermore, when both difficult intubation and difficult mask ventilation were anticipated, 42.9% of patients were scheduled for airway management by direct laryngoscopy. Our findings further underline the clear association between difficult mask ventilation and difficult intubation reported in previous studies .
The group of patients whose tracheas were intubated had a higher 30-day mortality compared with the group of patients who underwent mask ventilation. Many patients were included in both groups, as they underwent both mask ventilation and intubation. However, this finding illustrates that patients whose tracheas were intubated, and particularly the ones not also mask ventilated (e.g. undergoing rapid sequence induction), were more likely to have a bad outcome, possibly due to more severe underlying co-morbidity that might have influenced their management and outcome.
The proportions of difficult tracheal intubation and difficult mask ventilation were low compared with previous studies [8, 11, 13, 25]. This may partly be due to the fact that there is no international consensus on how to define difficult intubation, and it is often equated with, and described by, the laryngoscopic view classified by Cormack and Lehane .
We assume that anaesthesiologists’ subjective predictions were based on one or several known predictors of difficult intubation. However, the diagnostic accuracy of these predictions were poor, compared with reported studies on stand-alone tests, and with objective risk scores combining several predictors of difficult intubation [6, 11, 27]. Prior studies have been conducted under rigorous trial settings, and some even validated on the same study population, in contrast to our study, which reflects everyday clinical practice. Comparisons should therefore be made with caution.
The database does not contain data on the preparations made before intubation, such as having a more advanced intubation device available, and/or having a specialist in anaesthesiology present. Furthermore, the difficult airway is a continuum, from minor difficulties to the worst possible scenario, where intubation and mask ventilation are impossible. The DAD only allows answers of ‘Yes’ or ‘No’ to the questions ‘Anticipation of difficult intubation’ and ‘Anticipation of difficult mask ventilation’. It would have been preferable to have more differentiated information on the anticipated difficulties, to allow some insight into the airway planning made before induction of anaesthesia. This information gap may, to some extent, explain why we found that direct laryngoscopy was planned for such a high proportion of patients despite anticipation of difficulties.
This study has a number of possible limitations. The trial was conducted on a large cohort, reflecting daily clinical practice throughout Denmark, from a widespread population of surgical patients, and with a broad span of seniority among anaesthetists. The proportions of difficult intubation and difficult mask ventilation were very stable over the three-year period, but the data were not externally validated through other productivity logs or databases, and thus potentially unrecognised registration errors are possible. Most departments monitor the registrations of their patients, and do follow-up registrations on missing patients. However, we cannot rule out that some patients undergoing anaesthesia were never registered in the database, potentially resulting in an unknown number of missing data points.
Registrants in the DAD were unaware that the study was being conducted, and had no direct connection with the investigator group. However, each department providing data to the DAD could get access to pre-calculated data on the proportions of unanticipated intubation from their own department during the study period. The departments were not given specific data on the outcome, but data acquisition was in fact possible, if requested.
The conditions regarding tracheal intubation and mask ventilation were registered in the DAD following actual airway management. In case of an intubation score of ≥ 2 or difficult mask ventilation, it was mandatory to fill out difficult airway management details. The database was programmed so that it was impossible to record difficult airway management parameters before other mandatory anaesthesia variables, including finishing time of the anaesthesia. Unfortunately, this programming was not ideal, and may have created an incentive to register intubation scores < 2, in order to avoid the extra registration burden. Ideally, registration of airway management difficulties ought to take place in real time, i.e. immediately following airway management. The airway manager would thus be encouraged to register correctly, instead of leaving the registration to be completed, potentially by another anaesthetist, at a later time. This may have led to underreporting of difficult airway management.
We found a surprisingly high percentage of unanticipated difficult airway management. This may partly be explained by the fact that the DAD registration frame was set by default to ‘No’ regarding anticipated difficult intubation and difficult mask ventilation. Though it is continuously reinforced in the departments that the anaesthesiologists should register correctly, as well as record their anticipation of a difficult airway management, we cannot exclude that the default settings may have affected the registrations, thereby creating a higher proportion of unanticipated difficult airway managements. Furthermore, it may be a misconception to assume that every practitioner performed an airway assessment, every time.
The reported proportion of unanticipated difficult tracheal intubation in daily routine practice, ranging from 75 to 93%, underlines the importance of always being prepared for unexpected airway management difficulties. Prediction of difficulties remains a challenging task. There may be ample room for improvement, based on a rigorous, evidence based and systematic approach .
We thank The Danish Anaesthesia Database and the participating departments.
The trial is funded by independent grants by: The TRYG foundation; Nordsjællands Hospitals research council; The Capital Regions funds for research; DASAIMs fund.