Human factors in the development of complications of airway management: preliminary evaluation of an interview tool

Authors


Correspondence to: R. Flin

Email: r.flin@abdn.ac.uk

Summary

The 4th National Audit Project of the Royal College of Anaesthetists and the Difficult Airway Society (NAP4) analysed reports of serious events arising from airway management during anaesthesia, intensive care and the emergency department. We conducted supplementary telephone interviews with 12 anaesthetists who had reported to NAP4, aiming to identify causal factors using a method based on the Human Factors Investigation Tool (HFIT). We identified contributing human factors in all cases (median [range] 4.5 [1–10] per case). The most frequent related to: situation awareness (failures to anticipate, wrong decision) (nine cases); job factors (e.g. task difficulty; staffing, time pressure) (eight cases); and person factors (e.g. tiredness, hunger, stress) (six cases). Protective factors, such as teamwork and communication, were also revealed. The post-report HFIT interview method identified relevant human factors and this approach merits further testing as part of the investigation of anaesthetic incidents.

‘Human factors’ refers to the study of environmental, technical, organisational, psychological and physiological influences on human performance. The role of human factors in the safe and efficient delivery of anaesthesia is recognised, illustrated by a recent editorial in this journal [1].

The 4th National Audit Project (NAP4) collected case reports of major complications related to airway management during anaesthesia, in the emergency department or in intensive care, occurring in the UK between September 2008 and August 2009 [2, 3]. It used a web-based data form to collect technical information and free text. One question was on contributing factors (defective knowledge or judgement, poor teamwork, defective training, equipment problems). The free text also gave some insights into human factors issues [4], but the form was not designed primarily for this purpose. The assessors concluded that human factors issues relating to the individual or the team were present in 75 cases (40%) and in 25% of that subset, were a major contributor to the poor outcome. Poor judgement was considered to have contributed in 56% of the cases and inadequate teamwork in 14%. After the NAP4 project had started, we decided to conduct a supplementary, pilot study to test whether a systematic analysis tool might generate additional information on the human factors aspects of these difficult airway management cases.

Human factors have been studied in relation to surgical [5] and anaesthetic incidents [6] including difficult airway management [7, 8]. However, there is no standard method for human factors investigation in anaesthetic incidents and we decided to examine the suitability of a human factors tool used in other work settings. There are several instruments available for coding human factors aspects of incidents [9]. We chose the Human Factors Investigation Tool (HFIT) [10] as it was based on factors relevant to anaesthesia, (e.g. a threat/error model [11] and situation awareness [12]). It was devised for oil and gas drilling operations where monitoring plays a key role and events can escalate very rapidly [13]. The HFIT has 314 questions designed to collect information on 28 types of human and organisational factors, related to four components of an accident trajectory (Fig. 1):

Figure 1.

Human Factors Investigation Tool model of incident causation and direction of analysis. Reproduced from [10], with permission from Elsevier.

  1. Threats (12 factors) – underlying work or personal conditions that may be causal.
  2. Situation awareness (7 factors) – cognitive processes preceding an action error.
  3. Action errors (7 factors) – errors occurring immediately before the incident.
  4. Error recovery mechanisms (2 factors) – actions that averted an accident.

The model indicates that accidents occur when a person makes an ‘action error’, for example, omitting to carry out a critical task. Action errors typically occur from lack of situation awareness, e.g. not noticing the problem. Deficiencies in situation awareness can arise from ‘threats’ such as inadequate procedures or fatigue. The HFIT method provides a systematic interview structure to explore aspects of an event related to underlying human factors. If one of these components is present, then further ‘probe questions’ can be asked. Practitioners from industry found the HFIT to be helpful for incident investigation [10, 14]. The objective of this pilot study was to interview anaesthetists who had reported cases to NAP4, using an adapted version of the HFIT to structure the interview protocol. We aimed to determine if this method could identify causal human factors issues associated with critical difficult airway events.

Methods

Potential participants were located in only 40% of UK NHS hospitals (i.e. where NAP4 cases had been reported), but we aimed to send invitations to volunteers in all UK NHS hospitals, to ensure anonymity for those hospitals that had reported cases to NAP4. Research ethical approval was received from University and NHS Research Ethics Committees and the Research and Development (R&D) Department, Royal United Hospital NHS Trust, Bath. Individual approval was also required from the R&D Departments in each UK NHS hospital (n = 328); 199 hospitals ‘approved’ the study. We estimated that 200 incident reports would be submitted, and that 10% of respondents might volunteer for the additional interview, giving an anticipated sample size of 20, suitable for this type of study [15]. After Ethics Committee and R&D Department approvals were granted (over a year after NAP4 was completed), 13 anaesthetists who contributed reports to NAP4 agreed to take part. Due to a technical problem, one interview was not recorded and so 12 interviews were analysed.

We sent letters of invitation by electronic mail to these hospitals, either to individuals who had reported cases to NAP4; to the local reporter; or to all anaesthetists, depending on directions from that hospital's R&D department. The letter invited individuals who had reported cases to NAP4 to take part in a one-hour telephone interview [16]. Volunteers were sent information on the purpose of the study and were reminded that they should not identify the patient, hospital or members of the care team. Interviews were conducted by a psychologist (EF), audio-recorded (with permission) and transcribed for analysis.

We asked each interviewee to give a description of the event, then asked follow-up questions from the HFIT schedule (Table 1). The questions addressed each category in turn, except for the error recovery category for near misses, as the cases reported to NAP4 were true incidents rather than near misses. The format was sufficiently flexible to allow emerging topics to be covered in more depth, with questions selected to probe particular aspects of the account. We made some amendments to the interview tool following a review by a consultant anaesthetist not involved in NAP4, and also after the first two interviews, where we recognised that some of the tool terminology did not fit these clinical events (e.g. the use of the term ‘error’ [17]). Examples of questions from the final version are: ‘Can you comment on your own and team members’ level of concentration or attention prior to the event?'; ‘Were the roles and responsibilities of the team members clear?’ We asked respondents to state their sub-specialty in anaesthesia, length of experience, details of training on management of the difficult airway, prior experience of the type of case they were describing and what lessons had been learnt. The interview schedule is available from the first author.

Table 1. Simplified Human Factors Investigation Tool categories for coding anaesthetic events
HeadingSubheading and descriptor
Action errorsActions performed – anaesthetist, surgeon or other team member takes inappropriate action (e.g. gives a wrong drug, does not attach a required monitor)
Situation awarenessProblem detection – delays/difficulty in detecting critical change in state
Task cognition – failures in attention, concentration, problem solving, decision making, memory
ThreatsPolicies, standards & procedures
Work preparation – limitations in plans, strategies, contingency plans
Job factors – task difficulty, inappropriate staffing levels, work pressure
Person factors – fatigue, stress, hunger, improper motivation, physical incapability, feeling unwell
Competence & training
Communication – interpersonal or technical failures
Team work – lack of shared mental model, role confusion, poor coordination
Supervision and leadership – ineffective, no pre-task briefing, inadequate monitoring or instruction of trainee
Work situation – conditions at work site, e.g. heat, noise, lighting, too many people crowding the workspace, night shift
Human–machine interface – design flaws, operability, alarms
Tools and equipment – suitability, availability, maintenance, access
Organisational/safety culture – management, systems, norms

Each transcript was analysed line by line [18] to extract the human factors issues, then coded by a psychologist (RF) using the 15 elements (Table 1). A second psychologist coded four transcripts to test inter-rater reliability. Our focus was on contributory factors, but we also noted mitigating or protective factors without formally analysing them.

Results

All cases except one involved adult patients in the anaesthetic room or the operating theatre. Case type and procedural details are not given to protect anonymity, but involved orthopaedic, general, ENT and maxillofacial surgery. Anaesthetic procedures (performed awake and/or during general anaesthesia) included laryngeal mask airway (LMA) placement, tracheal intubation and tracheostomy. In several cases, a difficult procedure had been anticipated. Respondents were experienced anaesthetists and some were trainers on difficult airway courses. The interviews lasted between 24 and 54 min (mean 40 min). Inter-rater reliability of the dual coding of a third of the transcripts (Cohen's kappa) was kappa = 0.93, indicating high agreement [19] and therefore the remaining eight transcripts were coded by one psychologist.

The contributory human factors issues identified, with frequency of occurrence, are shown in Table 2, although the number of times a specific factor was mentioned within each case is not listed as this was influenced by questions asked and repetition on the part of the respondent. We identified a median (range) of 4.5 (1–10) human factors per case. The issues that were mentioned most frequently related to situation awareness (failures to anticipate, wrong decision) (nine cases), in the threat category: job factors (e.g. task difficulty; staffing, time pressure) (eight cases) and person factors (e.g. tiredness, hunger, stress) (six cases). Illustrative quotes from interviews are listed in Fig. 2. Examples of action errors were coded in four interviews, typically when the interviewee said that an anaesthetic procedure was not carried out correctly (e.g. completing all the steps of tracheal intubation correctly) and in one case it referred to a surgical procedure.

Table 2. Human Factor Investigation Tool elements identified as a potentially causal factor (at least once)
 Tool elementCase codeTotal
ABCDEFGHIJKL
  1. AE, action error; SA, situation awareness; T, threat.

AEActions performedx x  x   x  4
SAProblem detection     xx     2
Task cognitionxxxx xxx  xx9
TPolicies, standards and proceduresx           1
Work preparationx      x x  3
Job factorsxx xx  xx xx8
Person factorsx    x x xxx6
Competence & trainingx xx x    x 5
Communicationx  x x x x  5
Teamwork   x   x x  3
Supervision and leadershipx x       x 3
Work situationx        x x3
Human–machine interface            0
Tools and equipment            0
Organisational/safety culture            0
Figure 2.

Illustrative quotes from interviews, by category.

‘Situation awareness’ has prominence as a separate category in the HFIT due to the importance of task-related cognitive skills, such as attention, anticipation and decision making in safety-critical work domains [13]. This category consisted of two elements: failures in problem detection (minor delays to recognise the situation in two cases); and failures in task cognition in nine cases. These cognitive skills are essential for safe administration of anaesthesia [20, 21] so it is not surprising that the respondents mentioned their own cognitive failures. These included not anticipating airway difficulties (e.g. regurgitation). Also decisions were discussed (e.g. to use a LMA rather than to intubate) that, with the benefit of hindsight, might not have been optimal. Several mentioned, that in future, they would increase their risk estimation for certain types of patient, or would pay more attention to weak signals that were, in retrospect, indicating this type of problem.

‘Threats’ is the term from the aviation threat/error model [11] and these 12 factors relate to aspects of the organisation, work environment, equipment, team and the individual that could influence (threaten) work performance by increasing the chance of an error. ‘Job factors’, which include the nature of the task and resources to complete it, were identified in eight interviews. Given the nature of these cases, many of which included severe, unusual conditions, it was not surprising that task difficulty was mentioned for several events. The staffing issues that were referred to included fewer staff available at night or having to work with less experienced staff, or delays in receiving information such as scans.

‘Person factors’ were recorded in six interviews, and included self-reporting of tiredness, the length of time since the anaesthetist had last eaten, stress and feeling unwell. ‘Competence and training’ factors were noted in five cases, where the anaesthetists described possible limitations in competence of trainee anaesthetists or trainee surgeons or support staff, e.g. operating department practitioners. Several respondents discussed the difficulties of balancing the trainees' need to practise techniques against possible risks to the patient relating to their level of competence. One anaesthetist commented that the lack of standardisation in anaesthetic practice sometimes could make it difficult for trainees to know which technique was most appropriate for a given condition. ‘Communication factors’ were found in five interviews, usually within the operating theatre team (e.g. need for more assertive communication with a surgeon who was exerting pressure to proceed) or relating to the need for more pre-operative discussion (e.g. consulting an experienced colleague).

‘Work preparation’, such as inadequate planning, was discussed as a limiting factor in three cases, typically due to a lack of available time, e.g. an emergency or a case added unexpectedly to the end of a list. ‘Supervision and leadership’ was mentioned in three cases, referred to in conjunction with issues relating to competence and training. Examples included, allowing a trainee to attempt a particular procedure, or needing to monitor team members more. Another aspect mentioned was that the lead anaesthetist should not have made assumptions that members of the anaesthetic (or surgical) team knew how do to a task or how to do it sufficiently quickly and therefore they felt they should have briefed them more explicitly. ‘Teamwork’ was referred to in three cases, usually relating to failures in coordination and communication between the lead anaesthetist and one or more surgeons. For example, in one case, the anaesthetist reflected that time should not have been spent discussing with a surgeon as to whose instruments had caused a problem. In another case, the respondent suggested that the ‘authority gradient’ had not been clear.

‘Work situation’ in the industrial version of the tool refers to the work environment, e.g. extreme heat, cold, inadequate lighting. None of these were mentioned, but in two cases, both unusual clinical conditions, the anaesthetists felt that too many personnel had been present during the event. In one case, this created difficulties for those engaged in critical tasks and in another, it created a distraction (observers at the window of the anaesthetic room door). The other factor coded here related to the time of day, i.e. the early hours of the morning.

‘Procedures and policies’ were mentioned as a potential contributing factor in one case, relating to a lack of a protocol for a condition, but problems relating to the categories of ‘human–machine interaction’, ‘tools and equipment’ and ‘organisational factors’ were not mentioned in any of this small sample of interviews.

Human factors aspects of the cases that were protective or mitigating were not formally analysed, but the interviewees frequently mentioned that there had been cooperative teamwork, clear communication [22], strong leadership and beneficial preparation, and that colleagues had willingly responded to requests to help with difficult cases. In many cases, it was clear that the anaesthetist had demonstrated appropriate situation awareness by anticipating complications or rapidly noticing a change in condition. Furthermore, whilst the interviewees were sometimes critical of their own cognitive skills, there were reports of decision making that had provided rapid actions that enabled an effective response to a deteriorating condition.

Finally, an unexpected finding was that many respondents found the experience of further analysis of incidents, by taking part in the interview, valuable in itself. One interviewee referred to the personal recalibration of risk tolerance that can occur following a serious event (Fig. 2, final quote) and our opinion is that an opportunity to examine an incident in a structured manner and more detail is beneficial.

Discussion

We have identified human factors issues in each of our sample of interviews with anaesthetists reporting cases to NAP4 – an average of four per case. This contrasts with the NAP4 analysis, which identified human factors issues in only 40% of cases and suggests that a structured interview will identify more issues than an on-line submission. The most frequently mentioned factors were aspects of situation awareness (e.g. failures to anticipate, wrong decision), job factors (e.g. task difficulty, staffing, time pressure) and person factors (e.g. tiredness, hunger, stress).

The concept of a post-report interview came from industrial confidential reporting systems, such as ASRS (Aviation Safety Reporting System (www.asrs.arc.nasa.gov). The ASRS, which deals with all aspects of aviation in the USA, receives around 5000 reports per month. The UK has a similar system called CIRAS (Confidential Incident Reporting and Analysis System) for the rail sector (www.ciras.org.uk). In these systems, practitioners make a confidential (but not anonymous) report of an event, such as a near miss or an incident, to a reporting organisation which is independent of their employer. Depending on the content, the reporter may be asked to take part in a telephone interview (‘call back’) with a specialist investigator so that additional information, especially relating to human factors components, can be gathered. Thereafter, the incident report, containing any new information from the interview, is de-identified before it is entered onto the system database and a summary may be published in the reporting bulletin. This approach adds additional time and costs to the reporting system, but these organisations believe it provides return on investment as part of the information gathering for their safety management systems. Although efforts have begun to address some of the problem areas identified in NAP4 – for instance, wider use of capnography in emergency situations [23], national strategies for managing patients with tracheostomies [24] and for the management of tracheal extubation [25, 26], only the last mentioned [26] mentions human factors, and even then only in passing. We believe that a greater emphasis on human factors will allow the technical aspects of care to be applied even more successfully.

This was a preliminary study and only a small sample was obtained, so the results need to be regarded as indicative. Investigations into industrial accidents typically reveal a mixture of protective conditions and successful human actions [27, 28] as well as failures and error-enforcing conditions. We only analysed human factors issues that contributed to ‘failures’; as much, if not more, might be learnt from human factors issues that enable or contribute to the rescue of similar situations. ‘Call back’ interviews, such as we conducted, should ideally be performed soon after the event while the reporter has a recent memory of what happened. In our study, very long delays were incurred due to multiple, variable requirements from individual hospitals' R&D Departments.

The anaesthetists who took part in the study appeared to provide frank, self-critical accounts of their cases and this may explain the frequency of situation awareness and personal factors. The reliance on volunteers could have created a sampling bias in that those who chose not to take part may have been confident that their actions had all been appropriate. It is also possible that only where anaesthetists felt there were human factors issues in their case did they volunteer to be interviewed. Nevertheless, the experience of taking part appeared to be positive and the respondent's comments on risk recalibration mentioned above echo the metaphor of the ‘unrocked boat’ [29]: organisations experiencing a period devoid of ‘bad events’ gradually drift towards a less cautious stance more geared to productivity until there is a serious accident [e.g. [30]], when they immediately switch to prioritising safety. Projects such as NAP4 highlight to the anaesthetic profession that risks relating to organisational and human factors, as well as to clinical conditions, are ever present. As our analysis is based on only one report of the event (from the lead anaesthetist), rather than interviews with all those present, it is probably not surprising that their responses frequently related to individual factors such as situation awareness and physiological state. It is possible that there is a degree of (reverse) attribution error [31] in these accounts, with anaesthetists overestimating their own contribution to the event and underestimating the role of situational factors. Likewise, we can only speculate why no issues were reported relating to equipment [32], human–machine interaction or organisational factors; it is possible that a larger study might reveal these categories.

The HFIT required a degree of customisation to suit anaesthetic events. For instance, given the importance of anaesthetists' cognitive skills, the situation awareness category would be improved by including more elements from the original tool (e.g. decision making and assumption) rather than only using problem detection and task cognition. Another consideration is that some text segments could be coded under more than one heading (e.g. communication, teamwork), suggesting that some elements might be combined. Nevertheless, the language of the system was generally applicable. Other human factors investigative tools [9] or the Anaesthetists' Non-Technical Skills [21] framework can code behavioural aspects and these might have given different results. It was beyond the scope of this study to test more than one method.

The results suggest that there may be merit in the incorporation of a more qualitative component (such as the tool we have described here) within future large quantitative surveys. The HFIT interview appeared to be suitable for case discussion with anaesthetists and enabled coverage of a wide range of factors that may have contributed to adverse events in difficult airway management. This preliminary finding replicates both previous research [8, 33] and the views of experts [34].

Acknowledgement

This study was partly funded (Aberdeen University authors) by the Scottish Patient Safety Research Network (Scottish Funding Council, SRD grant to Flin). We would like to express our gratitude to the anaesthetists who took part in this study and to Prof Tanja Manser and Drs Ronnie Glavin, Kathryn Mearns and Lucy Mitchell for their assistance with design and analysis.

Competing interests

No competing interests declared.

Ancillary