Evaluation of airway equipment: man or manikin?


The past decade has seen significant changes in the area of airway management. The difficult airway continues to be an elusive problem, with an incidence that has remained largely unchanged and with complications arising from its management that remain a leading cause of anaesthetic morbidity and mortality [1]. The anaesthetic profession has responded with the publication of structured guidelines for difficult airway management [2]. Industry has not lagged behind, with a proliferation of new airway devices aimed to provide anaesthetists with alternatives to conventional airway management techniques. Cook listed 14 different types of supraglottic devices in 2003 [3], whilst a recent meta-analysis mentioned another 13 types of intubating aids including six videolaryngoscopes, two optical bougies and two airway conduits [4]. Add to that new designs of tracheal tubes, exchange catheters, stylets, lightwands and bougies and one has a long list that has probably grown by the time this editorial is in print. There has also been a move towards single-use equipment and for every device that is developed, one can be certain that a disposable version will soon follow. So, how does the discerning modern anaesthetist decide which device to use? In this era of evidence- based medicine, it should simply be a matter of looking at the supporting research. The gauntlet has been taken up rather enthusiastically by anaesthetists whose prolific articles are rivalled only by the burgeoning number of devices.

This decade has also seen changes in the way research is conducted. The Research Governance Framework [5] was introduced in an attempt to regulate and maintain the quality of published research. The Central Office for Research Ethics Committees (COREC) was established in 2000 in order to streamline the research ethics process and in 2007 was incorporated along with the local research ethics committees (REC) into the National Research Ethics Committee (NRES) [6]. The process of setting up a clinical trial in humans is complex, and obtaining ethical approval is a difficult and long-winded process that can, at best, take up to 3–6 months. To add to the difficulties involved in conducting clinical research, shift patterns and limitation of trainees’ working hours result in many trainees’ completing their training without ever being involved in a single research project.

What, then, should be the minimum evidence to label an airway device as efficacious and fit for purpose? Unlike introducing a new drug, there are no requirements for airway devices to undergo trials in humans before being introduced into the market. Cook, addressing this question in an editorial in Anaesthesia [3], suggested that a new airway device should undergo a three-stage process: in stage 1, devices would be evaluated ‘on the bench’ and in specifically designed manikins; in stage 2, a rigorous pilot study would take place (in humans) to determine whether the device is effective and safe; and in stage 3, the device would be compared in a randomised controlled trial (in humans) against the current gold standard for the procedure for which it is expected to be used (in the case of supraglottic airways, the ‘classic’ laryngeal mask airway). Stage 1 would probably not require ethical approval; stages 2 and 3 would require both ethical approval and written consent.

Many researchers have addressed these difficulties by finding an innovative solution: manikin studies. It is quite easy to see why manikin studies are attractive to researchers. Not all research ethics committees consider approval necessary for these studies and if approval is applied for it is usually easily obtained [7–10], as the issues around participant information, consent and recruitment involve staff and not patients. Furthermore, there are no adverse effects that might potentially halt the trial and the study can be completed in days, rather than months or years. There is certainly an important place for manikin studies in anaesthesia. For example, in some studies patients are not essential and manikins will suffice [11, 12]; others may require conditions that cannot be recreated easily in patients and manikins can offer an excellent alternative [13–15]. However, we are disturbed by a growing trend amongst researchers to circumvent Cook’s suggested stages 2 and 3, by extrapolating results from the evaluation of new airway devices on manikins to humans.

We conducted a search using ‘manikins’ or ‘mannequins’ as keywords in the pubmed database (http://www.ncbi.nlm.nih.gov/sites/entrez?db=pubmed) and the websites of five premier anaesthetic journals, from January 2000 to July 2010. The results were analysed for studies in which manikins were used to evaluate or compare airway devices. A total of 57 such publications were found: 40 in Anaesthesia, 10 in the European Journal of Anaesthesiology: three each in the British Journal of Anaesthesia and Anesthesia and Analgesia; and one in Anesthesiology. The newest entry into this arena is the paediatric manikin: a search for ‘paediatric manikin’ studies comparing airway devices generated four papers in the last 2 years. Dare we say that having grown tired of comparing adult devices in adult manikins, researchers have now turned their attention to paediatric devices in paediatric manikins? This is exemplified by a recent publication [8] that looked at minimum and optimum light requirements for laryngoscopy in paediatric anaesthesia, in which data were obtained from 50 anaesthetists performing laryngoscopy on a child manikin. The authors had published a study [9] in the previous year that looked at minimum and optimum light requirements for a Macintosh laryngoscope, obtaining data from 50 anaesthetists performing laryngoscopy on an adult manikin. Both studies, not surprisingly, concluded that ‘anaesthetists can see the larynx at very low light levels’, that ‘optimum illumination was significantly greater than minimum illumination’, and that ‘a brighter laryngoscope may not be a better one’. One would expect from the discussion in the first paper, which stated that reflection and the presence of mucous membranes are important contributory factors in patients, that a follow-up study in patients may have generated data of more use to the anaesthetic community than a confirmation of the same results in a paediatric manikin, made of the same material.

Many of the manikin studies have used time to intubation, or time to insertion of the device, as a primary endpoint of success; however, the time to insert a device in a manikin can be significantly different from that in a patient. Misiak et al. attempted to determine if the conditions created during simulation in a manikin are similar to those in humans [16]. The authors conducted the study in two stages, the first involving insertion of a laryngeal tube in a manikin and the second comparing the conditions during ventilation in manikins and anaesthetised patients. They demonstrated that insertion of the laryngeal tube and obtaining a seal were significantly more difficult in patients than in manikins, attributing this difference to population diversity, a factor that could not be simulated.

Other studies have used contact with the teeth during insertion of an airway device in manikins to conclude that one device has a higher incidence of trauma than another [17, 18]. Similarly, Asai compared two different laryngoscopes in three simulated circumstances of restricted laryngoscopy, concluding that in situations where access to the patients’ head is restricted, the Pentax Airway Scope (Pentax, Tokyo, Japan) is more effective than the Macintosh laryngoscope [15]. There is no evidence to allow for such extrapolation of results from manikins to patients; however logical the trend of thought expressed, it is still a supposition. To labour the point, one has to only look at a study by Howes et al. into insertion of the LMA Supreme™ (LMAcompany, St Hellier, Jersey) by novices in manikins and patients [19]. In the manikin phase of the study, all participants inserted the device successfully. Ninety-six per cent of participants inserted it on the first attempt, with a median (IQR [range]) time to lung ventilation of 15 s (12–20 [8–75] s). In the patient phase of the study, insertion was successful in only 86% of cases on the first attempt, and the time to successful placement was 34 s (26–40 [18–145] s).

Common arguments supporting these studies are: manikins provide an unchanging environment from one participant to another; the set-up remains constant between attempts, allowing the generation of more reliable comparative data; and scenarios such as difficult intubations are uncommon and clinical investigation is fraught with ethical problems. We believe that many of the above arguments are the very reasons that make these studies unsuitable for extrapolation to human populations, which are diverse, relatively unpredictable and clinically challenging.

Manikin studies often use the ‘get out of jail’ clause by pointing out in their discussion that one of the limitations of their study was its conduct in manikins, so that further clinical studies would be needed to evaluate the effectiveness of the device in patients. We could not find a study in which the authors followed up their published manikin-based study with a patient-based study, to confirm the initial results, and we would be pleased to be corrected should one exist. Further, such papers cite each other as references to justify the continued use of manikins. A recent publication stated ‘that simulation of intubation scenarios in an anatomically correct manikin has been widely used for similar studies in the past and has proven a reliable surrogate for the clinical context’ [20]. This is a dangerous trend as it could lead to the creation of unsubstantiated data that then provides the basis for future studies.

Manikins have several limitations, especially when it comes to studies involving airway devices. Despite the close resemblances, even the advanced high fidelity simulation manikins are unable to recreate the feel and finer aspects of human airway anatomy. An interesting study by Hesselfeldt et al., and perhaps the only one to attempt to evaluate the airway of the SimManTM full-scale patient simulator (Laerdal, Stavanger, Norway), concluded that the manikin’s airway was generally realistic but significantly differed from the human airway in important aspects, and the user should be aware of these aspects in order to obtain maximum benefit from training and evaluation and when using the simulator for testing new equipment and techniques [10]. A series of studies by Cook et al. evaluated four commercially available manikins for various supraglottic devices [21–24]. The authors concluded that the manikins’ performance for all supraglottic devices was unequal and that this had implications for training in the use of the devices and for studies assessing their performance in manikins. There was also a possibility of variation between individual manikins from the same manufacturers. Comments noted were ‘poor laryngeal anatomy’, ‘rigidity of tissues’, ‘stiff and unrealistic’ and ‘poor face mask ventilation’.

In very much the same way as the trainee graduates from a manikin to a patient when using a new airway device, it is time for serious researchers to move on to study patients rather than manikins. Call a halt to these prolific manikin-based studies and anaesthesia will be the better for it.

Competing interests

No external funding and no competing interests declared.