The authors declare no conflicts of interests and have no financial disclosures.
Development and pilot testing of an operative competency assessment tool for pediatric direct laryngoscopy and rigid bronchoscopy†
Article first published online: 22 OCT 2010
Copyright © 2010 The American Laryngological, Rhinological, and Otological Society, Inc.
Volume 120, Issue 11, pages 2294–2300, November 2010
How to Cite
Ishman, S. L., Brown, D. J., Boss, E. F., Skinner, M. L., Tunkel, D. E., Stavinoha, R. and Lin, S. Y. (2010), Development and pilot testing of an operative competency assessment tool for pediatric direct laryngoscopy and rigid bronchoscopy. The Laryngoscope, 120: 2294–2300. doi: 10.1002/lary.21067
- Issue published online: 22 OCT 2010
- Article first published online: 22 OCT 2010
- Manuscript Accepted: 11 MAY 2010
- Manuscript Received: 26 JAN 2010
- pediatric airway;
- educational measurement;
- residency Laryngoscope;
- Level of Evidence: 3b.
To develop and pilot test an objective assessment of technical skills instrument for evaluation of residents' surgical performance of pediatric direct laryngoscopy and rigid bronchoscopy, with emphasis on feasibility, validity, and interrater agreement.
Prospective, unblinded educational quality-improvement project combining paired and unpaired observations.
Objective Structured Assessment of Technical Skills (OSATS) evaluation instruments were developed from detailed faculty input. Two instruments were created: a surgical checklist and a global assessment of surgical performance. The instruments were tested in a simulation bronchoscopy course and in the operative suite. Paired observations were used to determine interrater agreement, whereas multiple evaluations were used to calculate construct validity and internal consistency.
Forty-four assessments were completed for 19 residents. Seven faculty members evaluated residents as they performed laryngoscopy and bronchoscopy in an animal simulation laboratory and in the operating room. The evaluation tool was found to be feasible. The interrater agreement for both instruments was significant at 80.4% to 84.6% (P ≤.0001). Construct validity was confirmed with increasing mean global and task specific scores by postgraduate year (P <.0001). Internal consistency, measured with Cronbach alpha, was high at 0.968.
This pilot study suggests that a reliable and valid instrument for objective evaluation of surgical competency can be developed for pediatric direct laryngoscopy and rigid bronchoscopy. This instrument can be used for formative and summative feedback of operative performance. In addition, it was easy to use and valid in a limited evaluation; however, larger studies are required to validate its utility. Laryngoscope, 2010