A sociotechnical systems analysis of aircraft aerodynamic stall events

In 2009, Colgan Air Flight 3407 had a fatal crash in New York due to an aircraft aerodynamic stall. Previous reports had placed the actions of the crew as the cause of the incident; however, this work provides a sociotechnical systems analysis of the events that led up to the fatal accident. An Accimap analysis provides a top‐down systemic analysis of Flight 3407, considering the high‐level governmental and regulatory agencies involvement. An online survey with 47 airline pilots provides a bottom‐up review of pilots’ perceptions and decision‐making in response to aerodynamic stall events. Combining the two approaches generates a holistic approach to managing aerodynamic stall events. Analysis of Flight 3407 identified contributory factors within the higher regulatory and company levels. Furthermore, questions of appropriate training were raised by the pilots within the survey results regarding the immediate response to a stall event which commonly affords a startle and/or surprise response. The impact of “startle and surprise” on the pilots’ response to this situation was identified as a key area to focus on, with design and training recommendations provided. We consider these within the context of recent training recommendations in the industry.


| INTRODUCTION
On February 12, 2009, Colgan Air Inc. operating as Continental Connections 3407 was a scheduled commercial flight on an instrument approach to Buffalo-Niagara International Airport when it crashed into Clarence Center, New York, ~5 nautical miles northeast of the airport.The aircraft in operation was a Bombardier DHC-8-400 (from hereon Q400) and all 49 occupants on board were killed on impact, as well as a resident in the house that was hit on the ground.It was determined that the Captain's inappropriate response to the stick shaker (which warns a pilot of an impending aerodynamic stall) led to the aircraft entering an aerodynamic stall from which it did not recover (National Transportation Safety Board [NTSB], 2010).
The timeline (in Eastern Standard Time) of events leading up to, and including, the accident is presented in Table 1.

| Aircraft aerodynamic stall events
An aerodynamic stall is characterized by the sudden loss of lift.As the aircraft's angle of attack increases, the maximum lift coefficient is reached at the critical angle of attack.After this, the lift generated by Hum Factors Ergon Manuf. 2024;34:16-30. the wing decreases as the separated region of aerofoil travels further to the leading edge (Anderson, 2011;Shevell & Schaufele, 1966).Such an event is extremely dangerous and it can lead to loss of aircraft control if it is not correctly responded to.It is still recognized that fully developed aerodynamic stalls remain the leading cause of loss-of-control accidents (CASA, 2020).Characteristics of an aerodynamic stall include extreme and random roll characteristics, and buffeting of the airframe (due to turbulent airflow; Shevell & Schaufele, 1966).In the Colgan Air Flight 3407, the Captain's inappropriate response to the stick shaker was deemed, by the NTSB (2010) investigation, to be the leading cause of accident.
Aviation is known for citing high rates of accidents caused by human error (Shappell & Wiegmann, 2003), yet it is acknowledged from the sociotechnical systems perspective that placing the blame on the individual is not an accurate or effective view of accident causation (Dekker, 2003;Hamim et al., 2020a;Read et al., 2021;Salmon et al., 2012).It is widely accepted that the accidents, which occur in complex and dynamic sociotechnical systems, such as aviation, are impacted by a range of interacting human and systemic factors (Read et al., 2021;Salmon et al., 2011).Although these factors may not be as quick to diagnose or attribute blame as "human error," the sociotechnical systems domain has developed multiple methods that aim to assess and understand the complexity involved in accidents with the intention of providing measures to prevent them from happening again (e.g., Leveson, 2004;Rasmussen, 1997;Wiegmann & Shappell, 2003).The classification of the Colgan Air Flight 3407 as being primarily attributable to pilot error prevents more detailed exploration of the involvement of the wider circumstances which could have influenced the event.
These circumstantial factors may be important to preventing similar incidents from occurring; therefore, it is important to conduct a detailed systemic analysis.This work sought to understand other contributory factors surrounding the captain's response in the incident, through the use of sociotechnical systems methods.
There are a variety of frameworks and methods, which consider the many levels of a sociotechnical system (Leveson, 2004;Rasmussen, 1997;Wiegmann & Shappell, 2003).Rasmussen's Risk Management Framework (Rasmussen, 1997) describes the various levels within the system involved in safety and production management, and allows the acknowledgment that degradation of system defences and the migration of organization behavior can occur at all levels (Rasmussen, 1997).Rasmussen (1997) outlined the Accimap method, which visually represents the actions and actors involved in the system and their interactions surrounding an accident event.The method typically consists of six organizational levels as follows: government policy and budgeting; regulatory bodies and associations; local area government planning and budgeting; technical and operational management; physical processes and actor activities; and equipment and surroundings.First, actors involved in the accident are identified (developing the Actor Map), then, the actions at each level are identified and linked between and across the levels, producing the Accimap.The Accimap method is popular within accident analysis T A B L E 1 Timeline of events leading up to the Colgan Air 3407 accident (NTSB, 2010).| 17 and has been used not only within aviation (Thoroman et al., 2020), but multiple domains (Cassano-Piche et al., 2009;Hamim et al., 2020b;Salmon et al., 2010;Salmon et al., 2014).Its comprehensive nature to review responsibility across the sociotechnical system, will allow further insight into the factors involved within accidents; especially where improvements and recommendations are warranted.

| Startle and surprise
After experiencing long durations of automated flight aided by the autopilot, the pilot can experience startle and surprise when sudden situations require a rapid switch to an active role (Landman et al., 2017a).The pilot's response to an aerodynamic stall is one of the many examples where the effects of startle and surprise can play a role in the way the flight crew respond to an unfolding situation.
Particularly given the levels of automation in modern aircraft which function without pilot inputs, for example, the low airspeed function on the auto throttle.
Startle is characterized by a physiological response to a highly salient stimulus (Landman et al., 2017b), such as the onset of the stick shaker.The reaction is brief and often involves an acute increase in stress (Landman et al., 2017b).This increase in stress can cause muscle tension, which can lead to abrupt physical responses by the flight crew (e.g., handling of the control column/side stick).Following a startle there is a period of disruption to coherent cognition as expectations of normality are breached and actions may occur without realizing (Bureau d'Enquêtes et d'Analyses pour la Sécurité de l'Aviation Civile, 2012; Martin et al., 2016).In cases where there is a significant threat (e.g., Colgan 3407), the pilot will experience a fear-potentiated startle that will cause a full stress reaction with significant impairment in cognition (Martin et al., 2016).
Surprise is an emotional and cognitive response to events (Landman et al., 2017a).Schema Theory (Bartlett, 1932) is a cognitive psychology theory that has been used to aid explanation of surprise events (Landman et al., 2019).A schema is an organized mental pattern of thoughts or behaviors to help classify world knowledge (Neisser, 1976).Such schema are triggered and shaped from situations within the environment (Neisser, 1976;Plant & Stanton, 2012).If a mismatch occurs between what the individual perceives should be happening, based on their triggered schema, and what is happening, an element of surprise can be induced (Landman et al., 2017a).The cyclical nature between a person and the surrounding environment forms the basis of Neisser's Perceptual Cycle Model (PCM;Neisser, 1976).The key components of the PCM are "Schema," "World," and "Action" (Neisser, 1976).Schema (mental templates) are initially triggered by information from the World through a bottom-up process.Utilizing our individual schema, which are activated via expectations and past experiences relating to the surrounding world, a top-down process then occurs whereby actions are conducted to respond to the world (Parnell et al., 2021).This can then affect the world and trigger further schema, continuing the cyclical process.
Current understanding of startle and surprise is largely focused on the individual's response and their direct impact on the outcome of incidents.This leads to incident analysis and interventions that are largely user-centric.In commercial aviation incidents, this often means targeting the pilot with more training and procedures.Yet, startle and surprise related events are particularly difficult to train for as they require presenting the pilots with unexpected and random events (Landman et al., 2018) et al., 2013).Predictable situations require very little sensemaking, which is a key process in coping with surprise (Landman et al., 2018).
Simulator studies have shown that when presented with an unexpected aerodynamic stall, pilots' responses vary and errors can be made when executing the aerodynamic stall recovery template (Casner et al., 2013;Landman et al., 2017b;Schroeder et al., 2014).
Outcomes from these studies indicate the importance of focusing on reframing and sensemaking capabilities in training, to increase the likelihood of a successful recovery.Landman et al. (2018)  incident and gain an understanding of how the situation could have been better managed with the training of current pilots.We therefore take a mixed-methods approach, using the Accimap analysis method to review all actions and interactions by the sociotechnical actors involved in the outcome of the event, and an online survey to capture pilot feedback on their training and understanding of aerodynamic stall events.
To investigate the Colgan Air Flight 3407 from a systems perspective, the Accimap method (Rasmussen, 1997) was used.Additionally, an online survey was conducted, utilizing the Schema World Action Research Method (SWARM, Plant & Stanton, 2016) to capture current pilots thoughts and possible responses to a scenario such as that encountered on Colgan Flight 3407.The outputs of this survey could then be used to generate a PCM (Neisser, 1976) for an aerodynamic stall on approach.The use of both the Accimap analysis and online survey allows for comparisons to be made between the actions made by the crew of Colgan Air 3407 and the participants of the survey who have undergone more recent training procedures.

| Equipment
The NTSB conducted an investigation into the Colgan Air accident, the report of which (NTSB, 2010) is freely available online and was used as the primary source for creating the Accimap.The Accimap was created using Microsoft Visio.

| Actor Map
The Actor Map depicts the system's relevant causal actors (i.e., individuals and organizations) within the accident and allows the actors who are involved in the incident to be determined.Therefore, the construction of the Accimap is a development of the produced Actor Map.Actors were identified from information in the NTSB report (NTSB, 2010).Revisions were made as to the actor's relevance to the crash and the use of "umbrella" terms comprising of more than one actor, for example, Colgan Air being made of multiple actors such as the Colgan Vice President.

| Development of the Accimap
After further analysis of the NTSB ( 2010

| Questions
The survey was formed of 18 questions (Supporting Information: Appendix A), comprising of both open text answer and closed choice questions.This was designed to elicit as much information in as few questions as possible and optimize the number of participants likely to respond to the survey.

| SWARM method
SWARM (Plant & Stanton, 2016) was used as a framework for the questions in the online survey; this was developed within the aviation domain, providing prompts designed to enable the participants to reflect on the three elements and critical decision-making in the PCM: schema, action, and world.The SWARM was developed and validated within the aviation domain (Plant & Stanton, 2016, 2017).It has been applied to understand pilot decision-making in response to critical events (Parnell et al., 2019;Plant & Stanton, 2017), as well as being used to generate requirements for future practices and aircraft cockpit displays (Parnell et al., 2021).The PCM is the continuous cycle of contextual feedback, which shapes actions through current understanding of events (Rankin et al., 2016).The schema prompts allow participants to reflect on the role of past experiences and expectations.The action prompts refer to information regarding the specific actions taken during the situation.Finally, the world prompts allow information about the external environment to be gathered (Plant & Stanton, 2016).The value in the SWARM is that is looks at the role of context and the wider environmental factors that can influence behavior and performance, as well as past experience which is particularly relevant to the study of startle and surprise.
There are 101 prompts available for use in the complete SWARM approach, yet Plant and Stanton (2016) suggest adapting these to use only those relevant to the research aims.The prompts were down-selected after analysis of the Colgan 3407 Report (NTSB, 2010) and the FAA stall recovery template (FAA, 2015).In total, seven prompts were applied in the survey questions (see Supporting Information: Appendix A).These prompts were asked in response to the following scenario: "You are on a twin-engine aircraft during its final approach (~2000ft AGL), your engine power levers are set at just above idle.The aircraft's stick shaker activates, and autopilot disconnect horn sounds.This is unexpected…" (Supporting Information: Appendix A).
The scenario was based upon the Colgan 3407 accident.Limited detail was given to allow for pilot interpretation.For example, in the case of Colgan 3407, they were flying in moderate icing conditions.If such information was included in the scenario, even though it did not cause the crash of Colgan 3407, it may have misled the participants as to the cause of the aerodynamic stall.

| Participants
Sixty-eight participants (67 males, 1 female) with an Airline Transport Pilot Licence or Commercial Pilot Licence for a dual engine aircraft completed the online survey.However, an exclusion criterion was created where only data from participants who had fully completed the survey was used.This led to a final participant number of 47 ( 46males and 1 female) The largest proportion of the participants held the job title of Captain (59.57%); this was followed by Senior First Officer (23.40%) and then First Officer (12.77%), and 4.26% responding with other.Given the time of the survey coincided with the coronavirus disease-2019 pandemic, participants were asked for information on their current airline or the airline they had worked for.
For their current airline, participants worked for British Airways (n = 15); Turistik Union International (n = 1); Aer Lingus (n = 1); Easy Jet (n = 1); Austrian Airlines (n = 1); Vueling (n = 1); prefer not to disclose (n = 4).The question was open ended to enable the participants to enter what they believed was most relevant; hence, some participants did not give a specific airline, for example, low-cost carrier (n = 3).For airline, they had worked for this included Flybe (n = 4); Eastern Airways (n = 1); Thomas Cook (n = 1); and Cathay Pacific (n = 1).Further information on pilot age, flight hours, and length of licensure can be found in Table 2.
The research was approved by the research institute's Ethical and Research Governance Office (ERGO ID: 61929).Consent was obtained from the participants at the beginning of the survey.

| Data analysis
As mentioned above, 21 responses were excluded, as they did not fully complete the survey.The pilots report from Section 2 of the survey were reviewed in detail to understand the key themes that the pilots discussed.The transcripts were then coded back to the Schema World Action taxonomy to get an understanding of the different aspects of these components the participants mentioned in their responses.This was done as a thematic analysis.The participants feedback could then used to develop a PCM of their decision-making.Descriptive analysis was completed on the responses to the single choice answer questions.

| Accimap
A total of 33 actors across the 6 levels of the Accimap were identified to have been involved within the events leading to the Colgan 3407 accident.The Actor Map in Various environmental factors were evident at the equipment and surroundings level, including the flight conditions, a lack of lowspeed awareness cues on the Q400, as well as no auditory stall warning in addition to the stick shaker activation.The crash and the postcrash fire were not included in Figure 1 but would be present at this level, the crash itself is represented by the box labeled "Impact." At the physical processes and actor activities level, the actions of the Colgan Air Flight 3407 flight crew in the period before the crash are captured.As noted in the NTSB report, there are multiple factors related to the inappropriate response of the Captain to the activation of the stick shaker, including the sudden and continued aft control column input, which consequently meant the aircraft entered an aerodynamic stall.The actor activities level also highlights the crew's complacency, including not completing the relevant checklists at the specific time during the flight.They also did not adhere to the sterile cockpit rule and there was a high likelihood that the crew were experiencing some level of fatigue.These are all events that are shown to have a direct impact on the incident, which may be why the NTSB report highlights the role of "human error" in the cause of the crash.However, when the level above is reviewed within the Accimap, the technical and operational management level, the Therefore, the Accimap shows how contributing factors within the higher levels of the system lead to the adverse actions of the flight crew (continuous aft movement of control and overriding of stick pusher) and therefore these higher level factors can be considered facilitators of the outcome.In other words, this event can be attributed to systemic factors, not human error.

| Online survey
In-depth qualitative responses to the survey displayed similarities between airlines in the way aerodynamic stalls are trained, as would be expected in relation to the requirement for UPRT training since the Colgan Air accident.These include high altitude stalls, midaltitude stalls, and stalls on approach in landing configuration.Results from the online survey found that 63.8% of participants have experienced feeling startled and/or surprised in their daily operations.Although a distinction between the two was not made and so we could not discern which may be more common.Primarily, participants noted feeling confused.
"Confusion and anxiety, followed by comfort of established procedures kicking in mentally" (Participant 1).
The majority stated that startle and/or surprise is included in their recurrent training in comparison to initial training (see Table 4).This is as expected due to its recent introduction (EASA, 2013;FAA, 2015) Table 4 provides the answers to the single choice questions.This shows how startle and surprise has only recently been introduced into training given the higher percentage for recurrent training.The high percentage of "yes" responses to the first two questions within Table 4 shows that the participants would be highly influenced by the aerodynamic stall training they received regarding the scenarios noted above.
Unfortunately, only one female participant completed the online survey.Due to the method of recruitment across social networks a target of equal gender representation could not be met.Therefore, the results of this survey should be interpreted as mainly male-centric.
Gender imbalance is evident within commercial airline pilots (CAA, 2018; FAA, 2019), which is the likely reason for this bias in the data.More effort is needed within the aviation domain to encourage females into careers, as well as include their views and data within aviation research.

| PCM analysis of a hypothetical stall on approach event
With stringent training and Standard Operating Procedures regarding aerodynamic stalls, the responses from the hypothetical stall on approach had a lot of similarity between participants.The benefit of applying the SWARM protocol was to capture pilots input into the types of information they would be looking for in the environment, the actions they would want to take and also their internal thought processes.It was encouraging that there was a lot of overlap in these areas from the responses, as the training has installed coherence across pilots.The insights given help understand the utility of training and if any further support can be given to pilots. Applying the SWARM prompts to the survey responses allowed them to be coded into the elements which make up the PCM and give an insight into the pilots decision-making process.
Table 5 displays examples of such coding, where one prompt from each PCM element is used.
The similarity in responses allowed for the data from participants to be placed onto an aggregated PCM diagram, shown in Figure 2 ( Plant & Stanton, 2015).The provided scenario in which the stick shaker sounds, and the autopilot disconnect horn sounds aids for the initial "World" element of the cycle from which the cycle commences by triggering the relevant schema.Participants reported that, due to the unexpected nature of the scenario, they would question whether the warning is real or spurious.In the sense that the warning could have been caused by a malfunction (e.g., pitot tube blockage).
However, this would then be validated through a quick scan of the | 23 emphasizing the need to identify the stall immediately to be able to recover and avoid entering a full aerodynamic stall.

| DISCUSSION
This research has applied a sociotechnical systems approach to the Colgan Air 3407 incident to gain a greater understanding of the systemic factors that led to up to the fatal event.Although previous NTSB reports highlight pilot error as the cause of the event, an Accimap analysis has shown the role and responsibility of other actors at the company, regulation, and governance levels.
Accusations of human error limit the exploration of these higherlevel factors and can prevent lessons from being learnt (Read et al., 2021)

| Combined approach
The Accimap in Figure 1

| Implications
Replication of surprise has been achieved within highly controlled scenarios in the simulator (Landman et al., 2017b;Schroeder et al., 2014) but the tight controls are still likely to limit the application of the scenario to real world events (Casner et al., 2013;EASA, 2015).Thus, training is only likely to be able to provide limited support to the pilot within such challenging events.Other ways to support the pilot through the management of these known responses at critical points are therefore needed, such as displays and technological interventions that provide support and inform the pilot effectively to update their schema to the match the scenario actually being experienced.
CASA (2020) note that recognition and deliberate actions to reduce the aircrafts angle of attack is of significant importance.
Similarly, participants' responses from the online survey identify that detection and recognition of an aerodynamic stall through the early signs and symptoms is of great importance.Such as through active monitoring of PFD, which was not displayed by the crew of Colgan Air 3407 (see Figure 1).The crew's lack of active Importantly, none of these statements relate to human error but a breakdown across the levels of the system identified in the Accimap.These levels are now considered in relation to recommendations.

| Training developments and recommendations
Consolidation of information from the Colgan 3407 Accimap and the aggregated PCM has generated systemic recommendations.These are presented in Table 6, including a justification and the actor(s) responsible.Many involve approaches that require interaction between multiple levels of the system to enact change and prevent events such as Colgan Air 3407 from happening again.Importantly, links are also made between levels of the sociotechnical system and how they relate to the pilot's perceptual cycle.This shows the interaction between the Accimap levels and the cognitive processing of the individual, which has long been a missing element of the Accimap.
Using the SWARM taxonomy, elements of the Schema, Action, and World shown to be directly impacted by the recommendations have been highlighted.This shows that greater training in the effects of startle and surprise, as well as aerodynamic stall events, needs to be considered for implementation by the very top management and governance levels, as it will consequently impact on the schema of trained past experience in the pilot.This training should enable them to recognize the early signs of an event such as a stall and manage their expectations accordingly.EASA (2015) noted that increasing the frequency of startle and surprise training was beneficial for pilots, including more discussion on these events during simulator briefings and an increase in percentage of startle and surprise being included in UPRT recurrent training.Yet, the frequency of physically practicing techniques to overcome the effects of startle/surprise in the simulator was not be seen.Further, the use of unexpected scenarios during training could not be determined, despite the EASA (2015) recommendations.Advanced active monitoring training will also need to be enacted by the higher levels and enforced by regulatory bodies as well as within individual airlines.The impacts of this will shape the pilots' schema and enhance their interactions with the information they have available to them in the "world" on the aircraft cockpit displays, which will, in turn, enable enhanced system monitoring actions.Improvements to the displays within the cockpit itself suggest that bottom approaches, focused on the lower equipment and environment levels, can also be effective in enhancing the world information available to the pilot to allow them to make more informed decisions.

| CONCLUSION
This work has displayed the importance of applying system-based accident analysis methods to accidents.The application of the Accimap to Colgan 3407 showed that there were many contributory factors, across different levels of the sociotechnical system, that impacted on the aircraft crash.Despite the NTSB identifying the primary cause of the crash being the Captain's inappropriate response to the activation of the stick shaker, there were many wider system implications that facilitated this behavior and which therefore hold responsibility.
Responses from 47 pilots in an online survey showed the repetitive nature of aerodynamic stall training and the recent Clear methods to be used to overcome these effects for any event, which are practiced in simulator at an increased frequency.Allows pilots to understand the possible effects of Startle/Surprise: negative reframing; potential for severe cognitive disruption; aid pilots in the understanding of decision-making processes; its effects in past accidents.It can be inferred that the startle and surprise response played a negative role in the Captain of Colgan Air 3407 response to the aerodynamic stall.
Nearly 2/3 of online survey participants noted feeling startled or surprised in their daily operations.
Training should focus on the skills and sensemaking capabilities to overcome psychophysiological events in contrast to specific upset events (CASA, 2020;Casner et al., 2013;Landman et al., 2017b;Schroeder et al., 2014) Possible inclusion of training aircraft involvement.Casner et al. (2013), Landman et al. (2017b), andSchroeder et al. (2014) have all identified errors when pilots perform stall recovery procedures when surprised.Landman et al. (2018) found pilots who were provided with variable training used throttle and airspeed more effectively in novel situations.
Vast majority of training has involved standardized and predictable responses to non-normal events (CASA [2020]), with 80.9% of participants noting that their airline repeated scenarios used in aerodynamic stall training.
There is difficulty in knowing how the pilot would respond to an unknown unexpected situation.The training received by the crew of Colgan Air 3407 lacked variation and inclusion of aircraft specific features.The inclusion of unexpected scenarios may lend to positively influencing the appraisal of such event.Despite some airlines using techniques such as placing the aircraft in a upset state and told to recover, the pilot has the knowledge that the aircraft will be in an upset state.PFD to include further low speed cues, e.g., more prominent airspeed trend vector Allowing earlier detection of changes in aircraft speed.
Airspeed trend vector gives the aircrafts predicted speed in 10 s if the current acceleration rate is maintained.A more prominent vector which does not obscure the airspeed value and is more visible would mean it is easier for pilots to identify changes and trends in their airspeed.
) report, the Accimap was developed through combining the actions of the actors into the previously created Actor Map.This process consisted of reviewing the NTSB report (NTSB, 2010) multiple times and the actors and actions throughout highlighted.The events directly within the timeframe of the accident were focused on when developing the Accimap and the actions of the flight crew, which were represented within the end-user level.From here, the actors and actions across the other levels of the accimap that related to the events within the accident itself, including pre-and postevent details where relevant, were extracted and placed into the relevant levels of the Actor Map/Accimap.The Accimap developed in this report utilizes the framework developed byRasmussen (1997).The actions were placed into one of the six relevant system levels.Each action was linked to one or more other actions using arrows to display their relations within the happenings of the accident, hence capturing the causal flows of activities, which are embedded within the accident.The Actor Map and Accimap were verified by two Human Factors researchers with a combined total of 12 years' experience in the field.Any disagreements between the initial coding and dual verification were resolved with discussion.The final Actor Map and Accimap represent full agreement.2.2 | Online surveyAn online survey was developed to gain pilots expert knowledge in relation to aerodynamic stall training as part of UPRT, as well as gathering experiences of the effects of startle and surprise training in the real world.
behavior of the pilot can be related back to contributory factors within the training program and the training records that were kept.There were also oversights made by Colgan Air in checking pilots training deficiencies or keeping an update of their proficiencies.This is particularly relevant to the Captain of Colgan Air 3407, who did not disclose two disapproval certificates to Colgan Air that would have warranted further training.Differences in stored data were also evident between the airline and the FAA.Destroying of training records is important as it prevented the ability to review training due to training records on the flight crew which would have shown the Captain's track record.Across this level, shortcomings at the company level are shown to have contributing factors at the lower levels of the system that include a reduction in the crews performance and the progression to the crash scenario.Further responsibility for the event is shown at the local area government, planning, budgeting, and company management level.This shows fundamental issues in the flight crews training provided by Colgan Air.Including areas within the aerodynamic stall training, which was not within the training program, such as the use of reference speeds bugs and consistent stick pusher training.Many of the issues at this level can be related back to oversights within the Regulatory bodies, including the FAA and the guidance that they give in aviation training.
. The Accimap revealed multiple contributory factors within the training and administration at Colgan Air, which provides context for the crew's actions.An online survey with 47 airline pilots provided greater insight into the training currently given to pilots, as well as their requirements in relation to an aerodynamic stall scenario.Thus, this research has provided both a top-down analysis of the contributory systemic issues within the Colgan Air 3407 incident, as well a bottom-up analysis to capture the pilots' perspective on the nature of this event and how it could be effectively managed.This work, therefore, extends accimap analysis by assimilating it with insights from current pilots, to understand how the event may be managed following their recent training to determine if an event such as the Colgan Air would be prevented from happening again, or if further training needs are required.
shows the complex interaction of actions across the multiple levels of responsibility.Here, it is evident that factors within the definition of training requirements at high level regulation levels contribute to the actions of the pilot in the Colgan Air incident.Colgan Air did not include startle and/or surprise in any form of training at the time of the accident.This has been represented in the local area government planning and budgeting, company management level of the Accimap.The recent introduction F I G U R E 2 Perceptual Cycle Model (PCM) response for a hypothetical stall on approach.ATC, air traffic control; IAS, indicated airspeed; VSI, vertical speed indicator. of more regulated startle/surprise training by airlines such a KLM and the Netherlands Aerospace center suggest that this type of training can be beneficial (EASA, 2015).Therefore, startle and/or surprise training could have given the Captain of the Colgan Air Flight 3047 the ability to recognize his cognitive state and deal with the serious situation effectively.Yet, the Accimap method is unable to fully explore cognitive failures or responses such as the startle and surprise responses in this aerodynamic stall incident.Salmon et al. (2012) note that it is the flawed decisions which are represented at the physical process and actor activities level but the failures in cognition which could cause these flawed decisions are not considered.That is why the possibility that it occurred within the Accimap is only revealed through the sudden and severe aft control column input upon the stall occurring.Conducting a survey with airline pilots is therefore useful to close this gap in understanding the responses that pilots would make within a situation similar to Colgan Air 3047.In addition to detailing the ways that aerodynamic stalls, startle and surprise are currently trained.Results from the online survey showed that airline pilots train for aerodynamic stalls in a repetitive manner and 80.9% agreed that their airline repeats the scenarios used for such training.The events posed to the flight crew of Colgan 3407 differed to their training.Aerodynamic stall training provided by Colgan Air lacked the use of autopilot, reference speeds switch, and the stick pushers use and purpose (NTSB, 2010).Hence, the training that the pilots experienced would not have provided an accurate schema from which to manage the real-world event that they experienced.Further, the Captain continuously overrides the Stick Pusher with aft input on the control column (see Accimap physical processes and Actor activities level).Plant and Stanton (2012) applied the PCM to British Midland Flight 92 (G-OBME) and found similar conclusions that the event was outside the pilots' expectation and training.The repetitive nature of training protocols has been previously criticized for being predictable and thus requiring little sensemaking (Landman et al., 2018).Something that UPRT has tried to overcome and new training procedures that aim to tackle startle/surprise have started to look at the different ways in which training can combine to improve pilots' responses (CASA, 2020; EASA, 2015).These new training provisions combine classroom training with simulator and simulator training.This aims to enhance the pilots awareness of their responses to critical and sudden events and manage them more effectively by asking pilots to explicitly carry out counter measure techniques (EASA, 2015).Yet, it is still very difficult for simulated training to replicate the startle and surprise responses to the same degree as in the real world; and airlines are still struggling with FSTD constraints (CASA, 2020; EASA, 2015).The participants within the survey were aware of these complexities."It's very difficult to generate a scenario in the sim where the crew are surprised by a sudden UPRT scenario.We are attempting to train them, not catch them out.Obviously if they self-generate a UPRT scenario then I would let it play out but short of rapid windshear encounters it is difficult to mirror startle and surprise in a simulator."(Participant 14-Training Captain).
monitoring and response to low airspeed cues (physical processes and actor activities level of Accimap), highlights this issue.For example, the First Officer did not input key icing terminology Into the Aircraft Communications Addressing and Reporting system, which lead to the incorrect airspeed being returned.The Captain should have questioned the value, which he did not.Yet, although clear monitoring issues occurred; there was no error detection system to indicate to the crew the returned speeds did not match with the reference speeds switch position.Further, poor active monitoring allowed the aircrafts speed to continually loose speed, while altitude hold mode was engaged (aircraft adds nose up trim to maintain altitude).It raises the question of whether additional low speed cues (which the Q400 lacked), such as a more prominent airspeed trend vector, would have enabled earlier detection of their decreasing airspeed, rather than relying on the stick shaker as the initial world trigger.In addition to training surrounding startle/surprise related events, active monitoring training in association with the Evidence Based Training (EBT) could help pilots know where to look within scenarios such as stall events.Parnell et al. (2019) showed that retrieval error was the most common type of error to occur, where an individual receives incorrect information through erroneous information or misinterpretation.Survey responses noted the possibility of the stall warning (stick-shaker and/or aural warning) being spurious and relating to an error.This reinforces the need for rigorous active monitoring to ensure that, if it is spurious, it can be identified quickly and dealt with appropriately.When viewing the Colgan Air 3407 Accimap, it can be viewed that, if the flight crew had responded to STURGESS ET AL. | 25 the cues through active monitoring; then many of the causal flows which are linked to the activation of the stick shaker (Equipment and Surroundings level) would not have occurred.These can be viewed within the actions linked to "Crew did not respond to cues" in the Physical processes and actor activities level.EASA (2015) notes that active monitoring is still not well developed in most airlines and would enhance flight crews startle and surprise prevention skills.Likewise, in the Regulatory Bodies and Associations level, it can be seen that there was poor implementation of training for active monitoring.This work therefore suggests that the following should be asked in relation to the crew of Colgan 3407 and whether the aerodynamic stall could have been avoided entirely if: -The crew had known about their aircrafts decreasing speed earlier; -The crew had received further monitoring training to aid them in change/error detection; and -The Captain had received EBT where his lack of basic flying skills was identified and specifically dealt with.

A
limitation of this work is that the Accimap was only validated with Human Factors researchers using the incident report.Validation with an airline pilot would be beneficial to review in the future to understand how current training practices, or those from other airlines may currently differ and the impact that this may have had on the events of Colgan Air 3407.The use of an online survey within this work was able to gain some information on the type of training that pilots undergo.It is clear that active monitoring plays a large role in the startle/surprise response but this study did not capture detail on the active monitoring behavior or training that pilots receive on this.Future work should look at this in more detail.These should take the form of qualitative interviews to enable more detailed data collection from the pilots' perspective.This work could be used to inform and develop a training framework of sorts which could be used to aid airlines in active monitoring training and how this relates to startle and surprise responses.Future work should also seek to conduct an updated review of the startle and surprise training programs since the updates suggest in 2015 by the EASA.Again, qualitative data on the experience of this training and providing observations would be a useful method of data collection here to understand the effectiveness of this training.
Participant data for (i) age (years), (ii) hours logged for a commercial airline, and (iii) how long (years) they have held an ATPL/CPL.
Table3presents all the actors that were involved in the event, from the Airline Pilots Association at the top to the numerous equipment, artifacts, and environmental factors that were directly involved in the incident at the bottom.The interactions between these actors within the Colgan Air 3407 incident are presented in the Accimap in Figure1.T A B L E 2 T A B L E 3 Colgan Air 3407 Actor Map.
. Many participants noted that the training was part of their Crew Resource Management training, briefing before simulator sessions and ground-based training days e.g.: "It is discussed a lot.Colgan Air 3407 Accimap (file attached for larger version of this.AirMet, Airmen's Meteorological Information; ALPA, Air Line Pilots Association; FO, First Officer; FOQA, Flight Operational Quality Assurance Program; FRMS, Fatigue Risk Management System; GIA, Gulfstream International Airlines; LOSA, Line Operations Safety Audit.Responses to selection of questions from Section Two of online survey.
"Would your training for aerodynamic stalls influence the expectations you would have if such event were to occur in flight?" 95.7% 4.3% "Have you experience feeling startled/surprised by an unfolding situation in your daily operations?" 63.8% 36.2% "Has the effect of startle and/or surprise in relation to UPR been included in your initial training?"48.9% 51.1% "Has the effect of startle and/or surprise in relation to UPR been included in your recurrent training?" Training recommendations for Aerodynamic Stalls and Startle and Surprise.
T A B L E 6 . Further, participants noted that as current startle and surprise is included as part of a 3-year cyclical training process versus as part of every recurrent training.EASA (2015) displayed that such training is most effective when by all crews and become part of a standard response and hence increasing training frequency to bring further structure.29.8% of survey participants stated that Startle and/or Surprise has not been included in UPR in recurrent training and 51.1% in initial training.The introduction methods to overcome these effects should be introduced from ab initio and higher frequency of training maintained throughout career.
(Landman et al., 2022) importance of identifying the early signs of aerodynamic stalls, hence practicing cue recognition is key to have greater understanding of this(Landman et al., 2022).CASA (2020)provided guidelines that training must include "handson" exposure to stalls that are fully developed; yet airlines are still battling with FSTD limitations (CASA [2020]; EASA [2015]).For many airline pilots, the last time they experienced a real aerodynamic stall would be in initial training in a certified singleengine aircraft.The possible inclusion of this during airline recurrent training (UPRT) may aid in the development of the pilots schema and update their trained past experience.Advancements to active monitoring training focusing specifically on the human factors elements (e.g., startle, surprise, situational awareness, fatigue) including simulator practice/demonstrations. Active monitoring promotes earlier detection of errors; therefore, further training can only be beneficial.The crew of Colgan Air 3407 displayed poor active monitoring skills, missing key ques from their aircraft.Pro-active monitoring should be highly emphasized (e.g., busier flight stages such as approach).EASA (2015) notes that active monitoring is still not well developed in most airlines and will enhance flight crews startle and surprise prevention skills.