Using the Perceptual Cycle Model and Schema World Action Research Method to generate design requirements for new avionic systems

The Schema World Action Research Method (SWARM) has previously been used as a means to explore the underlying decision‐making processes involved in retrospective incident reports. The approach has been fruitful in capturing all interacting processes involved in managing incidents. This paper proposes that SWARM may also be used prospectively within the early phases of the design lifecycle for new behavioral‐based intervention strategies. Six pilot interviews were conducted to explore pilots' perceptual cycle processing when faced with a suspected engine oil leak. The aim was to explore whether there may be any deficiencies within current practise and explore ways in which pilots may be better supported in dealing with abnormal system parameters such as this. A number of design recommendations are proposed for a new avionic system capable of supporting and guiding pilots through the decision‐making process.

The processes underpinning decision-making are typically explored in a reactive manner, that is to say research explores why accidents occur or, explore why operators make mistakes, after an event has taken place. However, it can be argued that models of decision making may also be used proactively. Models of decision making can be useful when applied early on in the design and development of new flight deck technology to provide a more descriptive and complete account of what pilots actually do, or have done in the past (Dillon, 1998). There are numerous models of natural decision making (NDM) within the literature. Considerable debate remains over how decision-making processing can be best conceptualized and presented (Jenkins, Stanton, Salmon, Walker, & Rafferty, 2010). Lipshitz (1993) identified nine different models of NDM. This included the most popular, the Recognition-Primed Decision Model (RPDM; Klein, 1989). The RPDM model argues that decisions are based upon the recognition of critical information and the utilization of prior knowledge (Klein 1989(Klein , 1998. When people need to make a decision, they quickly match the situation to previous patterns that they have learned (Klein, 2008).
This process enables the decision-maker to recognize whether the situation is typical or familiar (Klein, 1989). If the situation assessed can be matched to a prior event (experienced or trained), the decision-maker is able to utilize their previous experiences in dealing with the situation. Kahneman (2011) argues that this culminates in fast, instinctive, and emotional responses. According to Naikar (2010), RPDM proposes that decision-makers will evaluate each individual option until a satisfactory option is found. This mental simulation allows the decision-maker to evaluate the adequacy of the proposed action. The exploration of alternatives stops when the operator reaches a satisfactory option, the action of satisficing (Klein, 1998). Yet, this does not mean it is always the most suitable option and inexperienced operators may need to reject several options before selecting an action (Klein, Calderwood, & Macgregor, 1989;Naikar, 2010). This approach has been useful in understanding decision making across a variety of domains including rail (Stanton & Walker, 2011) and truck drivers (Salmon, Read, Stanton, & Lenné, 2013).
Yet, the critique of the RPDM states that interactions to the decision making are solely within the individual's cognitive processing and the model does not capture the moderating effect of the environmental interactions (Plant & Stanton, 2014). Furthermore, RPDM does not fully capture the connection between the decision and the external environment, which is vital to understand why a decision was made at the time (Dekker, 2006). Excessive focus is placed on the decision as processed in the mind of the individual (Plant & Stanton, 2014. In this respect, the Perceptual Cycle Model (PCM; Neisser, 1976) is thought to give a better account of how decisions are made relative to the psychological processing of the individual and the environmental context surrounding it (Plant & Stanton, 2014. For this study, the aspects of the environment that influence the decision making were crucial in understanding how the pilot may be better supported by their surroundings to make enhanced decisions. A schema is an organized pattern of thoughts and/or behaviors that help organize our knowledge of the world (Neisser, 1976).
Schemas, therefore, provide a template in which we formulate mental representations of the world that can be used to guide future behavior (Chalmers, 2003;Plant & Stanton, 2012). According to the literature, there are five defining features of a schema; meaningful organization, embedded with other schemas, change dynamically as information is received, reorganized based upon incoming data and they are gestalt mental representations (Anderson, 1977;Norman, 1981). Schema Theory provides a valuable explanation for how we interact with the world (Stanton & Walker, 2011). The PCM (Neisser, 1976) offers a visual representation of how "schema" is embedded in a reciprocal, cyclical relationship between an individual and their environment. In this manner, it identifies that the schema that are based on the training and experience of the individual determines the actions that the individual will perform in a set scenario. Yet, it also dictates that present information in the world can assist in constructing the schema. This is in agreement with others who support the notion of sensemaking, whereby situation-specific beliefs are thought to be activated by contextual factors that guide behavior when generic belief networks require additional support (Attfield & Baber, 2017;Baber, Chen, & Howes, 2015). Therefore the PCM suggests a way to understand how environmental sampling, combined with pre-existing biases can influence and guide behavior; but also how subsequent information within the environment determines how an individual perceives and gains understanding within specific situations. The PCM process is represented in Figure 1.
The PCM places emphasis on understanding the processes involved in decision making, rather than focussing entirely upon decision output. Rather than focusing solely on the decision making within the head of an individual, it places the individual and their schema within the wider decision-making environment. It, therefore, supports the notion that cognition is distributed across the wider system within which it is occurring (Hutchins, 1995(Hutchins, , 2000Stanton, Salmon, Walker, & Jenkins, 2009)  | 67 their account of only the individual level of analysis (Reason, 2000).
Incorporating the role of the wider system surrounding the decisionmaking process allows for a greater understanding in how decision making can be supported within the environment that it is conducted (Plant & Stanton, 2012;Stanton et al., 2009). This is important to this area of study because it enables us to see where and how pilots may be better supported throughout the decision making process. This paper was interested in understanding airline pilot decision making in relation to a suspected engine oil leak to see if any further information may be able to assist the pilot in managing the event. The PCM was proposed as an effective tool for gaining a full representation of all factors impacting on the decision from the training that pilots receive, the information that they are presented within in the cockpit as well as the actions that they take.
Understanding perceptual cycle processing can however be extremely complex (Plant & Stanton, 2013). One approach to reducing this inherent difficulty is to use the Schema World Action Research Method (SWARM; . SWARM is an interviewbased method designed specifically for aeronautical decision making, originally developed to elicit information about perceptual cycle processes. Its development aimed to capture the interaction between internal schemata and external world information to fully inform the development of the perceptual cycle process . The method was developed and validated through interview data from airline pilots . SWARM facilitates the collection of data from subject matter experts in relation to the three categories of the PCM; schema, action, and world. Each of these has subtypes. Schema is comprised of six subtypes (direct past experience, trained past experience, vicarious past experience, declarative schema, analogical schema, and insufficient schema. Action has 11 subtypes (aviate, navigate, communicate, system interaction, system monitoring, environment monitoring, concurrent diagnostics, decision action, situation assessment, nonaction, and standard operating procedures). The world subtype also has 11 subtypes (natural environmental conditions, technological conditions, communicated information, location, artefacts, display indications, operational context, aircraft status, the severity of problem, physical cues, and absent information). Within each of these subtypes are further interview prompts that draw out wide-reaching information from subjects to capture the complete processing surrounding the decision making. The full repository of 95 prompts can be found in Plant and Stanton (2016), although the authors recommend not using all of these prompts but reviewing them to determine those relevant to the analysis required. SWARM was the first attempt to develop a methodology that could facilitate the collection of data and practically apply it to the PCM (Neisser, 1976). Previously, the PCM has been used to explore systemic decision-making processes in a retrospective analysis of incidents or accidents (Banks, Plant, & Stanton, 2018;Plant & Stanton, 2012;Stanton & Walker, 2011). However, the authors propose that by using SWARM, the PCM may also be used prospectively during the early phases of the design lifecycle, meaning it may be used to generate data that can inform design requirements for future flight deck technologies.
Displays within commercial airline cockpits have remained largely unchanged for a number of decades (Harris, 2011). This stagnation remains despite it being universally recognized that new sensor-based technology could be used to provide flight crews with more information about aircraft status, potentially offering flight crew more information which could be used to inform safety-critical decisions (Harris & Stanton, 2010;Salas, Maurino, & Curtis, 2010). Green (1990) stated that aircraft equipment can be more closely tailored to human requirements and so it makes sense to explore and highlight possible deficiencies in current procedures and displays on the flight deck. This is because it has long been established that there is a connection between the information that is available to an operator and the subsequent quality of their decisions (Jenkins, Boyd, & Langley, 2016;Jenkins et al. 2010). Thus, within this paper, we seek to explore the underlying mechanisms of pilot decision making in abnormal operating scenarios using the PCM (Neisser, 1976) and SWARM  to provide insight into the possible deficiencies within current flight deck architecture and propose how pilots may be better supported through access to timely, accurate information from new avionic systems.

| Participants
Six commercial airline pilots were recruited to take part in this study (two females and four males), aged between 26 and 35 years (M = 30.17, SD = 3.02). All participants were qualified fixed-wing airline transport pilot licence or commercial pilot license pilots with an average 3692 h flight experience (SD = 570.39) and had held their licenses for an average of 8.08 years (SD = 1.59). Interviews lasted for approximately 1.5 h and participants were reimbursed for travel and time spent participating in the study. The study was ethically approved by the university's Ethical Research Governance Office.

| Procedure
Participants were first introduced to the aims of the research before being asked to give informed consent to take part. The participants were then presented with a hypothetical scenario relating to a suspected aircraft engine oil leak. This stated the following.

| An incomplete maintenance action has resulted in an oil leak in the aircraft engine: You are in cruise
The rationale behind focusing on an engine oil leak scenario is that despite being rare, they can have serious repercussions on flight operators, maintenance teams, and passengers when handled inappropriately (ATSB, 2012(ATSB, , 2017Parnell et al., in press). In addition, the presentation of engine system parameters has remained largely unchanged for a number of decades with analog displays simply being replaced by digital equivalents (Harris, 2011). This means that in current practise, pilots only become aware of abnormal system parameters when threshold limits have been met via their on-board alerting systems. At the point of notification, pilots have few options available to them and they must either throttle back or shut down the engine completely, to prevent the engines from reaching oil starvation (ATSB, 2012(ATSB, , 2017. Engine shutdowns in particular have significant implications for the flight operator both in terms of economics and safety so there is great value, both from operations and economic perspectives, in exploring how such situations can be prevented (Parnell et al., in press). The limited options available to pilots within this scenario was also ideal for the explorative nature of the current work.

| Schema World Action Research Method
Following the presentation of the scenario, participants were invited to take part in a semi-structured interview using exemplar prompts from the SWARM repository. To get their initial thoughts on the scenario and their response they were asked to give a brief initial overview of their thoughts in relation toward the scenario, prompted by two questions: 1. Currently, how would you be informed of an oil leak in an engine? 2. How would you respond to it? While these questions were not part of the SWARM interview prompts, they allowed the participants to speak openly about their perception of the scenario as well as ask any further details from the researchers. After this, the SWARM prompts began.
A down selected set of SWARM prompts were selected from the original 95 as suggested by Plant and Stanton (2016), not all are relevant to all events so down selection allows the interview to be more efficient. Plant and Stanton (2016) state that preferably this would use the top five subtypes for each PCM category. The SWARM prompts were reviewed by Human Factors Experts to determine which would be relevant to the scenarios presented in the study which is in line with the guidance. A total of 37 of the SWARM prompts were selected for the interview. Example relevant prompts included "What would you be looking at on the technological system during the scenario?"; "Would you require information from others?" and "What information would you use to assess the severity of the problem?" Participants were asked to speak openly and honestly and in as much depth as possible. This enabled a full account of their anticipated response to the scenario, capturing their schema representations, the information they would access from the world, and the actions they would be taking.

| Data analysis
All interviews were audio-recorded and subsequently transcribed by the researchers. Transcripts were then reviewed by the researchers.
The comments from the interviews were coded to the Schema, Action, or World feature of the PCM. As the method utilized the SWARM methodology the application of the reports to the PCM model was straightforward, as the relevant prompts related to the relevant PCM category, see Stanton (2013, 2016) for further details on this process. The process of mapping the interview reports on to the PCMs was conducted in an iterative manner until the researchers were satisfied that the mapped decision-making process accurately reflected the interview data. The data were used to develop an amalgamated representation of the pilots' PCM for the given scenarios. As the pilots are highly trained and the procedures that they conduct on the flight deck are heavily standardized and regulated, the reports were largely similar. The generated PCM containing the combined responses of the pilots was then reviewed by an independent subject matter expert, with over 10 years of flight experience and a strong background within the Human Factors discipline to ensure accuracy.

| Generating design requirements
The second part of the interview asked participants to consider how a future Engine Monitoring Assistant (EMA) system may influence their responses to the previous scenario. Here the updated scenario stated the following: "An incomplete maintenance action has resulted in an oil leak. An automated system detects a non-normal change in system parameters and notifies you of a non-critical oil leak i.e. sufficient levels to complete flight to intended destination safely" Participants were again asked key SWARM prompts of relevance to generate data that could also be used to inform the development of a PCM that could inform how participants thought a future system could assist them. From the PCMs and the transcriptions, the researchers reviewed how the pilots could be further supported with their decision making to inform design recommendations. Importantly, participants were not given specific details of how this system may work, rather they were asked to determine how they may want it to work and what utility it may have.

| RESULTS AND DISCUSSION
The interviews generated extensive accounts of how trained and experienced airline pilots would respond to the hypothetical scenario of an engine oil leak. Pilots undergo stringent training on a regular basis and they have a set of standard operating procedures that BANKS ET AL.
| 69 dictates how they respond to engine events such as the engine oil leak. The reports that pilots gave therefore followed similar decisionmaking processes and could be aggregated in one PCM. This was also the approach used by Plant and Stanton (2014) who aggregated data from 20 pilots into one composite PCM, demonstrating that the model can account for all decision making data. Due to applying the SWARM, the accounts were comprised of information that detailed the components that inform the PCM. This included the comprising element of the participant's schema, including any previous experience of an event, their trained experience, and knowledge of oil leakage. The world information captured all available information in the environment, including the visual presentation of information in the cockpit and communicated information from relevant others.
They also detailed all relevant actions that they would take in their response to this event. Through the application of the SWARM interviews to the PCM, the information could be easily categorized into the three PCM components.

| PCM of current processing in response to an aircraft engine oil leak
The final amalgamated PCM of current practise is presented in Figure 2 and outlines the perceptual cycle processes of a pilot when they are subjected to dealing with a suspected engine oil leak. The information available in the "world" acts as the impetus for diagnosis and option generation. To first diagnose and understand the problem presented to them, pilots would attend to any warning messages on the Electronic Centralized Aircraft Monitor (ECAM) system, also sometimes referred to by some manufacturers as the Engine Indicating and Crew Alerting System (EICAS). This system provides data to pilots on the status of a variety of aircraft systems, as well as providing warnings and alerts when parameters reach unusual levels.
These warnings are color-coded to differentiate their urgency for attention and action by the pilot. Participants reported that when receiving an ECAM/EICAS message in this scenario they would seek to validate it by checking the most appropriate system parameters, in this case, oil pressure, oil quantity, and oil temperature. All of these artefacts are available in the "world" via the Engine Display. Appraisal of these dials would enable the pilots to determine whether the warning is genuine or spurious. Particularly on older aircraft, sensors are vulnerable to exhibiting false indications and therefore the presence of a single abnormal reading may not be a valid indication of failure (Flight Safety Foundation, 2001). Thus, to confirm a genuine failure, flight crews must inspect multiple sources of information. Five participants suggested they might also seek assistance from the cabin crew in terms of performing a visual inspection of the engines. This would provide an opportunity to gather more "world" based information that would be used to trigger relevant "schema," previous experience in dealing with available information. This process shows how information in the "world," that is the presence of a master warning, can go on to trigger relevant "schema." For example, the F I G U R E 2 Amalgamated Perceptual Cycle Model of pilots' current approach to dealing with an engine oil leak presence of the master warning suggests something is seriously wrong, leading to subsequent "action" which includes checking affected engine system parameters. Further detail is given in Figure 2.
When it came to making a decision, all six pilots spoke about the utilization of the "DODAR" decision aid (Diagnose, Options, Decision, Assign task, Review) or variations of such, for example, T-DODAR (Time, Diagnose, Options, Decision, Assign task, Review) to help them systematically reach a decision (Walters, 2002). The variation in the DODAR come about due to airline specific guidelines, yet each uses a similar tool to guide the pilots' decision making. Each element of the tool details key requirements and the order dictates the order the requirements should be fulfilled. "Diagnosis" is the requirement to gather as much information as possible to determine and confirm the problem. "Options" required the pilot(s) to consider all possible alternatives relating to all possible actions. "Decide" requires the pilot(s) to come to the most appropriate selection of action(s). Once the decision is made, they must then "Assign tasks" to allocate tasks between the pilot flying and pilot monitoring. The "Review" aspect then denotes that the resulting consequences and emerging situation should be continuously reviewed to ensure the desired outcome is achieved. It was evident from the interviews that the DODAR aid was central to the decision making process. The detail they gave regarding the DODAR decision aid linked into the SWARM interviews to allow participants to discuss the factors that guided their decisions and the resulting actions.
3.2 | PCM of future proposed processing in response to an aircraft engine oil leak In Scenario 2, all six pilots recognized that an early warning from an EMA system would indicate that minimum thresholds had not yet been met. For example, one pilot stated, "Time is everything in an aircraft, with failures, with everything. It buys you time so you can sit, think, discuss with the company, come up with a plan and have a bit more time before it is critical." (Participant 2). Participants stated that they would still utilize the DODAR decision-making tool. This is an airline regulated decision tool that pilots must use to guide their decision making and therefore it would still be applied when the EMA is present. In this scenario, the flight crew would begin monitoring and trending oil to rule out the potential for spurious sensor readings as soon as the assistant system alerted them to a possible issue. With engine parameters continuing to show an abnormal downward trend, the flight crew would use their prior experience (both through operational activities and training) to determine that an oil leak was indeed occurring. In terms of a pilots PCM, Figure 3 demonstrates that information in the world (i.e., the system triggers alert in relation to the oil system) can trigger underlying schemata (i.e., EMA provides an advisory warning that may be wrong and therefore requires further investigation), which then goes on to influence action (i.e., monitor and trend oil system parameters to confirm what the problem may be). Importantly, the majority of the F I G U R E 3 Amalgamated Perceptual Cycle Model of pilots' future proposed approach to dealing with an engine oil leak with an automated system assistance BANKS ET AL.

| 71
PCM remains similar to that presented in Figure 1 (i.e., pilots would utilize the same information in the world which would go on to trigger the same underlying schema). As one pilot stated; "I would do the same checks. I would have a look for myself…" (Participant 4).
However, this is likely to be a product of the strict training a pilot goes through. For example; "…I think that is something that pilots always do. We have to check, check and double check" (Participant 1).
With this in mind, it is mainly the "Action" phase of the PCM that changes as a result of EMA implementation which is what we would expect.
Whilst in the first scenario (current practise), pilots would choose to shut down the engine, the general consensus amongst the pilots for the second scenario (with the EMA) would be to accept the information provided by the pilot decision aid and therefore execute the operational limitation. Pilots would then reevaluate whether they would still be able to reach their destination, based upon their monitoring and trending of oil quantities. A key difference between the current processes in Figure 2 and anticipated processes in Figure 3 are that pilots are better informed with the future system on the remaining level of oil and if they can reach their destination. Pilots suggested that if they could receive more up-to-date information regarding the oil level in the case of a leak then there would be less ambiguity and therefore, they could make more informed decisions about shutting the engine down. Importantly, pilots did not think that the addition of an EMA would slow down a pilots response to an incident. Instead, the provision of an early warning would enable pilot responses to be more considered. The process of diagnosis, the options that are available, the allocation of tasks, and key decision points remain very similar.

| Design recommendations
Once an understanding of pilots actions had been collected using the SWARM technique and subsequently mapped using PCM, it was possible to use this information to generate design concepts that would enable easier access to appropriate information, and therefore reduce the likelihood of the suspected oil leak having a significant impact on flight operations. Table 1 lists a number of possible design recommendations that could be implemented on the flight deck that may reduce the requirement for, and number of, in-flight engine shutdowns as a result of abnormal engine system parameters. These recommendations are based upon providing the flight crew with reliable and accurate data relating to the on-going status of engine system parameters using the EMA.
As suggested in Table 1, an additional potential benefit relating to the diagnosis of oil leaks is supporting pilots in choosing an appropriate route diversion and an appropriate airfield for landing.
Diversions are typically less of a problem for short-haul operators, operating in heavily populated and serviced locations, such as Europe and the United States, whereby maintenance bases are easily accessible. Yet, it is important to remember that choosing an appropriate, nonmaintenance landing site is based on multiple factors.

| Evaluation
SWARM can be seen as a similar methodological approach to the Critical Decision Method (CDM; Klein et al., 1989), in that it relies on verbal reports to elicit understanding. As a consequence, similar to CDM, one of the main limitations of SWARM is the reliance on verbal reports Stanton, & Young, 2005) to represent the cognitive processing of the decision-maker. There are known issues with memory alteration and decay when data are collected this way (Klein & Armstrong, 2005;. The interviews were conducted in a classroom-type environment where there were no cues to the cockpit. If the studies were conducted within the cockpit it is true that the pilots may have had more cues to assist them in reporting their decision-making process. This should be reviewed in future work. The interviews were based on a hypothetical scenario that the pilots may not have directly experienced in the real world. They did, however, report experience in the simulator of similar events that would trigger the same standardized responses that they reported in response to the oil leak scenario. It is often the case that pilots are trained not to respond to the cause of an engine failure, but the resulting impact that it has on the aircraft functionality. This was why the pilots were able to give detailed accounts of BANKS ET AL.
| 73 the procedures they would conduct to diagnose and contain the scenario. The standardized procedures facilitated the amalgamated PCM representation from SWARM interviews. The validation of the models by an independent subject matter expert also credits the robustness of the PCMs.

| CONCLUSION
The main purpose of this paper was to explore the potential utility of using the SWARM interview technique to elicit value within the product development lifecycle. Using an engine oil leak as an exemplar scenario, the interviews sought to provide insight into the current procedures pilots must undertake, their thought processes and the sources of information that pilots access when dealing with such a scenario. Not only did SWARM enable the production of a PCM for this process but the SWARM technique also enabled researchers to generate insight into how pilot decision making can be better supported in such events. In addition to this, the same approach was used to determine how pilots may want to be assisted in the future and from these key design recommendations were generated. This represents a much more proactive approach to technology, and specifically the cockpit display, the design that engages end-users in the design process at a much earlier stage of the design lifecycle than is typical. Such engagement can lead to designs that are more suitable for implementation and installation within a cockpit environment and can more directly address end-user needs. While the focus within this study was on aviation, it is possible that similar approaches can be taken in other domains to facilitate the early engagement of users in the design process. This includes areas such as rail and road where the PCM has already been shown to be effective.