Decision‐Making in Multi‐Agency Groups at Simulated Major Incident Emergencies: In Situ Analysis of Adherence to UK Doctrine

When major incident emergencies occur in the UK (e.g., a terrorist attack), senior representatives convene from local emergency services, civil resource organizations, health agencies, and government. These groups make decisions that minimize the societal and economic impacts of major incidents. Their adherence to UK doctrine was assessed using video footage of 18 groups responding to a major incident in a simulation suite, and six groups responding to a large&#8208;scale exercise. There were marked between&#8208;group differences in the sequencing of decision&#8208;making activities, and limited consideration of alternative courses of action. These results provide an impetus for future policy, guidance, and training to address (a) between&#8208;group inconsistency in decision processes and (b) the lack of consideration of alternative courses of action.


| MA JOR IN CIDENT EMERG EN CIE S: CONTEMP OR ARY UK CONTE X T
Recent terrorist attacks (e.g., Manchester Arena, Borough Market, and Houses of Parliament), the Grenfell Tower fire, and natural disasters are examples of major incident emergencies in the UK (henceforth major incidents). A consistent lesson that has been identified from recent analyses of major incidents is the need to develop interoperability: "the capacity of organisations to exchange operational information and to use it to inform decision-making" (NPIA, 2009, p. 12; for a recent review, see House, Power, & Alison, 2014; see also, Alison & Crego, 2008;Comfort, 2007;Pollock, 2013). For example, Pollock (2013) identified the failure of different agencies (e.g., local emergency services, civil resource organizations, health boards, and government) to work together at a strategic level, as an issue in the UK and elsewhere. One important component of the approach to improving interoperability in the UK is embodied in multi-agency groups who are convened and charged with making decisions that help to minimize the societal and economic impacts of major incidents. These Strategic Coordinating Groups (SCGs) include senior representatives of the relevant agencies, including local emergency services, civil resource organizations, health agencies, and government. Similar regional approaches have evolved elsewhere (see, for example, Palm & Ramsell, 2007;Wimelius & Engberg, 2014). SCGs consider the incident in its wider context, define and communicate the overarching policy and strategy for the emergency response, and monitor progress towards defined objectives. This strategy-setting role extends beyond the initial response to the incident and includes formulating a media and communication strategy as well as horizon scanning to facilitate the recovery stage of an incident. SCGs receive nationally endorsed guidance and training, a central pillar of which is the Joint Decision Model (JDM; Figure 1). This UK practitioner model was introduced to support the capacity of multi-agency groups to work together. The model was based on the Police National Decision Model, and before that the Conflict Management Model (National Decision Model, https:// www.app.college.police.uk/app-content/national-decision-model/ the-national-decision-model/#application).
The JDM is the normative decision-making framework that is used to guide the response to major incidents in the UK. It is documented in the nationally agreed Joint Doctrine that provides guidance to agencies that are required by the Civil Contingencies Act, DOI: 10.1111DOI: 10. /1468

Decision-making in multi-agency groups at simulated major incident emergencies: In situ analysis of adherence to UK doctrine 2 | SUPP ORTING INTEROPER AB ILIT Y: THE JDM
Strategic Coordinating Group members come from a range of organizations and major incidents are infrequent, which means that these groups face issues that differ from those faced by expert teams, who frequently work together on familiar tasks (e.g., cockpit flight crews: Stout, Cannon-Bowers, Salas, & Milanovic, 1999;nuclear operations personnel: O'Connor, O'Dea, & Flin, 2008; ship navigators: Hutchins, 1995a; for a review, see Flin, 1996).
The ad hoc nature of SCGs allows them to rapidly assemble in the affected location with key local knowledge, but it also means that they might not have worked together at a major incident (cf. Rouse, Cannon-Bowers, & Salas, 1992; see also, Cannon-Bowers & Salas, 2001), which could reduce their capacity to develop a coordinated plan of action (Stout et al., 1999). The JDM was adopted in an attempt to meet these challenges: to enhance interoperability, engender a shared representation of an incident (cf. Hutchins, 1995a, 1995b, Nickerson, 1993, and enable a coordinated response through shared superordinate goals (cf. Power & Alison, 2017a, 2017bRouse et al., 1992;Stout et al., 1999).
The JDM describes five categories of decision-making activity, which support the superordinate goals of working together, saving lives, and reducing harm. These activities are as follows: (1) gather information and intelligence; (2) assess risks and develop a working strategy; (3) consider powers, policies, and procedures; (4) identify options and contingencies; and (5) take action and review what happened. The five activities can be broadly aligned to the more generic decision-making processes of situation assessment (activity 1), plan formulation (activities 2-4), and plan execution (activity 5; e.g., Lipshitz & Bar-Ilan, 1996;van den Heuvel, Alison, & Power, 2014).
The JDM also describes a consistent sequence in which these activities should occur, moving from gathering information to taking action through the three intermediate activities. While there has been no formal assessment of whether the JDM engenders such a consistent approach to decision-making in SCGs, there is evidence from the field of naturalistic decision-making suggesting that it is unlikely to do so.

| NATUR ALIS TIC DECIS ION -MAKING
The field of naturalistic decision-making is concerned with understanding how people make decisions in real-world contexts, including those where the decisions involve high stakes, are time pressured and occur in a context of uncertainty (e.g., Klein, 1993Klein, , 2003Klein, , 2008Klein et al., 2003;see also, Doya, 2008;Gigerenzer, 2007;Gureckis & Goldstone, 2006;Hodgkinson & Healey, 2008;Shafir, 1994;Tversky & Kahneman, 1974). That is, the field is concerned with exactly those conditions in which SCGs operate. To take one example, the study of decision-making in individual firefighters has revealed marked departures from normative models that are similar to the JDM: Firefighter decision-making often reflects past experience and is recognition-primed and intuitive rather than being based on a process of evaluation and reflection. This observation was originally highlighted through an analysis of the retrospective reports of firefighters (Klein, Calderwood, & MacGregor, 1989; see also, Klein, 1998); but it is also evident when decision-making was investigated in situ at real emergency incidents (Cohen-Hatton, Butler,  Rake & Njå, 2009) and simulated incidents . Similarly, the failure to consider alternative interpretations of evidence has been widely documented in laboratory studies (e.g., Wason, 1966) and in police investigations (e.g., Dando & Ormerod, 2017). Indeed, Dando and Ormerod (2017) have highlighted the fact that current practice in the police (in particular the use of decision logs) might constrain the explicit generation of alternative interpretations of evidence by (less experienced) police detectives investigating serious crime.
There is already evidence suggesting that experienced practitioners do not follow normative models of decision-making; and there is a broader literature which has identified the pitfalls and benefits of making decisions in groups rather than individually (for a recent review, Bang & Frith, 2017). Thematic analysis of interviews with multi-agency group members, after they have participated in simulated major incidents, alongside other evidence which suggests that timely action can be constrained by decision inertia (e.g., Power & Alison 2017a, 2017bAlison et al., 2015; see also Janis, 1972Janis, , 1982Janis & Mann, 1977). Here, decision inertia refers to "a process of (redundant) deliberation over possible options and in the absence of any further useful information" (Power & Alison, 2018); which could interact with whether a group is tolerant of uncertainty or not (e.g., Frenkel-Brunswik, 1949; see also, van den Heuvel et al., 2014; for reviews, see Furnham & Marks, 2013;Hillien, Gutheil, Strout, Smets, & Han, 2017). Moreover, the separation of "assess risks and develop a working strategy" from "identify options and contingencies" within the JDM might have the unforeseen consequence of limiting rather than encouraging the consideration of alternative options (cf. Dando & Ormerod, 2017;Wason, 1966). On a different tack, Bang and Frith (2017) have argued that group decision-making might be improved by combining individuals with different decision-making styles: explorers, who sample the available information and decision space in order to select the optimal decision, and exploiters, who commit to a course of action without such an analysis, but based on the prior success of that action. At the moment, there is no formal assessment of the decision-making styles of SCG members. Consequently, the likelihood that different SCGs will contain a combination of individuals with different decision-making styles is a matter of chance.
However, there has been no evaluation of the use of the JDM by SCGs in the UK. Without such an evaluation, there is no basis upon which to determine the utility of the JDM in (a) reducing decision inertia or the issues identified by Pollock (2013) or (b) harnessing the benefits and avoiding the pitfalls of group decision-making more broadly (Bang & Frith, 2017).

| S IMUL ATED MA JOR IN CIDENTS
To provide a detailed (real-time) analysis of the use of the JDM, we studied multi-agency groups who faced the same simulated major incidents. Previous studies of how multi-agency groups respond to simulated major incidents have made use of immersive simulation suites in which the groups are convened and respond to a virtual event (e.g., Crego, 1996;see also, Power & Alison, 2017a, 2017bvan der Haar, Koeslag, Euwe, & Segers, 2017; for a review, see Alison et al., 2013). In Study 1, this approach was adopted in order to investigate the use of the JDM by 18 SCGs who were responding to the same simulated incidents within a single critical meeting. The meetings were recorded, and the decision-making sequences, involving the five JDM activities, were derived from these recordings. Coding the sequences in this way provides a common frame of reference for

| Study 1
Eighteen multi-agency groups attended 2-day national training and exercise events in Study 1 (nine groups from Exercise Wales Gold 1 and 9 groups from Exercise Wales Gold 2; see Table 1 for additional information). The total number of participants in Study 1 was 147.
Each training event consisted of opportunity samples of participants who had applied and were selected by their agencies to take part on the basis of prior involvement in a SCG, having been on call to attend Electronic copy available at: https://ssrn.com/abstract=3605648 a SCG, or the possibility of future involvement in a SCG based on their rank. Participants were employed by the emergency services (Ambulance Service, Fire and Rescue Service, and Police), Health Boards, Local Government, Natural Resources Wales, and Public Health Wales, and ranged in seniority from Chief Executive level to a recently promoted Police Chief Superintendent. The experience of participants varied considerably: Some had served as a member of a SCG or had undertaken some relevant training to prepare them for such a role, while others had received no formal training for a role in SCGs. However, the groups were composed of participants who could, if an immediate need arose, be part of SCGs faced with a major incident. The participants from Study 1 provided informed consent for their participation in accordance with ethical approval through Cardiff University; and those in Study 2 (from Exercise Unified Response, EUR) provided informed consent through their agencies.

| Study 2
The participants in the SCGs in Study 2 represented the emergency services (City of London Police, London Ambulance Service, London Fire Brigade, Metropolitan Police), Civil Contingencies Secretariat,

City of London Corporation, Department of Communities and Local
Government, Environment Agency, Transport for London, and London Underground Limited. They were selected on the basis of their seniority and experience. There was a core set of participants that attended each of the six SCG meetings, and there was some turnover in participants over the course of the exercise. For example, the Chair of the SCGs changed across the meetings: SCG meetings 1 and 2 (a male representative of the Metropolitan Police), and meetings 3-6 (a female and a male representative of the Local Government; see Table 1 for additional information). A majority of the participants in Study 2 participated to more than one SCG meeting; but the total headcount (including multiple attendances by the same person as part of the count) was 142.

| Study 1
At the start of each training event, all participants took part in an interactive session that lasted for approximately one hour. This session covered the consistent and nationally recognized UK emergency command control and coordination structures (Cabinet Office, 2013), the decision-making role of SCGs, and the use of the nationally endorsed model for making collective decisions in an emergency (i.e., the JDM). The virtual timeline of the scenarios extended from the afternoon of Day 1, when the incident was declared to 3 months later through intermediate time points (evening of Day 1, Day 2, and 1 week later).

| Hydra simulations
The Study 1 scenarios were delivered using Hydra immersive simulation systems (Alison et al., 2013;Crego, 1996). Hydra provides a "syndicate room" for each group, which was equipped with a large screen projector, PC, wireless keyboard and mouse, printer, and CCTV. Large posters of the JDM were displayed prominently on the wall of each of the syndicate rooms and were visible to all participants. The PC ran a communication interface that was permanently displayed on the projector screen and delivered information updates ("injects") and tasks to the groups. Exercise control staff also received and responded to all written communications sent by the groups using the Hydra communicator. GoPro cameras (GoPro Hero 3, Half Moon Bay, USA) and CCTV were used to record meetings in Studies 1 and 2.

| Study 1 scenarios
Both scenarios were multi-faceted, dynamic, and involved time pressure. The scenarios were developed so that there were no explicitly correct or incorrect critical decisions and that all agencies would be engaged. They were managed by control room staff who delivered scripted updates on the scenario at pre-defined times via tasks, video and audio clips, and printed documents. The first scenario began with a video update from the Police tactical commander. This update stated that there was a large-scale chemical fire at an industrial site and the nearby road network and railway line were closed due to the resulting plume of smoke. The scenario then developed into a serious environmental and economic incident with media impacts that required decisions on longer-term recovery issues regarding health impacts, housing, decontamination, and economic recovery. The second scenario also began with a video update from the Police tactical commander, which included information about a crash between a passenger train and a truck carrying a hazardous substance. The crash caused many fatalities and injuries to passengers. Within the first hour of the incident, a fire ignited, burning the hazardous substance and sending a toxic plume of smoke over a residential area. Across the two days of the event, the groups took part in meetings that were approximately 45-60 min, during which they made decisions in response to the evolving incident. The analysis for Study 1 was conducted on the critical second SCG meeting. In this meeting,  Note. The mean number of participants in each group together with the range, and the ratio of female to male participants.

| Coding of activity
The audio-video recordings were coded using the categories of activity from the JDM: gather information and intelligence (e.g., "We need more information about how injuries were sustained"); assess risks and develop a working strategy (e.g., "What is the risk of evacuating people?"); consider powers, policies, and procedures (e.g., "Can we control the airspace to legally prevent helicopters from taking footage?"); identify options and contingencies (e.g., "If we move people out and the fire burns for two weeks where will they go?"); and take action and review what happened (e.g., "Initiate mutual aid plan"). These activities were coded directly from the videos, and the codes were accompanied by notes on the same spreadsheet, which described the content of the coded activities. The activities were scored at the level of the group, independently of the individual. The fact that the meetings were chaired meant that the meetings had a coherent structure that was readily coded as a sequence of activities. Isolated comments that were either irrelevant or were not part of the discussion (e.g., informal asides, which were infrequent) were excluded from the analysis. The coding was conducted on two separate occasions (by B.W.), which resulted in a small number of the activities (<5%) being re-classified. An independent assessor (R.C.H.) then confirmed the reliability of the coding on a sample of 30 observations from each study (≈95% agreement with B.W.). A lag sequential analysis (Sackett, 1979;see also, O'Connor, 1999) was used to derive the primary data of interest: the sequences of transitions between different decision-making activities in the group meetings. In this analysis, the decision-making activities were coded as a continuous stream, with repetitions of the same category removed. The lag sequential analysis stopped at the end of the SCG meetings.
The resulting sequence of decision-making transitions was then compared to the binary transitions within the JDM (Figure 1).

| Study 1: What is the distribution of decisionmaking activities during independent SCG meetings?
We first considered the frequency with which a given category of activity occurs without consideration of what happened before or after that category of activity occurred. The overall frequencies are shown in Figure 2 in the form of stacked columns. The leftmost column shows the mean number of each of these categories for the 18 groups in Study 1. Three categories of activity dominated: gather information, develop strategy, and take action; while the frequencies of the remaining categories (consider powers and identify options) were low. These frequencies were analysed using a mixed ANOVA, with activity (e.g., gather information) as a withinsubjects factor (with five levels), and study as a between-subjects factor (with two levels: Exercise Wales Gold 1 or 2). This analysis confirmed that there was a main effect of type of transition (F(4, 64) = 72.97, p < 0.001, η 2 p = 0.82), no effect of Wales Gold 1 or 2 (F(1, 16) = 2.74, p = 0.12, η 2 p = 0.15), and no interaction between these two factors (F < 1). Pairwise comparisons, with a Bonferroni correction, confirmed that there were significant differences between each pair of activities (ps < 0.005) with the exception of between gather information and take action (p > 0.50). The pattern of statistical significance was as follows: develop strategy > gather information = take action > identify options > consider powers.

| Study 1: Is the sequencing of decision-making activities consistent across SCGs?
The analysis of the results from Figure 2 showed that there was con- Electronic copy available at: https://ssrn.com/abstract=3605648 groups. The mean frequencies of transition from one category to each of the other categories are shown in Table 2, together with the range of frequencies. Inspection of this table shows that the number of transitions involving the three main categories (gather information, develop strategy, and take action) were higher than those involving the other categories (consider powers and identify options).
This simply reflects the fact that these categories of activity were more frequent. Of more interest, is the fact that there was marked variability in the frequency of transitions involving the three remaining categories (i.e., gather information, develop strategy, and take action) across the different groups; as is evident from the ranges (shown in brackets). For example, while the mean number of transitions from gather information to develop strategy was 5.39 the range was between 2 and 10 transitions across the groups. The basis of this variability is explored in the next section.
A principal components analysis (PCA) assessed whether the variability in decision-making sequences was simply noise or had some underlying structure. PCA is a data reduction technique that identifies interrelationships (i.e., structure) between a set of variables, and through this process reduces the set to a smaller number of variables (called components or classes). For the three dominant categories (gather information, develop strategy, and take action), there are six possible binary transitions (e.g., develop strategy->take action).
For each of the 18 groups, the frequencies with which each of the six transitions occurred are the primary data: a matrix of numbers with 18 rows (one for each group) and six columns (the possible sequences). PCA examines the extent to which any of the six sequence types (the columns) can be combined because the values in them are correlated. If the variability identified in Section 6.2 was noise, then this would be evident as the values being randomly arranged across the columns and no structure would be revealed by PCA. In fact, the PCA (which converged in three iterations and used a varimax rotation and Kaiser normalization) revealed two classes with eigenvalues of >1: Class 1 can be labelled "action-oriented" and involved four of the transitions: develop strategy->take action, take action->develop strategy, take action->gather information, and gather information->take action (all factor loadings > 0.76). Class 2 can be labelled "information-oriented" and involved the two remaining transitions: gather information->develop strategy and develop strategy->gather information (both factor loadings >0.95; there were no cross class loadings > ±0.13). That is, the variability in the six sequences could be reduced to two components or classes, labelled "action-oriented" and "information-oriented." These two transition classes accounted for 74% of the variance in the six transitions between the three activities. A schematic that presents the action-oriented and information-oriented classes of transition is depicted in Figure 3. 1

| Study 2: What is the distribution of decisionmaking activities across successive SCG meetings?
The overall distribution of activities across each of the six SCG meetings in Study 2 is also shown in Figure 2 (SCGs 1-6 from Exercise Unified Response). The distribution of the categories in these meetings was similar to those in the left-most bar (i.e., Study 1). In spite of the differences in the number of participants in the groups in Studies 1 and 2, and the different scale of the exercises,

Gather information Develop strategy Consider powers Identify options Take action
Gather information  ps < 0.01); and there were no differences between the frequencies of transition involving the activities within each of these two sets (i.e., gather information = develop strategy = take action > consider powers = identify options).

| Study 2: Does the sequencing of decisionmaking activities change across successive SCG meetings?
The analysis of the variability in the sequencing of decision-making activities that was performed in Study 1 cannot be conducted on the results of Study 2, because only one SCG was making decisions at a given time point, as the exercise developed. However, a complementary analysis can be conducted that allows the development of the transitions between the three dominant activities to be tracked across successive meetings of this large-scale exercise. Figure 4 depicts this evolution in the frequencies of transitions from: gather information to either develop strategy or take action (upper panel); develop strategy to either take action or gather information (middle panel); and take action to either gather information or develop strategy (lower panel). Inspection of the upper panel shows that during each SCG, gather information was more likely to be followed by develop strategy than take action (binomial test, p < 0.05). The middle panel shows that during each SCG, develop strategy was more likely to be followed by take action than gather information (binomial test, p < 0.05). Finally, the lower panel shows that while in the first SCG, take action was more likely to be followed by develop strategy than by gather information, by the final SCG this pattern of transitions had reversed and take action was more likely to be followed by gather information than develop strategy (Fisher's exact probability test, p < 0.05). The final observation shows that the sequencing of decision-making activities changes over the life of an extended incident.

| SUMMARY AND PRIN CIPAL RE SULTS
Strategic Coordinating Groups play a central role in how the UK responds to major incidents. A pillar of UK national operational guidance is the JDM. We investigated the use of the JDM in multi-agency groups in simulated major incidents created in Hydra suites (Study 1) and in a large-scale live-play exercise (Study 2). Decision sequences were generated by first categorizing group activities within the meetings in terms of the five categories that form the basis of the JDM (Figure 1), and then examining the transitions between these categories. There was a consistent distribution of the five activities across the different groups, with three dominant activities (gather information, develop strategy, and take action), and two categories of activity that were relatively infrequent (consider powers and identify options). However, this consistent distribution belied marked between-group differences in the nature of the transitions between the dominant activities. There were two classes of decision-making transitions involving (a) develop strategy and take action, and take action and gather information, and (b) gather information and develop strategy. To give a concrete illustration, two SCGs might have a similar overall number of transitions involving developing a strategy, but for one group these based on transitions to-and-from take action whereas for another they might be based on transitions to-and-from F I G U R E 4 Study 2. Frequency of transitions from gather information to either develop strategy or take action (upper panel); from develop strategy to either take action or gather information (middle panel); and from take action to either gather information or develop strategy ( Electronic copy available at: https://ssrn.com/abstract=3605648 gather information. Finally, the analysis of a series of meetings across an extended incident showed that our form of analysis was sensitive to changes in the distribution of decision-making activities across the event. Most notably, while taking action was most likely to be followed by revisiting strategy development in SCG meeting 1, it was most likely to be followed by gather information in SCG meeting 6. Further work involving extended incidents will be needed to understand the basis of this change, and whether it is a consistent feature of different groups or is itself subject to variability between groups. For example, the change in decision-making sequences between SCG meetings 1 and 6 might reflect the changing nature of the issues faced early and late in a major incident, or an increasing need for gather information about the immediate (or ongoing) consequences of actions. In the following discussion, we focus on three facets of our studies: The variation in how the groups approached the simulated incidents (Study 1); the fact that there was limited consideration of powers or identification of options in all groups (Studies 1 and 2); and the nature of the methodology employed here. We conclude by examining the implications of our results for policy, guidance and training.

| D ISCUSS I ON
Our investigation provided the first analysis of the use of the JDM to support decision-making in multi-agency groups. The groups were faced with realistic, simulated major incidents. We did not evaluate the effectiveness of the different groups, including the decisions that they made (cf. van der Haar et al., 2017), but rather the process of decision-making (cf. e.g., Klein, 1993Klein, , 2003Klein, , 2008. The findings summarized in the immediately preceding section have clear theoretical and operational significance.

| Between-group variability in transition sequences
There were marked between-group differences in the transitions between the three main categories of decision-making activity. The groups differed in the extent to which their sequences were "actioncentred" or "information-oriented." These labels are not intended to describe a process of decision-making, but are simply a convenient and theoretically neutral way of labelling the different classes of decision-making sequences that were evident across the groups. The basis for these group differences cannot be determined from the present results, but some speculation can be offered. For example, thematic analysis of interviews with members of multi-agency groups, after they have participated in simulated major incidents, revealed high levels of decision inertia (Power & Alison 2017a, 2017bAlison et al., 2015; see also Janis, 1972Janis, , 1982Janis & Mann, 1977). In Study 1, decision inertia might be reflected in high levels of the information-oriented component and low levels of the action-oriented component. Our results thereby complement those based on thematic analysis of interviews with individual group members; but they also suggest that decision inertia differs markedly across different SCGs that are responding to the same incident. The same form of argument can be made about a given group's tolerance for uncertainty or ambiguity, which might be reflected in their tendency to exhibit frequent transitions between gather information and develop a strategy (e.g., Frenkel-Brunswik, 1949;see also, van den Heuvel et al., 2014; for reviews, see Furnham & Marks, 2013;Hillien et al., 2017). These group differences, whether they involve decision inertia or tolerance of uncertainty, are likely to be based upon differences in the composition and characteristics of the groups or the disposition of the chair (van der Haar et al., 2017). In this context, a recent analysis of the benefits and pitfalls of group decision-making is directly relevant. Bang and Frith (2017) highlighted the possibility that differences in the tendency of individuals within a group to explore or exploit information could affect that group's overall tendency to continue to gather information rather than to act on the basis of existing information (Frank, Doll, Oas-Terpstra, & Moreno, 2009;Tversky & Edwards, 1966). It seems plausible to link the information-oriented and action-oriented classes of decision-making sequences to group differences in exploration and exploitation: Groups with many explorers being more likely to exhibit information-oriented sequences and those with many exploiters being more likely to exhibit actionoriented sequences. To test this analysis would require the decision-making styles of the individuals to be assessed prior to their allocation to groups: Would groups that are composed of explorers or exploiters (or a mixture of the two) differ in their decision-making sequences, and if so would these differences impact on critical decision-making? We will return to this issue in the final section of the discussion where we offer some specific suggestions about how the issue of the variability in the sequencing of decision-making activities could be addressed in future policy, guidance, and training.

| Limited consideration of powers or identification of options
A consistent feature of the SCGs was the limited consideration of powers or identification of alternative options. This finding might simply reflect that there was a shared understanding of the available powers for a given situation. However, this explanation is much less plausible in the case of the failure to identify different options. In Study 1, the number of actions taken were many and varied (e.g., to evacuate buildings, to activate the mass fatalities plan), and yet there was little consideration of alternative options.
Moreover, in Study 2 the potential for identifying different options across the developing incident was manifold, and yet there was little or no attempt to do so in the six SCG meetings. This failure to explicitly identify different options has also been observed in studies of individual decision-making, where decisions have been characterized as being experience-based (e.g., Gigerenzer, 2007;Shafir, 1994;Tversky & Kahneman, 1974) or recognition-primed (Klein, 1993(Klein, , 2003see also, Doya, 2008). The failure to consider alternatives has been associated with poor group decision-making (for a review, Walker, McLeer, & the DAMOCLES group., 2004). In the context of SCGs, the limited identification of options and contingencies could be generated in a variety of ways. For example, a need to maintain group harmony could serve as a constraint on the evaluation of alternative courses of action and contingencies (Janis, 1972(Janis, , 1982; and the extent to which the chair is perceived as inclusive and the environment psychologically safe might moderate the tendency of individuals to contribute alternative views or speak up (e.g., Bienefeld & Grote, 2014). There are a variety of ways in which encouraging the explicit consideration of alternative options could be addressed in future policy, guidance, and training, which will be considered in the final section of the discussion.

| Methodological considerations
Some of the results presented in the previous paragraphs were enabled by the fact that there was a relatively large sample of groups in Study 1 (i.e., 18) engaged in the same scenarios. However, one cost of the increased reproducibility that the use of simulations affords is that they lack some of the features of real major incidents; including the fact that group decisions at real major incidents have consequences (e.g., saving lives, preventing further casualties and damage to property and the environment). This is a limitation of Study  Figure 2). This similarity-across exercises of very different scales-supports the view that our results are of relevance to real major incidents; but complementary analyses of real SCG meetings, dealing with a range of different major incidents, is the only way to determine whether or not this view is accurate.

| Implications for future policy, guidance, and training
With the limitations noted above borne in mind, the results from Studies 1 and 2 do provide a context-sensitive basis upon which to develop future policy, guidance, and training. One obvious target is to modify policy and guidance to ensure that (a) options and contingencies are consistently explored, and (b) the rationale for prospective courses of action are routinely and explicitly assessed against goals, anticipated consequences, and a risk/benefit analysis (cf. . Cohen-Hatton and Honey (2015) showed that training firefighters to use such explicit assessments (which they called "decision controls"), before committing to a course of action, increased the use of reflective decision-making relative to recognition-primed or intuitive decision-making. The implementation of such decision controls in a group decision-making context might also yield greater reflective decision-making, involving appropriate consideration of alternative options and goals.
However, the between-group variation in decision-making processes, coupled with the infrequent evaluation of options and contingencies, also highlights a need to consider the dynamics of group working rather than the development of prescriptive models of decision-making per se. In fact, there are relatively simple techniques (e.g., considering views from outside of the group; Janis, 1972Janis, , 1982 that enhance the quality of group decision-making in some contexts (Lovallo & Kahneman, 2003;Ministry of Defence, 2013; see also , Newell, Lagnado, & Shanks, 2015), and which could be integrated into how multi-agency groups respond to major incidents (cf. Exercise Unified Response Evaluation Report, 2017, pp. 142-143). However, there is another potential route to changing group dynamics that is based on a recent theoretical analysis of group decision-making that was briefly mentioned above. Bang and Frith (2017) presented an analysis of how the past experience of group members could be integrated with new information to affect group decisions. Their (Bayesian) analysis is broadly consistent with the naturalistic decision-making approach, which is also concerned with how previous experience primes decisions in the face of uncertain information (i.e., recognition-primed or intuitive decision-making; Klein, 1993Klein, , 2003Klein, , 2008; see also Doya, 2008;Gigerenzer, 2007;Gureckis & Goldstone, 2006;Salas, DiazGranados, & Rosen, 2010;Tversky & Kahneman, 1974). In the case of multiagency groups, bringing together representatives from the relevant agencies might well increase their ability to work together effectively (but see, Power & Alison, 2017a, 2017b. However, Bang and Frith (2017) argued that group decision-making might also benefit from the combination of different types of individual decision maker: specifically from the combination of explorers and exploiters. There are clearly pitfalls associated with being either an explorer (who might not reach a decision in a timely fashion) or an exploiter (who might quickly reach the wrong decision), and Bang and Frith (2017) reasoned that "A mixture of such diverse individuals can create advantages for the group." As already mentioned, this claim needs to be assessed experimentally, but it has clear practical implications for assembling effective groups in a variety of contexts, including at major incidents (cf. Polzer, Milton, & Swann, 2002;Roberge & van Dick, 2010). At present, the selection of individuals that come together to respond to major incidents (simulated or real) in the UK is not based on any formal assessment of their individual approaches to decision-making. The foregoing analysis suggests that such selection might provide a means of increasing the consistency and efficacy of decision-making processes in multi-agency groups.
To conclude: The JDM is a central to the UK public sector. The results of Studies 1 and 2 suggest that future policy, guidance, and training should focus on ways to enable the JDM to be used more consistently and effectively by SCGs. Greater consistency could be achieved by ensuring that SCGs include a balance of individuals with different decision-making styles (i.e., explorers and exploiters; see Bang & Frith, 2017), which might also reduce decision inertia ( Electronic copy available at: https://ssrn.com/abstract=3605648 to investigate whether the decision-making processes of SCGs, and groups more generally, are affected by the decision-making styles of their members. At a more specific level, our results suggest the need for SCGs to give more consistent and explicit consideration to alternative plans of action. The use of decision controls has proven effective in modifying decision-making in incident commanders , but this technique has yet to be formally assessed in SCGs or other groups. The sample size for Study 1 (n = 18) is low for a PCA, but complementary analyses using simple correlations to assess relationships among the 6 sequences (for which the n is suitable) provided statistical support for the components derived from the PCA.