Exploring protective decision‐making in the context of impact‐based flood warnings

Previous research has found that users fail to comprehend flood hazard warning messages as well as the possible impacts of the forecast events. A proposed way to improve understanding and uptake is the implementation of impact‐based warning services. However, even though extensive qualitative research has been done on the provision of these impact‐based warnings (IBWs) for different types of hazards and extreme events, there is still little scientific evidence that the additional information on impacts in flood warnings positively affects decisions made by the general public. The research reported here explores whether including messages of potential impacts increased the general public's likelihood of making a protective decision when provided with a flood warning. Nine hypothetical flood scenarios were presented to participants who were then asked to score their likelihood of making a specific protective decision. Participants were either presented IBWs or hazard flood warnings (HWs). Results show that IBWs indeed led to higher likelihoods of participants making a protective decision than HWs. Interestingly, we found that key factors identified in previous studies as influential elements in the decision‐making process had little impact on their decision. These factors include gender and previous experience with flood warnings.


| INTRODUCTION
Early warning systems are a critical element for flood risk reduction (World Meteorological Organization, 2018).If properly designed and disseminated, flood warnings can empower citizens and communities at risk by enhancing their preparedness and their response capacity during a flood event in order to reduce the possibility of personal injury or loss of life (Verkade, 2015;World Meteorological Organization, 2018).In past years, the national meteorological and hydrological services have made significant advances on their capacity and precision to forecast events in terms of the hazard and its associated impacts (Weyrich, Scolobig, Bresch, & Patt, 2018).Nowadays, forecasts have higher accuracy and provide longer lead times, allowing citizens at risk to take appropriate protection actions with sufficient time (Aldridge, Gunawan, Moore, Cole, & Price, 2016).Nevertheless, human casualties remain high (Morrow & Lazo, 2015).The reported situations where citizens did not react to warnings keep increasing, regardless if timely warnings were issued in the area (National Academies of Sciences, Engineering, and Medicine, 2018;Weyrich et al., 2018;World Meteorological Organization, 2015a).
Warnings are not received "passively" by individuals; they are processed, interpreted and evaluated according to their personal and socioeconomic context (Mileti, 1995;Parker, Priest, & Tapsell, 2009).Previous works suggest that socio-demographic factors, such as gender, and knowledge characteristics about the hazard may also influence an individual's protective behaviour.However, others have found contradictory results.Casteel (2016) reported that background knowledge of tornados and severe weather had no impact on the individual's decision to evacuate, whereas Ripberger, Silva, Jenkins-Smith, and James (2015) and Parker et al. (2009) found different results.The later linked people's inappropriate response to their level of knowledge, and their perception of the different flood risk elements, precisely, but not limited to, the individual's understanding of the content of the flood warnings message.Previous research has also shown that citizens are more likely to believe and respond to a warning if they understand what the warning is trying to communicate and how it personally affects them (Morss et al., 2016;O'Sullivan et al., 2012).
Receiving an official flood warning is not enough to guarantee a response from individuals (Casteel, 2016).Stakeholder consultation has found that some citizens fail to understand water hazard messaging, especially the overly technical terms currently used in warnings to express risk and flood magnitudes (National Oceanic and Atmospheric Administration, 2015;O'Sullivan et al., 2012).Flood warnings must be written in terms that can be easily understood by a non-technical audience to support citizen's understanding and self-protective behaviours (Casteel, 2016;Mileti, 1995).
A solution that has been proposed to overcome these challenges in risk communication is the implementation of impact-based warnings (IBWs).In this study, all warnings that contain information about potential impacts on a specific sector due to floods are IBWs.These warnings have been actively promoted by various organisations, such as the World Meteorological Organization, as a promising solution to address this problem, on the assumption that they generate more protective responses (World Meteorological Organization, 2015a).However, limited experimental research exists that allows for verification of this assumption.
To the knowledge of the authors, at the moment, there is little scientific evidence that IBWs do increase the likelihood that the public will take a protective action during a flood situation.The present research aimed to address this gap in an experimental form.

| BACKGROUND
IBWs come forward as a technique to improve user's understanding of warnings in order to help trigger protective responses from the public (World Meteorological Organization, 2015b).Previous research on IBWs has focused on the influence of the added impact information on an individual's intended behavioural response and warning understanding.Nevertheless, different approaches on how to design IBWs exist for different types of hazard.
In the United States, the National Weather Service (NWS) has released a product called "IBW" for tornados, thunderstorms and hurricanes (Morss et al., 2016); however, in the case of the NWS product, extreme consequence language such as "IF YOU LIVE IN AN AREA OF RISK AND YOU STAY, YOU MAY DIE" and "COMPLETE DESTRUCTION OF ENTIRE NEIGHBOURHOODS" is embedded in their warnings.Casteel (2016) and Ripberger et al. (2015) administered experimental surveys to discover if IBWs for tornados influenced the participant's likelihood of making a protective decision.In both experiments, they found that individuals were more likely to take protective action when presented with IBWs; yet, the two experiments yielded different results on how previous knowledge influenced the participant's decision.For hurricanes, Morss et al. (2016) performed a similar experimental study to analyse the text of the IBWs product, where they also found that the impact information increased the likelihood of evacuation.However, the warnings used in all three studies are based on impacts that have been exaggerated to increase the sense of danger.Warnings with overblown language may provoke defensive attitudes, including denial and reluctance of taking any action on real emergencies (Morss et al., 2016).Potter et al. (2018) and Weyrich et al. (2018) have also done similar studies on intended actions for thunderstorms and strong winds with a different approach to IBWs.Nevertheless, in these experiments, the added impact information on IBWs does not include extreme consequence language, and thus, this factor must be taken into account when analysing and comparing results between studies.Potter et al. (2018) reported that IBWs were easier to understand but did not find an effect on increasing actual protective behaviour, whereas Weyrich et al. (2018) reported an increase on improving intended behavioural response.Prior contradictory results on IBWs reaffirm the complexity of studying the individual's decision-making process that leads to intended behaviour and the importance of not generalising findings across weather hazards.Recommendations to effectively communicate flood risk do not necessarily transfer over to effective tornado risk communication (National Oceanic and Atmospheric Administration, 2015).As mentioned previously, no experimental work has been found where the influence of IBWs for flood hazards was examined, and the research presented here was designed to address this issue.

| APPROACH
Consistent with the presented work on IBWs, a roleplaying experimental survey was proposed as a method to explore protective decision-making in the context of IBWs across nine hypothetical flood scenarios.Previous research on IBWs and risk communication have used responses to hypothetical warning situations as a rough proxy for actual behavioural intentions to real warnings based on the theory of planned behaviour (TPB) (see Casteel, 2016;Morss et al., 2016;Potter et al., 2018;Ripberger et al., 2015).Proposed by Ajzen (1985), this theory postulates that a person's intention to perform (or not to) a behaviour is the immediate determinant of a real action.Therefore, based on prior work in the field, in this study, the participant's responses to the nine hypothetical flood scenarios were used as a proxy for actual behavioural intentions.

| Participants
Participants for the experiment were recruited using the crowdsourcing internet marketplace Amazon Mechanical Turk (MTurk) and limited to European nationals and residents.Four preliminary steps were taken to improve the quality and reliability of the survey data obtained from MTurk.Firstly, participants were prevented from taking the survey more than once.Secondly, the participants' approval rate qualification, which is the percentage of previous assignments submitted by the participant in the platform that have been approved, was set to be above 95%.Thirdly, time requirements were specified in the instructions to ensure participants spent the proper amount of time doing the experiment.Participants were encouraged to spend a minimum of 8 min on the experiment and submissions that were completed in less than 5 min were discarded.And fourthly, a control question was made with the intention of verifying that the participants were paying attention, a control mechanism proposed in several user guidelines related to MTurk (Black, 2016).Scenario 9, in which the participant assumed the role of a public parking lot costumer, was used as a control scenario.There was only one possible answer, zero, as the scenario's geographical setting does not match the target location of the warning.If participants failed the above control question, all their answers were discarded and thus, not included in the final data analysis.
Finally, to increase confidence in the reliability of the results provided by MTurk, control surveys were made in a supervised environment with students from the Institute for Water Education in Delft, The Netherlands (IHE Delft).The data obtained from this control group was, however, used exclusively for establishing the reliability of the full experiment, and was not included in the final analysis of results.

| Materials
Participants were exposed to the nine hypothetical flood scenarios presented in Table 1.The information provided in each scenario was divided into two paragraphs.The first paragraph introduces to the participants their current role and responsibilities.This section aims to increase the level of realism of the experiment by informing the participants that the role they are playing comes with important obligations.The second paragraph explains to the participants that they were are also responsible for deciding if a specific-scenario protective decision should be made in response to the flood warning they have just received, and that this decision could have both positive and negative consequences.All the participant's roles intentionally position them as being responsible for others during the flood scenario, thus introducing a sense of accountability for their safety.By means of an example, the instructions for the restaurant owner (Scenario 2) are shown below: You are the owner of a popular restaurant which is located in North Modesto.The weekends are your most profitable days.The safety of your employees is your responsibility as well as keeping the restaurant operating effectively at all times!In your role as owner, you have to decide if the restaurant should be closed in response to the flood warning you just received.This is an important decision since keeping the restaurant open or closed has both positive and negative consequences!On a scale of 0 to 10, where 0 means "I definitely WOULD NOT close down the restaurant" and 10 means "I definitely would close the restaurant," please indicate how likely it is that you would close down the restaurant based on the previous information.
Figure 1 presents the structure of the IBWs used for this study, divided into four sections: (1) general information, (2) hazard information, (3) impact information and (4) recommendations.The Hazard flood warnings (HWs) followed the same structure, excluding Section 3. The material, therefore, consisted of two groups of warnings, IBWs and HWs, which both included the nine hypothetical flood scenarios.Colours, flood risk matrices and warning tags are omitted from the warnings as the intent was to study the influence of the impact text, without the presence of other warning components.
The visual design of Sections 1, 2 and 4 was based on the layout of flood warnings issued by the NWS.However, not all components of the NWS warnings were used and the content inside each section followed the guidelines of previous work on IBWs (see World Meteorological Organization, 2015b, 2018).For the content of Section 3, the flood impact table for the general public used by the UK's Meteorological Office was chosen as a base.Since the scenarios in this study were not restricted to a real geographical location and the warnings were aimed at citizens, the UK's Meteorological Office flood impact table for the general public and their categories were considered appropriate for Section 3.
Finally, Section 2 included the probability of precipitation (POP) component.As seen in Table 1, POPs below 50% were intentionally coupled with high flood impacts to focus on the influence of the additional impact text in the likelihood of making a protective decision, even if the POP could be considered low by the participants.

| Procedure
The online survey application, Limesurvey, was used to code and publish the experimental survey online and collect the responses.In order to identify and group participants, questions such as gender and previous experience with flood warnings were asked at the beginning of the experimental survey.These two variables were later used to analyse the obtained data.
All participants were exposed to the nine hypothetical flood scenarios.However, they did not all receive the same type of warning.Participants were randomly assigned to one of two groups; IBWS or HWS.Splitting participants F I G U R E 1 Impact-based flood warning structure into two independent groups helped reduce the bias in the experiment and created a baseline for analysing and comparing the correspondent results between the two types of flood warnings.Since every participant received all nine scenarios for a particular type of flood warning, they were unaware of how the other type of warning was structured.
After reading the initial instructions, participants were randomly assigned to one of the two created groups and proceeded to read the nine hypothetical flood scenarios, which were presented in a randomised order.This last step helped decrease possible recency effects, where the participants select the most recently "acceptable" response option; and eliminate any underlying answer trend.For every scenario, participants entered a number between 0 and 10 to score their likelihood of making a specific protective decision.
At the end of the experimental survey, a unique completion code was generated in order to verify and approve the work done by participants coming from MTurk.After finishing the experimental survey, participants were directed to the official webpage of MTurk, where they provided their code.This last step allowed to link the participant's unique code to their experimental survey general information, such as completion time.After checking if the submitted experimental survey passed the established quality standards as explained in the participant's section, the work done by the participants was marked as "approved" in MTurk.

| RESULTS
The results obtained from the experimental survey were analysed using a two-way analysis of variance (ANOVA).Table 2 presents the two sets of analysis that were performed, in which gender, past flood warning experience and type of flood warning (IBWs or HWs), were chosen as independent variables to study their combined and individual interaction effects on the likelihood of making a protective decision (dependent variable).All tests for non-homogeneity of variance were non-significant (p > .05)as assessed by the Levene's test of equality of variance, as well as all tests for normality as assessed by Shapiro-Wilk's test (p > .05).
A total of 338 people responded to the experimental survey.Of those that participated, the majority were female (202) in the age range of 18-28 (132) and indicated that they had been exposed to a flood warning (210).In total, 169 participants were assigned IBWs, with an equal number being assigned HWs.

| Overall results
Figure 2 presents the mean likelihood, on a scale from 0 to 10, of participants indicating that they would take a protective action for all scenarios after reading IBWs and HWs.In overall, the participants presented with IBWs had a higher likelihood of making a protective decision than those exposed to HWs.However, the mean likelihoods of making a protective decision of participants are not constant across scenarios.Both types of warnings follow a similar trend, with highs and lows for the same hypothetical flood scenarios.This suggests that the mean likelihood of making a protective decision is also influenced by the type of scenario participants were presented within the experimental survey.Nevertheless, in all scenarios, IBWs generated higher likelihoods of making a protective decision than HWs.

| Test 1: Type of flood warning and flood warning experience
Previous experience with flood warnings was chosen as the first independent variable, alongside the type of  warning, to evaluate the influence on the participant's likelihood of making a protective decision.Results of the two-way ANOVA test show that the combined interaction effect of the type of warning and previous experience on the likelihood of making a protective decision was not found to be statistically significant; F(1, 334) = 2.570, ρ = 0.110, η 2 p = 0.008.Therefore, an analysis of the main effect of each individual independent variable was performed.The main effect for the experience was not statistically significant, F (1, 334) = 0.556, ρ = 0.456, η 2 p = 0.002.However, the main effect for the type of warning was found statistically significant F(1, 334) = 61.100,ρ = 0.000, η 2 p = 0.155.These data are shown in Table 3.
The above results can be better understood if the mean likelihoods of making a protective decision are interpreted with the help of Figure 3.It can be seen that IBWs generated higher mean likelihoods of making a protective decision than HWs, regardless of the participant's previous experience, further supporting that the type of flood warning effect on the dependent variable is statistically significant.
The difference in likelihoods between the experienced and non-experienced groups for each type of warning is relatively equal, which suggests that previous experience with flood warnings does not appear to have an influence on the likelihood of participants making a protective decision.Subjects with no experience were slightly more sensitive to IBWs than subjects with experience.However, in this study, the difference is not large enough to be found significant.

| Test 2: Type of flood warning and gender
Gender was chosen as the second independent variable, alongside the type of flood warning, to study their effects on the participant's likelihood of making a protective decision.Similar to Test 1, the combined interaction effect between type of flood warning and gender on the likelihood of making a protective decision was not found to be statistically significant, F(1, 333) = 1.347, ρ = 0.247, η 2 p = 0.00.Therefore, gender and type of warning were analysed separately to determine if one of the two chosen variables had any individual effect on the participant's likelihood of making a protective decision.As seen in Table 3, from the two variables, again, only the type of warning was found to be statistically significant F (1, 333) = 51.35,ρ = 0.00, η 2 p = 0.140.The same pattern of responses seen when analysing the previous experience variable is present with gender.As seen in Figure 4, IBWs generated higher likelihoods of making a protective decision, irrespective of the gender.Both female and male participants had an almost equal mean likelihood of making a protective decision regardless of the type of flood warning presented to them.The above results suggest that the participant's gender does not have an influence on the likelihood of making a protective decision.For this study, between the two chosen independent variables, only the type of flood warning has a significant effect, and gender does not.

| DISCUSSION
This study was designed to answer the question: Do Impact-Based flood warnings increase the likelihood that citizens will take more protective decisions in potentially Across all hypothetical flood scenarios, the additional impact information provided in the IBWs influenced participant's behaviour towards a higher likelihood of making a protective decision in response to the warning received, regardless of previous knowledge and gender.One of the most common assumptions done in the field of forecasting and warning services is that citizens will take protective actions if presented with timely and accurate hazard information (Morrow & Lazo, 2015).However, previous research in the risk communication field suggests that this is not enough.A critical question in risk communication should be "Do people get the message and understand what it means to them?" Warning clarity directly influences the understanding stage of the public's warning response process; if a warning is understood by the users, the likelihood that they would make a protective decision increases (Mileti, 1995).In the case of IBWs, these services aim to influence the public's response by translating the common technical language used in flood warnings into clear potential impacts users may experience due to the expected hazard and thus, help them undertake more protective decisions (Global Facility for Disaster Reduction and Recovery, 2016).In this study, the additional impact text included in the IBWs increased the likelihood that users would take more protective decisions, thus providing support and validation to the fundamental objectives behind IBWs.
Previous experience and gender were chosen as independent variables to explore, alongside IBWs, their influence on the participant's likelihood of making a protective decision.These two variables were selected due to previous, substantial and contradictory, research on the influence of these parameters in the decisionmaking process of citizens at risk (see Casteel, 2016;Potter et al., 2018;Ripberger et al., 2015;Weyrich et al., 2018).However, the primary objective of this research was to study IBWs, and thus, these two variables are discussed in an exploratory manner.
The empirical findings on the influence of past experience are mixed, suggesting that it can have a positive, negative, or no influence on protective decisions (Morss et al., 2016).How past experience influences participant's protective decisions are determined by many different aspects, such as the recency and severity of events, if property damage was experienced and if evacuation was needed (Morss et al., 2016;Parker et al., 2009).Interestingly, previous experience with flood warnings was found to have little impact on the subject's decision of taking protective action.Flood warnings for significant severe events are not a common daily experience.Users might, therefore, be exposed to such warnings once or twice in their lifetime (Parker et al., 2009); thus, due to the rarity of these events, even if a person has had previous experience with flood warnings, they can hardly be considered to be experts in how to interpret the information.Casteel (2016) and Weyrich et al. (2018) present similar results for IBWs, where background knowledge and experience were also not found to influence the participant's decision to take a protective decision.This result is encouraging as it validates the results presented here and further suggests that the additional impact text that is included in the IBWs provides clearer and more understandable information, regardless of the participant's previous experience.
Gender is a complex socio-demographic factor which could influence the decision-making process in dangerous situations.Previous research on gender is, however, inconclusive.Studies on IBWs that have also studied gender, like Potter et al. (2018), Morss et al. (2016) and Ripberger et al. (2015) have found that females are more likely to take a protective decision, whereas Weyrich et al. (2018) recent research concluded that gender does not have an influence on the decision-making processes.Social characteristics such as socioeconomic and work status, household composition and partnership status to name a few, are some potential explanations for gender differences in protective decision-making (Bateman & Edwards, 2002).Detailed characteristics regarding the participant's social context were, however, not asked in this experiment, as it was outside the scope of this study and thus, gender and its interaction with the type of warning is discussed only in an exploratory manner.The above decision may explain why the results did not reflect the existence of any potential influence of gender in this research.Regardless of the participant's gender, IBWs did generate a higher likelihood of making a protective decision than HWs, with gender having no effect on their final scores.The context in which an individual receives a warning can also influence their response behaviour.As seen in Figure 2, both types of flood warnings do follow almost the same response trend, displaying highs and lows for the same scenarios.The lowest likelihoods of making a protective decision are found for the Scenarios 2 and 6, where the participant takes on the role of a restaurant owner or a taxi driver.These were the only ones where the role was one of ownership, with responsibility for their daily income.The results suggest that when participants are accountable only for their own safety and income, they would rather take the risk of being flooded than the risk of losing part of their daily revenue.This phenomenon observed in this study can also be seen in real life.The ride-hailing company Uber, which offers a form of taxi service, was accused in 2014 of increasing their tariff during extreme weather events, such as Hurricane Sandy (see Luckerson, 2014; State of New York Office of the Attorney General, 2014).Instead of cancelling the service and taking shelter, more Uber cars were deployed, and these charged extra due to the increase in demand.Participants had a similar behaviour when presented with the taxi driver scenario, which had the lowest likelihood of making a protective decision of all the scenarios irrespective of the type of warning participants were presented with.As a result of this behaviour, drivers of motor vehicles have been identified as an important target audience for their inappropriate and dangerous attitude during flood events (Drobot, Benight, & Gruntfest, 2007;Parker et al., 2009).Ruin, Gaillard, and Lutoff (2007) found that motorists have a higher tendency of underestimating the flood risk which can lead to a higher likelihood of warnings being ignored or dismissed.As in real life, the participants seem to have engaged in similar dangerous attitudes liked the ones linked to motorists; however, IBWs still generated higher likelihoods than HWs in these scenarios.
Besides the type of scenario and roles, another factor that could have influenced the variation of likelihoods in Scenarios 2 and 6 is the POP inside the hazard information section of the flood warnings.These two scenarios had the lowest POP with 25% and 35%, respectively, and yielded the lowest likelihoods of making a protective decision.The scenarios with the highest POP (Scenarios 7 and 8, each with 60%), had the highest overall likelihoods.These findings are similar to those presented by Demeritt (2012) in which users reported that they would take a wait-and-see approach and not react to a flood warning unless the POP of an event is very high (above 50%) whereas Priest et al. (2011) defined this threshold closer to 70%.
Therefore, because of these two possible influences, an internal analysis within each scenario was made to isolate the effect of the additional impact information on the participant's likelihood of making a protective decision.Besides the impact text, all other sections within the hypothetical flood scenarios were kept the same in the experiment, including the hazard section and the POP.As seen in Figure 2, in Scenarios 2 and 6, which have a POP lower than 40%, IBWs generated a mean likelihood of making a protective decision above 5.0, almost 6.0, whereas HWs scores for the same scenarios are close to 4.0.The results of this study for HWs support the findings of Demeritt (2012), with participants being less likely to make a protective decision when the POP is low, with a likelihood almost 1.7 below their IBWs scenario counterpart.This variation in likelihoods between the two types of warnings further suggests that IBWs influenced participant's behaviour towards a higher likelihood of making a protective decision, even when the POP was below 50%, which is considered as a potential wait-and-see limit in previous studies (see Demeritt, 2012;Priest et al., 2011).Additionally, in the scenarios with a POP above 50%, participants in the IBWs had on average a likelihood of 1.2 higher than when exposed to only HWs.These results suggest that regardless of the scenario and their associated POP, IBWs are found to increase the likelihood of making a protective decision.

| LIMITATIONS
The current experimental survey uses behavioural intentions as a proxy for actual intentions based on the TPB.Previous research on IBWs (see Casteel, 2016;Morss et al., 2016;Potter et al., 2018;Ripberger et al., 2015;Weyrich et al., 2018) have all used behavioural intentions to infer people's actual reactions towards warnings.However, as stated by Ajzen (1985), TPB only predicts behaviours on which the individuals (a) believe they have the necessarily abilities and resources to successfully carry them out and (b) have a high amount of control.Selfefficacy or the belief about one's ability to successfully perform an activity, and evacuation barriers during flood emergencies may inhibit people from evacuating in real situations (Morss et al., 2016).Therefore, it can be argued that intentional responses may differ from actual responses in real life situations in which individuals might not know how or are unable to react (Jonkman, Maaskant, Boyd, & Levitan, 2009;Parker et al., 2009;Yoe, 1994).Nevertheless, psychological research has shown that there is a statistically significant correlation between actual and intended behaviours (Armitage & Conner, 2001).Ripberger et al. (2015) demonstrated that behavioural intentions are a decent proxy for actual behaviour in their IBWs tornado study.Although the results are based on behavioural intentions as a proxy, they can help to demonstrate general relationships between the main chosen variables, type of warning and the user's likelihood of making a protective decision, and create a foundation for future research made on real events experienced by the participants.
In the context of this research, a valid observation is, What is considered an "appropriate protective reaction" during an emergency?This is a subjective matter, as a warning can generate a protective reaction but not necessarily an effective one, depending on the situation.For example, an IBW may be issued for an upcoming severe storm event; but what if this results in a false alarm?Residents in an area would be mobilised, and emergency measures would be put into place for an event that did not happen.It can be argued that even though the warnings did generate a reaction in this context, the outcome is not effective or appropriate from an economic point of view, as resources were allocated for unnecessary emergency actions.However, in this study, the focus was to study if the impact text increased the overall likelihood of users making protective decisions to secure themselves and their property (e.g.avoid areas where flooding is expected) regardless if the forecasted event results in a false alarm.
Not precisely a limitation, is the usage of MTurk as a crowd-sourcing resource for data collection.MTurk is considered one of the most relevant crowdsourcing markets used by researchers; approximately 15,000 papers containing the word "Mechanical Turk" were published between 2006 and 2014 (Chandler & Shapiro, 2016).The survey was disseminated through this online platform so it could reach more people in different locations, optimising time and money resources.Research on MTurk indicates not only that the data gathered from "MTurk workers" is as trustworthy as those obtained from traditional sampling methods (e.g., recruiting volunteers at institutes), but most importantly, that the quality of the data meets or surpasses the psychometric standards (validity, reliability/errors of measurement, fairness in testing), requisite to published research (Buhrmester et al., 2011;Chandler & Shapiro, 2016).However, using online platforms for data collection limits the researcher's control on external factors that could negatively impact participant's responses.In this study, participants were anonymous "workers" from an online platform and took part in an unsupervised environment, and thus, it was not possible to ensure that other external variables, such as noise or constant interruptions, were not influencing their performance.Consequently, to improve the reliability on the data collected, the most habitual procedures for MTurk are (a) to include an Attention Check Question to either increase the attention from whom is responding to the survey, or to omit the responses from the inattentive ones (see Aust et al., 2012;Buhrmester et al., 2011;Downs, Holbrook, Sheng, & Cranor, 2010;Oppenheimer, Meyvis, & Davidenko, 2009) and (b) to restrict sampling to high-rated workers (above or equal to 95%) to guarantee good enough scales of reliability, socially desirable responses, central-tendency bias, and the replicability of known effects (Peer, Vosgerau, & Acquisti, 2014).Both of these procedures were applied during the data collection phase.Additionally, control surveys in a supervised environment were carried out to compare the results between the two groups.The results from this control group followed a similar pattern as the ones obtained by MTurk, providing further confidence in the reliability for the entire experiment.
Lastly, the main goal of this research was to study the influence of the additional impact text on user's likelihood of making a protective decision and thus, previous experience and gender were explored in less depth and detailed than the two main variables.To better understand the role of IBWs in emergencies, future research should focus on how relevant factors in the risk communication field influence user's protective decision-making in the context of IBWs across different types of communities and societies.

| CONCLUSIONS
Assessing what a user needs in order to take relevant protective decisions during a potentially dangerous situation is a complex task.If addressed incorrectly or incompletely, it can create a gap between what forecasters and officials think citizens need and what citizens actually need from a flood warning in order to increase their likelihood of undertaking protective actions.IBWs have been promoted as a promising solution to address this gap but, Do they actually generate more protective responses than HWs? Role-playing inside an experimental survey was used as a method to explore the likelihood of participants making protective decisions in hypothetical flood scenarios when presented with IBWs versus HWs.The results presented here demonstrate that IBWs generate higher likelihoods of making a protective decision than HWs in potentially dangerous flood situations.The additional impact text used in the IBWs of this study provided sufficient discriminate information to influence participant's behaviour towards protective actions, regardless of the participant's previous experience with flood warnings, type of scenario and gender.
Finally, the results can provide important quantitative support and validation to the work already done on the field of impact-based flood forecasting and warning services and open the door for further discussion and future re-evaluations towards developing a multi-hazard system that helps to trigger a more effective response from the general public while reducing impacts.

DATA AVAILABILITY STATEMENT
Data sharing is not applicable to this article as no new data were created or analyzed in this study.

ORCID
Erika R. Meléndez-Landaverde https://orcid.org/0000-0001-5564-3589 U R E 2 Mean likelihood of making a protective decision for both types of warnings across all scenarios Mean likelihood of making a protective decision for each type of warning with gender as a secondary independent variable T A B L E 3 Two-way ANOVA outputs for the two tests dangerous situations than Hazard flood warnings?Based on the results presented, the answer is a resounding Yes.