Use and perception of weather forecast information across Europe

Although European studies have become more common in recent years, published research on perception and use of weather information has been dominated by studies from the United States with some scattered contributions across Europe. The present study gives a broad European context, by providing perspectives from 18 countries and several user professions as well as from 14 National Meteorological and Hydrological Services (NMHSs), and by combining new insights from probabilistic forecasting, warning and interaction between NMHSs and their users. These insights are based on two surveys undertaken in the framework of the EUMETNET Nowcasting (E‐NWC) Programme, where EUMETNET represents the European Meteorological Services' Network: one survey for the participating NMHSs in the E‐NWC Programme, and the other one for their respective users. Both surveys were distributed in autumn 2019, and open for responses until spring 2020. Several findings from the surveys support conclusions of previous research, for example, concerning the perception of probabilities or taking measures in case of severe weather (many users would start their preliminary measures at a probability level of 60%). Although most of the NMHSs and their users are in regular contact, there is room for increasing the frequency of face‐to‐face meetings between them. Nearly one‐third of NMHSs never meet face‐to‐face with users from the public. The two surveys indicate that there might be benefits of increased collaboration and sharing of data between European NMHSs to be able to offer their users more training, and to learn from each other in areas where insight already exists.


| INTRODUCTION
In the same way as weather has always played a significant role to humankind in their everyday activities (Wiston & Mphale, 2018), NMHSs have since their origin been there to cover societal needs and to provide information to protect the inhabitants lives and properties. Despite this long and interconnected dependency between NMHSs and their users, common questions to discuss in the meteorological community still are "do the users understand our forecasts?" and "what are their needs and preferences"? Luckily, over the years, there have been numerous attempts to provide answers to the questions and to enlighten the community. If we dig a little deeper, previous research on the topic can be separated into different subfields of weather forecasting. Actually, the body of research is substantial, and only examples are shown below. The subfields following below are selected based on what can be regarded as important areas for NMHSs and their users, where the responsibility to issue weather warnings is central, with examples from different countries. Very recent casessuch as the catastrophic floods in Western Europe between 12 and 17 July 2021showed that despite accurate forecasts of meteorological parameters it was not possible to prevent casualties (Cloke, 2021). Such cases might also motivate improvements in communicating severe weather risks and forecast accuracy, taking into account the user's current perception.
Therefore, a subfield that has caught much attention for many years is related to forecast uncertainty. NMHSs have always been interested in the accuracy and reliability of their forecasts (Sivle, 2016), and it is fair to assume the same holds for the users of the forecasts. Various studies related to the topic have been performed in different countries and for different user-groups. The subfield of forecast uncertainty has several subfields on its own. For example, some research concentrates on the interpretation of probability statements and numbers (e.g., Gigerenzer et al., 2005;Hanrahan & Sweeney, 2013), other research focuses on how people interpret uncertainty into deterministic information (e.g., G omez et al., 2021;Sivle & Kolstø, 2016). In addition, several studies focus on the impact of the uncertainty or probabilistic information on decision-making. For example, Joslyn and LeClerc (2012) concluded that participants from the University of Washington (USA) using forecasts containing uncertainty information made better decisions compared with those using only deterministic information. Mass et al. (2009) suggested that the public in the United States is widely interested and understands probabilistic weather forecasts if clearly presented. The user's perception of the uncertainty was also intensively studied among emergency managers in Germany (Kox et al., 2015). They revealed that still a large percentage (85% of respondents) would seek deterministic information, though the uncertainty information was generally appreciated. Moreover, the respondents would take measures by rather high probability thresholds, for example, 55% of participants would decide to take action only if the probability of a storm occurrence would exceed 70%.
Another subfield with a large body of research is related to severe weather and warnings. Part of the research is crosscutting and involves both questions related to uncertainty, as exemplified above, and warning information. The practices of NMHS concerning criteria, dissemination and verification of severe weather warnings in Europe were summarized by Rauhala and Schultz (2009); however, the user perception was only marginally discussed. An example of research is described in Demeritt (2012), studying various aspects of perception and use of warning information, including a risk matrix, in the United Kingdom. One suggestion mentioned in this report was to allow the setting of personal thresholds for the level of rainfall alerts that the user deems necessary, to reduce the number of irrelevant alerts. Warnings on severe weather are often issued when meteorological parameters are expected to exceed certain thresholds (Holzer et al., 2015). These thresholds can depend on the country's climate, the purpose of warning and so forth. The assumed impact of reaching or exceeding such thresholds plays an increasing role in the decisions of forecasters or crisis managers, too. There are several studies on impact-based forecasting, its understanding and influence on the decisions and risk mitigation. It is expected that the risk of severe weather is often underestimated because of lack of information about its impacts. Surveys on simulated events (e.g., risk of thunderstorm during a mountain hike) indicate that impact forecasts should lead to more adequate response (Weyrich et al., 2018). Another study based on real postevents showed also some level of ambiguity in the responses, depending on the meteorological education of the respondent or his/her perception of vulnerability to weather risk (Taylor et al., 2019). Kox and Thieken (2017) studied people's willingness to act, and found that this was more likely if the expected weather is more severe or the property at risk is of higher value. A recent survey with 32 European NMHSs showed that 31% of respondents were already issuing impact-based forecasts and 50% were planning to do so within 5 years (Kaltenberger et al., 2020). Among major problems in implementation of such forecasts/warnings, they identified lack of infrastructure and missing national hazard strategy and national hazard partnerships. Such partnerships can provide authoritative, consistent and useful information to responder communities and governments, aiming to reduce impact from natural disasters (Hemingway & Gunawan, 2018).
These examples of subfields demonstrate the importance of getting a deeper insight into forecast uncertainty and severe weather forecasts in a broader European context, as well as more information about the use and requirements from both NMHSs' and users' perspectives. Therefore, one important point in the two surveys discussed in this paper was the way of presenting probabilistic information and warnings by NMHSs and their reception by users. The respondents were also asked about the role of different communication tools as well as about the importance of training sessions. Both perception and requirements of users are discussed in the present paper, as well as the cooperation between the users and NMHSs by confronting these two perspectives. The results show insights from many different countries across Europe, as well as many different professions as the asked users were numerous.
The background and the method are presented in Section 2. Findings from the surveys are reported and discussed in Sections 3, 4 and 5. First, insights on usergroups and their interaction with NMHSs can be found in Section 3. Thereafter, two sections are centred on use and perception of two main topics: probabilistic products (Section 4) and warnings (Section 5). Finally, conclusions are presented in Section 6.

| BACKGROUND AND METHOD
The first EUMETNET activity, which aimed to map the status of nowcasting in Europe, was done in the frame of the "Nowcasting Activity" from 2012 to 2013. Within this programme, an overview of nowcasting systems and activities at European NMHSs, methods of dissemination and requirements on future developments were prepared. The programme continued in the frame of EUMETNET under the name ASIST (Application-oriented analysis and very short-range forecast environment) in the years 2014-2018. One of the programme working areas was especially focusing on the user aspects in providing nowcasting applications and on the communication between NMHSs and users. However, the information gained about the user's point of view was only indirect at that time, reflecting the experiences of the NMHSs (ASIST, 2018). It was concluded that the nowcasting user-groups in Europe and their requirements should be identified with more details and more insights on the way they use and understand the forecast outputs.
Therefore, during the ongoing third programme phase (2019-2023), one task for the E-NWC Programme is to get insight into user-groups' needs and requirements concerning nowcasting and seamless prediction systems (from minutes to a few days ahead). Since 26 different countries (NMHSs) take part in the E-NWC Programme, this was a great opportunity to get a broad European context on user perspective across different topics. Hence, it was decided to conduct two surveys: one survey addressed to the NMHSs themselves and another to their users. The development of both surveys was an iterative process, starting in summer 2019, where draft questionnaires were shared with the expert team of E-NWC to discuss questions, formulations and other issues related to the surveys. After several rounds of interactions and adjustments to improve the questionnaire, the surveys were distributed via Survey Monkey (2021) in autumn 2019, and answers were collected until spring 2020. The cornerstones of the E-NWC surveys are listed in Table 1.

| Details of the NMHS survey
The NMHS survey counted 47 questions (denoted with capital letter "Q" in the text and the respective number of the question), of which 25 were mandatory. It was sent to 26 NMHSs, one in each of the countries taking part in the E-NWC Programme. The questionnaire was sent to the NMHSs only in English, which is not the native language of most of the respondents, so different interpretations of some of the formulations are possible. Fourteen (from 26) NMHSs answered the survey (see Figure 1, on the top): eight NMHSs answered all questions, and on most questions, 10 answers were received. The length of the questionnaire could be the reason for a lack of responses on the non-mandatory questions, and probably also for not always elaborating on open text questions.
The NMHSs were asked about recent trends in the development of their nowcasting systems and applications. However, some of these topics are not covered in this paper.

| Details of the user survey
The user survey counted 39 questions (denoted with small letter "q" in the text and the respective number of the question) and was translated by the respective NMHSs to their national language to ease the answering process to the users. Nevertheless, there is no guarantee that different interpretations did not take place, since the meaning of words can differ slightly between languages. NMHSs distributed the user questionnaire to their respective users such as authorities, users from specialized professions, users from transportation and the general public. Hence, some contact between the NMHSs and the users might have existed in advance. Responses from 232 users were collected, of which 135 completed the survey (18 countries, 15 different languages, see Figure 1, at the bottom). This is not so many to represent the broad net of different types of users of meteorological data and services in Europe. Unfortunately, in some of the countries, no responses by users were achieved. However, in some others, more than 30 answers were received (e.g., 37 in Estonia and 32 in Norway), almost 30 in Hungary, Sweden and Poland, and around 10 answers in Austria, Croatia and Serbia (see Figure 1 for details).
The user survey was inspired by some of the published studies mentioned before: for example, Morss et al. (2008) and their questions 11 and 12; Hanrahan and Sweeney (2013) and their questions 7, 9 and 13, and Kox et al. (2015) and their questions 1, 2, 6, 9, 10 and 11 (see also Table S1 in the Supporting information for more details). Especially the latter survey was used for comparison on some topics with previous research (e.g., concerning the perception of probabilities in case of approaching severe weather, and lead time to start with F I G U R E 1 Maps depicting countries participating in NMHS survey (upper figure) and user survey (lower figure); for the latter according to answers in question 1 (q1): "In which country are you working?" (221 answers, 11 skipped). Table (right) lists the participation of countries in the NMHS and user surveys respectively and the number of user survey respondents in the given country. Altogether 14 countries participated in the NMHS survey, and 232 responses were obtained from users in 18 countries preliminary measures). Thus, in addition to questions related to nowcasting, the respondents were asked about short-range forecasting (deterministic vs. probabilistic information), warnings and impact forecasting, and communication and training issues. Its purpose was to investigate whether the findings of the earlier studies were similar for a broader group of users and countries in Europe. For compatibility reasons, the same structure or terms have been used in some of these questions. For example, q15 was analogical to question 6 in Kox et al. (2015). Similarly, the wording in question q13 concerning the perception of probability (possible, likely, very likely) was used, since this is applied in several European languages in relation with weather forecasts.
Both questionnaires are available online as Appendix S1 and Appendix S2.

| Limitation of language
The answers from both surveys were extracted and inserted in Excel as different sheets and thereafter combined and analysed as a whole. As the user survey has been translated to the different national languages with support of NMHSs, the information from open questions containing text was also provided in the respective national language. The E-NWC team had to translate this back into English, so content might also differ slightly from the original meaning. Nevertheless, in case of potentially ambiguous (for non-meteorologists) terms like "storm," "thunderstorm" or "precipitation," "certain amount of precipitation" in the question q15 of the user survey, the respondents clearly distinguished these terms (see Section 3.4).
Because of the possible lack of representativeness of the answers, all results are marked with the number of answers to the respective question (in the figure text), and the charts show the answers in percentages of the answers of that particular question.

| MAIN USER-GROUPS AND THEIR RELATION TO WEATHER FORECASTING
The user-groups were manifold: 60% are authorities, of which the majority worked in civil protection and Emergency Services (response to q2 of the user survey). Twenty percentage of the users were involved in specialized professions such as district heating service, public space cleaning, local government, sport activities, as well as in energy and water supply and industry and constructions. Fifteen percentage of the users were from the general public, and only 2% were in transportation (e.g., railway or aviation operations). However, from the NMHS point of view, the range of their services was very broad and they provide weather information to different transport sectors. For example, 8-10 NMHS (67%-83% of all responding) disseminated regular and specifically suited forecasts for air traffic controllers or for users in shipping (question Q1 of the NMHS survey).

| Interaction between NMHSs and their users
As the communication between meteorologists and different user-groups is important for both parties in developing the forecasting systems and products, NMHSs were asked about the frequency of contact on phone or other communication tools and face-to-face meetings with users (Q2 and Q3). Contacts are relatively frequent between NMHSs and authorities (50% in daily contact and 40% at least once a week, and 40% of NMHSs meet face-to-face one to three times within a month) and also with the transportation (20% daily) and meet the general public (30% of NMHSs meet daily or at least 1-3 times a week). However, 30% of NMHSs never meet face-to-face with users from the general public, 20% never meet with users from transportation, and 10% never meet with businesses and specialized professions.
NMHSs were asked which nowcasting information they provide to their specific user-groups like (i) authorities, (ii) businesses and specialized professions, (iii) transportation, (iv) general public and (v) others (Q5). Altogether, 27 various products were mentioned explicitly, predominantly usual meteorological parameters (temperature, wind, precipitation, cloudiness) and severe weather warnings on thunderstorms, floods, blowing snow, gale and so forth. The resolution of the products is 1 Â 1 km or 2.5 km, the update frequency varies from 5 min to 2 h. Some of the NMHSs seem to provide exactly the same nowcasting information to the different user-groups. From the user side, many weather parameters, especially wind, temperature, rain/snow and thunderstorms, are already in use in a short time frame (1-12 h). Parameters such as tornadoes, turbulence, icing, hail, lightning/thunderstorm, humidity/dew point as well as fog and clouds have been highlighted as being useful in the future (see Figure 2, q10). Possible links between the size and resources (budgets) of respective NMHSs and the number of parameters provided to users (Q12) or the use of data assimilation types were statistically investigated (not shown). However, these relationships seemed to be rather ambiguous. Deeper research of factors influencing the applied technologies, number and quality of products disseminated by NMHSs to users was beyond the scope of this study (one could, e.g., consider differences due to geography, climate, legislation, etc.). Figure 3 shows that 15% of the users regularly attend training sessions with their meteorological institute, and 31% of the users occasionally attend such training sessions (q37). However, 30% of the respondents say that they do not attend specialized training/workshops/postevent reviews with their meteorological service. There was no place for the respondents to provide more information about why, but of course, this is not very surprising, as training sessions are quite resource-demanding, and one of the user-groups in the survey is the general public. Since relatively few users answered the survey, and some of them probably already were in contact with the NMHS, the actual number taking part in training sessions might be lower. One user commented that he or she is probably not aware of training sessions / seminars. Most of the NMHSs (80%) claimed that they occasionally organize post-event reviews with users, but no one reported that they have consultations on a regular basis (Q32, 10 answers).

| Training and verification
When asking the users if they need more training, 55% of the users answered "yes," while 45% answered "no" (q38, 119 answers, 113 skipped). Many of the users also explained in which fields more training could be needed: theoretical knowledge about meteorology itself, background of forecasting, interpretation of forecasts, cost-loss relationships, specific topics such as extreme weather events and communication.
Verification of weather forecasts provided to users usually increases the confidence in the products. In general, 70% of the NMHSs provide verification results to their users (Q26, 10 answers). Verification of deterministic weather forecasts and weather warnings are provided to the users on a regular basis or occasionally (57% and 43%, respectively; Q27, 7 answers). Thirty-three percentage of the NMHS respondents do not provide verification results for nowcasting products and the same for probabilistic forecasts. However, 17% of the NMHSs provide verification results for probabilistic forecasts on a regular basis, 17% provide them rarely and 33% only occasionally. In case of nowcasting products, these verifications are occasionally (67%) sent to the user, depending on the user's needs. The NMHSs were also asked if they provide information about the added value of nowcasting products compared with numerical weather prediction (NWP) output to their users (Q30, 10 answers). While 40% of the F I G U R E 2 q10 "Please indicate (1) the kind of weather parameters that are in use already on a short time frame (1-12 h) within your decision-making procedure, and (2) the kind of weather parameters that would be useful on a short time frame within your decision-making procedure in future" (159 answers, 73 skipped) NMHS inform their users about the added value, 40% do not.
Some of the users might not be aware of the training and verification opportunities given by the NMHSs. However, organizing such training can be resource-demanding. Hence, this might be an opportunity for NMHSs to support and learn from each other, since training material already exists in some of the countries. One platform already exists under the EUMETNET umbrellanamely EUMETCALaiming at collaboration in training activities (Chiariello & Nietosvaara, 2009).

| Communication channels
As communication tools, most prominently usedaccording to the user survey (q22, 130 answers)are websites (82%), followed by email (42%) and telephone (38%). Less used are SMS/mobile alert systems, social media and electronic systems (like chats). Probably, such systems are less available to users as some of the NMHSs do not use social media. Similarly, Zabini (2016) reports that a large percentage of the population now obtains weather information from their mobile phones, and this will affect the way weather forecasts are communicated to, and used by, people. For instance, mobile phones allow users to receive specific, personalized and updated information.
The NMHSs answered a similar question (Q4): "Which channel(s) do you use for providing weather forecasts and nowcasting information tailored to your users?" (10 answers). One hundred percentage of the NMHSs use websites to reach the public and authorities; in addition, 100% use email to communicate with the authorities. Social media is used by 70% of the NMHSs to inform the public and 50% of the NMHSs to inform the authorities. Chat or other interconnection systems are also in extended use to communicate with authorities (80%).
Some other communication tools were mentioned by the users such as an internal system for the respective company or video conferencing. Forty-four percentage of the users answered that weather information on social media is worse than from an official source, and 34% say that forecasts on social media are complementary (q23, 126 answers). Forty-two percentage of the users responded that video explainers would be helpful in a high-impact weather situation (q24, 126 answers). Such video explainers are already in use in the United States (e.g., The Weather Channel (2017,2021). The use and role of social media, for example, Twitter, in providing the information about weather risks were also discussed in Morss et al. (2017) and Ripberger et al. (2014).
In summary, all NMHSs answered that they use websites to disseminate their forecasts, and this was reported to be the dominant channel in the user survey. Forecast information is not shared on social media by all NMHSs, and this provides one possible explanation for why the users report them to be less used than other channels. There might, however, need to be increased focus on (socio)-technological advances and new media by NMHSs, both to get the message out to the users but also to understand communication in the modern information environment, as suggested by Morss et al. (2017). The audiencethose affected by the weather and the riskare suggested to be incorporated as partners in a co-production process. q37 (user survey). Do you attend specialized trainings/workshops/post-event reviews with the meteorological service as well as consultation?
F I G U R E 3 q37 "Do you attend specialized training/workshops/postevent reviews with the meteorological service as well as consultation?" (124 answers, 108 skipped)

| General confidence of users in weather prediction
The confidence of users in the accuracy of a deterministic 2-day weather forecast is high for all mentioned parameters such as temperature, a certain amount of precipitation, precipitation (regardless of the amount), thunderstorm and storm (windstorm, snowstorm, etc.) (See Figure 4; q15, 146 answers, 86 skipped). In more detail, 88% of the users have high or very high confidence in a 2-day forecast of temperature and 82% of the users have high or very high confidence in a 2-day forecast of precipitation, but are less confident in a certain amount of precipitation (53% high or very high). For comparison, in an earlier survey concerning emergency services in Germany (Kox et al., 2015), 70% of respondents were highly confident in predicting chances of precipitation but only 50% in forecasts of the amount of precipitation. Furthermore, Morss et al. (2008) found highest confidence in temperature (1-day forecast) followed by precipitation. For a 5-day forecast (q16, 146 answers, 86 skipped), there is a general shift towards lower confidence, but there is still high confidence for at least three parameters: temperature (7% of the users answer very high confidence), amount of precipitation and storm. A lower confidence in forecast information with increasing lead time is consistent with other studies reporting that many users in general expect forecast uncertainty to increase with increasing lead time (e.g., Joslyn & Savelli, 2010;Kox et al., 2015;Morss et al., 2010).

| PROBABILISTIC PRODUCTS
Ensemble information provides better communication of the forecast uncertainty to users. Some research is crosscutting and touches into both user perception of probabilistic forecasting and aspects of decision-making (Morss et al., 2008;Morss et al., 2010 andStephens et al., 2019). Uncertainty information is especially important for "complex" parameters or weather situations with high variability in space and time (e.g., convection). Exactly 70% of the NMHSs (Q15 in total 10 answers) answered that they provide probabilistic weather forecasts to their users.

| Benefits for users of probabilistic information
Exactly 66% of the users think that probabilistic information makes the forecast more trustworthy compared with q16 (user survey). How high is your confidence in the accuracy of a 5-day forecast with respect to parameters F I G U R E 4 Left: q15 "How high is your confidence in the accuracy of a 2-day forecast with respect to each of the following parameters." Right: q16 "How high is your confidence in the accuracy of a 5-day forecast with respect to each of the following parameters" (for both questions there were 146 answers, 86 were skipped). 98% of 146 answers answered the question for temperature, 93%-95% answered the two questions regarding precipitation, and 88%-90% answered the two last phenomena, thunderstorm and storm, respectively a deterministic forecast (q19, 140 answers, 92 skipped). This finding is in line with other studies reporting that uncertainty information improved decision quality and increased trust in the forecast (Hanrahan & Sweeney, 2013;Joslyn & LeClerc, 2012), because such forecasts seem less "wrong" than single-valued forecasts failing to verify (Joslyn & LeClerc, 2012). Eighty-seven percentage of the respondents think that nowcasting and probabilistic information make the forecast more valuable (q20, 140 answers, 92 skipped). It shows that it is important for the NMHSs to provide probabilistic information. However, it is also important to keep in mind that some users might think that probabilities would be meaningless and confusing, or that they should only be used for severe weather (Hanrahan & Sweeney, 2013).
Most of the users in this survey are professional or expert users, and might have other demands than the general public. Forty percentage of the users think that nowcasting and probabilistic information give more detailed and accurate information in time and space compared with a standard single value weather forecast (q21, 140 answers, 92 skipped). Even 39% of the users are convinced that nowcasting and probabilistic information help them to make faster, better and easier decisions. This was a star ranking question from 1 to 4 stars, where the weighted average was 3.1. Still, 21% of the users see little or no benefit of this information, and might even be confused. As reported in Section 3.2, half of the users responded that they need more training in different fields. Hence, providing training in the fields of nowcasting and probabilistic information to certain users might increase the value of this information. Nevertheless, providing nowcasting and probabilistic information should be done with care and in dialog with users, and perhaps as a supplement to other information rather than a replacement.
Some of the users provided additional comments. They stated that a single value forecast may facilitate the decisionmaking procedure (but it can have a high error) compared with a range of values that makes decision-making less efficient/effective (but which might have a lower error). Knowing the uncertainty of a forecast supports the decision makers in dimensioning their measures.

| Presentation and use of probabilistic information
Eighty-six percentage of the NMHSs respond that their probabilistic products are based on user requirements to some degree, but 14% of the NMHSs (that means one respondent of the seven answers on Q18) respond that they do not find a way to ensure that their users understand the probabilistic information received. However, 80% of the NMHSs use regular consultation with the users to help them to better understand the uncertainty of the forecasts (Q20, 10 answers) and 40% organize specialized training/workshops/post-event reviews.
The user survey showed that maps are the preferred way to receive a probabilistic product (64%) followed by graphs (51%) and percentages (42%, q18, see Figure 5). Other studies have also asked for preferences for uncertainty formats. For instance, Morss et al. (2008) found that their respondents preferred percentage probability and non-numerical text rather than relative frequency and odds. In another study, users from the civil protection were found to be more familiar with the depiction of uncertainty in graphs (meteograms), while the interpretation of probabilistic maps (i.e., distribution of probabilities of exceeding a threshold of certain parameter) was found to be more difficult (Wastl et al., 2018). Notably, a number of studies have shown that what users prefer to view is not necessarily the same as what works best for them (Kinkeldey et al., 2014).
As shown in Figure 6, most NMHSs respond that they provide probabilistic information to different user-groups as ranges or in the form of text; graphs or maps are used less in certain areas (e.g., transportation). Symbols are more used to show probabilistic information to the general public than to other users.
Three of the NMHSs (42%) do not know how the users like or use the probabilistic weather products (Q19, 7 answers). On the other hand, three of the NMHSs (42%) confirm that the user feedback of probabilistic weather products is good, and one NMHS even says that it is very good.

| Probability notation as terms and numbers
The users were asked to indicate the probabilities they associated with the terms Possible, Likely and Very likely. (q13; "Imagine your meteorological service states the onset of an upcoming storm in your region with the indications possible/likely/very likely. Which probabilities would you associate with this forecast? Scale: 0%-100%; e.g. possible: 20-40…"). There was a quite wide variety of answers to q13 (Figure 7). All answers that were not clearly indicated as a percentage were excluded from the analysis. Most of the users gave an interval of percentages for each of the terms, as a lower and an upper level. Some gave one number, for example, 50%, which is giving both "lower" and "upper" as a level of 50%. Thus, Figure 7 shows the spread in the answers. The  F I G U R E 6 Q17 "And how do you present probabilistic product(s) to the user?" seven NMHSs specified the form of probabilistic products for authorities, five for users in business and transportation, six for the general public. Percentages shown in the figure refer to the number of respondents in each category. "Other" was only an option to select, but it was not a field to be specified upper level of "possible" has most outliers from 0% to 100%. Some of the outliers can be caused by misunderstanding or a really quick answer due to the long survey (e.g., for "possible" the mean value is from 22% (lower) to 41% (upper), but the range is from 10% to 50%). On average, there is no overlap in how the terms were interpreted by the users, even though we see that the upper range of "possible" coincides with the lower range of "likely." For comparison, Kox et al. (2015) also showed a high spread of interpretation, but the mean values were: 36% (possible), 58% (likely) and 79% (very likely). The notion of the terms "possible," "likely" and "very likely" have a slightly different meaning in the respondents national languages, when translated, than in English. Nevertheless, the information from Figure 7 shows that using terms, without numbers, it can be interpreted differently among the users, without knowing if this is a result of different risk acceptance.
There are several other studies showing a range of interpretations when people are asked to put numerical values to uncertainty statements (Budescu et al., 2014;Morgan, 2014;National Research Council, 2006). The interpretations were personal based on experiences or even personality (e.g., risk seeking vs. risk averse).
When NMHSs use a certain statement in the communication with their users, they cannot expect users to interpret it the same way as they do. To some extent, it is possible to reduce the range of interpretations by combining numerical and verbal expressions, as the Intergovernmental Panel on Climate Change (IPCC) does in its Assessment Reports (https://www.ipcc.ch/ reports/). Still, there is no guarantee they are interpreted similarly. It starts already with the translation of a 70% probability of precipitation taken from the numerical model to "likely," and for a forecaster telling that "there is a chance of showers today" based on his/her subjective assessment of model data, observations, experiences and more. Similarly, two meteorologists would most likely also put different numerical values to the same "chance of rain" for the same day (Stewart et al., 2016).
Probabilistic information can be, and is, provided by NMHSs in different formats, such as maps or graphs. Neither this study nor previous studies (e.g., Wastl et al., 2018) in the subject can give an unambiguous answer on which format is best suited to communicate uncertainty information. Rather than being one single format suited to all, it appears that different user-groups and different situations require different formats; for instance, using maps in some situations to some users, F I G U R E 7 q13: "Indications of possible/likely/very likely. Probabilities associated with a forecast for an upcoming storm in your region? Scale: 0%-100%; for example, possible: 20 (lower)-40 (upper)" (a box plot figure in analogy to Wallsten et al., 1986 and their fig. 4) and graphs or non-numerical text in others. Also in line with previous research (e.g., Kox et al., 2015), the users in the survey associated different numerical probabilities with the non-numerical probability terms. Combining multiple representations of uncertainty information can provide benefits for the meaning-making process (Sivle & Uppstad, 2018). Importantly, regular contact with users to get insight in their needs, requirements and perceptions are necessary to increase the uptake and use of the information.

| WEATHER WARNINGS
Most NMHSs in Europe cooperate in the METEOALARM project (see https://meteoalarm.org), and thus show active weather warnings to all interested users. Moreover, impact forecasting is expected to gain in importance: Approximately 30% of the users think that impact forecasting will be more useful than a pure meteorological warning in the future; however, 55% of the users need both (q26, 131 answers, 105 skipped).

| Challenges with warnings
When dealing with weather warnings, problems encountered are numerous (Figure 8, q25, this question could have multiple answers). Thirty-four percentage of the users think that too many warnings make a "dulling effect," 32% of the users have difficulties when dealing with uncertainties in weather forecasts and warnings, and 32% of the users have to deal with inappropriate warnings with respect to geographical characteristics of the warning area. Similar results are reported elsewhere; for example, Helbig (2019) found in his qualitative study of use of warnings in Norway that warnings are sometimes issued for too large areas. If there are warnings for areas without danger, this can reduce trust in the warnings (Helbig, 2019). Kox et al. (2015) also report that too many warnings may result in a dulling effect, and that dealing with uncertainty in warnings can be demanding for some users. However, Mu et al. (2018) report that providing impact level and likelihood level helped participants to make better decisions. According to Morss et al. (2017), technological advances are changing how people exchange and interact with information. In an interconnected dynamic system, people's perceptions and use of information evolve in social interactions in the physical and digital worlds (Morss et al., 2017). Hence, it is interesting that 28% of users in this survey observed conflicts between official warnings and other information sources, which can be a potential problem in the decision process because clarification of such differences takes time and false or incorrect information can increase vulnerability to the threats (Torpan et al., 2021). However, more research is needed in this area to understand the process of communication and meaning-making in a modern technology environment better.
Other problems encountered were that information has to be filtered or that it may be difficult to follow the changes in warning levels in case of adverse phenomena. Moreover, the warnings might be fixed to predetermined geographical areas, and more flexible local adaptation is in demand.

| Taking action
The severe weather events that are causing most of the users' "take action" are strong wind and gusts (72%), heavy rain connected with thunderstorms (69%), followed by snow (41%), glazed frost (34%) and continuous rain (30%). Hail and "other" seem to cause less taking action of users (18% and 13%, respectively; see Figure 9, q29). In the category "other," the users mentioned the following events, which are causing most of their "taking action": freezing rain with the possibility of a negative impact on the power system, sea level, frost, no rain (drought), low temperature, the expected rise of the Danube water level in Budapest, sleet, avalanches/landslides blocking roads, fog (below 200 m) + freezing fog, frosts in spring, thawed frost (below À15 C), heat (above 30 C), blows and snowstorms, cold, sea waves, fog, reduced visibility, high flows, low water levels/drought, and high fire risk.
"Taking action" meant for the majority (78%) of respondents more intense observation, for example, of weather information (q27, 129 answers, 103 skipped). Users selected also actions as forwarding information (e.g., potential damage assessment), personnel preparatory measures (e.g., assemble staff members on call) and measures to reduce risk (coordination, making contingency plans, acquiring special equipment). The latter three possibilities were chosen by 39%-42% of respondents. Other types of action were mentioned by 11% of respondents. More specifically, some users analyse the situation of the respective area (river basin) and foresee possible conflictive areas or points. Another respondent has to adjust the plans and activities in some situations or he/she is conducting collaboration conferences in case of more serious warnings.
The European Severe Storms Laboratory (ESSL) provides annual reports on severe weather events that result in injuries and deaths. For 2019, severe wind caused most injuries (38%), followed by lightning (30%), hail and tornadoes (8%), and avalanches and heavy rain (7%). A total of 406 fatalities were reported, with heavy rain (36%) on F I G U R E 9 q29 "Which severe weather event is causing most of your 'taking action'?" (124 answered, 104 skipped) top followed by avalanches (25%), lightning (21%) and severe wind (16%; cp. ESSL, 2020). It seems that there is some correspondence between the events causing injuries and deaths and the events users in this survey find important to "take action" about. This awareness and ability could indicate that the respondents are highly interested in weather information, and might not represent the general public.

| Importance of probability thresholds for preliminary actions
Probability thresholds could be important for preliminary actions of users, so one question was q34: "A storm is forecasted for the next day. On which probability would you or your department/agency/organization generally start with preliminary measures? Based on a probability of XX%." The same question was included in the questionnaire of Kox et al. (2015). When a storm is forecasted for the next day, most of the users (35 users of 124 users answering q34, corresponding to 28%) would start their preliminary measures at a probability level of 60% (see Figure 10).
Moreover, 24% of the users would start at a probability level of 40% already, and in total 67% of the users would have done preliminary measures at 60% or lower probability level (for comparison, 45% of respondents in Kox et al. (2015) would take preliminary measures at 60% or lower). However, 23% of the users would start with measures only at a probability level of 80%. In the category "other," the users mentioned 0%, 1%, 70%, and that personnel preparation measures are always taken, but physical measures only with probabilities of more than 50%.
Comparison of the user survey results with Kox et al. (2015) is presented in Table 2 for percentages of participants who would take action at forecast probability bigger or equal to 40, 60 or 80%. It must be stressed that differences between the E-NWC and Kox et al. (2015) survey results could also be caused by the fact that the latter was organized specifically for the civil protection authorities. Nevertheless, in both surveys, one can see the tendency to take measures by rather high probability levels taking into account large uncertainty in forecasting extreme or small-scale severe events, which are often forecasted with relatively low probability (e.g., Kuchera et al., 2021, or Bouttier & Marchal, 2020 can be found in Konold (1989), suggesting that some persons assign 0% with meaning "no," 50% with "I don't know", and 100% with meaning "yes." Intermediate values can be associated with these anchors, and hence, for example, 70% can be judged sufficiently close to 100% to warrant a "yes" or "go" in the decision-making process (Konold, 1989). Taking the same question as before but with the additional information that the event is likely to happen in the countryside with low density of population (q35, 124 answers), only 20% of the users (compared with 28% of the users before) would start their preliminary measures at a probability level of 60%. Twenty-two percentage of the users would start at a probability level of 40% already, and 27% of the users at a probability level of 80%. In the category "other," the users mentioned percentages of 30%, 70% or that it does not depend on where the event is forecasted. One respondent stated that the locations of the power grids are relevant and exist in sparsely populated areas, or it is in the countryside that users operate, which is why probabilities are the same as stated in the question before. When taking the additional information that the event is likely to happen at a place where many people would be affected, 23% of the users would start their preliminary measures at a probability level of 60%. Twenty-three percentage of the users would start at a probability level of 40% already, and 21% of the users at a probability level of 80%. Specifications in the category "other" were, for example, 30%, 50%, 70% or that the user prepared himself or herself anyway or that the threshold is the same and is not depending on the location or the density of the population (see Figure 10).
The main idea behind these three questions was, whether the density of population or number of people potentially affected would have an impact on the decisions and thresholds. Some past severe weather accidents were combined with high outdoor concentration of people (Horv ath et al., 2007) and some hazard impact models or visualization take population density into account (Robbins & Titley, 2018). It can be concluded that in the countryside the number of users who start with preliminary measures is higher only at high probability levels of T A B L E 2 Comparison of E-NWC user survey results with the survey for emergency services in Germany (Kox et al., 2015) Question number (q: E-NWC survey K: Kox et al., 2015) Question ( Note: The selected questions were formulated in the same way or very similarly in both surveys. Only percentages of high and very high confidence are shown for parameters in q15 (question 6 in Kox et al., 2015). The statistics was provided with all exactly specified probability thresholds (118 cases, including 4 from the "Other" category in q34).
an event, while the probability level of the event does not highly affect the number of users starting with their preliminary measures at locations where many people would be affected.

| Lead time for preparatory measures
In the responses to question q30, 1 h lead time was mostly preferred for intensified observation (e.g., of weather information) and to forward information (about potential damage assessment, evaluation of need of assistance, etc.), which was indicated by 40% or 35% of all responding users, respectively. However, for personnel preparatory measures (assemble staff members on call) and non-personnel preparatory measures (making contingency plans, acquiring equipment, etc.), more time is needed. The 6-h lead time was mostly selected by 31% of respondents for the former and 29% for the latter category (see Figure 11). On the one hand, this means that issuing a warning 1-6 h before an event gives many users sufficient time to take action. In the survey of Kox et al. (2015), the range for the organizational lead times needed for preparatory action was large (from 10 min up to 7 days) but the median was about 3 h. On the other hand, several respondents in their survey also stated that short warning lead times did not always correspond with their needs and identified this as a major communication problem. It seems that 1-h lead time is needed for most of the users' to start with first mitigation measures for all the different parameters like strong wind and gusts, heavy rain, snow, glazed frost, hail, continuous rain and heat (q31). Some more comments were provided by the users in the category "other." For an open-air event, users can handle 6-12 h; or there is no need for such a long preparation time; or responsible actors probably need more than 6 h to be able to take active measures, they can call a conference within about 2 h after notice; or in some cases users need only less than 30 min lead time! The optimal lead time in case of tornado warnings was discussed in the survey of Hoekstra et al. (2011), where it was shown that even relatively short lead times (15 min) would be sufficient to get into shelter but longer (30 minÀ1 h) lead time would give bigger sense of security and more choices on how to react according to the respondents. It must also be taken into account that additional time can be needed for authorities for dissemination and evaluation of the warning information.
Apparently, the length of the possible warning lead time is dependent on the situation and type/scale of the severe weather, despite recent progress in forecasting local events, for example, flash floods (Golding, 2009). In summary, too many warnings are reported by some users in the survey to make a "dulling effect," and some experience difficulties when dealing with uncertainties in warnings. This supports findings reported by for instance Kox et al. (2015), despite the use of different thresholds. Further similarities with their findings could be identified concerning probability thresholds for starting preliminary measures and lead time, although the variability in the responses was high. Similar to Kox et al. (2015), many users have time to prepare if they are warned approximately 1-6 h in advance, but others require several days of lead time to be sufficiently prepared.

| CONCLUSIONS
The uniqueness of this study lies in the broad European context with several countries participating, and in the combined perspectives of NMHSs and users on some of the same topics. Several countries across Europe took part in the two surveys (NMHS survey and user survey); 14 respondents from overall 26 NMHSs taking part in the E-NWC Programme, and 232 users from 18 countries and different user-groups. The total number of responses from users is relatively low (1000 responses is often a typical number in quantitative surveys to allow for generalization), given the number of countries and potential forecast users, hence the findings cannot be generalized to a larger population, but the data have a value supporting previous research in a broad European context. In addition, the data provide insight into perspectives across Europe and to some extent allow for comparison with responses from NMHSs.
Previous surveys at national level or with specific usergroups (e.g., civil protection) already contained questions related to user perception of forecast accuracy, probability or threshold to take action. Also in the presented bigger context, one recognizes similar traits in understanding forecast probability. There is, for example, a tendency to take measures to mitigate severe weather impact by probability thresholds as high as 60% or 80% (see Table 2 and Section 5.3). One reason is probably the lack of knowledge of what such reasonable/objective thresholds might be for some particular type of severe weather or NWP model. Although NMHSs provide verification of probabilistic forecasts or organize training sessions and post-event reviews (see Section 3.2), this usually does not take place on regular basis. Brief explanation of probability or intensity of the forecasted event in the form of text, which is sometimes provided by meteorological offices, can be helpful but allows a wide range of interpretation (Abraham et al., 2015;Lenhardt et al., 2020).
Findings from the two surveys indicate that there might be benefits of increased learning and sharing between European NMHSs. Thus, they might be able to offer their users more training, and to share experiences in areas where insight already exists (e.g., from local surveys). Another recommendation for the NMHSs is to have contact with certain user-groups to improve the quality of services and uptake of information, especially regarding probabilities, weather warnings and impact of the weather. Similar conclusions were also made by Fundel et al. (2019), showing that better understanding of probabilistic products and their use in decision-making can be reached by deeper cooperation with the users and with visualization specifically designed for risk communication.
For future research, it is interesting to conduct similar dual surveys (to NMHSs and users) in other continents than Europe, for comparison. It would be positive to link the future NMHS and user surveys in order to get more information, whether the recent developments in NWP and related applications bring the desired effect and are properly used. The findings of the user survey could be useful in transdisciplinary projects, especially with civil protection (e.g., to revisit probability thresholds in severe weather mitigation). In nowcasting or very short-range forecasting programmes, it is important to clarify the most problematic weather events and lead times required for taking measures (e.g., for evacuation of citizens, etc.). Such activities could be more focused on protecting crowds, which are especially vulnerable to severe weather. It is also of interest to collect data and reports from earlier user studies conducted by European NMHSs, analyse them as a whole, and publish them to make the findings accessible to a larger audience. Nevertheless, this is not enough, and to achieve sound communication in a modern media environment, increased focus and research on how people exchange and interact with information in an interconnected dynamic system is needed. This must be done in close contact between NMHSs and their users (also using methods of dialog, chats, social media, etc.).