How Low Should We Alert? Quantifying Intensity Threshold Alerting Strategies for Earthquake Early Warning in the United States

We use a suite of historical earthquakes to quantitatively determine earthquake early warning (EEW) alert threshold strategies for a range of shaking intensity targets for EEW along the United States West Coast. The current method for calculating alert regions for the ShakeAlert EEW System does not take into account variabilities and uncertainties in shaking distribution. As a result, if the modified Mercalli intensity (MMI) level used to determine the extent of the alert region (the alert threshold) is the same as the target intensity threshold, the alert region will be too small to include all locations that require alerts even if earthquake source parameters are estimated accurately. Missed alerts can be reduced by using a lower alert threshold than the target threshold. This expands the alert region, increasing the number of precautionary alerts issued to people who experience shaking below the target level. We determine alert thresholds that optimize this tradeoff between missed and precautionary alerts for target thresholds of MMI 4.0–6.0 using 143 M5.0–7.3 earthquake ShakeMaps as ground truth. We examine the quality of each alerting strategy relative to the target MMI, where we define alert quality metrics in terms of both the area and population alerted. Optimal alert thresholds maximize correct alerts while limiting most precautionary alerts to regions that are likely to still feel some shaking. We find these optimal alert thresholds also maximize warning times. This analysis presents a quantitative framework ShakeAlert can use to communicate alerting strategies and performance expectations to ShakeAlert users.

• We determine optimal alert thresholds for the ShakeAlert Earthquake Early Warning System for a range of target shaking intensity levels • Early warning alert regions produced using an intensity threshold at the target level cannot alert most (>60%) people who need alerts • Optimal alert thresholds can achieve >95% correct alerts with nearly all precautionary alerts to places that still feel some shaking

Supporting Information:
Supporting Information may be found in the online version of this article.

of 20
Earthquake early warning (EEW) is particularly subject to the uncertainties caused by limited real-time information, as strong shaking occurs within seconds after an earthquake begins. In order to issue alerts as quickly as possible, EEW systems are automated and typically employ simplistic approaches to rapidly forecast shaking intensity once earthquake ground motions are detected (Allen & Melgar, 2019). The ShakeAlert EEW System for the West Coast of the United States (U.S.) rapidly determines an earthquake's magnitude and location, and then EEW alert regions are determined by computing the extent of prespecified median-predicted modified Mercalli intensity (MMI) levels given that magnitude and location Kohler et al., 2020;Thakoor et al., 2019). However, ground-motion variabilities caused by source, path, and site effects can amplify shaking at a given location above its median-predicted value or attenuate shaking to be less than the median predicted. This creates a large range of uncertainty in the distance over which a given level of shaking intensity can occur even when the earthquake's magnitude is perfectly known. For a M6+ earthquake, one standard deviation of ground-motion model uncertainty translates into a 100 km uncertainty in the alert distance using an MMI 3.5 alert threshold .
In EEW, there is typically some critical level of shaking for which people or infrastructure want to be alerted. This could be the level at which shaking is felt, or the level that might be potentially damaging to a critical piece of infrastructure. We will refer to this critical shaking level for which warnings are needed as the target shaking intensity or target threshold. (A list of definitions of important EEW terms such as "target threshold" is in Table 1) The actual spatial distribution of shaking exceeding the target shaking will be complex. The ShakeAlert EEW System does not attempt to forecast the precise areas that will experience shaking greater than the target threshold. Instead, the system alerts a circular region whose perimeter is based on the distance at which the median-predicted shaking is forecast to equal a specified intensity level (Thakoor et al., 2019). Given these simplifications, if the MMI level used to compute the radius of the alert region (the alert threshold) is the same as the target threshold, there will be locations outside of the alert region that experience larger-than-expected shaking exceeding the target threshold. This situation creates missed alerts even if the earthquake's magnitude is estimated accurately Saunders et al., 2020). Additionally, setting the alert threshold MMI at the target threshold level may also prevent locations that will experience strong shaking in an earthquake from receiving an alert before shaking at the target level arrives (i.e., a timely alert) because of the time it takes for an earthquake rupture to evolve and thus for its magnitude to be estimated accurately (Minson et al., 2018).
An ideal alerting strategy is one that alerts a high percentage of the locations that experience shaking above the target level while minimizing sending alerts to locations that experience shaking below the target level. To reduce missed alerts at and above the target level, EEW systems can account for the uncertainties inherent in rapid ground-motion modeling by employing an alerting strategy where the MMI level used to generate the alert region is lower than the target threshold for the system. This expands the size of the alert region, reducing missed alerts to locations that experience target-level intensities and providing more timely alerts to locations outside of the initial late alert zone (the near-epicentral region that will not receive a timely alert because strong shaking will arrive before an alert can be issued; Kuyuk & Allen, 2013;McBride et al., 2020;Wald, 2020). However, minimizing missed alerts increases the number of precautionary alerts (sometimes referred to as "false" ground-motion alerts) to people who experience shaking below the target level. Thus, it is important to find an alerting strategy that optimizes this tradeoff between missed alerts and over-alerting.
The strategy of using an alert threshold lower than the target threshold is employed in EEW systems around the world. For example, the Japan Meteorological Agency's (JMA) EEW system aims to alert for JMA intensity 5L (approximately equivalent to MMI 7-8) but has an alert threshold of JMA Intensity 4 (approximately equivalent to MMI 6-7; Hoshiba et al., 2008Hoshiba et al., , 2010. In Mexico City, the soft sediments the city is built on cause extreme site amplifications when subjected to seismic waves, making the city's structures and dense population particularly vulnerable to earthquakes (Espinosa Aranda et al., 1995). Thus, the EEW system in Mexico is tailored to alert for any earthquake that is likely to cause felt shaking in Mexico City to better ensure that alerts are issued for the earthquakes that may cause damage (Suárez et al., 2018).
In the U.S., the ShakeAlert System's stated goals are to rapidly detect and issue warnings for potentially damaging earthquakes (Given et al., 2014Kohler et al., 2018Kohler et al., , 2020. However, specific ShakeAlert alerting target intensities for the public have been unclear and alert thresholds have changed in time in response to unanticipated negative real-time alert performance (Cochran & Husker, 2019 (Cochran & Husker, 2019;Lin, 2019), implying that target shaking levels for public EEW alerts may also include non-damaging shaking intensities. Further decreases have been made to the operational ShakeAlert alerting thresholds without any stated changes to the overall goal of alerting for potentially damaging shaking (Kohler et al., 2020).
In this study, we examine ShakeAlert-specific alerting strategies for a range of target thresholds, with the alert threshold a set amount lower than a target MMI. Our goal is to quantitatively determine optimal ShakeAlert EEW alert thresholds for a range of target levels such that missed alerts are minimized and precautionary alerts are reduced as much as possible. We systematically compute alert regions for different alert threshold strategies for a ShakeMap catalog of 143 M5.0+ earthquakes that have produced MMI 4.0+ shaking along the U.S. West Coast (Figure 1). We use the ShakeMaps for these earthquakes as the ground truth to evaluate the alert quality Terms pertaining to EEW a alerting strategies

Term Definition
Alert Distance The radius of the alert region relative to the earthquake's epicenter.

Alert Region
The area that is issued an EEW alert. In the ShakeAlert System, this is defined as the extent of the alert threshold MMI b given median-expected ground-motion distributions computed from ground-motion models using the earthquake's estimated magnitude and location.

Alert Threshold
The MMI level for which the EEW alert region is defined. The alert threshold may not be the same as the target threshold.

Alerting Strategy
The choice of alert threshold given the target threshold. Here, we aim to find optimal alerting strategies where the alert threshold is lower than the target threshold.
Post-Alert Message Follow-up notification sent to EEW system users in the event of an earthquake or a false alert that provides detailed information about why an alert went out (or did not go out). See McBride et al. (2020) for the specific post-alert message templates developed for the ShakeAlert System.

Target Threshold
The minimum MMI level for which EEW alerts are needed. Here, we consider target thresholds ranging between MMI 4.0-6.0.
Terms pertaining to EEW alert quality

Term Definition
Alert Quality The breakdown of correct, missed, and precautionary alerts for a given alert region based on the observed distribution of target threshold shaking (here, according to the earthquake's ShakeMap). Alert quality can be computed in terms of area and/or population exposed to target threshold shaking.
Correct Alert A location is issued an alert that experiences shaking at or above the target threshold. Correct alerts can be further categorized in terms of whether the alert is timely or late.
False Alert An alert is issued but no local earthquake has occurred. This alert is issued in error.

Late Alert
A location received an alert after shaking at or above the target threshold arrives at that location.
Over-Alerting A situation where the alert region contains substantial amounts of precautionary alerts in order to reduce missed alerts.
Missed Alert A location is not issued an alert but does experience shaking at or above the target threshold.
Precautionary Alert A location is issued an alert but experiences shaking below the target threshold. Precautionary alerts can be further categorized in terms of the likelihood for people to still perceive some shaking and recognize it as an earthquake. Here, we define likely felt shaking as MMI ≥3.0 and likely not-felt shaking as MMI <3.0. In our analysis, all locations beyond the ShakeMap boundary are assumed to experience likely not-felt shaking, so all alerts to these locations are categorized as precautionary alerts.
Timely (Useful) Alert A location received an alert before shaking at or above the target threshold arrives at that location (here, assumed to be when the S-wave arrives at the location). The goal for any EEW system is that all alerts are timely alerts. a EEW: Earthquake Early Warning. b MMI: Modified Mercalli Intensity. and the alert timeliness produced by the different alert regions. We assess the alert quality in terms of the spatial coverage of the alert (i.e., whether the alert region produced by that alert strategy contains a high percentage of the locations that experience shaking above the target level), and we assess the alert timeliness by whether the alert to a location arrives before the S-wave arrives at that location. This is an optimistic view of EEW performance because shaking exceeding the target threshold, especially if the target threshold is low, could arrive in the P-wave (e.g., Minson et al., 2018).
While we find that we cannot reduce both missed and precautionary alerts, given their tradeoff, we find that there are optimal alert thresholds for different target intensities such that high percentages of correct alerts can be achieved without significant alerting to locations that are unlikely to experience felt shaking from the earthquake. This means that people who were, strictly speaking, unnecessarily alerted because shaking never exceeded the target threshold might still perceive the alert as beneficial (or at least understandable) since they likely felt shaking from the earthquake. Cost-benefit analyses suggest alerting strategies that favor over-alerting (i.e., substantial precautionary alerts; Table 1) perform better than strategies that have substantial missed alerts, especially if the public's tolerance to false or precautionary alerts is high Saunders et al., 2020). Surveys of public expectations of EEW systems in Japan and New Zealand also suggest that over-alerting is acceptable when the public is aware that an earthquake has occurred (Becker et al., 2020;Nakayachi et al., 2019), further motivating our special treatment of over-alerting to locations with felt shaking. We also find that the preferred alert threshold in terms of overall alert quality also maximizes the available warning times to locations that experience the target intensity.

Methods
We take two approaches to assess the ability of a given alerting strategy to provide high-quality alerts for the earthquakes in our catalog. First, we assess the alert quality in terms of the spatial extent of the alert region assuming accurate magnitude and location estimation. Second, we evaluate the timeliness of a given alerting strategy by considering the expected evolution of the magnitude estimate as a function of P-wave observation time. We perform both comparisons using alert quality metrics computed in terms of the area that required alerts for a given target threshold as well as in terms of population. Details of these methods are below.

Target and Alerting MMI Thresholds
ShakeAlert determines alert regions by the predicted extent of a specified MMI level given the estimated magnitude of the earthquake; we call this the MMI alert threshold (Table 1). Depending on the alerting strategy, the alert threshold may be equal to or less than the minimum MMI level for which EEW alerts are targeted. In previous papers, this MMI level is referred to as the damage threshold (e.g., Minson et al., 2018), but we refer to this here as the target threshold (Table 1) as some of our target thresholds are non-damaging levels of shaking.
In our analysis, we consider target thresholds between MMI 4.0-6.0 in 0.5 MMI unit increments. MMI 4.0 falls under a category of "light" shaking that is not damaging but felt by most people, and generally described as strong shaking by the people who report it (Stover & Coffman, 1993; Figure 2). MMI 4.5 and MMI 5.0 shaking fall under the "moderate" shaking category, where objects can fall from shelves and people report difficulty standing or moving around (Stover & Coffman, 1993). MMI 5.5 and 6.0 shaking fall under "strong" shaking, where windows can break and damage to unreinforced masonry can occur (Wood & Neumann, 1931).
For each target threshold, we consider a range of alerting strategies where the MMI threshold used to compute the alert region is 0-2.5 MMI units lower than the target in 0.5 MMI unit increments. We do not consider MMI alert thresholds below 2.0 where MMI is not defined. This consideration only affects the lowest alert threshold for the MMI 4.0 target threshold, which would call for alerting at the MMI 1.5 level, something that is undefined as it is halfway between MMI 1 "not felt" and MMI 2 "few felt". Operationally, ShakeAlert uses alert thresholds of MMI 2.5, MMI 3.5, and MMI 4.5 to compute the alert regions that are delivered to the public through EEW mobile phone apps, the Wireless Emergency Alert (WEA) System, and Google/Android, respectively (Kohler et al., 2020;McBride, Smith, et al., 2021).

Alert Region Computation
We compute the alert regions for every combination of target threshold and alert threshold for each earthquake in our catalog. Our alert region computation follows the ShakeAlert operational procedure for the alert contour product (Thakoor et al., 2019) with updates to the ground-motion models used. Our alert region calculation procedure takes the following steps: 1. Rapid magnitude and epicenter estimates are computed by ShakeAlert.
Here, we assume that ShakeAlert estimates these values accurately according to the values in the Advanced National Seismic System (ANSS) catalog. (While this assumption is an idealization of the ShakeAlert System, our focus for this work is on optimizing the alert regions produced by the EEW system, not optimizing the source characterization). 2. The magnitude and epicenter estimates are input into ground-motion prediction equations (GMPEs) to estimate the median-predicted peak ground acceleration (PGA) and peak ground velocity (PGV) with distance. We use four NGA West-2 GMPEs (Abrahamson et al., 2014;Boore et al., 2014;Campbell & Bozorgnia, 2014;Chiou & Youngs, 2014), where, following the approach used by the ShakeAlert System, each GMPE was implemented individually, assuming: a set source depth of 8 km (from assumptions in the EEW source-parameter estimation algorithms); a set V S30 site condition (average shear-wave velocity in the top 30 m) of 500 m/s corresponding to the average V S30 across the ShakeAlert reporting region; and a strike-slip rupture on a vertical fault with a stress drop of 5 MPa and a Brune-type circular rupture (Brune, 1970) to estimate the source radius and hence depth-to-top of rupture. We then average the four PGAs and PGVs respectively to obtain a mean PGA and mean PGV with distance. 3. The output peak ground motions are then converted to MMI as a function of distance using ground-motion-to-intensity conversion equations (GMICEs; Equations 6 and 9 of Worden et al., 2012). 4. The distance with MMI corresponding to the MMI alert threshold is chosen as the alert distance. This alert distance is the radius of the circular alert region, which is centered at the earthquake's epicenter. Operationally, this alert region perimeter is then approximated as an octagon (Kohler et al., 2020;Thakoor et al., 2019), but we keep this perimeter circular for our analysis. Following ShakeAlert operations, we truncate the alert region perimeters to the California, Oregon, and Washington boundaries McGuire et al., 2021). Including this within-state restriction to our analysis also prevents over-weighting offshore earthquakes, where only a portion of the region that experiences shaking above the target threshold may be onshore.
The alert distances using the above procedure for a range of alert thresholds and earthquake magnitudes are summarized in Figure 3.

Alert Quality Definitions
For each earthquake and alerting strategy, we determine alert quality totals in terms of both area and population. For a given alert, we categorize each location throughout the ShakeMap as a correct, missed, or precautionary alert depending on whether the location was issued an alert as well as the MMI experienced at that location according to the ShakeMap (see Table 1 for full definitions). For cases where the alert region extends beyond the boundary of the ShakeMap (i.e., alerts are issued to locations where intensity ground-truths are not available), we assume the ground motions in those locations correspond to not-felt shaking (MMI <3.0) and thus consider alerts in those locations to be precautionary.
While precautionary alerts are typically referred to as False Alerts (e.g., Meier, 2017;Minson et al., 2018Minson et al., , 2020Saunders et al., 2020) or False Positives (e.g., Cochran et al., 2018;Kohler et al., 2018Kohler et al., , 2020Meier et al., 2020), the nomenclature used here follows suggested public communication terminology for EEW-specific post-alert follow-up messages . The use of false alert is reserved for an alert that is not associated with an earthquake, such as one issued in error (Table 1). However, in the cases considered here, an earthquake has indeed occurred and alerts are needed to some regions.
The precautionary alerts can be further categorized in terms of the likelihood for people to still perceive some shaking from the earthquake. While people who receive a precautionary alert may take unnecessary protective actions due to shaking being insignificant at their location, if they still perceive shaking they will have immediate confirmation that an earthquake has occurred. Information that an earthquake has occurred and that an alert was likely needed somewhere builds tolerance of over-alerting (Becker et al., 2020;Nakayachi et al., 2019). As discussed in Saunders et al. (2020), a high tolerance to precautionary alerts in public EEW suggests a preference for alerting strategies that favor over-alerting to reduce missed alerts.
Thus, we aim to find alert thresholds for our range of target thresholds that maximize correct alerts to locations that experience target threshold shaking (while potentially alerting locations that are likely to still feel shaking) and minimize alerts to locations that are unlikely to feel any shaking at all. We consider MMI ≥3.0 as likely to still feel shaking, especially if people are indoors (Figure 2; Wood & Neumann, 1931;Dewey et al., 1995), while we consider MMI <3.0 to be where shaking is unlikely to be perceived by most people and recognized as an earthquake (Dengler & Dewey, 1998). In our alert quality analysis for a given alerting strategy, we first consider the number of missed alerts relative to the total number of precautionary alerts. Following this, we consider the number of missed alerts relative to the number of precautionary alerts to locations where shaking is unlikely to be perceived.

ShakeMap Catalog Ground Truth
We assess the alert quality of these different MMI alert threshold strategies using a catalog of United States Geological Survey (USGS) ShakeMaps as the ground truth. We have curated a ShakeMap catalog with 143 M5.0-7.3 earthquakes (1980-2020) that have produced MMI 4.0+ shaking somewhere within California, Oregon, and Washington ( Figure 1). These ShakeMaps were produced by the National Earthquake Information Center (NEIC) using the most recent version of the USGS ShakeMap software (version 4; Worden & Wald, 2016). All Shake-Maps incorporate finite fault slip information, ground-motion observations from seismic sensors, and USGS "Did You Feel It?" (DYFI) reports of observed shaking intensities, where available (Wald et al., 2021;Worden & Wald, 2016).
The ShakeMaps in our catalog have good completeness at MMI ≥4.0 shaking (Figure 1d). This is expected as ShakeMap development has traditionally emphasized mapping the distribution of higher MMI values to aid in hazard assessment and rapid response to regions of damage (Wald et al., 1999). Because a significant portion of the ShakeMap boundaries in our catalog contain intensity observations in the MMI 2.5-3.7 range, using these ShakeMaps as ground truth for MMI <4.0 shaking may underestimate the extent of these MMI values. This means that we may overestimate the precautionary alerts issued to likely not-felt locations, as we assume that all locations outside the ShakeMap boundary experience not-felt ground motions. However, this assumption is conservative.
SAUNDERS ET AL.

Estimating Alert Timeliness
In addition to the overall alert quality, we also consider the expected timeliness of an EEW alert when finding an optimal alert threshold for a given target threshold. Alert timeliness is determined by whether that alert arrives before the shaking at that location exceeds the target threshold, typically during the S-wave (Table 1). A location is issued an EEW alert when the MMI predicted at the location (MMI predicted ) is larger than the alert threshold (MMI alert ) considered in a given alerting strategy: When an earthquake is detected by the ShakeAlert system, t alert at a location reflects the time at which the location is inside the alert region computed using the given alerting strategy and the available magnitude estimate. The latest time for when an alert can be considered useful at a location is when the shaking intensity at that location (MMI observed ) reaches the target threshold level: The warning time at a given location is thus the difference between t last and t alert plus the time it takes for the alert to be delivered to that location: The timeliness of an alert at a given location (t warning ) is affected by many factors. Some factors depend on the physics of the earthquake, such as when shaking at or above the target threshold arrives (t last ) as well as how magnitude estimation (which is controlled by rupture evolution and high-frequency amplitude saturation) affects t alert (e.g., Minson et al., 2018;Trugman et al., 2019). Other factors can depend on the EEW system itself. These include: the station configuration, and data telemetry latencies, and algorithm computation times, which affect earthquake detection and magnitude estimation; and alert delivery latencies. The delays caused by technical aspects of the ShakeAlert System and alert delivery mechanisms can significantly affect the timeliness of the alert (e.g., Allen et al., 2020;Behr et al., 2015;Chung et al., 2020;Stubailo et al., 2020;Wald, 2020). However, in this work, we follow Minson et al. (2018) and only consider the magnitude evolution as the main factor in alerting time. By focusing on just the factors affected by the earthquake physics, we can quantify the minimum possible alert times for each earthquake. Thus, these results represent the best performance that any EEW system could provide given the physical limits of how fast earthquake ruptures evolve and shaking propagates as well as the inherent variability of ground motion.
For each earthquake in our catalog, we use a 1-D velocity model (Kennett & Engdahl, 1991) and the earthquake's hypocenter to estimate P-and S-wave arrivals at all points in the ShakeMap grid (Beyreuther et al., 2010;Crotwell et al., 1999). We assume MMI target arrives at a location at the start of the S-wave, so t last becomes the S-wave arrival time at that location. Following Minson et al. (2018), we assume that the rupture evolution can be tracked perfectly as soon as P-waves reach the surface (starting at time t p ). Magnitude is then estimated as a function of the P-wave observation time: where we use the magnitude saturation analysis presented in Trugman et al. (2019) to define this relationship ( Table S1 in the Supporting Information S1). At each time step, the magnitude estimated using that P-wave observation time is used to compute the size of the alert region for a given alert threshold, which updates until the magnitude estimate equals the catalog magnitude. This final alert update reflects the maximum extent of the alert region for a given alerting strategy.

Individual Earthquake Example Alert Quality Calculation and Alert Strategy Comparison
For each earthquake in our catalog, we compute the alert regions for the range of target thresholds using a range of alert thresholds and evaluate the quality of these alerts using the earthquake's ShakeMap. An example of this alert quality calculation is shown in Figure 4 for the 1989 M6.9 Loma Prieta earthquake using a target threshold of MMI 4.5 and alert thresholds of MMI 4.5, 4.0, 3.5, and 3.0. As we are only considering one target threshold 9 of 20 in this example, we color the ShakeMap directly in terms of alert quality relative to the MMI 4.5 target threshold (Figures 4a-4d). Comparisons of the alert quality totals for this example demonstrate how different amounts of correct, missed, and precautionary alerts are achieved for area and population using the same alert threshold (Figures 4e and 4f). These comparisons also illustrate how the M6.9 Loma Prieta earthquake produced much higher ground-motion amplitudes than expected from median shaking distributions (e.g., Hanks & Brady, 1991), perhaps due to a high stress drop. The MMI 4.5 alert threshold, the case where the alert threshold equals the target threshold, achieves only 24% and 30% correct alerts in terms of area and population, respectively. As the alert threshold MMI decreases, the size of the alert region increases and includes more of the regions that experience MMI 4.5+ shaking. The MMI 3.5 alert threshold achieves nearly 0% missed alerts for both area and population (0.3% and 0.2%, respectively) but requires issuing precautionary alerts to nearly as many locations as those who experience MMI 4.5+ shaking.
We can generalize this alert quality visualization to all target thresholds and alert thresholds for the 1989 M6.9 Loma Prieta earthquake (Figures 5 and 6). Examining the range of alert regions compared to the ShakeMap Loma Prieta earthquake for each alert threshold with the earthquake's ShakeMap colored by alert quality relative to the shaking distribution of the MMI 4.5 target threshold. ShakeAlert public EEW alerts are currently only available in California, Oregon, and Washington, so we truncate alert regions that extend beyond these borders (seen in d). The bar charts show the alert quality totals in terms of (e) area and (f) population for each alert threshold. The percentages on top of each bar indicate the total percentage of correct alerts for each alert region. As a note, in total, about 60,000 km 2 experienced MMI 4.5+ shaking, and 10.5 million people are included in that area, which gives the total necessary alerts for that target threshold in terms of area and population.
SAUNDERS ET AL.

10.1029/2021EF002515
10 of 20 demonstrates that for all target thresholds, the distribution of shaking at a given target threshold (for example, MMI 5.5+) extends beyond the boundary of the alert region computed using the same MMI as the alert threshold ( Figure 5a). To visualize this in terms of both area and population exposed to different MMI levels, we compute the cumulative MMI distribution in 0.5 MMI unit increments as a function of epicentral distance in terms of area and population (Figures 5b and 5c). As we are considering the point-source-based alert regions in our analysis, the epicentral distance corresponds to the radius of the alert region (the alert distance). Given a target threshold MMI, these MMI distributions can be converted into alert quality as a function of alert distance ( Figure 6). For the same target threshold, 100% correct alerts may be achieved at different alert distances when considering alert quality in terms of area and population (Figures 6a and 6b). In both cases, reducing missed alerts requires a significant portion of the alerted region to be precautionary alerts (Figures 6c-6f).

Results
We aim to find an optimal alert threshold for each target threshold such that missed alerts are minimized (<5%) and precautionary alerts are reduced as much as possible. First, we consider the quality of the different alerting strategies in terms of the spatial distribution of the results. Then we examine the expected alert timeliness of these different strategies. In both cases, we consider the total area and population distributions for a given target MMI threshold summed over all earthquakes in the catalog. In our catalog, larger magnitude earthquakes generally have a larger relative contribution to the total area that experiences shaking at a specific target threshold compared to smaller earthquakes, and this difference increases as the target threshold level increases (Figure 7). This is skewed from what is expected in a Gutenberg-Richter earthquake distribution, where the contributions to different shaking levels are nearly equal for M5.5+ earthquakes (Minson et al., 2021), due to the under-representation of smaller events in the available ShakeMaps (Figure 1c). Relative contributions of target threshold shaking in terms of population are closer to equal across magnitudes at lower target thresholds, while at higher target thresholds, M5.5-6.0 and M6.5-7.0 earthquakes have the highest relative contributions compared to the other earthquakes in our catalog (Figure 7). The total area and population exposed to shaking at lower target thresholds is much higher than the total area and population exposed to shaking at higher target thresholds. Loma Prieta earthquake. The ShakeMap is used to compute the cumulative distribution of MMI in terms of (b) area and (c) population as a function of the radius of the alert region (alert distance; same as epicentral distance). The shaded gray regions represent the area and population outside of the ShakeMap boundary that are inside the alert region of a given alert distance (seen in this example for the cumulative MMI in terms of area at alert distances of ∼450+ km). The alert regions for a M6.9 earthquake using different alert thresholds are shown by the circles on the ShakeMap and vertical lines on the cumulative MMI distribution plots.

Figure 6.
Using the cumulative MMI distribution with alert distance (Figure 5), alert quality metrics can be computed as a function of alert distance for different MMI target thresholds: (a and b) percentage of correct alerts as a function of alert distance (i.e., the percentage of the area or population that should have been alerted that actually was alerted); (c and d) percentage of the alert region that is over-alerted as a function of alert distance (i.e., the percentage of the alerted area or population that represents precautionary alerts rather than correct alerts); (e and f) total number of precautionary alerts as a function of alert distance. This figure shows the alert quality metrics for the same example earthquake in Figure 5 (the 1989 M6.9 Loma Prieta earthquake), and the left and right columns show alert quality in terms of area and population, respectively. The dots and corresponding horizontal lines in (e and f) indicate the total number of necessary alerts for a given target threshold. As the alert distance increases, more locations that experience shaking at the target level are included inside the alert region (i.e., the correct alert percentage increases). However, the number of locations inside the alert region that experience shaking below the target level also increases with increasing alert distance, where the amount of over-alerting can be many times more than the number of necessary alerts for a given target threshold, especially at large alert distances. Similar to Figure 5, the vertical lines indicate the alert distances for a M6.9 earthquake using MMI alert thresholds between MMI 2.0-6.0. For an example alert threshold of MMI 3.5 and an example target threshold of MMI 4.0, these plots indicate that alerting strategy obtains 80% correct alerts in terms of area (90% in terms of population), where 20% of the alerted area is to precautionary alerts (10% alerted population is over-alerted).

Alert Quality Considering Spatial Distribution Only
Similar to previous analyses (e.g., Minson et al., 2019;Saunders et al., 2020) as well as our individual earthquake examples (Figures 4-6), we find that there is a tradeoff between minimizing the number of missed alerts and minimizing the number of precautionary alerts for a given target threshold. We find that the sum total over the earthquakes in our ShakeMap catalog produces similar alert quality results in terms of area ( Figure 8) with those in terms of population ( Figure S1 in the Supporting Information S1). We first consider precautionary alerts as those sent to areas that experience MMI less than the target MMI threshold (Figure 8, circles). When the alert threshold is the same as the target threshold (Figure 8a), the alert regions have substantial missed alerts, at 60%-80% missed alerts for alert targets of MMI 4.0-6.0, respectively, albeit low percentages (<10%) of precautionary alerts. Lowering the alert threshold relative to the target threshold increases the size of the alert region ( Figure 3) and thus increases the correct alert percentage at the target level, reducing missed alerts (i.e., progressing through the panels in Figure 8). In general, lower target thresholds (cool colors in all panels of Figure 8) achieve lower percentages of missed alerts at alert thresholds closer to the target threshold value. For example, Figure 8 shows that an MMI 4.5 target (purple dots) has <5% missed alerts using an alert threshold of 1.0 MMI units lower than the target threshold, while an MMI 6.0 target (yellow dots) requires an alert threshold of 2.0 MMI units lower than the target threshold in order to reach <5% missed alerts.
This behavior is likely due to a combination of factors. Due to the nature of the GMICEs employed in the ShakeAlert System, 0.5 MMI unit increment differences in alert thresholds result in smaller alert distance changes at higher MMI levels (MMI >4.0) compared to smaller MMI levels (MMI <4.0). For example, a M7.0 earthquake will have a ∼40 km increase in alert distance between MMI 4.5 and MMI 4.0 alert thresholds while there will be an ∼100 km increase in alert distance between MMI 4.0 and MMI 3.5 alert thresholds (Figure 3). These differences decrease at smaller magnitudes, but the general trend remains the same. Additionally, for higher magnitude earthquakes that produce larger areas of high-intensity shaking, finite-fault effects may reduce the ability for point-source-based alert regions to easily encompass the whole region of target threshold shaking.
Increasing the size of the alert region also substantially increases the amount of over-alerting to locations that experience shaking below a given target threshold. Even alerting strategies where the alert thresholds equal the target threshold can contain some precautionary alerts (∼10% of the alert area for MMI 5.5 and MMI 6.0 targets; Figure 8a). At alert thresholds where the missed alerts are <20%, the majority of the alert region can be over-alerted (Figure 8c). For alert thresholds where missed alerts are minimized (<5%), precautionary alerts can compose nearly all (75%-95%) of the alerting region (i.e., Figures 8c-8f). And given the differences in total population exposed to different target thresholds in the catalog (Figure 7), the total population over-alerted for an MMI 4.0 target threshold will be much larger than the total population over-alerted at higher MMI targets for an individual earthquake.
We next relax the requirement that precautionary alerts occur whenever the experienced MMI is strictly less than the target threshold and instead consider alerts to be precautionary only when they are delivered to regions that are unlikely to experience felt shaking at all (MMI <3.0). This implies that users may perceive the alerts as correct as long as they feel some shaking, even if that shaking level is less than the target MMI communicated by the EEW system. When we examine the percentage of the alert region that includes locations that are unlikely to experience felt shaking (MMI <3.0), we find a more encouraging result (Figure 8, diamonds). We highlight this subset of precautionary alerts because these will require post-alert follow-up messaging for the recipients to understand that an earthquake occurred and alerts were likely needed somewhere rather than the alert was a false alert . The number of alerts to likely-not-felt locations resulting from a given alerting strategy depends on the alert target. For the MMI 6.0 target threshold, none of the alert thresholds tested resulted in precautionary alerts to likely-not-felt locations. For the other target thresholds, precautionary alerts to likelynot-felt locations are issued when MMI alert ≤ 3.0 for that target threshold. We observe that the cases with precautionary alerts to likely-not-felt locations are cases that have no missed alerts at a given target threshold, and that generally <5% missed alerts had already been achieved using a higher alert threshold for that target threshold. The MMI 4.0 target threshold is an exception to this, where <5% missed alerts cannot be achieved without having ∼20% of the alerted area as likely-not-felt locations. Overall, we find that once missed alerts are minimized, using a lower alert threshold results in a substantial number of alerts to likely-not-felt locations (20%-60% of the alert region).

Figure 8.
Alert quality (by area) tradeoff between missed alerts and precautionary alerts for different target thresholds (different colors) for a given MMI threshold alerting strategy (different subplots). The y-axes of these plots show the percentage of the alerted regions that are precautionary, and thus show the amount of overalerting resulting from a given alerting strategy. Circles indicate precautionary alerts to all regions that experience shaking below the alert target, while diamonds indicate precautionary alerts only to regions that are unlikely to experience felt shaking (MMI <3.0). This is for the sum total over all earthquakes in the catalog. Ideal alerting strategies produce no missed alerts as well as no precautionary alerts to regions that are unlikely to feel shaking. Note that the MMI 4.0 target is not included in (f) because we do not consider alert thresholds below the category of perceived shaking. The alert quality results in terms of population are very similar and are shown in Figure S1 in the Supporting Information S1.
SAUNDERS ET AL.

Alert Quality Including Timeliness Estimates
Now, we examine the expected alert timeliness for these different alert threshold strategies. For each alerting strategy, we compute the fraction of required alerts at a target threshold with a minimum theoretical warning time, where positive warning times indicate the alert arrives at a location before the predicted S-wave arrival time and negative warning times indicate the S-wave is expected to arrive before the alert. As with the alert quality comparisons, we find that the alert timeliness summed over all earthquakes in the catalog produces similar results in terms of area ( Figure 9) and population ( Figure S2 in the Supporting Information S1).
Our idealized EEW system assumptions result in no negative warning times for the alert threshold strategies tested. However, as we observed in the alert quality comparisons (Figure 8), there are substantial amounts of required alerts that receive no warning (missed alerts) when the alert threshold is similar to the target threshold.
Decreasing the alert threshold relative to the target threshold increases the fraction of the region that experiences target shaking that receives an alert. This increases the fraction of alerts with a given minimum available warning time both because the fraction of correct alerts increases and because some locations are receiving an alert earlier. For example, using a target threshold of MMI 6.0+, changing the alert threshold from MMI 5.5 Figure 9. Alert timeliness totals for different target thresholds (different colors) for a given MMI alert threshold strategy (different subplots), summed over all earthquakes. The y-axis shows the fraction of required alerts by area for a given target threshold, while the x-axis shows the minimum available warning time. Using a target threshold of MMI 4.5+ and an alert threshold of 0.5 MMI units below the target (purple line in plot b), 50% of the MMI 4.5+ area receives >5 s warning while 70% of the MMI 4.5+ area is issued alerts (30% missed alerts). Lowering the alert threshold to 1.0 MMI units below the target level (c) yields 98% correct alerts to MMI 4.5+ areas with 82% of the area receiving >5 s warning. Decreasing the alert threshold to 1.5 MMI units below the target (d) achieves 100% correct alerts for the MMI 4.5+ and slightly increases the fraction of area with >5 s warning time, but further decreasing the alert thresholds (e and f) yields no increase in the fraction of area with >5 s warning time. This behavior is similar for the other target thresholds and is similar for the alert timeliness in terms of population ( Figure S2 in the Supporting Information S1).
SAUNDERS ET AL.

10.1029/2021EF002515
15 of 20 ( Figure 9b) to MMI 4.5 (Figure 9d) increases the percentage of correct alerts from 45% to 90% and increases the percentage of area with 5.0+ seconds of warning from 0% to 50%. However, there is an upper limit on the maximum available warning time at a given location, defined by the S-wave arrival time at that location and the time of the first EEW alert issued for the earthquake. In general, lower target thresholds have longer minimum available warning times compared to higher target thresholds. We find that once 100% correct alerts are achieved for a given target threshold, further decreasing the alert threshold does not noticeably increase the available warning times to a given target threshold as the locations that are closest to the source (and have the fastest S-wave arrivals) are already included in the initial alert region.
Our results indicate that the alert thresholds that maximize warning times are the same as the alert thresholds that optimize overall alert quality. These optimal alert thresholds for each target threshold tested are summarized in Table 2. For the MMI 4.0 and 4.5 target thresholds, we find an optimal alerting strategy is to have the alert threshold be 1.0 MMI units less than the target; for the MMI 4.5 target threshold, the difference is 1.5 MMI units; and for the MMI 5.5 and 6.0 target thresholds, the optimal strategy is 2.0 MMI units less. We also find for the MMI 6.0 target threshold that there is no additional penalty in increased alerts to likely-not-felt shaking locations when using an alert strategy of 2.5 MMI units less than the target. That all implies that for MMI 4.5, 5.0, 5.5, and 6.0 target thresholds for earthquakes within the magnitude range considered in this analysis (M5.0-M7.3), we find an alert threshold of MMI 3.5 is optimal; for the MMI 4.0 target threshold, an alert threshold of MMI 3.0 is optimal. While these strategies produce substantial over-alerting, these alert thresholds minimize missed alerts (<5%) while limiting precautionary alerts primarily to locations that are still likely to feel some shaking.

Discussion
In an ideal EEW system, alerts for strong shaking are delivered to locations that will ultimately experience strong shaking, while locations that will experience shaking below the target threshold do not receive alerts and are spared from taking unnecessary protective actions. However, the immediacy with which EEW alerts must be issued after an earthquake detection in order for alerts to be timely prevents current EEW systems from using complicated ground-motion modeling approaches that could more accurately capture expected ground-motion variabilities. In the U.S., the ShakeAlert System uses rapid magnitude and location estimates alongside fixed source and site parameter assumptions (e.g., depth and V S30 ) to estimate the median-predicted extents of a range of MMI levels (Kohler et al., 2020;Thakoor et al., 2019). The MMI extent that corresponds with the alert threshold becomes the radius of the alert region that is used by ShakeAlert partners to deliver warnings to the public. This MMI threshold alerting approach has established tradeoffs between missed and precautionary alerts (e.g., Meier, 2017;Minson et al., 2018Minson et al., , 2019Saunders et al., 2020), and misunderstandings about the alerting strategy used have led to adjustments in the operational ShakeAlert alert thresholds (Cochran & Husker, 2019;Lin, 2019).
Here, we conducted a detailed analysis of alert threshold strategies for different target intensity thresholds in terms of both the alert quality and expected alert timeliness using a large ShakeMap catalog of historical U.S. West Coast earthquakes, focusing on finding optimal alert thresholds for the alert perimeters used for public EEW in the ShakeAlert System. Importantly, we find that alerting strategies where the alert threshold equals the target threshold cannot provide alerts to most (>60%) of the locations (or people) that experience shaking at target threshold levels even with no magnitude or location errors. To reduce missed alerts, the alert threshold used to generate the alert region needs to be a lower MMI level than the target threshold. We determine optimal alert thresholds such that the missed alerts for a given target threshold are minimized (<5%) while precautionary alerts are issued to locations that are likely to still feel some shaking from the earthquake (Table 2). We find that these alert thresholds also maximize the available warning times for the target thresholds.
Our analysis demonstrates that the EEW alerting strategy employed in the current ShakeAlert System for public alerts cannot reduce missed alerts without also increasing precautionary alerts. This means that alerts cannot be issued only to regions that experience damaging shaking levels, the levels most desired for receiving EEW alerts (Becker et al., 2020;Nakayachi et al., 2019); alerts to locations that experience non-damaging shaking  are unavoidable to reduce missed alerts at damaging target thresholds. However, negative public reaction to the lack of EEW alerts in Los Angeles County during the 2019 Ridgecrest, California, earthquake sequence hints that the U.S. public may appreciate alerts at non-damaging levels of shaking as well (Cochran & Husker, 2019;Lin, 2019;Ruan et al., 2020). Additionally, surveys for the Earthquake Network Initiative's EEW app found that users value alerts for small earthquakes even when the shaking did not cause damage and alerts for these earthquakes arrived late (Bossu et al., 2021). This implies a high public tolerance to precautionary alerts and allows us to favor alerting strategies that over-alert in order to reduce missed alerts at higher target thresholds compared to the alert threshold. Using this approach, our results demonstrate that we can choose alert threshold strategies such that we can minimize missed alerts to <5% while also issuing precautionary alerts to regions that are still likely to feel some shaking from the earthquake (Table 2). However, it is important to note that just because these lower alert thresholds will contain substantial precautionary alerts does not mean that everyone who experiences felt shaking will receive an EEW alert.
The ShakeAlert System currently issues alerts delivered through the WEA System at the MMI 3.5 alert threshold (Kohler et al., 2020), which corresponds well with our optimal alerting strategies ( Table 2). A strategy of setting the alert threshold at MMI 3.5 will minimize missed alerts and maximize timeliness to locations that experience target shaking of MMI ≥4.5 as well as reduce precautionary alerts to likely-not-felt locations ( Figure 10)  . Alert timeliness totals in terms of (a) area and (b) population for different target thresholds using the MMI 3.5 alert threshold, summed over all earthquakes in the ShakeMap catalog. As in Figure 9, the y-axis shows the fraction of required alerts by area for a given target threshold, while the x-axis shows the minimum available warning time.
SAUNDERS ET AL.

10.1029/2021EF002515
17 of 20 For ShakeAlert System users that use MMI 4.5 as an alert threshold, our analysis suggests that this alert threshold may be suited for very high MMI target thresholds not considered in this analysis (i.e., only the highest damaging MMI levels). With such high implied intensity targets, more detailed cost-benefit analyses may need to be considered (e.g., those in Minson et al., 2019). Alert regions using an MMI 4.5 alert threshold will also not be as resilient to errors in epicenter estimation compared to smaller alert thresholds due to the shorter alert distances at the MMI 4.5 alert threshold level (Figure 3). For example, a 20 km error in the epicenter estimate for a M5.5 earthquake may cause the MMI 4.5 alert region to not even include the actual epicenter for that earthquake. Such errors will cause substantial missed alerts in locations that experience the highest shaking amplitudes, not counting any additional errors in the alert region caused by magnitude estimation errors (which depend on the epicenter estimation).
The MMI 2.5 alert threshold used in ShakeAlert System operations for issuing public alerts via cell phone EEW apps is lower than the optimal thresholds found for the target thresholds tested here. This suggests that the MMI 2.5 alert threshold is optimized for non-damaging shaking levels that might not be felt by everyone alerted.
Indeed, alert quality comparisons using this alert threshold (Figure 8) indicate that alert regions produced using an MMI 2.5 alert threshold contain substantial precautionary alerts to likely not-felt locations (∼50% of the alerted region).
It is important to reiterate that these optimal alert thresholds produce alert regions with significant over-alerting, where the majority of the alerted locations are precautionary alerts. Totaled over all events in this analysis, the MMI 3.5 alert threshold produces alert regions where approximately 70%, 85%, 90%, and 95% of the alert region are precautionary alerts for target thresholds of MMI 4.5, 5.0, 5.5, and 6.0, respectively ( Figure 8). However, nearly all of these precautionary alerts are to locations likely to still experience some shaking (MMI ≥3.0). This indicates that people who receive an EEW alert are more likely to be alerted unnecessarily during a particular earthquake (Minson et al., 2018. Effective communication of the alerting strategy as well as informative follow-up messages after earthquake alerts are issued can help maintain the high precautionary alert tolerance necessary for enabling over-alerting strategies that reduce missed alerts at target threshold levels (Becker et al., 2020;McBride et al., 2020;Nakayachi et al., 2019).
The optimal alert thresholds for the target thresholds considered here depend on the earthquakes represented in the ShakeMap catalog. For the magnitudes included in our analysis (M5.0-7.3), we were able to find optimal alert thresholds using point-source-based alert regions. For larger earthquakes than those represented in our catalog (i.e., M7.5+ earthquakes), point-source based alert regions may not be sufficient due to finite-fault effects as well as due to magnitude underestimation caused by saturation effects from rapid near-field magnitude estimation. For these earthquakes, line-source-based alert regions such as those produced in ShakeAlert using the FinDer EEW algorithm (Böse et al., 2018;Kohler et al., 2020;Thakoor et al., 2019) will be preferred.
Our analysis considered an idealized EEW system to isolate changes to alert quality exclusively due to changes in the alert threshold in order to optimize the specific alerting strategies used by the ShakeAlert System. These assumptions included accurate earthquake magnitude and epicenter estimation, no specific station distribution, and no additional system or alert delivery latencies aside from magnitude saturation. Epicenter estimates for within-network events typically have low errors; however, out-of-network events can have epicenter estimation errors of >100 km (Chung et al., 2019;Hartog et al., 2016;Kohler et al., 2020). ShakeAlert magnitude estimations can also be inaccurate, especially when the epicenter is inaccurate. Errors in earthquake magnitude can significantly affect the size of the alert region, especially for lower alert thresholds and larger earthquake magnitudes ( Figure 3). While magnitude overestimation will produce larger alerting regions and may result in more precautionary alerts, magnitude underestimation will produce smaller alerting regions and may result in more missed alerts. Errors in the earthquake's epicenter will also misplace the alert region and may also result in increased missed alerts and precautionary alerts.
EEW system and alert delivery latencies can be on the order of several seconds (Allen et al., 2020;Behr et al., 2015;Chung et al., 2020;Stubailo et al., 2020), which impact the size of the late alert zone around the earthquake's epicentral region where shaking is the strongest Wald, 2020). As the available warning times are generally much shorter for smaller magnitude earthquakes (M <6) compared to larger magnitude earthquakes (M6+) due to differences in the extent of target level shaking, these latencies may have bigger impacts on the timeliness of EEW alerts for smaller earthquakes compared to SAUNDERS ET AL.

10.1029/2021EF002515
18 of 20 larger earthquakes for a given target threshold. While a detailed analysis of the implications of these latencies is beyond the scope of this paper, it is important that these latencies are communicated to the users of EEW systems when discussing alerting strategies and system performance expectations.

Summary and Conclusions
For the MMI threshold alerting strategy employed in the ShakeAlert EEW System, the MMI level used for the alert threshold must be lower than the MMI threshold targeted by the EEW system in order to account for ground-motion uncertainties and reduce missed alerts at target shaking intensities. Appropriate alert thresholds for specific target thresholds previously had not been quantified for the ShakeAlert System. We find optimal alert thresholds for a range of target intensity thresholds through a detailed analysis of alert quality and expected alert timeliness. For this analysis, we used a large catalog of ShakeMaps of historical M5.0-7.3 U.S. West Coast earthquakes as ground truth and considered an idealized EEW system (i.e., correct magnitude and location estimates as well as no additional latencies aside from magnitude evolution) to isolate changes in alert quality and timeliness due to the different alert thresholds tested. Optimal alert thresholds minimize missed alerts to locations that experience target threshold shaking while limiting precautionary alerts to locations that are likely to experience some shaking from the earthquake. We find an alert threshold of MMI 3.5 is optimal for target thresholds of MMI 4.5, 5.0, 5.5, and 6.0, while an alert threshold of MMI 3.0 is optimal for a MMI 4.0 target, though the MMI 3.5 alert threshold still correctly alerts >85% of the area (and population) experiencing MMI 4.0+ shaking. These optimal alert thresholds maximize the available warning time to locations that experience target shaking for the range of earthquake magnitudes tested and correspond with an operational alert threshold used in the ShakeAlert System. Thus, we suggest that the ShakeAlert System adopt MMI 4.5 as a target threshold with the current WEA System alert threshold of MMI 3.5 for communicating ShakeAlert System performance. This work provides a quantitative framework that can be used by the ShakeAlert System to communicate alerting strategies and performance expectations to users of the EEW system, without necessarily changing the level at which alerts are currently sent.

Data Availability Statement
Earthquake source information and ShakeMaps were obtained from the U.S. Geological Survey (U.S. Geological Survey, 2017; https://earthquake.usgs.gov, last accessed March 2021). Population data are the 2020 estimate from the Center for Internal Earth Science Information Network -CIESIN -Columbia University (2018) and National Aeronautics and Space Administration (NASA) Socioeconomic Data and Applications Center (SEDAC)'s Gridded Population of the World, v.4 (GPWv4) available at https://sedac.ciesin.columbia.edu/data/set/gpw-v4-population-count-rev11 (last accessed May 2021). All analysis and figures were created using Python (https://www. python.org).