NOAA's National Water Model: Advancing operational hydrology through continental‐scale modeling

The National Weather Service (NWS) Office of Water Prediction (OWP), in conjunction with the National Center for Atmospheric Research and the NWS National Centers for Environmental Prediction (NCEP) implemented version 2.1 of the National Water Model (NWM) into operations in April of 2021. As with the initial version implemented in 2016, NWM v2.1 is an hourly cycling analysis and forecast system that provides streamflow guidance for millions of river reaches and other hydrologic information on high‐resolution grids. The NWM provides complementary hydrologic guidance at current NWS river forecast locations and significantly expands guidance coverage and water budget information in underserved locations. It produces a full range of hydrologic fields, which can be leveraged by a broad cross section of stakeholders ranging from the emergency responder and water resource communities, to transportation, energy, recreation and agriculture interests, to other water‐oriented applications in the government, academic and private sectors. Version 2.1 of the NWM represents the fifth major version upgrade and more than doubles simulation skill with respect to hourly streamflow correlation, Nash Sutcliffe Efficiency, and bias reduction, over its original inception in 2016. This paper will discuss the driving factors underpinning the creation of the NWM, provide a brief overview of the model configuration and performance, and discuss future efforts to improve NWM components and services.

activities.When distributed across the country, however, even this seemingly large number of forecast locations (3800) leaves wide swaths of the domain without forecast coverage, with only 100,000 out of 3,500,000 miles of river covered by a forecast.These spatial gaps include a wide variety of areas such as power generation sites, ecologically sensitive regions, cities and recreation areas-locations where guidance on water flow is critical.Variation in approach also handicaps development efforts, as there is no single, uniform nationwide system which can act as a unified platform for model development.
Lastly, the current suite of regional RFC hydrologic models are manually labor-intensive and do not represent the full set of coupled inland-coastal-cryosphere-groundwater processes necessary to fully capture the hydrologic cycle.While the streamflow forecasts from these models are, in and of themselves, highly useful, the holy grail for emergency responders and many end users is flood inundationnamely the extent of the ground which is covered by flood waters.Without direct knowledge of this quantity via maps of surface inundation for all portions of their area of interest, end users lack key, actionable intelligence allowing them to understand and react to an ongoing or forecast flood event.
Against this backdrop, the National Academy of Sciences (NAS) issued a report in 2012 with guidance on the improvements needed in the NWS hydrologic forecasting enterprise (National Research Council, 2012).A sampling of the critical recommendations includes: • Implement a new hydrologic modernization effort.
• Improve pathways for collaboration and accelerate research to operations efforts.
• Transition RFC forecasters away from the execution of labor-intensive models and towards an "over the loop" role, enabling a shift in focus to model and product development, forecast interpretation, and decision support.
• Add value to hydrologic forecasts through the use of more advanced models, data assimilation (DA) and employment of more sophisticated ensemble techniques.
Combined with the previously noted forecast system shortcomings, these NAS recommendations firmly established the outlines for a new NWS operational hydrologic forecasting capability-the NWM.These outlines were translated into a physical system via a timely combination of five main resources: 1.The Integrated Water Resource Science and Services (IWRSS) consortium-consisting of Federal Water Agency partners (NOAA, USGS, USACE, FEMA) and seeking to develop shared plans for a virtual or physical center to advance water resources prediction nationwide; 4. The NWS-CUAHSI Summer Institute program at the National Water Center-providing a testbed for continental-scale real-time hydrologic modeling; and 5. Congressional programmatic mandates to improve hydrologic forecasting-providing the resources needed to accomplish hydrologic modeling and product modernization These driving forces provided the comprehensive mix of science, programmatic, staffing and funding resources needed to create the operational NWM hydrologic modeling system.This system delivers guidance which complements the essential and skillful forecasts already produced by RFCs, as well as addresses the needs of an expanded set of end users and critical mission areas.

| OPER ATIONAL MODEL CONFIG UR ATION
The operational configuration of the NWM was driven by the wide variety of end user needs including flood and drought forecasting, water resource management and emergency response.Several configurations of the NWM are run operationally on the NOAA Weather and Climate Operational Supercomputing System (WCOSS) computing platform over the range of modeling domains depicted in Figure 1.These configurations include: (1) A simulation of the past several hours using the best available observed forcings and streamflow DA to establish initial conditions for forecasting, which we call the "CONUS Analysis and Assimilation" cycle, (2) CONUS Short-Range 18 h deterministic forecast, (3) CONUS Medium-Range 10 day ensemble forecast, (4) CONUS Long-Range 30 day ensemble forecast, (5) Hawaii and Puerto Rico/USVI Analysis and Assimilation current snapshot, and (6) Hawaii and Puerto Rico/USVI 48 h short range forecast (Table 1).Open-loop (no DA) configurations of the Analyses, CONUS medium-range, and island short-range forecasts are executed as well.

| Continental United States configurations
The accuracy of hydrologic forecasts are heavily influenced by the quality of the model initialization (Li et al., 2009).As such, the various analysis and assimilation (AnA) configurations which initialize each NWM forecast play a critical role in the modeling process.Over the United States, these initial conditions come from a set of three separate AnA cycles-two linked to the NWM short-and medium-range forecasts, and one linked to the NWM long-range forecast.Each of these analysis cycles is started from the previous AnA iteration, making them continuously linked rather than event-based in nature, and eliminating the need for spin-up simulations.Further details of these configurations are given in Appendix A.
The Short-Range Forecast configuration produces guidance for forecasters and emergency responders who deal with events such as flash floods and other situations which can rapidly evolve based on changing meteorological and hydrologic conditions.It cycles hourly, is forced with meteorological data from the High-Resolution Rapid Refresh (HRRR) and Rapid Refresh (RAP), is initialized from the latest AnA solution, and produces hourly deterministic forecasts of streamflow and hydrologic states out to 18 h.The Medium-Range Forecast configuration which is executed four times per day at 00Z, 06Z, 12Z, and 18Z, is initialized from the latest AnA solution and is forced with Global Forecast System (GFS) model output.Unlike the deterministic Short-Range configuration, the Medium-Range configuration produces ensemble guidance.Member 1 of this configuration is forced with model output from the most recent GFS NWP model run, and extends out to 10 days.While starting from the same set of initial conditions as member 1, members 2-7 ingest time-lagged forcing data from successively older forecast cycles of the GFS.These six members extend to 8.5 days, creating an overall seven member equal-weighted medium range ensemble forecast with an 8.5 day forecast horizon.At 30-days in length, the longest set of NWM forecasts is produced by the Long-Range Forecast.As with the Medium-Range configuration, the Long-Range configuration is a forcing-based ensemble which cycles at 00Z, 06Z, 12Z, and 18Z.However, it differs in that it consists of four members which are each forced with a different Climate Forecast System (CFS) forecast member from a single CFS model cycle.
Adding to the standard configurations outlined above, several open-loop or "no-DA" configurations are executed as well.These open loop analyses lack the assimilation of USGS stream gauge observations that occur in the standard analysis, and provide end users with insight into the native model behavior, unaffected by observation-based corrections.These open loop analyses are also used to initialize corresponding CONUS medium-range, and island short-range forecasts.
All of the analyses and forecasts are executed with the same 1 km resolution land surface grid, and use the same NHDPlus-based hydrofabric supporting routed channel output at over 2.7 million stream reaches.Additionally, the Analysis, Short-, and Medium-Range simulations rely on identical 250 m CONUS routing grids to perform surface and sub-surface (non-channelized) routing.

| Hawaii and Puerto Rico/USVI configurations
The NWM v2.1 operational Hawaii and Puerto Rico/USVI configurations feature several important differences from the CONUS configurations discussed above.These differences are driven by the selection of observation-and model-based forcing data which is available for these island domains.The most significant difference is that only Analyses and Short-Range forecasts are produced-the latter extending out to a 48-h forecast horizon.Further, whereas the CONUS Analysis and Short-Range cycles are driven in part by data from the HRRR and RAP models, the island configurations draw meteorological input from (1) Shortwave and Longwave Radiation: North American Model Nest (NAM-Nest; Carley et al., 2017;Rogers et al., 2017) and (2) Remaining Fields: WRF-ARW models.The final difference lies in the spatial resolution of the Hawaii and Puerto Rico NWM configurations.While they rely on the NHDPlus-based hydrofabric and feature the same 1 km base land surface grid resolution, the surface and subsurface routing grid is set at 100 m, versus the 250 m resolution used over the CONUS, reflecting the high topographic relief of the islands.As occurs in operations over the CONUS, real-time USGS streamflow observations are assimilated into the Island Analysis configuration.Additionally, MRMS precipitation is ingested as forcing into the Hawaii Analysis, while the Puerto Rico/ USVI Analysis relies upon the HIRESW-with both island Analysis configurations using precipitation from the NAM-NEST when their primary sources are unavailable.The remaining Analysis forcing fields are obtained from the NAM-NEST.

| NWM S TRUC TUR AL OVERVIE W
Functioning across the variety of domains and forecast configurations described above, the NWM is a multi-faceted modeling system as opposed to a singular hydrologic process model.It consists of the community WRF-Hydro modeling system, a detailed geospatial-hydrofabric set of data layers, a Meteorological Forcing Engine (MFE) and an automated model calibration system.Each of these components is flexible and extensible, with structural support for the addition of new modular process modeling capabilities.WRF-Hydro is a community modeling framework that was initiated in 2003 as the 'Noah-distributed' modeling system (Gochis & Chen, 2003) and encompasses several modules and datasets, which together allow for efficient and robust, continental-scale, high-resolution simulations of the Nation's hydrologic systems.Representations of infiltration, soil hydraulic physics, evapotranspiration, snowpack accumulation and ablation, lateral overland and subsurface flow, baseflow and river channel flow exist inside the model within various model components.
These processes can each run on different spatial frameworks (e.g.rectilinear grids, catchments, river reach vectors and reservoir objects) thus providing flexibility in process representation and computational efficiency.The multi-scale WRF-Hydro system has been developed for application on large clusters and high performance computing systems so is applicable to large-domain, high-resolution simulations.It is currently used for a wide range of hydrometeorological research and operational forecasting applications (Arnault et al., 2021;Givati et al., 2016;Lahmers et al., 2019;Pal et al., 2020;Rummler et al., 2019;Senatore et al., 2015Senatore et al., , 2020;;Verri et al., 2017;Xiang et al., 2017;Yucel et al., 2015).

| NWM hydro-geo fabric
Each of the components within WRF-Hydro are supported by a detailed hydro-geospatial fabric of inputs-the means by which the modeling system represents the geometry and topology of river networks and the description of the land surface and shallow subsurface at scales appropriate for NWM simulations and end-use applications.The NWM V2.1 modeling domain extends from central Mexico to central Canada, and includes the major Hawaiian Islands, Puerto Rico, and the U.S. Virgin Islands (Figure 1).Land surface processes are modeled using a 1 km gridded resolution over all domains, with surface overland flow and saturated sub-surface flow routed on a 250 m grid over the CONUS, and on a 100 m grid over Hawaii, Puerto Rico, and the U.S. Virgin Islands.In the NWM configuration of WRF-Hydro, channel routing of runoff occurs on a vector-type network of channel reaches (Table 1).The NWM channel routing network is largely derived from the National Hydrography Dataset Plus Version 2 (NHDPlus; McKay et al., 2012), which is based on an integration of the medium-resolution National Hydrography Dataset (NHD), National Elevation Dataset (NED), and National Watershed Boundary Dataset (WBD).While a high-resolution version of NHDPlus has become available, the NWM leverages the 1:100,000 scale flowlines, catchments, waterbodies, gauge associations, and value-added attributes of the Medium Resolution NHDPlus for operations, providing streamflow output for over 2.7 million river reaches nationwide.
While the NHDPlus Version 2.1 dataset underpins the vast majority of the features within the NWM hydrofabric, a number of modifications were necessary to support the spatial requirements of the NWM's extensive domain.The large catchments representing upstream areas in Canada and Mexico were replaced with hydrography and elevation-derived catchments roughly equivalent in scale to NHDPlus.
Additionally, divergent flow paths were eliminated, Strahler stream orders re-calculated, and a continuous flow network was ensured by examining all interior network endpoints and reconnecting the network wherever possible.In this way, a seamless computational river network was constructed which could support operational hydrologic needs across the country.In areas tributary to the U.S., the channel networks and catchment delineations were custom-derived and linked to the NHDPlus networks.In areas beyond tributary regions, no channel routing is performed in the NWM.The USGS-World Wildlife Federation 90 m 'Hydrosheds' topography dataset was used in these regions where NHDPlus data was unavailable (Lehner et al., 2008).

| Column land surface physics
The core physics of the multi-scale WRF-Hydro modeling system consists of multiple process modules including a column land surface model, modules for overland and saturated subsurface lateral flows, a bucket-type parameterization for baseflow, a channel flow module, and a reservoir routing module.The column land surface model used in the NWM is an updated version of the community NoahMP land surface model (Niu et al., 2011;Yang et al., 2011).NoahMP is itself a multi-physics model with numerous process representation options for the accounting of canopy energy exchange, snowpack, runoff and infiltration, groundwater, dynamic vegetation, surface aerodynamic exchange and land cover description.The time-step for the execution of NoahMP is identical to that of the meteorological forcing input which is 60 min.

| Overland flow formulation
While the NoahMP land surface model is configured to run on a 1 km rectilinear grid, the overland and saturated subsurface flow routines execute on a 250 m CONUS (100 m over the Hawaii and Puerto Rico/Virgin Island domains) rectilinear grid that is forced to edge-match the 1 km grid.Infiltration-capacity excess from NoahMP which exceeds a pre-defined retention depth value is allowed to flow overland using an option within WRF-Hydro that is an explicit formulation of the diffusive wave simplification of the full St.Venant equation for overland flow, similar to Julien et al. (1995;Gochis et al., 2020).The one-dimensional steepest descent gradient estimation option within WRF-Hydro is used in the NWM to minimize forecast latency.To satisfy Courant constraints in the diffusive wave formulation, when executed at a 250 m grid spacing an overland flow routing time step of 10 s is used.Overland flow, with depths exceeding the specified retention depth, and intersecting 250 m grid cells that have been designated as 'channel' grid cells, is sequestered as channel inflow.Overland flow-derived channel inflow for each NHDPlus channel reach is aggregated from all channel grid cells within each NHDPlus catchment associated with each NHDPlus channel reach.We note here that the 250 m overland and subsurface (described below) routing functions are used in all NWM configurations except the long-range, 30-day forecast configuration due to computational constraints in completing the long-range forecasts.Instead of performing the high-resolution overland and subsurface routing, the long-range configuration simply aggregates surface runoff and base-flow across NHDPlus catchments and the stream inflow contributions.This more simplistic formulation greatly reduces computational expense in forecast production but both reduces the kind of outputs produced by the model and also requires a separate calibration due to differing process representations.]

| Saturated subsurface flow formulation
Similar to overland flow, shallow saturated subsurface flow is executed on a 250 m rectilinear grid (100 m over the island domains).A Boussinesq approximation is employed as in Wigmosta et al. (1994) which results in an effective 2-dimensional calculation of saturated subsurface lateral transport.In versions 1.0-2.1 of the NWM, subsurface soil saturation characterization is determined from the 2 m deep, 4-discretization soil moisture representation in NoahMP.In this soil column, the water table is defined as the top-most discretization which features saturated soil.
As such, the subsurface flow routine in WRF-Hydro does not represent perched lateral transport.The 60-min time-step of saturated lateral flow is synchronized with the execution of the NoahMP time step.Soil moisture accounting following lateral transport can result in full column saturation and exfiltration of soil water to the land surface where it may subsequently participate in overland flow.Numerical details of the saturated subsurface flow formulation are provided in Gochis et al. (2020).

| Baseflow formulation
Deeper groundwater contributions to streamflow are represented using a relatively simplistic, non-linear, conceptual baseflow bucket model.
The spatial unit of the baseflow buckets is that of the uniquely-defined NHDPlus catchments.Inflow into the buckets is derived from drainage through the NoahMP soil column and is calculated as a scaled free-drainage boundary flux.A spatial weighting of NoahMP grid cells encompassed within the NHDPlus catchments is used to estimate which grid cells and fractions of grid cells contribute soil column drainage to each bucket.The baseflow bucket then discharges baseflow to the upstream node of the corresponding NHDPlus stream segment associated with each NHDPlus catchment using an empirical exponential storage-discharge formulation, where the discharge from the bucket to the river channel is specified as a function of the fractional storage capacity of the bucket.

| Channel flow
Channel inflow components coming from overland and baseflow are ingested into the channel at the upstream node of the relevant NHDPlus channel reach.Channel flow is then estimated using the iterative Muskingum-Cunge hydrologic routing formulation adapted from Chow (1959) as described in detail by Read et al. (2023), and flows are routed down the NHDPlus-defined channel and reservoir network.

| Reservoir accounting
Given the thousands of reservoirs spread across the country, and the degree to which they can impact streamflow across all flow regimes, accurate representation of reservoirs is critical for NWM analyses and forecasts.More than 5000 of these reservoirs are already represented in the NWM, but in a fairly simplistic fill-and-spill manner.This basic approach does not mimic the real-world manual operation of reservoirs, and instead uses fixed orifice discharge and reservoir spillway estimation methods.Given the impractical nature of obtaining and applying thousands of operating functions, a new module was developed in NWM v2.0 to simulate the anthropogenic impacts of reservoirs with a twopronged automated approach for use at several hundred of these locations.The approach uses a persistence-type treatment whereby USGS and USACE-supplied streamflow measurements at, or immediately downstream of, reservoirs are used to correct NWM reservoir states and releases.Additionally, RFC-supplied observations and forecasts of reservoir outflow are assimilated into the NWM for use in the NWM analysis and short-and medium-range forecasts (Figure 2a).Future versions of the NWM will leverage this flexible ingest mechanism to assimilate RFC-sourced observations and forecasts of diversion-related flow.

| Real-time streamflow DA
The NWM has assimilated real-time streamflow observations, specifically volumetric flow rates, since its v1.0 inception.The purpose of assimilating streamflow observations is to help mitigate the impact of background model error propagating downstream of observed flow locations in the NWM's analysis cycle and on its forecast initial conditions (Figure 2b).The value of the NWM streamflow DA in reducing model forecast errors increases proceeding downstream on streams and rivers which have progressively more gauges, due to the influence the gauges have on conditioning upstream flows.given full weight, while observations between 15 min and 2 h old are given proportionately lower weights.Observations older than 2 h are not used.The NWM also relies on the data quality and error flags accompanying real-time USGS streamflow observations.Generally, for well-managed gauging sites with long periods of record and well-established rating curves streamflow observations with high quality rankings can be considered reliable estimates of streamflow.It is acknowledged that there may be instances where site and flow dynamics may be evolving disproportionately and may not be detected until future stream surveys.In such instances, there may be additional uncertainty in the model.Similarly, during the forecast cycles of the NWM, the influence of the observation in the forecast is relaxed as the model iterates away from its initial state.At this initial state, the model flow solution is forced to match the observed flow solution, with the influence of the streamflow observation decaying to zero by 2 h into the forecast.Streamflow observations are applied only on the local stream reach where the observation is made, and then allowed to propagate under the model's flow solution downstream from the observation location.Streamflow DA is underpinned with data drawn from the USGS NWIS data service.While the NWM is able to ingest data from over 8000 gauge sites, the availability of observations varies with time, with many fewer gauge discharge values available during the winter due to frozen river conditions.

| Meteorological forcing prescription
The NWM is not interactively coupled to a parent atmospheric model.Rather, it ingests meteorological forcing from a variety of external sources including Multi-Radar Multi-Sensor (MRMS, Zhang, Howard, et al., 2016) radar-gauge and Stage IV Multisensor Precipitation Estimator (MPE) observed precipitation data, HRRR (Dowell et al., 2022), RAP (Benjamin et al., 2016), NAM-Nest, High-Resolution Window Forecast System Advanced Research Weather Research and Forecasting (HIRESW WRF-ARW; Skamarock et al., 2008), GFS (NWS, 2022b) and CFS (Saha et al., 2014) Numerical Weather Prediction (NWP) forecast data.Each of these forcing sources is ingested into the NWM MFE which is a set of unique configurations of the more general WRF-Hydro MFE (https:// github.com/ NCAR/ WrfHy droFo rcing ).The NWM MFE performs source data ingest, variable and units standardization, spatial interpolation, downscaling, statistical bias correction, product layering, and file output.The NWM requires seven meteorological variables; precipitation rate, 2-m air temperature, 2-m specific humidity, 10 m wind speed, surface pressure, incoming shortwave radiation, and incoming longwave radiation.Because all of the meteorological variables come from gridded analyses or other models whose grid structures are different from the NWM, each variable is interpolated to the 1 km NWM land surface model grid.In addition to spatial interpolation, several variables are subjected to downscaling and/or statistical bias correction within the MFE.When downscaling is needed, temperature, humidity and pressure terms are downscaled using climatological lapse rate adjustments while incoming shortwave radiation is downscaled using terrain adjustments to incoming solar radiation for terrain slope and aspect variations (terrain shading is not performed) on the 1 km model grid (Garnier & Ohmura, 1968;Zangl, 2005).A description of each variable, its required units and the processing operations applied to it are provided in Table 2.The MFE also applies a preferential layering approach to incoming precipitation datasets, the details of which are provided in Appendix A.
The Long Range configuration of the NWM utilizes statistically-processed model output from the CFS as forcing.The availability of a long reforecast archive of the CFS system provides a large, stable climatological database that makes the enhanced bias correction of CFS data possible.NCEP CFSv2 outputs covering the full set of required NWM forcings are first post-processed to correct biases in the raw CFSv2 data, and then downscaled and disaggregated to the standard space-time resolution of NWM MFE inputs.This bias correction approach is based on the quantile mapping (Panofsky & Brier, 1958;Wood et al., 2002) of CFSv2 outputs to match the North American Land Data Assimilation (NLDASv2) climatology (Cosgrove et al., 2003;Xia et al., 2012).Parametric quantile mapping is used for all variables, with a Dirac delta function to model instances of zero precipitation, and a Weibull distribution to model non-zero precipitation.A Gaussian distribution is used for 2-m temperature, surface pressure, and downward longwave radiation; a gamma distribution for specific humidity and wind speed; and a mean scaling of non-zero shortwave radiation.The parameters of the distributions are estimated on the CFSv2 grid and varied for all grid points across forecast hours for each specific initialization time, with a centered ±2-day moving window applied to increase sample sizes.This bias-correction procedure yields four cycles per day of bias-corrected CFS data which then force the 6-hourly NWM Long-Range ensemble.
TA B L E 2 Description of each NWM forcing variable, required units and processing operations applied for the AnA, Short-Range Forecast, Medium-Range Forecast (MRF), and Long-Range Forecast (LRF) configurations.

| Model parameter specification
As a process-based, high resolution, spatially-distributed hydrological modeling system, the NWM has a very large number of model parameters which must be specified.The parameter specification process has three primary steps: 1. Initial parameter specification.

| Calibration and regionalization
The Across the CONUS domain, a total of 14 model parameters (Table 4) are calibrated with an iterative Dynamically Dimensioned Search approach (Tolson & Shoemaker, 2007).A similar set of parameters, minus the MFSNO melt factor, are calibrated for the Hawaii and Puerto Rico domains.In this technique, the model is cycled over the calibration period 300 times to minimize an objective cost function.For the NWM, the objective cost function is a weighted Nash-Sutcliffe Coefficient of Efficiency (NSE) consisting of equal parts NSE (Nash & Sutcliffe, 1970) and NSE calculated for the log of the discharge (NSElog) using hourly streamflow observations.Once direct calibration is complete, parameters for the rest of the NWM domain are then derived via a regionalization process.
The regionalization process transfers optimized parameters from calibrated basins to the portions of the domain which cannot be directly calibrated due to a lack of observations or flow complications from water management.The basis for this procedure is the concept that basins featuring similar vegetation, topography, geology, soil properties and climate will have similar hydrologic parameters, and will be characterized by similar streamflow responses.To accomplish this regionalization, one of two techniques is applied, as determined through multi-year test simulations.For the full-physics NWM configuration (i.e. with the 250 m overland and subsurface routing functions active), the degree of similarity between each uncalibrated and calibrated basin is measured with the Gower's distance metric (Gower, 1971), while for the long-range NWM configuration an ecoregion-based approach is leveraged.Once matches are made between the directly calibrated donor basins and the uncalibrated receiver basins, parameters are mapped to the remainder of the NWM domain.

| NWM OUTPUT
Output from the NWM encompasses a wide range of hydrologic variables, which vary between modeling configurations based on end user needs and computational resource limitations.Real-time output totals nearly 1 Tb per day and is produced in a fully automated 24 × 7 operational fashion, the cadence of which varies by forecast configuration.By contrast, only one long-term retrospective run is released with each NWM version.Forced with AORC meteorological data, and executed without DA, the NWM v2.1 long-term retrospective run spans the period 1979-2020.
As extensive as the list of NWM output fields is, it is of very limited use unless it's paired with effective dissemination and visualization services.Towards this end, OWP has partnered with NCEP, NCAR, CUAHSI, and Big Data partners to leverage a set of foundational capabilities that will expand over time.The chief outlet for basic distribution of NWM data is the NOAA Operational Model Archive and Distribution System (NOMADS) hosted by NCEP.This publicly accessible system distributes data from models run operationally on the NOAA WCOSS, including the NWM.Full-domain NetCDF NWM output files are hosted on NOMADS and are made available for download via ftp and https protocols.Complementing this basic distribution are post-processed files, targeted for ingest into the NWS RFC operational forecast environment.
In addition to these operationally supported data sites, there exist a growing number of joint data ventures with both academic and private entities.These collaborations with Big Data partners and CUAHSI are especially important, given the challenges associated with distributing and processing the large amount of data produced by the NWM.

| MODEL PERFORMAN CE
NWM simulation performance is assessed through a variety of approaches and metrics.Output from real-time forecast, historical refore- Yes version-over-version evolution as new features are added (Table 5).Together they help focus development efforts on where they are most needed, and inform forecasters as to the level of confidence which characterizes NWM forecasts.

| Streamflow verification-Retrospective output
Assessment of NWM streamflow is based on comparison against USGS hourly streamflow observations.As with verification of meteorological models, the number of model output points (over 2.7 million) dwarfs the number of available observation sites (~10,000).However, the sequentially connected nature of many of the stream reaches acts as a mitigating factor, allowing downstream gauges to be used to verify, by proxy, nearby upstream stretches of the river.This effectively reduces the spatial gap in verification data, but does not address another verification challenge-anthropogenic influences.Given the profound impact that dams, diversions and other similar elements have on natural streamflow, USGS data are divided up into two NWM verification sets: (1) The full USGS observation collection of approximately 10,000 stations, and (2) A reference stream gauge dataset from the Geospatial Attributes of Gages for Evaluating Streamflow version II (GAGES-II; Falcone, 2017) database, which includes a subset of non-or lightly-regulated streams (approximately 1600 stations).
Using the full USGS dataset as the baseline, Figure 4 displays NWM hourly percent bias values as CONUS-aggregate histograms.Output is from a continuous retrospective simulation using the full-routing formulation of the model.This validation run was executed from 2013 to 2016, directly after the 2008 to 2013 NWM calibration period, and separate from the long-term 1979-2020 retrospective run.It was driven with AORC meteorological forcing data, and did not assimilate USGS streamflow observations.Progressing from NWM V1.0 through V2.1, each new version of the NWM has seen a related reduction in simulated streamflow bias.In particular, while 31% of verification sites have a bias of ±20% (the most favorable category) in V1.0, 52% of those same sites reach that level of accuracy in V2.1.This improvement is accompanied by a favorable tightening of the histogram distribution, with a decrease in wet and dry extremes.The spatial distribution of the errors are somewhat uneven, with wet biases more prevalent in the middle of the CONUS than in other regions.Removing the influence of streamflow regulation from the assessment, Figure 5 displays bias statistics from the same simulation, but this time validated against the GAGES-II subset of observations.The same general findings hold, although performance levels are predictably higher given the lack of complications introduced by reservoirs and diversions.Bias levels of ±20% are seen in 36% of validation locations in V1.0, increasing to 68% by V2.1 of the NWM.
An assessment of hourly model correlation and normalized Nash Sutcliffe Efficiency (NNSE, Nossent & Bauwens, 2012) provides additional insight into NWM performance.Verification results at GAGES-II locations are presented in Figures 6 and 7 using an approach similar to that taken with bias.As with bias, a strong improvement in results is seen version-over-version.While only 18% of verified reaches have a correlation of 0.8 or greater in V1.0, improvements between versions leads to a value of 52% by V2.1 (Figure 6).Similarly, only 43% of verification basins have an hourly NNSE of 0.6 or greater in v1.1, rising to 81% in v2.1.The spatial variation in results for both metrics is also somewhat similar to that of bias, with lower values noted over the midsection of the CONUS.
Verification of the simplified NWM configuration used in Long-Range CONUS applications (i.e.without the 250 m terrain routing) was also conducted for GAGES-II locations over this same retrospective time period.Summarizing results from V1.2 through V2.1, Figure 8 shows the improvements in streamflow correlation between model versions.Overall, model results are less favorable than those seen in Figure 7, which is to be expected given the lack of explicit surface and sub-surface routing in this simplified, long-range configuration of the NWM. the version in which it was implemented.Here, 51% of the verification sites show a streamflow bias of less than ±20%, while 48% (79%) of sites display an hourly (daily) streamflow correlation greater than 0.6 (Figure 10).

| Snowpack verification
The NWM contains a multi-layered, energy-balance-based snowpack formulation which tracks snowpack conditions on the 1-km LSM grid over the entire model domain.The performance of the snowpack representation in each model version has been evaluated against Natural Resources Conservation Service (NRCS) SNOwpack TELemetry (SNOTEL) station observations and against the NOAA OWP gridded Snow Data Assimilation System (SNODAS) analysis product (Carroll et al., 2006).Figure 11 shows a comparison of model performance between v2.0 and v2.1 of the NWM against these observations for retrospective run spanning 2011-2017.Figure 11a shows that, in general, a modest bias Improvements in snowpack conditions are attributed to improvements in the snow physics formulation related to snowpack storage and release of melt water as well as to improved calibration of some of the snowpack model parameters during automated calibration.

| Streamflow verification-Real-time operational output
Of equal importance with the preceding suite of verification statistics derived from the assessment of NWM retrospective validation simulations is a parallel set of statistics based on real-time operational NWM forecast output.These assessments yield information which then guides use of the model by forecasters and other end users.Unlike the historical runs which are forced with observed precipitation, real-time NWM forecasts are forced with NWP-based precipitation.This changes the error profile of the model significantly, with input precipitation now an increased source of error.
Focusing on the short-range configuration, Figure 12 depicts the improvements in performance between NWM versions 2.0 and 2.1.The sizable overestimation of peak discharge in v2.0 (especially at longer lead times) is greatly reduced in v2.1, with the distribution of the median absolute percent peak bias centered near zero.This is mirrored by improvements in event-type statistics including probability of detection (POD), false alarm ratio (FAR) and critical success index (CSI).Peak streamflow timing error is similar in v2.1 and v2.0, with the forecast streamflow too quick to peak.
As highlighted in Figure 13, overall improvement is also noted in the medium-range real-time forecast configuration.Peak bias and peak timing errors improve during the first 6 h, but change little in the hours that follow.Model improvement is more pronounced when examining categorical flood prediction scores.Here, notable decreases are seen in the FAR, accompanied by sizable increases in the CSI.Similar results are found for days 4-10, with the medium-range ensemble means exhibiting higher scores (not shown).Focusing on the performance of The NWM's second domain outside of the CONUS-Puerto Rico-is new for v2.1, and so there is no prior NWM version against which to compare results.However, it is still instructive to examine the performance of this first implementation.Figure 16 shows that peak discharge is generally underestimated across all lead times, while the median peak timing error is relatively small (median within ±3 h).Assessing the output from a categorical event perspective, it can be seen that this first implementation of the Puerto Rico domain displays reasonable categorical flood forecast skill, with a POD of 0.7 and a FAR of 0.42.These scores support initial use of the model, complementing information streams that are already available, and future versions of the NWM will improve upon these scores.
An in-depth comparison of NWM and RFC-produced streamflow forecasts has the potential to increase understanding of the relative merits of each approach and aid in the effective use of each source of data.Such a study would provide insight into the use of a complementary mix of data, across various hydrologic situations in time and space.However, a meaningful intercomparison is hampered by the many differences between NWM and RFC forecast systems, as well as from RFC-to-RFC.These increase the complexity of an assessment and interpretation of the results to such an extent that it moves beyond the scope of this overview article.In particular, RFC forecasts are produced over differing spatial regions, at a different cadence, using differing input forcing data sources and with forecaster modifications to forcing inputs, model states and parameters.A direct and informational comparison against the NWM would entail conducting reforecasts or reanalyses with both platforms using the same forcing data, but would still fail to capture important differences in model performance for the aforementioned reasons.Until completion of a study focusing exclusively on such an in-depth and multifaceted assessment, the summary statistics in this NWM overview article serve to highlight the current performance characteristics of the NWM, and provide both a basis for use and a foundation for a future, in-depth intercomparison article.

| Operational enhancements
Established in 2016, the NWM is a relatively young modeling system.The NWM continues to undergo cyclic upgrades so as to improve and expand version-over-version capabilities guided by the assessments described above.Certain foundational components of the NWM are improved with each upgrade, while other components are added for the first time, or may only receive sporadic updates.Improvements which accompany each new model version include updated calibrated parameters, fixes to the hydro-fabric connections and waterbody attributes, along with updates to the workflow, model and MFE to improve execution robustness and efficiency.
Nowhere is the drive towards improved representation of hydrologic processes more important than along coastal regions.In this area, over 100 million people currently lack a forecast of the integrated impacts of freshwater, storm surge, waves and tidal flooding.Within version 3.0 of the NWM, a new routing capability will support linkage of NWM freshwater modeling capabilities to a coastal-estuary model, the Semiimplicit Cross-scale Hydroscience Integrated System Model (SCHISM) (Zhang, Ye, et al., 2016).Atmospheric forcing will be drawn from the existing set of NWM forcing data described above, while ocean forcing will be drawn from the Surge and Tide Operational Forecast System (STOFS; Funakoshi et al., 2012) and Probabilistic Tropical Storm Surge (P-SURGE; Taylor & Glahn, 2008) models.This approach will be applied Service provision will be further improved via first-time NWM coverage for Alaska, wherein an NWM configuration is scheduled for operational deployment over the Cook Inlet and Copper River Basin regions (Figure 17).Several upgrades were introduced into the NWM in order for it to better function in this cold land process-dominated area.Chief among these are a linkage with a snow-ice model (Vionnet et al., 2012) and the capability for ingest of RFC-provided forecasts of streamflow from glacial lake outbursts.Combined with forcing and hydrofabric datasets tailored to the needs of this particular domain, this implementation will bring much-needed comprehensive and complementary hydrologic guidance to the south-central Alaska region.
Complementing the foundational enhancements above will be a shift to updated land cover and channel datasets, improved treatment of infiltration, the creation of a dynamic parameter update capability, and use of the National Blend of Models as an input forcing source.
Taken together, the wide-ranging process modules and forcing enhancements discussed above combine with the NWM's underlying distributed modeling structure to support key nationwide hydrologic applications in a way not before possible, and connect back to the driving factors behind the NWM's creation.A prime example of this is the advent of coast-to-coast flood inundation mapping (Aristizabal et al., 2023).
Critical for emergency managers in times of extreme flow, this application leverages NWM analyses and forecasts to provide distributed inundation information down to the neighborhood structure level.The NWM's fine scale NHD-based stream network, along with its representation of multiple processes including coastal total water level, supports seamless production of this information from summit to sea, and will continue to evolve with each iteration of the NWM.

| Community assessment and development
To date a number of published works have documented various performance aspects of the NWM.These works have analyzed several aspects of the skill of reservoir inflow forecast performance (Viterbo, Read, et al., 2020) and snow model performance (Garousi-Nejad & Tarboton, 2022).Viterbo, Mahoney, et al. (2020) explored the ability of the NWM to forecast specific high-impact flooding events while Tijerina et al. (2021) provided a multi-metric assessment of NWM model performance in comparison with another state-of-the-art hydrologic model.The research community surrounding the NWM has also produced a number of other papers centering on development aspects for specific processes within the NWM, including the improvement of the NWM for semi-arid environments (Lahmers et al., 2019(Lahmers et al., , 2022)), the development of hydro-geo-fabric data for the Great Lakes region (Mason et al., 2019), the assimilation of remotely sensed vegetation data (Elmer et al., 2022), the enhancement of channel routing physics (Read et al., 2023) and the impact of tile drainage processes on NWM simulation performance (Valayamkunnath et al., 2022).

2.
The new National Water Center in Tuscaloosa, Alabama-acting as a catalyst and collaborative focal point for hydrologic research and operations support (Title III of Public Law No: 116-271-Coordinated Ocean Observations and Research Act of 2020, 2020); 3. The creation of the Weather Research and Forecasting (WRF) Hydrological Modeling Extension (WRF-Hydro; Gochis et al., 2020) at the National Centers for Atmospheric Research (NCAR)-providing a flexible, community-based hydrologic modeling foundation; A simplified, computationally-efficient nudging DA was selected for use in the operational NWM.Real-time streamflow observations from the USGS NWIS data service are quality screened and directly assimilated, or inserted, into the analysis cycle of the model.Assimilation temporal weighting factors are applied to the observations, which relax the model solution back to its open-loop, or unassimilated, value.Streamflow observations deemed to be of high quality that were recorded within 15 min of the model analysis time are

F
I G U R E 2 Impact of assimilating River Forecast Center (RFC)-supplied reservoir outflow data (a; left) and USGS streamflow observations (b; right) on NWM v2.1 medium-range streamflow forecasts.

3.
Parameter regionalization.Initial parameter specification is performed during model setup using landscape classification information for soils, vegetation, land use, channel hydrography, lakes and reservoirs.The datasets for these landscape classifications come from a variety of data sources which are listed in Table3.Initial model parameter estimates are spatially-mapped to the NWM using these classifications and model parameter tables which associate model parameters with a specific classification (e.g.leaf area index, soil hydraulic conductivity, soil porosity, canopy roughness heights, etc).There currently is no sub-grid land cover/vegetation or soils type classification within the NWM, and each model grid cell has a unique classification and set of parameter values.TA B L E 3Land surface datasets used in NWM v2.1.Depiction of basins directly calibrated using streamflow observations in NWM v2.1.
significant physics and parameter upgrades inherent in each version of the NWM necessitate recalibration of the NWM before each version enters into NWS operations.The calibration period for the CONUS domain of the current operational version of the NWM (v2.1) extends from October 2008 through September 2013, with a validation period spanning October 2013 through September 2016, and 2007 used for spin-up.Meteorological forcing is drawn from the Analysis of Record for Calibration (AORC) dataset developed by the NWS OWP (Fall et al., 2023).To avoid complications introduced by water management activities, direct calibration is limited to 1365 lightly regulated basins selected from USGS, California Department of Water Resources (CADWR) and Canadian sites (Figure 3).Complementing this CONUS calibration, 47 basins are calibrated over Puerto Rico and 29 basins are calibrated across the Hawaii Islands, each using multi-year NAM-Nest-based forcing datasets unique to those domains, with observed precipitation supplied by the local RFC.The calibration and validation periods for Hawaii are, respectively, January 2005 to January 2010 and January 2010 to January 2014.Similarly, for Puerto Rico, calibration spans January 2009 to January 2014, with validation from January 2014 to September 2017.
cast, and retrospective analysis runs each provide a different window into NWM skill, and so are all assessed as part of the research to development to operations pipeline.These assessment activities document not only the skill of a particular version of a model but also the TA B L E 4 List of model parameters calibrated in NWM v2.1.Calibration scalar multiplier column indicates whether the parameter is adjusted uniformly within a calibration basin by a scalar multiplier during the parameter optimization process.
Complementing the CONUS verification is a parallel assessment covering the NWM domains outside of the CONUS, namely Hawaii and Puerto Rico.Over the relatively new Hawaii domain, great strides in performance were made between V2.0-when the domain was first TA B L E 5 Overview of major new features implemented within each version of the NWM.border, first detailed calibration of parameters, increased forecast cycling frequency and length V1.2 (2018) Improved calibration and streamflow data assimilation, additional domain expansion along CONUS border V2.0 (2019) Expansion to Hawaii, seven-member medium-range CONUS ensemble, use of RFC MPE precipitation, improved calibration, increased reservoir representation, improved forcing downscaling and model physics V2.1 (2021) Expansion to Puerto Rico/US Virgin Islands and Great Lakes Drainage domains, implementation of reservoir outflow data assimilation, forcing upgrades, new open-loop configurations, improved physics and calibration V3.0 (2023) Total water level coastal coupling component, expansion to Alaska, ingestion of National Blend Models forcing, improved runoff module, and improved physics and calibrationintroduced-and v2.1.Figure 9 depicts a significant reduction in the model's overall wet bias, with biases remaining high in areas with significant anthropogenic impacts (e.g.diversions, urbanization) and particularly dry regions.An increase in correlation (not depicted) is noted as well.Across the NWM's other island domain, which includes Puerto Rico and the USVI, strong initial performance metrics are noted in v2.1,

F
Hourly NWM streamflow bias (%) from version 1.0 through version 2.1.Statistics are for the 2013-2016 post-calibration validation period are based on the Gauges II subset of CONUS USGS gauges.F I G U R E 4 Hourly NWM streamflow bias (%) from version 1.0 through version 2.1.Statistics are for the 2013-2016 post-calibration validation period are based on the full set of CONUS USGS gauges.

F
I G U R E 6 Hourly NWM streamflow correlation from version 1.0 through version 2.1.Statistics are for the 2013-2016 post-calibration validation period are based on the Gauges II subset of CONUS USGS gauges.F I G U R E 7 Hourly NWM streamflow Normalized Nash Sutcliffe Efficiency (NNSE) from version 1.1 through version 2.1.Statistics are for the 2013-2016 post-calibration validation period are based on the Gauges II subset of CONUS USGS gauges.improvement in modeled snowpack performance-as quantified by local model minus observed values and aggregated across RFC regional SNOTEL locations-was achieved in v2.1 for most, though not all, regions.Figure11bshows the Central Rocky Mountain, regionally averaged time-series of snow water equivalent for the two NWM versions and the SNODAS product.Modest improvement in v2.1 snowpack accumulation volume is found both in terms of the regional mean and max/min range of values during the 2011-2017 retrospective assessment period.F I G U R E 8 Hourly NWM streamflow correlation from version 1.2 through version 2.1.Statistics are for the 2013-2016 post-calibration validation period are based on the Gauges II subset of CONUS USGS gauges and are computed using the simplified Long-Range configuration of the NWM.F I G U R E 9 Hourly NWM streamflow bias (%) for versions 2.0 and 2.1.Statistics are for the 2010-2013 post-calibration validation period are based on the full set of Hawaii USGS gauges.

F
Hourly NWM streamflow bias (%) and correlation for version 2.1.Statistics are for a 2009-2017 evaluation period are based on the full set of Puerto Rico USGS gauges.F I G U R E 11 Assessment of NWM v2.0 and v2.1 snowpack against SNOwpack TELemetry (SNOTEL) (percent bias, left; a) and Snow Data Assimilation System (SWE volume, right; b) data for 2011-2017 retrospective assessment period.In (b), line colors denote model, line styles denote statistic.streamflow prediction downstream from reservoirs, Figure 14 isolates the impact of upgrades made to the reservoir module.In these two representative examples, the application of observation persistence and the ingestion of RFC-sourced reservoir forecasts in NWM v2.1 lead to sizable benefits.Switching focus to the NWM's offshore domains, large version-over-version improvements in Hawaii streamflow prediction skill are shown in Figure 15.The large overestimation of peak discharge in v2.0 is significantly reduced in v2.1 across all lead times, and the peak timing error F I G U R E 1 2 Median peak bias (%, top left), median peak timing error (hours, top right), and categorical flood verification metrics (bottom right) for NWM v2.1 versus NWM v2.0, for NWM short-range configuration.Calculated with data from July 1 to August 9, 2020.F I G U R E 1 3 Median peak bias (%, top left), median peak timing error (h, top right), categorical flood verification metrics (bottom left) for member 1 of the NWM medium-range configuration, with categorical scores calculated for approximately 3000 sites.Additionally, correlation by lead time member 1 and the ensemble mean for both NWM v2.0 and v2.1 (bottom right).Calculated with data from July 1 to August 9, 2020.distribution has a smaller range in v2.1 than v2.0.Additionally, categorical flood detection is greatly improved in V2.1 (similar results for hours 13-48, not shown), with a reduction from 23 false alarms in V2.0 to zero in v2.1.Improvements in the selection and processing of source precipitation data as well as improvements in calibration methods are attributed to this increase in skill over Hawaii.

F
Relative change in NNSE from NWM v2.0 to v2.1 at sites with three different sources of reservoir outflows for both the short-range (top row) and medium-range (bottom row) forecast configurations.Note that the plots show the version-on-version change in NNSE, not the actual NNSE values, and are calculated with data from July 1 to August 9, 2020.F I G U R E 1 5 Event-based assessments by lead time for NWM v2.0 and v2.1 Short-Range forecasts over Hawaii domain.Median peak bias (%, left), median peak timing error (hours, upper right), and table of false alarms (bottom right).Calculated with data from July 1 to August 9, 2020.
over the East, Gulf and Pacific coasts of the CONUS, along with the coastlines of the Hawaii, Puerto Rico, and US Virgin Islands, within the AnA, Short-Range, and Medium-Range forecast configurations.It will provide enhanced guidance to emergency responders and will improve the accuracy of NWM-based flood inundation maps along the coast.F I G U R E 1 6 Event-based assessments by lead time for NWM v2.1 Short-Range Forecast over Puerto Rico domain.Median peak bias (%, left), median peak timing error (hours, upper right), and table of false alarms (bottom right).Calculated with data from July 1 to August 9, 2020.

F
I G U R E 17 Depiction of the new NWM Alaska domain, to be released in version 3.0 of the model.Now at version 2.1, with NWM v3.0 arriving in late 2023, the continued success of the NWM ultimately depends on continual improvement of model accuracy and capabilities-something which, in turn, depends on a vibrant and robust development community with a clear and efficient link to the NWS operational pipeline.With this in mind, OWP is embarking upon a major, multi-year effort, to build on and expand the NWM development and applications community.OWP and agency partners are improving upon the existing modularity of NWM processes and are establishing the Next Generation Water Resources Modeling Framework (NextGen)(Ogden et al., 2021).NextGen is a standardsbased model interoperability framework that allows for the use of spatially varying modeling techniques which are the most appropriate for each region.Utilizing the Basic Model Interface(Hutton et al., 2020) coupling standard, NextGen facilitates linkage to a wide range of community-sourced modules.The NextGen framework also employs the National Hydrologic Geospatial Fabric Reference Hydrofabric(Bock et al., 2022) developed jointly by USGS and NOAA.This reference hydrofabric adopts the WaterML version 2.0 Hy_Features data model(OGC, 2017).The overarching end goal for all of these activities is a system which supports community development and funnels innovation into a common standards-based platform that can be leveraged for both research and operations across a wide range of scales and applications.The NOAA-NWS OWP anticipates that Version 4.0 of the NWM will use the NextGen Framework beginning in late 2025 or 2026.7 | CON CLUDING THOUG HTSWith the NWM in NWS operations since 2016, the hydrologic community has a nationwide platform which will further both operational and research interests.The NWM features forecast horizons from 18 h to 30 days, covers the CONUS and nearby regions along with Hawaii, Puerto Rico and US Virgin Islands, and soon south-central Alaska.It provides hydrologic output for over 2.7 million stream reaches and land surface output on high-resolution 100 m to 1 km grids.The model continues to evolve through version-over-version upgrades, providing powerful guidance to complement the vital forecasts already being produced by the RFCs, and filling in spatial gaps where little to no hydrologic information was previously available-a critical advance, as population growth combined with aging infrastructure increases hydrologic vulnerability.The model's density of coverage ensures a gap filling, quantifiable increase in the proximity of guidance, with a current CONUS domainwide average distance to an NWM output location of less than 975 meters.This supports access not only to relevant NWM streamflow and related hydrologic guidance, but to key advances in seamless flood inundation mapping products.Beyond increases in coverage, the NWM's unique mix of high spatial resolution, multiple forecast horizons, nationwide domain, operational robustness and accessible code allow it to serve as focal point for government agencies across the water spectrum, from NOAA to the USGS, USACE, EPA, USDA, FEMA, NASA, and the USBR.At the same time, increasing links with academia and private industry are providing the foundation for a rich and responsive development environment.Combined with Big Data partnerships and strengthened use of GIS-based services to lower data access barriers, the NWM provides information needed to further the NWS mission of protecting lives and property, and to support a broad range of other hydrologic applications in ways not before possible.AUTH O R CO NTR I B UTI O N S Conceptualization; funding acquisition; methodology; project administration; resources; supervision; writing -review and editing.Ryan Cabell: Brian Cosgrove: Conceptualization; data curation; formal analysis; funding acquisition; investigation; methodology; project administration; resources; software; supervision; validation; visualization; writing -original draft; writing -review and editing.David Gochis: Conceptualization; data curation; formal analysis; funding acquisition; investigation; methodology; project administration; resources; software; supervision; validation; visualization; writing -original draft; writing -review and editing.Trey Flowers: Conceptualization; funding acquisition; methodology; project administration; resources; supervision; writing -review and editing.Aubrey Dugger: Conceptualization; data curation; formal analysis; funding acquisition; investigation; methodology; project administration; resources; software; supervision; validation; visualization; writingreview and editing.Fred Ogden: Conceptualization; formal analysis; investigation; methodology; writing -review and editing.Tom Graziano: Conceptualization; funding acquisition; methodology; project administration; resources; supervision; writing -review and editing.Ed Clark: data curation; formal analysis; investigation; methodology; software; validation; visualization.Nels Frazier: Conceptualization; data curation; formal analysis; investigation; methodology; software; validation; visualization.Camaron George: Conceptualization; data curation; formal analysis; investigation; methodology; software; validation; visualization.Rich Gibbs: Conceptualization; data curation; formal analysis; investigation; methodology; software; validation; visualization.Liliana Hernandez: Conceptualization; data curation; formal analysis; investigation; methodology; software; validation; visualization.Donald Johnson: Conceptualization; data curation; software.Ryan Jones: Funding acquisition; project administration; resources; supervision.Logan Karsten: Conceptualization; data curation; formal analysis; investigation;