Data assimilation experiments inform monitoring needs for near-term ecological forecasts in a eutrophic reservoir

Ecosystems around the globe are experiencing increased variability due to land use and climate change. In response, ecologists are increasingly using near-term, iterative ecological forecasts to predict how ecosystems will change in the future. To date, many near-term, iterative forecasting systems have been developed using high temporal frequency (minute to hourly resolution) data streams for assimilation. However, this approach may be cost-prohibitive or impossible for forecasting ecological variables that lack high-frequency sensors or have high data latency (i.e., a delay before data are available for modeling after collection). To explore the effects of data assimilation frequency on forecast skill, we developed water temperature forecasts for a eutrophic drinking water reservoir and conducted data assimilation experiments by selectively withholding observations to examine the effect of data availability on forecast accuracy. We used in-situ sensors, manually collected data, and a calibrated water quality ecosystem model driven by forecasted weather data to generate future water temperature forecasts using FLARE (Forecasting Lake And Reservoir Ecosystems), an open-source water quality forecasting system. We tested the effect of daily, weekly, fortnightly, and monthly data assimilation on the skill of 1 to 35-day-ahead water temperature forecasts. We found that forecast skill varied depending on the season, forecast horizon, depth, and data assimilation frequency, but overall forecast performance was high, with a mean 1-day-ahead forecast root mean square error (RMSE) of 0.94°C, mean 7-day RMSE of 1.33°C, and mean 35-day RMSE of 2.15°C. Aggregated across the year, daily data assimilation yielded the most skillful forecasts at 1-7-day-ahead horizons, weekly data assimilation resulted in the most skillful forecasts at 8-35-day-ahead horizons. Within a year, daily to fortnightly data assimilation substantially outperformed monthly data assimilation in the stratified summer period, whereas all data assimilation frequencies resulted in skillful forecasts across depths in the mixed spring/autumn periods for shorter forecast horizons. Our results suggest that lower-frequency data (i.e., weekly) may be adequate for developing accurate forecasts in some applications, further enabling the development of forecasts broadly across ecosystems and ecological variables without high-frequency sensor data.


44
In the face of increasing ecological variability due to climate and land use change (e.g., 45 Gilarranz et al., 2022, Malhi et al., 2020), ecological forecasting is increasingly being used for 46 3 understanding and predicting future ecological change (Carey et al., 2022d, Lewis et al., 2022. 47 Here, we define ecological forecasts as predictions of future environmental conditions with 48 quantified uncertainty (see Carey et al., 2022d, Lewis et al., 2022. Applications of ecological 49 forecasts can improve understanding of ecosystem processes (e.g., carbon cycling, Bett et al., lake water temperature, tick abundances, forest net ecosystem production, beetle communities) 58 before the data have been collected (Thomas et al., 2023a). 59 Many near-term (daily to decadal) ecological forecasts are produced using the iterative, 60 near-term forecasting cycle, in which models are updated as new observational data become 61 available to generate forecasts into the future with quantified uncertainty (Dietze et al., 2018). 62 The process of updating forecast models with newly available data, termed data assimilation week-ahead forecasts of reservoir methane emissions with and without weekly DA and found 71 that the accuracy of forecasts with DA was 44 -128% higher than forecasts without DA over a 72 five-month forecasting period. Despite the usefulness of DA for improving forecasts, however, 73 the optimal frequency of observations for updating ecological models to produce skillful 74 forecasts is not well characterized. 75 While there are a number of best practices proposed for applying the near-term, iterative observations. For example, if weekly or fortnightly DA yielded similarly accurate lake dissolved 82 oxygen forecasts as daily DA, then water quality forecasting systems could be developed for 83 lakes that have weekly or fortnightly routine monitoring program data without needing expensive 84 high-frequency sensors, thereby enabling forecasts to be generated for many waterbodies 85 globally. 86 Currently, many automated ecological forecasting systems rely on high-frequency 87 sensors to assimilate data at each time step and generate accurate forecasts (e.g., Baracchini et (Steere et al., 2000). Furthermore, some remotely sensed variables may only be 94 available as satellite orbits and weather conditions (e.g., cloud cover) allow (e.g., Herrick  root biomass, litter fall, soil respiration), each with different collection frequencies, to identify 122 the relative importance of these data sources in constraining long-term carbon dynamics, but did 123 not consider how different frequencies of the same dataset could affect forecast skill. Piazzi et al.

124
(2018) assimilated multiple snow observations at two different frequencies (3 and 24 hours) for 125 predicting different snow-related variables (e.g., depth, density, and snow water equivalent), and 126 Ziliani et al. (2019) performed DA tests using 1-20 second assimilation of water depth data to 127 assess water level forecast skill, but neither considered the effect of less frequent assimilation 128 (e.g., >24 hours). Massoud et al. (2018) performed DA tests using a wider range of temporal 129 frequencies (e.g., ~3-34-day abundance data) to predict plankton community dynamics, but did 130 not consider the effects of DA across spatial scales (i.e., how DA affects forecast skill across 131 multiple sites or depths in an aquatic ecosystem). As a result, further work is needed to quantify 132 the utility of increased observation and DA frequency over both time and space to forecast 133 performance in ecological systems with varying sensitivities to initial conditions.

134
Among ecosystems, freshwater lakes and reservoirs are particularly important systems 135 for developing near-term forecasts because they provide essential ecosystem services, including 136 drinking water, food, irrigation, and recreation (Carpenter et al., 2011, Meyer et al., 1999

168
Forecasting system overview 169 We applied the Forecasting Lake And Reservoir Ecosystems (FLARE) forecasting  Forecast generation via FLARE can be summarized by four steps (Figure 1). First, 10-176 min resolution water temperature data were collected by sensors deployed in the reservoir 177 (Figure 1 step 1). Second, these data were transferred to the cloud and stored in a GitHub 178 repository, where they were downloaded daily and made available for DA (Figure 1 step 2). 179 Simultaneously, 1 to 35-day-ahead NOAA meteorological forecasts were downloaded daily as 180 driver data for the reservoir hydrodynamic model to generate the water temperature forecasts.

181
Third, during the forecast generation step, DA was used to update initial conditions and 182 parameters with the most recent observations using an ensemble Kalman filter, a numerical 183 9 approach that allows for the updating of model states and parameters using data (Evensen, 2003) 184 (Figure 1 step 3a). Following DA, the reservoir hydrodynamic model was initialized with the 185 updated model states and parameters to produce 1-35-day-ahead forecasts for each 0.5 m depth 186 interval across the water column (Figure 1 step 3b). Finally, forecast skill was assessed by 187 comparing observed vs. predicted water temperatures for each daily forecast at each depth 188 (Figure 1 step 4). We repeated steps 3a-4 for daily, weekly, fortnightly, and monthly intervals of   it takes for data to become available for modeling after they are initially collected, we used the 215 daily mean in our forecasting application. Following quality checks, these data were integrated 216 into the FLARE forecasting system to produce depth-specific daily water temperature forecasts.          Data assimilation frequency altered forecast output and parameters over time 381 We were able to successfully forecast water temperature throughout the water column 382 over the year using DA to update model states and parameters (Figures 4-5). Across all depths, 383 DA constrained uncertainty by updating initial conditions with the most recent water temperature 384 observations. Forecast uncertainty for the lower DA frequencies was strongly dependent on the 385 time since last assimilation (Figure 4). On average, forecast variance at the one-day horizon 386 across 2021 for forecasts with daily DA was 1.56℃ while mean forecast variance at the one-day 387 horizon for forecasts with monthly DA was 3.25℃. 388 We observed that DA frequency altered parameter evolution of the forecasts (Figure 5).

389
The daily DA frequency resulted in more variable parameter estimates through time for all three 390 tuned parameters, reflecting the more frequent adjustment that occurred each time data were 391 assimilated. Importantly, parameter evolution for forecasts with daily DA yielded very different 392 estimates than the weekly, fortnightly, and monthly DA forecast frequencies ( Figure 5). For 393 example, the evolution of the longwave radiation scaling parameter (longwave) over the 365-day 394 forecast period showed that forecasts with weekly, fortnightly, and monthly DA frequencies  (Figure 6).

414
The skill of all forecasts degraded as the forecast horizon increased, but the decrease in 415 performance was greatest for daily DA forecasts, such that forecasts generated using monthly, fortnightly, and monthly forecasts never exceeded that metric for any of the 1-35-day-ahead 420 horizons. These results were consistent across forecast evaluation metrics, including the CRPS 421 metric that evaluates the full ensemble forecast (Appendix S1: Figure S2).

426
Forecast skill was generally best at 9 m regardless of horizon or DA frequency. Aggregated 9 m 427 forecast skill was 1.29 ± 1.80℃, followed by aggregated 5 m forecast skill (1.63 ± 1.85℃), and 428 aggregated 1 m forecast skill (1.69 ± 1.82℃). As expected, forecast skill generally decreased 429 with horizon, with a mean 1-day-ahead forecast RMSE of 0.80 ± 1.20°C, mean 7-day RMSE of 430 1.15 ± 1.60°C, and mean 35-day RMSE of 1.99 ± 2.17°C. However, we observed an exception to 431 this pattern for 1 m mixed forecasts, which is further described below.

432
On average, forecast skill was slightly better (as indicated by smaller RMSE) during the 433 stratified period than during the mixed period, aggregated among all depths and horizon 434 regardless of DA frequency (aggregated mixed RMSE = 1.59 ± 1.57°C, stratified RMSE = 1.46 435 20 ± 2.13°C; Figure 7). Forecast skill was more variable among forecast horizons than depths in the 436 mixed period, whereas forecast skill was variable across both depths and horizons in the 437 stratified period (Figure 7). In the stratified period, forecast skill was best at 9 m, with relatively 438 similar skill over the forecast horizon (Figure 7f). In the mixed period, forecast skill varied very 439 little among depths aggregated across horizons (Figure 7a, c, e), with consistently greater 440 decreases in skill with increasing horizon than in the stratified period, except for at 1 m. Forecast  Lower frequency DA forecasts consistently had more total uncertainty (Figure 8). We 454 found that the differences between uncertainty for daily and monthly DA were largest at 1-day-455 ahead horizons and largely converged by the end of the 35-day horizon (Figure 8). At 1 m depth, 456 total uncertainty was similar between the mixed and stratified periods across the 35-day horizon, 457 but at 5 and 9 m, total uncertainty was on average higher in the stratified than mixed period. Both 458 21 RMSE and total variance were similar for forecasts run with and without initial conditions 459 uncertainty included (Appendix S1: Figures S3-S4).

460
Forecasts with less frequent DA had a greater contribution of initial condition uncertainty 461 to total forecast uncertainty during the first few days of the forecast horizon. However, overall, 462 initial conditions uncertainty contributed a minimal proportion of the total uncertainty for 463 forecasts generated with daily DA (Figure 9). At the 1-day-ahead forecast horizon, daily DA 464 initial conditions uncertainty was 0% of total uncertainty, whereas initial conditions uncertainty   In this study we found that less frequent DA (e.g., weekly, fortnightly, and monthly DA) 491 sometimes led to more skillful water temperature forecasts than daily DA for all depths during  Our work is consistent with studies that have found that the optimal DA frequency often 501 matches that of the forecast model timestep (e.g., Derot et al., 2020, Woelmer et al., 2022. For 502 example, during both the mixed and stratified periods, daily DA was always better for 1-day-503 ahead forecasts, but was often outperformed by weekly DA at 8-day-ahead forecast horizons 504 23 ( Figure 6). Because water temperatures were homogenous among all depths during the mixed 505 period, water temperature variability among all depths was likely driven by air temperature 506 variability, ultimately making it more challenging to predict water temperature across depths as 507 the forecast horizon increased. During the stratified period, however, less frequent DA could still 508 generate accurate surface and mid-depth water temperature forecasts. The increased importance 509 of daily DA at bottom depths during the stratified period is likely because of the increased 510 thermal stability at bottom depths associated with thermal stratification (Figure 7). This pattern is 511 in contrast with other water temperature forecasting studies that have found daily DA necessary 512 for improving the skill of forecasts in the middle of the water column around the thermocline 513 (Baracchini et al., 2020a), but is likely explained by the overfitting of both the daily longwave 514 radiation and the epilimnetic sediment temperature parameters ( Figure 5). 515 We note that there are many ways to quantify skill beyond the 2℃ RMSE threshold used 516 here. We chose to use RMSE because it is a commonly used metric by lake modelers to   Our findings are similar to other water temperature lake and reservoir forecasting studies. We found that initial conditions uncertainty contributed a substantial proportion of total 568 uncertainty for weekly, fortnightly, and monthly DA, but only during the first few days of the 569 forecast horizon. From 6-23 day-ahead horizons, the contribution of initial conditions decreased 570 to <1% across all DA frequencies, depths, and times of year (Figure 9). We observed that high-571 frequency DA was required for skillful 9 m stratified forecasts, while weekly DA was sufficient 572 for other depths and times. This finding may be because the contribution of initial conditions 573 26 uncertainty decreases more rapidly within the first few days of the forecast horizon for the daily 574 DA forecasts at 9 m in the stratified period. For all other depths and times of the year, the rate at 575 which initial conditions uncertainty decreases is greater for weekly, fortnightly, and monthly 576 DA, resulting in more similar performance of daily and weekly DA early in the forecast horizon 577 (Figure 9). However, more frequent DA may not always improve forecast performance, 578 especially when initial conditions uncertainty is not the dominant source of uncertainty, as seen 579 at longer horizons. Given that initial conditions uncertainty predominated at the beginning of the 580 forecast horizon, it is likely that total forecast uncertainty at longer horizons was primarily 581 influenced by uncertainty in model process, model parameters, and/or meteorological driver data 582 ( Figure 9). Conversely, the dominant source of uncertainty for weather forecasting is typically 583 initial conditions uncertainty given the inherent instability of atmospheric processes (Dietze,584 2017b), which is why more frequent DA often substantially improves meteorological forecast 585 skill.

586
Other lake and reservoir water quality forecasting studies have found that model driver  Our results suggest that weekly DA may suffice for some lake and reservoir water 644 temperature forecasting applications, with the caveat that more frequent DA often improved 645 water temperature forecast performance at short forecast horizons. However, we only assessed 646 forecast skill for a single reservoir and ecological variable for only one year, and therefore note 647 the limitations of extending these results to other systems and variables. Additionally, updating 648 model parameters and initial conditions too regularly can lead to overprediction biases when 649 forecasting, which may explain why weekly rather than daily DA resulted in more skillful water 650 temperature forecasts in the mixed period (see Lin et al., 2021). Finally, because we did not 651 quantify the contribution of all sources of uncertainty, we can only identify the relative role that 652 DA has on reducing initial conditions uncertainty. Future studies that consider the role of other 653 sources of uncertainty will improve our understanding of DA on total forecast uncertainty. This study emphasizes the importance of DA for improving ecological forecast skill and 657 has implications for forecasting efforts among a wide range of ecosystems and ecological 658 variables. We argue that weekly observations of water temperature are likely "good enough" to 659 set up a skillful forecasting system for many reservoir management applications, while daily DA