Potential Predictability of the Madden‐Julian Oscillation in a Superparameterized Model

The Madden‐Julian Oscillation (MJO) is a promising target for improving sub‐seasonal weather forecasts. Current forecast models struggle to simulate the MJO due to imperfect convective parameterizations and mean state biases, degrading their forecast skill. Previous studies have estimated a potential MJO predictability 5–15 days higher than current forecast skill, but these estimates also use models with parameterized convection. We perform a perfect‐model predictability experiment using a superparameterized global model in which the convective parameterization is replaced by a cloud resolving model. We add a second “silent” cloud resolving component to the control simulation that independently calculates convective‐scale processes using the same large‐scale forcings. The second set of convective states are used to initialize forecasts, representing uncertainty on the convective scale. We find a potential predictability of the MJO of 35–40 days in boreal winter using a single‐member ensemble forecast.

10.1029/2023GL105705 2 of 9 Waliser et al. (2003).In these experiments, instead of computing skill using model forecast errors from observations, the observations are replaced by another simulation from the same model as the forecasts.These experiments can be considered a "best case scenario" for prediction skill if the forecast model had a perfect mean state and near-perfect initial conditions (H.Kim et al., 2018).Previous perfect-model predictability experiments found a potential MJO predictability of up to 35-45 days when using state-of-the-art ensemble forecast models (Neena et al., 2014;Xiang et al., 2022;S. Wang et al., 2019;Wu et al., 2016).
Most predictability estimates use coarse resolution models that parameterize convection.J. Zhu et al. (2020) showed that the potential predictability of the MJO is strongly dependent on convective parameterization scheme, with a more realistic (though still imperfect) scheme being more predictable by up to 15 days.The impressive 35-40-day MJO predictability in ECMWF ensemble forecasts (S.Wang et al., 2019) has been attributed to improvements in convective parametrization and model physics (Vitart, 2014), though the ECMWF still uses a parameterization for convection.Further, the literature varies substantially in how they generate initial conditions for the forecasts runs, which clouds the interpretability of potential predictability estimates from multi-model forecasts (Vitart, 2017).
An alternative to using a convective parameterization is to simulate the MJO in fine resolution, global cloud-permitting models (Miyakawa et al., 2014;Zavadoff et al., 2023).The MJO in these models is more realistic and yields prediction skill of nearly 30 days with a single ensemble forecast (comparable to an 11-member ensemble ECMWF forecast).However, these models are extremely computationally expensive and can only feasibly simulate comparatively few MJO events (<100 in the above studies), restricting their ability to assess the potentially significant differences in predictability for different MJO characteristics and background states (Wu et al., 2023).A more computationally practical alternative is to use a multiscale modeling framework that couples a coarse resolution global model to a fine resolution cloud resolving model (Hannah et al., 2015;Randall et al., 2016).These multiscale models are much more efficient than global cloud permitting models, and the multiscale framework naturally permits separating convective versus large scale influence on predictability.As yet, no studies have used a multiscale model to extensively estimate MJO potential predictability.
In this manuscript, we perform a perfect-model predictability experiment using a superparameterized version of CAM (SPCAM), a multiscale model that replaces the convective parameterization in CAM with a cloud resolving component.We utilize the multiscale structure of superparameterization to generate initial conditions for the forecast runs by imposing a perturbation on the convective scale.Section 2 describes the model and forecast generation procedure, then defines the analyses used for estimating predictability.Section 3 shows the results of the predictability experiments, and Section 4 discusses the results in context of previous work and their limitations.

Model and Initial Conditions
We assess the potential predictability of the MJO using a superparameterized version of the Community Atmospheric Model (SPCAM) (M.F. Khairoutdinov & Randall, 2001;M. Khairoutdinov et al., 2005).The global climate model (GCM) component of SPCAM is CAM4 with 1.9°latitude x 2.5°longitude resolution, 26 vertical levels, and a timestep of 30 min.Sea surface temperatures are prescribed using monthly climatology.Each larger GCM-scale grid box has a 2-D cloud resolving model (CRM) embedded within it.The CRM calculates the cloud-scale processes explicitly, replacing the convective parameterization.Each CRM has 32 horizontal gridboxes aligned east to west in each larger GCM gridbox, with 4,000 m horizontal resolution and 24 vertical levels that align with the lower levels of the GCM.The CRM timestep is 20 s.The CRMs use a one-moment SAM microphysics parameterization scheme based on M. F. Khairoutdinov and Randall (2003).The CRMs are running simultaneously with the GCM but at a shorter timestep; large-scale tendencies from the GCM and convective-scale tendencies from the CRMs are passed between the models at each GCM timestep.
SPCAM has been shown to simulate a relatively realistic MJO, especially compared to CAM (H.Zhu et al., 2009).Benedict and Randall (2009) and M. Khairoutdinov et al. (2008) describe some of the main improvements and lingering biases of SPCAM, which we summarize here.SPCAM reasonably simulates the observed wavenumber-frequency spectrum of subseasonal tropical waves, including the MJO.The general structural evolution and geographic span of the MJO is well represented in SPCAM.The main bias of SPCAM is that it 10.1029/2023GL105705 3 of 9 overestimates the strength and variability of the MJO, especially over the Western Pacific, in opposition to most GCMs that have a weak MJO.
To generate initial conditions for the forecast runs, we add a second CRM to each GCM gridbox in SPCAM, following the idea of "multiple-instance superparameterization" in Subramanian and Palmer (2017); Jones et al. (2019).During the control simulation, we modify the radiative and convective components of SPCAM to run a second "silent" CRM in parallel to the standard CRM in each gridbox.At each timestep, the GCM passes the same large-scale information to both CRMs.Each CRM independently calculates convective tendencies as in the standard SPCAM, but only the first CRM passes information back to the GCM.The second silent CRM simply outputs its state to a file.Each CRM uses CRM-scale information from its own previous timestep and thus stays spun up throughout the simulation.The GCM only receives information from one continuous CRM, so the control run output is essentially a standard SPCAM simulation.Independence between the two CRMs comes from their initialization by a random temperature perturbation near the surface on the first timestep, which is the standard for initializing SPCAM; afterward, they run continuously and independently.See Figure S1 for a visual schematic of the control run setup.
The silent CRM information is used as the initial perturbation for the forecast runs.The GCM-scale information used in the forecast restarts is the same as from the control run, so the only initial perturbation of the forecast is from the convective scale.This allows for scale separation of the initial perturbation and can be thought of as simulating a situation where we have perfect information of the GCM-scale conditions with uncertainty in the convective scale.Subramanian and Palmer (2017); Jones et al. (2019) examined how running multiple CRM components affects ensemble spread and found the stochastic-like nature of the CRMs better represents observed variance of deep convection in the tropics than convective parameterizations.
For this project, we use a 20-year control simulation.60-day single ensemble member forecasts are initialized using the silent CRM information in a regular version of SPCAM every 5 days during boreal winter (October -April), resulting in 798 forecasts.We restrict analysis to boreal winter due to the shown lower predictability of boreal summer intraseasonal convective signals (S. Wang et al., 2019), but this analysis could be repeated for a similar study of boreal summer events.

Analyzing Predictability
To isolate the MJO signal from our forecast runs, we use a version of the OLR-based MJO index (OMI), first developed in Kiladis et al. (2014) and modified in Weidman et al. (2022).The OMI is an EOF-based index that uses the two leading principal components (PCs) of 30-96-day filtered daily tropical OLR to track the strength and propagation of the MJO through time.For forecasting purposes, we use a real-time version of the OMI (ROMI) as described in Kiladis et al. (2014); S. Wang et al. (2019).In brief, the ROMI is calculated by removing the mean OLR anomaly of the previous 40 days from the raw OLR anomaly and then taking a 9-day running average of the result, tapered to 7-, 5-, 3-, and 1-day running averages at the end of the forecast.This is to remove high and low frequency signals beyond that of the MJO.The filtered/smoothed OLR is then projected onto a set of seasonally-varying OLR-based EOFs calculated from an independent 40-year SPCAM run, following the EOF rotation procedure in Weidman et al. (2022).The OLR anomaly is calculated by removing the mean and first three harmonics of the seasonal cycle from each gridpoint, again calculated from a 40-year SPCAM run.
Many studies use the real-time multivariate (RMM) index (Wheeler & Hendon, 2004) for forecasting purposes.The RMM is based on EOFs describing the zonal structure of OLR and zonal wind at 200 and 850 hPa; however, Straub (2013) showed that the RMM underrepresents convection compared to zonal wind, causing the RMM to miss MJO-like convective signals and poorly predict desired forecasting variables such as precipitation and surface temperature (S.Wang et al., 2019;Kumar et al., 2020;Hannah et al., 2015).In addition, the lack of meridional structure in the RMM confounds the MJO signal with equatorial Kelvin waves (Roundy et al., 2009).For these reasons, we use the OMI here; results do not substantially change when using the RMM.
Following previous MJO predictability studies, we use three metrics for determining MJO predictability: the bivariate correlation coefficient (COR), root mean squared error (RMSE) and signal to noise, all of which are calculated using the first two PCs of the ROMI.

COR and RMSE
Following for example, Lin et al. (2008), the COR is calculated as and root mean squared error (RMSE) as where a 1,2 (t) are the first two PCs of the ROMI for the control simulation t days after forecast initiation, b 1,2 (t) are the PCs of the ROMI for the forecast, and N is the total number of forecasts.By convention, forecasts are considered skillful until COR falls below 0.5.The COR is essentially the weighted similarity of the ROMI, while the RMSE includes information about amplitude differences (Wilks, 2019).

Signal to Noise
COR and RMSE quantify the difference between the control and forecast runs on each day.Another assessment of predictability is to relate the strength of the MJO signal to the day-to-day variability of the tropics (Waliser et al., 2003).As long as the MJO signal remains stronger than the background noise, it could be predictable.We quantify noise as the mean squared error as in previous studies.Since the MJO is a propagating signal with a period of 30-60 days, the "signal" is calculated as the average amplitude of the ROMI within a sliding window that approximately captures a full MJO event.Previous work showed that signal is insensitive to window size, so we follow previous studies and use a 51-day window (Neena et al., 2014;Waliser et al., 2003).Signal is defined as where again a 1,2 are the two PCs of the control run and L is 25 for a window size of 51.Since the control and forecast runs are essentially separate runs of the same model, we expect signal to be similar if the forecast ROMI was used; we use the control for convenience when calculating the sliding window.

Results
To visualize how the MJO simulation differs between the control and forecast runs, we plot the two components of the ROMI on a phase diagram for four example forecasts compared to the control run over the same period (Figure 1).A typical active MJO trajectory will propagate counterclockwise (eastward) outside of the dashed circle (representing an amplitude greater than 1), such as in the two cases on the right in Figure 1.The example forecasts here are chosen arbitrarily to represent forecasts initiated during active MJO periods in Phases 1/8, 2/3, 4/5, and 6/7, but are broadly representative of forecast behavior.The exact timing and pattern differs for each forecast, but in general the control and forecast trajectories track together closely for about 15 days before the amplitude or phase of the forecast run starts to drift from the control.From these examples, we see that the MJO signal remains strongly coherent for at least 10-15 days after the initial convective perturbation in the forecast run.
We use the bivariate COR and RMSE to quantitatively assess MJO predictability for each day after the forecast initiation (Figure 2).Both metrics are also recalculated using only forecasts that were initiated on an active or inactive MJO day as defined by a ROMI amplitude above 1.14, the median amplitude over the 20-year period.The 95% confidence interval is calculated using a Student's t test.The initial strength of the MJO does not significantly affect either of the metrics in Figures 2a and 2b; the COR of all three cases fall below 0.5 between days 37-40.The COR shows nearly perfect correlation for 10-15 days, corroborating the high similarity between the control and forecast trajectories in Figure 1.After the first few days, the RMSE increases approximately linearly until reaching a saturation around days 37-40.
Perhaps a more operationally relevant metric is to look at the predictability of the MJO based on target date; in other words, how far ahead could a strong or weak MJO event be predicted (Xiang et al., 2015).COR and RMSE are recalculated based on the strength of the MJO on the target date and shown in Figures 2c and 2d.A lag of −5 means the forecast was initiated 5 days before a strong or weak MJO target date.The correlation is significantly larger for strong targets compared to weak targets between days 10-30 based on the 95% confidence interval, though they all converge at 0.5 COR between 37 and 40 days.The RMSE is about the same for both strong and weak MJO targets until the error saturates after day 40.This is likely because although the correlation is stronger for strong targets, the MJO signal itself (the amplitude of the ROMI) is also larger, so the mean error remains consistent.
Several studies have suggested that predictability is dependent on target phase of the MJO (S.Wang et al., 2019).COR is recalculated by separating forecasts by initial phase and target phase using the same methodology as above.Correlations for all initial/target phases and active MJO initial/target phases are plotted in Figure 3.There is little phase dependence in COR before 20-25 days; afterward, there is slightly higher predictability for initial Phase 2 and target Phase 5-6 for all and active MJO periods, and slightly decreased predictability for target Phase 2. Phase 2 corresponds to enhanced precipitation over the Indian Ocean, and Phase 5-6 aligns with precipitation transitioning from the Maritime Continent to the Western Pacific, generally 15-20 days after Phase 2. Several other models have shown worse skill for MJO events that propagate across the Maritime Continent, a phenomenon known as the Maritime Continent barrier (Abhik et al., 2023;Du et al., 2023;S. Wang et al., 2019).SPCAM does not seem to have this problem and in fact finds these events (which are often strong events) more predictable.
Signal and noise (mean squared error) is our final predictability metric (Figure 4).The MJO signal is interpreted as predictable as long as the signal is larger than the error.Again, results seem to be insensitive to strength of the initial MJO signal; the signal intersects with error between days 35-40 for all three cases.Since we are using a perfect-model experiment, signal should be nearly constant through lead time; the small increase (decrease) at the beginning of the period for strong (weak) events is because we are restricting our set of forecasts   to initially stronger (weaker) MJO signals, and this feature dissipates as the error surpasses the signal.Error slowly increases until it saturates just after surpassing the signal.In general, the signal to noise metric aligns with the conclusions drawn from COR and RMSE, implying a predictability of the MJO in boreal winter of 35-40 days.

Summary and Discussion
By replacing the convective parameterization with a cloud resolving component and using a novel technique to initialize the forecast runs using convective-scale uncertainty, we find the MJO potential predictability in SPCAM to be about 35-40 days using a single-member ensemble forecast.This estimate is comparable to the highest performing ensemble forecasting models (W.Wang et al., 2014;Xiang et al., 2022;Wu et al., 2016), though these models find predictability closer to 20-30 days when using single-member ensemble forecasts (Neena et al., 2014;H.-M. Kim et al., 2014).Lim et al. (2018) found prediction skill increased by 1-5 days (depending on the model) when increasing the number of ensembles from one to three; we expect an ensemble forecast in SPCAM generated from a set of multiple silent CRMs to increase potential predictability by a similar amount.Further, many prior estimates use the RMM to quantify predictability, rather than the ROMI.When repeating analyses with the RMM, we find a decreased potential predictability by 2-4 days, which is comparable to previous studies (S.Wang et al., 2019).However, the broad findings or interpretations do not largely change when using a different MJO index.
MJO predictability in SPCAM is not strongly dependent on initial phase or amplitude, but has slightly higher predictability for strong target dates in Phases 5-6.The stronger predictability of MJO events passing over the Maritime Continent is in opposition to most other models in which the MJO tends to dissipate too frequently over the Maritime Continent (S.Wang et al., 2019).Abhik et al. (2023) suggested that this deficiency arises from issues with MJO simulation due to model physics, rather than an inherent predictability barrier.Our results corroborate this hypothesis, since SPCAM more readily simulates MJO propagation across the Maritime Continent and is also biased toward a more active MJO in the Western Pacific.
The potential predictability limits found by SPCAM may not be possible to reach in the real world because the MJO and mean state in SPCAM do not perfectly replicate observations.Since MJO events in SPCAM are in general too vigorous and propagate too easily across the Maritime Continent, our findings may be an overestimation of MJO predictability.Most forecast models have the opposite problem of a weaker and slower MJO than observed, especially over the Western Pacific, which would likely result in an underestimation of predictability.In addition, we use prescribed sea surface temperatures, which simplifies our interpretation of the results and reduces computational expense, but restricts our understanding of how MJO predictability is influenced by interactions with different ENSO background states (Mengist & Seo, 2022).Previous studies have shown how including a coupled ocean component helps drive MJO propagation and can increase predictability of precipitation and OLR after 10 days (Pegion & Kirtman, 2008).Coupled ocean influence on MJO predictability is a complicated topic of current research and dependent on model formulation and convective representation (DeMott et al., 2015).Here we only simulate MJO events driven by atmospheric internal dynamics which only represents a portion of observed MJO events (Fu et al., 2015); investigating MJO predictability in a coupled version of SPCAM would be a fruitful topic for future work.The inherent predictability of the real atmosphere is likely not properly represented by any modeling approach, yet our use of a more realistic convective representation and novel forecast initiation method hopefully adds a useful datapoint for our understanding of MJO predictability.
Although it is conventional to use an MJO index to quantify MJO predictability, bivariate indices only capture a condensed slice of actual MJO behavior.The goal of improving MJO forecasts requires an understanding of the phenomenon and the modeling components that are most important for improving those forecasts.This study is a first step in utilizing scale separation in a multiscale model for assessing MJO predictability.With this framework, future work should include a thorough analysis of which physical aspects of the MJO lead to its predictability, or conversely, which aspects of biased MJO simulation in models lead to decreased forecast skill

Figure 1 .
Figure 1.MJO phase trajectories of the ROMI for four example forecast cases.Green dashed lines are the control simulation and purple lines are the forecast runs.Filled circles represent the beginning of the forecast, with squares every subsequent 5 days for a total of 50 days.

Figure 2 .
Figure 2. Bivariate correlation (left) and RMSE (right) between control and forecast runs by time after forecast initiation.(Top) Using the ROMI principal components at each day, calculated for all forecasts (green), strong MJO initial days (orange) and weak MJO initial days (purple).(Bottom) Same, but using strong and weak target dates.Shading represents a 95% confidence interval.The dashed lines are at 0.5 (left) and  √ 2 (right).

Figure 3 .
Figure 3. Bivariate correlation as a function of forecast lead time for different MJO phases, separated by initial (top) and target (bottom) date.Using data from all forecasts (left) and only strong MJO events (right).

Figure 4 .
Figure 4. MJO signal (dashed) and mean squared error (solid) as a function of forecast lead time for all forecasts (green), strong initial days (orange), and weak initial days (purple).Shading represents a 95% confidence interval.