A Perspective on the Future of CMIP

The Coupled Model Intercomparison Project (CMIP) has demonstrated the importance of climate modeling for climate research and its usefulness for climate services. The latter has increased CMIP's operational burden, so much so that serving IPCC has become its animating force. Attempting to satisfy an operational mandate through a coordinated research project diminishes both the service and the research. Regaining the initiative will require CMIP to transition the quasi‐operational system it has developed to an operational setting. Doing so would allow CMIP to focus on developing an international scientific agenda to encourage and exploit advances in climate modeling.

• Continuing to develop the Coupled Model Intercomparison Project (CMIP) as a quasi-operational system disguised as a research activity, serves neither science nor society • The system of climate information provision being sustained by CMIP needs to be operationalized as a service • Freeing CMIP from its operational burden will allow it to focus on the pressing need to strengthen climate science

Supporting Information:
Supporting Information may be found in the online version of this article.community.Efforts to improve standardization (e.g., cf conventions) and experimental protocols remain essential, and require continued attention, but these are hardly what has come to animate CMIP.

Where Have All the Flowers Gone?
Today what mostly animates CMIP is the perceived need to serve the IPCC.This determines how its phases are branded, how it is timed, how it is structured, and how it is funded (Eyring et al., 2016;Lamarque, 2022;Meehl, 2023;Stouffer et al., 2017).Because the models used in CMIP are largely the outgrowth of past research activities, this operational burden is disproportionately borne by the research community.
Ten years ago, a major motivation for my contributions to the design of CMIP6 was my belief that CMIP's growing operational burden could be tamed and subordinated to the search for new systematics.I was wrong.CMIP6 had less spark than CMIP5, and for reasons that are now easier to understand, even less can be expected from a possible CMIP7.Not only have the systematics of the class of models CMIP targets become more stable, we now also know that projections of warming and regional changes are sensitive to "minor treatments" (Kawai et al., 2022;Zhu et al., 2022), albeit not systematically so (McWilliams, 2007).By refuting the idea of large-scale control, these sensitivities undermine a long-standing ambition of CMIP, which has been to support parameterization development.As a result parameterization, as an expression of theory, has given way to machines learning to play games of error compensation (Balaji et al., 2022;Couvreux et al., 2021).
The absence of new systematics also explains why a posteriori re-scalings that aim to reduce spread (Andronova & Schlesinger, 2001;Knutti et al., 2006), by weighting or culling models, have not proven successful (Lee et al., 2021).The culling problem is illustrated by the case of NCAR's high-sensitivity climate model projections, as part of CMIP6.Despite clearly failing a test (to reasonably simulate the last-glacial maximum) designed to cull models (Hargreaves, 2021), and for reasons that have since been attributed to model error (Zhu et al., 2022), output from this model continues to be distributed and used (often without justification).The lack of a satisfactory solution to the culling problem was addressed in the Fifth assessment report of the IPCC WG1, which broke with the tradition of using CMIP model projections to estimate uncertainty when assessing aerosol forcing (Boucher et al., 2013).The success of subsequent assessments has largely come from adopting a similar approach (Bellouin et al., 2020;Forster et al., 2021;Sherwood et al., 2020;Stevens et al., 2016).None of the above is an argument against continuing to perform routine climate projections using a scientifically mature class of models.It just questions the benefit of constituting this as a high profile research activity.

The Operational Burden
An operational activity is something that provides a regular service for external users.While it often requires, and usually benefits from, ongoing research and development, it differs from the self-serving nature of research activities.As a result, agencies like the European Centre for Medium Range Weather Forecasts, which are mandated and funded to provide such services, tend to have a different character than research labs, like my own.For one, the service contribution of the scientists supporting the operational activities are often valorizing through greater pay, or easier access to a long-term perspective.
The fear that, by giving it an operational face, we diminish CMIP's ability to express research is not new.Already in preparing for CMIP5 it was recognized that pressure to synchronize research to support an operational mandate risked detracting from innovative developments.To mitigate against this outcome CMIP5 introduced tiered experiments (Taylor et al., 2012).The failure of the "Tiers" to lessen the burden of synchronization, motivated a finer grained program of experimentation and more autonomy of the MIPs through the introduction of the DECK in CMIP6 (Eyring et al., 2016).By creating mechanisms to benchmark new model versions, as they were released, the DECK hoped to decouple the development of models for research, from their operational use, while maintaining a connection between the two (Eyring et al., 2016).
The DECK also failed to break the subordination of the research activities to the timelines and protocols of CMIP's growing operational mandate.It didn't lead to models being finalized and benchmarked on their own time-scale, and programs of scientific experimentation remain coupled to operations.The reasons are three-fold: (a) most of the research groups presently responsible for the modeling lack the rigor and reason to finalize model versions in the absence of an operational imperative; (b) many labs favored bundling the science and the service, often with the argument that doing so attracts funding for the science; (c) third parties benefit from a monolithic CMIP, as they can use it to more easily attract funding for service activities, for example, access to archives, output data requests, standard model evaluation, and model documentation.These formidable forces of attraction are have caused research activities in past CMIP phases to collapse under CMIP's operational mandate.
In retrospect it now seems clear that attempting to establish and maintain operational rigor within the research environment is counter productive.Operational rigor emphasizes routine to ensure the timely and quality controlled provision of a product, which research and development is constituted to support.Scientific rigor emphasizes experimentation in support of critical thinking.Trying to serve both does justice to neither.Also problematic, is that because research goes hand-in-hand with training, providing operational service through a research activity changes the quality of the training and diminishes the professional prospects of those being trained (Jain et al., 2022).

Operationalize What Is Operational
There is no longer any reason for CMIP to organize research activities as a substitute for the professional provision of climate projections to meet the manifold needs of users, that is, an operational service.Procedures for developing forcing data sets, for tuning and running well characterized models based on these forcing, for analyzing the results of the simulations in the face of uncertainty, and for establishing the extent to which models can distinguish among different types of forcing, are all well established and ripe for operations.Reconstituting these activities in an operational setting, whose scope (including the ongoing necessity for research and development) would be determined by its funding, would lead to a more sustainable and better quality of service (Jakob et al., 2023).To the extent this is constituted as a distributed international activity, it will need coordination, but not by the research community, that is, not by CMIP.

Reconstitute CMIP to Support Research
As an activity of the World Climate Research Program, CMIP should focus on doing what only it can do, which is to help answer specific scientific questions through agile coordination of international modeling activities.Relinquishing its operational aspirations, and re-establishing a scientific focus would open CMIP to new horizons (Figure 1).For instance, it could finally embrace a true diversity of models selected and developed based on their suitability to address specific scientific questions.A more singular scientific focus would give CMIP the dexterity required to support and link innovative programs of numerical experimentation (Schmidt et al., 2023;Stevens  If the operational provision of climate information is important, and I think it is, it must be provided as a professional operational service.Failing to do so is suffocating the science and short changing society.et al., 2019;Wing et al., 2018), while at the same time advancing standards and best practices.Most importantly, by focusing on research, CMIP could use its capital to raise the profile of our science, and thereby help researchers gain access to the resources they require to collaborate and work most effectively.As it stands, the imposition of artificial timelines, a preoccupation with a single type of model, and the lack of a scientific narrative are inevitable byproducts of CMIP's alignment with IPCC assessment cycles, and diminish CMIP as a research activity.

Recapturing the Initiative
Before there was agreement about the basic facts of climate change, CMIP helped the IPCC communicate the state of scientific understanding.CMIP's coordinated program of experimentation, using the latest versions of our most sophisticated models, drew focus to their unwavering systematics.In this way CMIP5, by virtue of its similarity to CMIP3, helped the IPCC's Fifth Assessment Report demonstrate that the bedrock of our understanding did not depend on the details of how one represented the climate system, laying the basis for the landmark 2015 Paris agreement.
This accomplishment made for a powerful message-perhaps too powerful.Today many people, and even scientists, conflate the CMIP systematics with the systematics of nature, rather than what they are: grist for the mill of science.Although CMIP includes state-of-the-art models, it's systematics can't reflect them, and it lags even further behind in terms of the state-of-the-science, which is more multifaceted.Nonetheless, the sheer magnitude of the effort and its close coupling with the IPCC, gives the activity a de facto reference status.This wastes time and talent, as paper after paper feels the pressure to give the CMIP ensemble due diligence; it discourages the exploration of novel approaches, which might lead to more than incremental improvements; and it retards the operationalization of climate information provision (Jakob et al., 2023).More fundamentally it miscommunicates scientific ambition.Routine might be helpful for the societal discourse; capturing the public imagination and winning the competition for the best minds requires inspiration.Using CMIP as a Trojan horse to fund research might be a short-term solution for gaps in funding, but it dulls our ability to articulate what the science has left to offer.
Moving the operational modeling activities to operations, and reconstituting CMIP as a research activity, would provide a strong counterweight against the forces of attraction that, in the past, have caused CMIP to collapse upon itself.Avoiding another collapse will be paramount if we wish society to be well served by what we know, and science to be well stimulated by what we don't.Yvonne Schrader (at my own institute) is thanked for the inspiration and creative execution of Figure 1.Kerry Emanuel, Jonathan Gregory, Isaac Held, Helene Hewitt, Markus Jochum, Brett McKim, Tiffany Shaw, Marty Singh, Adam Sobel, Philip Stier, Pier Luigi Vidale, Masahiro Watanabe are thanked for comments on earlier drafts.Longer exchanges with Sandrine Bony, Tomoki Miyakawa, Robert Pincus, and V. (Ram) Ramaswamy were particularly influential in helping me understand how best to present my ideas.Acknowledging collegial contributions is not meant to construe an endorsement of my ideas.Dave Schimmel, in his role as editor, as well as Karl Taylor and two further, albeit anonymous, reviewers, are thanked for their time and thoughtful comments, which along with my responses, are provided by the AGU as supplement to this article.This essay benefited from the time and hospitality afforded me while visiting the University of Tokyo's Atmosphere Ocean Research Institute.Additional support was provided from the EC Horizon2020 program through NextGEMS (Grant 101003470) and the Project WarmWorld (Grant 01LK2202B) of the German Ministiry of Education and Research (BMBF).

Figure 1 .
Figure 1.Coupled Model Intercomparison Project needs to let go of the projections, and return to the science.If the operational provision of climate information is important, and I think it is, it must be provided as a professional operational service.Failing to do so is suffocating the science and short changing society.