Comparisons Between CMIP5 and CMIP6 Models: Simulations of Climate Indices Influencing Food Security, Infrastructure Resilience, and Human Health in Canada

The warming climate can considerably affect socioeconomic activities and environmental health conditions in Canada. Climate models play a key role in evaluating the impact of climate change and developing adaption and mitigation strategies corresponding to Canadian regions. This study compares the behavior of climate models participating in the Climate Model Intercomparison Project Phase 6 (CMIP6) with their CMIP5 predecessors in representing a set of climate indices relevant to Canada’s agricultural productivity, infrastructure resilience, and environmental health. Our results show that although CMIP5 and CMIP6 multi‐model ensemble mean values for the considered indices are almost similar, the behavior of individual CMIP5 and CMIP6 models or even the model pairs of the same modeling center can be different across Canada. Moreover, the CMIP6 models do not necessarily outperform CMIP5 in comparison to NOAA and NCEP reanalysis datasets in simulating the annual mean and coefficient of variation values for the indices during the historical period. The comparisons between models’ simulations also reveal that the envelope of estimated values based on individual CMIP6 models does not cover or overlap with their CMIP5 counterparts or even with other CMIP6 models. Therefore, the CMIP6 models with higher number of simulations do not necessarily provide a larger range of projections over Canadian regions. The divergence between CMIP5 and CMIP6 models’ behavior is more obvious under the 8.5 W/m2 forcing in the long‐term horizon. Overall, the estimated sign, magnitude, and spatial pattern of changes in the climate indices depend on the considered climate model and forcing scenario.

human health, particularly in northern regions are identified as key areas at high risks in the upcoming future (Council of Canadian Academies, 2019). Therefore, an improved understanding of climate change impacts over these sensitive areas is essential to propose reliable adaptation strategies.
General Circulation Models (GCMs), from the Coupled Model Intercomparison Project (CMIP), are the most advanced tools to project climate conditions (Randall et al., 2007). The CMIP5 provides GCMs' outputs under different Representative Concentration Pathways (RCPs) (Stouffer et al., 2017;Taylor et al., 2012). Individual and ensemble of CMIP5 models have been used to assess the impact of climate change for example, on a series of indices related to agriculture (Bock et al., 2018;Grenier et al., 2019), infrastructure design (Hassanzadeh et al., 2019;Jaramillo & Nazemi, 2018), and natural hazards (Gaur et al., 2018;Tam et al., 2019) in Canada. These studies find that the sign and magnitude of changes depend on the considered climate models, and overall large uncertainties exist in the projections. More specifically, the models show warm as well as cold biases over northern latitude, and wet biases in cooler regions (Sheffield et al., 2013;Sillmann et al., 2013;Wazneh et al., 2020).
The new state-of-the-art CMIP6 models' simulations have recently become available and aim to fill the gaps in CMIP5, most importantly, to better represent physical processes at smaller scales O'neil et al., 2016;Stouffer et al., 2017). Moreover, CMIP6 offer a wider range of forcing than CMIP5 due to inclusion of the Shared Socio-Economic Pathway (SSP) scenario matrix, which considers broader socio-economic conditions for the future Riahi et al., 2017;Stouffer et al., 2017). In brief, the SSPs are parallel approaches to RCPs accounting for mitigation and adaptation to the diverse challenges in our societies and offer a possibility for researching groups to specifically assess the impacts of these challenges' responses (Hausfather, 2019). In addition, CMIP6 provides a higher number of simulations for the same forcing to better represent internal variabilities Merrifield et al., 2020;Pascoe et al., 2019;Stouffer et al., 2017). The performance of multiple CMIP6 models in representing precipitation and temperature conditions are assessed in different regions (Akinsanola et al., 2020;Almazroui et al., 2020;Z. Chen et al., 2020;Ha et al., 2020;Jiang et al., 2020;Masud et al., 2021;Wang et al., 2020) and at the global scale (Fan et al., 2020;Na et al., 2020) by considering one simulation per model. Understanding the differences between behavior of CMIP5 and CMIP6 models is essential due to their critical role in evaluating the impact of climate change. Various studies show that CMIP5 and CMIP6 multi-model ensemble temperature (and precipitation) values tend to be roughly similar at the global scale (see H. Kim et al., 2020;Seneviratne & Hauser, 2020). In addition, CMIP6 models have globally improved their historical representations of climate extremes indices compared to CMIP5 models (H. Di Luca et al., 2020). Studies at the smaller scale for example, in China (H. , India (Gusain et al., 2020), and Australia (Grose et al., 2020) also show a better representation of spatial pattern and seasonality of climate variables by CMIP6 in comparison to CMIP5 models during the historical period. Although using an ensemble of simulations per climate models are recommended to represent GCMs' "forced response" Maher et al., 2019;Martel et al., 2018), most recent studies use only one or a limited number of simulations per GCM for their CMIP5 and CMIP6 comparison (H. Chen et al., 2020;Di Luca et al., 2020;Grose et al., 2020;Kim et al., 2020;H. Zhu et al., 2020). Only a few studies analyze CMIP6 simulations (Papalexiou et al., 2020;Schurer et al., 2020;Shrestha et al., 2020;Yazdandoost et al., 2020) or even compare multi-simulation ensembles of CMIP5 and CMIP6 models (Cook et al., 2020;E. Zhu et al., 2020). Nevertheless, to the best knowledge of authors, no specific study on comprehensive assessment of CMIP5 and CMIP6 simulations over Canada has been done yet.
This study aims to compare the CMIP5 and CMIP6 simulations in estimating a set of climate indices related to agriculture security, infrastructure sustainability, and environmental health in Canada during the historical and future periods. In particular, the specific objectives of this study are to understand whether the considered CMIP5 models can be replaced by CMIP6 models and if an individual CMIP6 model, containing a large number of simulations, can be solely used for impact assessment in Canada. Section 2 introduces the CMIP5 and CMIP6 models, two reanalysis datasets that are used to benchmark GCMs' performances, as well as six climate indices analyzed in this study. Section 3 presents the results of inter-comparisons during the historical and future periods. Discussions about uncertainties in projected changes in the climate indices and further directions for impact assessments are provided in Section 4.

CMIP5 and CMIP6 Models
The daily precipitation (P), maximum temperature (Tmax), and minimum temperature (Tmin) datasets for seven CMIP5 and seven CMIP6 models, a total of 389 simulations, are obtained from https://esgf-data. dkrz.de/projects/esgf-dkrz/ in fall 2020. Table 1 shows the name of climate modeling centers, GCMs and their spatial resolutions as well as number of simulations and future scenarios. These climate models are selected due to their large number of simulations available for the studied variables. For more information on these CMIP models, please refer to https://pcmdi.llnl.gov/. Moreover, RCPs 4.5 and 8.5 for CMIP5 and SSPs 2-4.5 and 5-8.5 for CMIP6 models are considered as future scenarios. Both SSPs and RCPs used in this study present the same end of the century forcing (4.5 and 8.5 W/m 2 ), which are measures of the Earth's heating surface as a function of greenhouse gases concentration, aerosols, clouds, and changes in land surface (Van Vuuren, 2011). In brief, CMIP6 SSP 5-8.5 presents higher values (around 20%) of CO 2 emission than RCP 8.5 for the whole 21st century. While SSP 2-4.5 starts with higher CO 2 emissions than RCP 4.5 and ends up with less emissions, with a slower and more continuous decline by the end of century (Hausfather, 2019). This study considers a historical period of 1961-1990 as well as short-term (2021-2050) and long-term (2071-2100) future horizons. Moreover, the daily P, Tmax, and Tmin from two different reanalysis datasets, namely NCEP (Kalnay et al., 1996) and NOAA (Compo et al., 2011;Slivinski et al., 2019Slivinski et al., , 2020 are considered here to represent the historical climate conditions (Sillman et al., 2013) over Canada and benchmark the performance of CMIPs during this period. Due to the mismatch among the spatial scale of climate models and reanalyzes, the Nearest Neighbor algorithm (Schulzweida et al., 2006) from the Climate Data Operator (https://code.zmaw.de/projects/cdo) is considered to regrid the model outputs to a common resolution of 1.5°.

Climate Indices and Their Significations
Canada is a large country with diverse topographic features and climatic conditions (Jeong et al., 2016). Changes in Canada's climate can potentially affect the length of growing season, presence of frozen soil, cycles of dry and wet years, heatwaves, forest fires, drought, and flood events, among others, with great BOURDEAU-GOULET AND HASSANZADEH Note. The information for CMIP5 and CMIP6 are respectively shown in light and dark gray rows. Canada is among the top 10 agricultural exporters in the world (Turcotte, 2013). Therefore, assuring Canada's crop production in the future is essential not only to feed the ever-increasing global population but also to support the country's economic growth (Qian et al., , 2020. GDD and MSR, calculated for the months of May to October, are one of the important factors affecting the soil moisture and crop conditions over the growing season (Smith et al., 2013). The GDD is commonly used to estimate the length of growing season for various crops. Here, the GDD with a base temperature of 5° is considered, which is relevant for the forage and canola production, two of the main crops in the country (Prairie Climate Centre, 2019; Qian et al., 2020). The MSR, rainfall during the growing season, is also critical for crop growth and yield (Grenier et al., 2019;Huber & Gillespie, 1992). Changes in the characteristics of GDD and MSR can potentially influence the food security at local, regional, and global scales (Lemmen et al., 2008;NRTEE, 2010).
FTC and AMP are critical for design and maintenance of Canadian infrastructures. An FTC occurs when Tmax is higher than 0 °C and Tmin is lower than −1 °C during the same day (Prairie Climate Centre, 2019). Changes in the historical characteristics of FTCs can potentially affect the land conditions such as soil stability and erosion, and cause damages to existing roads, buildings, power lines, and mining sites (Council of Canadian Academies, 2019; Guest et al., 2020;Mohammed et al., 2020;Palko, 2017). Thawing permafrost has already affected the existing infrastructure in Canadian North, where mostly indigenous communities reside and whose culture and livelihood revolve partly around the existing FTCs (Bush et al., 2019; NR-TEE, 2010). In addition to FTCs, the characteristics of AMPs are important for design, maintenance, and upgrade of various infrastructure including reservoirs and urban drainage systems (Mailhot et al., 2010;Yigzaw et al., 2013). Floods are the most catastrophic natural hazards in Canada and the federal government spends a large portion of disaster assistance budget for flood management (IBC, 2015). Therefore, an increase of AMPs can potentially augment the chance of flooding at urban and regional scales (Hassanzadeh et al., 2014;Henstra & Thistlethwaite, 2017).
Canada also has a long history of heatwaves, which have caused death and hospitalization of several hundreds of people, especially in major cities (Bustinza, 2013   also young and even physically active individuals (Government of Canada, 2020). Moreover, increases in the number of hot and dry days can affect the ecosystems and cause forest fires with major environmental consequences. Therefore, it is important to assess the evolution of MNH and MCD to develop efficient adaptation strategies, specifically in the areas, where the risk of heatwaves is higher (Council of Canadian Academies, 2019).

Intercomparison Procedure
For our analyses, the long-term annual average as well as Coefficient of Variation (CV) values for the six climate indices are calculated at 1.5° grids for NOAA and NCEP, as well as each simulation of CMIP5 and CMIP6 models over the historical and future periods. The "multi-simulation ensemble mean" for a given climate model in each grid represents the averaged long-term annual mean (and CV) values over all simulations, that is, one value per model. The "multi-model ensemble mean" values are then calculated by taking the average of multi-simulation ensemble mean (and CV) values over all seven climate models for CMIP5 (and CMIP6), that is, one value per project. The behavior of models based on multi-simulation ensemble mean values (and CVs) as well as multi-model ensemble mean values (and CVs) are compared at the grid scale across Canada. Moreover, the difference between the simulations of the climate models over 10 Canadian climatic zones is evaluated. These climatic zones, shown in Figure S1, are the East Artic, Great Lakes, Mackenzie Valley, Canadian Maritimes, Northern Plains, Northeast Forest, Northwest Forest, West Coast, Western Cordillera, and Yukon Territory. For more information on these zones, please refer to Plummer et al. (2006) and Mladjic et al. (2011).

Results
In the following sections, the behavior of CMIP5 and CMIP6 models is compared during the historical and future periods over Canada. In brief, Sections 3.1-3.3 answer the following questions during the historical period: Do CMIP5 and CMIP6 multi-model ensemble mean values (and CVs) for the six indices have similar magnitude and pattern of variations as reanalyzes over Canada? Is there any difference between the behavior of individual CMIP5 and their corresponding CMIP6 model pairs? Are the envelopes of CMIP6 and CMIP simulations comparable? Sections 3.4-3.6 address similar questions but for the future period.

CMIPs' Multi-Model Ensemble Mean Versus Reanalyzes During the Historical Period
Little difference exists between CMIP5 and CMIP6 multi-model ensemble mean GDD, MSR, FTC, AMP, MNH, and MCD indices over Canada during the historical period -see Figure 1. CMIP6 precipitation-based estimates are slightly higher in Northern Canada while temperature-based values are smaller in Southern Canada compared to CMIP5 (except for GDD  Figure S2. Same as average values, the difference between estimated multi-model ensemble mean CVs of for CMIP5 and CMIP6 is almost negligible. Moreover, the two reanalyzes present almost similar patterns and magnitudes of CVs, except for MSR and FTC. The largest divergence between CVs of reanalyzes and CMIPs is observed for the MNH index.

CMIP5 Versus CMIP6 Individual Models During the Historical Period
Here, the multi-simulation ensemble mean and CV values, estimated by individual CMIP5 models are compared with their CMIP6 counterparts from the same modeling center over Canada during the historical period. Figure 2 displays   models is observed for MCD. This is logical as both precipitation and temperature values are required to estimate this index. Therefore, the difference between estimated values by CMIP5 and CMIP6 models for these two variables propagates to the results. These analyses reveal that even though the multi-model ensemble mean values of CMIPs are almost alike (Figure 1), the estimations by individual CMIP5 and CMIP6 models can be considerably different.
The divergence between multi-simulation ensemble mean CVs, estimated by CMIP5 and CMIP6 models from the same modeling center can be significant in various cases -see Figure S3. These differences within and among modeling centers are not spatially consistent. It is interesting to note that even though the multi-model ensemble CVs, shown in Figure

CMIP5 and CMIP6 Simulations During the Historical Period Compared to Reanalyzes
Here, the objective is to understand whether the observed differences between CMIP5 and CMIP6 model pairs signify an improvement in the CMIP6 outputs over Canada or not. For this purpose, the estimated long-term annual average values of climate indices by climate models' simulations are compared with NOAA and NCEP over Canada's 10 climate zones. Therefore, for each simulation, the grid-based values are averaged over the given zones. Figure 3 illustrates the boxplots for simulations of CMIP5 (black) and CMIP6 (red) models as well as expected values based on NOAA (dotted line) and NCEP (dashed line) for the six indices over the East Arctic, North East Forest, and Great Lakes. Figures S4 and S5 Figure 3 and Figures S4 and S5). Although CMIP6 models do not outperform the CMIP5 models in BOURDEAU-GOULET AND HASSANZADEH 10.1029/2021EF001995 8 of 17 representing the AMPs, they perform better in estimating the MNH values. CMIP5 and CMIP6 models overestimate the MCD values of reanalyzes in northern regions.
Likewise, Figures S6-S8 display the CVs of the six indices based on all simulations of CMIPs as well as reanalyzes. The ensemble of CVs based on CMIP6 simulations do not necessarily cover or overlap with the CMIP5. However, the spread and values of CVs are usually similar for both CMIPs and are close to the reanalyzes, except for the MNH index for which some models present important span and estimates-see fifth left column in Figure S6. Had/UK pair tends to overestimate the CVs for GDD in all zones. In addition, both CMIPs often overestimate the CVs for MSR in comparison to reanalyzes.

CMIP5 Versus CMIP6 Multi-Model Ensemble Mean During the Future Period
The CMIP5 and CMIP6 multi-model ensemble mean values for the six climate indices are compared under different future scenarios over short-and long-term horizons in Figures S9 and 4, respectively. Despite relative similarities between CMIP5 and CMIP6 multi-model ensemble mean values during the historical period, there are some differences between their future magnitudes. For instance, although the values under RCP 4.5 and SSP 2-4.5 for the short-term horizon look almost identical at first glance, CMIP6 presents a slight increase in GDD, MSR, AMP, and MCD indices over the Central Canada ( Figure S9). The same observation can also be made under RCP 8.5 and SSP 5-8.5. Higher numbers of FTCs are projected in Northern Canada under SSP 5-8.5 than under RCP 8.5. Interestingly, under forcing 4.5 W/m 2 MNH values by CMIP6 are lower than CMIP5. The opposite can be seen under the 8.5 W/m 2 forcing, where MNH projections by CMIP6 are larger for the whole country. For the end of the century under forcing 4.5 W/m 2 , both multi-model ensemble mean values again present resembling patterns and values, however there is an increase in CMIP6 values over Northern and Central Canada except for FTC and MNH -see Figure 4 two left columns. Under the 8.5 W/m 2 forcing in the long-term horizon, CMIP6 projects an increase in all indices except for FTC. Similarly, Figures S10 and S11 exhibit the CMIP5 and CMIP6 multi-model ensemble mean CVs over the short-and long-term horizons under different scenarios, respectively. It is interesting to note that MNH has a significant increase in mean values projected by the end of century under forcing 8.5 W/m 2 over Central Canada as seen in Figure S11 fourth column.

CMIP5 Versus CMIP6 Individual Models in the Future Period
The relative difference between projections of CMIP5 and CMIP6 models from the same modeling center is shown for two sets of forcing during 2021-2050 (Figures S12 and S13) and 2071-2100 ( Figures 5 and 6), respectively. The black areas in these figures display the grids, where CMIP6 values are more than 100% larger than their corresponding CMIP5 ones and gray grids present the opposite. Considering each index, it is obvious that there is a divergence between projections of CMIP5 and CMIP6 individual models. In fact, for most of these indices the relative difference between the estimates of the pairs can be close or more than 100 percent, as an example see the dark areas for GDD, MNH, and MCD indices. Thus, using an individual climate model may lead to over or underestimation of future changes in the climate indices across Canada. Moreover, same as the historical period, the relative difference among CMIP5 and CMIP6 models is not resembling among modeling centers, regions, and indices. Another observation is that for the same index, the sign and pattern of relative difference between the estimates of most model pairs for the same modeling center are almost similar between the historical and future periods. Moreover, the magnitudes of differences between CMIP5 and CMIP6 models' averages are often larger under 8.5 W/m 2 than 4.5 W/m 2 forcing during the 2071-2100 period.
Likewise, Figures S14-S17 present the relative difference between CVs of CMIP5 and CMIP6 models for the four future experiments during the short and long-term horizons, respectively. Contrary to the multi-model ensemble mean CVs (Figures S10 and S11), the relative difference between multi-simulation mean CVs of CMIP5 and CMIP6 models is considerably large. For all projections and forcing scenarios, the spatial pattern of CVs for precipitation related indices tends to be random while for temperature related indices the CVs often present a consistent spatial pattern, especially for the GDD index. Like mean values, the relative difference for CVs is comparable for the same index and modeling center during short-and long-term periods as well as future scenarios, except for GDD and MNH. These observations again highlight the existing divergence between projections of individual CMIP5 and CMIP6 models, which cannot be observed by analyzing the multi-model ensemble mean values.

CMIP5 Versus CMIP6 Models' Simulations During the Future Period
In this section, we compare the simulations of CMIP5 and CMIP6 models over the  do not necessary provide a wider envelop of projections. For instance, the envelope of AMPs by CanESM2 with five realization is larger than its corresponding CMIP6 model (CanESM5) with 50 simulations-see the fourth left column in Figure 8. In addition, CMIP6 projections tend to have higher values than CMIP5 under 4.5 and 8.5 W/m 2 scenarios for most of the indices (except FTC) and zones. Interestingly, projected values by CMIP6 SSP 2-4.5 are larger than CMIP5 RCP 8.5 in some cases. This implies that using CMIP5 models solely, even under the pessimistic future conditions, may lead to underestimation of changes in the indices (e.g., see the projections by CanESM for MSR over Mackenzie Valley in Figure S22). Some of CMIP5 and CMIP6 models provide contradictory future projections for the same index in the same region. For instance, while CanESM2 simulations project lower AMPs than its historical values over the East Arctic zone, CanESM5 projects an increase in this index during the 2071-2100 period. UKESM1.0-LL presents abnormally large values for temperature-based indices over all zones under the forcing 8.5 W/m 2 for the short-term horizon ( Figures S19-S21).
BOURDEAU-GOULET AND HASSANZADEH 10.1029/2021EF001995 11 of 17 In addition, Figures S24-S31 exhibit the CV values for the six indices over the 10 Canadian climatic zones in the future. In general, the estimated CVs for all scenarios and projections tend to be similar for the same climate index in the future. Moreover, the sign and magnitude of changes in CV values between the historical and future periods depend on the considered index, climate model, and climatic zone.

Discussion and Conclusions
As Canada loses its cool, an improved understanding of climate change impacts over the country's society, economy and environment is required. The outputs of the state-of-the-art CMIP6 models have become recently available, therefore, their performance over CMIP5 models for climate change impact assessment have been questioned. Therefore, in this study, the behaviors of 389 simulations from seven CMIP6 and seven CMIP5 models in representing six important climate indices are compared across Canada during the BOURDEAU-GOULET AND HASSANZADEH 10.1029/2021EF001995 12 of 17 historical and future periods. Our analyses show that there is no clear superiority of CMIP6 over CMIP5 models based on their performance in representing the mean and CV characteristics of climate indices in comparison to NOAA and/or NCEP during the historical period. Moreover, none of the estimated values by individual CMIP5/CMIP6 models necessarily match with reanalysis datasets. In fact, the simulations generally tend to underestimate the GDD and overestimate the MSR. There are also models that present more considerable biases for some indices and zones, such as IPSL-CM6A-LR for MSR and AMP or Miroc6 for MNH. Moreover, comparisons between climate models' simulations during the historical and future period reveal a considerable change in the climate indices by the end of the century. These changes include an increase in GDD in Southern regions, an increment in MNH over the Great Lakes and Northern Plains zones, as well as changes in FTCs and precipitation-based indices across Canada. These changes can potentially cause challenges, for example, for agriculture, infrastructure, as well as human and environmental health depending on the considered Canadian region. However, further analyses are needed to develop adaptation and mitigation strategies to protect Canadian communities and ecosystems and support the country's economy.
In the future studies, bias correcting both CMIP5 and CMIP6 models' simulations is required before using them for impact assessments (Masud et al., 2021;Prairie Climate Centre, 2019). Due to the existence of various bias-correction methods (Maraun & Widmann, 2018), the uncertainties associated with their implementation should be also addressed (Cannon et al., 2015;Teutschbein & Seibert, 2012). In addition, although considered reanalyzes can be employed for bias correction, usage of combined datasets such as MSWEP (Beck, Wood, et al., 2019) or diverse sources such as satellite and gauge observations is also recommended (Beck et al., 2017;Donat et al., 2014;Faramarzi et al., 2015;Schurer et al., 2020;Sillmann et al., 2013) as reanalysis datasets often present numerous biases (Beck, et al, 2017;Donat et al., 2014). Furthermore, even though regridding is commonly used to compare different climate datasets in the same resolution, this process adds uncertainty to the comparison BOURDEAU-GOULET AND HASSANZADEH 10.1029/2021EF001995 13 of 17 procedure. This is because regridding can alter the statistical characteristics of model outputs, thus modify models' performance scores (Accadia et al., 2003). In addition, the nearest neighbor method is used in this study for regridding. It would be interesting to investigate the differences between the simulations caused by using other regridding approaches, such as bilinear interpolation or first and second order conservative remapping (P. W. Jones, 1999;Shea, 2014).
It should also be noted that the intercomparisons done in this study are based on the long-term annual mean and CVs. Hence, further analyses for example, at monthly and seasonal scales can be performed to better explore the inter-annual variability of climate variables/indices. Moreover, usage of larger sets of climate indices and/or more specific models is recommended for proper assessment of climate change impact on different sectors. For example, advanced models such as the Decision Support System for Agrotechnology Transfer module (J. W. Jones et al., 2003) can be used to evaluate crop growth and yield by taking into accounts economic and environmental factors in the future (Qian et al., 2020). Hydrological and water resources system models can be also utilized to project streamflow and assess flood risk (Seiller & Anctil, 2016). In addition, more profound investigations of impacts on the environment such as analyses of the Canadian Forest Weather Index System indices are needed to better assess climate change impacts on forest fires (Bush et al., 2019;Stocks et al., 2002).

Data Availability Statement
The datasets for this research are available online in https://esgf-data.dkrz.de/projects/esgf-dkrz/ for the CMIP5 and CMIP6 model simulations, as well as https://psl.noaa.gov/data/gridded/reanalysis/ for NOAA and NCEP reanalysis datasets.