• Open Access

A comparison of global marine surface-specific humidity datasets from in situ observations and atmospheric reanalysis

Authors


ABSTRACT

The accurate estimation of near-surface marine-specific humidity is necessary for climate and air–sea interaction applications. Available estimates of monthly mean-specific humidity spanning the past 50-years are based on a variety of sources including in situ observations, atmospheric reanalyses and datasets that blend many different data sources. Eight specific humidity datasets are compared and little consensus emerges as to mean values, regional variations and changes over time. For large area averages the datasets do show consistency in their interannual variations and, in the Extratropics, in their seasonal cycles. Adjustments applied to in situ observations from ships are shown to be smaller than differences among the datasets and in well-sampled regions and periods the in situ data are able to highlight biases in the reanalysis-based specific humidity estimates. Near surface-specific humidity estimates from two recent atmospheric reanalysis projects show markedly different responses in Tropical-specific humidity to the assimilation of satellite radiance measurements that became available in 1999. There is less confidence in reanalysis-based estimates of specific humidity over the ocean than over land. However the in situ-based humidity analyses have suffered in recent years with a reduction in observation numbers and lack of information on observation methods and heights. Consequently near-surface-specific humidity remains relatively poorly known over the oceans.

1. Introduction

Water vapour is the most abundant radiatively active gas in the atmosphere and exerts a strong feedback on the climate system (Trenberth et al., 2007). Near-surface humidity contrasts drive evaporation from the land and ocean and exert an important influence on the global energy and hydrological cycles. Despite this, surface humidity has historically received less attention than surface temperature, partly because of difficulties associated with humidity measurement. However, recent progress in adjustment and quality assurance of humidity records over both the land and the ocean has resulted in the use of surface humidity records to increase confidence in estimates (Kennedy et al., 2010; Willett et al., 2012) and attribution (Willett et al., 2007) of climate change.

An important motivation for the quantification of marine surface humidity is the estimation of heat and freshwater fluxes between the atmosphere and ocean. Direct measurement of turbulent fluxes is expensive and there are few measurements available (Brunke et al., 2011). Therefore in global marine flux datasets the ‘bulk’ form of the fluxes is often used where the air–sea latent heat flux depends on the specific humidity contrast between the sea surface (qs) and the air above it (q). The accepted benchmark for surface net heat flux accuracy on large space scales is ±10 Wm−2 over monthly to seasonal time scales (WCRP, 1989; Webster and Lukas, 1992; WGASF, 2000; Weller et al., 2004; Fairall et al., 2010), implying determination of individual components of the net heat flux to a few Wm−2. Where turbulent fluxes are parameterised this sets limit on the mean bias of input specific humidity observations of order 0.3 g kg−1 (Fairall et al., 1996; Bradley and Fairall, 2007). Other applications, for example global climate change or annual ice mass calculations, require even greater accuracy (Bourassa et al., 2010).

Humidity of the near-surface atmosphere is designated an Essential Climate Variable (ECV) by the Global Climate Observing System (GCOS, 2010). Observational requirements for climate applications are given in Table 1. Over the ocean, near-surface atmospheric humidity is measured by the Voluntary Observing Ships (VOS) programme and by a small number of fixed platforms (Woodruff et al., 2011). A small number of marine humidity observations date from the 1850s but data are extremely sparse until the 1950s and data coverage peaks between the 1970s and 1980s. Observations from the VOS and its predecessors are made routinely, initially in support of marine climatology (Maury, 1851) and more recently also in support of weather forecasting (Kent and Ingleby, 2010). Observations are concentrated in the major shipping lanes despite efforts to recruit ships operating in data sparse regions to the programme. Marine surface humidity observations are collated in the International Comprehensive Ocean–atmosphere Data Set (ICOADS, Woodruff et al., 2011) and have been used to produce gridded datasets and analyses of marine surface humidity (Da Silva et al., 1994; Josey et al., 1999; Bourassa et al., 2005; Dai, 2006; Willett et al., 2008; Berry and Kent, 2009). Each of the in situ datasets differs in the version of the source dataset, whether or not buoy observations are included, in the adjustments applied to VOS humidity observations and in the methods used to quality control (QC) and to produce a gridded analysis from scattered observations.

Table 1. Observational requirements from the WMO Rolling Review of Requirements. From http://www.wmo-sat.info/db/requirements (accessed 16 April 2012)
 GoalThreshold
Uncertainty (g kg−1)1%2%
Horizontal resolution25 km100 km
Temporal resolution30 min6 h

Atmospheric reanalysis model output has become an important resource for understanding the marine atmosphere. Advances in atmospheric modelling, data assimilation techniques and in the observing system have led to sustained improvements in reanalyses over time (Trenberth et al., 2010). Reanalyses provide an internally consistent representation of the atmosphere, but are limited by computational resources, simplifying assumptions in the model physics and can evolve their own climate especially in regions or periods which lack adequate observations to constrain the model (Bosilovich et al., 2011). Because of uncertainty in poorly constrained quantities such as surface fluxes (Rienecker et al., 2011) some researchers have developed blended products, combining reanalysis output with other observational data sources in an attempt to improve the representation of near-surface flux-related variables (Yu and Weller, 2007).

Because in situ measurements of marine surface humidity are sparse in many regions (Berry and Kent, 2009) there have been many attempts to retrieve humidity from satellite observations (Kubota et al., 2002; Bentamy et al., 2003; Chou et al., 2003; Andersson et al., 2010; Roberts et al., 2010; Shie et al., 2010), these products will be considered in another paper. This article will present an evaluation of several of the marine surface-specific humidity fields provided by in situ and reanalysis-based global datasets. Section 2 describes the available in situ observations of humidity from ships and buoys. Section 3 summarizes the characteristics of 10 marine surface-specific humidity datasets of which 8 are compared in this paper. Section 4 details the processing applied to the datasets to allow this comparison to be made along with adjustments that may be applied for observational height and measurement bias. Section 5 presents the results of the dataset comparison and some conclusions are drawn in Section 6.

2. In situ measurements of marine surface humidity

2.1. Voluntary observing ships

The VOS use a variety of different measurement methods for humidity but metadata indicating observational method is not contained within the ships meteorological reports. Instead, information on humidity measurement methods is obtained from World Meteorological Organisation (WMO) Publication No. 47 (Pub. 47, Kent et al., 2007) and this information has been appended to individual ship observations in the ICOADS (Woodruff et al., 2011). Kent et al. (2007) list the methods that could be entered for the type of humidity sensor and its exposure between 1970 and 2004. The metadata for 2004 was the last year available to Kent et al. (2007) and by this date there was an increase in electric methods of measurement as automated measurements became more common. The most common method is psychrometric: paired wet- and dry-bulb thermometers exposed in either a naturally ventilated screen or a mechanically ventilated psychrometer. Lack of entries for individual VOS in Pub. 47, combined with missing ship identifiers within reports, results in a substantial number of ICOADS humidity reports that do not have any humidity metadata associated with them. This problem has been compounded in recent years by the withholding of ship callsigns from real-time reports in response to security and commercial concerns by the operators of ships participating in the VOS programme. From December 2007 ICOADS contains no callsigns (Woodruff et al., 2011) although some identifier information may become available in the future.

Measurement of humidity at sea requires considerable care and VOS observers are issued with detailed instructions (Met Office, 1995; WMO, 2008). A handbook detailing requirements and methods for research quality observations describes best practices for measuring humidity at sea (Bradley and Fairall, 2007). Observational accuracy of a subset of the VOS providing additional observational metadata in the period 1988–1990 was analysed by Kent et al. (1993a). The VOS humidity data were classified according to whether they were measured using a mechanically ventilated instrument (denoted psychrometer) or by a naturally ventilated instrument (denoted screen). It concluded that psychrometer-measured humidities were on average drier than those measured using screens. There was some evidence that humidities measured by an Ocean Weather Ship (OWS) were drier still, but regional effects on the comparison could not be ruled out. Although VOS air temperature observations contain significant diurnal biases due to heating of the ship environment (Kent et al., 1993b; Berry et al., 2004; Berry and Kent, 2005), this is not expected to cause comparable biases in measurements of dew point temperature or specific humidity (Kent et al., 1993a; Kent and Taylor, 1996). It should be noted, however, that relative humidity will be underestimated if calculated using the erroneously high air temperature (Dai, 2006). A further potential source of bias occurs if an adjustment to a standard reference height is applied using estimates of atmospheric stability based on air temperature observations containing significant diurnal biases. The methods to measure humidity are changing over time and there has not yet been an assessment of the electric sensors that have recently come into more common use (Kent et al., 2007).

Measurements require adjustment to a standard reference height as in the surface layer the water vapour content decreases with distance from the sea surface. Adjustment from the measurement height to the marine standard reference height of 10 m is typically a few tenths g kg−1. Monin–Obukhov similarity theory (MOST; Smith, 1980, 1988) is typically used to make the adjustment and requires an estimate of the stability of the atmospheric surface layer. Stability is estimated from the near-surface wind speed, air temperature, humidity and the sea surface temperature (SST). Measurement heights for the VOS are not contained within the ship report but are available from Pub. 47. Although the height of the humidity sensor is not reported it can be estimated from available information: observing platform height (from 1968); barometer height (from 1995) and air temperature sensor height (from 2002), see Kent et al. (2007) for further information. Because of the lack of ship identifiers measurement heights have not been associated with individual ICOADS observations from December 2007.

2.2. Moored buoys

Because of the need to replenish reservoirs for wet-bulb measurements, autonomous humidity measurements from moored buoys have been a relatively recent development. Challenges include power limitations, stability in the presence of contaminants and durability (Weller et al., 2008). Trials of humidity sensors on operational moored buoys in the late 1980s revealed problems with reliability (Breaker et al., 1998). Sensors were added to the Tropical Atmosphere Ocean moored buoy array in 1989 and were assessed as having root-mean-square errors in relative humidity of 4% (McPhaden et al., 1998). Improved sensors were deployed in 1993 on 2 buoys in the Gulf of Mexico and were shown to meet WMO standards for reliability (Breaker et al., 1998) and humidity is now measured on many moored buoys (McPhaden et al., 1998, 2009; Bourlès et al., 2008). Buoy observations are typically made at heights of about 3 m for the Tropical arrays but many Extratropical buoys are larger and therefore measurement heights may be greater outside the Tropics. For the Tropical moored buoy arrays, measurements from the Pacific are within 10° of the Equator, measurements from the Atlantic and Indian Oceans are within 20° of the Equator. Extratropical buoys are usually coastal. Measurements of humidity from buoys were excluded from the analysis of Berry and Kent (2009) but included by other analyses (Da Silva et al., 1994; Josey et al., 1999; Bourassa et al., 2005; Willett et al., 2008).

3. Marine surface humidity datasets

3.1. In situ datasets

3.1.1. UWM/COADS

The longest in situ humidity dataset was produced by the University of Wisconsin-Milwaukee (UWM) based on the Comprehensive Ocean–atmosphere Data Set (COADS, the predecessor to ICOADS) Release 1 (Woodruff et al., 1987). The UWM/COADS dataset (Da Silva et al., 1994) covers the period 1945–1989 and presents an analysis of monthly means on a 1° grid for the global ocean. It is noted that only features with scale greater than 770 km (1300 km below 40°S) are retained by the analysis (Da Silva et al., 1994). The observations are interpolated to a regular grid using successive correction of 1° monthly box averages (following Cressman, 1959) that does not account for the varying uncertainty of the box averages (Kent et al., 2000). Specific humidity is adjusted to a 10 m reference height (q10) from an assumed measurement height of 20 m (A. M. da Silva, pers. comm. 2012). No adjustments are applied to the humidity data for observational bias. QC is applied using COADS ‘trimming flags’ to remove data outside of 3.5 standard deviations of a climatological median. No ice mask is applied but in areas of climatological ice cover (Alexander and Mobley, 1976) a tighter QC limit of 2.8 standard deviations is applied.

3.1.2. NOC Surface Flux dataset v1.1

NOCv1.1 (previously called the SOC Air–Sea Flux Climatology) is based on COADS Release 1a (Woodruff et al., 1993). The analysis uses successive correction but with a minimum radius of 331 km (c.f. 770 km used in UWM/COADS) and presents monthly means on a 1° grid for the period 1980–2005. Kent et al. (2000) estimated the spatial resolution of the NOCv1.1 dataset to have a mean resolution of about 4° latitude by 6° longitude with a modal value of 3° in each direction. QC uses the ICOADS trimming limits to exclude data outside of 4.5 standard deviations from a long-term median value. Adjustments to the humidity observations from screens (defined following Kent et al., 1993) are made by applying a linear correction to the dew point temperatures (Josey et al., 1999) and adjusted to give q10. Height and measurement method metadata from Pub. 47 are used, where available. In the absence of metadata, it is assumed that the measurement height for ships is 18 m and that one third of the observations is made using naturally ventilated screens. The data, methods and adjustments applied in the construction of NOCv1.1 are summarised and compared to the methods used in NOCv2.0 by Berry and Kent (2011).

3.1.3. FSU v3.0

The methods used to generate the Florida State University (FSU) Flux Dataset version 3.0 are described by Bourassa et al. (2005) and Smith et al. (2011). The input observations are from ICOADS Release 2.2 (a minor update from Release 2.1; Worley et al., 2005). FSU v3.0 is available from 1978 to 2004 on a monthly 1° grid; we use the version that is unfiltered in time (as used by Smith et al., 2011; S. R. Smith, pers. comm. 2012). QC is applied based on 3.5 standard deviations difference from the UWM/COADS dataset (Da Silva et al., 1994). In regions where the standard deviation in the UWM/COADS datasets is limited by poor sampling, minimum standard deviation limits are prescribed (Bourassa et al., 2005). A variational method using multiple constraints with objective weights is used to generate the fields (Bourassa et al., 2005). Specific humidity is adjusted to give q10 assuming a measurement height of 20 m and no other adjustments are applied.

3.1.4. Dai

This combined land and ocean dataset covers the period December 1975 to May 2005 and was based on ICOADS Release 2.1 (Worley et al., 2005) plus ICOADS near real-time updates from 2002. Dew point, air temperature and pressure observations are range checked and those observations outside 4.5 standard deviations of a 1° grid box mean for the entire period excluded. Specific humidity and relative humidity are calculated and subject to similar range and standard deviation checks. Those observations passing QC are averaged in 4° × 5° latitude/longitude seasonal grid boxes (Dai, 2006). Adjustments are not applied for measurement height or for observational bias. However, relative humidity is calculated using air temperatures adjusted for solar heating biases following Kent et al. (1993) noting that the specific humidity was calculated using unadjusted air temperatures (Kent and Taylor, 1996). Because not all the information required to adjust the air temperature is available for the entire period of analysis, unadjusted air temperatures are used to calculate relative humidity anomalies. Dai (2006) notes that although the adjustments to air temperature change the diurnal cycle and mean values of air temperature and relative humidity, the impacts on the interannual and longer-term variability are small. Analysis of the resulting dataset indicated a statistically significant reduction over time in relative humidity over the ocean. The analysis also indicated statistically significant upward trends in specific humidity over much of the Atlantic and parts of Indian and Pacific Oceans.

3.1.5. HadCRUH

The HadCRUH dataset is a combined land and ocean specific and relative humidity anomaly only dataset presented on a 5° monthly grid covering the period 1973–2003. The presentation of anomalies avoids the additional uncertainty inherent in the calculation of absolute values (Dai, 2006). Over the oceans, HadCRUH is based on ICOADS Release 2.1 (Worley et al., 2005) supplemented with ICOADS near real-time updates from 1998 onward. No adjustments are made for either height or observational bias. Specific humidity and relative humidity are calculated using unadjusted dew point and air temperatures. Heating biases in air temperatures should not affect the calculation of specific humidity (Kent and Taylor, 1996) but will result in an underestimate of the relative humidity. However, this should have a smaller effect on the relative humidity anomalies presented in HadCRUH than on the mean values (Dai, 2006). QC is applied to the marine data using differences from climatology, outlier and neighbour consistency checks, iterating to produce new climatology estimates created using only data passed by the previous round of QC until only a small number of observations are removed (Willett et al., 2008). Gridding methodology follows Rayner et al. (2006). In mixed ocean-land 5° grid boxes the land and marine estimates are averaged weighted according to the proportional presence of land and ocean in the box, but with a minimum contribution from each of 25% (Willett et al., 2007). There is no indicator in the dataset to differentiate between land, ocean and mixed grid boxes.

Analysis of the marine anomalies (Willett et al., 2007) showed an increase over time in specific humidity and a decrease in relative humidity statistically significant at the 1% level for global data. It was suggested that the decrease in relative humidity may have been at least partly due to a moist bias, apparent in the humidity data prior to 1982 and possibly related to changes in observing practice, but this could not be confirmed (Willett et al., 2008). Willett et al. (2012) showed that for the period 1976–2003 the variability and trends in marine q10 calculated from HadCRUH and Dai (2006) are similar.

3.1.6. NOC Surface Flux dataset v2.0

NOCv2.0 is based on ICOADS Release 2.4 (Worley et al., 2005) up to and including 2006 and ICOADS Release 2.5 (Woodruff et al., 2011) thereafter including preliminary products from 2008. The dataset as analysed here covers 1971–2010. Only ship observations are used. Adjustments are applied to give q10 using known heights where available (Kent et al., 2007) or default values based on a 2° area monthly gridded dataset of humidity measurement heights (Berry and Kent, 2009). Bias adjustments are applied to screen-measured humidities by reducing the specific humidity estimates by 3.4% (Berry and Kent, 2011). Where no measurement method was available a partial adjustment was made based on the fraction of observations with known method within the same month and 10° area made using screens (Berry and Kent, 2009). Due to the loss of callsign information in ICOADS, from 2007 onwards the last calendar month with information was used to estimate the fraction of screens in use and the fractional correction to apply, i.e. the data for January 2006 were used for all subsequent Januarys etc. The dataset is constructed using Optimal Interpolation (OI) with a Gaussian space scale of 300 km and a time scale of 3 d to produce daily fields on a 1° grid. The random uncertainty used to weight q10 relative to the background field of q10 is 1.3 g kg−1 calculated following Kent and Berry (2005). Uncertainty estimates are produced for each day and grid box and are combined to give monthly mean values and uncertainties accounting for the correlation in errors between days (Berry and Kent, 2011).

Trends in q10 from NOCv2.0 are similar to those shown by Dai (2006) and in HadCRUH over the period 1976–2003 (Willett et al., 2012). Differences are due to the adjustments applied to NOCv2.0 and are particularly noticeable prior to about 1985.

3.2. Reanalyses based datasets

3.2.1. 20CRv2

The 20th Century Reanalysis version 2 (20CRv2; Compo et al., 2011) provides 6-hourly, daily and monthly fields of q2 on a 2° grid from 1871 to 2010. 20CRv2 is a global atmospheric circulation dataset, assimilating only surface pressure reports over both the land and ocean and using observed monthly sea-surface temperature and sea–ice as boundary conditions. It uses an Ensemble Kalman Filter data assimilation method with background ‘first guess’ fields supplied by an ensemble of forecasts from a global numerical weather prediction model. This directly yields a global analysis every 6 h as the most likely state of the atmosphere, and also an uncertainty estimate of that analysis through the spread of the ensemble. The SST and sea–ice come from the HadISST dataset (Rayner et al., 2003) and marine pressure observations come from ICOADS Release 2.4 (from 1952) and ICOADS Release 2.5 (from 1871). No humidity observations are assimilated into 20CRv2.

3.2.2. ERAI

The European Centre for Medium-Range Weather Forecasts (ECMWF) Interim Reanalysis (ERAI) covers the period 1979 to near present (November 2011 at the time of writing) and the version analysed here is available on a 6-hourly 1.5° grid. There were problems with the representation of the hydrological cycle in the previous ECMWF reanalysis (ERA40, Uppala et al., 2005) and improvements have been developed (Andersson et al., 2005) and implemented in ERAI (Dee et al., 2011). In ERAI a four-dimensional analysis provides fields based on forecast model output with data assimilation (ECMWF, 2007), followed by a separate OI analysis of near-surface parameters, including q2 and 2 m temperature (T2; Dee et al., 2011). Several different SST and sea–ice concentration datasets are used with changes in July 2001, January 2002 and February 2009 (Dee et al., 2011).

Many different observational sources are assimilated into the forecast model but over the oceans, observations from the microwave sounding instruments [Special Scanning Microwave/Imager (SSM/I) and the Advanced Microwave Sounding Unit-B (AMSU-B)] dominate the humidity analysis (Andersson et al., 2007). The 2 m OI analysis uses background values derived from the lowest model level using MOST. Surface observations of marine humidity from ships and buoys are used in the 2 m OI analysis without any adjustments (A. J. Simmons, pers. comm. 2008; Simmons et al., 2010). Doubts about the quality of input observations of temperature and humidity from ships meant that background rather than assimilated fields were used in a recent climate assessment (Willett et al., 2012). The water cycle in ERA-Interim is thought to be too intense over the oceans (Dee and National Center for Atmospheric Research Staff, 2012).

Temperature and specific humidity are prognostic variables (ECMWF, 2007) but 2 m dew point temperature (Td2) is archived rather than q2. Td2 is computed from the estimate of q2 and surface pressure. Td2 uses the saturation formulation with respect to water: if the resulting Td2 is higher than the T2, Td2 is set equal to T2 (ECMWF, 2007; D. P. Dee., pers. comm. 2012).

3.2.3. MERRA

The National Aeronautics and Space Administration Modern Era Retrospective-analysis for Research and Application (MERRA) had a particular focus on improving the representation of the hydrological cycle compared to previous reanalyses (Rienecker et al., 2011). MERRA data is available from 1979 to present both as 0.667° × 0.5° 6-hourly fields and monthly mean fields. The version analysed here is the monthly dataset. MERRA makes values of both q2 and q10 available. MERRA assimilates many different observation types (Rienecker et al. 2008, 2011) including in situ humidity from radiosondes, ships and buoys and satellite radiances from several different satellite sources (Rienecker et al., 2008) of which the most important for humidity estimation are SSM/I and AMSU-A (Robertson et al., 2011). MERRA is forced at the sea surface by SST and sea–ice concentrations from the weekly 1° analysis of Reynolds et al. (2002) interpolated to each model time step (Rienecker et al., 2011). Atmospheric temperature and humidity at the lowest model level are prognostic variables and are affected by the boundary layer parameterisation and by assimilated observations (Roberts et al., 2012). The thickness of the lowest model level is variable with a mean value over the ice-free ocean of approximately 130 m, varying from about 120 m at high latitudes to about 134 m near the Equator. Humidity at the 2 and 10 m levels is calculated as a diagnostic variable using similarity theory with values at the surface and the lowest model level. It should be noted that at low wind speeds and especially in stable conditions, estimates at these heights may be decoupled from surface values (Smith, 1988).

The water budget in MERRA has been studied by Bosilovich et al. (2011) who note the impact of the assimilation of data from AMSU starting in November 1998. Analysis of MERRA output before and during the assimilation of AMSU data shows substantial changes in mean values, annual cycles and phase of variables related to the hydrological cycle. Assimilation of AMSU-A data leads to greater total column water over the ocean and an intensification of the hydrological cycle. The analysis increments, which indicate the changes that observations make to the model forecast, show the largest changes are in tropical and high latitude regions. The assimilation of observations is performed to correct biases in the estimated atmospheric state due to deficiencies in the model physics. The analysis increments have complex dependencies on the ability of the model to represent the processes that define the atmospheric state at any particular place and time and also on the availability and accuracy of observations that may constrain the model. Analysis increments have been used by Robertson et al. (2011) to examine the impact of observations on water fluxes in MERRA. For zonally averaged (land and ocean) analysis increments of vertically integrated moisture, the impact of first SSM/I in August 1987 and then AMSU-A in November 1998 is clear (Robertson et al., 2011). It should be noted that similar discontinuities, particularly in evaporation, are also present in other reanalyses such as the National Centers for Environmental Prediction (NCEP) Climate Forecast System Reanalysis (CFSR, Wang et al., 2010).

3.3. Blended datasets

3.3.1. OAFlux

The Objectively Analyzed air–sea Fluxes for the Global Oceans (OAFlux) is a blended product combining humidity fields from reanalyses and satellites (Yu et al., 2008) covering the period from 1958 to mid-2011. The National Centres for Environmental Prediction Reanalysis version 1 (NCEP1, Kalnay et al., 1996) is used throughout the analysis. NCEP version 2 (NCEP2; Kanamitsu et al., 2002) is used from its start in 1979. The ECMWF 40-year Reanalysis (ERA-40; Uppala et al., 2005) is used from 1958 to mid-2002. The satellite surface humidity fields constructed by Chou et al. (2003) from the SSM/I series of sensors were used from mid-1987 to the end of 2000. The three reanalysis models provide q2 whilst the SSM/I estimates are representative of q10. The SSM/I estimates have been adjusted to a 2-m reference height, i.e. q2, using the COARE3 algorithm (Fairall et al., 2003) and the near-surface humidity from OAFlux is representative of q2 (Yu and Weller, 2007). Weights for each field are calculated objectively using research-quality buoy observations (Yu et al., 2008) and are: NCEP1 = 1; NCEP2 = 1; ERA-40 = 1.3; SSM/I = 0.4. It should be noted that many different types of observation at the surface and through the atmosphere are assimilated into the reanalyses in differing combinations and with different data availability, QC and weighting. It is therefore difficult to determine the dependency of the OAFlux humidity estimates on any particular observation type.

4. Data processing

4.1. Regridding and masking

Each of the available datasets has been regridded onto the same 1° latitudinal/longitudinal monthly grid (Table 2 and 3). NOCv1.1a and Dai (2006) are not analysed in this paper. NOCv2.0 covers a longer period than NOCv1.1a and the methods used to calculate humidity fields have now been updated. The Dai (2006) dataset has not been made publically available. UWM/COADS, FSUv2.0, NOCv2.0 and OAFlux are already available on this grid (Table 2) and are ocean-only so no further processing is required. Ice free, ocean only grid boxes for MERRA, ERAI and 20CRv2 were selected using the land and ice masks supplied as part of those datasets. Ocean only grid boxes were selected from HadCRUH using the ERAI land mask. HadCRUH and 20CRv2 are available on 5° and 2° grids respectively and the 1° values have been assigned as the value from the nearest 5° or 2° grid box. q2 is not available for ERAI and has been calculated from the archived 6-hourly, 1.5° latitude/longitude values of Td2, T2 and surface pressure following Henderson-Sellers (1984). The q2 estimates have then been averaged to give estimates of monthly mean q2. For a limited period estimates of q10 have been calculated using surface pressure, Td2, T2, 10-m wind speed and SST data. The monthly 1.5° grid box values are then linearly interpolated to the monthly 1° grid. MERRA q2 and q10 values for "open water" grid boxes were linearly interpolated from the 0.667°x0.5° grid onto the 1° grid.

Table 2. Dataset characteristics
DatasetInput dataIn situ humidity platformsPeriodGridReference height (m)Height adjustmentBias adjustmentCoverage
UWM/COADSCOADS R1 & 1aShips

Buoys

1945–19891° monthly10From 20 mNoneOcean
NOCv1.1aCOADS R1a plus updatesShips

Buoys

1980–20051° monthly10Based on Pub. 47Yes (Josey et al., 1999)Ocean (with mask)
DaiICOADS R2.1 plus updatesShipsDecember 1975 to May 20054° × 5° seasonal10NoneNoneGlobal
HadCRUHICOADS R2.1 plus updatesShips

Buoys

1973–20035° monthlyanomalyNoneNoneGlobal
FSUv3.0ICOADS R2.2Ships

Buoys

1978–20041° monthly10From 20 mNoneOcean (with mask)
NOCv2.0ICOADS R2.4 & 2.5Ships1971–20101° monthly10Based on Pub. 47Yes (Berry and Kent, 2011)Ocean
20CRv2ICOADS R2.4 & 2.5None1871–20102° monthly2N/AN/AGlobal
ERAISee Dee et al. (2011)Ships

Buoys

1979–20111.5°2NoneNoneGlobal
MERRASee Rienecker et al. (2008)Ships

Buoys

1979–20110.667° × 0.5° monthly2, 10NoneNoneGlobal
OAFluxSee Yu et al. (2008)Ships

Buoys

January 1958 to July 20111° monthly2NoneNoneOcean

Figure 1 shows masks that have been applied to the datasets to account for the presence of land, sea–ice and regions of sparse data availability. NOCv2.0 and UWM/COADS are both masked according to the estimates of total uncertainty in NOCv2.0. 1° grid boxes where the average q10 uncertainty is greater than 1 g kg−1 or the air temperature uncertainty is greater than 1 °C are excluded (Figure 1(a)). FSUv3.0 does not provide estimates in regions with particularly poor sampling (Figure 1(b)). HadCRUH has been masked to only include 5° grid boxes where it contains data for every month (Figure 1(c)). Where only reanalysis-based datasets are compared a common mask is applied based on the sea–ice concentration fields from NOCv2.0, which are themselves based on Reynolds et al. (2002; Figure 1(d)). The mask is invariant over time and excludes data where the sea–ice concentration is above zero in any month during the analysis period. Where in situ datasets are compared a mask combining the NOCv2.0 and the FSUv3.0 masks is applied (Figure 1d)

Figure 1.

Masks applied to the datasets: (a) NOCv2.0 and UWM/COADS; (b) FSUv3.0; (c) HadCRUH; (d) reanalysis sea–ice and land mask (dark grey as used for 20CRv2, ERAI, MERRA and OAFlux), and combined in situ mask (light grey as used for UWM/COADS, NOCv2.0 and FSUv3.0).

4.2. Adjustments to marine humidity observations

Some datasets have adjustments applied to account for expected differences due to measurement method and height (Table 2). Adjustment is problematic because the required metadata on observation heights and methods is not always available and there is little high-quality validation data for humidity over the ocean (Roberts et al., 2012).

Comparison of humidity datasets is also complicated by the difference in reference heights chosen, the range of different humidity variables provided and the nonlinearity of the relationships among humidity, temperature and pressure. Over the ocean the reference height used to adjust observations is typically 10 m, for land observations it is typically 2 m ("screen height").

The effects of adjustments to VOS-specific humidity measurements are examined using the NOCv2.0 dataset (Berry and Kent, 2009, 2011). Three versions of the dataset have been constructed, one with no adjustments applied, one with only the height adjustment applied and one with the full adjustment for both height and measurement method. The height adjustment of humidity data increases the specific humidity estimates on average. This is because most of the measurements are made above the marine reference height of 10 m (Kent et al., 2007) and specific humidity typically decreases with height in the surface layer. In contrast the adjustments made for measurement method act to decrease the estimated specific humidity.

Both adjustments depend on the value of specific humidity itself, so are larger in warmer regions where the specific humidity is high. The adjustments also reflect regional and temporal differences in measurement height and methods (Kent et al., 2007). The availability of height and measurement method metadata varies greatly over time. This is sometimes because the metadata itself is missing, but is often because metadata cannot be associated with individual ICOADS reports because ship identifiers are missing. For most of the period after 1971 the availability of metadata improves over time, but after 2007 ICOADS callsigns and hence humidity method metadata are missing. Where metadata cannot be associated with individual reports adjustments are made based on available information. This is inherently uncertain and relies on the assumptions that the proportions of observations from the different methods are the same for the known and unknown cases. Similarly, the unknown heights are assumed to be similar to the known heights. The observing system characteristics are assumed to vary slowly over time. Recently Kennedy et al. (2011) approached the problem of unknown metadata by generating an ensemble of equally likely realisations of SST fields to account for uncertainty in metadata and adjustments. A similar approach would help to improve our understanding of uncertainties in bias-adjusted humidity fields but has not yet been attempted.

Figure 2 shows time series of the specific humidity adjustments based on NOCv2.0 for the period 1971–2006 for which metadata are available. The adjustment for bias in the screen-measured-specific humidities scales with humidity so is largest in regions and seasons of higher humidity, but modulated by the relative proportions of screens and psychrometers. Figure 3 shows averages over the period 1971–2006 of the adjustments. Annual averages of the height adjustment applied to the NOCv2.0 dataset over the masked region shown in Figure 1(a) increase by 0.11 g kg−1 between 1971 and 2006 (the specific humidity is increased in 1973 by ∼0.08 g kg−1 and in 2006 by ∼0.19 g kg−1). This picture is complicated by the adjustments applied for measurement method that are larger (on average the humidity is dried by ∼0.26 g kg−1) and have a more complicated spatial and temporal structure (Figures 2 and 3). The most important effect of the bias adjustments is an overall decrease of the specific humidity with a strong regional dependence in the size of the decrease. It should be noted that if no height correction is applied or the heights are assumed constant over time a small artificial drying trend would be present in the data due to the increase in observing height with time.

Figure 2.

Adjustments applied to NOCv2.0 dataset over the period 1971–2006 (g kg−1). Annual mean (left panels); seasonal cycle relative to annual mean (centre panels); mask showing data averaged (right panels). Solid line is the height adjustment, dashed line is the bias adjustment and the dotted line is the combined height and bias adjustment. (a) 25°N–75°N; (b) 25°S–25°N; (c) 75°S–25°S; (d) 75°S–75°N.

Figure 3.

Adjustments applied to NOCv2.0 dataset over the period 1971–2006 (g kg−1). (a) Height adjustment; (b) bias adjustment; (c) combined height and bias adjustment; (d) zonal mean adjustments averaged over 1971–1975. Solid line is the height adjustment, dashed line is the bias adjustment and the dotted line is the combined height and bias adjustment; (e) as (d) but for 2001–2005.

It should also be noted that the adjustments applied to NOCv2.0 are not well-represented by a linear trend and variations in the bias adjustment are small compared with both the annual cycle and the interannual variability.

4.3. Surface humidities in MERRA and ERAI

For the period 1989–2009 monthly mean q10 was calculated using 6-hourly values from ERAI using MOST and the transfer coefficients of Smith (1980, 1988). MERRA provides values for both q2 and q10 and differences between these values (Δq2–10) are shown in Figure 4 along with Δq2–10 calculated for ERAI. Mean Δq2–10 has a strong zonal dependence and is about 50% larger for MERRA than for ERAI (Figure 4). Roberts et al. (2012) showed that compared with research vessel observations the specific humidity difference in MERRA between the sea surface and 10 m (Δqs–10, denoted QSQA by Roberts et al., 2012) was overestimated at low specific humidity values (e.g. by 1 g kg−1 for specific humidities below 3 g kg−1) and underestimated at higher specific humidity values (e.g. by nearly 1.5 g kg−1 for specific humidities above 9 g kg−1). The values of Δqs–10 as calculated for ERAI are on average larger than those for MERRA in each of the ranges of specific humidities quoted by Roberts et al. (2012). There is a small decrease in Δq2–10 in the latter part of the MERRA analysis which is particularly noticeable in the tropics (not shown) and may be related to the assimilation of radiances from the AMSU-A starting in late 1998 (Bosilovich et al., 2011; Robertson et al., 2011; Roberts et al., 2012). In contrast, the ERAI values of Δq2–10 calculated using MOST increase slightly over the period 1989–2009.

Figure 4.

MERRA and ERAI q2q10 (g kg−1) 1990–1999. (a) MERRA average; (b) MERRA January (solid) and July (dotted) zonal averages; MERRA monthly mean over ice-free ocean; (d–f) as (a–c) but for ERAI.

Figure 5(a) shows global average differences between MERRA and ERAI values of qs, q2 and q10. qs has been calculated from MERRA SST by Roberts et al. (2012) for the period 1979–2007 and for ERAI in this study for the period 1989–2010. The global mean difference between in qs in MERRA and ERAI increases by slightly less than 0.1 g kg−1 at the start of 2002 which coincides with a change in the SST and sea–ice concentration forcing fields used by ERAI (Dee et al., 2011). Global mean q2 and q10 both increase in MERRA relative to ERAI over the period 1989–2007 (see also Figure 4(c) and (f)). Figure 5(b) shows the zonal average difference between qs in MERRA and ERAI, smoothed with a 12-month moving average filter. ERAI qs is typically higher than that from MERRA near the equator and at high latitudes and MERRA qs higher elsewhere. The change in the ERAI SST forcing in 2002 is clear from Figure 5(b), after 2002 zonal annual mean qs in MERRA is higher than in ERAI at most latitudes. There is an increase from 2001 in the annual cycle of the differences between MERRA and ERAI in both q2 and q10 for which the reasons are unclear.

Figure 5.

(a) monthly mean MERRA-ERAI specific humidity (g kg−1) averaged over area shown in Figure 1(d): dotted = q10; solid = q2 and dashed = qs. (b) MERRA-ERAI zonal mean qs over same region as (a) and with 12-point moving average filter. Solid line is zero contour.

Figure 6 shows MERRA ocean-only analysis increments for vertically integrated water vapour (liquid and ice water increments are much smaller) averaged in three periods: prior to the launch of SSM/I (period 1); SSM/I available but prior to the launch of AMSU-A (period 2) and after the launch of AMSU-A (period 3). Also shown are zonal averages of the ocean-only increments with a 12-month moving average filter applied. We note that these are total column water vapour amounts not near-surface humidity and that transport processes may lead to changes that are non-local, including over land (Bosilovich et al., 2011 note increased ocean to land transport of water after 1999). In period 1, the analysis increments are negative (i.e. drying) at high latitudes and in the tropics and positive (i.e. wetting) in mid-latitudes (more strongly in the Northern Hemisphere where the in situ observing system is concentrated). Localised regions of positive moisture increments are seen, some of which are probably associated with radiosonde observations on Pacific Islands. Period 2 (SSM/I) shows an increase of the negative moisture increments in high latitudes and of positive increments in mid-latitudes. In the tropics, there are strong regional variations that give an overall reduction in drying increments over period 2. Period 3 gives the most dramatic change (Bosilovich et al., 2011; Robertson et al., 2011; Roberts et al., 2012) with widespread positive increments and only localised regions where the increments remain negative. The impact of the changing observing system on the integrated water vapour is clear from a Hovmöller diagram of analysis increments smoothed with a 12-month moving average filter (Figure 6(d)).

Figure 6.

MERRA analysis increments for integrated water vapour (kg m−2 s−1 × 106): (a) pre-satellite period (1979–1986); (b) SSM/I period (1988–1997); (c) AMSU period (1999–2010); (d) zonal average of ocean-only data with 12-month moving average filter applied, vertical lines indicate start of SSM/I data in August 1987 and AMSU in November 1998. The thick line in all panels shows the zero contour.

5. Results

5.1. Reanalysis-based products, ocean/ice masking only applied

Figure 6 shows annual average values and seasonal cycles of q for the reanalysis products in three latitude bands. In the Northern region (25°N to 75°N, Figure 7(a)) the interannual variability is similar between all the products, but the mean values show substantial differences. All the products (ERAI, MERRA, 20CRv2 and OAFlux) have a reference level of 2 m, MERRA q10 is also plotted. MERRA q2 is highest averaged over the region, the q10 values from MERRA agree better with the q2 values from the other datasets. The other datasets show similar area annual average q2, agreeing to within about 0.1 g kg−1 until the mid-1990s. After this time the q2 estimates diverge: ERAI shows the smallest increase in humidity and OAFlux a slightly larger increase than either 20CRv2 or MERRA. By 2010, the gap between ERAI and OAFlux averaged humidities has increased to about 0.3 g kg−1. The seasonal cycle in the Northern Extratropics is large, about 6 g kg−1 and there is good agreement amongst the different products.

Figure 7.

Annual time series and seasonal cycles for reanalysis-based datasets. Note the different y-axis scaling. (a) 25°N–75°N; (b) 25°S–25°N; (c) 75°S–25°S; (d) 75°S–75°N. OAFlux q2 (dark solid); ERAI q2 (mid dotted); 20CRv2 q2 (light solid); MERRA q2 (light dotted); MERRA q10 (light dashed).

In the Tropics (25°S to 25°N, Figure 7(b)) q2 is higher and differences among products larger compared with those to the North although interannual variations remain similar. OAFlux and MERRA q2 are now much more similar although MERRA q2 is increasing more rapidly than OAFlux. q2 from ERAI and 20CRv2 remains more similar to q10 from MERRA. Changes over time in OAFlux and 20CRv2 remain similar, both showing an increase over the entire period. Since the 1980s, q2 in ERAI remains fairly constant (representing a drying in relative humidity as temperature has increased) and MERRA shows the strongest increase among these datasets in the Tropics. The annual cycle is much smaller than in the Northern Extratropics at less than 1 g kg −1. ERAI shows the strongest annual cycle in the Tropics and 20CRv2 the weakest. 20CRv2 shows peak humidities a month later than the other datasets and the minimum humidity seen in the other products in August is not seen.

In the Southern Extratropics (75°S to 25°S, Figure 7(c)), there is less correspondence between the interannual variability among the reanalysis products, probably as they are less constrained by the in situ observing system in this region. As in the Northern Extratropical region ERAI q2 is driest of all the reanalysis-based products and MERRA shows the highest q2. OAFlux shows the strongest increase in q2 over time. There is fairly good agreement in the representation of the seasonal cycle amongst the reanalysis-based products.

Annual averages for the global ice-free ocean are shown in Figure 7(d). 20CRv2 and OAFlux show similar magnitudes and change over time with OAFlux showing a slightly larger increase. Globally MERRA q2 starts highest and shows the strongest increase over time. ERAI shows the lowest q2 of any of the products in the global average (only MERRA q10 is lower) and also shows very little change over time. The difference between MERRA and ERAI q2 increases from 0.4 to 0.8 g kg−1 over the period 1979–2010. The global seasonal cycle is small ∼0.4 g kg−1.

Figure 8 shows the differences between decadal zonal averages of q2 for the reanalysis-based products using OAFlux as a reference. Only OAFlux and 20CRv2 are available over the full period from 1960 and 20CRv2 is not suitable for use as a reference as in these decadal averages there is a signature of the spectral nature of the 20CRv2 model evident as oscillations in long-term mean values (G. P. Compo, pers. comm. 2010). A 5° latitude moving average filter has therefore been applied to the zonal average of the 20CRv2 data plotted in Figure 8 to reduce the visual impact of the fluctuations. Filtered and unfiltered values are shown in Figure 8(b), filtered values only in Figures 8(c–f). It should be noted that OAFlux is not expected to be homogeneous over the analysis period as the input reanalyses show inhomogeneity over their period of record (Sterl, 2004) and the availability of different input products varies over the OAFlux period of record (Yu et al., 2008). We also note that OAFlux shows the strongest increase in q2 of any of the products in the Extratropics.

Figure 8.

Zonal mean differences from OAFlux for five decades 1960s–2000s, ice and land masks applied as shown in Figure 1(d). (a) OAFlux values 1960s–2000s; (b) 20CRv2 – OAFlux q2, zonal mean for 1960s shown with and without filtering in latitude. Panels (c–f) show filtered values only; (c) product − OAFlux q2; zonal mean for 1970s; (d) as (c) for 1980s; (e) 1990s; (f) 2000s.

Relative to OAFlux zonal decadal mean, q2 from 20CRv2 is dry in tropical regions in every decade from the 1960s to the 2000s. Agreement between 20CRv2 and OAFlux is closer outside the Tropics but systematic differences are evident and remain similar in character throughout the five decades. For the 1980s comparison 20CRv2, MERRA and ERAI are all available and all show lower q2 than OAFlux in the Tropics. MERRA shows higher q2 in mid-latitudes and ERAI q2 is lower than OAFlux in the decadal zonal mean at all latitudes and all available decades. Agreement in zonal q2 and its temporal evolution among the datasets is weak. The increase in MERRA q2 relative to OAFlux, ERAI and 20CRv2 in the tropics in each decade is striking. This is consistent with the analysis increments for column-integrated water vapour (Figure 6) which show that the assimilation of observations acts to increase the water vapour content of the MERRA atmosphere.

Figure 9 shows changes in decadal, zonal mean humidity for each product relative to its 1980s zonal mean. Plots in the first column present data masked as in Figures 7 and 8 (ice and land only), the plots in the other columns present data masked appropriately for the in situ products and are discussed in the next section. There is very little consistency among the reanalysis-based products as to how the zonal patterns of q2 change over time.

Figure 9.

Zonal mean q2 anomalies from 1980s average. Dark dotted, 1960s; solid, 1970s; dark dashed, 1990s; light solid, 2000s. First column (a, d, g, j) is for ice-free ocean, columns 2 and 3 are for combined in situ masked data as shown in Figure 1(d). Note that values are not plotted where the full decade is not available and HadCRUH contains only anomalies.

5.2. All products, masking for data uncertainty applied in addition to land/ice mask

Figure 10 shows the same information as Figure 7 but includes both in situ and reanalysis-based products and is masked as shown in Figure 1(d) and indicated in each row of Figure 10. Relatively high values for the FSUv3.0 product are seen in all regions, as noted by Smith et al. (2011). FSUv3.0 would be expected to contain higher q10 than NOCv2.0 as the adjustments applied to the NOCv2.0 dataset act, on average, to reduce q10. However, the size of the adjustments (Figures 2 and 3) is smaller than differences between NOCv2.0 and FSUv3.0 (Figure 10). Differences between FSUv3.0 and the UWM/COADS dataset are also large in their period of overlap suggesting that the adjustments are not the cause of these differences.

Figure 10.

Annual time series and seasonal cycles for reanalysis-based and in situ datasets for masked regions shown. Note the different y-axis scaling. (a) 25°N–75°N; (b) 25°S–25°N; (c) 75°S–25°S; (d) 75°S–75°N. OAFlux q2 (dark solid); ERAI q2 (dark dotted); 20CRv2 q2 (light solid); MERRA q2 (light dotted); MERRA q10 (light dashed), NOCv2.0 q10 (mid dotted); FSUv3.0 q10 (mid dashed); UWM/COADS q10 (dark dashed).

The datasets shown in Figure 10 are a mixture of estimates of q2 and q10. Although there is uncertainty in Δq2–10 (Figure 4), 2 m values are expected to be higher than 10 m values, and Δq2–10 should be largest in regions of high specific humidity. Figure 10 does not show a clear separation of products representing q2 and q10 indicating that uncertainties due to near-surface humidity gradients are only one contribution to the systematic differences among the datasets.

The changes seen in MERRA and ERAI bound those in the in situ datasets. In the Northern Extratropics (Figure 10(a)), the change in q10 over time in NOCv2.0 is more similar to that seen in OAFlux, 20CRv2 and MERRA than the smaller increase shown in ERAI. The difference between NOCv2.0 and UWM/COADS in the Northern Extratropics and Tropics (Figure 10(b)) in their period of overlap is consistent with the size of the bias adjustments applied in NOCv2.0 (Figure 2). In the Tropical average (Figure 10(b)), the increase in NOCv2.0 q10 is similar to that seen in OAFlux and 20CRv2 q2. Compared with NOCv2.0 q10 the increase seen in MERRA q2 is too large and that seen in ERAI is too small. There is only a small amount of data available in the Southern Extratropical comparison (Figure 10(c)) but as in the Northern Extratropics the changes seen in NOCv2.0 q10 are more similar to those in OAFlux and 20CRv2 rather than MERRA or ERAI. The difference between NOCv2.0 and UWM/COADS is larger than the bias adjustments applied to NOCv2.0 and is likely the result of a combination of adjustments with additional data available in NOCv2.0 and the difference in dataset construction methods used. The global comparison (Figure 10(d)) is dominated by data in the Northern Extratropics due to the in situ sampling pattern.

Figure 11 is similar to Figure 8 but for the in situ data products with regional masks applied (as shown in the last column of Figure 11). Again, OAFlux is used as a comparison reference and decadal zonal differences from OAFlux are plotted. Relative to OAFlux q2 in the 1960s UWM/COADS q10 is slightly drier in the Northern mid-latitudes, similar to 20CRv2. However, in the tropics UWM/COADS is higher than OAFlux whereas 20CRv2 is lower. The differences (relative to OAFlux) in the tropics would be further increased if Δq2–10 was taken into account (Figure 4). In the 1970s, NOC data become available and unadjusted, height-adjusted and fully adjusted versions of the NOCv2.0 dataset are plotted. The north–south variation seen in UWM/COADS relative to OAFlux is not seen in the unadjusted NOC values so is likely a result of increases in available data (COADS Release 1/1a vs ICOADS Release 2.4/2.5) or analysis method rather than adjustments applied to the observations in the NOC dataset. The main inputs to OAFlux in the 1970s are ERA-40 and NCEP1 that both make use of unadjusted humidity values and differences between OAFlux q2 and the fully-adjusted NOC q10 are small in the decadal zonal mean. In the 1980s, the FSU product becomes available and values are higher than the other in situ-based products as seen previously in Figure 10. Zonal differences from OAFlux are also much larger than for the other in situ datasets. The bias adjustments cause NOC-adjusted product to be lower than OAFlux in the tropics and diverges from the unadjusted NOCv2.0 and the UWM/COADS products. Similar features are seen in the 1990s and 2000s. Comparing Figures 8 and 11 (noting that the horizontal scales have the same range but different limits), it is clear that there is little consensus between the estimates of specific humidity from the different in situ and reanalysis-based products, their regional variations and long-term changes.

Figure 11.

Zonal mean differences from OAFlux for five decades 1960s–2000s, for in situ-based products masked as shown in Figure 10(d). (a) OAFlux values 1960s–2000s; (b) 20CRv2 – OAFlux q2, zonal mean for 1960s shown with and without filtering in latitude. Panels (c–f) show filtered values only; (c) product − OAFlux q2; zonal mean for 1970s; (d) as (c) for 1980s; (e) 1990s; (f) 2000s.

Figure 9 shows the decadal zonal changes in each reanalysis-based (column 2) and in situ dataset (column 3) relative to the dataset mean over the 1980s for the masked region shown in Figure 10(d). Also included in Figure 9 is the change in the HadCRUH anomaly dataset between the 1980s and the 1990s. This is similar to that seen in both the NOCv2.0 and FSUv3.0 datasets. When the changes are compared relative to each datasets own zonal mean for the 1980s its hard to find a consistent picture of decadal changes. Changes in q2 between the 1960s and the 1980s in UWM/COADS are more similar to those in 20CRv2 than OAFlux. UWM/COADS and NOCv2.0 show similar change between the 1970s and the 1980s, which again are more similar to 20CRv2 than OAFlux. NOCv2.0, FSUv3.0 and HadCRUH show similar changes between the 1980s and the 1990s but the reanalysis-based datasets all show different changes and none are similar to the changes seen in the in situ datasets.

The in situ q10 estimates are all based on similar observations, although the volumes of data and how they are adjusted and combined to give monthly fields might be expected to give differences in the final estimates of order 0.1 g kg−1. The increase in MERRA q2 of nearly 0.8 g kg−1 between the 1980s and the 2000s at the Equator is not seen in any of the other datasets and ERAI is the only dataset to show a decrease in q2 at the same time and place. The in situ-based datasets are fairly sparsely sampled in the Tropics but there are some well-sampled shipping lanes in Tropical regions (Berry and Kent, 2009). Large-scale decadal averages should therefore be broadly representative of conditions which suggests that the changes seen in MERRA and ERAI in the Tropics may not be realistic.

6. Discussion and conclusions

A comparison of eight different datasets has revealed substantial structural uncertainty in marine near-surface humidity, a GCOS ECV. Traditionally, humidity observations have been made on VOS but their sampling has never been sufficient to produce global estimates of marine surface humidity to the accuracy required for surface flux estimation (WCRP, 1989; Berry and Kent, 2009). VOS have, however, provided observations of useful accuracy for decades in well-sampled regions (Berry and Kent, 2009). This is now at threat due to declining numbers of VOS and a lack of the observational metadata required to make adjustments to the humidity observations for height and observational bias (Kent et al., 2007; Berry and Kent, 2011). Increasing automation on VOS has led to a change in instrument types and the impact of these changes has not yet been assessed, partly because of the lack of observational metadata. Observations from moored buoys supplement the VOS observations from the 1990s but only limited assessments of their characteristics have been made (McPhaden et al., 1998).

The FSU humidities are about 0.8 g kg−1 higher than the other in situ datasets; the reasons for this are currently unknown. The differences between the NOCv2.0 and UWM/COADS in situ datasets are consistent with the size of the adjustments applied to the NOCv2.0 dataset and other differences due to input data and construction methods. The uncertainty estimates in the NOCv2.0 dataset in the best sampled regions are ∼0.2 g kg−1 which is substantially smaller than the differences amongst the reanalysis-based datasets. There are also substantial differences amongst the datasets in terms of magnitude, regional mean values and changes over time.

The evolving observing system is known to impact both in situ and reanalysis datasets. ERAI and MERRA are both examples of state-of-the-art atmospheric reanalysis but show very different characteristics in terms of how marine surface-specific humidity changes over time. MERRA shows moistening over the period from 1979, especially in the Tropics, but ERAI indicates much smaller changes. The response of MERRA to the assimilation of observations can be examined using analysis increments and a change in the analysis increments for integrated water vapour in response to the availability of first SSMI and then AMSU-A has already been noted (Robertson et al., 2011). In contrast, observations from SSMI and AMSU-B have the most impact on the ERAI analysis (Andersson et al., 2007). There is an increasing divergence of q estimates from the two reanalyses over time at all latitudes. Trends are not quantified here because they are dominated by the model representation of El Niño-Southern Oscillation variability in q due to the short (∼30 year) period of record. The OAFlux blended analysis combines data from older reanalyses and satellite estimates of q with weights derived from comparison with in situ observations (Yu et al., 2008). OAFlux is closer in to the in situ datasets (UWM/COADS and NOCv2.0) than to the reanalyses despite being representative of q2 rather than q10.

The in situ datasets UWM/COADS and NOCv2.0 show more similarity to OAFlux in magnitude and regional means than OAFlux does to the other reanalysis-based datasets. The zonal, decadal differences between OAFlux and 20CRv2 are similar from the 1960s to the 2000s suggesting that 20CRv2 shows a fairly consistent model bias over time relative to OAFlux. As noted earlier the response of MERRA and ERAI q2 to the increasing availability of satellite observations acts in opposite directions in Tropical regions. OAFlux is based on earlier versions of reanalyses which would not have been able to assimilate the new observations from AMSU. It is interesting that there appears to be better agreement between NOCv2.0 and OAFlux than with either MERRA or ERAI. OAFlux ingests ERA-40, NCEP1 and NCEP2 which are thought to be poorer estimates of surface humidity over land at least (Simmons et al., 2010). In the 2000s, MERRA zonal average q2 becomes more similar to OAFlux (and hence the in situ datasets) than in previous decades that may indicate that the assimilation of satellite observations has acted to some extent to correct model error in MERRA which would make trend estimation from MERRA inaccurate.

Although there is poor agreement amongst the different datasets as to the mean near-surface humidity, the representation of interannual variability and seasonal cycles over large areal averages is more consistent. This is because uncertainties in anomalies will be smaller than those in absolute values. Whilst there is uncertainty in the adjustments required for in situ observations due to the lack of observational metadata and the need to assess new sensor types the uncertainties are smaller than the differences seen between ERAI and MERRA. On the basis of the results presented here, the structural uncertainty (i.e. that due to choice of data, QC and gridding methodology) in the marine-specific humidity and its changes over time are large and unlikely to meet the accuracy requirements described in the introduction.

Comparison of humidity datasets is problematic because of the nonlinear relationship between the different variables that can be used to represent humidity and temperature, pressure and atmospheric stability. This means that any conversion between humidity variables, or adjustment to different reference heights, is only accurate when calculated at the native model resolution. It would facilitate long-term comparisons if monthly mean values of at least q and rh were made available by dataset providers at the standard marine reference level of 10 m in addition to the land reference level of 2 m above sea level. The need to convert between different reference levels adds uncertainty to the comparisons presented, but presently other differences between the datasets are larger.

Given the large differences amongst the different datasets it seems unlikely that any of the data sources will be reliable enough to provide humidity estimates accurate enough to allow the estimation of monthly mean fluxes to better than 10 W m−2. This accuracy may, however, be achievable using in situ data for some regions and periods.

To improve the accuracy of the in situ q10 estimates it will be necessary to revisit the adjustments applied to the observations including an assessment of newer types of VOS humidity sensors. It will also require the development of an improved methodology to better handle missing metadata, including the exploration of dataset ensemble methods. An assessment of the accuracy of humidity measurements on moored buoys in the field would also provide valuable information. One resource that is so-far underutilised is the use of Research Vessel humidity measurements to provide an estimate of absolute biases in specific humidity.

Acknowledgements

The International Comprehensive Ocean–atmosphere Data Set (ICOADS) archive of surface marine observations underpins the datasets used in this study. Scott Woodruff, Sandy Lubker, Steven Worley and Eric Freeman have all provided help and expert advice when needed. We are grateful to the National Center for Atmospheric Research for making the NOCv2.0 data available from the Computational and Information Systems Laboratory Research Data Archive (http://rda.ucar.edu/datasets/ds260.3/). UWM/COADS data were obtained from the International Research Institute for Climate and Society/Lamont-Doherty Earth Observatory Climate Data Library (iridl.ldeo.columbia.edu/). HadCRUH data were downloaded from the Met Office Hadley Centre observations datasets webpages (http://www.metoffice.gov.uk/hadobs/index.html). FSUv3.0 was downloaded from the Center for Atmosphere–ocean Prediction Studies of Florida State University (http://coaps.fsu.edu/RVSMDC/FSUFluxes/index.php). Twentieth Century Reanalysis V2 data provided by the NOAA/OAR/ESRL PSD, Boulder, Colorado, USA, from their Web site at http://www.esrl.noaa.gov/psd/. OAFlux data were provided by the WHOI OAFlux project (http://oaflux.whoi.edu) funded by the NOAA Climate Observations and Monitoring (COM) programme. ERA Interim data were downloaded from the ECMWF data server. MERRA data were obtained from the Global Modeling and Assimilation Office (GMAO) and the NASA Goddard Earth Sciences (GES) Data and Information Services Center (DISC). We would like to thank the anonymous reviewers for their help in improving this article. This study received funding through the NERC National Centre for Earth Observation (NCEO) and the National Oceanography Centre's National Capability programme.

Ancillary