SEARCH

SEARCH BY CITATION

Keywords:

  • marine meteorological data;
  • ship data;
  • buoy data;
  • data rescue;
  • sea surface temperature;
  • sea level pressure;
  • humidity;
  • metadata

Abstract

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. The national and international contributing community
  5. 3. Release 2.5
  6. 4. Data characteristics and unresolved issues
  7. 5. Data and product access
  8. 6. Community data products derived from ICOADS observations
  9. 7. Future plans
  10. 8. Conclusions
  11. Acknowledgements
  12. References

Release 2.5 of the International Comprehensive Ocean-Atmosphere Data Set (ICOADS) is a major update (covering 1662–2007) of the world's most extensive surface marine meteorological data collection. Building on extensive national and international partnerships, many new and improved contributing datasets have been processed into a uniform format and combined with the previous Release 2.4. The new data range from early non-instrumental ship observations to measurements initiated in the twentieth century from buoys and other automated platform types. Improvements to existing data include replacing preliminary Global Telecommunication System (GTS) receipts with more reliable, delayed mode reports for post-1997 data, and in the processing and quality control (QC) of humidity observations. Over the entire period of record, spatial and temporal coverage has been enriched and data and metadata quality has been improved. Along with the observations, now updated monthly in near real time, Release 2.5 includes quality-controlled monthly summary products for 2° latitude × 2° longitude (since 1800) and 1° × 1° boxes (since 1960), together with multiple options for access to the data and products. The measured and estimated data in Release 2.5 are subject to many technical changes, multiple archive sources, and historical events throughout the more than three-century record. Some of these data characteristics are highlighted, including known unresolved errors and inhomogeneities, which may impact climate and other research applications. Anticipated future directions for ICOADS aim to continue adding scientific value to the observations, products, and metadata, as well as strengthen the cooperative enterprise through expanded linkages to international initiatives and organisations. Copyright © 2010 Royal Meteorological Society


1. Introduction

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. The national and international contributing community
  5. 3. Release 2.5
  6. 4. Data characteristics and unresolved issues
  7. 5. Data and product access
  8. 6. Community data products derived from ICOADS observations
  9. 7. Future plans
  10. 8. Conclusions
  11. Acknowledgements
  12. References

The International Comprehensive Ocean-Atmosphere Data Set (ICOADS) contains marine surface data spanning the past three centuries. The observational archive and monthly summary products were first made available 25 years ago (Slutz et al., 1985; Woodruff et al., 1987). Each subsequent release (see Appendix) has brought notable improvements that have benefited research, education, and commercial operations. The continuing objectives that remain are to collect as much original estimated (non-instrumental) and measured in situ data as possible; to treat each observation systematically—preserving data source identification and measurement metadata with each record, and converting units and coding schemes to a uniform set—to perform basic quality checks; and to freely distribute the data and products worldwide. As it contains observations from many different observing systems encompassing the evolution of measurement technology over hundreds of years, ICOADS is probably the most complete and heterogeneous collection of surface marine data in existence.

The expanding national and international community providing data, and otherwise contributing to ICOADS development, is reviewed in Section 2 of this paper. Section 3 describes ICOADS Release 2.5 (1662–2007) and the continuing monthly observational updates for 2008 onwards. Section 4 discusses selected data characteristics of Release 2.5 and highlights some outstanding data issues. Sections 5 and 6 briefly review access to ICOADS data and products and community-derived products respectively. Section 7 covers future goals and challenges for ICOADS and conclusions are in Section 8.

2. The national and international contributing community

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. The national and international contributing community
  5. 3. Release 2.5
  6. 4. Data characteristics and unresolved issues
  7. 5. Data and product access
  8. 6. Community data products derived from ICOADS observations
  9. 7. Future plans
  10. 8. Conclusions
  11. Acknowledgements
  12. References

ICOADS has grown from a US-centric effort between the National Oceanic and Atmospheric Administration (NOAA)—its Earth System Research Laboratory (ESRL) and National Climatic Data Center (NCDC)—and the National Center for Atmospheric Research (NCAR) to an expanded United States and international partnership. This growth has added value to ICOADS in many ways. New data sources have been uncovered and contributed; expert knowledge has enabled data improvements spanning the early sailing fleets, ship and measurements metadata, and quantification of data uncertainty; to name just a few examples. Data provision, collation, and distribution remain the responsibility of the founding partners, but other countries and international organisations—including the Joint World Meteorological Organisation (WMO)–Intergovernmental Oceanographic Commission (IOC) Technical Commission for Oceanography and Marine Meteorology (JCOMM)—now make noteworthy contributions.

From the operational and research areas, ICOADS incorporates observations from a variety of existing activities. Real-time Global Telecommunication System (GTS) data are provided by the NOAA National Centers for Environmental Prediction (NCEP), marine surface meteorological observations and near-surface profile temperatures are contributed by the NOAA National Oceanographic Data Center (NODC) through the World Ocean Database (WOD), and environmental US moored buoy and Coastal Marine-Automated Network (C-MAN) data are provided by the NOAA National Data Buoy Center (NDBC).

Data from tropical moored buoy arrays—specifically, the Tropical Atmosphere Ocean/TRIangle Trans-Ocean buoy Network (TAO/TRITON) array in the Pacific (McPhaden et al., 1998) and the Prediction and Research Moored Array in the Atlantic (PIRATA) (Bourlès et al., 2008)—are provided by NDBC, the NOAA Pacific Marine Environmental Laboratory (PMEL) and the Japan Agency for Marine-Earth Science and Technology (JAMSTEC). The Center for Ocean-Atmospheric Prediction Studies (COAPS) at Florida State University contributes high-resolution measurements from some ocean research vessels (R/Vs) (Gould and Smith, 2006). Integrated Science Data Management (ISDM), of Fisheries and Oceans—Canada, provides quality controlled (QCd) drifting (plus some moored) buoy data.

The JCOMM global collecting centres (GCCs) distribute worldwide delayed mode (DM) (i.e. paper or electronic logbook) Voluntary Observing Ship (VOS) data on a quarterly basis, currently from about 26 contributing members (Woodruff et al., 2009). The National Oceanography Centre, Southampton—UK, contributes quality-checked WMO (1955) ship metadata (Kent et al., 2007). Beyond the most active recent and ongoing major programs noted above, there are many others, including substantial contributions from Europe and Asia (see e.g. Worley et al., 2005 and Woodruff et al., 2005).

In the historical arena, the RECovery of Logbooks And International Marine data (RECLAIM; Wilkinson et al., 2010), Climatological database for the World's Oceans (CLIWOC) (García-Herrera et al., 2005), and Atmospheric Circulation Reconstruction over the Earth initiative (ACRE; http://www.met-acre.org/) are all multinational efforts that have augmented ICOADS with seventeenth to twentieth century data and metadata. For example, in recent joint projects, the UK Met Office and NOAA Climate Data Modernization Program (CDMP) (Dupigny-Giroux et al., 2007) have provided digitised and quality-checked data from logbooks (e.g. Brohan et al., 2008) and historical publications held in UK archives.

Continuity and coordination for ICOADS have been promoted by two series of meetings that began over a decade ago: The JCOMM Workshops on Advances in Marine Climatology (CLIMAR) were held in Vancouver, Canada, 1999, Brussels, Belgium, 2003, Parker et al., 2004 and Gydnia, Poland, 2008 (Charpentier et al., 2008). Alternating approximately biennially with CLIMAR, the Workshops on Advances in the Use of Historical Marine Climate Data (MARCDAT) have been held in Boulder, USA, 2002, and Exeter, UK, 2005 (Kent et al., 2007). Sponsors of past meetings have included WMO, the Global Climate Observing System (GCOS), and the Climate Variability and Predictability Project (CLIVAR). These meetings have done much to guide the development of ICOADS.

3. Release 2.5

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. The national and international contributing community
  5. 3. Release 2.5
  6. 4. Data characteristics and unresolved issues
  7. 5. Data and product access
  8. 6. Community data products derived from ICOADS observations
  9. 7. Future plans
  10. 8. Conclusions
  11. Acknowledgements
  12. References

Release 2.5 (R2.5) data processing was completed in July 2009. This marks a major update where 238 million (M) reports from the preceding Release 2.4 (R2.4) were merged together with new data, subjected to duplicate analysis and elimination, quality checked against climatologies, and preconditioned to fix specific problems and remove suspect data. The resultant output was then compared with R2.4. R2.5 has 261M records in the International Maritime Meteorological Archive (IMMA) format covering 1662–2007. This archive is a blend of DM and GTS data, where DM records are preferentially selected in cases where duplicates are detected.

The IMMA format (Woodruff, 2007) has been recommended as a standard to JCOMM and is ideal for marine surface data spanning many centuries. It is an ASCII-based format with a fixed ‘core’ set of the most commonly reported meteorological variables sufficient for most users, also including the time, location, and individual platform identification (e.g. ship callsign or WMO buoy number, if available). In addition to the IMMA core, an arbitrary number of data ‘attachments’ can be made. Ship metadata, original data records, QC flags, special ICOADS source-tracking information, platform type, and a variety of other ancillary metadata are preserved in these attachments.

A new procedure to extend ICOADS to near real time is also coincident with R2.5 providing IMMA records based on the best available GTS receipts (currently based exclusively on NCEP data). This forward extension to ICOADS is updated monthly, and lags real-time by 2 to 6 weeks. These data are subjected to basic quality checks and the IMMA core fields are populated to the maximum extent possible. This product is replacing an existing near-real-time ASCII-formatted product containing only a limited selection of data fields.

R2.5 greatly improved ICOADS by extending the period of record backward to 1662 (i.e. by 122 years, albeit with sparse coverage), and forward through the end of 2007, and generally increasing the number of available records over the whole period. Twelve major new or replacement historical data sources were incorporated (Figure 1) and numerous other data sources reprocessed (the Appendix provides R2.5 data composition details). Some new sources were specifically digitised for ICOADS, e.g. US Marine Meteorological Journals (MMJ), 1878–1894, 1.8M records. A few data sources replaced earlier versions, e.g. the World Ocean Database 2005 (WOD05), 1772–2005, 7.9M. Others such as the UK Marine Data Bank (MDB) (previously blended into periods outside 1950–1979; Woodruff et al., 2003) partially duplicated reports already in ICOADS, but also provided some unique or improved contributions.

thumbnail image

Figure 1. Major historical data sources added to ICOADS Release 2.5. Horizontal green lines illustrate the time range of the original data sources. The annual numbers of reports are plotted as curves, blue for the previous R2.4, and red for R2.5. For clarity, the vertical scale is truncated at 9M; years 2005–2007 have 13M, 15M, and 16M total reports (not visible) in R2.5, respectively. Data coverage prior to 1800 is very sparse

Download figure to PowerPoint

Based purely on the number of records, regardless of which parameters might be available, R2.5 has increased the global spatial coverage. Figure 2 illustrates important increases in global sampling in R2.5 distributed over the full period since the 1860s. The 1880s shows outstanding improvement (up to 9%), primarily due to addition of the MMJ collection (Figure 1), which provides extensive observations throughout the diurnal cycle (plus accompanying detailed instrumental metadata and instructions). Deutscher Wetterdienst (DWD) historical archive subsets helped increase coverage from the late 1800s through 1914, and the 2003 Edition of the Japanese Kobe collection accounts for major increases around the 1920s. The UK Royal Navy Ship's Logs (Brohan et al., 2008) helped fill in the data-sparse period (1938–1947) around World War II, and the UK MDB helped enhance coverage particularly during the 1960s and 1970s.

thumbnail image

Figure 2. Annual percentage ocean area sampled for SST for R2.5 (red curve) compared to R2.4 (blue curve) (right axis). Annual percentage increase in global ocean area sampled for R2.5, compared to R2.4 (bars, left axis)

Download figure to PowerPoint

While these increases had limited impact on total ocean area sampled (still peaking at about 60% around 1980), they should add confidence to long-term climate estimates. Focusing on the 1880s, for example, three important variables—sea surface temperature (SST), sea level pressure (SLP), and relative humidity (RH)—show greatly increased global coverage relative to Release 1 and to the previous update of that period (Figure 3). The nearly void RH coverage prior to R2.5 was addressed by a QC modification discussed in Section 6.

thumbnail image

Figure 3. Decadal (1880–1889) totals of sea level pressure (SLP) observations (upper) illustrating data additions between Releases 1 (completed in 1985), 2.0 (2002), and 2.5 (2009). The colours show the number of observations in a 2° box per decade. Grey indicates 1–10; dark blue indicates 11–20; …; orange indicates 641–1280; red indicates 1281 or more. (Middle and Lower) As for (upper) except for SST and RH

Download figure to PowerPoint

4. Data characteristics and unresolved issues

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. The national and international contributing community
  5. 3. Release 2.5
  6. 4. Data characteristics and unresolved issues
  7. 5. Data and product access
  8. 6. Community data products derived from ICOADS observations
  9. 7. Future plans
  10. 8. Conclusions
  11. Acknowledgements
  12. References

An important objective with each new Release is to replace GTS data with higher quality DM data. Normally this has limited effect on temporal and spatial coverage, but occasionally DM data can more appreciably augment coverage. This is illustrated for the 1998–2006 period, where global coverage increased 2% or more with respect to R2.4 (Figure 2). This effect arises largely because no DM ship data had been added to ICOADS for that period (owing to project resource limitations) since 2002.

This impact can be segregated into three broad platform categories: observations from ships, moored buoys, and drifting buoys. In R2.4, all ship data from 1998 and onward were derived from GTS receipts, whereas in R2.5 about 50% of data in this period are from DM collections (Figure 4). Here DM data from the GCCs (and other sources) have replaced the GTS data for 1998 onward and additional new data have been added from historical sources throughout the period of record (e.g. numbers of reports increased 30% or more around World War II through blending the UK Royal Navy Ship's Logs; Figure 4).

thumbnail image

Figure 4. Annual distribution (1937–2007) of the DM and GTS ship data (mainly VOS and some R/Vs) mixture in R2.4 (top; note: 2007 not shown because R2.4 ended in May) and R2.5 (bottom). The curve in the top panel contrasts the resultant R2.5 distribution

Download figure to PowerPoint

The differences in moored buoys between R2.4 and R2.5 are relatively minor when contrasting annual record counts and data sources (Figure 5). It is noteworthy though that minor decreases in R2.5 have been achieved with better duplicate elimination, e.g. 2005–2006. The mixture of sources has not changed much, but the tropical moored buoys were replaced with a new DM version acquired in 2007, in which importantly the air moisture measurements were translated into IMMA core fields for the first time.

thumbnail image

Figure 5. Annual distribution (1971–2007) of the DM and GTS moored buoy data mixture in R2.4 (top; note: 2007 not shown because R2.4 ended in May) and R2.5 (bottom). The curve in the top panel contrasts the resultant R2.5 distribution. DM deck numbers: TAO/TRITON and PIRATA (143–145), ISDM (714), and NDBC (876–883). GTS deck numbers: NCEP (793–794 and 893), plus small additions from historical US Navy and Air Force sources (555 and 888)

Download figure to PowerPoint

The drifting buoy records in R2.5 are dominated by DM data from ISDM (Figure 6). The 2005–2007 real-time data in R2.4 have been replaced with fewer DM reports in R2.5. The partial and duplicate reports in the real-time data stream have been consolidated at ISDM (through removal of duplicates plus some compositing of fragmentary receipts). However, it is interesting to note that a few GTS records remain unique, augmenting global coverage during 1998–1999. This highlights the importance of including all available data sources and using robust duplicate elimination procedures and rules to arrive at the ‘best’ possible set of observations to support a wide range of applications (e.g. both drifting and moored buoy SSTs are critical for calibrating and validating infrared and microwave satellite retrievals).

thumbnail image

Figure 6. Annual distribution (1978–2007) of the DM and GTS drifting buoy data mixture in R2.4 (top; note: 2007 not shown because R2.4 ended in May) and R2.5 (bottom). The curve in the top panel contrasts the resultant R2.5 distribution. DM deck numbers: ISDM (714); data (1980–1996) from drifters with deep drogues collected by the Institut für Meereskunde, U. of Kiel, Germany (715). GTS deck numbers: NCEP (793–794)

Download figure to PowerPoint

In addition to ship and buoys, ICOADS encompasses data from a variety of other in situ observing systems (Figure 7). Until the middle of 1970s, ship-recorded data (mostly VOS plus some R/Vs) dominate. Since then, the contribution from buoys and other automated platforms has rapidly increased, while ship records begin decreasing in 1990 (Kent et al., 2006). Superficially this appears satisfactory, but sampling characteristics of these different systems reveal important concerns. VOS equipped with observers and instruments typically report measurements for many different variables, and mostly sample along the world's major shipping routes. Meteorological and oceanographic R/Vs explore a wide range of ocean areas, but generally do not visit the same locations with the regularity of the VOS fleet. Moored buoys sample only a small portion of the global ocean, but do so with high frequency, an increasing compliment of instruments over time, and potentially with good accuracy in the post-recovery calibrated DM archives. Drifting buoys have near ubiquitous coverage, but sample only a few parameters, and, since they are generally not recovered, some measurement sensor drift might go undetected and result in decreased accuracy.

thumbnail image

Figure 7. Annual distribution (1937–2007) of major platform types in R2.5 shown as millions of reports per year. For clarity the vertical scale is truncated at 9M; years 2005–2007 have 13M, 15M, and 16M total reports (not visible) in R2.5, respectively. The red line curve shows the R2.4 annual counts. Ships (mainly VOS plus some R/Vs), buoys and oceanographic are self explanatory, Ocean (permanent) Station Vessel = OSV, Coastal–Marine-Automated Network = C-MAN, ocean drilling rigs/platforms and other small entities = other, and unidentified platform types = missing. (Figure adapted from Woodruff et al. (2008); ship photo courtesy of www.ShipPhotos.co.uk)

Download figure to PowerPoint

Because ICOADS incorporates so many data sources, it is very difficult to address all the issues related to observing system changes and anomalies that indicate potential error. R2.5 has resolved or improved upon many outstanding issues, but some still remain (and some, surely, are as yet undiscovered). The following are several known outstanding major issues (traceable back to individual sources in ICOADS using Tables AII and AIII in the Appendix):

  • (a)
    SLP biases have been previously identified [Mitchell T. in Diaz et al. (2002), Ansell et al. (2006)] in the US Maury collection (1784–1863). These are among the earliest available SLP data, and thus are crucial for estimation of early global fields (e.g. Küttel et al., 2010). Further evaluation and testing of schemes to correct the problem are needed, also prior to future blending of additional German Maury (1845–1867) data (Wilkinson et al., 2010) possibly impacted by similar biases.
  • (b)
    Data from oceanographic profiling instruments are an important supplement to the near-surface data. However, data from each instrument type have different characteristics that can create time- and space-dependent biases. For R2.5, near-surface profile temperatures were selected from the depth closest to 4 m and ⩽10 m. In previous Releases, the scheme started at the shallowest depth in a profile and used the first temperature value at any depth ⩽3 m. Neither approach is ideal. Better schemes, which could be instrument type dependent, are needed. This is especially important to take greatest advantage of shallow depth measurements from the ARGO array.
  • (c)
    On the basis of comparisons with duplicate reports from other sources, some Shipboard Environmental Acquisition System (SEAS) data included in R2.5 for 2000–2004 appear to be erroneous (e.g. wind speeds and temperature elements). Details of the problem remain under investigation and any erroneous data will need to be replaced in the future.
  • (d)
    International Maritime Meteorological Tape (IMMT) format data from the GCCs were assumed by ICOADS processing to be ship data, with platform type set accordingly, but many recent data were subsequently found to come from other platform types. A separate ‘Observation Platform’ field shows the magnitude of this incorrect assumption (Figure 8). The problem is greatest from the late 1990s through 2007 when some buoys and other automated ocean stations have been designated as ships.
  • (e)
    The three historical archives of NDBC moored buoy and C-MAN data within NOAA (NDBC, NCDC, and NODC) need to be thoroughly intercompared and reconciled. There are gaps, and possibly errors, among all three archives, stemming from independent maintenance at the three centres for more than 30 years.
  • (f)
    Data from the new tropical Indian Ocean Research Moored Array for African–Asian–Australian Monsoon Analysis and Prediction (RAMA; McPhaden et al., 2009) have only been integrated into ICOADS to a limited extent, and future Releases will strive to incorporate a more complete collection.
  • (g)
    Some undetected duplicates exist in ICOADS due to spatial mislocations or other differences in reported time/space location. We do not currently have a good estimate on the frequency of embedded duplications (nor of the inverse problem of removal of reports erroneously identified as duplicates). This difficult problem could be better addressed with the application of track-checking software including parameter value checks against climatologies and accurate land–sea masks, when resources allow.
  • (h)
    The association of metadata with some recent VOS reports is being obstructed by GTS (and to a more limited extent DM) callsign masking for commercial and security reasons. Solutions meeting the requirements of ship operators, NMHSs, and climate users are urgently being sought (Woodruff et al., 2009). Meanwhile, however, one impact has been the complete masking since December 2007 of all ship reports in the NCEP data used as the primary GTS input to ICOADS (since NCEP does not have the mandate to implement selective unmasking based on confidential time-varying lists).
  • (i)
    Recent GTS data also contain other important problems or data omissions, which should be addressed through reprocessing as resources permit, including the following:
    • A special total cloudiness code (N = 9, indicating ‘Sky obscured by fog and/or other meteorological phenomena’) is important for cloud analyses (e.g. Hahn et al., 1995, 1999), but was omitted from NCEP GTS data for approximately October 1999–7 May, 2002, due to the lack of any recognised representation until then in WMO's Binary Universal Form for the Representation (BUFR) of meteorological data.

    • RH was extracted from the NCEP GTS buoy code only starting with 2005 data, and some untapped RH observations may also be available from the GTS ship code.

    • Useful buoy platform and instrumental metadata, including engineering and technical parameters (e.g. buoy type) and QC information, may be available in the GTS buoy code.

    • Voluminous meteorological observations associated with US and international Tide Gauge stations remain untapped in GTS datastreams since approximately 2000.

thumbnail image

Figure 8. Distribution of the Observation Platform field in deck 926 [International Maritime Meteorological (IMM) Data from the GCCs; field available in the current IMMT and IMMA formats]. As discussed in the text, during R2.5 processing a separate platform type field was set for all deck 926 data to indicate ship (PT = 5). The Selected, Supplementary, and Auxiliary VOS categories are defined by WMO (1990)

Download figure to PowerPoint

5. Data and product access

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. The national and international contributing community
  5. 3. Release 2.5
  6. 4. Data characteristics and unresolved issues
  7. 5. Data and product access
  8. 6. Community data products derived from ICOADS observations
  9. 7. Future plans
  10. 8. Conclusions
  11. Acknowledgements
  12. References

ICOADS data and products are maintained by the founding US partners with free and open access worldwide, and no restrictions on redistribution. There are a variety of access schemes and data formats available through the project website (http://icoads.noaa.gov/). The observations (e.g. of available SST and air temperatures, humidity, SLP, wind speed and direction, wind wave and swell heights, directions and periods, cloud cover, types and heights, and weather codes; plus reported metadata elements) are available in the IMMA format or as simple ASCII format text files. The monthly summary statistical products are available in binary, netCDF, and ASCII text files. Parameter, temporal, and spatial subsetting are available for the observations and statistics that result in ASCII-formatted files. Subsetting and plotting also augment the service of netCDF files. These services address the requests of over 400 distinct users each year.

The data processing performed to create new ICOADS Releases is largely driven by availability of additional DM data. To support research that extends beyond the formal end date of a new Release (e.g. R2.5's 2007 end date) basic ICOADS format conversion and data processing is applied to the GTS datastream received from NCEP. This processing is carried out on a monthly basis and provides users with a near-real-time ICOADS option for the observational data (IMMA format). Routine production and updating of the monthly summary statistical products is also planned once sufficient resources can be located. These updates are integrated into the services described above.

6. Community data products derived from ICOADS observations

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. The national and international contributing community
  5. 3. Release 2.5
  6. 4. Data characteristics and unresolved issues
  7. 5. Data and product access
  8. 6. Community data products derived from ICOADS observations
  9. 7. Future plans
  10. 8. Conclusions
  11. Acknowledgements
  12. References

ICOADS forms the basis for many derived data products, and is regarded worldwide as the long-term marine surface reference dataset. There is a wide product range that includes the simple statistical summaries of the observations, gridded analyses, bias-adjusted analyses, multi-source blended analyses where ICOADS is one component, and assimilation input for global atmospheric reanalyses.

The ICOADS' monthly summary statistics are produced for 2° latitude × 2° longitude (since 1800) and 1° × 1° boxes (since 1960). Eight ‘observed’ variables (SST and air temperature, winds, SLP, total cloudiness, and RH; after the climatological outlier trimming) and 14 derived variables are summarised with a set of 10 statistics, e.g. mean, median, and number of observations (Worley et al., 2005). No attempt is made in these products to account for observing system changes and measurement biases, but the many variables are treated consistently and the observation types grouped in different ways and with different levels of outlier trimming applied to provide a good first-order environmental assessment.

One major QC change was made in the processing of RH for R2.5. For a data value to be used in the monthly summaries, it must pass a check against climatological ‘trimming’ (QC) limits (Slutz et al., 1985). However, the available 1854–1909 trimming limits are essentially missing (Figure 9), hence RH during this period could not be checked and monthly summaries of humidity variables (all dependent on RH) were not calculated. To mitigate this problem, the climatological trimming limits for 1910–1949 were used for all RH data prior to 1910. R2.5 therefore, for the first time, offers humidity fields extending back into the nineteenth century (Figure 3). Other variables are impacted to a lesser degree, as illustrated by the presently available January long-term trimming median values for SST, SLP, and RH, for the three consecutive trimming periods, 1854–1909, 1910–1949, and 1950–1979 (Figure 9). These data voids in the trimming limits exist because insufficient ICOADS data were available over those regions when they were constructed by 1985, and project resources have not been adequate yet to upgrade those original limits, e.g. making use of newly rescued data in those regions.

thumbnail image

Figure 9. January smoothed long-term median values for relative humidity (RH; %), sea level pressure (SLP; hPa), and sea surface temperature (SST; °C). The values are plotted separately for the three Release 1 (Slutz et al., 1985) trimming periods (1854–1909, 1910–1949, 1950–1979) from top to bottom (note: limits from the closest period are currently used for trimming data prior to 1854 or later than 1979). As discussed in the text, the general absence of RH trimming information for the earliest period was resolved for R2.5 processing by utilization of the trimming limits for the following (1910–1949) period for data prior to 1910

Download figure to PowerPoint

International research teams have taken special interest in specific variables and continue to improve upon the basic ICOADS statistics. There have been many significant works published and data made available to the public. We highlight just a few as a representative set. The visual estimated wave data have been processed and summarised for the global ocean (Gulev and Grigorieva, 2006). Similarly, a widely used climate research cloud data product, the Extended Edited Synoptic Cloud Reports from Ships and Land Stations Over the Globe (EECRA) (Hahn et al., 1995, 1999), has been created.

NOAA incorporates ICOADS as a component of several global gridded SST products (Woodruff et al., 2008): Extended Reconstructed Sea Surface Temperature (ERSST) 1854–present (Smith et al., 2008), Weekly Optimum Interpolation SST 1982–present (Reynolds et al., 2002), and Daily Optimum Interpolation SST 1985–present (Reynolds et al., 2007). Complementary work using ICOADS is done at the Met Office Hadley Centre to create the Hadley Centre Sea Ice and SST (HadISST) 1870–present (Rayner et al., 2003) and associated products (Rayner et al., 2006), the ocean portion of SLP in HadSLP 1850–2004 (Allan and Ansell, 2006), and other global temperature assessments.

Fluxes at the ocean–atmosphere interface are critically important for understanding the earth system. Although modern instrumental systems are relatively accurate for measuring fluxes, in situ estimates for the long term and with wide area coverage are only possible from VOS that record a full compliment of sea surface conditions, i.e. SST, air temperature, wind, humidity, cloud, and weather information. In these circumstances, instrument type and placement are critical metadata that can lead to improved accuracy. Many global ocean flux datasets have been derived from ICOADS in the past two decades (da Silva et al., 1994; Josey et al., 1999; Smith et al., 2004; Bourassa et al., 2005; Berry and Kent, 2009).

During the past decade, a significant portion of our scientific climate change understanding has come from projects that have produced global atmospheric reanalyses. The models used in these projects assimilate vast quantities of in situ and remotely sensed data. Invariably ICOADS has been a significant part of the reference dataset for the ocean surface. NOAA, the European Centre for Medium-Range Weather Forecasts (ECMWF), the Japan Meteorological Agency (JMA), and the US National Aeronautics and Space Administration (NASA) have all taken advantage of ICOADS for their reanalyses. Communication between reanalysis centres and the ICOADS developers is excellent. Each new reanalysis is based on the most recent ICOADS update, thereby taking advantage of any improvements in data quality or quantity based on the efforts of marine data experts. ICOADS therefore reduces efforts required for data preparation and quality control for reanalysis projects. In a complimentary manner, reanalyses efforts (e.g. Compo et al., 2006) also uncover data problems in ICOADS that feedback to the developers and lead to future improvements.

Overall, as ICOADS has been regularly sustained as an international collection of global ocean observed data and widely utilised in all of these key applications, it has had significant influence on a wide spectrum of research and understanding of the earth system.

7. Future plans

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. The national and international contributing community
  5. 3. Release 2.5
  6. 4. Data characteristics and unresolved issues
  7. 5. Data and product access
  8. 6. Community data products derived from ICOADS observations
  9. 7. Future plans
  10. 8. Conclusions
  11. Acknowledgements
  12. References

Many new historical ship datasets exist in various stages of preparation and should be added to ICOADS as soon as feasible in order to continue to support the wide range of community data products as well as critical downstream applications (e.g. international climate change assessments). The datasets that are most ready for addition are in digital form and require format translation and quality checks to prepare them for the data processing steps that merge new sources into the archive. Other datasets exist only as document images and are yet to be fully digitised. Even further removed from being available, there are many collections in printed (or published) form that if imaged and then digitised would improve the historical coverage. The ICOADS project, together with programs including RECLAIM, ACRE, and CDMP, helps discover and track these sources, and when possible arranges for imaging, digitisation, and ultimately contribution to ICOADS (see Wilkinson et al., 2010).

We also seek to improve the representation of modern ship and buoy data in ICOADS. The meteorological and oceanographic R/Vs are under-represented in the archive. Recent work at COAPS and through the Shipboard Automated Meteorological and Oceanographic System (SAMOS) initiative is improving upon this situation and has added data from the World Ocean Circulation Experiment (1990–1998) into R2.5 (Figure 1). Expansion and support for this effort would be an important asset for ICOADS, and procedures are being developed to make this a routine part of SAMOS data archival. We anticipate that additional R/V observations (from 2005 to present) will be available for inclusion in a planned 2011 ICOADS update. The inclusion of meteorological observations from the WOD has also been helpful in this regard.

For DM moored buoy data and metadata, ICOADS could benefit significantly from more tightly coordinated international data management. Currently the DM data are heavily fragmented and lack a uniform format and metadata standards. In contrast, many drifting buoy data are well managed internationally through ISDM in their role as the Regional Oceanographic Data Center/Drifting Buoys.

DM data from VOS and from the VOS Climate (VOSClim) project are critical components for ICOADS. A planned modernisation of the DM VOS system under JCOMM (Woodruff et al., 2009) could lead to multiple benefits: increasing the quantity of full meteorological reports from ships, providing a secure environment for the handling of time-sensitive information such as ship identifiers, creating more accurate marine reference data and products, and potentially even better observational data as new QC schemes are initiated.

The longstanding need for general QC improvements for ICOADS (Wolter, 1997), coupled potentially with improved standardisation in conjunction with JCOMM data management changes (Woodruff et al., 2009), remains an important and under-resourced problem, as the results of QC can have broad impacts on the data and products provided for research. As discussed in Section 6, globally complete climatological trimming fields could be highly beneficial, facilitating the blending and prompt utilisation of newly recovered data from remote regions. ICOADS has experimented with modernised ‘adaptive’ QC of SST, based on advanced techniques described by Smith and Reynolds (2003). But these techniques proved complex to implement operationally, and for other variables appeared likely to require substantial retooling.

Data collected over multiple centuries use many different observation methods and a variety of modern observing systems. These changes will reduce data assessment accuracy unless the observing system heterogeneities can be addressed. Metadata about ships, buoys, instrument locations, sampling methods, and observing practices and codes are necessary to begin making bias adjustments, corrections, and error estimates. ICOADS has traditionally captured and carried the basic metadata that is attached to each reported record. However, the amount of metadata available this way is highly dependent on the observing system. We have improved upon this basic set by including selected VOS metadata (WMO, 1955; Kent et al., 2007), but much more work is needed. Improvement and expansion of the ship metadata would be helpful and, critically, methods and formats need to be devised (operationally under JCOMM) so a time history of buoy metadata can be archived and uniquely linked to the individual records.

Reanalysis (and operational Numerical Weather Prediction) data assimilation models are another source of untapped metadata. For example, differences between the observed value and the final model analysis can serve as a quality check on the observed data. Random and systematic outliers often stand out against the model field. Metadata of this type can serve multiple purposes: to evaluate and possibly correct the archive, e.g. as systematic errors can sometimes easily be accounted for by mispositioned records; provided to users as additional QC feedback information that can be factored into their studies; and likewise so that subsequent reanalyses can take advantage of previous reanalyses.

Adding data from the nineteenth century and before to ICOADS is also presenting new challenges in metadata handling. There is often considerable uncertainty about how these early data should be processed and converted to the standard IMMA format. As a result, it is crucial to preserve the original data along with the IMMA reports and to maintain records of how conversions were done. This is likely to require refinements to the IMMA data format as well as the technology for storing the observations.

The international community has had organised discussions about a program that would create an ‘advanced’ version of ICOADS. Community experts have done significant work on specific variables and time periods to enhance homogeneity across observing systems, estimate the uncertainty of observations, and improve QC (e.g. track checking). These activities typically result in analysed (gridded) datasets. The proposed idea is to make the underlying observations used in these improved datasets readily available to all ICOADS users. Although plans are in an early stage, it is envisaged that a group of active researchers with experience using ICOADS would assume responsibility for making data adjustments, metadata, and other derived information available alongside the original data fields, which will remain unaltered. Proposed contributions would be vetted by a coordination group, and a unified interface would inform the users about the latest updates, and provide flexible data access. The proposed system would be dynamic and would evolve as recommended adjustments are evaluated, reviewed, and refined. These observations could support a new broad set of statistical or analysed summary products, but there are many details to be fleshed out and suitable funding across the international groups needs to be identified.

More can also be done to aid users in accessing the data. Rapidly served queries can easily be supported by the combination of a web interface and connected databases holding all the ICOADS data. NCDC plans to initiate such a system in the future. Also, interoperability can be supported by OPeNDAP and THREDDS servers; technologies made available by Unidata (http://www.unidata.ucar.edu/). Another option includes the GrADS Data Server (GDS, http://www.iges.org/grads/gds/). Currently neither option is offered for ICOADS from the US partners both because funding support is not available and because computer security concerns need to be addressed. To better facilitate international availability of ICOADS data and products, links between these technologies and the new WMO Information System (WIS) and WMO Integrated Global Observing Systems (WIGOS; http://www.wmo.int/pages/prog/www/wigos/) are also planned.

8. Conclusions

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. The national and international contributing community
  5. 3. Release 2.5
  6. 4. Data characteristics and unresolved issues
  7. 5. Data and product access
  8. 6. Community data products derived from ICOADS observations
  9. 7. Future plans
  10. 8. Conclusions
  11. Acknowledgements
  12. References

ICOADS continues to be significantly improved through periodic Releases, including contributions from a supportive and growing international community, and new linkages developing with JCOMM and WMO operational programs such as WIGOS. The user community is substantially larger, and greatly benefits from this freely available sustained data collection integrated together from many heterogeneous, in situ observing systems. Support for ICOADS at the current set of contributing institutions and laboratories should be proudly recognised and additional support should be added if possible. This is an open community and new participants are welcomed.

As part of this involvement, we hope that nations and organisations taking and holding marine surface observations can continue to examine the pathways by which their information is shared. As far as possible, all such observations (historical and contemporary) should be added to ICOADS. Similarly, research projects and research vessel operators should take account of the marine surface data they collect and ensure there is a data pathway for it to be included in ICOADS. These high-quality and possibly high-resolution data can have the effect of increasing the accuracy of large-scale analyses.

The importance of ocean observing system metadata cannot be overemphasised. We are at the beginning of, hopefully, a long improving trend to use metadata to better understand the ocean measurements and to reduce the uncertainty in environmental assessments. Legacy metadata need to be collected, systematically organised (formatted), and eventually made available with the ICOADS reported observations.

This is also true for metadata for today's operational systems. Metadata for all types of buoys and automated stations are currently not easily included or linked with large global collections like ICOADS. The existing VOS platform and instrumental metadata (WMO, 1955) only become available in DM and do not cover as many vessels as would be optimal, but represent a substantial resource. JCOMM and the international community should continue to take action to improve collection and digital distribution of observing system metadata for all platform types.

The CLIMAR and MARCDAT meeting series are invaluable in sustaining the international community around ICOADS and guiding its future. These meeting series should continue. Increased investment should also be made in the ICOADS US core team. There are numerous readily available benefits: contributions from international researchers could be capitalised to support all users, data preparation for insertion into ICOADS could be accelerated, additional data-access tools could be deployed to further expand easy data use in the growing community doing earth system research, and more frequent Release updates and near-real-time archive extensions would bring improvements to the users more rapidly.

Appendix

Past release history and detailed R2.5 data composition

Table AI summarises the ICOADS Releases, data temporal coverage, issuance dates, and other information.

Table AI. ICOADS Release history [update of Table I in Worley et al. (2005)]. For each Release, any references, the total temporal coverage, any updates and extensions, and the composition (for Release 2.0 forward, only), in terms of the DM and real-time (RT; GTS-only) archives, are listed
Release nameReferencesResultant period (issuance year)Updates and extensions (issuance year)
Release 1Slutz et al. (1985), Woodruff et al. (1987)1854–1979 (1985)1980–1991 (1987–1992)a
Release 1aWoodruff et al. (1993)1980–1992 (1993)1992–1993 (1995)b
   1990–1995 (1997)
   1980–1997 (1999)
Release 1b 1950–1979 (1996)1970c (1999)
Release 1c 1784–1949 (2001) 
Release nameReferencesResultant period (issuance year)Main update scope (other amendments)dDM archiveRT archive
  • a

    Following Release 1 (Slutz et al., 1985; Woodruff et al., 1987), ‘interim’ products, constructed using simplified procedures and preliminary input data, were first issued in 1987 (covering 1980–1986) and then extended, on an approximately annual basis, to finally cover 1980–1991.

  • b

    Woodruff et al. (1993).

  • c

    Minor corrections for October–November 1970.

  • d

    Main update scope lists the temporal range of reprocessed or extended data, while ‘other amendments’ refers to the temporal range impacted only by less significant changes (such as the addition of QC or metadata).

  • e

    New Release nomenclature adopted for the combination of Releases 1c, 1b, and 1a (no new data or products).

  • f

    March–December 1997 observational data were also processed.

  • g

    The official R2.4 period was extended with preliminary observational data (only) through 7/2007 (by 6/2008), through 12/2007 (by 12/2008), and through 12/2008 (by 2/2009).

  • h

    As discussed in the text, the preliminary observational data (only) are now updated monthly.

Release 2.0Woodruff et al. (1998, 2003)1784–1997 (2002)(none)e1784–1997 
Release 2.1Worley et al. (2005), Woodruff et al. (2005)1784–2002 (2003)1998–2002 (1784–1997)1784–19971998–2002f
Release 2.2 1784–2004 (2005)1998–2004 (1784–1997)1784–2004 
Release 2.3 1784–2005 (2006)2005 (1999–2004)1784–20042005
Release 2.4 1784–5/2007 (2007)g1998–5/2007 (1784–1997)1784–20042005–5/2007
Release 2.5Worley et al. (2009)(and this paper)1662–2007 (2009)1662–20071662–2007(Ongoing)h

Table AII documents the complete R2.5 composition in terms of the ‘decks’ (originally referring to punched card decks) included. Qualitatively, the impact of each deck can be assessed by its period of record and number of records included in ICOADS. Similarly, Table AIII provides a breakdown of the R2.5 data mixture by source identification (SID), which either provides cross-cutting information within a given deck or identifies broader collections of decks. These fields are critical for data mixture assessment and problem analysis by the ICOADS project staff; and they can be useful for data users, in addition to more general categorisations, such as provided by the platform type field (e.g. ship, buoy).

Table AII.. R2.5 deck composition [update of Table II in Worley et al. (2005)]. For each deck number, the description, starting and ending years, and number of reports (in thousands) after final blending are listed. Decks entirely new to (or replaced in) R2.5 are numbered in bold. Further details about individual decks are available on the ICOADS website
DeckDescriptionStartEndRpts K
  • a

    Deck 145 contains daily-averaged data, and up to the early 1990s TAO deck 144 contains average estimates for 2–8 h depending on the buoy instrument package and power requirements.

  • b

    Except new decks 245–267 within this range are described individually (including their temporal range and report counts, not included here) below. Deck 215 within this range is believed to be derived from the same original German punched cards as deck 192. Some of the other decks within this range were entirely new to ICOADS for the 1950–1979 period (but are not numbered in bold).

  • c

    A mixture of US- and foreign-keyed data exists in deck 927 prior to 1980; starting about 1980 deck 927 is believed to contain only US-keyed ships.

110US Navy Marine19451951633
116US Merchant Marine194519636 866
117US Navy Hourlies1952196411
118Japanese Ships No. 1 (Kobe Collection Data keyed in 1961)193019531 727
119Japanese Ships No. 2 (Kobe Collection Data keyed in 1961)19511961904
128International Marine (US- or foreign-keyed ship data)1950197814 537
143Pacific Marine Environmental Laboratory (PMEL) Buoys1976197713
144TAO/TRITON and PIRATA Buoys (from PMEL and JAMSTEC)a198520047 192
145PMEL (Daily) Equatorial Moorings and Island Stationsa1979199117
150Pacific (US Responsibility) HSST Netherlands Receipts1939196185
151Pacific (US Responsibility) HSST German Receipts18621960206
152Pacific (US Responsibility) HSST UK Receipts1855196115
155Indian (Netherlands Responsibility) HSST186119601 068
156Atlantic (German Responsibility) HSST185219615 564
184Great Britain Marine (194 extension)19531961344
185USSR Marine IGY19571958111
186USSR Ice Stations1950197020
187Japanese Whaling Fleet1946195610
188Norwegian Antarctic Whaling Factory Ships193219392
189Netherlands Marine19391959232
192Deutsche Seewarte Marine185519395 944
193Netherlands Marine180019386 276
194Great Britain Marine18561955457
195US Navy Ships Logs19411946598
196Deutsche Seewarte Marine (192 extension)19491954143
197Danish (and Other) Marine (Polar)1871195623
201–255UK Met. Office (MetO) Main Marine Data Bank (MDB)b1854199415 212
245Royal Navy Ship's Logs (keyed by 2007)193619551 423
246Antarctic Expeditions: Print./Published (held at Met. Office)1898194035
247Atmospheric Circ. Reconstructions over the Earth (ACRE) Data1872187616
281US Navy Monthly Aerological Record (MAR)19261945187
555US Navy Fleet Num. Met. and Oceano. Center (FNMOC; Monterey) Telecom.196619732 213
666Tuna Boats1970197517
667Inter-American Tropical Tuna Commission (IATTC)197119971 148
700UK Met. Office VOSClim GTS BUFR Data2003200710
701US Maury Collection178418631 346
702Norwegian Logbook Collection18671889201
704US Marine Meteorological Journals Collection (1878–1894)187818941 761
705US Merchant Marine Collection (1912–1946) (500 series)191019461 014
706US Merchant Marine Collection (1912–1946) (600 series)191019442 062
707US Merchant Marine Collection (1912–1946) (700 series)19131941425
714Canadian Integrated Science Data Mgmt. (ISDM; formerly MEDS) Buoys1978200757 274
715German Deep Drifter Data (via ISDM; originally from IfM/Univ. Kiel)198019961 031
720Deutscher Wetterdienst (DWD) Marine Met. Archive18761914976
730Climatological Database for the World's Oceans (CLIWOC)16621855261
731Russian S.O. Makarov Collection180418913
732Russian Marine Met. Data Set (MORMET) (rec'd at NCAR)188819957 873
733Russian AARI North Pole (NP) Stations1937199198
734Arctic Drift Stations1893192412
735Russian Research Vessel (R/V) Digitisation193620001 789
736Byrd Antarctic Expedition (keyed by Hollings Scholars)192919341
740Research Vessel (R/V) Data Quality-Evaluated by FSU/COAPS1990199856
749First GARP Global Experiment (FGGE) Level IIb197819796
761Japanese Whaling Ship Data (CDMP/MIT digitisation)1946198420
762Japanese Kobe Collection Data (keyed after decks 118–119)188919403 135
780NODC/OCL World Ocean Database (WOD) (and formerly Atlas, WOA)177220057 738
792US Natl. Cntrs. for Environ. Pred. (NCEP) BUFR GTS: Ship Data199820075 889
793NCEP BUFR GTS: Buoy Data (transmitted in FM 13 ‘SHIP’ code)1998200710 545
794NCEP BUFR GTS: Buoy Data (transmitted in FM 18 ‘BUOY’ code)199820071 950
795NCEP BUFR GTS: Coastal-Marine Automated Network (C-MAN code) Data200520074 056
849First GARP Global Experiment (FGGE)19781979250
850German FGGE19781979146
874Shipboard Environmental (Data) Acquisition System (SEAS)19912007504
876–882US National Data Buoy Center (NDBC) Data19721979315
883US National Data Buoy Center (NDBC) Data1980200420 538
888US Air Force Global Weather Central (GWC)197319975 993
889Autodin (US Dept. of Defense Automated Digital Network)197219951 039
892US Natl. Cntrs. for Environ. Pred. (NCEP) Ship Data198019979 209
893NCEP Moored Buoy Data198619972 225
896NCEP Miscellaneous (OSV, plat, and rig) Data19801997575
897Eltanin196219631
898Japanese19541974121
899South African Whaling1900195564
900Australian19311979386
901FOSDIC Reconstructions (card images from 16 mm film)186819637
902Great Britain Marine (184 extension)1957196199
926International Maritime Meteorological (IMM) Data1954200725 372
927International Marine (US- or foreign-keyed ship data)c1970200711 138
928Same as 927 including Ocean Station Vessels (OSV)197019744
999US Air Force Environ. Technical Applications Center (ETAC)1967196937
Table AIII.. R2.5 source ID (SID) composition [update of Table III in Worley et al. (2005)]. For each SID number, the description, starting and ending years, and number of reports (in thousands) after final blending are listed. SIDs entirely new to R2.5 are numbered in bold. Generally data within a SID are confined to a single input format, whereas some decks have been received by ICOADS in different formats. Further details about individual SIDs are available on the ICOADS website
SIDDescriptionStartEndRpts K
1Atlas1800196932 713
2HSST Pacific18551961405
3HSST Indian186119601 068
4HSST Atlantic185219615 564
5Old TDF-11 Supplement B185419752 694
6Old TDF-11 Supplement C185519782 625
7Monterey Telecommunications19661969661
8Ocean Station Vessels (OSV)19451973822
9OSV Supplement1947197357
10MSQ 486 and 105 Omissions18541968172
13Eltanin196219631
14Japanese19541974121
15South African Whaling1900195564
16Australian19311970192
17International Maritime Meteorological (IMM) Data19561979224
18'70s Decade1970197912 183
19IMM '70s19781979< 1
20OSV Z ('70s)197019741
21Australian ('70s)19711979194
22NCDC: 1980–1984 Annual Receipts19821987135
23'70s Mislocated Data197319792
24Buoy Data19721979192
25–28NCDC: 1980–1985 Annual Receipts196219851 534
29NCDC: US Nat. Met. Center (NMC, now NCEP) Reconversion (1980–1992)198019928 201
30NCDC: 1980–1984 Period of Record196519844 192
32–33NCDC: Annual Receipts (and duplicates; starting in 1986)197419974 440
34–45NCDC: 1986–1997 Receipts (delayed)196919961 251
46–47International Maritime Met. (IMM) Tape Archive (1982-)196919957 117
50US National Data Buoy Center (NDBC) Data1980199712 770
51–52Russian AARI North Pole (NP) Stations1937199198
53First GARP Global Experiment (FGGE) Level IIb: Surface Marine Data197819796
56Russian S.O. Makarov Collection180418913
57Russian Marine Meteorological Data Set (MORMET) (rec'd at NCAR)188819937 873
59UK IMM Corrections (1982–1989)198219891 552
60French International Maritime Met. (IMM) Corrected19541988159
63Canadian ISDM (formerly MEDS) Buoys (July 2005 archive extended Dec. 2008)1978200757 274
64Russian Research Vessel (R/V) Digitisation: Marine Surface193620001 153
65Russian Research Vessel (R/V) Digitisation: Marine Actinometric19472000637
66Pacific Marine Environmental Lab. (PMEL) TOGA/TAO Buoys19851992236
67PMEL (Daily) Equatorial Moorings and Island Stations1979199117
68Arctic Drift Stations1893192412
69US Maury Collection178418631 346
70Inter-American Tropical Tuna Comm. (IATTC) Porpoise Obs. Logs19791997736
71IATTC Fishing Logs19711997413
72IMM Tape Archive from WMO Global Collecting Centre (GCC) (1994 format)198219973 808
77NCDC: US National Cntrs. for Environ. Pred. (NCEP) Reconversion (1994–1997)199419972 609
78NCDC: US-keyed Logbook Data Reconversion (TD-9972; keyed during 1996–1997)19871997307
79US Air Force Global Weather Central (GWC): DATSAV2 format198019971 469
84US Merchant Marine Collection (1912–1946): Full QC191019441 927
85US Merchant Marine Collection (1912–1946): Partial QC191019461 246
90UK Met. Ofc. (MetO) Main Marine Data Bank (MDB): Flatfile 1 (no cardimage)185619949 272
91MetO MDB: Flatfile 1A (Flatfile plus cardimage data)185419795 413
92MetO MDB: Flatfile 1B (no Flatfile match; data derived from cardimage)1855197869
93MetO Historical Metforms (1935–1939): Flatfile 1C (data from cardimage)19351939457
95Japanese Kobe Collection Data (IMMT format; 2003 Edition)188919403 135
96Norwegian Logbook Collection18671889201
98US Merchant Marine Collection (1912–1946): Full QC (CLICOM system)19141944328
100NCEP BUFR GTS: Operational Tanks: Converted from Original Message199819992 198
103NCEP BUFR GTS: Dumped Data: Converted from BUFR1999200720 241
110UK Met. Office VOSClim GTS BUFR Data2003200710
111Shipboard Environmental (Data) Acquisition System (SEAS)19912007438
112IMM Tape Archive from WMO GCC (IMMT-2 or IMMT-3 format)198220077 990
113International Marine (US-keyed ship data)19922007533
115Japanese Whaling Ship Data (CDMP digitisation)1946198420
116Japanese Whaling Ship Data (MIT digitisation)19511976< 1
117PMEL TAO/TRITON and PIRATA Research Archive Hourly Average Data199020013 394
118PMEL TAO/TRITON and PIRATA Research Archive 10-Minute Average Data199620042 746
119JAMSTEC TRITON Hourly Average Data19982004595
120PMEL TAO/TRITON and PIRATA Research Archive Hourly Average SLP Data20002004222
121US National Data Buoy Center (NDBC) Data (obtained from NCDC 2005–2007)199820047 768
125US Marine Meteorological Journals Collection187818941 761
126Royal Navy Ship's Logs (keyed by 2007)193619551 423
127Antarctic Expeditions: Print./Published (held at Met Office)1898194035
129Byrd Antarctic Expedition (keyed by Hollings Scholars)192919341
130Research Vessel (R/V) Data Quality-Evaluated by FSU/COAPS: WOCE ver.3.01990199856
133Climatological Database for the World's Oceans (CLIWOC; Release 2.1)16621855261
134Deutscher Wetterdienst (DWD) Marine Meteorological Archive: Compo Subset18841914580
135DWD Marine Meteorological Archive: Newly Digitised Data18761902395
136DWD Marine Meteorological Archive: HISTOR Data18821899< 1
137NODC/OCL 2005 World Ocean Database (WOD05) updated through 13 Dec. 2007177220057 738
138ACRE Data: Challenger Expedition1872187616
139German Deep Drifter Data (via ISDM; originally from IfM/Univ. Kiel)198019961 031

Acknowledgements

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. The national and international contributing community
  5. 3. Release 2.5
  6. 4. Data characteristics and unresolved issues
  7. 5. Data and product access
  8. 6. Community data products derived from ICOADS observations
  9. 7. Future plans
  10. 8. Conclusions
  11. Acknowledgements
  12. References

ICOADS Release 2.5 has benefited from financial support, and contributions of new observations, from a range of sources: We are grateful to the Deutscher Wetterdienst, the NOAA Climate Program Office (CPO), the NOAA Climate Database Modernization Program (CDMP), and the Joint DECC, Defra and MoD Integrated Climate Programme - DECC/Defra (GA01101), MoD (CBC/2B/0417_Annex C5).

References

  1. Top of page
  2. Abstract
  3. 1. Introduction
  4. 2. The national and international contributing community
  5. 3. Release 2.5
  6. 4. Data characteristics and unresolved issues
  7. 5. Data and product access
  8. 6. Community data products derived from ICOADS observations
  9. 7. Future plans
  10. 8. Conclusions
  11. Acknowledgements
  12. References
  • Allan RJ, Ansell TJ. 2006. A new globally-complete monthly historical gridded mean sea level pressure data set (HadSLP2): 1850–2004. Journal of Climate 19: 58165842.
  • Ansell TJ, Jones PD, Allan RJ, Lister D, Parker DE, Brunet M, Moberg A, Jacobeit J, Brohan P, Rayner NA, Aguilar E, Alexandersson H, Barriendos M, Brandsma T, Cox NJ, Della-Marta PM, Drebs A, Founda D, Gerstengarbe F, Hickey K, Jónsson T, Luterbacher J, Nordli O, Oesterle H, Petrakis M, Philipp A, Rodwell MJ, Saladie O, Sigro J, Slonosky V, Srnec L, Swail V, García-Suárez AM, Tuomenvirta H, Wang X, Wanner H, Werner P, Wheeler D, Xoplaki E. 2006. Daily mean sea level pressure reconstructions for the European–North Atlantic region for the period 1850–2003. Journal of Climate 19: 27172742, DOI: 10.1175/JCLI3775.1.
  • Bourlès B, Lumpkin R, McPhaden MJ, Hernandez F, Nobre P, Campos E, Yu L, Planton S, Busalacchi A, Moura AD, Servain J, Trotte J. 2008. The Pirata Program: history, accomplishments, and future directions. Bulletin of the American Meteorological Society 89: 11111125.
  • Berry DI, Kent EC. 2009. A new air-sea interaction gridded dataset from ICOADS with uncertainty estimates. Bulletin of the American Meteorological Society 90: 645656, DOI: 10.1175/2008BAMS2639.1.
  • Bourassa MA, Romero R, Smith SR, O'Brien JJ. 2005. A new FSU wind climatology. Journal of Climate 18: 36923704.
  • Brohan P, Allan R, Freeman JE, Waple AM, Wheeler D, Wilkinson C, Woodruff S. 2008. Marine observations of old weather. Bulletin of the American Meteorological Society 90: 219230.
  • Charpentier E, Harrison DE, Keeley JR, Kent E, Mietus M, Rayner N, Rutherford M, Swail V, Woodruff S. 2008. Third JCOMM Workshop on Advances in Marine Climatology (CLIMAR-III). MeteoWorld, December 2008, WMO: Geneva.
  • Compo CP, Whitaker JS, Sardeshmukh PD. 2006. Feasibility of a 100-year reanalysis using only surface pressure data. Bulletin of the American Meteorological Society 87: 175190.
  • Diaz H, Folland C, Manabe T, Parker D, Reynolds R, Woodruff S. 2002. Workshop on advances in the use of historical marine climate data. WMO Bulletin 51(4): 377380.†.
  • Dupigny-Giroux LA, Ross TF, Elms JD, Truesdell R, Doty SR. 2007. NOAA's climate database modernization program: rescuing, archiving, and digitizing history. Bulletin of the American Meteorological Society 88: 10151017.
  • García-Herrera R, Können GP, Wheeler DA, Prieto MR, Jones PD, Koek FB. 2005. CLIWOC: a climatological database for the World's Oceans 1750–1854. Climatic Change 73: 112.
  • Gould WJ, Smith SR. 2006. Research vessels: underutilized assets for climate observations. EOS Transactions 87: 214215.
  • Gulev SK, Grigorieva V. 2006. Variability of the winter wind waves and swell in the North Atlantic and North Pacific as revealed by the voluntary observing ship data. Journal of Climate 19: 56675785.
  • Hahn CJ, Warren SG. 1999. Extended Edited Synoptic Cloud Reports from Ships and Land Stations Over the Globe, 1952–1996, NDP-026C. Carbon Dioxide Information Analysis Center, Oak Ridge National Laboratory: Oak Ridge. http://cdiac.ornl.gov/ftp/ndp026c/ndp026c.txt.
  • Hahn CJ, Warren SG, London J. 1995. The effect of moonlight on observation of cloud cover at night, and application to cloud climatology. Journal of Climate 8: 14291446.
  • Josey SA, Kent EC, Taylor PK. 1999. New insights into the ocean heat budget closure problem from analysis of the SOC air-sea flux climatology. Journal of Climate 12: 28562880, DOI: 10.1175/1520-0442(1999)012 < 2856:NIITOH > 2.0.CO;2.
  • Kent EC, Berry DI, Woodruff SD, Taylor PK. 2006. Voluntary Observing Ships: a vital observing system in decline. CLIVAR Exchanges 11(3): 2021.
  • Kent EC, Woodruff SD, Berry DI. 2007. WMO Publication No. 47 metadata and an assessment of observation heights in ICOADS. Journal of Atmospheric and Oceanic Technology 24: 214234.
  • Kent E, Woodruff S, Rayner N, Arbetter T, Folland C, Koek F, Parker D, Reynolds R, Saunders R, Smolyanitsky V, Worley S, Yoshida T. 2007. Advances in the use of historical marine climate data (Second International Workshop on Advances in the Use of Historical Marine Climate Data). Bulletin of the American Meteorological Society 88: 559564.
  • Küttel M, Xoplaki E, Gallego D, Luterbacher J, García-Herrera R, Allan R, Barriendos M, Jones PD, Wheeler D, Wanner H. 2010. The importance of ship log data: reconstructing North Atlantic, European and Mediterranean sea level pressure fields back to 1750. Climate Dynamics. DOI: 10.1007/s00382-009-0577-9. (in press).
  • McPhaden MJ, Busalacchi AJ, Cheney R, Donguy J-R, Gage KS, Halpern D, Ming J, Julian P, Meyers G, Mitchum GT, Niiler PP, Picaut J Reynolds RW, Smith N, Takeuchi K. 1998. The Tropical Ocean-Global Atmosphere (TOGA) observing system: a decade of progress. Journal of Geophysical Research 103: 1416914240.
  • McPhaden MJ, Meyers G, Ando K, Masumoto Y, Murty VSN, Ravichandran M, Syamsudin F, Vialard J, Yu L, Yu W. 2009. RAMA: The Research Moored Array for African–Asian–Australian Monsoon Analysis and Prediction. Bulletin of the American Meteorological Society 90: 459480, DOI: 10.1175/2008BAMS2608.1.
  • Parker D, Kent E, Woodruff S, Dehenauw D, Harrison DE, Manabe T, Mietus M, Swail V, Worley S. 2004. Second JCOMM Workshop on Advances in Marine Climatology (CLIMAR-II). WMO Bulletin 53(2): 157159.†.
  • Rayner NA, Brohan P, Parker DE, Folland CK, Kennedy JJ, Vanicek M, Ansell T, Tett SFB. 2006. Improved analyses of changes and uncertainties in sea surface temperature measured in situ since the mid-nineteenth century: the HadSST2 data set. Journal of Climate 19(3): 446469.
  • Rayner NA, Parker DE, Horton EB, Folland CK, Alexander LV, Rowell DP, Kent EC, Kaplan A. 2003. Global analyses of sea surface temperature, sea ice, and night marine air temperature since the late nineteenth century. Journal of Geophysical Research 108(D14): 4407, DOI:10.1029/2002JD002670.
  • Reynolds RW, Rayner NA, Smith TM, Stokes DC, Wang W. 2002. An improved in situ and satellite SST analysis for climate. Journal of Climate 15: 16091625.
  • Reynolds RW, Smith TM, Liu C, Chelton DB, Casey KS, Schlax MG. 2007. Daily high-resolution blended analyses for sea surface temperature. Journal of Climate 20: 54735496.
  • da Silva AM, Young CC, Levitus S. 1994. Atlas of Surface Marine Data 1994: Algorithms and Procedures, vol. 1, NOAA Atlas NESDIS 6, US Government Printing Office: Washington, DC; 83 pp.
  • Slutz RJ, Lubker SJ, Hiscox JD, Woodruff SD, Jenne RL, Joseph DH, Steurer PM, Elms JD. 1985. Comprehensive Ocean-Atmosphere Data Set; Release 1. NOAA Environmental Research Laboratories, Climate Research Program: Boulder; 268 pp. (NTIS PB86-105723).†.
  • Smith TM, Reynolds RW. 2003. Extended reconstruction of global sea surface temperatures based on COADS data (1854–1997). Journal of Climate 16: 14951510.
  • Smith TM, Reynolds RW, Peterson TC, Lawrimore J. 2008. Improvements to NOAA's historical merged land-ocean surface temperature analysis (1880–2006). Journal of Climate 21: 22832296.
  • Smith SR, Servain J, Legler DM, Stricherz JN, Bourassa MA, O'Brien JJ. 2004. In-situ based pseudo-wind stress products for the tropical oceans. Bulletin of the American Meteorological Society 85: 979994.
  • Wilkinson C, Woodruff SD, Brohan P, Claesson S, Freeman JE, Koek F, Lubker SJ, Marzin C, Wheeler D. 2010. RECovery of Logbooks And International Marine Data: The RECLAIM Project. International Journal of Climatology DOI: 10.1002/joc.2102 (in press).
  • Wolter K. 1997. Trimming problems and remedies in COADS. Journal of Climate 10: 19801997.
  • Woodruff SD. 2007. Archival of data other than in IMMT format: The International MaritimeMeteorological Archive (IMMA) Format. Second Session of the JCOMM Expert Team on Marine Climatology (ETMC), Geneva, Switzerland, 26–27 March 2007, JCOMM Meeting Report No. 50, 68101. Available from: http://icoads.noaa.gov/e-doc/imma/.
  • Woodruff SD, Diaz HF, Elms JD, Worley SJ. 1998. COADS release 2 data and metadata enhancements for improvements of marine surface flux fields. Physics and Chemistry of the Earth 23: 517526.†.
  • Woodruff SD, Diaz HF, Worley SJ, Reynolds RW, Lubker SJ. 2005. Early ship observational data and ICOADS. Climatic Change 73: 169194.
  • Woodruff SD, Diaz HF, Kent EC, Reynolds RW, Worley SJ. 2008. The evolving SST record from ICOADS. In Climate Variability and Extremes during the Past 100 Years, vol. 33, Advances in Global Change Research, BrönnimannS, LuterbacherJ, EwenT, DiazHF, StolarskiRS, NeuU (eds.) Springer: Netherlands; 6583.
  • Woodruff SD, Lubker SJ, Wolter K, Worley SJ, Elms JD. 1993. Comprehensive Ocean—Atmosphere Data Set (COADS) Release 1a: 1980–92. Earth System Monitor 4(1): 18.†.
  • Woodruff SD, Scott N, Berry DI, Bourassa MA, Charpentier E, Gulev S, Haar H, Kent EC, Reynolds RW, Rosenhagen G, Rutherford M, Swail V, Worley SJ, Zhang H-M, Zöllner R. 2009. Surface in situ datasets for marine climatological applications. Draft Community White Paper, OceanObs'09, Venice, Italy, 21–29 September 2009.
  • Woodruff SD, Slutz RJ, Jenne RL, Steurer PM. 1987. A comprehensive ocean-atmosphere data set. Bulletin of the American Meteorological Society 68: 12391250.
  • Woodruff SD, Worley SJ, Arnott JA, Diaz HF, Elms JD, Jackson M, Lubker SJ, Parker DE. 2003. COADS updates and the blend with the UK Met Office Marine Data Bank. In Advances in the Applications of Marine Climatology—The Dynamic Part of the WMO Guide to the Applications of Marine Meteorology. WMO/TD-No. 1081 (JCOMM Technical Report No. 13), WMO: Geneva; 310 (CD-ROM).†.
  • WMO. 1955. International List of Selected, Supplementary and Auxiliary Ships. WMO-No. 47. WMO: Geneva. (Serial publication; recently annual. Editions prior to 1966 were entitled International List of Selected and Supplementary Ships).
  • WMO. 1990. Manual on Marine Meteorological Services. vol. I, WMO-No. 558, Global Aspects, World Meteorological Organization: Geneva.
  • Worley SJ, Woodruff SD, Lubker SJ, Ji Z, Freeman JE, Kent EC, Brohan P, Berry DI, Smith SR, Wilkinson C, Reynolds RW. 2009. The role of ICOADS in the sustained ocean observing system. Draft Community White Paper, OceanObs'09, Venice, Italy, 21–29 September, 2009.
  • Worley SJ, Woodruff SD, Reynolds RW, Lubker SJ, Lott N. 2005. ICOADS Release 2.1 data and products. International Journal of Climatology 25: 823842, DOI: 10.1002/joc.1166.
  • Note: references followed by † are available online at: http://icoads.noaa.gov/publications.html or from the underlying selected research bibliography