The contribution of Bruce Ingleby was written in the course of his employment at the Met Office, UK, and is published with the permission of the Controller of HMSO and the Queen's Printer for Scotland.
For centuries, mariners have recorded meteorological information as part of the day-to-day information essential to navigation. These early observations of the weather from ships' logbooks now form a unique climate record (Woodruff et al., 2010). For example, logbooks kept by officers from the seventeenth and eighteenth centuries held in the archives of Britain, Holland, France, and Spain were digitised and made widely available in the Climatological Database for the World's Oceans (CLIWOC) project (García-Herrera et al.,2005). These early logbooks typically recorded the vessel's speed, the winds, and weather information such as precipitation, the state of the sea and sky, and thunder and lightning. Navigation was not precise due to the lack of accurate clocks: navigating using the positions of astronomical features (celestial navigation) did not allow dead reckoning. Additionally, many different zero meridians were in use at this time, which complicated the allocation of correct positions to the data. Despite the many difficulties, these observations were collated to produce charts and atlases of the winds, temperatures and currents (Figure 1) which were used by mariners to plan voyages.
The value of this information for efficient and safe navigation was recognised by Lt Matthew Fontaine Maury of the US Navy. Maury also realised that consistency was key to the utility of the data recorded. In August/September 1853, he convened a conference in Brussels for the purpose of establishing a uniform system of meteorological observations at sea, and of concurring in a general plan of observation on the winds and currents of the ocean. This was the first international meteorological conference and led to the establishment, 20 years later, of the International Meteorological Organization, the predecessor of the World Meteorological Organization (WMO). The Brussels conference established the procedures for the marine observing system: for example, the Beaufort scale was adopted as the standard for the consistent reporting of wind force based on ship speed and sails carried. Another important development was the adoption of a standard logbook format for recording the observations.
Early reports comprise descriptions of weather, prevailing winds and navigational information. These are first supplemented by instrumental observations in the late eighteenth century. As the quality and suitability of instruments for marine use improved, increasing numbers of quantitative instrumental observations were made and from about 1850 there are enough observations of sea-surface temperature (SST), air temperature and surface pressure to produce gridded datasets for the global ocean. Examples of long-term datasets include those from the Met Office Hadley Centre (Allan and Ansell, 2006; Rayner et al.,2003; 2006; Met Office, 2010a) and the US National Oceanic and Atmospheric Administration (Reynolds et al.,2002; NOAA, 2010). To enable such datasets to be used for studies of climate variability and change, it is important to understand the evolving instrumentation and observing practices that may influence the observations.
Measurements from ships
The routine collection of meteorological observations from ships is now jointly co-ordinated by the WMO and the Intergovernmental Oceanographic Commission through the Voluntary Observing Ships (VOS) Scheme (VOS, 2010). Ships from many different countries of registration are recruited by Met. Services into their national VOS Fleet based on the willingness of the ships' officers to perform the observations, the typical routes followed by the ship, and the regularity of port visits. Most VOS are issued with instruments by the recruiting Met. Service although there are different categories of operation and some ships carry more limited instrumentation. The Port Meteorological Officer (PMO) plays a vital role, identifying ships for recruitment, visiting VOS to check instrumentation, calibrating the barometer, supplying consumables such as barograph charts or logbooks, and discussing any observational problems with the Master and officers. Without their enthusiasm and dedication both the quantity and quality of VOS reports declines.
Although individual National Meteorological Services recruit ships to the scheme, there is substantial international cooperation. For example, many of the European Met. Services operate a joint surface marine programme to rationalise observations and achieve economies of scale. More informally, the different Met. Services cooperate by visiting ships recruited by other nations when they visit their local ports. Participation is voluntary: the observers are generally not paid to take weather observations, and the scheme relies on the goodwill of the shipping companies and ships' officers. The benefits to the ship are through contributing to the quality of weather forecasting and ship routing information and the availability of meteorological measurements to aid navigation.
VOS make observations of surface wind speed and direction, SST, air temperature, humidity, atmospheric pressure, cloud (including types) and wave parameters, and significant weather. Of these SST, air temperature, humidity and atmospheric pressure are measured by meteorological instruments, while waves, clouds and significant weather are estimated visually. Wind reports represent a mixture of measurements and visual data. Almost 30% of reports from ships without automatic weather measurement systems still indicate use of visual estimates: however, many of these (with the exception of Dutch recruited ships) have winds that appear too strong for ten metres and are better treated as measured winds (Ingleby, 2010). Preferred methods of measurement change over time and electronic logbooks are increasingly replacing paper versions. Electronic logbook software is used to format manual observations for transmission and storage, calculate derived parameters (e.g. the true wind) and perform simple quality control. The introduction of electronic logbooks has helped to reduce errors in the observations such as those associated with the calculation of the true wind from the measured wind and ship speed and direction (Smith et al.,1999). As marine instruments have become more robust an increasing number of automated weather systems (AWS) are being installed on VOS. However, it is not always easy to find well-exposed locations for the sensors, and high-quality AWS able to make the full range of observations are expensive.
The instruments used by the VOS include: wet and dry bulb thermometers exposed in marine screens or sling psychrometers; a range of different types of anemometer, and barometers and barographs. Methods of measurement have tended to change over time: for example, visual observations of wind parameters are being replaced by measurements from anemometers and bucket measurements of SST by engine intake or hull sensors. Ingleby (2010) found that automated ship reports are generally higher quality than manual ship reports, partly due to fewer errors in data processing and transmission and also because remote reading technology has allowed instruments to be located in better-exposed positions remote from the bridge. The voluntary nature of the ship observing programme means that data quality can sometimes be a problem and these issues have been discussed for over a century. The PMO system is vital in ensuring that ships' officers understand how the observations should be made and that the instruments are properly installed and regularly checked and calibrated.
The need for additional information or metadata to accompany VOS reports on how the observation was made has long been recognised. A list of all VOS and their instrumentation has been available since 1955 using information collected by PMOs and supplied by participating Met. Services. Early editions of the list have been digitised by the US Climate Database Modernization Program and all the ships' particulars and instrumentation information are now available electronically and provide an invaluable resource for climate research. Of particular importance are information on heights of measurement for winds and temperatures, and measurement methods for air temperature, humidity and SST. Only limited metadata describing observing methods can be transmitted in the real-time report, but ideally this can be linked to the extensive range of information available ashore. To associate the report with the correct information, each ship report must include a valid call-sign. Any report transmitted with a blank call-sign, or generic call-sign such as ‘SHIP’, cannot have the full range of measurement methods assigned and any corrections properly applied ashore. Lack of ship identifiers also means that quality assessments and monitoring feedback are not possible and data quality will therefore suffer.
Despite these efforts to ensure data quality, adjustments to the data are required. Observations of pressure, wind speed, air temperature and humidity all need to be referenced to a common height. For pressure, adjustment to sea level is calculated by the observers on the ship (Met Office, 1995). Measurements of wind speed, air temperature and humidity are all reported as observed and adjustment to the required reference level is performed ashore. Modelling of the effects of air flow distortion by the ship on wind measurements is becoming common for research vessels (Moat and Yelland, 2008) and some progress has been made for VOS (Moat et al.,2006). Daytime air temperature observations are affected by solar heating of the surrounding ships infrastructure, and biases are largest for poorly exposed sensors (Berry and Kent, 2005). Methods of measurement can also lead to important differences in observations. There are differences between measurements of SST made using buckets, engine intakes, and hull sensors (Kent et al.,2010b), of wind speed made using anemometers and from visual estimates (Thomas et al.,2008) and of humidity measured in screens or using psychrometers (Berry and Kent, 2009a). Some types of observational error can be modelled and corrections applied to reduce bias: for example, differences between instrument types and adjustments for measurement height. However, some quite large observational errors arise from ship-to-ship differences which are less predictable. Examples might include large changes in ships' loading unaccounted for in adjustments for heights, pressure biases due to wind effects where the barometer is not connected to a static head, and any calibration offsets in the ships instruments. These types of errors are identified through data monitoring ashore and the PMO is responsible for following up any remedial action required. The VOS Climate (VOSClim) project was set up to try to understand and improve the quality of ship meteorological observations. The photographs in Figure 2 were collected as part of VOSClim to help relate the characteristics of the observations to the characteristics of the ship and the instruments.
The coverage of observations from VOS has been in decline over the past 20 years (Kent et al.,2006). This is at least partly due to the increasing reliance of Numerical Weather Prediction (NWP) on observations from satellites and drifting buoys. Observation numbers have now stabilised, but often reports are less complete than in the past. For example, the increasing number of observations from automated systems has led to a decrease in the proportion of reports containing elements which require manual input, such as visual estimates of cloud parameters, sea state, and weather codes. Although the VOS now only comprise one element of the surface marine observing system, their ongoing role in providing marine information is increasingly being recognised. It is clear that VOS data and VOS-based datasets remain widely used: to monitor and understand climate change; for the validation, calibration and analysis of satellite observations of SST, precipitation, wind, cloud, air temperature and humidity; providing information on air-sea interaction and atmospheric stability, and modelling applications including reanalysis, NWP and forcing fields for ocean models. New initiatives to improve data quality and integration with other observing programs should ensure that the VOS remain an important contributor to the Global Climate Observing System in decades to come (Kent et al.,2010a).
Other types of measurements
Ships making reports on an opportunistic basis have made an important contribution to the observing system over centuries, but coverage is inevitably patchy as observations cannot be targeted except through selectively recruiting ships operating in particular regions. There are some places where ships just do not go. Improving technology has allowed the deployment of buoys and installations on platforms which can, at least partially, fill these gaps.
The first moored buoys were deployed operationally in the late 1970s around the coast of North America. The robustness of the buoys and their instruments and the range of parameters they are able to measure reliably have increased over time. The deployment and maintenance of the moored-buoy network is hugely challenging and expensive, especially for moorings in deep water and harsh environments. There is a wide range of types of moored buoy (Figure 3). In exposed regions the buoys used are extremely large: the largest buoys deployed by the US National Data Buoy Center are discus buoys of 12 metres diameter. With the exception of the tropical arrays of moored buoy almost all moorings are deployed fairly close to the coast (Figure 4).
Moored buoys have the potential to provide high-quality observations of a wide range of meteorological variables. There are considerable difficulties, however, in making long-term measurements from buoys, including proximity to the sea surface, wave-induced buoy motions, sheltering in wave troughs and power limitations (Weller et al.,2008). It is particularly challenging to maintain stability of calibration for humidity sensors in the presence of the inevitable sea spray, salt and organic contaminants. For the best quality data we need the data from the buoys to be monitored; effective calibration and sensor replacement programmes must be in place. The metadata required to make best use of these data are typically not readily available in the data archives. The difficulties in maintaining surface moored buoys mean that there can often be data gaps of many months before faulty sensors can be replaced. Moored buoys are expensive to maintain and cannot yet be reliably deployed in harsh environments. Vandalism is a problem in some regions and leads to significant data loss, particularly in the tropics. In extratropical regions, damage and data loss is more likely to be caused by winds, waves and icing.
Drifting buoys typically provide observations of a limited range of meteorological variables. Some only measure SST and position, about half additionally report atmospheric pressure, and a small number report air temperature or wind speed. The data quality is reasonably good for SST and atmospheric pressure; it is less good for air temperature and winds. As drifting buoys are often deployed in data sparse regions the opportunities for data intercomparisons and quality checks are often limited.
The number of observations of SST and pressure from drifting buoys has increased dramatically in recent years. For the case of SST this has been largely driven by the requirement to use in situ observations to derive bias corrections for satellite measurements of SST to remove the effects of changes in the atmospheric composition. These bias-adjusted SST fields are needed for many applications, including as forcing for weather forecasts. The increasing number of pressure observations from drifters is for assimilation into weather forecasts. Surface drifters are relatively cheap and easy to deploy (Figure 5). They are particularly valuable for providing observations outside the main shipping lanes and over the extratropical deep oceans where the moored buoys cannot reliably be deployed. Drawbacks include the narrow range of parameters sampled, limited power, lack of sampling in regions where the ocean surface currents diverge, and oversampling when drifters get caught in eddies. Lack of post-calibration is also a problem as drifters are rarely recovered. A rare example was a drifter ceremonially deployed to celebrate the drifter network meeting its design target of 1250 buoys: global drifter 1250 travelled from Canada to France over the course of 500 days. It was recovered by a French Naval vessel RHM Tenace and returned to Meteo-France (Figure 5).
Figure 4 shows the locations of observations for a single day in July 2009 for SST, air temperature, pressure and wind speed. SST shows the densest network with observations from ships, moored buoys and drifters covering much of the ocean. The pressure network is also fairly complete, but there are some major gaps in the Tropics and Southern Hemisphere. Both air temperature and wind speed show some even bigger gaps as most drifters do not measure these variables. This is particularly a problem for air temperature and also humidity, which, unlike wind speed, cannot be reliably measured by satellites.
Figure 6 shows observations of SST, air temperature, surface pressure and wind speed made on the 1 July 2009, as collated in the International Comprehensive Ocean-Atmosphere Data Set (ICOADS v2.5: Woodruff et al.,2010). It is clear that, although observations are received from most areas of the ocean, the density of observations varies quite dramatically. For SST, there are enough data from ships, moored buoys and drifters to define the large-scale field with a single day's observations. Pressure is fairly well sampled, but there are some substantial gaps. Air temperature is only measured on ships and moored buoys, and sampling is therefore patchy. Wind speed shows higher small-scale variability and so one day's data does not provide a simple picture of conditions, although some features such as the wind speed gradient across the tropical Pacific are clear.
From observations to forecasts
Early in the twentieth century, observations from ships began to be transmitted in real time via wireless telegraphy for use in weather forecasting. It also became possible for warnings of dangerous conditions to be transmitted back to the ships. At the 1929 meeting of the Safety of Life at Sea (SOLAS) Convention, provision was made for the international encouragement of meteorological observation at sea. Almost all surface observations are now transmitted back to Met. Services as soon as they are made, usually via satellite, and are shared with other forecast centres. A minority of reports, from ships' paper or electronic logbooks, are available only after a significant delay and these ‘delayed-mode’ reports are archived and used in climate research rather than weather forecasting. Limited information describing methods of measurement is transmitted with the observation. The additional information or metadata, discussed earlier, is increasingly being associated with observations and used to adjust observations, for example to adjust measured wind speeds to ten metres height.
For weather forecasting the time of receipt of observations is critical. Cut-off times for the Met Office NWP systems are: 90 minutes for North Atlantic data in the regional forecast system, 150 minutes for the global forecast system and 6 hours for global ‘update’ analyses. SST analyses and sub-surface analyses for ocean forecasting are performed at about 0600 utc for the previous day. Delayed sub-surface observations for one or two days earlier are used with reduced weight. Other Met. Services use broadly similar cut-off times, which have tended to become shorter over the years.
Automated real-time quality control systems are used to reject observations with large errors as determined by comparison with model forecast information and other nearby observations. There is also a track check: 1–2% of manual ship reports have position errors, but the percentage is lower for automated ship reports. Additionally, routine monitoring of data quality and consistency is performed using comparisons of observations with model forecast information. Ships/variables with persistent problems are not used in the NWP systems, with the list of exclusions being updated monthly. Information from the monitoring is used to advise PMOs of data quality issues requiring their attention and is also exchanged between Met. Services. The UK Met Office is a WMO-designated lead centre for monitoring the quality of surface marine data. Results of this monitoring are compiled and distributed at monthly and six-monthly intervals (Met Office, 2010b).
Weather forecasts use a high-resolution SST field as the lower boundary condition. The Met Office system is called Operational Sea Surface Temperature and Sea Ice Analysis (OSTIA, http://ghrsst-pp.met office.com/pages/latest_analysis/ostia.html). The OSTIA system uses high-resolution satellite data together with in situ observations to produce a daily SST field with approximately six-kilometre horizontal resolution. Both satellite and in situ data are needed: the satellite provides the high-resolution detail but can be biased on larger spatial scales by changes in atmospheric composition – for example, in the presence of volcanic aerosols or Saharan dust. The coarser resolution sampling from the in situ network is needed to make corrections for these effects (Figure 7).
Until recently the only atmospheric variables used from ships and buoys in most NWP systems were pressure and wind, but the Met Office is now using temperature and humidity information as well (Ingleby, 2010). The information is combined with that from a forecast and other observations – including satellite estimates of oceanic winds. Figure 8 shows an example pressure map from the Met Office forecast. The locations of observations are indicated by dots. The colours represent the difference between the observation and the model background field. The background is the previous 6-hour forecast, bi-linearly interpolated in space and linearly interpolated in time (using the 3-, 6- and 9-hour forecasts) to allow the best use of asynoptic observations. Over densely sampled land regions, the differences between observations and background are small. The largest differences are very probably due to problems with the observations and will have been excluded from the forecast analysis. The background pressure fields are generally very good and only small adjustments to them are made in the analysis process. This can be seen in the comparison of reported and background values for a buoy in the Bay of Biscay (Figure 9). The depth of the low pressure on 21 December was underestimated by the model and there are differences at the end of the month but mostly the reported and forecast pressures agree very well. The temperatures follow each other fairly closely, but there are differences of detail.
After their use in forecasts, the observations are archived and later supplemented or replaced by observations from a parallel data system designed to produce long-term data sets for climate applications (Met Office, 2010c). The Met Office Hadley Centre maintains long-term datasets of SST (Figure 10), night marine air temperature, and pressure (Met Office, 2010a).
Whilst for NWP the most important parameters are air pressure and SST, the full range of observations is needed for other applications. An obvious example is the calculation of heat and freshwater fluxes, or exchanges, between the atmosphere and ocean. Flux calculation requires knowledge of variables including wind speed, SST, air temperature, humidity, pressure and clouds (Berry and Kent, 2009b; Fairall, et al., 2010). Figure 11 shows estimates of the net heat exchange over the past 40 years calculated from ship data alone. These calculations are very sensitive to errors in the data: the currently available in situ and satellite observations are not accurate or dense enough to give estimates of net heat exchange consistent with observed changes in heat content of the upper ocean.
We can expect that trends towards automation will continue, along with initiatives to improve data quality and the availability of more information describing the observations. As the skill of weather forecasts improves, the quality of observations needed will increase. Adjustments to the data, for example correcting winds to a common reference height, are starting to become routine in forecasting as they have long been in climate applications. Some benefits may also come with better quality control of the observations, but higher quality data will clearly help all applications. More targeting of marine surface observations in important or under-sampled regions to improve forecast quality is likely. The challenge will be to ensure that increased automation and changes in sampling are not to the detriment of climate applications and air-sea interaction research.
The authors acknowledge with gratitude the many unpaid observers who have contributed to the long record of marine measurement. The authors also thank the reviewers for their help in improving the manuscript. The contribution of Elizabeth Kent was funded by the NERC Oceans2025 programme (Theme 10, SO9).