Between-observer variation in the application of a standard method of habitat mapping by environmental consultants in the UK

Authors


Andrew Cherrill (e-mail andrew.cherrill@sunderland.ac.uk).

Summary

1. In the UK, Phase 1 survey is a standard method of habitat mapping that has been used widely for environmental assessment and management planning. In this paper we make the first rigorous test of the precision with which environmental consultants apply the technique.

2. Six ecologists surveyed independently the same upland site in northern England. In pairwise comparisons between maps, spatial agreement was found to average 25·6% (with a range of 17·3–38·8%) of the area of the study site. The numbers of land cover types that were identified ranged from 13 to 21. Four or more surveyors agreed on the classification of 19% of the study site, while the area of land upon which all six agreed was only 7·9% of the study site. Spatial errors in the positioning of habitat boundaries occurred, but were a relatively minor source of the differences between maps. The majority of differences between maps were due to classification errors. Land cover types with similar species compositions were most frequently confused.

3. Spatially referenced field ‘target notes’ giving additional information on the vegetation mapped in each survey varied in number between 18 and 56. The contents of target notes were inadequate to allow a retrospective assessment of mapping decisions. The total numbers of species listed in target notes varied between surveys from 25 to 145. Sorenson's similarity for species lists derived from pairs of surveys ranged from 18·8% to 63·7%, and was not related to spatial agreement between surveys.

4. Time spent at the field site was not a correlate of any aspect of the results or cost of the survey. Three surveys conducted by members of a professional institute for ecologists were the most expensive, and also recorded larger numbers of target notes and species than the other surveys. However, their maps were no more similar than other pairs of maps.

5. Analysis of the survey results and comparisons with other methods of vegetation mapping suggest that mapping precision could be increased by (i) placing a greater emphasis on use of aerial photographs and other extant map data prior to (and during) field work; (ii) making greater provision for mapping of mosaics and increasing the level of floristic information in habitat definitions; (iii) recording a greater number of more detailed target notes in the field; and (iv) providing office-based support to assist in the interpretation of aerial photographs, and the cross-checking of field surveyors’ preliminary classifications against the contents of target notes and habitat definitions. The current application of the Phase 1 approach by environmental consultants places too great a reliance on decision-making by the (frequently) unsupported lone surveyor whilst in the field.

Introduction

Mapping of biological resources is a valuable tool in aiding the identification of conservation priorities, and has an important role in monitoring and planning to minimize the deleterious impacts of projects and policies (Harding 1992, 1994; Morris 1995). However, all databases (spatial and otherwise) are likely to contain errors, and it is important that the scale and nature of these errors are known (Allen 1981; Harding 1994; Rich & Woodruff 1995; Williams 1996). This paper therefore focuses on the reliability of a method of vegetation mapping that has been used extensively in Britain. Comparisons with alternative methods are drawn and recommendations made that are likely to be widely applicable.

Vegetation mapping is intimately linked with vegetation classification. Kuchler (1967) identifies two types of interaction. In the first, field data on a series of stands are obtained, and these data are used to construct a novel classification that is particularly appropriate for the mapping of that locality. In the second, an extant classification is available before field work commences and vegetation is classified within the existing framework. This approach ensures that surveys in different regions and years use a common framework, enhancing communication and comparative analyses (Kuchler 1967, 1988a). The application of extant classifications is of particular value in conservation and land use planning, where an appreciation of the overall distribution of natural resources is required (Uhlig & Jordan 1996; Banner et al. 1996; Smith & Carpenter 1996). In Britain, two comprehensive national vegetation classifications, the National Vegetation Classification (Rodwell 1991) and Phase 1 system (Nature Conservancy Council 1990), have been developed and applied widely for these reasons. Phase 1 survey forms the focus of this paper.

Classes in the Phase 1 classification are defined on the basis of a range of factors, including land use, soils, physiognomy of vegetation and the presence of species characteristic of particular edaphic conditions (Nature Conservancy Council 1990). Because of the range of criteria used in the Phase 1 classification, the classes are arguably best referred to as land cover types, rather than vegetation types. None the less, the aims of Phase 1 survey are to identify sites of potential wildlife conservation value and those requiring more detailed survey. County-wide Phase 1 surveys have proved useful in the setting of conservation priorities, and are also recommended as a basis for the initial assessment of the impact of planning applications (Wyatt 1991; Institute of Environmental Assessment 1995; Morris 1995).

Recently, Cherrill & McClean (1995) reported a study in which an area of approximately 25 km2 was surveyed independently by two organizations. Errors in the classifications in one or both maps were found to account for at least 41% of the total area. The result emphasized the need for incorporating quality control measures in the design of surveys, but also the need for further work to assess the extent of errors in Phase 1 data. The study can be seen against a background of increasing concern that the quality of scientific (and particularly ecological) input to environmental assessment and planning is often inadequate in the UK (Coles 1993; Treweek 1996; Thompson, Treweek & Thurling 1997). Similar concerns have also been raised in Australia (Warnken & Buckley 1998), Canada (Beanlands & Duinker 1984) and North America (Reichhardt 1999).

The present study compares six Phase 1 vegetation maps of an upland area in England. Five of the six surveys were conducted by professional consultants. The causes of differences between surveys are identified, and comparisons between the Phase 1 methodology and alternative techniques used in Europe and North America (Kuchler 1967; Kuchler & Zonneveld 1988) are used to suggest recommendations, which it is hoped will significantly enhance the objectivity of vegetation mapping in the UK.

Methods

The phase 1 classification

The major divisions in the classification are ‘woodland and scrub’, ‘grassland and marsh’, ‘tall herb and fern’, ‘heathland’, ‘bog’, ‘swamp, marginal and inundation’, ‘open water’, ‘coastland’, ‘rock exposure and waste’ and ‘miscellaneous’ (incorporating boundary features and highly artificial land cover types such as buildings, arable crops and amenity grasslands). Within each division there are subdivisions, which in many cases are further subdivided to give a hierarchical structure to the classification.

The Phase 1 land cover types are defined primarily on the basis of the dominant and characteristic plant species, but for certain land cover types this information is supplemented by soil and land use characteristics. ‘Woodlands’, for example, are subdivided into ‘broadleaved’, ‘coniferous’ and ‘mixed woodlands’. Each of these subdivisions is then split into ‘semi-natural’ and ‘plantation’ land cover types, reflecting land use considerations. Depth of peat is a factor in the definition of ‘mire’ land cover types, with all ‘mires’ occurring on peat deeper than 0·5 m. Certain grasslands are divided into ‘acid’, ‘neutral’ and ‘basic’ types, although soil conditions are inferred from the presence of indicator species rather than direct measurement of soil pH. Degree of agricultural improvement is a key factor in the classification of bogs and grasslands. For example, ‘blanket bogs’ that have been damaged by heavy grazing, burning and drainage, and which consequently have an altered species composition, are classed as ‘modified’. In the classification of grasslands, ‘unimproved’ types have not been sown and are dominated by native species in mixtures determined by soil, climate and traditional low-intensity grazing or cutting. ‘Improved grassland’ and ‘poor semi-improved grassland’ are those which have been heavily influenced by drainage, fertilizers, herbicides and sowing of species of agricultural value. The latter are marginally more species-rich that the former, but are of low potential conservation value. ‘Acid’, ‘neutral’ and ‘basic semi-improved’ grasslands form a transitional group that has been influenced significantly by agricultural practices, but which retains species that are characteristic of ‘unimproved grassland’ and soils of the area. Descriptions of the Phase 1 land cover types are provided by the Nature Conservancy Council (1990).

Phase 1 field methodology

Surveyors identify areas of relatively homogeneous vegetation and assign each ‘parcel’ to one of the land cover types in the classification whilst on site. Boundaries often follow artificial linear features, but in open unenclosed areas, and where there is variation within fields, surveyors draw their own boundary lines. Aerial photographs are used where available, particularly for locating complex boundaries that are not marked by artificial boundaries. Land cover types are mapped on to 1 : 10  000 Ordnance Survey (OS) maps in the field using standard colour or alphanumeric codes.

There is no requirement for soil survey in the methodology, although a knowledge of local soils is advantageous. Surveyors are advised to familiarize themselves with existing information (e.g. soil maps) before commencing work in the field. In large-scale surveys involving teams of botanists, training in groups and working in pairs is recommended to standardize recording.

Information on mapped land cover types and interesting features smaller than the minimum mappable area (0·1 ha at 1 : 10 000) is recorded in ‘target notes’ linked to numbered dots on the map. Target notes may include information on habitat type, dominant plant species, other noteworthy species (e.g. those of conservation value), site management and desirability of further survey. The number of target notes required is not specified in the Phase 1 manual, although in general as many as possible should be recorded in the time available (Nature Conservancy Council 1990). On completion of field work, target notes are typed up and a final neat version of the field map is prepared.

Selection of surveyors

Eight environmental consultancy companies known to undertake Phase 1 survey were contacted and asked to submit an estimate for a Phase 1 survey of the study site (for which a map and brief description were provided). From the replies, five consultants (responsible for surveys A–E) were selected to encompass the range of estimates (Table 1).

Table 1.  The employment status of surveyors, membership of the Institute of Ecology and Environmental Management (IEEM), survey effort and the cost of each survey
Survey effortCost of survey
Survey
Surveyor(s) employed by consultancy
Surveyor(s) IEEM member
Days
Number of surveyors
Hours
£
AYesYes12201500
BYesNo1324400
CYesYes1217800
DYesYes24221600
EYesNo1324600
FNoNo1215300

In addition to the five professional consultants, an individual with Phase 1 experience, currently employed by a governmental conservation agency, agreed to carry out a sixth survey (survey F) (Table 1). For brevity, the surveyors and surveys for which they were responsible will be referred to by the same letter (hence surveyor A completed survey A, and so on).

Three of the surveys (A, C and D) were conducted by individuals who were members of the Institute of Ecology and Environmental Management (IEEM), which represents the interests of professional ecologists in the UK. Although IEEM has no statutory powers, members share the aim of improving the quality of environmental assessments. Membership is based on peer-review of experience and qualifications.

Organization of the field surveys

Surveyors were asked to use the standard Phase 1 approach (Nature Conservancy Council 1990). The report was specified to include a colour-coded 1 : 10 000 scale Phase 1 map, target notes and summaries identifying areas of potential conservation value requiring further survey (reported elsewhere), and any problems encountered in carrying out the field survey.

The study site, of approximately 4 km2, was at the ADAS Experimental Husbandry Farm, Redesdale, Northumberland (Grid Reference NY8396), and included land rising from the banks of the River Rede to an altitude of 350 m a.s.l. Surveyors had free access to all parts of the farm. A range of land uses was present, including intensified (i.e. ‘improved’) and unintensified (i.e. ‘unimproved’) grazing, and several broadleaved and coniferous woodlands. A small number of additional fences, not shown on the OS map (Fig. 1), were present, but the study site was characterized by large enclosures (particularly in the south).

Figure 1.

The base map of the study site showing information available to all surveyors. The field marked X is referred to in the text (as Field X).

Field surveys were conducted between 1 July and 1 August 1996. Survey dates were arranged so that surveyors would not be present on the site at the same time. On one day this system failed and two surveyors (B and C) were on site at the same time. Surveyors were required to sign a visitors’ book, before entering or leaving the study site. This provided information on survey duration (Table 1). Colour aerial photographs of the site taken in 1995, and an office to facilitate their use, were made available to the surveyors.

Preparation of field maps for analysis

Surveyors’ maps were compared with each other using a Geographical Information System (GIS). All digitizing was performed by a single operator using the ARC/INFO GIS. Boundaries on the OS map used as a basis for mapping by all surveyors were digitized to produce a standard ‘base map’ (Fig. 1). This map was then copied to produce six identical base maps. Individual surveyor's maps were registered to the base map and additional boundaries drawn by the surveyors were digitized. Parcels of land were labelled with a code representing their Phase 1 land cover types. In the case of the Phase 1 land cover type ‘scattered trees’, parcels were coded according to the type of vegetation within which the trees occurred. These non-standard Phase 1 cover types have been allocated codes prefixed A.3.

Copies of the original Phase 1 maps were made, and in these new maps the land cover types were aggregated into a smaller number of broad land cover categories. In the new map, boundaries between parcels sharing the same land cover category code were dissolved. The list of land cover categories reflected the hierarchical nature of the Phase 1 classification of cover types, but also the experience of the authors (Cherrill & McClean 1995). Thus, for example, recently sown grass leys, ‘improved grassland’ and ‘poor semi-improved grassland’ were grouped in the ‘pasture/silage’ category. Nature Conservancy Council (1990) classifies recently sown grass leys as ‘arable’, rather than grassland, but leys were included in the ‘pasture/silage’ category because they are dominated by sown species that are also abundant in ‘improved’ and ‘poor semi-improved grassland’.

Copies of the maps of the original and aggregate classifications were made and modified by adding line buffers. Buffering involved enclosing the land lying within 25 m of each boundary by the addition of arcs running parallel to those already present. The 50-m wide ‘buffer zones’ were eliminated from certain of the analyses described below. This approach reflects the acceptance that field surveyors could not be expected to mark boundaries on their field maps with absolute accuracy. Moreover, it is accepted that many boundaries are not distinct, but are better regarded as zones of change (or ecotones) between related vegetation types. Buffering enabled a comparison of the cover types assigned to the ‘core areas’ of parcels of land, thereby minimizing any subtle spatial differences between maps in the location of boundaries. The choice of a 25-m buffering distance was ultimately subjective, but reflected the authors’ prior analyses using a range of alternative distances (A. Cherrill & C. McClean, unpublished data), the resolution of Phase 1 mapping at 1 : 10 000 and the authors’ (albeit subjective) interpretation of the width of ecotones present at the study site.

All boundaries were buffered, except those which were present in the original base map. Areas lying within buffer zones were labelled with a new code. The unbuffered maps containing the maps of the original and aggregate classifications, plus their buffered copies, were stored as separate layers within the GIS, resulting in four separate maps for each survey.

Agreement between pairs of maps

The agreement between maps was quantified by overlaying them within ARC/INFO as follows: (i) pairwise comparison of the unbuffered maps of the original land cover types; (ii) pairwise comparison of the buffered maps of the original land cover types; (iii) pairwise comparison of the unbuffered maps of the aggregate land cover categories; and (iv) pairwise comparison of the buffered maps of the aggregate land cover categories.

The results of each overlay were presented as a matrix of correspondence in which the (Mi,Nj)th cell gave the area of cover type i in map M classed as cover type j in map N. In each matrix, cover types were placed in the same order along rows and columns. The percentage of land that lay within the diagonal cells of each matrix was used as an index of the extent of the agreement between pairs of maps.

Construction of matrices of correspondence was identical for all overlays, except that where buffering had been used, land lying within a buffer zone in either (or both) maps was excluded. Agreements for pairs of unbuffered maps were expressed as a percentage of the total area included in the overlay analysis. In contrast, the area of agreement for each pair of buffered maps was expressed as a percentage of the land lying outside of the buffer zones in those maps (rather than as a percentage of the site as a whole).

For each pair of surveyors’ maps, four estimates of overall agreement were calculated from the comparisons detailed in (i), (ii), (iii) and (iv) above. In addition, four estimates of agreement for a single field (referred to as Field X, and shown in Fig. 1) were also calculated from these overlays. Field X has an area of 0·27 km2 and is reverting to semi-natural grassland, having been intensified in the past.

Agreement between all six maps

The overall agreement between the six surveys was assessed by combining all six maps. This was done for maps containing unaggregated and aggregate land cover classifications, with and without buffers, such that four new maps were created. In each of the new maps, all land parcels had six land cover codes (one from each of the six maps). In analyses involving the buffered maps, areas contained within a buffer zone in one (or more) of the map layers were excluded from the analysis. The number of occurrences of each code within each parcel was then computed. Each parcel was relabelled with the maximum number of occurrences recorded for any cover type occurring within that parcel. The total areas labelled 1 (agreement between none of the maps), 2, 3, 4, 5 and 6 (agreement between all six maps) were calculated for each of the four new maps.

Agreement for individual land cover types

Agreement for individual classes in a classification could be calculated from a matrix of correspondence in two ways, differing in whether the marginal row or column total is used as a basis for interpretation of the figure in the diagonal cell (Story & Congalton 1986). In the present study, map M was compared to five others, resulting in five matrices of correspondence. Five estimates of the agreement of cover type i were therefore calculated by expressing the five absolute areas of agreement for i (i.e. one from each matrix) as a percentage of the total area of i in the reference map M. Identical calculations were also performed using each of the other five maps as the reference map. These estimates for cover type i were then summarized as median and range for each reference map (i.e. n = 5 in each case).

Target notes

The numbers and contents of target notes in the six maps were compared (i) at the level of the whole study site (by pooling information across target notes within each survey), and (ii) for the single field referred to as Field X (Fig. 1).

Comparisons for the whole study site

The contents of target notes were quantified by recording the numbers and identities of plant species in each note. Use of measures of species abundance was noted. The identities of plant species recorded in target notes were tabulated in a single list for each survey. Sorenson's similarity index was calculated for pairs of lists derived from different surveys (Southwood 1966).

Comparisons for a single field

An important consideration in this study was the extent to which the contents of target notes could be used to understand the decisions made by surveyors in assigning parcels of land to a particular land cover type. Target notes for Field X were therefore compared between surveys and cross-referenced with the Phase 1 land cover definitions. To illustrate this analysis, extracts from the Phase 1 manual and the surveyors’ target notes are quoted. In these quotations, and throughout the paper, nomenclature follows Stace (1991).

Survey effort

Surveyors were required to sign a visitors’ book on arrival and departure at the study site. The number of hours spent on site and the numbers of days on which visits were made were each used as estimates of survey effort. These figures were also multiplied by the number of surveyors to give two further estimates of survey effort. It was not possible, however, to distinguish time spent in the field from that examining the aerial photographs in the farm office. Several surveyors borrowed the photographs overnight and this was noted.

Problems encountered during surveys

Surveyors were asked to include a section in their reports identifying problems encountered in carrying out the field survey. Further information was also extracted from the target notes.

Results

Aspatial analyses

Comparisons for the whole study site

In total, 35 land cover types were recorded by the six surveys. The number of land cover types recorded in individual maps ranged from 13 to 21 (Table 2). Each survey also included small areas of land that were not classified.

Table 2.  The areas of land cover types recorded in each surveyor's map as a percentage of the study site's total area. Land cover types are grouped under 13 broad categories (+ = percentage area is less than 0·05). The codes are from Nature Conservancy Council (1990)
Map
Phase 1 land cover types (and codes)ABCDEF
Broadleaved woodland
 Broadleaved semi-natural woodland (A.1.1.1.)4·15·04·83·33·34·5
 Broadleaved plantation woodland (A.1.1.2.)0·80·00·11·41·70·1
 Felled broadleaved woodland (A.4·1.)0·00·00·0+0·00·0
 Sub-total4·95·04·94·85·04·6
Coniferous woodland
 Coniferous semi-natural woodland (A.1.2.1.)0·00·20·00·00·00·0
 Coniferous plantation woodland (A.1.2.2.)0·00·10·30·20·20·7
 Sub-total0·00·30·30·20·20·7
Mixed woodland
 Mixed semi-natural woodland (A.1.3.1.)0·20·00·00·00·00·0
 Mixed plantation woodland (A.1.3.2.)0·60·40·00·50·20·0
 Sub-total0·80·40·00·50·20·0
Scattered trees
 Trees in improved grassland (A.3.1.)0·00·00·0+0·00·0
 Trees in semi-natural neutral grassland (A.3.2.)0·00·0+0·00·00·0
 Trees in marshy grassland (A.3.3.)0·00·00·40·10·00·0
 Trees in unclassified land cover type (A.3.4.)0·00·0++0·00·0
 Sub-total0·00·00·50·20·00·0
Rough grassland
 Unimproved acid grassland (B.1.1.)0·02·119·60·416·110·5
 Semi-improved acid grassland (B.1.2.)15·45·14·011·27·74·3
 Unimproved neutral grassland (B.2.1.)0·01·00·10·00·30·0
 Semi-improved neutral grassland (B.2.2.)16·111·72·724·07·20·0
 Marshy grassland (B.5.)2·01·123·00·310·84·2
 Sub-total33·520·949·535·942·119·0
Pasture/silage
 Arable (grass ley) (J.1.1.)0·57·90·50·07·70·0
 Improved grassland (B.4.)11·78·511·916·39·514·5
 Poor semi-improved grassland (B6.)0·04·55·10·00·022·2
 Sub-total12·221·017·516·317·236·7
Tall herb and fern
 Bracken (C.1.1.)0·40·70·50·81·00·4
 Ruderal tall herbs (C.3.1.)0·10·00·00·00·40·1
 Non-ruderal tall herbs (C.3.2.)0·00·00·00·00·10·0
 Sub-total0·50·70·50·81·50·5
Heathland
 Dry acid dwarf shrub heath (D.1.1.)0·05·60·80·01·40·1
 Wet acid dwarf shrub heath (D.1.2.)0·00·012·20·00·016·3
 Dry dwarf shrub heath/acid grass mosaic (D.5.)0·014·50·00·027·40·8
 Wet dwarf shrub heath/acid grass mosaic (D.6.)46·428·412·20·00·00·0
 Sub-total46·448·425·10·028·817·2
Mire
 Blanket bog (E.1.6.1.)0·00·00·00·00·01·1
 Wet modified bog (E.1.7.)0·00·10·013·71·00·0
 Dry modified bog (E.1.8.)0·00·00·025·81·217·6
 Acid/neutral flush (E.2.1.)0·00·90·1+0·00·0
 Sub-total0·01·10·139·62·218·6
Swamp
 Swamp (F.1.)0·00·00·0+0·00·0
Rock exposure
 Acid/neutral natural rock exposure (I.1.4.1.)0·00·20·00·01·00·0
 Quarry (I.2.1.)0·00·00·0++0·0
 Mining Spoil (I.2.2.)0·00·00·0+0·00·0
 Sub-total0·00·20·0+1·10·0
Built
 Built (buildings, metalled roads, etc.) (J.3.6.)1·51·51·51·61·51·5
Unclassified land
 Unclassified land0·10·50·2+0·11·1
Total100·0100·0100·0100·0100·0100·0

The land cover types with the greatest areas within individual maps were ‘wet dwarf shrub heath/acid grassland mosaic’ (maps A and B), ‘marshy grassland’ (map C), ‘dry modified bog’ (map D), ‘dry dwarf shrub heath/acid grassland mosaic’ (map E) and ‘poor semi-improved grassland’ (map F). Of these cover types, only ‘marshy grassland’ was recorded in all six maps (with areas ranging from 0·3% to 23·0% of the study site). The other three ‘dominant’ cover types were each recorded in three maps only.

Overall, six land cover types were recorded from all maps. A further four were recorded from five maps. These figures included the cover types ‘built’ (buildings, roads and unmetalled tracks), ‘broadleaved semi-natural woodland’ and ‘conifer plantation’, which were shown on the original base map (Fig. 1). Combining the 35 land cover types into 12 broader categories increased the consistency of mapping, with eight land cover categories being recorded in at least five maps. However, some large differences between maps remained. For example, the areas of ‘heathland’ and ‘mire’ ranged from 0% to 46·4% and 0% to 39·6%, respectively, while the area of ‘pasture/silage’ varied threefold, from 12·2% to 36·7% (Table 2).

Comparisons for a single field

The six surveyors identified seven land cover types, but four of these were found in a single map only. ‘Marshy grassland’ alone was recorded in more than two surveys (Table 3). The land cover types with the greatest areas within individual maps were ‘semi-improved acid grassland’ (map A), ‘semi-improved neutral grassland’ (maps B and D), ‘marshy grassland’ (maps C and E) and ‘poor semi-improved grassland’ (map F). Aggregating land covers into broader categories resulted in a substantial increase in the consistency of mapping, with five of the six maps agreeing that over 90% of the field was ‘rough grassland’ (Table 3).

Table 3.  The areas of land cover types recorded in Field X in each surveyor's map as a percentage of the field's total area. Land cover types are grouped under the broad categories shown in Table 2. The codes are from Nature Conservancy Council (1990).
Map
Phase 1 land cover types (and codes)ABCDEF
Rough grassland
 Semi-improved acid grassland (B.1.2.)100·00·00·00·021·30·0
 Semi-improved neutral grassland (B.2.2.)0·092·30·0100·00·00·0
 Marshy grassland (B.5.)0·00·099·40·078·321·9
 Sub-total100·092·399·4100·099·621·9
Pasture/silage
 Improved grassland (B.4.)0·00·00·60·00·00·0
 Poor semi-improved grassland (B.6.)0·00·00·00·00·078·1
 Sub-total0·00·00·60·00·078·1
Tall herb and fern
 Bracken (C.1.1.)0·00·00·00·00·40·0
Mire
 Acid/neutral flush (E.2.1.)0·07·70·00·00·00·0
Total100·0100·0100·0100·0100·0100·0

Spatial agreement between pairs of maps

Comparisons for the whole study site

Agreement between the Phase 1 land cover type maps was low (Table 4). The maximum agreement between any pair of maps was 38·8%, while the lowest agreement was 17·3%. When land cover types were aggregated into broad land cover categories, the mean agreement between maps was increased from 25·6% to 56·4% for unbuffered maps (paired t-test, t = 10·52, P < 0·0001), and from 27·7% to 59·1% for buffered maps (paired t-test, t = 9·83, P < 0·0001). Adding buffers raised the mean agreement from 25·6% to 27·7% for the original land cover maps (paired t-test, t = 7·44, P < 0·0001), and from 56·4% to 59·1% for maps of broad land cover categories (paired t-test, t = 8·56, P < 0·0001). With the application of both techniques the greatest agreement achieved between any pair of maps was 79·3% (Table 4). Aggregation of similar land cover types was much more effective at increasing agreement than the use of buffers.

Table 4.  The spatial correspondence between pairs of surveyors’ maps assessed using percentage agreement (%). Correspondence was assessed for maps with and without buffering of boundaries, and for maps with and without aggregation of land cover types into broad categories
Maps of unaggregated land cover typesMaps of aggregated land cover types
Pair of mapsUnbuffered maps (%)Buffered maps (%)Unbuffered maps (%)Buffered maps (%)
A–B38·842·574·979·3
A–C29·530·969·271·1
A–D32·033·144·645·7
A–E21·122·673·776·5
A–F18·119·140·641·5
B–C29·532·761·765·4
B–D22·025·038·139·9
B–E24·626·165·970·2
B–F17·319·846·650·6
C–D19·120·754·457·3
C–E25·826·471·375·5
C–F27·729·154·356·1
D–E23·627·454·457·5
D–F34·237·550·652·2
E–F20·722·046·248·0
Mean25·627·756·459·1
SD6·36·712·313·1

Comparisons for a single field

The maximum agreement between any pair of maps for Field X was 92·3%, but in 10 out of 15 pairwise comparisons spatial agreement was zero (Table 5). Aggregation of land cover types resulted in significant improvements in agreement for both unbuffered (paired t-test, t = 5·10, P < 0·0002) and buffered maps (paired t-test, t = 4·78, P < 0·0003). However, adding buffers had little effect on agreement for either the original maps (paired t-test, t = 0·28, P = 0·78) or maps of broad aggregated land cover categories (paired t-test, t = 1·89, P = 0·08) (Table 5). In some cases, agreement was reduced by the addition of buffers. Visual inspection of the maps for this field indicated that differences in the location of boundaries were so large that they could best be attributed to classification errors, rather than differing perceptions of the same ecotones (Fig. 2).

Table 5.  The spatial correspondence between pairs of surveyors’ maps for Field X assessed using percentage agreement (%). Correspondence was assessed for maps with and without buffering of boundaries, and for maps with and without aggregation of land cover types into broad categories
Maps of unaggregated land cover typesMaps of aggregated land cover types
Pairs of mapsUnbuffered maps (%)Buffered maps (%)Unbuffered maps (%)Buffered maps (%)
A–B0·00·092·3100·0
A–C0·00·099·4100·0
A–D0·00·0100·0100·0
A–E21·35·999·6100·0
A–F0·00·021·918·7
B–C0·00·091·7100·0
B–D92·3100·092·3100·0
B–E0·00·092·0100·0
B–F0·00·021·926·9
C–D0·00·093·4100·0
C–E77·794·099·0100·0
C–F21·919·022·519·0
D–E0·00·099·6100·0
D–F0·00·021·918·7
E–F18·319·521·919·3
Mean15·415·971·373·5
SD29·633·636·238·8
Figure 2.

Phase 1 land cover types recorded in Field X by surveys A, B, C, D, E and F. The locations of target notes are indicated by the solid triangles (numbers refer to Table 11).

Spatial agreement between six maps

Overlay of the six maps revealed that the area of land that was classified as the same land cover by all surveyors was only 7·9% of the total study area (Table 6, Fig. 3). This figure included the areas of roads, tracks and buildings that were shown on the base map (Fig. 1). Other land cover types contributing to the agreement between all six maps were ‘improved grassland’ and ‘broadleaved semi-natural woodland’ (representing 3·9% and 2·5% of the study area, respectively). These cover types were found in the north of the study site, while in the south there was less agreement between maps on the classification of land used for rough grazing (Figs 1 and 3).

Table 6.  The overall agreement between six maps assessed as the area of land (as percentage) for which two or more surveys agreed upon the classification of land cover
Maps of unaggregated land cover typesMaps of aggregated land cover types
Number of maps agreeingUnbuffered maps (%)Buffered maps (%)Unbuffered maps (%)Buffered maps (%)
12·20·80·00·0
250·441·12·41·0
328·531·622·318·5
46·07·528·526·4
55·17·628·931·5
67·911·517·822·7
Area included in analysis (km2)4·032·314·032·73
Figure 3.

The areas of land upon which different numbers of survey maps agreed in terms of classification to Phase 1 land cover type.

Aggregating Phase 1 land cover types into broad land cover categories raised the area of overall agreement to 17·8% (again including an area of 1·5% representing the ‘built’ land cover type). Land cover categories contributing to the agreement were ‘pasture/silage’, ‘rough grassland’ and ‘broadleaved woodland’ (contributing areas equivalent to 10·1%, 2·6% and 3·6% of the study area) (Fig. 4).

Figure 4.

The areas of land upon which different numbers of survey maps agreed in terms of classification to broad aggregate land cover category.

In Table 6 the detailed effects of aggregating land cover types and introducing buffers can be seen on the pattern of agreement between the six maps. Thus, for example, the area of land upon which four or more surveyors agreed was 19·0% of the study site, while after aggregation of land cover types this figure rose to 75·2%. Excluding land that lay within 25 m of a boundary in one or more map reduced the areas included in the analyses of the buffered maps (Table 6). However, as seen in earlier analyses, introduction of buffers had relatively little effect on levels of agreement, indicating that only a relatively small proportion of mapping errors was closely associated with boundaries. Even after buffering of maps of aggregate land cover categories, the area of complete agreement between the six maps was equivalent to only 22·7% of the land lying outside of buffer zones, although four or more surveyors were in agreement on the classification of 80·6% of this area (Table 6).

Spatial agreement for individual land cover types

Estimates of spatial agreement for individual land cover types varied according to which map was used as the reference data set (Table 7). The analyses also showed wide variation in agreement for individual cover types when different maps were compared to the same reference map. Thus, for example, when map F was used as the reference data set, agreement for ‘conifer plantation’ varied from 0% to 38·2% with a median of 21·5%. However, when map E was used as the reference data set, a median figure of 85·1% agreement was recorded for the same land cover type (Table 7).

Table 7.  Median percentage agreements (with minimum and maximum values in parentheses) for individual Phase 1 land cover types using each map as a reference for comparison with the other five (n = five in each case, – = cover type absent from reference map)
Reference map
Land cover typeABCDEF
Broadleaved semi-natural woodland86·675·880·191·190·479·8
(76·5–93·8)(60·6–89·3)(64·2–92·3)(89·1–98·4)(83·5–94·3)(62·4–94·9)
Broadleaved plantation15·40·08·97·5100·0
(0·0–87·9) (0·0–83·4)(0·0–99·5)(0·0–84·5)(0·0–100·0)
Felled broadleaved woodland0·0
(0·0–0·0)
Coniferous semi-natural woodland0·0
(0·0–0·0)
Coniferous plantation90·756·465·385·121·5
(0·0–100·0)(0·0–100·0)(0·0–100·0)(0·0–85·1)(0·0–38·2)
Mixed semi-natural woodland0·0
(0·0–0·0)
Mixed plantation31·9
(0·0–55·1)
51·3
(0·0–93·9)
44·0
(0·0–80·5)
85·9
(0·0–100·0)
Trees in improved grassland0·0
(0·0–0·0)
Trees in semi-improved neutral grassland0·0
(0·0–0·0)
Trees in marshy grassland0·0
(0·0–0·0)
0·0
(0·0–0·0)
Trees in unclassified land0·0
(0·0–0·0)
0·0
(0·0–0·0)
Unimproved acid grassland23·03·515·25·04·6
(0·0–37·9)(0·0–31·8)(0·0–62·8)(0·0–38·9)(0·0–43·2)
Semi-improved acid grassland12·815·64·42·827·516·5
(0·0–24·4)(6·1–41·5)(0·0–62·3)(0·1–27·9)(0·1–48·8)(4·1–43·9)
Unimproved neutral grassland0·0
(0·0–0·0)
0·0
(0·0–0·0)
0·0
(0·0–0·0)
Semi-improved neutral grassland3·0
(0·0–70·8)
3·4
(0·0–68·9)
14·9
(0·0–80·0)
30·0
(0·0–47·3)
0·0
(0·0–100·0)
Marshy grassland2·70·04·76·90·50·0
(0·0–10·6)(0·0–100·0)(0·2–30·4)(0·0–62·6)(0·1–64·7)(0·0–37·4)
Arable (grass ley)100·0
(0·0–100·0)
6·3
(0·0–32·4)
100·0
(0·0–100·0)
6·5
(0·0–33·3)
Improved grassland92·893·593·469·646·374·8
(35·4–98·5)(51·5–99·5)(36·9–97·4)(50·7–88·2)(43·7–86·9)(49·6–98·8)
Poor semi-improved grassland0·00·00·0
(0·0–62·7)(0·0–55·1)  (0·0–9·5)
Bracken20·00·339·728·122·446·5
(0·5–33·3)(0·0–4·0)(0·0–46·8)(3·6–42·8)(0·6–32·6)(0·0–60·6)
Ruderal tall herbs0·00·00·0
(0·0–59·6)   (0·0–20·5)(0·0–100·0)
Non-ruderal tall herbs0·0
(0·0–0·0)
Dry acid dwarf shrub heath0·50·00·20·0
(0·0–12·3)(0·0–56·6) (0·0–48·1)(0·0–35·0)
Wet acid dwarf shrub heath0·0
(0·0–28·5)
0·0
(0·0–21·3)
Dry dwarf shrub heath/acid grassland mosaic0·00·00·0
(0·0–62·4)  (0·0–33·0)(0·0–58·6)
Wet dwarf shrub heath/acid grassland mosaic0·0
(0·0–48·0)
0·0
(0·0–78·5)
0·0
(0·0–92·6)
Blanket bog0·0
(0·0–0·0)
Wet modified bog0·0
(0·0–0·0)
0·0
(0·0–0·7)
0·0
(0·0–10·3)
Dry modified bog0·0
(0·0–51·2)
0·0
(0·0–100·0)
0·0
(0·0–75·3)
Acid/neutral flush0·0
(0·0–0·0)
0·0
(0·0–0·0)
0·0
(0·0–0·0)
Swamp0·0
(0·0–0·0)
Acid/neutral natural rock exposure0·0
(0·0–60·8)
0·0
(0·0–10·1)
Quarry0·0
(0·0–0·0)
0·0
(0·0–0·0)
Mining spoil0·0
(0·0–0·0)
Built98·7100·0100·095·598·6100·0
(98·7–99·8)(100·0–100·0)(100·0–100·0)(95·5–96·6)(98·6–98·6)(100·0–100·0)
Unclassified land37·212·214·80·032·16·7
(0·0–58·0)(4·3–29·9)(0·0–32·3)(0·0–100·0)(0·0–63·2)(1·9–13·1)

With few exceptions there was low correspondence between surveys for individual cover types. Consistently high levels of agreement between maps for individual land cover types were achieved for ‘improved grassland’, ‘broadleaved semi-natural woodland’ and ‘built’ land cover types only (Table 7). Several blocks of woodland were shown on the base map (Fig. 1). However, there was some disagreement as to their composition (‘broadleaved’, ‘mixed’ or ‘coniferous’) and origin (‘semi-natural’ or ‘plantation’). Agreement was, not surprisingly, greatest for the ‘built’ land cover type. Where the spatial agreement for ‘built’ deviated from 100% this reflected the small areas of hard surfaces and outbuildings that were added to the base map by three surveyors.

Combining land cover types into broad land cover categories generally resulted in increased estimates of agreement (Table 8). None the less, the effect of combining cover types into cover categories was variable. The greatest contrast was seen for the land cover categories ‘heathland’ and ‘mire’. Before aggregation ‘heathland’ and ‘bog’ cover types exhibited low (and typically zero) median agreements (Table 7). However, after aggregation the new category ‘heathland’ showed greatly elevated levels of agreement, while no improvement was seen for the ‘mire’ category (Table 8).

Table 8.  Median percentage agreements (with minimum and maximum values in parentheses) for broad land cover categories using each map as a reference for comparison with the other five (n = five in each case; – = cover type absent from reference map)
Reference map
Land cover categoryABCDEF
Broadleaved woodland90·289·791·692·987·994·5
(86·7–91·0)(87·2–90·5)(89·3–92·8)(92·3–93·9)(83·9–88·6)(92·1–96·7)
Coniferous woodland64·278·987·985·230·1
(0·0–66·8)(0·0–100·0)(0·0–100·0)(0·0–85·2)(0·0–38·2)
Mixed woodland23·6
(0·0–40·8)
51·3
(0·0–93·9)
44·0
(0·0–80·5)
85·9
(0·0–100·0)
Scattered trees0·0
(0·0–0·0)
0·0
(0·0–0·0)
Rough grass78·481·554·273·263·861·3
(20·9–80·1)(23·1–81·8)(34·6–66·3)(32·5–89·5)(28·7–78·0)(25·3–90·2)
Pasture/silage96·969·189·995·092·244·7
(88·5–98·9)(51·6–93·0)(68·9–95·9)(72·6–97·5)(68·7–97·0)(31·6–53·0)
Tall herb and fern19·00·439·728·119·138·6
(0·5–29·6)(0·0–4·0)(0·0–59·1)(3·6–47·1)(0·0–24·3)(0·0–71·1)
Heathland52·2
(0·0–90·3)
49·3
(0·0–86·5)
61·1
(0·0–96·4)
53·3
(0·0–100·0)
81·1
(0·0–93·8)
Mire0·00·00·00·00·0
(0·0–0·0)(0·0–0·0)(0·0–43·0)(0·0–100·0)(0·0–91·4)
Swamp0·0
(0·0–0·0)
Rock exposure0·0
(0·0–60·8)
0·0
(0·0–0·0)
0·0
(0·0–9·8)
Built98·7100·0100·095·598·6100·0
(98·7–99·8)(100·0–100·0)(100·0–100·0)(95·5–96·6)(98·6–98·6)(100·0–100·0)
Unclassified land37·212·214·80·032·16·8
(0·0–58·0)(4·3–29·9)(0·0–32·3)(0·0–100·0)(0·0–63·2)(1·9–13·1)

Target notes

Comparisons for the whole study site

The number of target notes associated with each survey varied between 18 and 56 (Table 9). The content of most target notes was restricted to a brief list of species, with subjective assessments of the abundance of subsets of these. Within each report, the numbers of species recorded varied between target notes; few target notes listed a large number of species, and many target notes listed few. There was also significant variation between surveys in the number of plant species listed in target notes (Kruskal–Wallis anova, H = 33·5, d.f. = 5, P < 0·001). Survey C recorded a median of 11 species per target note, while the median for survey B was three (Table 9).

Table 9.  The number and content of target notes associated with Phase 1 survey maps A–F
Map
ABCDEF
Number of target notes501833562427
Median no. of species per note6·03·011·04·04·04·0
(25th – 75th percentile range)(2·0–14·0)(1·0–5·0)(6·5–21·0)(1·0–8·0)(1·3–10·8)(3·0–8·0)
Number of species identified overall10025145837064
Number of species of:
 Trees12917939
 Woody shrubs358836
 Herbs44359302528
 Grasses2212617189
 Rushes438462
 Sedges825653
 Other monocotyledons211220
 Ferns315214
 Horsetails103100
 Bryophytes1013473

The total number of plant species identified in target notes varied almost sixfold between surveys, from 25 (in survey B) to 145 (in survey C) (Table 6). Overall, 205 species were identified across all six surveys. Numbers of sedges recorded varied from two to eight, while the numbers of grasses and herbs recorded varied from one to 26, and three to 59, respectively. Few species of bryophytes were identified in each survey. The similarity of the plant species lists derived from all target notes recorded in each survey ranged from 18·8% to 63·7% (with a mean of 43·4%) (Table 10).

Table 10.  Sorenson's similarity for pairs of species lists derived from target notes associated with each Phase 1 survey map
Pair of maps comparedNumbers of species recorded
Map 1Map 2Map 1Map 2Both mapsSimilarity S (%)
AB100251625·6
AC1001457863·7
AD100835054·6
AE100704552·9
AF100643846·3
BC251451618·8
BD25831527·8
BE25701021·1
BF25641636·0
CD145836758·8
CE145705652·1
CF145645249·8
DE83703849·7
DF83643649·0
EF70643044·8

The information recorded on the abundance of the species was variable. Only one survey (B) explicitly identified a system of scoring species abundance used in the target notes, the stated method being DAFOR (dominant, abundant, frequent, occasional, rare). However, the meaning of the initials was not explained and in the target notes only ‘dominant’ and ‘abundant’ were used (in addition to ‘sparse’, a term not included in the DAFOR approach). In this report, approximately 50% of references to individual species were accompanied by an indication of the species’ abundance at the locations identified by the target notes.

Terminology relating to species abundance in surveys A, D, E and F was variable, with ‘dominant’, ‘frequent’ and ‘occasional’ being used inconsistently, along with a wide range of alternatives, e.g. ‘lots of’, ‘some’, ‘scattered’ or ‘extensive patches of’, and ‘main higher plants are’. In these four reports, between 10% and 50% of references to species within target notes included information (however, vague) on their abundance.

Survey C was the only report within which almost all references to species within target notes (i.e. over 95%) gave an indication of the species’ abundance. This report was also consistent in its use of terminology, which adhered to the DAFOR system, although this was not stated in the text.

Comparisons for a single field

The number of target notes recorded in Field X varied from zero (surveys B and F) to five (survey D) (Fig. 2). The manner in which target notes were used to link data to the maps also differed between surveys. The single target notes recorded in surveys A, C and E each gave a description of the field as a whole. In survey D, variation within the field was described by a number of target notes, and information therein was more closely related to their precise locations on the map (Table 11).

Table 11.  Target notes recorded in Field X in each survey. (The location of each target note is shown in Fig. 2; no target notes were made for this field in surveys B and F)
Survey A
Target note 1 –‘Predominantly acid grassland including Agrostis canina, Agrostis capillaris, Agrostis stolonifera, Anthoxanthum odouratum, Deschampsia cespitosa, Festuca spp., Nardus stricta and Poa trivialis. Interspersed with rushes including Juncus articulatus, Juncus conglomeratus, Juncus effusus and Juncus squarrosus with occasional Carex nigra. Mosses included Sphagnum sp. and Polytrichum commune. Herbs included Cirsium palustre and Potentilla erecta. Occasional patches of neutral grassland comprising Cynosurus cristatus, Holcus lanatus and occasional Lolium perenne. Herbs included Cerastium fontanum, Cirsium vulgare, Conopodium majus, Plantago lanceolata and Rumex acetosella. Ditches across eastern part of field lined by Juncus spp. and shallow fast flowing water. Remainder dry.’
Survey C
Target note 1Marshy grassland; semi-improved acid grassland; running water.‘Rush-pasture interspersed with acid grassland covers main part of field, including abundant Juncus effusus, J. acutiflorus, Holcus lanatus, Cynosurus cristatus, Trifolium repens, frequent Ranunculus acris, R. repens, occasional Equisetum palustre, Ranunculus flammula, Deschampsia cespitosa, Juncus articulatus, rare Climacium dendroides, Carex ovalis. Semi-improved grassland on northern slope with abundant Agrostis capillaris, Anthoxanthum odouratum, Cynosurus cristatus, Festuca rubra, frequent Holcus lanatus, Hypochaeris radicata, Juncus squarrosus, Nardus stricta, rare Poa humilis. Stream with slabs of rock in bed, Ranunculus flammula by the water but no associated rush-pasture.’
Survey D
Target note 1 – Semi-improved neutral grassland. ‘Rushy pasture. Remnant acidic grassland species present but sward dominated by neutral species due to enrichment from sheep-grazing. Species include: Festuca ovina, Cynosaurus cristatus, Bellis perennis, Cerastium fontanum, Holcus lanatus, Ranunculus repens, Trifolium repens, Lolium perenne, Cirsium sp., Rumex acetosa, Juncus squarrosus, Carex curta, Anthoxanthum odouratum, Juncus effusus, Deschampsia cespitosa and Poa spp.’
Target note 2 – Marginal vegetation. ‘Vegetation on steep slope alongside stream, dominated by Juncus acutiflorus and other Juncus species. Similar to marshy grassland in the next field to the east. Debatable whether should be classed as ‘marginal vegetation’ or ‘marshy grassland’ due to slope.′
Target note 3 – Semi-improved neutral grassland.‘Field with many new drainage channels. Large patches of Juncus effusus with occasional Urtica dioica. Locally enriched mosaic of acid and neutral grassland but more species-poor than stream-sides. Occasional patches of Juncus squarrosus and Nardus stricta but mostly Juncus effusus, Cirsium sp., Trifolium repens, Cerastium fontanum, Cynosaurus cristatus, Anthoxanthum odouratum, Poa sp. and Festuca ovina. Upper slopes have a greater proportion of Juncus spp., including Juncus acutiflorus, but these areas are not considered to be marshy grassland as they are scattered.’
Target note 4 – Semi-improved neutral grassland.Juncus effusus along stream-side. Large patch of Cirsium sp.’
Target note 5 – Marginal vegetation.Juncus effusus along stream-side.’
Survey E
Target note 1 –‘A semi-improved field with impeded drainage leading to a prolific growth of rushes, especially Juncus effusus with J. conglomerata, J. squarrosus and Cirsium palustre. Deschampsia cespitosa and Holcus lanatus are also abundant. J. acutiflorus present along the stream which runs through this field. Ditches also contain Polytrichum commune and Sphagnum species.’

The target notes for surveys A, C and D gave the impression that the field was a mosaic of different land cover types (Table 11). However, this was not reflected in the accompanying maps that recorded the surveyors’ perceptions of the dominant vegetation type (Fig. 2). Survey A, for example, mapped the field as ‘semi-improved acid grassland’, but noted that patches of ‘semi-improved neutral grassland’ were present. Survey C noted the presence of ‘semi-improved acid grassland’ within ‘marshy grassland’, but mapped only the latter. In Survey D the grassland was said to have affinities to ‘semi-improved acid grassland’, yet the entire field was mapped as ‘semi-improved neutral grassland’ (Table 11).

In each of surveys A, C and D, species listed as occurring in ‘acid’ or ‘neutral’ grasslands included a few given by the Phase 1 manual (Nature Conservancy Council 1990) as indicative of the opposite pH. For example, Juncus squarrosus (when dominant indicative of ‘acid’ grassland) occurred in ‘neutral’ grassland in survey D, while Deschampsia cespitosa (when dominant indicative of ‘neutral’ grassland) occurred in ‘acid’ grassland in survey A (Table 11). The absence of abundance data, however, prevents a retrospective interpretation of the appropriateness of the mapped land covers.

‘Marshy grassland’, which was mapped in three surveys, includes ‘vegetation with a greater than 25% cover of Juncus acutiflorus, J. effusus, J. inflexus, Carex species or Filipendula ulmaria’ but excludes ‘grazed Juncus effususHolcus lanatus/Deschampsia cespitosa grasslands, which should be classified under neutral grasslands, B2’ (Nature Conservancy Council 1990). None of the target notes stated the percentage cover of Juncus species, although these rushes were evidently conspicuous to surveyors C, D and E (Table 11). Survey D alone inferred the presence of grazing livestock, while surveys A, C and E also recorded that both H. lanatus and D. cespitosa were present (Table 11). Overall, the target notes suggested that some combination of ‘semi-improved acid grassland’, ‘semi-improved neutral grassland’ and ‘marshy grassland’ may have been appropriate for all or part of Field X. It is clear that most differences within this field involved similar vegetation types, as confirmed by the substantial improvement in agreement between maps after aggregation of related land cover types (Table 5). The remaining differences included small areas of ‘improved grassland’, ‘flush’ and ‘bracken’, which could arguably have been overlooked. The mapping of ‘poor semi-improved grassland’ by surveyor F was an exception, however. ‘Poor semi-improved grassland’ has affinities with ‘improved grassland’ and ‘semi-improved neutral grassland’, but differs from other semi-improved grasslands in lacking species indicative of underlying soil pH (Nature Conservancy Council 1990). Mapping of almost 80% of the field as ‘poor semi-improved grassland’ was difficult to reconcile with the target notes recorded in other surveys.

Survey effort

Five of the surveys were conducted by a single surveyor. Survey D involved two surveyors. Time spent on site varied from 17 to 24 h spread between 2 and 4 days (Table 1). Four measures of survey effort were the number of days, the number of hours and each of these multiplied by the number of surveyors. None of these measures was significantly correlated with cost of survey (P > 0·05 in each case). Neither cost nor any of the four measures of survey effort were significantly correlated with any of the following parameters used to characterize the surveys: number of target notes, number of land cover types, median number of species recorded per target note, or number of species recorded in total (P > 0·05 in each case).

The three surveys performed by members of the IEEM (A, C and D) were the three most expensive (Table 1). They also yielded the highest numbers of both target notes and numbers of species. In other respects, their results were similar to those produced by non-members (Table 9).

Pairwise comparisons of surveys generated 15 estimates of spatial agreement from map overlay analyses, and 15 estimates of similarity based on comparison of species lists using Sorenson's index (Tables 4 and 10). Those estimates of spatial agreement derived from pairwise comparisons in which both surveyors were members of IEEM (n = 3, x = 25·3%, SD = 6·4%) were not different from those derived from pairs in which one or neither were members (n = 12, x = 26·9%, SD = 6·8%) (t = –0·36, P = 0·75). Sorenson's similarity scores derived from the pairwise comparisons in which both surveyors were members of IEEM (n = 3, x = 59·03%, SD = 4·55%) however, were significantly higher than those derived from comparisons in which one or neither were members (n = 12, x = 39·49%, SD = 12·87%) (t = –4·29, P < 0·002).

Surveys were conducted over a period of 32 days from 1 July to 1 August 1996. The number of days between the mid-points of pairs of surveys was not correlated with either agreement derived from overlay of maps (rs = –0·318, P > 0·20, n = 15) or Sorenson's similarity based on comparison of species lists (rs = 0·424, P > 0·10, n = 15). There was therefore no evidence that temporal separation influenced either measure of similarity between pairs of surveys. There was also no correlation between spatial agreement and similarity of species lists (rs = –0·011, P > 0·50, n = 15).

Use of aerial photographs was noted in five survey reports (B, C, D, E and F), and photographs were borrowed overnight during surveys A and E. However, the extent to which photographs were used in map production could not be deduced. The similarity of maps A and E was unremarkable when compared with other pairs of maps (Table 4).

Problems reported in conducting the survey

Surveyors were asked to report problems encountered in completing the survey. Responses relating to generic mapping problems were (i) mapping of boundaries where vegetation graded between different types (two surveys); (ii) spatially referencing field observations to the base map in areas with an undulating land form and an absence of land marks (two surveys); (iii) mapping of complex habitat mosaics (one survey); and (iv) distinguishing closely related vegetation types using the habitat definitions in the Phase 1 manual (Nature Conservancy Council 1990) (four surveys). Combinations of land cover types that surveyors identified as being difficult to distinguish were (i) ‘semi-improved acid grassland’ and ‘semi-improved neutral grasslands’; (ii) ‘semi-improved neutral grassland’ and ‘marshy grassland’; (iii) ‘marshy grassland’ and ‘wet dwarf shrub heath’; (iv) ‘wet dwarf shrub heath’ and ‘mires’; (v) ‘heaths’, ‘modified bog’ and ‘unmodified bog’; and (vi) ‘marshy grassland’ and ‘marginal vegetation’ beside a stream.

Discussion

This paper examines the effect of observer variability on the precision of habitat mapping using the Phase 1 method, a technique that is widely used in the UK. An earlier study revealed a spatial agreement of only 44% between a pair of Phase 1 habitat maps of the same area produced in consecutive years (Cherrill & McClean 1995). A closer agreement was expected in the present study, which was conducted in the same region and with a similar range of habitat types, yet the maximum spatial agreement between any pair of the six maps examined was lower (Table 4). The only areas of habitat upon which all surveyors agreed were ‘improved grassland’ and ‘broadleaved semi-natural woodland’. However, even for these habitats, agreement between pairs of maps was in some cases less than 75% of their total areas in each map (Table 7). The poorest agreement was recorded for areas of rough grazing, which extended over more than 50% of the site (Figs 1 and 3). Agreement between surveys in an intensively managed lowland agricultural landscape might therefore be expected to be higher than recorded in this study. None the less, the results are a cause for concern because over 70% of the land surface of England has been mapped using Phase 1 survey (Wyatt 1991), yet there have been no previously published studies of the type reported in this paper (Cherrill & McClean 1995).

Two basic types of mapping error are possible in habitat mapping, namely misclassification of land cover types and spatial displacement of boundaries (Cherrill & McClean 1995). These types of error are difficult to distinguish because spatial errors can give the impression of classification errors where polygon boundaries drawn by different surveyors do not coincide. Inevitably there are difficulties in referencing field observations to the base map in upland areas with few land marks. Moreover, semi-natural vegetation types often merge across ecotones, making the precise identification of boundaries difficult. Buffering can be used to remove land lying close to the boundaries drawn by field surveyors, and in this study led to a significant increase in agreement (P < 0·0001). Vegetation within an ecotone is evidently more likely to be misclassified than that lying further away, but the improvement in agreement from buffering was relatively small (Table 4).

The limited improvement achieved through buffering is perhaps not surprising in view of the fact that the initial agreement between pairs of maps was on average only 25·6% (Table 4), and that there were large differences in the absolute areas of land cover types between maps (Table 2). The maps for Field X are typical of much of the study site in that there is little evidence that surveyors recognized similar gradients or discontinuities in species composition within fields (Fig. 2). Such large differences in boundary location are arguably best viewed as a secondary symptom of differing perceptions of the types of vegetation present, rather than spatial errors.

Aggregation of cover types increased agreement dramatically for Field X, and for the study site as a whole (Tables 4 and 5). However, despite the reduction from 35 Phase 1 cover types to 12 broad categories, the mean agreement achieved was still only 56·4% for the whole site (Table 4). Overall, four of the 12 broad cover categories (‘heathland’, ‘mires’, ‘pasture/silage’ and ‘rough grassland’) represented at least 90% of the study site (Table 2), emphasizing that there remained considerable confusion between even these broad categories of land. Notwithstanding the difficulties of separating spatial and classification errors, the results of this analysis support those of Cherrill & McClean (1995), who tentatively concluded that the most significant cause of poor agreement between Phase 1 maps was classification error caused by differing interpretations of the vegetation and Phase 1 land cover definitions.

The Phase 1 land cover definitions are in many cases vague, and can only be applied with a degree of interpretation. For example, the distinctions between ‘unimproved’, ‘semi-improved’ and ‘improved grasslands’ are unclear, and are described in relative terms by Nature Conservancy Council (1990). Thus, ‘unimproved grassland’‘should not have had sufficient applications of fertilizer or herbicide, or have been so intensively grazed or drained, as to alter the sward composition significantly’. In contrast, ‘improved grasslands’‘have lost many of the species which one could expect to find in an unimproved sward’ and have a bright green, lush sward dominated by agricultural grasses, with a low diversity of forbs. Only a handful of species are listed as indicators of each type of grassland, and the Nature Conservancy Council (1990) manual points out that these grasslands form a continuum ‘so that it is not possible to define each with precision’. It is perhaps not surprising therefore that separation of grassland cover types using the Phase 1 definitions was identified as a cause of concern in the surveyors’ reports. However, problems also occurred for other cover types, and the level of detail provided in the definitions of the grassland types is typical of the classification as a whole. Thus, the definition of ‘wet dwarf shrub heath’ makes general observations on the changes in species compositions to be expected along transitions to ‘mire’, but one of the few definitive observations is that the latter occur on peat more than 0·5 m deep. This is difficult to assess in the field over large areas, and changes in vegetation may not always reflect variation in underlying soils.

A further difficulty identified by surveyors, and identified in our analysis of their target notes, was the treatment of mosaics. The Phase 1 manual (Nature Conservancy Council 1990) gives little guidance on the scale of observation to be used. One type of ‘marshy grassland’ is defined as having greater than 25% cover of rushes, yet while these species may be locally dominant, they may be an insignificant component of the vegetation at the whole-field level. The identification of the cover type depends on the spatial resolution at which mapping is conducted. Similar difficulties arise in the mapping of ‘heathland’, which is defined as having greater than 25% cover of dwarf shrubs. Moreover, while the Phase 1 classification has a code for the mapping of dwarf shrub heaths as a mosaic with acid grassland, neither the relative contributions of dwarf shrubs and grasses, nor the appropriate scale of observation, is defined for the mapping of this cover type. More generally, the Phase 1 classification makes no provision for other types of mosaic, but these are encountered in the field (Table 11).

In this study surveyors were aware of the problems inherent in the Phase 1 approach but the results suggest that they greatly underestimated the degree of subjectivity involved. On a positive note, although many combinations of land cover types were confused, the most frequently occurring differences involved pairs of land cover types that were ecologically related. Indeed, detailed examination of the target notes for Field X suggests that the surveyors’ perceptions of the vegetation were more similar than might be assumed from comparison of the maps alone. Several surveyors noted the complex nature of the vegetation and the contribution of species typical of ‘acid’, ‘neutral’ and ‘marshy grasslands’ (Table 11). However, in drawing the maps, surveyors gave differing emphasis to each component (Fig. 2). This may have reflected differences in the routes followed by the surveyors as they walked through the field. It is very likely that most fields were traversed only once or twice, and that only a small proportion of each field was physically inspected by each surveyor. This may explain why small, but apparently conspicuous features, such as the areas of ‘improved grassland’ and ‘bracken’ recorded in Field X by surveyors C and E, were recorded by some surveyors but overlooked by others (Fig. 2).

The time spent carrying out a survey has been shown to influence the number of species recorded in floristic surveys (Woodell 1975; Rich & Smith 1996). In the present study, however, measures of survey effort based on time were not correlated with any aspect of the results, or surprisingly the cost of the surveys. None the less, the three most expensive surveys were conducted by members of a professional body (IEEM) and these surveys also recorded the largest numbers of species and target notes, suggesting that cost and professional status influenced the intensity of a survey (if not its duration). The three longest species lists were also the most similar in pairwise comparisons. This could be expected, because large samples taken from a finite species pool are by chance likely to share more species than small samples taken from the same species pool. It was notable, however, that in terms of spatial agreement, surveys conducted by members of IEEM were no more similar than other surveys. Without ‘ground-truth’ against which to assess the accuracy of the vegetation maps, it is impossible to assess which survey represented best value for money. The authors did not undertake a seventh survey, because it would have been difficult to prove it was more accurate than those presented here.

A characteristic of Phase 1 survey that distinguishes the approach from others, and which may make the method particularly susceptible to observer error, is the extent to which mapping decisions are made by the surveyor whilst in the field. In the phytosociological tradition of continental Europe, boundaries are identified in the field but detailed floristic data are also collected (Poore 1955; Kuchler 1967). Moreover, where an extant classification is applied, the vegetation types are described in detail as floristic tables, and allocation of new field data within the existing framework can be assisted by statistical analysis (e.g. Hill 1989).

An alternative approach, used widely in North America and Canada, is to locate boundaries in a preliminary desk study, using existing maps and remote sensed data (aerial photographs or satellite imagery) showing land form, land use, soils, geology and vegetation physiognomy to identify parcels of uniform cover and/or environmental conditions (Bajzak & Roberts 1996; Banner et al. 1996; Smith & Carpenter 1996). These analyses rely heavily on use of GIS (Uhlig & Jordan 1996). Field work involves visiting parcels to describe the environmental conditions and vegetation in detail. The type of data collected depends on the nature of the classification within which parcels are to be assigned (Kuchler 1988b). However, field work is characterized by the careful documentation of data for later analysis. In this respect the approach is similar to that of continental plant ecologists, and data collected may include detailed phytosociological records (Hanson & Hargrave 1996).

In comparison to Phase 1 mapping, the two broad approaches described above each place greater emphasis on the detailed recording of data in the field, and its subsequent analysis as part of a desk study. Less emphasis is placed on the surveyor making decisions in the field, and careful documentation of field observations makes the process of classification more transparent and amenable to retrospective analysis. The GIS-based approach emphasizes the role of desk studies prior to field work, while an important feature of the phytosociological approach is the use of detailed floristic tables to define vegetation types.

The following observations and recommendations are made with regard to the Phase 1 survey.

1. The Phase 1 manual recommends that surveyors train in groups and work in pairs during large-scale surveys. Revisiting sample areas to check mapping accuracy is also advised. These are sensible suggestions, but they cannot be applied readily to many consultant ecologists who frequently work alone on relatively small projects. None the less, consultancy companies should ensure that their employees are well-qualified and undergo training to maintain field skills and keep abreast of best practice.

2. The use of geological and soils maps is already recommended in the Phase 1 manual (Nature Conservancy Council 1990). Their use should increase. The use of aerial photographs for preliminary habitat classification and drawing of habitat boundaries as part of a desk study are also currently recommended (Nature Conservancy Council 1990) but should become mandatory. Field surveyors should be supported by staff trained in air photograph interpretation.

3. The definitions of Phase 1 land cover types should be expanded and clarified to include more detailed floristic data (particularly with regard to the abundance of species in different vegetation types). Further guidance on the scale of observation is needed for the mapping of mosaics. The range of mapping classes might also be increased to include a wider range of mosaic types.

4. Field surveyors should make at least one target note for every parcel mapped. All target notes should include floristic data, information on land use and management practices, and also comment on the extent of spatial variation within the parcel. Floristic data should as a minimum list all dominant, abundant and frequent species (although these terms require further definition, see point 3 above) and include estimates of abundance for all species recorded.

5. Preliminary field-based assignment of parcels to a land cover type should be confirmed by more detailed analysis of target notes by experienced office-based staff working in collaboration with surveyors, thereby reducing reliance on decision-making in the field.

6. Implementing our recommendations would increase significantly the cost and duration of Phase 1 surveys. However, unreliable surveys are of limited practical use and also undermine the credibility of others. Further research is required to investigate the most effective way of improving the quality of vegetation mapping: our recommendations are untested.

Acknowledgements

Thanks are due to the consultants, the staff at Redesdale Experimental Husbandry Farm, the University of Sunderland for funding, Dr P. Eady and three referees for helpful comments. Ordnance Survey maps were copied under licence.

Received 29 July 1998; revision received 24 August 1999

Ancillary