Museum collections are reservoirs of past and present biodiversity (Brooke 2000; Guralnick & Cleve 2005; Lister et al. 2011). Modern genetic methods are increasingly applied to museum specimens, providing for example, insight into phylogenetic placement of recently extinct species (Cooper et al. 2001; Shapiro et al. 2002), DNA sexing that clarifies taxonomy (moa; Bunce et al. 2003), composition of population samples (moorhen, Lee & Griffiths 2003), adaptive allele frequencies (blowfly, Newcomb et al. 2005) and specimen provenance (penguin, Boessenkool et al. 2010). Isotope and trace-element analysis of teeth, feathers and other tissues has similarly been applied to museum specimens to elucidate ontogenic movements and dietary shifts (e.g. sperm whales, Mendes et al. 2007; seabirds, Norris et al. 2007), migratory patterns (Hobson et al. 2010), palaeoenvironmental change (Newsome et al. 2010; Uno et al. 2011) and responses to environmental change such as the transport of contaminants (Horton et al. 2009; Vo et al. 2011). Recent developments have extended analysis of museum specimens to include historical artefacts made from biological materials and thus probe history (Hartnup et al. 2011; Foley et al. 2012; Rogers et al. in press). There is now potential to repatriate important cultural artefacts and human remains, and reveal linkages between biodiversity and culture (Clarke et al. 2006). Such applications, however, require that sampling is applicable to the question at hand. Studies of both natural history specimens and historical artefacts are sensitive to sampling biases that distort inference from even the most sophisticated downstream analysis. A recent study on Māori feather cloaks held in museum collections throughout the world illustrates the need to carefully contextualize molecular data alongside other ecological or ethnographic evidence. Using this case study, we highlight three major areas of concern applicable to studies of museum artefacts and specimens: sampling biases (museum policies, collector effort and data accuracy); reference data sampling and subsequent analysis (mismatched sampling, data pooling); and use of historical evidence (parallel analyses, historical documents). We discuss some possible solutions.
Museum collections are increasingly subjected to scientific scrutiny, including molecular, isotopic and trace-element analyses. Recent advances have extended analyses from natural history specimens to historical artefacts. We highlight three areas of concern that can influence interpretation of data derived from museum collections: sampling issues associated with museum collection use, methods of analysis, and the value of cross-referencing data with historical documents and data sets. We use a case study that focuses on kiwi (Apteryx spp.) feather samples from valuable 19th century Māori cloaks in New Zealand to show how sampling and analysis challenges need to be minimized by careful design. We argue that aligning historical records with scientific data generated from museum collections significantly improves data interpretation.
Provenance of biological material in artefacts
We begin with the construction of hypotheses for studies using artefacts. Without known provenance, the scientific and historical value of any artefact is severely diminished. Yet it cannot be assumed that artefacts and the materials from which they are made have the same origin. The case study of interest (Hartnup et al. 2011) concerns two New Zealand icons, one biological and the other ethnological, and the very special cultural linkage between them. The study describes the use of ancient DNA methods to recover short mitochondrial hypervariable region one (HVRI) DNA sequences from 849 kiwi (Apteryx spp.) feathers, subsampled from 109 kahukiwi (cloaks) held in museum collections in England and New Zealand. These prestigious cloaks were produced by indigenous Māori weavers in New Zealand using fibres from endemic plants and the feathers of the endemic, flightless kiwi (Apteryx spp.) in the 19th century (Pendergrast 1987). Hartnup et al. used this sampling to develop a ‘new understanding of the methods used by Māori to collect materials for cloak construction, as well as an understanding of the traditions of cloak making itself’. A primary conclusion was that the eastern North Island was the most prolific of cloak-making areas in 19th century New Zealand. Is this a valid inference from the data, and more importantly is this a meaningful interpretation of the history and traditions of cloak weaving? It is unclear at the outset whether the authors wish to infer kahukiwi weaving locations using kiwi DNA from feathers in the cloaks (which implies that feathers were collected from kiwi and woven at the same location) or to propose an alternative hypothesis of regional trading and exchange of materials or kahukiwi themselves, as described in oral history (in which case feathers cannot be used as indicators of weaving location). The assumption that the biological materials used to create an artefact also reflect the production location might be wrong. A carefully designed sampling strategy is essential to uncover historical weaving locations.
Sampling bias in museum collections
Any circumstance that causes a sample to be unrepresentative of the underlying population cannot provide a reliable basis for hypothesis testing. Typically, some form of randomized sampling is used (Southwood 1976) but there are several reasons why museum collections can fail in this respect. The composition of museum collections is dictated by collecting strategies that have often been, in the past at least, opportunistic and nonrepresentative (Pyke & Ehrlich 2010), based on the voracity of collectors with diverse personal objectives. Even current acquisition strategies are frequently not systematic if, for example, ‘…. acquisitions are made to maintain and improve... [the]… cultural and historical record of the world’s cultures and civilizations, and generate public interest in the past and present’ (British Museum Policy on Acquisitions 2011).
Many researchers recognize that museum collections suffer from the limitations of presence-only data, but do not recognize that understanding the reasons for data absence is critical. Absence of an artefact might reflect a true historical absence at a particular location, imperfect detection, failure to collect in that area (collector bias) or subsequent loss/damage of specimens. Imperfect detection occurs where objects or species are present but not detected by observers, for example where species are rare or cryptic, or sacred artefacts are not revealed or sold to collectors. More importantly, collections of natural history specimens and artefacts such as kahukiwi reflect the nonrandom interests, geographic movements and history of individual collectors and are thus frequently assembled in a nonrepresentative manner. This type of collector bias can also result if items are sought specifically for their rarity, quality or accessibility (Guralnick & Cleve 2005). Collector bias thus leads to the spatial biases frequently seen in museum and herbaria collections which have, for example, an over-representation of records that map accessible sites on road and river networks (Margules et al. 1994; Guralnick & Cleve 2005) or an over-representation of rare, and dearth of common, species. In the case of the kahukiwi study, although the number of kahukiwi sampled appears large (n = 109), the sample will be spatially biased if it reflects uneven collecting effort. Our information searches suggest that in New Zealand, many Māori artefacts and natural history specimens held in museums were obtained from a limited pool of late 19th and early 20th century European collectors (Appendix S1, Supporting information). These collectors were often associated with specific tribal groups with whom they had long-standing relationships. For example, Elsdon Best, a long-time staff member of the Dominion Museum (Wellington, New Zealand), spent years recording tribal history in the eastern Bay of Plenty (Best 1898, 1908), and collected natural history specimens and artefacts there. Best sent at least one feather cloak from this region to the museum (Evening Post, 29 May 1899; Tamarapa 2011). The kahukiwi deposited in museums thus reflect the history and affiliations of the collectors rather than regional intensity of weaving activity; a thorough examination of provenance (if known) can lead to a better understanding of this bias.
Gaps in artefact collections can also be better understood if placed in an historical context. For example, land closures during the 1845–1872 land wars between Māori and government troops led to 3.5 million acres of western North Island being inaccessible to Europeans (Belich 1986). Ornithologist Walter Buller finally obtained permission from the Māori king to hunt kiwi in the Waikato in 1882 but noted that, ‘owing to our strained relations with the ‘King Party’, no European had been admitted into this part of the country for many years’ (Buller 1888). Nonetheless, evidence from tribal archives indicates that Waikato leaders wore kahukiwi in the 19th century, so collector bias was probably a significant cause of artefact absences in this case (Fig. 1A). Sampling kahukiwi held in private collections (that often have well-known provenance based on oral histories) might address data absence issues from this region, and allow a test of the hypothesis that the eastern North Island was the centre of kahukiwi weaving in early New Zealand.
Other unquantified sources of error can arise for the most assiduous of researchers. Curators may mislabel specimens, donors provide incorrect information and curators can modify artefacts by repairing them. Errors in transcription and relabelling, confusion of records during storage and even fabricated information (Lee & Griffiths 2003; Boessenkool et al. 2010) can mislead. Curation practices such as the preparation and modification of material from museum specimens and artefacts can affect sampling procedures. Older biological material such as bird skins may have initially been preserved in arsenic (potentially limiting the quality of DNA extracted; Payne & Sorenson 2003), produce erroneous data owing to degradation (Staats et al. 2011) or have high levels of inorganic mercury (Vo et al. 2011). Weavers mend feather cloaks, and reuse feathers from damaged cloaks (Davis 2007). For instance, a former curator at Wanganui Museum (where many kahukiwi were sampled) reported in 1901 that a moth-damaged peacock feather cloak in the museum collection was mended by adding feathers (Wanganui Chronicle, 1901). We caution that practices carried out by collectors, artefact makers or museum curators cannot be ignored, and we need instead to investigate analytic techniques that accommodate the likelihood of nonrepresentative data.
Nonindependence of presence data in museums can also influence interpretations, where multiple samples are collected from a single location but the replication is not recorded. In New Zealand as elsewhere, curators travelled the country to obtain museum specimens of fauna and flora and cultural artefacts, before exchanging these with colleagues to build up collections (Appendix S1, Supporting information). Similarly, curators sometimes purchased items from commercial dealers with well-established sources [e.g. from sellers of Māori ‘curios’ (Day 2005)]. The inclusion of specimens for analysis selected from museums at different geographic locations does not ensure sampling independence, and can instead yield artificial inflation of sampling from specific districts, with consequences for subsequent data analysis. This difficulty can also apply to modern data sets that have been compiled in a nonrepresentative manner.
Historical and reference data sampling
Assembling DNA reference data that meaningfully assess historical populations is challenging, particularly where modern populations are reduced in size and range. Modern reference data can have fragmentary spatial coverage that is mismatched with historical distributions (Guralnick & Cleve 2005; Fig. 1A). In the case study, Hartnup et al. (2011) obtained a reference set of modern kiwi DNA sequences with which to compare the kahukiwi data and thus identify likely provenance for kiwi feathers. To identify the spatial origins of kiwi feathers from kahukiwi using DNA sequence data, a sample that accurately represents the potential sources of material is required. Kiwi today are endangered birds, localized in distribution. However, fossil remains indicate kiwi were once widespread (Robertson 2003; Fig. 1). Prior to intensive European settlement in the 19th century, kiwi ranged throughout New Zealand, only becoming seriously threatened following the population expansion of introduced European dogs, pigs, rats and mustelids (King 1984). Thus, it is not possible to sample the distribution of genetic diversity of kiwi during the time that kahukiwi were being made. Nonetheless, comparisons of feather DNA from museum natural history specimens with those obtained from artefacts as well as modern sampling can extend knowledge of haplotypes in time and space.
The modern kiwi reference data set used by Hartnup et al. (2011) did not include data from the western and southern North Island (including the Waikato area; Fig. 1A). It is therefore not possible to accurately infer the origin of kiwi feathers woven in kahukiwi. It is only possible to identify which of the modern reference sites that have been sampled are more likely sources. Although one mitochondrial HVRI haplotype (haplotype 8) was dramatically overrepresented in kahukiwi (36%), as compared to modern population reference samples (2.1%), bias in museum accessions and/or modern sampling could be responsible (Fig. 1B). This mismatch was inappropriately interpreted as a change in haplotype frequency over time, even though the chi square test used suggests only a difference between the haplotype frequencies in the two samples (kahukiwi and modern kiwi population). Attempting to interpret the chi square statistic as a change in population genetics is not only dubious but confounds the objective of the study, which is to show that feathers for kahukiwi were drawn from a subset of the spatially distributed kiwi population genetic diversity.
Pseudoreplication through repeated sampling of the same artefact can lead to inaccurate estimates of haplotype frequencies. Related individuals in artefacts and modern populations respectively will skew estimates of haplotype frequency (Fig. 1B). Furthermore, identifying the origin of biological material does not imply that artefacts have a similar origin, although comparisons of feather and cloak origins for feathers from kahukiwi with known provenance could provide a test of this hypothesis. In the kahukiwi data, feathers used in the construction of a single cloak were estimated to have been derived from more than one geographic area in 15% of cases. Precluding substantial kiwi population shifts, Hartnup et al. (2011) plausibly suggest either a trade in feathers or hunting trips to different parts of the country. Trade in other resources used in cloak making, such as flax (Phormium tenax), was well established in the North Island (Wehi 2006), so analysing flax and feather data from the cloaks together could illuminate trade patterns (Harwood 2011). For instance, parts of the eastern Bay of Plenty with rich feather resources (such as Ruatāhuna, in the midst of lowland forest with kiwi habitat) were famously recognized as unsuitable for growing flax (Wehi 2009).
Records of likely provenance, cultural traditions and historical context can often be identified for artefacts. Complementary historical and ethnographic research can help estimate the importance of caveats associated with survey and sampling bias. Nonetheless, reference to historical events that are peripheral to the question under study also deflects appropriate interpretation of data. For example, it is unlikely that early 19th century wars in New Zealand had a major impact on kahukiwi weaving, as implied by Hartnup et al. (2011), given that feather cloak weaving probably reached its zenith in the late 19th century (Pendergrast 1987). Close examination of provenance details for individual specimens or artefacts can help establish the extent of nonindependence. For example, we found that at least half of 16 kahukiwi sampled from Hawke’s Bay museum were gifted by members of a single family (Table S1, Supporting information). Similarly, all four kahukiwi sampled from the Waikato Museum were woven by members of one family. In addition, these four kahukiwi are of late 20th-century origin (D. Pike, pers. comm.), and their inclusion in a study of 19th-century kahukiwi is puzzling. Their age highlights the long-term and heterogeneous nature of museum collections.
Museum specimens and artefacts are typically far removed from their geographical and cultural context. Nevertheless, publicly available historical records can usefully verify known details including provenance, and reveal other details. We searched the online New Zealand National Library Collection of English language newspapers, ‘Papers Past’ (http://paperspast.natlib.govt.nz/cgi-bin/paperspast), between 1835 and 1945 for commentaries on kahukiwi. This investigation exposed information on 19th-century weavers, potential locations for feather cloak making and ceremonial events where kahukiwi were exchanged or gifted. Other historical sources such as parliamentary records and personal journals of collectors can be usefully accessed from library archives.
Increasing the reliability of conclusions
So how can the reliability of estimates and conclusions be enhanced? Design of appropriate hypotheses that can be tested using the available specimens or artefacts is paramount, and a number of strategies can help limit uncertainty in conclusions. Careful morphological examination of specimens can provide solutions in some cases: for example, comparisons between ancient DNA data and morphological data increased confidence in results that assigned provenance to a purported Cape lion (Barnett et al. 2007). It is also clear that investing time in verification of specimen data can increase the reliability of findings. Boessenkool et al. (2010) presented a case in which genetic analyses of yellow-eyed penguins revealed previously unsuspected inaccuracy in locality and subspecies assignment. Mistakes of this type in locality data could confound inferences of historical population connectivity, or effective population size estimates. Boessenkool et al.’s molecular data indicated incorrect assignation of individual specimens, supporting a mainland New Zealand, rather than sub-Antarctic, origin for eight penguin specimens. Boessenkool et al. then identified nonindependence in these data; all eight specimens were collected by one collector (H.H. Travers) and held at the same museum (the American Museum of Natural History). Historical records and field notes from sub-Antarctic voyages were then examined to assess the likelihood that collectors had visited the sub-Antarctic islands around the collection dates on specimen labels. This investigation corroborated genetic evidence that the eight yellow-eyed penguin specimens under scrutiny were unlikely to have originated from the sub-Antarctic Islands. More generally, this inaccuracy led Boessenkool et al. to suggest that all museums specimens deposited by this collector should be carefully examined.
Verification can use data from historical distributions of other specimens or species that are in some way related to the question to assist interpretation. Horton et al. (2009) were limited by a lack of historic data in their study of mercury contaminants and stable isotopes in polar bears through time. The data were therefore divided into two primary time periods (pre and post 1950). Horton et al. justified this decision based on abrupt changes in Hg levels observed in other species sampled around this time period, which probably reflect increased anthropogenic inputs of Hg to the atmosphere. In addition, Horton et al. removed four outliers from their data set, as they considered these museum samples were probably affected by the preservative mercuric chloride. Finally, Horton et al. consolidated their findings by compiling a data set of stable isotope and contaminant values in other high latitude species across historic and modern times with which they could compare their data. Other researchers have similarly used historical records of ecological parameters such as water quality and contamination, changes in ecosystem size and food web structures to inform studies of species decline (Hobson et al. 2010).
In many cases, conclusions about past geographical distributions of species can benefit from the addition of historical evidence. Possible sources of information include fossil records and oral histories as well as written notes by early naturalists. Ross et al. (2006) tested hypotheses of greater prairie chicken population expansion, important for conservation management, using genetic data from museum specimens. Early historical records that identified prairie chicken populations inhabiting the plains before agricultural modification by settlers added support to their conclusions (Ross et al. 2006). Similarly, a stable isotope study of historic shifts in bald eagle diet used multiple sources of data to verify conclusions (Newsome et al. 2010). These researchers excavated faunal remains from a historically occupied nest to reconstruct bald eagle diet at one site, and researched historical documents to verify changes in land use and history, such as sheep ranching, that might also impact diet and the local abundance of potential prey.
However, historical evidence may add complexity to data interpretation. For instance, using the kahukiwi example, the conclusion that little spotted kiwi (Apteryx owenii) were rare but present on New Zealand’s North Island during the feather cloak-making period because a little spotted kiwi feather was present on one cloak (Hartnup et al. 2011) is contentious when all the evidence is examined. Little spotted kiwi previously inhabited the mainland of New Zealand but are now restricted in distribution to the extreme south of the New Zealand mainland and a few offshore islands (Colbourne 2005). However, little spotted kiwi also occurred on D’Urville Island in Cook Strait, between the North and South Islands. This island was well known to Māori during the 19th century (MacKay 1859; Colbourne 2005). The little spotted kiwi feather identified in the kahukiwi could alternatively have originated from feather trading amongst Māori or the movement of an individual weaver. Historical evidence in this case adds a layer of complexity that could contradict the original conclusion.
There are likely to be many more molecular and isotope studies of artefacts and natural history specimens in the near future. We applaud the new and interesting efforts to scientifically verify artefact materials and thus extend knowledge of their provenance and associated cultural traditions. Nonetheless, we emphasize that many studies using museum collections, including the study highlighted here, would benefit from careful sampling design, clear statements about the limitations of the available data, and confirmation of related cultural and ecological details that may impact findings. We caution against the assumption that identification of provenance for biological material equates to identification of artefact construction sites. Potential steps to illuminate this issue might include analysis of geographic origins of other biological material found in the same artefact. It has been suggested that molecular and isotopic approaches have the potential to discover a wealth of lost information from cultural artefacts worldwide (Hartnup et al. 2011). We agree, but conclude that thorough consideration of post hoc sampling issues, and sampling and source data distributions, is vital to attaining robust results. Scientific findings that address the knowledge of indigenous peoples can have serious cultural, tribal and political impact, as indigenous peoples worldwide seek to retain and rediscover highly valued cultural traditions (Stumpe 2005). Complementary ecological and historical research can corroborate important results inferred from studies using museum collections, and indeed add new insights. Such rigorous methodology will benefit both science and culture.
PMW is supported by FRST postdoctoral fellowship MAUX0905. Emma Bean of the National Kiwi Trust, Rainbow Springs Kiwi Encounter, Rotorua, provided data from the kiwi parent database. We thank T. Cracknell (Hawke’s Bay Museum and Art Gallery), A. Holloway (Horniman Museum), D. Pike (Waikato Museum), R. Te Kanawa (Te Papa) and P. Nugent-Lyne (Whanganui Regional Museum) for their interest and assistance. David Penny and Phil Battley made useful comments on an earlier draft of the manuscript.
P.M.W. is a postdoctoral fellow using stable isotope analysis to answer ecological questions. She currently works on endemic New Zealand crickets (tree weta), but also has an active interest in conservation biology and TEK of indigenous peoples. H.W. is a linguist interested in discourse analysis, ethics and the digitization of indigenous knowledge. S.A.T. addresses questions in evolutionary ecology and genetics, using New Zealand invertebrates and birds, with his group at Massey University http://evolves.massey.ac.nz/. His research questions are generally centred on the evolution of species.