Laboratory studies on surface sampling of Bacillus anthracis contamination: summary, gaps and recommendations

Authors


Correspondence

Greg F. Piepel, Pacific Northwest National Laboratory, PO Box 999, Richland, WA 99352, USA. E-mail: greg.piepel@pnnl.gov.

Summary

This article summarizes previous laboratory studies to characterize the performance of methods for collecting, storing/transporting, processing and analysing samples from surfaces contaminated by Bacillus anthracis or related surrogates. The focus is on plate culture and count estimates of surface contamination for swab, wipe and vacuum samples of porous and nonporous surfaces. Summaries of the previous studies and their results were assessed to identify gaps in information needed as inputs to calculate key parameters critical to risk management in biothreat incidents. One key parameter is the number of samples needed to make characterization or clearance decisions with specified statistical confidence. Other key parameters include the ability to calculate, following contamination incidents, the (i) estimates of B. anthracis contamination, as well as the bias and uncertainties in the estimates and (ii) confidence in characterization and clearance decisions for contaminated or decontaminated buildings. Gaps in knowledge and understanding identified during the summary of the studies are discussed. Additional work is needed to quantify (i) the false-negative rates of surface-sampling methods with lower concentrations on various surfaces and (ii) the effects on performance characteristics of: aerosol vs liquid deposition of spores, using surrogates instead of B. anthracis, real-world vs laboratory conditions and storage and transportation conditions. Recommendations are given for future evaluations of data from existing studies and possible new studies.

Introduction

In 2001, letters containing Bacillus anthracis (BA) contaminated areas of government office buildings in Washington, DC and postal facilities that processed the letters. A congressional inquiry and Government Accountability Office (GAO) investigation (Government Accountability Office 2005a,b) identified two main concerns regarding the methods used to characterize and clear buildings. One main concern was the reliance on sampling specific locations where it was thought BA would be found. This type of sampling approach is referred to as judgmental (or targeted) sampling. The GAO reports identified the need to use statistically based sampling so that when all results are negative, a building (or area within a building) can be cleared with a specified level of statistical confidence. The second main concern was that methods used in the steps of the sampling and analysis process (SAP) (i.e. sample collection, storage/transportation, processing/extraction and analytical) were not validated. The lack of validated methods raised questions about the reliability of the negative sampling results obtained.

Several organizations are working to address the GAO concerns, including the U.S. Department of Homeland Security (DHS), U.S. Environmental Protection Agency (EPA), Centers for Disease Control and Prevention (CDC), National Institute of Standards and Technology (NIST), U.S. Department of Defense (DoD), Federal Bureau of Investigation (FBI), Pacific Northwest National Laboratory (PNNL) and Sandia National Laboratories (SNL). Some key activities to address the GAO's main concerns are the following:

  1. Developing a sampling guidance document that describes appropriate uses of judgmental and statistical sampling approaches
  2. Validating methods associated with the SAP
  3. Developing formulas to estimate the BA contamination, quantify the bias and uncertainty of the estimate, calculate the confidence in characterization and clearance decisions, and calculate the number of samples necessary to achieve the desired uncertainty and confidence in such decisions.

For conciseness in subsequent discussions, these are referred to as Activities A, B and C.

Activities B and C require quantitative information on the performance of sampling and analysis methods for indoor surface samples. Such information can be obtained from laboratory and field studies, although laboratory studies are best suited for certain performance measures because the level of contamination can be better controlled. Hence, we identified 20 previous laboratory studies that were conducted to assess the performance of swab, wipe, and vacuum sample collection methods and subsequent sample storage/transportation, processing/extraction and analysis methods for surfaces contaminated by BA or surrogates. We summarized the test conditions and results of the studies and assessed whether there were any substantive gaps. The summary information, gaps and needs that were found, and recommendations made to address the gaps and needs, are presented and discussed in this article. Edmonds (2009) also reviewed the literature and discusses difficulties, needs and perspectives for future work. Later, in this article, we compare our results with those of Edmonds (2009).

The next section discusses how Activities B and C motivated the work discussed in this article. The following section identifies the previous laboratory studies that were considered in this effort and summarizes their characteristics and results. The final section discusses the gaps and needs that were identified and makes several recommendations for future studies and evaluations.

Motivation for summarizing and assessing the gaps of previous laboratory studies with Bacillus spores

The motivation for summarizing the performance characteristics of swab, wipe and vacuum surface-sampling methods was to use the results as inputs to Activities A, B and C. Activities A and C involve developing sampling strategies, approaches and formulas to provide high confidence in the results of characterizing building contamination and clearing an uncontaminated or decontaminated building for reoccupation. Statistical and combined judgmental and statistical (Sego et al. 2010) sampling approaches provide for calculating the number of samples necessary to achieve the desired confidence for detecting contamination and clearing uncontaminated or decontaminated buildings. These approaches also provide for calculating the confidence in characterization or clearance decisions based on sampling results following contamination incidents. Well-quantified performance characteristics are required to select sampling approaches and methods, as well as perform the calculations mentioned.

Activity B involves validating methods in the SAP for BA contamination in buildings. Validating a method consists of several actions (ISO/IEC 2005), two of which include (i) quantifying the uncertainty (repeatability and reproducibility) and accuracy of results obtained by the method and (ii) quantifying relevant performance metrics, such as recovery efficiency (RE), false-negative rate (FNR), false-positive rate (FPR) and limit of detection (LOD). Hence, it is important to understand how much of this type of information exists for the previous laboratory studies that investigated sampling of surfaces contaminated with BA or surrogates. This article summarizes estimates of the above performance metrics and corresponding uncertainties reported in the references that document the studies.

The FNR plays an important role in Activity C, because a higher FNR will reduce the confidence in contamination detection and clearance decisions, which can be offset by taking more samples. Further, the overall RE of a sampling process contributes to determining the overall FNR. Here, ‘overall’ means over all steps in the sampling process: sample collection, storage/transportation, processing/extraction and analysis (e.g. culturing and counting). Having sufficient experimental data to determine the overall RE and FNR (and their uncertainties) for swab, wipe and vacuum samples on various surfaces at various levels of contamination (and any other influencing factors) is important in assessing risks (of failing to detect spores or erroneously clearing an area) and in developing sampling strategies and approaches.

Standard statistical formulas assume that the overall FNR = 0 when calculating (i) the number of samples required to achieve the desired confidence for a characterization or clearance sampling goal and (ii) the uncertainty and confidence associated with a characterization or clearance decision using a specific sampling approach implemented following a contamination incident. When FNR = 0, the formulas account only for the uncertainty in results associated with the specific type of statistical or hybrid sampling approach being used. However, the overall FNR is affected by the concentration of the contaminant and anything in the sampling process that might yield a false negative, including (i) the RE of a sampling method (e.g. swab, wipe or vacuum), (ii) the RE of storage or transportation steps, if applicable, (iii) the RE of the processing/extraction step (i.e. extracting the contaminant from the sample medium) and (iv) the uncertainty of the analytical/detection method and equipment. Hence, it is important in laboratory studies to quantify the REs and uncertainties affecting the results of a method at each step in the process, so that the overall RE and overall FNR can be determined.

Standard statistical formulas for calculating (i) and (ii) in the previous paragraph can be extended to address situations in which the overall FNR > 0. The Visual Sample Plan (VSP) software (VSP Development Team 2010) implements many statistical- and nonstatistical-sampling approaches and methods for environmental sampling. VSP generally implements standard formulas to calculate the numbers of samples to address situations in which the overall FNR = 0. Extended formulas (applicable when the overall FNR > 0) have only been developed for a statistical, grid-sampling approach for detecting contaminated areas of specified size (so-called hotspot sampling). PNNL plans to develop extended formulas for calculations (i) and (ii) when FNR > 0 with other statistical-sampling approaches recommended in a multiagency sampling strategy document (not yet completed). Adequate information on the FNR performance of sampling and analysis methods is needed as input to this work.

Previous study data indicate the overall RE and FNR may depend on the surface concentration of BA contamination, how the contamination is deposited on a surface, the surface material, the specifics of a sampling method and possibly other factors (references cited in the first paragraph of the following section). Hence, laboratory studies should investigate the performance of the SAP for the most commonly used sampling methods over a range of BA (or surrogate) surface concentrations, different deposition methods, different surface materials, etc. In fact, several such studies have been performed, but important questions include whether the studies adequately (i) cover the combinations of factors that may affect results and (ii) quantify performance measures of the methods and conditions studied.

Summary of anthrax-related laboratory studies

Previous laboratory studies were identified that used (i) swab, wipe and vacuum-sampling methods to collect BA (or surrogate) contamination from various porous and nonporous surfaces and (ii) culture and count methods to quantify the contamination. Only BA/surrogate contamination and count methods were considered to keep the scope of the effort manageable. We focused on laboratory studies (as opposed to field studies or data from BA contamination incidents) because they provide a basis for quantifying actual contamination levels, and using those values to estimate REs and other method performance metrics such as FNR. The 20 studies included in our summary and gap assessment are: Buttner et al. (2001, 2004a,b), Rose et al. (2004), Hodges et al. (2006), Nellen et al. (2006), Brown et al. (2007a,b,c), Quizon et al. (2007), Almeida et al. (2008), Frawley et al. (2008), Montgomery and Camp (2008), Valentine et al. (2008), Edmonds et al. (2009), Estill et al. (2009), Hodges et al. (2010), Einfeld et al. (2011), Krauter et al. (2012), and Rose et al. (2011).

The characteristics and results of relevant tests in each study are summarized in four tables, each with ‘a’ and ‘b’ parts. The ‘a’ part of each table summarizes the characteristics of the studies, while the ‘b’ part summarizes the results of the studies. The ‘b’ parts of tables are further divided into three categories of information: (i) Recovery concentration results – Mean and %RSDs (ii) Recovery efficiency (RE) – Mean and %RSDs and (iii) LOD, FNR and FPR. All per cent relative standard deviation (%RSD) values are rounded to zero decimal places. Table 1 describes the study characteristics and results in the columns of the summary tables. In each table, different studies are represented by groups of rows, with rows corresponding to the tests performed within a study. Because the four tables that summarize groups of tests and results are long, they are not included in this article. They are included in the Appendix of Piepel et al. (2011) as Tables A.1 (swab sampling), A.2 (wipe sampling), A.3 (vacuum sampling) and A.4 (one storage/transportation study that did not involve surface sampling). Condensed summaries of Tables A.1 to A.4 are presented in this article as Tables 25, with ‘a’ and ‘b’ parts as described previously. Acronyms and abbreviations used in Tables 25 are defined in Table 6.

Table 1. Descriptions of columns in tables summarizing the characteristics and results of laboratory studies on surface sampling of Bacillus anthracis and surrogates
Study characteristics (in Tables 2a, 3a, 4a and 5a)
ReferenceLead author (Year) citation of publication that documents the study
Test no.A number of the form x.y where x is 1, 2, 3, … for each study, and y = 1, 2, 3, … for the tests within a study
AgentB. anthracis or related surrogate
Agent depositionMethod used to deposit agent on test material
Agent concentrationConcentration or amount of agent on surface
Swab/Wipe/Vacuum typeType or material of sample collection medium
Wetting agentSwab or Wipe: Liquid, if any, used to wet or premoisten the sampling material
Vacuum techniqueVacuum: Technique in vacuuming (e.g. pattern and pressure applied)
Relative humidityPer cent relative humidity in chamber or laboratory during testing
Surface typeMaterial type of surface sampled
Surface area sampledArea of surface sampled
Extraction liquidLiquid used to extract spores from the sample
Extraction methodMethod used to prepare a sample and extract the contaminant
Culture method/mediumMethod/medium used to culture samples
No. labsThe number of laboratories (labs) that participated in a study
No. test runsThe number of test runs (set up and performed separately at different times)
Total no. test samplesTotal number of samples tested (i.e. over labs, runs and samples within runs)
Study results (in Tables 2b, 3b, 4b and 5b)
Recovery Concentration Results – Mean and %RSDs
ReferenceSame as for ‘Study characteristics’ tables
Test no.Same as for ‘Study characteristics’ tables
Mean (CFU cm−2)Mean surface concentration recovered
Lab %RSDLab-to-lab per cent relative standard deviation, which includes the variation from preparing the samples, extraction and analysis.
Run %RSDPer cent relative standard deviation from replicate runs of a test performed at different times.
Sample-within-run %RSDPer cent relative standard deviation from replicate tests performed at the same time (in one run)
Total %RSDTotal per cent relative standard deviation (including Lab, Run and Sample-within-run)
Recovery efficiency (RE) – Mean and %RSDs
RE mean (%)Mean recovery efficiency
RE lab %RSDLab-to-lab per cent relative standard deviation of RE
RE run %RSDRun per cent relative standard deviation of RE
RE sample-within-run %RSDSample-within-run per cent relative standard deviation of RE
RE total %RSDTotal per cent relative standard deviation of RE (including Lab, Run, and Sample-within-run)
LOD, FNR, and FPR
Positive resultHow a positive result (detection) was defined (e.g. CFU ≥ 1)
LOD definitionHow the limit of detection (LOD) is defined
LODValue of the limit of detection
LOD SD or 95% CI (CFU cm−2)Standard deviation or 95% confidence interval for the LOD
FNRFalse-negative rate (FNR) based on controlled tests where the sampled surface was known to be contaminated but yielded a negative result
FPRFalse-positive rate (FPR) based on controlled tests where the sampled surface was known to be uncontaminated but yielded a positive result

In reading the publications that document the studies summarized in this article, we noticed many differences in the way tests and calculations were performed and reported. The main differences are documented in Tables 25 (and Tables A.1 to A.4 of Piepel et al. 2011), but some differences were too detailed to document. A common difference was in the units used to report results. Whenever possible, we performed calculations using information in the publications to convert results reported in a given set of units to the common set of units used in the ‘b’ parts of Tables 25 (and Tables A.1 to A.4). There were also experimental and calculational differences in the ways studies used positive controls to obtain the ‘actual’ contamination values used as the denominator in calculating RE. For example, some studies had positive controls colocated with every sample, which allows RE values to be calculated using the colocated sample and positive control results. This way of calculating RE values can correct for inadvertent differences in contaminant deposition or other aspects of testing. In some studies without colocated positive controls, the average result of positive controls was used as the denominator to calculate RE values. This can inflate the uncertainty of the RE values because of any inadvertent differences.

Table 7 documents key study characteristics and results from Tables 25. Specifically, it summarizes the numbers of the 20 laboratory studies that varied several study factors and reported various performance metrics. Table 7 shows that 13, 12 and 5 studies investigated swab, wipe and vacuum sampling, respectively. Some studies investigated more than one type of sampling. Only Almeida et al. (2008) investigated storage/transportation effects on sampling results. An additional storage/transportation study has been completed, but the results have only been partially released in two poster presentations (O'Connell et al. 2010; Perry et al. 2010). The information in those references was not complete enough to include the study in this article.

The ‘zero’ entries in Table 7 denote combinations of study characteristics/results that were not addressed by the 20 studies. Table 7 shows that few of the 20 studies had results available from varying the factors: (i) agent (contaminant), (ii) agent deposition method, (iii) sample collection medium, (iv) wetting agent for the sampling medium (swab and wipe), (v) storage/transport conditions and (vi) processing/extraction method. Laboratory-to-laboratory uncertainties of REs were reported only by Estill et al. (2009) for swabs, wipes and vacuum samples; Hodges et al. (2010) for swab samples; and Rose et al. (2011) for wipe samples. Run-to-run uncertainties were quantified by Estill et al. (2009) for swab, wipe and vacuum samples; Edmonds et al. (2009) and Hodges et al. (2006) for swab samples; and Montgomery and Camp (2008) for vacuum samples. Even when uncertainties reported in some studies include laboratory-to-laboratory and/or run-to-run uncertainties, the numbers of laboratories and runs were typically small, indicating these sources of uncertainty are poorly estimated. Laboratory-to-laboratory and run-to-run uncertainties are expected to be substantial contributors to uncertainty and were not quantified in most of the studies. Hence, the uncertainties reported by many studies can be expected to underestimate the total uncertainties in surface-sampling results. Also, the uncertainties may be underestimated because of smaller uncertainties when applying sampling and analysis methods in controlled laboratory environments than may occur in actual contamination incidents.

LOD and FNR/FPR results are rarely reported in the laboratory study results, as shown in Table 7. As discussed previously, the FNR and RE (each of which may be a function of contaminant concentration and other factors) play a key role in the framework for determining the number of samples necessary to achieve a characterization or clearance goal with a specified confidence. Hence, it is important to experimentally quantify how RE and FNR depend on contaminant surface concentration and other sampling and environmental factors. Table 7 shows that only a few of the 20 studies varied the contaminant at different surface concentrations. Most of these studies investigated three or fewer concentrations, although Hodges et al. (2006) investigated six and Krauter et al. (2012) investigated eight. Montgomery and Camp (2008) [vacuum results], Estill et al. (2009), Hodges et al. (2010), Rose et al. (2011) and Krauter et al. (2012) found that RE did not depend on surface concentration for the range of concentrations tested. However, Hodges et al. (2006) and Edmonds et al. (2009) reported that RE increased as concentration increased. Of the studies that investigated different surface concentrations, only Krauter et al. (2012) presented equations for FNR as a function of surface concentration (for each of six surface materials). Estill et al. (2009) mentioned using probit regression to develop a probability of detection (equivalent to FNR) curve as a function of contaminant concentration, but did not present the results. The lack of FNR data (and of FNR and RE curves as functions of surface concentration, surface material and other influencing factors) is a significant gap in all studies except Krauter et al. (2012).

The studies summarized in Tables 25 have large ranges of mean RE values and sample-within-run uncertainty (%RSDRE) values. The following ranges of RE and %RSDRE exclude data from direct-inoculation tests in which surfaces were not sampled with a swab, wipe or vacuum. For swabs, RE ranged from 0 to 92·7% and %RSDRE ranged from 6 to 550. For wipes, RE ranged from 1 to 97% (excluding a questionable 120% value in the Estill et al. 2009 study) and %RSDRE ranged from 0 to 316. For vacuums, RE ranged from 0·02 to 36% and %RSDRE ranged from 10 to 130. These ranges of %RSDRE values represent the uncertainty in RE results from replicate tests performed at the same time, generally by the same samplers and the same laboratory personnel (because that is what most studies reported). Hence, these %RSDRE ranges do not include all relevant sources of variation and can be expected to underestimate the total uncertainty.

The wide ranges of RE values summarized earlier are a result of the effects of several factors varied within and across the studies (e.g. contaminant surface concentration; contaminant deposition method; surface material being sampled; materials and specifics of swabs, wipes, and vacuum socks; wetting agent for swabs and wipes; specifics of sample preparation and extraction methods and counting method) as shown in Tables 2 (a)5 (a). Krauter et al. (2012) also indicate that FNR values can be affected by such factors. Hence, the dependence of RE, FNR, LOD, and their total uncertainties (including all sources of uncertainty in the SAP) on such factors needs to be quantified. Ideally, all data from each of the swab, wipe, and vacuum studies would be combined and a statistical analysis performed to estimate the effects of the quantitative and qualitative test factors on RE, FNR and LOD. However, FNR and LOD were not reported in enough studies to do this. Even for the widely reported RE, the number of test factors is large with test factors sometimes having many possible values/options. Also, there may have been interactive effects on performance metrics between some test factors in the studies. Although it was beyond the scope of this work to attempt such a statistical analysis of the combined data, our initial assessment is that there are not enough data for this exercise to be successful given the large number of factor combinations and possible interactions in the swab, wipe and vacuum studies.

Table 2 (a). Summary of test conditions for swab-sampling studies. Acronyms and abbreviations are defined in Table 6 and the footnotes
Reference (lead author and year)No. testsAgentAgent depositionAgent concentration(s)Swab typesWetting agentSurface typesSurface area sampled (cm2)Extraction liquidExtraction methodCulture method, mediumNo. labsNo. test runsNo. samples/test
No.Range (CFU cm−2)
  1. aFive analysts total from two labs.
  2. bAluminium, V2A steel, MLI foil, Kapton®, Teflon®.
Estill 20096BA-SDA30·03–2MFBBTSS, Cpt103BBTV+SFP, TSAB32–424–36
Hodges 20066BA-SLD60·4–6000MFPBSTSS10PBSTVPlate/FP, TSAB2aNR15–45
Rose 200424BA-SLD11937·5Ct, P, R, MFDry, PBSTSS25·8PBSTU, V, SPlate, TSAB1NR10
4BA-SLI11E+4 CFU/swabDI: Ct, PE, R, MFNANoneNAPBSTVPlate, TSAB1NR10
Frawley 20086BA-SLD11E+2–1E+5 CFU/samplePEDry, PBST/TrP, W, Cl1, NRPBSTVPlate, SBA1NR12–24
7BA (4)LD150 CFU/sampleCtPBST/TrP, G, F, M, Cpt, B, Ct ClothNRNRNRPlate, NR1NR11
Brown 2007b4BAtrDA2102–103, 105–106RDWSS25BBTS, H, VPlate, PF1NR20
1BAtrLI11E+6 CFU/swabDI: RNANoneNABBTS, H, VPlate, PF1NR20
1BAtrLD12·0E+5NANADI: SS6·25BBTS, HPlate, BHIA1NR24
Edmonds 200916BAtrLD11·0E+5Ct, DP, R, MFDWG10PBSTrV, SSprP, NR13–428–40
16BAtrDA11·0E+9Ct, DP, R, MFDWG, SS, PC, VL10PBSTrV, SSprP, NR1324–30
4BAtrLD44·77E+3–2·52E+6DPDWG10PBSTrV, SSprP, NR1330
Valentine 200816BSLD29·03E+4, 2·82E+5Ct, PF, PE, DPPBSTP, O, PEUF, Cpt104·04PBSTVPlate, TSA11010
4BSLD29·03E+4, 2·82E+5Ct, PF, PE, DPPBSTMonitor25PBSTVPlate, TSA11010
Buttner 20012BSLD11·48E+6SKPBSTG5PBTVPlate, TSAC133
8BSDA1100–1000SK, CtPBSTVL, Cpt(3)32·49PBTVPlate, TSAC133
8BS+PC2DA1100–1000SKPBSTVL, Cpt(3)32·49PBTVPlate, TSAC122
Buttner 2004a6BAtrDA1107·6–1076·4SSPKNRVL, W, M929NRHMPlate, TSA133
3BAtrDA1107·6–1076·4SSPKNRVL, W, M929NRHMPlate, TSA112
Buttner 2004b4BAtrDA1NRSSPK, CtNRM, W100, 317NRHM, ShakePlate, TSA1NR4, 8
Quizon 20074BAtrWA1NRPEUFPBSTPWB, SS, VL, W100PBSTV, SSP/FP, TSA144, 10
Nellen 20062BS-168LD11000 CFU/swabDI: Ct, RNANANAPBSUntreatedPlate, TSA1315
6BS-168LD11000 CFU/swabDI: Ct, RNANANAPBSV, S, V+SPlate, TSA1315
4BS-168LD14Ct, RDWPD25PBSNR, H+V+SPlate, TSA1315
10BS-168LD116RDWb25PBSNR, H+V+SPlate, TSA133
Hodges 20106BA-SLD31·4, 15·3, 1607·2MFPBST, PBST+SS26PBSTVFP, TSAB121118–120
6BA-SLI336–33300 CFU/swabDI: MFNA, PBST+NANAPBSTVFP, TSAB12124–48
Table 2 (b). Summary of test results for swab sampling studies. Acronyms and abbreviations are defined in Table 6
Reference (lead author and year)No. testsRecovery concentration results – Mean and %RSDsRecovery efficiency (RE) – Mean and %RSDsLOD, FNR and FPR
Mean (CFU cm−2)Lab %RSDRun %RSDSample-within-run %RSDTotal%RSDRE mean (%)RE lab%RSDRE run %RSDRE sample- within-run%RSDRE total %RSDPositive result (CFU)LOD definitionLOD (CFU cm−2)FNRFPR
  1. aCombined estimate of ‘run’ and ‘sample-within-run’ uncertainties.
  2. bThe precision (variance within labs) reported by Hodges et al. (2010) is listed in this table as sample-within-run uncertainty.
  3. c≥1 CFU ml−1.
Estill 200960·0012–0·180–1100–9857–46073–4733·4–14·00–1100–11076–55081–560≥1LOD950·4 (SS) 1·9 (Cpt)NR0·037
Hodges 200660·1–2900NRNR27–100NR31·7–49·1NANA, 28–3321–91NA≥1LOD901·20–0·27NR
Rose 200424NRNANRNRNA0·1–43·6NANR18–200NANRNRNRNRNR
4NRNANRNRNA83·8–93·9NANR7–12NANRNRNRNRNR
Frawley 20086NRNANRNRNA0·2–5·5NANR30–63NANRNRNRNRNR
7NRNANRNRNA0–15NANR28–144NANRNRNRNRNR
Brown 2007b4NRNANRNRNA35·5–45·6NANR34–67NA≥1NR1NRNR
1NRNANRNRNA75·6NANR16NANRNRNRNRNR
1NRNANRNRNA99·9NANR0·1NANRNRNRNRNR
Edmonds 200916NRNANRNRNA42·5–89·1NA11–338–16NANRNRNRNRNR
16NRNANRNRNA51·5–75·5NA10–358–20NANRNRNRNRNR
42·0E+2–2·4E+6NA19–29aaNA42·1–92·7NANR19–29NANRNRNRNRNR
Valentine 200816NRNANRNRNA0·6–6·6NANR25–150NANRNRNRNRNR
4NRNANRNRNA0·5–2·7NANR18–84NANRNRNRNRNR
Buttner 20012NRNANRNRNA68·6–73·5NANR6–7NANRNRNRNRNR
882–748NA7–107aaNANRNANRNRNANRNRNRNRNR
884–1264NA18–85aaNANRNANRNRNANRNRNRNRNR
Buttner 2004a6340–711NA24–92aaNANRNANRNRNANRNRNRNRNR
3523–704NANR8–55NANRNANRNRNANRNRNRNRNR
Buttner 2004b4840–1200NANR11–28NANRNANRNRNANRNRNRNRNR
Quizon 200744170–5670NA17–36aaNANRNANRNRNANRNRNRNRNR
Nellen 200620·29NA17aaNA13·4NA17–18aaNANRNRNRNRNR
61·42–2·04NA1–6aaNA65·8–94·5NA3–7aaNANRNRNRNRNR
40·95–1·75NA25–33aaNA35·9–66·1NA27–34aaNANRNRNRNRNR
100·73–2·72NA24–56aaNA19·8–73·7NA26–52aaNANRNRNRNRNR
Hodges 201060·48–1064·32NRNRNRNR15·8–83·112·1–24·2b9–57b15–59cNR0·800–0·083
60·71–87·27NRNRNRNR27·9–73·926·7–46·8b15–40b32–57cNRNR0–0·170–0·146
Table 3 (a). Summary of test conditions for wipe sampling studies. Acronyms and abbreviations are defined in Table 6 and the footnotes
Reference (lead author and year)No. testsAgentAgent depositionAgent concentration(s)Wipe typesWetting agentSurface typesSurface area sampled (cm2)Extraction liquidExtraction methodCulture method, mediumNo. labsNo. test runsNo. samples/test
No.Range (CFU cm−2)
  1. aPC2 and PC4 = Penicillium chrysogenum at 102 and 104 CFU cm−2 on test coupons.
  2. bPlate with growth medium, or filter plate with TSA.
Estill 20093BA-SDA30·03–2SpongeBBTSS929BBTA+C+V+SFP, TSAB3327
3BA-SDA30·03–2SpongeBBTCpt929BBTA+C+V+SFP, TSAB32–418–36
Brown 2007a4BADA2102–103, 104–105PR gauzeDWSS, PWB25BBTS+H+VPlate, PF1NR20
1BALI1NRDI: PR gauzeDWNoneNABBTS+H+VPlate, PF1NR40
1BALD12·0E+5NANADI: SS25BBTS+HPlate, BHIA1NR24
Buttner 20011BSLD11·48E+7SpongePBSTGlass32·49PBSTHSPlate, TSAC133
12BS,BS+PC2, BS+PC4aDA11E+2–1E+3SpongePBSTVL, Cpt-R, Cpt-S, Cpt-C32·49PBSTHSPlate, TSAC12, 32, 3
Valentine 200812BSLD190349·9Ct, HCW,PRPBSTP, O, PEUF, Cpt104·04PBSTV+CPlate, TSAC11010
3BSLD1282,000Ct, HCW,PRPBSTMonitor25PBSTV+CPlate, TSAC11010
Buttner 2004a12BAtrDA1107·6–1076·4Swipe, HWPBSTVL, W, M929NRHMPlate, TSA133
6BAtrDA1107·6–1076·4Swipe, HWPBSTVL W, M929NRHMPlate, TSA111, 2
Buttner 2004b4BAtrLI210, 18·2BiSKitDry, PBSTM10 000PBSTBiSKitPlate, TSA1NR4, 8
6BAtrDA1NRBiSKitDry, PBST, NRM, W10 000PBSTBiSKitPlate, TSA1NR4, 8
Frawley 20086BA-SLD11E+2–1E+5/sampleGauzeDry, PBST/TrP, W, Ct, Cloth1, NRPBSTVPlate, SBA1NR12, 24
Einfeld 201110BAtr, BAtr+GrimeDA19,55E+0–6·81E+4PRDW, RH = 10-15, 82-90SS, G, Marble100BBTS+H+VPlate, PF12, 3, 524–60
Quizon 20074BAtrWA1NRPRPBSTPWB, SS, VL, W900PBSTC+V+SSP/FP, TSA144, 10
Krauter 201227BAtrLD80·0025–1·8538Sponge stickNBSS, CerT, VL645·16PBSTSt+C+V+Sb119, 10
27BAtrLD80·0078–0·1537Sponge stickNBFL, PW, PLCP645·16PBSTSt+C+V+Sb119, 10
Rose 20113BA-SLS30·0135–17·123DI: Sp. stickNB+ATDSS645·16PBSTSt+CSP/FP, TSAB9117 –18
3BA-SLS30·0405–51·37Sponge stickNB+ATDSS645·16PBSTSt+CSP/FP, TSAB9154–63
1BA-SLS115·5Rayon gauzePBSTSS645·16PBSTSt+CSP/FP, TSAB1110
1BA-SLS115·5Sponge wipeDE brothSS645·16PBSTSt+CSP/FP, TSAB1110
1BA-SLS115·5Sponge wipeBBSS645·16PBSTSt+CSP/FP, TSAB1110
1BA-SLS115·5Sponge stickPBSTSS645·16PBSTSt+CSP/FP, TSAB1110
3BA-SLS30·0155, 0·155, 15·5PEF spongePBSTSS645·16PBSTSt+CSP/FP, TSAB1315
Montgomery 20084BA-SEWD1NRPR gauze, pad 2 wipe tech'sPBSTrHVAC filter100PBSTrShake+CPlate, NR12, 32–5
Table 3 (b). Summary of test conditions for wipe sampling studies. Acronyms and abbreviations are defined in Table 6 and the footnotes
Reference (lead author and year)No. testsRecovery concentration results – Mean and %RSDsRecovery efficiency (RE) – Mean and %RSDsLOD, FNR and FPR
Mean (CFU cm−2)Lab %RSDRun %RSDSample- within- run %RSDTotal %RSDRE mean (%)RE lab %RSDRE run %RSDRE sample- within-run %RSDRE total %RSDPositive result (CFU)LOD defi-nitionLOD (CFU cm−2)FNRFPR
  1. aCombined estimate of ‘run’ and ‘sample-within-run’ uncertainties. In cases where the data supported estimating each source of uncertainty and combining them, it was not possible to tell whether these calculations were correctly implemented. A common, but incorrect, method is to calculate a single standard deviation from data including two or more sources of uncertainty.
Estill 200930·017–0·3509–270–4561–20080–20018–3114–290–1353–16057–160≥3LOD950·15NR0·133
30·084–0·5009–260–5237–18060–18221–1200–110–6340–22075–220≥1LOD950·009NR0·0
Brown 2007a4NRNANRNRNA25·2–39·2NANR32–59NA≥1CFU ml−1NR3·6–4·2NRNR
1NRNANRNRNA93·2NANR9NA≥1CFU ml−1NRNRNRNR
1NRNANRNRNA99·9NANR0·1NA≥1CFU ml−1NRNRNRNR
Buttner 20011NRNANRNRNA74·3NANR7NANRNRNRNRNR
12127–1283NA0·6–101aaNANRNANRNRNANRNRNRNRNR
Valentine 200812NRNANRNRNA1·4–7·9NANR18–84NANRNRNRNRNR
3NRNANRNRNA2·6–4·2NANR22–38NANRNRNRNRNR
Buttner 2004a12711–6950NA0–44aaNANRNANRNRNANRNRNRNRNR
6506–883NANA3–95NANRNANRNRNANRNRNRNRNR
Buttner 2004b42·05–3·35NANRNRNA11·3–18·4NANRNRNA≥1 CFU ml−1NR0·0042–0·01NRNR
6115–590NA14–84NRNANRNANRNRNANRNRNRNRNR
Frawley 20086NRNANRNRNA0·2–6·6NANR46–70NANRNRNRNRNR
Einfeld 201110NRNANRNRNA18·0–96·8NA29–96aaNANRNRNRNRNR
Quizon 200746670–14615NA8–20aaNA39·5, NRNANRNRNANRNRNRNRNR
Krauter 2012270·0003–0·994NANR6·6–316NA12·5–75·5NANR6·6–316NA≥1LOD95, LOD900·013–0·0380–0·9330·0
270·0–0·084NANR0–316NA4·0–76·9NANR0–316NA≥1LOD95, LOD900·015–0·0510–1·00·0
Rose 201130·0064–13·34NRNRNRNR46·1–77·914–48NA11–5418–72≥1NRNR0–0·1590–0·056
30·013–15·48NRNRNRNR24·4–32·420–31NA20–6928–76≥1NR0·031NRNR
1NRNRNRNRNR30·8NANA50NANRNRNRNRNR
1NRNRNRNRNR32·3NANA30NANRNRNRNRNR
1NRNRNRNRNR26·8NANA21NANRNRNRNRNR
1NRNRNRNRNR36·3NANA25NANRNRNRNRNR
 3NRNRNRNRNR26·0NANA38NANRNRNRNRNR
Montgomery 20084NRNANRNRNA14·5–19·9NA10–370–27NANRNRNRNRNR
Table 4 (a). Summary of test conditions for vacuum-sampling studies. Acronyms and abbreviations are defined in Table 6 and the footnote
Reference (lead author and year)No. testsAgentAgent depositionAgent concentration(s)Vacuum filter typeVacuum techniqueaRelative humidity (%)Surface typesSurface area sampled (cm2)Extraction liquidExtraction methodCulture method, mediumNo. labsNo. test runsNo. samples/test
No.Range (CFU cm−2)
  1. aR = rough, G = gentle, VS = vertical slide, TD = tilt drag, TP = tilt push. See Table 6 for remaining acronyms.
Estill 20093BA-SDA30·03–2HEPA sockP2DNRSS929BBTA+C+V+SFP, TSAB1327
3BA-SDA30·03–2HEPA sockP2DNRCpt929BBTA+C+V+SFP, TSAB12–418–36
Brown 2007c8BADA21E+2–1E+3 1E+4–1E+5HEPA filterPP2DNRSS, PWB, Cpt, Concrete100BBTS+H+VPlate, PF1NR13–24
Einfeld 20116BAtr, BAtr +grimeDA11·31E+3–8·78E+4PE filterNR36–48, 77–90Marble, Concrete100BBTS+H+VPlate, PF13–533–59
Quizon 20076BAtrWA1NRHEPA sockSVHNRCelT, PWB, SS, VL, W900PBSTC+V+SSP/FP, TSA144–10
Montgomery 20081BA-SEWD17·98E+2HEPA sockRNRHVAC filter309·68PBSTrShake+CNR, NR13NR
4BA-SEWD37·98E+2, 9·0E+4, 9·1E+6TECFG, RNRHVAC filter400PBSTrShake+CNR, NR139, NR
3BA-SEWD2167, 201TECFVS, TDNRCpt400PBSTrShake+CNR, NR113, 6
2BA-SWD1111TECFVS, TPNRCpt400PBSTrShake+CNR, NR113
Table 4 (b). Summary of test results for vacuum sampling studies. Acronyms and abbreviations are defined in Table 6 and the footnote
Reference (lead author and year)No. testsRecovery concentration results – Mean and %RSDsRecovery efficiency (RE) – Mean and %RSDsLOD, FNR and FPR
Mean (CFU cm−2)Lab %RSDRun %RSDSample-within-run %RSDTotal %RSDRE mean (%)RE lab %RSDRE run %RSDRE sample-within-run %RSDRE total %RSDPositive result (CFU)LOD definitionLOD (CFU cm−2)FNRFPR
  1. aCombined estimate of ‘run’ and ‘sample-within-run’ uncertainties.
Estill 200930·0043–0·118–380–5082–9884–1163·7–5·516–310–573–9079–92≥3LOD950·44NR0·267
30·0011–0·0620–300–5744–17065–1703·7–6·30–180–8950–130100–130≥3LOD950·28NR0·067
Brown 2007c8NRNANRNRNA16·4–36·1NANR28–90NA≥1 CFU ml−1NRNRNRNR
Einfeld 20116NRNANRNRNA10·0–19·7NANR50–129NANRNRNRNRNR
Quizon 20076890–5200NA9–20aaNA4·4, NRNANRNRNANRNRNRNRNR
Montgomery 20081NRNANRNRNA2·42NA1·324NANRNRNRNRNR
4NRNANRNRNA1·16–3·52NA16–2813–25NANRNRNRNRNR
3NRNANRNRNA0·20–0·48NANA10–60NANRNRNRNRNR
2NRNANRNRNA2·26, 1·69NANA18, 32NANRNRNRNRNR
Table 5 (a). Summary of test conditions for storage and stability tests. Acronyms and abbreviations are defined in Table 6
Reference (lead author and year)No. testsAgentaAgent depositionbAgent concentration (CFU ml−1)Sampling medium typebWetting agentbRelative humiditySurface type and area sampledbStorage conditionsExtraction liquidaExtraction methodaCulture method, mediumNo. labsNo. test runsTotal no. test samples
AdditivecTemp. (°C)No. days
  1. aThe BA Sterne spores were diluted with PBS (10 mmol l−1 phosphate, 138 mmol l−1 NaCl, and 2·7 mmol l−1 KCl, pH 7·4) containing 0·01% Triton® X-100, and vigorously mixed by vortexing.
  2. bThis study used only liquid samples containing the agent, so neither deposition onto surfaces nor sampling of surfaces was involved.
  3. cAdditives to sterile water: Ethanol = ethanol 20% (v⁄ v), EDT = ethylendediaminetetraacetic acid, 10 mmol l−1, pH 8·0, Phenol = phenol 1% (v⁄ v), PBSTr = PBS containing 0·01% (v ⁄ v) Triton × 100.
Almeida 20081BA SterneLiquidNRNANANRNANone40NANAPlate, LBANR3 lots19
15BA SterneLiquidNRNANANRNANone, Phenol, EDTA. Ethanol, PBSTr40, 182, 279NANAPlate, LBANR13
4BA SterneLiquidNRNANANRNANone-20, -80182, 279NANAPlate, LBANR13

Gaps, needs and recommendations

This section discusses several gaps and needs identified based on the summaries of laboratory studies of swab, wipe and vacuum-sampling and analysis methods in Tables 25. Recommendations are made for additional laboratory studies and other evaluations to address these gaps and needs. In what follows, ‘performance results’ refers to RE, FNR, LOD and their uncertainties, considering uncertainty contributions from all steps of the SAP.

Realistic vs laboratory conditions

Price et al. (2009) emphasize the importance of quantifying the performance of methods (sample collection, storage/transportation, processing/extraction and analysis) under realistic conditions and not just highly sanitized laboratory conditions. Some studies investigated other material added with the BA or surrogate spores: Penicillium chrysogenum organism (Buttner et al. 2001), grime (Einfeld et al. 2011) and silicon dioxide (Brown et al. 2007a,b,c; Estill et al. 2009). The latter studies did not perform tests without silicon dioxide added, so no conclusion can be made about its effect. Buttner et al. (2001) and Einfeld et al. (2011) found that REs were the same or better with the other materials added to the Bacillus contaminants they investigated. Data from field studies (e.g. Amidan et al. 2007; Piepel et al. 2009) and the 2001 BA letter incident that involve realistic conditions are available, but do not provide for accurately assessing RE and FNR in the way that laboratory studies do. We recommend that more work be performed to supplement the studies of Buttner et al. (2001) and Einfeld et al. (2011) and quantify the effects on method performance results of surfaces contaminated with other material and/or biological organisms in addition to BA. Ideally, such studies should be performed with three levels of each other organism/material (none, low and high) to better assess the effects of such factors on sampling performance results.

Bacillus anthracis vs surrogates

Previous laboratory studies have used BA or one of several surrogates, but never investigated BA and one or more surrogates in the same study (see Tables 2 (a)5 (a)). The number of test variables whose values differ among studies is large enough that the existing studies do not provide for quantifying the relationships between performance results of swab, wipe and vacuum methods with BA and with its surrogates. Hence, we recommend that more work be performed to establish the relationships between sampling performance results for surrogates and BA. One way to do that would be to use BA and key surrogates in a new laboratory study to establish these relationships for various combinations of sampling conditions (i.e. various concentrations and surface materials) for a each set of sampling methods (sample collection device, storage and transportation, extraction and analysis). Another option would be to measure, for BA and its surrogates, organism properties relevant to sampling and resuspension. The data could then be used to establish the relationships for sampling and analysis performance results between BA and its surrogates.

Deposition method

Previous laboratory studies generally used one of two contaminant deposition methods: (i) contaminant in liquid (water, ethanol or a mixture of the two) that was allowed to dry after deposition by wet aerosol or liquid drops and (ii) dry aerosol. Only one swab study (Edmonds et al. 2009) compared REs for different deposition methods. No wipe or vacuum studies varied the deposition method. Edmonds et al. (2009) found that for high contaminant concentrations, REs differed significantly depending on liquid vs dry deposition, with the difference depending on the swab-sampling method. We recommend that more work be performed to quantify the difference in performance results of swab, wipe and vacuum-sampling and analysis methods as a function of the contaminant deposition method, as well as functions of other influencing factors (e.g. contaminant concentration).

Surface materials

A wide variety of surface materials were investigated in the swab, wipe and vacuum-sampling studies summarized in Tables 24. However, some studies investigated only stainless steel or a limited number of surface materials. Method performance results were generally reported separately for each surface material, without any attempt to develop relationships between performance results and characteristics of the surface materials. The exception was Krauter et al. (2012), who compared mean REs, FNR and LOD to the roughness index of the surfaces investigated. We recommend using existing data where possible, or performing new studies to develop equations that relate method performance to one or more surface-characteristic variables (as well as any other influencing factors). Such equations can then be used to estimate method performance for various surface materials without additional experimental work.

Table 5 (b). Summary of results for storage and stability tests. Acronyms and abbreviations are defined in Table 6
Reference (lead author and year)No. testsRecovery concentration results – Mean and %RSDsRecovery efficiency (RE) – Mean and %RSDsLOD, FNR and FPR
Mean (CFU ml−1)Lab %RSDRun %RSDSample- within-run %RSDTotal %RSDRE mean (%)RE lab %RSDRE run %RSDRE sample- within-run %RSDRE total %RSDPositive result (CFU)LOD definitionLOD (CFU cm−2)FNRFPR
  1. §Combined estimate of ‘run’ and ‘sample-within-run’ uncertainties.
Almeida 200816·19E+9NA17425NA111NA2224NANRNRNRNRNR
151·00E+8–1·95E+8NANA4–86NA71–86NA2224NANRNRNRNRNR
41·20E+8–1·61E+8NANA7–84NANANA6–55NANANRNRNRNRNR
Table 6. List of acronyms and abbreviations
AAgitation (during processing/extraction)
AlumAluminium
ATDArizona test dust
BBrick
BABacillus anthracis
BA (4)Four strains of Bacillus anthracis
BA-SBacillus anthracis Sterne
BAtrBacillus atrophaeus
BAtr + grimeBacillus atrophaeus +  standard grime (Arizona test dust + diesel carbon + oil + biologicals)
BBButterfield buffer
BBTButterfield buffer with Tween
BHIABrain heart infusion agar
BiSKitBiological Sampling Kit
BSBacillus subtilis
BS + PC2, BS + PC4Bacillus subtilis with 102 or 104 CFU cm−2 P. chrysogenum background contamination
CCentrifuging (during processing/extraction)
CelTCeiling tile
CerTCeramic tile
CFUColony forming unit
CIConfidence interval
CptCarpet
Cpt-CCarpet, commercial loop
Cpt-RCarpet, residential cut-pile
Cpt-SCarpet, soiled residential cut-pile
Cpt(3)Cpt-C, Cpt-R, Cpt-S
CtCotton
Ct clothCotton cloth
DADry aerosol
DE brothDey Engley broth
DIDirectly inoculated
DPDacron® polyester
DWDe-ionized, distilled, or sterile water
EWDDrops of agent in 50% ethanol, 50% water
EtOHEthyl alcohol, enthanol
FFormica®
FLFaux leather
FNRFalse-negative rate
FPFilter plate
FPRFalse-positive rate
GGlass
HHeat treatment (extraction/processing)
HCWHS II cleanroom wiper
HEPAHigh-efficiency particulate air
HMHand mixed
HSHand stomached
HVACHeating, ventilation and air conditioning
HWHeavy wipe
LBALuria broth agar
LDLiquid drops
LILiquid immersion
LODLimit of detection
LOD90, LOD95Lowest concentration with 90 or 95% probability of detection
LSLiquid spread
MMetal
MF, MfoamMacrofoam
MLI foilMultilayer insulation foil (aluminized Kapton)
NANot applicable
NBNeutralizing buffer
NB+ATDSponge wipe premoistened with neutralizing buffer, wipe samples collected, then inoculated with Arizona Test Dust
NRNot reported
OOak wood
PPlastic
P2DPulled one direction and perpendicular direction
PBSPhosphate buffered saline
PBSTPBS with Tween
PBST+PBST + Arizona Test Dust + Bacillus atrophaeus + Staphylococcus epidermidis
PBSTrPBS-TritonX-100
PBST/TrPBS with 0·1% Tween (PBST) or 0·1% Triton-X (PBSTr)
PBTPotassium phosphate buffer w/Tween
PCPolycarbonate
PDPetri dish
PEPolyester
PE filterPolyethylene filter
PEFPolyester foam
PEUFPolyester upholstery fabric
PFPetrifilm
PLCPPlastic light cover panel
PP2DPush-pull one direction, then other
PRPolyester-rayon blend
PR gauzePolyester-rayon blend gauze
PU foamPolyurethane foam
PWPainted wood
PWBPainted wallboard
RRayon
RERecovery efficiency
RHRelative humidity
%RSDPer cent relative standard deviation
SSonication
SBASheep blood agar
SDStandard deviation
SEStandard error
SKSwab kit
SPSpread plate
Sp. stickSponge stick
Sp. wipeSponge wipe
SP/FPSpread/filter plate for high/low concentrations
SprPSpiral plate
SSStainless steel
SSPKSwab sample processing kit
StStomached
SVHS strokes, vertical then horizontal
TECF3M Trace Evidence Collection Filter
TSATrypticase soy agar
TSABTrypticase soy agar with 5% sheep blood (also referred to as TSAII)
TSACTrypticase soy agar + 100μg cycloheximide per ml
UUntreated
VVortexing (during processing/extraction)
VLVinyl or vinyl tile
WWood
WAWet aerosol
WDDrops of agent dispersed in water
Table 7. Numbers of the 20 surface-sampling studies that investigated various factors and reported specific results
Number of studies that varied the.. .Swabs (13)aWipes (12)aVacuums (5)a
RecbREcLODfFNR/FPRgRecbREcLODfFNR/FPRgRecbREcLODfFNR/FPRg
Mean/ %RSDdLab/Run %RSDeMean/%RSDdLab/Run %RSDeMean/%RSDdLab/Run %RSDeMean/%RSDdLab/Run %RSDeMean/% RSDdLab/Run %RSDeMean/%RSDdLab/Run %RSDe
  1. aNumber in parentheses is the number of studies summarized.
  2. b‘Rec’ denotes the agent recovered by surface sampling.
  3. cRE denotes recovery efficiency.
  4. dMean/% RSD denotes the mean and within-run%RSD.
  5. eLab/Run %RSD denotes lab-to-lab %RSD and run-to-run %RSD.
  6. fLOD denotes limit of detection.
  7. gFNR/FPR denotes false-negative and/or false-positive rate.
  8. hStorage results are available in Almeida et al. (2008), but sampling media were not part of that study.
Agent (contaminant)101000101000001000
Agent deposition104101100000001000
Agent concentration519233415253113111
Sample collection medium type305000103000000000
Wetting agent103101103010NANANANANANA
Surface type514111617132215111
Surface area sampled101000001000001000
Storage/transport conditionsh000000000000000000
Processing/extraction method404010000000000000
no. labs313223212222000000

Storage and transportation of samples

Only two previous studies have investigated the effects of storage and/or transportation conditions on method performance results. Almeida et al. (2008) investigated the effects of different additives, storage times and storage temperatures on the stability of BA in water. However, that work is not directly relevant to surface sampling, which would involve storing and/or transporting swab, wipe and vacuum samples to laboratories (which would then perform the processing/extraction and analysis steps). A second study investigated the effects of storage and transportation factors on swab samples, but only partial results have been released in two poster presentations (O'Connell et al. 2010; Perry et al. 2010). Because that study only addressed swab samples, we recommend that additional studies or evaluations be performed to quantify how sampling and analysis methods for wipe and vacuum samples are affected by storage and transportation methods.

Recovery efficiency of sample collection vs processing

Several studies investigated the contributions of sample collection and sample processing/extraction to the overall RE of a method. For directly inoculated Petri dishes, Buttner et al. (2001) reported sample collection efficiency, processing efficiency and overall RE for two swab and one wipe sampling methods. For all three, the majority of the inefficiency came in the processing step, not the sample collection step. Results from other studies that investigated and reported REs for sample collection and processing steps were similar in some cases (Rose et al. 2004), while in other cases, the sample collection inefficiency was similar to or larger than processing inefficiency (Buttner et al. 2001; and Nellen et al. 2006; Brown et al. 2007a,b). In all cases, processing/extraction inefficiency is large enough that optimizing the specifics of the processing/extraction step would be very important in maximizing RE. The optimal processing conditions may depend on the sample collection material, the extraction solution and the methods of dissociation (e.g. vortexing and sonication). While it is desirable to understand how steps of the SAP affect the overall RE and improve methods when possible, this is not a ‘gap’ from the standpoint of adequately quantifying the overall performace of methods in a given SAP. Hence, we recommend additional work of this type only when it is desired to improve the performance of a particular type of surface sampling (e.g. vacuum sampling).

RE and FNR as functions of concentration and other influencing factors

Few of the 20 studies reported FNR data and only Krauter et al. (2012) developed relationships between low contaminant concentrations and the FNR performance (for the sponge-stick wipe method). Similar work is needed for other surface sampling and analysis methods that may be used when BA surface concentrations are low. Quantifying the FNR-concentration relationships is critical because FNR > 0 significantly affects (i) the number of samples required to achieve desired confidences in characterization and clearance decisions and (ii) the confidence in characterization and clearance decisions made using sample and analysis data following a contamination incident. On the other hand, if surface concentrations are at levels much higher than when false negatives begin to appear, then the magnitude of RE and its dependence on surface concentration are not very important if the goal is to merely detect contamination (vs to accurately quantify the amount/concentration of contaminant). We recommend that additional work be performed to quantify the FNR performance (as a function of lower contaminant concentrations, surface material characteristics and other relevant factors) of one or two primary sets of SAP methods, for each of swab and vacuum sampling.

Estimates of main sources of uncertainty

Previous laboratory studies did not capture the main sources of uncertainty affecting performance results. Many of the studies investigated only short-term, within-run uncertainties (repeatability) and did not investigate run-to-run or laboratory-to-laboratory uncertainties (reproducibility). Hence, the estimates of uncertainty in performance measures for most of the studies in Tables 25 can be expected to underestimate the total uncertainty. Further, in some studies that captured more than one source of uncertainty, the statistical measures of uncertainty reported may have been improperly calculated. A common error is to calculate the standard deviation (or %RSD) from a set of data subject to more than one source of uncertainty when the data provides for separately estimating the uncertainties. The correct approach is to use statistical variance-component estimation to separately estimate the standard deviation (or %RSD) for each source of uncertainty and then properly combine the separate estimates into an estimate of total uncertainty. The studies that best met the goals of quantifying repeatability and reproducibility uncertainties are the swab and wipe validation studies (Hodges et al. 2010 and Rose et al. 2011, respectively). We recommend that additional studies of this type be designed and performed to capture repeatability and reproducibility uncertainties in the steps of the SAP.

Control samples

Control samples (e.g. positive, negative, sample medium blanks and laboratory blanks) are necessary to quantify performance measures such as RE, FNR and FPR. Although the studies summarized in this article included control samples, FNR and FPR were seldom reported. Positive controls for deposition load used to calculate RE should be colocated with each surface sample, although this was not carried out for several of the studies. Colocating a positive control with each test sample enables calculating the RE for each surface sample, if necessary. This approach corrects for unavoidable systematic variation in contaminant deposition in a chamber or controlled test area, so that such differences do not inflate the uncertainties of RE. Hence, we recommend colocating a positive control with each test sample in designing laboratory studies for surface sampling.

To close this section, we briefly compare the results in this article to those in the mini-review article by Edmonds (2009). He summarized REs for swab, wipe and vacuum methods from selected studies, whereas this article summarizes in tabular form the test conditions and several performance measures (RE, FNR, LOD, FPR and uncertainties) from 20 laboratory studies. The tabular summaries provided for indentifying several gaps and needs not discussed by Edmonds (2009). There is little overlap of this article with Edmonds (2009), because even for the few ‘needs’ addressed in both articles, the discussions are quite different. On the other hand, Edmonds (2009) discusses some difficulties, needs and perspectives that are different than discussed in this article. Hence, this article and the Edmonds (2009) article provide a comprehensive review of the performance of swab, wipe and vacuum surface-sampling methods and gaps/needs and recommendations for additional work in the future.

Conclusions

In each of the topic areas discussed in the preceding section, there have been useful results from the 20 studies summarized in this article. The studies of Hodges et al. (2010) and Rose et al. (2011) are noteworthy because they substantially validated the swab and wipe methods recommended by CDC (at the time of this article). The study of Krauter et al. (2012) provided FNR performance results for the CDC-recommended wipe method that had not been addressed by Rose et al. (2011). The study discussed by O'Connell et al. (2010) and Perry et al. (2010), when published, will address a gap related to the effects of storage and transportion on performance of the CDC-recommended swab method. The studies of Buttner et al. (2001) and Einfeld et al. (2011) concluded that grime and another organism, respectively, did not affect the performance of the swab and wipe, respectively, sampling methods they investigated. All of the studies have contributed to the body of knowledge on performance of swab, wipe and vacuum surface-sampling methods for BA contamination.

Although 20 laboratory studies were reviewed and summarized in this article, some of the results are for methods that are out-performed by other methods and hence would not be used in the present time. In other cases, the studies that have been performed are limited and only partially address a topic. Hence, we identified in the previous section several remaining needs and made recommendations for future work or studies. Several gaps/needs/recommendations involve quantifying the difference in surface-sampling performance results for

  1. surrogates vs BA (no studies to date)
  2. liquid vs aerosol deposition (only Edmonds et al. 2009 with varying results)
  3. other material (e.g. dust and grime) or organisms in addition to BA (only Buttner et al. 2001 and Einfeld et al. 2011)
  4. different surface materials as a function of material characteristics (only Krauter et al. 2012).

Only Krauter et al. (2012) have quantified the FNR performance of a method (sponge-stick wipe) as a function of concentration for various surface materials. So, additional work of that type for other methods is needed to be able to assess the confidence that there is no contamination given a set of negative sample results. Finally, any future studies should consider colocating positive control samples with each test sample and be designed to provide for quantifying reproducibility uncertainty and not just repeatability uncertainty.

While additional work is recommended to fill gaps/needs in several topic areas, such work could involve new evaluations of existing data (some of which may not have been published), as well as new experimental studies. It is hoped that this review article will spur work of both kinds, so that the combination of previous and new results provide for adequately quantifying the performance characteristics (e.g. RE, FNR, LOD and uncertainties) of swab, wipe and vacuum-sampling methods, accounting for all the steps of the SAP.

Acknowledgements

The Pacific Northwest National Laboratory (PNNL) work summarized in this article was funded by the Biological Research and Development Branch of the Chemical and Biological Division in the Science and Technology Directorate of the US Department of Homeland Security (DHS). We also acknowledge the US Department of Energy's Young Women in Science program, which funded the work of student intern Rebecca Hu. PNNL is a multiprogram national laboratory operated for the US Department of Energy by Battelle under Contract DE-AC05-76RL01830.

The authors acknowledge Jayne Morrow (NIST), Paula Krauter (SNL), and Brent Pulsipher (PNNL) for reviewing the technical report from which this article was adapted, as well as Landon Sego (PNNL) for reviewing the report and the article. Finally, we acknowledge Maura Zimmerscheid for excellent copy editing. We also thank the authors of several publications we cited who provided (i) copies of presentations or prepublication copies of articles, (ii) additional data or calculations, (iii) additional information not included in their publications and (iv) feedback on initial tabular summaries of their work. These include Jamie Almeida (NIST), Mark Buttner (University of Nevada), Kenneth Cole (US Army Dugway Proving Grounds), Wayne Einfeld (SNL), Cheryl Estill (CDC), Misty Hein (CDC), Robert Knowlton (SNL), Paula Krauter (SNL), Laura Rose (CDC), and Nancy Valentine (PNNL).

Versions of Tables 2 (a)5 (a) with larger font size and less white space are available in the downloadable Piepel et al. (2011).

Ancillary