• Benchmark analysis;
  • BMD;
  • BMDL;
  • bootstrap confidence limits;
  • dose-response analysis;
  • isotonic regression;
  • toxicological risk assessment


  1. Top of page
  2. Abstract

Estimation of benchmark doses (BMDs) in quantitative risk assessment traditionally is based upon parametric dose-response modeling. It is a well-known concern, however, that if the chosen parametric model is uncertain and/or misspecified, inaccurate and possibly unsafe low-dose inferences can result. We describe a nonparametric approach for estimating BMDs with quantal-response data based on an isotonic regression method, and also study use of corresponding, nonparametric, bootstrap-based confidence limits for the BMD. We explore the confidence limits’ small-sample properties via a simulation study, and illustrate the calculations with an example from cancer risk assessment. It is seen that this nonparametric approach can provide a useful alternative for BMD estimation when faced with the problem of parametric model uncertainty.


  1. Top of page
  2. Abstract

1.1. Benchmark Analysis

In quantitative risk risk assessment, the benchmark approach for estimating low-dose risk after exposure to a hazardous stimulus has seen substantial growth in familiarity and acceptance since its proposal in the mid-1980s.[1] The method takes a function, inline image, relating the response to exposure at dose inline image, and manipulates components of this posited model to yield a benchmark dose (BMD) at which a specified benchmark response (BMR) is attained. (If the exposure is measured as a concentration, one refers to the exposure point as a benchmark concentration, or BMC.) The BMD is then further manipulated—using, e.g., uncertainty factors or modifying factors [2]—to arrive at a level of acceptable human or ecological exposure to the hazard or to otherwise establish low-exposure guidelines. One important modification is the use of lower inline image confidence limits on the BMD—called benchmark dose (lower) limits or simply BMDLs [3]—in order to incorporate statistical variability of the BMD point estimate into the calculations. Where needed for clarity, the notation adds a subscript for the BMR level at which each quantity is calculated: BMD100BMR and BMDL100BMR; inline image. With this, many analysts use the BMDL or BMCL as a point of departure in quantitative risk assessment, [4, 5] and these quantities are employed for risk characterization and management by a variety of agencies, including the U.S. Environmental Protection Agency (EPA), the U.S. Food and Drug Administration (FDA), the Organisation for Economic Co-operation and Development (OECD), and many others. The application of benchmark analysis for quantifying and managing risk with a variety of toxicological endpoints is growing in both the United States and the European Union. [6-9]

A common data-analytic application of the BMD occurs with proportions, say, inline image, where inline image is the number of subjects expressing the adverse event under study, inline image is the number of subjects tested, and each ith proportion is observed at a corresponding exposure dose inline image. This is the “quantal response” setting, and it is common in carcinogenicity testing, environmental toxicity analysis, and many other biomedical risk studies. [10] With quantal data the basic statistical model is the binomial: inline image, where inline image is the function representing the probability of response at dose inline image. For risk-analytic considerations the exposed subjects' differential risk adjusted for any spontaneous or background effects is often of interest. This leads to consideration of excess risk functions such as the extra risk inline image.[2] The BMD is determined by setting inline image BMR over inline image and finding the smallest solution. When applied to data, the method is a form of inverse nonlinear regression and except for the use of an excess risk function upon which to base the inversion, is similar to estimation of an “effective dose” such as the well-known median effective dose, ED50.[2]

1.2. Model Dependence

A recognized concern with benchmark analysis is its potential sensitivity to specification of the dose-response function, inline image. A wide variety of parametric forms has been proffered for inline image when estimating BMDs. Most operate well at (higher) doses near the range of the observed quantal outcomes; however, these different models can produce wildly different BMDs at very small levels of risk when applied to the same set of data. [11, 12] Traditional strategies to avoid parametric model dependencies in low-dose risk analysis generally rely on the so-called no-observed-adverse-effect level (NOAEL) or similar variants, which does not require specification of inline image. Substantial statistical instabilities have been identified with use of the NOAEL for risk estimation, however, and contemporary analysts recommend against use of this dated technology. [13, 14] (Indeed, the BMD was originally suggested as a more-stable statistical alterative to the NOAEL.[1]) Needed is a modern quantitative methodology that can produce reliable inferences on acceptable exposure levels to the hazard under study, but that can avoid estimation biases and instabilities resulting from uncertain model specification. [15]

Recent work with the BMD has led to some intriguing advances, including model averaging techniques motivated from both statistical frequentist [16-19] and Bayesian [20-22] perspectives. Alternatively, in order to avoid specification of inline image we describe below a (frequentist) nonparametric estimator for the observed dose-response pattern. Such nonparametric curve estimation has seen only limited implementation in low-dose risk assessment. Krewski et al.[23] made perhaps the earliest substantial effort, but they did not connect to the modern machinery of benchmark analysis. A selection of works have applied semiparametric models for estimating BMDs, where certain portions of the model are parametrically specified. For example, Bosch et al.[24] combined a nonparametric comparison measure with a parametric dose-response specification such as the well-known probit model. Later, Fine and Bosch[25] applied quasi-likelihood-type estimating equations to extend this semiparametric model framework. These works focused on continuous measurements, however, and not quantal data. For the quantal setting Wheeler and Bailer[26] considered model-free specifications for the dose response by exploring monotone cubic B-splines. This is similar to our approach below, except that they employed hierarchical, parametric Bayesian estimation of the dose-response parameters; also see Guha et al.[27]

Rather than appealing to the hierarchical Bayesian paradigm, Dette and colleagues [28, 29] (along with the references found therein) discussed nonhierarchical, kernel-based nonparametric estimation for effective doses, such as the ED50, from a quantal-response experiment. As noted above, ED50 estimation shares many similarities with BMD estimation, although Dette and his co-authors did not connect their work to problems in benchmark analysis. We consider here related methods based on the work of Bhattacharya and Lin.[30] Those authors produced an adaptive, isotonic, regression-based method for estimating nonparametric effective doses in the larger area of bioassay analysis, but also considered explicitly the benchmark dose problem. Their approach extended an earlier work of Bhattacharya and Kong,[31] where the ED50 was estimated by inverting a model-free estimator of the original risk function inline image. We adapted the Bhattacharya and Kong approach for use with excess risk functions such as inline image to construct nonparametric BMDs and BMDLs.[32] That previous work presented the theoretical features of the method and gave a corresponding BMDL based on a nonparametric bootstrap; herein we focus on application of our fully nonparametric BMD for use in toxicological risk assessment. Section 'MODEL-INDEPENDENT BENCHMARK ANALYSIS' reviews the nonparametric estimator, while Section 'BMDL PERFORMANCE EVALUATION' evaluates the small-sample performance of the bootstrap-based BMDLs via a broad-scale simulation study. Section 'EXAMPLE: FORMALDEHYDE CARCINOGENESIS IN LABORATORY ANIMALS' illustrates the methods with an example from cancer risk assessment, and Section 'DISCUSSION' ends with a contemplative discussion.


  1. Top of page
  2. Abstract

2.1. Nonparametric BMD Estimation

We continue to assume a binomial structure for quantal-response data: inline image, inline image, inline image. The sample proportions are written as inline image and we assume the dose values are ordered such that inline image. Rather than rely on any parametric specifications on the dose-response function, however, we follow our previous construction [32] and only oblige inline image to satisfy simple continuity and monotonicity assumptions. That is, we require inline image to be a continuous function with nonnegative first derivative inline image over inline image.

We view the monotonized sequence inline image as the preliminary estimation target, for which the unique maximum likelihood estimator (MLE) is the isotonic sequence:[33]

  • display math(1)

This ensures inline image, and may be obtained via the well-known pool-adjacent-violators (PAV) algorithm. [34] To find a model-independent, isotonic estimator of the function inline image, we construct a linear interpolating spline from the PAV estimates inline image:

  • math image(2)

for inline image, and inline image. From this, we build an isotonic estimator for inline image as the corresponding linear interpolator connecting the suitably adjusted, monotonized, point-wise estimates of the extra risks at each inline image:

  • math image(3)

for inline image, and inline image. If inline image satisfies the monotonicity and continuity constraints we impose above, so will the corresponding inline image.

Now, for notational convenience, denote BMD100BMR as inline image. Given the model-independent, isotonic estimator in Equation (3), a similarly model-independent estimator for inline image is available by inverting inline image at the specified BMR and calculating the smallest positive solution; i.e., inline image. Since inline image is a form of continuous, linear interpolating spline, we can write this model-independent estimator in closed form:

  • math image(4)

(We discuss below possible strategies to accommodate the unusual case when BMR inline image.) We showed [32] that inline image possesses desirable statistical qualities: under suitable regularity conditions it converges to the target value inline image, is asymptotically unbiased, and has a large-sample distribution converging to a Gaussian (“normal”) form.

2.2. Model-Independent Bootstrap BMDLs

Unfortunately, we also discovered [32] that the asymptotic variance of inline image depends critically on the underlying risk function inline image. Since under our basic premise we are unwilling to fully specify the form of inline image, this makes it difficult to employ the asymptotic features of inline image for constructing a corresponding BMDL. For building confidence limits on an ED50, Bhattacharya and Kong[31] encountered an analogous predicament with their similarly constructed, isotonic, inline image estimator. Their solution was to appeal instead to the bootstrap, [35] and their nonparametric bootstrap-based inferences achieved respectable stability. Following their lead, we also consider the bootstrap to build model-independent BMDLs.

Appeal to bootstrapping for construction of confidence limits in risk assessment has seen increasing application, and is becoming an accepted approach for calculating critical risk-analytic quantities such as the BMDL. Most authors employ it under some form of parametric specification, however. [36-39] Here, we follow our previous construction [32] and describe a bootstrap-based approach for building BMDLs with the model-independent estimator from Equation (4). In keeping with our general estimation strategy, we do not assume any parametric form for inline image, although we do continue to impose our continuity and monotonicity assumptions. We also continue to assume a binomial parent distribution for the data.

To perform the nonparametric bootstrapping, we begin with the original proportions inline image and resample each m-dose quantal response inline image times. That is, in the bth resample at each inline image we generate the pseudorandom variate inline image from Bin.(inline image), inline image. We then apply the PAV algorithm to the inline images to yield the monotonized bootstrap sequence:

  • display math

From these we produce an isotonic bootstrap estimate of the extra risk via Equation (3). Denote this as inline image. We then apply Equation (4) to find a bootstrapped inline image. (For simplicity here we suppress the BMR subscript on inline image, but it is understood that all these operations are conducted for a fixed, prespecified BMR.) Following previously validated suggestions [40, 37] for the number, B, of bootstrap resamples, we work with inline image 2,000.

Two special cases that require attention occur when inline image or when inline image at any i. If so, the resampling will always produce inline image or inline image, respectively, generating no new bootstrap information. [41] Various remedies for this are possible; [38] in keeping with our nonparametric strategy, we refrain from applying any parametric-model constructs. Instead, we add a small constant inline image to the numerator and twice it to the denominator of the observed proportions inline image. That is, whenever inline image or inline image, we replace inline image with inline image. This shrinks the proportion away from 0 or from 1 and toward inline image. (Shrinkage toward any fixed value is admittedly arbitrary, and we chose inline image simply as an objective, default, shrinkage target. If in practice a different target were available based on experiment- or stimulus-specific considerations, it could easily be employed instead.) We experimented with a variety of possible values for ε, and found that only a slight amount of shrinkage was necessary to stabilize the bootstrap calculations. We settled on inline image, i.e., whenever inline image or inline image, replace inline image with inline image.

To find a lower inline image confidence limit on the BMD, we collect the inline images together to produce a bootstrap distribution for inline image. From this, we apply the well-known percentile method:[42] order the B bootstrapped inline image values into inline image and select as the BMDL the lower αth percentile: inline imageinline image.

This represents a model-independent, nonparametric BMDL for use in risk assessment practice.


  1. Top of page
  2. Abstract

Previously [32] we studied the performance of our model-independent BMDL, inline image100BMR, via a series of Monte Carlo simulations. The BMDL exhibited generally stable coverage characteristics, but some undercoverage—i.e., coverage rates below the nominal inline image% level—occurred for very small sample sizes and very shallow dose-response patterns. Our earlier focus was on introducing the isotonic estimation method and exploring its theoretical properties, however, and so our study was limited to a small, illustrative selection of dose-response functions. We recognized that more complete evaluations were needed to study the operating characteristics of our model-independent BMDL. Here, we expand upon our previous results to study how the isotonic method operates over a broader range of possible dose-response models/shapes.

3.1. Simulation Design

Our focus is on the empirical coverage characteristics of the bootstrap-based confidence limit inline image100BMR. We set the BMR to a standard default level BMR = 0.10, [9] and operate at 95% nominal coverage. We employ either four dose levels, inline image, corresponding to a standard design in cancer risk experimentation, [43] or six dose levels, inline image, expanding on the four-dose geometric spacing. We also include a modified six-dose design inline image that gives less focus to doses near inline image, to study how/if this affects the BMDL inline image100BMR.

For simplicity, equal numbers of subjects, inline image, are taken per dose group. We fix the total sample size at (or as near as possible to) inline image 4,000, producing per-dose sample sizes of inline image for the four-dose design and inline image for the six-dose designs.

For the underlying dose-response patterns, we employ models corresponding to a variety of functions available from the U.S. EPA's BMDS software program for performing BMD calculations.[44] Table I provides a selection of four two-parameter and four three-parameter dose-response models. These are taken from a collection of dose-response forms chosen by Wheeler and Bailer[36] in their studies of (parametric) estimators for the BMD. (Wheeler and Bailer did not include the log-logistic model (3B) in all their calculations, although they did present it as a possible data generating model. They also presented a possible data generating model based on the gamma cumulative distribution function (c.d.f.), but did not use it in their calculations. In a similar vein, we do not consider the gamma model here.) In our earlier work, [32] we studied a subset of the models in Table I: the two-stage (3A), log-probit (3C), and Weibull (3D). Thus, for these three models the simulation outcomes we give here replicate the results we reported previously. Notice that certain models impose constraints on selected parameters; these are listed in Table I to correspond with typical constraints we find in the environmental toxicology literature. In the table, for the log-probit model (3C) we define inline image. Note that the quantal-linear model (2A) may also be referred to as the “one-stage” model (a form of “multi-stage” model) or as the “complementary-log” model. This may equivalently appear as inline image, where inline image

Table I. Selected Quantal Dose-Response Models Common in Environmental Toxicology
ModelCodeinline imageConstraints/Notes
Quantal-linear2Ainline imageinline image, inline image
Quantal-quadratic2Binline imageinline image, inline image
Logistic2Cinline image
Probit2Dinline imageinline image is the N(0,1) c.d.f.
Two-stage3Ainline imageinline image, inline image
Log-logistic3Binline imageinline image, inline image
Log-probit3Cinline imageinline image, inline image
Weibull3Dinline imageinline image, inline image

To set the simulation parameters for each model, we fix the risk inline image at three doses: inline image. Based on typical patterns seen in cancer risk assessment, [41, 45] background risks at inline image are set between 1% and 30%, and the other risk levels are increased to produce a variety of (strictly) increasing forms, ending with high-dose risks at inline image between 10% and 90%. Using the specifications at inline image and inline image, we solve for two unknown parameters. This completes determination of the two-parameter models (2A–2D). For the three-parameter models (3A–3D), we additionally employ the response specification at inline image to solve for the third unknown parameter. The actual specifications and resulting parameter configurations for the various models are given in Table II; we also include the corresponding values of inline image.

Table II. Models and Configurations (Including True BMD, ξ10, at BMR = 0.10) for the Monte Carlo Evaluations
ConstraintR(0) =
 R(inline image) =
 R(1) =
Quantal-linear (2A)β00.01010.01010.10540.05130.35670.1054
Quantal-quadratic (2B)γ00.01000.01000.10000.05000.30000.1000
Logistic (2C)β0−4.5951−4.5951−2.1972−2.9444−0.8473−2.1972
Probit (2D)β0−2.3263−2.3263−1.2816−1.6449−0.5244−1.2816
Two-stage (3A)β00.01010.01010.10540.05130.35670.1054
Log-logistic (3B)γ00.01000.01000.10000.05000.30000.1000
Log-probit (3C)γ00.01000.01000.10000.05000.30000.1000
Weibull (3D)γ00.01000.01000.10000.05000.30000.1000

For each parameter configuration (labeled A–F), 2,000 individual, pseudobinomial, data sets are simulated and as noted in Section 'Model-Independent Bootstrap BMDLs', inline image 2,000 bootstrap samples are generated from each data set to produce the corresponding empirical coverage value. Notice then that the approximate standard error of the estimated coverage over all 2,000 simulated data sets is inline image, and this never exceeds inline image. All of our calculations are performed with the R statistical programming environment. [46]

3.2. Infinite BMDs

An unusual artifact we uncovered while conducting our Monte Carlo computations was that for some very shallow dose-response patterns, calculation of a nonparametric BMD can sometimes break down. One obvious case is when the BMR is set too high, so that the isotonic extra risk estimator never reaches the desired benchmark response over the range of the doses, i.e., inline image, for all inline image. If so, there is no solution to the BMD-defining relationship inline image. Of course, if the extra risk were estimated using a fully parametric (nondecreasing) function one would simply extrapolate the function outside of the dose range to find the solution. To imitate this strategy with our nonparametric estimator, suppose that inline image from Equation (3) is linear and strictly increasing along its final segment between inline image and inline image. Then if inline image, we simply extend this final line segment past inline image until it crosses the horizontal BMR line, and solve for inline image at that intersection point. While admittedly one should apply any such extrapolations past the range of the data with great caution, this strategy nonetheless allows us to report an objective estimate for the BMD in this unusual case.

The extrapolative estimate for the BMD will still fail if the final line segment from Equation (3) is flat, i.e., if inline image (or when inline image, etc.). When this occurs, the data are in effect telling us that the observed dose response cannot attain the BMR, no matter how large x grows. Correspondingly, in such an instance we simply drive the estimator inline image to ∞, or, equivalently, report it as undefined. In the extreme, this also occurs if the inline images all equal each other. In this case, Equation (3) will produce inline image, so we again are forced to drive inline image to ∞.

Somewhat perniciously, this issue of undefined or infinite BMDs was not uncommon with some of the very shallow dose-response configurations in Table II, especially configurations “A” and “C.” Particularly at the smaller sample sizes, such shallow configurations could even produce nonmonotone response patterns in the simulated data, despite the fact that the underlying inline image function was strictly increasing. This forced the PAV algorithm to “flatten out” the estimated extra risk over a large portion of the dose range, and when this occurred near the upper end of the range we encountered the infinite-BMD phenomenon.

Operationally, when any “infinite” inline image was observed in our bootstrap procedure, we set the estimate equal to machine infinity. If this occurred for more than inline image of the bootstrapped inline images, we defined inline image100BMR itself as “infinite,” and viewed this as failure to cover the true inline image.

3.3. Simulation Results

We summarize the empirical coverage results from our Monte Carlo study in a series of tables. Recall that we fix BMR = 0.10 and operate at nominal 95% coverage. Each initial table displays empirical coverage results recorded for all model configurations and sample sizes except the quantal-linear model (2A); we will discuss model 2A in greater detail below. Table III presents the results for the geometric four-dose design with inline image 0, 0.25, 0.50, 1.0: therein, coverages lie near and generally within Monte Carlo sampling variability of the nominal 95% level. Some undercoverage is observed at the lowest per-dose sample size of inline image and/or with the (shallow) response configurations “A” and, to a lesser extent, “B”; however, conservative overcoverage is at least as prevalent. Indeed, averaged across the seven models (2B–2D and 3A–3D) in Table III, the empirical coverages as a function of sample size are at or near the 95% nominal level: 94.22% for inline image, 94.76% for inline image, 95.84% for inline image, and 96.95% for inline image 1,000.

Table III. Empirical Coverage Rates of Nonparametric Bootstrap BMDL inline image10 from Monte Carlo Evaluations Under Geometric Four-Dose Design for Selected Dose-Response Models Given in Table I
Model CodeSample Size, NABCDEFRow Means
  1. Model 2A is discussed separately. Nominal coverage rate is 95%.


Table IV presents the results for the geometric six-dose design with inline image 0, 0.0625, 0.125, 0.25, 0.50, 1.0: coverage patterns therein appear slightly more stable on average than those seen in Table III, although there are also more extreme cases of undercoverage. These again occur at the lowest per-dose sample size inline image and/or with the (shallow) response configuration “A.” Overall, average empirical coverages across the seven models 2B–2D and 3A–3D are again close to nominal: 94.61% for inline image, 95.96% for inline image, 96.88% for inline image, and 97.15% for inline image 1,000. As might be expected, decreasing the per-dose sample sizes while increasing the number of doses yields greater stability at larger sample sizes, but this effect appears to reverse somewhat when sample sizes drop.

Table IV. Empirical Coverage Rates of Nonparametric Bootstrap BMDL inline image10 from Monte Carlo Evaluations Under Geometric Six-Dose Design for Selected Dose-Response Models Given in Table I
Model CodeSample Size, NABCDEFRow Means
  1. Model 2A is discussed separately. Nominal coverage rate is 95%


Results for the modified six-dose design with inline image are given in Table V. The pattern of coverage is roughly similar to that in Table IV, although somewhat larger undercoverage is evidenced at inline image. Overall, average empirical coverages across the seven models 2B–2D and 3A–3D are 93.30% for inline image, 95.19% for inline image, 96.00% for inline image, and 96.20% for inline image 1,000. At least for these designs, deemphasizing doses near to zero appeared to have limited impact.

Table V. Empirical Coverage Rates of Nonparametric Bootstrap BMDL inline image10 from Monte Carlo Evaluations Under Modified Six-Dose Design for Selected Dose-Response Models Given in Table I
Model CodeSample Size, NABCDEFRow Means
  1. Model 2A is discussed separately. Nominal coverage rate is 95%.


On balance, our simulation results exhibit generally stable large-sample coverage characteristics for the model-independent, bootstrapped limit inline image100BMR at the standard level of BMR = 0.10. Slight undercoverage is evidenced in selected instances, more so when sample sizes are small. We previously identified a possible explanation for this behavior, [32] where we found that the PAV-based estimator in Equation (4) exhibits slight negative bias when applied to convex response patterns. Negative bias in the estimator can translate to more conservative lower confidence bounds for the BMDL. On a relative scale the bias was not exceptional, however, and in fact is not wholly unexpected: bias can be a recurring issue with isotonic regression estimators. [47] Indeed, the effect moderated as sample sizes increased. Nonetheless, this suggests that the method should be applied when sufficient data are available to help validate its asymptotic motivation.

We also conducted Monte Carlo coverage evaluations for the case of BMR = 0.01. While less common, this smaller BMR may be employed in practice when sufficient data are available to support inferences at extreme low doses. [14, 9] Our results (not shown) were generally similar to those seen above; in particular, the coverage rates again drove toward the nominal 95% level as the sample size increased. At the lowest sample sizes, however, they appeared much more variable, and often in the direction of undercoverage. We encountered a few empirical coverage rates that dropped below 50% with some of the low-response-rate models, especially configurations “A” and “B.” This is, in fact, consistent with practical benchmarking experience: when response rates are very small at low doses, and if the inline images do not counter by being fairly large, insufficient information will be available to perform effective inferences if the BMR is set very low. We therefore urge caution in practice when employing these methods with very small sample sizes, particularly with low-response patterns.

3.4. Simulation Results for the (Concave) Quantal-Linear Model

Our Monte Carlo results for the quantal-linear model (2A) differed from the general trends seen with the other seven models in Table I, hence we discuss these separately. Among all the models we study, the quantal-linear model (2A) is the only strictly concave form. That is, since model 2A has inline image and inline image, its first derivative is nonnegative: inline image for all inline image. As with the rest of the models in Table I, this produces an increasing dose response. (For the trivial case where inline image the dose response will be flat and of no risk-analytic interest.) For inline image, however, the second derivative is inline image for all inline image and therefore the dose response increases at a decreasing rate: a concave function.

This feature impacts our linear interpolator. A concave response function will, more often than not, produce concave dose-response patterns in the inline images from Equation (1), and linearly interpolating a concave-increasing pattern can lead to underestimation of the extra risk. This translates as overestimation of inline image. The corresponding lower confidence bounds would, in turn, be driven up. If variation in the data is tight this could push them past the true, underlying value of inline image, collapsing the coverage rates. With very small numbers of doses, m, and large per-dose sample sizes, inline image, the rates can drop well below their nominal level. In theory the issue would quickly be remedied by increasing m, since the theoretical properties of Equation (4) obtain as both inline image and m grow large without limit. With small m, however, the potential undercoverage can be dramatic.

This effect was evidenced in our Monte Carlo coverage evaluations. Table VI presents Model 2A's small-sample coverage rates for our model-independent, bootstrap BMDL inline image100BMR at the standard level of BMR = 0.10, and with nominal coverage set to 95%. Notice the large numbers of entries below 0.95 (the zero coverage value, 0.0000, for the four-dose design under configuration “F” at inline image 1,000 is not a typographical error). In Table II, one can determine that concavity in Model 2A increases as the configuration index moves from “A” to “F.” Thus two clear trends emerge in Table VI: (i) coverage performance worsens as the concavity of the model increases, and (ii) adding more doses improves the performance more often than it debilitates it. In addition, comparing the two six-dose designs shows that as more sample information is placed closer to the true BMD—which of course is impossible in practice without knowledge of that true value—the coverage locates closer to its nominal level.

Table VI. Empirical Coverage Rates of Nonparametric Bootstrap BMDL inline image10 from Monte Carlo Evaluations for Quantal-Linear Dose-Response Model (2A) Given in Table I
Model CodeSample Size, NABCDEFRow Means
  1. Nominal coverage is 95%.

  Geometric Four-Dose Design 
  Geometric Six-Dose Design 
  Modified Six-Dose Design 

The coverage results in Table VI warn that our PAV-based benchmark estimator should be applied to concave dose-response patterns with caution. For shallow response patterns (configurations “A”–“C”), coverage is roughly similar to that seen in Tables IIIV. With greater concavity comes greater instability, however, leading to extreme degradation with the extremely concave configuration “F.”

This begs the question, how much concavity is too much? As reviewed above, the second derivative of inline image measures concavity in the response. Thus as a first approximation we can quantify the concavity in any data set by calculating how the slope of the isotonically estimated response function in Equation (2) changes from inline image to inline image.

Specifically, given PAV-based estimates inline image from Equation (1), write the slope between each adjacent pair as inline image. The change in these slopes is inline image. Then, e.g., the average change in slope, inline image quantifies concavity in the isotonically estimated response: smaller (more negative) values of inline image indicate greater concavity. Note, however, that concavity far away from the BMD has little effect on the coverage for the BMDL inline image100BMR. More pertinent for our purposes is a measure of local concavity near the estimated benchmark point. For instance, suppose the calculated BMD lies between two dose values. Then, to measure local concavity we might average the two corresponding values of inline image associated with those bracketing doses. That is, define a measure of local concavity, inline image, as:

  • math image(5)

The latter specification for inline image when inline image is based on our suggestion in Section 'Infinite BMDs' to extrapolate along a straight-line segment to define inline image beyond the upper range of the data. A possible alternative in this case is to take inline image. In any case, similar to inline image, smaller (more negative) values of inline image indicate greater local concavity.

We computed inline image for each design/parameter configuration combination in Table VI to study how this measure correlates with unstable coverage performance. (Since we had available the true dose-response information from Table II, we used inline image in place of inline image and inline image in place of inline image for the calculations.) The results appear in Table VII. As expected, the concavity measure for the shallow configurations “A”–“C” is fairly close to zero. The remaining, more-curvilinear configurations show smaller (more negative) local concavity, and correspond to patterns of greater coverage instability in Table VI. As a first step, therefore, we recommend that when the data indicate local concavity dropping below about inline image for inline image doses or inline image for inline image doses, our PAV-based linear interpolator may not be appropriate for use in constructing BMDLs. (But, see the discussion in Section 'DISCUSSION', below.)

Table VII. Local Change in Slope, inline image from Equation (5), Under Quantal-Linear Dose-Response Model (2A) Given in Table I Across Configurations from Table II
Geometric four-dose−0.0128−0.0511−0.0635−0.3341−0.5768−2.5721
Geometric six-dose−0.0128−0.0615−0.0765−0.5112−0.8273−3.7932
Modified six-dose−0.0127−0.0499−0.0618−0.4365−0.7736−3.5018


  1. Top of page
  2. Abstract

Formaldehyde, CH2O, is a well-known industrial compound, exposure to which can be extensive in a variety of occupational and environmental settings. To explore the toxic and carcinogenic potential of the chemical, Schlosser et al.[48] reported on nasal squamous cell carcinomas observed in laboratory rats after chronic, two-year, inhalation exposure. The CH2O exposure dose, x, is actually a concentration (in ppm) here, and so technically we will compute BMCs based on the quantal carcinogenicity data. Six CH2O concentrations were studied: inline image 0.0, 0.7, 2.0, 6.0, 10.0, and 15.0 ppm. Since intercurrent mortality can occur in such chronic-exposure studies, the final tumor incidences were adjusted for potential differences in animal survival. Table VIII lists the survival-adjusted proportions.

Table VIII. Formaldehyde Carcinogenicity Data[48]
Exposure Conc. (ppm), inline image0.
Adjusted tumor incidence, inline image000321150
Animals at risk, inline image1222712611334182

Of interest is calculation of the BMC, and more importantly a 95% lower confidence limit, BMCL, to help inform risk characterization on this potential carcinogen. (Our analysis here is intended primarily to illustrate the nonparametric, model-independent methodology, and not to supersede the larger risk analysis reported by Schlosser et al.[48] or by any previous authors regarding CH2O carcinogenicity.) We make only the (reasonable) assumption that the true dose response is continuous and monotone nondecreasing over inline image, and operate at BMR = 0.10. From Table VIII, the observed response is already nondecreasing, so the per-concentration PAV estimates inline image are simply the observed proportions: inline image. Constructing the isotonic extra risk estimator in Equation (3) and setting it equal to BMR = 0.10 produces the model-independent estimate inline image ppm.

For the BMCL we apply the percentile bootstrap as described in Section 'Model-Independent Bootstrap BMDLs'. To do so, we first check that local concavity near the BMC as measured by Equation (5) is acceptable: we see inline image so take inline image. From Table VIII we find inline image, and inline image, with inline image and inline image. Thus, inline image. This is near zero and positive, indicating slight local convexity in the neighborhood of inline image. Hence, no problematic issues with potential dose-response concavity are evidenced and we proceed with our nonparametric bootstrap BMCL.

For the bootstrap, we generate inline image 2,000 resamples from the original data, and find no occurrences of infinite inline images. Our 95% BMCL is the lower 5th percentile from this distribution: inline image10 = 6.332 ppm. By comparison, Schlosser et al.[48] reported a BMC10 of 6.90 ppm with corresponding 95% BMCL10 = 6.25 ppm under the log-probit model, a BMC10 of 6.40 ppm with corresponding 95% BMCL10 = 6.22 ppm under a form of the Weibull model, and a BMC10 of 6.40 ppm with corresponding 95% BMCL10 = 6.22 ppm under a form of multistage model. All sets of values rest in similar ranges, and provide comparable points of departure for conducting further risk-analytic calculations on formaldehyde carcinogenicity. Thus, while our model-independent approach operates similarly to established parametric analyses for these data, it also provides an added benefit: it frees the risk assessor from uncertainties about the quality of any parametric assumptions made to support the benchmark computations.


  1. Top of page
  2. Abstract

Herein, we consider a model-independent, nonparametric method for estimating BMDs in quantitative risk analysis. Placing emphasis on cancer risk assessment, we describe an approach for estimating the BMD without call to any specific parametric dose-response models, relying only on the assumptions that the underlying response is continuous and monotone nondecreasing. [31, 30] Lower confidence limits (BMDLs) using this estimator are derived from nonparametric bootstrap methods, building upon previous explorations into bootstrap resampling for benchmark inference. [49, 37] Based on a Monte Carlo study, we find that the BMDLs exhibit relatively stable coverage for reasonably large sample sizes, but that some undercoverage can occur for very small sample sizes and very shallow dose-response patterns. Since the method is based on linearly interpolating the nonparametrically estimated risk function, the undercoverage is exacerbated if the dose-response pattern is highly concave and the number of doses is small; in such cases we cannot recommend use of this procedure. (But, see below.)

When sufficient data are available to support the nonparametric constructions, risk analysts can apply our results to build inferences on the BMD that avoid concerns over parametric model adequacy, expanding past the many, varied parametric models seen in practice. This extended operability can lead to improved risk analytic decisionmaking in carcinogenicity testing and other adverse-event risk assessments.

Of course, some caveats and qualifications are in order. The percentile method we used for finding the BMDL inline image100BMR is a basic approach for constructing bootstrap inferences. While the method appears generally stable at larger sample sizes, its mixed performance with very small samples might be improved by moving to more-complex bootstrapping strategies. For instance, the bias-corrected, accelerated (BCa) bootstrap [35] is a well-known alternative to the percentile method, so we evaluated the BCa approach under the standard four-dose design using our convex-model Monte Carlo configurations from Section 'Simulation Design'. We found that the resulting BMDLs exhibited slightly tighter empirical coverages at smaller sample sizes and with the more problematic, shallow, response patterns. Improvements were not seen across all configurations studied, however. (Details are available in a separate document.[50]) From this, we can recommend use of BCa-based BMDLs when samples sizes are very small and/or for shallow response patterns; however, the basic percentile method appeared to operate adequately in the majority of cases we studied.

As we note in Section 'Model Dependence', a number of model-robust competitors to our nonparametric BMDL exist, primarily focused on (parametric) model-averaging techniques. While a complete comparison among all these methods is beyond the scope here, we can make a direct comparison with a frequentist model-averaged (FMA) BMDL proposed by Piegorsch et al.[19] Their method employed a weighted average of BMD estimators over an “uncertainty class” of different parametric models for inline image. Information-theoretic weights were used to construct the model-averaged BMD point estimate, inline image; an associated standard error, inline image; and from these, a large-sample Wald-type lower confidence limit, inline image, where inline image is the upper-α critical point from a standard normal distribution. Assuming the uncertainty class is constructed to capture an appropriate set of potential models for inline image, Piegorsch et al. found that their FMA BMDL exhibits substantial model-robustness for building benchmark dose lower limits.

Comparison of the FMA BMDL with our nonparametric BMDL inline image100BMR is possible here: fortunately, Piegorsch et al. employed the exact same models and parametric configurations as in our Table II for their own small-sample simulation study of inline image's coverage characteristics. (They considered only the geometric four-dose design, but they did use the same BMR = 0.10 and nominal 95% confidence level as we have here.) Thus, it is possible to explicitly compare the empirical coverage rates they achieved under their parametric FMA BMDL with those we find under our nonparametric BMDL. We plot in Fig. 1 the associated pairs of empirical coverages for all 192 model/configuration/sample size combinations represented in Table III and for Model 2A, with the geometric four-dose design (Table VI). In the figure, empirical coverage for the FMA BMDL is plotted on the vertical axis and that for our nonparametric BMDL is plotted on the horizontal axis. (Note the different axis scales.) If both methods operated perfectly, we would expect a tight cluster of points to lie at the crossing of the two “95% nominal coverage” lines in the figure. Instead, the graphic portrays a more complex scenario. It corroborates the near-to-nominal coverage for our nonparametric BMDL seen in Table III, but it also highlights the few distressingly low coverage values under Model 2A in Table VI: see the points lying far to the left of the vertical rule marking nominal coverage. Indeed, the horizontal scale is lengthened by these few low-coverage points, making it difficult to visualize the larger comparison. To compensate, Fig. 2 repeats the plot with all the Model 2A points removed. The finer detail illustrates that by excluding Model 2A, very few cases occur where both methods produce significant undercoverage (since the lower left quadrant in the plot is almost empty), and that the FMA BMDL is often more conservative than our nonparametric BMDL (since the upper right quadrant in the plot is the most dense). Indeed, the upper half of the plot is far more populated, indicating the more-conservative coverage of the FMA BMDL. (This was, in fact, explicitly recognized by Piegorsch et al.:[19] their method included a conservative approximation for inline image to avoid cases of severe undercoverage, and this is evident in Fig. 1: to within Monte Carlo sampling error, only a handful of points lie below the nominal 95% level.) This conservatism does provide for some added “safety” in use of inline image: in contrast to the nonparametric BMDL studied here, the FMA BMDL does not fall victim to problems of benchmark overestimation with concave dose-response patterns. As Piegorsch et al. readily admitted, however, their FMA BMDL is dependent on proper development of a pertinent uncertainty class of parametric models. In cases where this is not possible, the nonparametric BMDL we suggest here can serve as a model-robust alternative, when employed cautiously with concave-increasing data (as described in Section 'Simulation Results for the (Concave) Quantal-Linear Model').


Figure 1. Empirical coverage rates between nonparametric BMDL inline image10 from Section 'Model-Independent Bootstrap BMDLs' and parametric model-averaged BMDL[19] at BMR = 0.10 under geometric four-dose design using configurations from Table II. (Note difference in scales.) Horizontal and vertical lines indicate nominal 95% coverage level. All models from Table I are included.

Download figure to PowerPoint


Figure 2. Empirical coverage rates between nonparametric BMDL inline image10 from Section 'Model-Independent Bootstrap BMDLs' and parametric model-averaged BMDL [19] at BMR = 0.10 under geometric four-dose design using configurations from Table II, excluding Model 2A from Table I. Horizontal and vertical lines indicate nominal 95% coverage level.

Download figure to PowerPoint

It is also important to emphasize that our nonparametric method requires both the number of doses m and the per-dose sample sizes inline image to grow large. [32] In practice, however, resource and design constraints can conspire to restrict one or both of these factors. With small m in particular, the dose spacings are often sparse and wide; this can compromise the ability of our nonparametric linear interpolator to adequately describe the dose-response pattern. Indeed, Muri et al.[51] report that the most prevalent study design they found for estimating a BMD among 20 pesticide risk analyses employed only inline image doses, and did so almost twice as often as the next most-common design (which was inline image). When studying the U.S. EPA's Integrated Risk Information System Database (, Nitcheva et al.[52] found that that the most common number of doses used among 91 rodent carcinogenicity studies was even lower, at inline image. (Clearly, designs with as few as inline image doses/concentrations spread the dose placement farther apart and limit our isotonic estimator's ability to capture information in the data. We do not recommend application of our techniques to such sparse, minimally informative designs.) Next most common was again inline image.

Referring to our Monte Carlo results in Table III, we see that with inline image doses the nonparametric BMDL's coverage characteristics are relatively stable if sufficient numbers of subjects/dose, N, are employed. Still, one questions whether we can gain greater information about the pattern of dose response, and therefore about inline image, if we increase the number of doses. Somewhat more-stable coverage patterns were seen with inline image doses in Tables IV and V. So, can increasing m to, say, 10 doses improve the small-sample operating characteristics of the BMDL if resource constraints force the inline images down to perhaps only 10 subjects/dose? For example, Bhattacharya and Lin[53] were able to show that an adaptive nonparametric estimator based on inverting Equation (2) proved competitive in such a setting. To examine how this extends to BMD estimation with extra risk functions, we repeated our Monte Carlo evaluations for all eight models in Table I with inline image doses spaced evenly between inline image and inline image. We again studied constant per-dose sample sizes, N, using inline image 10, 20, 40, and 400 to compare with the total sample sizes in Table III. (All other aspects of the calculations for the bootstrap BMDL inline image10 remained the same.) The results appear in Table IX; note the inclusion of Model 2A at the top of the table.

Table IX. Empirical Coverage Rates of Nonparametric Bootstrap BMDL inline image10 from Monte Carlo Evaluations Under Equi-Spaced 10-Dose Design for Dose-Response Models Given in Table I
Model CodeSample Size, NABCDEFRow Means
  1. Nominal coverage rate is 95%.


The patterns of coverage in Table IX appear roughly comparable to those in Tables IIIVI: only with configuration “A” at inline image does the coverage consistently weaken. Indeed, a highly encouraging indication is the improved stability in coverage for Model 2A. With inline image doses the 2A rates now appear commensurate with patterns seen for most other models—even at the more-concave configurations—extending the indications seen in Table VI. At least as regards estimation and inferences in benchmark risk assessment, this provides strong encouragement to include larger numbers of doses for characterizing the response when designing modern dose-response studies. [54, 55, 22]


  1. Top of page
  2. Abstract

Thanks are due two anonymous referees and the area editor for helpful and supportive comments on an earlier version of this material. Portions of the work were conducted while the fourth author was with the Department of Mathematics at the University of Arizona. These results represent part of the second author's Ph.D. dissertation with the University of Arizona Graduate Interdisciplinary Program in Applied Mathematics. The research was supported by grant #R21-ES016791 from the U.S. National Institute of Environmental Health Sciences. Its contents are solely the responsibility of the authors and do not necessarily reflect the official views of this funding agency. The authors declare no other forms of competing interests.


  1. Top of page
  2. Abstract
  • 1
    Crump KS. A new method for determining allowable daily intake. Fundamental and Applied Toxicology, 1984; 4:854871.
  • 2
    Piegorsch WW, Bailer AJ. Analyzing Environmental Data. Chichester: John Wiley & Sons, 2005.
  • 3
    Crump KS. Calculation of benchmark doses from continuous data. Risk Analysis, 1995; 15:7989.
  • 4
    Kodell RL. Managing uncertainty in health risk assessment. International Journal of Risk Assessment and Management, 2005; 14:193205.
  • 5
    Izadi H, Grundy JE, Bose R. Evaluation of the benchmark dose for point of departure determination for a variety of chemical classes in applied regulatory settings. Risk Analysis, 2012; 32:830835.
  • 6
    U.S. General Accounting Office. Chemical Risk Assessment. Selected Federal Agencies' Procedures, Assumptions, and Policies. Report to Congressional Requesters Number GAO-01-810. Washington, DC: U.S. General Accounting Office, 2001.
  • 7
    European Union. Technical Guidance Document (TGD) on Risk Assessment of Chemical Substances Following European Regulations and Directives, Parts I–IV. Technical Report Number EUR 20418 EN/1-4. Ispra, Italy: European Chemicals Bureau (ECB), 2003.
  • 8
    Organisation for Economic Co-Operation and Development (OECD). Draft Guidance Document on the Performance of Chronic Toxicity and Carcinogenicity Studies, Supporting TG 451, 452 and 453. Paris: Organisation for Economic Co-Operation and Development, 2008.
  • 9
    U.S. EPA. Benchmark Dose Technical Guidance Document. Technical Report Number EPA/100/R-12/001. Washington, DC: U.S. Environmental Protection Agency, 2012.
  • 10
    Piegorsch WW. Quantal response data. Pp. 20652067 in El-Shaarawi AH, Piegorsch WW (eds). Encyclopedia of Environmetrics, 2nd ed., Vol. 4. Chichester: John Wiley & Sons; 2012.
  • 11
    Faustman EM, Bartell SM. Review of noncancer risk assessment: Applications of benchmark dose methods. Human and Ecological Risk Assessment, 1997; 3:893920.
  • 12
    Kang S-H, Kodell RL, Chen JJ. Incorporating model uncertainties along with data uncertainties in microbial risk assessment. Regulatory Toxicology and Pharmacology, 2000; 32:6872.
  • 13
    Chapman PM, Caldwell RS, Chapman PF. A warning: NOECs are inappropriate for regulatory use. Environmental Toxicology and Chemistry, 1996; 18:7779.
  • 14
    Kodell RL. Replace the NOAEL and LOAEL with the BMDL01 and BMDL10. Environmental and Ecological Statistics, 2009; 16:312.
  • 15
    West RW, Piegorsch WW, Peña EA, An L, Wu W, Wickens AA, Xiong H, Chen W. The impact of model uncertainty on benchmark dose estimation. Environmetrics, 2012; 23:706716.
  • 16
    Moon H, Kim H-J, Chen JJ, Kodell RL. Model averaging using the Kullback information criterion in estimating effective doses for microbial infection and illness. Risk Analysis, 2005; 25:11471159.
  • 17
    Faes C, Aerts M, Geys H, Molenberghs G. Model averaging using fractional polynomials to estimate a safe level of exposure. Risk Analysis, 2007; 27:111123.
  • 18
    Wheeler MW, Bailer AJ. Comparing model averaging with other model selection strategies for benchmark dose estimation. Environmental and Ecological Statistics, 2009; 16:3751.
  • 19
    Piegorsch WW, An L, Wickens AA, West RW, Peña EA, Wu W. Information-theoretic model-averaged benchmark dose analysis in environmental risk assessment. Environmetrics, 2013; 24:143157.
  • 20
    Bailer AJ, Noble RB, Wheeler MW. Model uncertainty and risk estimation for experimental studies of quantal responses. Risk Analysis, 2005; 25:291299.
  • 21
    Morales KH, Ibrahim JG, Chen C-J, Ryan LM. Bayesian model averaging with applications to benchmark dose estimation for arsenic in drinking water. Journal of the American Statistical Association, 2006; 101:917.
  • 22
    Shao K, Small MJ. Statistical evaluation of toxicological experimental design for Bayesian model averaged benchmark dose estimation with dichotomous data. Human and Ecological Risk Assessment, 2012; 18:10961119.
  • 23
    Krewski D, Gaylor D, Szyszkowicz M. A model-free approach to low dose extrapolation. Environmental Health Perspectives, 1991; 90:279285.
  • 24
    Bosch RJ, Wypij D, Ryan L. A semiparametric approach to risk assessment for quantitative outcomes. Risk Analysis, 1996; 16:657666.
  • 25
    Fine JP, Bosch RJ. Risk assessment via a robust probit model, with application to toxicology. Journal of the American Statistical Association, 2000; 95:375382.
  • 26
    Wheeler MW, Bailer AJ. Monotonic Bayesian semiparametric benchmark dose analysis. Risk Analysis, 2012; 32:12071218.
  • 27
    Guha N, Roy A, Kopylev L, Fox J, Spassova N, White P. Nonparametric Bayesian methods for benchmark dose estimation. Risk Analysis, 2013, in press. doi:10.1111/risa.12004.
  • 28
    Dette H, Neumeyer N, Pilz KF. A note on nonparametric estimation of the effective dose in quantal bioassay. Journal of the American Statistical Association, 2005; 100: 503510.
  • 29
    Dette H, Scheder R. A finite sample comparison of nonparametric estimates of the effective dose in quantal bioassay. Journal of Statistical Computation and Simulation, 2010; 80: 527544.
  • 30
    Bhattacharya RN, Lin L. An adaptive nonparametric method in benchmark analysis for bioassay and environmental studies. Statistics and Probability Letters, 2010; 80:19471953.
  • 31
    Bhattacharya RN, Kong M. Consistency and asymptotic normality of the estimated effective doses in bioassay. Journal of Statistical Planning and Inference, 2007; 137:643658.
  • 32
    Piegorsch WW, Xiong H, Bhattacharya RN, Lin L. Nonparametric estimation of benchmark doses in quantitative risk analysis. Environmetrics, 2012; 23:717728.
  • 33
    Ayer M, Brunk HD, Ewing GM, Reid WT, Silverman E. An empirical distribution function for sampling with incomplete information. Annals of Mathematical Statistics, 1955; 26:641647.
  • 34
    Silvapulle MJ, Sen PK. Constrained Statistical Inference: Order, Inequality, and Shape Constraints. New York: John Wiley & Sons, 2004.
  • 35
    DiCiccio TJ, Efron B. Bootstrap confidence intervals (with discussion). Statistical Science, 1996; 11:189228.
  • 36
    Wheeler MW, Bailer AJ. Properties of model-averaged BMDLs: A study of model averaging in dichotomous response risk estimation. Risk Analysis, 2007; 27:659670.
  • 37
    West RW, Nitcheva DK, Piegorsch WW. Bootstrap methods for simultaneous benchmark analysis with quantal response data. Environmental and Ecological Statistics, 2009; 16:6373.
  • 38
    Buckley BE, Piegorsch WW, West RW. Confidence limits on one-stage model parameters in benchmark risk assessment. Environmental and Ecological Statistics, 2009; 16:5362.
  • 39
    Moon H, Kim SB, Chen JJ, George NI, Kodell RL. Model uncertainty and model averaging in the estimation of infectious doses for microbial pathogens. Risk Analysis, 2013; 33:220231.
  • 40
    Moerbeek M, Piersma AH, Slob W. A comparison of three methods for calculating confidence intervals for the benchmark dose. Risk Analysis, 2004; 24:3140.
  • 41
    Bailer AJ, Smith RJ. Estimating upper confidence limits for extra risk in quantal multistage models. Risk Analysis, 1994; 14:10011010.
  • 42
    DiCiccio TJ, Romano JP. A review of bootstrap confidence intervals. Journal of the Royal Statistical Society, Series B (Methodological), 1988; 50: 338354.
  • 43
    Portier CJ. Biostatistical issues in the design and analysis of animal carcinogenicity experiments. Environmental Health Perspectives, 1994; 102 (Suppl. 1):58.
  • 44
    Davis JA, Gift JS, Zhao QJ. Introduction to benchmark dose methods and U.S. EPA's benchmark dose software (BMDS) version 2.1.1. Toxicology and Applied Pharmacology, 2012; 254:181191.
  • 45
    Buckley BE, Piegorsch WW. Simultaneous confidence bands for Abbott-adjusted quantal response models in benchmark analysis. Statistical Methodology, 2008; 5: 209219.
  • 46
    R Development Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing, 2011.
  • 47
    Zhao O, Woodroofe M. Estimating a monotone trend. Statistica Sinica, 2012; 22:359378.
  • 48
    Schlosser PM, Lilly PD, Conolly RB, Janszen DB, Kimbell JS. Benchmark dose risk assessment for formaldehyde using airflow modeling and a single-compartment, DNA-protein cross-link dosimetry model to estimate human equivalent doses. Risk Analysis, 2003; 23:473487.
  • 49
    Zhu Y, Wang T, Jelsovsky JZH. Bootstrap estimation of benchmark doses and confidence limits with clustered quantal data. Risk Analysis, 2007; 27:447465.
  • 50
    Xiong H. Nonparametric statistical approaches for benchmark dose estimation in quantitative risk assessment. Ph.D. dissertation, Program in Applied Mathematics, University of Arizona, Tucson, AZ, 2011.
  • 51
    Muri SD, Schlatter JR, Brüschweiler BJ. The benchmark dose approach in food risk assessment: Is it applicable and worthwhile? Food and Chemical Toxicology, 2009; 47:29062925.
  • 52
    Nitcheva DK, Piegorsch WW, West RW. On use of the multistage dose-response model for assessing laboratory animal carcinogenicity. Regulatory Toxicology and Pharmacology, 2007; 48:135147.
  • 53
    Bhattacharya RN, Lin L. Nonparametric benchmark analysis in risk assessment: A comparative study by simulation and data analysis. Sankhya, Series B, 2011; 73:144163.
  • 54
    Sand SJ, Victorin K, Falk Filipsson A. The current state of knowledge on the use of the benchmark dose concept in risk assessment. Journal of Applied Toxicology, 2008; 28:405421.
  • 55
    Öberg, M. Benchmark dose approaches in chemical health risk assessment in relation to number and distress of laboratory animals. Regulatory Toxicology and Pharmacology, 2010; 58:451454.