Eric D. Hsi, MD, Department of Clinical Pathology, L-11, Cleveland Clinic Foundation, 9500 Euclid Ave, Cleveland, OH 44195, USA. E-mail: firstname.lastname@example.org
Summary. CD38 expression on chronic lymphocytic leukaemia (CLL) cells is a poor prognostic factor, however, methods for measuring this vary. The QuantiBRITETM flow cytometry (FC) system yields an absolute antigen expression value (antibodies bound/cell, ABC) and may be useful in standardizing CD38 expression analysis. We evaluated cryopreserved pretreatment CLL cells for CD20 ABC, CD38 ABC, and percentage of CD38+ B cells from 131 patients requiring therapy. The 92 patients (70%) with ≥ 100 CD38 ABC had worse overall survival (OS; 34% at 5 years) compared with those with < 100 CD38 ABC (70% at 5 years, mortality hazard ratio 2·30, 95% confidence interval 1·28–4·12; two-tailed P = 0·003). Among the 64 patients with < 30% CD38+ cells, OS of the 25 with ≥ 100 ABC was worse than that of the 39 with < 100 ABC (P = 0·018). OS of patients with < 30% CD38+ cells and ≥ 100 ABC was actually similar to that of patients with ≥ 30% CD38+ cells. BrightCD20 expression (≥ 20 000 ABC) was not associated with a worse OS (P = 0·10). The presence of ≥ 100 CD38 ABC in CLL patients requiring therapy is an unfavourable prognostic factor for OS and quantitative FC may be superior to percentage CD38+ cell assessment. Prospective trials are required to determine more precisely the prognostic significance of absolute expression levels in fresh CLL cells.
B-cell chronic lymphocytic leukaemia (CLL) is the most common type of human leukaemia (Foon & Gale, 1987; Rozman & Montserrat, 1995). It is a disease of older adults with a median age at diagnosis of 65 years. Morphologically, CLL cells are small round lymphocytes, with a condensed chromatin pattern, that are almost indistinguishable from normal mature lymphocytes. Immunophenotypically, CLL is characterized by expression of CD5, CD19 and CD23 (Harris et al, 1994). The disease course is typically indolent with many patients dying with disease rather than of disease. However, a substantial number of patients do die of progressive disease. Clinical staging systems have been devised to help predict the course of these patients (Rai et al, 1975; Binet et al, 1981). A shortcoming of these systems is the progression of some intermediate-stage patients that is not predictable by current methods (Zwiebel & Cheson, 1998).
In late 1999, CLL was shown to be separable into two types, based on the mutational status of the variable region of the immunoglobulin heavy chain (IgH) gene (Damle et al, 1999; Hamblin et al, 1999). Hamblin and colleagues reported that CLL patients with unmutated IgH genes had a worse survival than patients whose CLL cells had mutated IgH genes (Hamblin et al, 1999). They sequenced the IgH genes of leukaemic cells from 84 patients with CLL and correlated their findings with clinical features. Forty-five per cent showed sequence homology with the nearest germline V(H) gene; 55% showed somatic mutation. Unmutated V(H) genes were associated with advanced stage and progressive disease. Survival was significantly worse for patients with unmutated IgH genes, independent of clinical stage. These data were supported by the companion report by Damle et al (1999), who showed that CD38 expression status correlated with IgH mutational status. CLL patients with unmutated IgH genes and higher CD38 expression showed significantly shorter survival and poorer response to chemotherapy compared with those with IgH mutations and low CD38 expression. Subsequent studies have confirmed the clinical significance of CD38 expression (Damle et al, 1999; D'Arena et al, 2001; Ibrahim et al, 2001; Matrai et al, 2001) and recent studies suggest it is a prognostic factor independent of IgH mutational status (Hamblin et al, 2000, 2002; Jelinek et al, 2001).
Flow cytometric evaluation of CD38 would be preferable to determining IgH mutational status for prognostication in CLL because of the technical complexity and expense of the latter analysis. In order to bring this type of testing into the clinical arena, a reproducible method must be available that will give similar results in different laboratories. As it stands, varying reagents, instrumentation and subjective determination of positive percentages may result in ambiguity of this test. Antigen quantification has the potential to yield reproducible results with a low coefficient of variation (Gratama et al, 1998; Hultin et al, 1998).
We tested a series of cryopreserved CLL samples, derived from patients included in co-operative group trials, for CD38 expression using the QuantiBRITETM quantitative flow cytometry system. This system provides an absolute measure of antigen expression (antibodies bound per cell, ABC) and makes use of a highly purified phycoerythrin (PE) CD38 reagent with a 1:1 fluorochrome to protein (F:P) ratio and multilevel calibrated beads with known absolute PE fluorescence. This system has already shown to be useful in other clinical settings such as human immunodeficiency virus (HIV) monitoring (Davis et al, 1998; Iyer et al, 1998). The purpose of this study was to determine whether CD38 expression levels remained a significant prognostic factor in a multicentre CLL patient population that required therapy, and to determine whether an absolute expression level could be defined that appeared to predict clinical outcome.
Materials and methods
Patient samples. Patients in this study were registered on one of two clinical trials conducted by the South-west Oncology Group (SWOG) for the treatment of patients with B-cell CLL of high risk (Rai stage III or IV), or of intermediate risk (Rai stage I or II) with evidence of active disease. Study SWOG-9108 comprised the SWOG's participation in the recent intergroup phase III trial comparing fludarabine (Flud), chlorambucil (Chlor) and Flud plus Chlor (Rai et al, 2000). SWOG-9706 was a phase II study of high-dose cyclophosphamide (Cy) followed by Flud. Each patient signed an informed consent form approved by his or her local Institutional Review Board. This study included all patients who met the eligibility criteria of their respective clinical trials and for whom viable cryopreserved pretreatment lymphocytes were available from the SWOG Lymphoid Repository. Samples were shipped on dry ice from the SWOG Repository to the Cleveland Clinic by overnight express mail and were received in good condition. Excess fresh and frozen cells from 10 additional CLL patients were used to examine the effect of cryopreservation on CD38 ABC.
Flow cytometry. Cells (1 × 106) were resuspended in 100 µl of phosphate-buffered saline and 1% bovine serum albumin (Sigma Chemical, St Louis MO, USA), and were stained with each of the following antibodies: CD5-fluoroscein isothiocyanate (FITC, 10 µl), CD19-allophycocyanin (APC, 5 µl), CD45-perdinin chlorophyll protein (PerCP, 10 µl) and 20 µl of either CD20-PE or CD38-PE. The CD20-PE and CD38-PE were QuantiBRITETM reagents (Becton Dickinson, San Jose, CA, USA). Cells were incubated for 15 min at room temperature, washed and analysed on a FACSCaliburTM flow cytometer (Becton Dickinson), according to manufacturer specifications. CD19/CD5/CD45-positive cells were analysed for CD20 and CD38 quantitative antigen expression using the QuantiBRITETM system. The multilevel calibrated QuantiBRITETM fluorescent beads enable the construction of a standard curve for antigen quantification. Using software provided by the manufacturer (quanticalcTM; Becton Dickinson), measured sample fluorescence was converted to ABC. The percentage of CLL cells expressing CD38 (%CD38+) was also determined using auto-fluorescence of cells that had not been stained with a PE-conjugated antibody as the negative control to set the threshold for CD38 expression. The PE fluorescence threshold was set so that less than 2% of unstained events were in the positive range.
Statistical analysis. Demographic and clinical data were collected and evaluated according to standard procedures of the Cancer and Leukaemia Group B (for SWOG-9108) or SWOG (for SWOG-9706). Absolute antigen expression levels were analysed in relation to other variables using standard non-parametric techniques: Wilcoxon's rank sum test for comparisons between groups and Spearman's rank order coefficient for estimating correlations with quantitative factors. The other factors used in the analysis included clinical trial (SWOG-9108 versus SWOG-9706), age, haemoglobin (Hb), platelet count (PLT), leucocyte count (WBC), peripheral lymphocyte percentage and absolute count, and Rai stage. Overall survival (OS) was measured from the date of registration on the patient's clinical trial until death from any cause, with observation censored at the date patients were last known to be alive for those not known to have died. Response to induction chemotherapy was assessed according to criteria proposed by the National Cancer Institute (NCI)-sponsored working group prior to study SWOG-9108 (Cheson et al, 1988). Duration of response was measured from the date of initial response until disease relapse or progression, or death from any cause, with observation censored at the date of last contact for patients last known to be alive without report of relapse. Distributions of OS and response duration were estimated using the method of Kaplan and Meier, and analysed in relation to antigen expression levels and other factors using proportional hazards regression (PH) models (Kaplan & Meier, 1958; Cox, 1972). The standard PH model, relating an outcome to a covariate such as antigen expression level, assumes that the event hazard rate varies as a log linear function of the covariate. As this assumption might not be true for CD38 and CD20 expression levels, these variables were analysed both as quantitative (‘continuous’) variables and in categorized form.
A total of 209 patients were registered on studies SWOG-9108 (n = 158) and SWOG-9706 (n = 51). Of these, 131 (63%) met the inclusion criteria for this study, i.e. met all eligibility criteria for their clinical trial, and had cells available for CD38 and CD20 analysis (91 from SWOG-9108 and 40 from SWOG-9706). Table I summarizes the patient characteristics. Among the 131 patients, 32 were women (24%) and ages ranged from 35 to 81 years (median 60 years). Sixty-four patients (53%) had high-risk CLL (i.e. Rai stage III or IV) and most (110 patients) had good performance status (0–1). Treatment arms were evenly distributed. From the SWOG-9108 trial, 24 patients (18%) were treated with Flud, 36 (27%) with Chlor and 31 (24%) with Flud plus Chlor. The 40 patients (31%) from the SWOG-9706 trial were treated with Cy + Flud. The 131 included and 78 excluded patients were generally similar, although the included patients had higher leucocyte counts and absolute peripheral lymphocyte counts (results not shown), probably reflecting the fact that patients with higher counts were more likely to have cells available for study.
Table I. Demographic and clinical characteristics of 131 CLL patients.
Patients (n = 131)
Modified Rai stage
Initial induction chemotherapy
SWOG-9108 Flud + Chlor
SWOG-9706 Cy + Flud
Platelets (× 109/l)
WBC (× 109/l)
Peripheral lymphocytes (%)
Peripheral lymphocytes (× 109/l)
Pretreatment cryopreserved CLL samples were retrieved from the SWOG Lymphoid Repository and analysed for CD20 and CD38 expression. Quantitative flow cytometry, gated on CD19/CD5 dual expressing cells showed a variable expression of CD20 and CD38 (Table II). There was wide variation in CD20 expression, which ranged from 609 to 104 732 ABC with a median of 9025. The median CD38 expression was 216 ABC with a range of 66–6245. Expression levels of the two antigens were significantly, though modestly, positively correlated (Spearman's rank order correlation coefficient 0·27, P = 0·0015; Fig 1). Antigen expression was not significantly greater in the high-risk (HR) compared with intermediate-risk (IR) patients for either CD38 (median 243 ABC for HR and 179 for IR; P = 0·45) or CD20 (9256 vs 8638 ABC; P = 0·24, Wilcoxon two sample test). There was no significant difference in expression of CD38 between SWOG-9108 (median ABC 189) and SWOG-9706 (median 413 ABC; P = 0·27). However, CD20 was somewhat higher in SWOG-9706 patients compared with SWOG-9108 patients (median of 12 094 and 8177 ABC respectively; P = 0·024).
Table II. Distributions of absolute CD38 and CD20 expression levels in 131 CLL patients.
Absolute antigen expression level (ABC)
10 000–19 999
≥ 20 000
Sixty-seven of the 131 patients have died, and the remaining 64 have survived between 13 and 103 months (median 39 months). Table III shows how OS varied among categories defined by CD38 and CD20. There was no significant trend of OS in relation to CD38 expression level when the latter was treated as a continuous variable in the standard PH model (P = 0·88). However, as shown in Table III, patients with ≥ 100 CD38 ABC had mortality hazard ratios of about two or greater compared with those with < 100 CD38 ABC. Thus, using a cut-off of 100 ABC to define CD38 positivity, the 92 CD38-positive CLL patients were found to have a significantly worse OS compared with the 39 CD38-negative patients [estimated hazard ratio 2·30, 95% confidence interval (CI) 1·28–4·12, P = 0·003]. The estimated probability of survival at 5 years was 70% (95% CI 55–85%) for CD38-negative patients, but only 34% (22–46%) for CD38-positive patients (Fig 2). This survival difference could not be explained by confounding effects of demographic or clinical factors as CD38 expression was not strongly correlated with age, Hb, PLT, WBC or absolute lymphocyte count (Table IV). In addition, CD38 expression did not differ markedly in relation to modified Rai stage (median 243 ABC for HR and 179 for IR; P = 0·45).
Table III. Overall survival of 131 CLL patients by absolute CD38 and CD20 expression levels.
Two-tailed P-value for significance of correlation coefficient.
PLT (× 109/l)
WBC (× 109/l)
Periph. lymphocytes (%)
Periph. lymphocytes (× 109/l)
A similar survival analysis was done for CD20 expression. Overall, there was a marginally significant trend of decreasing OS in relation to increasing CD20 expression level (P = 0·045). However, as shown in Table III, OS was actually somewhat better in patients with intermediate levels of CD20 expression (5000–19 999 ABC) compared with those with either lower or higher levels. Consequently, a cut-off could not be found that segregated patients into groups with distinct prognoses for OS. Similar to CD38 expression, CD20 expression was not significantly correlated with other clinical factors (Table IV), and did not differ significantly between patients in the HR (median 9256 ABC) and IR (median 8648 ABC) groups (P = 0·24).
Twenty of the 131 patients achieved complete remission (CR) with their initial chemotherapy regimens and another 44 achieved partial responses (PR). Further analyses were performed to assess whether response to initial induction chemotherapy or response duration varied in relation to CD38 or CD20 expression. There were no statistically significant trends for CR, total response (CR or PR) or response duration in relation to the absolute expression level of either antigen (results not shown).
The effect of cryopreservation on CD38 expression was investigated in 10 CLL samples in which antigen expression was evaluated in freshly obtained cells and paired cryopreserved specimens. This demonstrated a variable effect with an overall decrease in antigen expression. The cryopreserved paired samples had a median of 79% of the CD38 expression level compared with the freshly measured cells (range 49–131%, data not shown).
Comparison of CD38 quantitative flow cytometry data to percentage CD38+ cells
Figure 3 displays the relationship between the percentage of CD38+ cells and CD38 ABC. As 30% is a commonly used cut-off for determining CD38-positive versus CD38-negative CLL patients, we compared this level to our data using 100 ABC as the cut-off. The overall concordance was 81%: 39 patients were classified as CD38 negative by both criteria and 67 were classified as CD38 positive by both criteria (Table V). Twenty-five (19%) patients were discrepant, all with < 30% CD38+ cells but ≥ 100 CD38 ABC. Similar results were obtained using 20% CD38+ cells as criterion of CD38 positivity, as was done by some investigators (results not shown) (Ibrahim et al, 2001; Durig et al, 2002).
Table V. Comparison of percentage CD38+ and CD38 ABC data (30% cut-off).
CD38+ cells (%)
CD38 antibodies bound per cell
Patients with ≥ 30% CD38+ cells had somewhat poorer OS than those with < 30%: the 5-year OS was 32% (95% CI 45–71%) compared with 58% (95% CI 17–46%). The estimated hazard ratio for this difference was 1·57 (95% CI 0·96–2·56, P = 0·072). Further analyses were conducted to investigate whether percentage CD38+ cells and CD38 ABC conveyed different prognostic information for OS. Figure 4 displays the estimated distributions of OS for the three groups defined by the two criteria (see also Table VI). The heterogeneity of the OS among the three groups was marginally significant (P = 0·027). Of interest, among the 64 patients with < 30% CD38+ cells, the 25 with ≥ 100 CD38 ABC had much worse OS than the 39 patients with < 100 CD38 ABC (two-sided P = 0·018). Indeed the results shown in Fig 4 suggest that the OS of the 25 discrepant patients was similar to that of the patients with ≥ 30% CD38+ cells. This suggests that among the patients who are considered CD38 negative based on percentage positive cells there is still a subgroup that can be identified by CD38 ABC with poor prognosis for OS. Similar results were obtained using 20% rather than 30% as the criterion for CD38 positivity (results not shown).
Table VI. Survival comparison of concordant and discordant patient groups as determined by percentage CD38+ and CD38 ABC.
Number of patients
Probability of surviving 5 years
Based on Cox's proportional hazards regression model, P = 0·027 for heterogeneity of survival among the three groups.
< 30%, ABC < 100
< 30%, ABC ≥ 100
≥ 30%, ABC ≥ 100
B-cell CLL is an indolent leukaemia with a variable clinical course and survival ranging from months to well over 15 years (Rozman & Montserrat, 1995). Clinical staging systems have been helpful in predicting survival (Rai et al, 1975; Binet et al, 1981), but other useful prognostic indicators, particularly in low-to-intermediate stage patients, are needed (Zwiebel & Cheson, 1998). CD38 expression on CLL cells is of particular interest as it is much more easily evaluated in clinical laboratories than other molecular genetic parameters, such as IgH mutational status. Flow cytometry is currently widely available in most hospitals of at least moderate size, whereas the availability of routine fluorescence in situ hybridization and DNA sequencing is much more limited.
There may be some disagreement as to whether CD38 expression correlates with IgH mutational status (Damle et al, 1999; Hamblin et al, 2000) and a recent study suggests they are independent factors (Hamblin et al, 2002). However, the prognostic utility of CD38 expression has been confirmed in other recent studies (D'Arena et al, 2001; Ibrahim et al, 2001). Interestingly, these studies used varying methods for defining CD38-positive and CD38-negative cells (Table VII). For example, Damle et al (1999), D'Arena et al (2001) and Del Poeta et al (2001) used the FACScaliburTM flow cytometer and a Becton-Dickinson PE-conjugated antibody with a 30% cut-off relative to an isotype control. Ibrahim et al (2001) used an APC-conjugated antibody and a 20% cut-off for defining positive cases. Durig et al (2002) also used a 20% cut-off. Furthermore, a mix of fresh and cryopreserved cells was used in these studies. If treatment protocols are to be based on CD38 expression, a more uniform testing procedure is desirable.
Table VII. Comparison of previous studies of CD38 expression in CLL.
The QuantiBRITETM system may provide a solution to this problem (Davis et al, 1998; Iyer et al, 1998). This system can accurately quantify an absolute antigen expression level, taking into account variations in reagent purity and instrument performance. Using the software provided by the manufacturer and calibrator beads that only need to be run once per day, this system represents a rapid and practical way to standardize expression analysis, and its utility in CD38 expression quantification in HIV patients has been shown (Iyer et al, 1998).
We retrospectively analysed a series of cryopreserved pretreatment CLL samples for CD38 and CD20 expression using this quantification system. Unlike other series (Damle et al, 1999; D'Arena et al, 2001; Del Poeta et al, 2001; Ibrahim et al, 2001), our patient population consisted of patients requiring therapy for their disease. Patients with CD38 expression of ≥ 100 CD38 ABC had worse OS compared with those with < 100 CD38 ABC. This effect could not be explained by confounding as CD38 expression was not correlated with Rai stage or other factors such as age, Hb, PLT, WBC, or absolute lymphocyte count. Thus, CD38 expression, as determined by the QuantiBRITETM system, can be successfully used in CLL patients. Furthermore, CD38 expression appears to remain a significant prognostic indicator in the setting of a multi-institutional trial. This could be further refined with better characterization of the binding characteristic properties of this particular CD38 clone (binding stoichiometry), to determine the actual number of CD38 molecules per cell that confers a worse prognosis (Davis et al, 1998). Such information could be useful in further biological studies of the functional significance of this level of CD38 expression. The selection of 100 ABC to define CD38 positivity was based on examination of the results in Table III. Therefore, it must be viewed with caution and the prognostic significance of this categorization should be validated in a separate data set.
No clear correlation was observed between OS and CD20, another antigen known to be variably expressed in CLL (D'Arena et al, 2000; McCarron et al, 2000). This lack of significance in CD20 expression should be considered in the context of the treatment used in this series. Specifically, it is uncertain whether CD20 expression levels are of importance in predicting response to monoclonal antibody treatments directed at CD20 itself. Heterogeneity of CD20 expression exists not only among different CLL patients but also between anatomic sites (Huh et al, 2001). CD20 expression appears to predict in vitro sensitivity to complement-mediated lysis, one potential mechanism of action for anti-CD20 monoclonal antibody therapy (Golay et al, 2001; Treon et al, 2001). Further studies of the clinical significance of the expression levels of CD20 and other complement pathway cell surface molecules such as CD55 and CD59 are needed in CLL patients treated with anti-CD20 antibody.
We recognize that this and other studies (Damle et al, 1999; Ibrahim et al, 2001) are limited by the use of cryopreserved cells. In our hands, cryopreservation and subsequent thawing appears to decrease antigen expression, the extent of which does not appear to be predictable. This is contrary to the study of Ibrahim et al (2001), whose correlation study was performed on the basis of the percentage of positive cells and may be one reason that these investigators found no relative decrease in expression. Given the variability of decreased expression in cryopreserved cells, these results should be viewed with caution when attempting to extrapolate to fresh cells. Prospective studies are needed to define the CD38 ABC value in freshly isolated cells that is associated with a poor outcome.
When comparing quantitative methods to the standard percentage CD38+ measurement, we found a substantial proportion of patients that were discrepant, according to whether they would be classified as ‘CD38 positive’. Using the most common cut-off of 30% as the threshold for stratifying patients, three groups could be discerned: two concordant (≥ 100 CD38 ABC with ≥ 30% CD38+ cells and < 100 CD38 ABC with < 30% CD38+ cells) and one discordant (≥ 100 CD38 ABC but < 30% CD38+ cells). In this sample of 131 patients, none had ≥ 30% CD38+ cells with < 100 CD38 ABC. The survival of the discrepant group appeared to be similar to that of patients ≥ 30% CD38+ cells. This suggests that there is a subgroup group of CLL patients with < 30% CD38+ cells who have poor OS that would be detected by CD38 ABC assessment. Again, these findings should be interpreted with caution and prospective studies would be required to validate this finding. Our data is supported by a recent report demonstrating a better discrimination between CLL patients with good and poor OS using CD38 antigen quantification compared with a 30% cut-off (Mainou-Fowler et al, 2001).
Finally, we note that the proportion of patients showing increased CD38 expression as defined by ABC, 70% with ≥ 100 CD38 ABC, is somewhat higher in our study than the 30–50% in previous studies (Damle et al, 1999; D'Arena et al, 2001; Del Poeta et al, 2001; Ibrahim et al, 2001). Considering the criterion of ≥ 30% CD38+ cells, our data is closer to the literature, with 51% of patients classified as CD38 positive. This relatively high proportion of CD38-positive patients may be due to technical factors. Such factors include variation in antibody fluorochrome to protein ratio, fluorochrome used and whether cryopreserved cells were used. Also, decreased antigen density in cryopreserved cells may have compressed the range of expression and decreased the discrimination between positive and negative patients. Another possible explanation is that our study population consisted of high-risk and intermediate-risk CLL patients requiring treatment. This may have biased us toward more patients with CD38 expression. This would suggest that CD38 is indeed a powerful predictor of prognosis, even in poorer risk CLL. The fact that we are on the upper end of CD38-positive CLL patients by the percentage criterion may also lend some additional support to our finding that CD38 ABC may identify a subgroup of patients with < 30% CD38+ cells with poor OS, as some of our ‘discrepant’ patients (≥ 100 CD38 ABC but < 30% CD38+ cells) might have been classified as CD38 positive by other groups.
In summary, our results suggest that absolute CD38 quantification on CLL cells is useful in the prognostication of CLL patients and it appears to be predictive of OS in CLL patients requiring treatment. In our data set, it appeared to have an advantage compared with the assessment of percentage CD38+ cells. In order for CD38 expression to be most reliably used as a stratification tool, an objective reproducible method should be agreed upon that provides meaningful information. We suggest this method as a practical way to accomplish this. The standardized calibrated beads with highly purified antibody offered by this system should allow more meaningful interlaboratory results, and interlaboratory reproducibility studies are underway. Using this method, CD20 expression levels do not appear to predict outcome. However, its value in predicting response to anti-CD20 therapy remains to be determined. Prospective studies are recommended in order to define the absolute CD38 expression level that is associated with poor outcome using freshly isolated CLL cells not only in patients requiring treatment but in low-risk patients as well.
The authors acknowledge the assistance of Dr Kanti R. Rai and the Cancer and Leukaemia Group B for providing the clinical data for the South-west Oncology Group patients. This investigation was supported in part by the following PHS Co-operative Agreement grant numbers awarded by the National Cancer Institute, DHHS: CA38926, CA32102, CA04919. Partial support was also provided by Becton Dickinson Biosciences. T.F. is an employee of Becton Dickinson Biosciences whose product was studied in this work.