Functional pathway analysis in acute myeloid leukemia using single cell network profiling assay: Effect of specimen source (bone marrow or peripheral blood) on assay readouts


  • How to cite this article: Cesano A, Rosen DB, O'Meara P, Putta S, Gayko U, Spellmeyer DC, Cripe LD, Sun Z, Uno H, Litzow MR, Tallman MS, Paietta E. Functional pathway analysis in acute myeloid leukemia using single cell network profiling assay: effect of specimen source (bone marrow or peripheral blood) on assay readouts. Cytometry Part B 2012; 82B: 158–172.

  • Conflict of Interest: A. C., D.B.R., P.O., S.P., U.G., and D.C.S. are employees of and holders of equity in Nodality Inc.



Single cell network profiling (SCNP) is used to simultaneously measure the effects of modulators on signaling networks at the single cell level. SCNP-based biomarker assays predictive of response to induction therapy and relapse risk in acute myeloid leukemia (AML) patients are being developed. Such assays have typically used bone marrow (BM) as the sample source of blasts. Because circulating peripheral blasts are detectable in ∼65% of AML patients and peripheral blood (PB) sampling is less invasive than BM sampling, this study was performed to assess the effect of sample source on AML blasts signaling as measured in SCNP assay.


SCNP using multiparametric flow cytometry was used to evaluate the activation state of intracellular signaling molecules in leukemic blasts under basal conditions and after treatment with modulators in 46 pairs of BM mononuclear cells/PB mononuclear cells. The relationship between readouts of modulated intracellular proteins (“nodes”) was measured using linear regression, Bland-Altman method, and Lin's concordance correlation coefficient.


The majority (156/161) of signaling nodes show strong correlations between paired PB and BM samples independently from the statistical method used. Notable exceptions were two PB samples with almost undetectable levels of circulating blasts compared with paired BM samples.


Our results demonstrate that specimen source (BM or PB) does not significantly affect proteomic signaling in patients with AML and circulating blasts. The ability to use PB as a sample source will facilitate the monitoring of cellular signaling effects following administration of targeted therapies and at time points when BM aspirates are not clinically justifiable. © 2012 International Clinical Cytometry Society

Acute myeloid leukemia (AML) traditionally has been classified based on cellular morphology after examination of bone marrow (BM) aspirate and biopsy specimens, immunophenotypic characteristics, cytochemistry staining patterns, and cytogenetics (1–4). More recently, molecular markers, such as the presence of either isolated nucleophosmin-1 (NPM-1) gene mutations or internal duplications of the FLT3 gene (FLT3ITD) in cytogenetically normal AML, or the presence of c-kit mutation in the core binding factor AML subgroup, have been used to further classify AML and inform on the intensity of consolidation therapy postremission (5–15). Although these markers offer directionally predictive information at a population level, their degree of accuracy at the individual patient level is still suboptimal. Because a wide range of chromosomal, genetic, epigenetic, and other molecular alterations ultimately converge at the level of protein function and cell signaling pathways, a functional biological characterization of AML based on the analysis of single cell signaling networks could ultimately provide increased accuracy for identifying disease heterogeneity and prognosis and for treatment selection.

Single cell network profiling (SCNP) using multiparametric flow cytometry measures changes in intracellular cell signaling upon exposure of live cells to extracellular modulators revealing network properties that would not be seen in resting cells (16–19) or in assays performed on fixed tissues. The potential usefulness of this technology to generate novel and clinically relevant information has been previously demonstrated (17, 20, 21). More recently, SCNP assay was used to predict the likelihood of response to standard induction chemotherapy in acute myeloid leukemia (AML) patients (22–24). All together, these studies demonstrated that functional pathway analysis of leukemic blasts can provide additional information distinct from other known AML prognostic factors, such as age, secondary AML, cytogenetics, and molecular alterations, and is potentially combinable with the latter to improve clinical decision making. Therefore, SCNP using multiparametric flow cytometry is emerging as a valuable tool to inform on therapy outcomes or relapse risks in both pediatric and adult AML.

The application of SCNP to clinical decision making requires the generation of high-content SCNP assays with robust, accurate, and reproducible results across operators and instruments. In addition, the assays must be characterized by longitudinal stability, as data will be compared across sequential pilot studies, training studies, validation studies, and in clinical patient care as results are delivered to physicians. The preanalytic, analytic, and postanalytic technical variables likely to play a significant role in SCNP assay performance and reproducibility have been previously reviewed (25) and include patient specimen source such as bone marrow mononuclear cells (BMMC) or peripheral blood mononuclear cells (PBMC).

In approximately two-thirds of AML patients, circulating blasts can be detected in PB making this sample source an attractive option due to its less invasive acquisition process compared with BM sampling. However, it is unknown whether or not certain biologic characteristics of AML blasts (such as cell signaling pathways and sensitivity to cytokine, DNA damage inducing or apoptosis causing modulators) are different in blasts collected from the BM samples versus those from the matching PB samples.

Here, we discuss the specific methodology and processes that were implemented to control for technical assay variability in SCNP with the main objective of the study to examine the effect of specimen source on proteomic signaling by comparing signaling readouts for viable paired BMMC and PBMC in a relatively large set of non-M3 AML samples.


Sample Disposition

This study used cryopreserved samples collected at the time of original AML diagnosis from subjects enrolled in the Phase 3 trial E3999 conducted by the Eastern Cooperative Oncology Group (ECOG) between 2003 and 2005. The sample set used for this study represents a subset of the subjects enrolled in E3999, which was selected based on the availability of stored aliquots of PB and BM samples.

In order to be included in the experimental analysis, each of the matched BMMC and PBMC specimens had to contain a minimum of two million viable cells post-thaw. Specimen quality could only be assessed upon thawing of the sample at assay initiation. Therefore, inclusion (or exclusion) of samples, in the study, was determined post-thawing and processing.

Supporting Information Figure S1 illustrates patient enrollment and sample disposition. The original protocol stated 49 donors were to be included in the study. Paired BM and PB samples, with associated clinical data, were available for 46 of the donors. Table 1 summarizes the baseline characteristics of all 46 donors included in the analysis. After thawing, each sample was divided into two aliquots, which were separately processed to assess assay reproducibility. Samples from all 46 donors were thawed and processed. To be included in this analysis, a donor was required to have at least one SCNP assay readout in paired BM and PB samples. All 46 donors met this criteria and were included in this analysis.

Table 1. Baseline Characteristics for Patients
Characteristic All
Patients (N = 46)
Number (percent) of donors 46 (100.0)
ECOG: Best confirmed induction responseCR14 (30.4)
mCR4 (8.7)
PD20 (43.5)
Unevaluable8 (17.4)
Last known survival statusDead46 (100.0)
ECOG performance status0 (Fully active)6 (13.0)
1 (Ambulatory and able to do light work)24 (52.2)
2 (Ambulatory, 100% self-care but cannot work)12 (26.1)
3 (Limited self-care, confined to bed or chair >50% of awake time)4 (8.7)
Disease typeDe novo AML30 (65.2)
RAEB-t2 (4.3)
Secondary AML14 (30.4)
CytogenicsMissing/indeterminate13 (28.3)
Intermediate22 (47.8)
Unfavorable11 (23.9)
Age at diagnosis (years)Mean (SD)69.5 (5.67)
SexMale28 (60.9)
Female18 (39.1)
RaceWhite44 (95.7)
Black2 (4.3)
Treatment receivedAC+ Zosuquidar26 (56.5)
AC+ Placebo20 (43.5)


In accordance with the Declaration of Helsinki, all patients provided written informed consent for the collection and use of their samples for research purposes. Institutional Review Board approval was obtained from Independent Review Consulting, Inc. (Approval No. 09068-01) on August 31, 2009. Clinical data were deidentified in compliance with Health Insurance Portability and Accountability Act regulations.

Study Design

This study investigated paired cryopreserved BMMC and PBMC samples from patients at least 60 years of age who had received cytarabine-based induction chemotherapy for newly diagnosed non-M3 AML and was performed in a blinded fashion to clinical outcomes. A paired sample was defined as BMMC and PBMC specimens collected from the same patient on the same day and prior to the initiation of induction therapy.

The assay measured cell signaling and apoptosis in separate 96-well plates. Figures 1A and 1B illustrate the signaling and apoptosis plate layouts, respectively. Paired BM and PB samples were tested on the same plate, along with instrument and laboratory work flow controls (see “Experimental Controls” section). Plates were processed in duplicate and repeatability data was analyzed (Fig. 1C). Experiments were performed on two instruments, BDTM LSR II (signaling plates) and BD FACSCantoTM II (apoptosis plates). Nine donors were processed per batch, with two samples (BM and PB) per donor. In total, 140 96-well plates were processed during the 3-week experiment, and data was collected from over 8,024 wells. The signaling and apoptosis plates that were run in duplicate are the focus of this analysis and correspond to 161 node-metric calculations (Table 2). The high-throughput nature of this experiment is illustrated schematically in Figure 1C.

Figure 1.

Experimental setup, including signaling and apoptosis plate layouts. A: Signaling plates layout. Each plate contains PB and BM samples from three AML patients (rows A–F). Experiment conditions are shown across the top, with an autofluorescence well (AF), followed by an unmodulated reference well (UM) and the associated modulated wells, shown color coded. A total of six modulated conditions and four associated unmodulated conditions are tested in this plate format. Column 12 contains rainbow control particles (RCPs) used to calibrate the fluorescence readouts on a plate-by-plate basis. Rows G and H contain cell line controls (GDM-1/U937 mix and RS4;11) used to monitor the modulation, fixation, permeabilization, and staining processes. Each plate is setup using a laboratory management system which populates sample, reagent, antibody cocktails, modulators, cytometer and other settings used to track information during the conduct of the experiment. B: Apoptosis plates layout. As in the signaling plates, each apoptosis plates contain a total of three paired PB and BM samples, cell line controls, and rainbow control particles (RCPs) in Column 12. One autofluorescence column is included as a reference. Unmodulated (UM) and modulated columns are used to assess response to Ara-C/Daunorubicin (Ara-C Dauno) and Etoposide. C: Each signaling and each apoptosis plate was run in duplicate, setup with the laboratory management system and data tracking was managed throughout the experiment.

Table 2. Comparison of BMMC and PBMC Measurements
MetricaMeasured populationReference stateBiology investigatedTotal nodesNodes where measurements of BMMC and PBMC are
EquivalentbLin equivalentcOtherd
  • a

    The subscripts represent the following: a = autofluorescence, m = modulated, u = unmodulated.

  • b

    The variance does not depend on the size of the observations; either the difference between BMMC and PBMC was not statistically significant at the 5% level or BMMC is a simple linear transformation of PBMC with R2 ≥ 80%.

  • c

    For nodes that did not satisfy the requirements of the Bland-Altman method, Lin's concordance correlation coefficient was applied to partition variation into precision (Pearson correlation) and accuracy (Cb). BM and PB were judged “Lin equivalent” if Pearson's correlation ≥ 0.6 and Lin's accuracy measure was ≥ 0.80 and the concordance correlation was at least 0.60.

  • d

    Test for independence indicates standard deviation depends on the size of the observation and Lin's concordance correlation coefficient was less than 0.60. Additional analysis is needed to verify the equivalence of BM and PB.

  • e

    Equivalent number of reference fluorophores.

ERFeCalibrated instrument fluorescence intensitiesNone 4115 (37%)23 (56%)3 (7%)
Basal = log2(ERFu/ERFa)UnmodulatedAutofluorescenceBasal pathway activity1615 (94%)1 (6%)0
log2Fold = log2(ERFm/ERFu)ModulatedUnmodulatedInduced pathway activity2218 (82%)3 (14%)1 (5%)
Total Phospho = log2(ERFm/ERFa)ModulatedAutofluorescenceTotal pathway activity2221 (95%)1 (5%)0
UaModulatedAutofluorescenceBasal pathway activity3822 (58%)15 (39%)1 (3%)
UuModulatedUnmodulatedInduced pathway activity2219 (86%)3 (14%)0
Total node-metrics   161110 (68%)46 (29%)5 (3%)

Sample Processing

SCNP assay

Three groups of measurements were performed for each sample: cell surface marker expression, intracellular signaling, and induced apoptosis (Table 3). Modulators, antibodies, and reagents were selected based on biological relevance and performance in previously conducted training studies [data not shown (22)].

Table 3. List of Nodes Tested
CategoryNode: modulatorNode: readout (antibody specificity)
ApoptosisDauno + Ara-CCleaved PARP
ApoptosisEtoposideCleaved PARP
ApoptosisNo modulatorCleaved PARP
ExpressionNo modulatorCD117
SignalingFLT-3 ligandp-Akt
SignalingHydrogen peroxidep-Akt
SignalingNo modulatorp-Akt

SCNP assays were performed as described previously (22). Cryopreserved samples were thawed at 37°C, washed in RPMI 60% fetal calf serum (FCS), and purified through ficoll density centrifugation. Mononuclear cells were washed in RPMI 1% fetal bovine serum (FBS) before staining with Aqua Viability Dye to distinguish nonviable cells. Cells were then counted, resuspended in RPMI 10% FBS, aliquoted to 100,000 cells/condition, and rested for 2 h at 37°C. For apoptosis assays, cells were incubated for 24 h with cytotoxic drugs (e.g., etoposide or Ara-C and daunorubicin) and restained with Aqua Viability Dye. For signaling assays, cells were incubated with modulators (Table 4) at 37°C for 15 min. After exposure to modulators, cells were fixed with 1.6% paraformaldehyde (final concentration) for 10 min at 37°C, pelleted and permeabilized with 100% ice-cold methanol, and stored at −80°C overnight. Subsequently, cells were washed with fluorescence-activated cell sorting (FACS) buffer (phosphate-buffered saline (PBS), 0.5% bovine serum albumin (BSA), 0.05% NaN3), pelleted, and stained with cocktails of fluorochrome-conjugated antibodies (Table 5). These cocktails included antibodies against 2–5 phenotypic markers for cell population gating (e.g., CD45, CD34, CD11b, CD15) and up to three antibodies against intracellular signaling molecules or against surface markers for an eight-color flow cytometry assay. Isotype controls or phosphopeptide blocking experiments were performed to characterize each phospho-antibody (data not shown). The laboratory process was carried out in parallel for all BMMC and PBMC paired samples.

Table 4. Modulators and Technical Conditions
ModulatorFinal concentrationModulator treatment durationManufacturer (location)
Ara-C0.5 μg/mL24 hSigma Aldrich (St. Louis, MO)
Daunorubicin100 ng/mL24 hSigma Aldrich (St. Louis, MO)
Etoposide30 μg/mL24 hSigma Aldrich (St. Louis, MO)
FLT3L50 ng/mL15 minR&D (Minneapolis, MN)
G-CSF50 ng/mL15 minR&D (Minneapolis, MN)
Hydrogen peroxide3.0 mM15 minJT Baker (Phillipsburg, NJ)
IL-2750 ng/mL15 minR&D (Minneapolis, MN)
PMA400 nM15 minSigma Aldrich (St. Louis, MO)
SCF20 ng/mL15 minR&D (Minneapolis, MN)
Table 5. Antibodies and Reagents
AntibodySpecies and isotypeManufacturer (location)Clone
CD11bMouse IgG1Beckman (Miami, FL)Bear1
CD15Mouse IgG1Biolegend (San Diego, CA)W6D3
CD34Mouse IgG1BD (San Jose, CA)8G12
CD45Mouse IgG1Beckman (Miami, FL)J33
cKit (CD117)Mouse IgG1DAKO (Carpinteria, CA)104D2
c-PARP(Asp214)Mouse IgG1, kBD (San Jose, CA)F21–852
Control IgRat IgG1MBL (Woburn, MA)1H5
Control IgMouse IgG1, kBD (San Jose, CA)MOPC-21
CXCR4 (CD184)Rat IgG1MBL (Woburn, MA)A145
FLT3 Receptor (CD135)Mouse IgG1BD (San Jose, CA)4G8
p-Akt (S473)Rabbit IgGCST (Danvers, MA)193H12
p-Chk2 (T68)Rabbit IgGCST (Danvers, MA)Polyclonal
p-CREB (pS133)Mouse IgG1, kBD (San Jose, CA)J151–21
p-Erk 1/2 (T202/204)Mouse IgG1BD (San Jose, CA)20A
p-Lck (Y505)Mouse IgG1BD (San Jose, CA)4/Lck-Y505
p-PLCγ2 (Y759)Mouse IgG1, kBD (San Jose, CA)K86–689.37
p-S6 (S235/236)Rabbit IgGCST (Danvers, MA)2F9
p-SLP-76 (pY128)Mouse IgG1, kBD (San Jose, CA)J141–668.36.58
p-STAT1 (pY701)Mouse IgG2aBD (San Jose, CA)4a
p-STAT3 (pY705)Mouse IgG2a, kBD (San Jose, CA)4/P-STAT3
p-STAT5 (pY694)Mouse IgG1BD (San Jose, CA)47
Non-Antibody Stainsn/an/an/a
Amine Aqua Viability Dyen/aInvitrogen (Carlsbad, CA)n/a
Streptavidin-Qdot 605n/aInvitrogen (Carlsbad, CA)n/a

Flow cytometry data acquisition and analysis

Flow cytometry data was acquired on a BD LSR II and/or BD FACSCanto II flow cytometer using the FACS DIVA software (BD Biosciences, San Jose, CA) and standardized using methods described previously (26). All flow cytometry data were gated with WinList (Verity House Software, Topsham, ME). Dead cells and debris were excluded by forward scatter, side scatter, and Amine Aqua Viability Dye measurement. Leukemic cells were identified as cells that fit the CD45 versus right-angle light-scatter characteristics consistent with myeloid leukemia blasts and that lacked the characteristics of mature lymphocytes, as described previously (22, 27).


In SCNP assay terminology, a “signaling node” is used to refer to a proteomic readout in the presence or absence of a specific modulator. For example, the response to FLT3L treatment can be measured using p-Akt as a readout. That signaling node is designated “FLT3L → p-Akt.” The normalized assay readouts for surface and intracellular markers are measured using two broad classes of calculated metrics that are applied to interpret the functionality and biology of each signaling node. They are referenced following the node, e.g., “FLT3L → p-Akt|Fold,” “G-CSF → p-STAT5|Total,” or “p-STAT5| Basal” and defined in detail below.

Supporting Information Figure S2A shows the workflow for calculation of the metrics used to quantify the assay readouts. In all cases, the raw instrument fluorescence intensities are converted to calibrated intensity metrics [equivalent number of reference fluorophores (ERFs)] (26, 28, 29). The calibration is applied on a plate-by-plate basis using the rainbow calibration particles as shown in Figures 1A and 1B. This correction ensures that data across the plate and between plates are calibrated to the same values, regardless of the instrument used for acquisition.

Two broad classes of metrics were developed to measure two distinct functional aspects of signaling proteins. The Fold metric class measures the magnitude of the responsiveness of a cell population to modulation relative to the same cell population in the reference well (i.e., unmodulated) by comparing the median fluorescence values of the responsive cell population to that of the reference population on a log2 scale. A value of zero would indicate overlapping populations and a value different from zero indicates the responsive population has shifted to higher fluorescence (positive values) or to lower fluorescence (negative values). The “U” metric class measures the fraction or proportion of a cell population that is responsive to modulation relative to the same population in the reference well by comparing the overlap of the responsive cellular population relative to the reference population evaluated on a cell-by-cell basis. This class is mathematically equivalent to an Area Under the Curve (AUC) metric (which is a scaled Mann-Whitney U metric) and is scaled to range from zero to one. For overlapping populations, roughly half of the cells in the modulated population have higher intensities than those in the reference population, so the U metric has a value of 0.5. A value different from 0.5 indicates the responsive population has shifted to higher fluorescence (values > 0.5) or to lower fluorescence (values < 0.5). This metric has an upper limit of 1.0, which represents the situation in which there is no overlap between the modulated and reference populations.

Examples of both metric classes are presented in Supporting Information Figure S2B to highlight the different and complementary information that the use of both metrics provides. The annotations in each of the plots show both metric values for different types of responses routinely seen in SCNP assay results. A summary of the various metrics showing the experimental and reference populations used for calculations and the relevant biology captured is shown in Table 2. For the current analysis, a total of 6 metrics were calculated, resulting in a total of 161 node-metrics.

In addition, for each sample, the cell health [percent healthy (PH)] is measured as the fraction of leukemic cells that are live and nonapoptotic (i.e., Aqua and c-PARP negative) as a fraction of the total number of intact cells.

Experimental Controls

Instrument controls

Linearity verification was performed daily for all fluorescence detectors on the cytometer using 8 Peak SPHERO™ Rainbow Calibration Particles (Spherotech, Lake Forest, IL). The slope, intercept, and R2 values obtained from the linear regression were used to standardize, qualify, and monitor the instrument during setup. They were also used to calibrate the raw fluorescence intensity readouts to control for instrument variability. For this experiment, the coefficients of variation (CV) of the rainbow particle bead peaks on both the BD LSR II and the BD FACSCanto II were under 2.1%. The dimmest peaks were not included in this calculation as they were considered to be near the baseline variance in the instrument.

Cell line controls

Three cell lines, GDM-1, U937, and RS4;11, were included as controls on each plate to ensure consistency in assay performance. Data from previous experiments show that the modulators used in this study, other than FLT3 ligand, induce robust signaling in GDM-1 and/or U937 cell lines. FLT3 ligand induces signaling in the RS4;11 cell line. In combination, the three cell lines provided plate-based controls for all modulator/antibody readout combinations used in this study. Therefore, these cell lines were used to identify potential technical variability at the modulation, fixation, and staining steps in the laboratory work flow. The assay performance was measured on the Uu metric (see “Metrics” for definition). In this study, CVs below 10% were observed for 49 of 54 (91%) signaling nodes and 5 of 8 (62%) apoptosis nodes. Higher variance in the remaining nodes was attributable to either technical issues such as cytometer clogs or an inherent lack of cell line responsiveness to specific modulators or poor signaling due to high levels of induced apoptosis and cell death.

Process controls

Several data analysis tools were developed to facilitate experimental setup and data tracking to ensure verification of data integrity at each step in the sample processing workflow (30). These tools were used to manage the sample, reagent, and instrument data from initial study design through processing and to unblinding of clinical data. Additionally, a set of internal software tools were developed to perform high-volume gating, to calculate all metrics employed, and to allow for data analysis, interpretation, and visualization. Together, these represent significant process efficiency improvements in SCNP assays.

Sample Quality Assessment

Sample quality was determined by calculation of the percent of cell recovery and percentage of healthy cells. Samples were thawed and ficolled on the day of the experiment, and total cell number and leukocyte count were determined and compared with the expected number of cells in the vials, as reported in the sample manifest. The fraction of live, nonapoptotic cells (Percent Healthy, see “Metrics”) was calculated from the unmodulated wells at the 6 h time point.

SCNP Assay Repeatability in AML Samples

Assay repeats were run on different plates on the same day. Teams of operators were fixed, but individual operators could vary for each step in the laboratory work flow. Following laboratory processing, linear regression and Bland-Altman analyses (31, 32) were performed to assess assay repeatability.

Determining Equivalence Between PB and BM

Three analysis methods were used to investigate equivalence between paired PB and BM readouts for signaling nodes, apoptosis nodes, and surface markers.

Linear regression

Linear regression with the BM result as the dependent variable and the PB result as the independent variable was performed.


Using the Bland-Altman method (31, 32), the relationship between the readouts difference (BM − PB) and the readouts average (BM + PB)/2 was determined by inspecting two plots of the data and by testing the hypothesis that the correlation of (BM − PB) and (BM + PB)/2 is equal to zero. The difference between BM and PB readouts was plotted, and the mean difference ± 2 standard deviations was calculated. The fitted regression line, the identity line (BM = PB), a 95% prediction ellipse about the point defined by the means of BM and PB sample readouts, and the 95% confidence band about the regression line were also computed.

Lin's concordance correlation coefficient

Lin's concordance correlation coefficient (33), a measure that incorporates accuracy (distance from the identity line) and precision [distance of each pair (PB, BM) readout from the best-fit linear regression line], was computed.

Identification of Outliers

Outliers were identified by review of linear regression plots. Results that were outside the 95% confidence interval for the linear regression fit were identified as outliers and subject to follow-up analysis.

Evaluation of the Impact of Sample Quality on Paired BM and PB Readouts Concordance

To evaluate the impact of sample quality on paired PB and BM readouts concordance, regression analysis was performed to include the PH as a covariate as follows:

equation image
  • • PB = node-metric values for PB samples.

  • • BM = node-metric values for BM samples.

  • • dPH = difference in PH between the two tissue types at 6 h.

This analysis used the 6 h time point because PH data was not collected at 0 h in this experiment and, based on previous studies (data not shown), the 6 h time point is an acceptable measure of sample quality (i.e., shows a good correlation with intrinsic cell capability to respond to modulation). Unfortunately, due to cell number limitation, data for PH at 6 h was not available for all samples, and therefore, this analysis was performed only on the subset of samples with paired 6 h PH data (N = 37).


Sample Quality Assessment

We assessed the sample quality as determined by the percent of cell recovery after thawing (N = 46 paired samples) and PH after 6 h (N = 37 paired samples, see “Metrics” section for definition). A total of 96% of the samples contained sufficient cells to analyze the nodes listed in Table 3. The median percent recovery, based on reported cryopreserved cell numbers from the sample manifest, was 51.3%, with a minimum of 1.56% and maximum of 462%. The median PH was 41.6%, with a minimum of 0% and maximum of 83.2%. Of the 37 samples, six PB and nine BM samples had PH < 25%. The importance of this observation is related to the association observed between the baseline level of apoptosis and cell signaling ability; specifically, samples with low PH (high levels of apoptosis) displayed poor induced signaling potential, whereas samples with high PH were shown to display high signaling potential. Figure 2 shows the relationship between PH and FLT3L-induced signaling of p-S6 using the fold metric. Samples with low percentage health showed little or no signaling. In addition, PH at 6 h is correlated between matched PB and BM samples (Figure 3) suggesting that similar biology and/or preanalytical variables influenced samples of both tissue types for each donor. Overall, these results show the appropriateness of these methods to measure and control sample quality for inclusion in SCNP assay.

Figure 2.

Comparison of the FLT3L-induced signaling changes in p-S6 on the fold metric (y-axis) as a function of the percentage of healthy cells (PH) (x-axis).

Figure 3.

Correlation of PH between BM and PB samples at 6 h time point. [Color figure can be viewed in the online issue, which is available at]

A number of outliers with lower correlation between PB and BM sample quality were observed (Fig. 3). Regression analysis was performed to evaluate the impact of sample quality on concordance between paired PB and BM signaling (see below).

SCNP Assay Repeatability in AML Samples

Linear regression analysis demonstrated excellent correlation between sample repeats. Seventy-eight percent (PB) and 68% (BM) of nodes displayed R2 > 0.80 for the Uu metric (Tables 6 and 7, respectively), and 73% (PB) and 68% (BM) of nodes displayed R2 > 0.80 for the fold metric (data not shown). Equivalent readouts from the two repeats (Slope = 1.0 and Intercept = 0.0) were observed for 38 out of 44 (87%) nodes.

Table 6. Repeatability Study in Peripheral Blood Samples
Uu metric
ModulatorAntibodyColorNaR2InterceptP (Intercept = 0.0)*SlopeP (Slope = 1.0)*
  • a

    The number of samples varies due to insufficient cells to test all the conditions in a prioritized order or due to insufficient cells in the myeloblast gate (e.g., in apoptosis not enough viable cells remain in a well after 24-h drug treatment).

  • *

    The combination of P (Intercept = 0) and P (Slope = 1) indicate if an equivalent relationship between BM and PB can be ruled out. If both of these P-values are above 0.05, an equivalent relationship (slope = 1.0, intercept = 0) is within the 95% confidence limits of the linear fit.

p-Chk2Alexa Fluor 647-A340.890.050.2420.950.251
p-Chk2Alexa Fluor 647-A380.740.060.2910.910.225
FLT-3 Ligandp-AktAlexa Fluor 647-A410.770.060.2180.890.157
p-S6Alexa Fluor 488-A410.970.030.1210.960.118
G-CSFp-STAT1Alexa Fluor 488-A390.860.060.1100.920.148
p-STAT5Alexa Fluor 647-A390.970.000.9331.010.383
Hydrogen Peroxidep-AktAlexa Fluor 488-A380.94−0.010.8671.010.388
p-SLP-76Alexa Fluor 647-A380.970.020.2240.960.180
IL-27p-STAT1Alexa Fluor 488-A410.950.010.6250.990.390
p-STAT5Alexa Fluor 647-A410.820.040.3010.930.224
p-ERKAlexa Fluor 647-A370.980.000.9440.990.387
p-S6Alexa Fluor 488-A370.98−0.020.2701.020.317
SCFp-AktAlexa Fluor 647-A410.960.020.2760.970.242
p-S6Alexa Fluor 488-A410.910.080.0150.890.030
Table 7. Repeatability Study in Bone Marrow Samples
Uu metric
ModulatorAntibodyColorNaR2InterceptP (Intercept = 0.0)**SlopeP (Slope = 1.0)**
  • a

    The number of samples varies due to insufficient cells to test all the conditions in a prioritized order or due to insufficient cells in the myeloblast gate (e.g., in apoptosis not enough viable cells remain in a well after 24-h drug treatment).

  • **

    The combination of P (Intercept = 0) and P (Slope = 1) indicate if an equivalent relationship between BM and PB can be ruled out. If both of these P-values are above 0.05, an equivalent relationship (slope = 1.0, intercept = 0) is within the 95% confidence limits of the linear fit.

p-Chk2Alexa Fluor 647-A290.840.010.8280.980.387
p-Chk2Alexa Fluor 647-A340.780.120.0330.830.041
FLT-3 Ligandp-AktAlexa Fluor 647-A410.710.160.0000.720.001
p-S6Alexa Fluor 488-A410.940.020.5770.970.315
G-CSFp-STAT1Alexa Fluor 488-A410.780.050.2660.930.265
p-STAT5Alexa Fluor 647-A410.910.020.5040.970.308
Hydrogen peroxidep-AktAlexa Fluor 488-A360.930.070.0470.920.062
p-SLP-76Alexa Fluor 647-A360.960.040.0070.930.029
IL-27p-STAT1Alexa Fluor 488-A420.950.050.0580.950.111
p-STAT5Alexa Fluor 647-A420.700.050.3290.890.194
p-ERKAlexa Fluor 647-A350.990.040.0060.940.004
p-S6Alexa Fluor 488-A350.980.040.0660.950.078
SCFp-AktAlexa Fluor 647-A410.920.020.5260.980.362
p-S6Alexa Fluor 488-A410.970.020.4110.980.263

A technical outlier was identified during the repeatability analysis based on low reproducibility metrics. Specifically, low reproducibility for FLT3L (R2 = 0.49, Table 7) was attributed to a cytometer clog. Overall, only two wells out of the 8,024 wells processed had a cytometer error.

Effect of Specimen Source on Proteomic Signaling

Equivalence was shown for the majority of signaling nodes (>90%) between matched PBMC and BMMC samples (Table 2 and Supporting Information Tables S1A–S1C and S2A–S2C). Simple linear regression models and bias estimates from the Bland-Altman analysis between BMMC and PBMC are shown in Tables 8 and 9 (unmodulated and modulated, respectively). The Bland-Altman plots for the Ua and Uu metrics are shown in Supporting Information Figures S3 and S4.

Table 8. Comparison of Bone Marrow and Peripheral Blood with Respect to Each Node for the Uaa Metric Comparing the Modulated and Autofluorescence Wells (No Modulator)
inline image
Table 9. Comparison of Bone Marrow and Peripheral Blood with Respect to Each Node for the Uua Metric Comparing the Modulated and Autofluorescence Wells
inline image

In total, 97% of the 161 node-metrics studied in matched BMMC and PBMC were judged equivalent using either the Bland-Altman method (Table 2, Supporting Information Tables S1B and S2B) or Lin's concordance correlation coefficient (Table 2, Supporting Information Tables S1C and S2C). For the majority of signaling node-metrics, the difference between BMMC and PBMC is either not statistically significant at the 5% level or a simple linear transformation equates the two methods on average.

Bland-Altman analysis of the basal signaling as evaluated using the Ua metric across all nodes and all donors revealed equivalence between PBMC and BMMC samples (Table 8) for all nodes except p-Erk-PE and p-S6-Alexa488 in the dimethyl sulfoxide (DMSO) vehicle. Visual inspection of the Bland-Altman plots (Supporting Information Fig. S3) for these two nodes indicate that most donors have high basal levels of expression of p-Erk and p-S6 relative to autofluorescence. With the exception of one datapoint in the p-Erk-PE, the data for both tissue types are tightly clustering around a Ua value between 0.85 and 1.0 and are consistent with each other.

Analysis of modulated signaling as evaluated using the Uu metric across all nodes and all donors revealed equivalence between PBMC and BMMC samples (R2 = 0.819, Fig. 4).

Figure 4.

Correlation of the SCNP signaling and apoptosis readouts on the Uu metric between BM and PB samples. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary. com.]

Several of the exceptions that displayed poorer linear regression correlations (R2 < 0.64) belonged to nodes with weak response to modulation (e.g., SCF → p-Erk|Uu).The response for those nodes is consistent between the two tissue types, as can be seen from the Bland-Altman analyses.

Identification and Analysis of Biological Outliers

Two biological outliers (ECOG023 and ECOG025) were identified using methods described above (see “Identification of Outliers” section) (Fig. 5A). Further analysis revealed that biological outliers displayed distinct leukemic cell populations between paired BM and PB samples. Specifically, ECOG023 and ECOG025 BM samples demonstrated a higher proportion of CD34+ cells compared with the paired PB sample (Fig. 5B). ECOG023 was collected from a patient with refractory anemia with excess blasts in transformation (RAEB-T). Because CD34+ cells respond to SCF, G-CSF, and IL-27 modulators, the marked difference in leukemic cell populations may explain the lower equivalence between PB and BM samples for these two patients.

Figure 5.

Process for identification of outlier samples. A: Outlier samples were identified by review of linear regression plots of signaling and apoptosis nodes. Results that were outside the 95% confidence interval of the linear regression fit were subject to additional analysis. As an example, the IL-27 → p-STAT1|Fold response is higher in BM compared with PB for ECOG023 and ECOG025. B: The gating plots were then visually inspected on the outliers identified from the linear regression plots. Results for CD11b versus CD34 on both PB and BM for ECOG023 and ECOG025 are shown. In the case of ECOG023 and ECOG025, the BM contains mostly CD34+ immature blasts, whereas the PB contains mostly CD11b++ mature blasts (i.e., “biologic outliers”).

Accounting for Sample Quality Improves the Concordance Between Paired BM and PB Samples

Signaling has been shown to be affected by sample quality (Fig. 2). In this study, differences in PB and BM sample quality, as measured by PH, were observed in a few donors (Fig. 3). Regression analysis, including PH as a covariate, was performed to evaluate the impact of sample quality on concordance between matched PB and BM readouts. The slope for the difference in PH between samples was found to be significant (P < 0.05, slope = 0.0) in 68% of Uu node-metrics (13/19; Table 10). This data shows that sample quality plays a significant role in the concordance between PB and BM sample readouts.

Table 10. Impact of Sample Quality on Concordance Between Matched Peripheral Blood and Bone Marrow Readouts
ModulatorModulator timeStainColorR2R2 when difference in PH is included as a covariateP (slope = 0) for PHaN
  • a

    The slope for the difference in PH between samples was found to be significant (P < 0.05, slope = 0.0).

AraC+Daunorubicin24 hcPARPFITC-A0.6700.7630.00331
AraC+Daunorubicin24 hp-Chk2Alexa Fluor 647-A0.8450.8450.86431
Etoposide24 hcPARPFITC-A0.6690.6830.24135
Etoposide24 hp-Chk2Alexa Fluor 647-A0.7380.7700.04435
FLT-3 Ligand15 minp-AktAlexa Fluor 647-A0.6120.6560.04139
FLT-3 Ligand15 minp-ErkPE-A0.6440.6710.09439
FLT-3 Ligand15 minp-S6Alexa Fluor 488-A0.7250.7850.00339
G-CSF15 minp-STAT1Alexa Fluor 488-A0.5740.6680.00339
G-CSF15 minp-STAT3PE-A0.7770.8480.00039
G-CSF15 minp-STAT5Alexa Fluor 647-A0.7770.8440.00039
Hydrogen peroxide15 minp-AktAlexa Fluor 488-A0.7270.7290.66138
Hydrogen peroxide15 minp-PLCγ2PE-A0.8410.8600.03438
Hydrogen peroxide15 minp-SLP76Alexa Fluor 647-A0.7410.7610.09638
IL-2715 minp-STAT1Alexa Fluor 488-A0.8570.8800.01239
IL-2715 minp-STAT3PE-A0.8080.8260.06639
IL-2715 minp-STAT5Alexa Fluor 647-A0.7340.7440.24839
PMA15 minp-CREBPE-A0.8360.8680.00737
PMA15 minp-ErkAlexa Fluor 647-A0.7680.8370.00137
PMA15 minp-S6Alexa Fluor 488-A0.8060.8420.00937
SCF15 minp-AktAlexa Fluor 647-A0.7810.8660.00039
SCF15 minp-ErkPE-A0.3970.4450.08739
SCF15 minp-S6Alexa Fluor 488-A0.7230.8290.00039


SCNP assay using multiparametric flow cytometry is emerging as a valuable tool to inform on therapy outcomes or relapse risks in AML (22–24). Although SCNP assay has previously been limited to academic settings (16–21), recent studies have shown the potential utility of this assay in clinical settings to inform decision making (22–24). To successfully transition this technology from the academic environment to the clinic, instrument, reagent, and assay procedure standardization were put in place. Specifically, standard instrument controls (rainbow beads) and cell line controls enabled the assessment of technical variability at the modulation, fixation, staining, and acquisition steps in the laboratory work flow thus allowing for the generation of reproducible results across operators, plates and time. These improvements are essential for the development of clinically applicable assays.

In addition, specific metrics were developed to describe and quantify the functional changes observed using the SCNP assay. Classic flow cytometry metrics measure “static” events using surface markers. In contrast, the metrics described in this paper are designed to quantify the biological effect of modulation over time, providing a dynamic and quantitative view of intracellular functional events. These metrics are calculated on a relative basis and compared across patients and tissue types, which underscores the importance of equipment calibration and standardization. Data analysis using a standard set of metrics is a critical component for assay reproducibility and a required step for moving the phosphoflow assay from the laboratory into clinical practice.

Moreover, preanalytical sample characteristics pose potential barriers to standardization in flow cytometry. In particular, functional assays require that the cells are not only viable but sufficiently “healthy” to signal. Standardization methods based on sample quality could allow for improved control of preanalytical variables and comparison of samples that were subject to different handling procedures (i.e., cryopreserved vs. fresh).

After the development and implementation of processes to control for technical variability within the assay, the effect of specimen source on proteomic signaling was assessed by comparing signaling readouts for matched viable BMMC and viable PBMC. Because of the way the experiments were designed, the differentiation between technical assay variability and biological variability in leukemic cells from paired BMMC and PBMC specimens was possible.

Overall, strong correlations between the majority of nodes in matched PB and BM samples were observed; correlations were particularly high in signaling nodes which had higher levels of changes under modulation. The few exceptions that were noted could be classified as either technical or biological “outliers.” Specifically, repeatability analyses identified a low frequency of technical outliers, which were attributed to a cytometer clog. The extremely low frequency of technical outliers (2 out of more than 8,000 assays) demonstrated that the instruments performed within expected levels of variance and that the modulation and staining processes were consistent among plates and across days.

In addition, two biological outliers were also identified in which the primary leukemic cell population identified in the BM was almost undetectable in the paired PB samples. This is in alignment with previous observations that a small number of AML patients will not have circulating myeloblasts, and therefore, these samples will not be usable for the SCNP assay. It is also of note that overall correlations may be further improved by adjustments to the gating strategy using two methods. First, one could use additional surface markers to better identify the leukemic myeloblasts only. Second, one could use subpopulations (i.e., immature and mature myeloblasts) for SCNP signaling readouts within those cell populations only. Despite these future efforts, we have demonstrated that a classifier based on the leukemic blasts as defined in this analysis has been validated and shown repeated accuracy in independent patient sample sets (22, 24).

An important observation in this study was the effect of sample quality (preanalytic variable) on intrinsic cell signaling capability; specifically, controlling for cell health improved correlation between matched PB and BM sample readouts. These results suggest that analysis of sample quality and cell health must be integrated into study design and qualification of samples and used to control/adjust the level of signaling. Currently, two methods have been used to account for cell health—gating on live, healthy cells (defined using both Live/Dead Aqua Stain and cleaved poly ADP ribose polymerase (PARP)) and normalization using a prespecified adjustment factor for cell health. We have previously reported on the successful application of a normalization adjustment during classifier development (24), which underscores the importance of standardized, prespecified processes for addressing cell health in signaling assays.

Although not the focus of this study, demonstrating a correlation of SCNP readouts between fresh and cryopreserved preparations is essential to applying these proteomic signatures to the clinical setting. Bridging studies, which compare the results of SCNP assays between paired fresh and cryopreserved samples, have been published (34), and results suggest that SCNP assays developed and validated using cryopreserved samples can be applied to fresh samples and integrated prospectively into frontline clinical trials and clinical practice. Further studies are ongoing.

In summary, our results demonstrate that specimen source (BM or PB) does not significantly affect proteomic signaling in patients with AML and circulating blasts. Therefore, PB myeloblasts can be used as a sample source for SCNP assays to identify functionally distinct leukemic blast cell populations. The ability to use PB as a sample source will greatly improve the utility of these assays, e.g., it can facilitate the analysis of target coverage and off-target activities during preclinical selection of lead molecules and early phase clinical trials and also guide dosing schedules after a drug candidate moves to clinical testing. Moreover, PB can be used as a sample source at time points when BM aspirates are not clinically justifiable. The development of instrument standards and process controls described here will accelerate the transition of SCNP proteomic assays from the academic laboratory to the clinic.


 We would like to thank all the participants in this study and Norman Purvis for his exceptional manufacturing work.