- Top of page
- Materials and Methods
- Statistical Analysis
Detection of C4d is crucial for diagnosing antibody-mediated-rejection. We conducted a multicenter trial to assess the reproducibility for C4d immunohistochemistry on paraffin tissue. Unstained slides from a tissue microarray (TMA) comprising 44 kidney allograft specimens representing a full analytical spectrum for C4d were distributed to 73 institutions. Participants stained TMA slides using local protocols and evaluated their slides following the Banff C4d schema. Local staining details and evaluation scores were collected online. Stained slides were returned for centralized panel re-evaluation. Kappa statistics were used to determine reproducibility. Poor interinstitutional reproducibility was observed (kappa 0.17), which was equally due to limitations in interobserver (kappa 0.44) and interlaboratory reproducibility (kappa 0.46). Depending on the cut-off, reproducibility could be improved by omitting C4d grading and only considering ± calls. Heat-induced epitope recovery (pH 6–7, 20–30 min, citrate buffer) with polyclonal antibody incubation (<1:80, >40 min) appeared as best practice. The BIFQUIT trial results indicated that C4d staining on paraffin sections varies considerably between laboratories. Refinement of the current Banff C4d scoring schema and harmonization of tissue processing and staining protocols is necessary to achieve acceptable reproducibility.
- Top of page
- Materials and Methods
- Statistical Analysis
Detection of C4d in the microcirculation of allografts is crucial for diagnosing antibody-mediated rejection (AMR; Refs. [[1-3]]). Detailed criteria for the evaluation of C4d staining in kidney transplants are described in the 2007 Banff update . At the 2009 and 2011 Banff consensus meetings, refinement of criteria for evaluating C4d staining in cardiac and pancreatic allografts was accomplished [2, 5, 6]. In routine practice, C4d is evaluated by either immunohistochemistry (IHC) on formalin-fixed, paraffin-embedded sections or by immunofluorescence on frozen sections. Despite its central importance in clinical decision making, little has been done to evaluate its reproducibility across laboratories [7-10].
Tissue microarrays (TMAs) have become a valuable tool in conducting external quality assessment trials in the area of IHC . TMAs allow multiple cores of paraffin-embedded tissue to be placed on a single glass slide. This slide can be tested in laboratories under identical staining conditions, thus eliminating confounders of interlaboratory variability assessments, while enabling a comprehensive assessment of numerous cases in a highly consistent, cost and time efficient manner .
In 2009, the Banff initiative for quality assurance in transplantation (BIFQUIT) was launched  and organized a multicenter BIFQUIT trial with the aim to assess and improve the standards for a reproducible C4d IHC assessment in human kidney transplant tissue. Here we present the results from the first such BIFQUIT trial.
- Top of page
- Materials and Methods
- Statistical Analysis
All statistical calculations were done in R as the analysis platform. For an explorative data analysis we used Bland–Altman plots which show on the x-axis the mean of the two scores (i.e. panel + participants C4d score/2), which essentially represents the best guess as to the “correct” result and on the y-axis the difference between the two scores (panel C4d scores – participants C4d scores; Ref. []).
Weighted kappa statistics were calculated for the interinstitutional, interobserver and interlaboratory agreement, as defined below, measuring the degree of agreement between reads and taking into account the degree of agreement that is expected due to chance alone . The general guidelines for interpreting the significance of kappa values are: <0 as indicating no agreement (or less agreement than expected by chance), 0–0.20 as slight, 0.21–0.40 as fair, 0.41–0.60 as moderate, 0.61–0.80 as substantial and 0.81–1 as near perfect agreement. Weighted kappa statistics, in which disagreements between observers are rated differently (bigger discrepancies carry greater weights), were calculated unless the variance in one respective reading was below 0.25 (i.e. 25%). For these, a prevalence-adjusted kappa was calculated, which adjusts the kappa statistic to take into account the higher expected agreement between observers associated with low variance.
Assessing the reproducibility of C4d IHC on paraffin sections
The overall reproducibility of C4d staining between institutions results from two major sources: the interlaboratory (influence of technical differences in the staining protocol applied at different laboratories) and interobserver reproducibility (influence of the subjective interpretation of the stained slide). The interlaboratory reproducibility is further influenced by pre-analytical (tissue processing) and analytical (staining protocol) variables.
In the real world diagnostic setting the overall variation for C4d assessment between institutions (i.e. the same patient tested at different transplantation centers) comprises variance originating from using different processing and staining protocols plus that from different observers applying the semi-quantitative Banff C4d scoring system. To this end, we calculated the kappa values by comparing the C4d scores provided by each participant form their locally stained slides to the corresponding C4d scores provided by the local participant from the reference case (i.e. the scores provided by the participant at the center that was identified by the panel as the “best C4d stain”). Mean kappa values were calculated for each tissue core and for all 78 participants.
To assess the influence of the technical component on the C4d results independent from the subjectivity of individual observers, we had the central panel evaluate all locally stained TMAs. Thus, interlaboratory agreement was calculated by comparing the panel consensus read for each tissue core to the panel read of the corresponding tissue core on the reference slide. Considering that the reference stain potentially is extraordinary sensitive, i.e. turns all cases included in TMA into diffusely C4d positive, we also calculated the interlaboratory reproducibility by comparing the panel read for each tissue core to the read of the majority of participants for this particular tissue core. By this approach we aimed to assess the reproducibility across a broad analytical spectrum for the test cases. In addition, We calculated the interlaboratory reproducibility using the full scale of C4d scores (C4d0-3) and using binary C4d positive/negative calls at various cut-offs (C4d0 vs. C4d1, 2, 3; Cd40, 1 vs. 2, 3; C4d0, 1, 2 vs. C4d3 (i.e. a focal vs. diffuse).
This component examined the impact on reproducibility resulting from the subjective application of the Banff C4d scoring system by different observers. The interobserver reproducibility was calculated by comparing the reads of the local participants to the consensus reads of the panel recorded on the same TMA slide. Mean kappa values were calculated from the 1936 tissue cores using the two different approaches as previously stated: using the full scale of C4d scores (C4d 0–3) and using binary C4d positive/negative calls (C4d0 vs. C4d1, 2, 3; Cd40, 1 vs. 2, 3; C4d0, 1, 2 vs. C4d3 at various cut-offs (i.e. focal vs. diffuse).
Assessing the influence of pre-analytical and analytical components on the C4d result
In order to assess the influence of tissue fixation and processing (pre-analytical component) on the C4d staining result, we compared the interlaboratory kappa values for those six tissue cores in the TMA obtained from the nephrectomy specimen with variable processing as described in Table 1. To assess the impact of different staining protocols and reagents (analytical component), the 15 laboratories with the best reproducibility, (i.e. the highest kappa-values for interlaboratory reproducibility) were compared to those fifteen with the worst in terms of the methods used for epitope recovery (enzyme vs. heat, high (EDTA buffer) versus low (citrate buffer) pH), antibody type, antibody dilution, incubation time and type of detection system. This class comparison approach was chosen over an unsupervised exploration of the raw data returned by the participants through the methods questionnaire, based on prior knowledge that IHC results in paraffin sections are primarily influenced by, tissue fixation, epitope recovery and antibody incubation/detection .
- Top of page
- Materials and Methods
- Statistical Analysis
Current classification systems include C4d staining as an essential component for diagnosing AMR [1-3]. In this multicenter, multinational Banff trial, we observed poor interinstitutional reproducibility for the C4d stain using IHC on paraffin section from renal allograft specimens. This was equally attributable to limitations in technically reproducing the stain as well as between observers using the current Banff C4d grading schema. The observations from this first BIFQUIT trial indicate that further refinement of the Banff C4d scoring schema and harmonization of tissue processing and staining protocols are necessary in order to achieve acceptable reproducibility for C4d staining on paraffin sections.
The need for ongoing proficiency testing and thorough standardization in diagnostic IHC is well recognized and part of national and international consensus guidelines [8-10, 15]. Class I IHC tests provide adjunctive diagnostic information to pathologists, while class II tests are so-called “stand alone” tests that are reported independently of other clinical or laboratory information. The results of these tests are frequently used as predictive biomarkers or companion diagnostics and clinicians rely on the results to stratify patients for therapies [7, 8]. In this regard, C4d IHC assumes an intermediate position. It is not a “stand alone” test for AMR, but the diagnosis of AMR cannot be rendered without C4d. It should be noted that alternative criteria for AMR are currently being evaluated in light of the recent recognition of C4d-negative AMR [5, 16]. At this time, the result of the C4d stain carries significant weight in the clinical decision-making and the presence or absence of C4d may serve as a biomarker for monitoring response to treatment and disease intensity/grade [17-19]. Therefore efforts should be made to make the test as reproducible as possible.
We observed limited reproducibility between institutions for C4d results, which was equally due to differences in how the stain was performed and how the stain was interpreted. The review panel consistently assigned higher C4d scores than the local participants, indicating that local pathologists may have adjusted their scoring to their local laboratory. Such local adjustments typically happen over time by correlating C4d results with in-house clinical feedback, morphology and antibody titers, or by taking into account prior knowledge (e.g. paraffin C4d IHC is less sensitive than immunofluorescence on frozen tissue [20-22]. To address this bias introduced by the panel read, which may be due to the fact that the panel comprised individuals potentially more familiar with paraffin stains than the average participant, we also assessed the reproducibility for a majority call of all participants. This only marginally improved the kappa value for interinstitutional reproducibility from 0.17 (slight) to 0.26 (fair) and the interlaboratory reproducibility from 0.46 (moderate) to 0.6 (moderate). In this BIFQUIT trial all laboratories stained the same tissue. There is the possibility that participating laboratories would have generated ‘better’ stains if the tissue was processed at their institution. However, minor changes in tissue processing, as included in the trial TMA (Table 1), did not significantly alter the results. Only severe mishandling of the tissue, i.e. severe underfixation (<1 h) and ethanol fixation had significant negative effects on C4d results. Most laboratories now use standardized formalin fixation and automated tissue processing. Thus, the pre-analytical components likely only have minor impact on the real world variance in C4d IHC.
It seems to be relatively safe to recommend a stringent and most sensitive staining protocol for C4d IHC with the aim of detecting as many AMR cases as possible. Few potentially false positive results were seen in this BIFQUIT trial (7/44 laboratories, mostly as C4d1, only two C4d3 false positives). We used histologically unremarkable tissue from tumour nephrectomies as negative controls, but the possibility of complement activation in the setting of ischemia cannot be entirely ruled out; this may have been detected by “sensitive” laboratories as positive C4d. However, the more likely explanation is that complement rich plasma stuck non-specifically on the endothelium, and was misinterpreted by the observers (including the panel) as specific C4d deposition in capillaries of native kidneys. Previous single center studies from Basel, Switzerland already described that C4d staining using immunofluorescence in frozen tissue is not only more sensitive but also more reproducible between observers . The interobserver kappa values between two observers from the same institution for frozen immunofluorescence stains were perfect (.0) and for paraffin sections only good (0.57–0.63). Better reproducibility on paraffin sections was observed with higher dilutions of the primary antibody at the expense of further decreased sensitivity. These data and our own observations indicate that with C4d IHC on paraffin section at high primary antibody concentrations, nonspecific pseudo-linear C4d stain in peritubular capillaries can occur as background staining and be misinterpreted as C4d positivity.
The motivation for introducing the Banff C4d scoring schema at the 2007 meeting  was to standardize C4d IHC and immunofluorescence reporting, thus making them more amenable for comparisons between studies to evaluate the clinical relevance of focal or negative C4d. A generally accepted notion is that detailed standard consensus criteria for histological assessment improve reproducibility and thus make data comparable . However, it stands to reason and previous studies have already validated that if a grading schema is overly complex, reproducibility is poor thus defeating the purpose of the entire system . As expected and previously shown in a smaller studies  and confirmed here, specimens showing intermediate C4d scores (i.e. C4d1 or C4d2) are less reproducible compared to those with negative or diffusely positive C4d. A simplified grading schema, i.e. a positive versus negative call (any C4d stain vs. no C4d stain) was associated with improved reproducibility. However, interobserver reproducibility remained weak if a positive/negative call was applied with cut-off at the focal/diffuse interface (C4d ≤ 2 vs. C4d3), i.e. the intermediate scores with greatest variation across all participants, potentially explaining the conflicting data in the literature regarding the significance of focal C4d positivity [18, 25-28]. Even using a highly simplified scoring schema (any C4d stain vs. no C4d stain, which essentially is a reduction ad absurdum), only 59% of the participants performed and evaluated the stain with at least a moderate interobserver and interlaboratory reproducibility. Altering the benchmark for C4d positivity, e.g. adjusting it to the majority and not the “best” participating laboratories improved reproducibility. However, this is at the cost of limited sensitivity of detecting C4d, since the “best” laboratories stained considerably more cases as C4d 3, than the majority of participants did. Furthermore, defining C4d cut-offs at which clinical interventions get triggered, represents a completely different task requiring appropriate validation. But before this can be scope of prospective studies, standardization of C4d staining is paramount, otherwise the threshold for therapeutic interventions need to be established and validated at each transplant center independently.
Regular participation in IHC quality assurance trials is known to lead to improved reproducibility following individual participant feedback . We provided each participant with individual performance feedback. This information together with best practice recommendations for staining and tissue processing (see Table 4) can lay the foundation for subsequent BIFQUIT trials with the aim to monitor improvement of the reproducibility of C4d IHC staining. However, a limitation of this first BIFQUIT trial is that the technical protocol details were reported very heterogeneously preventing an unsupervised statistical exploration with the aim to identify the true best practice. But international standardization of C4d IHC seems feasible, since only one polyclonal antibody is currently commercially available and standardized processing of core biopsies is achievable using automated tissue processors and strainers. Combining standardized staining protocols with computer based image analysis algorithms might then be the next step towards acceptable reproducibility of C4d. A recent single center experience in a small series of C4d positive cases observed a superior reproducibility using digital pathology and image analysis for C4d assessment compared to pathologists visually scoring the slides . The Banff working group for digital pathology could take this on as a task to generate a consensus image analysis algorithm for C4d IHC. The ultimate aim of such concerted Banff efforts would be to assure that every patient receives an identical C4d result independent of where his or her biopsy is obtained and evaluated.
- Top of page
- Materials and Methods
- Statistical Analysis
We thank Victoria Sheldon and Akshatha Raghuveer for an outstanding logistical support. This trial was supported by a research grant from Astellas Canada Inc. to M.M for facilitating Banff Working Group activities.
The following institutions contributed tissue for constructing the BIFQUIT C4d TMA:
Volker Nickeleit, Chapel Hill, USA; Verena Bröcker, Hannover, Germany; Parmjeet Randhawa, Pittsburgh, USA; A. Bernard Collins, Boston, USA; Heinz Regele, Vienna, Austria; Michael Mengel, Edmonton, Canada; Cinthia Beskow-Drachenberg, Maryland, USA; Surya Seshan, New York, USA.
The authors would like to thank all centers participating in the BIFQUIT trial for contributing their time and resources, and valuable feedback during and after the trial. We apologize to those participants to whom we were unable to deliver the trials slides (usually due to customs regulations) and those participants from whom we received the slides too late to be reviewed by the panel and included into this analysis.
- Centers who registered for the Banff C4d BIFQUIT trial:
- Department of Pathology, The Methodist Hospital, Houston, TX, USA
- The University of North Carolina, Department of Pathology and Laboratory Medicine, Division of Nephropathology, Chapel Hill, NC, USA
- The Ohio State University, Columbus, OH, USA
- Oregon Health & Science University, Department of Pathology, Portland, OR, USA
- Department of Pathology Dartmouth-Hitchcock Medical Center, Lebanon NH, USA
- University of Illinois Medical Center, Department of Pathology, Chicago IL, USA
- Anatomic Pathology, Laboratory Service, North Florida/South Georgia Veterans Health System, Gainesville, Fl, USA
- Huntsman Cancer Hospital University of Utah Department of Pathology, Salt Lake City, UT, USA
- Department of Pathology, Cedars-Sinai Medical Center, Los Angeles, CA, USA
- University of Virginia Health Sciences Center, Department of Pathology, Charlottesville, VA, USA
- Department of Pathology University of Chicago Medical Center, Chicago, IL, USA
- Pathology Department London Health Sciences centre, London, Ontario Canada
- Department of Laboratory Medicine St. Michael's Hospital, Toronto, Ontario, Canada
- Washington University School of Medicine – Department of Pathology and Immunology, Division of Anatomic Pathology, St Louis, MO, USA
- Department of Pathology University of Maryland Hospital, Baltimore MD, USA
- Dept. of Laboratory Medicine & Pathology Memorial Medical Center, Springfield, IL, USA
- Department of Pathology University of Texas Medical Branch Department of Pathology, Galveston, TX, USA
- HOSPITAL DAS CLÍNICAS- Prédios dos Ambulatórios Divisão de Anatomia Patológica, São Paulo,-Brazil
- Hospital Infantil de Mexico “Federico Gomez” Departamento de Patologia Calle, Mexico City, MEXICO
- Department of Cellular & Anatomical Pathology Derriford Hospital, Plymouth, UK
- Department of Cellular Pathology, John Radcliffe Hospital,, Oxford, UK
- Department of Clinical and Transplant Pathology Institute for Clinical and Experimental Medicine, Prague, Czech Republic
- Department of Pathology Health Sciences Centre, Winnipeg, MB, Canada
- Dept of Pathology, University Health Network, University of Toronto, Toronto, ON, Canada
- Department of Laboratories, Seattle Children's Hospital, Seattle, WA, USA
- ProPath, Dallas, TX, USA
- Dept Pathology, Foothills Medical Centre, Calgary AB, Canada
- Pontificia Universidad Católica de Chile Escuela de Medicina Departamento de Anatomía Patológica, Santiago, Chile
- Department of Pathology Oslo University Hospital, Oslo, Norway
- Transplantation Laboratory- HUSLAB Helsinki University Central Hospital, Helsinki, Finland
- Surgical Pathology Montefiore Medical Center, NY, USA
- Division of Transplant Pathology, University of Pittsburgh, Department of Pathology, UPMC-Montefiore Hospital, Pittsburgh, PA, USA
- Dept of Pathology – Presbyterian Hospital Weill Cornell Medical College, New York, NY, USA
- Institut fuer Pathologie Medizinische Hochschule Hannover, Hannover, Germany
- Klinisches Institut für Pathologie, Wien, Austria
- Department of Pathology Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA
- Department of Pathology Singapore General Hospital, Singapore
- Service de Pathologie CHUQ, Hôtel-Dieu de Québec, Québec City, Québec, Canada
- Department of Pathology, Princess Margaret Hospital, Hong Kong
- RUA JOSÉ MARIA DE OLIVEIRA CASACA, BAIRRO JARDIM MARIA CÂNDIDA SÃO JOSÉ DO RIO PRETO, SP, Brazil
- Dept. of Pathology HADASSAH MEDICAL ORGANIZATION, HADASSAH UNIVERSITY HOSPITAL KIRYAT HADASSAH, Jerusalem, Israel
- Emory University Department of Pathology Emory University Hospital, Atlanta, GA, USA
- Department of Cellular Pathology Barts and the London NHS Trust, London, UK
- Department of Pathology LSU Health Sciences Center, LA, USA
- Nephorology Department Hospital Vall d'Hebron, Barcelona, Spain
- Department of Pathology St. John Hospital & Med. Ctr. Detroit, MI, USA
- University of Arizona Department of Pathology, Tucson, AZ, USA
- 1st Department of Pathology Medical School National and Kapodistrian, Athens, Greece
- Imperial College Healthcare NHS Trust Hammersmith Hospital Dept Histopathology, London, UK
- Leiden University Medical Center Dept. of Pathology, Leiden, The Netherlands
- Department of Pathology Massachusetts General Hospital, Boston MA, USA
- Dept. of Pathology, University of Washington Medical Center, Seattle, WA, USA
- Department of Pathology Albert Einstein Medical Center Philadelphia, PA, USA
- Intermountain Central Laboratory, Salt Lake City, UT, USA
- Dept. of Pathology, UMC Utrecht, Utrecht, The Netherlands
- Pathology & Laboratory Medicine, St. Paul's Hospital, Vancouver, BC, Canada
- Wake Forest Univesrsity School of Medicine Dept. of Pathology, Winston-Salem, NC, USA
- PATHOLOGY Mayo Medical Laboratories, Rochester, MN, USA
- Dept. of Pathology University of Iowa Hospital, Iowa City, IA, USA
- Pathologische ontleedkunde UZ Leuven campus, Leuven, Belgium
- Clinical Pathology and Cytology Gula Stråket, Göteborg, Sweden
- Histopathology Department Mubarak Al Kabeer Hospital City of Jabriyah Governate of Hawally State of Kuwait
- Surgical Pathology QA and Compliance Fletcher Allen Health Care/University of Vermont, Burlington, Vermont, USA
- Department of Anatomical Pathology Austin Hospital, Heidelberg, Australia
- Rhode Island Hospital, Providence, RI, USA
- Institute of Pathology Faculty of Medicine University of Ljubljana, Ljubljana Slovenia
- Cellular Pathology – University Hospitals Birmingham NHS Foundation Trust The Medical School, Birmingham, UK
- Department of Histopathology, Postgraduate Institute of Medical Education and Research, Chandigarh, INDIA
- Consultant Pathologist MBC, Riyadh, Kingdom of Saudi Arabia
- Rua Candido Gaffree, Rio de Janeiro, Brazil
- Section of Pathology/Anatomía Patológica Fundació Puigvert, Barcelona. Spain
- Department of Pathology Baystate Medical Center, Tufts University School of Medicine, Springfield, MA, USA
- Department of Pathology Medical University of South Carolina, Charleston, SC, USA
- Servicio de Anatomia Patologica Hospital Universitario Miguel Servet, Zaragoza, Spain
- Department of Pathology and Laboratory Medicine University of Wisconsin-Madison, Madison, WI, USA
- Institute for Pathology University Clinic, Basel, Switzerland
- Pathology Department, The Johns Hopkins Medical Institutions, Baltimore MD, USA
- Department of Laboratory Medicine and Pathology, University of Alberta, Edmonton, Canada