Human Mutation

Department of Public Health and Primary Care, Biomedical Quality Assurance Research Unit, University of Leuven, Leuven, Belgium Gustave Roussy, Service de Génétique/ Pathologie Moléculaire, Villejuif Cedex, France Department of Pathology and Department of Human Genetics, Radboud University Medical Center, Nijmegen, The Netherlands University Medical Center Groningen, Department of Pathology, University of Groningen, Groningen, The Netherlands

current version, 19.01, publicly available from http://varnomen.hgvs. org (Johan T. Den Dunnen et al., 2016). Examples of 2016 updates include (a) the addition of brackets when amino acid changes are predicted without experimental evidence, (b) the use of the term "variant" instead of "mutation," (c) the use of "Ter" or "*" instead of "X" to indicate a stop codon, and (d) the use of "X" to indicate "any amino acid," as specified in the International Union of Pure and Applied Chemistry recommendations for amino acids nomenclature (International Union of Pure and Applied Chemistry, 1983).
These guidelines have been widely accepted in molecular diagnostics. Thus, they have been adopted by the Consensus recommendations of the College of American Pathologists (Gulley et al., 2007), the American College of Medical Genetics and Genomics, and the Association for Molecular Pathology (Richards et al., 2015). Public databases for germline (e.g., ClinVar) and cancer (e.g., COSMIC) variants have implemented the guidelines as well.
Currently, several tools for variant reporting which comply with HGVS nomenclature are available to guide laboratories. These may include public applications, such as Mutalyzer (Taschner & Den Dunnen, 2011) or VariantValidator (Freeman, Hart, Gretton, Brookes, & Dalgleish, 2018). HGVS software packages that run on Python (Wang et al., 2018) can also be used to manipulate sequence variants according to the nomenclature guidelines. Recently, a significant inconsistency in HGVS nomenclature was observed when comparing variant representations by three tools (SnpEff, Variant Effect Predictor, and Variation Reporter) that generate transcript and protein-based variant nomenclature (Yen et al., 2017). Moreover, discordances were also noted between annotations generated by Snpeff and Variant Effect Predictor, and those in major germline and cancer databases such as ClinVar and COSMIC.
Besides variant descriptions, HGVS advises to include a reference sequence to unambiguously relate the variant to the used coding sequence (Den Dunnen et al., 2016). These sequences are preferably locus reference genome (LRG) sequences. In case an LRG is not available or "pending," a RefSeq sequence is recommended (Macarthur et al., 2014;O'Leary et al., 2016). This is exemplified by a coding DNA transcript reference sequence (NM) frequently applied in routine practice for the detection of somatic variants in EGFR, KRAS, or NRAS. When reporting RefSeq sequences, the inclusion of a version number is important, as variants may refer to different genomic positions between reference sequences and between version numbers within one sequence.
External quality assessment (EQA) aims to monitor laboratory performance, compare results to international peers, and provide individual feedback to laboratories. Initial EQA schemes revealed a need to improve the consistent use of the appropriate nomenclature (Mueller, 2004;Touitou et al., 2009). For instance, assessment of reporting of cystic fibrosis transmembrane conductance regulator (CFTR) variants revealed that only 22% of the reports (n = 631) contained nomenclature conforming to HGVS recommendations, and 5% included nomenclature with the potential to generate interpretation errors in a clinical setting (Berwouts et al., 2011). Moreover, similar findings have also been reported for other indications, such as BRCA1 and BRCA2 analysis in breast cancer (Mueller, 2004), and hereditary recurrent fever (Touitou et al., 2009).
For predictive testing in non-small cell lung cancer (NSCLC), several European providers assess HGVS compliance with regard to variant descriptions and reference sequences in EQA schemes. A first comparative study between providers (Tack, Deans, Wolstenholme, Patton, & Dequeker, 2016) suggested that higher HGVS compliance for epidermal growth factor receptor (EGFR) variants was correlated with the duration of operation of the scheme. It was therefore suggested that additional analyses are necessary to investigate whether a learning effect could be driving this improvement. Another study of EQA data in 2016 revealed highly variable descriptions of EGFR variants, and a lack of using tools to verify the variant descriptions. It was suggested that education is likely to be the way forward to eliminate the observed variability in data reporting (Deans, Fairley, Den Dunnen, & Clark, 2016).
This manuscript evaluates HGVS compliance over time. This is the first study in predictive testing for NSCLC and metastatic colorectal cancer (mCRC) to include this extent of longitudinal scheme data, as well as KRAS proto-oncogene GTPase (KRAS) and NRAS proto-oncogene GTPase (NRAS) variants besides EGFR variants. This approach allows for an evaluation if continued education has indeed exerted a positive effect since 2016 as previously suggested. Additional information was collected in the course of the EQA schemes, which enabled us to investigate the effect of continued EQA participation, as well as of the used test method as contributing factor, and their efficacy in overcoming previously reported residual discrepancies in HGVS compliance.

| MATERIALS AND METHODS
The European Society of Pathology (ESP) and Gen&Tiss consortium organize yearly EQA schemes for variant analysis in EGFR for NSCLC, and KRAS/NRAS analysis for mCRC. Both schemes were organized according to the guideline on the requirements of EQA programs in molecular pathology (van Krieken et al., 2013) and the International Organization for Standardization (ISO) 17043 guideline for conformity assessment of proficiency testing (International Organization for Standardisation, 2010). Detailed organization of the EQA schemes has been previously described Keppens et al., 2018;Tack et al., 2015). Participants were asked to analyze a set of formalinfixed tissue samples by their routine procedures. Then, they completed an electronic datasheet about their test results and the applied methods.
The participants also submitted a written report with the molecular test result and interpretation, mimicking their routine practice.
Compliance with HGVS recommendations was scored from the written reports for the ESP EQA schemes, and from the electronic datasheets for the Gen&Tiss EQA schemes. Data from 351 international participants engaging in EQA schemes between 2012 and 2018 were included. The results and interpretation sections of the report were scored. If neither of these sections were available, nomenclature was scored from the provided list of tested variants.
For Gen&Tiss, data from 64 French laboratories were scored from EQA schemes run between 2013 and 2018, from entries in an open text field in the electronic results sheet.
All variants in the EGFR (NSCLC), KRAS, and NRAS (mCRC) genes were validated before sample distribution by a reference laboratory.
For evaluation of the variant descriptions according to HGVS, a team of international experts in molecular pathology agreed upon a predefined set of scoring criteria, based on HGVS key-points as presented in Table 1. These criteria have been harmonized between schemes and were already described in more detail in the paper by Tack et al. (2016). When necessary, results were rechecked by the Alamut software used by the respective reference laboratory or Mutalyzer. After reaching an expert consensus result, outcomes of the scheme were communicated to the participants in an individual report.
Nomenclature scores obtained during the schemes were then reevaluated, to ensure harmonized scoring between the two providers and all scheme years. HGVS compliance was evaluated only for cases that included a variant, but not for false-positive results in wild-type cases. Cases without nomenclature were also excluded, for example, in case of a technical failure, false-negative result, or descriptive sentences in the form of "we detected a variant." If a case comprised multiple variants, all variants were evaluated separately. Entries were then classified into "correct" and "incorrect" categories. Correct nomenclature according to the EQA scoring criteria, was defined as all variants reported in complete accordance with the HGVS guidelines. For incorrect nomenclature, a further division was made between (a) "type 1" small clerical errors (such as the omission of brackets or stop marks, spaces, etc.), (b) "type 2" errors with potential therapeutic impact (e.g., incorrect amino acid abbreviations, or using traditional "legacy" nomenclature), and (c) "type 3" errors, that include only reporting the variants at protein or nucleotide level.
Traditional nomenclature (e.g., T790M) was considered an error because of the absence of the nucleotide description and "p." prefix, however, a correct description with a one-letter amino acid abbreviation (e.g., c.2369C>T p.(T790M)) was considered correct.
Inclusion of the reference sequences with a version number was evaluated in the diagnostic report for both providers. Reference sequences which were correct according to the EQA scoring criteria, included the recommended LRG, or NM format with inclusion of the current version number at the time of assessment. Laboratories with an outdated version number were not penalized, but received a suggestive comment to raise awareness of the availability of a more recent version number.
Results were analyzed on two levels. First, the percentage of HGVS compliant nomenclature for each of the variants is separately provided. Secondly, HGVS compliance is shown on laboratory level, to evaluate the performance (a) over time, (b) relative to the number of EQA participations, and (c) in relation to the applied test methodology. Statistics were performed by SPSS v25.0 (IBM, Armonk, NY). χ 2 tests for contingency tables were applied, and Fisher's Exact test was used for cell counts below five. All significance levels were set at α = 0.05 with Bonferroni correction for multiple comparisons. The applied reference sequences for the description of the variants in this study were EGFR NM_005228.5, KRAS NM_033360.4, and NRAS NM_002524.5.

| Performance of different variants
In Table 1, we present an overview of the most important recommendations for variant reporting in NSCLC and mCRC, as well as examples of noncompliance with HGVS as observed in this study.
In total, 4,802 variant entries were evaluated during the ESP and Gen&Tiss EQA schemes combined, of which 749 (15.6%) did not include nomenclature (e.g., written in the form of "a variant was detected"). This resulted in 4,053 assessed variants, of which 12.1% was classified as complying with HGVS recommendations. There was no significant difference in the degree of compliance between both EQA providers (15.4%, n = 2,530 for ESP and 9.3%, n = 1,523 for Gen&Tiss; data not shown).
There was a large variability in the percentage of nomenclature reported in complete accordance with HGVS, displaying a wide range between the different variants. However, overall, there were no significant differences in the degree of HGVS compliance for reporting of variants in the EGFR (12.0%, n = 2,377), KRAS (12.4%, n = 1,217), and NRAS genes (11.9%, n = 459; Table S1).
In total, 64.4% (n = 4,053) of errors consisted of small (type 1) clerical errors like inclusion of a space, omission of a stop mark, or the absence or incorrect use of brackets on protein level. In 2.4% of entries, a larger (type 2) error was observed for which the "c." or "p." prefix on the nucleotide or protein level was omitted, traditional "legacy" nomenclature (e.g., T790M) was used, or an incorrect amino acid code was given (e.g., Tyr instead of Thr). For 2.0% of entries, the variant was described only on nucleotide or protein level (type 3). In 19.1% a combination of the abovementioned errors was made.
Errors with a potential therapeutic impact, for example, type 2 and type 3 errors or a combination, were observed more frequently for variants in the EGFR gene (35.6%, n = 2,377) compared with variants detected in the KRAS (18.1%, n = 1,217) or NRAS (16.8%, n = 456) genes (Table S1).

| Performance by time and EQA participation
On laboratory level, the percentage of HGVS compliance improved significantly (p < .001) over time from 2.0% (n = 94) in 2012 to 22.3% (n = 206) in 2018 (Table 3). An improvement was also observed after multiple EQA scheme participations (p < .001). Thus, 4.6% of cases (n = 614) were reported according to HGVS recommendations during a laboratory's first participation, compared to 19.5% of cases when laboratories had participated six times (n = 41; Table 3). Laboratories who reported variants in complete accordance with HGVS guidelines were significantly less likely (p < .001) to make an error in HGVS nomenclature in their subsequent participation (58.0% compliance compared with 11.6% for laboratories with non-HGVS compliance;  ■ Predicted consequences, that is, protein changes without experimental evidence (no RNA or protein sequence analyzed), should be given in parentheses since 2016 ■ Both three-(preferred) and one-letter amino acid code may be used ■ For all descriptions the most 3′ position possible of the reference sequence is arbitrarily assigned to have been changed ■ For deletions, insertions, deletions-insertions and duplications, the most 5′and most 3′ positions should be mentioned, and separated by an underscore on nucleotide and protein level, followed by their respective prefix: del, ins, delins, or dup ■ Specification of the nucleotide(s) or amino acid(s) (but not the number) is optional for deletions and duplications, but mandatory for insertions and deletion-insertions ■ When the exact protein change is unknown, "X" or "Xaa" is used to indicate Participants that reported an appropriate sequence were significantly (p < .001) more likely to do so again in the next EQA participation.

| Influence of testing methods
The reporting of variant nomenclature according to HGVS recommendations differs depending on the testing methodology used. In total, 9.6% and 9.9% of HGVS compliant entries were observed for users of a commercial (n = 768) or a noncommercial method (n = 717), respectively (both non-next-generation sequencing (NGS) based methods) (Figure 1a). Laboratories using NGS (n = 447) complied with the recommendations in 20.9% of assessed variants. Compliance was observed more frequently when applying noncommercial NGS panels compared with commercial NGS panels (24.6% n = 300 vs. 17.2%, n = 147). The difference between commercial and noncommercial methods was significant (p < .001) for EGFR and KRAS testing, but not for NRAS analysis (p = .385). An overview of the commercial test kits used by the laboratories in this study and the nomenclature used to describe the variants in the package inserts is given in Table S3.

| DISCUSSION
The aim of the ESP and Gen&Tiss EQA schemes for predictive biomarker testing in NSCLC and mCRC is to support diagnostic laboratories in the correct detection and reporting of molecular variants.

Performance between two participations
Correct in previous scheme 16.0 (25) 79.6 (49)*** 60.0 (5)* 55.6 (9)** 58.0 (88)*** Even though we observed a higher proportion of "type 2" errors for complex mutations, it remains to be elucidated how many of these include an infringement on the 3′ rule (which is defined as: 'for all descriptions the most 3′ position possible of the reference sequence should be assigned to indicate a change') by annotation tools.
Nevertheless, even when applying a commercial test kit, it is the final responsibility of the laboratory to comply with HGVS nomenclature when reporting the test results. Awareness and education of the laboratory staff are important at multiple levels, as predictive testing entails a cooperation of molecular biologists, clinical scientists in molecular pathology, and pathologists. First, in our experience, laboratories often apply traditional "legacy" nomenclature (such as "T790M" or "L858R") to cater for the needs of Laboratories who included a correct sequence with a previous version number were not penalized, but did receive a suggestive comment mentioning the availability of a more recent version number. Of more concern is the large fraction of entries for which no version number was included, similar to previously reported results (Tack et al., 2016). Inclusion of a version number is of utmost

ACKNOWLEDGMENTS
We would like to thank all participants to the ESP and Gen&Tiss lung and colon EQA schemes, the scheme assessors, the medical experts, the reference laboratories and regional scheme organizers. Our gratitude also goes to the European Society of Pathology, GFCO, and AFAQAP, for the joint organization of the EQA schemes. We are grateful for the financial support by the French National Cancer Institute (INCa), Roche, AstraZeneca, and Pfizer Oncology. We are grateful to dr. Jan von der Thüsen and Inne Nauwelaers for proofreading the manuscript. These schemes would not have been possible without our colleagues at the Biomedical Quality Assurance Research unit for the coordination of the schemes. (0000-0003-1290-1474), and E. S. (0000-0003-3655-143X) were responsible for technical expertise during the EQA schemes and conceiving of the nomenclature categories during the scheme. C. K., V.
T., K. D., and E. M. C. D. interpreted the data. C. K., E. S., and E. M. C. D.
were responsible for harmonizing the data, statistical analysis, and writing of the manuscript. All authors critically revised the manuscript for important intellectual content.

ETHICAL STATEMENT
The samples originated from tissue blocks of leftover patient material obtained during routine care, and were excluded from informed consent. The molecular testing performed in the schemes reflected the routine tests prescribed, as no additional molecular testing was performed. In addition, each scheme organizer signed a subcontractor agreement stating that the way in which the samples were obtained was conform national legal requirements for the use of patient samples from their biobank.

DATA AVAILABILITY STATEMENT
The datasets generated and/or analyzed during the current study are available from the corresponding author upon reasonable request.