Evolution of RAS testing over time: factors influencing mutation rates in metastatic colorectal cancer patients

Approximately 1.85 million new cases of colorectal cancer (CRC) were diagnosed in 2018 and is the third most commonly diagnosed cancer worldwide, representing 10.2% of all cancers [1,2]. In newly diagnosed cases, 20–25% will have already developed metastasis and up to 50% of all CRC patients eventually develop metastatic disease [2–6]. The 5-year survival rate for patients with metastatic CRC (mCRC) is approximately 14% [7].

Over the past 15 years, the introduction of monoclonal antibodies (mAbs), such as cetuximab and panitumumab into mCRC treatment has led to an improved overall and progression-free survival for many patients [8–12]. By binding to the EGFR, both cetuximab and panitumumab disrupt several cancerous signaling pathways involved in a variety of cellular processes such as control of cell growth and survival, cell cycle progression, cell proliferation and metastasis [13,14].

Observational studies have initially shown that mCRC patients with KRAS mutations in exon 2 (codons 12 and 13) are less likely to benefit from anti-EGFR therapies [15–21]. Further studies identified a number of additional mutations in exons 2, 3 and 4 of KRAS and NRAS, which impact on response to anti-EGFR therapies [9,22,23]. Therefore, in July 2013, the therapeutic indication for panitumumab was restricted by the European Medicines Agency (EMA) to patients with RAS (exons 2, 3 and 4 of KRAS and NRAS) wild-type mCRC tumors only (codons 12, 13, 59, 61, 117, 146). In November 2013, the EMA also determined that cetuximab could only be given to mCRC patients without a RAS mutation

Mutations in the BRAF gene occur in 6–15% of all CRC patients [24,25]. Testing for this gene is often combined with RAS testing for its prognostic value, rather than for predicting response to anti-EGFR mAbs, as there is no clear evidence of its predictive role at the moment [17,26].

It is not well understood whether or how the label changes impacted RAS testing for mCRC. Laboratories may have had to broaden their testing extensiveness, introduce new methods of higher sensitivity or test a higher number of samples. In addition to testing practices, improvement of the quality of laboratory testing is important to understand trends in laboratory testing and RAS mutation rates.

This study aimed to understand how RAS testing has evolved over time among mCRC patients in Europe and to evaluate whether laboratory performance is satisfactory 5 years after the EMA label changed. This manuscript consists of European-wide longitudinal data from two sources: external quality assessment (EQA) to evaluate misclassifications related to testing methodology; and a laboratory survey on the number of samples tested and RAS mutation rates in mCRC patients by methodology.

Material & methods

A retrospective observational study with two components was conducted: EQA data from the European Society of Pathology (ESP) Colon EQA scheme; and information was collected from a subset of laboratories participating in the ESP Colon schemes between 2013 and 2017 via a survey.

EQA scheme set-up

Data from the ESP Colon EQA schemes were collected between 2013 and 2018 according to ISO 17043 [27,28], from 239 unique laboratories from 44 different countries.

Each participant received three glass slides from ten formalin-fixed paraffin embedded tissue samples for pathological review, DNA extraction and mutation analysis. The preparation and validation of the samples was performed by regional scheme organizers until 2016 [29] and by one reference laboratory from 2017, following confirmation by another reference laboratory with experience in molecular pathology. All sample material was remaining resection specimens from real-life patients, carefully validated for containing a minimum of 30% neoplastic cells and sufficient DNA quantity and quality for testing by the participants.

Participants were asked to analyze these samples for RAS hotspot variants and optional BRAF hotspot variants using their routine procedures within 14 calendar days. In addition, participants provided information on the applied methodologies along with their testing results in an online datasheet. Results were assessed by a team of international experts in molecular pathology using pre-defined and harmonized scoring criteria [27].

EQA data analysis

For all analyses, only samples that contained neoplastic cells were included. In 2013, 18 tests were excluded from the analysis because the primer sequences used were unable to anneal to the template artificial DNA as a result of gene modulation technology. No samples from 2015 were included as there was not an EQA scheme organized in that year.

Misclassification rates were calculated as the total number of incorrect results, divided by the total number of performed tests, for which an outcome could be provided. Incorrect results included false-positive (FP) results (identification of a variant in a wild-type case, or an additional variant besides the correct variant), false-negative (FN) results (variant in the sample unidentified) and wrong mutations that affected the mutation rate (variant in another gene reported resulting in an FN or FP results for the respective gene). Wrong mutations that did not influence the mutation rate or possible therapy decision (e.g., NRAS c.34G>C p.[Gly12Arg] reported, instead of NRAS c.34G>T p.[Gly12Cys]), were not included in the ‘misclassification rate' definition. The true FN rates were calculated as the total of FN results, for samples containing a variant (with the exclusion of WT cases). To calculate misclassification rates related to test methodology, only methods by which at least 50 tests were performed in all scheme years are included. Technical failures are defined as samples for which no outcome could be given because of a test failure. The failure rate was calculated as the total number of technical failures divided by the total number of tests. Laboratories participated successfully to the EQA program if they obtained a score of at least 90% and made no major genotyping error, in other words, errors which could possibly lead to the administration of the wrong therapy.

Pathology data collection via a survey

All European centers that participated in the ESP Colon EQA schemes between 2013 and 2017 were invited to complete an online survey, to collect data on testing strategies, the number of samples tested annually for mCRC and the annual RAS mutation rate. The survey questions and responses are shown in Supplementary Data 1.

Average numbers of RAS samples tested per laboratory for a specific year were calculated from total numbers of RAS samples tested by all laboratories over the number of laboratories that provided information on that year. Only 6 months of data was available for years 2013 and 2017. To achieve comparable estimates per laboratory, the same testing rate was assumed for the remainder 6 months of these years, and hence, the means for these 2 years were doubled. The RAS mutation rate was calculated as the total number of samples of RAS mutant status over the samples with RAS known status:

Results

ESP Colon EQA schemes

From all unique laboratories participating in the ESP Colon EQA schemes between 2013 and 2018, 41.0% (98/239) were situated in a university hospital, 26.8% (64/239) in a community hospital, 18.0% (43/239) were private laboratories, 7.1% (17/239) related to an anticancer institute, 5.0% (12/239) were situated in a private hospital and 2.1% (5/239) were laboratories from the industry (Table 1).

Since 2013, an increase was observed in the codons tested in both KRAS and NRAS (Table 1), but full RAS testing (exon 2, 3, and 4 of KRAS and NRAS genes) had not been implemented by all laboratories in 2018. Table 1 shows how full RAS testing was adopted over time for different laboratory settings. In general, the percentage of laboratories who implemented full RAS testing increased between 2013 and 2018 from 43.1% (56/130) to 76.7% (86/112). Of the 112 participants in 2018, 82.9% (29/35) of community hospitals and 80.4% (37/46) of university hospitals included full testing compared with only 71.4% (5/7) of anticancer institutes and 62.5% (15/24) private laboratories/hospitals. It was also demonstrated that between 2013 and 2016 laboratories that applied full RAS testing produced a better EQA absolute score and pass-rate (Table 1). Laboratories which did not perform full RAS testing often did not test for alterations in exon 4 of both genes. In 2017 and 2018 no such variants were included in the EQA scheme. In addition, the scheme of 2018 included the NRAS c.38_40delinsACA p.(Gly13_Val14delinsAspIle) variant. Four laboratories in the ‘full RAS testing group’ were penalized for not correctly detecting this variant by next-generation sequencing, lowering the average score and pass-rate.

The number of NGS users in the EQA scheme increased between the 2013 and 2018 scheme from 6.9% (9/130) to 44.6% (50/112), in favor of non-NGS noncommercial methods which decreased from 37.7% to 9.8% The number of non-NGS commercial kit users remained stable (40.0% in 2013 vs 45.5% in 2018). Among NGS users, the use of a commercial panel increased over time, in favor of the use of laboratory developed primers (Table 1).

Across all EQA schemes 5471 tests were analyzable (without technical failures), of which 96.1% (5255/5471) were correct and 3.9% (216/5471) were misclassified. The percentage of misclassifications was the highest for tests performed on samples containing an NRAS variant with 6.0% (33/554) compared with KRAS variants with 4.7% (108/2306), BRAF variants with 4.9% (25/510) or wild-type samples with 2.4% (50/2101). A detailed overview of misclassification results per gene and variants is given in Table 2. The percentage of true false-negative results was 5.4% (152/3370), of which 4.6% (132/2860) for RAS and 3.9% (20/510) for BRAF testing (data not shown).

The percentage of misclassifications observed during the EQA scheme increased initially from 3.6% (42/1176) in 2013 toward 5.9% (71/1195) in 2014. Thereafter, misclassification rates decreased toward 1.6% (18/1100) in 2018 (Figure 1A). This coincided with an initial increase in FN results from 4.2% (28/669) in 2013 to 5.8% (42/719) in 2014 and 5.9% (42/709) in 2015. Thereafter, FNs decreased also to 2.6% (16/662) in 2018 (Figure 1B).

A decrease in misclassifications was also observed after repeated participation to EQA scheme rounds. Namely, for laboratories who participated only once (irrespective of the scheme year), 5.4% of tests (116/2156) were incorrectly analyzed compared with 0.2% of tests (1/422) for laboratories in 2018, who took part in all five successive scheme years (Figure 2A). The fraction of false-negative results declined simultaneously from 5.9% (75/1277) to 0.4% (1/254) between a first and fifth participation, respectively (Figure 2B).

A wide range of test methodologies was used by the participants for KRAS, NRAS and BRAF analysis with a variety of misclassification and failure percentages (Table 3). All genes combined, more misclassifications were observed in tests performed by non-NGS based commercial test kits with 4.3% (186/4372) compared with non-NGS/noncommercial (laboratory-developed) tests with 2.6%, (69/2693) and NGS-based techniques with 1.3% (29/2207).

To evaluate the influence of a change in test method, 25.5% (336/1320) of data points were excluded, as these were new laboratories and no method information was available from the previous EQA scheme. For the remaining 74.5% (984/1320) of data, a switch in testing methodology was observed in 35.5% (349/984) individual laboratory participations (data not shown). There was no observed difference in errors or technical failure rates between laboratories who changed and those who did not change test methodologies (data not shown).

In total, 1.6% (91/5562) technical failures were observed, which were more frequent for wild-type samples with 2.2% (48/2149) compared with cases containing KRAS (1.1%, 25/2331), NRAS (1.6%, 9/563) or BRAF (1.7%, 9/519) variants (data not shown) and for NGS (2.5%, 55/2207) compared with the other testing strategies (1.4%, 62/4372 and 1.7%, 47/2693 for commercial and noncommercial tests, respectively; Table 3). Technical failures diminished between 2013 and 2018 from 2.9% (35/1211) to 1.3% (15/1115 tests; Figure 1C) and for laboratories who participated to EQA more frequently (Figure 2C). Namely, participants on average reported 2.3% (50/2206) of technical failures during their first participation, compared with 0.3% (2/607) and 1.4% (6/428) during their fourth and fifth participation, respectively.

Pathology data collection via a survey

In addition to data from the EQA database, the 200 laboratories participating to the EQA schemes between 2013 and 2017 were invited to provide additional information via a survey. 95 laboratories initiated the online link of which 33 laboratories from 28 countries provided the information. Of those 33 laboratories, 21 laboratories provided information for the full study period, three (2013–2018) provided data for 2014 to 2017, four provided information for 2015 to 2017 and two provided data for 2016 to 2017.

The majority of participating laboratories (78.8%; 26/33) reported to test RAS on request by the oncologist only, whereas seven laboratories (21.2%; 7/33) reported routinely testing all CRC/mCRC patients (Q3 from survey, see Supplementary Data 1). For the remaining laboratories, testing was performed on request by either the oncologist or the pathologist (n = 2), or by other hospitals (n = 1). Between 2013 and 2017, the 33 laboratories provided information on 27,325 samples from mCRC patients, which were tested for RAS mutations. There was a small increase on the average numbers of mCRC samples tested from 246.5 in 2013 to 271.1 in 2018 (Table 4). A total of 18 (54.5%) laboratories reported to have a ‘notable’ increase in the number of samples tested during the period of 2013 to 2017, whereas three reported a ‘notable’ decrease (Q5 from survey, see Supplementary Data 1). The most common reasons the laboratories provided for ’notable’ testing increases were ‘more routine testing occurring’ with 50.0% (9/18) and ‘larger catchment area’ with 22.2% (4/18) (Q5a from survey, see Supplementary Data 1). A total of 17% (3/18) of laboratories thought both an increase in routine testing and testing a larger catchment area resulted in the increase. A total of 11% (2/18) of laboratories reported other reasons for this increase. The overall RAS mutation rate in the tested samples was 45.2% (12,341/27,325, 95% CI: [44.6%–45.8%]). There was no consistent change between 2013 and 2017 with the annual estimates varying between 42.6% (998/2343; 95% CI: [30]) in 2013 and 47.0% (3586/7636; 95% CI: [45.8%–48.1%]) in 2016 (Table 4).

Discussion

The EMA label for anti-EGFR mAbs was extended to full RAS testing in 2013. This study aimed to provide an overview of RAS biomarker testing in Europe, in relation to mutation rates and testing practices, based on data from the ESP Colon EQA program and a European-wide survey. The results demonstrated an overall RAS mutation rate in the tested samples of 45.2% (12,341/27,325, 95% CI: [44.6–45.8%]), which is in line with other observational studies of RAS mutations in Europe [24,31]. Although this is substantially lower than the 55.9% (95% CI: [53.9–57.9%]) reported in some clinical trials [32]. A possible explanation could be that in the study by Peeters et al. the majority of patients were tested in the context of second-line therapy or best supportive care, whereas this is unknown for our current study. Moreover, as RAS status was determined for 1014 patients (compared with more than 27,000 cases in our study), the chance of obtaining a RAS mutation positive case would be higher. The inclusion of this large dataset on tested samples in this study, allows a solid estimation of the mutation rate, as even the rarest variants in these genes have been reported at a frequency of ≥1% [33].

There was no consistent change in RAS mutation rate over time. One would have expected that if laboratories are now testing for more alterations in RAS and are improving their experience and performance over time, the RAS mutation rate would have also increased. The results of this study clearly illustrate that this is not the case. A plausible explanation could be that alterations in exons 3 and 4 of KRAS and NRAS genes are rather rare and therefore have little impact on the total mutation rate. Given that information on specific rare variants rates is missing in the early years, further investigation may be required to evaluate mutation rates of less frequent variants in the different RAS codons to verify if mutations in specific codons were missed specifically in the first years. In addition, information on mutation rates before the change of the drug labels in 2013 might provide a good estimate of the mutations that were missed at the time that exon 4 mutations were not yet included. However, retrieving this information might be difficult and therefore, retesting of leftover samples may be necessary.

Most laboratories that participated in the EQA programs between 2013 and 2018 have implemented full RAS testing over time, however, KRAS and NRAS codons 59, 117 and 146 remain untested by a group of laboratories. Especially in the early years after 2013, laboratories had separate kits for KRAS exon 2 testing and secondary for the other KRAS exons and NRAS. If the patient had a KRAS exon 2 mutation, laboratories could save money by not performing further tests for the other exons. For NRAS for instance, the two most widely used kits for NRAS analysis Therascreen NRAS Pyro Kit (Qiagen DxS) and Cobas^®NRAS Mutation Test (Roche) only include codons 12, 13 and 61 of the NRAS gene (Supplementary Table 2). Eight out of nine laboratories, which are not yet offering full RAS testing anno 2018, declared to use a non-NGS based commercial kit, which does not include all codons. The other laboratory used a NGS-based commercial kit. Important alterations could be present in the codons for which testing is not fully implemented yet, such as the NRAS c.181C>A p.(Gln61Lys) variant, which accounts for 27.2% of NRAS mutations (COSMIC).

The laboratories offering full RAS testing right after the label change were typically university hospitals, with on average high scores during EQA participation (≥18/20 points). In a previous study from Tack et al., it was observed that laboratories that also perform testing in a research setting (as opposed to only in a clinical setting), and laboratories which are accredited, had a better performance for NRAS testing in the first years after the label change (2013–2014), with less analysis errors being were made [34]. In the ESP Colon EQA schemes, no points were deduced when a laboratory did not test for rare variants in exons 3 and 4. Even with this milder scoring system, laboratories with incomplete RAS testing scored worse than those with full RAS testing. From a clinical point of view, it is time for EQA programs for being stricter with regard to rare variants, and demand testing for all clinically relevant mutations. Participants are advised to apply a testing method covering all relevant mutations, and to review relevant literature as more mutations might become clinically relevant in the future.

In this study, an overall percentage of true FN RAS results of 4.6% (132/2860, all scheme years) was observed, which might have serious implications for patient safety by altering the decision on targeted therapy administration. (Figure 1B). It must be noted that EQA performance is only an indication of routine practice. However, recently it was shown that laboratories performing well in EQA also perform well in routine testing, stressing the importance of these results for daily practice [35].

The observed FN and overall misclassification rates initially increased over time and where a laboratory participated in more EQA schemes, resulted in an improvement toward 2018. This initial increase could be explained by the more stringent testing criteria for full RAS testing, or by the lack of laboratory experience with testing multiple markers or participating to the EQA scheme. The following improvement coincides with the shift toward full RAS testing and the more sensitive techniques. Higher FN or misclassification rates during the first year (2013) or first participation, were especially observed for NRAS, presumably because laboratories already had more prior experience with KRAS testing compared with NRAS generally. Also, more misclassifications occurred for RAS variants in codons 59, 61, 117 and 146 compared with codons 12 and 13 (Table 2), suggesting that although routinely tested for several years, laboratories have still not fully adopted less frequent mutations. In addition, more misclassifications were observed in 2018 for a complex KRAS deletion–insertion compared with other point mutations, of which 90.0% (9/10) included FN results (wild-type reported) and in 10.0% (1/10) a KRAS codon 146 variant was reported instead. Increased error rates for complex mutations have been described in non-small-cell lung cancer as well. Even though BRAF is not yet approved for predictive testing, performance for this biomarker was similarly compared with those observed for RAS analysis. While one would expect more misclassifications for laboratory-developed noncommercial techniques, due to their limited sensitivity, our analysis revealed that those methods performed slightly better when compared with commercial test kits (Table 3). This may be explained by the fact that commercial test kits often do not include all necessary codons for testing, as described earlier, and the danger of FP results with high-sensitive mutation specific analyses.

On the contrary, more technical failures were made by NGS compared with other test methodologies. Since all distributed samples contained at least 30% neoplastic cells, which is above the minimum requirement of all test methods used, the effect of the neoplastic cell percentage estimates prior to DNA extraction was not included to calculate the misclassification or technical failure rates in this study. Nevertheless, this estimation has previously been reported to be highly variable, which might be at the base of the higher failure rate for NGS as an accurate neoplastic cell content estimation helps to set a minimal read depth, because samples with a low percentage of tumor cells require higher coverages [36]. For the 0.1% (91/5562) technical failures the main reasons were a too low quality of the extracted DNA (34.1%, 31/91), DNA concentration too low (12.1%, 11/91) or a too low percentage of neoplastic cells in the sample (9.9%, 9/91). Survey results indicated that the laboratories are indeed taking the necessary quality control actions after DNA extraction prior to proceeding with the analysis itself, with the majority of respondents quantifying the DNA concentration by a spectrophotometer (54.4%, 18/33) or another fluorescence method (39.4%, 13/33) (Supplementary Data 1, Q4). The fact that laboratories report technical failures more frequently in wild-type cases could be explained by their insecurity of interpretation of the test result. Namely, in case of too low DNA concentration, a wild-type result does not exclude the presence of a mutation which was not detected, while detection of a variant can be interpreted with more certainty.

The study provided average number of samples tested per laboratory per year, which determined a limited increase of the average number of samples tested between 2013 and 2017. These estimates should be interpreted with caution as they do not strictly correspond to the proportion samples tested for RAS. This is because there is no fixed denominator as the patient catchment area for each laboratory can vary by year. Indeed, out of the 18 laboratories that reported a ‘notable’ increase in samples tested, 50.0% (9/18) was reportedly due to a larger patient catchment area whereas 22.0% (4/18) was due to more samples being tested (Q5 from survey, Supplementary Data 1). The latter is reflected by the increasing incidence rate of mCRC in Europe [37]. Although here the average number of tested cases per year is presented across all respondents, previous research has indicated that testing a larger number of samples per year can serve as a predictive factor for a better performance during EQA [34].

To the best of our knowledge, with approximately 27000 mCRC samples tested for RAS, this study provides the most robust estimates of real-world RAS mutation rates to-date. Moreover, it is the first time that mutation prevalences were linked to laboratory practices to further understand their impact. The results showed no trend of RAS mutation rate over time despite having a clear shift to more sensitive tests and increased quality of the testing methods. The overall landscape of RAS testing in Europe appears to be positive with highly sensitive and specific tests. Since testing techniques have no impact, this mutation rate can now be used for a future follow-up of the mutation rate in population of patients with mCRC. However, as there still appears to be a small amount of misclassification it is important that laboratories continue to improve their testing methods and should all implement full RAS testing. Although NGS is becoming the gold standard for full testing, technical parameters (DNA quality and percentage of neoplastic cells) must be considered to prevent technical failures. The current study only presented data from a laboratory point of view. It could be further studied whether clinicians correctly interpret the test results and translate it to the right choice for therapy.

Future perspective

It is suspected that there will be further studies that will confirm the RAS mutation rate on a large scale. Although new mutations with an effect on the response to anti-EGFR therapy are becoming known, the impact on the RAS mutation rate will be limited as these are rather ‘rare mutations’.

The performance of the laboratories is currently high, with 96.4% of the samples correctly analyzed, yet it is expected that the laboratories will strive toward 100% accuracy. This is important given the effect the test result has on the response to anti-EGFR therapy. A first step toward this increased performance will be the further expansion of the number of codons tested and also the implementation of broad-testing and sensitive methods. A possible shift to testing on liquid biopsy samples is also expected, for which even more sensitive methods are needed than for tissue samples. It is important that when implementing these novelties, standard procedures are used for test validation and -verification and internal quality control. Earlier results of EQA schemes showed that this ‘quality-way of thinking’ is already known in Europe, but that there is still room for improvement.