Fax: (815) 227-2630.
Validation of a tool for identifying women at high risk for hereditary breast cancer in population-based screening
Article first published online: 11 SEP 2006
Copyright © 2006 American Cancer Society
Volume 107, Issue 8, pages 1769–1776, 15 October 2006
How to Cite
Hoskins, K. F., Zwaagstra, A. and Ranz, M. (2006), Validation of a tool for identifying women at high risk for hereditary breast cancer in population-based screening. Cancer, 107: 1769–1776. doi: 10.1002/cncr.22202
- Issue published online: 3 OCT 2006
- Article first published online: 11 SEP 2006
- Manuscript Accepted: 16 JUL 2006
- Manuscript Revised: 7 JUL 2006
- Manuscript Received: 10 MAY 2006
- OSF Saint Anthony Foundation
- BRCA1 gene;
- BRCA2 gene;
- genetic counseling;
- genetic screening;
- risk assessment;
- mass screening;
- hereditary neoplastic syndromes;
- breast neoplasms
Recent scientific advances provide the opportunity to identify women in the general population at increased breast cancer risk and to offer effective early detection and disease prevention interventions.
A pedigree assessment tool (PAT) was designed to identify women in primary care settings who are at increased risk for hereditary breast cancer, including potential BRCA mutation carriers. The PAT is a simple point-scoring system based on family cancer history with points weighted to account for features associated with a higher probability that a BRCA mutation is present. The ability of the PAT and the Gail model to accurately identify potential BRCA mutation carriers in 3,906 women without a personal history of breast cancer presenting for a screening mammogram at a community hospital was tested.
Eighty-six (2.2%) women had a family history indicative of a high probability (>10%) that a BRCA mutation was present within the family. The PAT performed better than the Gail model in correctly assigning women to the “high BRCA probability” cohort. The area under the receiver operating characteristic (ROC) curve for the PAT was 0.9625 compared with 0.389 and 0.5861 for the Gail model 5-year and lifetime risk estimates, respectively. At the optimal threshold score, the PAT performed with 100% sensitivity and 93% specificity.
The PAT is a simple and accurate tool for identifying women at risk for the hereditary breast cancer syndromes that can be employed as part of a comprehensive breast cancer risk-screening strategy in the primary care setting. Cancer 2006. © 2006 American Cancer Society.
The last decade has produced major advances in the ability of healthcare providers to supply women with accurate information regarding their individual risk of developing breast cancer. Cancer risk assessment has emerged as an important component of cancer risk counseling.1–3 Several empirically derived breast cancer risk prediction models are now available that provide individualized, quantitative risk estimates based on easily acquired familial and nonfamilial risk factor data.4, 5 These risk-prediction models are now widely used in the clinical setting and have been thoroughly reviewed elsewhere.1, 2
Another important advance was the identification of the breast and ovarian cancer susceptibility genes BRCA1 and BRCA2.6, 7 Although germline mutations in these genes are responsible for only 2% to 3% of all breast cancer,8, 9 the extraordinarily high cancer risk associated with these mutations10, 11 underscores the importance of identifying women who harbor BRCA1/2 mutations. Several professional organizations and the US Preventive Service Task Force have recommended that genetic counseling and DNA testing be offered to individuals at high risk for autosomal dominantly inherited breast cancer susceptibility due to mutations in BRCA1 or BRCA2.12–14
Advances in cancer risk assessment have been accompanied by equally important progress in the field of cancer risk-reduction and early detection. The National Surgical Adjuvant Breast and Bowel Project's P-1 breast cancer prevention trial15 and several other studies16, 17 demonstrated the effectiveness of tamoxifen as a chemoprevention agent in high-risk populations. Subsequently, both the American Society of Clinical Oncology and the US Preventive Services Task Force advocated consideration of tamoxifen chemo prevention for high-risk women.18, 19 Data has also emerged confirming that risk-reducing mastectomy20–22 and oophorectomy23, 24 provide effective risk reduction in high-risk women. Advances in breast imaging such as breast magnetic resonance imaging have demonstrated the ability to substantially improve the efficacy of breast cancer screening programs in high-risk women by allowing earlier detection of clinically occult tumors.25, 26
The net result is that we now have tools that have the potential to substantially improve the health of women in our society by reducing breast cancer incidence through cancer prevention27, 28 and more effective early detection programs for high-risk women. For a prevention strategy to succeed, the intervention must be widely applied within the population. A reduction in breast cancer incidence in our society and the resultant public health benefit will only be realized if breast cancer risk-reduction interventions are offered to large segments of the population. Currently available prevention strategies are targeted to women with increased levels of risk.18–26, 29 This implies that a cancer risk-screening strategy is needed that can be applied to the general population to identify those at increased risk who might benefit from cancer risk-reduction interventions. The risk assessment tools employed in the screening strategy must identify women at all levels of risk. This includes women at moderate levels of risk who do not require formal genetic evaluation, but could benefit from chemoprevention and heightened surveillance protocols, as well as the small cohort of women with hereditary breast cancer risk who are candidates for referral to a genetic counselor. Furthermore, the strategy should accurately identify the subgroup of women with autosomal dominantly inherited susceptibility due to BRCA1 or BRCA2 and the subgroup with risk due to genes other than BRCA1/2. Women in both groups can benefit from genetic and cancer risk counseling, although DNA testing is currently recommended only for women with a high probability of harboring a BRCA1 or BRCA2 mutation.12–14
A modified version of the model of Gail et al.4 is widely used in this country for calculating individualized risk estimates for invasive breast cancer. The Gail model has many features that make it an excellent candidate for use in a population-based risk-screening strategy: it has been validated in several different datasets,30–32 all required risk factor data are easily obtained from participants, it incorporates both familial and nonfamilial risk factors, it provides both 5-year and lifetime risk estimates expressed as cumulative absolute risk rather than relative risk, it is now widely available for use on personal computers in the form of the “NCI Risk Disk” (available from the National Cancer Institute at http://cancernet.nci.nih.gov/bcra_tool.html), and the speed and ease of performing the risk calculation makes it well suited for the high throughput necessary in a screening test applied to a large population. However, several authors have raised concern over the ability of the Gail model to accurately identify the subgroup of women at high risk for hereditary breast cancer who require genetic counseling and may be candidates for DNA testing, since it does not incorporate several features associated with the hereditary breast cancer syndromes (young age of breast cancer onset, presence of breast cancer in the paternal lineage or in second/third-degree relatives, and the presence of ovarian cancer or male breast cancer in the family).1, 2, 33
We developed a pedigree assessment tool (PAT) designed to identify women in a primary care setting with family cancer histories suggesting a hereditary breast cancer syndrome, including, but not limited to, potential BRCA gene mutation carriers. In this article we report an analysis of this tool's ability to identify potential BRCA1 and BRCA2 mutation carriers in a population of women presenting for screening mammography, and we compare the performance of the PAT to the Gail model in its ability to accurately identify this high-risk population.
MATERIALS AND METHODS
The PAT is a simple point-scoring system which assigns point values for each case of breast or ovarian cancer within a family (Table 1). The actual numeric values assigned for each case of breast or ovarian cancer were arbitrarily selected, but they are weighted so that more points are assigned for cases with features associated with a higher probability that a BRCA mutation is present: early age of breast cancer diagnosis, ovarian cancer in the family, male breast cancer in the family and Ashkenazi Jewish ancestry.34–43 A PAT score is calculated for both the maternal and paternal lineage by adding together the point assignments for each case of breast or ovarian cancer in each genetic lineage. The final PAT score assigned to the participant is the higher of the maternal and paternal PAT scores.
|Breast cancer at age 50 or higher†||3|
|Breast cancer prior to age 50†||4|
|Ovarian cancer at any age‡||5|
|Male breast cancer at any age||8|
|Ashkenazi Jewish heritage||4|
Breast cancer risk information was collected by mammography technicians from women with no personal history of breast cancer who presented for screening mammography at a community hospital between August of 2001 and December of 2002. All participants gave written consent for the collection of breast cancer risk information, which is routinely collected by mammography technicians as part of the clinical service in our center. Risk factor data were then analyzed for this study retrospectively without patient identifiers.
The PAT score was calculated based on family history information provided by participants at the time of the screening mammogram. The modified version of the Gail model, as provided on the “NCI Risk Disk,” was used to calculate 5-year and lifetime risk estimates for the development of invasive breast cancer, based on information provided by participants at the time of the mammogram. Women were classified as “potentially increased risk” if at least 1 of the following criteria was present: 5-year Gail estimate of at least 1.7%, lifetime Gail estimate of at least 15%, or the presence of at least 1 case of breast or ovarian cancer in any family member. The family cancer history of all “potentially increased risk” participants was analyzed in order to identify women with a family cancer history indicating a high probability that a BRCA1 or BRCA2 mutation is present in their family, as predicted by the model of Frank et al.34 (utilizing prevalence data available online at myriadtests.com/mutprev, according to the “spring 2005” update). Women were classified as “high BRCA probability” if the prior probability of a BRCA1/BRCA2 mutation in their family was greater than or equal to 10% and “low BRCA probability” if the prior probability of a BRCA mutation was less than 10%.
We calculated the sensitivity, specificity, positive predictive value, and negative predictive value for 10 different PAT scores utilized as threshold values for classifying women in the “high BRCA probability” vs. “low BRCA probability” categories. The same calculations were performed using Gail model estimates using 10 different 5-year and lifetime risk estimates as threshold values. We then compared the performance of the PAT score with the performance of 5-year and lifetime Gail model estimates in their ability to correctly identify the “high BRCA probability” population by plotting a receiver operating characteristic (ROC) curve for each test. An ROC curve for a test is generated by varying the cutpoint used to distinguish between the “normal” and “abnormal” population and plotting the resulting true-positive rate (sensitivity) against the corresponding false-positive rate (1-specificity) for each cutpoint.44 A test that discriminates perfectly between the “normal” and the “abnormal” population would provide a cutoff value that perfectly separates the 2 populations. The resulting ROC curve would take the form of a right angle passing through the point (0,1) on an x-y coordinate system, and the area under the ROC curve would equal 1.0. The closer the area under an ROC curve comes to 1.0, the better the discriminating ability of the test. A test with no discriminating ability will generate an ROC curve that passes along the diagonal and would have an area under the ROC curve equal to 0.5.44 A test with these performance characteristics would be no more accurate than the flip of a coin at assigning individuals to the “normal” vs. “abnormal” population.
In all, 3,906 women provided risk information; 737 were classified as “potentially increased risk” (18.9%). Of these 737 women, 651 (16.7%) were categorized as “low BRCA probability” and 86 (2.2%) were categorized as “high BRCA probability,” by virtue of a < 10% vs. ≥ 10% prior probability that a BRCA mutation is present in their family (Table 2). The mean age of the entire “potentially increased risk” cohort was 59 years; the mean age of the “low BRCA probability” and the “high BRCA probability” cohorts were 60 years and 52 years, respectively. The racial distribution of the “potentially increased risk” cohort was 95% Caucasian, 3.3% African-American, 0.7% Hispanic, and 1% Asian. The “high BRCA probability” cohort was 88% Caucasian, 11% African-American, 0% Hispanic, and 1% Asian. (See Table 3 for additional comparisons of these cohorts.)
|Population||No. of women (%)|
|Potentially increased risk*||737 (18.9)|
|Low BRCA probability†||651 (16.7)|
|High BRCA probability†||86 (2.2)|
|Characteristic||Low BRCA probability||High BRCA probability|
|No. of women||651||86|
|Mean age, y||60||52|
|Family history of breast cancer (%)||451 (69)||77 (92)|
|Early onset breast cancer in family (%)||155 (24)||61 (71)|
|Two cases of breast cancer in family (%)||116 (18)||32 (37)|
|Three or more breast cancers in family (%)||28 (4.3)||20 (23)|
|Bilateral breast cancer in family (%)||9 (1.4)||6 (7)|
|Male breast cancer in family||0||4 (4.7)|
|Ovarian cancer in family (%)||29 (4.4)||39 (45)|
|Breast and ovarian cancer in family (%)||17 (2.6)||30 (35)|
|Jewish ancestry||1 (0.2)||0|
We then analyzed the ability of the PAT score and the Gail model 5-year and lifetime estimates to correctly identify the 86 women in the “high BRCA probability” cohort using 10 different cutoff values for all 3 tests (PAT scores of 3, 4, 6, 7, 8, 9, 10, 12, 14, and 16; 5-year Gail estimates of 1.0%, 1.5%, 2.0%, 2.5%, 3.0%, 4.0%, 5.0%, 6.0%, 7.0%, and 8.0%; lifetime Gail model estimates of 5%, 7%, 10%, 12%, 15%, 20%, 25%, 30%, 35%, and 40%). The sensitivity, specificity, positive predictive value, and negative predictive value were calculated for each cutpoint. Representative results are presented in Table 4. This analysis identified a PAT score of ≥ 8 as the optimal PAT threshold value for assigning women to the “high BRCA probability” category. At that cutoff value the PAT had a sensitivity of 100%, specificity of 93%, positive predictive value of 63%, and negative predictive value of 100%. We were unable to identify a 5-year Gail model cutpoint value with acceptable performance characteristics. The only cutpoint values with sensitivity greater than 50% (5-year estimate of 1.0% = 75% sensitivity and 5-year estimate of 1.5% = 59% sensitivity) were associated with specificities of only 5% and 10%, respectively. Increasing the stringency of the cutpoint improved the specificity, but sensitivity dropped to unacceptable levels. For example, with a 5-year Gail model estimate of ≥ 3% as a cutoff, specificity improved to 73%, but sensitivity dropped to 27%. A similar pattern was observed with lifetime Gail model estimates. No Gail estimate threshold value could be identified with both sensitivity and specificity greater than 55%.
|PAT score||5-Year Gail estimate||Lifetime Gail estimate|
|≥ 6||≥ 7||≥ 8||≥ 9||≥ 2.0%||≥ 3.0%||≥ 4.0%||≥ 5.0%||≥ 6.0%||≥ 15%||≥ 20%||≥ 25%||≥ 30%||≥ 35%|
|Predictive value (+) test||0.33||0.45||0.63||0.59||0.07||0.12||0.18||0.21||0.19||0.14||0.21||0.28||0.34||0.63|
|Predictive value (−) test||1.0||1.0||1.0||0.95||0.69||0.89||0.89||0.89||0.89||0.90||0.90||0.90||0.90||0.89|
The overall performance of the PAT score, 5-year, and lifetime Gail model estimates in correctly assigning women to the “high BRCA probability” cohort was analyzed by comparing the area under the ROC curves. The results of this analysis are presented in Figure 1. The area under the ROC curve for the PAT was 0.9625 compared to 0.389 and 0.5861 for 5-year and lifetime Gail estimates, respectively. In this analysis the PAT score performed closer to an “ideal test” for identifying women with potential autosomal dominant susceptibility than either 5-year or lifetime Gail estimates, since the area under the ROC curve for the PAT score most closely approached 1.0.
The data presented in this report indicate that the PAT has better discriminating ability than the modified version of the Gail model for identifying women in a screening mammography population who would benefit from genetic counseling and are potential candidates for BRCA gene mutation analysis. We recognize that the Gail model was not designed for that purpose and therefore the results may not be unexpected, and in fact the comparison may seem unfair or irrelevant. However, the Gail model is widely used as a risk screening tool in this country, and we are unaware of any other analysis of its ability to identify women with potential autosomal dominant breast cancer susceptibility in the general population. Euhus et al.33 evaluated the ability of the Gail model to accurately assign breast cancer risk in 213 women attending a specialized breast cancer risk clinic. The Gail estimates were compared to risk estimates generated with 3 additional models (the Claus model, BRCAPRO, and the Bodian tables). The authors concluded that the Gail model was “an appropriate risk assessment tool for most women attending specialized clinics” since only 13% of study participants required risk level reassignment after evaluation with the other 3 models. However, the authors further state that the inclusion of models better able to account for family history information is required to accommodate all women. The Gail model performed better in the Euhus study than in the study presented here. However, several important differences exist between the 2 analyses. First, our study population consisted of women in the general population selected only by their desire for a screening mammogram, whereas Euhus et al. studied women attending a specialized breast cancer risk assessment clinic. Second, the study of Euhus evaluated the performance of the Gail model in determining lifetime breast cancer risk relative to other models designed more specifically for assessing women with potential hereditary cancer risk. Our study,conversely, was designed to test the ability of the Gail model to accurately identify women in the general population who would be potential candidates for further evaluation in a Cancer Risk Clinic with genetic counseling and DNA testing, by virtue of a high probability that a BRCA mutation is present in their family. For these reasons, we do not believe that the data of Euhus et al. necessarily conflict with our results.
Our decision to compare the PAT to the Gail model rather than to other models designed more specifically for assessing hereditary breast cancer risk34–38, 45, 46 may also be questioned. Several of these model models have been thoroughly reviewed elsewhere.2 We believe that none of these other models are suitable for use in a population-based risk screening program applied at a primary care level for many reasons. First, most of these models require detailed information regarding ages of cancer diagnosis for all relatives with breast cancer, and many women do not have that information readily available when they present in a primary care setting. Second, several of the models predict only the probability of carrying a BRCA1 or BRCA2 mutation. We believe that it is also important to identify women with polygenic susceptibility as well as susceptibility due to high penetrance genes other than BRCA1 and BRCA2, since these women also benefit from cancer risk counseling and implementation of a cancer risk-reduction program. The models referenced above were not designed to identify this subgroup of women. In this regard, the fact that the positive predictive value associated with a PAT score of 8 was only 63% is actually desirable, since women with a “false-positive” score still benefit form cancer risk counseling, even though they are not candidates for DNA testing at the present time. Finally, several of these models are fairly complex and time-consuming to operate. We question whether they could be efficiently employed as screening tools in a primary care setting, given the large number of women who require assessment and the time constraints inherent with delivery of care in that setting. These models are extremely useful when employed for their intended purpose, i.e., for estimating BRCA mutation carrier probabilities in a cancer risk counseling session. Since they are unlikely to function well as part of a risk-screening strategy as proposed here, we believe that comparing them to the PAT for that purpose is not germane. In our opinion, the Gail model is the only other model that can realistically function with the high throughput demanded by population screening performed at the primary care level.
Another potential criticism of this study is the use of the Frank model34 as the “gold standard” against which the PAT and Gail model were judged. As stated above, we believe that any risk screening strategy applied at a population level must identify with a high degree of accuracy the small subgroup of women at risk of harboring BRCA mutations. Since current guidelines recommend consideration of BRCA1 and BRCA2 DNA testing for individuals with a prior probability of ≥ 10% of detecting a BRCA mutation,12, 13 we believe that the ability of a test to identify women in that cohort is an important and relevant measure of its utility as a risk-screening tool. We chose to employ the Frank model as the gold standard for identifying the cohort of women with ≥10% prior probability for several reasons. First, it is the largest dataset yet published that correlates clinical characteristics with the probability of detecting mutations in BRCA1 and BRCA2, with over 10,000 patients analyzed. Second, the laboratory method used in that analysis (full-length sequencing in all patients) is currently the most accurate technique available for identifying mutation carriers. Third, the patient population in that study is most similar to our population, since patients were accrued primarily from clinical rather than research settings, and family history information was ascertained in a manner similar to ours, i.e., by patient self-reporting and not confirmed by healthcare professional review of family medical records.
In summary, we have demonstrated that a simple point-scoring system (PAT) performs very well in identifying women in a screening mammography population who would benefit from referral to a cancer risk clinic for genetic counseling and consideration of DNA testing of appropriate family members. A PAT score of ≥ 8 provided 100% sensitivity with 93% specificity and is the optimal cutoff score for this purpose. While the Gail model performed less well in that particular task, it remains an important tool in a comprehensive risk-screening strategy by virtue of its ability to identify women with increased breast cancer risk due to nonfamilial risk factors and genetic influences other than BRCA1 and BRCA2. Furthermore, the FDA-approved indication for tamoxifen chemoprevention is based on the result of a Gail model risk estimation. We believe that the concept of a comprehensive breast cancer risk-screening strategy applied to large populations at the primary care level warrants further investigation, and that such a strategy could be effectively employed by combining the Gail model with a tool like the PAT. We have developed a computer software program that collects all the relevant risk factor data and calculates Gail model estimates and the PAT score. The program can be easily used by providers with little or no knowledge of breast cancer risk assessment, and can serve a triage function to identify women who require referral to a genetic counselor or specialized cancer risk clinic as well as to identify women at moderate levels of risk who do not require a formal genetic evaluation, but would be appropriate candidates for tamoxifen chemoprevention. We believe that this concept warrants further investigation and we are currently planning to test this tool in other populations to verify the results presented here.
Note Added in Proof
A computer program which simultaneously calculates a Gail Model estimate and a pedigree assessment tool score is available on request at no charge.
- 12Statement of the American Society of Human Genetics on genetic testing for breast and ovarian cancer predisposition. Am J Hum Genet. 1994; 55: i–iv.
- 13Statement of the American Society of Clinical Oncology: genetic testing for cancer susceptibility. J Clin Oncol. 1996; 14: 1730–1736.