Women with a family history of breast cancer have an approximately twofold elevated risk of the disease. Even though an array of genes has been associated with breast cancer risk the past two decades, variants within these genes jointly explain at most 40% of this familial risk. Many explanations for this ‘missing heritability’ have been proposed, including the existence of many very rare variants, interactions between genetic and environmental factors and structural genetic variation. In this review, we discuss how next generation sequencing will teach us more about the genetic architecture of breast cancer, with a specific focus on very rare genetic variants. While such variants potentially explain a substantial proportion of familial breast cancer, assessing the breast cancer risks conferred by them remains challenging, even if this risk is relatively high. To assess more moderate risks, epidemiological approaches will require very large patient cohorts to be genotyped for the variant, only achievable through international collaboration. How well we will be able to eventually resolve the missing heritability for breast cancer in a clinically meaningful way crucially depends on the underlying complexity of the genetic architecture.