I use explicit genetic models to investigate the importance of natural and sexual selection during sympatric speciation and to sort out how genetic architecture influences these processes. Assortative mating alone can lead to speciation, but rare phenotypes’ disadvantage in finding mates and intermediate phenotypes’ advantage due to stabilizing selection strongly impede speciation. Any increase in the number of loci also decreases the likelihood of speciation. Sympatric speciation is then harder to achieve than previously demonstrated by many theoretical studies which assume no mating disadvantage for rare phenotypes and consider a small number of loci. However, when a high level of assortative mating evolves, sexual selection might allow populations to split into dimorphic distributions with peaks corresponding to nearly extreme phenotypes. Competition then works against speciation by favouring intermediate phenotypes and preventing further divergence. The interplay between natural and sexual selection during speciation is then more complex than previously explained.