Males and females differ in their reproductive roles and as a consequence are often under diverging selection pressures on shared phenotypic traits. Theory predicts that divergent selection can favor the invasion of sexually antagonistic alleles, which increase the fitness of one sex at the detriment of the other. Sexual antagonism can be subsequently resolved through the evolution of sex-specific gene expression, allowing the sexes to diverge phenotypically. Although sexual dimorphism is very common, recent evidence also shows that antagonistic genetic variation continues to segregate in populations of many organisms. Here we present empirical data on the interaction between sexual antagonism and genetic drift in populations that have independently evolved under standardized conditions. We demonstrate that small experimental populations of Drosophila melanogaster have diverged in male and female fitness, with some populations showing high male, but low female fitness while other populations show the reverse pattern. The between-population patterns are consistent with the differentiation in reproductive fitness being driven by genetic drift in sexually antagonistic alleles. We discuss the implications of our results with respect to the maintenance of antagonistic variation in subdivided populations and consider the wider implications of drift in fitness-related genes.