Hybridization among organisms can potentially contribute to the processes of evolution, but this depends on the fitness of hybrids relative to parental species. A small, recently formed population of fur seals on subantarctic Macquarie Island contains a high proportion of hybrids (17–30%) derived from combinations of three parental species: Antarctic, subantarctic and New Zealand fur seals. Mitochondrial control-region data (restriction fragment length polymorphisms) and nine microsatellites were used to determine the species composition of breeding adults, and hybrid male fitness was measured by comparing reproductive success (number of genetically inferred paternities) of hybrid and pure-species territory males over 6 years. No correlations were found between male reproductive success and three genetic measures of outbreeding, but this may be due to a relatively small number of dominant males analysed. Territory males fathered 63% of pups, but hybrid males had lower reproductive success than pure-species males despite having the same ability to hold territories. A greater proportion of females in hybrid male territories conceived extra-territorially than those in territories of pure-species males, and most (70 of 82) mated with conspecifics. This suggests the presence of reproductive isolating mechanisms that promote positive assortative mating and reduce the production of hybrid offspring. Although we found no evidence for male sterility in the population, mechanisms that reduce lifetime reproductive success may act to decrease the frequency of hybrids. Our study has identified a disadvantage of hybridization — reduced reproductive success of hybrid sons — that may be contributing to the persistence of pure lineages at Macquarie Island and the temporal decline in hybridization observed there.