RAPDs were generated from plants of six populations of Isotoma petraea F. Muell. The species occurs on rock outcrops in southern and western Australia, with populations exhibiting different breeding systems, including complete autogamy, varying levels of outbreeding and complex hybridity. Non-metric multidimensional scaling (nMDS) analysis of the random amplified polymorphic DNA (RAPD) data set clearly resolved all populations. The Pigeon Rock population, which is home to both complex hybrid and structural homozygote plants, was divided into those two groups by the nMDS analysis. There was little diversity in highly autogamous populations, but levels were higher in the outbred Yackeyackine population. All complex hybrid populations and plants possessed numerous genetic system-specific RAPDs, some of which were shown to be held in fixed heterozygosity. Estimating GST using RAPDs has been problematical due to their dominance, and analytical methods usually rely on knowledge of the selfing rate or assume Hardy–Weinberg equilibrium. This assumption does not hold when populations exhibit fixed heterozygosity, and an alternative method, Shannon's Index, was used to partition genetic diversity. The distribution of genetic diversity fit expectations for an inbreeding species, with most of the variation (87.5%) occurring between populations. This compares to an average RAPD-based GST of 59.6% for inbreeding species generally and 15.5% for outbreeding species.