Twenty seven accessions of sorghum conserved in the national gene bank of Zambia, representing two of the three agroecological regions of the country, were investigated using simple sequence repeats (SSR) markers in order to determine the extent and distribution of its genetic diversity. We used 10 microsatellite primer-pairs, which generated 2–9 alleles per locus and a total of 44 alleles across the 27 accessions. The observed heterozygosity (HoP) among the accessions ranged from 0 to 0.19 with an average of 0.04 whereas the average expected heterozygosity (HeP) among accessions was 0.07 in line with the fact that sorghum is predominately inbreeder. The analysis of molecular variance (AMOVA) revealed that 82% of the total genetic variation was attributable to the genetic variation among accessions (FST= 0.824; p < 0.001) whereas the genetic variation within accessions accounted for 18% of the total genetic variation. AMOVA on sorghum accessions grouped based on four ethnic groups (Soli, Chikunda, Lozi and Tonga) associated with collection sites revealed a highly significant variation among groups (23%; p < 0.001). Although cluster analysis grouped most accessions according to their sites of collection, some accessions that originated from the same site were placed under different clusters. In addition to the extent and pattern of genetic diversity, consideration should also be given to other factors such as ecogeographic and ethnic differences when sampling sorghum genetic resources for rational and efficient conservation and utilization in the breeding program.