Knowledge on the spatial distribution of prokaryotic taxa is an essential basis to understand microbial diversity and the factors shaping its patterns. Large-scale patterns of faunal distribution are thought to be influenced by physical environmental factors, whereas smaller scale spatial heterogeneity is maintained by species-specific life-history characteristics, the quantity and quality of food sources and local disturbances including both natural and man-induced events. However, it is still not clear which environmental parameters control the diversity and community structure of sedimentary microorganisms mediating important ecosystem processes. In this study, multiscale patterns were elucidated at seven stations in the Oyster Ground, North Sea (54°4′N/4°E), 100 m to 11 km apart. These were related to biotic (e.g. multicellular organisms) and abiotic parameters (e.g. organic carbon content in the sediment) to establish the relationship between the distribution of both bacterial and archaeal communities and their environment. A relatively high variability was detected at all scales for bacterial and archaeal communities, both of which were controlled by different suites of biotic and abiotic environmental variables. The bacterial community consisted mainly of members belonging to the Gammaproteobacteria and the Fibrobacteres/Acidobacteria group. Members of the Deltaproteobacteria, Bacteroidetes and Actinobacteria also contributed to the bacterial community. Euryarchaeota formed the majority of archaeal phylotypes together with three phylotypes belonging to the Crenarchaeota.