Bacterial and archaeal communities in sediments obtained from three geographically-distant mud volcanoes, a control site and a microbial mat in the Eastern Mediterranean deep-sea were characterized using direct 16S rRNA gene analyses. The data were thus in relation to the chemical characteristics of the (stratified) habitats to infer community structure–habitat relationships. The bacterial sequences in the different habitats were related to those of Actinobacteria, Bacilli, Chloroflexi, Alpha-, Beta-, Gamma-, Delta- and Epsilonproteobacteria and unclassified bacteria, including the JS1 group. The archaeal sequences found were affiliated with those of the Methanosarcinales, Thermoplasmales, Halobacteriales and Crenarchaea belonging to marine benthic group I and B, as well as MCG group archaea. In each sample, the communities were diverse and unique at the phylotype level. However, at higher taxonomic levels, similar groups were found in different sediments, and similar depth layers tended to contain similar communities. The sequences that dominated in all top layers (as well as in the mat) probably represented organisms involved in aerobic heterotrophy, sulfide-based chemoautotrophy and methanotrophy and/or methylotrophy. Sequences of organisms most likely involved in anaerobic methane oxidation, sulfate reduction and anaerobic heterotrophy were predominantly found in deeper layers. The data supported the notion of (1) uniqueness of each habitat at fine taxonomic levels, (2) stratification in depth and (3) conservation of function in the sediments.