The phylogenetic diversity of archaeal 16S rRNA genes in a thermoacidic spring field of Ohwakudani, Hakone, Japan, was investigated by PCR-based analysis using a novel Archaea-specific primer designed in the present study. Clone libraries of archaeal 16S rRNA genes were constructed from hot water (78 °C) and mud (28 °C) samples by PCR using a newly designed forward primer and a previously reported forward primer with reverse primers. Most phylotypes found in the libraries from the hot water sample were related to cultured (hyper)thermophiles. The phylotypes and their detection frequencies from the hot water sample were similar for the libraries amplified with the two different primer sets. In contrast, phylotypes having a low similarity (<95%) to cultured Archaea were found in the libraries from the mud sample. Some of the phylotypes were relatively close to members of Thermoplasmata (80–93% similarity) and the others were not clearly affiliated with Crenarchaeota and Euryarchaeota, but related to Thaumarchaeota and Korarchaeota. The phylotypes and their detection frequencies were significantly different between the two libraries of the mud sample. Our results from the PCR-based analysis using the redesigned primer suggest that more diverse, uncultured Archaea are present in acidic environments at a low temperature than previously recognized.