One pharmacophore model and three quantitative structure–activity relationship models were developed on a series of benzimidazole and imidazole inhibitors of histone deacetylase 2. The goodness of hit score value of the best pharmacophore model was 0.756, which indicated that it is reliable to be used for virtual screening. The built pharmacophore model was used to search the NCI database. The hit compounds were subjected to molecular docking. The results showed that 25 compounds had high scores and strong interactions with histone deacetylase 2. In three-dimensional quantitative structure–activity relationship studies, good predictive models were obtained using comparative molecular field analysis, comparative molecular similarity indices analysis, and Topomer comparative molecular field analysis. Some putative active compounds were proposed based on compound no. 41. Twenty-six compounds had high scores and good interactions when they were docking into histone deacetylase 2.