Inhibition of dipeptidyl peptidase IV is an important approach for the treatment of type-2 diabetes. In this study, we reported a multistage virtual screening workflow that integrated 3D pharmacophore models, structural consensus docking, and molecular mechanics/generalized Born surface area binding energy calculation to identify novel dipeptidyl peptidase IV inhibitors. After screening our in-house database, two hit compounds, HWL-405 and HWL-892, having persistent high performance in all stages of virtual screening were identified. These two hit compounds together with several analogs were synthesized and evaluated for in vitro inhibition of dipeptidyl peptidase IV. The experimental data indicated that most designed compounds exhibited significant dipeptidyl peptidase IV inhibitory activity. Among them, compounds 35f displayed the greatest potency against dipeptidyl peptidase IV in vitro with the IC50 value of 78 nm. In an oral glucose tolerance test in normal male Kunming mice, compound 35f reduced blood glucose excursion in a dose-dependent manner.