On the basis of the enzyme-binding features of known potent inhibitors of dipeptidyl peptidase IV, novel α-aminoacyl-containing proline analogs (8Aa–8Ak, 8Ba–8Bj, 8Ca–8Ck, and 8Da–8Di) with the S configuration were designed, synthesized, and their activity profiled. Their structural features were determined by nuclear magnetic resonance (NMR) spectroscopy, low- and high-resolution mass spectroscopy. Five compounds (8Aa, 8Aj, 8Ch, 8Ck, and 8Dc) were shown to have promising inhibitory activities against dipeptidyl peptidase IV. Two of them, compounds 8Aa and 8Aj inhibited dipeptidyl peptidase IV with IC50 values of 4.56 and 8.4 μm, respectively, and displayed no inhibition at other dipeptidyl peptidase IV. The possible binding modes of compounds 6, 7, 8Aa, and 8Aj with dipeptidyl peptidase IV were also explored by molecular docking simulation. This study provides promising new templates for the further development of antidiabetic agents.