Looking for water-soluble inhibitors of matrix metalloproteinase-2 (MMP-2 or gelatinase A), we have previously reported compound 1, a potent MMP-2 inhibitor with a promising selectivity over the structurally homologous MMP-9 (gelatinase B). Here we report the results of Molecular Dynamics (MD) simulations for both gelatinases (MMP-2 and MMP-9), and for the corresponding MMP/1 complexes, in an attempt to shed light on the observed selectivity between the two enzymes. These studies indicated a higher plasticity of MMP-2 at the S1′ pocket and suggested an induced-fit effect at the “back door” of this pocket. On the basis of these observations, we designed 11 a–d to aid further discrimination between MMP-2 and MMP-9. Those compounds displayed notably lower inhibitory activities against MMP-9; in particular, 11 b proved to be over 100 times more active against MMP-2 than against MMP-9. MD simulations of the MMP/11 b complexes and thermodynamic integration calculations provided structural insight and relative binding energies consistent with the experimentally observed activity data. These findings demonstrate that structural differences in the S1′ pocket bottom permit an improvement in selectivity in the inhibition of MMP-2 over that of MMP-9; this is of great relevance for future structure-based drug design because MMP-2 is a validated target for cancer therapy, whereas MMP-9 plays both detrimental and protective roles in cancer. This study also supports the need to consider the dynamics of the S1′ pocket in order to achieve selectivity in the inhibition of MMPs.