hERG blockade is one of the major toxicological problems in lead structure optimization. Reliable ligand-based in silico models for predicting hERG blockade therefore have considerable potential for saving time and money, as patch-clamp measurements are very expensive and no crystal structures of the hERG-encoded channel are available. Herein we present a predictive QSAR model for hERG blockade that differentiates between specific and nonspecific binding. Specific binders are identified by preliminary pharmacophore scanning. In addition to descriptor-based models for the compounds selected as hitting one of two different pharmacophores, we also use a model for nonspecific binding that reproduces blocking properties of molecules that do not fit either of the two pharmacophores. PLS and SVR models based on interpretable quantum mechanically derived descriptors on a literature dataset of 113 molecules reach overall R2 values between 0.60 and 0.70 for independent validation sets and R2 values between 0.39 and 0.76 after partitioning according to the pharmacophore search for the test sets. Our findings suggest that hERG blockade may occur through different types of binding, so that several different models may be necessary to assess hERG toxicity.