Human tankyrases are attractive drug targets, especially for the treatment of cancer. We identified a set of highly potent tankyrase inhibitors based on a 2-phenyl-3,4-dihydroquinazolin-4-one scaffold. Substitutions at the para position of the scaffold′s phenyl group were evaluated as a strategy to increase potency and improve selectivity. The best compounds displayed single-digit nanomolar potencies, and profiling against several human diphtheria-toxin-like ADP-ribosyltransferases revealed that a subset of these compounds are highly selective tankyrase inhibitors. The compounds also effectively inhibit Wnt signaling in HEK293 cells. The binding mode of all inhibitors was studied by protein X-ray crystallography. This allowed us to establish a structural basis for the development of highly potent and selective tankyrase inhibitors based on the 2-phenyl-3,4-dihydroquinazolin-4-one scaffold and outline a rational approach to the modification of other inhibitor scaffolds that bind to the nicotinamide site of the catalytic domain.