How the antibodies of individual convalescent human sera bind to each amino acid residue at the antigenic sites of hemagglutinin (HA) of influenza viruses, and how the antigenic drift strains of influenza viruses are selected by human sera, is not well understood. In our previous study, it was found by a binding assay with a chimeric HA between A/Kamata/14/91 (Ka/91) and A/Aichi/2/68 that convalescent human sera, following Ka/91 like (H3N2) virus infection, bind to antigenic site A of Ka/91 HA. Here using chimeric HAs possessing single amino acid substitutions at site A, it was determined how those human sera recognize each amino acid residue at antigenic site A. It was found that the capacity of human sera to recognize amino acid substitutions at site A differs from one person to another and that some amino acid substitutions result in all convalescent human sera losing their binding capacity. Among these amino acid substitutions, certain ones might be selected by chance, thus creating successive antigenic drift. Phylogenetic analysis of the drift strains of Ka/91 showed amino acid substitutions at positions 133, 135 and 145 were on the main stream of the phylogenetic tree. Indeed, all of the investigated convalescent sera failed to recognize one of them.