Two studies are reported that examine whether children encode category-neutral information about target persons with respect to gender and race categories. In Study 1, using a semi-naturalistic adaptation of the ‘who said what?’ technique (Taylor, Fiske, Etcoff, & Ruderman, 1978), children of 5, 8 and 11 years were asked to recall peers' choices in a preference task. For all three age groups, significantly more within-sex than between-sex confusions were found, indicating that children had encoded neutral information about targets with respect to their sex. In Study 2, 5-, 8- and 11-year-old children were presented with a conventional ‘who said what?’ task in which they were shown four photographs, two of black children and two of white children, along with 16 statements attributed to the target children (i.e. four statements to each child). Following this, 16 statement cards had to be assigned to photographs of the four target persons to indicate ‘who said what’. Across all age groups there were significantly more within- than between-race confusions. The evidence from these two studies indicates that even for category-neutral information, gender and race play an important role in children's initial encoding of others' behaviour.