Cortical function and related cognitive, language, and communication skills are genetically influenced. The auditory brainstem response to speech is linked to language skill, reading ability, cognitive skills, and speech-in-noise perception; however, the impact of shared genetic and environmental factors on the response has not been investigated. We assessed auditory brainstem responses to speech presented in quiet and background noise from (1) 23 pairs of same sex, same learning diagnosis siblings (Siblings), (2) 23 unrelated children matched on age, sex, IQ, and reading ability to one of the siblings (Reading-Matched), and (3) 22 pairs of unrelated children matched on age and sex but not on reading ability to the same sibling (Age/Sex-Matched). By quantifying response similarity as the intersubject response-to-response correlation for sibling pairs, reading-matched pairs, and age- and sex-matched pairs, we found that siblings had more similar responses than age- and sex-matched pairs and reading-matched pairs. Similarity of responses between siblings was as high as the similarity of responses collected from an individual over the course of the recording session. Responses from unrelated children matched on reading were more similar than responses from unrelated children matched only on age and sex, supporting previous data linking variations in auditory brainstem activity with variations in reading ability. These results suggest that auditory brainstem function can be influenced by siblingship and auditory-based communication skills such as reading, motivating the use of speech-evoked auditory brainstem responses for assessing risk of reading and communication impairments in family members.