This article aims at evaluating indoor noise levels at home and investigating the factors that may influence their variability. An 8-day noise measurement campaign was conducted in the homes of 44 schoolchildren attending the public primary schools of Besançon (France). The presence of the inhabitants in the dwelling and the noisy events occurring indoors and outdoors was daily collected using a time–location–activity diary (TLAD); 902 time periods were analyzed. The indoor noise level increased significantly with the outdoor noise level, along with the duration of the presence or level of activity of the inhabitants at home. However, this effect may vary according to the period of day and the day of the week. Moreover, a significant part of the day and evening indoor noise level variability was explained when considering the TLAD variables: 46% and 45% in the bedroom, 54% and 39% in the main room, respectively. Our results highlight the complexity of the indoor environment in the dwellings of children living in an urban area. Combining the inhabitant presence and indoor noise source descriptors with outdoor noise levels and other dwelling or inhabitant characteristics could improve large-scale epidemiological studies. However, additional efforts are still needed, particularly during the night period.