It is well established that heat is associated with reduced sperm production, but the role of physiological variation in temperature has never been scrutinized in humans. We studied diurnal scrotal temperature and markers of male fertility in a population of couples planning their first pregnancy. Sixty men from a cohort of couples who were planning their first pregnancy were included and scrotal skin temperature was monitored during 3 days using a portable data recorder. Working hours and working postures were recorded daily in a questionnaire. Each man provided a fresh semen sample and the couples were followed for six menstrual cycles or until a clinical pregnancy was recognized. The median value of scrotal skin temperature was 33.3 °C in the daytime and 34.8 °C at night. In periods of sedentary work, the median temperature was on average 0.7 °C higher (SE=0.2 °C). In addition, scrotal temperature was higher in the daytime, in summer, and in leisure time compared with working hours. Median sperm concentration among men with more than 75% of their daytime readings above 35 °C was 33.4 × 106/mL, compared with 91.8 × 106/mL for men with less than half of their readings above 35 °C (difference 58.4; 95% CI: 25.9–77.8 × 106/mL). It is concluded that a sedentary position is a significant source of increased scrotal skin temperature, and even moderate and physiological elevation in scrotal skin temperature is associated with a substantially reduced sperm concentration. Sedentary work should be considered as an important potential confounder for reduced sperm count in epidemiological research.