Visceral fat is a risk factor for non-alcoholic fatty liver disease (NAFLD). A reduction in sex hormones is associated with increased abdominal fat. Thus, we investigated whether reduced testosterone (T) or oestradiol (E2) levels in men are associated with NAFLD and central obesity. The study involved a survey of 1,882 men between 20 and 60 years of age. We detected hepatic fat infiltration by ultrasound. Early morning serum was analyzed for total testosterone (TT), E2, sex hormone-binding globulin (SHBG), follicle-stimulating hormone (FSH) and luteinizing hormone (LH). Free testosterone (FT) was calculated using the Vermeulen method. In the studied population, the prevalence of NAFLD, FSH, LH and SHBG increased with age, TT and FT declined with age, and E2 remained stable. However, in the NAFLD group, TT remained stable, FT and E2 declined, and hepatic fat infiltration increased (P < 0.001 for both). Using multivariate analysis, a correlation was found between E2 and NAFLD, with an odds ratio of 0.954 (95% confidence interval: 0.946–0.967). E2 is one of the protective factors against NAFLD in healthy men. T has no significant correlation with NAFLD. Further investigation would be required to assess the clinical consequences of reduced E2 in men with NAFLD, particularly for men whose TT remained stable.