Diabetes mellitus (DM) has been reported to be associated with an increased risk of site-specific cancers; however, few studies have assessed associations of DM with both total and site-specific cancers in Japan. We examined the association of a history of DM with cancer incidence in a population-based prospective cohort study in Japan. A total of 14 173 men and 16 547 women over 35 years old, who completed a self-administered baseline questionnaire in 1992, were followed up for cancer incidence from September 1992 to March 2008. At baseline, 6.3% men and 2.9% women had a history of diabetes. A total of 1974 men and 1514 women were identified as newly diagnosed with cancer. Hazard ratios (HR) and 95% confidence intervals (CI) were determined using Cox proportional hazards models. After controlling for potential confounders, men with DM had a modest risk increase of total cancer occurrence compared with those without DM (HR, 1.09; 95% CI, 0.93–1.29). Increased risk of cancer of the liver (HR, 2.18; 95% CI, 1.27–3.74), bile duct (HR, 2.17; 95% CI, 1.01–4.66), and larynx (HR, 3.61; 95% CI, 1.16–11.2) in diabetic men were observed. In women, significant increased risk of total cancer (HR, 1.35; 95% CI, 1.06–1.73) and stomach cancer (HR, 2.15; 95% CI, 1.30–3.54) were observed among diabetic subjects. These data suggest that people with DM may be at increased risk of both total and some site-specific cancers.