The introduction of new treatments for esophageal cancer including surgery, chemotherapy, radiotherapy, or a combination of these modalities has not only improved patient survival, but may also increase the risk of the second primary cancers. The available evidence is conflicting with most risk estimates based on sparse numbers. Here we estimated standardized incidence ratios (SIRs) of second cancer among 24,557 esophageal cancer survivors (at least 2 months) in the Surveillance, Epidemiology, and End Results (SEER) Program between 1973 and 2007, who had been followed up for median 6.5 years (range 2 months–29.3 years). Second cancer risk was statistically significantly elevated (SIR = 1.34, 95% confidence interval [CI]= 1.25–1.42) among the survivors compared with the general population; the SIRs for cancers of oral and pharynx, stomach, small intestine, larynx, lung and bronchus, thyroid and prostate cancer were 8.64 (95% CI = 7.36–10.07), 2.87 (95% CI = 2.10–3.82), 3.80 (95% CI = 1.82–7.00), 3.19 (95% CI = 2.12–4.61), 1.68 (95% CI = 1.46–1.93), 2.50 (95% CI = 1.25–4.47), and 0.77 (95% CI = 0.65–0.90), respectively. Radiotherapy raised cancer risk of larynx (SIR = 3.98, 95% CI = 2.43–6.14) and thyroid (SIR = 3.57, 95% CI = 1.54–7.03) among all esophageal cancer survivors. For patients who had 5–9 years of follow up after radiotherapy, the SIR for lung cancer was 3.46 (95% CI = 2.41–4.82). Patients with esophageal cancer are at increased risks of second cancers of oral and pharynx, larynx, lung, and thyroid, while at a decreased risk for prostate cancer. These findings indicate that radiotherapy for esophageal cancer patients may increase risk of developing second cancers of larynx, lung, and thyroid. Thus, randomized clinical trials to address the association of radiotherapy and the risk of secondary cancer are warranted.