Risk factors for esophageal squamous cell carcinoma (ESCC) and esophageal adenocarcinoma (EAC) are well defined, while the role of diet in these conditions remains controversial. To help elucidate the role of particular dietary components, major bibliographic databases were searched for published studies (1990–2011) on associations between esophageal cancer risk (EC) and consumption of various types of meat and fish. Random-effects models and dose-response meta-analyses were used to pool study results. Subgroup analyses were conducted by histological subtype, study design, and nationality. Four cohorts and 31 case-control studies were identified. The overall pooled relative risk (RR) of EC and the confidence intervals (CIs) for the groups with the highest versus the lowest levels of intake were as follows: 0.99 (95% CI: 0.85–1.15) for total meat; 1.40 (95%CI: 1.09–1.81) for red meat; 1.41 (95%CI: 1.13–1.76) for processed meat; 0.87 (95%CI: 0.60–1.24) for poultry; and 0.80 (95%CI: 0.64–1.00) for fish. People with the highest levels of red meat intake had a significantly increased risk of ESCC. Processed meat intake was associated with increased risk of EAC. These results suggest that low levels of red and processed meat consumption and higher levels of fish intake might reduce EC risk.