Chemoradiotherapy (CRT) for esophageal cancer is disadvantageous because of a high locoregional failure rate. Detecting early small recurrent cancers at the primary site is necessary for potential salvage treatment. However, most endoscopists are inexperienced and therefore, a role for surveillance endoscopy after complete remission (CR) has not been established. We retrospectively evaluated serial surveillance endoscopic images from patients eventually proved to have primary-site recurrence in order to identify useful endoscopic features for early diagnosis. From January 2000 to December 2004, 303 patients with esophageal squamous cell carcinoma underwent definitive CRT, and 133 of them achieved CR. The surveillance endoscopic images stored at intervals of 1–3 months for the 16 patients with recurrence only at the primary tumor site and the 61 patients with no recurrence were collected for reexamination. Among 133 patients who achieved CR, 16 (12%) developed only local recurrence at the primary site. Thirteen of the 16 primary-site recurrent tumors (81%) appeared as submucosal tumors (SMT), with the remaining appearing as erosions or mild strictures. Of biopsy-proven recurrences, 81% were preceded by newly developed lesions such as SMT, erosions, or mild strictures detected by earlier surveillance endoscopies. For all 77 patients achieving CR with no metastasis, 86% of the evolving SMT with negative biopsies were eventually confirmed as cancer at later endoscopies. Thirteen of the 21 evolving lesions were subsequently confirmed as recurrent cancer. Early primary-site recurrence of esophageal cancer after a complete response to CRT is detectable with frequent endoscopic surveillance. SMT appearance is a useful endoscopic sign of early recurrence, as well as a predictor of subsequent diagnosis of recurrence.