This study aims to evaluate the clinical utility of image-guided Raman endoscopy for in vivo diagnosis of neoplastic lesions in the stomach at gastroscopy. A rapid-acquisition image-guided Raman endoscopy system with 785-nm excitation has been developed to acquire in vivo gastric tissue Raman spectra within 0.5 sec during clinical gastroscopic examinations. A total of 1,063 in vivo Raman spectra were acquired from 238 tissue sites of 67 gastric patients, in which 934 Raman spectra were from normal tissue whereas 129 Raman spectra were from neoplastic gastric tissue. The swarm intelligence-based algorithm (i.e., ant colony optimization (ACO) integrated with linear discriminant analysis (LDA)) was developed for spectral variables selection to identify the biochemical important Raman bands for differentiation between normal and neoplastic gastric tissue. The ACO-LDA algorithms together with the leave-one tissue site-out, cross validation method identified seven diagnostically important Raman bands in the regions of 850–875, 1,090–1,110, 1,120–1,130, 1,170–1,190, 1,320–1,340, 1,655–1,665 and 1,730–1,745 cm−1 related to proteins, nucleic acids and lipids of tissue and provided a diagnostic sensitivity of 94.6% and specificity of 94.6% for distinction of gastric neoplasia. The predictive sensitivity of 89.3% and specificity of 97.8% were also achieved for an independent test validation dataset (20% of total dataset). This work demonstrates for the first time that the real-time image-guided Raman endoscopy associated with ACO-LDA diagnostic algorithms has potential for the noninvasive, in vivo diagnosis and detection of gastric neoplasia during clinical gastroscopy.