During the last few decades, many studies have been performed on the early detection of cancer using noninvasive or minimally invasive techniques in lieu of traditional excisional biopsy. Early detection can make an immense difference because cancer treatment is often simpler and more effective when diagnosed at an early stage. Cancer detecting methods may help physicians to diagnose cancer, to dissect the malignant region with a safe margin, and to evaluate the tumor bed after resection. In this paper, the advanced hyperspectral imaging system has been assessed using infrared wavelengths region for tumor detection. We were able to identify an appropriate wavelength region for cancer detection, spatially resolved images, and highlight the differences in reflectance properties of cancerous versus non-cancerous tissues. The capability of this instrument was demonstrated by observing gastric tumors in 10 human subjects. The spectral signatures were extracted and evaluated in cancerous and non-cancerous tissues. Processing means with the standard deviation of the spectral diagram, support vector machine, and first derivatives and integral of in spectral diagram were proposed to enhance and detect the cancerous regions. The first derivatives in spectral region between 1226–1251 nm and 1288–1370 nm were proposed as criteria that successfully distinguish between non-cancerous and cancerous tissue. The results of this study will lead to advances in the optical diagnosis of cancer. (Cancer Sci 2011; 102: 852–857)