Retrieval of temperature, moisture profiles and surface skin temperature from hyperspectral infrared (IR) radiances requires spectral information about the surface emissivity. Using constant or inaccurate surface emissivities typically results in large temperature and moisture profile errors, particularly over semi-arid or arid areas where the variation in emissivity is large both spectrally and spatially. A physically based algorithm has been developed to retrieve a hyperspectral IR emissivity spectrum simultaneously with the temperature and moisture profiles, as well as the surface skin temperature. To make the solution stable and efficient, the hyperspectral emissivity spectrum is represented by eigenvectors, derived from the laboratory measured hyperspectral emissivity database, in the retrieval process. Experience with Atmospheric InfraRed Sounder (AIRS) radiances shows that simultaneous retrieval of the emissivity spectrum and the sounding improves the surface skin temperature and temperature and moisture profiles, particularly in the near surface layer.