One method of analyzing radiation and propagation above a surface is to employ an effective surface impedance to describe the effect of the boundary. In this paper an expression is derived for the effective impedance at grazing incidence of a slightly rough interface between air and a finitely conducting medium for which the Leontovich boundary condition is applicable. The perturbation technique of Rayleigh and Rice is employed, and attention is restricted to vertical polarization. The resulting effective surface impedance consists of two terms, the impedance of the lower medium when the surface is perfectly smooth and a term accounting for roughness. The latter term can be complex in general and depends on the strengths of the roughness spectral components present. The result is applicable to either deterministic periodic surfaces or random rough surfaces. Various alternate definitions of the effective surface impedance are examined and are seen to be equivalent. The analysis of power flow at the surface permits the interpretation of the interaction process in terms of scattered fields.