An accurate and efficient algorithm is proposed for solving two-dimensional electromagnetic scattering problems in a layered medium. As a natural extension of the previously developed fast inhomogeneous plane wave algorithm (FIPWA), this approach has several inherent merits, such as being simple, versatile, and error controllable. The basic idea is first to express the spatial Green's function, encountered in layered medium studies, as Sommerfeld-type integrals. By observing the similarity of these integrals with the integral representation of the free space Green's function, FIPWA can be applied with ease to accelerate the matrix-vector multiplication. The multilevel scheme has been implemented, and the computational complexity of O(N log N) is achieved. It is noted that the proposed approach is more accurate and more efficient than the existing fast multipole method coupled with the discrete complex image method.