The implementation details of a fast direct solver is described herein for solving dense matrix equations from the application of surface integral equation methods for electromagnetic field scatterings from non-penetrable targets. The proposed algorithm exploits the smoothness of the far field and computes a low rank decomposition of the off-diagonal coupling blocks of the matrices through a set of skeletonization processes. Moreover, an artificial surface (the Huygens' surface) is introduced for each clustering group to efficiently account for the couplings between well-separated groups. Furthermore, a recursive multilevel version of the algorithm is presented. Although asymptotically the algorithm would not alter the bleak outlook of the complexity of the worst case scenario,O(N3) for required CPU time where N denotes the number of unknowns, for electrically large electromagnetic (EM) problems; through numerical examples, we found that the proposed multilevel direct solver can scale as good as O(N1.3) in memory consumption and O(N1.8) in CPU time for moderate-sized EM problems. Note that our conclusions are drawn based on a few sample examples that we have conducted and should not be taken as a true complexity analysis for general electrodynamic applications. However, for the fixed frequency (h-refinement) scenario, where the discretization size decreases, the computational complexities observed agree well with the theoretical predictions. Namely, the algorithm exhibits O(N) and O(N1.5) complexities for memory consumption and CPU time, respectively.