In order to efficiently analyze an electromagnetic scattering problem with the surface integral equation approach, the matrix decomposition algorithm (MDA) is used to accelerate the matrix-vector multiplication when the corresponding matrix equation is solved by a Krylov-subspace iterative method. Although the MDA is more efficient than the direct solution, this paper presents a novel recompression technique to further reduce computation time and storage memory. The technique applies singular value decomposition (SVD) to the matrices of MDA. Using the novel recompression technique, a sparser representation of the impedance matrix is obtained, and a more efficient matrix-vector multiplication is implemented. The modified MDA is comparable with the multilevel matrix decomposition algorithm (MLMDA) and the matrix decomposition algorithm-singular value decomposition (MDA-SVD) in terms of computation time and memory requirement. Remarkably, the new formulation can reduce the computational time and memory significantly, with excellent accuracy.