This paper proposes a novel islanding fault detection method for distributed generation systems. Islanding fault detection of distributed generation systems in microgrids and smart grids is an essential security technology. The existing methods for islanding fault detection can be classified as passive methods (such as under/over voltage detection, under/over frequency detection, and voltage phase jump detection, based on natural effects of islanding) and active methods (impedance measurement, slip mode frequency shift, Sandia frequency shift and so on). Once the power consumed by a local load matches the power generated by a local inverter, the islanding phenomenon will be not obvious, and traditional passive methods may fail. In contrast, the active methods can overcome this disadvantage. However, active methods create extra disturbances that may degrade the quality of power generated by the inverter. By reducing the harmonics of the inverter output, coupled with the use of a novel filtering method, the proposed approach can be achieved without additional disturbance and does no nondetection zone islanding detection. In this paper, current harmonic compensation is applied to inverter control to minimize the inverter output harmonics, highlighting on the grid harmonics. Set-membership filter is developed to estimate the voltage harmonics. Islanding condition is detected by the changes of the specific order harmonics. The effectiveness of the proposed method is demonstrated by simulations. A Kalman filter contrast experiment is presented to confirm that the set-membership filter can more effectively detect islanding. Copyright © 2013 John Wiley & Sons, Ltd.