This paper describes the measurement of photovoltaic module performance over a range of temperatures and irradiances according to the international standard IEC 61853 Part 1. The purpose of this work is to assess the reproducibility of power matrix measurements obtained using two methods specified in the standard: under natural sunlight with a tracker, and with a solar simulator. A comparison of results using the third principal method (under natural sunlight without tracker) is also summarised for completeness. The same measurement techniques have been employed to measure four modules of different technologies, namely mono and poly crystalline Si, CdTe and CIS. The method used to vary the irradiance in the natural sunlight with tracker and solar simulator approaches is based on un-calibrated mesh filters. The uniformity and effect on spectrum of the mesh filters have been studied, and the impact of these on the measurements estimated. Measurements from all methods are compared over as much of the ranges as possible. The results show that for all modules, the reproducibility is within the estimated measurement uncertainty. The suitability of the different methods is discussed in light of the results and the limitations of the various methods as applied to different modules technologies. On the basis of the results, parts of IEC 61853 Part 1 will be introduced into the ISO 17025 laboratory accreditation at the European Solar Test Installation (ESTI). Copyright © 2013 John Wiley & Sons, Ltd.