Results of the IX International Spectroradiometer Intercomparison and impact on precise measurements of new photovoltaic technologies

Today's variety of photovoltaic (PV) technologies imposes new challenges to laboratories and industries to precisely measure the performance of devices and, consequently, to accurately estimate the energy yield once installed in a specific location. Spectroradiometry has become a key discipline for metrology applied to PV: Spectral irradiance is one of the three parameters according to which solar simulators are classified according to IEC 60904‐9; precise spectrum measurements are a key factor in the spectral mismatch calculation. Finally, energy rating calculations according to IEC 61853 involve spectral irradiance conditions different than the AM1.5G standard spectrum. To tackle these issues, since 2011, the International Spectroradiometer Interlaboratory Comparison (ISRC) takes place annually in different locations of Europe with the participation of laboratories, research institutes, and industry partners to assess spectral measurement capabilities and share good measurement practices and protocols. In this paper, several results of the 9th ISRC 2019 are presented, looking in particular at the impact on characterization of new technologies like organic devices (OPV), dye‐sensitized (DSSC), and perovskites.


| INTRODUCTION
Spectroradiometry has become a key discipline in order to achieve precise measurements in the photovoltaic sector, from top-level calibration laboratories to industry in the production chain 1,2 ; new challenges have to be tackled, considering the increasing importance of an energy rating approach 3 Since 2011, the International Spectroradiometer Interlaboratory Comparison (ISRC) takes place in different locations of Europe with the participation of top level laboratories, research institutes and industrail partners. [4][5][6] At present, the need for a precise measurement of the spectral irradiance is due to the ample and continuously increasing variety of technologies and materials used for cutting-edge PV devices, characterized by very different spectral responsivities with respect to the reference devices used for their calibration. 7,8 This latter can be crystalline Si (with or without optical filters for reducing spectral sensitivity) used for outdoor or in indoor solar simulators or pyrheliometers and cavity radiometers for outdoor calibration. Whatever sensor is chosen, the difference in spectral responsivity between the device to be measured and the reference sensor generates a spectral mismatch, unless the measurement is performed using exactly the reference spectrum (usually AM1.5G according to IEC 60904-3), which is most unlikely the case (especially in indoor measurements). In addition to measurements at Standard Test Conditions (STC), the energy rating standard IEC 61853 foresees performance predictions in different climates, 9 with spectral irradiance characteristics very different from the AM1.5G standard. Amongst them, the alpine region is characterized by very low values of air mass (AM), with rich ultraviolet (UV) content and very high irradiance peaks due to the lower atmospheric absorption, decreased Rayleigh scattering and low humidity. 10,11 This peculiarity suggested to locate the 9th ISRC at the premises of the Astronomical Observatory of Saint-Veran  12 The present work is focused on the spectroradiometers interlaboratory comparison describing the measuring systems, the data acquisition protocol, the data analysis techniques, and presenting relevant results. We will analyze the absolute spectral irradiance differences among acquired spectra and the impact of these differences on spectral mismatch correction factor (MF), parameter directly affecting the calibration performance of any PV device.

| EXPERIMENTAL APPROACH
In this section, the experimental apparatus used to carry out the intercomparison is described, focusing only on the spectroradiometers and not on the system used for primary calibration of reference cells.
During the last 10 years, the variety of spectroradiometers available on the market has considerably increased, ranging from relatively cheap instruments to top level ones. They are different in technology (single or multistage rotating grating monochromators, CCD array based instruments for fast measurements), in measurement range (from the standard 300-1,100 nm of single crystalline silicon detector to 280-2,500 nm of dual-and triple-detector), principle of operation (e.g., filter radiometers, measuring only the spectral contents of selected wavelength bands and, then, mathematically reconstructing the spectrum up to 4,000 nm), and measurement configuration (e.g., instruments configured to measure only DNI, instruments adaptable to measure GNI, DNI, and diffuse irradiance). A list of the instruments present at the intercomparison is shown in Table 1. Throughout this paper, the participants will be labeled anonymously with letters from "Lab A" to "Lab J" in random order (not alphabetical).
All the trackers had tracking accuracy of ±0.05 . Five of the instruments present at the campaign were able to measure only DNI, three were able to measure only GNI, and three were equipped with collimated tubes to switch from GNI to DNI measurements. Due to T A B L E 1 Overview of spectroradiometers, measured quantities, and ranges (participants explicitly named in alphabetical order here)  Table 2.
Most of participants calibrate their spectroradiometers making use of standard lamps, which guarantee high accuracy, reproducibility, and stability; however, the emission spectrum of a standard halogen lamp is extremely poor in the UV region and, in general, exhibiting an integrated irradiance of about 200-300 Wm −2 in the wavelength band from 250 to 2,500 nm. 13 The standard lamp low UV content is particularly relevant when measuring natural Sun spectrum at 3,000 m altitude where, due to the lower atmospheric absorption, very UV-rich spectra and high values of irradiance are measured. Regarding measurement uncertainties, Labs A, B, C, F, G, I, and J provided detailed uncertainty calculations for each wavelength or wavelength bands, whereas the others did not provide information on their uncertainties. As a consequence, a sensible and realistic performance statistics analysis has been possible only including those participants.

| SELECTED METHODS FOR SPECTRA COMPARISON
In order to compare the synchronously (to within 5 s) acquired spectra, several methods are available. In this work, the following methods were used: • Absolute values comparison: At each wavelength, the absolute values of spectra are compared against an assigned value or a value taken as reference. In this work, due to the insufficient amount of information on measurement uncertainties, and for continuity with analysis of previous ISRC editions, the values measured by Lab A are taken as reference to calculate deviations.
• Comparison of relative spectral content in defined wavelength bands: This approach does not compare the absolute values wavelength-by-wavelength (W-W), but the content of the spectra integrated in several wavelength bands. This method is worth to be considered because in IEC 60904-9, it is used to classify solar simulators. Moreover, it is a good approach to evaluate the agreement of spectra shapes.
• Comparison of a derived parameter: Instead of comparing directly the spectra, a parameter of interest is calculated, and then, this latter is compared. In the present work, the parameter of interest is the spectral mismatch factor (MF), which affects all performance measurements of any PV device.
Amongst the three approaches, the first is the most analytical;

| MEASUREMENT UNCERTAINTIES
Each participant was asked to provide its measurement uncertainties for spectral irradiance. Estimation of measurement accreditation scheme but nevertheless is important for every laboratory, in order to assess the reliability of its measurements. The detailed uncertainty budget was not asked, and it is not the purpose of this work to go into details of estimation of measurement uncertainty of spectral irradiance. 14 Hereafter (Table 3), a summary of the given measurement uncertainties is presented, having grouped in wavelength bands the uncertainties for Labs B, C, and F, because the former provided uncertainty values for each wavelength, and the latter in bands of 20-50 nm each. The plot in Figure 1 shows the detailed information provided by participants.
For performance statistics analysis, the detailed information will be used.

| Direct normal irradiance
In this section, an analysis of direct normal irradiance (DNI) spectra is presented. Each participant followed its own measurement timing and protocol, but in general, everyone was able to measure spectra every minute or few minutes. However, not all participants were able to measure throughout the entire day: Several participants were able to measure only in several time slots, due to difficulties in reaching the observatory (special transport means allowed only few passengers per time). To select spectra set suitable for comparison, the following filtering criteria have been applied: (1) check of sky stability condition as described above; (2) for each slot with stable sky, the number of available spectra per participant has been calculated; and (3)  Lab E has better agreement after 700 nm; Lab G is generally within where x is the measured value by a participant,x is the reference value, and U C (.) is the expanded uncertainties (k = 2 in case of Gaussian distribution); this equation assumes the statistical independence between x andx. The E N numbers are calculated for all wavelengths in the common range 350-1,025 nm and are shown in Figure 5.
Having chosen E N numbers as key parameter for performance assessment, the following rule has to be applied (as indicated in the ISO/IEC 17043): • E N ≤ 1: satisfactory agreement The classification in satisfactory or unsatisfactory depends not only on the difference between the two measured values of the same quantity but on the difference evaluated with respect to the declared uncertainties; it is possible therefore that a participant distant from the reference obtains satisfactory results while another participant closer to the reference obtains unsatisfactory E N scores, if the latter uncertainties are too small respect to the difference. Considering this aspect, performance statistics is a useful tool for each laboratory to assess its own calculated uncertainties. In the specific case, the agreement is unsatisfactory in the UV (which is a common issue) and after 700 nm. Throughout all the analyses, the reference for EN numbers calculation is Lab A, which has been used as reference in all the previous intercomparison campaigns. The consistency between Lab A and the average of spectra of all participants is proofed in Figure 6, showing the EN numbers between Lab A and the average (taken as reference with its calculated uncertainty) of the spectra acquired in the second time frame (having more available data).
The second time frame considered above was on day June 27th between 10:45-11:10 (UTC + 1) with Labs A, B, C, D, E, F, and H measuring DNI. The AM in this frame varied between 1.07 and 1.09.
The selection of spectra useful for comparison follows the same methodology already described; the suitable selection of data is shown in Figure 7. 1,100 nm, which might be due to the merging algorithm between the two sub-spectra portions acquired by the detectors of the instrument.
Last, Lab D measurement is dominated by noise after 1,000 nm. These   performed. Looking at Figure 9, it is evident that the mutual uncertainties between Lab A and Lab B are underestimated, being Lab B completely above the "good agreement" limit of E N = 1; regarding Lab F, the agreement with the reference is good except for some outliers in the NIR part of the spectrum. A hypothesis for this behavior can be due to dark measurements procedures and uncertainties, which play an important role in outdoor measurement campaigns where sensors internal to the instruments are continuously exposed to sunlight, and also, the instrument body is not kept at room temperature like in indoor operations. Therefore, the two Labs might have used different routines for dark measurements, whereas the mutual uncertainties seem to be quite well in agreement in this case. For Lab C, the agreement is generally good except for outliers in the NIR and UV regions.  Figure 10, the spectra selected for comparison are shown, for a total of 89 spectra, together with the averages per participant. Once again, the irradiance stability measured by the reference pyrheliometer is confirmed by the acquired spectra, which are very close to each other; however, looking at the bottom right plot in Figure 10, Lab I peak point around 1.4 W m −2 nm −1 seems too low. All cosine diffusers connected to optical fibers were mounted onto the same tracker, equipped with a mounting structure for the alignment, so it is improbable this difference to be due to misalignment; one of the possibilities could be related to the spectroradiometer calibration.

| Global normal spectral irradiance analysis
Looking at the W-to-W comparison of the average spectra shown in Figure 11, Lab D appears well in-line with Lab A (being the difference within ±10% throughout all the range except for UV before 350 nm and NIR near 1,100 nm), whereas Lab I measured around 20%-30% lower. This considerable difference however might not be dramatic in terms of bands (spectrum shape matters) and consequently also its impact on spectral mismatch factor (MF) calculation. Lab I has previously reported that the in-house calibration was performed with a deuterium lamp with primary calibration (traceable NPL) and an halogen lamp with unreliable calibration, provided by the lamp manufacturer. Lab I has later on performed interlaboratory comparisons and found a systematic deviation in the VIS/NIR range, which is consistent with the deviations observed in this section.
In terms of E N numbers, only Lab I provided measurement uncertainties, and therefore, the following graph shows only the performance statistics between the reference (Lab A) and Lab I. However, excluding Lab D, this operation can be extended up to 1,700 nm.
From the result shown in Figure 12, it is evident that are in agreement in 400-1,100 nm, the measurement is affected by a systematic deviation arising from the biased in-house calibration.

| Global horizontal spectral irradiance
For this analysis, the best time slot was during day 27th from 11:55 to 13:30 (UTC + 1). Figure 13 shows all the acquired spectra in this time frame, for a total of 196.
The graph of the average spectra ( Figure 13) evidences a slight change in shape near 500 nm, where Lab A crosses Lab J spectrum; this effect is better appreciated in the point-by-point analysis below ( Figure 14).
The agreement between the spectra is generally within 5% excluding the portion below 400 nm and around 1,400 nm due to the very low values there. Compared with previous absolute values comparisons, this agreement can be considered good. To assess the agreement of the two average spectra with the two mutual uncertainties, the E N numbers have been calculated; they are shown in Figure 15; it is evident that the uncertainties have been correctly estimated in the 300-400 nm part, whereas near 700 nm, they have been underestimated. The E N < −1 after 1,600 nm are attributable mainly to noise.

| Spectral contents in bands
For most of the calibration involving a PV device, only the knowledge of the spectrum shape of the light impinging on the reference and on the device under calibration is needed. One notable exception is the calibration of reference cell using the integrated spectral irradiance method. Spectroradiometers in photovoltaics are mainly used for classifying light sources (normally solar simulators) according to IEC 60904-9 or calculating the spectral mismatch factor, which will be described in the next section. In this section, the relative spectral content of the spectra in several defined bands of interest is analyzed: Two different sets of bands have been chosen for the analysis: The first is defined in the present IEC 60904-9 Ed.2 (see Table 4) and the second in the future IEC 60904-9 Ed.3, which at this moment is under approval process at the International Electrochemical Committee.
Both of them are (will be) used to classify solar simulators accordingly to their match with the AM1.5G standard spectrum defined in IEC 60904-3, but the second one will extend the range from 400-1,100 nm to 300-1,200 nm; this last is a more appropriate range for the calibration of recent PV device technologies, exhibiting a wider range of spectral responsivities.
The principle used to define the new bands is the energyequivalent content of all bands respect to the AM1.5G spectrum (each band contains about 16.6% of the spectral content of the 300-1,200 nm region); the challenge for the spectroradiometry community working in PV concerns the measurement range: Not all the spectroradiometers participating to the ISRC were able to measure on the entire 300-2,500 nm range. 16 The relative spectral content is calculated by integration using with the integration extrema (a i , b i ) specified in the above table, and E (λ) the measured spectrum. From Equation 2, it can be seen that multiplying the spectrum by a constant factor does not influence the relative spectral content, and therefore the correctness of the spectrum in absolute terms is not necessary, where effects distorting the shape if the spectrum are taken into account. The measurement uncertainties used previously (Table 3) cannot be directly transferred to the new quantities E bi ai , because of the effects due to integration. Several works have been done to study the error propagation between measured quantities and functionals having them as variables; apart for very basic cases, an analytical expression of the derived uncertainty is not possible, and numerical methods involving Monte Carlo simulations are more and more frequently used.

| Direct normal spectral irradiance
The same spectra sets used for the point-by-point absolute analysis will be analyzed in this section to assess the congruency between participants in terms of relative spectral content. Therefore, 472 spectra will be used in this section. For analysis in bands, the average spectrum for each participant in each time frame is considered. Figure 16 shows the calculated relative contents E bi a i for both the two bands sets specified in the two versions of IEC 60904-9. Here, Lab A is not acting as reference, and results of the other participants are not evaluated against it. In Figure 16, each group of columns represents the content for a specific band; numerically, above each group of columns, the average and max-min deviation of the corresponding values are indicated. In this case, given the very low number of points, the max-min deviation has been preferred to the standard deviation. Not all the participants comparing in the W-to-W analysis (see Figures 3 and 7) could be considered here: Looking at Table 5, only those participants complying with the different range requirements are considered here. Looking at the results, the deviations for each band are all within ±1%, although some of the spectra had considerable differences W-to-W, in the order of ±10% or even more. This uncertainty reduction is partly due to the effect of integration, which smooths errors due to spikes, and partly to the normalization (denominator of Equation 2). The spectral content looks very different using Ed.2 bands respect to Ed.3 bands: In the former, the energy content decreases going to the right part, were thought to have a constant contribution for the AM1.5G spectrum; here, the spectra were DNI close to AM 1.1). The agreements shown in Figure 16 look quite comfortable in view of using these instruments on solar simulators, but only a few of them are able to fully cover the 300-1,200 nm range (typically, this problem concerns single-detector instruments). In particular, amongst the three participants on day 26th measuring DNI, the comparison is possible only using bands of IEC 60904-9 Ed.2, whereas only Lab A is able to cover the Ed.3 range, and therefore this comparison is absent from

| Global normal and global horizontal spectral irradiance
The same approach is followed to analyze the relative spectral content of GNI spectra acquired on day 27th (Figures 10 and 13).
where S m (λ) is the measured spectral responsivity of the device under test (DUT),Ŝ λ ð Þ the spectral responsivity of the reference device, G m (λ) the measured spectral irradiance used to perform the IV characterization, andĜ λ ð Þ the reference spectral irradiance (here the AM1.5G). It is worth noting that the M F as defined in Equation 3 is the inverse of that defined in IEC 60904-7 for practical reasons: As defined here, M F > 1 means the spectral correction will increase the measured performance, M F < 1 the opposite. In this work the measured spectra already considered in previous analysis are used to calculate mismatch factors using as reference a crystalline silicon reference cell and several devices of different technologies, listed in Table 6 and whose spectral responsivities have been provided by ESTI laboratory. 17,18 For confidentiality, the various manufacturers are not indicated and all devices remain anonymous. Although during the ISRC several reference cells have been measured against cavity radiometers, the choice of using here a crystalline silicon cell instead of a pyrheliometer is due to the measurement range needed to calculate the mismatch factor, which is 300-1,200 nm and not 300-4,000 nm. If a cavity radiometer is used as reference, the measured spectra are normally artificially extended up to 4,000 nm using the corresponding AM1.5D spectrum, but this would add much more complexity without adding more information.
The spectral responsivities of all the devices used in this analysis are shown in Figure 19: Instead of reporting the normalized curves, here, the absolute values are showed, in order to appreciate not only the different wavelength regions but also qualitatively the different photocurrents. However, looking at all the subplots of Figure 20, there is a clear tendency of participants to be positioned in the same order respect to the others for all technologies, with Lab F tendentially higher than the other three.

| Spectral corrections from GNI spectra
The same approach has been followed to compare the impacts on spectral mismatch factor of GNI spectra, measured on day 27th. Figure 21 is the analog of Figure 20 for GNI spectra. In this case, the three participants are not always in the same relative position on the graphs; in the case of c-Si, the larger deviation observed  Intercomparison. This annual measurement campaign allows participants to share measurement procedures, good practices, and assess their capability of measuring spectral irradiance. Ten participants submitted their measured spectra (DNI, GNI, and GHI), each one accordingly to its measurement protocols and data analysis. Several time frames during two different days were selected as more suitable in terms of irradiance stability and number of acquired spectra for intercomparison analysis. Results show a very good agreement between participants in terms of relative spectral content in