Extensive evaluation and classification of low‐cost dust sensors in laboratory using a newly developed test method

Abstract An extensive evaluation of low‐cost dust sensors was performed using an exponentially decaying particle concentration. A total of 264 sensors including 27 sensors with light‐emitting diodes (LEDs) and 237 sensors with laser lighting sources were tested. Those tested sensors were classified into 4 groups based on the deviation from the reference data obtained by a reference instrument. The response linearities of all the tested samples for PM1, PM2.5, and PM10 were in excellent agreement with the reference instrument, except a few samples. For the measurements of PM1 and PM2.5, the lighting source, that is, LED or laser, did not show any significant difference in overall sensor performance. However, LED‐based sensors did not perform well for PM10 measurements. The 32, 24, and 16% of all the tested sensors for PM1, PM2.5, and PM10 measurement, respectively, are in the category of Class 1 (reference instrument reading ± 20%) requirement. The performance of the low‐cost dust sensors for PM10 measurement was relatively less satisfactory.

have been conducted, and there is still lack of standard calibration and evaluation methods for the sensors.
Several researchers adopted their own calibration and evaluation procedures to test the low-cost sensors including co-location measurements in ambient environment and lab-scale measurements using standard test particles or ambient airborne particles. 15,18,[20][21][22][23][24][25] However, the accuracy of measurement data has always been questioned by many researchers and scientists, 15,26,27 and the limited number of sensors have been assessed due to the time-consuming evaluation process, which might not be able to represent the overall performance of hundreds of low-cost sensors in the commercial market. Besides, the lack of standardized methods for performance evaluation of low-cost sensors results in difficulties in intercomparisons of the sensors evaluated in different studies. 28 Therefore, a standard test procedure should be required for these overflowing low-cost sensors.
In the year 2015, there was a big dispute in Korea about the accuracy of low-cost dust sensors. Since then many scientists and industrial people have been collaborating to establish a standardized method in testing a dust sensor. As a result, the Korean industrial standard "SPS-C KACA 0027-7269:2018" for a low-cost dust sensor test was introduced in May 30, 2018. Based on this standardized procedure, we tested the low-cost dust sensor extensively from June to December 2018, and we classified the sensors into 4 groups based on the accuracy of the sensor measurements by comparing a reference instrument. In this study, 264 sensors with three different types according to their deployment conditions were tested, that is, a "module type" sensor that is half-finished and cannot be used as it is a "standalone type" sensor that is ready to use and an "embedded type" sensor that is installed inside an air purifier or an air conditioner. Based on this classification, 53 module type, 126 standalone type, and 85 embedded type sensors, that is, total 264 sensors from 23 manufactures were systematically evaluated.
Since the low-cost dust sensors are operated by detecting a scattered light coming from a particle, we checked the sensor's lighting source effect on its performance. The 27 and 237 sensors out of the total 264 sensors have a light-emitting diode (LED) and a laser as their lighting source, respectively. It should be noted that this paper systematically addressed the performance of hundreds of sensors according to their PM concentration measurements at a glance. Our findings from the results based on the standardized sensor evaluation are expected to provide important insights and valuable guidelines to effectively select and develop low-cost sensors.

| TE S T ME THOD
Basically, we adopted an exponentially decaying particle concentration for the evaluation of low-cost dust sensors. The advantage of this concentration condition is that the testing time is relatively short, mostly taking less than 10 minutes, compared with other test methods using a constant particle concentration, which usually takes more than an hour. 29 Therefore, in this study, two types of test systems using the exponentially decaying particle concentration were applied. The first one is a chamber system as shown in Figure 1A. This method is very useful when testing an air purifier with an embedded type of a low-cost dust sensor. Due to the purifying process of the air purifier removing particles inside the test chamber, the particle concentration in the chamber will decrease exponentially as a function of time. The chamber dimension in Figure 1A is 5 × 5 × 2 m 3 . For testing a dust sensor in an air purifier using the chamber system shown in Figure 1A, the introduced particles in the chamber are well mixed by the air coming from the air purifier and if necessary, an auxiliary agitating fan can be used to ensure the uniform particle concentration in the chamber. In this case, one must be careful about the airflow velocity near the dust sensor. If the flow speed is high near the sensor then the dust sensor may not function properly. 30 The second test system is to employ a low air-speed duct with an exponentially decaying particle concentration as shown in Figure 1B. This system performs well for the test of a low-cost dust sensor with a small size and high sensitivity to an airflow around it. The details in the low-speed duct test method are well described in Kang et al 30 Briefly, generated particles by an atomizer are introduced to a particle mixing chamber until the particle concentration in the chamber reaches a certain level, and then the path between the aerosol generator and the mixing chamber is disconnected. After this step, only the clean air is supplied to the particle mixing chamber, and the particles with an exponentially decaying concentration are introduced to the test duct system. We used a flow straightener with a typical honeycomb structure before the test section to obtain the flow uniformity. The uniformity of flow velocity and particle concentration in the test section was confirmed before each test.
For test particles, 5 wt% potassium chloride (KCl) solution was used to generate KCl particles by an atomizer for both test systems.
The exemplary size distributions of the generated KCl particles as

Practical Implications
• There are lots of low-cost dust sensors available in the market or used by many researchers and citizen scientists for air quality evaluations; however, the reliability of these sensors has been questioned due to the absence of a standardized calibration method. The size distributions were measured by a Grimm 1.209 dust monitor (Model 1.209, Grimm Aerosol Technik Company), and this was used as a reference instrument in this study. The number mean diameter was 0.44 ± 0.02 μm, and the geometrical standard deviation of 1.62 ± 0.05 was maintained during the entire testing periods for 6 months, that is, from June to December, 2018. Tested low-cost sensors measured particle concentrations, and the data were compared to the data simultaneously obtained by the reference instrument.
The sensor performance was classified into 4 groups based on the deviation from the reference data as shown in Figure 3. For the evaluation of the sensors, we focused on their measurement accuracy and response characteristic. The accuracy of the sensors was estimated by the absolute concentration data at the lowest and highest concentrations, and the response characteristics to the rapidly changing concentration were assessed by the ratio of the slope obtained by the sensors to the one from the reference instrument.
F I G U R E 1 Schematics of low-cost dust sensor evaluation systems: (A) chamber test; (B) low-speed duct test F I G U R E 2 Potassium chloride particle size distributions as a function of elapsed time with an exponentially decaying particle concentration. The time interval is 4 min F I G U R E 3 Classification and acceptance limits of low-cost sensor performance The Class 1 acceptance limit is represented by a red solid line parallelogram in a semi-log graph. This is equivalent to the inside of ±20% of the regression line of the reference data (black dashed line). It should be noted that due to the exponentially decaying particle concentration, the concentration is represented in log scale on the y-axis. In the same manner, we set the limits as ±30% and ±50% of the reference data for Class 2 and Class 3, respectively. Class 4 is the one beyond the Class 3 acceptance criteria. To clarify the classification method, we showed an example in Figure 3. It can be seen that the measurement data of the test sensor for the highest and lowest concentrations during the test period meet the criteria for Class 3 and Class 1, respectively. In this case, we classify this low-cost dust sensor as Class 3. It should be mentioned that if there are experimental errors in the particle generation or flow system, the regression data obtained by the reference instrument shown in Figure 3 will not be straight in a semi-log graph.

| RE SULTS AND D ISCUSS I ON
Some of the sample test results are shown in Figure 4. Figure 4A-C are the sample test results for a "module type" sensor, "standalone type" sensor, and "embedded type" sensor, respectively. Based on the criteria introduced in the previous section, the tested sensors in Figure 4A-C were classified to Class 1, Class 3, and Class 2, respectively. Figure 5 represents the slopes obtained from the measurements using 78 PM 1 sensors including all three types, that is, module, standalone, and embedded types. It is clearly seen that the absolute values of the slopes obtained from the test sensors classified as Class 1, 2, and 3 are in good agreement with those from the reference instrument regardless of sensor types, lying on the 1:1 line. However, the obtained slopes from some of Class 4 sensors are slightly deviated from the reference data, for example, black stars in Figure 5A.
The linearity comparison for 125 PM 2.5 sensors with the three types was also performed, and the results are shown in Figure 6.
Except for some of Class 4 sensors, the other low-cost dust sensors have the similar slopes to the reference instrument, indicating the response characteristics of the sensors to the exponentially changing concentration are generally good. The degree of deviations from the 1:1 line shown in the module type sensors, that is, Figure 6A, was found to be larger than the other two types of sensors, especially when compared to the embedded type sensors in Figure 6C. The results for PM 10 measurements are represented in Figure 7, but due to some difficulties in manufacturing process, which might be related to failure of the satisfaction of certain criteria, embedded type sensors were not available during the test period. Therefore, Figure 7 only shows the results for the module and standalone types of sensors. We found that most of the Class 4 sensors with the module type showed significant deviations in slope while relatively good linearity was seen for the standalone type sensors.
In addition to the evaluation of linearity, we can clearly evaluate the degree of deviation for the measurements of high and low particle concentrations from the reference data. Figure 8 quantitatively represents the degree of deviation of PM 1 (Figure 8A-C), Figure 8D-F), and PM 10 ( Figure 8G,h) measurements relative to the reference instrument at the high and low concentrations. If a data point is placed within ±20% of the reference data at both high and low concentration, the sensor is classified as Class 1, which is depicted as a circular solid dot in a red rectangle. In the same way, the limit of Class 2 and 3 is represented as a blue and dark red rectangle, respectively. To be noted, some of the sensor data points that are out of the most outer boundaries, that is, over ±80%, are not shown. From the way of representing the data as shown in Figure 8 based on the testing method introduced in this study, one can easily evaluate the performances of sensors according to particle size and concentration.
The performance classifications of test sensors are summarized in Table 1 for different sensor types, that is, module, standalone, and embedded types. According to the results shown in Table 1, about one quarter of the low-cost dust sensors regardless of sensor type were classified into Class 1. As seen in the table, a considerable portion of the module type sensors, that is, 32.1%, belongs to Class 4, indicating the less accuracy compared with the standalone and embedded type sensors. Therefore, it can F I G U R E 8 Deviations of particulate matter (PM) concentration measurements from the reference data at high and low particle concentrations. C s and C r are the concentration measured by a test sensor and reference instrument, respectively: (A) PM 1 , module; (B) PM 1 , standalone; (C) PM 1 , embedded; (D) PM 2.5 , module; (E) PM 2.5 , standalone; (F) PM 2.5 , embedded; (G) PM 10 , module; (H) PM 10 , standalone type sensor be concluded that standalone and embedded type sensors were calibrated better by the final manufactures than the half-finished module manufactures. Figure 9 represents the percentages of sensor grade depending on PM for all test sensors. The accuracy of the low-cost dust sensors is higher for detecting smaller sized particles, that is, PM 1 , compared with PM 2.5 and PM 10 . For instance, the Class 1 satisfaction ratio for PM 1 , PM 2.5 , and PM 10 is 32%, 26%, and 16%, respectively. From the result, low-cost dust sensors cannot detect larger particles efficiently, which should be a main part needed to be improved near future. It should be noted that theoretically the optic system can easily detect large particles due to higher scattering  one possible explanation is that the low particle sampling efficiency and the significant transportation losses for larger particles inside a sensor. Therefore, the findings in this study also enlighten the importance of the proper designs of the sampling port and flow channel for minimizing the particle losses, which might affect the sensor performance significantly.
We classified the sensors' performance according to the type of light source in the sensors, that is, LED or laser, shown in Figure 10.
Only 27 samples adopted an LED as an illumination source, and most of the sensors used a laser, that is, 237 samples. As it is shown in Figure 10, no significant difference on sensor performance was found.
We also estimated the ratios of the low-cost sensors according to PMs in Figure 11, and detailed values are summarized in Table 2.
Interestingly, the effect of a light source on detecting different sized PMs was observed from the results showing that the lowcost sensors with an LED as a light source can hardly function on PM 10 measurement as shown in Figure 11C. This may be caused by the sampling mechanism of the LED type sensors. LED type sensors usually employ heating registers as an air mover. This register type air mover usually generates very low air speed inside the sensor that can hardly transport large-sized particles into the sensing zone.
In general, the sensor performance of the LED-based module type sensors is much less satisfactory than the embedded type sensors. This may imply that the sensors embedded in air purifiers have better algorithms or calibrations than the half-finished module sensors. The performance of the low-cost dust sensors according to the light source and the sensor types is summarized in Table 3. The assessment procedure for different types of sensors and the data shown in this study will provide manufacturers and users with valuable insights on the performance of low-cost dust sensors.

| CON CLUS ION
Low-cost dust sensor performance has been tested using an exponentially decreasing particle concentration that is recommended in   About one quarter of the tested sensors satisfied the Class 1 acceptance limit. However, the accurate PM 10 measurement by using the low-cost sensors was found to be relatively difficult to achieve, which might be caused from the difficulty in transporting large particles to detection zones. The comprehensive assessment presented in this study should be widely adopted for the further improvement of low-cost dust sensors in this high demand.