Noise simulation in classification with the noisemodel R package: Applications analyzing the impact of errors with chemical data

Classification datasets created from chemical processes can be affected by errors, which impair the accuracy of the models built. This fact highlights the importance of analyzing the robustness of classifiers against different types and levels of noise to know their behavior against potential errors. In this context, noise models have been proposed to study noise‐related phenomenology in a controlled environment, allowing errors to be introduced into the data in a supervised manner. This paper introduces the noisemodel R package, which contains the first extensive implementation of noise models for classification datasets, proposing it as support tool to analyze the impact of errors related to chemical data. It provides 72 noise models found in the specialized literature that allow errors to be introduced in different ways in classes and attributes. Each of them is properly documented and referenced, unifying their results through a specific S3 class, which benefits from customized print, summary and plot methods. The usage of the package is illustrated through four application examples considering real‐world chemical datasets, where errors are prone to occur. The software presented will help to deepen the understanding of the problem of noisy chemical data, as well as to develop new robust algorithms and noise preprocessing methods properly adapted to different types of errors in this scenario.

text, noise models have been proposed to study noise-related phenomenology in a controlled environment, allowing errors to be introduced into the data in a supervised manner. This paper introduces the noisemodel R package, which contains the first extensive implementation of noise models for classification datasets, proposing it as support tool to analyze the impact of errors related to chemical data. It provides 72 noise models found in the specialized literature that allow errors to be introduced in different ways in classes and attributes.
Each of them is properly documented and referenced, unifying their results through a specific S3 class, which benefits from customized print, summary and plot methods. The usage of the package is illustrated through four application examples considering real-world chemical datasets, where errors are prone to occur. The software presented will help to deepen the understanding of the problem of noisy chemical data, as well as to develop new robust algorithms and noise preprocessing methods properly adapted to different types of errors in this scenario.

K E Y W O R D S
attribute noise, chemical data, classification, label noise, noise models

| INTRODUCTION
Real-world data are often subject to errors or noise in a variety of fields. 1,2 Among them, datasets coming from chemical analyses and procedures in diverse applications, such as agri-food or medicine, may be particularly prone to imperfections. 3,4 These errors proceed from different sources associated with chemical practice, including human action and the instruments used, among others. 5,6 Thus, noisy data can come from human errors when performing experiments, such as inaccuracies when diluting ingredients or small spills when handling chemicals during transfer. 7 Another sources of error occur when estimating a measurement. For example, a mark may be exceeded when a beaker is filled to a specific volume or several experts may estimate different times in reactions involving a color change. 8 On the other hand, incorrect calibration of certain chemical instruments or equipment, which is not performed with proper regularity, can lead to inaccuracies in experiments and data collection. 5 Laboratory equipment limitations when making measurements, such as the precision usually associated with measuring flasks, are also a potential source of error. 5,8 In this context, applying the results of chemical procedures to create classification datasets may imply the introduction of noise in the form of corruptions in both output class labels and input attributes. 9,10 Learning from noisy data usually involves building less accurate and more complex models as a result of overfitting errors. 11 These facts have led to the development of a multitude of proposals to overcome the inconveniences of noise, such as robust algorithms or noise preprocessing techniques. 12,13 Robust algorithms 12 are methods characterized by being less influenced by noisy data. An example of a robust method is C4.5, 14 which includes pruning mechanisms to reduce the influence of errors in model building. Other well-known examples are Repeated Incremental Pruning to Produce Error Reduction (RIPPER) 15 and Random Forest (RF), 16 which are widely used in the specialized literature. On the other hand, within noise preprocessing techniques, the main representatives are known as noise filters. 13 They identify and remove noisy samples from the data, which has proven to be advantageous in many domains. 17 Furthermore, the separation of noise detection and learning avoids the usage of noisy samples during the classifier building process.
In R, 18 there is a variety of software packages implementing proposals belonging to the above approaches. 19,20 For example, robustDA 21 implements Robust Mixture Discriminant Analysis, which allows building robust classifiers when data suffer from noise in class labels. Similarly, the AdaSampling package 22 codes an adaptive sampling procedure, which is also applicable for learning with mislabeled data. The C4.5 and RIPPER robust methods can be found in the RWeka package, 23 whereas RF is implemented in randomForest. 24 Many of the aforementioned robust algorithms can also be found in the well-known caret package. 19 Related to these approaches, it is worth noting another type of robust techniques known as high-breakdown methods. 4,25,26 Some of them are implemented in the rrcov 26 and rrcovHD 27 packages. On the other hand, the cellWise package 28 contains procedures to detect cellwise outliers and robust methods to analyze contaminated data, 29,30 as well as functions to generate multivariate normal datasets with various types of outliers. 31,32 Among the software devoted to noisy data preprocessing, the fmf package 33 proposes a fast class noise detector, which provides a noise score for each of the samples in the dataset, whereas the UBL package 20 implements various filtering methods with a special focus on imbalanced learning.
The above algorithms, either robust or noise preprocessing methods, are designed to deal with data in which noise may have particular characteristics. 11,34 However, noise affecting real-world data and, particularly, chemical datasets, is usually not quantifiable, and its characteristics are unknown. In order to test the effectiveness of these methods in a controlled environment, noise models 35 have been proposed to introduce errors into datasets in a supervised way. The utility of noise models in the field of noisy data research is clearly reflected by their wide use in the specialized literature, as they are frequently employed in dozens of papers published each year in this area. 17,36,37 Because they facilitate controlling the type of noise injected into the data, as well as its amount and characteristics, these models allow the design of experimental frameworks appropriate to the objective of the study and draw significant conclusions from them. 9,36 Their usage allows, for example, studying the circumstances (types and levels of noise) in which classifiers are more affected or evaluating the effectiveness of noise preprocessing techniques to detect and treat the errors introduced. 12,17 However, the usefulness of noise models is not limited solely to the field of research. They also have a practical utility, because they can be used by any practitioner with the aim of analyzing the impact of noise on the particular dataset addressed. Even though the robustness of well-known algorithms has already been studied in previous works, it is important to note that their behavior is not always uniform and may depend on the specific data processed. 1,12,38 Thus, as a consequence of an exhaustive experimental study, Nettleton et al 12 concluded that the behavior of each technique depends on the type and percentage of noise, the class imbalance, and the characteristics of the datasets themselves. This fact implies that an algorithm known to be robust does not always have to offer the best result in all the datasets affected by noise. It should also be noted that variations in the parameters of algorithms can affect their robustness and that there are classification algorithms whose robustness under different noise circumstances has not been sufficiently studied. 1,39 Something similar occurs with noise preprocessing algorithms, whose efficiency depends on the characteristics of the processed data. 40 For the above reasons, in practice, when it is desired to study the impact of noise on new classification problems, it would be advisable to previously simulate such errors using noise models and thus analyze their impact in a controlled manner. Despite the importance of these models both in research and in practice for noisy data analysis, to date, most of them are not yet implemented in any software package. This paper presents the noisemodel R package, which provides the implementation of a total of 72 noise models for classification data found in the specialized literature. 35 It includes methods to introduce label noise, attribute noise, and both in combination. 10,41 The software is built following an S3 formulation of methods (with default and formula class alternatives), allowing the models to be applied in a unified and user-friendly way. The package is available from the Comprehensive R Archive Network (CRAN) at https://CRAN.R-project.org/package=noisemodel. It has complete documentation * available to the user, providing details and examples of use of each of the functions implemented. In addition to these usage examples, this paper shows the functionality of the package through the analysis of the impact of errors in different applications that make use of real-world chemical data. Thus, the noise models included in this software will provide the field of chemometrics with a solid support for the research and development of multiple applications related to noisy data, including: • deepening the study of the impacts of noisy chemical data in the field of classification; • providing tools proposed in the specialized literature to introduce errors simulating those in real-world data; • accelerating the development of new algorithms dealing with erroneous chemical data; • verifying how existing classifiers behave when some adverse effect causes inaccuracies in the data; • creation of data with controlled errors to test the effectiveness of preprocessing methods for noise treatment; and • easing the experimentation and reproducibility of research results related to noisy chemical data.
The rest of this paper is organized as follows. Section 2 presents an introduction to noise models in classification. Section 3 details the usage of the noisemodel package, from its installation to the application of each of its functionalities. Then, Section 4 shows practical examples to illustrate the use of the software on real-world chemical datasets. Finally, Section 5 concludes this work offering some final remarks.

| NOISE MODELS IN CLASSIFICATION
Consider a classification dataset composed by n samples x i (i f1, …, ng), m input attributes v j ( j f1, …, mg) and one output class label v 0 taking one value in L ¼ f1, …, cg. The value of the variable v j in the sample x i is denoted as x i,j . If x i,j is corrupted by the noise model, it is represented as x i,j . In classification, noise models aim at simulating errors in class labels and attribute values occurring in real-world datasets. 10,41 The main sources of label noise are often related to human error, primarily due to inaccurate information or subjectivity during the labeling process. 9 On the other hand, attribute noise is usually related to instrumental measurement errors. 11 As an example, Figure 1 shows a twodimensional version of the well-known iris dataset before ( Figure 1A) and after ( Figure 1B-F) the application of several noise models implemented in the noisemodel package considering a noise level ρ ¼ 0:2, that is, 20% of the values in the dataset are noisy.
Label noise models 17,34 are designed to corrupt the class label x i,0 of the samples x i in a dataset. One of the most widely used models in this group is Symmetric uniform label noise 41 ( Figure 1D), which introduces errors affecting any part of the domain. All samples x i have the same probability ρ that their class label x i,0 is corrupted. Once a sample x i is chosen to be mislabeled in a dataset with c classes, its noisy label x i,0 is determined by the probability mass function: Instead of giving equal noise chance to all samples, other models assign a higher probability to those near decision limits, where they share similar characteristics. 17 These models use the distance d i of each sample x i to the decision boundaries to estimate its noise probability based on different probability density functions (pdf). 9 For example, the pdf of the exponential (Equation 2, left) or the gamma distribution (Equation 2, right) have been used 9 : *https://cran.r-project.org/web/packages/noisemodel/noisemodel.pdf.
A different approach for borderline label noise is followed by Neighborwise borderline label noise 17 ( Figure 1E), which computes a noise score Nðx i Þ ¼ dðx i , x j Þ=dðx i , x k Þ for each sample x i based on the Euclidean distances d to its closest samples of the same (x j ) and different class (x k ). The n Á ρ samples with highest scores are then chosen to be mislabeled. Models have also been designed to introduce label noise in other parts of the dataset. For example, Hubnessproportional uniform label noise 34 ( Figure 1F) assigns a higher probability Pðx i Þ ¼ N k ðx i Þ=ðn Á kÞ of being corrupted to those samples x i closer to hubs in the dataset, with N k ðx i Þ being the number of times that x i is among the k nearest neighbors of any other sample.
Finally, there are models that follow other different strategies to introduce label noise. For example, in Symmetric center-based label noise, 42 the probability ρ i,j of confusing two classes i, j L is greater the closer they are F I G U R E 1 Application of label and attribute noise models on the iris dataset with a noise level ρ ¼ 0:2. Each color represents a class. Circles ( • ) symbolize clean samples, whereas crosses (Â) are noisy samples.
with d i,k the distance between the centers of classes i and k. Other approaches consider the usage of default values to mislabel samples, 43 the existence of a hierarchy among classes, 44 or the peculiarities of label noise in binary classification. 36 On the other hand, attribute noise models 45,46 are designed to simulate inaccuracies in attribute values x i,j ðj ≥ 1Þ in the dataset. Symmetric uniform attribute noise 39 ( Figure 1B) allows introducing errors affecting any part of the domain. It introduces n Á ρ errors in each attribute v j by randomly selecting a noisy value x i,j following a uniform distribution within the attribute domain: Other models introduce attribute noise in different areas in the dataset. 10,47 For example, Symmetric end-directed attribute noise 10 ( Figure 1C) introduces errors in the limits of the problem domain. Boundary/dependent Gaussian attribute noise 47 alters samples close to decision limits by means of zero-mean additive Gaussian noise. A Gaussian error distribution is also used by Symmetric Gaussian attribute noise, 38 in which each noisy value x i,j is obtained as with σ 2 ¼ k 2 ðmaxðv j Þ À minðv j ÞÞ 2 and k ð0,1. Note that, even though most attribute noise models alter different samples in each variable, some of them corrupt all attributes in each sample simultaneously. 46,47 An example of this type of models is Symmetric/dependent uniform attribute noise, 46 which chooses noisy values uniformly in the domain. An important question regarding noise simulation can arise when choosing which model may be the most suitable for the available dataset. Although the characteristics of noise occurring in real-world applications are difficult to know exactly, the different noise schemes have been proposed in specific contexts, which makes them more oriented to certain situations. 10,17 The most direct choice of model can be made depending on which type of variables (class or attributes) are most likely to be affected by errors in the dataset and, therefore, which type of noise is more interesting to study (label noise or attribute noise). However, as reflected above, each type of noise in turn allows for multiple ways of simulating its errors, for example, focusing on noise in decision boundaries, domain extrema, binary datasets, and so on. For a practical recommendation on the potential noise model to be used based on the knowledge of the problem treated, the characteristics of the available data, and the noise to be simulated, the reader may consult S aez. 35 3 | USAGE OF THE NOISEMODEL PACKAGE

| Installation
The noisemodel package is available from the CRAN repository at https://CRAN.R-project.org/package=noisemodel. It can be installed from the R command line by typing: This command also installs all the dependencies of the software, which can be checked in the Imports section of the CRAN website. In order to access all the package's functions, it must be attached in the usual way:

| Package content
The software offers the following functionality to the user: • a total of 72 functions implementing noise models for classification data used in the specialized literature; • the print, summary, and plot methods, which allow displaying the information on the noise introduction process concisely and visually; and • a two-dimensional version of the well-known iris dataset, as well as a variant with discretized attributes, which are used for illustrative purposes in the code examples of each of the models.
The software offers a wide range of alternatives for noise introduction, with 54 models to introduce label noise, 16 for attribute noise, and two combine both types of noise simultaneously. All the implemented noise models can be consulted through a complete Reference manual † associated with the package. The print and summary methods allow displaying an outline of the information associated with the noise introduction process, whereas the plot method graphically represents the resulting noisy dataset (these are discussed in Section 3.6). Finally, the 2D version of the iris dataset (see Figure 1A) maintains the numeric input attributes petal-length and petal-width, as well as its factor output label. In order to simplify these data in the application examples, duplicate (equal samples) and contradictory samples (those with the same input attributes and different class labels) are removed. The command data(iris2D) allow loading this dataset, creating the iris2D data.frame. The discretized version can be loaded using data(diris2D).

| Documentation
Each one of the functions in the noisemodel package has been properly documented using roxygen2. 48 The documentation allows consulting specific information of each available function, including a brief description of the purpose of the function (Description), the modes to call the function (Usage), an explanation of the input parameters of the function and types of variables (Arguments), a description of the function referring to its input arguments (Details), the output variables of the function (Value), a relevant contribution in which the implemented functionality was used (References), and a complete R code block with an example of how the function can be used (Examples).
The above information can be accessed from the Reference manual in the CRAN webpage. Furthermore, the documentation pages can be examined with the ? and help orders. For example, in order to access the documentation for the aforementioned Symmetric uniform label noise (sym_uni_ln), 41 one can type:

| Application of noise models
The noisemodel package provides two standard ways to call the noise introduction models: • a default method, which receives the input attributes in the x argument (a data.frame) and the class labels of each sample in the y argument (a factor vector), and • a formula class method, which receives the formula argument (an expression relating the input attributes and the output class) and the data argument (a data.frame containing the variables in the formula).
In addition to these parameters, each noise model function has as arguments a series of specific parameters, as well as a sortid parameter indicating whether the indices of clean and noisy samples should be sorted in the output (sortid = TRUE by default, but can be set to FALSE to save time applying the model).
The name of the function associated with each noise model is divided into three parts, which are separated by the symbol _ and are composed of three to four letters referring to each identifier of the tripartite nomenclature of noise models. 35 For example, the function name for Symmetric uniform label noise 41 is sym_uni_ln, for Hubness-proportional uniform label noise 34 is hubp_uni_ln, and for Gaussian-level uniform label noise 49 is glev_uni_ln. Below are some examples of the usage of the default function for these noise models along with their arguments in different colors (red for dataset arguments, blue for specific arguments, and brown for the index sorting argument): † https://cran.r-project.org/web/packages/noisemodel/noisemodel.pdf Table 1 shows the rest of function names for each noise model, along with a reference on which the implementation is based and the parameters for the default function. A description of each implemented noise model can be found in the Reference Manual associated with the package, as well as using the aforementioned ? and help orders, which also detail each of the arguments. Note that, in order to standardize the functions and make them applicable to all types of data, some may differ slightly from the exact approach used in the associated work. For example, the models used in Petety et al, 46 which are employed on data with binary attributes, have been adapted to deal with higher cardinality attributes.

| The ndmodel class
All functions implementing a noise model return an object of ndmodel class. This S3 class, which is designed to unify the return value of all noise schemes inside the noisemodel package, is a list containing 10 elements with the most relevant information of the process: • xnoise: a data.frame with the noisy input attributes.
• ynoise: a factor vector with the noisy output class.
• numnoise: an integer vector with the amount of noisy samples per variable. In order to verify the structure of a ndmodel object in a practical example, the resulting value returned by the sym_uni_ln method can be inspected: The output variable is a list with the 10 aforementioned elements. They can be easily obtained using the $ operator. The elements xnoise and ynoise are a data.frame and a factor vector of equal size as the original x and y arguments, respectively, but they contain the version with noisy values. Note that for label noise models, xnoise is equal to x and, for attribute noise models, ynoise is equal to y.
In this example, numnoise shows that 3, 2, and 5 samples of the setosa, versicolor, and virginica classes have been, respectively, corrupted. Finally, the rest of the elements (distr, model, param, and call) provide additional information about the noise introduction process, which is especially useful for the print, summary, and plot methods described in the next section.

| print, summary, and plot methods
In order to display the result of the noise introduction process contained in the ndmodel object after applying a noise model, specific print, summary, and plot methods have been implemented. The print function receives as argument the object of ndmodel class. After its usage, brief information about the noise injection process is displayed in the R console including the name of the noise introduction model, the parameters associated with the noise model and the number of noisy and clean values in the dataset. Thus, to use this function on the output object obtained above, one can simply type: On the other hand, the summary method receives as arguments both an object of ndmodel class and the showid parameter indicating whether the indices of noisy samples should be displayed. The information provided by summary is similar to that of print, but it adds some extra elements, such as the function call, the number of noisy samples per class/attribute, the number of clean samples per class/attribute, and the indices of the noisy samples (if showid = TRUE). The function call has the following form: Finally, the ndmodel class also allows displaying a graphical representation of the resulting noisy dataset using the plot method, which has the following structure: where • noise is a logical indicating which samples should be displayed on the graph, including the noisy samples (noise = TRUE), the clean samples (noise = FALSE) or all of them (noise = NA). • pca is a logical indicating if a PCA approach using singular value decomposition must be used before displaying the graph. This technique can be useful for visualizing data with more than two dimensions by plotting, for example, the principal components with the highest percentages of explained variance.

| Performance predicting clean and noisy samples
In order to exemplify the first application, data resulting from a chemical analysis of wines from three different cultivars in a region of Italy are used. 86 The dataset consists of 178 wine samples, in which the amounts of 13 wine constituents are determined, including alcohol, malic acid, magnesium, phenols, and flavonoids, among others. The training set will consist of 70% of the data, which are used to create a classifier using the C4.5 algorithm with the aim of predicting the cultivar from the chemical characteristics of the wine. Thus, 70% of the original dataset is selected and the model is created using the createDataPartition and train functions from the caret package, respectively: In the code above, data is a data.frame with the original dataset with the class label in its last column, tra is the training set, and model contains the C4.5 classifier (note that C4.5 is called J48 in the train function of the caret package). The model created is used to predict the cultivar in samples that will be received in the future from different laboratories devoted to chemical analysis (represented by the test set with 30% samples remaining). Imagine that, due to instrumental and transcription errors in the data, around 20% of the samples could contain random errors in their attributes. In order to simulate this situation and analyze the behavior of the classifier built in this scenario, noise is introduced into the test set (tst) using a noise model included in the noisemodel package. Note that this noise scheme (clean training/noisy test) could also occur in other applications and has been previously considered in relevant papers on noisy data. 1,12 The Symmetric/dependent uniform attribute noise model 46 (symd_uni_an) will be employed to simulate errors in this case, as it allows the attribute values to be randomly altered for a proportion level of the samples: Information about the noise introduction process can be obtained by using the commands print(tst_noise) or summary(tst_noise) described in Section 3.6. The graphical representation of the different types of samples in this noisy test data can also be obtained using the plot function (see Figure 2): F I G U R E 2 Results of applying the plot function on the test set in the first application example, with pca = TRUE and varying the noise argument. Its remaining parameters, xvar and yvar, are left with the default values 1 and 2, respectively. These graphs show three different groups for clean samples in the test set ( Figure 2B), whereas noisy samples are far from them ( Figure 2C). Finally, in order to study how the classifier created will behave in this scenario, the labels of the test data created with the noisemodel package are predicted and the classification accuracy is computed. For this, the xnoise and ynoise elements of the ndmodel class must be accessed: Similarly, because the indices of the clean and noisy samples in the test set are known through the idclean and idnoise elements of the ndmodel class, the accuracy classifying these types of samples can also be computed: The above results show that the classifier built has, in general, a test accuracy of 82.69% predicting the whole test data containing both clean and noisy samples. However, because noisy samples have been introduced in a controlled way, this result can be broken down according to the type of samples (clean or noisy) classified. Thus, whereas 97.62% of the clean test samples are correctly classified, only 20% of the noisy test samples are predicted without error. This analysis shows the need to reinforce the attention on this type of noisy samples in the future and apply mechanisms to detect errors during their classification due to their impact on system performance.

| Analysis of robustness to noise
Robustness refers to the ability of algorithms to build models that are less affected by noisy data. 87 In the second application example, the robustness of different classification algorithms when training with noisy chemical data is analyzed. In order to exemplify this scenario, a dataset with samples of Longquan celadon is used, 88 in which an analysis has been carried out to determine their chemical components, such as sodium oxide, magnesium oxide, silicon dioxide, or aluminum oxide. The classification task will consist of distinguishing real from imitations samples of Longquan celadon based on their chemical composition. First, similarly to Section 4.1, 70% of the data is considered for the training set and the remaining 30% for the test set: Because labeling is performed manually, let us assume that the distinction between types of samples can generate label noise. Because of this, in order to create a robust model from the data, practitioners decide to choose an alternative between two algorithms considered to be noise tolerant: RIPPER 15 or RF 16 (which are called JRip and rf in caret, respectively). In order to determine which of these algorithms is the most robust against noise in this domain, an experiment introducing noise into the class labels using the well-known Symmetric uniform label noise 41 (sym_uni_ln) is performed. Different noise levels, from 0 to 0.3 (by increments of 0.1), are introduced into the training set, keeping the test set unchanged. Thus, the impact of noise on the training process and the models created can be studied from variations in performance on the same test set. 39 To compute the test accuracy of each algorithm, RIPPER and RF, by corrupting the training set with each noise level ρ f0,0:1,0:2,0:3g, the following block of code can be used: Once the accuracy (acc) for each algorithm and noise level is obtained, an analysis of the robustness to noise of RIP-PER and RF in this noisy chemical dataset is performed. For this purpose, the Equalized Loss of Accuracy (ELA) 87 metric can be employed, which allows analyzing the robustness of classification algorithms against noise. It measures the loss of accuracy due to noise from a perfect score weighted by the accuracy with no noise: with ELA ρ being the value of the ELA metric for a noise level ρ, A 0 is the accuracy of a classifier without noise, and A ρ is the accuracy with a noise level ρ. Lower ELA values indicate a higher robustness of the classification algorithm. Thus, the ELA robustness measure is studied for each method and noise level, obtaining the following results: These results show that the robustness of both classification algorithms is affected as the noise level increases, because ELA worsens as the number of errors increases. In addition, RF is more robust than RIPPER in the studied example, because it obtains lower ELA values in all the noise levels. Thus, in case of building classifiers in this domain with noisy data, it is advisable to use RF instead of RIPPER because the models created will be less affected by errors in the data.

| Impact of noise on different attributes
The third application example considers a dataset focused on distinguishing 6 types of glass defined in terms of 9 variables, including their oxide content (such as sodium, iron, potassium, and others) and refractive index. 89 This classification problem is motivated by criminological research, because the correct identification of small pieces of glass can be used as evidence. In this case, the impact of noise on different attributes will be studied. After the training (tra) and test (tst) sets are created, noise is introduced into only one of their attributes using an asymmetric attribute noise model (asy_uni_an 46 ), which allows a different noise level to be injected into each attribute. In the experiment below, a noise level of 20% is considered and the classifiers are created with C4. 5 These results show that the noise that most deteriorates the classification performance is the one affecting Si and Ca, which obtain the lowest accuracies. On the other hand, noise in Al and the refractive index RI affect performance less. Thus, special attention should be paid to errors in those attributes that have a greater impact on performance in this domain to ensure that the classifier built is not greatly affected.

| On the randomness of experiments with noisy data
In the previous sections, both data partitioning and noise introduction are affected by randomness, which may affect the results and conclusions obtained. This section focuses on this issue and offers some recommendations for experimentation. In order to illustrate this aspect, consider the dataset with samples of celadon used above. After the training and test sets are created, 20% of noise will be introduced into the training set using the default method of the sym_uni_ln noise model. The result of the noise introduction process can be easily inspected using the print function: Similarly, the method of formula class can be used with the same noise model and level: In the code above, the formula argument allows indicating how attributes and class are related. The point . in the formula class$. can be employed to indicate the use of all input attributes to predict the class label. By inspecting this output with the print(outfrm) command, one can check that it is identical to what was previously obtained with the default method. However, it should be noted that most of noise models follow random processes. Thus, although the percentages of clean and noisy samples are the same in both calls (which is the result of using the same noise level), the corrupted samples may be different. This fact can be verified by setting the argument showid = TRUE in the summary function, which shows the indices of the noisy samples. For example, when inspecting the output of the default method using summary, the following information is displayed: Thus, both partitioning and noise introduction are usually random in experiments related to noisy data. In order to obtain the most robust results possible, it is recommended to carry out several iterations in experiments involving noise models and analyze the results obtained from all of them. A possible alternative to do this is to define a function experiment, which receives the original dataset and other parameters of interest as inputs, to partition the data, introduce noise and, finally, return the result of the experiment (for example, the classification accuracy). Then, it would be enough to call this function the desired number of iterations (niter) and study the distribution of the results in all the executions using descriptive metrics, such as mean and standard deviation. The repetition of an experiment with noisy data allows to increase the robustness of the results obtained and the significance of the conclusions reached. As an example, in an experiment with noisy data to analyze accuracy, code similar to the following could be used: In the code above, note the utility of setting the random seed with the set.seed() function before applying noise models to facilitate reproducibility of experiments. Following this approach, the analyses of the previous sections are repeated considering 50 iterations to generate different datasets in the noise introduction processes. Table 2 shows the results obtained in each application example of Sections 4.1-4.3 once all the iterations have been carried out.
The conclusions obtained in Sections 4.1 (prediction of clean and noisy samples) and 4.2 (robustness of classification algorithms) are maintained considering multiple iterations, although the results in absolute value may vary to some extent. The most notable contrast is the second application example in the comparison of RIPPER and RF with 30% noise, because the difference between the two is not as pronounced as in the individual experiment. Another important aspect is reflected by the standard deviations in Table 2: The first experiment shows that the greatest dispersion occurs when predicting noisy samples, whereas the second experiment shows that increasing the noise level also increases the dispersion of the robustness. These facts may be associated with the greater complexity in these scenarios.
The most disparate conclusions are obtained in the experiment analyzing the impact of noise on different attributes (Section 4.3). Even though the individual experiment showed that noise in the Si and Ca variables was the most detrimental to performance, the multiple iterations in Table 2 show a fairly homogeneous impact of noise on all attributes. In fact, the most damaging noise in this case is the one affecting the refractive index RI, which was one of the least affected variables in the individual experiment. These facts show the need to repeat several times experiments involving noise models in order to obtain meaningful conclusions based on the most robust results possible.

| CONCLUDING REMARKS
This paper has presented the noisemodel R package, proposing it as support tool in the analysis of impacts of errors related to chemical data. This contains the first extensive implementation of noise models for classification problems. In order to introduce the context and motivation for this work, the problem of noise with chemical data, their potential solutions found in related software, and the need to simulate it using noise models have been discussed. The models for introducing label and attribute noise have been briefly presented, offering an overview of possible alternatives in each case. Then, a detailed description of the noisemodel software has been provided, showing the installation process, contents of the package, and how to access the documentation of each function. Furthermore, the usage of the implemented functions has been illustrated, as well as their output, which is an object of ndmodel class. This class benefits from specific print, summary, and plot methods, which allow its content to be displayed. The usage of the software has been shown through four illustrative application examples considering real-world chemical datasets. In the first application, where data from the chemical analysis of wines have been used, the noisemodel package has been employed as part of the analysis to estimate the accuracy on different types of samples (clean and noisy). Then, the second application has focused on the analysis of the robustness of various classification algorithms in the domain of the chemical composition of ceramics, whereas the third application has studied the impact of noise on different variables that determine the composition of several types of glass. Finally, the fourth application example has discussed the importance of accounting for randomness in experiments involving noise models.
Note that the possibility of having and being able to easily use a large number of noise models proposed in the specialized literature in a single software package is an advance in this field, because many of these models do not even have a public code that allows their usage both by the scientific community and practitioners in any field. Noise models are commonly used in the field of noisy data research, because they facilitate the control of the amount and characteristics of noise to obtain meaningful conclusions from the experiments. 17,37 On the other hand, the use of noise models in practical applications to analyze the impact of errors in a specific dataset can also be recommended, because there are works showing that the general behavior of classifiers and noise preprocessing methods can vary depending on the characteristics of the processed data. 12,40 In the context of noisy chemical data, due to the frequency of errors in realworld applications and its consequences when obtaining knowledge from them, the noisemodel package represents a useful tool in noisy data analysis and validation of classifiers and preprocessing techniques. The software presented will help both to delve into the consequences of errors in chemical data and to facilitate the design of new noise treatment techniques to mitigate their negative effects.

DATA AVAILABILITY STATEMENT
The software associated with this paper is openly available from the Comprehensive R Archive Network (CRAN) at https://CRAN.R-project.org/package=noisemodel.