Abstract
 Top of page
 Abstract
 1. Introduction
 2. Literature review
 3. Proposed approach
 4. Numerical examples
 5. Conclusion
 Biographical Information
 REFERENCES
In many complex experiments, nuisance factor may have large effects that must be accounted for. Covariates are one of the most important kinds of nuisance factors that can be measured but cannot be controlled within the experimental runs. In this paper a novel approach is proposed, based on goal programming, to find the best combination of factors so as to optimize multiresponsemulticovariate surfaces with consideration of location and dispersion effects. Furthermore, it is supposed that several covariates considered in the experiment have probability distributions of known form. One objective is to find the most probable values of each covariate. For this purpose, a multiobjective mathematical optimization model is proposed and its efficacy is demonstrated by two numerical examples. Copyright © 2010 John Wiley & Sons, Ltd.
1. Introduction
 Top of page
 Abstract
 1. Introduction
 2. Literature review
 3. Proposed approach
 4. Numerical examples
 5. Conclusion
 Biographical Information
 REFERENCES
In the models used in the design and analysis of experiments, several input variables can affect the response, or output variables. If the input variables are known and are controllable factors, their levels can be set so as to optimize the functions of the observed responses. However, some of the input variables may be ‘nuisance’ variables which are not of interest and may not be controllable. Montgomery1 categorizes such variables into three classes:

Known and controllable variables whose effects are not of interest as a factor. The statistical technique of blocking can be used to eliminate the effects of these variables on the statistical analysis and thus on model building. Examples are systematic variations caused by shifts of workers or differences between manufacturing lines.

Known and uncontrollable variables are often called covariates. As the variables are suspected to be important, their values should be measured during the experiment. Inclusion of the individual effects of covariates and their interactions with other variables can help to improve the modeling of the outputs. The chemical properties of raw materials are an example.

Unknown and uncontrollable variables, where the relevance of the factor is unknown, and it may change levels during the experiment. Atmospheric humidity or age of feedstock are examples in this class. Randomization is the design technique used to avoid bias from such nuisance factors.
In the analysis of complex systems there are, almost invariably, several responses. Optimization of multiple quality characteristics (or response variables) is more complicated than optimization of a single one since we face different units, importance, and optimality directions. In this study a multicovariatemultiresponse model has been developed in which the covariates have stochastic patterns and the aim is to find the best arrangement of the input variables including covariates and factors so as to optimize the response variables. This paper is organized as follows. In Sections 2.1 and 2.2, a literature review is presented for both covariateoriented studies and stochastic multiresponse optimization (MRO). In Section 3, a general model for multicovariateMRO is proposed and in Section 4, computational descriptions and other properties of the model are illustrated by two numerical examples. Finally, conclusions and key remarks are presented in Section 5.
4. Numerical examples
 Top of page
 Abstract
 1. Introduction
 2. Literature review
 3. Proposed approach
 4. Numerical examples
 5. Conclusion
 Biographical Information
 REFERENCES
In this section, two cases are studied to illustrate the applications of the proposed approach. In the first experiment there are two factors and one normally distributed covariate that might affect two responses. The data are generated from three replicates of a Central Composite Design (CCD). Table II shows more information about these experiments.
Table II. Experimental data for case 1    Response variables 

Factors  Covariates  Expected value  Estimated variance 

A  B  X∼N(15, 4^{2})     

−1  1  19.589  35  8.65  3.51  0.79 
1  1  17.385  30  2.88  1.16  1.70 
−1  1  22.156  29  7.95  3.57  1.98 
1  1  9.569  43  1.97  2.75  0.75 
−1.414  0  20.328  40  5.61  3.99  1.47 
1.414  0  17.519  34  3.48  1.50  0.88 
0  1.414  19.712  29  8.31  2.33  1.66 
0  1.414  19.484  39  8.82  3.89  1.78 
0  0  11.995  39  5.50  1.65  1.00 
  Desired value  36  6.25  0  0 
  Standard deviation  5.22  2.664  1.0917  0.477 
As presented in Table II, the two responses (µ_{1}, µ_{2}) represent the location effects of the input variables, whereas the other responses represent the dispersion effects used in dual response modeling. The fitted response surfaces for a simulated data set, obtained using the MINITAB software package, can be written as:
where X is single covariate and it is known that X∼N(15, 4^{2}). The response variables are of type NTB with desired values . According to the response surfaces and desired values of each response, the mathematical model of the optimization problem can be formulated as
Subject to
In this example, the desired probability value of the covariate occurrence is initially set to 0.19 and, later, sensitivity analysis for this parameter and its effects on the results will be performed (this is an adjustable parameter that might be obtained by interaction between the analyst and decision maker). This parameter is a lower bound for the occurrence probability of covariates; greater values of this parameter make the feasible region tighter and the results more reliable. In this example all objective function sets have equal weights, that is w′_{i} = 1. Table III shows the optimum results for Case 1 obtained using the LINGO 8 optimization software.
Table III. Optimum solution for case 1 and comparisonOptimum values  A  B  X      f(c ± α) 

Proposed method  −0.2414  −0.714  13.8812  36  6.25  1.86  0.689  0.19 
Derringer22  −0.072  −0.256  12.97  36  6.252  1.54  0.85  0.17 
Kim and Lin23  0.4766  0.29  12.05  33.36  5.722  1.33  0.79  0.15 
Kovach and Cho19  0.0153  −0.0792  12.767  36  6.25  1.5  0.925  0.169 
Pignatiello9  −0.0605  −0.297  12.77  35.92  6.07  1.512  0.809  0.17 
Considering the experimental data in Table II and the optimum values in Table III, it is clear that the proposed method has satisfied all response function constraints, including two location effects (µ_{1}, µ_{2}), two dispersion effects , and one covariate expectation. Table IV shows the results of MANCOVA for the fitted response surfaces and also includes some measures for comparison. MANCOVA and its optimization phase have been performed in MINITAB 15 statistical software.
Table IV. Optimum solution for case 1 using MANCOVA in Minitab 15Optimum values  A  B  X      f(c ± α) 

MANCOVA (Minitab)  1  0.106  22.1563  31.5361  6.8535  2.8557  1.4109  0.041 
 0.26  1  17.52  34.32  6.25  2.468  1.156  0.162 
 1  1  9.569  42.71  4.58  2.416  0.578  0.08 
Proposed method  0.2414  0.714  13.8812  36  6.25  1.86  0.689  0.19 
For sensitivity analysis of the optimization model, contour plots of the response surfaces as functions of the input variable are shown in Figure 3.
In Figure 3, we have set the third variable at its optimum value. In the right section of Figure 3, B is set to −0.714 and in the left section, A is set to 0.2414. It is clear in Figure 3, that if the covariate increases above its optimum value, the feasible region of the model becomes tighter. But small changes in the values of the controllable variables do not lead to unacceptable response values. As mentioned above the proposed method not only maximized the expected value of the covariate, but also satisfied all of the constraints on the response variables including location and dispersion effects.
Previously the target value of covariate probability was set to 0.19. Increasing this value makes the problem infeasible and in order to perform sensitivity analysis on this parameter it has been varied in the reasonable range. Figure 4 shows the effect of this parameter on location and dispersion effects and Figure 4 shows the sensitivity of goal function and covariate probability with respect to this parameter compared to each other.
Figure 4 shows that in this example, increasing the desired value of covariate probability has an inverse relationship with goodness of optimal goal function which should be minimized. In addition, greater values of this parameter could lead to a more reliable result since the optimum covariate value may occur with higher probability. But in this example, if this parameter takes a value greater than 0.19, the model will be infeasible. The effect of changing this parameter on each response variable can be found in Figure 5. This figure shows that this parameter has different effects on and , while two location effects are not sensitive with respect to this parameter.
We now consider Case 2, a multicovariate model. As shown in Table V, the second case has two response variables (R_{1} and R_{2}), three factors (A, B, and C) and two covariates (C_{1} and C_{2}). The response variables are of type NTB and their desired values are . In addition, the covariates have the probability distributions
In this case we ran a single replication of the design. As a result, dispersion effects cannot be considered.
Table V. Experimental data for case 2RunOrder  PtType  Blocks  A  B  C  C_{1}  C_{2}  R_{1}  R_{2} 

1  1  1  1  1  1  2  0.384  52.034  2.34 
2  1  1  0  1.681793  0  3  0.387  42.525  2.55 
3  0  1  0  0  0  3  0.786  48.524  4.52 
4  1  1  1  1  1  6  0.684  39.321  9.31 
5  1  1  1  1  1  7  0.739  35.483  3.48 
6  1  1  0  0  1.681793  6  1.460  38.404  8.4 
7  1  1  0  1.68179  0  7  2.210  55.907  5.0 
8  1  1  1  1  1  8  2.240  44.107  1.7 
9  1  1  1  1  1  8  2.812  44.276  2.2 
10  0  1  0  0  0  9  3.124  46.538  6.5 
11  0  1  0  0  0  12  3.204  45.231  5.21 
12  1  1  1  1  1  10  5.042  44.942  4.42 
13  1  1  1.681793  0  0  11  5.566  49.217  4.27 
14  0  1  0  0  0  11  7.126  43.160  3.1 
15  0  1  0  0  0  11  6.987  45.874  4.87 
16  1  1  1.68179  0  0  9  8.675  48.103  4.1 
17  1  1  1  1  1  12  8.833  43.757  3.57 
18  0  1  0  0  0  16  10.303  44.434  4.44 
19  1  1  0  0  1.68179  16  18.878  49.860  9 
20  1  1  1  1  1  17  19.667  35.622  5.6 
The correlation between C1 and C2 is 0.865. Owing to the resulting collinearity between the predictor variables, we use PCA to provide a linear transformation to uncorrelated variables. Table VI shows the analysis using MINITAB 15 statistical software.
Table VI. Principal component analysis for two correlated covariatesVariable  C_{1}  C_{2}  Eigenvalue  Proportion  Cumulative 

PC_{1}  0.584  0.812  46.737  0.938  0.938 
PC_{2}  0.812  −0.584  3.078  0.062  1.000 
According to Table VI, the required transformation of the two covariates to make them uncorrelated is:
 (11)
 (12)
By inverting (11) and (12) the original values of covariates can be obtained from (13) and (14):
 (13)
 (14)
Using PCA the regression function between inputs and outputs can be fitted more reliably. The two fitted response surfaces are
In this case all response sets have equal weights of importance (w′_{i} = 1). Now the final GP model can be written as follows.
Subject to
 (15)
 (16)
 (17)
 (18)
 (19)
 (20)
Similar to the first example, the objective of this model is to minimize the summation of standardized weighted deviations of targets. Constraints set (15) calculates the deviation from the target of NTB response variables. The next set is to find the optimum covariates that satisfy the minimum acceptable probability for occurrence which equals 0.05 for both exponential and Poisson covariates. The effect of changing this parameter on the results was elaborated previously in the first example. The linear relation between PCs and covariates are entered in the model by relations (17). The next three constraint sets (sets 20–23) show the limits of the decision variables depending on their definitions.
Table VII gives the optimal values of each variable resulting from optimization of the proposed model. This problem has also been solved using Minitab software24. The Minitab software provides MRO module in which the desirability function method is used to analyze such problems.
Table VII. Optimization results—case 2Optimal values  A  B  C  PC_{1}  PC_{2}  C_{1}  C_{2}  R_{1}  R_{2}  f(C_{1})  f(C_{2} ± α) 

Proposed methods  0.536  0.67  1.655  5.418  7.19  9  0.2  40  5.5  0.125  0.077 
Desirability function  1.51  1.6818  0.18  —  —  17  19.66  40  5.5  0.013  0.0015 
As shown in Table VII, R_{1} and R_{2} reach their goals and the probability functions of the covariates have proper values. Although the desirability function method has also satisfied two response variables (R_{1}, R_{2}), it is obvious that the probability value of covariates occurrence resulted by the proposed method is considerably better than the desirability method. In other words, if we set factors as (A, B, C) = (1.51, −1.68, −0.18), we cannot expect that the responses will be satisfied unless the covariates are observed as (C_{1}, C_{2}) = (17, 19.66). Therefore, since these values of covariates occur with smaller probability, the factor setting will not be sufficiently reliable.
Biographical Information
 Top of page
 Abstract
 1. Introduction
 2. Literature review
 3. Proposed approach
 4. Numerical examples
 5. Conclusion
 Biographical Information
 REFERENCES
Taha Hossein Hejazi received an MSc in Industrial Engineering from Shahed University, Tehran. His primary research interests include quality control and engineering, optimization and design of experiments, computer simulation, metaheuristics for optimization and Multiple Criteria Decision Making. Also he will start PhD programs at Amirkabir University of Technology (Tehran Polytechnic) in the next semester and he is a member of the International Association of Computer Science and Information Technology (IACSIT).
Mahdi Bashiri is an Assistant Professor at Shahed University. He received his BS in Industrial Engineering from Iran University of science and technology in 1999 and his MS and PhD in Industrial Engineering from the Tarbiat Modares University of Tehran in 2001 and 2005, respectively. He visited the statistics department at London School of Economics and political sciences at 2004 for six months. His research interests are Multiple Response Optimization, Design of Experiments and Facility Location and Layout.
Kazem Noghondarian is an Assistant Professor of Industrial Engineering at Iran University of Science and Technology. He received his BSc from University of Neveda and his MSc and PhD from Arizona State University and University of British Columbia (UBC), respectively. His primary research interests include statistical quality control, quality management, design of experiments, data mining and simulation.
Anthony C. Atkinson is Emeritus Professor of Statistics at the London School of Economics and Political Science. He received his MA in Chemical Engineering from Cambridge University in 1960 and his MSc and PhD in Statistics from Imperial College, London in 1965 and 1970. His primary research interests include: clinical trials; generalized linear models; simulation; robust statistical methods; regression diagnostics and optimum experimental design. He was editor of the Journal of the Royal Statistical Society, Series B and an associate editor of Biometrika and Technometrics. Professor Atkinson is a Fellow of the American Statistical Association.