An earlier version of this paper was presented at the 8th International Amsterdam Multilevel Conference, Amsterdam Netherlands, March 2011. The author would like to thank Stephen G. West for his comments on the earlier version.
Original Article
Factorial invariance in multilevel confirmatory factor analysis
Article first published online: 17 MAY 2013
DOI: 10.1111/bmsp.12014
© 2013 The British Psychological Society
Issue
British Journal of Mathematical and Statistical Psychology
Volume 67, Issue 1, pages 172–194, February 2014
Additional Information
How to Cite
Ryu, E. (2014), Factorial invariance in multilevel confirmatory factor analysis. British Journal of Mathematical and Statistical Psychology, 67: 172–194. doi: 10.1111/bmsp.12014
Publication History
- Issue published online: 13 JAN 2014
- Article first published online: 17 MAY 2013
- Manuscript Accepted: 8 APR 2013
- Manuscript Revised: 28 MAR 2013
- Manuscript Received: 17 JAN 2012
Abstract
- Top of page
- Abstract
- 1. Introduction
- 2. Factorial invariance in single-level confirmatory factor analysis
- 3. Multilevel confirmatory factor analysis
- 4. Multilevel factorial invariance
- 5. Illustration
- 6. Summary and conclusion
- References
- Supporting Information
This paper presents a procedure to test factorial invariance in multilevel confirmatory factor analysis. When the group membership is at level 2, multilevel factorial invariance can be tested by a simple extension of the standard procedure. However level-1 group membership raises problems which cannot be appropriately handled by the standard procedure, because the dependency between members of different level-1 groups is not appropriately taken into account. The procedure presented in this article provides a solution to this problem. This paper also shows Muthén's maximum likelihood (MUML) estimation for testing multilevel factorial invariance across level-1 groups as a viable alternative to maximum likelihood estimation. Testing multilevel factorial invariance across level-2 groups and testing multilevel factorial invariance across level-1 groups are illustrated using empirical examples. SAS macro and Mplus syntax are provided.
1. Introduction
- Top of page
- Abstract
- 1. Introduction
- 2. Factorial invariance in single-level confirmatory factor analysis
- 3. Multilevel confirmatory factor analysis
- 4. Multilevel factorial invariance
- 5. Illustration
- 6. Summary and conclusion
- References
- Supporting Information
With the development of multilevel structural equation modelling (Goldstein & McDonald, 1988; Lee, 1990; Longford & Muthén, 1992; Muthén, 1990, 1994; Muthén & Satorra, 1995), multilevel confirmatory factor analysis is increasingly used in behavioural and social sciences (e.g., Cheung, Leung, & Au, 2006; Reise, Ventura, Nuechterlein, & Kim, 2005; Zimprich, Perren, & Hornung, 2005). Multilevel confirmatory factor analysis provides an approach to estimating and evaluating measurement models with multilevel data. Like other measurement models, issues of measurement invariance potentially arise in multilevel measurement models. Measurement invariance concerns whether the relationship between an observed measure and underlying latent construct is the same across different groups (Mellenbergh, 1989; Meredith & Millsap, 1992; Millsap, 1997). When the same measure is collected from qualitatively distinct groups (e.g., males and females; public and private school students) or at different time points from the same individuals in repeated measures designs (e.g., first grade and third grade), it is critical to establish measurement invariance in order for comparisons of the constructs to be meaningful. The admissible comparison that may be made across groups depends critically on the level of measurement invariance that can be achieved (Widaman & Reise, 1997).
Within the structural equation modelling tradition, measurement invariance is established by testing a hierarchical series of models that impose increasingly strict constraints on the hypothesized confirmatory factor analysis (CFA) model. Factorial invariance in CFA is typically examined at four different levels: configural, weak, strong, and strict invariance (described below). For single-level CFA models, a standard procedure for testing factorial invariance has become well established (e.g., Cheung & Rensvold, 1999; Meredith, 1993; Reise, Widaman, & Pugh, 1993). Numerous applications testing factorial invariance can be found in a variety of areas of psychological (e.g., Atienza, Balaguer, & García-Merita, 2003; Dauphinee, Schau, & Stevens, 1997; Thill et al., 2003), cross-cultural (e.g., Ang et al., 2009; Steenkamp & Baumgartner, 1998), organizational (e.g., Schaufeli & Bakker, 2004; Torkzadeh, Koufteros, & Doll, 2005), health (e.g., Ang, Shen, & Monahan, 2008; Gregorich, 2006; Malcarne, Fernandez, & Flores, 2005), and educational research (e.g., Edwards & Oakland, 2006; Green-Demers, Legault, Pelletier, & Pelletier, 2008).
Factorial invariance in multilevel CFA requires examining invariance of parameters in the level-1 model and that in the level-2 model. Multilevel data have a hierarchical structure such that individual observations are nested within clusters. In multilevel modelling, level 1 indicates the lowest level in the nested structure, level 2 indicates the next level within which the level-1 observations are nested, and so on. Multilevel factorial invariance introduces additional complexities that are beyond a simple extension of the well-established procedures for testing factorial invariance in single-level CFA. First, the group membership may exist at level 1 or at level 2. To illustrate, consider a two-level model in which the data are collected from students nested within schools. The researcher may be interested in testing factorial invariance at the school level (level 2), for example, between public and private or between religious and non-religious schools. Alternatively, the researcher may be interested in testing factorial invariance at the student level (level 1), for example, between boys and girls or between first- and third-grade students. Second, when group membership is at the lower level (level 1, here students), the level-1 group membership intersects the clustered structure of multilevel data. In this case, a methodological challenge arises: how can factorial invariance be tested across level-1 groups without losing the capability of multilevel modelling to appropriately adjust for the potential dependency arising from the clustering in multilevel data?
Despite the development of multilevel structural equation modelling, testing factorial invariance in multilevel CFA across multiple groups has not been fully established in the literature. Only a few studies have considered factorial invariance in multilevel CFA across level-2 groups (Davidov, Dülmer, Schlülter, Schmidt, & Meuleman, 2012; Kim, Kwok, & Yoon, 2012; Muthén, Khoo, & Gustafsson, 1997). Multilevel factorial invariance across level-1 groups has rarely been addressed, except in one recent study by Jak, Oort, and Dolan (2013a). Jak et al. proposed a five-step procedure which tests both invariance across level-1 groups and invariance across level-2 groups. For testing invariance across level-1 groups, their five-step procedure takes the approach of using the level-1 group membership as a covariate (referred to as a restricted factor analysis model in their paper), instead of taking a multiple group analysis approach.
The goal of this paper is to present and illustrate a procedure for testing factorial invariance in multilevel CFA model. The presented procedure takes a multiple group analysis approach. Therefore it is within the same framework as the well-known standard procedure for testing factorial invariance in single-level CFA. Also, as will be shown later, testing invariance in factor loadings is straightforward in the procedure presented in this paper, whereas it is less straightforward in the restricted factor analysis model, particularly when the invariance is to be tested for a large number of factor loadings simultaneously (see Jak et al., 2013a, for more details about the restricted factor analysis model).
The procedure for testing multilevel factorial invariance across level-2 groups is parallel to the procedure for factorial invariance in single-level CFA. Testing multilevel factorial invariance across level-1 groups, however, raises methodological challenges. This paper identifies two challenges and presents solutions to them. For both cases, testing multilevel factorial invariance is illustrated using an empirical data set. Given limitations of space, the presentation is limited to the contexts that are most widely used in the literature: two-level CFA models and factorial invariance between two groups.
2. Factorial invariance in single-level confirmatory factor analysis
- Top of page
- Abstract
- 1. Introduction
- 2. Factorial invariance in single-level confirmatory factor analysis
- 3. Multilevel confirmatory factor analysis
- 4. Multilevel factorial invariance
- 5. Illustration
- 6. Summary and conclusion
- References
- Supporting Information
Let y be a vector of observed measures and η be a vector of underlying latent factors. The measurement model in CFA specifies a set of linear relations between observed measures and underlying latent constructs:
- (1)
where τ is a vector of measurement intercepts, Λ is factor loading matrix, and ɛ is a vector of residuals. It is assumed that and . For the multiple group case, the measurement model can be written as
- (2)
where the subscript k indicates group membership. Assuming that the latent factors are uncorrelated with the residuals, the mean and covariance structure of y_{k} are reproduced respectively by
- (3)
- (4)
where , , and .
Factorial invariance is typically tested at the following four levels (Widaman & Reise, 1997):
- configural invariance – the dimension and the pattern of zero and non-zero loadings in Λ_{k} are the same across groups;
- weak invariance – Λ_{k} is invariant across groups (Λ_{k} = Λ for all k);
- strong invariance – Λ_{k} and τ_{k} are invariant across groups (Λ_{k} = Λ and τ_{k} = τ for all k);
- strict invariance: Λ_{k}, τ_{k}, and Θ_{k} are invariant across groups (Λ_{k} = Λ, τ_{k} = τ, and Θ_{k} = Θ for all k).
Figure 1 illustrates linear relations between an observed variable and a latent factor at four levels of invariance between two groups. Solid lines represent the linear relation between an observed variable and a latent factor in group 1; dashed lines represent the linear relation in group 2.
With configural invariance, no parameter associated with latent factors is comparable across groups. When weak invariance holds, the relations between observed and latent variables in two groups are depicted by parallel but non-overlapping lines, as in Figure 1(b). In this case, the covariance matrix of latent factors Ψ_{k} is comparable across groups (directly if the same identification method is used; indirectly if different identification methods are used across groups), because the variance and covariance are based on the deviation from the mean. When strong invariance holds, the two lines exactly overlap each other, as in Figure 1(c). Under strong invariance, (3) and (4) are simplified to
- (5)
- (6)
In this case, the means (α_{k}) and covariance matrix (Ψ_{k}) of latent factors are comparable across groups. In (5), any difference in the means of observed scores y between groups is due to the difference in latent factor means α and therefore the means of observed variables are comparable across groups. But the variances and covariances of observed scores y are due not only to the difference in Ψ but also to the difference in Θ_{k}. In Figure 1(c), the two lines exactly overlap each other but group 1 (indicated by crosses) shows larger residual variance than group 2 (indicated by circles). Finally, when strict invariance holds (Figure 1(d)), the two lines exactly overlap each other and also the residual variance is the same in both groups. In this case, (6) is simplified to
- (7)
With strict invariance, the means (α_{k}) and covariance matrix (Ψ_{k}) of latent factors are comparable across groups. As shown in (5) and (7), the group difference in both means and variances of observed scores is due to the group difference in means and variances of latent factors. In other words, all group differences on the observed scores are attributable to group difference on the latent factors. The means and variances of observed variables are comparable too.
3. Multilevel confirmatory factor analysis
- Top of page
- Abstract
- 1. Introduction
- 2. Factorial invariance in single-level confirmatory factor analysis
- 3. Multilevel confirmatory factor analysis
- 4. Multilevel factorial invariance
- 5. Illustration
- 6. Summary and conclusion
- References
- Supporting Information
Throughout this paper, I use individual as level-1 unit and cluster as level-2 unit in multilevel data. Group indicates the group membership across which the invariance is tested. Suppose that the data are from N individuals clustered within J clusters. The clusters are a simple random sample from a population of clusters, and the individuals are a simple random sample with each cluster. The number of individuals in the jth cluster is n_{j}, where . For the balanced case, n_{j} = n for all j. Let y_{ij} denote a data vector for individual i in cluster j. In multilevel CFA, the data vector y_{ij} is decomposed into two latent random components reflecting two sources of random variation in multilevel data: between-cluster random components (y_{Bj}) and within-cluster, between-individual random components (y_{Wij}):
- (8)
where and . Note that y_{Bj} and y_{Wij} are latent (i.e., not directly observed) components. All level-1 variables are subject to implicit, model-based decomposition in multilevel CFA.
The two-level CFA model specifies linear relations between y_{Bj} and underlying latent factors η_{Bj} at level 2, and linear relations between y_{Wij} and underlying latent factors η_{Wij} at level 1:
- (9)
where , , , , and .The two-level CFA model shown in (9) assumes the following:(a) , therefore (b) Σ_{Wj} = Σ_{W} for all j;(c)
In (9), the level-1 parameters (Λ_{W}, Ψ_{W}, Θ_{W}) do not have subscript j because the within-cluster covariance structure is assumed to be homogeneous across clusters (assumption (b)).1 With these assumptions, the mean and covariance structure of y_{ij} are reproduced by (10) and (11), respectively:
- (10)
- (11)
In two-level CFA, the mean structure is captured at level 2 and the level-1 model has no mean structure, as shown in (9) and (10).
4. Multilevel factorial invariance
- Top of page
- Abstract
- 1. Introduction
- 2. Factorial invariance in single-level confirmatory factor analysis
- 3. Multilevel confirmatory factor analysis
- 4. Multilevel factorial invariance
- 5. Illustration
- 6. Summary and conclusion
- References
- Supporting Information
I now consider establishing factorial invariance in multilevel CFA, first when the group membership is at level 2 and second when the group membership is at level 1.
4.1 Level-2 group membership
The procedure for testing multilevel factorial invariance across level-2 groups is parallel to the standard procedure for testing single-level factorial invariance. Because the group membership is at level 2, the data can be separated by group membership without altering the clustering in multilevel data. With level-2 group membership (k = 1, …, K), the two-level CFA model can be written as
- (12)
where the subscript (k) indicates the level-2 group membership, , ,, , and
Assuming multivariate normality, the maximum likelihood (ML) solution can be obtained by the following fitting function:
- (13)
where is the number of clusters in group k, θ is a vector of parameters, , in which , is the total number of individuals in group k, , and .
All four levels of invariance are testable in the level-2 model: configural, weak (), strong (, ),and strict (, , ). At level 1, weak invariance () is testable, but strong invariance is not because the level-1 model has no mean structure. A model testing invariance in between level-2 groups can also be specified and tested.
4.2 Level-1 group membership
Suppose the standard procedure is adopted to test multilevel factorial invariance across level-1 groups (e.g., boys vs. girls in typical mixed gender schools that have both male and female students). The standard procedure first separates the data into level-1 groups. Then multilevel CFA models are specified in each group and equality of parameters is tested across the groups. In this case, the individuals are separated into groups within each cluster because the level-1 group membership intersects the level-2 cluster. Therefore the decomposition of level-1 variables occurs separately in each level-1 group:
- (14)
where the subscript |k indicates that the decomposition occurs separately within each level-1 group. The between-cluster component y_{Bj|k} in (14) is not constant for all individuals within the same cluster (e.g., the between components for boys are different from the between components for girls within the same school). The decomposition shown in (14) fails to capture the dependency between members of different level-1 groups who belong to the same cluster. In order to maintain the capability of multilevel modelling to take dependency due to the clustered structure into account, the decomposition should follow (8), not (14).
When the group membership is at level 1, it is critical that the decomposition of the level-1 variables is not conditional on the level-1 group membership. Therefore the decomposition must occur before the data are separated into level-1 groups:
- (15)
where the subscript (k) within parentheses indicates that the data are separated after the level-1 variables are decomposed. The between-cluster component is constant for all individuals regardless of level-1 group membership as long as they belong to the same cluster (i.e., for all k). The level-2 model is equivalent regardless of level-1 group membership.
In (15), in each level-1 group is not necessarily zero. The level-1 model needs mean structure to represent the relative difference in means between level-1 groups. The two-level CFA model with multiple level-1 groups can be written as
- (16)
where , , , , , and . In the level-1 model, all four levels of invariance are testable: configural, weak (), strong (,), and strict (,,).The level-2 model is equivalent regardless of level-1 group membership.
Another methodological challenge in testing multilevel factorial invariance across level-1 groups is that an appropriate effective sample size should be used for the level-2 model. In (15) the effective sample size is J for all across all level-1 groups, and the effective sample size for in each level-1 group is smaller than J. In the simplest case, suppose that the school sizes are equal across all J schools and there are equal numbers of boys and girls in each school. The effective sample size would be 0.5J for and 0.5J for so that the effective sample size for the between-cluster model (which is equivalent between boys and girls) would be J. For a general case, the effective level-2 sample size can be obtained using weights that are based on the relative sizes of level-1 groups in each cluster. The weight in level-1 group k is obtained by
- (17)
where is the number of individuals in level-1 group k in cluster j and n_{j} is the total number of individuals in cluster j. The effective level-2 sample size for each level-1 group is obtained by . The sum of effective level-2 sample size across all level-1 groups is always J.
Assuming multivariate normality, the ML fitting function for two-level CFA for multiple level-1 groups can be written as2
- (18)
where θ is a vector of parameters, , , in which is the number of individuals in cluster j in group k, and . Note that (18) includes an additional term for within-cluster mean structure , which did not appear in (13). Also in (18), the between-cluster model [, ] does not depend on the level-1 group membership, that is, there is no subscript (k).
4.3 MUML estimation for testing multilevel factorial invariance
To illustrate testing multilevel factorial invariance across level-1 groups, I use Muthén's maximum likelihood (MUML: Muthén, 1989, 1990) estimation via a manual set-up for two reasons. First, MUML serves as a better vehicle for a didactic presentation, showing the decomposition of y_{ij} more explicitly. Second, there is currently no software package available to obtain ML solutions using the ML fitting function shown in (18). When the cluster sizes are equal for all clusters, the MUML estimates are equivalent to ML estimates. When the cluster sizes are not equal (i.e., unbalanced cluster sizes), MUML provides an approximated solution. The performance of MUML approximation theoretically depends on the level-2 sample size (i.e., number of clusters) and the variability of cluster sizes (Yuan & Hayashi, 2005). It has been empirically shown that MUML provides a good approximation when the number of clusters is 100 or larger (Hox, 1993; Hox & Maas, 2001).
MUML via multiple group analysis requires means of observed variables () weighted by the square root of the scaling parameter (c, described below), between-cluster covariance matrix (S_{B}), and pooled within-cluster (S_{PW}) covariance matrix as input data:
- (19)
- (20)
- (21)
S_{B} is an unbiased estimator of the weighted composite of the population within and between covariance matrices (Σ_{W} + cΣ_{B}), where c is a scaling parameter, defined by, or c = n for balanced cases. S_{PW} is an unbiased estimator of population within covariance matrix (Σ_{W}). The ‘trick’ in MUML via manual set-up is to use a multiple group analysis of single-level CFA models with two ‘groups’ (^{M}group hereafter).3 In the first ^{M}group, [, ] is fitted to [, S_{B}]. In the second ^{M}group, is fitted to S_{PW}. The mean structure in the second ^{M}group is zero. The effective sample sizes are J and N - J for ^{M}group 1 and ^{M}group 2, respectively. The within-cluster covariance structure is constrained to be equal between the two ^{M}groups. The input means are weighted by so that the mean and covariance structure for the between-cluster model are on the same scale. To extend the MUML approach to testing factorial invariance in two-level CFA, a multiple group analysis needs to be set up for twice as many ^{M}groups as the number of groups.
The following input statistics are required in order to use MUML estimation for testing multilevel factorial invariance across level-1 groups:
- (22)
- (23)
- (24)
- (25)
where is a scaling parameter in group k, is the total number of individuals in group k, the number of individuals in cluster j in group k, and the effective level-2 sample size in group k. A SAS macro to compute the weights for level-2 effective sample size, scaling parameters, and the input statistics can be found in the supporting information, available with the online version of this paper.
For MUML estimation via a manual set-up, 2k ^{M}groups are required. For example, in order to test multilevel factorial invariance between two level-1 groups, a multiple group analysis with four ^{M}groups needs to be specified. For group k = 1, and are submitted to ^{M}group 1; and are submitted to ^{M}group 2. For group k = 2, and are submitted to ^{M}group 3; and are submitted to ^{M}group 4. The effective sample sizes for ^{M}groups 1–4 are , , , and , respectively. Three sets of constraints are required. The first set is equality constraints for between two ^{M}groups within each level-1 group membership (e.g., ^{M}group 1 and ^{M}group 2 for group k). The second is for to be equivalent across groups (i.e., constrained to be equal between ^{M}groups 1 and 3). The third is a set of constraints necessary for model identification and test of factorial invariance in the hierarchy. The necessary constraints are shown in the example below.
5. Illustration
- Top of page
- Abstract
- 1. Introduction
- 2. Factorial invariance in single-level confirmatory factor analysis
- 3. Multilevel confirmatory factor analysis
- 4. Multilevel factorial invariance
- 5. Illustration
- 6. Summary and conclusion
- References
- Supporting Information
In this section, I illustrate testing multilevel factorial invariance using PISA (Programme for International Student Assessment) 2003 data (Lee, 2009; OECD, 2004, 2005). A two-level CFA model for five mathematics self-efficacy items (see Table 1)4 is shown in Figure 2. I first illustrate testing multilevel factorial invariance between two countries, New Zealand and Turkey (level-2 group membership), and then testing multilevel factorial invariance between male and female students in Turkey (level-1 group membership).
ICC in each country | |||
---|---|---|---|
Items | ICC | NZL | TUR |
Notes
| |||
Q31c Calculating how many square metres of tiles you need to cover a floor | .080 | .035 | .096 |
Q31b Calculating how much cheaper a TV would be after a 30% discount | .088 | .043 | .109 |
Q31a Using a train timetable to work out how long it would take to get from one place to another | .164 | .056 | .115 |
Q31d Understanding graphs presented in newspapers | .132 | .039 | .092 |
Q31 h Calculating the petrol consumption rate of a car | .063 | .037 | .083 |
5.1 Example: Multilevel factorial invariance between New Zealand and Turkey (level-2 group membership)
In New Zealand (NZL), 4,189 students were nested within 173 schools. School size ranged from 1 to 54 (mean = 24.21, standard deviation = 10.031). In Turkey (TUR), 4,030 students were nested within 159 schools. School size ranged from 2 to 35 (mean = 25.35, standard deviation = 7.865). ML estimation in Mplus 6.12 (Muthén & Muthén, 1998) was used.
The goodness of fit was examined for the overall model (i.e., the fit of the school-level and student-level models was examined simultaneously) and also for each level separately using the level-specific approach described in Ryu and West (2009). In multilevel CFA, the model fit statistics for the overall model may be dominated by the level-1 model because the sample size is typically larger at level 1, and therefore the lack of fit in the level-2 model may not be detected by the fit statistics for the overall model (see Hox, 2010; Ryu & West, 2009). The level-specific approach by Ryu and West utilizes partially saturated models to evaluate the model fit at each level separately (e.g., a partially saturated model in which the level-1 model is saturated and the level-2 model is specified as hypothesized is used to evaluate the fit of the level-2 model). In both countries, the model fitted well at both levels (see Table 2). In NZL, the proportion of variance accounted for by the common factor (i.e., communality) ranged from .348 to .589 at student level, and from .531 to .934 at school level. In TUR, the communality ranged from .261 to .465 at student level, and from .785 to .989 at school level.
New Zealand | Turkey | |
---|---|---|
Note
| ||
Overall model | ||
χ^{2} = 69.467, df = 10, p < .001 CFI = .989 RMSEA = .038 SRMR_{B} = .034, SRMR_{W} = .018 | χ^{2} = 59.223, df = 10, p < .001 CFI = .990 RMSEA = .035 SRMR_{B} = .022, SRMR_{W} = .017 | |
School-level model | ||
= 4.707, df = 5, p = .453 CFI_{PS_B} = 1.000 RMSEA_{PS_B} = .000 | = 10.839, df = 5, p = .055 CFI_{PS_B} = .986 RMSEA_{PS_B} = .086 | |
Student-level model | ||
= 61.036, df = 5, p < .001 CFI_{PS_W} = .989 RMSEA_{PS_W} = .037 | = 46.876, df = 5, p < .001 CFI_{PS_W} = .990 RMSEA_{PS_W} = .032 |
For the configural invariance model, the following constraints were imposed for model identification.5 In the school-level model, Q31c was chosen as the reference variable.6 For Q31c, the factor loading was fixed to 1 ( = = 1), and the measurement intercept was constrained to be equal between NZL and TUR ( = ). The factor mean was fixed to zero in one group (NZL here, = 0) and freely estimated in the other group () was. In the student-level model, the factor loading for the reference variable Q31c was fixed to 1 ( = = 1). The mean structure does not exist in the student-level model and therefore no specification was needed.
Factorial invariance at school level was tested by a series of models with increasingly added constraints in the school-level model. The model fit statistics and likelihood ratio test statistics (LR or Δχ^{2}: Bentler & Bonett, 1980) are summarized in Table 3a.7 Once again, the model fit was assessed in two ways: the overall model (i.e., school-level and student-level model simultaneously) and the school-level model (denoted by subscript PS_B in Table 3a) using the level-specific approach by Ryu and West (2009). In Table 3a, the strong invariance did not hold at the school level. The model was modified by removing the equality constraints on measurement intercepts for Q31a, Q31d, and Q31 h (StrongPartial). The strict invariance model with partial strong invariance (non-invariant intercepts for Q31a, Q31d, and Q31 h) yielded acceptable model fit statistics and this model was selected as an appropriate school-level model.
Model | χ ^{2} | LR test (Δχ^{2}) | CFI | RMSEA | SRMR_{W} | SRMR_{B} | CFI_{PS_B} | RMSEA_{PS_B} |
---|---|---|---|---|---|---|---|---|
Notes
| ||||||||
Configural | 128.706 (20), p < .001 | .990 | .036 | .018 | .029 | .989 | .041 | |
Weak | 133.916 (24), p < .001 | 5.210 (4), p = .266 | .990 | .033 | .018 | .037 | .986 | .039 |
Strong | 354.927 (28), p < .001 | 221.011 (4), p < .001 | .973 | .053 | .019 | .248 | .772 | .195 |
StrongPartial^{a} | 136.581 (25), p < .001 | 2.665 (1), p = .103 | .990 | .033 | .018 | .037 | .986 | .041 |
Strict | 151.965 (30), p < .001 | 15.384 (5) p = .009 | .989 | .031 | .018 | .064 | .968 | .054 |
Factorial invariance at student level was tested by increasingly imposing constraints in the student-level model. The model fit statistics and LR test statistics are summarized in Table 3b. For student-level factorial invariance, the model fit was assessed for the overall model and for the student-level model (denoted by subscript PS_W in Table 3b). Note that the chi-square test and the LR test were significant at p < .05 for all models and all model comparisons at the student level, because the sample size was large (8,219 students). As mentioned earlier, strong invariance is not considered at the student level as there is no mean structure in the level-1 model. The fit indices were not satisfactory for strict invariance model. The model was modified by removing the equality constraint on the student-level residual variance of Q31d (StrictPartial). The partial strict invariance model (non-invariant residual variance for Q31d) was selected as an appropriate student-level model.
Model | χ ^{2} | LR test (Δχ^{2}) | CFI | RMSEA | SRMR_{W} | SRMR_{B} | CFI_{PS_W} | RMSEA_{PS_W} |
---|---|---|---|---|---|---|---|---|
Notes
| ||||||||
Configural | 128.706 (20), p < .001 | .990 | .036 | .018 | .029 | .990 | .035 | |
Weak | 191.966 (24), p < .001 | 63.260 (4), p < .001 | .984 | .041 | .023 | .031 | .983 | .037 |
Strict | 677.705 (29), p < .001 | 485.739 (5), p < .001 | .941 | .074 | .041 | .050 | .935 | .054 |
StrictPartial^{a} | 328.468 (28), p < .001 | 136.502 (4), p < .001 | .972 | .051 | .030 | .039 | .970 | .044 |
Finally, the selected school-level and student-level models were combined. For the final model, χ^{2} = 354.675 (df = 39, p < .001), CFI = .974, RMSEA = .045, SRMR_{W} = .030, SRMR_{B} = .067. The estimates are shown in Figure 3. At school level, the non-invariant intercepts are interpreted as follows: the school-level components of Q31a and Q31d would be higher for schools in Turkey than those for schools with the same level of mathematics self-efficacy in New Zealand. In other words, given the same level of math self-efficacy, the observed scores of Q31a and Q31d would be higher for students who belong to schools in Turkey than for students who belong to schools in New Zealand. For Q31 h, the direction is the opposite: given the same level of mathematics self-efficacy, the observed scores of Q31 h would be lower for students who belong to schools in Turkey than for students who belong to schools in New Zealand. Therefore the means of Q31a, Q31d, and Q31 h are not comparable across the two countries. If strong invariance had held for all five items at school level, the difference in the common factor (SELFEFF_{B}) mean (0.193, as shown in Figure 3) would have been interpreted as the difference in school-level mathematics self-efficacy between NZL and TUR.
At student level, strict invariance did not hold for Q31d. The non-invariant residual variance is interpreted as follows: the difference in the student-level component of Q31d is larger for students in Turkey compared to the difference in the student-level component of Q31d associated with the same difference of mathematics self-efficacy for students in New Zealand. In other words, the same difference in the student-level component of Q31d does not reflect the same difference in the common factor SELFEFF_{W} between the two countries. Therefore the variance and covariance associated with Q31d cannot be compared between the two countries (i.e., the estimated variances of SELFEFF_{W}, 0.311 for NZL and 0.375 for TUR in Figure 3, are not comparable).
5.2 Example: Multilevel factorial invariance between male and female students (level-1 group membership)
The data used for this example were from 4,030 students – 1,748 females and 2,282 males – nested within 159 schools in Turkey. School size ranged from 2 to 35 (mean = 25.35, standard deviation = 7.865). The proportion of female students ranged from 0 to 1, the mean proportion was .4215, and the standard deviation was .2263. Before testing multilevel factorial invariance using MUML estimation, the estimates and standard errors obtained by ML were compared to those obtained by MUML. As shown in Table 4, the estimates and standard errors obtained by MUML were comparable to those obtained by ML, even though the school size was not balanced.
ML | MUML | ML | MUML | ||
---|---|---|---|---|---|
Note
| |||||
Student level | School level | ||||
Fixed to 1 | Fixed to 1 | Fixed to 1 | Fixed to 1 | ||
0.970 (0.030) | 0.970 (0.030) | 1.052 (0.063) | 1.051 (0.062) | ||
0.955 (0.030) | 0.958 (0.030) | 1.065 (0.064) | 1.052 (0.062) | ||
0.760 (0.029) | 0.760 (0.029) | 0.950 (0.071) | 0.950 (0.069) | ||
0.944 (0.031) | 0.946 (0.032) | 0.874 (0.072) | 0.875 (0.071) | ||
2.162 (0.026) | 2.137 (0.026) | ||||
2.039 (0.026) | 2.013 (0.026) | ||||
2.284 (0.026) | 2.257 (0.025) | ||||
2.158 (0.025) | 2.132 (0.025) | ||||
2.401 (0.025) | 2.382 (0.025) | ||||
0.394 (0.012) | 0.395 (0.012) | 0.004 (0.003) | 0.004 (0.003) | ||
0.354 (0.011) | 0.354 (0.011) | 0.001 (0.002) | 0.001 (0.002) | ||
0.334 (0.010) | 0.333 (0.010) | 0.001 (0.002) | 0.002 (0.003) | ||
0.522 (0.013) | 0.521 (0.013) | 0.007 (0.004) | 0.008 (0.004) | ||
0.486 (0.013) | 0.486 (0.013) | 0.015 (0.004) | 0.015 (0.004) | ||
ψ _{SELFEFFW} | 0.319 (0.016) | 0.318 (0.016) | ψ _{SELFEFFB} | 0.071 (0.012) | 0.075 (0.012) |
α _{SELFEFFB} | Fixed to 0 | Fixed to 0 |
The weights for effective level-2 sample size were .4215 and .5785 for females and males, respectively. The effective level-2 sample sizes were = .4215 × 159 = 67.0185 and = .5785 × 159 = 91.9815. The scaling parameters were = 5.1237 and = 4.9888. Four ^{M}groups are required to test factorial invariance between females (^{M}groups 1 and 2) and males (^{M}groups 3 and 4; see Figure 4). The effective sample sizes for ^{M}groups 1–4 were 67.0185, 1680.9815, 91.9815, and 2190.0185, respectively. Note that the sum of school-level effective sample sizes is equal to the number of schools (67.0185 + 91.9815 = 159).
For model specification, three sets of constraints were imposed. First, the student-level model was set equal between two ^{M}groups within each level-1 group membership (e.g., between ^{M}groups 1 and 2 for females, between ^{M}groups 3 and 4 for males). Second, the school-level model was set equal between females and males (e.g., [, ] constrained to be equal between ^{M}groups 1 and 3). The third is a set of constraints necessary for model identification and factorial invariance testing in the hierarchy. The constraints for configural invariance model are summarized in Table 5. Mplus syntax for configural invariance model can be found in the online supporting information.
Females | Males | |||
---|---|---|---|---|
^{M}group 1 | ^{M}group 2 | ^{M}group 3 | ^{M}group 4 | |
Notes
| ||||
School level | School-level model constrained to be equal between ^{M}group 1 and ^{M}group 3 | |||
= 1 | = 1 | |||
= 0 | = 0 | |||
Student level | Student-level model for females constrained to be equal | Student-level model for males constrained to be equal | ||
= 1 | = 1 | = 1 | = 1 | |
= | = | = | = | |
= 0 | = 0 |
Once again the school-level model is equivalent between females and males, and the factorial invariance between females and males at the school level is not of concern. Configural, weak, strong, and strict invariance were tested at student level. The model fit statistics and LR test statistics are summarized in Table 6. The configural invariance model fitted well. The weak invariance model also fitted well: the LR test was not significant even with the large sample size (Δχ^{2} = 4.580 for df = 4, p = .333). The strong invariance model yielded a significant LR test, but the fit statistics indicated good model fit (CFI = .974, RMSEA = .057, and SRMR = .056). Finally, for the strict invariance model, the LR test was significant (Δχ^{2} = 20.193 for df = 5, p = .001), but the fit statistics indicated acceptable fit (CFI = .971, RMSEA = .057, and SRMR = .058). Considering the sensitivity of the LR test statistic to large sample size and the other fit indices indicating good model fit, the strict invariance model was selected as an appropriate student-level model.
χ ^{2} | LR test (Δχ^{2}) | CFI | RMSEA | SRMR | |
---|---|---|---|---|---|
Notes
| |||||
Configural | 132.449 (35), p < .001 | .981 | .053 | .051 | |
Weak | 137.029 (39), p < .001 | 4.580 (4), p = .333 | .981 | .050 | .052 |
Strong | 185.306 (43), p < .001 | 48.277 (4), p < .001 | .974 | .057 | .056 |
Strict | 205.499 (48), p < .001 | 20.193 (5), p = .001 | .971 | .057 | .058 |
The estimated strict invariance model is shown in Figure 5. Both the means and (co)variances of the common factor (SELFEFF_{W}) are comparable between females and males. The estimated mean of SELFEFF_{W} was lower for males than females by –0.234 (p < .001). The estimated variance of SELFEFF_{W} was 0.292 and 0.331 for females and males, respectively. The equality of the variance between the two groups can be tested by adding an equality constraint on the variance of SELFEFF_{W}. The fit statistics for this constrained model were χ^{2} = 209.822 (df = 49, p < .001), CFI = .970, RMSEA = .057, and SRMR = .062. The LR test statistic (compared to the strict invariance model) was Δχ^{2} = 4.463 for df = 1, p = .035. Considering the sensitivity of the LR test to large sample size, it was concluded that the variance of SELFEFF_{W} was not different between females and males.
6. Summary and conclusion
- Top of page
- Abstract
- 1. Introduction
- 2. Factorial invariance in single-level confirmatory factor analysis
- 3. Multilevel confirmatory factor analysis
- 4. Multilevel factorial invariance
- 5. Illustration
- 6. Summary and conclusion
- References
- Supporting Information
This paper has presented and illustrated a procedure to test factorial invariance in multilevel CFA. To emphasize the key idea again, a simple extension of the standard procedure is not appropriate for testing multilevel factorial invariance across level-1 groups, because the dependency between members of different level-1 groups is ignored. In the procedure presented, the decomposition of level-1 variables is not altered by level-1 group membership, and therefore the dependency is appropriately taken into account among all individuals within the same cluster regardless of their level-1 group membership.
In testing factorial invariance in multilevel CFA, it is critical to distinguish at which level the group membership is. When the group membership is at level 2, multilevel factorial invariance can be tested using a simple extension of the standard procedure. Weak, strong, and strict invariance can be tested at level 2, and weak and strict invariance can be tested at level 1. Strong invariance is not of concern at level 1 because the mean structure is zero in the level-1 model. When the group membership is at level 1, multilevel factorial invariance can be tested using the procedure presented in this paper. The level-2 model is equivalent regardless of level-1 group membership. Weak, strong, and strict invariance can be tested in the level-1 model.
Factorial invariance is a question of whether the relationship between an observed measure and underlying latent factor is the same across different groups. In single-level confirmatory factor model with no cross-loading, one latent factor underlies each observed variable. The parameters (e.g., factor loadings, measurement intercepts) describe the relationships between observed measures and latent factors. In a multilevel confirmatory factor model with no cross-loading, typically two latent factors underlie each level-1 observed variable: one for the within-cluster random component and the other for the between-cluster random component of the level-1 observed variable. For example, in the two-level factor model shown in Figure 2, the observed variable Q31c is hypothesized to load onto the within-cluster factor SELFEFF_{W} and onto the between-cluster factor SELFEFF_{B}. For typical level-1 observed variables in multilevel data which have non-zero within-cluster and between-cluster variances, the relationship between the observed variables and underlying common factors consists of two parts. At level 1, the parameters describe the relationships between the within-cluster latent random components of the observed variables (e.g., Q31c_{W}) and common factor (e.g., SELFEFF_{W}). At level 2, the parameters describe the relationships between the between-cluster latent random components of the observed variables (e.g., Q31c_{B}) and common factor (e.g., SELFEFF_{B}).
When the group membership is at level 2, there are four possible outcomes: (a) an observed measure is measurement invariant (at the desired level of hierarchy) with both level-1 and level-2 common factors (e.g., Q31c is measurement invariant with both SELFEFF_{W} and SELFEFF_{B}); (b) a measure is measurement invariant with the level-1 common factor but not with the level-2 factor (e.g., Q31c is measurement invariant with SELFEFF_{W} but not with SELFEFF_{B}); (c) a measure is measurement invariant with the level-2 common factor but not with the level-1 factor (e.g., Q31c is measurement invariant with SELFEFF_{B} but not with SELFEFF_{W}); and (d) a measure is not measurement invariant with any of the underlying common factors. When the group membership is at level 1, the level-2 model is equivalent regardless of level-1 group membership. Therefore in this case there are two possible outcomes: a measure is measurement invariant with the level-1 common factor or not (e.g., Q31c is measurement invariant with SELFEFF_{W} or not). Since multilevel CFA assumes that the within-cluster and between-cluster models are uncorrelated, the required level of invariance depends on which latent factor selection or inference is made upon. Measurement invariance must be established at the corresponding level at which an inference is made. For example, in Figure 3, an inference made upon SELFEFF_{B} at school level would be invalidated by the non-invariance in measurement intercepts of Q31a_{B}, Q31 d_{B}, and Q31h_{B}. But the non-invariance in measurement intercepts for Q31a_{B}, Q31 d_{B}, and Q31 h_{B} would not be critical for the inference regarding SELFEFF_{W} at student level.
In conclusion, the procedure for testing multilevel factorial invariance across level-1 groups can be summarized as follows: the decomposition of level-1 variables should follow the clustered structure of multilevel data before the data are separated into level-1 groups; the level-2 model is equivalent regardless of level-1 group membership; the effective sample size should be used for the level-2 model; the level-1 model needs mean structure to represent the relative difference in means between level-1 groups; and weak, strong, and strict invariance can be tested in the level-1 model. The procedure presented can easily be implemented using standard SEM software packages. I hope this paper will motivate researchers and provide them with a viable tool to consider the issue of factorial invariance in multilevel CFA models.
- 1
Jak, Oort, and Dolan (2013b) present a test for the homogeneity of within-cluster measurement model across clusters in multilevel data (referred to as ‘cluster bias’ in their paper).
- 2
- 3
To avoid confusion, I use ‘^{M}group’ to indicate ‘groups’ in multiple group analysis for MUML estimation, ‘group’ to indicate the group membership between which the invariance is tested, and ‘cluster’ to indicate the level-2 unit in multilevel data.
- 4
Although the items were measured using 4-point scales, the distribution of the measures was not severely deviated from the normal. The skewness ranged from 0.003 to 0.609, and the excess kurtosis ranged from –0.831 to 0.316. The illustrated examples used MUML estimation to obtain an approximated ML solution assuming multivariate normality. When the normality assumption is violated, an alternative estimation method (e.g., maximum likelihood estimation with robust standard error [MLR]) is recommended.
- 5
There are alternative ways to identify the model. The interpretation of the results is easiest if the reference variable method is used. Alternative identifications do not affect the likelihood ratio statistic.
- 6
The reference variable method relies on the assumption that the factor loading for the reference variable is truly invariant across groups. In this example, the weak invariance model in which all factor loadings were constrained to be equal across groups was acceptable at both school and student levels. If weak invariance does not hold for any of the items, the test of invariance may yield different results depending on the choice of reference variable. In this case it is crucial that the loading for the reference variable is truly invariant across groups (see Cheung & Lau, 2012; Johnson & Meade, 2007; Johnson, Meade, DeVernet, 2009; Rensvold & Cheung, 1998).
- 7
CFI was obtained using a more restricted null model that is nested within the hypothesized model (Widaman & Thompson, 2003).
References
- Top of page
- Abstract
- 1. Introduction
- 2. Factorial invariance in single-level confirmatory factor analysis
- 3. Multilevel confirmatory factor analysis
- 4. Multilevel factorial invariance
- 5. Illustration
- 6. Summary and conclusion
- References
- Supporting Information
- 2008). Factorial invariance found in survey instrument measuring arthritis-related health beliefs among African-Americans and Whites. Journal of Clinical Epidemiology, 61, 289–294. doi:10.1016/j.jclinepi.2007.05.004 , , & (
- 2009). Cross-cultural invariance of the academic expectations stress inventory: Adolescent samples from Canada and Singapore. Journal of Adolescence, 32, 1225–1237. doi:10.1016/j.adolescence.2009.01.009 , , , , , , & (
- 2003). Satisfaction with Life Scale: Analysis of factorial invariance across sexes. Personality and Individual Differences, 35, 1255–1260. doi:10.1016/S0191-8869(02)00332-X , , & (
- 1980). Significance tests and goodness of fit in the analysis covariance structures. Psychological Bulletin, 88, 588–606. doi:10.1037/0033-2909.88.3.588 , & (
- 2012). A direct comparison approach for testing measurement invariance. Organizational Research Methods, 15, 167–198. doi:10.1177/1094428111421987 , & (
- 1999). Testing factorial invariance across groups: A reconceptualization and proposed new method. Journal of Management, 25, 1–27. doi:10.1177/014920639902500101nz , & (
- 2006). Evaluating multilevel models in cross-cultural research: An illustration with social axioms. Journal of Cross-Cultural Psychology, 37, 522–541. doi:10.1177/0022022106290476 , , & (
- 2012). Using a multilevel structural equation modeling approach to explain cross-cultural measurement noninvariance. Journal of Cross-Cultural Psychology, 43, 558–575. doi:10.1177/0022022112438397 , , , , & (
- 1997). Survey of attitudes toward statistics: Factor structure and factorial invariance for women and men. Structural Equation Modeling, 4, 129–141. doi:10.1080/10705519709540066 , , & (
- 2006). Factorial invariance of Woodcock-Johnson III scores for African Americans and Caucasian Americans. Journal of Psychoeducational Assessment, 24, 358–366. doi:10.1177/0734282906289595 , & (
- 1988). A general model for the analysis of multilevel data. Psychometrika, 53, 455–467. doi:10.1007/BF02294400 , & (
- 2008). Factorial invariance of the academic amotivation inventory (AAI) across gender and grade in a sample of Canadian high school students. Educational and Psychological Measurement, 68, 862–880. doi:10.1177/0013164407313368 , , , & (
- 2006). Do self-report instruments allow meaningful comparisons across diverse population groups? Testing measurement invariance using the confirmatory factor analysis framework. Medical Care, 44 (Suppl. 3), S78–S94. doi:10.1097/01.mlr.0000245454.12228.8f (
- 1993). Factor analysis of multilevel data: Gauging the Muthén model. In J. H. L. Oud & R. A. W. van Blokland-Vogelesang (Eds.), Advances in longitudinal and multivariate analysis in the behavioral sciences (pp. 141–156). Nijmegen: ITS. (
- 2010). Multilevel analysis: Techniques and applications (2nd ed.). New York: Routledge. (
- 2001). The accuracy of multilevel structural equation modeling with pseudobalanced groups and small samples. Structural Equation Modeling, 8, 157–174. doi:10.1207/S15328007SEM0802_1 , & (
- Measurement bias in multilevel data. Structural Equation Modeling. , , & . (2013a).
- A test for cluster bias: Detecting violations of measurement invariance across clusters in multilevel data. Structural Equation Modeling, 20, 265–282. doi:10.1080/10705511.2013.769392 , , & . (2013b).
- 2007). The role of referent indicator in tests of measurement invariance. Paper presented at the 22nd Annual Meeting of the Society for Industrial and Organizational Psychology, New York. , & . (
- 2009). The role of referent indicator in tests of measurement invariance. Structural Equation Modeling, 16, 642–657. doi:10.1080/10705510903206014 , , & (
- 2012). Testing factorial invariance in multilevel data: A Monte Carlo study. Structural Equation Modeling, 19, 250–267. doi:10.1080/10705511.2012.659623 , , & (
- 2009). Universals and specifics of math self-concept, math self-efficacy, and math anxiety across 41 PISA 2003 participating countries. Learning and Individual Differences, 19, 355–365. doi:10.1016/j.lindif.2008.10.009 (
- 1990). Multilevel analysis of structural equation models. Biometrika, 77, 763–772. doi:10.1093/biomet/77.4.763 (
- 1992). Factor analysis for clustered observations. Psychometrika, 57, 581–597. doi:10.1007/BF02294421 , & (
- 2005). Factorial validity of the Multidimensional Health Locus of Control scales for three American ethnic groups. Journal of Health Psychology, 10, 657–667. doi:10.1177/1359105305055311 , , & (
- 1989). Item bias and item response theory. International Journal of Educational Research, 13, 127–143. doi:10.1016/0883-0355(89)90002-5 (
- 1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58, 525–543. doi:10.1007/BF02294825 (
- 1992). On the misuse of manifest variables in the detection of measurement bias. Psychometrika, 57, 289–311. doi:10.1007/BF02294510 , & (
- 1997). Invariance in measurement and prediction: Their relationship in the single-factor case. Psychological Methods, 2, 248–260. doi:10.1037/1082-989X.2.3.248 (
- 1989). Latent variable modeling in heterogeneous populations. Psychometrika, 54, 557–585. doi:10.1007/BF02296397 (
- 1990). Mean and covariance structure analysis of hierarchical data (UCLA Statistics Series, No. 62). Los Angeles: University of California. . (
- 1994). Multilevel covariance structure analysis. Sociological Methods & Research, 22, 376–398. doi:10.1177/0049124194022003006 (
- 1997). Multilevel latent variable modeling in multiple populations. Unpublished manuscript. , , & . (
- 1995). Complex sample data in structural equation modeling. Sociological Methodology, 25, 267–316. doi:10.2307/271070 , & (
- 1998–2010). Mplus User's Guide. Sixth Edition. Los Angeles: Muthén & Muthén. , & . (
- OECD (2004). Learning for tomorrow's world: First results from PISA 2003. Paris, France: OECD.
- OECD (2005). PISA 2003 technical report. Paris, France: OECD.
- 2005). An illustration of multilevel factor analysis. Journal of Personality Assessment, 84, 126–136. doi:10.1207/s15327752jpa8402_02 , , , & (
- 1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin, 114, 553–566. doi:10.1037/0033-2909.114.3.552 , , & (
- 1998). Testing measurement models for factorial invariance: A systematic approach. Educational and Psychological Measurement, 58, 1017–1034. doi:10.1177/0013164498058006010 , & (
- 2009). Level-specific evaluation of model fit in multilevel structural equation modeling. Structural Equation Modeling, 16, 583–601. doi:10.1080/10705510903203466 , & (
- 2004). Job demands, job resources, and their relationship with burnout and engagement: A multi-sample study. Journal of Organizational Behavior, 25, 293–315. doi:10.1002/job.248 , & (
- 1998). Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research, 25, 78–107. doi:10.1086/209528 , & (
- 2003). Assessing the factorial invariance of Harter's Self-Concept measures: Comparing preadolescents with and without spina bifida using child, parent, and teacher report. Journal of Personality Assessment, 81, 111–122. doi:10.1207/S15327752JPA8102_03 , , , , , & (
- 2005). Confirmatory factor analysis and factorial invariance of the impact of information technology instrument. Omega, 33, 107–118. doi:10.1016/j.omega.2004.03.009 , , & (
- 1997). Exploring the measurement invariance of Book psychological instruments: Applications in the substance use domain. In K. J. Bryant, M. Windle & S. G. West (Eds.), The science of prevention: Methodological advances from alcohol and substance abuse research (pp. 281–324). Washington, DC: American Psychological Association. , & (
- 2003). On specifying the null model for incremental fit indices in structural equation modeling. Psychological Methods, 8, 16–37. doi:10.1037/1082-989X.8.1.16 , & (
- 2002). On normal theory based inference for multilevel models with distributional violations. Psychometrika, 67, 539–561. doi:10.1007/BF02295130 , & (
- 2003). Eight test statistics for multilevel structural equation models. Computational Statistics & Data Analysis, 44, 89–107. doi:10.1016/S0167-9473(02)00349-3 , & (
- 2005). On Muthén's maximum likelihood for two-level covariance structure models. Psychometrika, 70, 147–167. doi:10.1007/s11336-003-1070-8 , & (
- 2005). A two-level confirmatory factor analysis of a modified Rosenberg Self-Esteem Scale. Educational and Psychological Measurement, 65, 465–481. doi:10.1177/0013164404272487 , , & (
Supporting Information
- Top of page
- Abstract
- 1. Introduction
- 2. Factorial invariance in single-level confirmatory factor analysis
- 3. Multilevel confirmatory factor analysis
- 4. Multilevel factorial invariance
- 5. Illustration
- 6. Summary and conclusion
- References
- Supporting Information
Filename | Format | Size | Description |
---|---|---|---|
bmsp12014-sup-0001.pdf | application/PDF | 35K | Dataset 1: SAS macro to compute the input statistics for MUML estimation Dataset 2: Mplus syntax for testing factorial invariance in multilevel confirmatory factor analysis |
Please note: Wiley Blackwell is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.