A Note on the Incremental Validity of Integrity Tests beyond Standard Personality Inventories for the Criterion of Counterproductive Behaviour

Abstract

Although the validity of integrity tests for predicting the focal criterion of counterproductive work behaviour (CWB) is well-established, little research has yet addressed the incremental CWB-related validity of integrity tests beyond basic personality traits. The present research addresses this issue by re-analyzing data adopted from Marcus, Lee, and Ashton (2007), in which four different overt and personality-based integrity tests and the HEXACO-Personality Inventory (Lee & Ashton, 2004) were related to CWB. Integrity accounted for practically significant proportions of incremental variance beyond personality across all integrity tests, yet effect sizes of incremental validity dropped considerably if Honesty-Humility was added to traditional Big Five dimensions. In addition, findings suggest that CWB is best predicted by a combination of integrity and personality tests. Copyright © 2013 ASAC. Published by John Wiley & Sons, Ltd.

Rèsumè

Même si la validité des tests d'intégrité utilisés pour prédire le critère focal du comportement contreproductif au travail (CWB) est bien établie, peu de travaux se sont penchés sur la validité incrémentielle des tests d'intégrité lies au CWB au-delà des traits de personnalité de base. La présente recherche examine ce problème en ré-analysant des données adoptées de Marcus, Lee et Ashton (2007) dans lesquelles quatre tests d'intégrité différents ouverts et basés sur la personnalité et l'inventaire de personnalité d'HEXACO (Lee & Ashton, 2004) étaient reliés au CWB. Dans tous ces tests d'intégrité, on s'aperçoit que l'intégrité rend bien compte d'une bonne partie de la variance incrémentielle au-delà de la personnalité. Cependant, les ampleurs de l'effet de la validité incrémentielle chutent considérablement lorsqu'on ajoute l'honnêteté l'humilité aux cinq principales dimensions traditionnelles. Les résultats indiquent aussi qu'une combinaison des tests d'intégrité et de personnalité prédit mieux le CWB. Copyright © 2013 ASAC. Published by John Wiley & Sons, Ltd.

Integrity tests are a special type of personality measure specifically designed for use in personnel selection. These tests can be considered the prototype of what Ones and Viswesvaran (2001) labelled “Criterion-Focused Occupational Personality Scales” (COPS). As opposed to standard personality tests designed to measure general or specific personality constructs, construction of COPS is not so much focused on measuring theoretically well-defined personality traits, but rather on predicting specific work-related criteria. Often, COPS do not measure homogeneous personality traits but rather a heterogeneous blend of different constructs sometimes referred to as compound traits (e.g., Hough & Schneider, 1996). As the definition of a compound trait is not usually based on established personality theory but depends on relations of the focal behavioural criterion to personality, a possible disadvantage of COPS is that their content is typically more difficult to interpret than the content of standard personality tests. This theoretical or interpretative disadvantage, however, may be outweighed for practical purposes if COPS can be shown to predict practically relevant criteria with higher or incremental validity beyond what could be obtained by using more theory-driven personality tests.

The present research tests this incremental validity for one type of COPS—integrity tests—as related to those tests' focal criterion, which is counterproductive or deviant behaviour. Counterproductive work behaviours (CWB) may be defined as volitional acts by members of an organization that may harm other members or the organization as a whole by violating their legitimate interests (e.g., Sackett & DeVore, 2001). Exemplary behaviours include theft and fraud, skipping work, working under the influence of intoxicants, or any kind of illegitimate aggressive act directed at people or things. We try to answer our research question based on an existing data set collected by Marcus, Lee, and Ashton (2007). Whereas Marcus et al. focused on a theoretical issue, the present re-analyses are aimed at contributing to knowledge about the practical utility of combining integrity and standard personality tests for predicting CWB. We elaborate on these issues further in the following section.

Integrity Tests' Incremental Criterion-Related Validity beyond Personality

The criterion-related validity of integrity tests has been the subject of extensive research. Two meta-analyses addressed those instruments' validity as related to both CWB and general performance criteria (Ones, Viswesvaran, & Schmidt, 1993; Van Iddekinge, Roth, Raymark, & Odle-Dusseau, 2012). Despite this substantial body of research, evidence is still inconclusive for providing practitioners with advice as to whether they should include integrity tests in their selection procedure given that there might be alternatives and that resources in terms of time and money are limited. In discussing this issue, we focused on CWB for two reasons. First, this criterion was found to be of universal relevance to practitioners (e.g., Rotundo & Sackett, 2002). Second, integrity tests are specifically designed to predict CWB and relatively few alternatives exist that could replace integrity tests for this purpose. Yet standard personality inventories may have the potential to represent such an alternative, and, as mentioned earlier, they may have the advantage of being easier to interpret than integrity tests. It is therefore important to establish the incremental validity of integrity beyond personality tests for that particular criterion. Previous research provided mainly indirect and only partial evidence with regard to this research question. Pieces of this evidence are reviewed next.

Meta-analyses of integrity tests indicate that validity for CWB criteria depends, among other things, the type of integrity test used. One type of integrity tests, referred to as overt (Sackett, Burris, & Callahan, 1989), mostly measures attitudes and sometimes even admissions of behaviour, which implies a more or less straightforward relation to the focal criterion. Another type of integrity tests, labelled personality-based by Sackett et al. (1989), is typically composed of standard personality test items or scales empirically keyed to predict CWB. This type therefore tends to be less overtly related to the focal criterion. Overall, overt integrity tests were more strongly related to CWB criteria than were personality-based tests in both Ones et al.'s (1993: ρ = .55 vs. .32) and in Van Iddekinge et al.'s (2012: ρ = .38 vs. .27) meta-analyses, yet validation studies also differed between test types with regard to design, setting, and breadth of criteria (Ones et al., 1993). As these variables also moderated the observed criterion-related validity, it is difficult to compare the figures for both types of integrity tests directly. The few studies providing for side-by-side comparisons of both types of integrity tests in the same sample report validities ranging from partial confirmations of the meta-analytic trend (Woolley & Hakstian, 1993), to virtually identical coefficients (Marcus, Schuler, Quell, & Hümpfner, 2002), to advantages for personality-based measures (Neuman & Baydoun, 1998).

With regard to incremental validity beyond standard personality tests, evidence is also mainly indirect. Two meta-analyses (Berry, Ones, & Sackett, 2007; Salgado, 2002) addressed the validity of dimensions of the five-factor model (FFM) of personality for predicting CWB. Whereas most values in these analyses were low to moderate, two coefficients in Berry et al.'s (2007) study were above .40 (agreeableness related to interpersonal CWB, and conscientiousness related to organizational CWB), exceeding the mean validity of personality-based integrity tests reported by both Ones et al. (1993) and Van Iddekinge et al., 2012. By contrast, in an earlier meta-analysis restricted to conscientiousness, Murphy and Lee (1994) concluded that the criterion-related validity of both types of integrity tests cannot be accounted for by that FFM dimension. In perhaps the first primary study comparing integrity and personality directly, Neuman and Baydoun (1998) reported stronger incremental validity of FFM dimensions beyond an overt than beyond a personality-based integrity test (ΔR² = .18 vs. 07). Again, these findings appear inconsistent and are difficult to compare across studies.

Furthermore, findings from a number of studies (Lee, Ashton & de Vries, 2005a; Lee, Ashton, & Shin, 2005b; Marcus, Lee, & Ashton, 2007; Zettler & Hilbig, 2010) suggest that the FFM may not be ideal for assessing the full potential of basic personality dimensions for explaining CWB. These studies used an extension of the FFM—the HEXACO model of personality (e.g., Ashton & Lee, 2007)—which adds the dimension of Honesty-Humility (H-H) to rotated variants of the original FFM dimensions. Studies cited above consistently found H-H to outperform the other five dimensions in accounting for CWB. Lee et al. (2005a) also found H-H more strongly related to an overt integrity test than were the FFM dimensions (see also Marcus et al., 2007 for a discussion of this issue).

Marcus et al. (2007) confirmed this finding but also found a noteworthy difference between the two types of integrity tests. Whereas H-H added incremental criterion-related validity mainly beyond personality-based integrity, FFM dimensions added more validity beyond overt than beyond personality-based integrity tests (note that the latter finding is in-line with Neuman & Baydoun's, 1998, result reported earlier). These analyses addressed the theoretical research question asked by Marcus et al. (2007): What constructs account for the criterion-related validity of integrity tests? Their findings indicate that the criterion-related validity of the two types of integrity tests is partially due to overlaps with different personality traits. However, these analyses do not necessarily help practitioners in making necessary decisions: For predicting CWB, should I use integrity tests, personality tests, or both? If I use a combination of tests, which types should be part of the battery? Answering these questions would require moving beyond just the incremental validity of personality beyond integrity addressed in previous research. Rather, one would need to compare incremental validities of each procedure potentially used to predict CWB beyond that of alternatives (or supplements). This is the major objective of the present research, with special emphasis given to the practical significance of findings.

Present Study

We re-analyzed Marcus et al.'s (2007) dataset in order to answer our research questions. This data set is particularly well-suited because its design allows ruling out a number of possible idiosyncrasies in the findings. Data were collected in two different countries (Canada and Germany). Moreover, two different integrity tests of each type (overt and personality-based) were used. One test of each type was originally developed in the German language and then translated into English, whereas the other test was translated from its original English version into German. In intercultural research, this type of design is sometimes referred to as “combined emic-etic” and considered an ideal standard for assessing culture-specific and interculturally transportable findings (Benet-Martinez, 2007). Finally, the standard personality measure used in the data set was the HEXACO-PI (Lee & Ashton, 2004), which allows for tests of incremental validity beyond FFM dimensions with or without the additional dimension of H-H.

As Marcus et al. (2007) have already established the incremental validity of personality beyond integrity in that data set, our first focus was on establishing the increment of integrity beyond personality, a topic not directly addressed in previous research. Unlike basic personality inventories, integrity tests are specifically designed to predict counterproductive behaviours. As with other COPS, construction of integrity tests involves sampling content only from criterion-relevant variance in broad personality dimensions, and dropping those facets expected to be unrelated to criteria (e.g., Marcus, Höft, & Riediger, 2006). Sampling out criterion-irrelevant trait variance should translate into higher criterion-related validity (e.g., Schneider, Hough, & Dunnette, 1996). We therefore expected the following relationship and tested this hypothesis by means of hierarchical regression analyses:

H 1. Integrity test scores account for variance in CWB beyond what is accounted for by basic personality traits.

In addition to this general expectation, we explored the validity of various combinations of alternative integrity and personality tests. Given that research established the particular relevance of the H-H dimension beyond HEXACO versions of the FFM dimensions of personality for explaining CWB, one would expect that increments of integrity tests beyond personality become smaller if H-H is added to the FFM before entering integrity. Moreover, differences between types of integrity tests reviewed above suggest that this detrimental effect of adding H-H on the incremental validity of integrity tests is stronger for the overt than for the personality-based type, as overt integrity tests are most closely related to H-H. Finally, we assessed, in an exploratory fashion, whether using the full set of predictors included in this study (i.e., both types of integrity tests plus a full range HEXACO assessment) was worth the additional cost for practical applications, or whether a subset of predictors could provide for a similarly valid yet more cost-effective alternative. With all these specific research questions, we focused on practical (i.e., effect sizes) rather than statistical significance. We therefore refrained from stating our expectations on specific effects as formal hypotheses.

Method

Sample and Procedure

Data were collected in the various subsamples in slightly different ways (see Marcus et al., 2007 for a detailed description). Canadian participants were (N = 171 mean age 21.5 years, SD = 4.7; 26 % men) undergraduates, all but one of whom reported having prior work experience and thus could be surveyed on CWB. In Germany, data collection took place in two universities in the former eastern and western part of the country by means of a snowball procedure. Undergraduate participants were asked to complete a survey and distribute questionnaires to employed acquaintances (see Marcus et al., 2007 for measures taken to avoid cheating with this procedure). Across the two German subsamples, 424 employees recruited by student participants, 145 students with prior work experience, and 113 undergraduates without work experience took part in the study. The latter subgroup was dropped for the present analyses. The German employee sample was 46% men and the German student sample was 18% men. Mean age was 36.8 years in East German employees (SD = 11.2) and 23.5 years in East German students (SD = 5.5; this item was inadvertently dropped in the West German survey). We collapsed data across subsamples for the present report in order to increase statistical power of our analyses.1

Measures

Integrity

All participants completed the Inventar Berufsbezogener Einstellungen und Selbsteinschätzungen (IBES, Marcus, 2006)—an integrity test originally developed in the German language (see Marcus et al., 2007 for references to validation data published in English). The IBES is comprised of a 60-item overt and a 55-item personality-based part. Both parts are modelled after prototypical North American integrity tests, but admissions of prior misconduct are not included in the IBES. Participants in Canada and in the West German subsample further completed the 80-item CPI-Cp (Hakstian, Farrell, & Tweed, 2002), a publicly available personality-based integrity test based on the content of the California Psychological Inventory. Finally, East German participants were administered a German translation of the public-domain overt Employee Integrity Inventory (EII, Ryan & Sackett, 1987), which is widely used in research. The admission subscale was dropped from the EII, leaving 52 items to complete. All translations were prepared in two-step procedures by teams of bilinguals including native speakers of both languages involved.

Personality

All participants took a 96-item version of the HEXACO Personality Inventory (HEXACO-PI, Lee & Ashton, 2004), which measures each of the six dimensions with 16 items. Dimensions of the HEXACO model include Honesty-Humility, Emotionality, eXtraversion, Agreeableness, Conscientiousness, and Openness to experience. The HEXACO dimensions of X, C, and O, all largely overlap with their FFM namesakes, whereas E and A represent slightly rotated variants of FFM Neuroticism and Agreeableness, respectively. The H-H dimension, however, is unique to the HEXACO model and includes facets of sincerity, fairness, greed-avoidance, and modesty. Translations of HEXACO-PI items into German were prepared using procedures similar to those used with integrity tests.

Counterproductive behaviour

CWB was measured by self-reports using the Workplace Behavior Questionnaire (WBQ, Ashton, 1998). The WBQ is a short (eight items) but rather broad measure of CWB,which includes absenteeism, lateness, alcohol use, safety violations, use of company resources for private purpose, theft, giving away merchandise, and vandalism. Note that the WBQ does not include interpersonal aspects of CWB such as bullying or harassment and that it was not constructed to be a strictly unidimensional measure; yet, a principal component analysis indicated a large drop in eigenvalues from the first (2.9) to the second (1.4) component. The internal-consistency reliabilities (Alpha) of the WBQ were .74 in the German sample and .81 in the Canadian sample.

Results

Results from hierarchical regression analyses involving the four different integrity tests are reported in Tables 1 and 2 (for a table containing study descriptives and bivariate correlations, see Marcus et al., 2007). In a first set of analyses reported in Table 1, HEXACO variants of FFM dimensions are entered in the first step, which was then followed by entering each of the four integrity measures. The second set of analyses reported in Table 2 differs from the first set in that all six HEXACO dimensions (including H-H) are entered in the first step. In both Tables 1 and 2, we further report effect sizes (see below) of incremental validity for the respective personality dimensions beyond integrity, thereby reversing the order of entry. These values allow for direct comparisons of incremental validities between different types of predictors.

Table 1. Hierarchical Regressions of CWB on Integrity and HEXACO-FFM Dimensions

Criterion	CWB
Integrity test	Overt		Personality-based
	IBES-o	EII	IBES-pb	CPI-Cp
Entries corresponding to single variables are standardized regression coefficients (βs). Effect sizes emphasized in the text are given boldface. ΔR^2 values for personality are corrected for shrinkage due to capitalization on chance with multiple predictors. CWB: counterproductive work behaviours; FFM: five-factor model of personality; Pers-b: personality-based * p < .05, ** p < .01, *** p < .001
Step 1: HEXACO FFM only
Emotionality	-.10**	-.14*	-.10**	-.11*
Extraversion	.08*	.01	.08*	.11*
Agreeableness	-.12**	-.16*	-.12**	-.10
Conscientiousness	-.25***	-.43***	-.26***	-.20***
Openness to experience	-.00	.06	-.00	-.05
R	.310***	.498***	.311***	.271***
R^2	.096	.248	.096	.073
Step 2: Integrity
Emotionality	-.01	-.07	-.09*	-.08
Extraversion	.08*	.00	.00	-.01
Agreeableness	.01	-.05	.04	.04
Conscientiousness	-.10**	-.23***	-.06*	.02
Openness to experience	.04	.05	-.02	-.05
Integrity	-.44***	-.50***	-.37***	-.48***
R	.486***	.663***	.407***	.479***
R^2	.236	.440	.166	.229
ΔR^2	.140***	.192***	.069***	.156***
Effect size for integrity (partial r^2)	.155	.255	.076	.168
ΔR^2 for personality (entered after integrity)	.013**	.040**	.011*	.002
Effect size for FFM (partial r^2)	.017	.065	.013	.003
N	665	204	665	462

Table 2. Hierarchical Regressions of CWB on Integrity and All HEXACO Dimensions

Criterion	CWB
Integrity test	Overt		Personality-based
	IBES-o	EII	IBES-pb	CPI-Cp
Note: See Table 1 for explanations.
Step 1: HEXACO All
Emotionality	-.06	-.10	-.06	-.06
Extraversion	.00	-.07	.00	.03
Agreeableness	.01	-.02	.02	.03
Conscientiousness	-.20***	-.34***	-.20***	-.16***
Openness to experience	.05	.09	.05	.03
Honesty-humility	-.41***	-.39***	-.41***	-.39***
R	.483***	.603***	.482***	.445***
R^2	.234	.364	.233	.198
Step 2: Integrity
Emotionality	-.02	-.07	-.06	-.06
Extraversion	.03	-.03	-.03	-.03
Agreeableness	.05	-.01	.10*	.08
Conscientiousness	-.12**	-.23***	-.09*	-.02
Openness to experience	.06	.06	.03	.00
Honesty-humility	-.27***	-.15*	-.35***	-.25***
Integrity	-.29***	-.41***	-.23***	-.36***
R	.527***	.672***	.505***	.520***
R^2	.278	.452	.255	.270
ΔR^2	.045***	.088***	.023***	.072***
Effect size for integrity (f^2)	.059	.138	.034	.090
ΔR^2 for personality (entered after integrity)	.055***	.050**	.100***	.042***
Effect size for FFM (partial r^2)	.070	.081	.117	.054
N	665	204	665	462

In formally testing H1, it is apparent that integrity tests accounted for statistically significant portions of criterion variance beyond personality in all eight regression models run. Thus H1 is consistently supported, although effect sizes for the incremental validity of integrity tests vary substantially across analyses. For the present purpose, these effect sizes are best indicated by Cohen's (1988) measure of partial r² = ΔR²_{step 2}/ (1 – R²_{step 1}), which is essentially a measure of the proportion of incremental criterion variance relative to residual criterion variance remaining from previous steps in the regression analysis. Unlike the more common ΔR² measure, partial r² is linearly related to effect size and therefore better suited to comparisons across different regression models (cf. Aiken & West, 1991). According to Cohen's (1988) conventions, partial r² values around .02 indicate small effects, values around .13 are considered moderate, and values around .26 are large.

The increments observed for single integrity tests beyond FFM dimensions varied between .08 and .26, thereby covering almost the full range from small to large effects. The sample-size weighted mean across these four analyses is .142, which indicates a medium-size effect. By contrast, increments of HEXACO-FFM dimensions beyond integrity do not even surpass the conventional value for a small effect (sample-size weighted mean partial r² = .017). If H-H is added to the FFM before entering integrity, incremental validity of integrity tests shrink to less than half of value in previous analyses (sample-size weighted mean partial r² = .066). With H-H included, the HEXACO-PI adds even slightly more beyond integrity (sample-size weighted mean partial r² = .083) than vice versa. In absolute terms, the drop in integrity tests' incremental validity due to the addition of H-H was in fact more substantial for overt (change in partial r² .096 and .117) than for personality-based tests (.042 and .078). Yet this conclusion is qualified if the drop is expressed as a percentage of previous incremental validities (62 and 46 % for overt, and 55 and 46 % for personality-based tests). Rather, it seems as if both types of integrity tests have substantially less incremental validity beyond personality if H-H is included in the personality measurement.

Discussion

Summary

The analyses presented in this paper provide for a direct assessment of integrity tests' incremental validity beyond the basic personality dimensions of the HEXACO model in accounting for the focal criterion of integrity tests—CWB. Four different integrity tests representing both the overt and the personality-based category of integrity each significantly accounted for incremental variance of counterproductive behaviours. Those increments, although still significant, were considerably smaller if the personality dimension of Honesty-Humility was added to FFM factors before entering integrity test scores. Notably, HEXACO variants of the FFM dimensions alone did not meaningfully contribute to CWB beyond integrity, whereas the inclusion of H-H led to even slightly higher incremental validities for personality than integrity tests. In this situation, both personality and integrity tests consistently produced incremental validities of small to moderate effect sizes, indicating that neither type of test could fully substitute the other in accounting for CWB.

Contributions to Scholarship

Although the criterion-related validity of integrity tests seems well-established since Ones et al.'s (1993) seminal meta-analysis, it is far less clear by now why integrity tests are criterion-valid. In particular, the nature of the construct (or the constructs) measured by integrity tests has been the subject of some debate. Issues discussed in this regard include the homogeneity (vs. heterogeneity) of the integrity construct, its degree of breadth relative to hierarchically structured models of personality, as well as the nature of the personality constructs underlying integrity that may explain integrity tests' validity in predicting external criteria (see Marcus et al., 2006, 2007 for reviews).

Although we focused on applied issues, our study contributes to this discussion beyond Marcus et al.'s (2007) analyses, which had addressed theoretical issues with the same data set. Whereas Marcus et al.'s major finding was that the validity of the two types of integrity tests in part stems from different personality traits, present re-analyses highlight the finding that integrity tests and personality inventories may be related to CWB for partially different reasons. Although there is an obviously large overlap between the two domains, both integrity and personality tests account for nontrivial proportions of CWB beyond each other. This is noteworthy because it implies that not only does personality not fully explain why integrity tests are valid (as noted by Marcus et al., 2007, among others), but also that integrity tests do not cover the full potential of personality in explaining CWB. As COPS are typically constructed with exactly that objective, this finding may point to some deficiency, or an overlooked factor, in the construction of integrity tests. Although such a deficiency may be specific to the very integrity tests used in this research, the fact that four different tests were used, all of which were developed with the aim of being prototypical for the integrity domain, renders this alternative explanation not very plausible.

Rather, present findings point to the conclusion that a key factor for further improving the validity of COPS aimed at predicting CWB may be sought within the personality domain of Honesty-Humility. It is true, however, that the two original FFM dimensions that were changed most substantially in constructing the HEXACO model of personality—namely Neuroticism and Agreeableness (cf. Lee & Ashton, 2004)—were also among the original FFM dimensions previously found most relevant for both CWB and integrity (e.g., Berry et al., 2007; Marcus et al., 2006). Moreover, some of the variance of H-H is captured by the FFM Agreeableness factor, and it is possible that this variance is particularly important in the prediction of deviant behaviours. Future studies may address this question more directly by including measures of both the original FFM dimensions and their HEXACO versions in the same study. Such studies may also further investigate the role of narrower personality facets, which is beyond the scope of the present research note.

Applied Implications

For potential users of integrity tests as well as personality tests, it is of vital interest to know whether use of these tests “pays off” in terms of the utility in light of possible costs and benefits. The present research did not address the costs, yet financial costs of paper-and-pencil tests generally tend to be insubstantial and outweighed by even small gains in validity (e.g., Schmidt & Hunter, 1998; of course, this is not meant to deny that there are nonfinancial costs to consider as well). The present research did address some major benefits, though. If our findings hold, HEXACO versions of the FFM dimensions do not add meaningfully to the prediction of CWB beyond integrity (but see above for a qualification with regard to original FFM factors), whereas increments of integrity beyond this version of the FFM tend to fall in the range of moderate to strong effects. Thus, integrity tests clearly outperformed the HEXACO-FFM in accounting for CWB, and employers are not advised to expect general personality inventories covering those dimensions to substitute for integrity tests. Yet this situation changes substantially if H-H is added to the FFM. In that case, increments of both integrity beyond personality and vice versa tended to be small to moderate. If either type of test (integrity or personality inventory) were used as stand-alone, both would provide for roughly the same validity yet would not account for CWB to the extent possible. If even small gains in validity paid off in terms of utility, the best advice for employers based on the present results thus seems to include both integrity and standard personality tests in their selection batteries. This advice is based on the assumptions that full scope personality inventories come as packages and that items may change their psychometric properties if the order or context of presentation is changed. If that were not the case, regression coefficients presented in Table 2 suggest that a combination of integrity with just H-H, complemented by Conscientiousness if overt integrity tests are used, may provide for an almost equally valid yet more cost-effective assessment of propensity to conduct CWB.

Of course, CWB is not the only criterion of interest to employers; choices of selection methods should be based on more comprehensive accounts of different facets of job performance. Early meta-analytic results (Ones et al., 1993) suggested that integrity tests outperform standard personality inventories (Barrick, Mount, & Judge, 2001) in predicting general job performance, but more recently, Van Iddekinge and colleagues (2012) reported meta-analytic evidence that earlier estimates may have overrated the validity of integrity tests for that criterion. As we do not have data to address this issue, we refrain from discussing this point in more detail, yet emphasize that it is an important point to consider for practitioners in addition to findings on CWB as the criterion. However, CWB appears important enough to justify inclusion of selection instruments shown to be valid predictors of it.

Limitations and Directions for Future Research

Perhaps the most obvious limitation of the present research is that all variables were measured with self-reports. Although inflated correlations due to same source bias are a common concern in organizational research (e.g., Podsakoff, MacKenzie, Podsakoff, & Lee 2003), evidence does not always support this concern (e.g., Spector, 2006). Recent meta-analytic evidence from CWB research suggests that patterns of results tend to be very similar for self-report and non-self-report criteria if otherwise similar designs are compared (Berry, Carpenter & Barratt, 2012), although the absolute strength of observed relations may vary due to low detection rates or possible content overlap between criteria and predictors (cf. Sackett et al., 1989; Van Iddekinge et al., 2012). In our study, there was no content overlap of integrity tests with criteria, as admission scales were not included on the predictor side. Moreover, we compared one type of self-report predictor (integrity) with another type of self-report predictor (personality), which implies that possible inflation of bivariate correlations, if any, should not translate into systematic distortion of multivariate patterns of results.

Furthermore, our measure of CWB, although broad in content, did not include CWBs directed at single persons rather than the organization as a whole. Future studies may broaden the scope on the criterion side to include all facets of CWB.

Another feature of the present research, which may limit generalizability to applied settings, is the fact that our participants were not applicants but incumbents of job or academic settings, which is related to the fact that our data were cross-sectional. Indeed, higher validities of integrity tests for incumbent than applicant samples with CWB criteria were found by both Ones et al. (1993: ρ = .54 vs. .44) and Van Iddekinge et al. (2012: ρ = .45 vs. .22), although these findings did not generalize to performance criteria. It is thus possible that our predictor-criterion relations may be somewhat inflated for both personality and integrity tests, yet there is again no reason to expect that multivariate patterns of results (i.e., incremental validity of one predictor set over the other) are systematically distorted.

We are therefore confident that the consistency with which we found incremental validity for a whole set of integrity tests in two different countries indicates that this finding will generalize to different conditions as well. However, replication with a full set of predictors (personality and integrity) in an actual applicant setting still remains one avenue for future research. Other issues not addressed in the present paper include the addition of original FFM dimensions to their HEXACO variants, or selection of narrower facets of personality to provide for potentially more parsimonious accounts of CWB. Finally, research that moves beyond traditional traits may be needed to fully understand what explains the criterion-related validity of overt integrity tests in particular. The current findings suggest that there is still something to discover beyond general dimensions of personality.

Endnotes

1. 1

   This practice also facilitates comparison with the study by Marcus et al. (2007), who also collapsed data across samples. However, an anonymous reviewer suggested that culture (Canada vs. Germany) may have moderated our results. We explored this possibility by comparing simple slopes and effect sizes (partial r², see results section) for three integrity tests across the two countries (the EII was administered only in Germany). If integrity was added to the full HEXACO model (including H-H), incremental validity was larger in Germany than in Canada for the IBES overt (β = −.44 vs. -.26; partial r² = .134 vs. .034), virtually identical for IBES personality-based (β = −.21 vs. -.18; partial r² = .025 vs. 020), and larger in Canada for the CPI-Cp (β = −.30 vs. -.43; partial r² = .060 vs. 113). Adding integrity to FFM dimensions only yielded similar results. These findings do not appear to follow a consistent pattern and are therefore not discussed further. Detailed findings may be obtained from the first author.