Integrity tests are a special type of personality measure specifically designed for use in personnel selection. These tests can be considered the prototype of what Ones and Viswesvaran (2001) labelled “Criterion-Focused Occupational Personality Scales” (COPS). As opposed to standard personality tests designed to measure general or specific personality constructs, construction of COPS is not so much focused on measuring theoretically well-defined personality traits, but rather on predicting specific work-related criteria. Often, COPS do not measure homogeneous personality traits but rather a heterogeneous blend of different constructs sometimes referred to as compound traits (e.g., Hough & Schneider, 1996). As the definition of a compound trait is not usually based on established personality theory but depends on relations of the focal behavioural criterion to personality, a possible disadvantage of COPS is that their content is typically more difficult to interpret than the content of standard personality tests. This theoretical or interpretative disadvantage, however, may be outweighed for practical purposes if COPS can be shown to predict practically relevant criteria with higher or incremental validity beyond what could be obtained by using more theory-driven personality tests.
The present research tests this incremental validity for one type of COPS—integrity tests—as related to those tests' focal criterion, which is counterproductive or deviant behaviour. Counterproductive work behaviours (CWB) may be defined as volitional acts by members of an organization that may harm other members or the organization as a whole by violating their legitimate interests (e.g., Sackett & DeVore, 2001). Exemplary behaviours include theft and fraud, skipping work, working under the influence of intoxicants, or any kind of illegitimate aggressive act directed at people or things. We try to answer our research question based on an existing data set collected by Marcus, Lee, and Ashton (2007). Whereas Marcus et al. focused on a theoretical issue, the present re-analyses are aimed at contributing to knowledge about the practical utility of combining integrity and standard personality tests for predicting CWB. We elaborate on these issues further in the following section.
Integrity Tests' Incremental Criterion-Related Validity beyond Personality
The criterion-related validity of integrity tests has been the subject of extensive research. Two meta-analyses addressed those instruments' validity as related to both CWB and general performance criteria (Ones, Viswesvaran, & Schmidt, 1993; Van Iddekinge, Roth, Raymark, & Odle-Dusseau, 2012). Despite this substantial body of research, evidence is still inconclusive for providing practitioners with advice as to whether they should include integrity tests in their selection procedure given that there might be alternatives and that resources in terms of time and money are limited. In discussing this issue, we focused on CWB for two reasons. First, this criterion was found to be of universal relevance to practitioners (e.g., Rotundo & Sackett, 2002). Second, integrity tests are specifically designed to predict CWB and relatively few alternatives exist that could replace integrity tests for this purpose. Yet standard personality inventories may have the potential to represent such an alternative, and, as mentioned earlier, they may have the advantage of being easier to interpret than integrity tests. It is therefore important to establish the incremental validity of integrity beyond personality tests for that particular criterion. Previous research provided mainly indirect and only partial evidence with regard to this research question. Pieces of this evidence are reviewed next.
Meta-analyses of integrity tests indicate that validity for CWB criteria depends, among other things, the type of integrity test used. One type of integrity tests, referred to as overt (Sackett, Burris, & Callahan, 1989), mostly measures attitudes and sometimes even admissions of behaviour, which implies a more or less straightforward relation to the focal criterion. Another type of integrity tests, labelled personality-based by Sackett et al. (1989), is typically composed of standard personality test items or scales empirically keyed to predict CWB. This type therefore tends to be less overtly related to the focal criterion. Overall, overt integrity tests were more strongly related to CWB criteria than were personality-based tests in both Ones et al.'s (1993: ρ = .55 vs. .32) and in Van Iddekinge et al.'s (2012: ρ = .38 vs. .27) meta-analyses, yet validation studies also differed between test types with regard to design, setting, and breadth of criteria (Ones et al., 1993). As these variables also moderated the observed criterion-related validity, it is difficult to compare the figures for both types of integrity tests directly. The few studies providing for side-by-side comparisons of both types of integrity tests in the same sample report validities ranging from partial confirmations of the meta-analytic trend (Woolley & Hakstian, 1993), to virtually identical coefficients (Marcus, Schuler, Quell, & Hümpfner, 2002), to advantages for personality-based measures (Neuman & Baydoun, 1998).
With regard to incremental validity beyond standard personality tests, evidence is also mainly indirect. Two meta-analyses (Berry, Ones, & Sackett, 2007; Salgado, 2002) addressed the validity of dimensions of the five-factor model (FFM) of personality for predicting CWB. Whereas most values in these analyses were low to moderate, two coefficients in Berry et al.'s (2007) study were above .40 (agreeableness related to interpersonal CWB, and conscientiousness related to organizational CWB), exceeding the mean validity of personality-based integrity tests reported by both Ones et al. (1993) and Van Iddekinge et al., 2012. By contrast, in an earlier meta-analysis restricted to conscientiousness, Murphy and Lee (1994) concluded that the criterion-related validity of both types of integrity tests cannot be accounted for by that FFM dimension. In perhaps the first primary study comparing integrity and personality directly, Neuman and Baydoun (1998) reported stronger incremental validity of FFM dimensions beyond an overt than beyond a personality-based integrity test (ΔR2 = .18 vs. 07). Again, these findings appear inconsistent and are difficult to compare across studies.
Furthermore, findings from a number of studies (Lee, Ashton & de Vries, 2005a; Lee, Ashton, & Shin, 2005b; Marcus, Lee, & Ashton, 2007; Zettler & Hilbig, 2010) suggest that the FFM may not be ideal for assessing the full potential of basic personality dimensions for explaining CWB. These studies used an extension of the FFM—the HEXACO model of personality (e.g., Ashton & Lee, 2007)—which adds the dimension of Honesty-Humility (H-H) to rotated variants of the original FFM dimensions. Studies cited above consistently found H-H to outperform the other five dimensions in accounting for CWB. Lee et al. (2005a) also found H-H more strongly related to an overt integrity test than were the FFM dimensions (see also Marcus et al., 2007 for a discussion of this issue).
Marcus et al. (2007) confirmed this finding but also found a noteworthy difference between the two types of integrity tests. Whereas H-H added incremental criterion-related validity mainly beyond personality-based integrity, FFM dimensions added more validity beyond overt than beyond personality-based integrity tests (note that the latter finding is in-line with Neuman & Baydoun's, 1998, result reported earlier). These analyses addressed the theoretical research question asked by Marcus et al. (2007): What constructs account for the criterion-related validity of integrity tests? Their findings indicate that the criterion-related validity of the two types of integrity tests is partially due to overlaps with different personality traits. However, these analyses do not necessarily help practitioners in making necessary decisions: For predicting CWB, should I use integrity tests, personality tests, or both? If I use a combination of tests, which types should be part of the battery? Answering these questions would require moving beyond just the incremental validity of personality beyond integrity addressed in previous research. Rather, one would need to compare incremental validities of each procedure potentially used to predict CWB beyond that of alternatives (or supplements). This is the major objective of the present research, with special emphasis given to the practical significance of findings.
We re-analyzed Marcus et al.'s (2007) dataset in order to answer our research questions. This data set is particularly well-suited because its design allows ruling out a number of possible idiosyncrasies in the findings. Data were collected in two different countries (Canada and Germany). Moreover, two different integrity tests of each type (overt and personality-based) were used. One test of each type was originally developed in the German language and then translated into English, whereas the other test was translated from its original English version into German. In intercultural research, this type of design is sometimes referred to as “combined emic-etic” and considered an ideal standard for assessing culture-specific and interculturally transportable findings (Benet-Martinez, 2007). Finally, the standard personality measure used in the data set was the HEXACO-PI (Lee & Ashton, 2004), which allows for tests of incremental validity beyond FFM dimensions with or without the additional dimension of H-H.
As Marcus et al. (2007) have already established the incremental validity of personality beyond integrity in that data set, our first focus was on establishing the increment of integrity beyond personality, a topic not directly addressed in previous research. Unlike basic personality inventories, integrity tests are specifically designed to predict counterproductive behaviours. As with other COPS, construction of integrity tests involves sampling content only from criterion-relevant variance in broad personality dimensions, and dropping those facets expected to be unrelated to criteria (e.g., Marcus, Höft, & Riediger, 2006). Sampling out criterion-irrelevant trait variance should translate into higher criterion-related validity (e.g., Schneider, Hough, & Dunnette, 1996). We therefore expected the following relationship and tested this hypothesis by means of hierarchical regression analyses:
H 1. Integrity test scores account for variance in CWB beyond what is accounted for by basic personality traits.
In addition to this general expectation, we explored the validity of various combinations of alternative integrity and personality tests. Given that research established the particular relevance of the H-H dimension beyond HEXACO versions of the FFM dimensions of personality for explaining CWB, one would expect that increments of integrity tests beyond personality become smaller if H-H is added to the FFM before entering integrity. Moreover, differences between types of integrity tests reviewed above suggest that this detrimental effect of adding H-H on the incremental validity of integrity tests is stronger for the overt than for the personality-based type, as overt integrity tests are most closely related to H-H. Finally, we assessed, in an exploratory fashion, whether using the full set of predictors included in this study (i.e., both types of integrity tests plus a full range HEXACO assessment) was worth the additional cost for practical applications, or whether a subset of predictors could provide for a similarly valid yet more cost-effective alternative. With all these specific research questions, we focused on practical (i.e., effect sizes) rather than statistical significance. We therefore refrained from stating our expectations on specific effects as formal hypotheses.