Job Evaluation Tool (JET)
To identify critical work behaviors for managerial jobs across the globe, we used archival data from the Job Evaluation Tool (JET; Hogan Assessment Systems, 2009a). The JET represents one of the most extensively researched, reliable, and valid worker-oriented job analysis tools available (Foster, Gaddis, & Hogan, 2012). Included in the JET is the Competency Evaluation Tool (CET; Hogan Assessment Systems, 2009b), used to measure a taxonomy of work behaviors mapped to the Domain Model of Organizational Performance (R. Hogan & Warrenfeltz, 2003). This model provides a broad framework for organising work behaviors into four domains: (a) intrapersonal skills, including self-awareness, self-control, emotional maturity, and integrity; (b) interpersonal skills, including social skills, empathy, and developing and sustaining relationships; (c) leadership skills, including influencing, persuading, and building/maintaining a high performing team; and (d) business/technical skills, including planning, organising, and resource allocation. In the CET, Subject Matter Experts (SMEs)—high-performing incumbents or supervisors who provide a representative sample of occupational and demographic strata—are asked to indicate the extent to which each of 56 specific behaviors relates to performance for a target job. Raters evaluate each item using a 5-point scale ranging from 0 (Not associated with job performance) to 4 (Critical to job performance).
Hogan Development Survey (HDS)
The Hogan Development Survey (HDS; R. Hogan & Hogan, 2009) measures 11 dark side personality dimensions associated with job derailment. The 11 HDS scales and their definitions appear in Table 1.
Table 1. HDS Factors, Scales, and Definitions
|HDS Factor||HDS Scale||Concerns seeming …|
|Moving Away||Excitable||moody and inconsistent, being enthusiastic about new persons or projects and then becoming disappointed with them.|
|Skeptical||cynical, distrustful, overly sensitive to criticism, and questioning others' true intentions.|
|Cautious||resistant to change and reluctant to take even reasonable chances for fear of being evaluated negatively.|
|Reserved||socially withdrawn and lacking interest in or awareness of the feelings of others.|
|Leisurely||autonomous, indifferent to other people's requests, and becoming irritable when they persist.|
|Moving Against||Bold||unusually self-confident and, as a result, unwilling to admit mistakes or listen to advice, and unable to learn from experience.|
|Mischievous||to enjoy taking risks and testing the limits.|
|Colorful||expressive, dramatic, and wanting to be noticed.|
|Imaginative||to act and think in creative and sometimes unusual ways.|
|Moving Toward||Diligent||careful, precise, and critical of the performance of others.|
|Dutiful||eager to please, reliant on others for support, and reluctant to take independent action.|
A 1997 principal components analysis of the HDS scales resulted in the extraction of three global factors accounting for 62 per cent of variance in the matrix. The first factor, defined by the Excitable, Skeptical, Cautious, Reserved, and Leisurely scales, resembles the theme of “moving away from people” in Horney's (1950) model of flawed interpersonal tendencies. Together, these scales describe a pattern of managing one's insecurities by avoiding others. The second factor, including the Bold, Mischievous, Colorful, and Imaginative scales, corresponds with Horney's theme of “moving against people”, or managing self-doubt by dominating and intimidating others. Finally, the Diligent and Dutiful scales comprise Horney's theme of “moving toward people”, defined by managing one's insecurities by building alliances to minimise the perceived threat of criticism (R. Hogan & Hogan, 2009). This factor structure has been replicated multiple times using both US and global normative samples. The most recent results, obtained in 2008 using both Varimax and Direct Oblimin rotations, are available in the Hogan Development Survey manual (R. Hogan & Hogan, 2009).
The HDS is available in over 40 languages. To ensure that the assessment functions properly across languages and cultures, a rigorous adaptation process is followed (Hogan Assessment Systems, 2011b). This process begins with forward- and back-translations consistent with practices recommended in academic literature (e.g. Geisinger, 1994; Hambleton & Patsula, 1999; van de Vijver & Hambleton, 1996) and the International Test Commission's Test Translation and Adaptation Guidelines (Hambleton, 2001). This rigorous translation process ensures that translated items (a) read correctly in terms of grammar and syntax, (b) measure the same construct as original items, (c) possess cultural sensitivity and relevance, and (d) maintain the same strength of wording as original items (Hogan Assessment Systems, 2011b).
Following translations, equivalence analyses are conducted to compare English and translated forms (van de Vijver & Leung, 1997). First, Classical Test Theory bootstrapping analyses (Manly, 2007) are used to examine properties of items and scales. At the item level, endorsement rates and corrected item–total correlations for translated items are examined against 95 per cent confidence intervals from bootstrapped samples estimating true population parameters. For scale-level analyses, these methods are used to examine scale means and coefficient alpha reliability estimates. Item and scale parameters that fall outside these confidence intervals suggest that translation revisions may be needed. Together, these analyses ensure the functional equivalence of translated forms at the item and scale level.
Moreover, to verify the structural equivalence of scales, global factors, and the overall assessment, Procrustes analyses (Schönemann, 1966) are used to find congruence coefficients between English and translated forms of the HDS. With this analysis, researchers rotate a matrix of factor loadings from the translated assessment to assess similarity with a matrix of factor loadings from the original English assessment. Congruence coefficients are computed to determine the degree of similarity between the two structures, ranging from −1.0 (maximum inverse similarity) to +1.0 (maximum similarity). Some researchers (e.g. McCrae, Zonderman, Costa, & Bond, 1996; Mulaik, 1972) argue that a coefficient of .90 is indicative of acceptable congruence, with others (e.g. Lorenzo-Seva & ten Berge, 2006) suggesting that coefficients as low as .85 demonstrate fair similarity. Because this approach is widely applied for its ease of use and straightforward interpretation (e.g. McCrae & Terracciano, 2005; McCrae et al., 1996; Rodriguez-Fornells, Lorenzo-Seva, & Andres-Pueyo, 2001; Rolland, Parker, & Stumpf, 1998; Schmitt, Allik, McCrae, & Benet-Martinez, 2007), it provides a sound method for ensuring that the factor structure of the HDS remains intact across languages (Hogan Assessment Systems, 2011b).
Leaders can use the HDS to gain strategic self-awareness about how to improve their performance and relationships with others at work. The HDS measures characteristics that leaders often consider strengths. However, a manager may rely on these tendencies excessively when under stress, leading to detrimental effects on performance (Kaiser & Hogan, 2011). For example, a manager with a high score on the Diligent scale may view him- or herself as detail-oriented, but an over-reliance on these tendencies may lead his/her supervisor, co-workers, and subordinates to view these behaviors as nit-picky, perfectionistic, and micro-managing.
Research using the HDS illustrates its value in predicting performance. For example, in a sample of 290 incumbent managers, Benson and Campbell (2007) found negative relationships between elevated scores on Excitable, Skeptical, Cautious, Leisurely, Mischievous, and Imaginative scales and ratings of leader performance. In a multi-wave, multi-method longitudinal study of military cadets, Harms, Spain, and Hannah (2011a) found that scores on the Skeptical and Imaginative scales negatively predicted trajectories of leader development. In other words, young officers who questioned others' motives or were eccentric and odd in their approach to solving problems were rated poorly by superior officers across a variety of performance dimensions including judgment, communication, leadership, fairness, sense of duty, response to feedback, courage, conduct, unselfishness, army values, conscientiousness, and fitness. Some mixed results were also noted, with scores on other HDS scales (i.e. Cautious, Bold, Colorful, Diligent, Dutiful) relating positively with leader development over time and across multiple leadership dimensions. Researchers also note that the dark side personality dimensions measured by the HDS explain incremental variance in job performance beyond that explained by FFM personality measures (Harms, Spain, & Hannah, 2011b).