Self-report measures of personality have a number of theoretical and practical virtues (Paulhus & Vazire, 2007), and a substantial body of literature supports their predictive utility in both basic research and applied settings (e.g. Roberts et al., 2007). Nonetheless, concerns over the vulnerability of these instruments to distortion have been raised for over a half-century (Edwards, 1957). A sizeable literature has developed to identify those who might provide biased self-reports, as well as the contexts that promote bias in these measures. In their survey of the various efforts to identify biased responding in self-report, Paulhus and John (1998) argued that existing measures typically capture one or another of two distinct patterns of distortion. ‘Egoistic bias’ refers to the tendency to claim high social and intellectual status, whereas ‘moralistic bias’ refers to the tendency to deny socially deviant impulses and behaviours. Instruments assessing each of these tendencies measure individual differences in the degree to which a person is prone to a given manner of response. An assumption implicit to these instruments is that, at least within each category of bias (i.e. egoistic or moralistic), the pattern of biased response—that is, the traits that people exaggerate—will be the same from one individual to the next.
This implicit assumption may obscure an alternative source of bias. The present study investigates the role of individual differences in perceptions of trait desirability in producing bias in self-reported trait levels. Rather than expecting misrepresentation in self-report to follow only one of a limited set of patterns (egoistic or moralistic), we additionally expect individuals to self-report higher levels of traits to the extent that they view them as desirable. The conceptions of ‘desirable traits’ operationalized in existing measures were derived from aggregations of individual opinions on traits (e.g. Jackson & Messick, 1962); this literature appropriately refers to its topic as ‘socially desirable responding’ (SDR) because it is the average desirability of the trait in society that is thought to provide the motivation for distortion. Consistent with this terminology, we label the phenomenon studied here ‘idiographically desirable responding’ (IDR) to indicate that it is each individual's personal views of the relative desirability of traits that are expected to be an importantly distinct source of misrepresentation in their self-reported personality.
Whereas a vast, venerable, and growing literature explores SDR, investigations of anything resembling IDR have been much rarer. This may reflect a belief that few meaningful differences exist between individuals' perceptions of the desirability of traits. This belief has some basis, as mean inter-rater correlations for ratings of trait desirability typically surpass .6 (e.g. Krueger, 1996). Nevertheless, Sinha and Krueger (1998) found indications of possible influence of desirability ratings on the overclaiming of a given trait, showing that individuals who rated a trait as more desirable than did others also tended to claim higher levels of that trait. This study provided an important first step towards examining IDR, but could not resolve the question of whether this association truly represented bias; their results are equally consistent with a tendency of individuals to rate as more desirable the traits they actually possess. That SDR measures have long been shown to measure at least as much true variance as bias in self-rated traits (McCrae & Costa, 1983; Ones et al., 1996) highlights the challenges to interpreting any association between self-reported trait levels and any proposed method of identifying bias. What is needed to do so effectively is evidence from an external criterion.
Borkenau, Zaltauskas, and Leising (2009) carried out the only study we know of to use an external criterion in an exploration of IDR. This study departed from most previous work on the topic in two ways. First, instead of assessing the relative desirability of traits, it assessed perceptions of the optimal level of traits. Second, it investigated how individual differences in these perceptions predicted distortions in self-reported personality. The optimal level of traits (e.g. talkativeness and suspiciousness) was assessed on a 6-point Likert-type scale by each subject, resulting in an idiographic index labelled the perceived optimal trait level (POTL). Significant associations observed between these ratings and peer-reported trait levels indicated that differences in views of optimal trait levels were, in fact, associated with differences in actual personality. However, ratings of optimal trait levels also predicted variance in self-report trait levels not shared with peer report, indicating that POTL also represents a source of bias in self-report.
Borkenau et al.'s (2009) use of peer-reported personality levels as a criterion to indicate ‘true’ trait levels is consistent with recommended practice in the field (Paulhus & John, 1998) and has considerable empirical and conceptual support (Connelly & Ones, 2010). At the same time, self-reports and peer reports of personality are consistently shown to provide mutual incremental improvement in predictive validity (Connelly & Ones, 2010), indicating that at least some of the variance not shared between self-reports and peer report is reflective of true trait levels. Having established POTL as an indicator of true trait variance, the results of Borkenau et al. (2009) leave open the possibility that some (or even all) of the variance found in self-report but not peer report that is shared with POTL was in fact true trait variance. For this reason, the use of additional measures of ‘true’ trait levels would provide a more conclusive demonstration of the existence and import of IDR. While objective measures are infrequently used for traits such as extraversion and agreeableness, both SDR and IDR have obvious potential connections to attributes for which objective assessments are more widely employed, such as intelligence. Indeed, existing measures of SDR have already been shown to predict overclaiming of intelligence when compared to performance on an IQ test (Paulhus & John, 1998). The inclusion of such measures in addition to peer report would thus provide a more conclusive demonstration of IDR. For this reason, we included a performance test of intelligence in Study 2.
To quantify the extent of overclaiming present in self-reports, Paulhus and John (1998) suggested the use of self-criterion residuals (SCRs), in which bias is captured by the residual variance remaining after regressing a self-report measure on a criterion measure of the same variable. After the variance shared with the criterion has thereby been removed from the self-report, any variance due to biased responding is thought to remain in the residual. SCRs have been repeatedly used in studies of SDR measures (Paulhus et al., 2003; Pauls & Crost, 2003), and the relative effectiveness of SDR and IDR at identifying these residuals is therefore of interest. Further, such a comparison is needed to investigate the independence of SDR and IDR. The importance of IDR would be supported if it could predict variance in SCRs over and above measures of SDR.
Both SDR and IDR can be operationalized in several ways. A range of instruments assessing SDR have been developed in the past 50 years (Paulhus, 2002). With scales to assess both egoistic and moralistic bias, the Balanced Inventory of Desirable Responding (BIDR; Paulhus, 1991) is among the most used in recent years. Procedures to assess IDR are less well established. The optimal-trait-level approach employed by Borkenau et al. (2009) has much to recommend it, including the important insight that the desirability of most personality traits is not a strictly linear function. However, this approach does not have the ability to compare differences in the degree of desirability of different traits: two traits may be matched perfectly in POTL but differ markedly in the desirability of attaining that level. As SDR measures were typically developed with an eye towards the relative desirability of a given trait (e.g. Edwards, 1953), an IDR index based on individual differences in the perceived desirability of a trait may be the most direct method of comparison between SDR and IDR.
We have thus far stressed how individual differences in perceived trait desirability may lead to individual differences in which traits a given individual overclaims. We expect these effects to be evident among the population as a whole, with those who view a trait as particularly desirable the most likely to have overclaimed that trait. But there may also be individual differences in this tendency: that is, some individuals may be particularly prone to overclaim the traits they personally view as desirable.
Differences of this nature are plausibly connected with several extant constructs. For example, those who are particularly prone to overclaim the traits they personally view as desirable (IDR) may also tend to overclaim traits viewed as desirable by society at large (SDR). This would suggest a general willingness to overclaim desirable traits and would be supported if high scorers on SDR measures were also more prone to IDR.1 Further, just as those high in SDR tend to exhibit high self-esteem (Mar, DeYoung, Higgins, & Peterson, 2006), individual differences in IDR may show the same relationship, as those with a positive view of themselves may be particularly prone to believe they have the traits they view as desirable.
- Top of page
- STUDY 1
- STUDY 2
- GENERAL DISCUSSION
In Study 2, we looked to replicate and extend the results of Study 1. Whereas the latter had only one peer rater of trait levels, each trait in Study 2 was assessed with multiple raters, providing a more accurate picture of trait levels (Connelly & Ones, 2010). For a particularly rigorous test of IDR's ability to identify bias in self-report, we used an additional measure for one trait, assessing intelligence not only by peer report but also with an objective test. We also tested whether IDR's effects can be seen for traits outside of the mental domain by looking at physical attractiveness.
To establish the relative importance of bias due to IDR, its ability to predict overclaiming was compared against an established SDR measure. Finally, we explored the nature of individual differences in the tendency to engage in IDR. First, we hypothesized a connection between IDR and SDR, which would indicate the existence of a general willingness to claim desirable traits. Second, we hypothesized that those with a positive view of themselves would be more likely to believe they had the traits they viewed as most desirable, even to the point of overclaiming those traits.
Participants in this study were members of the Eugene–Springfield Community Sample (ESCS), a group that was recruited by mail from lists of homeowners in the Eugene–Springfield area. ESCS participants agreed to complete questionnaires delivered by mail over a period of 10 years beginning in 1994 in exchange for cash. In line with the community from which the sample was drawn, most participants identified as White (97%), with 1% or less identified as Hispanic, Asian American, and Native American, or declining to report their ethnicity. The sample is primarily middle aged (mean age = 51 years, SD = 12.36, range = 18–80) and reported a range of educational attainment, with a median of 2 years of post-secondary education completed. The measures used in the present study were assessed over a period of several years, reducing the total N available for analysis. We used all participants for whom data were available for self-report on personality and trait desirability, and two or more peer reports of personality. After excluding two participants whose responses indicated obvious inattention (all 100 trait desirability ratings were marked as ‘neither desirable nor undesirable’), our sample for the primary analyses was 541 (58% female), although a somewhat smaller portion of this pool had completed the additional measures of intelligence, SDR, and self-esteem. The 1520 individuals providing peer- report data were 61% male, ranging in age from 6 to 94 years (mean age = 48 years, SD = 17.81). Previous work on informant reports in this sample (DeYoung, 2006) showed no effect from removing younger raters from the sample, so the present analyses were conducted using all available peer reports.
Big Five personality traits were assessed with a mailing of the 44-item Big Five Inventory (John & Srivastava, 1999) and Saucier's (1994) 40-item Mini-Markers (SMM), with two items added to each instrument assessing physical attractiveness. These measures were also completed independently by two or three peers who knew the participants well and were asked to rate them. Both instruments were completed in Fall of 1998 using a Likert-type scale with answers ranging from 1 (strongly disagree) to 5 (strongly agree). Scores for Big Five personality and physical attractiveness for both self-report and peer report were obtained by taking the mean of all items for each trait from both the Big Five Inventory and SMM, yielding alphas between .84 and .94. As significant correlations are often found between IQ and Openness/Intellect (DeYoung, 2011), it is no surprise that many of the Openness/Intellect items have high face validity for assessing intelligence (e.g. ‘unintelligent’ and ‘perceptive’). A subset of Openness/Intellect items judged by the authors to be most directly relevant to IQ was selected, including Complex, Deep, Intellectual, Philosophical, Unintelligent, ‘Is ingenious, a deep thinker’, and ‘Likes to reflect, play with ideas’. Post-hoc analyses confirmed that mean peer report for each item correlated at .25 or above with objective IQ, and the resulting seven-item scale had alphas for self-report and peer report of .84 and .87, respectively.
To assess individual differences in beliefs about the value of certain traits, participants were asked to rate ‘how desirable or undesirable you feel it is for others to be or act this way’ for a list of 97 characteristics constituted primarily by personality traits. Responses were provided in a mailing in summer of 2001, with participants rating each characteristic on a 9-point Likert-type scale anchored by ‘very undesirable’ and ‘very desirable’. Twenty-five of these adjectives are included in SMM, but an additional 54 were shown to be associated with a Big Five trait in Saucier and Goldberg's (1996) analysis of the factor structure of personality adjectives. Scores for the desirability of Big Five traits were obtained by taking the mean of the adjectives associated in those reports with a given trait, yielding scales with the following properties: Openness/Intellect (11 items, alpha = .78), Conscientiousness (19 items, alpha = .85), Extraversion (17 items, alpha = .68), Agreeableness (21 items, alpha = .87), and Emotional Stability (11 items, alpha = .61). The desirability of physical attractiveness is indicated by a composite of the desirability of ‘attractive’ and ‘sexy’, which correlate .46. Finally, several of the items have high face validity for assessing IQ: the authors identified Complex, Insightful, Perceptive, Philosophical, and Unintelligent, with the resulting scale yielding an alpha of .66. Each of these came from the Openness/Intellect scale. Sixteen remaining adjectives that were assessed in this sample did not have loadings above .30 on any Big Five trait in Saucier and Goldberg's (1996) analysis and were thus not incorporated in the present study.
Participants in the ESCS completed Cattell's 16 Personality Factor Questionnaire (Conn & Rieke, 1994) in 1996. Factor B of this instrument is a 15-item performance test of intelligence consisting of knowledge and reasoning problems with multiple choice answers. Previous work has found the scale to correlate .57 with Full Scale IQ from the Wechsler Adult Intelligence Scale—Revised (Wechsler, 1981), supporting its use as an intelligence assessment (Abel & Brown, 1998).
Socially desirable responding
Tendencies towards SDR were assessed in the ESCS in 1998 with the BIDR (Paulhus, 1991). In this edition of the BIDR, two different dimensions of SDR are assessed: Impression Management (IM) and Self-Deceptive Enhancement (SDE). IM captures moralistic denial of socially deviant behaviours, whereas SDE measures egoistic overconfidence and claims to superiority. The alphas for these scales (IM = .82; SDE = .68) were comparable with that found in previous work (Li & Bagger, 2007).
The Rosenberg Self-Esteem Scale (Rosenberg, 1965) measures global feelings of self-worth and was administered in 1998 to ESCS participants with a 5-point Likert-type scale. The alpha reliability for this measure was .84 in the present sample.
Based on the rationale and methods described for Study 1, we obtained both WT-SCRs and WP-SCRs using peer report data for Big Five and attractiveness. For intelligence, we used objective performance as assessed by Cattell's Factor B in addition to peer report, as described further below. As noted earlier, WT-SCRs assess individual differences in the tendency to self-report trait levels differently than would be indicated by the criterion. In contrast, WP-SCRs identify differences within a person in the traits they most overclaim or underclaim.
The desirability ratings used in this study were also adjusted in the same fashion as Study 1 in order to reduce the role of response biases such as extreme responding (Paulhus & Vazire, 2007). We predicted the desirability for each trait from the desirability of the others in a regression and saved the residual, which thus represents the desirability of a given trait after removing the variance shared with the desirability ratings of other traits. (Because our assessment of IQ desirability derives entirely from Openness/Intellect items, it was not entered into the regression for other traits; its residual was thus obtained using all traits but Openness/Intellect as predictors.) It was these residualized desirability ratings that were then used in the analyses.
Trait desirability's association with single-criterion constructs
Replicating the results of Study 1, desirability ratings for the Big Five and attractiveness were correlated with self-report ratings for the same. As shown in Table 2, the associations between desirability ratings and their corresponding self-ratings were all highly significant, with a mean across the Big Five of .35. That is, the more an individual believed a given trait was desirable, the more of that trait he or she claimed to have. As with Study 1, these associations were highly specific, with a mean absolute value of Big Five cross-trait correlations of .05. Two such associations were significant at p < .05 (uncorrected for multiple testing) in both samples: those who viewed Openness/Intellect as desirable tended to report lower levels of Conscientiousness, whereas those who viewed Conscientiousness as desirable tended to report lower levels of Openness/Intellect. Outside the Big Five, beliefs about the desirability of attractiveness were modestly correlated with self-reported levels.
Table 2. Associations of trait desirability with trait levels and self-criterion residuals in adult community sample
| || ||Trait desirability|
| || || || || || || || |
| || || || || || || || |
The association between self-ratings of desirability and peer ratings of the target's personality is shown in Table 2 and was also highly significant, with a mean association across the Big Five of .24. The relatively low association found for Emotional Stability is consistent with previous work showing relatively low peer accuracy for this trait (Vazire, 2010), as any true variance in personality associated with desirability ratings for Emotional Stability may be harder to detect because peer reports are a less effective criterion for this trait than they are for others. It may also be influenced by the relatively low alpha (.61) noted earlier for our scale assessing the desirability of Emotional Stability. The mean absolute value of all Big Five cross-trait associations was .05, indicating that these associations were highly specific. Outside of the Big Five, peer reports of the target's attractiveness were uncorrelated with the target's perception of the desirability of attractiveness (.06, p > .1), although targets who assigned high desirability to Agreeableness were viewed as slightly more attractive (r = .18, p < .001). Interpretation of these results is difficult because of the low reliability of the two-item scale used to indicate the desirability of this trait.
As in Study 1, trait desirability ratings do not simply indicate true trait levels (indexed by the various criteria) but are also associated with systematic overclaiming of the trait levels involved. Table 2 shows the associations between trait desirability and WT-SCRs, with significant associations for all traits and a mean across the Big Five of .26. These associations were highly specific, with a mean absolute value of the cross-trait correlations of .03. One such association was significant at p < .05 (uncorrected for multiple testing) in both samples, with those viewing Emotional Stability as desirable tending to understate their level of Agreeableness. Outside of the Big Five, those who viewed physical attractiveness as desirable were more likely to overclaim the trait.
This sample was more heterogeneous than that used in Study 1, but controlling for participants' age, sex, and education produced no substantive changes to the pattern of results presented in Table 2.
Desirability and multiple-criteria constructs
As discussed earlier, SCRs represent any variance in self-reported personality not shared with the criterion and as such have been proposed as an indicator of the extent to which an individual overclaims or underclaims a trait (Paulhus & John, 1998). Nevertheless, as self-reports have been consistently shown to incrementally improve the predictive validity of peer report (Connelly & Ones, 2010), some of the variance in self-report that accurately reflects trait levels may not be shared with peer reports. To the extent that this is the case, some of the variance in SCRs represents true variance in trait levels. This represents a potential challenge to the interpretation of the association between desirability and SCRs as a demonstration for IDR. One way to surmount this challenge is to compare peer reports to a more objective criterion, which we did in the present context using the trait of intelligence, for which we have a performance-based measure as well as peer-report levels.
As shown in Table 3, intelligence behaves similarly to the traits discussed earlier: perceptions of the desirability of intelligence are associated with self-reported levels, true trait levels, and SCRs. Consistent with expectations, peer reports did not remove all true trait variance from self-reports: a WT-SCR created using peer report was modestly but significantly associated with performance on our intelligence measure (r = .16, p < .001). At the same time, trait desirability is significantly associated with a dual-criterion WT-SCR (which has had variance in self-report shares with peer report and with objective intelligence); in fact, this correlation is not smaller (p > .1) than was the correlation between desirability and the peer-report WT-SCR. Thus, the effects of IDR are not only present even when using multiple indicators of trait levels, but the addition of an objective measure of trait levels did not provide a significant reduction in the ability of trait desirability to predict overclaiming. The heavy reliance on peer-reported trait levels as criteria in studies of overclaiming (Paulhus et al., 2003; Pauls & Crost, 2003) thus appears to be well-justified, particularly given the difficulty of performance-based measurement of Big Five trait levels (Furr & Funder, 2007).
Table 3. Intelligence: Associations between desirability, trait levels, and self-criterion residuals (SCRs)
| ||Desirability||Self-rating||Peer rating|
|Self-rating||0.48|| || |
|Peer rating||0.38||0.53|| |
|SCR—peer and Cattell||0.27||X||X|
Socially desirable responding
The results reported earlier indicate that individual differences in perceptions of trait desirability are associated both with true differences in personality and with a tendency to overclaim traits viewed as desirable. Consistent with previous studies (Pauls & Stemmler, 2003), we found that two existing scales measuring SDR (IM and SDE; Paulhus, 1991) are associated with bias in personality self-report. As shown in Table 4, IM is associated with a tendency to overclaim these same characteristics as measured by WT-SCRs. SDE predicts a tendency to overclaim these traits as well as both Openness/Intellect and Intelligence. (We use the intelligence residual that removed the variance from both peer report and objective intelligence for the analyses presented in Tables 4 and 5.)
Table 4. Correlations of the Balanced Inventory of Desirable Responding scales with self-criterion residuals
| || ||Impression Management||Self-Deceptive Enhancement|
Table 5. Overclaiming predicted in multiple regression using Balanced Inventory of Desirable Responding (BIDR) scales and trait desirability
| || ||Beta||Sig|
Results from multiple regressions in which overclaiming of a given trait was predicted simultaneously by the desirability of that trait as well as IM and SDE are presented in Table 5. Desirability ratings remained significant predictors for their respective traits and were superior to IM and SDE in predicting overclaiming for Openness/Intellect, Extraversion, Agreeableness, and intelligence. To confirm the independence of IDR from SDR, we ran two stepwise multiple regressions for each trait. The first of these entered the BIDR measures first and then tested for improvements in R2 after entering trait desirability ratings. The second entered desirability ratings first and then tested for improvements in R2 after entering BIDR scores. In both of these analyses, all associations identified as significant in Table 5 produced significant improvements in R2, confirming the independence of SDR and IDR.
Individual differences in idiographically desirable responding
Participants in Study 1 varied in the extent to which desirability influenced a given participant's level of overclaiming or underclaiming trait levels. To replicate this result, we analyzed the associations between WP-SCRs with trait desirability levels within participants in Study 2. Within-subjects comparisons would not be adequately powered at the level of the Big Five, but as noted earlier, 25 of the 44 adjectives assessed in SMM were also included in the desirability assessment, with nearly equal representation across the Big Five (O:5, C:4, E:5, A:6, and ES:5). We obtained correlation coefficients for each participant that indicated the association between WP-SCRs and trait desirability levels for these traits. The mean and median correlations for the sample were both .26, with a considerable range around this value (range: −.41 to .87, SD: 0.21), replicating Study 1's finding of variation in the tendency to overclaim or underclaim traits based on desirability. Estimates of this value appear to be reliable, as indicated by a Spearman–Brown adjusted split-half reliability (based on odd-numbered and even-numbered items) of .60; this is lower than was found in Study 1, as might be expected based on the smaller number of traits available for analysis than in this sample.
Finally, this index of IDR-based bias was associated with existing constructs. High scorers also tended to be high in self-esteem (r = .17, p < .001) and to score highly on SDR (SDE r = .21, p < .001; IM r = .10, p < .05).