Does Conventional Implicit Association Test of Shyness Measure “Self‐Shyness” or “Others‐Shyness”?†
Abstract
The Implicit Association Test (IAT) measures implicit associations between attitude targets and attributes. Its structure and procedure facilitate investigation of the strength of associations between one target and attributes relative to that between the other target and the same attributes when two targets are contradictory (e.g., black/white and comfortable/uncomfortable). This structure can cause conceptual complexity about what the IAT measures, particularly when a counter category is not needed. Thus, using the Single‐Target Implicit Association Test (ST‐IAT), which allowed only one target category for pairing with attributes, this paper delineated the association measured in the conventional IAT for shyness: “self‐shy” or “others‐shy.” Seventy‐seven Japanese university students completed the self‐report shyness scale, the conventional IAT, and two ST‐IATs (i.e., self/others as target). Results showed that implicit shyness produced in the conventional IAT significantly and positively correlated with that in the self‐targeted ST‐IAT. Moreover, implicit shyness in the conventional IAT was significantly accounted for by those produced by the ST‐IAT with self as target and those calculated in the ST‐IAT with others as target in opposite directions.
Self‐report scales have long been adopted in countless psychological studies for fathoming individuals’ attitudes and personalities. However, due to limitations in self‐reporting, indirect measurements have been developed, such as the Implicit Association Test (IAT; Greenwald, McGhee, & Schwartz, 1998). The IAT has been utilized in various attitude and personality studies; however, a controversy exists regarding the concepts that it measures (Gawronski, LeBel, & Peters, 2007). Hence, the current study investigated the conceptual issue, specifically in the case of the IAT measuring shyness.
The IAT was initially developed by Greenwald et al. (1998) to indirectly and tacitly identify people's attitudes by having them perform a series of word categorization tasks. Participants take the test on a computer (for the paper‐format version of the IAT, see Lemm, Lane, Sattler, Khan, & Nosek, 2008). In the test, respondents are asked to classify each stimulus word appearing sequentially in the center of the screen to either the left or right category by pressing an assigned key on a keyboard. A left or right category is set a priori representing opposite meanings of certain targets (e.g., self or other), certain attributes (e.g., comfortable or uncomfortable), or both (e.g., self and comfortable or other and uncomfortable). In one IAT measuring self‐esteem, for instance, self and other are set as target categories while pleasant and unpleasant are set as attribute categories (Greenwald & Farnham, 2000). In the test, participants are instructed to respond as quickly and correctly as possible to gauge the strength of each target's association with attributes at an unconscious level. Specifically, in the main session, each target is paired with an attribute and another target is paired with the other attribute to form combined categories. In another main session, the matching pairs are reversed. For the self‐esteem IAT, for instance, participants respond to a combined task in which self and pleasant are presented together on one side of the screen whereas other and unpleasant are presented together on the other side, followed by a different task with the pairing reversed (i.e., self and unpleasant and other and pleasant). Participants are required to judge whether each stimulus word displayed at the center belongs to a left or right category. As the test is conducted in this controlled way, it is assumed that the shorter the response time, the more strongly each target is associated with the paired attribute. Response latency for each of the two combined tasks is compared to produce an IAT score. For the self‐esteem IAT, implicit self‐esteem is calculated as the relative speed in a combined task with self and pleasant as one superordinate category and other and unpleasant as the other and compared with that in the other reversed combined task. More importantly, when computed in this way, implicit self‐esteem is assumed to be a concept related to but different from explicit self‐esteem computed on a self‐report scale, but both suggest the more overarching concept of self‐esteem.
Other than the Self‐Esteem IAT, the test is applicable to a variety of attitude and personality studies, such as shyness. The Shyness IAT was initially proposed and developed by Asendorpf, Banse, and Mücke (2002). Their study successfully validated the IAT for the assessment of implicit shyness with German people. To measure shyness, the researchers prepared target categories of me and others, attribute categories of shy and non‐shy, and their relevant stimulus words. To demonstrate validity, especially predictive validity, they built a conceptual model called a double dissociation strategy between the implicit and explicit self‐concepts of shyness, and revealed that implicit shyness calculated by the indirect measurement predicted spontaneous (i.e., uncontrollable) shy behavior, such as tense body posture in a conversation with a stranger, whereas explicit shyness computed using the self‐report measurement predicted controllable shy behavior, such as shorter speech duration in the same situation. Namely, explicit and implicit shyness are each related to different aspects of shy behavior.
Aikawa and Fujii (2011) created a Japanese version of the test and revealed the validity with a Japanese sample. Instead of recording shy behavior expressed by participants, such as that recorded in Asendorpf et al.’s (2002) study, peer assessments of shyness‐related behavior of the participants were investigated to show the predictive validity. More specifically, explicit shyness computed by a self‐report scale predicted peer‐rated low praise‐seeking behavior (i.e., controllable shy behavior) whereas implicit shyness computed by the IAT predicted peer‐rated high interpersonal tension (i.e., spontaneous shy behavior), both of which were rated by the participants’ close friends. Fujii and Aikawa (2013) later replicated the predictive validity with a different Japanese sample. These results are conceptually consistent with Asendorpf et al.’s double dissociation model in that explicit shyness and implicit shyness are differently linked to each behavioral aspect.
The IATs described so far are sometimes controversial in terms of what association(s) or concept(s) are measured by the test. As discussed earlier, researchers need to prepare a pair of target categories to create the conventional IAT. For the self‐esteem or shyness IATs, basically, self (me) and other(s) comprise a pair of targets. On calculating IAT scores, one score is produced per participant for representing the implicit self‐concept. Thus, this score is possibly interpreted in two different ways: (a) strong association between one's own self and a certain attribute; and (b) weak association between other people and the identical attribute. In the case of implicit shyness, suppose that a person has responded to the Shyness IAT, resulting in a high implicit shyness score. There are two types of explanations to this outcome: the person may be unconsciously associating (a) himself/herself strongly with shyness in contrast to sociableness; and (b) others around him/her weakly with shyness as compared to sociableness. In either case, the person would score high on the IAT for implicit shyness. It is worth noting that the latter possibility can be problematic if researchers are interested in participants’ own self‐concept, which is not influenced by their evaluation of others’ level of the same personality trait. The current paper addresses this issue.
Single‐Target IAT
A couple of measurements have been advanced to deal with this conceptual issue. To exclude a counter category that is not always necessary, one recent paper proposed a derivative of the conventional IAT, that is, the Single‐Target IAT (ST‐IAT; Bluemke & Friese, 2008), to measure a single association between a target and an attribute. To measure implicit shyness involving only one's own self, self is set as a target category to be combined with shy in one session and non‐shy (or sociable) in another session.
Other attempts to identify a single association include the Go/No‐Go Association Task (Nosek & Banaji, 2001), Evaluative‐Priming tasks (Fazio, Sanbonmatsu, Powell, & Kardes, 1986), and the Single‐Category IAT (SC‐IAT: Karpinski & Steinman, 2006). In order to disassemble combined associations in the conventional IAT into “self‐shy” and “others‐shy” associations, the ST‐IAT was adopted for the current investigation because of structural and procedural similarity with the conventional IAT. Note that the SC‐IAT is conceptually equivalent to the ST‐IAT, and there is essentially no difference in the procedure between them except the following features (Bluemke & Friese, 2008; Karpinski & Steinman, 2006). First, the SC‐IAT, but not the ST‐IAT, sets a maximum response time that facilitates a sense of urgency for quick response. Second, the ST‐IAT, and not the SC‐IAT, includes an initial practice stage with only attribute words. Third, the ST‐IAT has fewer stimulus words in each block than the SC‐IAT. To ease participants’ burden, we decided to adopt ST‐IATs.
Purpose of the Present Study
The current study attempted to cast new light on the conceptual complexity inherent in the conventional IAT, particularly in the Shyness IAT. Specifically, we tried to decompose Shyness IAT scores into one part related to the self and the other related to others.
Concerning the relation between the IAT and the ST‐IAT, one large‐scale study compared seven indirect attitude measures (Bar‐Anan & Nosek, 2014). When measuring self‐esteem, their study revealed a significantly positive correlation between the IAT and the self‐targeted ST‐IAT (r = 21). Regarding shyness as well, we first expected that the IAT scores computed traditionally could be related at least to those with the self‐targeted ST‐IAT. Second, given the structure of the conventional IAT mixed between self and others, we expected that the IAT scores would be predicted by those computed using the two ST‐IATs with self and others as targets, and explored how they differed in their predictive value. In addition, for an explorative purpose, a self‐report scale of shyness was also included to investigate its relationship with each implicit shyness score.
Method
Participants
Seventy‐seven Japanese university students (41 males, 33 females, three unreported sex; Mage = 19.41 years, SDage = 1.98 years) participated in the current study. All participants were recruited through class sessions at their university. No data were excluded in the following analyses.
Materials
Data collection was all computerized with an online program comprising the following scale and tests. For IATs, we used Inquisit 5 Web provided by Millisecond Software.
Conventional IAT
We adopted a Japanese version of the Shyness IAT developed and validated in previous studies (Aikawa & Fujii, 2011; Fujii & Aikawa, 2013). It consisted of seven blocks, and in each block participants were instructed to respond as quickly and correctly as they could. They were asked to categorize every stimulus word that appeared sequentially at the center of the screen into a left or right category by pressing an assigned key based on its definitional meaning. Each time participants made incorrect responses, a red X was displayed at the center until they hit the correct key. Stimulus words used in this study were the same as those reported in Aikawa and Fujii (2011). Specifically, five words were used for each category; all of these are presented in Table 1.
| Self | Others | Shy | Sociable |
|---|---|---|---|
| Jibun | Yuujin | Uchiki na (Bashful) | Jishin no aru (Assured) |
| Jishin | Chijin | Hikaeme na (Moderate) | Susunde suru (Voluntary) |
| Watashi | Tanin | Tamerai gachi no (Hesitant) | Daitan na (Adventurous) |
| Wareware | Shiriai | Mukuchi na (Reticent) | Enryo no nai (Uninhibited) |
| Onore | Tomodachi | Enryo gachi na (Modest) | Uchitoketa (Familiar) |
- Note. The stimuli for self and others are presented in Japanese pronunciation because their subtle nuances cannot be clearly explained in English; they indicate either I/me, we/us, others, acquaintance(s), or friend(s).
The overview of the IAT is summarized in Table 2. In responding, the F and J keys on a keyboard were assigned to indicate left and right categories respectively. In Block 1, self and others were assigned as targets to the left and right categories, respectively, and participants were asked to judge whether each stimulus word belonged to the left or right category for 20 trials. In Block 2, shy and sociable attributes were, respectively, assigned to the left and right categories, and each stimulus word was to be categorized into the left or right category for 20 trials. In Block 3, self paired with shy was assigned to the left category while others combined with sociable was assigned to the right, and participants classified each stimulus word to the left or right category. After practicing the combined task for 20 trials in this block, participants proceeded to a main test session of Block 4 with the same key assignments in which they performed the combined task for 40 trials. In Block 5, the placement of self and others targets in Block 1 were switched: self in the right category and others in the left category. Participants classified stimulus words for 20 trials in this block. In the next session, Block 6 served as a practice with 20 trials. In this block, the matching of categories as in Block 3 was switched: others and shy shared the left category whereas self and sociable shared the right. In Block 7, participants classified stimulus words using the same criteria as in the preceding session for another 40 trials. The order of presentation was counterbalanced.
| Block | Task description | Left key assignment | Right key assignment | Number of stimuli |
|---|---|---|---|---|
| 1 | Target practice | Self | Others | 20 |
| 2 | Attribute practice | Shy | Sociable | 20 |
| 3 | Pair practice | Self + Shy | Others + Sociable | 20 |
| 4 | Pair test | Self + Shy | Others + Sociable | 40 |
| 5 | Target practice | Others | Self | 20 |
| 6 | Pair practice (reversed) | Others + Shy | Self + Sociable | 20 |
| 7 | Pair test (reversed) | Others + Shy | Self + Sociable | 40 |
- Note. To counterbalance the task order, approximately half the participants followed this procedure whereas the other half underwent the test in the following order: Blocks 5, 2, 6, 7, 1, 3, and 4.
As a basic procedure for calculating IAT scores, each participant's average response time in Blocks 3 and 4 was compared with that in Blocks 6 and 7. Shorter response time indicates a stronger association between a target category and an attribute category that shared the same position. For instance, a participant with shorter response latency in Blocks 3 and 4 (e.g., self paired with shy and others paired with sociable) than that in Blocks 6 and 7 (e.g., self paired with sociable and others paired with shy) would be assessed to have stronger associations between paired categories in Blocks 3 and 4 (e.g., higher implicit shyness).
Single‐Target IATs
Two ST‐IATs were used to measure a single association between one's own self and shyness/sociableness and that between others and shyness/sociableness. One ST‐IAT with self as target had five blocks (see Table 3). In Block 1, participants classified stimulus words into the left category of shy or the right category of sociable for 20 trials, identical to Block 2 in the conventional IAT. In Blocks 2 and 3, self and shy shared the left category whereas merely sociable was on the right. Twenty and 40 trials were conducted in Blocks 2 and 3, respectively. The pairing was changed in Blocks 4 and 5. In Block 4, stimulus words were classified into the left category of shy or the right and combined category of self + sociable for 20 trials. In Block 5, participants did the same categorization task as in Block 4 for 40 trials.
| Block | Task description | Left key assignment | Right key assignment | Number of stimuli |
|---|---|---|---|---|
| 1 | Attribute practice | Shy | Sociable | 20 |
| 2 | Pair practice | Self + Shy | Sociable | 20 |
| 3 | Pair test | Self + Shy | Sociable | 40 |
| 4 | Pair practice (reversed) | Shy | Self + Sociable | 20 |
| 5 | Pair test (reversed) | Shy | Self + Sociable | 40 |
- Note. To counterbalance the task order, approximately half the participants followed this procedure whereas the other half underwent the test in the following order: Blocks 1, 4, 5, 2, and 3.
In addition to the self‐targeted ST‐IAT, one more ST‐IAT was formed with others as target. In creating the test, self in the self‐targeted ST‐IAT was replaced with others with the other structure constant, except for stimulus words belonging to self or others. The order of presentation was counterbalanced. For calculating each ST‐IAT score, each participant's average response time in Blocks 2 and 3 was compared with that in Blocks 4 and 5 (see Results for more detail).
Trait Shyness Scale
The 16‐item Trait Shyness Scale (TSS; Aikawa, 1991) was used. Participants indicated the degree of agreement with statements such as “I am a bashful person” (Watashi wa hanikamiya de aru) and, for a reverse‐scoring item, “I can make friends soon” (Watashi wa atarashii yuujin ga sugu dekiru) on a 5‐point Likert scale ranging from 1 (not at all) to 5 (very much).
Procedure
All items were integrated into an online program in which the sequence was as follows: ST‐IATs, conventional IAT, and TSS.44 For the task order of the conventional IAT and ST‐IAT, Karpinski and Steinman (2006) advised against conducting the dual‐targeted IAT first—although in their paper the SC‐IAT was used—out of concern that participants doing so may carry over this mindset to any succeeding ST‐IAT even if a counter category were not displayed in the test.
Counterbalancing was done for the task order of (a) self and shy (or others and sociable) combined first versus self and sociable (or others and shy) combined first in the conventional Shyness IAT, (b) target and shy combined first versus target and sociable combined first in the two Shyness ST‐IATs, and (c) ST‐IAT with self as target first or ST‐IAT with others as target first. It took approximately 15 to 25 min to complete the computerized package.
Although all participants accessed a designated URL to start the program, the study was conducted in two ways. For one sample, participants in one class were notified of the URL by the class instructor and the program was started simultaneously by each student on desktop computers in the classroom. Participants in the other sample in a different class were informed of the study; those who contacted us later via email were given the URL and they completed the program at their convenience. All participants were informed about the voluntary participation and received a token equivalent to 500 Japanese yen. On completing the program, we corresponded with them via email to thank and debrief them.
Results
Data Processing
D scores were calculated for all IATs following the algorithm recommended in Greenwald, Nosek, and Banaji (2003) to represent the respective personality. For this, average response latencies and other relevant variables, such as standard deviations of response time in Blocks 3, 4, 6, and 7 of the conventional IAT, were considered for producing implicit shyness scores. Note that, as Greenwald et al.’s (2003) study revealed, including practice blocks in the calculation would produce better psychometric properties. For the conventional IAT, computed scores indicate the degree to which a self‐shy (or others‐sociable) association was stronger than a self‐sociable (or others‐shy) association. For the two ST‐IATs, the same algorithm was applied to Blocks 2 through 5 to compute implicit shyness scores. For the self‐targeted ST‐IAT, the scores represent the extent to which a self‐shy association was stronger than a self‐sociable association. This was also the case in the others‐targeted ST‐IAT. The internal reliability of the TSS was good (Cronbach's alpha = .85); average scores were used for self‐reported shyness while reverse scores were used where appropriate.
Correlation Analyses and Regression Analysis
We computed all possible correlation coefficients and descriptive statistics for all variables as shown in Table 4. Regarding intercorrelations among indirect measures, as hypothesized, a significantly positive correlation was found for the relation between implicit shyness computed using the conventional IAT and the ST‐IAT with self as target (r = .29, p < .01). The other correlations were not significant: r = −.17, p = .14, for the relationship between implicit shyness of the conventional IAT and the ST‐IAT with others as target; and r = .09, ns, for the relationship between the two ST‐IATs.
| 2 | 3 | 4 | M | SD | |
|---|---|---|---|---|---|
| 1. TSS | .13 | .01 | −.09 | 2.98 | 0.75 |
| 2. Conventional Shyness IAT | — | .29** p < .01. |
−.17 | −0.26** p < .01. |
0.41 |
| 3. Shyness ST‐IAT (Self) | — | .09 | −0.12** p < .01. |
0.30 | |
| 4. Shyness ST‐IAT (Others) | — | −0.04 | 0.31 |
- Note. Asterisks for mean scores indicate significant differences from their theoretical midpoints.
- * p < .01.
In our regression analysis, implicit shyness on the conventional IAT was regressed onto two types of implicit shyness on the ST‐IATs. As seen in Table 5, the regression model was significant, F(2, 74) = 5.34, R2 = .13, adjusted R2 = .10. More interestingly, implicit shyness on the self‐targeted ST‐IAT significantly and positively predicted implicit shyness on the conventional IAT involving both self and others as targets (β = .31, p < .01), whereas implicit shyness computed on the others‐targeted ST‐IAT only marginally and negatively predicted the same variable (β = −.20, p = .08). This regression analysis shows that implicit shyness scores calculated on the conventional dual‐target IAT could be attributed both to self‐ and others‐targeted components of implicit shyness but at the same time, the predictive value of each type of ST‐IAT score was mutually contradictory.
Relationships With Explicit Shyness and Ancillary Analyses
Explicit shyness on the TSS was not significantly correlated with any measure of implicit shyness: r = .13, ns, r = .01, ns, r = −.09, ns, for the conventional IAT, self‐targeted ST‐IAT, and others‐targeted ST‐IAT, respectively. As ancillary analyses, t tests performed to see if each shyness score departed from a theoretical midpoint of zero revealed that on the conventional IAT and the self‐targeted ST‐IAT, it was significantly lower than zero, t(76) = 5.42, p < .01, and t(76) = 3.46, p < .01, respectively. However, this was not true for implicit shyness on the others‐targeted ST‐IAT or for explicit shyness, t(76) = 1.20, ns, and t(76) = 0.21, ns, respectively. To summarize, these t tests all indicate that the association between self and shy was weaker than that between self and sociable in each Shyness IAT involving self.
Discussion
This study aimed to clarify the conceptual complexity in IAT scores. For this, by focusing on shyness in IAT, the conventional IAT and two of its derivatives were administered, and the resulting scores were correlated.
Relationships Among Implicit Shyness Measures
The results showed that implicit shyness calculated with the self‐targeted ST‐IAT was not significantly correlated with implicit shyness as assessed using the others‐targeted ST‐IAT. Thus, it can be speculated that, at an unconscious level, how people link their own self with a shyness‐sociable dimension is independent from how they perceive other people to be shy or sociable, suggesting discriminant validity of each ST‐IAT.
In the subsequent analysis, implicit shyness scores on the conventional IAT were successfully regressed onto two types of implicit shyness scores created on the two ST‐IATs. Thus, it is fair to infer that both components involving one's own self and ideas about others can contribute to the original scores. Simultaneously, the predictive value of the conceptual pair self‐shy/sociable seemed a little stronger than that of the conceptual pair others‐shy/sociable and they were mutually contradictory. To extend this inference, the scoring of the conventionally structured IAT for identifying other self‐concepts may be more influenced by an association of one's own self with the concept than by an association of other people with the same concept, and these directions may be opposite.
Correlation of Explicit Shyness With Implicit Shyness
The correlation between explicit shyness on the self‐report scale and implicit shyness on the conventional IAT was weak, consistent with previous findings (r = .12 in Aikawa & Fujii, 2011; r = .23 in Fujii, Sawaumi, & Aikawa, 2015b). This is also true for the correlation between explicit shyness and implicit shyness on the self‐targeted ST‐IAT (r = −.04 in Fujii, Sawaumi, & Aikawa, 2015a). There has been no similar previous research upon others‐targeted IAT so far, making us unable to discuss any replication issue. Hence, the current data replicated the correlation between implicit shyness on the IATs involving self and self‐reported explicit shyness.
Explicit attitudes and personalities are reported to differ greatly from implicit counterparts in some studies, but not in other studies. The explicit–implicit relations vary with multiple factors (e.g., self‐presentation as a moderator of the relation; Nosek, 2007). Future research should investigate the relationship between explicit shyness and implicit shyness in a broader context, such as experimental studies in which self‐presentation motivation is manipulated.
Mean Scores Deviating From Zero
Implicit shyness scores on the conventional IAT and those computed using the self‐targeted ST‐IAT were significantly distant from zero unlike scores on the others‐targeted ST‐IAT or self‐reported shyness. This suggests that the Japanese may not be as shy as previously assumed (Aizawa & Whatley, 2006), at least on an unconscious level. As Nosek (2007) summarized, social motives can moderate explicit–implicit relations: Hence, Japanese may self‐report a level of shyness in a way that conforms to Japanese norms, one of which is maintaining group harmony (Aizawa & Whatley, 2006). For that purpose, remaining silent and holding back their individual opinions—namely, appearing shy—is encouraged. This line of argument suggests that showing their shyness can at times be functional and preferred in Japanese culture. Thus, it is likely that Japanese may self‐report their level of shyness as being higher than how shy they actually feel themselves to be. When it comes to associating other people with shyness or sociableness (i.e., others‐targeted ST‐IAT), such tendency is not suggested in either direction. To advance personality research further, as in shyness research, it is beneficial to apply both explicit and implicit measurements to tap into people's personalities.
Future Directions
We suggest three future directions. First, it is worth noting that the study assumes a single attribute dimension from shyness to sociableness. There has not been any definitive answer as to whether being shy is contradictory to being sociable. In the original study of the Shyness IAT (Asendorpf et al., 2002), the shy–non‐shy distinction comprises an attribute spectrum whereas the Japanese version adopts shy and sociable as an attribute dimension for linguistic reasons. Researchers might want to explore any differences in the labeling and any possible effects on the resulting scores.
Second, as discussed earlier, the IAT has been used to explore people's unconscious attitudes and self‐concepts. Not only shyness but also other aspects of personality, such as self‐esteem (Greenwald & Farnham, 2000) and anxiety (Egloff & Schmukle, 2002), have been identified by the indirect measurement. For concepts other than shyness, future studies can examine how each IAT score is composed.
Third, to capitalize on the nature of the ST‐IAT, researchers might want to integrate a couple of IATs and ST‐IATs to better predict relevant behavior. On combining them, the indirect measurement can strongly predict the related behavior and in some cases proactively prevent problematic behavior prior to its occurrence. The IAT has its own advantages, one of which is its ability to circumvent social desirability bias. To use this beautiful feature effectively, future studies should investigate conceptual as well as practical implications of the IAT.
Conflict of Interest
The authors have no conflicts of interest directly relevant to the content of this article.
