“Who said it?” How contextual information influences perceived profundity of meaningful quotes and pseudo‐profound bullshit

Disclaimer/Complaints regulations If you believe that digital publication of certain material infringes any of your rights or (privacy) interests, please let the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the material inaccessible and/or remove it from the website. Please Ask the Library: https://uba.uva.nl/en/contact, or a letter to: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands. You will be contacted as soon as possible.


| INTRODUCTION
Don't believe everything you read on the Internet just because there's a picture with a quote next to it. Abraham Lincoln It is clear that these are not Lincoln's words; however, it is true that we are exposed to a cascade of information on a daily basis, much of which is taken for granted. This is especially the case nowadays, when 81% of people in the developed world uses the Internet (International Communication Union, 2017). People respond to the resultant information overload in different ways-one of which is to consider the context of the situation, or more specifically, the source of the information.
For example, classical studies showed that crediting different authors for a quotation can alter the perception of the message, so that the same statement mentioning revolution, for example, is differently perceived, depending on whether it is reported to be authored by Vladimir Lenin or Thomas Jefferson (e.g., Lorge & Curtiss, 1936).
This kind of contextual effect is evident across different disciplines, for instance, cognitive psychology (e.g., the framing effect; Tversky & Kahneman, 1991) or consumer behavior (Levin & Gaeth, 1988). Specifically, the power of the label has been demonstrated by marketing the same products with different labels. For example, people regard the same wine differently, depending on the price tag (Plassmann, O'Doherty, Shiv, & Rangel, 2008), or have different opinions toward the same beer depending on different information that is disclosed (Lee, Frederick, & Ariely, 2006). Similarly, different products with no label can be difficult to identify. Thus, in one study, violinists preferred new violins over Stradivari in a blind condition (Fritz, Curtin, Poitevineau, Morrel-Samuels, & Tao, 2012). Given that there is no widely accepted term for this phenomenon, we use the term "labeling effect." Although the labeling effect was first demonstrated for material stimuli (e.g., taste of food), it has been proven valid for cultural products as well. Bar-Hillel, Maharshak, Moshinsky, and Nofech (2012) showed that both experts and laymen evaluated poems more positively when they were attributed to famous poets rather than to a bogus poet. In the present research, we examine the labeling effect using different cultural products-meaningful quotes and pseudoprofound bullshit.

| Pseudo-profound bullshit
Bullshit as an academic term became better known after Frankfurt published his essay "On Bullshit", in which he proposed a new meaning of the term, namely, something constructed without concern for the truth, aiming to deceive and impress others in everyday life (Frankfurt, , 2005. Although a substantial part of the work on bullshit has been done from a philosophical standpoint (e.g., Carson, 2016), one certain type of bullshit-pseudo-profound bullshit-has recently become a subject of psychological research (Pennycook, Cheyne, Barr, Koehler, & Fugelsang, 2015). Pseudo-profound bullshit refers to a collection of buzzwords that follow a syntactic structure and constructed with the intention of impressing the reader. Similar to the work of Deepak Chopra, the prominent New Age figure, pseudo-profound bullshit is vague, empty, and essentially meaningless, but it is constructed such that it appears to convey a deeper, profound meaning by using an obscure lexicon (Pennycook et al., 2015).
In their seminal work, Pennycook et al. (2015) measured bullshit receptivity and found that openness to pseudo-profound bullshit is associated with paranormal, conspiracist, and religious beliefs, as well as with lower intelligence, lower analytical, and higher intuitive thinking style. These findings were replicated and extended using different samples (Čavojová, Secar a, Jurkovič, & Šrol, 2018;Hart & Graether, 2018). Furthermore, all these associations could be explained by the Openness/Intellect simplex model in which pseudo-profound bullshit is negatively related to the constructs of one extreme of the modelintelligence-while positively related to the constructs on the opposite (e.g., apophenia; Bainbridge, Quinlan, Mar, & Smillie, 2019). On a more behavioral note, individuals high on bullshit receptivity were less likely to engage in prosocial behavior (e.g., to volunteer for a charity; Erlandsson, Nilsson, Tinghög, & Västfjäll, 2018). Finally, the paradigm of individual differences in pseudo-profound bullshit research was also extended by investigating the association with political attitudes: support for Republicans and conservatism (Pfattheicher & Schindler, 2016), neoliberalism (Sterling, Jost, & Pennycook, 2016), and proneness to fake news (Pennycook & Rand, 2018) or ideology in general (Nilsson, Erlandsson, & Västfjäll, 2019).
However, given that previous research has yielded many similar findings, we suggest that changing the line of research might be more fruitful. Indeed, one suggestion was to focus on contextual factors (Pennycook et al., 2015). In this manner, Turpin (2018) approached pseudo-profound bullshit as a context that enhanced the perceived profundity of abstract art. Nevertheless, the focus of his research was on art, not pseudo-profound bullshit.
In the present study, we offer a new line of research by focusing on pseudo-profound bullshit across different contexts. Given that pseudo-profound bullshit is ambiguous by definition, we expected it to be influenced by contextual information. That is, because the same information can be interpreted differently based on the source (e.g., Bar-Hillel et al., 2012), we assumed that contextual information would increase the ratings of profundity by giving enhanced meaning to pseudo-profound bullshit. To examine this, we presented pseudoprofound bullshit in three different contexts: (a) isolated, (b) as allegedly uttered by a famous author, and (c) within a vignette (a short story or book excerpt). Additionally, we investigated how contextual information affects meaningful quotes in relation to pseudo-profound bullshit. Meaningful quotes were chosen because they have the same structure (length and form) as the pseudo-profound bullshit but convey real, deep meaning in terms of content.

| Method
The design of the study was preregistered via the Open Science Framework (OSF; osf.io/wnbq2/). All materials and the data are available on the same link.

| Pilot study
In order to preselect pseudo-profound items for the main study, we conducted a pilot study. A sample of 20 participants completed a Serbian-translated 30-item bullshit receptivity scale (Sterling et al., 2016). The sample item includes "Wholeness quiets infinite phenomena." Participants were asked to assess the profundity of each item on a five-point Likert scale (1 = not at all profound to 5 = very profound). The scale has good psychometric characteristics (Pennycook et al., 2015), and high reliability was yielded in our translated version (α = .96). Out of 30 items (M = 2.54, SD = 0.87), we selected 14 items with the highest profundity ratings to be used in the main study (see Appendix S1).

| Participants
Based on the G*power analysis (effect size = 0.15, α = .05, power = 0.9) we aimed at 140 participants. The participants were recruited online via Facebook groups (student groups and survey-completion groups) and Facebook pages of the mainstream news portals in Serbia (e.g., b92). Our final sample consisted of 144 participants (102 females, GLIGORI C AND VILOTIJEVI C 536 42 males, M age = 24.5, SD = 7.55) after excluding nine participants who failed the attention check question or completed the survey too fast (in less than 400 s). No data analyses had been performed before the participants were excluded.

| Materials and procedure
We report all measures that participants completed. They filled out the measures in the order in which they are reported here. After signing the consent form, respondents answered demographic questions about age and gender. Participants then answered questions on political issues, support for free market and disintegration.
We included these measures as part of a separate aim, and we will not discuss them further.
Next, participants rated profundity of three types of statements: "pseudo-profound bullshit," "meaningful quotes," and "mundane statements" ("profundity" was described as "having deep meaning and capability to be widely applied"). Ratings were given on a five-point Likert scale (1 = not at all profound to 5 = very profound). Statements were presented in different contexts, all written in italics with quotation marks.

| Pseudo-profound bullshit
Fourteen preselected items from the bullshit receptivity scale were presented in three different contexts: 1. Five of them (randomly selected) 1 were presented isolated, for example: We are non-local beings that localize as a dot then inflate to become non-local again. The universe is mirrored in us.
2. Five of them (randomly selected) were presented as allegedly uttered by a famous author who was chosen based on his prominence and item content, for example: We are non-local beings that localize as a dot then inflate to become non-local again. The universe is mirrored in us. -Dalai Lama 3. Four of them (randomly selected) were presented as a part of a vignette (short story), for example: It was an early winter morning when the old man went to get a ceremonial tree. As he was walking down the well-known path, he noticed a young man selling matches near the road. The young body shuddered in the rhythm of church bells that were ringing from afar.
The old man stopped and asked the young fellow why he was not in his warm home, a question to which the young man replied that he had no home. An astonishment took the old man's face, and the young man said to him: "We are non-local beings that localize as a dot then inflate to become non-local again. The universe is mirrored in us." The score for all pseudo-profound measures was calculated as the mean of the items (see Table 1 for descriptives). Two vignettes with meaningful quotes were included as fillers in order to keep the research goal hidden. These were not used in analyses.

| Meaningful quotes
We selected seven meaningful quotes made by famous authors.
These were presented isolated (e.g., "No man ever steps in the same river twice, for it's not the same river and he's not the same man.") or with an author (e.g. "No man ever steps in the same river twice, for it's not the same river and he's not the same man." Heraclitus). Scores were calculated as the mean of items (Table 1).

| Mundane statements
To control for the tendency to find profundity anywhere, we included four mundane statements (Pennycook et al., 2015). These statements are generally true and should not be rated as profound (e.g., "Newborn babies require constant attention.").

| Results
We report analyses performed on all forms together including all participants. Correlations between variables are given in Table 2. The reported range is true for three forms; in the fourth form, reliability was low (α = .33). The reliability was improved when one item was dropped (α = .47). However, neither the exclusion of this item nor form changed the pattern of results, so we report results on all measures.
High correlations between bullshit measures (rs > .62, ps < .001) demonstrate their similarity regardless of context. Bullshit scores were also moderately correlated with the profundity ratings of meaningful quotes (rs range from .33 to .46), but the higher correlation between meaningful quotes with and without an author (r = .61, p < .001) suggests the distinction between bullshit and meaningful quotes. Furthermore, mundane statements were not correlated with any other type of statements, suggesting that there is no single latent construct, such as proneness to find profundity anywhere.
To investigate the authorship effect on pseudo-profound bullshit and meaningful quotes, we performed a two-way repeated-measure analysis of variance (ANOVA) with authorship (with conditions: present and absent) and type of statements (with conditions: pseudoprofound bullshit and meaningful quotes) as within-subject factors.
The results (Figure 1) showed that the meaningful quotes were rated as more profound than the pseudo-profound bullshit, F(1,143) = 138.53, p < .001, η 2 = 0.49. Also, statements with authors were rated as more profound than those without, F(1,143) = 7.50, p = .007, η 2 = 0.05. However, given the interaction between factors (F(1,143) = 10.66, p = .001, η 2 = 0.07), the authorship effect was true only for pseudo-profound bullshit, whereas the meaningful quotes were rated the same regardless of whether an author's name was presented.
To further examine the contextual effects on the profundity of pseudo-profound bullshit, we compared bullshit presented as a vignette to bullshit with the author and bullshit isolated.

| Discussion
Study 1 showed that pseudo-profound bullshit is susceptible to the authorship effect-a statement attributed to a famous author increased its profundity ratings. However, with regard to our main objective, assessing the contextual effect, our study had several limitations. First, we did not have the full randomization of the presented stimuli, that is, we had not ensured that a participant could potentially complete any possible form. As stated before, this was not possible due to the technical limitations of the software used (Qualtrics).
Second, in order to shorten the length of the survey, we did not have a complete 2 × 3 design, as meaningful quotes used in vignettes were different from those presented alone or with an author. Finally, the number of items in our study (with forms having only three meaningful quotes per condition) was small, which resulted in low reliability.
Due to these limitations, we carried out an additional study to test the generalizability of the obtained effects.

| STUDY 2
In Study 2, we wanted to conceptually replicate the contextual effect from Study 1 and overcome some of its limitations with several modifications. First, we used a simpler, 2 × 3 mixed design in which participants were presented meaningful quotes and pseudo-profound bullshit (within-subjects factor) in only one of the three conditions (between-subjects factor): (a) isolated, (b) authored, and (c) as an excerpt from an authored book. This overcomes the limitation of the first study, which had four forms and which resulted in the comparison of different items from different contexts. Second, the authored context was modified by adding an occupation of the presented author next to his name. Third, we tried to improve the vignette condition by making vignettes look like real excerpts from the original author's work. While developing these, we attempted to mimic the original author's style in order to increase the excerpt's literary value.
We also added the author's name and the alleged book under the excerpt. Consequently, the vignette condition in Study 2 contained more information than the vignette condition in Study 1. The rationale for this modification was that including more information about the T A B L E 2 Correlations between profundity of pseudo-profound bullshit, meaningful quotes, and mundane items

| Method
The design of the study was preregistered via the Open Science Framework (osf.io/bxprj). All materials, database, and R code are available at osf.io/g2aq7.

| Materials and procedure
We report all measures that participants completed. After signing the consent form, participants filled out demographic information (gender, age, and education). Next, participants were randomly allocated to one of the three possible conditions. All three conditions consisted of five pseudo-profound statements and five meaningful quotes but varied across the context in which these were presented. Items were randomized within the conditions. Participants rated profundity of statements using a five-point Likert scale (1 = not at all profound to 5 = very profound). Profundity was described as "having deep meaning and capability to be widely applied."

| Pseudo-profound bullshit
We selected five items with the highest profundity ratings from Study 1 (M = 3.17). We chose the highest-rated items to avoid a floor effect, that is, items rated too low (due to the higher degree of meaninglessness or inappropriate translation). Selected items were presented in three different contexts across the conditions: 1. Statements were presented isolated, for example: We are non-local beings that localize as a dot then inflate to become non-local again. The universe is mirrored in us.
2. Statements were presented as allegedly uttered by famous authors (who were chosen to match the occupation of those authors selected for the meaningful quotes), for example: We are non-local beings that localize as a dot then inflate to become non-local again. The universe is mirrored in us.
Aristotle, ancient Greek philosopher 3. Statements were presented as a part of a book excerpt (vignette), for example: The soul primarily consists of the basis we live on, we feel and we think about. It is wrong to speak of it as a supernatural matter and its substrate. It is limited by the way in which life unfolds. However, although it is limited, the soul is not determined by the body. That is why I say two friends are the embodiment of one soul, placed in two different bodies. Those who are looking for others are actually looking for the second part of their soul. That is why the assumptions made by those who disprove the rupture of the soul and the body are right. 'We are non-local beings that localize as a dot then inflate to become non-local again. The universe is mirrored in us.' Aristotle, an excerpt from the 'Debate on the Soul'

| Meaningful quotes
We included five new meaningful quotes made by famous authors in order to test the generalizability of the effect. These items were also presented in three different contexts (isolated, authored, or book excerpts) across the conditions which were built in the same way as for pseudo-profound items.

| Results
In order to test whether our participants perceived meaningful quotes and pseudo-profound bullshit as separate constructs, we performed factor analysis. The factor analysis (method: minimum residual; rotation: oblimin) showed an expected two-factor solution in which all pseudo-profound bullshit items loaded on one factor (loadings >.45) while meaningful quotes loaded on the second factor (loadings >.49) without high cross-loadings (see OSF for factor loadings). Both measures showed good reliability (αs = .77 and .76).
Next, we performed mixed ANOVA with the statement type as a repeated-measure factor (with two levels: pseudo-profound bullshit and meaningful quotes) and context as a between-subjects factor (with three levels: isolated, author, and vignette). The results ( Figure 2) showed that context influenced the ratings of profundity, F (1,275) = 13.81, p < .001, η 2 = 0.09. However, meaningful quotes and pseudo-profound bullshit were rated as equally profound, F (1,275) = 1.86, p > .05, and there was no interaction effect, F (1,275) = .86, p > .05. Post-hoc tests (Bonferroni corrected) showed that the statements in the vignette and author condition were rated as more profound than those isolated, whereas there was no difference between vignette and author condition (see Table 3).
Although ANOVA yielded condition effect, classical ANOVA has a limitation of treating all stimuli (i.e., items) as equal, which might increase the possibility of type 1 error and invalidate the analysis (Judd, Westfall, & Kenny, 2012. Namely, standard ANOVA only includes random intercept for participants (by participant analysis), which is why we built a linear mixed model and added random intercept for items allowing different ratings of the items (adding by stimulus analysis). The model with random intercept for items was significantly better (Likelihood ratio test: χ 2 diff (1) = 122.98, p < .001, difference in Akaike Information Criterion (AIC) = 121, difference in Bayesian Information Criterion (BIC) = 114), and items did not have the same mean ratings (Likelihood ratio test(1) = 122.98, p < .001).
We also tested a model with random intercepts and slopes for items (for condition factor), but this model did not show a better fit than the  previous one (Likelihood ratio test: χ 2 diff (5) = 8.99, p > .05, higher AIC and BIC), indicating that the contextual effect was similar for all items (see Figure 3). The mixed model with the random intercept for items still yielded significant contextual effect of author and vignette condition (isolated vs. author: t = 3.62, p < .001, d = 0.25; isolated vs. vignette: t = 4.81, p < .001, d = 0.35).

| GENERAL DISCUSSION
In the present research, we tried to add to the small amount of literature on pseudo-profound bullshit, while offering a new paradigm.
Across two studies, we demonstrated that pseudo-profound bullshit is susceptible to the labeling effect-bullshit being rated as more profound when presented as being uttered by a famous author. On the contrary, this contextual effect for meaningful quotes was inconsistent, as profundity ratings were increased only in the second study.
The labeling effect for pseudo-profound bullshit is similar to the ratings of poems attributed to famous or bogus poets (Bar-Hillel et al., 2012). Although we did not investigate any underlying mechanisms of the effect, it is plausible to assume the similar process to those where expectation led to genuinely different feeling (e.g. Bar-Hillel et al., 2012;Lee et al., 2006). That is, after seeing a famous author's name next to the statement, participants might have been primed by the author's name and construed the meaning in the statement. However, the power of different authorities remains; it may happen that one is seduced by an authority from an unfamiliar field (e.g., art/Dali), whereas this could not be the case for the familiar field (e.g., physics/ Plank). Specifically, one of the directions for future research could be to examine whether certain authorities (i.e., based on occupation) have a larger or smaller impact on bullshit receptivity. Taken together, it would be beneficial to test whether this tendency is irrational or not (as in heuristics, for example).
Interestingly, increase in profundity was inconsistent for meaningful quotes as it emerged only in the second study. All meaningful quotes from Study 2 were taken as excerpts from particular authors' work, which makes them decontextualized. This might be the reason why there was a contextual effect on these quotes. As quotes usually depict the author's views represented by their own words on a certain topic (Conrad, 1999), this way of recruiting can constrain their application. Alternatively, short and widely applicable sayings (such as Latin phrases, e.g., "He conquers who conquers himself") might be immune to the contextual effect due to their life-oriented message and widespread use. This might be one of the avenues for future research.
Another possible path of label influence is through the contextualization of the statement. For example, when one reads a short story (or book excerpt), she might relate the bullshit to that story so that "non-local beings that localize as a dot" actually relate to the protagonists of the story (e.g., signifying the old man's unimportance in the world). Even though our data do not support these conclusions, vignette condition had higher absolute ratings than the isolated condition. It might be the case that our short stores in Study 1 did not have enough literary value to increase the profundity. Although the vignette condition (book excerpt) improved ratings in Study 2, it also contained the author's name, making it impossible to distinguish whether effect occurred due to the author or the excerpt. However, this condition had higher absolute values than the author-only condition, supporting our notions. These questions remain open for other researchers to answer. Surprisingly, in Study 2, meaningful quotes and pseudo-profound bullshit were rated as equally deep, which is in contrast to results from Study 1 and findings from bullshit research (e.g., Čavojova et al., 2018;Pennycook et al., 2015). One plausible reason is the selection of the deepest pseudo-profound items from the original 30-item scale.
Higher mean profundity ratings for bullshit items in Study 2 (M = 3.2 compared with M = 2.9 in Study 1) support this notion. Therefore, although the same five pseudo-profound items had similar ratings in two studies (M = 3.17 and M = 3.20), the mean bullshit score was lower in the first study as it contained other bullshit items that had lower ratings. Second, our selection of meaningful quotes does not necessarily guarantee their profundity-mean profundity ratings for meaningful quotes was lower in Study 2 (M = 3.3 compared with M = 3.7 in Study 1). That is, some of the meaningful quotes in Study 2 might seem like contemporary motivational quotes (e.g., "They always say that time changes things, but you actually have to change them yourself.") and therefore have lower ratings. Indeed, this quote had the lowest ratings along with the Dostoevsky's quote ("To go wrong in one's own way is better than to go right in someone else's.").
In conclusion, our results suggest that pseudo-profound bullshit is susceptible to contextual effects-attributing a statement to a famous person alters its perception. Although it might be only economically exploited (as in the case of New Age leading figures), other kinds of bullshit (for example, political), might be more dangerous. Demonstrating how easily people might evaluate pseudo-profound statements as more profound just because they were presented with an author's name; we should be aware of potential abuse of this type of effect.

CONFLICT OF INTEREST
Authors have no conflict of interest.

AUTHORS CONTRIBUTIONS
Vukašin Gligori c carried out the initial idea, conception and design; data collection, analysis, and interpretation; and drafting the article and revising it critically for important intellectual content. Ana Vilotijevi c carried out conception and design; data collection and interpretation and drafting the article and revising it critically for important intellectual content.

ORCID
Vukašin Gligori c https://orcid.org/0000-0001-7528-6806 ENDNOTES 1 We made four quasi-randomized parallel forms that contained different statements across different contexts; in one form, one pseudo-profound bullshit item was presented without context; in the second one, it was presented with alleged author, in the third one as a vignette, and so on. Similarly, one quote was presented without an author in one form and with author in another form. This was done because it was not possible to present interdependent statements within one block. To obtain the full randomization, one statement per page would have had to be presented. This would slow down the tempo, potentially causing disclosure of the research aim and more dropouts. Forms did not differ on any of the dependent variables (Fs < 1.86, ps > .14).
2 Although our preregistered criterion for completion time was ±2.5*median absolute difference; Leys, Ley, Klein, Bernard, & Licata, 2013), we opted for the former one as the median absolute difference cut-off indicated that no participants should be excluded as too fast (negative value), whereas the upper limit would have been 14 min, which we deemed as too short. The same pattern of results emerged when we reanalyzed the data set including preregistered criterion (270 participants) or all participants (292).

DATA AVAILABILITY STATEMENT
The design of the study was pre-registered via the Open Science Framework (link). All materials and the data are available on the same link. The design of the study was pre-registered via the Open Science Framework (link). All materials, database, and R code are available here ORCID Vukašin Gligori c https://orcid.org/0000-0001-7528-6806