Misinformation interventions decay rapidly without an immediate posttest

In recent years, many kinds of interventions have been developed that seek to reduce susceptibility to misinformation. In two preregistered longitudinal studies ( N 1 = 503, N 2 = 673), we leverage two previously validated “ inoculation ” interventions (a video and a game) to address two important questions in misinformation interventions research: (1) whether displaying additional stimuli (such as videos unrelated to misinformation) alongside an intervention interferes with its effectiveness, and (2) whether administering an immediate posttest (in the form of a social media

The spread of and belief in misinformation is a substantial societal problem (Lewandowsky et al., 2017;Roozenbeek & van der Linden, 2024), having been linked to a range of negative public health outcomes (Schmid et al., 2023), including reduced vaccine uptake (Loomba et al., 2021(Loomba et al., , 2024)).Proposed approaches to countering misinformation include both system-level solutions (such as introducing new legislation or changing recommender algorithms) and individual-level solutions (boosting competences, nudging people to promote better sharing behavior, debunking/fact-checking, and content labeling); see Kozyreva et al. (2024) and Ecker et al. (2022) for discussions.In recent years, prebunking (pre-emptive debunking) and psychological inoculation have gained prominence as a preemptive individual-level method of tackling misinformation (Lewandowsky & van der Linden, 2021).
Prebunking is an umbrella term for various approaches that seek to protect people against misinformation.Inoculation theory is a commonly used form of prebunking.Grounded in a medical analogy, inoculation interventions commonly consist of two components, which when combined should induce increased psychological resistance against future unwanted persuasion attempts: (1) a warning of an impending attack on one's beliefs (i.e., a forewarning), and (2) a pre-emptive refutation of the false or misleading argument, narrative, or manipulation technique that participants are "inoculated" against (Compton, 2013;Compton et al., 2021;McGuire, 1964;McGuire & Papageorgis, 1962).Inoculation interventions such as infographics, informative videos, and online games have shown promise as a way to build resistance against misinformation at scale (Basol et al., 2021;Cook et al., 2022;Lewandowsky & Yesilada, 2021).
Recent inoculation studies share a similar study design, with a posttest (usually in the form of an item rating task, e.g., rating the perceived reliability of a series of true and false/misleading social media posts) administered immediately after the treatment or control intervention. 1In fact, the majority of contemporary studies testing the efficacy of misinformation interventions (not just using inoculation) involve immediate posttesting (Johansson et al., 2022;Kozyreva et al., 2024;Ziemer & Rothmund, 2022).And although some longitudinal studies are available (Basol et al., 2021;Brashier et al., 2021;Maertens et al., 2021;Swire et al., 2017), 2 most of these studies have involved participants going through an intervention, then immediately taking a posttest, and then taking another posttest some time after.This study design is potentially problematic.In a recent study using three types of inoculation interventions (text, video, and game), Maertens et al. (2024) tested whether administering a posttest immediately after the intervention yields similar outcomes (in terms of the strength and longevity of the inoculation effect) as only administering the posttest after ~10, ~12, or ~30 days.They found that participants who watched an inoculation video and did a posttest immediately after were significantly better than a control group at discerning misinformation from nonmisinformation ~10 days later, with no evidence for decay.However, participants who only received a posttest after ~12 days (and not immediately) showed evidence for decay and did not perform significantly better than control group participants.Similar results were found for the game-based intervention, which was no longer effective after ~10 or ~30 days if no immediate posttest was administered, despite a previous study showing good longevity (up to 2 months) with regular testing (Maertens et al., 2021).In other words, there are indications that immediate posttests help strengthen the lessons from the intervention in memory, thus boosting the strength and longevity of the intervention's efficacy.This is in line with the broader educational psychology literature, which has established the importance of retrieval practice for durable learning and memory (Agarwal et al., 2021;Carpenter et al., 2022).Intermittent testing of memory has been found to be more effective for overall learning than providing additional study opportunities across more than 200 studies (Carpenter et al., 2022).The testing effect constitutes one of the most robust findings in the psychology of learning and education, arising in nearly all conditions in school and classroom settings (Agarwal et al., 2021).And indeed, Bird et al. (2015) found that people who actively rehearsed the contents of a video were substantially better at recall 8 and 18 days after viewing compared to a group that did no rehearsal. 3Even in the context of debunking approaches to misinformation, retrieval practice has been shown to improve memory of corrective information, although this improved memory does not necessarily translate to more accurate beliefs (Collier et al., 2023;Fazio et al., 2023).
Within the inoculation theory literature, Compton and Ivanov (2012) found that interrupting an inoculation treatment (in their study, after the forewarning but before the preemptive refutational content) to measure threat seemed to enhance the resistance conferred by the inoculation treatment, when compared to interventions that did not have this interruption.They note that "the interruption of the experimental design and the embedding of a threat scale in the inoculation message contributed to greater levels of threat and ultimately greater resistance (i.e., more negative attitudes toward the counterattitudinal attack)" (Compton & Ivanov, 2012, p. 10).More broadly, these findings raise the question of whether posttests (or other memory strengthening exercises) should be seen as a component of many types of learning-based antimisinformation interventions, which has substantial implications for their scalability.
In addition, the vast majority of studies into the efficacy of misinformation interventions have looked at how these interventions perform in isolation (Johansson et al., 2022).We therefore know very little about the extent to which distractions (such as watching one or more irrelevant videos alongside a video designed to help people better identify misinformation) reduce people's performance on subsequent relevant outcome measures.In their YouTube field study, Roozenbeek et al. (2022) administered their outcome measure a median of 18 h after participants watched their intervention as a YouTube ad; it is likely that participants were distracted by other stimuli during this time period, but they did not measure distraction explicitly.Previous research has generally shown that distractions are detrimental to memory (Blasiman et al., 2018;Craik, 2014;Middlebrooks et al., 2017).It is therefore possible that results from studies which only look at the efficacy of (learning-based) misinformation interventions in isolation overestimate this efficacy when distracting stimuli are taken into account, for instance in social media environments where distractions are omnipresent (Pennycook & Rand, 2019).
We test both of these possibilities in two preregistered experiments (N 1 = 503 and N 2 = 673), one with a video-based and one with a game-based intervention.All materials including data sets, cleaning, analysis, and visualization scripts (written in R), Qualtrics files, and other supplementary information can be found on our OSF page: https://osf.io/9frch/.

| STUDY 1
In study 1 (N = 503, preregistration link: https://aspredicted.org/W92_ MDY), we look at whether a previously validated video-based inoculation intervention (Roozenbeek et al., 2022) retains its effectiveness with and without an immediate posttest after 48 h.In addition, we test if viewing unrelated content alongside the inoculation intervention (two other videos about random topics) interferes with the intervention's efficacy.
Briefly put, while we find that there is no interference with unrelated content (which is good news as in the real world people are exposed to a large number of stimuli alongside any potential intervention), administering an immediate posttest matters a great deal.Participants who only received a posttest 48 h after viewing an inoculation video performed no better than a control group, whereas participants who did get an immediate posttest continued to substantially outperform the control group after 48 h.

| Measures
We conducted a preregistered randomized controlled trial on Prolific Academic using a 30-s long video-based inoculation intervention about emotionally manipulative language (see: https://inoculation.science/ inoculation-videos/emotional-language/) as the intervention of interest.
Emotionally manipulative language is a technique commonly used in misinformation (Carrasco-Farré, 2022;Zollo et al., 2015).Thus, in previous studies, this video-based inoculation intervention was shown to substantially improve people's ability to discern emotionally manipulative from neutral social media content, increase confidence in their ability to identify misinformation that utilizes this technique, reduce the perceived trustworthiness of manipulative content, and boost the quality of their news sharing decisions.
The inoculation effect in the present study was measured through a posttest, that is, an item rating task consisting of 10 social media posts, identical to how the intervention was tested in previous studies (Maertens et al., 2024;Roozenbeek et al., 2022); see Supporting Information S1: Table 7 for an overview of the item wordings.Each social media post was randomly selected to be either manipulative (i.e., using emotionally evocative language) or neutral (similar in content, length, and topic as the manipulative post but not making use of emotionally evocative language), so that each participant saw on average five manipulative and five neutral items (although this proportion varied per participant).Participants rated on a Likert scale, from 1 to 7 (1 = strongly disagree, 7 = strongly agree) the following: (1) Manipulativeness ("This post is manipulative"); (2) Confidence in their judgment ("I am confident about my assessment of this post's manipulativeness"); (3) Trustworthiness ("This post is trustworthy"); and (4) Sharing intentions ("I would share this post with people in my network").For the manipulativeness, trustworthiness, and sharing intentions measures, our key outcome variable of interest is discernment, calculated by taking the difference between the average scores for the neutral social media posts and the average scores for their manipulative alternatives.Positive discernment scores thus mean that a participant is generally able to correctly distinguish manipulative from neutral content, whereas negative discernment scores mean that a participant incorrectly identifies neutral content as more manipulative (or less trustworthy, etc.) than manipulative content.For the confidence measure, we look at ratings separately for the manipulative items and neutral items.See Supporting Information S1: Table 0b for descriptive statistics.For an extensive explanation of the development and validation of this item rating task, we refer to Roozenbeek et al. (2022).
To test for interference and posttest effects, we included four separate experimental conditions, two inoculation groups and two control groups.The two inoculation groups were both shown the inoculation video as well as two out of three possible control videos of similar length and aesthetic but unrelated to emotionally manipulative language (about how to make blueberry muffins; the importance of vaccines 4 ; and age-related macular degeneration).The two control groups were shown all three control videos.Video display order was randomized at the participant level.
We administered the same posttest at two different time points: immediately after watching the videos (T1) and/or 48 h after (T2).Group 1 was shown the inoculation video (alongside 2 control videos), then did an immediate posttest, and then did the same posttest again 48 h after.
Group 2 was the same as group 1, except they did not do an immediate posttest, only after 48 h.Group 3 was the same as group 1 except participants saw only three control videos rather than an inoculation video.Participants in group 4 were also only shown control videos butas with group 2-they did not complete an immediate posttest.
We also administered a series of demographic questions: gender, age, political ideology (1 being "very left-wing" and 7 being "very rightwing"), people's political positions on social and economic issues, education level, ethnicity, how often people check the news, social media use, and their favorite news outlet.Participants received an attention check which assessed if they remembered the videos they watched.They were given six options in total, four of which they might have seen as part of the study, and two not being part of the study.As preregistered, those who failed the attention check (by selecting one or both of the videos that were not in the study) were excluded from the final sample.This resulted in 47 participants being removed from the analysis.Participants who took the survey more than once or who did not complete the survey at T2 were also excluded.This study received ethical approval by the University of Cambridge Psychology Research Ethics Committee (#PRE.2022.113).See Figure 1 for an overview of the design of studies 1 and 2.

| Sample
We conducted an a priori power analysis to determine sample size, using the pwr package in R (power = 95%, α = .05and d = 0.400, based on the d = 0.439 effect size found for manipulativeness discernment in Maertens et al., 2024).The minimum sample needed to detect the presence of a main effect of inoculation of that size on manipulativeness discernment was N = 680.As preregistered, we collected a total of 800 initial participants at T0 (~200 responses per group), to account for an attrition rate of 30%.Participant recruitment was limited to the United States and balanced on sex (400 male, 400 female).Participants were paid 1.00 GBP for a 6-7 min survey (~£9 per hour), followed by £0.46 for a 4-5 min follow-up survey (~£6 per hour).After applying our preregistered exclusion criteria, we ended up with a final sample of N = 503, below the target (see Figure 1 for the sample sizes per group).The attrition rate between T1 and T2 was 32.4%.The final sample was 50.7% female, of studies 1 and 2. 48.1% male, and 1.0% nonbinary.The average age was 44.1 years (SD = 13.5),78.1% identified as white (21.9% as BAME; Black, Asian, or Minority Ethnic) and the most common political stance was Left (52.5%), followed by right (25.2%) and center (22.3%).The most frequent level of education was a Bachelor's degree (37.8%), followed by a high school degree (28.8%), a Graduate/Professional degree (15.8%), an Associate's degree (13.7%), a Doctorate (3.4%), and no high school degree (0.6%).
Participants were somewhat left-leaning on social issues (M = 3.22, SD = 1.81 on a 7-point scale) but more centrist on economic issues (M = 3.65, SD = 1.87).38.6% of participants reported checking the news once a day, followed by 37.4% who checked it multiple times a day.2.0% reported never checking the news.64.8% reported checking social media multiple times a day, followed by 20.7% who check it once a day.2.8% of participants reported never checking social media.See Supporting Information S1: Table 0a for a full overview.

| Hypotheses and analysis plan
We preregistered (https://aspredicted.org/W92_MDY) the following hypotheses: H1.People who watch a ~30 s inoculation video are better at differentiating emotionally manipulative social media posts from neutral social media posts, compared to those who watched a ~30 s control video, at T1 (immediately after the intervention).
H2.The same effects can be found at T2 (48 h after the intervention), independent of whether participants received an immediate posttest at T1 [H2a] or not [H2b].
Finally, we preregistered the following exploratory hypotheses: eH2.The presentation of the inoculation video with two other unrelated control videos will produce a generally reduced inoculation effect, compared to previous studies which present only the inoculation video in isolation (Maertens et al., 2024;Roozenbeek et al., 2022).
However, when we compare the inoculation group that did not receive an immediate posttest (group 2) and the control group without immediate posttest (group 4), we find no difference between both groups at T2, M diff = 0.150, t(259) = 0.802, p tukey = .854,d = 0.010, 95% CI [−0.145, 0.344].Furthermore, a (nonpreregistered) Two One-Sided (TOST) equivalence test with a Smallest Effect Size of Interest (SESOI) of d = 0.100 (very small) is not significant, t(259) = 1.334, p = .092,confirming statistical equivalence to zero, meaning that we can rule out the presence of any effects larger than d = 0.100 (Blasiman et al., 2018(Blasiman et al., , 2020)). 5These results show that the group that received an immediate posttest continued to outperform the control group after 48 h, whereas the group without immediate posttest did not.These results speak against H2b and lend support to eH1.While the inoculation with immediate posttest group (group 1) did not perform significantly better at discerning manipulative content at T2 than the inoculation group without the immediate posttest (group 2, M diff = 0.484, t (230) = 2.438, p Tukey = .071,d = 0.320, 95% CI [0.061, 0.579]) in absolute terms (as stipulated in eH1), they did perform better in relative terms, as the inoculation with posttest group displayed significantly stronger manipulativeness discernment at T2 than the control groups, whereas the inoculation group with no immediate posttest was no better at discerning manipulative content than control groups at T2.In addition, an exploratory multiple linear regression for manipulativeness discernment at T2, but with demographic variables included as covariates (see below), did show that the inoculation with immediate posttest group (group 1) performed significantly better at discerning manipulative content than the inoculation group without immediate posttest (group 2), p = .008;see Supporting Information S1: Table 6.Thus, we conclude that without an immediate posttest, any inoculation effect disappeared within 48 h.
However, there were no significant differences between the four groups for sharing discernment and confidence at T2.Similar results were reported by Maertens et al. (2024), who also noted a decay of inoculation effects on sharing intent and confidence, while the effect on trustworthiness remained significant over time.However, our results may also be due to the fact that our study was powered specifically to detect effects on manipulativeness discernment.See  2024), although we note that the effect for sharing discernment was not significant, potentially due to a lack of statistical power (see Supporting Information S1: Table 1).We therefore find no support for eH2: there appears to be no evidence that watching two unrelated videos alongside the inoculation video interferes with the inoculation effect.In other words, people retain the relevant information from the video even if they are exposed to stimuli containing entirely different information around the same time.
Finally, we conduct a series of exploratory analyses with our demographic variables.Using a multiple linear regression with manipulativeness discernment at T2 as the outcome measure, experimental condition as the independent variable, and age, gender, education level, political ideology, ethnicity, how often people check the news, and social media use as covariates, we find that the main effect of inoculation with immediate posttest on manipulativeness discernment remains significant (comparing the inoculation with immediate posttest group-group 1-against the control group with immediate posttest-group 3), standardized β = −.424,p < .001).In addition, we find a significant effect of ethnicity (β = .263,p = .017)such that in comparison with people who identify as white, BAME (Black, Asian, and Minority Ethnic) people have lower manipulativeness discernment; political ideology (β = −.180,p < .001)so that Conservatives have lower manipulativeness discernment than Liberals; and a small but significant positive effect of education (β = .090,p = .045),with people with higher levels of education also having higher manipulativeness discernment.None of the other covariates show significant associations with manipulativeness discernment at T2 (all p > .131).See Supporting Information S1: Table 6.immediately after the intervention play a role in the longevity of these effects.We find no evidence that additional stimuli interfere with the inoculation effect conferred by the video, with the inoculation effect not only being highly significant when interference is present, but also with a similar (and descriptively even higher) effect size.However, in line with earlier work by Maertens et al. (2024), we also find strong evidence that testing effects play a substantial role in even a highly effective intervention's longevity: participants who were not given an immediate posttest just after watching an inoculation video showed no improved discernment of manipulativeness compared to a control group a mere 48 h after viewing.As the relevant p value for this difference was about p = .854and we were able to rule out the possibility of any effects larger than d = 0.10, it is unlikely that this null finding is due to lack of power, but rather reflects a true "forgetting" of the lessons from the video.Nonetheless, this study also had several limitations: we only looked at a single intervention (a short video), which limits the generalizability of our findings, and more importantly, we were not able to establish a causal mechanism behind why decay sets in more rapidly without an immediate posttest.We address these limitations in study 2.

| STUDY 2
For study 2 (preregistration link: https://aspredicted.org/blind.php?x=Q6P_C88), 1066 US participants were recruited through Prolific Academic with representative interlocking hard quota for sex, age, and ethnicity (based on simplified US Census Bureau population group estimates from 2015), of which 964 completed the survey at T0, with 673 responses remaining after applying all preregistered exclusion criteria.Study 2 was similar to study 1 in scope and design (see Figure 1), but with two major changes: (1) the time delay was 10 days instead of 48 h, and ( 2 (with minor design changes, explained below).In the interest of brevity, we refer to Appel et al. (2024) for more details about the game's design and testing.

| Measures, sample, and hypotheses
Similar to study 1, we administered a posttest either immediately postintervention (T0) or 10 days after (T10), or both, resulting in four conditions: inoculation and control, with and without an immediate posttest.Based on a power calculation with 90% power, α = .05,and d = 0.31 (based on findings by Appel et al., 2024), we sought to recruit a total of 960 participants, or 240 per group.964 complete responses were collected (which required recruiting 1066 initial participants).They were paid 2.41 GBP (~£6 per h) for a 24min survey, followed by ~1.61 GBP (~£6 per h) for the ~16min follow-up survey.After applying our exclusion criteria, we ended up with a final sample of N = 842 at T0 and N = 673 participants who completed the study at both T0 and T10, a total attrition rate of 20.1% (n = 207 for control without immediate posttest, n = 188 for control with posttest, n = 136 for inoculation without posttest, and n = 142 for inoculation with posttest).Attrition rates were higher in both inoculation groups, likely because these participants were required to provide a password at the end of the game to ascertain that they had played the game all the way through (unlike participants who played Tetris); some participants did complete the entire study but did not provide the correct password and were thus excluded from the main analysis.Study 2 received ethical approval by the University of Cambridge Psychology Research Ethics Committee (#PRE.2022.120).
The posttest was identical to the one used by Appel et al. (2024) and consisted of 12 simulated social media posts, each of which was randomly either manipulative (i.e., using one of the four techniques learned in the game) or neutral (similar in content, length, and topic as the manipulative post but not making use of any manipulation techniques from the game), so that each participant sees on average six manipulative and six neutral items; this procedure is highly similar to the one used in study 1.We included three outcome measures per item: manipulativeness, confidence, and sharing intentions, assessed on the same 1-7 Likert scale used in study 1.See Supporting Information S1: Table 13 for the item wordings.Again following Appel et al. (2024), we also included a separate "manipulation technique recognition" task consisting of 16 fictitious vaccine-related headlines, balanced on two dimensions: true versus false, and manipulative versus not manipulative.For an extensive discussion on the differences between these dimensions (and susceptibility to false vs. misleading/manipulative news), we refer to Maertens, Said, et al. (2023).Four of the headlines in this task were both false and manipulative, four were false and not manipulative (i.e., merely containing a false claim without further markers or cues of manipulativeness, e.g., "the flu vaccine doesn't work"), four were true and manipulative, and four were true and not manipulative.
Participants were asked to rate each item as either true or false and as manipulative or nonmanipulative, and indicated which (if any) manipulation technique they believed the item contained from a drop-down menu, with only one option being correct.See Supporting Information S1: Table 8b for descriptive statistics (means and standard deviations per experimental group) and Supporting Information S1: Table 14 for the item wordings.
In addition, to investigate why decay may set in more quickly if no immediate posttest is administered, we included a battery of questions at the end of the survey (shown to all participants), assessing how well people remembered relevant information from the Bad Vaxx game ("memory"; e,g., "How many different misinformation techniques did you learn about in the game?"; control group participants are expected to perform at or near chance level as they are randomly guessing), how much they had come across content about vaccine misinformation in the past 2 weeks ("interference"; e.g., "In the past two weeks, I have heard conflicting arguments about vaccine misinformation [1 strongly disagree-7 strongly agree]"), how afraid they were of vaccine misinformation ("fear"; e.g., "Thinking about vaccine misinformation I feel fearful [1 strongly disagree-7 strongly agree]"), how motivated they were to resist vaccine misinformation ("motivation"; e.g., "Thinking about vaccine misinformation motivates me to resist misinformation [1 strongly disagree-7 strongly agree]"), how much they talked about vaccine misinformation with others ("talk"; e.g., "In the past two weeks, how often did you talk about or discuss vaccine misinformation with other people?"), how much vaccine misinformation means to them ("involvement"; e.g., which [term] best describes how much misinformation means to you?"), and how much they felt threatened by vaccine misinformation ("threat"; a composite measure consisting of the combined scores on the "fear," "motivation," "involvement," "talk," "apprehensive threat," and "accessibility" 6 scales) 7 ; see Supporting Information S1: Table 12 for the question wordings.These questions were the same as those used by Maertens et al. (2024); the motivational threat scale was taken from Banas & Richards (2017), and the apprehensive threat scale from Compton & Ivanov (2012).These measures allow us to explore the relative importance of memory, motivation to resist misinformation, and the extent to which people feel threatened by misinformation in explaining the intervention effect's decay over time.Finally, we administered the 20-item Misinformation Susceptibility Test (Maertens, Götz, et al., 2023), COVID-19 vaccine uptake intentions, vaccine safety perceptions (see Supporting Information S1: Table 12), and a set of demographic questions: age, gender, education level, US state (and region), political ideology and US political party affiliation, social media use, and news consumption.For the sample descriptives, we refer to Supporting Information S1: Table 8a.We preregistered the following hypotheses (main effects tested using ANOVA contrasts, as in study 1): H1. Playing a short online game about vaccine misinformation improves people's ability to detect manipulative social media content (discernment and technique identification) about vaccinations.
H2.The intervention effects decay faster over time when no immediate posttest is administered compared to when an immediate posttest is administered.

H3. The longevity of the intervention effects is
predicted by memory and motivation.

| Results
We show the results for our preregistered analyses for H1 and H2 in Table 1.
To summarize, we find support for H1: immediately after the intervention (T0), inoculation group participants were significantly better T A B L E 1 Study 2: Between-group comparisons (ANOVA contrasts) at T0 (immediately postintervention) and T10 (10 days after).Note: Significant differences at p < .05are marked in bold.t values are consistently positive, indicating that inoculation group scores are (descriptively or significantly) higher than control group scores on all outcome measures in the table.Note that all statistical comparisons are significant for H1, indicating that the Bad Vaxx game yielded an inoculation effect at T0.For H2, we find no significant between-group differences at T10 when no immediate posttest was administered, whereas 6/7 effects were significant at T10 when participants did receive an immediate posttest (4/7 with Tukey-corrected p Values).
than the control group at distinguishing manipulative social media content about vaccines from neutral content, were more confident in their assessment, and had better sharing discernment.In addition, there were significant between-group differences on people's ability to distinguish true from false and manipulative from neutral vaccine-related headlines, and were better at correctly identifying which manipulation techniques were used in these headlines (all p < .025).All effect sizes are between d = 0.250 and d = 0.477, indicating a small to near-medium effect.These results are in line with those reported by Appel et al. (2024).
We also find partial support for H2: the inoculation group that did not receive a posttest at T0 did not perform better than the control group at T10 (all uncorrected p > .052,all Tukey-corrected p > .210).In contrast, with the exception of the "manipulative vs non-manipulative" part of the manipulation technique recognition task (p = .268),the inoculation group that did receive a posttest at T0 performed much better at T10 than the control group that also received a posttest at T0 on all outcome measures (all p < .047).However, preregistered Tukey corrections render two of these comparisons no longer significant: manipulativeness discernment (p Tukey = .091)and sharing discernment (p Tukey = .191),potentially due to insufficient power; see Supporting Information S1: Table 9 for a full overview of post hoc comparisons.It should also be noted that a significant difference in performance at T10 between the inoculation group with immediate posttest at T0 and the inoculation group without immediate posttest would provide even stronger support of H2.
However, as in study 1, participants who played the inoculation game and received an immediate posttest did not display significantly greater manipulativeness discernment, confidence, or sharing discernment than those in the inoculation group without immediate posttest.Nonetheless, the pattern of results is clear and highly similar to study 1: 10 days postintervention, inoculation group participants who had received an immediate posttest performed substantially better at a battery of relevant outcome measures than a control group, whereas participants who had not received an immediate posttest did not.
With respect to H3, we find that inoculation group participants who received an immediate posttest had significantly better memory of the relevant content and lessons learned in the Bad Vaxx game compared to the equivalent control group both at T0 (p Tukey < .001,d = 2.730) and at T10 (p Tukey < .001,d = 1.356), indicating that participants both learned something from the game and still remembered these lessons 10 days later; see Supporting Information S1: Table 10.However, while we find that both motivation (p Tukey = .028)and generalized threat (a composite measure of our threat indices; p Tukey = .032)were significantly higher in the inoculation group at T0, this effect disappeared at T10.We find no significant between-group differences at T10 for the composite threat index, apprehensive threat, motivation to resist misinformation, fear (of vaccine misinformation), talk (how much people talked about or discussed vaccine misinformation), or issue involvement (how much vaccine misinformation means to people); all Tukey-corrected p > .062with the sole exception of "fear" (how afraid people are of vaccine misinformation), which was significantly higher in the inoculation group without immediate posttest than the control group without immediate posttest at T10 (p Tukey = .021).This indicates that threat, the role of which is to motivate careful (central) processing of the inoculation intervention material and to motivate the accumulation of additional information (e.g., continuing to internally counterargue or "think through" the issue; Banas & Richards, 2017;Compton, 2021;Compton & Ivanov, 2012), indeed plays a role immediately after the persuasive attack, but dissipates relatively quickly afterwards.See Supporting Information S1: Table 11 for a full overview. 8 When we compare both inoculation groups (with and without immediate posttest), we find that participants who had received an immediate posttest were significantly better at remembering the lessons from the game than those who had not (p Tukey = .003,d = 0.416) at T10; see Supporting Information S1: Table 10 and Figure 3.However, both inoculation groups did not differ significantly from each other at T10 for any of the motivation or threat measures (all Tukey-corrected p > .062,see Supporting Information S1: Table 11).This indicates that "forgetting" may play a role in the reduced effectiveness of the intervention at T10 when no immediate posttest is administered (i.e., hypothesis H2).These results provide partial support for H3 in that memory matters a great deal for the longevity of the inoculation effect, whereas threat and motivation appear to play a less important role at this later stage This is in line with the memory-motivation model of inoculation longevity proposed by Maertens et al. (2024), who also find that memory is by far the most important predictor of intervention decay and that threat plays a role early on (i.e., at the inoculation stage) in improving learning and memory (i.e., the role of threat is to motivate careful processing of the inoculation intervention).See Figure 3.

| General discussion and conclusion
Across two preregistered studies, we investigated (1) whether administering additional, irrelevant stimuli alongside a misinformation intervention interferes with its effectiveness, and (2) whether administering a posttest immediately after a misinformation intervention plays a role in the longevity of the intervention effect.With respect to the former, we find no evidence that interference is a significant problem, at least in the short term.This is a positive finding, as in the real world (such as when watching YouTube videos) people encounter all kinds of stimuli alongside any potential antimisinformation intervention, and it is helpful to know that the (immediate) inoculation effect conferred by the video is strong enough to withstand such (relatively minor) distractions (Blasiman et al., 2018;Craik, 2014;Middlebrooks et al., 2017).Although Roozenbeek et al. (2022) included a field test on YouTube where distraction was assumed to be implicit, here we validate this assumption more explicitly.
With respect to the latter, both studies show strong support for the notion that immediate posttests help strengthen the effect of the intervention and stave off decay for a substantial amount of time.We find these effects for two very different types of inoculation interventions, a video and a game, both of which have previously been extensively validated using the same outcome measures used in this study (Appel et al., 2024;Maertens et al., 2024;Roozenbeek et al., 2022).This shows that, in the absence of a further boost to memory, rapid decay can be expected even with highly robust and replicable intervention-outcome measure pairs.Maertens et al. (2024) propose a "memory-motivation" model of intervention effect retention, providing evidence for an exponential decay in effects if memory is not strengthened or "boosted."They also report that how well people remember the content of the intervention is by far the most important factor in how long the effect remains over time.Our study provides further support for this model.
Importantly, our study 2 shows that inoculation group participants who received an immediate posttest were significantly more likely than control participants to correctly remember the lessons from the intervention 10 days later, whereas inoculated participants who only went through the intervention without doing a posttest immediately afterwards performed no differently than controls.While it is important to note that we did not observe a statistically significant difference in performance between the two inoculation groups, even after a 10-day delay, this relative difference in performance nevertheless provides support for the idea that the inoculation effect decays quicker because participants forgot the relevant lessons more quickly if they did not receive an immediate posttest.Indeed, this model is in accordance with research on debunking approaches to misinformation, which has also established the importance of memory.Specifically, memory for a correction was shown to be strongly associated with reduced belief in the corrected misinformation (Fazio et al., 2023;Swire-Thompson et al., 2023).Thus, in line with previous findings on memory retention after watching a video (Bird et al., 2015) and broader research on the learning benefits of retrieval practice (Agarwal et al., 2021;Carpenter et al., 2022), we posit that a test of newly acquired information is a critical component for strengthening memory.And Leder et al. ( 2024) find F I G U R E 3 Boxplots with distribution and data jitter for "memory" (how well people remembered relevant information from the Bad Vaxx game at T10 (10 days after the intervention).A higher score indicates better performance on the battery of memory tasks.Note that inoculated participants who received an immediate posttest (at T0) and again at T10 scored significantly higher on the memory tasks than inoculated participants who only received a posttest at T10 (p Tukey = 0.003, d = 0.416).Control group participants who received no immediate posttest performed at chance level (as they were given no information whatsoever about vaccines or vaccine misinformation).We expect that control group participants who received an immediate posttest at T0 performed better at T10 than control group participants with no immediate posttest (despite not receiving the intervention) due to greater familiarity with the posttest demands and stimuli used.All between-group differences shown in the figure are significant at p Tukey < .003;see Supporting Information S1: Table 10.
preliminary evidence that memory-strengthening exercises administered at the end of an inoculation game boost memory and hence item task performance 1 week postintervention.
That said, there is evidence that the intervention effect persists at least for a modest duration even if no posttest is administered: Roozenbeek et al. ( 2022) conducted a randomized controlled field trial on YouTube with the same "emotional language" video used in the present study, where they administered a posttest (in the form of a single multiplechoice question) a median 18 h after watching the video as a YouTube ad.They report a significant (if relatively small) improvement in the correct identification of manipulative content, indicating that some of the inoculation effect still remains about 18 h after viewing.
Our findings also complement a line of inoculation studies that seek to optimize inoculation treatments with "booster shots," much like booster doses are administered to sustain vaccinations against disease, like COVID-19, something suggested in the earliest inoculation theory research (see McGuire, 1961).To date, most booster research in inoculation theory uses a second full inoculation treatment message some time after the initial intervention (e.g., Pfau et al., 2006).Whereas some have found enhanced resistance from using boosters (e.g., Pfau et al., 2006), others have not (e.g., Pfau et al., 1990) deployed (e.g., on social media).For instance, it may be very difficult to persuade social media users to not only watch an inoculation video (or other intervention) as an ad on YouTube, Facebook, or another platform, but also to then voluntarily engage in some kind of exercise that helps practice the lessons from the intervention.This, we believe, is a tall order: it is difficult enough to get people to watch ads or engage with (digital) literacy interventions to begin with; how many people will then also voluntarily agree to practice what they learned?We argue that these findings should prompt misinformation researchers to carefully think about intervention design and implementation.
For instance, future research may investigate how including exercises with real-time feedback or inductive learning in media literacy interventions or "fake news" games impacts their efficacy and longevity (Motz et al., 2022).One study by Leder et al. (2024) showed promising results for the potential of such feedback  3-5.
6 An example a question measuring apprehensive threat is "The idea that there is the possibility you may come in to contact with deception techniques that are so effective that they might cause you to be misled is [dangerous]".The accessibility question was worded as follows: "Compared to other issues, how often do you think about vaccine misinformation?";see Supporting Information S1: Table 12.
7 There are various conceptualizations of "threat" in inoculation theory; in some cases, it can refer to the recognition that one's position on an issue (e.g., the efficacy of vaccines) is vulnerable to future persuasive attacks (e.g., Banas & Richards, 2017;Compton, 2021;Compton & Ivanov, 2012), whereas in the present study and other studies (e.g., Basol et al., 2021;Maertens et al., 2024) threat refers to the extent to which the persuasive attack (e.g., vaccine misinformation) is perceived as threatening.

eH1.
People who receive an immediate posttest after watching a ~30 s inoculation [T1] are better at differentiating emotionally manipulative social media posts from neutral ones, compared to those who watched the inoculation video without an immediate posttest, when tested 48 h later [T2].

Figure 2
Figure 2 for boxplots and violin plots with data jitter.See Supporting Information S1: Tables 2-5 for the full results, including nonpreregistered Bonferroni-corrected p values (which do not change the results reported here).See Supporting Information S1: Figure 1 for a series of stacked density plots.Next, we test exploratory hypothesis eH2.At T1, the between-group effect size for manipulativeness discernment is d = 0.545, descriptively larger than the effect size of d = 0.439 obtained byMaertens et al. (2024) and in the same realm as the effect sizes found inRoozenbeek et al. (2022) of d = 0.49 and d = 0.67 (this paper included two separate studies with the "emotional language" inoculation intervention).Effect sizes for the other three outcome measures are also descriptively similar to those found byMaertens et al. (2024), although we note that the Using a previously validated video-based inoculation intervention, this study sought to address two open questions in misinformation interventions research: (1) whether providing additional, irrelevant stimuli alongside an intervention interferes with its effectiveness, and (2) whether posttests administered ) we used a game-based instead of a video-based inoculation intervention.Specifically, inoculation group participants played Bad Vaxx, a 10-min inoculation game about vaccine misinformation.Over the course of four levels, players are tasked with "defeating" four characters(Ann McDotal, Dr Forge, Ali Natural, and Mystic Mac), each of which represents a different manipulation technique that previous studies have identified as being commonly used in vaccine misinformation(Kata, 2010(Kata, , 2012)): emotional storytelling, fake expertise/pseudoscience, the naturalistic fallacy, and conspiratorial reasoning.Control group participants played Tetris.In a different study(Appel et al., 2024), Bad Vaxx was shown to substantially improve people's ability to discern vaccine misinformation from neutral social media content about vaccines; we consider the present study in part a replication of this work F I G U R E 2 Boxplots with distribution and data jitter for the four outcome measures at T2 (48 h after the intervention): manipulativeness discernment (a), confidence (b), trustworthiness discernment (c), and sharing discernment (d).
exercises to boost the longevity of Bad News, a different inoculation game to the one tested in the present study.Nonetheless, we also note that more research is needed to explore whether our findings hold up for text-based "boosting" interventions (such as media literacy tips or infographics).(M diff = 0.012, t(259) = 0.084, p Tukey > .999,d = 0.010, 95% CI [−0.234, 0.255]), trustworthiness discernment (M diff = 0.077, t(259) = 0.504, p Tukey = .958,d = 0.063, 95% CI [−0.307, 0.182]) and sharing discernment (M diff = −0.037,t(259) = −0.310,p Tukey = 0.990, d = −0.039,95% CI [−0.282, 0.206]) show similar conclusions.See Supporting Information S1: Tables Hertwig & Grüne-Yanoff, 2017;Kozyreva et al., 2024)nce, and perhaps, test-focused boosters will show more efficacy than repeated inoculation interventions have, much likeCompton and Ivanov (2012)found that incorporating a threat measure into the inoculation intervention message boosted resistance.In our view, our findings have substantial implications for the scalability of misinformation interventions that rely on participants learning and remembering new information (i.e., "boosts," seeHertwig & Grüne-Yanoff, 2017;Kozyreva et al., 2024): if merely watching a video, playing a game, or reading a piece of text (without an opportunity to practise what is learned) is not enough to prevent immediate forgetting of manipulativeness discernment skills, this requires a careful rethinking of how interventions are