Timing of descriptions shapes experience-based risky choice

Summary Risky decisions based on the combination of different sources of information (e.g., decisions from description-plus-experience) have mostly been ignored, as research has focused on examining each source separately. Across three experiments, we explore the intricate relationship between experience and description by manipulating when descriptive information about risky options is made available during an experience-based task. The results show that the amount of prior experience moderates the way that descriptive information is considered and integrated in the decision-making process: Descriptions affected behavior more when participants had little experience with the task, whereas their effect was less pronounced with extended experience. This relationship reversed when participants had access to foregone payoffs, with descriptions being considered more when participants had more time to interact with the task. Potential mechanisms and theoretical accounts are discussed with an emphasis on how the results and conclusions of the present work can be applied to the effective design of warning labels.

situations are ubiquitous in every day life, as we frequently make decisions relying on both descriptions and experience, such as doctors who rely on both their own personal experience and reports from academic journals when choosing between treatments. The majority of this "description-plus-experience" literature has looked at the interaction between the two sources of information when both are always available (e.g., Jessup et al., 2008;Plonsky & Teodorescu, 2020;Rakow & Miler, 2009;. Cognitive modelling analysis of description-plus-experience tasks has shown that the observed behavior can be adequately described by models based on experience alone Lejarraga & Gonzalez, 2011), unless descriptions provide novel useful information which cannot easily be inferred from feedback-thus requiring the incorporation of descriptions into the model (Weiss-Cohen et al., 2016. However, in all the previous description-plusexperience research, descriptions were present from the beginning of the task, and before experiential feedback is provided, influencing behavior from the first selection (see also Ben-Asher et al., 2013). When feedback was received in subsequent decisions (i.e., via experience), participants had already been informed by descriptions. Therefore, the extant research in description-plusexperience has mostly investigated the behavioral influence of experiences posterior to the influence of descriptions. This might be a common situation, as we are often asked to make decisions under risk where we are exposed to descriptive information before gathering any experience from our choices, such as reading instructions manuals before using a new device, reading reviews before going to a new restaurant, or reading the patient information leaflet before taking a new medication.
The reversed situation should not be overlooked, when prior experience is accumulated before any descriptive information is received. Individuals will often engage in risky behavior, such as riding a bicycle without a helmet or crossing the road without due attention, before seeing a government campaign describing the dangers of road accidents. Commuters have been hurriedly running in stations around the world and might one day stumble upon a poster describing the dangers associated with running down the stairs. These descriptions about risks can come in the form of warnings: Labels and signs which can be considered as descriptions of otherwise rarely experienced events with catastrophic consequences. Descriptive warnings are often used in an attempt to influence behavior and are typically combined with contradictory personal experiences. For example, "dangerno diving" signs newly placed in the shallow end of a pool will be seen by individuals with and without prior diving experience Goldhaber and DeTurck (1989). Both novice and experienced workers might come across new warning signs in a factory floor requiring the usage of personal protective gear. Other notable historical examples include drinking alcohol and smoking before legislation mandating the placement of risk warning labels on bottles and packaging, and driving experience before being exposed to safety belt usage promotional campaigns. Crucially, the amount of prior experience before encountering such descriptions and warnings can vary widely in similar situations: Some individuals might have been engaging in risky behavior long before reading a description of the risks for the first time, whereas others might see them earlier.
The literature on warnings includes prior experience as a potential moderating factor of their effectiveness (Laughery, 2006;Laughery & Wogalter, 2006;Rogers et al., 2000). Newall and Parker (2019) showed how more financially literate investors, who therefore have more prior experience with financial markets, were less likely to be influenced by descriptive warnings about investment choices. Wulff et al. (2017) suggested that the influence of warnings on future behavior is likely to depend on people's past and recent experience.
Similarly, Hertwig et al. (2018) proposed that rich experience may discount the effect of a warning.
In a description-plus-experience task, Barron et al. (2008) investigated the influence of prior experience on the risk-reducing impact of descriptive warnings. Participants were presented with descriptive information about the choice options either at the beginning of the task (no prior experience), or halfway through the task (prior experience). Participants were asked to choose repeatedly between two options, one lower-value sure alternative with guaranteed outcomes, and one risky alternative which returned either a higher-value frequent outcome (99.99% of the time) or a very rare (0.01%) negative event which led to catastrophic losses (as in most warning labels in everyday life, e.g., wearing safety goggles to protect your eyes from the very unlikely event of flying debris). These descriptive warnings had the desired effect of reducing selections of the risky choice; however, their effect was strongest for participants without prior experience and more subdued for participants who had prior experience (for similar findings in medical decision making, see also Miron-Shatz et al., 2010). Crucially, descriptions were framed as warnings, mentioning only the risky part of the options (i.e., the rare losses), instead of a full description of outcomes. The results showed that warning labels had a stronger impact on behavior when presented earlier, before any personal experience; the impact of warning labels on behavior appeared to be discounted when these were presented later, after personal experience had been accumulated.

| Discounted descriptions
The idea that descriptions can be discounted in the presence of experience, when the two are presented concurrently, has been suggested by Lejarraga (2010). According to the author, experience is preferred over description because the former is more natural to process, whereas the latter requires more costly cognitive effort-as a result, experience can outweigh descriptions when both are available (see also Hertwig et al., 2018). Previously, we have empirically demonstrated how descriptions and experience are combined in the decision-making process, with different weights allocated to each source of information (Weiss-Cohen et al., 2016. We proposed that the weights allocated to descriptions can be moderated by a number of factors, two of which we have explored so far: the plausibility of the descriptions in the face of experiential evidence (Weiss- Cohen et al., 2016), and the complexity of the task (Weiss -Cohen et al., 2018). We believe that prior experience can be another moderator for the discounting of descriptions, which could explain the behavior observed by Barron et al. (2008): The more prior experience (and information) an individual has accumulated, the lower the impact of descriptions that are presented later.
Theories of reinforcement learning (RL) allow for the diminishing impact of information over time and support the discounting of delayed descriptions. In RL models, learning rates govern how much new information is incorporated in previously learned beliefs (Sutton & Barto, 1998). In models with decreasing learning rates, information which is received later in the task has a smaller effect on learning than information received earlier (Yechiam & Busemeyer, 2005). Decreasing learning rates are commonly observed in most learned behavior, as it is a compromise between having fixed low or fixed high learning rates (Murata et al., 2002). High learning rates are useful when encountering new environments, such as the beginning of a task, when quick learning is desirable (Doya, 2002;Konstantinidis et al., 2015). However, high learning rates translate into new information easily overwriting old information and very volatile behavior which never stabilizes (Yechiam & Rakow, 2012). Lower learning rates can smooth out learning and ensure that old knowledge is preserved, thereby helping stabilize behavior, but they can slow down the learning process (Cohen et al., 2007). Therefore, a learning rate that starts high but reduces over time allows for quick, accurate, and efficient learning in static environments (Doya, 2002;Rakow & Miler, 2009). 1 This is also in line with our prediction that late-presented descriptions will be discounted in comparison to early-presented descriptions.
Bayesian theories of learning and information integration can also explain the diminishing effect of description. Bayesian agents update prior beliefs with new information to generate posterior beliefs. Stronger priors, supported by extensive information (e.g., more prior experience), are associated with lower uncertainty and are hardly influenced by new observed data; weak priors, supported by limited information, are more uncertain and will easily be influenced by new information (Griffiths et al., 2008). More prior experience can thus be associated with stronger prior beliefs. New information should influence Bayesian agents more strongly when they have limited prior experience and weaker priors. As in reinforcement learning, such processes are useful for smoothing out the learning process, reducing volatility over time (Griffiths et al., 2008). This has been confirmed with cognitive models that show that most learning happens when uncertainty is highest, such as the beginning of tasks or when encountering new scenarios (Dayan & Niv, 2008;Speekenbrink & Konstantinidis, 2015).
Another behavioral phenomenon which can interfere in the learning process is the hot stove effect (Denrell & March, 2001). 2 The hot stove effect states that bad outcomes reduce the attractiveness of an option, causing those options to be avoided (Biele et al., 2009). By avoiding an option after a bad outcome has occurred, the ability to collect more information about that option is severely constrained, as feedback from that option will no longer be observed. With no new information beeing collected, the bad outcome observed will remain the most recent (and most salient) observation associated with that alternative, decreasing its perceived value. As a result, this avoidance behavior can create an imbalance between good and bad outcomes, with the latter having stronger and longer lasting impacts on learning than the former. The earlier the bad outcomes are observed (and this avoidance begins), the stronger is the hot stove effect. Similar results have been reported in other purely experiential tasks where bad outcomes occur early in the course of a task. For instance, Fellows and Farah (2005) have shown that losses in the first few choices can have a greater, longer-lasting impact in risk avoidance than later losses (see also Huber et al., 2011). By extension, we can expect early descriptions (containing negative information) to trigger a stronger hot stove effect than late descriptions. This effect however only presents itself in tasks with partial-feedback (in which only the outcome from the selected option is provided after every trial): If full-feedback is available (in which outcomes for all options are given after every trial), the ability to accumulate information is not impacted, as individuals can still gather information from their foregone alternatives. Full-feedback can correct the biases introduced by bad outcomes, eliminating the hot stove effect (Luria et al., 2017;).

| Overview of experiments
Reinforcement learning and Bayesian updating can explain the discounting of descriptions observed by Barron et al. (2008) in the presence of prior experience. However, in their study, participants in the prior experience condition went through half of the task (50 trials) before being presented with descriptions, which was a binary experimental manipulation (i.e., no experience vs. prior experience). To the best of our knowledge, no empirical research has explored how different amounts of prior experience influence the impact of descriptive warnings on risk-taking behavior. Understanding how prior experience interacts with descriptions can help create better and more efficient risk communication and warnings.
In this paper, we seek to determine whether different amounts of prior experience can influence the level of discounting of descriptions, with participants being exposed to different amounts of prior experience before descriptions are shown. In three experiments, we control for the amount of prior experience before descriptions are revealed and availability of outcome feedback (partial or full-feedback procedure). We employ variations of a popular decisions-from-experience paradigm, the Iowa Gambling Task (IGT; Bechara et al., 1994;Steingroever et al., 2013), with the addition of descriptions. While descriptions have been added before to the IGT (Weiss- Cohen et al., 2018), our novel manipulation includes the introduction of descriptions at different points during the task, after participants have had the opportunity to accumulate different amounts of prior experience.
We predict that the impact of descriptions will be highest when there is no prior experience, replicating the results in Barron et al. (2008). In addition, we expect the amount of prior experience to moderate the impact of descriptions on behavior. Descriptions that appear later, when participants have already had a large amount of prior experience with the task, should not influence behavior as strongly as descriptions that appear early. If descriptions provide useful information that can help participants perform better on the task, such as in the form of warnings about risks to be avoided, then early presentation of descriptions should help improve performance more than later ones.
We also compare paradigms with full feedback and partial feedback, to explore if the relationship between prior experience and descriptions is influenced by the presence of foregone payoffs, via the hot stove effect. If it is, we expect the effect of prior experience to be strongest when partial feedback is given.

| EXPERIMENT 1
This first experiment is based on a simple alteration to the IGT. In the IGT, participants are asked to make decisions based on feedback alone, without any descriptions. We altered the traditional paradigm  Only positive bonuses were paid: 12 participants ended the task with negative bonuses which were not deducted from their final compensation. 4

| Design
Experiment 1 was a between-subjects design with six experimental conditions manipulating when descriptions were presented to participants: one experience-only condition, in which descriptions were not presented at all and participants learned the outcomes of their choices through experience only (E: N = 33); one descriptionbefore-experience condition, with descriptions available from the first trial, prior to any accumulation of experiential knowledge about the available options (DE: N = 33); and four experience-beforedescription conditions, with descriptions appearing at different points during the task (ED20: N = 34; ED40: N = 33; ED60: N = 33; and ED80: N = 30; the numbers after the ED represent the amount of trials before the presentation of descriptions). In the ED conditions, participants first went through a number of trials without any descriptions: In condition ED20, participants went through 20 trials without descriptions, with descriptions appearing from trial 21 onward; in ED40, descriptions were shown from trial 41 onward, and so forth. After appearing, descriptions remained present until the end of the task (Figure 1).

| Task
The task was based on the traditional IGT, with the addition of descriptions (DIGT; Weiss- Cohen et al., 2018). The original IGT outcome schedule was used, although without a predetermined fixed order as in the original task (Bechara et al., 1994, Figure 1). Instead, we used a pseudo-randomization approach in blocks of 20 trials each, ensuring that the correct proportion of cards was respected but in a randomized order in each block (Camilleri & Newell, 2011a;Weiss-Cohen et al., 2018). Participants could select from four decks of cards: two advantageous and two disadvantageous, in terms of their overall expected returns. The disadvantageous decks, traditionally labelled A and B in the IGT literature, provide higher short-term gains of 100 points for most of the cards, but overall negative long-term expected values of -25 points each, due to some cards providing some very large losses (-1,150 points in some cases). The two advantageous decks, traditionally labelled C and D, provide smaller shortterm gains of 50 points for most of the cards, and because they do not have the same large losses (the largest loss is -200 points), they return an overall positive long-term expected value of 25 points each.
The actual schedule of outcomes can be seen in the descriptions in Figure 1b. Feedback was provided as net outcomes, unlike the IGT where gains and losses were presented separately.
The four decks of cards were shown face down side by side, and their positioning on screen was randomized. The patterns seen on the backs of the decks were also randomly assigned. Partial-feedback was given after each trial, with only the outcome of the chosen option being revealed. The between-subjects conditions determined the presence and timing of the descriptions displayed underneath the decks on screen. As in the original IGT, participants started with 2000 points, with each card revealed adding or deducting points from the running total. Points were converted into a monetary bonus at a rate of US$0.10 per 1,000 points, and the running total in both points and money was always displayed at the bottom of the screen. The task was self-paced over 100 trials and was completed on average in 8.8 min (SD = 4.2). 3 All raw data and R analyses scripts from the three experiments can be found on-line at https://osf.io/khyeq/. 4 We also ran all the analyses after excluding the participants who reached negative accumulated points before the presentation of descriptions. The same patterns of results are found and the same conclusions are reached across all three experiments.

| Results
We analyzed selections from advantageous decks, defined as the number of cards selected from decks C and D for each block of 20 trials: This is a measure of task performance, with higher selections resulting from individuals being able to identify the advantageous decks (Steingroever et al., 2013). Two separate analyses were conducted: first, a comparison across all trials, to verify that the previous findings from Barron et al. (2008) and Weiss-Cohen et al. (2016 were replicated and that descriptions influenced behavior, by informing participants of risks and reducing selections from the riskier options; and second, an analysis measuring the magnitude of the impact of descriptions on behavior, by comparing selections of advantageous decks in the blocks of 20 trials immediately before and after descriptions appeared.

| All trials
The analysis across the entire task aimed to verify the overall influence of descriptions on choice behavior. We used a generalized F I G U R E 2 Results from Experiment 1: (a) Evolution of selections from advantageous decks for each block of 20 trials by condition; (b) log odds-ratios (logOR) of the impact of descriptions on change in selections from advantageous decks for each condition (after/before). The number after the ED is the trial in which descriptions were first revealed to participants. The error bars represent the standard errors of the mean [Colour figure can be viewed at wileyonlinelibrary.com] Therefore, we grouped the four ED conditions together as ED all . We subsequently compared E, DE, and ED all . The timing of appearance of descriptions significantly influenced task performance (E vs. DE vs. ED all : χ 2 (2) = 45.67, p < .001). A post hoc analysis showed a significant difference between the DE and E conditions (DE=89.8%, E=68.3%, z=3.94, p=.001), replicating the results in Weiss- Cohen et al. (2018), where participants performed better when provided with descriptions from the beginning of the task. There was also a significant difference between the aggregated ED all and the DE conditions (ED all = 63.6%, z = 5.00, p < .001), replicating the results in Barron et al. (2008), where descriptions presented from the beginning of the task led to better task performance than descriptions presented later in the task. There was no difference between the E and ED all condition (z = 0.68, p = .78), likely indicating that late presentations of descriptions led to similar overall task performance as no descriptions at all. This effect is confounded by the fact that in some of those ED conditions participants did not have access to descriptions for the majority of the duration of the task, therefore making a substantial part of the ED and E conditions similar. The effect of block was significant (χ 2 (1) = 75.12, p < .001) with a positive coefficient (b = 0.44, SE = 0.14, z = 3.13, p = .002), indicating that participants selected more advantageous decks as the task progressed. The interaction between trial and timing of descriptions was not significant (χ 2 (2) = 0.06, p = .97).

| Impact of descriptions
We focused next on the impact of descriptions, that is, the change in selections from advantageous decks, comparing the blocks of 20 trials immediately before and after descriptions appeared. For example, in condition ED60, the behavioral impact of descriptions was calculated by comparing behavior in trials 41-60, the block of trials immediately preceding the appearance of descriptions, against trials 61-80, after descriptions appeared. A generalized linear mixed-effects binomial model with a logit link function was used.
The fixed effects were the amount of prior experience, as measured by the timing of appearance of descriptions (ED20, ED40, ED60, and ED80, representing prior experience of 20, 40, 60, and 80 trials respectively), 5 with polynomial coding (i.e., linear, quadratic and cubic contrasts), and a binary factor identifying the presence of descriptions (i.e., identifying if selections were made either before or after descriptions appeared). The model also contained a random intercept for each participant.
Overall, there was a significant effect of the presence of descriptions (χ 2 (1) = 36.32, p < .001), with participants selecting more often from the advantageous decks after descriptions were displayed (Before = 53.4%, After = 62.2%, logOR = 0.36, SE = 0.06, z = 5.67, p < .001). The log-odds-ratio represents the change in the frequency of selections of the advantageous decks from before to after the descriptions first appeared: Positive values indicate an increase in selections of advantageous decks. The focus of this analysis was the interaction between the amount of prior experience (based on the timing of descriptions) and the presence of descriptions (before vs. after), which returned a significant effect (χ 2 (3) = 45.11, p < .001).
There was also an overall main significant effect of the amount of prior experience (χ 2 (3) = 7.88, p = .049). As the task progressed, participants selected on average more often from the advantageous decks, regardless of the presence of descriptions.

| Discussion
The results of this experiment confirmed our hypothesis that increased prior experience leads to reductions in the influence of descriptive information on choice behavior. When comparing the different ED conditions against the selections from advantageous decks observed before and after the appearance of descriptions, the hypothesis that more experience leads to a lower impact of descriptions was confirmed. The impact of descriptions was lower for participants who were presented with descriptions later in the task and therefore had more time to accumulate larger amounts of prior experience. Based on theories of reinforcement learning, such behavior (i.e., discounting of information received later in the task) can be explained by learning rates which reduce over time. Descriptive information that is presented early in the task is integrated at higher rates as a result of early high learning rates and influence behavior more. Later presentation of descriptive information, when learning rates are lower, is greatly discounted, and does not influence behavior as strongly.
Very late appearance of descriptions, at trial 80, was even deleterious to task performance, leading to a lower selection of advantageous decks (Figure 2b). This is a perverse influence of descriptions, resulting in a deterioration of task performance from the presentation of descriptive information. Similar results have been observed in previous research, with warning messages increasing risk-taking behavior in certain situations: the opposite of the desired effect of such warnings (Ben-Ari et al., 1999;Ferraro et al., 2005;Hansen et al., 2010). It is also possible that descriptions reminded participants of the higher winnings associated with the high-risk alternatives, which on average participants would be selecting less often towards the end of the task (Ginley et al., 2016).
There was no observable difference in average behavior across all trials between participants in the E condition and participants in the ED conditions. This highlights the importance of early presentation of descriptions to ensure their maximum influence, before any prior experience. Participants performed significantly better when descriptions were shown before any experience (DE), choosing the 5 The other two conditions, E and DE were not included in the analysis because they did not include a transition from no-description to description: They were either never present, or always present, respectively. advantageous decks more frequently throughout, in comparison to all other conditions. It appears that presentation of descriptions before any prior experience is an important moderator of the influence of descriptions in selection rates observed in this task, to ensure maximum behavioral impact (see Miron-Shatz et al., 2010). Experiment 1 also replicated previous research showing that descriptions influence behavior and help improve performance in complex tasks. In a complex task such as the IGT, which demands excessive cognitive effort to learn by experience alone (Stocco et al., 2009), descriptions are taken into consideration and integrated into the decision-making process, helping identify the advantageous decks (see Weiss-Cohen et al., 2018). Participants in the conditions with descriptions performed better on the task than participants in the condition with no descriptions. The findings from Barron et al. (2008), who showed that participants who had access to descriptions prior to any experience performed better than participants who received descriptions halfway through the task, were also replicated here with the significant difference between the DE and ED conditions.
The current findings might be obscured by the influence of learning effects. In this experiment, the information provided by descriptions matched the experiential feedback; thus, after increased exposure with the task (and learning of the advantageous strategy, e.g., in ED60 and ED80 conditions), 6 the appearance of descriptions did not add anything new to participants' knowledge. Consequently, the influence of additional information provided when the environment had been well-learned would be minimal. Previously, we showed that descriptions influence behavior more when they present individuals with novel information (Weiss- Cohen et al., 2016Cohen et al., , 2018. In the next experiment, we employ a task in which descriptions always provide novel information which cannot be learned by experience alone.

| EXPERIMENT 2
To ensure that descriptions always provided novel information, the second experiment was designed to be a task which could not be learned via experience alone. This was accomplished by introducing low-probability (rare) outcomes, which never occurred (i.e., not experienced by participants) before descriptions first appeared. When descriptions were shown to participants, they contained information about all possible outcomes, including these rare events that had never been observed experientially before and thus provided useful novel information. The outcome schedule in Experiment 2 closely followed the nature of the original IGT: Decks A and B provided frequent larger positive rewards of 250 points but rare large losses; whereas decks C and D provided frequent smaller positive rewards of 100 points and no rare large losses.

| Design
Experiment 2 was a between-subjects design with four conditions, manipulating the amount of prior experience before descriptions were presented to participants as in Experiment 1: ED20, ED40, ED60, and ED80, with the numbers after the ED representing the total number of trials before descriptions were first shown (N for each condition: ED20=35, ED40=34, ED60=33, ED80=33). Crucially, the descriptions warned participants of high-magnitude low-probability outcomes that only occurred in the last 20 trials of the task (trials 101-120) and therefore always provided novel information, because these rare events never occurred before the descriptions appeared. This approach also allowed for a "clean" block of 20 trials in every experimental condition, without any rare events, before and after descriptions appeared.

| Task
The task was self-paced, lasted 120 trials, and was completed on average in 9.22 min (SD = 3.56). Participants started the task with 5,000 points, 7 and points were converted into money at a rate of US$0.10 per 1,000 points. In order to ensure that the descriptions provided novel information, all the low-probability (rare) events occurred in the last 20 trials of the task. In the first 100 trials, all decks had an EV of zero; decks could be categorized as advantageous or disadvantageous based on the rare events occurring in the last 20 trials. Decks A and B were the high-risk options, because they were associated with large losses in the last 20 trials; and decks C and D were the low-risk options, because they did not produce any large losses in the last 20 trials (see Table 1).
The purpose of keeping the EV equal to zero up to trial 100 was to ensure that participants did not create a preference 6 Bechara et al. (1997) and Maia and McClelland (2005) report that by trial 50 in the IGT (i.e., halfway through the task) most participants already have a good conscious knowledge of the advantageous strategy (see also Konstantinidis & Shanks, 2014). 7 The larger number of starting points in comparison to Experiment 1 was to allow for the zero-EV nature of the task in the first 100 trials. As participants did not accumulate as many points during the task, the initial amount had to be increased to ensure an appropriate level of reward at the end of the task. towards any of the decks via experience alone, before descriptions appeared, which should produce almost similar choice rates between decks. However, it could be the case that participants would show a small preference towards the high-risk decks initially, as these decks produce larger frequent wins of +250 points compared to the smaller frequent wins of only +100 points in the low-risk decks.
This conflict between wins and losses is similar to the one in the traditional IGT, which leads to an observed preference towards the decks with higher frequent wins. When all trials are taken into account, the descriptions are representative of the cards contained within each deck, although not exactly matched, due to the rounding of the probabilities. However, prior to trial 100, the descriptions contradicted the experience. Contrary to Experiment 1, the descriptions always provided novel information to participants, because they included the rare events, which never appeared before the descriptions were shown.
For example, according to the descriptions in deck A, across all 120 trials, participants would observe two cards with −5,250 points (1.7% of the total number of trials, the descriptions were rounded for ease of understanding). These cards never appeared in the first 100 trials, but instead all of these high-magnitude low-probability loss cards appeared in the last 20 trials, thus completing the overall distribution of cards. In the case of deck A, this was done by replacing the −2,250 points cards in the last block with cards producing losses of −5250 points, and similarly for the other decks.

| Results
The impact of descriptions on selections from low-risk decks was analyzed with the same mixed-effects model as in Experiment 1, with two fixed effects: amount of prior experience (ED20, ED40, ED60, and ED80) and presence of descriptions (before vs. after).
The impact of descriptions on behavior can be seen in   Note. Probabilities were rounded. The descriptions were representative of the outcomes across all trials, however the rare events only occurred in the last 20 trials. The expected value for each individual deck in the first 100 trials (excluding the rare events) was zero for all decks. However, when taking into account the rare events presented in the last 20 trials (i.e., across all 120 trials of the task), the described EV in decks a and b were −60 points, and in decks c and d, they were +25 points.

| Discussion
The presence of descriptions in Experiment 2 influenced behavior in the predicted direction, replicating the findings in Experiment 1. The appearance of descriptions helped participants identify and select the low-risk decks more often, therefore avoiding the large losses associated with the high-risk decks. The reduction in selections from highrisk decks associated with the appearance of warning descriptions was moderated by the amount of prior experience, with larger reductions for participants with less prior experience. This adds further credence to our initial hypothesis that the amount of prior experience moderates the way that descriptive information is integrated into the decision making process.
In Experiment 1, this negative relationship between amount of experience and influence of descriptions could have been the result of incremental learning via experience over time, with descriptions not providing any novel useful information. This potentially confounding effect was eliminated in the current experiment, in which the descriptions always provided novel useful information that could not have been learned experientially. For the disadvantageous high-risk decks, the descriptive information acted as a warning against high-magnitude low-probability losses that never occurred before the descriptions appeared. Similarly for the advantageous low-risk decks, it provided information about rare treasures.
The observed pattern of the impact of descriptions might be the result of the hot stove effect (Denrell & March, 2001). Although the research on the hot stove effect deals with the influence of direct negative experiences, it is possible that the actual experience of bad outcomes is not needed, only its possibility (see also "mere presentation effect," Erev et al., 2008). Consequently, in D+E tasks, the descriptions might be acting as hot stoves, causing the options with bad outcomes to be avoided regardless of whether any bad outcomes have actually been experienced. Therefore, when a participant is presented with descriptions, they may act as if "burned" by the possibility of high losses with the high-risk decks and select them less frequently.
This lower frequency of selections amplify the effect of the descriptions, by not giving participants the opportunity to realize that in fact the high losses presented do not occur as frequently as expected, and until much later in the task. This can occur even when descriptions are true representations of the experience (as the ones used here), which should still create a mismatch between described and experienced outcomes: The expectation of individuals is generated from descriptions, in which rare events are overweighted; whereas the perception of individuals will be based on their experience, in which rare events are underweighted (Hertwig et al., 2004).
The hot stove effect however only presents itself in tasks with partial-feedback, as full-feedback will give participants the opportunity to gather information from the avoided options, correcting any potential biases. The possibility that the hot-stove effect can be a moderating factor is tested in the next experiment where full feedback is provided.

| EXPERIMENT 3
Although in Experiments 1 and 2 we employed partial-feedback, Experiment 3 introduced full-feedback, in order to explore if the influence of descriptions was a result of the hot stove effect. Although in paradigms with partial-feedback, participants are only shown outcomes for their selected options, in paradigms with full-feedback, outcome feedback is provided for all available options (i.e., including foregone payoffs).
Behavior in the same task (i.e., choice options with same characteristics) with partial-feedback and full-feedback can differ in predictable ways. The hot stove effect is only relevant in the context of tasks with partial-feedback, but not when full-feedback is provided (Camilleri & Newell, 2011b;Luria et al., 2017;. This is because the impairment of learning caused by the hot stove F I G U R E 3 Results from Experiment 2: (a) Evolution of selections from low-risk decks for each block of 20 trials by condition; (b) log oddsratios (logOR) of the impact of descriptions on change in selections from low-risk decks for each condition (after/before). The number after the ED is the trial in which descriptions were first revealed to participants. The error bars represent the standard errors of the mean [Colour figure can be viewed at wileyonlinelibrary.com] effect can only occur with partial-feedback: with full-feedback, individuals still have the opportunity to gather information about the options with bad outcomes that are no longer being selected. The hot stove effect bias is corrected by foregone payoffs, and the avoidance of options following bad outcomes is not sustained over long periods (but not completely eliminated, see also Worthy et al., 2013).
With the hot stove effect not taking hold in tasks with full-feedback, risk-taking is higher, leading to more risky choices overall (Grosskopf et al., 2006;Yechiam & Busemeyer, 2005. This is because the risky options are typically the ones with higher variance and more extreme outcomes: risky options tend to be the ones with the bad outcomes which will trigger the hot stove effect. The hot stove effect can artificially reduce the attractiveness of risky options in partial-feedback paradigms, whereas with full-feedback this bias is curtailed and the attractiveness of risky options is no longer artificially reduced. Therefore, the hot stove effect is associated with avoidance of high-risk options (Biele et al., 2009).
If the hot stove effect is behind the behavioral patterns observed from the delayed introduction of descriptions in the first two experiments, then we will observe a different pattern of behavior with fullfeedback. Participants will still be able to observe foregone feedback, and they might be attracted to select the high-risk decks (due to attractive wins occurring from selecting these decks) despite descriptions discouraging them from doing so. This should reduce the impact of the rare negative outcomes present in the descriptions. One potential hypothesis is that participants should select more often from the high-risk decks throughout the task, and the shift from the high-risk to low-risk decks after the appearance of descriptions should no longer be largest for earlier presentation of descriptions.

| Design
Experiment 3 used the same paradigm as Experiment 2, with four between-subjects conditions, manipulating the amount of prior experience when descriptions were presented to participants: ED20, ED40, ED60, and ED80, with the numbers after the ED representing the total number of trials before descriptions were first shown (N for each condition: ED20=35, ED40=36, ED60=36, ED80=33).

| Task
The task in Experiment 3 was the same as in Experiment 2, lasting 120 trials and was completed on average in 11.7 min (SD=5.6). The only alteration was the introduction of full-feedback: After each selection, feedback was presented for all four decks.

| Results
The impact of descriptions on selections from low-risk decks was analyzed with the same mixed-effects model as in Experiments 1 and 2, with two fixed effects: amount of prior experience (ED20, ED40, ED60, and ED80) and presence of descriptions (before vs. after).
We observed much higher rates of risk-taking behavior with fullfeedback in Experiment 3 than with partial-feedback in Experiment 2, with lower selections from the low-risk decks throughout the task, as shown in Figure 4a. This is consistent with behavior in similar tasks with full-feedback (e.g., Yechiam & Busemeyer, 2005. The main effect of the presence of descriptions, irrespective of when descriptions first appeared, was also significant (χ 2 (1) = 20.59, p < .001), with an increase in the selection from low-risk decks after descriptions appeared (Before=23.0%, After=28.5%, logOR=0.29, SE=0.07, z=4.45, p<.001). However, across all conditions, the impact of descriptions was lower than in Experiment 2, with a lower logodds-ratio, with translates into a lower increase in selection from lowrisk decks due to descriptions in Experiment 1 than in Experiment 2. Similar to Experiment 2, the main effect of amount of prior experience was not significant (χ 2 (3) = 5.67, p = .13). There was no difference in overall selections according to prior experience.

| Discussion
In Experiment 3, the influence of prior experience on the impact of descriptions did not follow the expected pattern as hypothesized in the introduction, and as observed in the previous experiments.
Instead, with full-feedback, we observed a behavioral pattern that was opposite to the one obtained with partial-feedback: When participants have access to complete feedback, the warning descriptions have a reduced impact when received earlier (with no prior experience), and an increased impact when received later (with more prior experience). 8 As the hot stove effect is bypassed with the introduction of full-feedback, the observed difference in behavior between full-feedback and partial-feedback confirms that the hot stove effect

| Attention
As observed by the lower odds-ratios in Experiment 3, across all experimental conditions, descriptions influenced behavior less when there was full-feedback, when compared to Experiments 1 and 2. One way in which this can be explained is by looking at how attention, a limited cognitive resource, is allocated. Full-feedback increases cognitive and attentional demand on the task, with considerably more information presented after each trial: in addition to a single obtained payoff, individuals also had to attend to three foregone payoffs after each trial (Ashby & Rakow, 2016). As shown by Weiss-Cohen et al.
(2018), descriptions do not influence behavior in tasks of high complexity, likely because individuals lack the attentional and cognitive bandwidth required to process them. Descriptions are more cognitively demanding than experience; therefore, it is expected that a preference is given to experience when cognitive load is higher (Lejarraga, 2010).
As individuals accumulate experience and develop a better understanding of the choice environment, their attention to feedback information reduces: this strategy is adapted to static environments, in which feedback become less informative, and therefore less important, with time (Ashby & Rakow, 2016;Grosskopf et al., 2006). This shift in attention can free cognitive resources to deal with other information, such as descriptions: therefore, later presented descriptions can have greater impact as they benefit from the lower attention given to feedback.

| Regret
The literature on decision affect theory indicates that choices are guided by emotions resulting from negative outcomes, such as regret and disappointment, both of which can be anticipated (pre-decisional) or experienced (post-decisional; Mellers et al., 1999). Disappointment can occur in partial-feedback environments, if the obtained outcome is worse than expected; whereas regret can only occur with full-feedback, when the outcome of the foregone option is better than that of the chosen option (Zeelenberg et al., 1996;Zeelenberg, 1999). Experi- F I G U R E 4 Results from Experiment 3: (a) Evolution of selections from low-risk decks for each block of 20 trials by condition; (b) log oddsratios (logOR) of the impact of descriptions on change in selections from low-risk decks for each condition (after/before). The number after the ED is the trial in which descriptions were first revealed to participants. The error bars represent the standard errors of the mean [Colour figure can be viewed at wileyonlinelibrary.com] Zeelenberg et al., 1996). More recently, by selectively manipulating exposure to foregone feedback, Plonsky and Teodorescu (2020) have experimentally confirmed that higher regret leads to higher risk-taking, even when participants have access to full descriptive information.
In Experiment 3, with full-feedback, we propose that descriptions triggered anticipated regret: individuals wanted to avoid the regret associated with experiencing a large loss (after choosing a high-risk deck), while foregoing the small frequent gain from an unchosen lowrisk deck. 9 This regret avoidance behavior, however, increases with experience. Cooke et al. (2001) suggest that individuals learn to anticipate and avoid regret over time, and adjust their future behavior to avoid repeated regret: more than one initial occasion of a regretful loss is needed (Ratner & Herbst, 2005). This idea of learned regret has been confirmed by neuroimaging research, which has shown that individuals become increasingly regret aversive over time, with an apparent cumulative effect of the experience of regret: participants' anticipated regret increased with experience (Coricelli, Critchley et al., 2005;Coricelli, Dolan et al., 2007). This accumulation and learning to react to anticipated regret can help explain why later descriptions were more influential than earlier ones in Experiment 3: higher anticipated regret triggered by later descriptions made participants shift more, whereas in early trials anticipated regret was low and not enough to make participants shift away from the high-risk decks.

| GENERAL DISCUSSION
The objective of the current work was to assess whether and how the accumulation of prior experience shapes the impact of descriptive information in decisions from description-plus-experience. Across three experiments, we found that the amount of prior experience moderates the influence of descriptions. The direction of this relationship was dependent on the amount of information provided at feedback: partial or full. In Experiments 1 and 2, with partial-feedback, there was a negative relationship between the amount of prior experience and the impact of descriptions on choice behavior: Descriptions influenced behavior more when participants had not accumulated prior experience; in such situations of reduced accumulation of experience with the task, choice behavior was more susceptible to influences caused by the descriptive information. In the conditions in which descriptions appeared in later stages of the task (as indexed by the amount of purely experiential trials), descriptions had a lower impact, with participants shifting less towards behavior predicted by descriptions. However, these patterns were reversed in Experiment 3, with full-feedback: when more information was provided after every trial, later descriptions influenced behavior more than earlier descriptions.

| Potential mechanisms
The behavioral patterns observed here can be explained by a bidirectional interaction between descriptive and experiential information and how this interaction influences the learning process: Prior experience can influence how future descriptions are integrated into the decision-making process; and, conversely, prior descriptions can influence how future experience is learned. Theories of reinforcement learning already include the idea that previously acquired knowledge interferes with the learning of new information (Wisniewski, 1995;Wisniewski & Medin, 1991). More recently, Hertwig et al. (2018) suggested that there should be influences on learning from both description and experience, which are two different but interacting ways of learning about the choice environment (Wulff et al., 2017).

| Prior experience influences how descriptive information is integrated
Theories of reinforcement learning show that prior knowledge can influence future learning, by encouraging the dismissal of information supporting new alternative hypotheses when they go against more established ones (Murphy & Allopenna, 1994): Old hypotheses will persevere even in the face of new information (Darby & Sloutsky, 2015;Lin & Murphy, 1997). With the accumulation of experience, the learning process can be attenuated and slowed down (Einstein, 1976), with new information being discounted, distorted, and at the extreme, even completely ignored. Confirmation bias is a related phenomenon that also leads to informational neglect and distorted learning: Individuals attend mostly to new information that confirms their previously established beliefs and ignore those that go against it (Klayman & Ha, 1987;Krizan & Windschitl, 2007;Pilditch & Custers, 2018). This behavior supports the idea that individuals seek to avoid cognitive dissonance (Anderson, 2003): "an individual tends to discard or mentally suppress information that indicates a past decision was in error" (Samuelson & Zeckhauser, 1988, p.39). Thus, the interference provided by prior experience can lead to a dampening of the influence of later information. Therefore, it is plausible that the more experiential information an individual has accumulated, the better established are their hypotheses about the environment, which will be more resistant to future contradictory information such as descriptive warnings about rare events.

| Descriptions influence how experiential information is integrated
The presence of descriptions can also bias the accumulation of experience and its associated learning. We believe that, in partial-feedback tasks, descriptions might be triggering the hot stove effect, with participants being vicariously "burned" by the bad outcomes presented in the descriptions. The hot stove effect severely constrains the learning process by reducing the likelihood of selections from undesirable options, after a sequence of bad outcomes; in partial-feedback 9 Participants could also experience regret from the lower positive outcomes of the low-risk decks, in comparison to the larger positive outcomes of the high-risk decks. We believe that this comparison is not as strong as the regret associated with the losses. First, the magnitudes of gains are lower: the large losses of the high-risk decks are likely to generate considerably more regret. Second, regret with regards to larger losses is stronger than those with regards to smaller wins (Kahneman & Miller, 1986).
environments, there are no opportunities for further learning from unselected options, therefore eliminating learning about how unnattractive these options really are (i.e., how often the stove is hot; Denrell & March, 2001). This process is likely to be related to the overweighting of rare events observed in decisions from descriptions, which would dictate that even though the described bad outcomes only occur rarely, individuals perceive them as happening much more often (Kahneman & Tversky, 1979). In contrast, in full-feedback paradigms, the hot stove effect is not sustained because individuals are able to learn from the foregone outcomes and therefore realize that the stove is not necessarily "hot" all the time. Eliminating the hot stove effect via full-feedback in our experiment was detrimental to performance, leading to more risk-taking, lower bonuses, and reduced impact of the descriptions, which acted as warning messages.
Although the hot stove effect is typically perceived as a negative bias (as it stops individuals from correcting biases introduced by early bad outcomes), in the case of warning messages, it is a welcome way of helping reduce risk-taking behavior.

| Influence of foregone payoffs
The introduction of foregone payoffs (full-feedback) eliminated the hot stove effect and helped identify it as one of the mechanisms behind the relationship between descriptions and experience. However, full-feedback also reversed the behavioral pattern from what was originally proposed under partial-feedback, raising new questions about how experiential and descriptive information interact. Across decision-making paradigms, the introduction of foregone payoffs has frequently changed behavioral patterns substantially (e.g., Denrell, 2007;Yechiam & Busemeyer, 2006;Yechiam & Rakow, 2012). We proposed two different accounts for the observed pattern reversal, both of which are associated with the availability of foregone payoffs.
First, attentional resources are under more demand in full-feedback paradigms. We proposed that experience gets attentional priority, and when experience requires too much demand, such as in full-feedback paradigms, then descriptions are not attended to. Therefore, it is only later in the task, after the attentional demands of feedback have reduced, that descriptions receive enough attention to influence behavior. Second, regret, which only occurs when foregone payoffs are known, accumulates over time and is learned with experience. As a result, anticipated regret increases as the task progresses. We propose that descriptions carrying negative information can trigger anticipated regret (with full-feedback); moreover, as prior experience is accumulated, anticipated regret increases, and that, as a result, participants react more to those descriptions. Future research should continue exploring how negative emotions, such as regret and disappointment, influences decisions differently under description, experience, and description-plus-experience.

| Limitations and extensions
One limitation of the current set of experiments is that participants might not trust the descriptive information. Specifically, after many trials in which a particular outcome has not been observed, a description including this outcome might be considered as deceitful. Lack of trust can also explain why descriptions had a lower impact on behavior in Experiment 3, as the full-feedback increased the amount of information available with which participants could dismiss the veracity of the descriptions.
This is a problem with any type of description which includes rare outcomes which are almost never observed (Weiss-Cohen et al., 2016). The more experience an individual has with the task, the easier it is to dismiss a description including events which rarely occur. The lack of compliance with warnings might be fomented by the proliferation of unnecessary warnings against dangerous outcomes even if they are extremely rare (and sometimes even impossible) to occur.
This overzealous approach to health and safety reduces the credibility of warnings. For example, Carson and Mannering (2001) noticed that road hazard signs warning individuals of the potential accumulation of ice on the road can be found in roads that rarely see snow or ice. It remains to be tested empirically if manipulating the trustworthiness, source, and nature, of descriptions can change the results observed here (see Pilditch et al., 2020).
Another potential limitation is that the descriptions used here were always present on screen after initially displayed. Therefore their impact on behavior might not be limited to the trials immediately following their initial appearance. In real life, it is common for warning signs to be seen only occasionally, for example, gambling machines employ pop-up messages which appear after a certain number of games are played (Ginley et al., 2017), and patient information leaflets are rarely read every time a patient takes the same medication (although some warnings, such as those in alcohol bottles and cigarette packs, are more constantly present, Rogers et al., 2000). This permanent or intermittent exposure to warning labels is likely to interact with the attention-based account for the impact of descriptions, and further research is needed to investigate how behavior is influenced by the removal of descriptions.

| Applications
Considering that warnings might be seen as descriptive information about catastrophic rare events added onto an individual's own personal experience (Hertwig et al., 2018), with the aim of encouraging a shift in behavior, our research can be useful for the creation of effective warning messages. Our findings reinforce the importance of earlier warnings for influencing behavior in situations where only partialfeedback is available. These are applicable for real world warning situations, because partial-feedback conditions are more typical in the real world, and only rarely we receive full-feedback about our foregone choices. We expand on the previous research by Barron et al. (2008), who had shown that warnings presented at the beginning of the task, before any experience, had a greater impact on behavior than descriptions presented halfway through the task. We have now shown that in addition to warnings before any prior experience, warnings post experience are more effective when the amount of experience is smaller. In order to influence experience-based risky choice, it is essential to present descriptions as early as possible. The more an individual accumulates prior experience, the lower the behavioral impact of any subsequent descriptive information. Therefore, although the research by Barron et al. (2008)

APP E NDIX A: DECK ANALYSIS
The traditional data-analytic approach for analyzing IGT-type tasks, which involves combining the alternatives into a simplified "advantageous" against "disadvantageous" comparison (which we also employed in our analysis), can hide variations among the underlying individual decks (Steingroever et al., 2013). For example, participants tend to prefer the decks with less frequent losses, regardless of longterm rewards (the frequency-of-losses effect; Ahn et al., 2008). Two patterns of choice arise from this phenomenon. Participants tend to select from deck B more frequently than expected, given it has negative EV, the highest volatility, and the largest losses (the prominent deck B effect; Lin et al., 2007) and select from deck C less frequently than expected, given it has positive EV, the lowest volatility, and the smallest losses (the sunken deck C effect; . In this appendix, we will present the results for individual decks which can expose more specific patterns of behavior.

Experiment 1
In the experiment-only (E) condition, the patterns observed are typical of traditional IGT tasks (Yechiam & Busemeyer, 2005, p. 389). We observe that participants avoided deck A and its large and frequent losses. We observe that participants selected more frequently from deck B than expected. We also observe that participants did not select as often from deck C as expected, and instead split their selections of advantageous decks between decks C and D.
With the introduction of descriptions from the beginning of the task, in the description-experience (DE) condition, we observe a reduction in the prominent deck B effect. A similar pattern was also observed in the IGT with descriptions in Weiss- Cohen et al. (2018, p. 225). It is often suggested that deck B is chosen frequently at the start until its large losses are observed, triggering a hot stove effect Denrell and March (2001); however, descriptions appear to help participants avoid this deck from earlier on, which can indicate an earlier triggering of the hot stove effect. Participants realize the attractiveness of deck C from the beginning of the task, likely by observing from descriptions that it has no negative outcomes, and choose from deck C more frequently from the start. The split between decks C and D remain, however. might be because the hot stove effect initially lead to an avoidance of selections from that deck (which can led to overweighting of rare event). The descriptions, when present, informed participants that the bad outcomes were less often than they might have predicted.

Experiments 2 and 3
In Experiments 2 and 3, the decks are different from the traditional IGT, so direct comparisons to Experiment 1 and prior research are not possible. However, we can attempt to map the new decks on to those in Experiment 1 and the traditional IGT according to their characteristics. Decks A and B have negative EV with high variance whereas decks C and D have positive EV and lower variance, similarly to Experiment 1. However, in comparison to the IGT, decks A and C now have less frequent losses (around one card in every 10), whereas decks B and D have more frequent losses (around two losses in every ten cards). Therefore, these decks are flipped in terms of frequency of losses in relation to Experiment 1. 10 In Experiment 2, we observe a clear preference for deck A, especially in the trials in which participants had to rely in experience alone, before descriptions were presented. This is the deck with the less frequent losses, and high recurrent wins, similar to the prominent deck B effect observed in the IGT. The introduction of descriptions led to a reduction in selections from the two negative-EV decks A and B, and an associated increase in selections from the advantageous decks, C and D. However, the shift away from deck B appears to be higher than the shift away from deck A, even though the magnitude of losses associated with the latter are much greater. Participants did select more often from deck D (the deck with more frequent losses) than deck C (which had less frequent losses). Therefore, they did not choose the deck with less frequent losses as observed in the IGT. This could be because the two decks had a relatively similar frequency of losses (10% and 20%) whereas the difference between frequency of losses of the decks in the IGT is considerably greater (10% and 50%).
In Experiment 3 with full-feedback, participants were more riskseeking, with considerably higher selections from deck B and, to a lesser extent, from deck A (as this was already the preferred deck 10 In addition, in Experiment 1, Deck C has no losses, only gain cards and "break even" cards with zero value. This is not a characteristic of any deck in Experiments 2 and 3, which always contain a combination of gain cards and loss cards.
under partial-feedback, and this remained the same), together with a reduction of selections from deck D, in comparison to partial-feedback. It is likely that the potential for regret from missing the high gains of decks A and B led participants to prefer these decks overall. This preference for the decks with higher gains (and associated higher risk-taking) is also observed in experiments which introduce full-feedback to the IGT (Grosskopf et al., 2006;Yechiam & Busemeyer, 2005. The appearance of descriptions with fullfeedback appear to lead to a reduction in selections from deck B and a shift towards deck D.

APP E NDIX B : INSTRUCTIONS
Reproduced below are the full set of instructions for Experiment 1, which were displayed before the task started. To ensure that participants read the instructions, this was followed by a set of four attention checks which needed to be answered correctly before proceeding: "How many decks of cards can you choose from?," "How much money can you earn in total, including the bonus?," "How many cards do you get to pick?," and "What's the objective of this task?" In the next screen you will see four different decks of cards side by side, face down.
I want you to select one card at a time from any deck you choose.
Each card is associated with rewards or losses. You will find out the cards as we go along. Each deck is formed of a different combination of cards.
It is important to know that the colors of the cards are irrelevant in this game and there is no way for you to figure out when you lose money. All I can say is that no two decks are alike, and some decks are worse than others.
You may find all of them bad, but some are worse than others.
No matter how much you find yourself losing, you can still win the maximum bonus if you stay away from the worst decks.
You are absolutely free to switch from one deck to the other at any time, and as often as you wish.
The goal of the game is to win as much money as possible, or avoid losing money as much as possible, by getting more cards with rewards and avoiding cards with losses.
You won't know when the game will end. You must keep playing until it tells you to stop.
Occasionally the computer might randomly decide to give you some hints by giving you more information about the cards contained within each deck.
You start with 2,000 [5,000] bonus points. After each card you select, if the card contains a reward, the amount will be added to your total points, and if the card contains a loss, the amount will be deducted.
At the end of the HIT, your final score will be converted to money at a rate of $0.10 for every 1,000 points.
Choosing wisely between the decks of cards, in order to maximise your points, will help you increase your bonus.
Please keep in mind that you need to complete the task until the end, and click the final orange submit button, to send in your bonus rewards back to Amazon. If you exit early or do not finish, you will not earn any money.»