Explicit incentives increase citizen science recordings

From tracking land‐use change to biodiversity loss, citizen science data have become a cornerstone for conservation. However, policymakers must understand the “data‐generating process” to make good use of existing citizen science data and encourage the production of useful new data. We analyze data from the two largest German online platforms for ecological observations to explore and quantify the effect of explicit incentives on volunteer recordings, created by a large‐scale prize competition on one of the platforms. We find 10% more recordings during the prize competition. Moreover, the effects of weather and weekends are attenuated during the competition period. Finally, the diversity of recorded species decreases. Our study shows the first statistical evidence that using explicit incentives increases the quantity of citizen science data. It highlights the need to further study the effect of explicit incentives on data quality and the engagement of citizens for conservation.

. In principle, one can leverage the potentially large number of observations to factor out measurement errors, and one can correct for potential bias when its direction and size can be estimated (Bird et al., 2018;Cameron & Kolstoe, 2022;van Strien et al., 2013).For both tasks, it is essential to understand the incentives and motivations of the people that produce the data (Arazy & Malkinson, 2021;Bird et al., 2018;Kelling et al., 2015;van Strien et al., 2013).
Here, we complement existing qualitative studies on volunteer motivations (Bowler et al., 2022;Larson et al., 2020;Maund et al., 2020;Rutten et al., 2017) by analyzing a large dataset of species sightings registered on two popular online platforms in Germany, naturgucker.deand iNaturalist.org.Specifically, we present the first study that tests whether explicit incentives, such as prize competitions or monetary rewards, affect the production of citizen science data.
Understanding how explicit incentives affect the production of citizen science data is both important and interesting.It is an important question because policymakers could use explicit incentives as tool to increase volunteer effort (Wood et al., 2011;Xue et al., 2016).Moreover, explicit incentives could be used to steer volunteer effort and thus optimize sampling designs (Callaghan, Poore, et al., 2019;Callaghan, Rowley, et al., 2019).It is a scientifically interesting question because it is unclear whether explicit incentives would work.Most volunteers are highly motivated by intrinsic reasons and share an ethos and understanding that their activities contribute to some greater good.It is therefore not obvious whether citizen scientists are at all receptive to explicit incentives.More than that, explicit incentives could even backfire as the extrinsic motivation that, for example, a prize competition creates may crowd out intrinsic motivations of volunteers (Bénabou & Tirole, 2003;Gneezy et al., 2011).Finally, there could be interactions between explicit incentives and other predictable external factors such as time constraints or weather, affecting the bias the latter factors introduce.

METHODS
We aim to describe the "data-generating process" of citizen science recordings.Starting from the incentives and constraints of an individual volunteer, we predict the extensive and intensive margin of volunteer effort.That is, does a given individual volunteer go out to record (extensive margin)?If yes, how much does she record (intensive margin)?
To answer these questions, we merge daily data from two online platforms for recording ecological observa-tions that are popular in Germany, naturgucker.de 1 and iNaturalists.org, 2 with weather data from the German meteorological service. 3 Previous studies suggest that weather is an important predictor of effort, with fewer recordings on rainy or cold days than on sunny or warm days (Bas et al., 2008;Brum-Bastos et al., 2018).Further, similar to the findings in other countries (Courter et al., 2013;Sparks et al., 2008), we expect a "weekend effect" with fewer recordings on working days than on the weekend.A unique feature of our dataset enables us to study the effect of explicit incentives: One of the platforms, naturgucker.de,conducted a large-scale prize competition between December 5, 2020, and January 7, 2021.Every sighting registered during this time was a lot in a raffle for a high-quality binocular (market value of about 1800 Euro) and other prizes. 4We include 6 weeks before and 7 weeks after the competition to have observations from dates with similar weather conditions and similar animal populations (many recordings in the online platforms come from transient, or migratory, species).That is, we consider the dates from November 1, 2020, to February 28, 2021 (120 days).There are 495 weather stations that have at least one observation and at least one recording in either online platform.In total, we have 59,348 valid station-date observations (see Figure 1 and Table SI-1 in the Supporting Information for summary statistics).
Our unit of observation is thus a weather station in location  at date .Our outcome variable is the number of recordings registered at date  in the vicinity of location .We regress the variables that capture weather, time constraints (working days), and explicit incentives (the prize competition) on the outcome variable, using a negative binomial hurdle model to account for characteristics of our count data (for details, see the Supporting Information [SI]).
To assess whether any effect of the prize competition is indeed causal, we need a valid counterfactual.  The announcement was publicized in the monthly newsletter and on the platform's social media pages.Naturgucker.dehas previously conducted several such competitions, with different modalities and targeting different audiences (such as a photo contest in the summer).The prize competition that we consider here is ideally suited to study the effect of extrinsic incentives on volunteer effort as it is indiscriminate to the type of recording, and because it takes place in the winter when there are no other special circumstances to account for (such as bird migrations or spring bloom).

F I G U R E 1
Map of Germany, showing the locations of the weather stations and recordings on the naturgucker.deand iNaturalist.orgplatforms from November 2020 to February 2021.end, we use the time series from the iNaturalist.orgplatform.We subtract the number of recordings in location  at date  in the iNaturalists.orgplatform from the number of recordings in the same location and at the same date in the naturgucker.deplatform.This gives us a time series of differences in the two datasets that implicitly controls for all location-and time-specific effects.Under the assumption that all other temporal trends affect the two platforms in the same way, the difference between the time period with and without the prize competition identifies the causal effect of the prize competition (hence this approach is often referred to as difference-in-difference method; Angrist & Pischke, 2009).
In addition to probing the robustness of our results in various subsamples (see the SI), we assess the "parallel trends assumption" in two ways.First, we aggregate the data and analyze the weekly averages of the differences before, during, and after the prize competition.We should observe no systematic differences before the competition.Second, we conduct a placebo test.That is, we randomly assign treatment status to dates outside the true competition period and test whether this placebo treatment is significant.The randomly assigned treatment should have no effect.

Explaining what drives observer effort
We first look at what explains volunteer recording in the naturgucker.dedata.We see that whether it is a working day or not has a strong effect, both on the extensive margin (whether there is a recording, column 1 of Table 1) and on the intensive margin (how many recordings there are, column 2 of Table 1).The chance that there is a recording for a given station-date combination is about 40% lower, and there are about 30% fewer recordings on working days than on weekends and holidays (SI, Figure SI-1).Furthermore, we find significantly more recordings when it is warmer and drier.
Turning to the effect of the prize competition in the naturgucker.dedata, we find a negative effect on the extensive margin (zero model, column 1).On the intensive margin (count model, column 2), in contrast, we find a positive effect.There are about 10% more recordings (conditional on that there is at least one recording) during the period of the prize competition compared to dates outside of it.
Looking at the regression results from the iNaturalist.orgdata, columns 3 and 4 in Table 1, we confirm the presence TA B L E 1 Results from the regression analysis, columns 1 and 2 show, for the naturgucker.dedata, the coefficients for predicting the whether there is a recording (the zero model) and the number of the recordings (the count model).Columns 2 and 3 show the results for the iNaturalist.orgdata.Columns 5 and 6 show results from a model specification that explores interactions of the prize competition and weather/weekend effects in the naturgucker.dedata. of a strong weekend effect and the weather effects found in the naturgucker.dedata.With respect to the prize competition, we find a negative correlation with the occurrence of recordings and no relation to the number of recordings.The latter result, the null effect on the intensive margin, is expected as there was no prize competition on the iNaturalist.orgplatform.The former result, the significant negative relation between the prize competition and the occurrence of recordings, likely picks up a general trend that is common to both databases (see the corresponding coefficient in column 1).Before we present the difference in differences analysis, we study potential interaction effects (columns 5 and 6 in Table 1).We find that the prize competition interacts with both weather and weekend effects.The former interaction is relatively weak and only concerns the extensive margin: It is more likely that a volunteer records an ecological observation when it rains during the period of prize competition than outside of it.The interaction with the opportunity cost of time is stronger.We find that the negative effect of a working day on the number of recordings is about halved during the prize competition.

Dependent variable
Both interaction effects point in the same direction: They suggest that the prize competition may attenuate the negative incentives of bad weather and high time costs during working days.

The causal effect of explicit incentives
There are, on average, about 8.67 more recordings per day and weather stations in the naturgucker.dedatabase than in the iNaturalist.orgdatabase during the control period.During the period of the prize competition, this difference is, on average, 9.95 recordings.The difference is highly significant (see the first row in Table 2).In other words, the prize competition increases the difference in the number of recordings per day and weather station by 1.28 recordings in absolute terms or by 15% in relative terms.
Figure 2 presents a clear picture of the treatment effect.It shows the differences in the number or recordings aggregated to the weekly level, for each week in the data.The prize competition started on Saturday, December 5,  so week 50 is the first full week in the treatment period (highlighted in blue in Figure 2).Clearly, the difference in the number of observations is larger than the pretreatment average (indicated by the dashed line).In fact, the weekly averages are higher during each week of the treatment period except for 1 week (Week 52).Interestingly, the data suggest that the treatment effect did not end immediately with the end of the prize competition but rather faded out slowly.The fact that the weekly averages are all very close to each other during the pretreatment period (Week 44-49) is comforting as it suggests that the parallel trend assumption is likely to hold.
To probe deeper, we conducted a placebo analysis, where we randomly assigned treatment or control status to the dates.We then computed the difference that would result if this placebo assignment were the treatment.Only five out of one thousand such draws exceed the observed treatment effect (Figure SI-3).
Having established that the prize competition has increased the number of recordings, we are interested in learning who responded to the explicit incentives and what other effects the treatment might have had.To this end, we study varying subsamples and consider other outcome variables (see Table 2).
First, we split the sample into those 10% of volunteers that contribute most (in fact, more than 85%) of the entries in the database, and those that are less active.We call the first group the "core contributors" and the second group the "occasional contributors."Two things stand out.One, the difference between the naturgucker.deand iNaturalist.orgdatabases is much larger for the core contributors than for the occasional contributors.Two, the prize competition apparently only affects the core contributors.For the occasional contributors, the difference between the control-period differences and the treatment-period differences is not significant.
Next, we take a closer look at what is being recorded.One concern with using explicit incentives could be that volunteers do not really spend more effort, but begin to record species that are common and easy to spot, or even begin to enter false data to increase their chance of winning the prize.As discussed in the SI, there are some recordings from the same volunteer of the same species at the same location on the same day.It is not possible to discern whether these are different individuals of the same species, or whether these were duplicates.We label these observations as potential "excess recordings" and create a variable that counts the lower bound of the "unique recordings."As can be seen in the fourth row of Table 2, the number of potential "excess recordings" more than doubles due to the prize competition.However, when looking at the difference in unique recordings (the fifth row of Table 2), we see that it is almost identical to the differences in our primary outcome variable, and the effect is statistically highly significant.This alleviates the concern that the treatment effect is driven by false recordings.
Further, we investigate whether there are any differences in the number and diversity of species that are being recorded.Indeed, we find that the number of species that are being recorded drops due to the prize competition.Also when we look at the species diversity that is recorded in either platform on a given day, we find it decreases due to the prize competition (p-value <0.001, two-sided t-test; see Figure SI-4).However, it is not the case that the prize competition only increases the number of very common species that are being recorded.When we remove the 10 most recorded species from the dataset (the last row in Table 2), we still find a strongly significant treatment effect.

DISCUSSION
Incentives affect the recording of ecological observations by volunteers.When it rains, it is less likely that there is an entry for a given location on either of two popular online platforms for nature observations in Germany.Similarly, when the opportunity cost of time is high because it is neither weekend nor a holiday, there are fewer recordings.In contrast, there are more recordings when entering a recording that earns the chance to win a high-prized binocular.That is, in addition to the importance of intrinsic motivation that is highlighted in the previous literature, our study shows that volunteers can also be motivated by explicit incentives.This finding is relevant for policymakers that want to use citizen science data and potentially steer its production.
A potential limitation of the data is that we cannot, in principle, exclude the possibility that users submit bogus recordings to increase their chance of winning the prize.Given the usual ethos and social norms among citizen science volunteers, we do not find this a plausible explanation.Further, our data suggest that potentially increased excess recordings do not drive the results as we observe a strong treatment effect also when excluding all recordings that could be a double count.That said, we do document that the prize competition changed what is being recorded.We find that the diversity of species that are recorded is lower during the treatment period.Our analysis thus highlights that more work is needed to research potential side effects of prize competitions on the data quality.Certainly, whether ecological observations can be classified as being "more valuable" and "less valuable" is debatable and depends on the specific purpose that a manager has in mind when using explicit incentives to motivate volunteer recordings.
Another potential limitation of the data is that characteristics of the volunteers are not known.On the one hand, this means that valid concerns for data protection and privacy are naturally respected.On the other hand, this means that one cannot investigate person-specific effects such as age or gender.An interesting alternative that one could explore is to classify volunteers by additional characteristics that they voluntarily supply, such as whether they use advanced photographic equipment or merely the camera of their smartphone (see Arazy & Malkinson, 2021, for an example of such an approach).
Because the prize competition was not deliberately designed to increase the number of recordings but rather to support the general visibility and popularity of the platform, documenting its effect opens many interesting questions.Would the effect be stronger if a prize competition were targeted at increasing the number of recordings?Could a prize competition be manipulated such that a specific type of recording is increased?Does the way how the competition creates explicit incentives matter?For example, there is a growing literature that documents gender differences in how people react to competition (Cassar & Rigdon, 2021;Niederle & Vesterlund, 2007), and using explicit incentives that are framed in terms of competition may interact with user identities in important ways.How do direct financial incentives affect citizen science recordings in comparison?
Extrapolating the local average treatment effect, we compute an implicit cost of 39 cents per additional recording. 5Increasing the number of recordings is not the only, and most often not the foremost aim of citizen science.Most platforms list engagement and increased stewardship as their primary goals (Ellwood et al., 2017;MacPhail & Colla, 2020).Moreover, when considering to use explicit incentives to steer the production of citizen science data, one has to be aware of the fact that the different platforms interact in a larger market.While we do not detect that the increase in recordings on the naturgucker.deplatform is associated with a decrease in recordings on the iNaturalist.orgplatform, it is not inconceivable that negative spill-over effects materialize at some point.
Given the disruptive global changes and dramatic declines in biodiversity that humankind is experiencing (and causing), it is imperative to have "all hands on deck."The involvement of citizens is indispensable to both document the biological consequences of global change (Soroye et al., 2018) and to create the engagement necessary to advocate and implement effective conservation (Crain et al., 2014).More work is needed to better 5 There are 1.28 more recordings per day and weather station due to the treatment, 6016 valid units of observation, and the total cost of the prizes were roughly 3000 Euro (in 2020).
understand how volunteer efforts can be best coordinated and catalyzed such that citizen science can unleash its full potential (Bonney et al., 2014).

A U T H O R C O N T R I B U T I O
Mean weekly differences in the number of recordings in iNaturalist.organd naturgucker.dedatabase.The treatment period (prize competition) is highlighted in blue, and the dashed line shows the pretreatment average.
N S F.D. and L.S. designed the research, S.M. and G.S-M. provided data and interpretation, F.D. analyzed the data, and F.D. and L.S. wrote the first draft of the paper; all authors approved the final draft.A C K N O W L E D G M E N T S This research has been funded by the Belmont Forum Project OceanFrontChange.We thank Hamsa Zazai for his excellent research assistance.C O N F L I C T O F I N T E R E S T S TAT E M E N T F.D. and L.S. declare no conflict of interest.S.M. and G.S-M. work for naturgucker.deand declare no conflict of interest.O R C I D Florian Diekert https://orcid.org/0000-0002-5986-4060R E F E R E N C E S
Difference in difference analysis: Results from two-sided t-tests, varying samples and outcome variables.
TA B L E 2