A replication of “ Representative bureaucracy and the willingness to coproduce ”

Research on symbolic representation suggests that citizen – state interactions might benefit from public organizations' representativeness. Recent experiments on symbolic gender representation provide contradictory findings regarding the influence on citizens' co-production intentions. This study conducts a wide replication based on new data to reexamine the positive impact of symbolic gender representation identified by Riccucci et al. (2016, Public Administration Review , 76(1), pp. 121 – 130). The applied survey experiment closely resembles the original design aspects. The experiment is set in criminal justice policy, a policy field featuring co-production of core public services such as prisoner rehabilitation. The results do not confirm a positive effect of symbolic gender representation on willingness to co-produce. Instead, several arguments point to citizens' perceptions of uncertainty related to the co-production context and procedures as a boundary condition for the effects of symbolic gender representation. female officials among the four cited individuals (all-male, mixed, all-female). Subsequently, participants indicated their willingness to co-produce with three items. Each item captured co-production activities related to the recycling initiative, such as the recycling of hard plastics. The survey experiment was administered to individuals of a research panel from the CivicPanel project, resulting in a sample of 733 participants. The statistical analysis applied ordinary least square (OLS) regression analysis to test the impact of the treatment manipulation. The results indicate a positive effect of gender representation on women's willingness to recycle hard plastics and heavy composting. There were no effects on male participants. Overall, women more willing to recycle when the announcements included female names (Riccucci et al., 2016).


Abstract
Research on symbolic representation suggests that citizenstate interactions might benefit from public organizations' representativeness. Recent experiments on symbolic gender representation provide contradictory findings regarding the influence on citizens' co-production intentions. This study conducts a wide replication based on new data to reexamine the positive impact of symbolic gender representation identified by Riccucci et al. (2016, Public Administration Review, 76(1), pp. 121-130). The applied survey experiment closely resembles the original design aspects.
The experiment is set in criminal justice policy, a policy field featuring co-production of core public services such as prisoner rehabilitation. The results do not confirm a positive effect of symbolic gender representation on willingness to co-produce. Instead, several arguments point to citizens' perceptions of uncertainty related to the co-production context and procedures as a boundary condition for the effects of symbolic gender representation.

| INTRODUCTION
Public organizations are critical in a democracy as they often bridge the distance between political entities and the population. Thus, citizens' perceptions of public organizations significantly influence citizen-state interactions (Sievert et al., 2020). Previous research argues administrative actions and decisions might be perceived as fairer and more just when street-level bureaucrats are representative of citizens or the population (Gade & Wilkins, 2013).
Recent studies refer to symbolic representation, suggesting a positive influence on citizens' perceptions of trust and legitimacy. Symbolic representation occurs when there is congruence between the salient characteristics of the public workforce (e.g., gender) and the population, indicating the organization's responsiveness . Overall, citizen-state interactions could benefit "where representation may change the attitudes and behaviors of the represented client" (Gade & Wilkins, 2013, p. 267), such as cooperation or compliance (Choi & Hong, 2020). However, previous research yields contradictory findings related to the hypothesized effects of symbolic gender representation.
A recent study by Riccucci et al. (2016) indicates that symbolic gender representation is an important quality for citizen-state interactions. Increasing the representation of female officials positively affected the willingness to co-produce in a recycling initiative. The results suggest that women were more willing to co-produce when encountering higher shares of female officials. These findings promise considerable benefits for public organizations, as symbolic representation may lead to better services as well as cost savings . However, Van Ryzin et al. (2017) conducted a recent replication with a different policy context that could not confirm the findings. Symbolic representation did not unfold in the context of emergency preparation.
Consequently, Van Ryzin et al. (2017) proposed additional replication studies "as an opportunity to further develop and test the underlying theory of symbolic representation to more deeply probe its mechanisms and contingencies" (p. 1376). Following this notion, the present study provides a wide replication (Walker et al., 2019) of the original research by Riccucci et al. (2016). The preregistered survey experiment is set in the policy field of criminal justice, a context that features the co-production of areas such as prisoner rehabilitation (Loeffler & Bovaird, 2020). Applying the judicial system allows adding meaningful contextual variation to the design (Tsang & Kwan, 1999) by testing the effects of symbolic representation in a context with greater organizational integration of the coproduction efforts (Nabatchi et al., 2017). This replication provides new data from a sample of 1000 citizens and allows extending the original analytical strategy with robustness tests, such as a manipulation check.
The replication's results do not confirm the findings outlined by Riccucci et al. (2016). In the present study, symbolic gender representation does not affect the willingness to co-produce. Equivalence testing (Lakens, 2017) indicates null findings based on the absence of meaningful treatment effects. This research contributes to the literature on symbolic representation by reasoning about potential boundary conditions. The discussion indicates positive effects of symbolic representation might be limited to specific settings. Based on all three empirical studies, the discussion suggests that symbolic representation might be contingent upon citizen's perceptions of uncertainty related to citizen-state interactions. Suppose citizens experience high levels of uncertainty, for instance, because they lack central information about procedures. In that case, the underlying lack of knowledge (Zhang et al., 2020) seems to overshadow the symbolic meaning of gender representation. Thus, if citizens miss central information to evaluate potential risks or costs, they might not factor in symbolic representation. The discussion of the disparate results across the three studies points to several research avenues. In particular, qualitative research methods promise an in-depth understanding of the underlying mechanisms.

| SYMBOLIC REPRESENTATION AND ORIGINAL STUDY
Representative bureaucracy rests on the normative assumption that the public workforce should resemble society (Bishu & Kennedy, 2020). Active representation refers to behavioral aspects and indicates that bureaucrats use their procedural discretion to support and benefit particular groups (Gilad & Dahan, 2020;Kennedy, 2014). Passive representation indicates whether a public organization's workforce mirrors the characteristics of the citizenry (Meier, 2019), for instance, related to gender (Groeneveld et al., 2020). Recently, the scholarly focus shifted to symbolic representation (Fay et al., 2020). Symbolic representation occurs when there is congruence between the public workforce's salient characteristics and the population . This research stream explores mechanisms that explain how representation affects citizen-state interactions and service delivery by influencing citizens' attitudes and behavior (Headley et al., 2021;. Several empirical studies examined symbolic representation focusing on citizen-state interactions. In this regard, recent studies addressed the implication for co-production (Riccucci et al., 2016), referring to mutual contributions by service providers and users (Osborne et al., 2021). Empirical research involving a behavioral focus on citizens in the co-production domain did not confirm the hypothesized effects without exception (Meier & Nicholson-Crotty, 2006;Van Ryzin et al., 2017).
In this regard, Riccucci et al. (2016) applied an indicatory experimental study to the policy context of recycling.
In particular, this influential study examined the effects of gender representation in an announcement on women's willingness to engage in the co-production of a recycling initiative. The expectation was that women's willingness to co-produce would increase if more female officials were present in depictions of an organization. The main argument is that the symbolic meaning of salient passive representation of a public organization should positively affect citizen-state interactions, independent from public officials' actions (Nicholson-Crotty et al., 2016;Riccucci et al., 2014). This causal process includes changes in citizens' perceptions and behavior within public encounters. The theoretical mechanism underlying this assumption includes increased legitimacy of the organization at hand (Gade & Wilkins, 2013). Once an individual-level representation of the person's social category (i.e., gender) is apparent, the person's view of the organization might be more favorable, increasing the likelihood of constructive behavior . Citizens should perceive the organization as aligned with social norms and, thus, more legitimate (Riccucci et al., 2014;Theobald & Haider-Markel, 2009). The authors expected that women should value greater representation of female officials, resulting in increased willingness to co-produce (Riccucci et al., 2016).
To test the hypothesized effects of symbolic gender representation, Riccucci et al. (2016) applied a survey experiment with a between-group design. The authors randomly assigned the participants to vignettes resembling an announcement for a recycling initiative. The vignettes include different fractions of female officials among the four cited individuals (all-male, mixed, all-female). Subsequently, participants indicated their willingness to co-produce with three items. Each item captured co-production activities related to the recycling initiative, such as the recycling of hard plastics. The survey experiment was administered to individuals of a research panel from the CivicPanel project, resulting in a sample of 733 participants. The statistical analysis applied ordinary least square (OLS) regression analysis to test the impact of the treatment manipulation. The results indicate a positive effect of gender representation on women's willingness to recycle hard plastics and heavy composting. There were no effects on male participants. Overall, women were significantly more willing to recycle when the announcements included female names (Riccucci et al., 2016).
Following this initial research, Van Ryzin et al. (2017) replicated the same survey experiment. To that end, the authors changed the policy context to emergency preparation, specifically applying an announcement for the organization "Citizen Corps." Based on a sample of 604 US citizens, there were no statistically significant effects in this latter study. The representation of female officials in the announcement did not increase co-production intentions. The authors discussed the importance of the applied policy context, indicating that the absence of significant effects might result from the policy area's characteristics. The authors suggested further replicating the original survey experiment ; this study seeks to answer their call.

| REPLICATION STUDY
Replication studies contribute to public administration research by either generalizing theoretical mechanisms or exploring potential contingencies (Walker et al., 2019). Since the two survey experiments examining symbolic representation provide contradicting findings, this replication aims at the latter. This study applies a wide replication of the original research (Riccucci et al., 2016) to explore the initial claims and introduce potential contingencies. Given the progressed state of empirical research, including the initial replication by Van Ryzin et al. (2017), this research does not include a narrow replication of the original study. The design focuses on an alternative policy area, which presents a considerable variation from the original research. This approach allows exploring contingencies but might not provide conclusive evidence for them. This trade-off seems justified to shed light on the contradictory findings while also moving forward. In sum, this study closely resembles the original between-group design (Riccucci et al., 2016), set in criminal justice policy. Following the notion of an empirical generalization and extension (Tsang & Kwan, 1999), this study involves variations for design aspects and analytical procedure. To ensure a comprehensive overview of the empirical studies, Table 1 outlines the central characteristics of the two previous studies and this replication study. The table focuses on presenting aspects related to the research designs, research contexts, as well as analysis and findings to explicate similarities and differences between the studies.

| Context
The judicial system features several organizations responsible for aspects related to the overarching process of jurisdiction. This includes the court system, prosecution, as well as penal system. Judicial systems heavily depend on voluntary participation and the co-production of services, especially related to prisoner rehabilitation and judicial procedures (Loeffler & Bovaird, 2020). Commonly, citizens participate by providing guidance related to reintegration.
This policy area differs from the previously used experiment contexts. It features a greater integration of coproduction efforts (Nabatchi et al., 2017). Whereas recycling implies activities in the domestic environment, coproduction in the judicial system requires more effort and citizens' commitment.
Most co-production activities in the judicial system are highly embedded within the respective organizations (Loeffler & Bovaird, 2020). The previous studies' mixed findings might be related to public service industries, policy fields, or the co-production context. Therefore, the voluntary efforts applied in this study resemble another type of co-production as compared with the initial study. Co-production in the judicial system includes implementing core tasks instead of merely complementing public service delivery (Brandsen & Honingh, 2016). Co-production in the judicial system involves activities to achieve offenders' reintegration and the encouragement of desistance (Loeffler & Bovaird, 2020). Both require extensive efforts and interaction with convicted individuals.
The most prominent co-production roles in the German judicial systems are lay judges, voluntary probationers, and correspondents. First, lay judges mark an integral component of the German judicial procedures. They primarily take on an advisory role, even though their voting power is equivalent to professional judges. The criminal justice system heavily relies on lay judges for court rulings (Matthews et al., 2018). In Germany, lay judges exceed the number of professional judges. Second, rehabilitation programs require voluntary probationers who cooperate with professional probationers (Walsh et al., 2020). Their area of responsibility is individual clients on probation either following a court ruling or after serving a certain amount of their prison term. Voluntary probationers focus on a limited number of clients and support reintegration by assisting with social and economic problems. Their main goal is to help clients to avoid recidivism. Third, co-production takes the form of mail-based communication with inmates.
This aspect focuses on building a bridge between society and those imprisoned. This role is least intrusive for those engaging in co-production related to the judicial system.
Overall, the present experiment design adds a further variation of co-production activities compared to the previous research. Lay judges and voluntary probationers face extensive integration in core public services (e.g., in court proceedings or when helping clients with administrative procedures), and correspondence with inmates includes personal contact. Representation might be more critical if participation in the agency is required compared to activities undertaken in the domestic environment.

| Experiment procedure
After a short introduction to the study and informed consent to the use of collected data for scientific purposes, participants saw a brief description of the public organization responsible for co-production in the experiment setting (i.e., ministry of justice). Following this brief description, participants indicated their generalized perception of the Afterward, participants were randomized into three groups while proceeding to the second part of the survey. In the second part, based on the randomization, each participant saw one of three announcements for co-production activities in the ministry's rehabilitation program. These vignettes resemble realistic announcements achieved through a professional layout, including the ministry's logo, colored background, and prominent text elements. The announcements featured quotations appealing to the participants to act as lay judges, voluntary probationers, or engage in mail correspondence with inmates. The featured quotes in all vignettes allow manipulating the displayed gender representation, resembling the original study by Riccucci et al. (2016).  Table 2 presents the sample structure relative to the general population. Participants matched the age and gender distribution of the general population. Half of the participants were women, and 23% were between 50 and 69 years of age. However, the overall sample contained fewer private sector workers and a higher share of unemployed people than the general population. People working in public and nonprofit sectors were also underrepresented. The sample exhibited higher educational levels, with roughly half of the participants having a higher education entrance qualification. Finally, the sample was highly diverse in terms of political orientation, with the mean being slightly right of the center.

| RESULTS
To start, testing for differences in the participants' age, gender, education, work, and political orientation can reveal whether the randomization was successful. In this regard, a chi-square test (χ 2 ) for gender and analysis of variance for the remaining variables examined whether the three experimental groups differ. The results in Table 3 suggest that the experimental groups exhibit no statistically significant differences (p > 0.05). The absence of significant differences indicates that the groups were statistically equivalent.
Furthermore, the check of the manipulation's effectiveness assesses whether the manipulation of gender representation caused changes in participants' perceptions of the number of female officials. Given that the manipulation included altering the gender distribution, participants should perceive higher shares of female officials if randomized into the "mixed" and "all-female" conditions. Table 4 outlines the OLS regression analysis with the manipulation check as the dependent variable and the treatment conditions as independent variables. The results suggest the treatments have a significant influence on perceptions of women's representation ("mixed" treatment: b = 1.14, SE = 0.08, p < 0.001; "all-female" treatment b = 2.31, SE = 0.08, p < 0.001) indicating that the treatment manipulation was successful. Table 5 presents the results for the treatment effects on the three dependent variables. The displayed effects indicate the predicted change caused by the treatment groups "female/male" and "all-female" compared with the group "all-male." First, the OLS regression analysis was conducted. Second, additional logistic regression models were used to outline the treatment's impacts on the dichotomized dependent variables. Identical to Van Ryzin The regression results in Table 5 indicate that the treatment manipulations, compared with the control group ("all-male"), have no statistically significant effect on the willingness to co-produce. Neither a balanced representation between male and female officials nor an overrepresentation of female officials fosters the willingness to co-produce in the judicial system. The results are similar for all three types of co-production behavior, indicating no differences between co-production as lay judge, probationer, or correspondent. Thus, varying the representation of female names in a co-production initiative does not affect participants' intention to engage in the judicial system.  Table 6 outlines the OLS regression results for gender subgroups, assessing whether female officials' representation affects male and female participants differently. Figure 2 presents the graphical results. Closely resembling the results for the full sample, the analysis shows no significant effects of gender representation. The treatments do not affect the subgroups differently, emphasizing that the representation of female officials does not affect coproduction intentions. Overall, gender representation did not affect the co-production intentions of both women and men.
Several robustness tests amend the statistical analysis. First, the nature of the dependent variables requires caution due to unknown distances. Thus, proportional ordered logistic regressions (Fullerton, 2009)  F I G U R E 2 Graphical display of the experimental results confirming the results from OLS and logistic regressions. Second, instrumental variables (treatment failed = 0, treatment successful = 1) were used to account for incomplete treatment compliance based on the manipulation check (Grimmelikhuijsen & Klijn, 2015). Two-stage least square regression models confirm the findings from the OLS regressions for the full sample and the gender subgroups.
The results merit advanced statistical computation to establish "the absence of a true effect" (Lakens, 2017, p. 355). Two one-sided tests (TOST) are suitable to examine the equivalence to zero for all investigated treatment effects (Lakens et al., 2020). Because previous studies do not yield adequate information to define the smallest effect of interest, the benchmarks introduced by Cohen (1988) were used, according to recent recommendations (Lakens, 2013). The analysis is based on Cohen's d = 0.2, indicative of a small effect. The TOST illustrate whether the observed treatment effects fall into the range between the lower and upper bound (À0.2; +0.2), indicating effects smaller than d = 0.2. If effects fall into this range, it can be considered "equivalent to the absence of an effect that is worthwhile to examine," indicating null findings (Lakens, 2017, p. 356). Table 7 outlines the TOST for all treatment effects. Five of the six treatment effects show statistically significant results. They can be considered equivalent to zero and, therefore, are not meaningful. The "female/male" group's treatment effect on the willingness to act as a probationer is not equivalent to zero. In this case, exclusively, the analysis cannot rule out the existence of a treatment effect.

| DISCUSSION AND CONCLUSION
This replication study complements previous research addressing the effects of symbolic gender representation on willingness to co-produce. The results are not in line with the initial findings presented by Riccucci et al. (2016). Symbolic gender representation did not affect participants' willingness to co-produce in the judicial system. Neither of the treatment groups that received announcements with increased representation of female officials exhibits greater willingness to co-produce. Furthermore, varying the representation of female officials does not affect participants regardless of their gender. These findings complement the initial replication by Van Ryzin et al. (2017), verifying that symbolic gender representation's expected positive effects are not generally applicable. The summary of findings highlights the importance of boundary conditions for symbolic representation (see also Headley et al., 2021). Comparing the empirical results based on the experimental designs indicates that the effects of symbolic representation may be context dependent.
Given the application of different co-production contexts and vignettes, the related characteristics merit discussion. Interestingly, recycling (the original study's experimental context) is not directly related to the organization displayed in the announcement. Instead, it implies co-production efforts in the domestic environment. Simultaneously, the co-production efforts in the present replication imply at least direct contact with the organization (Nabatchi et al., 2017). In this regard, citizens likely experience differing levels of uncertainty when confronted with public organizations (Herian et al., 2012). Procedures and specifications of activities are unclear, and individuals encounter less predictability. Previous research emphasizes the importance of issue-specific knowledge (Zhang et al., 2020), indicating that low uncertainty situations positively affect the likelihood of co-production behavior. This perspective suggests uncertainty might affect whether representation influences co-production intentions. For instance, working with strangers when providing reintegration sessions entails extensive ambiguity for citizens and could overrule symbolic representation. High levels of uncertainty reduce the importance of contextual information because people tend to engage less in automatic information processing (Tiedens & Linton, 2001). Citizens confronted with initiatives in both replication studies faced higher levels of uncertainty about the expected processes (Vanleene et al., 2020). Seemingly, citizens have less control if co-production is embedded in the organization; thus, women might be unresponsive to the representation of female officials. The symbolic gender representation does not provide implications for the actual co-production procedures and, therefore, does not increase the activities' predictability.
Future research should substantiate the discussed aspects by focusing on further policy domains and coproduction efforts. Additional replications could address varying levels of uncertainty as a contextual factor covering the broad spectrum of co-production types (Brandsen & Honingh, 2016). Continued research might consider other social categories , such as racial representation (Hong, 2020) and other shared identities between citizens and public officials. Moreover, additional research should test whether alternative content in announcements reduces uncertainty. In particular, manipulating the representation of citizens already engaged in the co-production initiative seems promising (Migchelbrink & Van de Walle, 2020).
To advance theoretical development, research on symbolic representation would benefit from a deeper understanding of underlying mechanisms related to individual meanings and identities (Merritt et al., 2020;Ospina et al., 2018). The contextual factors that drive uncertainty and citizens' susceptibility to representation remain unclear. Thus, how citizens incorporate symbolic representation into their cognitive and emotional processes requires attention. The experimental studies' disparate results indicate a considerable lack of understanding of how citizens construct identities and experience representation in citizen-state interactions. This includes a consistently static understanding of causal relations, disregarding social dynamics and social mechanisms' complexity (Raadschelders, 2011). Qualitative research designs could help uncover potential boundary conditions and illuminate identities and social construction (Bhanot & Linos, 2020). Such endeavors would allow exploring how individuals experience encounters with public organizations. For instance, ethnographic studies offer in-depth observations and analysis of citizen-state interactions (Ashworth et al., 2019;Cappellaro, 2017). Furthermore, approaches such as interpretative phenomenological analysis (e.g., based on semi-structured interviews) enable researchers to infer cognitive and emotional processes underlying the social mechanism shaping public encounters (Smith, 2019 preparedness), the present context requires higher levels of involvement. Acting as a lay judge or probationer implies extensive emotional and cognitive investments, considering the consequential decisions and challenging situations involved in these efforts-the original study required co-production through recycling behavior in the domestic environment. The results might be affected by participants' awareness of the high individual effort. In particular, symbolic representation might be less relevant in high-cost settings if citizens are less responsive due to the required effort. Still, the influence on the empirical results should be limited. The design featured a dependent variable requiring less effort (corresponding with inmates). This activity entails higher efforts compared to the original study but is much less intrusive than acting as a lay judge or probationer. Thus, if the level of investment produces the observed null findings, the treatment effect should vary based on the required efforts implied by the dependent variables. However, this is not the case as the statistical analysis provided consistent null findings. Second, the experiment context is by tendency male dominated, similar to the first replication study . Future studies should systematically vary the co-production activities to examine if gender-dominated contexts and co-production procedures affect whether citizens consider representation. Third, this study was conducted with a sample of German citizens. The national context might influence the results, because the importance of gender representation depends on economic development, culture, and further country-level factors (Falk & Hermle, 2018). However, data from the World Values Survey indicate that citizens in both countries have similar attitudes toward gender equality (Wernet, 2016).
In conclusion, the empirical results contrast the expectations derived from theory and previous studies. Symbolic gender representation did not affect co-production intentions in the context of prisoner rehabilitation. These findings point to potential contingencies for the effects of symbolic representation. The combined insights help to theorize about what might cause the variation. Uncertainty related to the different co-production contexts seems a suitable candidate. Co-production processes explicitly linked to the organization and, thus, characterized by high uncertainty for citizens, might not profit from symbolic representation.

ACKNOWLEDGMENTS
The author wants to thank those who provided feedback on earlier versions of this manuscript: Seung-Ho An, Mat-

CONFLICT OF INTEREST
The author declares no potential conflict of interest.
For the data and study materials, see https://doi.org/10.7910/DVN/6ZYXML. 2 The original job titles were gendered; for example, Richterin is the female version of a male Richter (Judge).

DATA AVAILABILITY STATEMENT
The data and replication material are openly available in Harvard Dataverse at https://doi.org/10.7910/DVN/ 6ZYXML.