On the interconnected nature of risk and responsibility in the research and development of new and emerging technologies

Risk analysis of new and emerging technologies requires innovative approaches that are agile, exploratory, and can accommodate broad stakeholder engagement and perspectives. Existing theories of risk governance and responsible innovation suggest that operationalizing guiding principles for engagement such as inclusion and reflection may provide a useful approach to the risk analysis of these technologies. Yet, methodologies to systematically assess how we might operationalize such guiding principles in risk analysis are limited in existing risk research. We contribute to filling this gap by demonstrating a practical methodology for examining and documenting how research and development (R&D) professionals operationalize inclusion and reflection in risk analysis and what value this provides to risk analysis in the R&D context. We use the Australian nanotechnology R&D sector as our case study, interviewing 28 experts to examine how R&D professionals have operationalized inclusion and reflection into their risk analysis practices, generating three findings. First, we describe how our research design enables the successful translation of theory into a methodology that supports an empirical assessment of the integration of these guiding principles into risk analysis practice. Second, we argue that successfully and systematically integrating inclusion and reflection in risk analysis fosters a wider understanding and identification of risk through the activation of multi‐actor and multi‐institutional stakeholder engagement processes. Third, we outline how this research depicts the outward‐facing and introspective nature of risk analysis.


INTRODUCTION
Risk is an everyday term that is commonly used to refer the potential for undesirable consequences arising from a given activity.However, this understanding of risk is limited as it fails to acknowledge that activities will give rise to both desirable and undesirable outcomes.Inherent to the risk concept, according to Aven and Thekdi (2022), are consequences and uncertainties, which capture the potential for both desirable or undesirable consequences to occur and the uncertainties associated with this.To demonstrate with an example, the use of mobile phones could lead to the outcomes that an individual desires (e.g., connecting with loved ones) or it could generate undesired consequences (e.g., eye fatigue due to prolonged exposure to blue light), or perhaps even both outcomes.When risk is considered, it is therefore critical to look at how such consequences and uncertainties might arise and particularly to examine how and who this will impact.
To solve risk problems-be they related to technologies, security, the environment, or health-risk analysis approaches are widely used.A primary role of risk analysis is to establish knowledge about risk and inform decisionmaking accordingly (Cox, 2013).Although there is no universally agreed definition of risk analysis (Aven, 2012), this research applies the definition developed by the Society for Risk Analysis, which defines risk analysis as including: "risk assessment, risk characterisation, risk communication, risk management, and policy relating to risk, in the context of risks of concern to individuals, to public and private sector organisations, and to society at a local, regional, national, or global level" (Aven et al., 2018, p. 8).Renn (2008) similarly characterized risk analysis as comprising the core activities of risk assessment, risk management, and risk communication.Risk analysis as science can be categorized into applied risk science and generic (or fundamental) risk science (Aven, 2018).The former contributes to knowledge generation for specific risk activities, whereas the latter focuses on knowledge related to methodologies, concepts, and principles of risk science.While identifying this study as generic risk science, we contribute to the generation and demonstration of a methodology focusing on the integrated nature of governance and risk analysis in the research and development (R&D) of new and emerging technologies.
New risks are being introduced as new technologies continue to advance.Even the best intended technology can pose such risks and questions about their potential consequences and the associated uncertainties.Risk analysis can be challenging with technologies that are emerging or under development, as the appraisal of consequences and uncertainties cannot entirely be determined based on prior knowledge (Maynard, 2015(Maynard, , 2016)).Embedded in this challenge are ethical considerations, such as how the consequences will be distributed, who has the authority to introduce a risk, and who will be responsible for mitigating and managing such risks (Doorn, 2015;Douglas et al., 2022a;Giddens, 1999).Questions related to the role of responsibility in risk analysis have been investigated (e.g., Kim & Kim, 2022;Liu & Du, 2022;Rickard, 2014).The scope of these investigations, however, is largely limited to psychological theories and examines the attribution of responsibility in case undesirable consequences occur.
There is a gap in studies that examine risk analysis approaches that enable the discussion and deliberation of responsibility during the R&D of new and emerging technologies.To explore the nature of the relationship between risk and responsibility, Kermisch (2012) conceptualized the integration of the moral dimension in risk research but offered no practical methodology to achieve this goal.The novelty of this article is to investigate an approach that aims to adopt a more forward-looking or anticipatory approach to responsibility in risk analysis that extends our notions of responsibility beyond those of blameworthiness and liability (i.e., the backward-looking or retrospective forms of responsibility as identified by van de Poel, 2011).This will enable R&D professionals of new and emerging technologies, such as scientists, engineers, and researchers, to consider potential consequences and uncertainties before the technologies are out of the laboratory and into people's hands (Klinke & Renn, 2014).Further, such an approach demands a new governance style that allows risk analysis to move beyond probability estimates and hierarchical controls and engage a wide range of stakeholders and multidisciplinary scientists and researchers (Klinke & Renn, 2019).Moreover, Hansson (2017) argued that there are multiple ways of assessing risks, and the evaluation of risks demands a governance approach that enables the exploration of all types of risks and the appropriate ways to address them.
The concept of risk governance combines risk analysis with governance mechanisms in order to make decisions about risks that account for the legal, institutional, social, and economic aspects of risk, along with the involvement of multiple actors and stakeholders (Renn, 2008).Risk governance allows the appraisal of risk from multiple perspectives at various levels and seeks a collaborative approach to responding to risk in a more holistic way (Malakar, Lacey, Twine, & Bauer, 2023).This is crucial in the context of the R&D of new and emerging technologies as in this context, there is a comparatively high level of uncertainty about the potential consequences and the stakeholders that may be affected.Therefore, the role of the R&D professionals becomes increasingly critical, not only to assess risks from diverse expert perspectives but also to contribute to the collaborative governance of those risks during the R&D process.
To achieve this approach to risk governance, van Asselt and Renn (2011) proposed that risk analysis must systematically include three principles: (i) communication and inclusion, (ii) integration, and (iii) reflection.In this study, we recognize that the operationalization of these guiding principles in the risk analysis of new and emerging technologies is a core responsibility of R&D professionals.This governance process also reflects the idea of "responsible innovation," which underscores the responsibilities of R&D professionals to perform reliable science that addresses societal risks and facilitates societal needs and aspirations (McCrea et al., 2022;Owen et al., 2012;Stilgoe et al., 2013).More importantly, we explore the operationalization of inclusion and reflection in risk analysis.These principles are less well established and more often seen as cross-cutting concepts that are not studied independently in the existing risk analysis literature.Additionally, these principles are found in both the established risk governance and emerging responsible innovation literatures, which suggests they function as linking or bridging concepts across risk and responsibility scholarships.Given this, we argue that an important contribution of this study is the demonstration of a practical methodology for examining and documenting how R&D professionals operationalize inclusion and reflection in risk analysis and what value this provides to risk analysis in the R&D context.In order to demonstrate this, we draw on the Australian nanotechnology R&D sector as a case study because nanotechnology as an emerging technology was expected to achieve societal, economic, and environmental advancements.
The article is structured as follows.Following this introduction, we briefly introduce the case study context, followed by our research methods, including the study framework, data collection, and analysis.We then present our results focusing on how inclusion and reflection were operationalized in risk analysis, who were involved in the operationalization, and what the underpinning objectives of the operationalization were.Our discussion explores the implications of our findings and the limitations of the research.Finally, we conclude with a brief reflection on the contribution of this research to risk analysis science and the broader context of the R&D of new and emerging technologies.

CASE STUDY CONTEXT: THE AUSTRALIAN NANOTECHNOLOGY R&D SECTOR
The possibility of manipulating matter at the nanoscale introduced nanotechnology as a critical driver for innovation (Rodríguez, 2017) and economic advancement (Read et al., 2016).This was accompanied by significant global investment in the sector (Roco et al., 2011).In Australia, investment in nanotechnology R&D similarly increased from the early 2000s, as it was expected to play a key role in revolutionizing the manufacturing sector (Australian Academy of Science, 2012;Australian Government, 2010).However, its development and use in some products faced worldwide scrutiny, including in Australia (Duncan, 2011;Faunce, 2010;Oberdörster et al., 2005).Despite the broad benefits of the technology and its application across numerous sectors, the associated risks have also received significant attention (Bosso, 2012;Pidgeon et al., 2011;Williams et al., 2010).This gave rise to a range of risk analysis studies concerning nanotechnology (e.g., Bostrom & Löfstedt, 2010;Kuzma & Priest, 2010;Williams et al., 2010).Given this experience with developing an emergent and disruptive technology, this research identified the Australian nanotechnology R&D sector as a suitable case study in which to apply and test a conceptual framework for demonstrating how Australian nanotechnology R&D professionals operationalized inclusion and reflection in their risk analysis over the last two decades.
Various initiatives were undertaken to advance nanotechnology R&D in Australia (Figure 1, the timeline is not drawn to scale).The possibility of nanotechnology R&D to benefit social sectors, such as health and the environment was recognized in various policy documents (PMSEIC, 2005).The Australian Government introduced schemes, such as Backing Australia's Ability, to channel government funding into nanotechnology R&D (Australian Government, 2007).The Australian Nanotechnology Network was established in 2005 to foster nanotechnology R&D collaboration between research agencies (national and international) and industry (Australian Nanotechnology Network, 2023).However, nanotechnology R&D met with some opposition, especially in relation to the use of nanoparticles in sunscreens, with concerns raised by various nongovernment organizations (Miller, 2008;Miller et al., 2006;Miller & Senjen, 2008).Following this, the scientific community conducted studies to investigate the negative consequences of nanoparticles on human biological systems (Barnard, 2010;Osmond & McCall, 2010).Additionally, a Senate inquiry was conducted in 2006 to assess how nanoparticles affect the safety of people in the workplace (Australian Government, 2006), which recommended the development of Australia's national strategy for nanotechnology R&D.A review of the then-current regulations was undertaken in 2007 (also known as the Monash report) to examine whether they were appropriate for safeguarding the health and safety of people who used or worked with nanotechnology (Ludlow et al., 2007).The Australian Government adopted an approach toward the responsible management of nanotechnology in 2008 (Australian Government, 2008).The three high-level objectives of this approach were (1) to protect the health and safety of humans and the environment, (2) to foster informed community debate, and (3) to achieve economic and social benefits from the responsible adoption of nanotechnology.In 2012, the first Australian national nontechnology research strategy was published, reflecting these three high-level objectives (Australian Academy of Science, 2012).

Study framework
In this study, we specifically examine the nature of how principles of inclusion and reflection interact with the established components of risk analysis (risk assessment, risk management, and risk communication).In doing so, we demonstrate a methodological framework that provides an empirical account of how R&D professionals have operationalized inclusion and reflection in risk analysis in the Australian nanotechnology R&D sector. Figure 2 shows the framework applied in this study.It has the following four components, each of which is described as follows: 1. Two principles adopted from risk governance and responsible innovation literatures (Owen et al., 2012;Stilgoe et al., 2013): inclusion and reflection (green boxes) (Malakar et al., 2022); 2. Practices of inclusion (multidisciplinary research and multi-stakeholder engagement) and reflection (reflective discourses and reflective decisions) (grey boxes); and 3. Three areas of operationalization: who, why, and how (orange box); 4. Three core elements of risk analysis, namely, risk assessment, risk management, and risk communication (blue box) (Aven, 2020;Aven & Thekdi, 2022).
First, we provide a brief description of the more established components of risk analysis within the framework.Risk assessment captures the assessment of the nature of multiple risks (Renn & Roco, 2006) and ways to evaluate them (Aven et al., 2018), including but not limited to occupational health and safety, environmental health, public health, and social and ethical risks.Risk management encompasses undertaking specific actions relating to policy and regulatory arrangements (Renn & Schweizer, 2009), including investment in risk research, and acting to prevent, mitigate, and adapt to identified risks.Risk communication involves the two-way exchange of knowledge, practice, and experience among various stakeholders, including experts, non-experts, and policymakers (Aven et al., 2018;Renn, 2015).
Second, the inclusion and reflection components are less well established and often seen as cross-cutting concepts that are not studied independently in the existing risk analysis literature.As noted, inclusion and reflection in scientific research and technology development feature in both risk governance and responsible innovation literatures.Therefore, we draw on these literatures and elaborate on these two components and the associated practices in more detail.
Inclusion in scientific research and technology development is about facilitating the participation of multiple stakeholders to ensure their perspectives are considered in the process of R&D and embedding responsibility into the innovation process (Lacey et al., 2020;Russell, 2013;Stilgoe et al., 2013;van Mierlo et al., 2020).Inclusion broadens out from a sole reliance on experts and enables a range of stakeholders to be involved in shaping ethically desirable and socially acceptable research and innovation pathways (Stilgoe et al., 2013;van Asselt & Renn, 2011); it can be organized prior to performing significant R&D activities (Joly & Kaufmann, 2008;Kuzma et al., 2008) or while they are being conducted (Rip & Robinson, 2013;Webler & Tuler, 2021).In risk analysis, inclusion has the potential to facilitate the representation of risks perceived by a range of stakeholders so that they may be systematically addressed.Central to inclusion is the ability to build the risk decision capacity of risk managers and make informed risk decision via deliberation with stakeholders (Cormick, 2012).Based on our review of literature, we identified the following two key inclusion practices in the context of new and emerging technologies: 1. Multidisciplinary research (i.e., inclusion of multiple scientific disciplines); and 2. Multi-stakeholder engagement (i.e., inclusion of public, industry, and government officials).
Reflection in scientific research and technology development enables the R&D professionals to revisit the values and norms underpinning their own assumptions and decisions (Espig et al., 2022;Stilgoe et al., 2013).In effect, it also allows R&D professionals to scrutinize their own work with a broader range of stakeholder inputs (Owen et al., 2013).Reflection in this framework denotes the iterative processes of reviewing risk perceptions and risk decisions to develop and implement adaptive measures to evolving circumstances.Reflection is particularly important in conducting the risk analysis of new emerging technologies because it helps to address the limited knowledge about the nature of risks that may be posed and the uncertainty associated with these technologies (Fisher et al., 2006;Fleming et al., 2021;Renn, 2008;Rip & Robinson, 2013).Based on our review of literature, we identified the following two key reflection practices in the context of new and emerging technologies: 1. Reflective discourses (i.e., the continuous and iterative discourses about how risks are identified and how risk-related decisions are made); and 2. Reflective decisions (i.e., the continuous and iterative processes to review and revise risk-related decisions and ensure that are integrated into subsequent practice).
We use these practices to examine the evidence of inclusion and reflection respectively in risk analysis.We structured our documentation of the evidence using the following three areas of operationalization: 1.The stakeholders involved in operationalizing inclusion and reflection in risk analysis (i.e., the "who"); 2. The objectives of using or adopting inclusion and reflection practices in risk analysis (i.e., the "why"); and 3.The processes by which inclusion and reflection occurred in risk analysis (i.e., the "how").

Data collection
We employ a case study approach in this research, which is an established method in risk science (e.g., Dow & Tuler, 2022;Lazrus et al., 2016;Pollard & Rose, 2019;Ram & Webler, 2022).Within this research approach, the aim is not to generalize findings but to document the insights for wider reference and application (Lapan, 2011).In the context of this study, we aim to demonstrate a practical methodology to systematically examine and document the operationalization of inclusion and reflection in risk analysis.
A qualitative method was used for data collection, using a semi-structured interviewing model (Minichiello et al., 2008).To recruit participants, we approached experts of significant national and publicly available Australian nanotechnology publications to invite their participation.Using a snowball sampling method (Marshall & Rossman, 2011), we also invited participants to suggest other potential experts to inform the research.A total of 28 interviews were conducted via telephone, audio-recorded with permission, and professionally transcribed for data analysis.Most participants were from universities (13) and publicly funded research organizations (13) with other participants from industry (1) and a regulatory body (1).For participant selection, a criterion of 5 or more years of experience in the Australian nanotechnology R&D sector was required.The experience of the partici-pants interviewed ranged from 5 to more than 15 years (5 years = 1 participant, 6-10 years = 3 participants, 11-15 = 9 participants, >15 years = 15 participants).
Participants had expertise across various fields, including nanometrology, nanotoxicology, nanofibers, environmental chemistry, nano ethics, public communication, and nano regulation.We developed an interview protocol, which covered the three components of risk analysis, that is, risk assessment, risk management, and risk communication.Topics covered under these components were: participants' work experience in nanotechnology; their practices to assess potential risks in their field; their involvement in informing risk management and other related policies; and processes of risk communication.No direct questions related to inclusion and reflection were asked because the objective was to surface the presence of inclusion and reflection practices within risk analysis without leading the participants' responses.

Data analysis
For data analysis, we employed a deductive approach, using the methodological framework described above (Figure 2).First, we created three themes of risk assessment, risk management, and risk communication from the interviews.
Responses from participants pertaining to these themes were identified and coded for analysis.As the questions in the interviews were around these three themes, the coding of responses was straightforward.

RESULTS
Participants identified a range of practices in relation to the risk analysis components, in which we were able to systematically identify evidence of inclusion and reflection.We describe the results in the following sections.

Operationalizing inclusion in risk analysis
In examining inclusion, we focused on identifying evidence of two inclusion practices in risk analysis.To reiterate, they were practices related to (1) multidisciplinary research and (2) multi-stakeholder engagement.Table 1 summarizes the risk analysis components and the evidence of inclusion across the three areas, that is, the "who," the "why," and the "how."This evidence of inclusion and how it was operationalized in each risk analysis component is described below with a selection of illustrative quotes from participants.

In risk assessment
Evidence of inclusion in risk assessment was mainly demonstrated through the involvement of scientists, industry, and government officials (the "who").Participants spoke of working with this group of stakeholders to measure toxicity, assess risk perceptions, and optimize risk assessment processes (the "why").The main approaches (the "how") that participants employed to operationalize inclusion were multidisciplinary scientific studies.According to participants, the involvement of these stakeholders increased their knowledge for assessing risks in nanotechnology R&D.
The following quote from a nanomaterial scientist describes a scientific collaboration with an international university, which helped them understand the risks of a particular nanomaterial.
The other project that I worked on was a collaboration with [the] University of [deidentified]… [for] assessing whether carbon nanotubes over a longer period might have asbestos-like qualities that have shown in the short term.(I 26) Although risk assessment was traditionally understood as the domain of scientists, the involvement of stakeholders beyond the scientific community (albeit in scientific studies) indicated that practice was shifting toward more inclusive engagement of other expert stakeholders and their experiences (such as industry and government officials) to assess the risks associated with nanotechnology R&D.This was exemplified by one participant who worked in nanomembranes and reported how they benefited from collaboration with a government agency to assess nanotechnology risk: Safe Work Australia was quite involved [in our study] … they made a lot of contributions to getting a much better idea of where the risks lie, and some ideas on how to mitigate them.(I 7) Irrespective of a range of stakeholders involved in risk assessment, the main approach described by participants was including expert stakeholder input into scientific studies.This suggests that risk assessment was a core part of the scientific studies that participants undertook, and an activity that demonstrated inclusion enabling risk analysis.

4.1.2
In risk management In risk management practices, participants discussed their involvement with government officials, scientists, industry, and public (the "who").As risk management encompasses policy and regulatory arrangements and government funding, most participants identified the involvement of government officials.Three objectives (the "why") for integrating inclusion in risk management identified from the interviews were to contribute to national nanotechnology risk policies, generate funding for international nanotechnology risk research, and address stakeholder concerns.These engagements mainly occurred through two approaches: scientific studies and stakeholder consultations (the "how").Participants explained how consultations with regulatory bodies in Australia to manage the risks of nanomaterials to people working in laboratories had helped identify effective regulations.This is illustrated in the following quote from a nanotoxicology researcher: SafeWork Australia and National Industrial Chemicals Notification and Assessment Scheme [currently known as Australian Industrial Chemicals Introduction Scheme], for example, [we] worked closely together to try and classify whether certain [nano]materials were carcinogenic, or to identify what their safety standards would be for those working with these materials.(I 13) Engagement with government agencies also helped the scientific community to secure funding to conduct studies with international agencies, which, in turn, enabled international knowledge exchange in the field.One university researcher highlighted the intersection of their national and global collaborations as follows: I coordinated Australia's involvement in that [OECD] program…so that we could get access to the data that other people around the world were generating, pretty much leveraging off …the budget from…Australian regulators.I TA B L E 2 Operationalizing reflection in risk analysis.

Stakeholders-the who Objectives-the why Approaches-the how
Risk assessment

In risk communication
In risk communication, participants discussed their involvement with industry, public, and media (the "who").These activities occurred mainly to address stakeholder concerns, provide education, and support industry (the "why"), for which stakeholder consultations and scientific studies were conducted (the "how").According to participants, the public had asked questions about the use of nanoparticles in cosmetics, and risk communication aimed to inform the public of the science and address their concerns, as described by one participant, who worked in the field of nanoelectrons.
There were some concerns raised by the public, but at the same time what we did was that we went around the country and organised pub- Similarly, participants also spoke of engaging with the public by organizing informal meetings at public places and communicating science through audio and visual media.
There were always events like National Science Week… [and]…putting displays into museum-s…we liked to broadcast breakthroughs on news and media…this showed that Australian scientists were working on important problems [to the public].(I 10)

Operationalizing reflection in risk analysis
In examining reflection, we focused on identifying the evidence of two reflection practices in risk analysis.To reiterate, they were practices related to (1) reflective discourses and (2) reflective decisions.In the interviews, participants dis-cussed their risk analysis practices with examples.Table 2 summarizes the risk analysis components and the evidence of reflection practices across the three areas, that is, the "who," the "why," and the "how."This evidence of reflection and how it was operationalized in each risk analysis component is described below with a selection of illustrative quotes from participants.

4.2.1
In risk assessment We found the evidence of reflection in risk assessment occurred exclusively among scientists (the "who").The objectives of practicing reflection in risk assessment were to address stakeholder concerns, update risk perceptions, and ensure laboratory safety (the "why").To achieve these objectives, participants explained how they reviewed and revised their decisions about the risk levels and toxicity of nanotechnology R&D (the "how").Several of the participants repeatedly emphasized the public's concerns over the use of nanoparticles in consumer products.To address these concerns around changing risk perceptions, participants described the reassessment and reclassification of the toxicity of some nanoparticles to respond to such concerns.Referring to the scientific study that they were conducting to reclassify nanoparticles, one nanomaterial researcher explained: …in our carbon nanotube research, we found that some forms of carbon nanotubes did actually degrade in biological fluid, but then others did not and still have that asbestos like effect.(I 26) Evidence of reflection in risk assessment was also described in terms of designing risk assessment approaches to ensure safety in laboratories.One participant who worked in environmental chemistry reported the continuous discussion and reflection they had within their science team in relation to risk assessment protocols: The question was, is an artificial or a manufactured nanomaterial different from a natural one?And so that was the question we were trying to answer.Do we need to actually come up with a different way of assessing their risk and then on the environment?(I 18)

4.2.2
In risk management In risk management, two groups of stakeholders, that is, scientists and government officials (the "who"), were identified in relation to reflection practices.The objectives of integrating reflection in risk management were to address stakeholder concerns, ensure laboratory safety, safeguard the safety of consumers, and strengthen regulations (the "why").These objectives were achieved by four approaches: reviewing policy and regulatory priorities, funding risk projects, reclassifying toxicity, and banning chemicals (the "how").The use of nanoparticles in consumer products raised stakeholder concerns, and participants reported of working with government bodies to reclassify the toxicity of certain nanoparticles.
Additionally, the process of reviewing existing regulations was identified and one participant, who worked in an Australian regulatory body, highlighted their work with scientists and government agencies to assess the safety of consumers and people working in laboratories: We ran a project to look at the readiness of different regulatory frameworks to manage nanotechnology.[The project] looked at the main regulators in Australia, in relation to areas that nanotechnology is having an effect on, such as industrial chemicals, therapeutic goods, agricultural, veterinary chemicals and food related chemicals, and looked at what those regulations do, how they do it, and how they might pertain to safety of consumers and scientists.(I 11) Additionally, participants noted that the toxicity of nanoparticles was not fully understood at the time, and nanoparticles were treated much like any other chemicals.As the risk studies pointed toward the risk of nanoparticles to humans, participants explained that they carried out additional studies to reclassify the toxicity of these chemicals.In one case, a government body banned the use of a particular nanomaterial.It was because of the findings of their studies, one participant who practiced law mentioned that regulatory bodies were keen to make reflexive decisions to address loopholes and improve existing regulations.
Most of the regulators did actually fix up the problems.It's not that things weren't regulated, it's just that there were small details that meant it wasn't regulated as thoroughly as it could have been.So, they all went through and corrected all of those things.(I 15) Participants also described government agencies revising their research funding priorities to the safety issues of nan-otechnologies.Consequently, participants explained they got funding from a government research agency to design a new risk research project just to study the effects of nanoparticles in humans.

In risk communication
The evidence of reflection in risk communication occurred via engagement with the public (the "who").This often occurred in response to the growing concerns the public were expressing about the safety of nanotechnology (the "why").This resulted in taking reflective actions by participants to change the strategy of science communication (the "how").One participant who worked in science communication reported that nanotechnology was a much hyped phenomenon at the time.Scientists had high expectations of its potential and were experimenting with the use of nanoparticles in various consumer-focused products.At the time, there were also studies documenting the health and environmental impacts of nanoparticles.That is why, their project reviewed the science communication strategy and adopted a proactive rather than reactive approach.
We were trying to get ahead of public concern by being open [with the public] and trying to be foresightful about what people might think, and therefore interrogate that, and also be a little bit more cautious while informing the public about the adoption of the new technology.(I 8)

Summarizing the "who," the "why," and the "how"
This section summarizes the findings related to the three areas (stakeholders, objectives, and approaches).We adopted this systematic approach to identifying the "who," "why," and "how" as a way of building a robust methodology and evidence base, which would not only enable the identification of inclusion and reflection practices but also demonstrate how they were being operationalized into risk analysis processes.As noted earlier, inclusion and reflection are often treated as cross-cutting concepts that are rarely given focused attention or studied independently in the risk analysis literature.This can mean that the evidence of how they are included in risk analysis, or not, has been largely anecdotal or implicit.In Figure 3, we show the evidence of inclusion and reflection that was identified across the three risk analysis components in our case study.We used frequency analysis, where the thickness of the strands reflects the number of participants who identified the evidence of inclusion and reflection practices in their responses.
The analysis of stakeholders (the "who") shows that various stakeholders were involved in operationalizing inclusion and reflection in the risk analysis of nanotechnology R&D.Inclusion practices engaged more diverse stakeholders F I G U R E 3 Visual representation of the (A) "who," (B) "why," and (C) "how" of operationalizing inclusion and reflection in risk analysis.
(media, industry, public, and government officials) beyond the scientific community in comparison to reflection practices (which tended to occur between scientists and government officials).In Figure 3A, "Scientists," "the public," and "government officials" are highlighted as they were involved in integrating both inclusion and reflection into one or more risk analysis components.More specifically, government officials were identified as key players in implementing both inclusion and reflection into all risk analysis components; scientists in risk assessment and risk management; and the public in risk communication and risk management.We also highlight "industry" in the figure, as their involvement was identified in implementing inclusion into all risk analysis components.
The analysis of objectives (the "why") shows the greatest range of responses (Figure 3B).These objectives cover technical (laboratory safety, measuring toxicity, and risk procedures), regulatory (national risk policy, securing international funding for risk research, strengthening regulations, and consumer safety), and societal domains (educating the public, supporting industry, addressing societal concerns, and risk perceptions).The figure highlights two objectives that showcase relationships with multiple risk analysis components.
First, one objective that was repeatedly discussed by participants was "addressing stakeholder concerns."Additionally, it was identified as an objective of integrating both inclusion and reflection into two or more risk analysis components (inclusion-risk management and risk communication; reflection-all three components).The results suggest that as stakeholders started raising questions about the use of nanoparticles into consumer products, it triggered engagement with (above-identified in Figure 3A) stakeholders to (re)examine and (re)assess the risks, develop (or revise) regulatory arrangements, and communicate the risks (or review risk communication strategies).Similarly, "risk perception" is also highlighted in the figure, as it was identified as an objective underpinning the integration of both inclusion and reflection into risk assessment.According to participants, they engaged with stakeholders to assess and reassess the perceived risks of nanotechnology R&D.Second, we have also highlighted the objective of "lab safety" because it enabled the integration of reflection into risk assessment and management.To ensure the safety of people who worked in labs, participants spoke of reassessing the toxicity of nanoparticles and contributing to improvements in regulatory frameworks.
Finally, the analysis of approaches (the "how") shows that diverse approaches helped integrate inclusion and reflection into risk analysis processes (Figure 3C).There were eight approaches identified by participants; six were related to reflection, which demonstrates the identified approaches tended to be more reflexive than inclusive.The figure highlights three approaches, "scientific studies," "stakeholder consultations," and "reclassification of toxicity" being linked to all three risk analysis components.We found that it was largely through "scientific studies" and "stakeholder consultations" that participants integrated inclusion in risk analysis.According to participants, they became involved with scientists and government officials to assess nanotechnology risks and contribute to managing those risks.Stakeholder consultations were reported to involve government officials, industry, the public, and media.These consultations were targeted at communicating nanotechnology risks.The approach of "reclassifying toxicity" implemented reflection into both risk assessment and risk management.Participants also described how their reassessment of the toxicity of certain nanoparticles informed Australian regulators to review and improve regulations.

DISCUSSION
This study demonstrates a practical methodology for examining and documenting how R&D professionals operationalize inclusion and reflection in risk analysis.Below, we discuss the three key findings from this study.

Translating theory into practice supported by empirical assessment
New technologies continue to emerge, and there are risks inherent to these technologies: what consequences (desirable and undesirable) will ensue, and what are the uncertainties associated with the consequences (Bowman et al., 2017)?Risk analysis helps to solve risk problems (Aven & Thekdi, 2022), but the risk analysis of new and emerging technologies demands approaches that enable consideration of multiple risks, wider stakeholder engagement, ethical responsibility, and support an R&D process that is agile (Douglas et al., 2022b;Hankins, 2015;Renn et al., 2020;Torgersen & Fuchs, 2017).We employed a framework, drawing on the theories of risk governance and responsible innovation, to demonstrate that such approaches can be systematically applied.In our case study, we explored a methodological approach that would systematically identify and build the evidence to show how and by who principles of inclusion and reflection were being operationalized into risk analysis processes.Although it has been argued that risk analysis already includes some features of risk governance, we show there is value in systematically exploring how theories of risk governance and responsible innovation can be examined empirically in risk analysis contexts (Aven & Renn, 2020).
We argue that the methodology employed in this study can help generate new insights that enhance risk analysis practices about multidisciplinary research (e.g., engagement with scientists from multiple disciplines) and multi-stakeholder engagement (e.g., engagement with government officials, industry, public, and media).Additionally, this research identifies practical ways in which reflective discourses (e.g., reviewing and reexamining previously held knowledge) and reflective decisions (e.g., decision to reclassify toxicity) about the potential risks of these technologies can be identified and integrated into risk analysis processes.We contribute to the risk analysis literature by demonstrating a method to document: (1) the stakeholders involved (i.e., the "who"), (2) the objectives of this integration (i.e., the "why"), and (3) the process by which this integration occurred (i.e., the "how"), we have explored a systematic and evidence-based approach to elements of risk analysis that are seldom given any dedicated attention (i.e., they are seen to be cross-cutting or implicit in risk analysis).This allowed us to identify not only how inclusion and reflection can be operationalized in risk analysis but also identify opportunities this type of integration might offer to risk analysis.For example, we found the limited involvement of the public in risk assessment and risk management.
This suggests that there are opportunities for exploring the role of public involvement in these risk analysis components.The results also identified that although there were less reflection practices associated with risk communication in our case study, there may be opportunities to consider how reflective processes and decision-making could be better captured in the risk analysis processes of new and emerging technologies.
An important contribution of our study lies in the translation of theory into methodological practice and demonstrates an approach to the empirical identification and assessment of inclusion and reflection.Although we tested this methodology in the case of the Australian nanotechnology R&D sector, we argue that this approach is equally applicable to the R&D of other new and emerging technologies and hence believe that this methodology contributes to generic risk analysis science.

Fostering decisions about risk through multi-actor and multi-institutional processes
Existing risk analysis literature already identifies the importance of involving a range of stakeholders in assessing, managing, and communicating risks (Linkov et al., 2018;Malakar, Lacey, Twine, & Bauer, 2023;Renn, 2015).However, the uncertainties related to the consequences of technologies that are new or emerging can be comparatively high (Abbott, 2013), thereby making stakeholder engagement processes more challenging.For instance, it may be uncertain what the impact will be, who it will affect and how, and how these risks might be best mitigated and managed, and whose responsibility that might be.Technology development is a process that involves multiple actors and institutions; decisions about risks, therefore, should arise from multi-actor and multi-institutional processes (Linkov et al., 2018;Renn, 2015).As these stakeholders might have different understandings of risks, a comprehensive risk analysis can only be achieved by engaging them in the process (Dalton-Brown, 2016;Jones et al., 2014;Malakar, Lacey, Twine, McCrea, et al., 2023).Our study shows that a systematic approach to the identification and understanding of inclusion and reflection can be used to identify the full range and type of stakeholder roles in risk analysis.For example, we were able to identify how stakeholders perceived risks and what roles they played in risk analysis.More specifically, we found that scientists, government officials, and industry were considered critical in risk assessment, risk management, and risk communication, respectively.
There are multiple ways to assess risks (Hansson, 2017), particularly when uncertainties are high.The evaluation of risks needs to be performed as uniformly as possible, by exploring all types of risks and addressing those risks (Hansson, 2017).We found that exploring how stakeholders went about operationalizing inclusion and reflection in risk analysis also revealed a wider range of risks and an associated understanding of those risks across various domains.For example, the nanotechnology R&D professionals who participated in our research identified a range of technical, regulatory, and social risks.Our results also identified who plays key roles in addressing these risks (i.e., scientists, government officials, and industry) along with how they go about doing that.
The existing literature on scientific research and technology development calls for innovation pathways to be responsible toward society.It is argued that this is achieved by addressing perceived risks and concerns and embedding societal values and aspirations in scientific innovation and technology development (Jarmai & Vogel-Pöschl, 2020;McCrea et al., 2022;Stirling, 2007).Our research found that addressing stakeholder concerns was identified as a key objective in operationalizing both inclusion and reflection in risk analysis.Such efforts formed part of the outreach activities designed to address stakeholder concerns, which additionally provided an opportunity for research participants to engage with multiple stakeholders, reflect on broader understandings of risk and make subsequent decisions about how to manage risk in nanotechnology R&D.This demonstrates that operationalizing inclusion and reflection may also be a valid pathway for facilitating the integration of societal values in risk analysis and as a way of making decision about the risk of new and emerging technologies.However, further exploration of the societal value generated from these processes is needed.

Building both the outward-facing and introspective nature of risk analysis
A robust risk analysis process should evolve as the contextual circumstances require it (Seno-Alday, 2018), and this may include changes caused by new studies, regulations, and public opposition, for example.Depending upon the relevant circumstances, the boundary of risk analysis may need to be readjusted, and this may involve looking at who to engage with and how so as to accurately capture and represent a wider understanding of the relevant risks (van der Vegt, 2018).This iterative process combines the outward-facing nature of risk analysis as it is applied to different use cases to inform risk decisions but also the constant improvement of the science itself as we continue to expand our understanding and practice of risk analysis.The operationalization of inclusion and reflection in risk analysis may serve both purposes.For example, our research shows that inclusion practices in risk analysis were more outward-facing by nature and included a wide range of stakeholders beyond the science community, such as government officials, industry, the public, and the media.By contrast, reflection practices tended to be more introspective and largely involved scientists and to a lesser extent, government officials.The introspective nature of reflection in risk analysis of nanotechnology R&D enabled research participants to recategorize the toxicity of chemicals from low to high risk and ban unsafe materials from being used.
In our study, inclusion was operationalized through more traditional and science-focused approaches, whereas reflection was through more diverse approaches that included questioning, reviewing, and revising technical and regulatory aspects.Given the limited knowledge about how nanoparticles might affect human health at the time, participants identified the importance of conducting scientific studies and involving multidisciplinary scientists to assess potential consequences and uncertainties.It is unsurprising that the importance of scientific studies was identified as a predominant approach among the research participants, the majority of whom were scientists.In relation to reflection, reviewing policies and regulations enabled research participants to identify regulatory gaps and adapt plans and strategies to bridge these gaps in the changing circumstance.These findings suggest that by exploring inclusion and reflection in context, there may be benefits that can be demonstrated in contextual applications of risk analysis (i.e., leading to improved risk governance and decisions).But similarly, as we become more proficient in applying these empirical approaches to concepts of inclusion and reflection and making them explicit in our theory, methodology, and practice, we will also contribute to the ongoing development of risk analysis science.

CONCLUSION
The concepts of risk and responsibility are interconnected; studies exist that explore the role of responsibility in risk analysis.These studies, however, are largely limited to psychological theories and examine the attribution of responsibility in the event that undesirable consequences occur.The novelty of this study, therefore, is to investigate an approach that transcends from retrospective forms of responsibility toward a more practical forward-looking or anticipatory approach to responsibility in risk analysis.Risk analysis of new and emerging technologies requires innovative approaches that are agile, exploratory, and can accommodate broad stakeholder engagement and perspectives.Existing theories of risk governance and responsible innovation identify that the principles of inclusion and reflection are critical to conducting risk analysis for such technologies.Although risk scholars argue that risk analysis already includes some features of risk governance, in this study, we show there is value in systematically exploring how theories of risk governance and responsible innovation can be examined empirically in risk analysis contexts.Our study contributes to the risk analysis literature by demonstrating a methodology that not only systematically identifies these principles in risk analysis but also creates an evidence base that identifies the stakeholders, their objectives, and the approaches they adopt for assessing, managing, and communicating risks.We applied this conceptual and methodological framework using the Australian nanotechnology R&D sector as a case study and found this approach enabled the translation of the abovementioned theories into documenting inclusive and reflexive risk analysis practice.
We found that the systematic integration of inclusion and reflection also fosters a wider understanding of risks through multi-actor and multi-institutional processes.Further, this approach contributes to both the outward-facing and introspective nature of risk analysis.Finally, although we initially developed and tested this methodology with the Australian nanotechnology R&D sector, which allowed us to look back across a 20-year R&D period, we believe the approach could be improved through application in other technology contexts, particularly those that are grappling with currently emerging and newer stages of technology development.

F
I G U R E 1 Major events and timeline of nanotechnology research and development (R&D) in Australia.F I G U R E 2 Methodological framework used in the study.
lic consultations and open forums [to share our science].(I 19) Operationalizing inclusion in risk analysis.
TA B L E 1