Understanding “influence”: An empirical test of the Data‐Frame Theory of Sensemaking

This paper reports findings from a study designed to gain broader understanding of sensemaking activities using the Data/Frame Theory as the analytical framework. Although this theory is one of the dominant models of sensemaking, it has not been extensively tested with a range of sensemaking tasks. The tasks discussed here focused on making sense of structures rather than processes or narratives. Eleven researchers were asked to construct understanding of how a scientific community in a particular domain is organized (e.g., people, relationships, contributions, factors) by exploring the concept of “influence” in academia. This topic was chosen because, although researchers frequently handle this type of task, it is unlikely that they have explicitly sought this type of information. We conducted a think‐aloud study and semistructured interviews with junior and senior researchers from the human–computer interaction (HCI) domain, asking them to identify current leaders and rising stars in both HCI and chemistry. Data were coded and analyzed using the Data/Frame Model to both test and extend the model. Three themes emerged from the analysis: novices and experts' sensemaking activity chains, constructing frames through indicators, and characteristics of structure tasks. We propose extensions to the Data/Frame Model to accommodate structure sensemaking.


Introduction
Exploring sensemaking, i.e., how people make sense of problems, sheds light on the development of a repertoire of skills to deal with novel situations. As Bates (2010) puts it: "People-even including Ph.D. scholars-develop what search skills they have incidentally to their primary efforts at research or problem-solving, and often fail to develop a conscious repertoire of search skills and techniques to help them over difficult stages." Our aim in the work reported here was to test one of the prominent models of sensemaking- Klein et al.'s (2007) data/frame (D/F) model-by applying it to a kind of sensemaking task to which it has not previously been applied (as described below) in order to test its generalizability.
The process of sensemaking involves cycles and activities that have been reported in prior studies on sensemaking models and theories. Studies by Russell, Stefik, Pirolli, and Card (1993), Dervin (1999), Pirolli and Card (2005), and Klein et al. (2007) have been widely used to explain how people interact with information and make sense of a problem situation. These studies present a theoretical explanation of the processes people go through to understand narratives or processes. For example, intelligence analysts may explain a variety of phenomena by reconstructing sequences of events and their causal relationships (Pirolli & Card, 2005).
The motivation for this study was to learn how novice and experienced researchers make sense and construct an understanding of their academic communities to better support those processes. Not all sensemaking tasks involve unraveling a narrative: They can also involve understanding a structure (i.e., structure sensemaking). Existing models of sensemaking have not been widely tested with different types of tasks. Within the sensemaking literature, this constitutes an area for further exploration-an area to which this paper contributes.
The study reported here presents behaviors in an academic context when solving four challenging sensemaking tasks concerned with constructing an understanding of how a particular academic community is organized: which are the main components and parts (e.g., people, contributions, sources), how these components and parts are interrelated, and what the structure is. This extends previous work reported by Pontis and Blandford (in press), which focused on the roles of professional and domain expertise in shaping people's sensemaking processes. In that earlier work, Klein et al.'s (2007) D/F model was used as a sensitizing framework, but was not the focus of analysis. In the work reported here, the focus of analysis is specifically on the D/F model and on the extent to which it accounts for the data. Our intention was to test how well the model generalizes to structure tasks, having originally been developed for narrative tasks. The D/F model has been previously used in information science contexts to exemplify the iterative process involved in information seeking (Jansen & Rieh, 2010), to describe the complex cognitive activities involved in information processing through visual representations (Parsons & Sedig, 2014), and to understand "the cognitive process and mechanisms of . . . sensemaking" (Zhang & Soergel, 2014, p. 1733. Although Klein et al.'s (2007) D/F model is one of the dominant models of sensemaking, it has not been tested with sensemaking tasks that involve making sense of structures or organizations. In this paper, we code and analyze cases looking for evidence supporting the idea of frames, and other key components of the model, but also for cases that indicate contradictions or further details that did not feature in the original model. Saturation was achieved after four cases (16 tasks).
The aim of this study was to test whether the D/F model could be used to understand nonnarrative sensemaking tasks. Outcomes from applying the D/F model in structure sensemaking tasks may allow the development of betterinformed tools or systems to support such sensemaking, and contribute to science research by presenting aspects involved in the sensemaking of structure tasks. In addition to providing a set of recommendations to expand or embellish the theory to describe activities of broader types of sensemaking tasks, our study contributes to information behavior research by exploring how a specific population that frequently deals with structure sensemaking tasks (i.e., academics) interacts with information and constructs an understanding to solve those tasks. Findings can be applied to better understand the "interaction between people and content in information systems" (Jansen & Rieh, 2010) expanding the information seeking and information retrieval literature.

Related Work
Sensemaking involves knowledge construction from information and gaining understanding of a problemsituation. The process has been studied in various disciplines including information science (e.g., Dervin, 1999), humancomputer interaction (HCI) (e.g., Pirolli & Card, 2005;Russell et al., 1993), organizational studies (e.g., Weick, 1995), and organizational behaviors and information dominance (e.g., Klein, Moon, & Hoffman, 2006). Sensemaking starts when a person realizes the ambiguity or inadequacy of their current understanding of a situation or when they want to expand their understanding of what is going on in a situation. They then make a "deliberate effort" to seek new information to construct a broader or new understanding , or to combine their understanding with the understanding of others (Dervin, 1999). People examine the environment (reality, situations, events) by interacting with a small amount of the information they encounter. This is an active interaction that involves discovering connections among data (e.g., people, places, events) and giving meaning to experiences. Through the process of sensemaking, people "integrate what is known and what is conjectured, to connect what is observed with what is inferred" (Klein et al., 2007, p. 114). Sensemaking can be used for a varied range of situations from problem detection, connecting dots and forming explanations to projecting future states, and identifying problems; people follow various strategies to gain knowledge, draw inferences, and make predictions from data.
Although existing models of sensemaking have commonalities (e.g., they describe similar steps, actions and phases involving information seeking and gathering, and information retrieval and interaction), they have been developed from different perspectives. For example, Russell et al. (1993) approach sensemaking as "a process of modeling" while Dervin's (1999) theory sees it as "a process of education" (Kolko, 2010).
These differences in perspective can be illustrated by comparing Dervin's theory of sense-making (1999), which originated in the contexts of information science, information needs, and use, with Klein's theory of sensemaking (1997), which is the analytical theory we use in this study. Table 1 compares the core aspects of both theories.
Dervin uses the term sense-making and Klein uses the term sensemaking to refer to their respective theories. In this paper, we use the former spelling when we refer to Dervin's theory and the latter spelling when referring to Klein's theory. Klein's D/F model has been introduced to express "the core process of sensemaking as it occurs in high-end, proficient problem solving" (Moore & Hoffman, 2011, p. 145), whereas Dervin's presents "a theoretic net, a set of assumptions and propositions, and a set of methods that have been developed to study the making of sense that people do in their everyday experiences" (Savolainen, 2006(Savolainen, , p. 1117. Although using different terminologies, both theories discuss the construction of new knowledge to solve a problem (Klein, 1997) or fill a gap (Dervin, 1999), while using frames (Klein, 1997) or verbings (Dervin, 1999) to guide the processes of information seeking and use. Dervin's theory of sense-making is based on "gapbridging," which begins when an individual faces a problematic situation ("gap-facing") (Savolainen, 1993, p. 17). The theory is built on "the metaphorical triangle of situation-gap-uses in that gap-bridging stands for the process which results in various outcomes of information seeking and use" (Savolainen, 2006(Savolainen, , p. 1117. People "construct bridges" (find answers) to fill the gap of knowledge/ information; "the attempts to bridge [the] gap are preceded by gap-facing and gap-defining, followed by the consideration of gap-bridging strategies and tactics" (Savolainen, 2006(Savolainen, , p. 1120. "The construction of the 'informational' bridge consists of identifying, finding, and combining various elements such as ideas, beliefs, and narratives" (Savolainen, 2006(Savolainen, , p. 1121. As in Klein's (1997) theory, Dervin argues that people use prior experiences (internal sources) to anticipate possible outcomes of gap-bridging, but when they cannot rely on their knowledge they look for external familiar sources until they find the information that helps them fill the gap. Although the frame is changed and adapted through cognitive activities in the D/F theory, gapbridging strategies and tactics are used to determine ways to construct the most appropriate bridge in the sense-making theory. A difference between both theories is that the "nature of these strategies and tactics is not discussed in greater detail in sense-making" (Savolainen, 2006(Savolainen, , p. 1122, whereas in Klein et al.'s theory (2007) cognitive activities that people follow to make sense of a situation are explained in detail.
In this study, we analyze the D/F model because it considers sensemaking as a process of problem-solving based on sensemakers' experiences and interpretations, and because it provides a clear explanation of common activities involved in the process. In the next sections we describe in more detail the D/F Theory of sensemaking, and discuss other relevant theories for this study. Klein (1997) introduced the D/F theory of sensemaking ( Figure 1) which is derived from his work on a naturalistic decision-making framework for understanding military decision-making. This theory contemplates the creation of internal, cognitive representations when people are making sense of a situation. Building on Minsky's work (1977), Klein et al. (2007) argue that elements are explained when they are fitted into a structure that links them to other elements. Minsky (1977, p. 355) explained that when we are making sense of a situation we "select from memory a structure" of information. Memory is the place where we store our repertoire of frames. Both theories use the term "frame" to refer to the data-structure (Minsky, 1977) or explanatory structure  that defines entities and describes their relationship with other entities. A frame can take the form of: a story (explaining a chronology of events), a map (explaining location, showing distances and directions), a script (explaining roles), or a plan (describing a sequence of intended actions) . Sharing a similar view of explanatory structures, Russell et al. (1993) see those representations as external constructions, referred to as schemas, rather than internal representations.

Frames as Explanatory Structures of Sensemaking
When making sense of a task, we need a structure (e.g., frame) to direct the search for ways to gain new understanding, for example, seeking information (Pirolli & Card, 2005;Russell et al., 1993). These structures define and give meaning to the elements of a situation, and guide sensemakers in the process of filtering (relevant from irrelevant) and interpreting data. Various studies have analyzed the selection or identification of that initial structure (e.g., Minsky, 1977;Bodnar, 2005;Klein et al., 2007;Russell et al., 1993;Pirolli & Card, 2005). Whereas Minsky (1977) argued that the situation determines the selection of one frame or another, Klein et al. (2007, p. 118) stated that "the data identify the relevant frame, and the frame determines which data are noticed." Klein et al. added that "neither of these [frame or data] comes first" as the "data elicit and help to construct the frame and the frame defines, connects and filters the data" (2007, p. 118). Both authors argue that that structure (frame or schema) explains the data and guides the search for more data, reflecting the sensemaker's accumulated experiences. Similarly, Attfield and Blandford (2011) describe internal and external representations (explanatory structures) as a consequence of possessing an understanding of the task domain and going through a "bottom-up and top-down" sensemaking interaction. Even if the initial frame  Klein et al. (2007) and Moore and Hoffman (2011). is poor or mistaken, it guides the search for data until new information helps to identify initial errors and redirect the search.
During the sensemaking process, the initial selected frame adapts as we encounter more information. Klein et al. (2007, p. 118) describe the frame as an active structure that evolves as new information becomes available, and that can be "improve [d] or replace[d] (. . .) to obtain more relevant data" through cognitive sensemaking activities. This paper explores academics' sensemaking activities to identify, select, and use frames to make sense of the structures of both familiar and unfamiliar domains, and gain an understanding of influence. As in previous studies Sieck et al., 2007), we investigate the relationship between the selection of frames and domain expertise.

Cognitive Sensemaking Activities
Broadly, sensemaking activities relate to organizing, collecting, and accessing information to construct the necessary understanding. More specifically, sensemaking involves "learning about new domains, solving illstructured problems, acquiring situation awareness, and participating in social exchanges of knowledge" (Pirolli & Russell, 2011, p. 1). Originally, Klein et al. (2007, pp. 132-142) described seven cognitive activities involved in the D/F model: 1. Mapping data and frame: This activity involves connecting the data and a frame. The identification and selection of frames is a conscious process, which depends on the data and information that is available, and on the individual's aims, repertoire of frames, and attitude (e.g., commitment to a task). 2. Elaborating a frame: Initial frames are expanded until anomalies become apparent giving an indication that the frame needs to be replaced. Otherwise we explore the initial frame, searching to add details and fill in slots. 3. Questioning a frame: If while working with the frame we encounter data which are inconsistent with that frame, we may decide that that frame needs to be replaced although we may not be certain whether the frame is incorrect. All we have found are inconsistencies that lead us to start questioning the frame. 4. Preserving a frame: When inconsistencies previously found are considered not relevant or strong enough to dismiss the frame we have been working with, that frame is preserved. A frame should not be preserved if indicators of inaccuracy emerge during the exploration process. 5. Comparing frames: In some cases we consider more than one frame at the same time. Then we compare those frames to fully appreciate the dimension of the task. Klein et al. (2007) report that three is the maximum number of alternative frames we can work with at the same time but do not provide an explanation of this. 6. Reframing: When we accumulate inconsistencies and contradictory evidence, we need to replace the frame. In other words, the structure we were using to guide our data search needs to be changed as it was misleading. In some cases, we consider data elements that we have previously discarded for our frame, but that now we find relevant as new cues emerge. 7. Constructing or finding a frame: When we encounter a situation that does not make sense for us, or our initial frame cannot be reframed, we seek and construct a new frame.
According to the D/F model, the process of sensemaking can start with any of these activities, as they do not correspond to a specific order, that is, the order responds to sensemakers' needs. Klein et al. (2007) added that sensemaking can involve a few activities or all of them and that each sensemaking activity involves actions that define and update the frame and the data.

Narrative and Structure Sensemaking
Sensemaking tasks are often associated with the reconstruction of events (e.g., incident analysis) or understanding processes (e.g., how steps or actions occur). There is a variety of sensemaking tasks corresponding to those characteristics, as illustrated in the introduction to a special issue on sensemaking (Pirolli & Russell, 2011, p. 2): "Examples of sensemaking activities include understanding features, costs, service plans, and trade-offs in consumer decision making (e.g., buying a cell phone); collecting, organizing, and comprehending information about a medical condition, treatment options, and trade-offs in order to choose a treatment (Bhavnani, 2002;Bhavnani, Jacob, Nardine, & Peck, 2003); analyzing a subject matter domain in order to develop an efficient and effective training course (Russell et al., 1993); and collecting and analyzing open source publications to determine the likelihood that a foreign nation is developing biological weapons (Pirolli & Card, 2005)." When making sense of any of the above situations we gain an understanding through searching for and finding relevant information and constructing a chain of events to build a story. The D/F model has been successfully applied to represent the sensemaking process of developing situations in the intelligence analysis context (Moore & Hoffman, 2011), in which multiple themes or storylines were happening simultaneously, such as scenarios based on peacekeeping operations in Bosnia (Sieck et al., 2007), and navigational problems, for example, how do I go from A to B (Sieck et al., 2007). Also, it has been applied to understand operative sensemaking in a medical context (Stewart, Dominguez, & Lawrence, 2011), for studying problem detection (Klein, Pliske, Crandall, & Woods, 2005) and incident analysis . These studies have explored narrative tasks, and in some cases, real-time sensemaking (Wahlström, Karvonen, & Norros, 2013); however, very few studies have investigated how individuals make sense of structure tasks. Instead of investigating structure sensemaking with more traditional information-seeking tasks, we explored it using influence tasks because they are intrinsic to the nature of academia and frequently dealt with by academics. This paper investigates structure sensemaking in experimental conditions, by studying how academics gain an understanding of a scientific community structure.
Sensemaking structure tasks. Academics seek and make sense of information in well-defined situations (e.g., looking for a specific book) and ill-defined situations (e.g., looking for any relevant information when starting a new project). In the latter situations, people need to determine from where to start the search (define the problem). In this study, we explore situations dealing with the problem of "influence" as an example of structure sensemaking. In academia, situations with a similar problem frequently occur at the beginning of a PhD investigation, when recruiting researchers, when assigning roles in a new project, or looking for key authors when learning about a new project or preparing a class on an unfamiliar topic. In all those situations, academics evaluate how influential other academics are, but most likely they would go through this process without making it explicit that they were seeking that characteristic.
Influence tasks are considered here structure sensemaking tasks because academics first, explicitly or implicitly, need to make sense of the structure of the academic domains under investigation in order to complete the tasks (Pontis & Blandford, in press).
Scientific community organizational structures. Whitley (2000) and Becher and Trowler (2001) present organizational features of different scientific fields (e.g., HCI and chemistry), providing a clear view of their structure and key components. Whitley (2000) describes scientific fields as organizations that control knowledge production and knowledge evaluation, which deal with high degrees of (technical and strategic) task uncertainty. The internal structure of a scientific field includes components and relationships. Research sites and groups, specialisms, schools of practice, tasks, processes of coordination, conflicts, members, and roles are some of those components. Knowing the structure of a community helps understand the interdependencies and relationships between the different components (Whitley, 2000). With the influence tasks of our study we explored strategic task uncertainty by asking participants to gain an understanding of the structure of both a familiar and an unfamiliar scientific field. We refer to this type of task as structure tasks because they explore the internal structure of an organization, in contrast to narrative tasks, in which they explore the reconstruction of a story (e.g., chain of events).

Study
This study tested the D/F model  to gain an understanding of academics' sensemaking processes and analysis activities when working on structure tasks focused on the problem of "influence." Academics (of different levels of seniority) were asked to understand and define familiar and unfamiliar domains in terms of current and future influential community members. This study addressed (a) how academics construct knowledge of their community through building an understanding of the concept of influence (being influential and becoming influential), (b) what sensemaking activities academics undertake while making sense of a community that they are already intimately familiar with and one that they are outsiders to, and (c) which sensemaking activities of the D/F model are manifested in their sensemaking journeys and which ones are not apparent. This analysis is based on the same data set as that reported by Pontis and Blandford (in press); in that paper, the focus is on how novices and experts respond to tasks within and outside their areas of expertise, and how they use internal and external knowledge sources, whereas in this paper the focus is on the applicability of the D/F model to this kind of sensemaking task. For completeness, we include details of the data gathering method here, although it is only the analysis that differs between the two papers.

Methods
We designed an experimental study with academics from the HCI domain to investigate how they made sense of structure sensemaking tasks. A think-aloud session (Ericsson & Simon, 1984) followed by a debriefing semistructured interview was conducted with each participant at University College London (UCL) labs. The shortest think-aloud session lasted 50 minutes, and the longest was 1 hour and 30 minutes. Participants were given four tasks to complete using both traditional tools (e.g., paper, pencil, post-its, etc.) and a computer with full Internet access. We did not specify particular search software or tools to be used during the session. Both think-aloud sessions and interviews were audiorecorded and transcribed verbatim for analysis, and participants' interactions with the computer were recorded using screen capture software.

Participants
We recruited 11 participants from UCL. Six participants were students from the master of science in human computer interaction (HCI) course with 1 to 5 years of research experience, which we refer to here as professional expertise. They have basic HCI knowledge and no chemistry knowledge at all. We refer to subject matter knowledge as domain expertise. The remaining five participants were members of the staff at the UCL Interaction Centre (UCLIC) with more than 10 years of research experience and more than 5 years actively working in HCI. Dorst (2004) and Kennedy (1987) stress that after 5 years of actively working in a specific domain, people develop a repertoire of experiences to handle problems without supervision, and therefore can be considered experienced. Consequently, in this paper we refer to participants with little professional and domain expertise as novices, and those with more than 5 years of experience in both types of expertise as experts.
Participants' demographic information has been anonymized and coded with a combination of letters and numbers summarized in Table 2.
To increase validity and objectivity, none of the participants in the study was familiar with the tasks in advance or had a previous personal relationship with the researcher.

Study Structure
The study was organized in three parts (Figure 2). Before the beginning of the session we gave participants a short questionnaire to gather demographic information; we asked about their research experience (professional expertise) and background domain knowledge (domain expertise). Then we gave participants an anagram as a warm-up task (Ericsson & Simon, 1984, pp. 376-378) to help them practice verbalizing what was going on in their minds (see Appendix A1). We gave participants two words (one at a time) from which they had to rearrange the letters to come up with an English word while thinking aloud. After this, we gave participants four influence tasks (see Appendix A2) aimed at exploring one familiar (HCI) and one unfamiliar domain (chemistry). We selected HCI and chemistry as the two domains of the study because participants were unlikely to have any experience in the latter, and both domains are different from each other in that chemistry is a much larger and consolidated domain, and HCI is a smaller research specialty. This contrast of domains was found useful to challenge participants throughout the tasks and gather rich experiences.
For all tasks, participants had to construct an understanding of the concepts of "being influential" and "becoming influential"; we did not provide definitions of the terms during the study. Although at the beginning of the session we stressed that there were "no right or wrong answers" to each participant, to achieve full completion of each task they needed to provide the names of three influential academics. Tasks 1 and 3 were referred to as identification tasks because participants were asked to identify current influential candidates, and Tasks 2 and 4 were referred to as prediction tasks, as participants were asked to identify who might be future influential candidates. For Tasks 1 and 2 participants made sense of the HCI domain, and for Tasks 3 and 4 they did so of the chemistry domain.
Following the D/F model, we gave tasks in the same order to help participants first make sense of the bigger picture (elaborating) before dealing with the search for more specific types of information (inferring) . HCI tasks were given first, helping participants concentrate on the search for the requested information, get used to thinking aloud, and be at ease with the task type of the study. Tasks were provided to participants one at a time. Only when the current task was completed did we give the next task. This way, we allowed participants to get immersed in each task. When a participant was feeling blocked with the current task, they moved forward to the next task. Some participants returned to complete earlier tasks after doing later tasks, and some of them revised their responses at the end of the session.
The last part of the study involved a semistructured debriefing interview with each participant. Debriefing interviews provided further details, which enriched notes and expanded observations made during the sessions (Charters, 2003). In addition, participants gave richer descriptions of unclear parts, silences, and other relevant aspects of their thinking processes. We used these data to validate interpretations during the analysis of the transcripts.

Data Analysis
We collected 44 tasks, which varied in length, in number and type of sensemaking activities, in ways of interacting with the computer, and in ways of solving the tasks, but provided a rich description of participants' sensemaking processes and activities. Transcripts from tasks were initially analyzed using thematic analysis (Braun & Clarke, 2006), and a second layer of analysis was conducted applying the D/F model to guide the analysis. The results of the  Pontis and Blandford (in press), and the focus of the present paper is on the latter layer of analysis; however, reference to findings from the initial analysis is also made and indicated when necessary.
In this paper, we report on factors that will help support sensemaking of structure tasks: which D/F sensemaking activities were manifested in the process of making sense of such tasks, the sequence of those activities, and how participants constructed their initial frames. For that, we selected four tasks from two novices (N4 and N6) and two experts (E2 and E3) that we considered richer and representative of each group of expertise. This decision was made based on our knowledge gained from the thematic analysis: We selected tasks that (a) took average time for completion, discarding shortest and longest tasks, with (b) different levels of completion (tasks with two or three names identified, and tasks with no identified names at all), and with (c) the use of soft and hard evidence to make sense of the tasks. Then we manually coded the selected 16 tasks following Klein et al.'s (2007) sensemaking activities and terminology. We stopped in-depth analysis after 16 tasks because no new information was revealed in the analysis of task 13 onwards, and the preceding thematic analysis had not highlighted any properties of other task transcripts that would make them particular candidates for additional in-depth analysis. Consequently, we concluded that theoretical saturation had been reached by this point. The analysis involved activity categorization, semantic analysis, and mapping. First, we agreed with another researcher how to categorize participants' sensemaking processes into activities using the D/F as a framework (Figure 1), making sure that we both were interpreting and applying the framework in the same way. We coded transcripts and screen capture videos independently using five variables: user's expertise, data set, action type, influence indicator, and sensemaking activity. In the analysis, we used 11 sensemaking activities: nine activities described by Klein et al. (2007), and two emerging activities not contemplated in the theory. Each researcher created analysis logs with the coded data ( Table 3) that we then compared; resolving any semantic discrepancies until we reached consensus and interrater agreement.
Then we conducted a semantic analysis coding participants' understanding of influence. Transcripts were semantically coded as follows: each time participants provided an indication of what influence would mean by them or of trying to make sense of any of the above concepts we highlighted those words. For example, participant N6's description when she found an article provided initial indications of what influence meant for her: There is actually an article for children about famous scientists, and I'm going to look at that, because it probably gives some of the basic people, important people there [Chemistry domain].
We inferred that for N6 an influence candidate needed to be famous and needed to be a chemist. In addition, for this participant being famous was understood as being important. We referred to these qualities as influence indicators. From this analysis, influence indicator types for identifying and predicting influential community members emerged, as summarized in Figure 3. Influence indicators were considered the main data components of participants' explanatory structures or frames. The last step was mapping participants' sensemaking journeys using the D/F model terminology. We mapped performed activities horizontally from left to right as chains to indicate related activities. We used a box to represent each sensemaking activity, and arrows to indicate how they were connected to each other and ordered (Figures 6 and 7). Black arrows indicate that a piece of information was found useful and triggered further activities, and gray arrows indicate that participants went back to a previous state of the process, often discarding the information found. Pontis and Blandford (in press) describe these moments as "turning points" in the sensemaking process. Chains were also numbered and organized vertically from top to bottom to reconstruct the sensemaking structure of each task. Below we explain the coding rationale.
We began the coding when participants were reading the task aloud and expressing an understanding of what they were asked to do. This activity was mapped as "recognize the situation" (Figure 4A), followed by the box "constructing the frame" ( Figure 4B) representing the moment of realization of how much participants already knew about each task. Figure 5 represents a sensemaking chain from N6's journey for Task 3 and it reads as follows: • Black solid stroke boxes indicate elaboration activities, such as "seek and infer data" (5a), "add and fill slots" (5b), and "discard data" (5d). • Light gray boxes indicate incoming information referred to as "data points" (5c), building on Moore and Hoffman's (2011) work. In this analysis, we considered two types of "data points": information found and used to identify a candidate influencer, and information found to expand participants' initial frame (i.e., identify new influence indicators). • Dashed stroke boxes indicate questioning activities, such as "gauge data quality" (5e), "judge plausibility" (5f), and "detect inconsistencies" (5g). • Darker gray solid boxes indicate choices of preselected candidates and final decisions (5h). These last two sensemaking activities are not originally considered in the D/F model.
Finally, we triangulated sensemaking transcripts with screen captures and participants' debriefing interview transcripts to determine when participants started a search to find information that may lead them to a candidate influencer (e.g., open browser), what type of keywords or terms they selected (e.g., keywords: influential chemists), and when they changed the initial seeking strategy (e.g., selected a new term/s or keyword such as Marie Curie). We also paid attention to and coded explicit actions (typically interactions with the computer), and observable actions (e.g., thinking, reading, explaining) to gather insights into the type of actions involved in each sensemaking activity.

Results
In this analysis, we explored how participants made sense of influence tasks and whether the D/F model sensemaking activities were manifested in the academic domain structure context. Overall, although participants performed most of the sensemaking activities, excluding the activities of "seeking and inferring new relationships," "tracking anomalies," and "seeking a new frame," some additional activities were manifested as well. In addition, we elaborate on how a domain is organized, what it is to be influential, and what it is to become influential: concepts that emerged from the analysis. In this paper, we present the sensemaking journeys of four participants (N4, N6, E2, and E3); we include as examples the journey maps for Task 3 of two of those participants (N6 and E2; Figures 6 and 7, respectively). These are representative of the processes of novices and experts across the four tasks of the study. When experiences from the remaining seven participants are used to complement and expand findings, this is clearly indicated.

Narrative Versus Structure Sensemaking Tasks
In narrative sensemaking tasks one piece of new knowledge helps understand the story and individuals can build on it to construct further understanding. In structure tasks, the process varies, as Figures 6 and 7 show. Participants could not follow a thread or construct a narrative because there was no story to learn or reconstruct. The aim of these tasks was to find a concrete piece of information (three influence candidates), which is a very different aim from gaining an understanding of how a series of events happened, as in narrative tasks.
The influence tasks of this study involved unraveling the structure of the HCI and chemistry domains, after having identified some parameters to distinguish what was relevant from irrelevant (indicators), key domain components (members, literature, sources), but also how to relate them (interdependencies and relationships), and assess their level of relevance (which components are more important). The sensemaking maps shown in Figures 6 and 7 indicate different journeys between novices and experts. Pontis and Blandford (in press) report differences between the sensemaking processes of both, mainly in terms of level of engagement, and the ways they made final decisions (arbitrary vs. informed decisions). Further differences emerged from this analysis, as illustrated in the above figures and discussed below.
For tasks concerning participants' own field of expertise, the main difference was that experienced participants relied mostly on their previous knowledge to complete the tasks (soft evidence), while novices' sensemaking processes were typically longer because they needed to find hard evidence on which to base their decisions (Pontis & Blandford, in press). Findings of novices and experts indicate different sensemaking processes, as discussed below.
The process of N6 is simpler, with six data points, than that of E2, with 15 data points. In addition, E2's process is more elaborated, in that she judged the plausibility of data points, discarded those she found irrelevant, and dug deeper into data points she found relevant (e.g., Data points 14 and 15). The higher number of data points discarded also shows a lack of conviction from the information found. As a result, E2 needed more convincing information before even preselecting a candidate, while N6 preselected six names from six data points.
Moreover, as illustrated in Figures 6 and 7, novice participants made sense and made decisions in shorter journeys than experts, or they needed less exploration in order to make a decision. Experts frequently explored a data point, performing various activities before discarding it, and tended to elaborate the initial frame further, searching for additional information to fill knowledge slots or verify their initial hunches or ideas. They also expressed high degrees of concern when they could not trust a source, and questioned its origin. In addition, experts evaluated incoming information in terms of their understanding and whether they could judge the relevance of that information based on their background knowledge.
We also found similarities between novices and experts. When both came across a name repeatedly (e.g., Marie Curie and John Dalton, in the cases of N6 and E2, respectively), they sought further information before making a final decision. In some cases, such as E3, participants gauged the quality and relevance of the information found in the searches. The sensemaking journeys showed a high number of turning points and discarded data points, indicating the lack of a narrative flow.
As previously reported (Pontis & Blandford, in press), in all 44 tasks we found that participants only worked with one frame. This characteristic points out a difference from narrative sensemaking in which sensemakers frequently track two or three alternative frames simultaneously until they choose one as the most relevant . Hence, instead of frames, we referred to the identification of candidates as frame instantiations, building on Russell et al. (1993), because participants kept or expanded the initial explanatory structure. In the following sections we discuss frame instantiations in more detail, analyze the main sensemaking activities manifested in the processes, and indicate those which emerged from this study but that were not originally taken into account by the D/F model.

Recognizing and Constructing the Frame
Understanding influence concepts. In different ways, at the beginning of each task, the four participants first recognized the situation and what they were asked for. To recognize the problem, they read aloud the task brief, and assessed whether their background knowledge could be of any help or was related to the problem domain. Figure 8 illustrates the thought of participant N6 at the beginning of Task 3.
After that, participants started to construct an understanding of what "being influential" meant. For later tasks, in most cases, participants expanded or adapted their previous understanding of "influence" to the task requirements (e.g., prediction instead of identification). In terms of the D/F model, this initial understanding acted as the explanatory structure (frame) that defined what information participants needed to seek, and whether and how their previous experiences could be utilized to solve the current task. In Klein et al.'s (2007) terms, participants' initial understanding was used to develop the initial frame.
Initially, we designed the task as focused on understanding the concept of "being influential"; however, two other concepts emerged from the analysis. As Figure 8 shows, participant N6 begins Task 3 by trying to make sense of "chemistry." To operationalize an academic domain, first it is necessary to narrow it down by identifying and learning its structuring components (e.g., members, subdomains, trends, authoritative sources) (Pontis & Blandford, in press). As participants were unfamiliar with Tasks 3 and 4 domain, they first needed to understand the concept of "chemistry" in order then to make sense of the problem of influence: With task 1 [HCI domain], I was able to think HCI and then sub-theme games and then try to search more specifically, but with this task, there's just the general heading of chemistry, so it's quite difficult to know which sub-category of chemistry I should actually be looking in.-[Task 3-E3] It's such a big area. I don't know how to narrow it down and I don't . . . even looking at these journals, kind of, going, okay, so those are the impact factors; great. Who's published in those, and then I'm trying to look at that and cross-reference it with the students and not finding much, or finding one, but okay, if they each have a publication, who's the most influential?-[About Tasks  In line with previous work (Becher & Trowler, 2001;Whitley, 2000), this indicated that unraveling the organizational structure was the first step for making sense of an academic domain. By starting her search with Marie Curie, participant N6 narrowed down the chemistry domain to a member component.
A third concept emerged that participants had to make sense of which was related to Tasks 2 and 4. Indicators used for those tasks, for example, age, indicated the use of different criteria to make sense of the problem of "becoming influential." The difference between the concepts of "being influential" (identifying) and "becoming influential" (predicting) is elaborated further in the next section.
Defining influence indicators. Each participant used a set of indicators to define the meaning of influence, which were the data components of the frame acting as anchors points . Klein et al. (2007, pp. 122-123) stated that the first "one or two key data elements we experience sometimes serve as anchors for creating an understanding. These anchors elicit the initial frame, and we use that frame to search for more data elements." In the study, participants determined and used indicators to construct the explanatory structure to direct their search processes. Indicators helped them decide what keywords and search strategies were the appropriate ones to find candidates. When initial indicators were not rich enough to help participants begin a search and connect data with the frame, participants revised their initial idea of what it meant to be an influencer, and determined more indicators. Participants made sense of the problem of influence by operationalizing two different concepts: what they thought a current thought leader was (being influential) and what a future "rising star" should be (becoming influential), and by defining what characteristics they should have (indicators). They described an influential candidate as someone who has: some sort of recognition (e.g., peer recognition through citation count, prizes awarded), made contributions to knowledge or big developments, influenced and enlightened others' way of thinking or works, and has had an active presence in his/her area of expertise, produced a consistent amount of work, and planted the seed for something (e.g., started producing something) or was the founder of, for example, a course, a discipline. A future influential candidate was thought to: be of a young age, be investigating/ working on new/emerging areas as opposed to more traditional areas, be producing work with potential/ promising future application, be at an early stage in the field, be related to (e.g., working or studying with) already influential people, have a varied range of contacts and networking skills, and belong to a prestigious institution (e.g., university, Royal Society, research group, company).
The above characteristics can be grouped into five types of indicators for identifying and predicting influential candidates that participants considered for solving the tasks: 1. Candidate's professional expertise: Intrinsic characteristics of academia, from having a record of publications and citation counts to attending top conferences and belonging to a research group. 2. Candidate's domain expertise: Aspects directly related to a domain, from belonging to the domain and having done some sort of work related to it to having done seminal work or be doing something to contribute to its development. 3. Candidate's education: Universities and research centers which the candidate has attended and been awarded degrees from, and also candidate's year of graduation and age.
4. Candidate's research community: Other researchers' thoughts and opinions, and whether the community has somehow recognized candidates' work by awarding prizes or deeming them as influential, respectable, or up-and-coming. 5. Candidate's personal relationship with the participant: Whether the sensemaker knows a candidate, has worked with them, or has heard of them in class or during a conference; and whether that person has influenced their own work. Figure 3 summarizes the five indicator types described above, and indicates which specific indicators were used for each task by each of the 11 participants. All participants used four or five indicator types to identify and predict candidates. In line with Whitley's (2000) theory, candidates' professional expertise was the type of indicator most used by participants to assess their degree of influence, followed by candidates' domain expertise. Having had some sort of contact either at work or having heard of a candidate in class, and candidates' education background were considered more reliable aspects to measure influence than community recognition of their works by awards or appraisals. The last indicator type was mostly used for tasks in which participants were familiar with the domain and have had more direct contact with their peers.
A learning curve emerged throughout the four tasks. When participants arrived at the last tasks they had already constructed an understanding of influence, and they just expanded that frame with the particularities of the last tasks. For example, for Task 1, participants E2 and N6 considered someone influential when they "have done seminal work," and for Task 2, a future influential academic when they would "be doing interesting stuff in the area" or "exciting work." Then, participants E2, E3, N4, and N6 determined as an influence indicator "to belong to the HCI domain" for Task 1, and they used that same indicator for Task 3, but rephrased as "to be a chemist." Participant E3 explained how Task 1 experience helped her for addressing Task 3: I kind of had more of an idea of where to go for Task 1. With Task 3, because it's a completely different domain, I was more lost in terms of where to start, but because I had done Task 1, I tried to apply some of the same things to Task 3. So in particular, I was thinking about . . . before, I was doing influential in terms of publications, so who's published a lot. Also, with Task 1, the people identified, they've been involved in conferences and journal papers. One of the people in Task 3, there was someone that was an editor for a journal, so that was a kind of a similar strategy there as well. I think the main difficulty was because I didn't know the sub-domain, I didn't know where to specialize in chemistry.- [E3] Prediction tasks demanded more indicators than identifying tasks. Participants also determined different indicators within each type related to selecting future influential peers and not established ones. In particular, this situation occurred mainly during Task 2. For example, participants E1 and E2 paid attention to the dates of published work for solving that task, making sure that they were choosing candidates who they knew had published recently, and therefore indicated they were active researchers.
Participants combined indicators to construct their frames. The D/F model describes the construction of frames and the search for data as actions happening simultaneously (chicken/egg relationship; Moore & Hoffman, 2011). However, we observed the recognition of a frame and construction of that structure occurring first, and being necessary to start a productive search. This is shown in Figures 6  and 7 in which the first action participants performed was to recognize the problem followed by starting to construct the frame.
Connecting data with the frame through indicators. After gaining an understanding of "influence" and defining indicators, participants adopted searching strategies to find data responding to those indicators in order to connect them with the frame. For each type of task, participants sought people with a combination of 2 to 12 of the indicators shown in Figure 3. This highlighted that indicators were essential to start connecting data with the frame, as participants used them as a way to assess and measure a candidate's influence. Participants reported that the lack of a set of indicators to determine degrees of influence was one of the main reasons why they found prediction tasks (2 and 4) as "secondguessing the future" (E4).
By searching data, participants enriched and expanded the initial explanatory structure, until they found community members who could be considered influential. In other words, they were elaborating the frame, but also identifying instantiations of that frame. In the next section we discuss this concept.

Finding Instantiations: Elaborating the Frame and Identifying Candidates
The D/F model states that sensemakers elaborate the initial frame until they find a relevant explanation that fits that structure. However, we observed that because of the nature of the tasks, there was more than one possible explanation to the task frame. This is in line with the work of Russell et al. (1993): Participants identified information of interest to select an influence candidate which was a representation or example of the explanatory structure. In some cases, participants stopped pursing those instantiations and considered them final candidates after a short pause for reflection. This situation is exemplified with Chain 16: John Dalton from E2 in Figure 7. In other cases, participants decided not to preserve a found instantiation because they were not convinced enough that it was addressing all indicators, and opted to seek a new candidate (e.g., Chain 5: Louis Pasteur-N6, Figure 6). When this occurred, some participants decided to elaborate the frame further by identifying more indicators that may help them find more suitable candidates (e.g., Chain 4a and 4b: Candidate's research-N6, Figure 6), while other participants kept the frame as it initially was and used it to identify the remaining candidates (e.g., N4 and N3).
The D/F model states that data components used as anchors for one frame should not be used as such in a second frame to avoid conceptual strain. This is in line with the analysis presented here: Participants worked with only one frame across the four tasks because they worked with or recombined the same anchors throughout the sensemaking process of the four tasks.
Once participants were able to connect data and frame, they started elaborating that frame until they found a candidate (instantiation). In line with the D/F model, participants expanded the frame by seeking and inferring data. Participants performed this sensemaking activity on many occasions, from trying to define a search strategy or find a new strategy, to looking for a specific source of information. Occasionally, they filled slots from the initial frame when they discovered new relevant information, which strengthened their understanding of someone they were considering a candidate influencer. However, they discarded a piece of information when it was considered irrelevant (E2, E3), time-consuming to investigate further (N4), or when participants could not assess the relevance (E2, E3).

Questioning Instantiations: Source Origin and Credibility
As previously discussed, some participants asked themselves many questions after identifying a potential influence candidate (instantiation) and judged the plausibility of their selected candidates (instantiations); other participants questioned the candidate's expertise and magnitude of their achievements, while other participants were more concerned about the information source from which the candidate name was found. In this latter case, participants E2 and E3 questioned the plausibility, origin, and credibility of the sources. When the source of information could not be trusted, participants gauged the quality of the data. This was a frequent occurrence among more experienced study participants who sought additional sources of information as evidence to support potential candidates when they did not fully trust the initial source. This is shown in Figure 6 with "Judge plausibility" boxes.
A prior study (Pontis & Blandford, in press) found that experts relied on their background knowledge when possible to assess a candidate's record of influence contributions and thus support the influence status. This second layer of analysis shows that some participants also gauged the relevance of the information found against the influence indicators and their prior knowledge. In addition, experts who were highly engaged with the tasks connected the incoming information to previous experiences or knowledge: there is a guy called Democritus, and I'm pretty sure he had some interesting ideas about particles and things like that. Want to find out a little bit more about him to try and remind myself, but these are the foundations, I guess.-  E2 and E3 expressed frustration when they could not understand the magnitude of a candidate's discovery or contribution, because of the lack of the necessary domain expertise. In the cases of N4 and N6, they primarily judged the plausibility of frame instantiations, rather than the origin or credibility of a data point.

Reframing Instantiations and Choosing Final Candidates
The nature of the sensemaking tasks of this study did not require participants to construct a story or a narrative to make sense of the problem. This meant that participants worked with only one frame (although they made sense of three concepts), in contrast to narrative sensemaking tasks, in which sensemakers compare up to three frames (explanations) Moore & Hoffman, 2011). None of the participants reframed the initial explanatory structure (frame) and constructed a new one; instead, they expanded the same structure throughout the four tasks by adding or adapting influence indicators (anchors) (Figure 3). That is, instead of working with more than one frame, participants worked with more than one frame instantiation (candidate name).
Participants selected between three and eight possible candidates for each task, although most selected the requested three only. Participants searched for one candidate at a time, and only in a few cases did participants compare the instantiations in order to make a final decision. We observed this mainly for Tasks 3 and 4 when participants preselected a higher numbers of candidates. Lack of confidence in making final decisions is a consequence of participants lacking the necessary knowledge to assess the relevance of incoming information (E2 and E3) (Pontis & Blandford, in press). In the cases of N4 and N6, they pursued candidates they considered could be influential until they discovered some information that made them change their mind and discard their current candidate.

A Data/Frame Model for Influence Structure Tasks
In the above sections we discussed the sensemaking cognitive activities that emerged from our study. Now we present those activities using the D/F model introduced in Figure 1 as a structure, but building on it to represent structure sensemaking. As Figure 9 illustrates, we introduce two modifications to the original model: consideration of frame instantiations and expansion of the number of cycles from three to five.
The first modification to the model is the focus of sensemaking activities and cycles on frame instantiations (in this case, influencer candidates) rather than on frames. As explained above, the fact that participants worked with only one frame may indicate a significant difference from narrative sensemaking in which sensemakers tend to contemplate more than one frame. The current D/F model does not provide a way to represent the elaboration of instantiations, which may be a distinctive characteristic of structure sensemaking. Therefore, we rephrased the activity headings to stress this point by including the word "instantiation" (Figure 9).
Then, to the initial cycles of preservation, elaboration, and reframing, we added the cycles of analysis and decisionmaking. To avoid visual clutter in Figure 10, we excluded arrow edges in the circles representing these two cycles, but both are influenced by activities from other cycles, and explained in detail as follows. The analysis cycle involves the discovery of information responding to influence indicators and the understanding of the problem situation ( Figure 10). This cycle is closely related to seeking, inferring, adding, and discarding activities, but the focus is on how sensemakers interact with and analyze the incoming information (data points) in terms of its correspondence with the initial explanatory structure and anchors (indicators). In this cycle the sensemaker identifies the appropriate nuggets of information that are directly related to the particular task and those which are less connected but help construct the bigger picture (Qu & Furnas, 2008).
The decision-making cycle involves activities related to making some sort of frame instantiation choice, preselecting frame instantiations, and making a final judgment about a frame instantiation ( Figure 11). This last activity also indicates the end of the sensemaking journey and, in some cases, the completion of a task, while the previous activities indicate progress in the process. The D/F model represents a process with "no box that says 'start here,' " as Moore and Hoffman (2011) pointed out, nor box that says "end here." However, we found that both "boxes" were necessary to accurately represent the full sensemaking journey of structure tasks. As Figures 6 and 7 show, the recognition of the situation and construction of an initial explanatory structure constituted the starting point, and the end point was clearly indicated by all participants either when they found all three requested candidates or when they decided to abandon a task.

Discussion
The tasks used in this study dealt with the problem of influence in two particular domains for which various answers were appropriate. Participants made sense of a problem which presented a high degree of uncertainty for many of them, by understanding who and what were more influential than others. Using indicators as anchors, participants assessed knowledge production, and validated the relevance of contributions, some of which were completely foreign to them. By doing this, they were constructing an understanding of relationships among people, and unraveling the organizational structure of the domains, which allowed them to identify the requested influential candidates for most of the tasks.

Understanding Structure Sensemaking
The sensemaking process of structure tasks is not linear. Participants made sense of the tasks through an iterative process of elaborating one frame, and questioning incoming information and instantiations (Figures 6 and 7). Building on data points was sometimes impossible or extremely hard, resulting in journeys with many turning points. In most cases, to identify an instantiation (candidate), participants had to go back to the beginning of the process (connect data with frame) and start a new search (seek and infer data), until they found a piece of information or source (data point) that was considered relevant.
Novice and expert sensemaking for this type of task presented both differences and commonalities. Experts went through more elaborate sensemaking journeys, in some cases performing a higher number of activities and turning points, which made them analyze more data points. Experts judged the quality and relevance of both incoming information and preselected instantiations. On the other hand, most novices preselected a higher number of instantiations that were later compared in order to make the final decisions. Most tasks solved by novices were completed with surface exploration, which translated into fewer returns to previous states (turning points), and in many cases, the absence of questioning activities. Both novices and experts based their decisions on the same influence indicator types (Figure 3), determined similar indicators, and considered cues (e.g., names) that repeatedly came up in search results. In general, both tended to use a similar number of indicators across the four tasks, although a few novices made decisions based on simpler frames constructed with fewer indicators.
The structure sensemaking tasks demanded different cognitive activities and levels of expertise. Identification Tasks 1 and 3 were less demanding in terms of professional expertise than predicting Tasks 2 and 4. Expanding that finding, we also observed that Tasks 2 and 4 demanded more cognitive activities, which resulted in more complex sensemaking journeys. In addition, these tasks demanded a higher number of indicators, in some cases, different or complementary to the ones for Tasks 1 and 3 (Figure 3).

Applying the D/F Model to Structure Sensemaking
The identification and selection of frames is a conscious process, which depends on the data and information that are available, and on the individual's aims, repertoire of frames, and attitude (e.g., commitment to a task). We described how participants constructed an initial explanatory structure (frame) based on their initial understanding of influence and combining indicators. Then we discussed how that structure evolved and, in some cases, expanded as new information was found and participants' learning about the concept increased. The initial frame was actively enriched as participants identified new influence indicators (anchors). In some cases, participants experienced a learning curve from the first task to the last one, indicated by the fact that they used previously identified anchors as the starting point for later tasks.
In some cases, it was hard to clearly decompose participants' sensemaking processes into the D/F model activities, because frequently more than two activities occurred simultaneously. We found activities in line with the model and others not originally contemplated: judging and gauging data relevance, analyzing data, and making decisions. By presenting sensemaking activities as chains, we stressed the need for considering initial and end points in the process. Building an understanding which leads to the construction of an explanatory structure emerged as an essential initial activity or starting point for making sense of a structure task. Participants used that understanding to select from their memories or find the most suitable frame to direct their sensemaking processes and start connecting incoming data.
In Klein et al.'s (2007) model, sensemaking activities do not have a specific order, which indicates that the process of sensemaking can begin with any of the activities. Nevertheless, the fact that participants could not start the process until they managed to construct an explanatory structure highlights that a certain order may be implied for some types of sensemaking activities. Similarly, Moore and Hoffman (2011) indicated the need of an "exit point" in the model, which in our case is when a task is completed or abandoned. We highlight this need by adding a cycle referring to making choices and final decisions ( Figure 11). In addition, we highlight the need for a frame activity to code participants' steps related to making sense of and understanding the task itself, referred to as "recognize a situation."

Conclusions
This study explored structure sensemaking by applying the D/F model to the analysis of influence tasks. As the study reported here was conducted in experimental settings with tasks specifically designed to explore structure sensemaking, it is not possible to assess how the findings generalize to other structure sensemaking tasks. Although the tasks investigated in this study are common in various academic contexts, we cannot be sure how widely the findings generalize.
Notwithstanding, as explained by Whitley (2000), all scientific fields deal with a certain degree of task uncertainty as exemplified by the ones in this study. In particular, this study explored strategic task uncertainty; that is, the uncertainty of the hierarchy/internal structure within a scientific field. In general, broadening the understanding of scientific field structures contributes to the understanding of the social organization of the sciences; and, in particular, the problem of influence sheds light on other intrinsic aspect of sciences: how scientists validate knowledge, and how the different components of a scientific field related to and depend on each other. To solve the problem of influence (within a familiar and an unfamiliar domain) participants needed to unravel the hierarchy of the fields under investigation, and determine criteria to assess the validity of the information they encountered (e.g., whose researchers were the most prestigious, which trends were more likely to be up-andcoming, what journals were the most important). Knowing these criteria and the steps researchers go through to identify the hierarchy within a field may decrease the level of social or strategic task uncertainty in scientific fields.
The D/F model provided a framework to make the process of learning the internal structure of a scientific field more tangible and understandable, in that key activities, steps, and components involved in that process were identified. The development of the extended model provided a possible framework for structure sensemaking in the modern sciences, and the identification of the necessary repertoire of search skills for dealing with strategic task uncertainty. Further studies are needed to validate the extended model; for example, applying the model to a broader range of structure sensemaking tasks and with a sample of researchers from more diverse backgrounds (e.g., art and design, social sciences, natural sciences). Nevertheless, we found that, as well as being suitable for representing narrative sensemaking, the D/F model gives a reasonably complete account of structure sensemaking, but we also identified further activities and cycles that are needed to deliver a fuller, more complete representation of structure sensemaking.