Understanding “influence:” an exploratory study of academics' processes of knowledge construction through iterative and interactive information seeking

The motivation for this study was to better understand academics’ searching and sensemaking processes when solving exploratory tasks for which they lack pre‐existing frames. We focus on “influence” tasks because, although they appear to be unfamiliar, they arise in much academic discourse, at least tacitly. We report the processes of academics at different levels of seniority when completing exploratory search tasks that involved identifying influential members of their academic community and “rising stars,” and similarly for an unfamiliar academic community. 11 think‐aloud sessions followed by semi‐structured interviews were conducted to investigate the roles of specific and general domain expertise in shaping information seeking and knowledge construction. Academics defined and completed the tasks through an iterative and interactive process of seeking and sensemaking, during which they constructed an understanding of their communities and determined qualities of “being influential”. The Data/Frame Theory of Sensemaking was used to provide sensitising theoretical constructs. The study shows that both external and internal knowledge resources are essential to define a starting point or frame, make and support decisions, and experience satisfaction. Ill‐defined or non‐existent initial frames may cause unsubstantial or arbitrary decisions, and feelings of uncertainty and lack of confidence.


Introduction
The way people seek information to make sense of a situation is a complex process that involves many factors (Baldwin & Rice, 1997;Cooke, 1999;Diriye, 2011;Dorst, 2004;Dreyfus, 2004;Gwizdka & Lopatovska, 2009;Hsieh-Yee, 1993;Kuhlthau, 1991Kuhlthau, , 1999Nahl & Tenopir, 1996;Vakkari & Hakala, 2000). People have different sen-semaking and searching strategies to build understanding and make decisions. When tasks are not precisely defined or they sit within an unfamiliar domain, the identification of the required information can be challenging because task doers first need to understand and define the problem and may not be able to rely on their existing knowledge. To investigate undefined situations, we designed exploratory search tasks that involved the concept of "being influential." Academics frequently deal with that type of task (e.g., recruitment of new researchers, distribution of tasks in a project, referencing authors, seeking prominent discoveries), but it is unlikely that they have explicitly searched for that type of information.
In this study, we explored the relationship between expertise and search behavior, focusing only on domain expertise (Wildemuth, 2004) and professional expertise but excluding search expertise. Although the effects of domain expertise and search expertise on information seeking have been previously studied (Jenkins, Corritore, & Wiedenbeck, 2003), the effect of professional expertise (regardless of subject specialism) has not. However, we are aware that there is a correlation between these three dimensions of expertise: All typically correlate with maturity. We investigated the role of domain and professional expertise in information seeking and sensemaking for exploratory search tasks. Expertnovice differences may lead to varied ways of seeking information, building an understanding of a community, and influencing the thinking process and the rationale for identifying and predicting thought leaders. The study explored how academics constructed understanding as well as made sense of and defined the concept of "being influential" to solve an assigned set of exploratory search tasks (Diriye, 2011) aimed at identifying current and future peers with that characteristic. Understanding the journey that academics took to find the requested information was the aim of the study, "rather than the information per se" (Diriye, 2011, p. 132 -″per se″ is in italicts in the original document). The exploratory study reported here provides insights into how academics build understanding, use prior knowledge, interact, make sense of familiar and unfamiliar domains, and what problem-solving strategies they adopt in poorly defined situations. The data/frame (D/F) model of sensemaking (Klein, Phillips, Rall, & Peluso, 2007) was used as the framework of analysis, along with findings built on previous studies (Diriye, 2011;Kuhlthau, 1991Kuhlthau, , 1999Vakkari, 2002;Warwick, Rimmer, Blandford, Gow, & Buchanan, 2009;Wildemuth, 2004) and added insights of academics' information seeking and sensemaking processes. The study also offered new insights about the role of professional and domain expertise in the construction of initial frames of an undefined problem situation and in distinguishing relevant from irrelevant frames to determine adequate information (i.e., identifying current and future leaders).
We first present a review of related work that informed the design of our study and data analysis. Next, we present our definitions of expertise, novices, and experts and the exploratory tasks, along with the analytic rationale and theoretical framework used to draw conclusions and indicate how they relate to established literature. We then report on the study design, the main findings, and our conclusions.

Related Work
To investigate people's sensemaking and searching strategies for exploratory tasks for which they have different levels of existing (domain and professional) expertise, we designed tasks that involved identifying influential members and "rising stars" in both familiar and unfamiliar academic communities.

Academic Community Structure
Prior studies (Fagerberg & Verspagen, 2009;Fry & Talja, 2007;Whitley, 2000) have defined the structure of a scientific field as including well-defined domain boundaries, research objects, problems, topics and techniques, results and discoveries, resources, and a communication system (e.g., conferences, journals, publications). Supporting Whitley's views, Fagerberg and Verspagen (2009, p. 219) highlighted the latter, as well as having "common standards (for what is good work and what is not) and a merit-based reward system (that promotes the good work)" as key components of an academic community. Similarly, Faisal (2008) argued that an academic community involves members (e.g., authors, peers, students), their interests, and the literature that they produce (e.g., publications, books, papers).
The production of academic literature, including books, e-books, papers, and conference presentations, is increasing daily and becoming widely accessible through the Internet and online library access, among other channels (Martin & Quan-Haase, 2013). Every day, academics interact with literature domains to make sense of problem situations and solve a wide range of activities. Academics gain understanding by becoming familiar with their community components (e.g., members, interests, literature, communication systems) and through previous knowledge, informationseeking activities, reading, and solving problems (Faisal, 2008). They probably have a tacit understanding of who the current thought leaders and influential peers are, and may be expected to predict who would become one in the future (e.g., when involved in recruitment of new academics). However, most academics are unlikely to have done an explicit search to identify influential members of the community. Understanding the sensemaking process undertaken to discern influence is the focus of this article.

Exploratory Search Tasks
Literature has revealed that regardless of what channels, tools, or systems are used, information seeking is frequent activity in academia that reflects different needs based on purpose and scope (Cottrell & Eisenberg, 2001;Fagerberg & Verspagen, 2009;Tenopir et al., 2011) and is seen by academics and researchers as a daily task activity (Marchionini, 2006). Kuhlthau (1991, p. 361) described the informationseeking process as a "constructive activity of finding meaning from information in order to extend his or her initial state of knowledge on a particular problem or topic." Informationseeking tasks have been widely studied, and definitions vary (Belkin, Marchetti, & Cool, 1993;Diriye, 2011;Kim, 2008Kim, , 2009Marchionini, 2006;Vakkari, 1999). They range from basic fact-finding to "guide short-term actions" (e.g., What time is the next train to Bristol?) to more complex ones involving "related concepts that help us understand phenomena or execute complex activities" or gaining knowledge that may evolve as expertise develops (Marchionini, 2006, p. 42). Diriye (2011) described two broad types of search tasks: exploratory and known-item search tasks. However, all information-seeking tasks are considered as exploratory tasks to some extent, involving elements from known-item search tasks (Diriye, 2011). Known-item search tasks are closed tasks with a definite answer (Kim, 2008) and the location of a specific and well-defined document or fact (Cottrell & Eisenberg, 2001). Previous studies have described exploratory tasks as "ill-defined" (Diriye, 2011, p. 28), "vaguely structured" (Kim, 2008, p. 174), and presenting open-ended problems aimed at generating some sort of data set or informing actions as the outcome of the process. This indicates a need for building an initial understanding and making sense of the problem to identify the information required to construct appropriate solutions.
Required information and answers are sought through a wide range of channels and sources (Byström & Järvelin, 1995) and are determined by searchers' perceptions (Vakkari, 1999). Although academics have used literature as a principal resource for understanding a community, there also are other possible online and personal channels and sources (Fry & Talja, 2007). To locate pieces of information, academics use their prior knowledge, search online for electronic sources (e.g., research centers' websites, digital libraries, scholarly homepages), or pursue more traditional channels (e.g., libraries, conferences, and peer discussions) (Fry & Talja, 2007;Kumar & Kumar, 2008;Martin & Quan-Haase, 2013). Information seeking is only one component of the sensemaking process through which academics gain an understanding of their community.
In the following sections, we introduce prior work related to sensemaking, information seeking, and the role of expertise in knowledge construction.

The Sensemaking Process in a Nutshell
The sensemaking process has been widely studied (e.g., Chi & Card, 1999;Cottrell & Eisenberg, 2001;Dervin, 1999;Klein, Moon, & Hoffman, 2006;Klein et al., 2007;Pirolli & Card, 2005), and many models have been defined. Dervin (1999) defined sensemaking as the process of bridging assumptions of incomplete understanding about reality, in which individuals combine their own understanding of reality with the understanding obtained (sense made) by others. Other authors have defined sensemaking as the process of understanding a situation or problem by searching for new information (Kuhlthau, 1991) and discovering connections among data (e.g., people, places, events) (Klein et al., 2006(Klein et al., , 2007, which occur when there is a "deliberate effort to understand events" (Klein et al., 2007, p. 114). Throughout this process, an individual filters relevant from irrelevant information, gives meaning to experiences, moves from data to an interpretation, and constructs broader understanding of a specific situation. This "nonlinear, dynamic, holistic, and flowing" process (Foster, 2004, p. 235) involves pattern creation and pattern discovery. When trying to make sense of a situation, people follow various strategies to expand their initial knowledge or gain new knowledge, draw inferences, and make predictions from data (Klein et al., 2006). Three interwoven activities are involved in the sensemaking process: "physical, actual actions taken; affective, feelings experienced; and cognitive, thoughts concerning both process and content" (Kuhlthau, 1991, p. 362).
In this study, we used the D/F theory (Klein et al., 2006(Klein et al., , 2007 as a framework of analysis. The theory was introduced by Klein (1997) and emerged from his work on the naturalistic decision-making framework for the information dominance domain. This theory introduces the concept of creating internal, cognitive representations when people are making sense of a situation. Klein et al. (2007) argued that elements are explained when they are fitted into a structure that links them to other elements. The term frame is used to refer to that explanatory structure that defines entities and describes their relationship with other entities. A frame is the initial "perspective, viewpoint, or framework" that people have when they are trying to gain understanding of a problem situation and can be expressed as "stories, maps, organizational diagrams, or scripts" (Klein et al., 2006, p. 88 (for both quotes)). "Frames define what count as data" and change as data are collected. People construct understanding when they distinguish the important frames from the irrelevant ones, and determine which are useful (Klein et al., 2006, p. 90).
Information seeking in sensemaking. The majority of current sensemaking models refer to two intertwined cycles. The former deals with seeking and gathering data or elaborating a frame (Klein et al., 2006); the latter deals with making sense and understanding or "reframing" those data. Several studies have been conducted to investigate those cycles and gather insights to understand how sensemakerssuch as analysts (Chin, Kuchar, & Wolf, 2009;Kang, Gorg, & Stasko, 2011), lawyers (Attfield & Blandford, 2011), and academics (Faisal, 2008;Foster, 2004)-transform, organize, store, and use information from different sources to elaborate a frame, then question, evaluate, and reconsider that initial frame based on new data, and finally find a solution or solve a problem. Other studies have mostly analyzed the first cycle and focused on the information-seeking and -retrieval processes, search behaviors, tactics, and strategies (Belkin et al., 1993;Kim, 2008Kim, , 2009Kuhlthau, 1991;Vakkari, 1999). They typically attend less to the use of information and the process of building knowledge.
Few prior studies have focused on the analysis of professional expertise and knowledge construction, but some studies have explored this dimension of expertise in the context of information seeking and retrieval. Vakkari (2002) found that domain expertise determined searchers' ability to distinguish relevant from irrelevant information obtained from search. In addition, his work revealed that professional expertise determines searchers' relevance criteria, which help them identify the hierarchical structure of the information and relevant connections between data components. These findings have indicated that the more experienced task doers are in accessing various information sources (search expertise) and the more specific knowledge that they have about the problem subject area (domain expertise), the more expert they will be considered to be. Another topic of past research has been the relationship between search behaviors and expertise. Hsieh-Yee (1993) investigated the use of search tactics in online search, controlling both search and domain expertise. She found a correlation between subject or domain expertise and the type of search tactics that searchers use based on the searchers' experiences. Expanding on this work, Wildemuth (2004) explored the role of searchers' domain knowledge on their formulation of search tactics. Her work has explained how searchers' strategies are composed of search moves, terms, and concepts, and how those evolved when new searchers gained new knowledge. She found that searchers' domain expertise influences search behaviors and the information-seeking process. Vakkari (1999Vakkari ( , 2002 examined information retrieval actions and systems and the role of search and subject knowledge on the information-seeking process. He argued that "both subject and search knowledge contribute in the formation of the search outcome" (Vakkari, 2002, p. 110), and stressed that individuals with expert subject (domain) knowledge have developed the skills to define a more precise starting point to solve a task.
Most of these studies have explored the first two dimensions of expertise-search and domain, not explicitly considering the third one, professional expertise. Our study addresses this gap by investigating the influences of different levels of professional and domain expertise in the problem-solving process of exploratory search tasks; we do not assess or control for search expertise. In addition, we investigated how expert professionals behave when they have to solve an undefined task in which they cannot rely on their domain expertise, and whether novices can use their background knowledge to make sense of the tasks.
Despite both experts and novices going through the same sensemaking process and employing "the same types of logical and abductive inferencing" (Klein et al., 2007, p. 126), research has indicated that the former have a more diverse and robust repertoire of strategies while the latter tend to employ a more basic approach. Similarly, Chi et al. (1981, p. 122) explained that "expert-novice differences may be related to poorly formed, qualitatively different, or nonexistent categories in the novice representation." Prior studies in this subject (Chi et al., 1981;Cooke, 1999) have found that experts' problem-solving rationale tends to be based on principles while less experienced individuals tend to look at problems from a more literal point of view. As a result, experts "see the underlying similarities in a great number of problems, whereas the novices see a variety of problems that they consider to be dissimilar because the surface features are different" (Chi et al., 1981, p. 130). Expanding on this finding, Warwick et al. (2009Warwick et al. ( , p. 2413 reported that novices tend to "use what expertise they have to support the retention of familiar strategies and limit both the effort and scope of information seeking." Our study expands previous findings by adding another layer of analysis to the role of expertise in the sensemaking process. Sternberg (1994), Kuhlthau (1998), Klein et al. (2006), Faisal (2008, and Warwick et al. (2009) found that the years of experience and level of expertise in a particular domain can have a considerable effect on the way a task is completed and on task doers' behavior. Cooke (1999) associated expertise with the level of pattern recognition and the way domain knowledge is organized in memory. Nonetheless, internal factors are tightly interwoven, making it difficult to unpack them, as the study reported here illustrates. Therefore, as noted in prior studies, there is a correlation between the three dimensions of expertise: search, domain, and professional. Many studies have focused on the role of search expertise by comparing the search strategies of novice and experienced users of a specific medium (e.g., web search: Hsieh-Yee, 1993;Jenkins et al., 2003) or a specific databases (e.g., Magazine ASAP: Nahl & Tenopir, 1996). In contrast, we studied the role of domain expertise in a freeform search, not restricted to a particular database. Some of the previously mentioned studies measured participants' domain knowledge and task success based on right or wrong answers, or solutions presented; however, our focus is on gaining understanding of academics' sensemaking journeys, without judging the quality of their responses.

Study
This study aimed to gain insights on how academics with different levels of seniority (master's students and established academics) interact with familiar and unfamiliar literature domains to identify current and future thought leaders through exploratory search tasks. This study has been designed to answer the following questions: RQ1: How do academics build an understanding of their community, in terms of identifying • community leaders and those who have significant influence within a community or subcommunity • "rising stars" who are likely to be the next generation of leaders RQ2: What are the processes academics engage in while making sense of a community • that they are already intimately familiar with? • that they are outsiders to? RQ3: What are the differences in terms of expertise (a) of the community (domain expertise) and (b) expertise as academics (professional expertise)?
• Does our study support the hypothesis that experts focus more on underlying principles while novices focus more on the surface characteristics of a problem situation?
This study examined how academics (a) gain an understanding of both their own and external communities, (b) interact with familiar and unfamiliar information sources, and (c) solve exploratory search tasks by identifying current and future community leaders. We specifically examined academics' information-seeking strategies, knowledgeconstruction process, and sensemaking journeys.

Study Design
Eleven think-aloud sessions (Ericsson & Simon, 1984) followed by debriefing semistructured interviews were conducted with academics with different levels of seniority from the human-computer interaction (HCI) domain in labs at University College London (UCL). Each think-aloud session lasted from 50 min to 1 1 ∕2 hr. We designed nonintrusive prompts as a strategy to encourage participants to openly share thoughts while solving the tasks. When participants spent more than 10 s in silence, we showed them a "KEEP TALKING" sign to remind them to verbalize thoughts without interfering too much with their thinking process (Charters, 2003). After we explained the study and each of the tasks, we sat behind, not across from, the participants to minimize bias (Charters, 2003). While they were solving the tasks and verbalizing their thoughts, we took notes on their sensemaking process, information-seeking strategies that they used, and the terms searched. To complete the tasks, participants were provided with both traditional tools (e.g., paper, pencil, Post-it notes, etc.) and a computer with an Internet connection and access to basic computer search and text tools (i.e., Google, Microsoft Word, and PowerPoint, etc.). Participants were not requested to use specific tools during the experiment. Think-aloud sessions and interviews were audio-recorded, and screen-capture software was used to record participants' information-seeking actions and interactions with the computer.
Study tasks. At the beginning of each session to help participants get used to verbalizing thoughts while solving a problem, we gave everyone a warm-up task (Ericsson & Simon, 1984, pp. 376-378), which was to solve an anagram. Participants were asked to form new words by rearranging the letters of given words and to think aloud while solving the challenge. After the warm-up task was completed, participants were asked to complete four exploratory tasks (Table 1), which were realistic, but challenging, and similar to ones that they deal with every day at work or during their studies. Our working definition of exploratory search tasks was based on the work of Diriye (2011) (discussed earlier): An information search problem that is motivated by a poorlydefined, or vague information need, and exacerbated by a poor understanding of the domain terminology and information space structure. It seeks to foster learning and understanding to inform an action, or produce some knowledge product. (p. 123) In this study, exploratory search tasks were designed according to external (search objective, search activities, conceptual complexity, procedural complexity) and internal (domain, professional, and search expertise) factors summarized in Table 2. We explored and controlled the role of domain and professional expertise in information seeking and sensemaking, but we did not investigate search expertise.
The tasks required academics to explore familiar and unfamiliar domains, make sense of undefined problems, and construct an understanding of the concept of "influence." To complete the tasks, participants were asked to identify three current and three future influential academics from the HCI and chemistry domains. No definitions of the terms "being influential" or "becoming influential" were provided to avoid bias, but we clearly pointed out to all participants at the beginning of the study that there were "no right or wrong answers." Therefore, tasks were considered undefined.
Tasks were provided to participants in the same order based on the D/F, in which first making sense of the broader picture (exploring/elaborating) is needed before searching for more specific types of information (predicting/inferring) (Klein et al., 2007). Familiar domain tasks were given first to help participants concentrate on the search for the requested information, allow them to get used to verbalizing the thinking process, and feel at ease with the experimental setting of the task. We gave participants one exploratory search task at the time and waited until it was completed before giving the next one to allow them to fully focus on each task. However, some participants moved forward to the next task when they were feeling stuck with the current one, and, in some cases, completed it after the last task. All participants had the opportunity to revise their responses at the end of the session. Semistructured debriefing interviews. After the tasks were completed, participants were debriefed in a semistructured interview that investigated in more depth their experience during the session (Charters, 2003). We used the observations noted during the session as the structure for the debriefing interview. Participants were asked to define what "being influential" meant for them, which criteria or indicators they used to determine the influential names, and the reasons for choosing certain authors while disregarding others. Then, we asked which information-seeking strategies they used in each task, and which task they described as the most difficult and why (the interview guide is provided in the Appendix). These questions allowed us to gain more details on our previous observations and clarify participants' thoughts about ambiguous parts, pauses, and other interesting aspects of their problem-solving process.

Participants
For the purpose of this study, the operational definition of expertise considered participants' domain expertise as the knowledge that they possessed in the subject domains under investigation (HCI and chemistry) (Wildemuth, 2004) and professional expertise as their research experience (regardless of their search expertise) based on the years that they have been in academia after completing their undergraduate degree (Dorst, 2004;Kuhlthau, 1998). We defined a novice as someone just starting to get familiar with the HCI domain and who had spent less than 5 years in academia; that is, had "done little independent [problem-solving] in an academic context" (Warwick et al., 2009(Warwick et al., , pp. 2403(Warwick et al., -2404. We considered experts as those who had been actively working in the HCI domain and who had been working in academia for more than 5 years. Therefore, the more experienced participants were in solving academic problems and the more knowledge they had about the problem subject matter, the more expert they were considered to be. In this study, "double experts" (Warwick et al., 2009(Warwick et al., , p. 2403 were defined as participants having both strong domain knowledge and research experience while "double novices" were those with little experience in. Participants were recruited following network sampling (Sirken, 1998) by publicly announcing the study to the junior research population from UCL meeting the eligibility criteria described earlier (i.e., master of science in HCI) and by sending an open call to UCL Interaction Centre members of staff to recruit experienced researchers. Both groups of participants were recruited as domain insiders (in HCI) and domain outsiders for the second domain of study (chemistry).
The 6 participants recruited as novice researchers (firstyear master of science students) self-reported to have between 1 and 5 years of research experience and considered themselves as beginners in the HCI domain. Only 1 participant of this group indicated having more than 10 years of research experience, but exclusively related to the social sciences. The other 5 participants were recruited as experienced researchers; each reported having more than 10 years of experience in different areas of HCI and actively working as academics for more than 5 years. Therefore, expert academics were all domain experts, in that they were academics from HCI with more than 5 years of experience, but they were not subject matter experts for chemistry, the second domain investigated in this study. All novice academics had some degree of familiarity with the HCI domain, but did not describe themselves as experts, and 1 of them had more than 5 years of research experience in a different domain. Table 3 summarizes participants' demographic information.
Participants' demographic information was anonymized and coded with a combination of letters and numbers. A letter indicates participants' level of experience-(E) for expert and (N) for novice-while a number is used to indicate the order in which participants took part in the study within each group. Throughout this article, each time we  (Diriye, 2011). Our study focused on investigating the role of domain and professional expertise, but we did not control participants' information-seeking expertise.

External factors Search objective
Create a knowledge product or shape an action through searching, browsing, learning, and investigation. "ill-defined," "vaguely structured," and open-ended problems Search activities Higher level search activities such as analysis, comparison, comprehension, and evaluation as well as more undirected search behaviors such as exploratory browsing Conceptual complexity Task uncertainty, a priori determinability, or how much of the task's requirements, process, and outcomes can be determined beforehand Procedural  refer to a specific participant's experience group or excerpt, we abbreviate that information using acronyms. For example, E1 denotes the first (1) participant of the expert group (E), and N2 refers to the second (2) participant of the novice group (N).

Data Analysis
We collected qualitative data sets from think-aloud tasks, observation notes, and debriefing interviews. Each data set was transcribed verbatim and analyzed using thematic analysis (Braun & Clarke, 2006). We modified the interview guide based on what had happened in that participant's think-aloud session, and data gathered from the interviews helped validate our interpretations when analyzing the verbal protocols and develop an initial set of codes to start a top-down analysis.
Initially, data sets were manually coded using 12 categories (search key terms, sensemaking strategy, associations/ organizations/universities, other supporting sources, citations, contributions, verification, inferred thought, new trends, soft information, time, and credibility). These categories were refined and grouped into fewer categories, and new ones not contemplated at the beginning also emerged (e.g., the concept of influence indicators). First, we looked at how each participant constructed understanding of the communities, connected ideas, and inferred thoughts and the search sources. We found that participants went through similar moments while trying to determine current leaders and predict future ones. We considered moments with similar objectives and actions (e.g., define influence, find one name, filter a list, select one name) a phase. Klein et al.'s (2006) model was used as a framework to verify and define each phase and to delineate participants' sensemaking journey. We identified the following phases: define, search and select, filter and determine, and verify and decide.
Using the phases as main categories, we then looked for specific characteristics in each phase and how the phases were structured and connected to each other. Key components of the sensemaking journey (narrowing down the domain, having a starting point) emerged from the data sets as well, indicating nonlinear processes (turning points, cycles of verification, uncertainty). We judged the linearity of the process on the basis of the number of turning points. We defined these points as the number of times that participants went back to a previous state and repeated actions (e.g., start a new search from scratch or select new set of search keywords).
Then, we looked for ways in which participants dealt with the concept of influence and constructed understanding by determining influence indicators or criteria. An indicator was defined as a characteristic that an influential person should have or something that they should have accomplished.
We analyzed the different channels participants used to find, filter, identify, and decide names. We judged this distinction on the basis of where participants found the information. When participants found relevant information drawing on previous encounters or experiences, we judged this information as coming from an internal channel (Byström & Järvelin, 1995). When participants found useful information as the result of a search using the computer, we described this information as coming from an external channel. Information from online searches found through external channels (e.g., online magazines, conference websites, Nobel Prize lists) was defined as hard information or evidence, and information from background knowledge accessed through internal channels was defined as soft information or evidence. When either soft or hard information was used to verify ideas, hunches, or names found on the Internet, we considered that information as supporting evidence. Supporting evidence constituted the reasons for making a final decision, and a decision was considered final when an influential academic was identified.
We also analyzed the participants' perceptions of task complexity. We used subjective parameters based on task doers' assessment (Byström & Järvelin, 1995;Liu, Liu, Yuan, & Belkin, 2011) because we were more interested in the information-seeking process than in the results. This means that to measure task complexity, we used participants' perceptions of task difficulty reported at various stages of the study: at the beginning of the task (pretask) and at the end, once the task was completed (posttask). Participants' comments during task performance were later expanded during the debriefing interview, developing a good understanding of how they perceived the complexity of all tasks.
Finally, we conducted a comparative analysis of processes, behaviors, experiences, and perceptions between novice and expert participants, and among tasks. We first compared expert and novice participants' insights and experiences, then compared those experiences between domain insider tasks (1 and 2) and outsider tasks (3 and 4). Finally, we compared participants' journeys in identifying tasks (1 and 3) with those in predicting tasks (2 and 4).

Constructing Knowledge of Academic Communities
Participants pointed out the need to narrow down the domain under investigation to begin making sense of the task. Expanding the work by Faisal (2008), various components that are part of an academic community also emerged from this study: members, trends, discoveries, literature, and authoritative sources. Figure 1 lists the structuring components of an academic domain indicated by the participants.
We found that when participants' level of domain expertise was high, they had the appropriate knowledge to narrow down a domain. This helped them distinguish specialized areas or subdomains and, therefore, made a domain tractable to determine suitable answers for the tasks. When domain expertise was low, the domain under investigation became extremely broad, with "too many researchers" (N5) to identify the most influential ones. This situation was most frequently observed among both experts and novices when working on Tasks 3 and 4 with the chemistry domain. For those same tasks, experts experienced difficulty identifying relevant subdomain components such as main lines of investigation, important research centers, or what discovery could be considered a breakthrough. To some extent, this situation made experts behave as novices, as they could not rely on their domain expertise. As one participant noted, "The way I would approach that would be to try to narrow the question down so it's much more precise. But, again, without knowing anything about the domain, it's really difficult to do" [Task 3-E4].
Professional expertise was essential to understand "how [components] might relate to one another" (E1), as this type of expertise provided participants with relevant criteria to evaluate the information found through searches. Similarly, Vakkari (2002) referred to this knowledge as the appropriate judgment to distinguish relevant from irrelevant terms, or in this case, influential community members. When professional expertise was poor, novices lacked the necessary criteria to select one name over another. All found names were perceived as equally important, and novices were not able to identify a hierarchical structure or determine which name could be more influential than the other: "It's not difficult to identify some famous names, but just there are too many names and too many lists of names. I just don't know how to tell that one is more influential than another" [Task 3-N5].
In other words, when participants had both high domain and high professional expertise, they were familiar with the discipline subdomain components and the academic structure, which let them understand the connections among those components. This knowledge facilitated the identification and prediction of influence indicators, and eventually of thought leaders. This was the case for experts in Tasks 1 and 2; however, all participants found Tasks 3 and 4 more challenging, as none of them had any domain expertise and experts could not fully rely on their professional expertise.

Different Starting Points to Understanding Influence
The first step in the participants' sensemaking process was to frame and understand the problem. To accomplish that, participants constructed an understanding of what "influence" meant by defining the concept of influence and determining influence indicators. Indicators helped participants create an understanding and construct an initial starting point. Moreover, in most cases, indicators helped determine the strategy used to find the required three influencial candidates.
Some participants did not verbalize the need to find influence criteria (indicators) to guide and direct the search, and this could be described as an inner search, mainly conducted in their heads and revealed at the end of the study during the interview. In other cases, the definition of influence indicators was explicit, as participants constructed understanding in parallel while determining the search strategy and starting online searches. We discuss these different behaviors in the following sections.
Defining the concept of influence was considered the initial phase of the sensemaking process for both novices and experts. Once an indicator or set of indicators was defined, participants looked for cues or anchor points (Klein et al., 2006) in the data to initiate the search of candidates, including keywords, names, specific websites, or resources. Because influence indicators were determined mostly in explicit ways before starting a search, there was no explicit computer interaction at the beginning of the sensemaking process in many cases.
Sensemaking behaviors. We observed three types of sensemaking behaviors at the beginning of the process. In most cases, the behaviors changed across tasks; therefore, some participants belong to more than one of these groups: • Participants who started straightaway to seek information using online searches without explicitly defined influence; • Participants who, first, explicitly defined what "being influential" meant for them and then started a search; and Some participants exhibiting the first type of behavior started a blind information search without "really thinking" [N1] or having in mind what they were looking for or having something to guide the search. Some of them did not explicitly construct an initial understanding of influence, and thus appeared to not have elaborated a thoughtful way to get into the domain. We found this behavior mostly among novices, but also among experts in Tasks  Conversely, some other participants who started online searches with no explicit definition of influence indicators did not report struggles during the process. They seemed to have a predefined idea of who could be influential and, during the interview, reported that they drew upon some prior influence criteria and that they used online searches to expand that initial knowledge.
A second group of participants first described what "being influential" meant to them and determined an initial set of influence criteria that they then used as the first step to make sense and gain understanding of the task (Moore & Hoffman, 2011). The defined criteria acted as a starting point while the search helped participants expand that initial knowledge and confirm whether those initial thoughts were relevant to the current task: The next generation of the most influential academics . . . How do I figure that out? Up and coming, I guess. Let's see, I guess one of the first things to do would be to check universities and maybe have a look at some of the PhDs or MSc's or something like that.  A third group of participants acquired the knowledge needed to define a starting point while seeking information. This process helped them make a connection with previous experiences and move to more productive information-seeking strategies that eventually concluded with the identification of the required names: Again, it's people that I've known and met because it's such a small research community, so I'd think, who do I expect to make the biggest impact, judging from their kind of promising track record at the moment? So what I'm trying to do is think of the different people that I know, but also I'm trying quickly to dismiss the people that are already established. So these are the people that are up-and-coming and I think therefore will be next generation, big in that area. [Task 2-E1] These three behaviors indicated that defining a starting point is essential to making sense of a problem situation and determining a useful information-seeking strategy. In line with the D/F theory, this starting point is the element that directed participants to start connecting searched data with their previously determined influence indicators. We found that the lack of an explicitly defined starting point or identified initial frame (Klein et al., 2006) resulted in the absence of a mindful search strategy. In other words, we found that when influence indicators were not somehow determined prior to the beginning of the search, a need to pause emerged to define some sort of indicators: I need to think exactly what this next generation of most influential researchers/academics is. Again, I might think that if I pause a bit and I think what does this exactly mean, so maybe the next generation could be related to the new areas of research in chemistry, so it's more about those people looking at those fields.  In the next section, we discusses how the role of expertise influenced the starting point of sensemaking.
The role of expertise. As reported earlier, to make sense of an academic domain, scholars need to either have the knowledge to narrow down the domain or find a starting point to do so. Novice and expert participants used various types of information sources to define that "point of reference" (E2), "entry point" (N3), or "starter reference" (E1). For HCI domain tasks, most participants relied on their domain knowledge based on previous experiences to find the necessary minimal knowledge to "start to get a feel for what's important in a field" (E1): I had a starting point. I had [DN]'s book from a past course, so that kind of started me off. So I had an initial entry point, I suppose. [Task 2-N3] With the questions relating to my field [Tasks 1 and 2], I have that, kind of, direct knowledge to draw upon. So, I, kind of, used that as a baseline, to an extent, of these are people that I look up to or think are doing good things. [E2] As noted previously, for Tasks 3 and 4, all participants were considered novices with little or no task domain (chemistry) knowledge. Consequently, starting points were harder or impossible to determine in a few cases, resulting in a more challenging identification and prediction of names. In most of these cases, these two tasks were described as "harder" (N1, N3), "more difficult" (N2, E4), or "more challenging" (N6) than were Tasks 1 and 2. For Task 3, most participants needed to start "from scratch" (E3) and construct a reference point relying on information beyond their background knowledge. For this task, a few participants decided to draw on soft knowledge gained in high school after some attempts to elaborate a start from hard information sources while in other cases, first tasks provided the previous knowledge needed to start making sense of the current task: and what I heard, and if I think of this, I will perhaps think about the ones that discovered the DNA structure. For all tasks, when participants could not recall any prior soft information, they looked for some sort of hard information that would give them a domain overview (E1, E2, E3), hierarchical influential order of community members (N3, N6), or lists of research trends (N5, N4) and thus guide their sensemaking process.
Most participants based their choices on authoritative sources of information, such as names obtained from higher education institutions, prestigious conferences, research groups, and awarded prize lists. We observed this pattern mainly for Task 2, for which many participants looked into prestigious conferences (e.g., E5, N6, E2), organizations and Royal Societies (N4), and universities (E4, E3, N6), and based their decisions on the information found in those external hard sources. To some extent, we inferred that participants saw authoritative sources as having the same role as a domain insider or peer.
We found that participants who could define a starting point were then able to determine possible sets of answers for most of the tasks whereas not having a starting point resulted in the inability to solve a task at all. This situation was evident with three extreme cases in which, due to the lack of a starting point, participants (E1, E2, N5) could not find ways to make a final decision with which they would be satisfied. Participant E2's explanation of her Task 4 performance illustrates this situation: I don't know how to decide whether they're doing something promising or not, because I don't have a point of reference to say, oh that sounds like it might be something really influential within the field, do you see what I mean? So, I can pick this guy, but it feels arbitrary, or I could pick one of the students I was looking at before, but I feel like I'm just picking to pick something.  Domain expertise was the type of expertise mostly required for finding a starting point. The relevance of domain expertise was shown when both novice and expert participants experienced difficulties in determining an entry point in the chemistry domain tasks but went through a smoother process in Tasks 1 and 2. When participants were familiar with the task domain, they tended to use soft information to determine a starting point. They had to search for hard information in unfamiliar task domains, as they could not draw on previous experiences.
The search for a starting point constituted an important phase in the process, often following the definition of influence indicators reported in this study as the initial phase. Influence indicators let participants determine a starting point and an information searching strategy, but participants also used those indicators to direct the selection of search keywords and concepts. In terms of the D/F model, defining a starting point would be similar to identifying an appropriate initial frame to direct a search for further data, and influence indicators would help determine what data to connect with the frame.
In the next section, we discuss further phases of participants' sensemaking process for identifying and predicting peers.

Identifying and Predicting Influential Community Members
After defining influential indicators and finding a starting point, participants' sensemaking involved various phases until they made final decisions. To some extent, all participants moved through the phases of seeking, filtering and selecting influential names. Some participants also verified preselected influential names and only made final decisions once they had achieved a degree of conviction based on supporting evidence. In terms of the D/F theory, the first three phases (seeking, filtering, and selecting) that emerged from this study were concerned with the elaboration of frame instantiations (influential names), and the verification phase involved activities of judging the plausibility and gauging the quality of those frame instantiations. Having influence indicators and a starting point helped discard data or give further consideration to data that they found through keyword searches. Incoming information was used to expand the initial starting point, or, using Klein et al.'s (2006, p. 90) words, to "fill in missing parts of the [initial] frame." In most cases, participants built understanding of one name at the time rather than making a list of possible influential academics and comparing them to make the final choices. Only Participant E5, who determined citation count as the main influence indicator, decided on her final names after comparing each candidate's citation counts of a final set of four.
Three causes emerged that made participants pause and go back to a prior phase: when they were feeling lost or stuck and unable to find any relevant names in line with their influence indicators, when the initial search strategy was not found useful enough to identify one or all required names, or when they could not support findings with any type of evidence. Participants referred to these moments of the process as "turning points" to "seek and infer new data" (Moore & Hoffman, 2011), which in many cases resulted in a change of the information search strategy. When turning points occurred, participants determined a (new) starting point, defined more influence indicators, or sought different supporting information. The following fragments exemplify participants' turning points during the process: Ah, so it's an associate professor, so they're not really up and coming. Okay. Maybe I should change my strategy. Another way I could try to find people is try to think of different research groups, and then to look at the PhD students and researchers that are part of the group. [Task 2-E3] I don't recognize any of the names, so I should probably think of a different strategy of doing this.  The sensemaking process concluded when participants either identified the three requested names per task or decided to abandon a particular task after several unsuccessful information-seeking efforts. In the case of the latter, participants could not find the necessary information to satisfy the task requirements or make informed decisions. As described earlier, we observed this situation when participants could not narrow down the domain under investigation or determine a starting point.
Participants needed various cycles of defining, seeking, analyzing, and verifying, alternating with going back to previous phases (turning points) to construct the necessary knowledge to identify and predict influential members of the academic community. Although novices went through fewer cycles and in some cases skipped any verification phase, these characteristics indicated a high level of iteration involved in the process of building an understanding of an academic community. A change in the information-seeking strategy tended to occur when participants did not find worthy findings or were not convinced by the supporting evidence. In most cases, verification sensemaking activities helped participants make informed decisions.
In the next section, we discuss the role of expertise throughout the process reported here.
The role of expertise: Level of engagement. Once participants located soft or hard information sources (e.g., an article, a blog, a journal) from where they could potentially identify influential names, both novices and experts actively examined the information that they found. However, experts manifested a higher level of engagement with the tasks than did novices. In line with Kuhlthau's work (1999), comments from more experienced participants revealed a desire to understand and reason rather than merely complete the tasks; in most cases, comments from novices indicated an eagerness to finish the tasks rather than to achieve the most suitable solutions or gain new knowledge. Novices' lack of interest in questioning the credibility of sources or looking for additional sources to confirm their findings further demonstrated their haste in getting the tasks done. A similar situation was found by Warwick et al. (2009) and was referred to as "cognitive economy." Supporting that study, novice participants selected "information sources and search strategies" within their "comfort zone," but avoided more time-consuming or less familiar strategies. In addition, the majority of those novice participants tended to complete the tasks by making arbitrary decisions: But there's too many people, it's a big list, how am I supposed to know who was the best? Well, I am going to say, you know what? I'm just going to make an arbitrary decision, sort of, because I want to, because one of them went to my university, so, I'm going to say him. [Task 1-N4] Okay, I'm going to go for a different strategy, actually. I'm going to go away from Google Scholar, because I think that's too difficult for me to find this question out. [Task 1-N3] Yes, I thought about [citation counts], but that's a very long process.  The role of expertise: Arbitrary and informed decisions. In some cases, participants used evidence to inform and support decisions. Most novices made final decisions based on the finding of minimum relevant information, such as by choosing three names from a list with the 100 most influential academics in the HCI domain. Conversely, more experienced study participants sought an evidence-supported explanation to help them make informed final decisions. In other words, experts elaborated their frames further. We observed that experts experienced uncomfortable feelings each time they had to make arbitrary judgments and felt that they needed complementary information to be able to make the right decisions. As noted earlier, when some experts were not satisfied with the information found, they could not make a decision. Consequently, they decided to stop the search, reporting that what they had found was not adequate to complete the current task: So looking through this list, I don't know any of them. I could arbitrarily pick from the ones that had Nobel Prize listed because they must be smart, but I don't want to do that. I don't want to do that. I feel like I need a connection, and that's such a biased way of doing this. I feel like I need a connection of knowing the story behind some of these in order to make my decision.  On the other hand, when expert participants made final decisions based on supporting evidence, they were able to develop a rationale to justify and explain the reasons behind those decisions. All experts drew upon soft information to define starting points in those tasks where their domain expertise was high (Tasks 1 and 2). Most of them explained that to determine the influential names, they "thought about [their] PhD research and who [were] the people that [they] read up on the most" and "whether they attend conferences and whether they were involved in any kind of networking" (E1, E2, E3, E5). This indicated a strong role between domain knowledge and soft information for expert academics. The following fragment exemplifies the reasoning process to determine influential peers of E3: So he's just got his first research grant, so he's still very young in terms of being a researcher, and it's a major Starting Independent Researcher Grant, so he must have a really good idea, in which they've decided to give him some funding. And it describes here the scheme aims to identify and support the very best and creative early career independent researchers, so there must be something promising in his work, where they thought they would give him this grant. So I've put him down as one of the next generation.  Moreover, supporting Kuhlthau's findings (1999), expert participants were more concerned with the origin of sources than were novice participants, which indicates that for experts, finding credible and reliable information sources was essential to determine satisfying responses. This was demonstrated by the fact that experts tended to rely more on information coming from authoritative sources (e.g., Royal Society of Chemistry, American Psychological Association, Cambridge University) and discard information found in generic websites (e.g., about.com, Wordpress blog posts). For Tasks 1 and 2, we observed that most expert participants knew beforehand specific websites (e.g., CHI Conference), and key names and terms (e.g., HCI games research) that they considered useful to narrow down the domain and help them focus on specific HCI subdomains.
Although most novice researchers reported not being concerned with making arbitrary decisions, two of them (N2 and N5) did pay attention to supporting information with credible and informed decisions. Interestingly, both of them, as opposed to more experienced participants, questioned the validity of soft information and described it as "not rational" (N5) because answers would have been "just" based on their opinion and people they have met. Therefore, "it may not be right" (N5). Instead, they sought "some specialized website or specialized publications" from which they "can find [that] type of information," such as "a specialized online magazine" (N2). In short, they were looking for authoritative sources of information, as expert participants did. We elaborate on this point in the following section.
We have discussed the use of soft and hard information in experts' and novices' sensemaking process, the level of engagement in the tasks, and the degree of arbitrariness for both of them when making final decisions. In the next section, we report levels of task completion and participants' feelings experienced during the tasks and in relation to the achieved results.

Uncertainty and Satisfaction in the Selection of Influential Community Members
Participants self-reported varying degrees of satisfaction and confidence in relation to how happy they felt with their responses, even in cases when they could not achieve full completion of a task (e.g., when they determined only one or two of the three requested names). They described themselves as being "very happy" (E1) or "happy" (E2, E4), "confident" (E3), "not sure" (E3, E4), "less confident" (N3), and "not convinced" (N5) or having not "much conviction" (N6) with their performance and final responses to each task. Based on their testimonies, three degrees of confidence emerged, and this spectrum represents the main feelings that participants experienced during the think-aloud sessions and how certain they felt about the results. Table 4 describes each of the three confidence degrees, although in some cases, the boundaries could not be precisely defined.
Based on an analysis of think-aloud protocols and debriefing interviews, Table 5 illustrates three dimensions of participant task performance: participants' satisfaction and confidence with each of their responses, the type of information source used to make a final decision, and levels of task completion. Smiley icons indicate the degrees of confidence and satisfaction, as shown in Table 4. Icons on white background indicate names identified using solely soft information, icons on light grey background indicate names identified using both soft and hard information, and icons on dark grey background indicate the use of only hard information in identifying or trying to identify influential names. An "X" icon indicates that a name could not be identified; for a given task, one to two "X" icons mean that a task was partially completed, and three "X" icons mean that the task was not completed at all. Table 5 indicates that participants experienced more than 1 degree of confidence during the session, in the majority of the cases feeling more confident with names identified in the familiar domain (Tasks 1 and 2) than in the unfamiliar domain (Tasks 3 and 4). Participants who relied on either soft information or combined soft and hard information to inform final decisions also were the ones most satisfied with their responses. They were equally confident and happy that the names they had determined were or would be somehow influential: I'm sure that he will be one of the ones in the next generation of most influential chemistry researchers.  On the other hand, those participants who relied merely on hard information to determine the responses and those whose decisions were arbitrary reported that they were "not sure" or did not feel confident with some of the results: I can't with any certainty back up the task two and four names.
[Tasks 2 and 4-N4] I wouldn't be sure that they are actually the most influential, so I'm going to give these names but I can't tell for sure that they are the most influential.  When participants could not identify or predict the name of a thought leader, they felt frustrated and dissatisfied with their performance: I think I'm starting to get a bit fed up with it. I'm struggling to know how to make a decision.  Professional expertise seems to have played a decisive role in undefined tasks when participants could rely on soft information as the main source (Tasks 1 and 2 for experts; Task 1 for novices). The majority of experts relied entirely on soft information to make final decisions while the rest combined soft and hard information to complete the tasks. Although some novices did draw upon their previous experiences to solve Task 1, when the problem became more demanding cognitively (Task 2), they needed to seek hard information. Moreover, Tasks 2 and 4 demanded more professional expertise to evaluate the degree of relevance and influence of peers' work and achievements. This knowledge was found necessary to make an informed prediction.
In addition, Table 5 shows a much larger difference in experts' feelings about insider-outsider tasks than in novices' (i.e., between Tasks 1 and 2, and Tasks 3 and 4). This aspect is in line with experts' need to gain understanding and make informed decisions, stressed in the previous section, rather than to solely complete tasks. In accordance with prior work by Wildemuth (2004), Table 5 indicates the need of familiarity with the subject area to find an appropriate solution, although N5 could not rely on her novice domain knowledge to find any influential person for Task 2.
In the previous section, we noted that having both robust professional and domain expertise and making decisions based on soft information sources appear to have been determinant factors for experiencing highly confident and happy feelings with task performance. This finding appears to be supported with a much larger difference in feelings between experts and novices in Task 2 ( Table 5): The former were mostly satisfied and happy with their decisions informed by soft information while responses from the latter group were mainly uncertain and based on hard information. The basic level of professional expertise (research knowledge) appears to have been the cause of frustrating feelings among novices, as some of them found it hard to distinguish a hierarchical order between peers (relevance criteria). Results from Task 3 demonstrate the difficulty generated in experts when they cannot draw on their prior knowledge. Although most of them were able to find influential names based on inferences drawn from hard information, their levels of satisfaction were not as high as for tasks in which they were highly familiar with the domain and therefore could rely on soft information (Tasks 1 and 2). Novices also experienced lower levels of satisfaction with their responses for Task 3, but were higher overall than that of the experts. This is in line with findings reported in the previous section in which many novices did not question the credibility of hard information whereas others expressed doubtful feelings for soft information and did not want to rely on it.
In Task 4, for which both groups of participants' domain knowledge was equally low, most participants appear to have experienced similar feelings of uncertainty, in line with Task 3. However, it denotes a higher level of uncertainty for experts, highlighting the fact that expert participants needed some sort of explanation or justification (supporting evidence) to be able to make final decisions. As reported previously, novice participants did not seem concerned with gaining new knowledge but with completing the tasks, even if by making arbitrary decisions. Consequently, some experts made the choice to abandon Task 4 or leave it incomplete while all novices completed the task even when they were not confident with the responses. Those experts who did complete the task were uncertain with regard to the influential peers they identified, and only E5 was highly confident with the choices.
Previous sections provided a detailed description of the role of domain and professional expertise in sensemaking and information seeking, and of how decisions were informed by two types of supporting evidence (hard and soft information). The type of knowledge resource used and the degree of arbitrariness in decisions both emerged as the factors that influence participants' levels of satisfaction with the responses and indicate a distinct difference between novice and expert participants, as Table 5 shows. In the following section, we discuss the relationship among these factors.

Discussion
We have reported information-seeking strategies and learning and understanding processes that academics went through when examining familiar and unfamiliar knowledge domains to complete undefined exploratory search tasks-undefined in the sense that participants of this study were not provided with definitions of what "being influential" meant when asked to identify individuals with that quality. We described that process as iterative and interactive with a variable number of turning points, augmented with the difficulty of formulating a starting point or finding an initial frame, and the level of domain expertise. In this section, we summarize our findings and note the need for starting points to construct understanding of a problem situation. We introduce and discuss the relationship between the quality of evidence (soft and hard) and internal factors (level of uncertainty, and domain and professional expertise) involved in the identification of current leaders and rising stars.

Making Sense of the Academic Community
To answer our first question, we gathered robust evidence to shed light on how academics construct understanding of their communities and make sense of an undefined problem situation. In terms of Klein et al.'s (2006), D/F theory, we did not provide participants with an initial frame of the meaning of being influential but instead let participants use and expand their own frames to begin solving the tasks. Consequently, participants learned about the problem (being influential) while defining key qualities (influence indicators) and proposing solutions (current and future thought leaders). This process involved cycles of understanding, defining, seeking, questioning, analyzing, filtering, evaluating, and verifying. These actions are in line with those of previous studies by Kuhlthau (1991), Foster (2004, and Klein et al. (2006).
In consonance with prior theories (Bodnar, 2005;Chi & Card, 1999;Klein et al., 2006;Pirolli & Card, 2005), our study provides further evidence that academics needed to find, define, or construct a story, map, schema, or some other type of structure to start making sense of the data, referred here to as a starting point or an initial frame (Klein et al., 2006). Klein et al. (2006, p. 90) stated that initial frames are then completed as "data are acquired." Similarly, Kuhlthau (1991) explained how "through a series of choices" (p. 361) people complement "what they already know or have experienced" (expertise) with "new information" from various sources (information seeking) to "construct their view of the world" (construct understanding and make sense of a situation; p. 362).
Building on those prior studies, we presented various ways in which academics constructed starting points, and described how, by finding adequate data, they constructed other frames, referred to as "reframing" (Moore & Hoffman, 2011). We discussed how, when academics attempted to move further in the sensemaking process having no influence indicators (anchors), ill-defined frames, or no initial frames, they were forced to stop and define or revise the first steps. Conversely, when indicators and starting points were clearly defined, they guided academics' informationseeking process and helped them decide which information was relevant and which to discard and therefore make pertinent decisions and final choices. In other words, frames guided academics' search: When academics were not able to construct a suitable initial frame for the problem situation, they ended up either with arbitrary solutions, incomplete solutions, or no solution at all, as was exemplified by Task 4 for E1 and E2 and Task 2 for N5 (Table 5).
When possible, participants drew on their domain knowledge from personal experiences to complete the tasks. In previous sections, we indicated how soft information was used to find a starting point. The fundamental role of soft information in helping academics formulate initial frames when tackling undefined problems and making sense of exploratory search tasks expands previous studies by Kuhlthau (1999). On the other hand, when participants were less familiar or unfamiliar with the task domain (Tasks 3 and 4), they could not draw on their previous knowledge to formulate an initial frame. The lack of specific domain expertise results in poor understanding of the internal structure of a domain. Consequently, most of the components of an academic community (members, trends, contributions, literature) are unknown. Professional expertise was found necessary to understand how academic components relate to each other, providing the knowledge to assess the level of influence among community members.
When participants possessed low domain knowledge, hard information sources were used as the entry point to learn about the unfamiliar community (chemistry) and its components. In this study, hard information sources were accessed through the Internet and search engines. Nonetheless, in most cases, information obtained from the Internet was described as a form of "self-publicity," but with "not a lot there" to "tell [the researcher] that that person was more influential, or less influential, than another person who worked somewhere else" (E4). In other words, the majority of participants found that hard information alone was neither sufficient nor credible enough to help them develop a starting point or evaluate the credibility of information. Participants stressed qualitative insights or face-to-face peer communication as core component of their sensemaking process.
In line with prior studies, we found that the use of combined soft and hard information is necessary to gain a thorough understanding of the academic community. Kuhlthau (1991, p. 361) stated that "formal organized sources from information systems interact with informal sources from everyday life experiences" to enhance understanding. Similarly, Faisal (2008, p. 71 (for all quoted material in this sentence)) explained that the understanding process of a domain consists of both explicit (and more objective) steps such as identifying community members, their research interests, and "who collaborated with whom on a piece of work" and implicit (and more subjective) steps such as defining "who is influential in a particular domain, or what piece of work or idea changed the course of a field's development."

Professional and Domain Expertise
Another objective of this study was to explore the differences between novices' and experts' sensemaking processes. Prior studies have indicated that more experienced individuals draw on knowledge acquired previously across many domains (Chi et al., 1981;Cooke, 1999;Klein et al., 2007;Kuhlthau, 1991Kuhlthau, , 1999Pirolli & Card, 2005) whereas more novice ones seem to deal with problems at a surface level (Chi et al., 1981;Cooke, 1999). Findings from the study reported here are in line with those of prior studies but add new detailed insights about the role of domain and professional expertise in the sensemaking process of influence tasks.
Prediction Tasks 2 and 4 were described as harder than were Identification Tasks 1 and 3. Identifying rising stars involved some domain expertise to have a sense of the community and identify key components, but also a high level of professional expertise to determine hierarchical structures and evaluate the relevance of peers' achievements, awards, and/or discoveries. On the other hand, candidate names for identifying thought leaders could be inferred by having basic domain knowledge and minimum professional expertise.
When making sense of an academic community, initial frames are fundamental to make the breadth of the domain more manageable. To understand a community, it is necessary to have domain expertise regarding components and to have professional expertise regarding how these components are related to each other. Both types of knowledge are fundamental when understanding influence and identifying current and future influential community members. The more experienced a researcher is in a particular domain, the more familiar he or she is with those components. The more research experience an academic has, the more familiar he or she is with the internal hierarchical structure of his or her academic community.
Another interesting finding is that research or academic experience alone (professional expertise) is not sufficient to solve tasks in which domain expertise is not specific. This was evident when some of the most experienced participants of this study experienced feelings of uncertainty and eventually failed to solve some of the tasks from the unfamiliar domain (Table 5). To some extent, experienced academics acted as novices when they were dealing with undefined situations in unfamiliar domains while expecting to be able to apply expertise. Initially, experienced academics tended to use the same strategies used to solve familiar domain tasks, but the lack of specific domain knowledge to draw on made it difficult-and in some cases impossible-to get familiar with community components (Figure 1), and therefore identify who was or would be an influential individual or who had made a remarkable discovery. To make pertinent decisions, experienced academics tried various strategies to find adequate information to help them construct understanding of the chemistry domain, but all of them opted for keeping the task(s) unsolved if they were not satisfied with the information found or if they might have needed to make arbitrary decisions.
Our study demonstrated that both experienced and novice academics use frames as the thread to explore information sources, gain understanding, make inferences, and in most cases, come up with solutions with which they were happy. However, the experts expressed a need to construct a frame involving a thoughtful process to complete the tasks whereas novices expressed no such need and instead presented answers just to complete the tasks.
The way both groups of researchers used frames to achieve task completion also was different. More experienced researchers tried primarily to gain understanding in addition to completing the tasks whereas novices mostly focused on merely solving the tasks. This situation did not indicate a lack of understanding of the tasks but a lack of engagement. Novices exhibited a similar behavior in a prior study focused on search expertise (Warwick et al., 2009), in which more novice participants manifested cognitive economy instead of searching for the best possible answer. In our study, a similar situation was evident when most novices decided to stop the information-seeking process after they achieved satisficing responses. In addition, they were reluctant to use unfamiliar search strategies because they were considered time-consuming, even when that strategy would have provided more adequate solutions.
Finally, we found a different analytical strategy toward the selection of information sources between more and less experienced researchers. The former chose and prioritized authoritative sources of information over unknown ones whereas the latter were less concerned with finding trusted sources. Among novice participants, most decisions were made arbitrarily, even when they had not found sufficient information or were not fully certain about the adequacy or credibility of the information found.

Confidence in Decision Making
Work by Kuhlthau (1991Kuhlthau ( , 1998 has indicated that uncertainty is expected at the beginning of and during an exploratory search situation. Diriye (2011) found that when a problem situation is undefined and the domain search task is unfamiliar to task doers, they may experience a considerable degree of uncertainty. Similarly, most participants in our study experienced uncertainty at the beginning of the tasks, which increased when the familiarity with domain tasks decreased. This initial feeling of uncertainty can be seen as the result of having to tackle an undefined problem situation (Kuhlthau, 1991).
When informed choices were derived from constructed understanding and evidenced-based decisions, uncertainty was replaced by feelings of satisfaction and highly confident answers. Similarly, when decisions were based on soft sources of information, feelings of satisfaction increased. However, frustration and dissatisfaction were recurrent feelings among experienced participants when they could not rely on their domain expertise and needed to base decisions primarily on hard information sources. Another trigger of frustration was when more experienced participants were unable to achieve a solution with which they were happy, which contrasts with the novices' satisficing feeling discussed earlier.

Conclusions
This study investigated how academics gain understanding of their communities while tackling exploratory search tasks. Six novice and 5 experienced researchers were recruited to complete think-aloud sessions followed by semistructured interviews. Data sets gathered provided a detailed picture of how academics of different levels of seniority manage and explore information when they are looking to identify the current and the next generation of influential community members within and outside their domains of expertise. Key findings indicated the relevance of formulating an initial frame to obtain deep understanding. Soft information emerged as the major source from which more experienced academics construct a starting point and informed decisions. We also found that robust domain knowledge allows decisions to be based on soft sources of information, or on a combination of soft and hard information, and feelings of confidence and certainty prevail over feelings of satisficing. We argue that a combination of soft and hard information sources may lead to deeper understanding and to higher levels of confidence in decision making. Other findings expanded and enriched previous studies on sensemaking, and on the novice-expert's sensemaking differences, indicating the need for future studies on the subject.
This study has limitations that need to be recognized. The number of participants and the fact that most of them were female make the findings indicative rather than applicable to a broader population; male academics may present different behaviors. Studies with academics from other domains may find different sensemaking behaviors. In addition, this was a laboratory study with assigned tasks. Although the tasks were designed to reflect real needs of academics, they were artificial tasks that were completed in an artificial setting, so we cannot be sure how broader contextual factors may have influenced behavior in normal practice.