SEARCH

SEARCH BY CITATION

Abstract

  1. Top of page
  2. Abstract
  3. THE SPECIAL CONTEXTUAL DEMANDS OF KNOWLEDGE WORK
  4. THE NEED FOR ADDRESSING KNOWLEDGE WORK AS ON-GOING CREATIVE PROCESSES
  5. HOW MOST APPROACHES TO USER STUDIES MISS THE MARK
  6. USING SENSE-MAKING METHODOLOGY AS AN ALTERNATIVE RESEARCH APPROACH
  7. EXEMPLAR APPLICATION OF SENSE-MAKING INTERVIEWING TO KNOWLEDGE CREATION WORK
  8. CONCLUSIONS
  9. Acknowledgements
  10. References

Designing approaches to support knowledge intensive work has been documented to be critical and costly. Research has shown that knowledge workers frequently evaluate KM efforts as missing the mark. They are too often left without the help they need for constructing knowledge-based solutions. Knowledge workers point to failures not so much in retrieving topically-perfect-information but rather to communication gaps, such as systems that do not address work demands and informing needs in complex, changing, and sometimes elusive situations. This research used an interviewing approach informed by Dervin's Sense-Making Methodology. The aim was to allow digging deeply to understand hidden depths of informing practices that rarely have come to light in user studies. The ultimate aim is to design systems that support knowledge creation anchored to knowledge worker informing practices and to the situationality of these practices. The purpose of this paper is to present an exemplar study focusing on the challenges of doing user research in such a way that it usefully informs the design of knowledge management systems intended for use in knowledge-intensive work in the for-profit context. Sense-Making Methodology is presented as an alternative and more powerful approach to studying knowledge creation work.


THE SPECIAL CONTEXTUAL DEMANDS OF KNOWLEDGE WORK

  1. Top of page
  2. Abstract
  3. THE SPECIAL CONTEXTUAL DEMANDS OF KNOWLEDGE WORK
  4. THE NEED FOR ADDRESSING KNOWLEDGE WORK AS ON-GOING CREATIVE PROCESSES
  5. HOW MOST APPROACHES TO USER STUDIES MISS THE MARK
  6. USING SENSE-MAKING METHODOLOGY AS AN ALTERNATIVE RESEARCH APPROACH
  7. EXEMPLAR APPLICATION OF SENSE-MAKING INTERVIEWING TO KNOWLEDGE CREATION WORK
  8. CONCLUSIONS
  9. Acknowledgements
  10. References

The design of support for knowledge workers1 whose primary activity is to create knowledge to ground actions of internal and/or external customers (e.g. consultants, business analysts, market and competitive intelligence analysts) is critical activity. Designs need to guarantee that knowledge workers invest minimum effort (e.g. cognitive, emotional, time) in finding bits, chunks or complete information resources that can help users meet their informing needs in order to create knowledge-based solutions and products. Designs should ideally allow knowledge workers to focus primarily on combining, interpreting, using and analyzing information resources, and in applying the results to their customers' needs. These needs range from creating suitable and effective knowledge-based analyses, studies, plans, strategies or recommendations. Design, thus, should focus not just on “delivering” information resources but on providing information inputs that will be informative to intended users and supportive of the ways in which they inform themselves.

At the same time, design must account for the two most marked characteristics of the contexts within which knowledge-intensive organizations operate. One of these is high competitiveness; the second is high complexity. Knowledge-intensive organizations (e.g. focusing on management, biopharmaceutics, marketing, and competitive intelligence) operate in intensely competitive environments. Their core competitive advantage is the knowledge-based solutions and products created by their knowledge workers. Any loss of time or misuse of efforts severely impacts the economic bottom line – the quality of knowledge workers' customers decisions and competitiveness. It has been well documented that knowledge workers spend excessive time struggling to gather useful inputs – attempting to locate relevant documents, scanning through irrelevant ones, erroneously including error-filled documents, and reinventing information resources that have somehow become “lost.”2

What makes knowledge work more challenging, however, is not merely the competitiveness of the organizational environment but rather that this risky competitive scenario plays out amid high complexity. Alvesson (2004) put it well: knowledge work is “the use of knowledge for achieving a high level of rationality in situations of complexity” (2004, p. 222) and “the exercise of professional judgment in the effort to solve complex, frequently unique problems” (p.23). Hence, competitiveness in knowledge-intensive organizations is heavily based on the learning and unlearning capacities of workers and on their skills in translating these into customized solutions that provide added value to customers.

Clearly, the challenges knowledge workers face are not merely document retrieving and gathering activities oriented to locating relevant and error-free information. Rather, somehow systems that support knowledge work must be designed to support creative, unstructured and adaptive informing-in-action and the capacities of knowledge workers to put information bits, chunks, and documents together in uniquely responsive ways. As knowledge-intensive businesses are highly dependent on the results of their knowledge workers' informing practices, any investment in strategies to support these practices must, at a minimum, be grounded in an understanding of these unique challenges.

Yet, there is accumulating evidence of how investments in supporting knowledge workers' informing practices are missing the mark. For example, the KPMG Consulting (2002) large-scale study of KM programs in 423 organizations (in the UK, Europe and USA) clearly suggested that the primary reasons users saw system designs as not helping and not meeting expectations involved a variety of communication gaps. Users saw the KM program as not sufficiently integrated into everyday working practices; as too complicated; as providing too little personal benefit. Users also challenged that there was lack of training; lack of learning time; and a general lack of communication. User critiques, thus, focused not on information retrieving and gathering but rather on how useful and usable the KM systems were to them in their everyday work practices.

THE NEED FOR ADDRESSING KNOWLEDGE WORK AS ON-GOING CREATIVE PROCESSES

  1. Top of page
  2. Abstract
  3. THE SPECIAL CONTEXTUAL DEMANDS OF KNOWLEDGE WORK
  4. THE NEED FOR ADDRESSING KNOWLEDGE WORK AS ON-GOING CREATIVE PROCESSES
  5. HOW MOST APPROACHES TO USER STUDIES MISS THE MARK
  6. USING SENSE-MAKING METHODOLOGY AS AN ALTERNATIVE RESEARCH APPROACH
  7. EXEMPLAR APPLICATION OF SENSE-MAKING INTERVIEWING TO KNOWLEDGE CREATION WORK
  8. CONCLUSIONS
  9. Acknowledgements
  10. References

In line with the above, a growing number of observers have conceptualized knowledge work with such terms as: ambiguous, elusive, lacking in structure, emergent, situational, disorderly, non-linear, and unpredictable.3 All of this suggests a need for designers to base their designs to support knowledge work on how knowledge workers go about navigating their chaos-filled journeys. Yet, in fact, most KM system designs4 (e.g. information or content management systems) do not focus on how knowledge workers actually create knowledge or inform themselves and how they use informative inputs to help in these processes. Rather, most KM systems focus on information access defined as gathering and retrieving. In addition, most KM systems are predominantly organized by type of document, source, industry, or knowledge domain and concentrate only on the end products of informing processes – what traditionally have been called “knowledge” or “information.”5

At the same time, there have been a great many projects acknowledging the need to introduce other design structures that are more responsive to knowledge worker needs. A large roster have emerged, including open forums for discussion and sharing; repositories of methodologies and tools; expert directories organized by skill level; employee directories; best practice repositories; and a host of alternative category systems for organizing documents by nature and type.6 Yet, when looked at as a whole what marks these efforts is that they add a host of alternative categories of access to documents that are already seen as difficult to navigate by KM workers. This does not mean that the alternatives are not potentially useful. Rather, it means that something more is needed because essentially these approaches are implementing design by what Bontcheva et al. (2006, pp. 142-3) critiqued as “text string.” As they explained it: “knowledge workers need information defined by its meaning [semantic], not by text strings, that is relevant to their needs and their current context, they need to find ‘not just documents’, but sections, chunks within digests of information created from multiple documents”.

Within this context, there have been a number of innovative introductions in enterprise information access that attempt to address semantic meaning and build it into KM systems. These innovations have involved using a wide variety of the flexible capacities of electronic technologies to develop and implement enterprise semantic search, taxonomies, ontologies, semantic annotations, automatic indexings and classifications, text mining, natural language processing, visualization, clustering, summarization, and so on.7 Again, these innovations clearly have potential but still in themselves do not address the issue that the challenges knowledge workers face are not merely having new ways to find and retrieve potentially relevant information bits, chunks, and documents but rather to make sense in navigating unique, complex, and uncertain terrains in ways that the solutions and outcomes they create provide value to customers' needs.

These innovations have proceeded usually without knowledge worker inputs about how they use KM systems to support their on-going informing practices. A host of alternative “access” categories are added yet these are still driven not by knowledge worker practices but by top-down defined and isolated categories – although now isolated in a variety of different ways. Prusak & Weiss (2007, p. 38) summed the situation up as “an unhappy reality”. While there have been many projects focusing on supporting knowledge work, this support has positioned users primarily as consumers and receivers within the context of established business processes, person hierarchies, divisions, tasks, and topics. These assumptions turn knowledge workers into “statistical artifacts” because they “can not cope with users' agency, leaving no space for users to define themselves and act in their own interests…. [I] individuals cannot escape speaking from a position - as a designer, researcher, investor, sales person, or user - and designers need to understand the positions from which their stakeholders understand their world…. [We] need to realize that all stakeholders, by the very fact of their living and asserting themselves, are experts on their own lives, and this applies to designers and users as well. Designers and their stakeholders merely understand differently, and the issue is how human-centered designers can bring these different kinds of understandings in design” (Krippendorf, 2006, p. 64–66).

HOW MOST APPROACHES TO USER STUDIES MISS THE MARK

  1. Top of page
  2. Abstract
  3. THE SPECIAL CONTEXTUAL DEMANDS OF KNOWLEDGE WORK
  4. THE NEED FOR ADDRESSING KNOWLEDGE WORK AS ON-GOING CREATIVE PROCESSES
  5. HOW MOST APPROACHES TO USER STUDIES MISS THE MARK
  6. USING SENSE-MAKING METHODOLOGY AS AN ALTERNATIVE RESEARCH APPROACH
  7. EXEMPLAR APPLICATION OF SENSE-MAKING INTERVIEWING TO KNOWLEDGE CREATION WORK
  8. CONCLUSIONS
  9. Acknowledgements
  10. References

Discussion above has argued that approaches to user-oriented design have focused on the nouns that drive KM systems – business processes, person hierarchies, divisions, tasks, type of document and topics. Much of this design work has proceeded without what has come to be called user studies. Yet, ironically, even in the context of user studies, by far the majority of studies have the same foci. Even when focusing on users, attention has continued to center on such static attributes as demographic, psychological, and geographic descriptions of users, all conceptualized as across time-space identifiers of users as individuals. Examples include: job titles, departmental membership, career level, knowledge domain, level of expertise. As Dervin put it as early as the late 1980s (Dervin, 1989/2003, p. 47): “probably 95% of the available studies on users of communication systems rely on this set of categories…and much the same emphasis has been placed in system design”. Recent systematic literature reviews completed by the authors of this paper have confirmed that user studies for the most part focus on static attributes of users, tasks, domains, and information problems11.

Indeed these are the dominant and traditional ways social sciences have addressed the study of human beings. Yet, from the perspective of our argument above – that we need to focus on how users navigate complex and elusive situations and the informing practices that allow users to do so – these approaches can be seen as across time-space stereotypes. Instead of focusing where the moments of informing occur, these approaches focus not on informing behaviors (internal and external) but on derivative behavior (e.g. search strategies, tactics and performance) that ignore the more powerful explanatory aspects of the hows and whys of user informing activities at specific moments of time when they intersect with potential inputs. The result is a level of understanding that does not reach the phenomenological and interpretive worlds of users. The resulting explanations of human differences are too far removed from the time-space moments when users are acting on their needs.

It is important to emphasize here what we mean by across time-space versus time-space bound portraits of users. (Dervin, 2003). We are implementing an approach in this paper that mandates an emphasis not on users per se but on users-acting-in-situations and, in particular, on their informing actions in specific moments of time-space within their situations. The focus on users as such is an across time-space emphasis; the focus on user informings-in-situations is a time-space bound emphasis. The former can only capture habit patterns, inflexibilities, and responses to rigid system constraints; the latter is able to attend to changes and flexibilities in human behavior. This does not mean that attention to habit patterns and other across time-space explanations are not useful. Rather, it means they are not enough for us to effectively design systems to serve user informing needs. This is so even when a variety of across time-space attributes are strung together in complex and lengthy predictive equations that tap increasingly finer granularities. An example might be describing a sub-set of users as members of a particular group with a high level of experience in using search engines, an analytic cognitive style, and a highly complex task to perform. Here the assumption is that stringing together attributes encompassing smaller and smaller groups of users will make it easier to design for that particular group.

However, results of our literature reviews suggest that understanding users in these ways is sufficient only for a limited set of purposes. It can explain, for example, users and how they structure and perform searches. However, it is not powerful enough to explain deeper and explanatory practices – how and why users inform themselves the ways they do, and what kinds of helps they need in these informing processes. As explained by Dervin, Reinhard, Song & Reed (2006, p. 4) “The problem…is that all the variability that is human adaptation to changing situations and human constructing of new options for themselves is relegated to error”. A basic assumption of the approach we use here is that there are systematic patternings in human situation-facing hows and whys that have been left unstudied because of the approaches we have been using. In the next section, we explain how we see Dervin's Sense-Making Methodology as aiming to provide an approach that can attend systematically to time-space bound patterns of user informings.

USING SENSE-MAKING METHODOLOGY AS AN ALTERNATIVE RESEARCH APPROACH

  1. Top of page
  2. Abstract
  3. THE SPECIAL CONTEXTUAL DEMANDS OF KNOWLEDGE WORK
  4. THE NEED FOR ADDRESSING KNOWLEDGE WORK AS ON-GOING CREATIVE PROCESSES
  5. HOW MOST APPROACHES TO USER STUDIES MISS THE MARK
  6. USING SENSE-MAKING METHODOLOGY AS AN ALTERNATIVE RESEARCH APPROACH
  7. EXEMPLAR APPLICATION OF SENSE-MAKING INTERVIEWING TO KNOWLEDGE CREATION WORK
  8. CONCLUSIONS
  9. Acknowledgements
  10. References

In the remaining sections of this paper we use the terms “sense-making” and “informing” interchangeably even though in relevant literatures they appear with a large array of confounded and contested definitions. For our purposes, the terms point to the practices, actions, or activities, both internal and external, that humans (in this case knowledge workers), use to bridge gaps within and between their inner phenomenological and interpretive worlds and the outer worlds of their environments including their societal and work contexts and the information systems and inputs they encounter.

Based on the arguments in sections 1, 2, and 3, we have concluded that understanding how and why knowledge workers make sense and inform themselves as they navigate changing, complex, and often elusive situations is necessarily a core foundation for designing systems more responsive to their needs. This requires a research approach that goes beyond attention to across time-space attributes of users, their knowledge domains, organizations, and tasks. The research approach we have applied is Dervin's Sense-Making Methodology (Dervin & Foreman-Wernet, 2003). Under development since 1972 and first named as such in 1983, the term “Sense-Making Methodology” is now used to point to the methodological edifice that drives the approach and the terms “sense-making” and “sense-unmaking” are used to point to the phenomena the approach is designed to study. Sense-Making has been developed to inform the design of research, systems, and practices for any context in communicative ways. What is meant by this is that design for communicating purposes must be based not only on transmission assumptions but also on communication assumptions. The former addresses concerns for placing the right “messages” (e.g. documents, information bits) in the right places at the right time. The latter addresses concerns of how to design for the realities of how communicating works for living, breathing, humans who necessarily must act in their worlds as evolving, changing, interpreting agents and not as empty buckets into which right “messages” can be dropped.

Drawing on a variety of complexity and chaos theories, Sense-Making has assumed since its inception that there are patterns to be found underneath or hidden within the macro across time-space categorizations of persons, organizations, domains, situations, and tasks that now dominate both research and design. Sense-Making's joint focus on research and system/practice design is an especial advantage for our purposes. In Sense-Making, all these activities are seen as requiring disciplined foundations based on communicating principles, implemented methodologically and dialogically. Methodologically means systematically implementing assumptions about the nature of people, communicating, informing, and sense-making activities in research and design methods. Dialogically means that part of the systemization of Sense-Making methods is a mandate to address communication as dialogue rather than as transmission. Dialogue in this sense does not refer to spontaneous chat (Dervin, 2007) and sharing. Rather it refers to the disciplined uses of communicating procedures that allow the interfaces between users and systems, both in research and in design/practice, to be implemented communicatively and systematically in ways that allow users to be heard and insofar as possible to act on their own terms.

In the history of the applications of communication to research and system/practice design, a series of logics have evolved over time moving from top-down designs to bottom-up involvements. (Dervin, 2008). Top-down designs are essentially based on assumptions that the noun hierarchies that organize systems and expertise at the top are sufficient for communication purposes. Since research has shown repeatedly that these assumptions do not work, designs have moved to focusing on sub-sets of potential users organized into an ever smaller and smaller sub-sets: domains, organizational types, task foci, user demographic and personality characteristics, and so on. The latest developments in this roster of noun hierarchies is emphasis on users divided into community and cultural groups.

Yet, this ever-increasing division of potential users into finer and finer sub-sets has not worked. Now driven by the astonishing capacities of emergent electronic technologies, we see two forces emerging. One of these is more of the same top-down logic but now using technologies to build ever-increasing alternative hierarchies of nouns (e.g. edifices of search terms arrayed, for example, by frequency of use). It is assumed that these alternative arrays of categories will provide entry points for users to meet their unique needs. The difficulty from a communication standpoint is that these hierarchies are still constrained within top-down defined expertise and too far removed from sense-making activities. A second force that is emerging is, indeed, a genuine alternative communication logic based on principles of spontaneous interaction. In short, users are invited to interact with systems and each other in voluntary ways – they comment, chat, add notations, blog, play games, vote, and so on. Some of these more communicative approaches have been used in KM system designs although clearly these are approaches far more familiar in marketing, mass media, and popular culture settings.

Communicatively, however, while these spontaneous “sharings” are more communicative, they still primarily manifest shallow and stereotyped views of user sense-makings. Resting as they do on fast paced chat and patter, these approaches rarely give participants time to dig deeply into their as-yet-unarticulated selves to mine the ways in which they struggle with external constraints and implement informing practices based on embodied practices and tacit knowledge. From its inception, Sense-Making has proposed that communicative practice, systems, and design require a different logic based on disciplined communicating for dialogue anchored in conceptualizations of human universals of movement through time-space. This emphasis in Sense-Making draws heavily on the theoretical writings of a number of philosophically and methodologically oriented social science and humanities scholars.8

The essential mandates of the Sense-Making Methodology, then, are to understand individuals in terms of human universals of movement through time-space, rather than allow our understandings to be constrained by organizational or researcher viewpoints and expertise. The mandate is to understand users not only as individuals but to do so in non-individualistic ways – to focus on sense-making actions, both internal and external, because this is where user encounters with inputs take place. Sense-Making assumes that system design and practice must ultimately be based on usings, not uses. In short, inputs uses are not a noun to be assessed only after the fact, at the end of the use process. Rather, it is a series of micro-moments made up of using activities. Inputs uses are always a verb or as said in Sense-Making language a “verbing.” (Dervin 1993/2003). More importantly, it isn't a “verbing” but rather multiple “verbings” and the methodological trick is to be able to zone in on these verbings in ways that we can genuinely hear users and apply the results of our listenings in system and practice design.

In order to understand and approach users in this way, Sense-Making Methodology is based on research methods for conceptualizing and operationalising variables, data collection and data analysis. Of particular concern for this paper are the interviewing approaches for data collection. Figures 1 and 2 below encapsulate the core premises of the methodological approach in easy-to-access metaphoric ways. The figures pull together the assumptions Sense-Making makes about people, sense-making, and how context, sources, and inputs intersect in the sense-making practices of users. A caution in their presentation is that the metaphor's accessibility masks what is indeed a complex methodology (Naumer, Dervin & Fisher, 2008) and is not intended to be a short-hand tool for understanding how to use the methodology.

thumbnail image

Figure 1. Sense-Making metaphor (Dervin, 2006).

Download figure to PowerPoint

thumbnail image

Figure 2. Sense-Making triangle: the core of the metaphor

Download figure to PowerPoint

In Figure 1, we see a human being moving through time-space in context and situation. The human, by mandate of the human condition, faces gaps at every point in time-space movement. Most of these “gaps” are bridged with habits and repetitions. The gaps of particular interest for information and knowledge management systems are those where sense has “run out.” Sense-makers must somehow construct new or revised sense or reclaim old ones. These sense-making processes involve both internal verbings (e.g. remembering, reflecting, thinking, emoting, relating, questioning) and external verbings (e.g. scanning, seeking, connecting, chatting, asking, gathering). As they face their gaps, sense-makers encounter (on purpose or accidentally) sources of inputs (e.g. information, documents, database systems, media, sources such as peers) that allow them to construct bridges over the gaps (e.g. insights, ideas, intuitions, emotions). Sense-makers constantly evaluate the inputs they encounter based on attributes relevant (i.e. relevances) to them – do the inputs help? do they hinder? In a world assumed to be at least in part disorderly, it is rare that a sense-making bridge is seen by a sense-maker as complete for situation-facing. Rather, sense-makers make do (sometimes called sufficing) with less than complete understanding because they must. Uncertainty is assumed to be characteristic of the human condition. All of this is framed within a moving time-space such that each new time-space moment mandates the making of another sense-making bridge.

Figure 2 pulls out the essence of this metaphor in terms of how it is used in interviewing practice. The foundational interviewing approaches of Sense-Making all focus on sense-making instances – intersections of situation-gap-bridge-outcome anchored in particular time-space moments. One given sense-making instance flows in time-space into another. These instances are the core foci of all Sense-Making interviews. How sense-making instances are addressed in interviews depends on research purposes but all are based on some form of what is called the Sense-Making Triangulation (Dervin, Reinhard, Song & Reed, 2006). This triangulation is used over and over again as part of the methodological emphasis on giving informants time to think deeply beyond surface stereotypes about their situations and to share deep articulations.

For the empirical study discussed below, the sense-making instance is called an informing moment or instance. Each moment of informing is anchored situationally in time and space. This means that it is assumed that knowledge workers are constantly navigating between macro aspects of the organizations, situations, and tasks they confront and micro aspects of moments of informing activities. In this way, knowledge worker informing is a result of the congruence of how they see and define their situations, gaps, necessary bridges, and sought for outcomes. While most traditional research focusing on information seeking and use (including much that now uses the terms sense-making) have limited gap-bridging to problem solving and decision making, Sense-Making studies have documented the variety of different ways in which sense-makers make sense of how they are stopped in their sense-making movements and how sense-makers often reach out for information inputs even in sense-making instances that information systems have deemed irrelevant to their purview. It is for this reason, as will be shown in the exemplar application described below, that Sense-Making interviewing does not limit informants to discussing things normatively labeled as “information” or any particular information systems or technologies.

EXEMPLAR APPLICATION OF SENSE-MAKING INTERVIEWING TO KNOWLEDGE CREATION WORK

  1. Top of page
  2. Abstract
  3. THE SPECIAL CONTEXTUAL DEMANDS OF KNOWLEDGE WORK
  4. THE NEED FOR ADDRESSING KNOWLEDGE WORK AS ON-GOING CREATIVE PROCESSES
  5. HOW MOST APPROACHES TO USER STUDIES MISS THE MARK
  6. USING SENSE-MAKING METHODOLOGY AS AN ALTERNATIVE RESEARCH APPROACH
  7. EXEMPLAR APPLICATION OF SENSE-MAKING INTERVIEWING TO KNOWLEDGE CREATION WORK
  8. CONCLUSIONS
  9. Acknowledgements
  10. References

The project we report here as an exemplar is informed by the mandates of the methodology described above. In particular, a Sense-Making Methodology approach to interviewing was strictly followed. Our specific purpose here is to illustrate how interviewing was implemented both theoretically and operationally. We focus on the design of the interview in this section as well as an overview of the project's purpose. In the final section, we present a preliminary extraction of qualitative themes that have emerged from interviewing. In our conclusion, we relate these specifically to the challenges of responsive design.

The on-going larger project on which this paper is based focused on knowledge worker' informing practices – the hows and whys of their sense-makings as they navigated situations that required that the inputs they used were not only appropriate to whatever external standards were applied but at the same time informative to their internal sense-making activities and to what they understood as the demands of complex, ever-changing, and often elusive situations. While the interviewing necessarily has intersected with each knowledge worker as an individual who was informing self about self the research focus has not been individualistic behavior as such but rather patterns in informing activities as knowledge workers went about creating knowledge and knowledge-based solutions and products for internal or external customers. The main objective, thus, has been to identify knowledge workers' informing practices – how and why they overcame gaps in specific gappy-situations as they saw them, and, most specifically, how and why they used inputs to help overcome these gaps. It is at the intersection of these foci that we assume research can add strengths to the design of enterprise information systems to support informing practices in knowledge-intensive work.

The sample of informants was obtained among active knowledge workers in England, persons employed in knowledge-intensive-firms in the for-profit context. The sample of 40 informants were obtained by nonprobability purposive sampling, identified by our first author at specific events (business conferences) and in scientific and business publications, and also based on networking recommendations. For all informants, the primary focus of their jobs at the time of interviewing was the creation of knowledge, such as analyses, plans and strategies, to support internal and external customer actions. These actions included whatever outcomes customers put on the plate – e.g. launching a new product, assessing a new market, sizing up competitors, evaluating human resources, using technologies in alternative ways. Informants included business analysts, management consultants, and market and competitive intelligence analysts and consultants.

In line with the mandates of the Sense-Making Methodology as described above, interviewing addressed a series of smaller and smaller units of analysis. The interview started, as is normative practice, with identifying informants as units of analysis – their organizational contexts, levels of education and expertise, and other demographic measures. The interview itself then flowed in the following steps that involved identifying and describing ever and ever smaller units of attention.

  • 1.
    ONE IMPORTANT WORK SITUATION: Informants were asked to select a situation in which they created a knowledge-based product for a customer. They were asked to describe what happened, what the situations involved, and how they saw their journeys through these situations as stopped or blocked. The unit of focus here became: informant-in-situation.
  • 2.
    CRITICAL UNDERSTANDINGS IN THE ONE SITUATION: Informants were asked to describe the core understandings they needed to construct in the important situation. They could describe more than one. Here the unit of analysis became: informant-seeking-understanding-in-situation.
  • 3.
    CRITICAL QUESTIONS NEEDING ANSWERS FOR EACH CRITICAL UNDERSTANDING: Informants were asked to describe for each critical understanding, the most critical and demanding questions they needed to answer when they were trying to construct each of the understandings and how critical each question was to getting to that understanding. Here the unit became: informant-question-asking-to-gain-understanding-in-situation.
  • 4.
    INPUTS USED TO TRY TO ANSWER EACH QUESTION: Informants were asked to describe what inputs they used to try to answer each question. Helpfulness of each input was explored and evaluated. Here the unit became: informant-using-inputs-in-trying-to-answer-question-in-situation.

We have purposively laid the units of analysis with clumsy language above to illustrate the variable ways researchers can enter informant narratives. Sense-Making studies have used a wide variety of units of analysis smaller than the person and the person-in-situation. The structure has always been designed to allow systematic analyses, qualitative and quantitative. But the intent of the interviewing design is far greater than an emphasis on capabilities for doing systematic analyses. The intent is to allow informants to describe the “hidden depths” (Dervin, 1999a/2003, p. 158) of sense-making processes. In the project reported here this was the primary aim. Each situation-gap-input-helps convergence was conceptualized as a sense-making instance – a micro-moment when a knowledge worker encountered at a specific moment of knowledge creation activity in time-space a gap and an input seen as potentially useful in bridging that gap. The aim of the research was to investigate deeply the gaps knowledge workers saw and what happened. These methodological lenses were designed to microscopically focus on informant gaps in specific situations without loosing the aspects of the whole.

This is illustrated in Figure 3 below showing four levels of triangulation applied to each situation-gap-input-helps sense-making instance. The result of using this logic in interview design was that informants were able to comfortably and deeply articulate their moments of informing while the interviewer got knowledge workers' deeper verbalizations in highly structured yet qualitative and open-ended ways. The process was one intended to go to the depths of the essences of knowledge worker informing practices –depths at which we assume that the greatest gains can be made as we invent alternative ways to conduct user studies to effectively inform system design. The same protocol was applied in all interviews, adapting queries to the specific contexts and experiences and varying the sequence of interviewing queries to each informant's articulation style at a particular interviewing moment. As is mandated by Sense-Making Methodology, all interviewing queries referred back to the specific situation, although in different breadths of time-space.

thumbnail image

Figure 3. The four levels of Sense-Making Triangulation nested within the larger units-of-analysis

Download figure to PowerPoint

Despite the fact that aim of this study was to inform design of enterprise information systems interviewer queries never focused on the existence or use of technologies or information systems of any particular kind unless these were introduced by informants. Informants were invited to explain which inputs they used; these inputs could be anything (e.g. insights, experiences, texts, database systems). Even when informants asked the researcher “Inputs…like information?” the answer was “Any kind of inputs.” This was done purposively because Sense-Making studies have found that focusing user research on “information” constrains users to describing “information” as it is currently organized by our systems. All interviews were focused on experiences that were seen as real to informants and that were explored situationally. Within situations, informants were invited to talk about what the situation involved, what helped and what they struggled with. By focusing both on failures and successes – another mandate of the Sense-Making Methodology – the result was comfortable interactions in which informants felt able to display themselves at different moments in time-space as intelligently strategic, or constrained and struggling, or even confused and befuddled.

Because the goal was to elicit quality interviews and to provide a quality exemplar of the application of Sense-Making, some 16 different pre-tests were executed. The first author of this paper worked directly with the second author for this project phase. An average individual in-depth interview took 1.5 hours, with a range from 1.2 to 2.5. All the final interviews were conducted by the first author in rooms reserved at each workplace. All interviews were recorded with informant permission.

The approach to interviewing in this project was one of the most complex of the Sense-Making Methodology approaches. Most often described as a “micro-moment, micro-element” interviewing approach it is one of the interviewing forms intended especially for complex, highly involving situations. In addition, the first author as interviewer was trained in the articulation styles required by Sense-Making. These included: achieving an interested but neutral detachment, never inserting nouns that had not been put on the interviewing plate by informants, allowing informants time to think deeply and articulate experiences, and adapting to the experiences and styles of informants in each interviewing segment. Intent was to cover the protocol as flexibly as possible. The kinds of questions asked that invited informants to describe essences of sense-making practices and connections they saw allowed for this flexibility. Informants did not stray off into discussion of non-essentials yet at the same time it was informants and not the interviewer that controlled substantive interviewing content.

An important aspect of the interviewing approach that clearly showed in this context was Sense-Making's emphasis on bringing to consciousness and thus to articulation understandings that were unarticulated, embodied, tacit, and/or unconscious. In the interviews for this project, the effects of this emphasis were evidenced by long time periods taken by informants to answer the deepest questions as they reflected. Informants clearly exhibited that they did not have well-structured answers on the tip of their tongues. Yet they willingly moved from possibly shallow and stereotyped thoughts and ready-made answers and dove deeply into their interpretive selves in what were genuine attempts to articulate their own experiences. One event illustrating this conclusion was an informant who was asked about constraints and barriers in a specific situation, and then said that the barriers were related to the “group…working on it…I could describe vaguely some ideas to X or…anybody else in the group, but they didn't really know what I was looking for… So…the constraints and barriers were…I had to do all the search myself…. was I looking in the right directions?” (Informant #12). This deep digging was also evidenced by comments about the interview that were made by the informants after it. As but one example: “Your method of relating the question to different… [situations]…makes you think about…it did help me.” (Informant #23).

Figure 4 encapsulates the interviewing process as described above and highlights how the interview implemented the two primary mandates of Sense-Making for interviewing – questioning “methodologically” and “dialogically.”

thumbnail image

Figure 4. Questioning methodologically and dialogically: how questions were grounded in metatheoretic assumptions.

Download figure to PowerPoint

What is meant by questioning “methodologically” is that interviewing implemented the specific mandates in Sense-Making to anchor sense-making instances in specific moments in time-space. This is illustrated on the left side of Figure 4 where attention is focused on how a specific input helped in answering a specific question. This bracketing of informing practices in time-space is what allowed the interviewing protocol to channel to the essences of informing practices. Being “methodological” is also illustrated on the right hand side by how constraints, barriers, and struggles informants saw themselves as facing were always anchored in the context of their situations. It is this way of framing interviewing that moved the process away from only an individualistic approach and allowed the researcher to come to understand how context was working itself out for informants in given situations at given moments. What is meant by questioning “dialogically” is that Sense-Making mandates the use of a variety of specific procedures for disciplining interviewer communicating so it is dialogic, so it invites articulation by informants on their own terms and not on terms set top-down by systems or researchers representing systems. Some of these dialogic procedures are emphasized in the bottom row of Figure 4.

CONCLUSIONS

  1. Top of page
  2. Abstract
  3. THE SPECIAL CONTEXTUAL DEMANDS OF KNOWLEDGE WORK
  4. THE NEED FOR ADDRESSING KNOWLEDGE WORK AS ON-GOING CREATIVE PROCESSES
  5. HOW MOST APPROACHES TO USER STUDIES MISS THE MARK
  6. USING SENSE-MAKING METHODOLOGY AS AN ALTERNATIVE RESEARCH APPROACH
  7. EXEMPLAR APPLICATION OF SENSE-MAKING INTERVIEWING TO KNOWLEDGE CREATION WORK
  8. CONCLUSIONS
  9. Acknowledgements
  10. References

Our purpose here has not been to report findings from this project but rather to focus on how the project's interviewing approach implemented a methodology designed specifically to address limitations identified in dominant approaches to user studies. It is instructive, however, to highlight a very few of the themes that have emerged that illustrate the strengths of the interviewing approach.9 Numbers in parentheses after quotes refer to specific informants.

(a) The gaps knowledge workers identified in inputs often focused on the inputs being in the right “territories” but not answering the specific questions they had.

*…information I need is about markets although what we have…is about projects. (#02)

*…there might be some statistics but [they didn't] necessarily have the depth of discussion or understanding about why, what's meaningful in those numbers, what's behind those numbers. (#22)

(b) The ways in which knowledge workers said they were helped by inputs almost always went beyond the content or the topic of the input.

*…by knowing… current issues, I could identify…areas where I need to do…more work and [get]… more explanation… (#29)

*…helped me paint the whole picture… helped me picture and visualize the market. (#29)

(c) The bridges that knowledge workers used to overcome gaps informatively were not necessarily what systems usually call information. Informants, for example, frequently turned to colleagues for informal inputs.

*… they could tell me what they want to know. “OK, I am working in the car industry, but what I want to know is….” (#29)

*…because of their…practical experience…they know…the potential problems and what to expect…do… (#20)

(d) Knowledge workers saw given inputs as limited bridges that helped as limited pieces of the larger puzzles that are the knowledge workers' informing quests. Each puzzle piece was for them part of a web of connections of diverse meanings created by the use of diverse inputs.

*…one piece of each of these puzzles…is not the whole jigsaw…. they are component parts. They…relate to each other which allowed us to refine an appreciation of the overall picture…. One piece in isolation doesn't allow…that. ( #08)

*…it was a piece of the puzzle and it helped to make sure we had a good triangulation of different views and different types of information, of different sources of information”. (# 22)

(e) Knowledge workers clearly exhibited that they were not trying to find right or absolute answers but rather answers that helped them to design their own answers to what they saw as particular situational needs.

*particularly the question we were talking about relies so much in interpretation…I feel like the exercise to have to look across a set of things that… are at least partially appropriate to what I need…is part of the process…it forces me to think critically about what helps and what doesn't. (#22)

*…having a defined process basically I find… quite restrictive, it doesn't give you the freedom to determine and choose what's right and what's wrong. (#02)

(g) In many senses, what emerged was a sense of knowledge workers as “knowledging workers” because they showed how they were constantly moving between different informing situations and different strategies for handling them.

* [time passing]… that's part of the process we've developed. We know we need to get here at the first pass, where we have some idea of where we need to be and there are some inner other steps that might depend on what we find‥ (#22)

*…what we also did is that we pulled on people internally who had previously worked at [company X] and know the company well. So we actually had a roundtable discussion. And we did that, later on in the process, when we already had a better view, a more refined view… and we really had a guess of more hypotheses that we really wanted to test about what they were doing and why. And I think that made that kind of round table conversation much more useful… …and the conversation was interesting because it generated some…new insights that we didn't have before. But it also allowed us to test those against what we had already found. ( #22)

Our primary conclusion is that the Sense-Making informed approach to interviewing allowed us as researchers to learn about aspects of user informing practices that rarely have come to light in other user studies. The listings above are a small slice of those emerging from this project. The interviewing approach allowed our first author as interviewer to dig deeply, to understand the hidden depths of user informing practices. In particular, the approach allowed the interviews to elicit understandings of knowledge work that were explanatory of situationally anchored informing practices rather than derivative of top-down imposed frameworks. In addition, the approach forced the interviewer to listen respectfully, giving time for informants to think deeply and to articulate their tacit and embodied experience and thoughts. Even though highly constrained by time and challenged by how hard it can be to articulate tactic, embodied practices, when informants realized they were allowed to fully express themselves – strengths and weaknesses, wisdoms and confusions, stumples and successes – surprising and rich exchanges took place.

It is the fundamental assumption of this paper that responsive design can orient itself to knowledge worker informing practices as disclosed in these interviews. The journey from the implementation results of this interviewing approach to design has only just begun. Prior Sense-Making projects have implemented design applications in other contexts but in fact these have been modest to date.10 In the midst of growing edifices of hierarchically based noun categorizations that are proliferating with the use of electronic tools, Sense-Making studies have documented the urgent need to focus attention in situationally-anchored informing practices. One of the future outcomes of this project is to design a responsive KM system.11 This initial effort will, of course, be a baby step. Many such efforts are needed to zero in on the systematic patterns in informing that exist at micro-levels below users, systems, tasks, and domains as macro categories. Needed as well are other alternative approaches to interviewing that address knowledge workers not merely as users but as “knowledging workers” applying their myriad informing practices to creating value-added knowledge-based solutions and products.

References

  1. Top of page
  2. Abstract
  3. THE SPECIAL CONTEXTUAL DEMANDS OF KNOWLEDGE WORK
  4. THE NEED FOR ADDRESSING KNOWLEDGE WORK AS ON-GOING CREATIVE PROCESSES
  5. HOW MOST APPROACHES TO USER STUDIES MISS THE MARK
  6. USING SENSE-MAKING METHODOLOGY AS AN ALTERNATIVE RESEARCH APPROACH
  7. EXEMPLAR APPLICATION OF SENSE-MAKING INTERVIEWING TO KNOWLEDGE CREATION WORK
  8. CONCLUSIONS
  9. Acknowledgements
  10. References
  • Alavi, J., Leidner, D. (2001). Review: knowledge management and knowledge management systems: conceptual foundations and research. MIS Quarterly, 25 (1), 107136.
  • Alvesson, M. (2004). Knowledge work and knowledge-intensive firms. USA: Oxford University Press.
  • Apostolou, D., Mentzas, G. (1999). Managing corporate knowledge: a comparative analysis of experiences in consulting firms, Part I. Knowledge and Process Management, 6 (3), 129138.
  • Bontcheva, K., Davies, J. Duke, A. Glover, T., Kings, N., Thurlow, I. (2006). Semantic information access. in Davies, J., Studer, R., Wareen, P. (ED), Semantic web technologies: trends and research in ontology-based systems, 139-169. England: Wiley.
  • Ciborra, C. (2002). The Labyrinths of information: challenging the wisdom of systems. Oxford: Oxford University Press.
  • Davenport, T. H. (2005). Thinking for living: how to get better performance and results from knowledge workers. Boston, Massachusetts: Harvard Business School Press.
  • Davies, J., Weeks, R., Krohn, U. (2004). QuizRDF: search technology for the semantic web. in Davies, J., Fensel, D., Harmelen, F. V. (ED). Towards the semantic web, 133143. England: Wiley.
  • Dervin, B. (2008, in press). Hidden passions, burning questions: the other side of so-called mass audiences. in Foreman-Wernet, L. & Dervin, B. (ED), Audiences and the arts: Communication perspectives. Cresskill, NJ: Hampton Press.
  • Dervin, B. (2007). Focus groups for participatory research: design using systematic dialogic principles drawn from Sense-Making Methodology. Presented at International Association for Media and Communication Research meeting, Paris, France: UNESCO, July 23–25. (For access: contact: dervin.1@osu.edu).
  • Dervin, B. (1999a/2003/2003/2003/2003). Sense-Making's journey from metatheory to methodology to method: an example using information seeking and use as research focus. in Dervin, B., Foreman-Wernet, L. (ED), Sense-Making Methodology reader: Selected writings of Brenda Dervin, 133163. Cresskill, NJ: Hampton Press.
  • Dervin, B. (1999b/2003). Chaos, order, and Sense-Making: a proposed theory for information design. in Dervin, B., Foreman-Wernet, L. (ED). Sense-Making Methodology reader: Selected writings of Brenda Dervin, 325340. Cresskill, NJ: Hampton Press.
  • Dervin, B. (1993/2003). Verbing communication: mandate for disciplinary invention. in Dervin, B., Foreman-Wernet, L. (ED). Sense-Making Methodology reader: Selected writings of Brenda Dervin, 101110. Cresskill, NJ: Hampton Press.
  • Dervin, B. (1989/2003). Users as research inventions: how research categories perpetuate inequities. in Dervin, B., Foreman-Wernet, L. (ED). Sense-Making Methodology reader: Selected writings of Brenda Dervin, 4760. Cresskill, NJ: Hampton Press.
  • Dervin, B., Reinhard, C.D., Song, M., Reed, S.J. (2006). Interviewing. In Dervin, B., Reinhard, C.D., Kerr, Z.Y., Song, M. & Shen, F.C. (ED). Sense-Making the information confluence: the whys and hows of college and university user satisficing of information needs. Phase II: Sense-Making online survey and phone interview Study. Report on National Leadership Grant LG-02-03-0062-03 to Institute of Museum and Library Services, Washington, D.C. Columbus, Ohio: School of Communication, Ohio State University. Retrieved 25 September, 2007 from http://imlsproject.comm.ohio-state.edu/imls_reports/imls_PH_II_report_list.html
  • Duke, A., Glover, T., Davies, J. (2007). Squirrel: an advanced semantic search and browse facility. In The Semantic web: research and applications, Lecture Notes in Computer Science, 4519, 341355. Germany: Springer-Verlag.
  • Feldman, S., Sherman, C. (2001). The high cost of not finding information. IDC Information and Data, Framingham, MA, USA: IDC.
  • Feldman, S., Villars, R. L. (2006). The information lifecycle management imperative. IDC Information and Data, Framingham, MA, USA: IDC.
  • Glaser, B. G. (1992). Emergence vs. forcing: basics of grounded theory analysis. Mill Valley, CA: Sociology Press.
  • Feldman, S., et al. (2005). The Hidden costs of information Work. IDC Information and Data, Framingham, MA, USA: IDC.
  • Kuhlthau, C., (1999). The role of experience in the information search process of an early career information worker: Perceptions of uncertainty, complexity, construction, and sources. Journal of the American Society for Information Science, 50 (5), 399412.
  • KPMG Consulting (2000). Knowledge management research report 2000. Retrieved November 27, 2005 from http://www.insite.cz/data/kpmg_km_report2000.pdf [Accessed September, 2007].
  • Krippendorf, K. (2006). The semantic turn: a new foundation for design. USA: Taylor & Francis Group.
  • Manafy, M., McKellar, H. (2007). Enterprise search sourcebook. Medford, N.J.: Information Today.
  • Markus, L. M., Majchrzak, A. and Gasser, L. (2002). A design theory for systems that support emergent knowledge processes. MIS Quarterly, 26 (3), 179212.
  • Naumer, C.M., Fisher, K. & Dervin, B. (2008). Sense-Making: a methodological perspective. Presented at CHI2008 Workshop on Sense-Making, Florence, Italy, April 5.
  • Prusak, L., Weiss, L. (2007). Knowledge in organizational settings: how organizations generate, disseminate, and use knowledge for their competitive advantage. in Ichijo, K., Nonaka, I. (ED), Knowledge creation and management: new challenges for managers, 3243. New York, New York: Oxford University Press.
  • Souto, P.C. N. (in preparation). Doctoral dissertation. University of Tampere, Finland.
  • Stenmark, D. and Lindgren, R., (2004). Integrating knowledge management systems with everyday work: design principles leveraging user practices. In Proceedings of HICSS-37, Big Island, Hawaii, January 5–7, 2004. Retrieved 21 November, 2006 from http://www.viktoria.se/∼dixi/publ/oskms02.pdf
  • Werr, A., Stjernberg, T. (2003). Exploring management consulting firms as knowledge systems. Organization Studies, 24 (6), 881903.
  1. 1

    2Knowledge or information workers are used interchangeably. Kuhlthau (1999, p.399) defined the latter as focusing on “seeking, gathering, and interpreting information…to provide value-added information as the basis for making decisions and judgments critical to the function of an organization or individual, frequently within the context of market profitability. In this situation, information seeking is a process of constructing new knowledge and understandings to add value to an enterprise” (Kuhlthau, 1999, p.399), Feldman, S., et al. (2005).

  2. 2

    2 See, in particular: Manafy & McKellar, 2007; Feldman & Villars, 2006; IDC, 2006; Feldman & Sherman, 2001

  3. 3

    3 See, for example: Davenport (2005), Alvesson (2004), Ciborra (2002). Stenmark, & Lindgren (2004) and Markus et al. (2002).

  4. 4

    4 KM system was defined by Alavi & Leidner (2001) as “a class of information systems applied to managing organizational knowledge…. systems developed to support and enhance… organizational processes of knowledge creation, storage/retrieval, transfer, and application”.

  5. 5

    5 See, for example: Werr & Stjernberg (2003).

  6. 6

    6 See, for example: Apostolou & Mentzas, 1999.

  7. 7

    7 See, for example: Duke et al. (2007), Davies, et al. (2004).

  8. 8

    8 See, Dervin (1999a/2003), Naumer, Fisher & Dervin (2008), and http://imlsproject.comm.ohio-state.edu/imls_papers/asist06_panel_infoR_02quotes.ppt.

  9. 9

    9 The thematic analysis used here is informed by Glaser (1992) and involved intersecting grounded theory driven inductive dig of informant transcriptions with the deductive categories form Sense-Making studies and the extensive literature reviews of user studies that informed this project. The approach to extracting quotes was “smooth transcription” where non-fluences and other grammatical errors common to spoken language were corrected.

  10. 10

    10 See, in particular: Dervin, B. (1999b/2003).

  11. 11

    11 See: Souto, P.C.N. (in preparation).