User research and testing of FRBR prototype systems

Authors


Abstract

This study is part of a three-year ongoing IMLS-funded project on the research and development of FRBR-based retrieval systems to support user tasks and to facilitate effective information seeking. The major focus of this study is on user research and testing of current FRBR prototype systems. A sample of three such FRBR prototypes are evaluated and compared by 72 users from both public and academic library settings. This user study will help identify useful options for FRBR implementation and contribute toward the development of more effective FRBR systems in the future. The findings of this study will also reveal current FRBR implementation issues and possible solutions from users' perspectives.

Background

Current library catalogs and other information retrieval systems do not support all the functions of the catalog as stated by theoreticians such as Cutter and Lubetzky (Cutter, 1904; Lubetzky, 2001). Even after years of development since the mid-1970s and extensive research since the 1980s, searching these catalogs is still difficult for users (Borgman, 1996; Campbell & Fast, 2004; Halcoussis et al., 2002). Some of the problems cited in previous studies include the ways in which library materials are perceived based on current cataloging practices and the manner in which records are created and related following these practices (Yee, 2005).

IFLA's final report on Functional Requirements for Bibliographic Records (FRBR) (IFLA, 1998) has been widely embraced by library communities and is shaping the direction of future cataloging standards, and consequently, library practice and system development. In contrast with previous cataloging standards and practice, FRBR focuses more upon users as its framework is built upon taking user tasks into consideration. Such a user focus offers great opportunities for creating retrieval systems that better support user information seeking (Le Boeuf, 2002; Tillett, 2005).

However, FRBR is essentially a conceptual model that is open to a variety of interpretations and implementations. Current implementation efforts have been largely exploratory in nature and vary in many aspects regarding user interface and display, system features, FRBR model focus, collection, and other technical system implementation details. As noted by Carlyle (2006), these differences in FRBR implementations are due to the various ways in which cataloging rules are interpreted and the various designs of the individual catalog systems.

An extensive review of current implementation research and development reveals that current implementation efforts have mainly been focused upon system development and how to adapt existing cataloging standards and practices to support user tasks as defined in FRBR (O'Neill, 2002; Ayres, Kilner, Fitch, & Scarvell, 2003; Mimno, Crane & Jones, 2005; Ayres, 2005). Very few projects have actually either conducted or reported user studies on their developed FRBR systems, although several have had plans for such user testing. In addition, up to this point there have been no evaluative comparisons of existing FRBR prototype systems.

Research Questions

This study is part of a three-year ongoing IMLS-funded research project on the research and development of FRBR-based systems to support user tasks and to facilitate effective information seeking. The major focus of this study is to understand users' experience with existing FRBR-based prototype systems, and to evaluate whether these systems support user tasks as defined in the FRBR model. In particular, the following questions will be explored:

  • Do the FRBR prototype systems support user tasks as defined by the FRBR model?

  • Do the FRBR prototype systems support these user tasks better than regular OPAC systems do?

  • What functions do users find helpful when searching FRBR prototype systems?

  • What factors contribute toward differences in user performance when using the FRBR prototype systems?

  • How could existing FRBR prototype systems be improved to facilitate user information seeking?

Method

Although FRBR implementation efforts are occurring in both library and non-library settings, this study focuses upon the FRBR prototype systems specifically related to online library catalogs both for comparison purposes and for the development of a FRBR online library catalog at a later phase of the project based on the results of this user research. The FRBR-based catalogs included in the user study are:

These systems contain some FRBR-like features based on the model for end users and support similar user tasks to be tested for comparison purposes for this study. Participants of this study include 72 users from both the public (36 participants) and academic (36 participants) library settings. Each system is evaluated by 12 public library users and 12 academic library users, with a total of 24 users for each system.

Search tasks are designed around the following general user task categories as defined by FRBR: a) to find materials that correspond to the user's stated search criteria; b) to identify an entity; c) to select an entity that is appropriate to the user's needs; and d) to acquire or obtain access to the entity described (IFLA, 1998). In addition, participants are asked to search their regular online catalogs and the FRBR systems with their own tasks.

Data about user interactions with the FRBR prototype systems will be gathered through a multi-method approach, including screen-captures, pre- and post-search surveys, focus group discussions, and eye movement tracking.

Findings and Discussion

This research is currently in progress. The data collection and analysis are scheduled to be completed in late spring and late summer, respectively. FRBR user evaluation and research is critical in developing effective systems that support user information seeking. This user study will help identify useful options for FRBR implementation and contribute toward the development of more effective FRBR systems in the future. The findings of this study will also reveal current FRBR implementation issues and possible solutions from users' perspectives.

Ancillary