Evaluating a metadata-based term suggestion interface for PubMed with real users with real requests
Article first published online: 18 NOV 2010
Copyright © 2009 American Society for Information Science and Technology
Proceedings of the American Society for Information Science and Technology
Volume 46, Issue 1, pages 1–18, 2009
How to Cite
Tang, M.-C., Wu, W.-C. and Hung, B.-W. (2009), Evaluating a metadata-based term suggestion interface for PubMed with real users with real requests. Proc. Am. Soc. Info. Sci. Tech., 46: 1–18. doi: 10.1002/meet.2009.1450460238
- Issue published online: 18 NOV 2010
- Article first published online: 18 NOV 2010
This paper reports results of an evaluation study of MAP (Multi-faceted Access to PubMed), a metadata induced query suggestion interface for PubMed bibliographic search.
A novel evaluation methodology was used to address the challenges involved in evaluating an IIR (Interactive Information Retrieval) system such as the MAP interface. The most significant aspect of this methodology is that, instead of using assigned tasks common in traditional IR evaluation, it asks real users with real search requests to search with real systems in an experimental setting. Several performance measures were created based on which comparisons were made between MAP and PubMed baseline. MAP was shown to perform better in several of these measures, especially when the search requests had not been attempted before.
The finding pointed to search characteristics as an important intervening variable in IIR evaluation. The advantages of and potential threats to our methodology were also discussed.