Measuring consistency for multiple taggers using vector space modeling

Abstract

A longstanding area of study in indexing is the identification of factors affecting vocabulary usage and consistency. This topic has seen a recent resurgence with a focus on social tagging. Tagging data for scholarly articles made available by the social bookmarking Website CiteULike (www.citeulike.org) were used to test the use of inter-indexer/tagger consistency density values, based on a method developed by the authors by comparing calculations for highly tagged documents representing three subject areas (Science, Social Science, Social Software). The analysis revealed that the developed method is viable for a large dataset. The findings also indicated that there were no significant differences in tagging consistency among the three topic areas, demonstrating that vocabulary usage in a relatively new subject area like social software is no more inconsistent than the more established subject areas investigated. The implications of the method used and the findings are discussed.