• Bailey, P., Craswell, N., Soboroff, I., Thomas, P., de Vries, A. P., Yilmaz, E. (2008). Relevance assessment: are judges exchangeable and does it matter. In Proceedings of the 31st Annual international ACM SIGIR Conference on Research and Development in information Retrieval, 667674.
  • Baron, J. R., Lewis, D. D., Oard, D. W. (2006). TREC-2006 legal track overview. In Proceedings the Fifteenth Text REtrieval Conference (TREC 2006). Retrieved May 24, 2010 from
  • Barry, C. L., Schamber, L. (1998). Users' criteria for relevance evaluation: a cross-situational comparison, Information Processing and Management, 34 (2–3), 219236.
  • Cleverdon, C. W. (1970). The effect of variations in relevance assessments in comparative experimental tests of index languages. Technical Report ASLIB Part 2, Cranfield Institute of Technology.
  • Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 3746.
  • Cooper, W. S. (1971). A definition of relevance for information retrieval. Information Storage and Retrieval, 7 (1), 1937.
  • Hedin, B., Tomlinson, S., Baron, J. B., Oard, D. W. (2009). Overview of the 2009 TREC legal track. In Proceedings the Eighteenth Text REtrieval Conference (TREC 2009). Retrieved May 24, 2009 from
  • Huang, X., Soergel, D.. (2006). An evidence perspective on topical relevance types and its implications for exploratory and task-based retrieval. Information Research, 12 (1). Retrieved May 26, 2010 from
  • Huang, X. (2009) Topical Relevance, Rhetoric, and Argumentation: A Cross-Disciplinary Inquiry into Patterns of Thinking and Information Structuring PhD Dissertation, University of Maryland. Retrieved May 26, 2010 from∼xiaoli/XH_DisserationAbstract.pdf
  • Landis, J. R., Koch, G. G. The measurement of observer agreement for categorical data. Biometrics 1977, 33, 15974.
  • Oard, D. W., Hedin, B., Tomlinson S., Baron J.B. (2008). Overview of the TREC 2008 legal track. In Proceedings of the Seventeenth Text REtrieval Conference. Retrieved May 24, 2010 from:
  • Voorhees, E. M. (1998). Variations in relevance judgments and the measurement of retrieval effectiveness. In Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 315–323.
  • Wang, J., Sun, Y., Mukhtar, O, Srihari, R. (2008). TREC 2008 at the university at buffalo: legal and blog track. In Proceedings the Seventeenth Text REtrieval Conference (TREC 2008). Retrieved May 24, 2009 from
  • Wang, J., Sun, Y., Thompson, P. (2009). TREC 2009 at the university at buffalo: interactive legal e-discovery with Enron emails. In Proceedings the Eighteenth Text REtrieval Conference (TREC 2009) Retrieved May 24, 2009 from
  • Wang, P., Soergel, D. (1998). A cognitive model of document use during a research project. Study I. Document selection. Journal of the American Society for Information Science and Technology, 49 (2), 115133
  • Wilson, P. (1973). Situational relevance. Information Storage and Retrieval, 9 (8), 457471.