• expert raters;
  • taxonomy;
  • kappa


A comprehensive taxonomy categorizing passive behaviors in people with dementia was developed and revised through the use of expert raters. The taxonomy was first derived from the synthesis of 15 empirical studies that addressed this phenomenon, then was rated by an expert panel of six nurse-scientists with expertise in dementia and neuroscience research. This article describes the application of two measures of agreement, multiple rater kappa and proportion of agreement for multiple raters, calculated using Stata and SPSS, to evaluate and revise the taxonomy. The method proved useful for estimating the content validity of the taxonomy and provided evidence of stronger agreement among raters for the revised and final forms of the taxonomy. Nurse researchers will find this methodology to be an efficient, practical approach to applying measures of agreement for a variety of purposes, including taxonomy development. © 2001 John Wiley & Sons, Inc. Res Nurs Health 24:336–343, 2001