Common practice in calculating within-group dispersion (VARwg and SDwg) and within-group agreement (rwg) includes first estimating within-group variance at the item level, then averaging across item-level variances. Using classical test theory, we illustrate problems with the common approach. Estimating dispersion and agreement via scale scores is recommended, and implications of item-specific and raterspecific variance for indexing substantive dispersion constructs are discussed.
