Please support Web standards. Have a look at this page on the Web Standards Project site.
LeBreton, J.M., Burgess, J.R.D., Kaiser, R.B., Atchley, E.K., & James, L.R. (2003). The restriction of variance hypothesis and interrater reliability and agreement: Are ratings from multiple sources really dissimilar? Organizational Research Methods, 6, 78-126.
Conventional wisdom has it that superiors, peers, and subordinates provide unique assessments because each group has a somewhat different perspective on performance. Researchers often point to the moderate correlations between ratings from each source as evidence for this. We argue that low between-source correlations are an artifact of most managers being rated between 3 and 5 on typical 5-point scales. We further argue that correlations between sources (e.g., superior-subordinate) are about as high as are correlations between raters from the same source (subordinate-subordinate). Our analyses using Monte Carol simulations as well as two large independent field samples indicate support for our hypotheses and reveal that different sources do not furnish ratings that are nearly as unique as typically thought to be.
Article in copyrighted peer-review journal: Organizational Research Methods
Filed under: Journal & Magazine Articles