Publisher
University of Tennessee at Chattanooga
Place of Publication
Chattanooga (Tenn.)
Abstract
This research project assesses whether the number of behavioral anchors increases rater accuracy in performance evaluations. Specifically, it observes whether a behavioral anchor rating scale (BARS) with five behavioral anchors will be more accurate in scoring performance evaluations than BARS with three behavioral anchors. The accuracy of the ratings is assessed in terms of scale alphas, absolute differences between ratings and true score estimates, and over and under rating. In addition, this study explores whether rating accuracy varies among raters with different degrees of conscientiousness. All participants will be recruited from Middle Tennessee State University and Amazon’s MTurk. The study will use a between-subjects design to assess the accuracy of performance evaluations ratings on a computerized assessment. Participants will be randomly assigned to evaluate the performance of University professors using either BARS with five behavioral anchors or BARS with three behavioral anchors. The study will use videos as stimulus material which will involve University professors lecturing about a topic in a course at the undergraduate level. Prior to viewing the stimulus material and providing performance ratings, all participants will complete a short personality inventory, specifically assessing four facets of conscientiousness, by responding to items based on the HEXACO personality model. Performance evaluation ratings are often used in organizations to make decisions regarding promotion, training, and compensation. Therefore, the outcome of this study could provide organizations with a better performance evaluation technique that yields increased rater accuracy.
Date
10-24-2020
Subject
Industrial and organizational psychology
Document Type
posters
Language
English
Rights
http://rightsstatements.org/vocab/InC/1.0/
License
http://creativecommons.org/licenses/by-nc-nd/4.0/
Included in
Will the Number of Behavioral Anchors Increase Rater Accuracy in Performance Evaluations?
This research project assesses whether the number of behavioral anchors increases rater accuracy in performance evaluations. Specifically, it observes whether a behavioral anchor rating scale (BARS) with five behavioral anchors will be more accurate in scoring performance evaluations than BARS with three behavioral anchors. The accuracy of the ratings is assessed in terms of scale alphas, absolute differences between ratings and true score estimates, and over and under rating. In addition, this study explores whether rating accuracy varies among raters with different degrees of conscientiousness. All participants will be recruited from Middle Tennessee State University and Amazon’s MTurk. The study will use a between-subjects design to assess the accuracy of performance evaluations ratings on a computerized assessment. Participants will be randomly assigned to evaluate the performance of University professors using either BARS with five behavioral anchors or BARS with three behavioral anchors. The study will use videos as stimulus material which will involve University professors lecturing about a topic in a course at the undergraduate level. Prior to viewing the stimulus material and providing performance ratings, all participants will complete a short personality inventory, specifically assessing four facets of conscientiousness, by responding to items based on the HEXACO personality model. Performance evaluation ratings are often used in organizations to make decisions regarding promotion, training, and compensation. Therefore, the outcome of this study could provide organizations with a better performance evaluation technique that yields increased rater accuracy.
Department
University of Tennessee at Chattanooga. Dept. of Psychology