Performance Appraisal

Proximal variables - structural

Dealing with the configuration or makeup of the appraisal itself (ex. appraisal dimensions or frequency of appraisals)

DeNisi, A. S., & Sonesh, S. (2011). The appraisal and management of performance at work

"...although rating accuracy is important, it may not be as important as employee perceptions of accuracy and fairness. Satisfaction and perceptions of fairness are important for employee motivation." Performance Management System -Ongoing communication and feedback -Focus more on developmental aspect -Creates a standard for employees

DeNisi & Sonesh (2011) II

* Regardless of the focus, any feedback on performance - whether it's for administrative purposes or development - needs to be timely, frequent, and include plans for goal setting o Setting smart goals is key; may also need to distinguish between the types of goals.

What affects raters

-Raters may have different judgments and ratings -They want to create group cohesiveness -They want to give higher ratings to motivate subordinates -Rater agreeableness - the rater does not want to be seen as the bad guy -Supervisors may not want a negative reflection of themselves -They want to avoid confrontation -Time constraints -They are focused on financials

6 Steps for a Performance Appraisal System

1. Behavior vs. Results (Outcomes) 2. Composite Criterion vs. Multiple Criterion 3. Objective vs. Subjective Measure 4. Typical vs. Maximum Performance 5. Dimensionality of Job Performance 6. Dynamic vs. Stable

Factors that can help you receive feedback from a source

1. Having input in development of the feedback system 2. Information awareness about the system 3. Person giving feedback is credible 4. Person giving feedback is high on expertise and competency 5. Trustworthy source 6. Recommendations for improvement 7. Giving specific and timeliness feedback

Sources of Feedback Variables That Might Determine Impact of Indirect Observations of Performance

1. Source of the report (colleague, customer, etc.) 2. Initiator of the report (Supervisor requested?) 3. Form of the report (Descriptive vs. evaluative; detailed vs. summary) 4. Medium (letter, conversation) 5. Referent (behavior vs. results) 6. Timing (time elapsed since episode) 7. Motive for the report 8. Consistency of report observations

DeNisi and Sonesh (2011) III: Rater Goals

1. Task performance goals are purely motivated to increase performance levels and that are directly tied to the organization's goals in conducting appraisals 2. Interpersonal goals focus on ways to maintain or improve positive climate of a work group and maintain feels of equity 3. Strategic goals in which the manager uses the appraisal to increase his or her standing in the organization 4. Internalized goals in which the manager tries to provide as accurate appraisals as possible

Viswesvaran & Ones (2000) V

1. the mean performance of individuals changes over time 2. the rank order of individuals on performance changes over time 3. the correlations of performance indicators with external variables change

Kluger, A. N., & DeNisi, A. (1998). Feedback interventions: Toward the understanding of a double-edged sword.

Although FIs improve performance on average, they reduce performance in more than one third of the cases Found no evidence that information about failure (negative FIs) and information about success (positive FIs) have differential effects, on average, on performance. In summary, the data suggest that, at least under certain circumstances, FIs can impair performance and that the processes through which FIs affect performance require more than simple explanations.

Tziner (2002) II

BOS also appears to minimise communication barriers, role ambiguities, lead to clearer and more observable goals and consequent ratee commitment to carry them through. The feedback process with BOS is generally more focused, factual, objective, and unbiased, and thus more likely to garner specific job-related goals that positively influence appraisee attitudes to the overall PA process, and their consequent work satisfaction. BOS, unlike GRS, focuses on specific behavior rather than ambiguous, subjective criteria.

Roch, S. G., Sternburgh, A. M., & Caputo, P. M. (2007). Absolute vs. relative performance rating formats: Implications for fairness and organizational justice II

BOS was perceived as the most fair of all formats, reaching significance in Study 2. The relative percentile method (RPM) was found to be the most fair relative method. Takeaway: Even though a rating format may have psychometric advantages, ratee perceptions are another important variable to consider.

Wiersma, U., & Latham, G. P. (1986). The practicality of behavioral observation scales, behavioral expectation scales, and trait scales

BOS were preferred to BES by subordinates and managers, and in all but two instances, BOS were viewed as superior to trait scales. Trait scales were felt to be as good if not better than BES. Appraisal instruments need to be seen as useful by the people who use them. A second questionnaire administered to attorneys indicated that BOS would be easier to defend in the courtroom than either BES or trait scales. "Over time BOS users reported higher satisfaction because of fewer personality disputes, enabled appraisers to justify low ratings, improved feedback, better overall reviews "

Norm-Referenced Ranking Paired Comparison

Compare two employees until you find the better out of the two. Keep comparing each pair in the organization. Disadvantage: Time consuming because you have to look at all possible combinations

Viswesvaran, C., & Ones, D. S. (2000). Perspectives on models of job performance cont. IV

Each performance dimension is complexly determined (jointly by ability and personality) and that it is impossible to specify a sole cause or antecedent of a particular dimension of job performance. However, we should note that the general factor obtained in our model of job performance implies that there are some common determinants across different job performance dimensions. That is, different performance dimensions are likely to have common individual differences antecedents. Given a large body of research in work psychology, two individual differences variables that would fit the bill are: cognitive ability and conscientiousness

Rater Affect

Feelings during the appraisal process Positive affect leads to higher ratings Feelings about the person and whether you like them Liking the subordinate may not actual affect the appraisal, but you like the person because they perform well

Atwater, L. E., Brett, J. F., & Charles, A. C. (2007). Multisource feedback: Lessons learned and implications for practice

First, the positive relationship between attitudes toward using the feedback and reactions emphasizes that human resource professionals should understand the importance of paying attention to how they introduce and implement the MSF process in their organizations. Second, reactions to negative feedback were not transitory mood states with minimal implications for leadership development, but rather influenced subsequent behavior.

Proximal variables - process

Have a direct impact on the performance appraisal process (ex. impression management, accountability, or supervisor-subordinate relationship)

Rater Errors and Biases

High in agreeableness leads to more lenient ratings High in conscientiousness leads to more accurate ratings Halo focuses on an individual person (good in one area, so give good ratings on everything

Tziner, A., & Kopelman, R. E. (2002). Is there a preferred performance rating format? A non-psychometric perspective

How does the rating format affect employee attitudes and reactions? Behavioral rating methods appear to have the advantage over graphic scales. However, largely due to their precise nature, the specific drawbacks of BARS indicate it is less preferable than BOS or GRS. Psychometric advantages of BOS over GRS seem to be superior reliability and validity. Several investigations yielded little differences between BOS and GRS in certain dependent variables (ex. rater satisfaction)

Smither, J. W., London, M., & Reilly, R. R. (2005). Does performance improve following multisource feedback? A theoretical model, meta-analysis, and review of empirical findings

Improvement in direct report, peer, and supervisors performance ratings over time is generally small. Performance improvement is most likely to occur when feedback indicates that change is necessary, recipients have a positive feedback orientation, perceive a need to change their behavior, react positively to the feedback, believe change is feasible, set appropriate goals to regulate their behavior, and take actions that lead to skill and performance improvident

Sackett, P. R., Zedeck, S. & Fogli, L. (1988). Relations between measures of typical and maximum job performance I

In order to capture maximum performance • Have to know you're being evaluated • Be aware of what it means to maximize your performance, what you need to do • The actual performance measurement should be done over a short enough amount of time to maintain maximum performance Work sample tests measure maximum and not typical performance.

Cleveland et al. (1989) II

Information from performance appraisal had the greatest impact on salary administration, performance feedback, and the identification of employee strengths and weaknesses. Furthermore, when asked to indicate three activities or decisions for which appraisal had the greatest impact, over 50% of the respondents cited salary administration and performance feedback.

Dynamic vs. Stable

Is the criterion dynamic or stable? Stable means that you are required to perform the same things over time, while dynamic means that job requirements will change. Most of the time, it is more dynamic.

Mixed Standard Scale

Items for different dimensions/traits are not presented in orders/they are not grouped by dimension. Raters are asked to rate better than, equal to, or worse than the behavior described. Supposed to make things easier Disadvantage: Complex, difficult to score, difficult for the rater to give feedback and for the ratee to understand. Research has not supported the scale as being simpler. Not as commonly used as the other scales.

Conceptual criterion

Job performance

Viswesvaran, C., & Ones, D. S. (2000). Perspectives on models of job performance cont. III

Job performance is an abstract construct. First, this implies that someone cannot point to something physical and concrete and say that it is job performance. Second, there are many manifestations that could indicate job performance.

Atwater, L. E., Brett, J. F., & Charles, A. C. (2007). Multisource feedback: Lessons learned and implications for practice V

Leaders who received low rating but overrated themselves were more motivated than those that received low ratings and rated themselves low. However, they also had more negative reactions. Comparing the sources of feedback, improvement was only found in bosses' ratings of feedback recipients. Positive changes in leaders' performance orientation were related to positive changes in employee engagement.

Jawahar (1997) II

Moderator analyses indicated larger differences between ratings obtained for administrative and research purposes when performance evaluations were made in field settings, by practicing managers, and for real world subordinates. Administrative ratings were more lenient than research ratings when managers (not students) in real organizations (not lab settings) rated real (not paper people or videotaped people) superiors (not subordinates).

Viswesvaran, C., & Ones, D. S. (2000). Perspectives on models of job performance cont. II

Models of job performance postulating specific, stand alone dimensions developed to apply across jobs can be grouped primarily around three broad dimensions: task performance, OCBs, and CWBs.

Proximal variables

More about the actual organizational policies, performance appraisal, the relationship between the supervisor and subordinate. Divided into process and structural. Affect ratee and rater behavior.

Distal variables

More broad and can have effects on performance appraisal and the process Not as salient as proximal variables Affects ratee behavior and rater behavior Ex: organizational culture, climate, economic conditions

How the purpose affects the accuracy of ratings

More lenient if it is for administrative decisions More accurate if for developmental purposes Solutions -Multiple raters -Taking average of admin and developmental ratings

Murphy (2008) II

Murphy (2008) notes that a mediating model of PA tends to be the better model to use, as it takes into account the goals and objectives of the raters and acknowledges that they have an active role in the process. "These models suggest that raters must be given both the tools and the incentive to perform well as measurement instruments and that systematic efforts to remove the negative consequences of giving honest performance ratings are needed if we hope to use performance ratings as serious measures of job performance." If raters are given more active consideration along with the tools and training need to do their jobs, then perhaps there would be a stronger relationship between performance and performance ratings.

Objective vs. Subjective Measures

Objective measures may not be applicable to all jobs, you may not be able to obtain a number, you may not be able to capture all numbers, and may be deficient because you are not capturing everything that job performance contains.

Borman, W. C., Bryant, R. H., & Dorio, J. (2010). The measurement of task performance as criteria in selection research. I

Objective measures: Associated with several issues Can lead to bias - Isn't truly objective because people decide what to measure (contamination, difficult to measure objectively) Criterion contamination Time and equipment constraints for work samples May not apply to all jobs o Ex. production rates, scales, work samples and job knowledge tests

Kluger, A. N., & DeNisi, A. (1998). Feedback interventions: Toward the understanding of a double-edged sword. II

One way practitioners can minimize the risks of FIs is using them in combination with goal-setting intervention. Providing FIs that relate to previously established goals is likely to direct attention to the task at hand and not to the self. In sum, positive feedback can have negative consequences: to reduce this keep most of the focus of the good performance on other ways they can apply this to other behaviors, give them goals, focus the praise on task specific reasons, think about the amount of positive feedback and the person's ego

Behaviorally Anchored Rating Scales (BARS)/Behavioral Expectation Scale (BES) Difference

Only difference between BARS and BES is that BES includes "expected" in statements BES may get at potential behavior, while BARS may get at actual behavior.

Borman, W. C., Bryant, R. H., & Dorio, J. (2010). The measurement of task performance as criteria in selection research. IV

Overall, Project A research supports strong relationships between cognitive ability, spatial ability, and perceptual/psychomotor ability and task performance.

Ratee Issues

Participation increases perceptions of justice. More likely to make a change in their performance.

Criterion measures

Performance dimensions

Viswesvaran, C., & Ones, D. S. (2000). Perspectives on models of job performance

Positive correlation between task performance, contextual performance, organizational citizenship behaviors, counterproductivity and organizational deviance are pointed out. Cognitive ability contributes to overall job performance through its effects on learning job knowledge and mastery of required skills

Organizational Justice

Procedural justice - The perceived fairness of the process and procedures -How did they come to this rating? -How did they decide the raise amount? Distributive justice - The perceived fairness of outcomes .If the process isn't fair, the decision won't be seen as fair. Interactional - How you perceive you are being treated

Norm-Referenced Ranking Overall

Rank employees from best to worst Disadvantage: Have to rank everyone in each slot, even if several people may be a three. Time consuming and difficult with large employee population.

Absolute vs. Relative Ranking

Relative (Ranking) formats require raters to rate individuals relative to one another. Absolute (Ratings) formats involve raters rating each ratee against an absolute standard.

Behavioral Observation Scale (BOS)

Reporting the frequency of behavior Supposed to be more objective, but they are perceptions of frequency Opponents say it provides more subjectivity Advantage: Supposed to provide more clarity because you have actual behaviors defined

Borman, W. C., Bryant, R. H., & Dorio, J. (2010). The measurement of task performance as criteria in selection research. III

Research has generally shown small differences between rating formats in terms of level of rater errors, reliability, validity, or accuracy. Research has shown that rater error training is usually successful in reducing psychometric error, error training does not improve the quality of ratings when interrater reliability or accuracy is used as a criterion, and frame-of-reference training increases rating accuracy.

Boswell (2002) II

Results indicated no effect on employee attitudes (satisfaction, satisfaction with supervisor, and awareness of development) H5: Employees in the separated use PA will report they intend to use less development than those in the traditional system. Supported.

Roch, S. G., Sternburgh, A. M., & Caputo, P. M. (2007). Absolute vs. relative performance rating formats: Implications for fairness and organizational justice

Results indicated that not only are absolute formats perceived as more fair than relative formats, but differences in fairness perceptions also occur among relative and absolute formats. Rating format influences procedural justice, especially when outcomes are perceived as fair. Format had a greater influence on procedural justice perceptions when the outcome was relatively fair (i.e., high performance rating), than when it was relatively unfair (i.e., medium rating)

Behavior vs. Results

Results isn't always possible to obtain in performance appraisal measures because it is an objective measure of performance (Ex. number of sales).

Murphy, K. R., & Constans, J. I. (1987). Behavioral anchors as a source of bias in rating. II

Results suggest that behaviorally anchored scales are not necessarily more objective or less prone to bias than are scales without behavioral anchors. Anchors need to representative of behaviors. The raters need to be reminded that the behaviors on the anchors are examples.

DeNisi and Sonesh (2011) IV

Setting specific, yet attainable goals, and talking about ways to achieve them. Set goals by date, and meeting to see if goals have been met. Goal setting increases the effectiveness of feedback interventions Negative feedback can backfire Need to provide constructive criticism Vague feedback can be detrimental Untimely feedback can be detrimental Feedback does not always result in higher performance

Graphic Rating Scales

Simplest scale format Most commonly used in organizations No behaviors are attached to it Advantage: Easy to develop Disadvantage: No context/raters get little to no structure when it comes to the anchors. Lack of clarity. Creates a lot of subjectivity because it's based on what people's perception of excellent, average, and poor are. Does not define scale anchors.

Atwater, L. E., Brett, J. F., & Charles, A. C. (2007). Multisource feedback: Lessons learned and implications for practice IV

Specific findings: Individuals with higher self-efficacy were more likely to engage in follow-up activities and took more positive steps toward change. Text feedback was reacted to significantly less favorably than numeric feedback. Individuals were less motivated. Those who reacted negatively to the feedback on the first round got worse, and had more developmental feedback on the second admin.

Borman, W. C., Bryant, R. H., & Dorio, J. (2010). The measurement of task performance as criteria in selection research. II

Subjective: Generally preferred and involve ratings o BARS, BSS, BOS, CARS BARS (1 anchor) Behavior summary Scale (multiple anchors) Sample rating format Behavior observation scale (looking at perceptions of frequency of behaviors) Computer adaptive rating scales

Sackett, P. R., Zedeck, S. & Fogli, L. (1988). Relations between measures of typical and maximum job performance II

Supervisor ratings reflect global judgment rather than dimension-specific. These supervisor ratings also correlate strongly with max performance rather than typical performance (unexpected). It seems likely that as job complexity increases, the role of ability as a determinant of typical performance will increase. Thus, the correlation between typical and maximum performance may be substantially higher in more complex jobs than in the current study

Dimensionality of Job Performance

Task vs. Contextual You can generalize task, contextual, and withdrawal behavior across jobs Task performance is based on the different duties that are part of your job description Contextual performance is individual behavior that goes above and beyond and is not recognized by the formal appraisal system Counterproductive is anything that can harm the organization

RPM

The RPM,requires that rankings be made on 101- point scales where 50 equals average. For each perfor- mance dimension, all of the ratees of a given rater are considered relative to one another and ranked on a 0-100 scale.

Viswesvaran, C., & Ones, D. S. (2000). Perspectives on models of job performance cont. I

The developmental context of job performance dimensions can be categorized as either a. stand-alone, specific or b. part of a larger set of dimensions. The occupational focus of job performance models can either be 1. limited to specific jobs 2. applicable across jobs These aspects combined create 4 types of models of job performance.

Cleveland, J. N., Murphy, K. R., & Williams, R. E. (1989). Multiple uses of performance appraisal: Prevalence and correlates I

The four purposes of performance appraisal: • Within-Individuals (more developmental, identifying and using information about individual profiles of strengths and weaknesses, training needs) • Between-Individuals (comparing individuals in terms of their overall performance levels, administrative decisions) • Documentation (justify personnel decisions) • Systems maintenance (workforce planning, implement and evaluate human resource systems in organizations, goal identification, program evaluations)

Cleveland et al. (1989) III

The majority (70%) of the organizations sampled indicated that performance appraisals had at least a moderate impact on both between- (administrative) and within-individuals (developmental) comparisons. It is plausible that many organizations are implementing a single appraisal system to accomplish multiple, incompatible goals.

Levy, P. E., & Williams, J. R. (2004). The social context of performance appraisal: A review and framework for the future

The predominance of PA research on psychometrics but not how the organizational context or other factors affect the rater, how distal and proximal variables affect ratings, rater affect, and what increases ratee motivation (merit pay).

Murphy, K. R., & Constans, J. I. (1987). Behavioral anchors as a source of bias in rating.

The results confirmed the hypothesis that behavioral anchors can be a significant source of bias in performance ratings. When BARS contained incidents that had actually been observed by the rater, but that were not representative of the ratee's performance, performance ratings were biased in the direction of those unrepresentative anchors. The results suggested that anchors do not bias observation and encoding of ratee behavior, but may bias memory for behavior.

Atwater, L. E., Brett, J. F., & Charles, A. C. (2007). Multisource feedback: Lessons learned and implications for practice III

The role of organizational context is critical to the success of MSF (ex. organizational cynicism will interfere). Organizations need to assist MSF participants in understanding how the feedback orientation fits with the organization's goals and initiatives and how it aligns with HR activities. Acceptance and trust in appraisal and feedback are critical. Some individuals benefit from MSF while others do not.

Subjective Measures

There are two types of subjective measures. Ratings (criterion-referenced) and rankings (norm-referenced)

Murphy, K. R. (2008). Explaining the weak relationship between job performance and ratings of job performance

There is a weak relationship between job performance and ratings of job performance "... showing that factors other than ratee performance explain systematic variance in performance ratings."

Atwater, L. E., Brett, J. F., & Charles, A. C. (2007). Multisource feedback: Lessons learned and implications for practice II

These findings reinforce the need for organizations to consider how they facilitate feedback distribution and how they encourage developmental activities following feedback. Our findings show that those leaders who improved were more likely to see subsequent changes in employee attitudes. This result indicates that with regard to organizational outcomes, MSF can do more than just develop leaders. It can have a positive ripple effect upon others in the organization.

Boswell, W. R., & Boudreau, J. W. (2002). Separating the developmental and evaluative performance appraisal uses I

This article examined the effects of separating the evaluative and developmental components of PA. o Employees were assigned to one of two groups: (1) formal appraisal to be administered by the supervisor once-removed and used only for evaluation (treatment group) (2) performance appraisal conducted as in the past, by the immediate supervisor for both development and evaluation (control group)

Jawahar, I. M., & Williams, C. R. (1997). Where all the children are above average: The performance appraisal purpose effect I

This article looked at whether there was a Performance Appraisal Purpose Effect (PAP). When you are evaluating for administrative, you are more lenient. When you are evaluating for development, less lenient. This article found that there is a PAP. Performance evaluations obtained for administrative purposes were, on average, one-third of a standard deviation larger than those obtained for research or employee development purposes.

Trust

Trust the supervisor more, more accepting of the ratings Higher justice perceptions

Criterion problem

Trying to figure out how to define your criterion of interest. In performance appraisal, it is performance.

Typical vs. Maximum Performance

Typical performance is primarily affected by ability and personality or motivation. Maximum performance is primarily affected by ability.

Behaviorally Anchored Rating Scales (BARS)/Behavioral Expectation Scale (BES)

You are given behavioral examples of different levels of performance for the dimension and the scale anchors. Supposed to at least give the high, middle, and low levels of performance. Advantages: Should provide more clarity because everything is defined Disadvantage: May be limiting and too specific. Time consuming to construct, and therefore expensive.

Norm-Referenced Ranking Forced Distribution

You put employees into top percentage, bottom percentage, and middle. Advantage: Often only need to evaluate the top and the bottom, then everyone else is in the middle. Disadvantage: May be a big difference between the top middle and bottom middle, or big differences in any of the other categories. You are not getting detailed information about each employee.

Composite Criterion vs. Multiple Criterion

You will likely want a composite criterion for administrative purposes, and a multiple criterion measure for developmental purposes. Research has support that performance is multidimensional.

Previous performance appraisal research...

assumed that raters want to be accurate, so we just need to focus on having accurate rating scales. They ignored that the rater is an active instrument. (Murphy & Cleveland, 1995)

Sources of Feedback Direct Observation

best carried out by managers, supervisors you only see certain behaviors, and you may only see maximal performance if incumbents know they are being observed if being evaluated by peers, said peers are likely to see typical performance and can give better judgments issues: it takes time; supervisors may also only make their decisions based on initial observation (negative incidents may stand out more than average - likewise with especially positive incidents)

Multisource Feedback: Sources of Information Upper Management

might be important for succession planning negative: amount of time to observe; could be more biased because they're just hearing everything through the grapevine positive: separating feedback - getting administrative from upper, and then developmental from the supervisor; might be more objective and unbiased

Multisource Feedback: Sources of Information Subordinates

need at least 3 (based on research) positives: provide accurate feedback on managerial style negatives: concerns regarding anonymity

Multisource Feedback: Sources of Information Peers

negatives: they could be friends/enemies, and not give accurate ratings based on that positives: better to assess typical performance

Multisource Feedback: Sources of Information Internal/External Customer

positive: valuable to check on how internal processes are affecting external factors; can get a better understanding of word-of-mouth negative: problems may be outside of employee's control; more focused on results instead of behaviors; more prone to extremes; accuracy of feedback is questionable Ex. of internal customer: recruiter working with a hiring manager Ex. of external customer: recruiter working with people outside of the organization

Multisource Feedback: Sources of Information Self

positive: you have the most information on what you've done; can be used to compare against other peoples' perceptions of themselves negative: inflated ratings can be used best in a developmental, as opposed to administrative should not be used on its own

Multisource Feedback: Sources of Information Supervisors

positives: higher ratee satisfaction; most common for giving PAs; better understand your role and your job tasks/responsibilities negatives: depending on time/distance, supervisors may not be the most accurate judges; may also only see maximal performance

Judgment

private evaluation

Rating

public evaluation

Multisource Feedback: Sources of Information 360 degree feedback

should involve at least 2 of the sources, but may use all of them positive: lots of different feedback and different perspectives (some self-ratings involved) negative: unsure of which ratings/perspectives are most important; disagreement among sources may cause this to backfire issues: how to provide the ratings (how do you combine them? Does one get rated more heavily? Who actually gets the ratings?); developmental vs. administrative; how to choose raters (random selection vs. choosing your own)

Direct Observation Aids

using BARS as a reference (issues with this) using BOS may be a better alternative (in terms of litigation), depending on the frequency with which your observing them also suggested to train raters how to observe and to make them aware of biases ex. confirmation bias, and how to avoid it how feasible is this? - use multiple methods: diaries, multiple raters

Sackett, P. R., Zedeck, S. & Fogli, L. (1988). Relations between measures of typical and maximum job performance III

• MAIN TAKEAWAY: Low correlation between maximum and typical. This means that you need to distinguish maximum and typical performance. • Supervisor ratings were more correlated with maximum performance. Employees are more likely to increase their performance when they know their supervisor is nearby. • Managers may care more about whether employees can "turn it on" during busy times