PSY 3711
To ensure feedback acceptance, what can we do?
Have feedback from someone who is trustworthy, credible, powerful, an expert in the area or from multiple sources
There are two fundamental problems that can plague a performance appraisals system. What are they?
(1) Contamination Irrelevant source of information "contaminate" the performance appraisal Example: knowledge of the persons college GPA causes higher ratings (2) Deficiency The appraisal does not include all important aspects of job performance Example: appraisal of task performance with no questions relevant to contextual performance
Name four specific pieces of information that we can obtain from the O*NET. Describe how one of them might be useful for a human resource intervention.
(1) skills (2) knowledge (3) abilities (4) tasks One piece of information that might be useful for HR intervention is the "skills" piece of information so HR is able to create training programs accordingly
A classic study of typical versus maximal performance was discussed in class. This supermarket study by Sacket et al (1988) found what? Describe the basic design and results.
-245 employees were tested 1 month after training for their maximum performance in scanning items from 6 standardized shopping carts; they measured the average rate of the item scanning. -Workers were consistent from week to week, relatively low correlations between typical and maximum performance measures were found
In general terms, describe what a meta-analysis does and why they are conducted.
-A quantitative synthesis of multiple studies on similar studies with a goal to create a single super study -Conduct these to see if differences between studies are due to sampling error or something else or if they are real -Also give more accurate estimate of the relationship
Kuncel and Beatty (2013), Thinking at Work paper. The define ability, aptitude, and achievement measure. What are their definitions for each (you may quote or paraphrase them for this answer).
-Ability - a sample of a person's current behavioral repertoire -Achievement - gains in ability over a period of time after an opportunity to learn -Aptitude - and ability that is associated with the amount of gain (learning) demonstrated after an opportunity to learn
In class, the number of correct versus incorrect decisions was presented as a introductory model for evaluating the efficacy of a personnel selection system. This model oversimplifies reality in a number of ways. List and describe three of them.
-Assumed that people apply in large groups: people look at many different jobs and might not decide to apply -Based on linear model which isn't always true, because some personality characteristics show nonlinear relationships -Job performance is not one thing, it's multidimensional (meaning different dimensions predict success)
You have been asked to develop a performance appraisal system that measures the in-class performance of students. Develop and describe some of its features.
-Behavior based study - measuring frequency and quality of important in-class participation -Category method - creating a checklist of participation such as raising their hand or taking notes -Doing a long term study over the course of the semester would allow use to gather average ability and motivation to perform well in class
What was Spearman's g theory of intelligence? On what did he base this?
-Believed that intelligence is one signal entity -Based off of conducted ability tests in many different subjects and concluded that there was a single overarching ability that accounted for most important differences across measures -So performance on a mental task = g + s s = something specific to a particular measure
What are the perspectives sampled by a multi-rater feedback instrument? What can be gained from obtaining information from all 4 perspectives?
-Boss, supervisor, or bosses -Customers or clients -Subordinate -Peers/coworkers -Self -Obtaining information from all perspectives allows for different aspects of a person's performance in different contexts to be accounted for in decision making
The critical incident method has been mentioned as a useful tool for research and/or practice in several areas of personnel psychology. First, what is the critical incident method? Second, name two areas in which it can be applied. Third, explain how it would be used in each area.
-CIM - a collection of specific examples of effective and ineffective behaviors for a particular job. These incidents are gathered and sorted into a taxonomy with the goal of creating a model of the job -CIT can be applied to training because it can allow for systems to be designed based on knowledge of what is effective and ineffective performance for a specific job. These training programs can target certain situations that are noted in the critical incidence. -CIT can also be applied to hiring or promotion selection because it gives insight into what skills and abilities are needed to perform well in a certain job, and can be used to select the best hires
Sackett and Laczo (2003), Job and Work Analysis Describe three ways in which competency modeling differs from traditional job analysis methods.
-CM focuses on attributes rather than activities, JA focuses on both -CM can vary from a single job to an entire organization, JA is focused on a single job -CM is more prescriptive/future oriented, and JA is just descriptive
Three general job performance models were discussed in class. Name two of them and explain one similarity and one difference between them.
-Campbell and Borman/Motowidlo -Similar in that they are both concerned with task and contextual dimensions -Different in that campbell listed many specific determinants of performance (such as demonstrating effort and supervision), while Motowidlo grouped his determinants into two broad dimensions (task performance and contextual performance)
Describe 3 general types of rating errors
-Central Tendency, Leniency, Strictness Error - tendency to rate all people as average, above average, or below average, respectively -Halo effect - leniency tendency for an individual I.e. tend to rate everything about a specific individual/class of individuals well *differs from leniency effect b/c that's a tendency to rate everyone the same -Contrast effect - performance of others affects judgement of current rating target. Example: poor performance from other workers results in a higher rating for you
Campbell's model of job performance was discussed in class. What are the three aspects of motivation within that model?
-Choice to perform -Level of effort -Persistence of effort
List the three direct determinants of job performance. What is the relationship between a direct and indirect determinant? (From the Campbell model of job performance).
-Declarative knowledge (DK) - knowledge, facts, information, and "knowing what to do" -Procedural Knowledge (PK) - Skill or knowing how to do a particular job -Motivation (M) - effort choices directed toward a goal with three components (choice to perform, level of effort, persistence of effort) -Performance on a particular component of a job is the direct function of these three determinants.. However, these direct determinants can be influenced by other indirect determinants, such as personality, interests, education, etc. (the indirect determinants affect the direct determinants which in turn affects performance)
What was Thurstone's multiple intelligences theory of intelligence? (be specific)
-Factor analytic research identified a set of 7 group factors: primary abilities theory -There was a recognition that both the general factor and group factors needed to be included in a comprehensive model of abilities (opposite of Spearman's), but he viewed group factors as most important -Thurstone identified intelligence as a cluster of abilities and proposed 7 group factors of intelligence that he coined as "primary mental abiltiies"
List three of the conditions for effective feedback?
-Feedback must be focused on specific aspects of performance or performance determinants -Feedback should be informative -It should not focus on the general outcome. This directs attention away from the task and toward ego-centered self evaluations.
What is the difference between a halo rating error and a leniency rating error?
-Halo - tendency to be lenient in rating an individual -Leniency rating error - tendency to rate all people the same (favorably)
Borman and Motowidlo proposed a two dimensional model of job performance. One dimension is task performance and the other is contextual performance. Given three examples of contextual performance.
-Helping and cooperating with others -Volunteering to carry out task activities that are not formally a part of the job -Following organizational rules and procedures even when personally inconvenient
For what dimensions of performance does cognitive ability tend to be a superior predictor? For what dimensions does cognitive ability tend to be less effective (although still positive)? Cite evidence.
-High predictor: skill, knowledge, task-based (grades) -Low predictor: motivationally determined outcomes, avoiding counterproductive work behaviors (whether they finish a degree) -Evidence: Figure from Kuncel & Beatty (2013) Data from McHenry et al. (1990)
Let's assume we have hired an analyst to do a job analysis based on her observations of our mid level managers. At the same time we have also obtained information from questionnaires we handed out to senior management characterizing the tasks of the mid-level managers. What might we learn from the questionnaire that we might not have learned from the observations? What might we learn from the observations that we might not have learned from the questionnaire?
-In the questionnaire, we might learn more standardized information to get a basic job description -In the observation, we might learn things about the job that are often looked over or forgotten because they are seemingly small tasks (incumbent doesn't think about it). -We also might observe biases that incumbents might not have been honest about or aware of in the questionnaire
What are the 4 general goals of recruiting?
-Increase the number of applicants. More choice on who to hire generally gives better results -Increase the proportion of applicants with the necessary KSAs qualifications = more qualified people -Increase other characteristics of the applicant group to achieve diversity or other organizational goals -Convince desirable applicants that your company is a great place to work
What 3 characteristics of performance appraisal systems are associated with greater legal defensibility?
-Job analysis is used in development of system -Evaluators have written instructions -Appraisal results reviewed with employees
Kuncel and Beatty (2013), Thinking at Work paper. Differentiate linear, asymptotic, and curvilinear relationships in terms of the practical interpretations we would draw from the relationships (you are welcome to draw graphs to aid your answer). Which of the three best characterizes the relationship between cognitive ability and performance at school/work?
-Linear - more is always better, maximize that characteristic, best characterizes the relationship between cognitive ability and performance -Asymptotic - all you need is a cutoff - beyond that point, more is irrelevant/functionally equivalent -Curvilinear - range beyond which people get worse - need to know where that point is, wouldn't want someone who is too neat/tidy that it would get in the way
What are the three conditions that allow for the greatest number of correct decisions for personnel selection?
-Most people are acceptable to begin with (high base rate) -We can be very picky (low selection ratio) -There is a good predictor of future performance (strong correlation called predictive validity)
What are the 4 "key aspects" of performance appraisals? Given an example of each.
-Objective - no rating errors like halo effect, whoever is conducting the study is unbiased or the study has the ability to be conducted blindly (assign each employee a number) -Measureable - a test can be given to measure knowledge or ability in a certain area -Clearly understood - standards are laid out simply and clearly so that there is no room for misunderstanding -Ideally: standards are communicated to employees and are explainable by the manager
Compare and contrast the Old School view of job performance with the exciting new worldview of job performance. Why is this important?
-Old school view: there must be an "ultimate criterion" of job performance (single measure) -New worldview: job performance is not one thing; it's multidimensional. To adequately study and measure job performance we must consider many dimensions. -Importance: this is important because it moderates the relationship between different variables predicting it. Some are much better than others. It also gives a framework for evaluating training programs.
Differentiate "performance" from "effectiveness" and "productivity" according to the Campbell model definitions. Use examples if you think they will aid your definition.
-Performance - behavior that is under the direct control of the individual that is directed toward organizational goals (picking up the phone and calling potential clients to make a sale) -Effectiveness - an evaluation or outcome of the results of performance (sales volume) -Productivity - ratio of effectiveness to the cost of achieving that level of effectiveness (sales volume/overhead costs)
Job analysis is a broad set of technologies that differ in many ways. Describe three different dimensions along which job analyses can differ.
-Purpose of data collection (performance evaluation, entry level selection/hiring, training program development) -Type of information collected (task vs. KSAO) -Method of data collection (interview, observation, questionnaire)
What are the features of a good theory?
-Repeatable, observable, and testable -Good theories may also use clearly defined constructs and operationalization
What is a "retranslation" and why would we do one?
-Retranlsation involves new SME's (subject matter experts) sorting the incidents collected into an existing model from which the incidents were previously sorted by another group of SME's. -The purpose of this is to see if there is consistency in the way the incidences are sorted into the model, or if the model needs to be altered. A new set of SME's gives a new and unbiased look at the incidents.
Why do we often do correlational studies in psychology rather than true experiments?
-Running a true experiment may be impossible (i.e. can't manipulate social class) or unethical -What we are studying is naturally occurring, so can't do an experiment -Correlational studies are best for studying multiple factors at once and job performance is multivariable
Wernimont & Campbell (1968), signs-samples-criteria paper. Describe the differences between signs, samples, and criteria, and provide an example of each.
-Signs - behaviors in repertoire (ACT score because it is used as a predictor of how a student can do in college academically, but it is not an actual measurement of how a student will do, just related to the behavior) -Samples - predictor that comes as close as possible to measuring a behavior (i.e. a simulation. An internship would be an actual measure of how someone would do in that job) -Criteria - the actual performance, best predictor because it's actual performance/outcomes
What are the three major pieces of information we attempt to gather in a critical incident? Why do we gather these three?
-Situation: context for the incident -Behavior or action: actual behavior the worker exhibited during the incident -Outcome: consequence of the behavior *Important to gather all 3 in order to make a model of the job (excellent source of data for obtaining rare, but important events)
Describe two advantages of effective recruiting. That is, give two examples of why it is worth spending some time on effective recruiting.
-Strategic goals like diversity -More people will apply which helps with selection ratio, leading to more choice
In what situation might we want to do a task job analysis? a KSA job analysis? Give an example for each.
-Task - situation where management is deciding whether or not to hire a part-time position to assist another full-time position. This analysis could be used to identify how someone's time is divided up among tasks, and if a part-time position could help to alleviate their time spent working on frequent but minor tasks that take away from more major tasks. -KSA job analysis - situation where someone is deciding whether or not to launch a career in a specific field. This analysis could be used to identify what knowledge, skill, abilities and other background information is required for a certain job, and could help someone decide if they would be a good fit for this career choice.
What is the RIASEC? List the names of each of the 6 elements. Why is this important for I/O psychology?
-The RIASEC is a career classification system that classifies people on the basis of 6 interest areas: realistic, investigative, artistic, social, enterprising, and conventional -RIASEC is important because personal interests have important relationships with people's vocational choices, satisfaction, and job performance
Kuncel and Beatty (2013), Thinking at Work paper. They specify the situations where a specific measure of ability is more likely to demonstrate stronger predictive power than a general measure. List and describe three of these situations.
-The ability measure captures critical job knowledge and skill - performance on the job or in training is particularly reliant on the content domain sampled by the ability measure -Compensatory knowledge cannot be applied to permit performance - there is only one way to do the job correctly, and other knowledge will not get one around this fact -The ability measure captures knowledge that is necessary to permit additionally learning after hiring - if learning advanced statistical techniques is needed after hiring, those without a good working knowledge of algebra will be in trouble on Day 1
What is the evidence for the validity and generalizability of general mental ability?
-There is a positive relationship between cognitive ability scores and job performance -Mental ability is moderated by complexity and dimensions of job performance
Why might differentiating between Task and Contextual performance be important?
-They can specify on performance appraisals or for when you are hiring someone for a job and need to know which is more important --Personality is a better predictor of contextual performance --Experience is a better predictor of task performance
What are the three most basic types of performance appraisals?
-Trait based: evaluations of a person's internal characteristics --Personality, attitude, interests, general abilities -Behavior based: evaluations of the frequency or quality of a person's behaviors --Acts with integrity, presents material clearly -Results based: performance outcomes --Volume of sales, number of patents
Describe three pieces of information a job analysis can yield.
-What tasks are done on the job -The importance and frequency of different tasks for job -Identifies specific knowledge, skill, and abilities necessary for competent performance on the different tasks that comprise a job
Is cognitive ability linearly related to job performance? What evidence supports your statement?
-Yes, Coward and Sackett tested the relationship between intelligence and job performance over a substantial sample size and found overwhelming support for a linear relationship between ability and job performance -Found no evidence of this nonlinearity or "good enough" level (the graph didn't level off but was rather linear)
What is adaptive performance? List four aspects (or sub-dimensions) of adaptive performance that have been proposed.
Adaptive performance is the ability to continue working successfully despite changing surroundings either with creative/flexible behaviors or by learning --Handling emergencies --Handling work stress --Solving problems creatively --Dealing with unpredictable work situations
Excluding considerations of cost, list one advantage and one disadvantage of using questionnaires to gather job analysis data.
Advantage - can obtain more standardized information Disadvantage - Less likely to make employees feel important
What is a major advantage of a multiple hurdle selection system? What is a disadvantage of a multiple hurdle selection system?
Advantage - evaluate/weeds out large numbers of applicants in a cost/time effective manner Disadvantage - might result in the elimination of outstanding candidates who are excellent in all areas except for one (no second chances)
What is an advantage and a disadvantage of the critical incident method as a job analysis technique?
Advantage - obtains rare but important events Disadvantage - can be very time consuming and labor intensive
List two advantages and two disadvantages of using observation to gather job analysis data.
Advantages -Avoids incumbent attempts to bias process -May witness aspects of the job incumbents do not think about Disadvantages -Less effective for some complex jobs because behaviors are difficult to observe -Requires considerable investment of time
List two advantages and two disadvantages of using interviews to gather job analysis data.
Advantages -Easy to follow up on questions -Good for obtaining great deals of detail Disadvantages -Can be very time consuming (time = $) -Very dependent on interviewer skill
A study finds a zero correlation between two variables based on a large and properly executed study. What can we conclude? What can't we conclude?
Can conclude that there is no linear relationship between the two variables. We can not conclude that there is no causation because correlation can not determine whether one variable caused another to occur.
Name the four elements in the taxonomy of human abilities describe by Fleishman and colleagues. Given an example of each.
Cognitive: intelligence Psychomotor: digital movements Perceptual: visual acuity Physical: strength
What are two major moderators of the relationship between cognitive ability and job performance? Explain why one of them occurs?
Complexity - more complex jobs require more cognitive ability -Occurs because jobs tend to differ in the level of complexity of the data, people, and things elements of tasks that workers need to accomplish Dimensions (personality and interest)
Compare and contrast the advantages and disadvantages of the concurrent and predictive validation and quasi-predictive strategies.
Concurrent - use the selection system on our current group of job incumbents and then obtain a measure of their job performance -Pros: --Convenient: workers are already there --Low risk: we are not making decisions using the unproven selection system -Cons: --Unrealistic: job incumbents may behave differently than job applicants because they ---Already know the job ---Have little to lose --Not predictive: data gatherers at same point in time Predictive - use selection system on a group of applicants, make no decisions using the system, randomly select people for the job, then measure performance -Pro: --Powerful: we can make strong claims about the validity because there is no criterion contamination --Low risk: if the system doesn't work or is biased no harm done -Con: --Missed opportunities: if the system does work we may have made some bad hires --Not practical: organizations hire us to help them make decisions, not put data in drawers Quasi-predictive -Pros: --Potential gains: if we think the system works we can reap its benefits right away --Practical: don't have to waste incumbents time -Cons: --Criterion contamination: predictor performance is known, this may bias evaluations of performance --Risk: may wind up making bad decisions and/or decisions that get the company sued
Turnover can be functional or dysfunctional for an organization. List two costs to the organization and one scenario when turnover can be functional.
Costs: (1) recruiting new applicants and (2) training new employees An example of functional turnover is when poor performers leave, allowing better performers to be hired
Describe the two types of incorrect decisions. Create a simple selection diagram with a predictor of moderate to strong validity and identify their locations in it.
False negative - when you don't hire someone who could have been a great worker False positive - when you hire someone who turns out to be a bad worker
Draw and label the scenario where there is a predictor with a strong validity. Draw in the selection ratio and acceptable performance base rate in such a way to create the situation where there are no false positives.
Decrease the selection ratio so that less people are hired, and so by using the top down method, only the very few best would be hired (decrease from .5 to .1) ?
Describe 2 of the decisions made by potential job applicants that can affect the performance of a selection system?
How to perform - this will affect the scores and who offers are given to Accept/decline an offer - affects who is hired
If we wanted to create a situation in which there were no false negatives (assuming a predictor that has a strong validity) what, in words, would we have to do to attain this situation. You may supplement your description with a picture.
Increase the base rate of what is acceptable performance (horizontal line), increase from .5 to .9 ?
Job analyses provide useful information. The results of a job analysis can be used to inform other areas of practice. One of them is personnel selection. Describe how the information from a job analysis can be used for this area of practice.
Information from a job analysis can be used for personnel selection by identifying which attributes and skills are necessary to perform the job successfully. This can be used with assessment tools to evaluate the potential employee for the position
What are the two different kinds of participation in performance appraisals and what influence do they appear to have on the person being evaluated?
Instrumental - employee can influence outcome of appraisal Value-expressive - employee can express opinions about procedure (no guarantee of influence)
There are two major types of job analysis. They are the worker oriented and job oriented approaches. Give two examples of the types of information gathered by each approach. (That is, four examples, 2 for each approach).
Job oriented -What tasks are done on the job -How frequently these tasks are performed Worker oriented -What knowledge, skills, and abilities are requirements for a job -Level of proficiency workers need for various KSA's
List four desirable characteristics we would want to have in a selection system.
Maximally valid - gives the strongest possible prediction of job performance Unbiased Cheap/inexpensive Quick to gather/efficient
Robinson and Bennett (1995) came up with a typology of workplace deviance. What are the two dimensions? Also, describe one phase of their study.
Minor vs. serious deviance -Minor: not serious, not unethical, not harmful to the company, not harmful to individuals -Serious: unethical, harmful to the company and individuals Interpersonal vs. organizational: -Interpersonal: not harmful to organization, harmful to individuals -Organizational: not harmful the individuals, harmful to the organization Phase 1 -Asked 70 individuals to describe two incidents of deviant behavior -Edited to create pool of 45 behaviors
Name two things that would have a negative correlation with each other and name two things that would have a positive correlation.
Negative -Amount of time spent studying, amount of sleep you get -Hours spent on your phone and the phone's battery percentage Positive -# of minutes running and # of calories burned -hours worked and money earned
What are the reasons that some experts have called for the end of the traditional performance appraisal?
Not timely (immediate feedback better) Prone to judgement bias (e.g. recency) Not flexible for shifting goals (poor goal setting) May not be perceived as fair (damage to satisfaction)
What typically determines the selection ratio?
Number of people hired and number of applicants
What are the three major pieces of information we attempt to gather in a critical incident? What other useful information could we collect?
Other useful information - frequency of incident, importance/weight of incident, organizational impact
Borman and Motowidlo proposed a model of job performance. List and describe the dimensions in their model.
Proposed that performance could be conceptualized in terms of two broad dimensions: task performance (actual job / core requirements) and contextual performance (going above and beyond basic job description). They argued that contextual performance was often overlooked and even ignored when considering the measurement of job performance.
What are the four dimensions along which cognitive ability tests differ?
Purpose Domain covers: topic Breadth Recency of learning
List and describe two of the three traditional validation strategies discussed in class.
Quasi-predictive (most common) -Use the new selection system to make hiring decisions -Examine the correlation between the system and subsequent job performance for those selected by the system -If we get a positive correlation between the selection system and subsequent performance we have some evidence that the system predicts Retrospective -We collect information about a group of job incumbents (holders) -We happen to have old information about how people performed sometime in the past -We then examine the correlation between the selection system and performance that happened earlier in time (in the past) -We have no information about how other non-incumbents would have done -We have no information about how people have changed since the information was collected about their performance in the past
What is a realistic job preview? What does research suggest they tend to accomplish?
RJP gives a potential applicant a realistic sense of the nature of the job, allowing them to decide if they would be a good fit. Research suggests that RJP may actually lower initial job expectations, increase self self-selection, and increase job satisfaction/performance.
Psychologists have often assumed that raters will do their best to give accurate assessments during a performance appraisal. What has research found to be true at least some of the time. Give details.
Raters can sometimes be focused on certain political or personal goals and have little interest in being accurate in an absolute sense
Recruiting does more than let people know a job is available. Describe what other effect recruiting has on potential applicants.
Recruiting gives candidates a perception of the organization and influences applicants' willingness to apply and accept jobs
What are the causes and consequences of restriction of range?
Restriction of range - when the sample we are studying has less variability than the population of interest, so there may be weaker effects / relationships observed (if one exists)
Robinson and Bennett (1995) came up with a typology of workplace deviance. What are the two dimensions? Give an example of a counterproductive work behavior that might fall in each of the 4 quadrants.
Severe organizational: embezzlement Minor organizational: employee taking a slightly longer break Severe interpersonal: violence or sexual harassment Minor interpersonal: mildly rude to another employee
Give 4 reasons for conducting a performance appraisal.
Termination - documenting reasons Feedback for development Downsizing - deciding who to keep Promotion - identifying people to advance in the future
What can we learn from a multi-rater feedback system that we would not obtain from a traditional performance appraisal (15 minute chat with the boss).
We can learn about other aspects of a person's performance that the boss doesn't see. The evaluation may also be more reliable because of there being more perspectives
Assume I have a powerful selection system (r = .65). I am in a situation where the base rate of acceptable performance is very high. I am also in a situation where I will need to hire 90% of all applicants. Characterize the kinds of gains that I will get with my selection system. Draw the selection situation with the base rate and selection ratio line.
We have to hire essentially everyone who comes through the door and we may have a couple "wrong decisions" but everyone is mostly qualified/would be fine to begin with
Assume that you are in a hiring situation where you have a large number of job applicants for a small number of positions. Assume that the base rate of acceptable performance is right around .50. Given this scenario, what would have to be true for the selection system to result in an equal number of correct and incorrect decisions. Draw a diagram to illustrate your point.
We would have a 0 correlation, no increasing linear relationship. It would be a square or a perfect circle When it's random, r = 0, there is no way of sorting out who will do well (draw a circle)
Describe 2 of the decisions made by the hiring organization that can affect the performance of a selection system?
What predictors to use - this will affect if someone applies, depending on if they think they have a shot Who to give an offer to - this will affect who can accept and ultimately who is hired
