Selection and Recruitment
Why is a test neither valid nor invalid? Explain.
A test is only as valid (or invalid) as the context for which it is designed to be used in. The validity of a test is dependent on the quality of the data it collects (error measurement), the inferences that are drawn from the test, and the veracity of the construct it intends to measure
Why are there so many ways to estimate reliability?
there are different sources of error variance that can impact tests. Source of error can only be captured in specific ways; therefore, must use a specific estimate of reliability appropriate to examine a specific potential source of error variance (e.g., rater error, items, construct change over time).
criterion
what is meant by employee success on the job Defined by thoroughly studying the jobs for which a selection system is being developed to determine how to measure job success. Most frequently used - supervisory ratings of work performance. Must be important to the job and must be appropriately measured
What basic principles should an organization follow to develop a selection program that is both legally defensible and ensure hiring the most qualified applicants?
• Perform a job analysis and define the essential job functions • Have a thorough understanding of legal policies for selection decisions, such as: Civil Rights Act, ADA, ADEA, Uniform Guidelines • Understand accommodations that may be necessary • Identify if your selection process has disparate or adverse impact • Periodically reassess selection procedures to determine if selection requirement are still appropriate • Be familiar with and use validation processes • Document everything
Name the factors which indicate an activity is essential to the job, according to the ADA.
• The job exists to perform the activity. • Only a limited number of other employees are available to perform the activity, or the activity can be distributed among a limited number of employees. • The activity is highly specialized and the person in the position is hired for the special expertise or ability to perform it.
Describe (verbally) how each of the following reliability estimates is calculated/correlated? test-retest, parallel or equivalent form test, internal consistency, interrater reliability
• test-retest: have people take a test, wait a time interval, and then take test again and look at correlation between scores: people should be at the same level of a construct overtime • Using two forms of measurement - each form of measure has different items, but the questions assess the attribute being measured in the same way • internal consistency: look at the correlation between responses on items that are believed to assess the same construct (want items in same test, or subtest, to be highly correlated) • interrater reliability: need multiple raters assessing same individuals; raters assessing same construct on based on same criteria should provide similar ratings (look at percentage agreement or correlation between raters' scores)
Explain how criterion contamination could affect validity coefficients through either spurious inflation or deflation of the "true"validity coefficients.
■ Occurs when scores on a criterion are influenced by variables other than the predictor ■ Can lower or raise the validity coefficient ■ Classic example: total dollar sales used in judging salesperson performance; but variables like geographic location contaminate data Impacts individual performance: How you measure your criterion is just as important as how you measure your predictors.
How do you conduct a job analysis when a job does not exist but is being created?
○ Analyze similar jobs to identify current tasks and WRC's that might be applicable (research existing jobs) Subject matter experts knowledgeable about the job are assembled in a workshop to discuss how future issues are likely to affect the job ○ Information on expected future tasks and WRCs is collected from individuals knowledgeable about these expected job changes
What can companies hope to accomplish with recruiting?
● Goal is to attract as many qualified people as possible with as little cost as possible ○ More qualified applicants in pool → better selection decision Establish a reputation for employee standards
What should be the goals of any job analysis conducted for selection purposes?
● Identify employee specifications (KSAs) or WRCs necessary for success on a job ● Select or develop selection procedures that assess these important application WRCs to forecast those job candidates who are likely to succeed on the job ● Develop criteria or standards of job performance that represent employee job success. Legally defensible
Must a validity coefficient be large to be useful? Explain.
A high validity coefficient is not enough to make a basis for using a measurement. It may not matter how high it is, it will have minimal effect on the efficiency of your selection. the measures usefulness depends on the situation in which it is employed
Describe the recruitment process and how the different stages narrow the number of individuals under consideration.
Applicant population: Every individual who could be interested in the job Applicant pool: Those who actually do apply Job candidates: Those who showed the best job performance potential through the selection procedure Hires: Those offered the job position
Identify and discuss some of the ways that recruitment on the Web can be done.
Companies can post jobs on their own website, job announcement websites such as monster.com, social networking websites, and they can screen for applicants using search functions on linkedin.com. Employee testimonials.
validity
the degree to which available evidence supports inferences made from scores on selection measures.
Why is a time interval between predictor and criterion data collection both a strength and a weakness of a predictive validation strategy?
■ Applicants, rather than incumbents, may have a higher and more realistic level of motivation when they complete the predictor measure • Differences between individuals and subsequent applicants with respect to job tenure are not a problem with the predictive approach, since the same level of job experience applies to both ■ We also won't have restriction of range (like promotions, or terminations of the incumbent pool). ■ Weakness - the time interval required to determine the validity of the measure being examined ■ Difficult to accumulate large amount of data needed to conduct an appropriate statistical analysis o Decline depends on construct. Over time, more things are subject to change at the individual difference level that will affect job performance. The greater the time interval, generally speaking, the lower the validity coefficient. o E.g. cognitive ability scores predict initial job performance (due to learning), over time motivation will better predict job performance now that people have learned the job.
Evaluate the statement, "Production Data are a preferred source of data for performance ratings, since they are usually gathered for other business purposes
Data may not be at the individual level. Consensus in sales work is that the most often used measure of sales performance, dollar sales volume, is closely related to the characteristics of the territory that is worked.
How does a broader definition of selection influence the way an organization manages its HRMS
If selection has a broader definition, organizations can differentiate between closely related steps of the HRMS. Including the relation selection has to recruiting in the definition could bridge gaps between the two steps. By doing this an organization can have consistency in its HRMS
How can an organization get the greatest amount of useful information about an applicant at the least cost?
Use application forms and resumes to pre-screen applicants ito get basic information about applicants at a small cost. Pre-screening applicants early in the process will reduce the amount of applicants that have to be interviewed face-to-face.
Why is reliability important in human resource selection?
Reliability has to do with identifying random error. Unreliability can change the rank order of test scores, or, error variance could cause the rank order to change. hire different types of people each time which is problematic because you're affected people's lives based on error variance.
An organization has been accused of disparate impact discrimination through the use of a statistical argument. Provide two alternative explanations for a statistically unbalanced work force that are not due to illegal discrimination.
1. Stock or flow statistics: compare proportions of demographic group to the result of selection decisions. Base it on the relevant labor market (RLM) 2. Base it on job performance. Clearly models that selection is due to job performance and not illegal discrimination.
Describe the steps involved in an empirical cross-validation. What is the desired outcome?
1. A large group of people on whom predictor and criterion data are available is randomly divided in 2 groups. 2. A regression equation is developed on one of the groups ("weighting groups") 3. The equation developed on the weighting group is used to predict the criterion for the other group (called the "holdout group"). 4. Predicting criterion scores are obtained for each person in the holdout group. When regression equations are applied to a new group, the predictive accuracy of the equations will almost always fall. This "shrinkage" in predictive accuracy occurs because the new group is not identical to the one on which the equations were developed. Because of the possibility of error, the equation has to test for shrinkage before their implementation in selection decision making. --desired outcome: regression equation developed on one sample to predict the scores of the other sample, then the regression is cross-validated. It will never be exact, but it could have minimal error and be appropriate for another group.
Why has job analysis received so much attention in court cases? What can we infer from court cases that will help us design legally acceptable job analysis?
Because companies wouldn't do a job analysis prior to creating job descriptions/requirements. This would limit the applicant pool and cause adverse effects. For using a selection procedure: you must link the relationship to job performance Example Griggs v. Duke powers: They required a written test/high school diploma when in reality, the job didn't require a high school diploma. There is no one job analysis that is standard. Before choosing a method, consider the possibility of going to court and choose a method that would be easily defensible.
You are consulting with an owner of several small convenience stores. She wants you to conduct a job analysis study to help develop valid selection measures for each of the positions found in the stores. Select an appropriate job analysis and write a brief, nontechnical explanation of your choice for the store owner
Combine task analysis and CIT: Identify important job tasks Identify important WRCs Show critical job task and WRC information to SMEs Rewrite the critical incident into a selection interview question Develop a key for scoring responses to the interview question
What is the correction for attenuation? Why is it used? When should it be used?
Correction for attenuation is a formula that will yield validity scores as if the data of interest were not range restricted. The correction should be performed when any form of range restriction occurs within the sample, as any form of range restriction negatively impacts validity. Criteria and predictors may have their ranges restricted, but methods for correction in selection exist only for predictors. Make sure you are reporting the correction otherwise you are misrepresenting the validity.
A male scored three points lower than a female on a selection test. The female was hired. The male filed a discrimination case on the basis of unfair use of test score for selection. What evidence would he have to provide to support his claim?
Could show that the test is unreliable enough for him to pass, on any other day, he would have scored higher than the female candidate - thus his score was within the level of acceptability based on confidence intervals around the test's reliability results. Three points out of 100 using a test with a confidence interval of +/-4 would be an example. Could also show systematic differences across males and females.
Why are criterion measurement issues as important as predictor measurement issues in selection?
IIf you cannot accurately assess the current state of an applicant then you cannot ensure how well that applicant will perform on the job. You will also be unable to defend the use of the test for selection. Criterion measurement issues are as important as predictor issues, if not more so. If we cannot accurately measure job performance itself we cannot succeed in predicting it or supporting any other predictions.
How do you conduct a job analysis for a team job in which the team member is asked to play multiple roles, be cross-trained, and perform different tasks based upon the needs of the moment?
If they are clearly defined roles, you conduct a job analysis for each role. If they are not, they you basically treat all roles as one giant job, and conduct the analysis on that. From the initial analysis (at the ground level) you determine tasks, KSAOs, WRCs, etc. Then you conduct another job analysis on individuals performing the fluid job role [composite of tasks from the first job analysis(es)], to determine if there are additional WRCs that are related to an incumbent's ability to perform the fluid role. Additional information can be gathered from O*NET, or other job databases to supplement any gaps in the analysis.
What characteristics should useful selection criteria have? What is the general conclusion regarding how violations of these characteristics will likely affect the validity coefficient?
Individualization: Measure must collect data about performance that the individual controls Relevance: Individuals are employed primarily to perform well on the critical or important parts of a job - these pasts should be included in job performance measures Measurability: Must be possible to generate a number that represents the amount or quality of the work performed Variance: Scores that are generated must have differences among them Violations of these criteria generally lead to lower, insignificant coefficient which can lead to inaccurate inferences (selection decision)
What are the similarities and differences in applicant information collected from internal and external sources? Are more accurate selection decisions made on internal candidates? Why or why not?
Internal: much smaller applicant pool, but a wealth of knowledge on the applicant is also available through job performance data External: yields a large applicant pool with a lot of information on each applicant internal hiring can typically be more accurate bc job performance data may be objectively more reliable than resumes and interviews as applicants may mold their resume and interview skills to fit the position
If selection patterns for specific jobs indicate demographic differences, what two options does the organization have?
Options 1: Cease use of the selection device in favor of something better or Develop assessments/selection procedures that equally valid at predicting performance but cause significantly less (if any) adverse impact option 2: Defend selection practices via evidence that it is valid (even if adverse impact has occurred). Involves conducting a validation study to support that the selection instruments are job-related and best predictor of performance with a high level of content validity
One writer describes selection as "hypothesis testing". Explain
Robert Guion describes the initial stages of developing a selection program as a hypothesis. The development includes collecting data on job activities/tasks and outcomes to determine the work-related characteristics needed to succeed on the job. The selection program also determines a way to measure work-related characteristics of all applicants and compare the applicants to one another. The testing portion of Guion's statement is to validate the hypothesis.
Describe, in words, what rxy = .59, p = .05 means. Discuss these separately and jointly.
Rxy is the validity coefficient for predictor x and criterion y. This validity coefficient tells us how useful our results are by demonstrating how strong or weak a relationship is. Our validity coefficient is significant, as indicated by the p=.05, which means that it is highly likely (95% chance) that this is a non-zero validity coefficient. The validity coefficient, rxy, shows that our predictor Is useful and explains some of the differences in employee work performance. We have to square the validity coefficient .59, which means 35% of the variance in job performance can be explained by scores on the sales ability inventory.
Explain why coefficient alpha estimates of reliability are not appropriate for speed tests.
Score on one item compared to score on the rest of the items on a test. That's an item total correlation. Leads to coefficient alpha. You can't compare one item to others with a speed test because unanswered questions will score a 0 and cannot be compared. Non-answers do not indicate a lack of ability, rather they indicate a lack of total speed.- preferable would be test-retest or parallel forms estimates of reliability.
What is the difference between selection and hiring?
Selection is the process of testing to determine which applicants will perform well on the job. Hiring is the process of offering a position to an applicant and then onboarding them if they accept.
How does an organization's strategy influence its selection process?
Selection research versus selection practice. most organizations do not practice evidence-based human resource management.
Why is task performance still the primary type of job performance measure? What other methods should be used?
Task performance consists of Production Data and Judgemental Data. It is still the primary type of performance measure because production data is easy to obtain and understand, and judgemental data is best when a job does not produce tangible, countable data. Other methods to are adaptive performance (how well an employee handles change), OCBs, and CWBs.
Describe in words what r2xy = .76 means. Is this "good" for a selection measure? Why or why not?
little r = correlation. Squaring r produces the coefficient of determination, which accounts for the variance between the criterion and the predictor. A .76 coefficient of determination means that 76% of the variance in job performance (y) can be explained by the selection measure (x).
What is the first step necessary to develop selection measures? Explain why it must be the first step?
The first step is performing a job analysis. a job analysis identifies the work-related characteristics (WRCs) necessary to perform a job successfully. Once the attributes have been defined, applicants can be measured against these attributes and predictions of performance can be made.
What are the most commonly used predictors and criteria for selection? Why do you think they are most commonly used?
predictors: tests, interviews, biographical data questionnaires, application blanks, and assessment centers. criteria: absenteeism, turnover, and OCB's. error rates, number of goods produced, dollar sales, and speed of performance. Most common: supervisor ratings of performance
How is the standard error of measurement used?
The standard error of measurement represents the amount of random error in a measure, with higher values indicating more error and less reliability. SEM is a necessary component when expressing reliability of a measure. SEM can also be used to identify if discrepancies exist between scores (e.g., is the difference between applicant scores really due to aptitude, or chance?).
What is the interaction among selection, recruitment, compensation, and training?
The type of training depends on the skills and abilities of the applicants selected. Compensation is based off the skills of the selected applicant. Recruitment and selection deal with placing people into jobs
reliability
the degree of dependability, consistency, or stability of scores on a measure used in selection research (either predictors or criteria). Determined by the degree of consistency between two sets of scores on the same measure
You have two hours to find an existing selection measure to use for retail sales positions. Where will you look? What is the most likely source for finding such a measure?
Under the time constraint, an online search will be the best way to find an existing measure (as opposed to books, journals, publisher catalogs, or professional associations).
Explain the debate over the situational specificity hypothesis versus the validity generalization hypothesis.
Using the situational specificity hypothesis as a basis for studies can be more legally defensible because they are specific to both the job and the organization. Validity generalization less legally defensible because they are prone to statistical errors. You can't always do a high quality validation study so you'll need to do a validity generalization. (professor at ucf example)
What is the likely impact on rxy of each of the following factors in a predictive validation study?
a. Restriction of range impacts job performance when dealing with tenure. What happens if there's variability of job tenure? More variability. b. Do the employees look like the applicants? Unrepresentative employees deal with range restriction.
Why is effective measurement important to human resource selection?
allows selection managers to systematically apply rules which accurately represent people's attributes/traits/KSAOs. • It yields differences among applicant's scores that are more likely to be caused by individual differences rather than test-taking conditions, administration, or scoring flaws • Measurement of an attribute among job candidates allows us to make accurate/valid predictions about their job performance • Measurement/scoring rules also enable companies to standardize selection processes
predictor
employee attributes (WRCs) identified as being important for predicting job success. Must be relevant to the job and they must be appropriate ways to measure the employee WRCs identified as representing job success
