Exam 3

Multiple cutoffs

Assumptions A nonlinear relationship exists among the predictors and the criterion—a minimum amount of each predictor attribute is necessary. Predictors are not compensatory—lack or deficiency in any one predictor attribute cannot be compensated for by having more of another. Advantages Candidates in pool are all minimally qualified. Easy to explain to managers Disadvantages Requires assessing all applicants on all predictors Identifies only the minimally-qualified candidates Best Uses When physical abilities are essential for performance When a minimum level of performance on each predictor is necessary to perform job safely (e.g., aircraft takeoffs and landings)

Multiple hurdle

Assumptions Each applicant must sequentially meet the minimum cutoff or hurdle for each predictor before going to the next predictor. Failure to pass on the next predictor eliminates the applicant from consideration. An alternative procedure is to calculate a composite multiple regression for each applicant at each successive hurdle. Whenever the probability of success for an applicant drops below some set value, the applicant is rejected. Advantages All candidates are minimally qualified on predictors Do not have to apply all predictors to all candidates "Funnel" Disadvantages Establishing validity for each predictor Time required to apply predictors sequentially Best Uses When extensive post-hiring training is required When an essential KSA must be present at hiring With numerous applicants and expensive procedures

Bootstrapping

Bootstrapping is based on the assumption that the people can make sound judgments, they are not able to articulate how they did it Through regression analysis infer the weights used by the decision maker to arrive at ranking

Core Self Evaluation (CSE) (not a key word)

CSEs consist of 4 frequently studied traits: Self-esteem Generalized efficacy Locus of control Emotional stability Those who are confident, feel good about themselves, control their anxiety, and see work as interesting, are more productive at work CSE a predictor of success across many jobs

Fitness-for-duty tests

Compares an individual's response times and accuracy while operating a computer game to the individual's previous scores on the game.

Interview

Conscientiousness is the single most frequently measured WRC in the interview Interviewer ratings are moderately related to job performance The Appropriate Use of the Interview Limit assessment of personality to traits needed for job through job analysis Behavior description interview appropriate

Recommendations for Use

Define Personality Traits in Terms of Job Behaviors It is necessary to have a job-related basis for measuring personality Job analysis information can identify job tasks, interactions, working conditions, etc. From these data, an HR specialist should be able to infer personality traits needed Define Work Effectiveness as Retaining Productive Employees Focus on the Five-Factor Model to comprehensively measure an applicant's personality, ensuring predictive validity Other researchers recommend using facets to identify a more specific personality trait Relevance of the Work Context Both personality and the situation determine behavior The Appropriateness of the Selection Instrument Use only tests with enough developmental information to show that it measures the personality traits it is said to assess Review reliability, item statistics, etc. Some tests sold expensive but useless Can respondent learn "correct" responses? Better to use instrument for which appropriate responses are not apparent Source of Measurement Observer ratings predict performance better than self-reports; use the interview, letters of reference, and headhunters who talk to former associates Incremental Validity over Other Selection Measures Superior prediction results when personality is combined with other predictors Trade-offs between Validity and Disparate Impact Personality tests produce few large differences between demographic groups, avoiding disparate impact; continue to monitor results Faking "Good" during Selection Faking good does not substantially alter predictive validity; but including warnings in the instructions that faking on the test can be detected tends to discourage it Cheating during Selection -We recommend that even web tests be proctored to enhance standardization of the testing procedures and to secure test security Using "At-Work" Personality Tests -Contextualize personality items so that they measure the test-taker's behavior at work; leads to superior predictive validity Realizing Prediction Gains over Time -Benefits will improve with employee's age We have drawn on personality psychology to help you use personality data appropriately; this requires understanding: 1. traits vary greatly in the extent to which they influence behavior 2. the situation also has an important influence on a person's behavior 3. measuring personality is challenging; try to reduce "faking good" or cheating on Web exams

Behavioral dimensions

Dimensions Begins with a job analysis to identify clusters of job activities important to the job Each cluster specific, observable, comprising job tasks that are related in some way (dimensions) These measured by the AC devices Dimension Definition Oral Communication -- Effectively expressing oneself in individual or group situations (includes gestures and nonverbal communications) Planning and Organizing -- Establishing a course of action for self or others in order to accomplish a specific goal; planning proper assignments of personnel and appropriate allocation of resources Delegation Utilizing subordinates effectively; allocating decision making and other responsibilities to the appropriate subordinates Control Establishing procedures for monitoring or regulating the processes, tasks, or activities of subordinates; monitoring and regulating job activities and responsibilities; taking action to monitor the results of delegated assignments or projects Decisiveness Expressing a readiness to make decisions, render judgments, take action, or commit oneself Initiative Actively attempting to influence events to achieve goals; showing self-starting actions rather than passive acceptance. Taking action to achieve goals beyond those called for; originating action Tolerance for Stress Maintaining a stable performance under pressure or opposition Adaptability Maintaining effectiveness in varying environments, with various tasks, responsibilities, or people Tenacity Staying with a position or plan of action until the desired objective is achieved or is no longer reasonably attainable Traditional Assessment Devices AT&T ACs often used various traditional tests and interviews, including mental ability, projective personality, and paper-and-pencil personality tests Expensive to score; rarely used now But ACs commonly use an in-depth interview (the background interview) Interview structured, focused on previous job behaviors; formal scoring system; effective

Drug tests

Drug & alcohol use of concern to employers since 1960's Drug testing increasingly common, especially for those with safety sensitive jobs or government contracts Drug use: negatively associated with job performance Positively correlated with accidents, injuries, absences, involuntary turnover, job-withdrawal behaviors Any usage information gathered from self-reporting may be understated One 2006 study found: 1. Marijuana used by 11.3% of workforce 2. Cocaine used by 1.0% of workforce 3. One of four psychotherapeutic drugs used by 4.9% of the workforce 4. Marijuana use had caused impairment to10.6% of workforce 5. Cocaine use had impaired 0.9% of workforce 6. One of four psychotherapeutic drugs impaired 2.2% Paper-and-Pencil Tests Resemble paper-and-pencil integrity tests Limited by prohibition on self-incrimination Hair Analysis Requires screening and confirmation tests Allows for longer detection periods Paper-and-Pencil Tests Overt types ask such questions as: Do you think it is okay for workers to use "soft" drugs at work if it does not cause poor job performance? In past 6 months, how often have you used marijuana at work? In past 6 months, have you brought cocaine to work even though you did not use it at work? Almost no research on validity of this test Chemical tests should be completely accurate False positives may occur in an initial test, but should be eliminated by follow-up test But these tests have limitations: Positive result means only presence of drug above threshold level set for detection Does not determine how much of the drug was used, how frequently, how long ago, the circumstances or the level of impairment in performance People differ in absorption, metabolism and tolerance Not correct to assume that presence of drug means it affected performance

Contingent predictors

Emotional intelligence (EI) A broad construct composed of attributes beyond those of personality, and includes nearly all non-cognitive predictors, and some that appear to be cognitive measures. ??

Banding

Establishing Bands Two methods used to determine whether scores differ, each requiring calculation: (a) standard error of measurement method (b) standard error of differences Fixed and Sliding Bands In fixed bands (top-score referenced bands) use the top applicant score as the starting point Fixed and Sliding Bands (cont.) Sliding bands also based on top applicant's score Then the band is recalculated using the next highest score; sequential Selecting Within a Band Once bands created, other specifications used to choose people within the same band: Interpersonal skills Job experience Work performance, etc. Advantages of Banding Employer has more flexibility in hiring decisions Banding helps employer take into account factors not measured by traditional selection methods The very narrow perspective of maximum economic gain from than top-down selection ignore the social and financial impacts of not having a diverse workforce Concerns Regarding Banding Banding makes it possible to choose a larger number of minorities, but debate re whether using banding to identify equally qualified people, then choosing on the basis of minority status will withstand legal challenge But when minority status used with other specifications when selecting Problem if banding cannot be used to reduce adverse impact, but banding leads to loss of economic utility

Strategies for Selection Decision Making (not kw)

Even when an organization collects reliable and valid selection procedure data, poor implementation can undermine the entire selection system Do not combine "gut instincts" with the selection data Keep a record of your decision-making process, and audit it occasionally Which employees performed well? Which did not? Answering these questions will improve employment decision making: 1. For a specific selection situation, what are the best methods for collecting predictor information on job applicants? 2. Because we often collet applicant information using more than one predictor, how should we combine scores? 3. Once we have a total score, how should this score be sued to make selection decisions? ( Modes for Combining Predictor Information ) We all make selection decisions in our daily lives, but we sometimes make errors This is measurement error In personnel selection, decision makers collect information from job applicants using predictors that reflect one of two different measurement philosophies Once data is collected, the information must be combined to reach a selection decision When predictor data are combined with gut instincts, predictor data have been combined judgmentally When choosing selection procedures, ask: (a) should procedure be sequential or not? (b) should the decision be compensatory or not? (c)should the decision be based on ranking applicants or on banding? Assess job and nature of work performance; what determines success? Implement the strategy systematically using a mechanical strategy Often not possible for small businesses to build objective strategies Must build a systematic way of combining selection procedure data Bootstrapping is based on the assumption that tho people can make sound judgments, they are not able to articulate how they did it Through regression analysis infer the weights used by the decision maker to arrive at ranking Whatever method you choose, it is critical procedures be perceived as fair

Overt integrity tests

Examples include the Personnel Selection Inventory, the Reid Report, the Stanton Survey Research shows the typical employee-thief: (a) is more tempted to steal (b) engages in many common rationalizations for theft (c) would punish thieves less (d) often thinks about theft-related activities (e) attributes more theft to others (f) shows more inter-thief loyalty (g) is more vulnerable to peer pressure to steal

False positives

False Positives - 40 to 70% of applicants commonly fail integrity tests, and a large number of those may be incorrectly classified It may be in the interest of the publisher to avoid false negatives, so may set low cutoff score It may be in the interests of the employer to avoid false positives, so may set high cutoff score Because of implications of false positives, use of a personality inventory better than integrity test Chemical tests should be completely accurate False positives may occur in an initial test, but should be eliminated by follow-up test

False negatives

False negatives (erroneous rejections) Initially appear unacceptable but would have performed successfully if hired

False positives

False positives (erroneous acceptances) Appear acceptable but, once hired, perform poorly

Big Five personality dimensions

Five-Factor Model, five core traits: Extraversion (sociability, assertiveness, optimism) Emotional stability (calmness, confidence) Agreeableness (courtesy, flexibility, trust) Conscientiousness (responsibility, dependability) Openness to experience (intellect, culture) Each trait composed of many facets Extraversion I tend not to say what I think about things. Agreeableness I tend to trust other people. Conscientiousness I approach most of my work steadily and persistently. Stability Whenever I'm by myself, I feel vulnerable. Openness to Experience I enjoy eating in new restaurants I know nothing about

Graphology

Graphology The analysis of the handwriting of an individual in order to infer personality traits and behavioral tendencies. Graphologist The individual who performs the analysis of handwriting features: Size of letters, slant, width, zones (top, middle, and bottom), regularity of letter formation, margin, pressure, stroke, line of letters (upward, straight, downward), connections of letters, form of connection, and word and line spacing.

Hair analysis

Hair Analysis Samples of hair including the root are taken from applicants to be tested More expensive, but use rising Drug use can be detected over a longer period of time than urine testing Each inch of hair shows usage for 90 days For cannabis, 1 to 3 months

Genetic Testing (not key word)

In 2003, Human Genome Project completed mapping of 25,000 genes Specific genes linked to certain diseases For some diseases, link is strong; for others, weak These results will continue to have important implications for health care and employment Implications of genetic testing for employers when selecting among job applicants Some employers have used genetic information to predict which most likely to miss work due to illness, disease Analysis of human DNA, chromosomes, proteins or metabolites that detect genotypes, mutations, more Produces genetic information on (a) individuals (for our purposes, job applicants) (b) family members of these individuals (c) the health histories of the individuals' family members Before federal law, based on genetic information: employers denied jobs, fired existing employees insurers denied coverage based on pre-existing condition Before protective laws were passed, employers refused to hire affected applicants to avoid health insurance costs Health insurers used genetic information to deny insurance or refused to pay for treatment, calling genetic information a "pre-existing" health condition Case involving employers' unauthorized testing of black applicants for sickle cell anemia raised issue in federal government The Genetic Information Nondiscrimination Act of 2008 (GINA) Title II prohibits employers, and others from discriminating against employees & job applicants because of genetic information: "An employer may never use genetic information to make an employment decision because genetic information is not relevant to an individuals' current ability to work" But complaints filed with the EEOC Neuroscientists studying the brain have begun to examine relationships between neural circuits and development of diseases (Alzheimer's), behaviors (racism, cooperativeness) and personality characteristics (extraversion, introversion) One day, it will be possible to determine the probability that certain neural wirings are linked to specific outcomes The same concerns arise here as did with genetic testing and the human genome

Integrity

Integrity testing is useful in testing for certain undesirable work behaviors There is some disagreement regarding the strength of the relationships between integrity test scores and certain undesirable employee behaviors "Bad" behaviors are categorized as counterproductive work behaviors (CWBs)

Powerful situations

Interaction of Traits and Situations Powerful Situations Cause individuals to interpret events in the same way Create uniform expectancies of appropriate behavior Provide incentives for the performance of a behavior Require commonplace skills Weak Situations Cause individuals to not uniformly interpret events in the same way Do not create uniform expectations of desired behavior Do not offer incentives for one type of behavior Require a variety of skills

Weak situations

Interaction of Traits and Situations Powerful Situations Cause individuals to interpret events in the same way Create uniform expectancies of appropriate behavior Provide incentives for the performance of a behavior Require commonplace skills Weak Situations Cause individuals to not uniformly interpret events in the same way Do not create uniform expectations of desired behavior Do not offer incentives for one type of behavior Require a variety of skills

Proactive personality

Is a dispositional approach toward taking initiative at work and effecting environmental changes. Has been shown to predict salary, the rate of promotions, use of transformational leadership, entrepreneurial intentions, and job performance. Has been shown to relate to conscientiousness and extraversion, and to a lesser extent, openness to experience and emotional stability. Represents a larger construct than the Big Five

False negatives

It may be in the interest of the publisher to avoid false negatives, so may set low cutoff score Major drawback to polygraph is frequency of false-positive results (Example: 68%) There are also false negatives

Liability

Limitations Reactions other than guilt can trigger an emotional response There are a variety of countermeasures examinees can use to avoid detection: Tensing muscles, biting tongue, flexing toes, etc. Mental countermeasures like relaxation Major drawback to polygraph is frequency of false-positive results (Example: 68%) There are also false negatives

Cutoff scores

Look at Angoff Method Cutoff scores are not required by legal or professional guidelines; thus, first decide if a cutoff score is necessary. There is not one best method of setting cutoff scores for all situations. If a cutoff score is to be used for setting a minimum score requirement, begin with a job analysis that identifies levels of proficiency on essential knowledge, skills, abilities, and other characteristics. (The Angoff method is one that can be used.) If judgmental methods are used (such as the Angoff method), include a 10 to 20 percent sample of subject matter experts (SMEs) representative of the race, gender, shift, etc. of the employee (or supervisor) group. Representative experience of SMEs on the job under study is a most critical consideration in choosing SMEs. If job incumbents are used to develop the cutoff score to be used with job applicants, consider setting the cutoff score one standard error of measurement below incumbents' average score on the selection procedure. Set cutoff scores high enough to ensure that at least minimum standards of job performance are achieved.

OAR Decision Rule (not kw)

Look at dimensions - most important - versus ready now/ready with development/not ready

Employee Polygraph Protection Act (1988)

Made it illegal in selection (with the exception of specific circumstances) to: Directly or indirectly require, request, suggest, or cause any employee to take or submit to any lie detector test to render a diagnostic opinion about the honesty of an individual Use, accept, refer to, or inquire about the results of any lie detector test of any job applicant or current employee Discharge, discipline, discriminate against, or deny employment or promotion to (or threaten to take such adverse action against) any prospective or current employee who refuses, declines, or fails to take or submit to a lie detector test

Legal Issues Drug Testing

Major questions about drug testing: 1. it represents an invasion of privacy True, but only an issue if the employer is a governmental institution 2. it constitutes unreasonable search & seizure Limited application to private employers; but does apply to governmental institutions 3. is a violation of due process If authorized, professionally conducted testing not a violation 4. drug users are protected under the ADA ADA protects former drug users in rehabilitation, but excludes those currently using illegal drugs 5. Testing may violate the Civil Rights Act If unequal treatment occurs; also possible adverse impact 6. Testing my violate the National Labor Relations Act Under NLRA, collective bargaining agreement between employees and employer governs

Methods

Measurement difficult because we are trying to quantify intangible constructs through use of inferred data Most commonly used methods: Self-report inventories (produce insignificant or lower validity coefficients than other source) Projective Techniques Judgment of interviewers Judgment of other observers (especially former coworkers or supervisors) Inventories These are written responses of a person Many such measures, differing substantially We discuss two major types: Self-report inventories or questionnaires Projective techniques Research shows that contextualized personality items predict job performance better Self-Report Questionnaires A series of brief items asking respondents to use a multiple-choice format to indicate personal information about thoughts, emotions, past experiences (Ex: "I am happy") Answer choices: agree, undecided, disagree Five-Factor Model Most commonly used are NEO-Personality Inventory, and Personality Characteristics Inventory (PCI) PCI has 150 multiple choice items using Five-Factor inventories; (Answers: agree, undecided, disagree) Other Self-Report Personality Inventories Others available, not all of which use the Five-Factor Model Forced-Choice Inventories Test-takers must choose the most liked item of 2-4 equally desirable items; faking reduced A few studies yield predictive validities as good as or better than Likert-scale self-report scales Issues to be resolved, but another option

Mechanical information collection

Mechanical versus Judgmental Mechanical combination increases the accuracy of prediction when weighting predictor scores. Accuracy of statistical models increases as data on more applicants are added to available data. Systematic and thorough judgmental decision makers can do only as well as a statistical model. Decision makers add error when judgmentally combining both judgmental data and objective data. Statistical models make an allowance for overconfidence errors and reduce the impact of individual biases on decision outcomes.

Multiple regression

Multiple regression shows the maximum linear association between two or more predictors and a criterion Applicant's scores can be entered into an equation: Look it up Because it is possible to compensate fro low scores on one predictor by high scores on another, multiple regression may be referred to as a compensatory method Multiple regression makes 2 basic assumptions: (a) predictors are linearly related to the criterion (b) the predictors are additive and can compensate for one another Multiple regression strategy has several advantages: It minimizes errors in prediction and combines predictors to yield best estimate of applicant's future performance It is very flexible, and can be modified to handle nominal data, nonlinear relationships, or both Regression equations can be constructed using the same predictors weighted differently, or different predictors Multiple regression strategy also has disadvantages: Some statistical issues difficult to resolve If a small sample size is used to determine regression weights or predictors are correlated with one another, weights will not be able from one sample to the next and standard errors of weights increase; cross validation essential If no preliminary screening, multiple regression requires assessing all applicants on all predictors, which can be costly with a large applicant pool

Unit Weighting (not kw)

Multiple-regression model called a proper linear model because predictors are identified to maximize the relationship with the criterion An improper linear model is chosen by some non-optimal means (based on a theory, job analysis results, opinions of SMEs, etc.) With sample sizes pf up to 100, unit weights and regression weights perform about equally When sample sizes are less than 75, unit weights perform better

Legal issues

No adverse impact found with use of personality tests, but two legal issues: 1. ADA prohibits pre-employment medical exams; because psychologists can use personality tests to diagnose mental health deficiencies, such tests could be seen as medical exams EEOC: If test includes scales with psychiatric disorders, likely to be a medical exam If test focuses on general traits, not a medical exam In recent case, court ruled company's use of Minnesota Multi-Phasic Personality Inventory (MMPI) which asked questions that could reveal a mental disability, test was held to be a "medical exam" under the ADA, and illegal as a pre-employment test Court noted that tests that measure honesty, integrity, etc. do not qualify as medical exams; under this ruling, Five-Factor tests would not constitute medical exams 2. Privacy is an issue for employees for public institutions, as the are protected by the U.S. Constitution in their employment Several states have constitutional privacy protection acts or a statutory right to privacy; these apply to both public and private employers In one case, a personality test asking questions about an applicant's sexual inclinations or religious views for store security positions were banned Faking in Personality Inventories Empirical evidence is contradictory as to whether candidates intentionally alter their responses to get hired Contradiction may be caused by design of research Little evidence that faking affects the predictive validity of personality inventories Self-reports may be less accurate than anticipated for reasons other than faking Web-based personality tests result in high levels of internal consistency; scores are equivalent to paper-and-pencil scores Also evidence of equivalence between proctored and unproctored examinations, but professional standards require proctoring Further innovation in progress Monitor innovations Monitor EEOC implications

Observer ratings (not kw)

Observers can rate an applicant's personality even after brief conversation, with modestly high agreement on rating by one who knows applicant Observer ratings show a persons' public self, the way we are perceived by others Self-reports reflect our perceptions of self Observer ratings enhance prediction of job performance, often twice as well as self-reporting Ratings better from those who know the subject Obtain reputation-infused ratings from supervisors or coworkers Perhaps a headhunter can visit with people who worked with the candidate A letter of reference may also served as a way to assess the candidate's personality Study: A structured letter of reference predicted job performance better than self-reporting

Oral fluid tests

Oral swab taken from inside of individual's mouth is analyzed for presence of drugs

In-basket exercise

Performance (Simulation) Tests In-Basket A paper-and-pencil test designed to replicate administrative tasks of the job under consideration. Applicants must indicate what action is to be taken regarding memos in the basket and are interviewed about their decisions TO: Mick Mount FROM: Murray Barrick RE: Your Work Performance I know that you are always open to new experiences. So am I. That is why I switch jobs and organizations about every two years. However, you have to stop being so agreeable and saying yes to everyone on the team who asks you to do something. You are doing so many different tasks that you are becoming unstable. What you have to do is pay attention to details!! For high-level job performance you have to be conscientious. You should list all the work objectives that you are trying to reach, make plans to achieve each one, and check off each plan when you complete it. Also quit trying to act like an extravert. You aren't a salesman

Leaderless group discussion (LGD)

Performance (Simulation) Tests Leaderless Group Discussion (LGD) A small group of applicants are each assigned roles and charged with resolving either a competitive or a cooperative scenario. The group must produce a written report that specifies the action to be taken by the company relative to the scenario. Problem: Your organization has decided to start an Executive Education and Development unit. The first step is for this group to choose a manager of this unit. Only current employees will be considered. Assigned: Role #1 Alan Witt, Vice President of Research Your goal is to get your assistant Roger Griffin chosen as the manager. He has worked in education and development for many years so you think he can do the job. However, he may not be favored by some of the others in the group for two reasons. Some think that he can only teach executives how to avoid turnover among their subordinates. Others suspect that he is already paid too much and would be really overpaid in the new position. You should convince others that Roger is mesomacro enough to handle the position and that money means nothing to him. You know that he would do this for the same amount that he is making now.

Work Samples

Perhaps the earliest form of simulations Ex: tests of typing and editing a business letter; Automobile repair; carpenters, masons Work sample simulations are high-fidelity simulations Test Job Motor: Machine operator - Lathe , Drill press, Tool dexterity ,Screw board, test Packaging Clerical worker - Shorthand , StenographicTyping, Mechanic - Blueprint reading , Tool identificationInstalling beltsRepair of gearboxInstalling a motorVehicle repair Tracing trouble in a complex circuit Electronics technicianInspection of electronic defectsElectronics test

Personality

Personality measures in selection are one of the most complex and rapidly changing topics in the field Personality affects success at work mostly by enhancing social engagement and work motivation It encompasses a person's relatively stable way of thinking, feeling, and acting in a variety of situations Use of personality tests is increasing Personality equated with social skill Work psychologists instead focus on traits linked to motivational work habits Two personality traits - conscientiousness and emotional stability - valid for all jobs Recently, agreeableness emerged as predictor for nearly all jobs Extraversion & openness to experience, valid for some jobs Some still believe personality not a predictor Predictive validities still low, 0.20 or lower But growing consensus: Personality characteristics can be grouped into five broad dimensions, Five-Factor Model Managers believe personality traits matter at work Recent meta-analytic data show these traits can be relevant predictors of work effectiveness But growing consensus: (cont.) Because personality not highly correlated to other selection tools, it contributes validity Little or no adverse impact; scores comparable across racial and ethnic groups Even small benefit from personality trait benefits employer every day Conscientiousness, emotional stability & agreeableness increase about 1/3 to ½ standard deviation from late 20s to early 50s

Personality-based measures (under integrity test)

Personality-Oriented Measures Idea for this test is that employee theft is just one element in a larger syndrome of antisocial behavior which may include dishonesty, theft, drug and alcohol abuse, vandalism, violence, more Assumption is that there is a common personality pattern, and it can be identified A subset of items are identified as an integrity scale, and these are used to measure the test-taker's integrity Test less transparent; reduces faking Rejected applicants need not be rejected for dishonesty

Projective techniques

Projective Techniques Judgment of interviewers Judgment of other observers (especially former coworkers or supervisors) Projective Techniques Require verbal responses, but major differences; questions and answers are intentionally ambiguous, instructions brief and vague Respondents presented with inkblots or pictures and asked to make up a story about them Or given sentence stem and asked to complete: "My father..." or "My favorite..." Respondents' projections expose personality The Validity of Projective Techniques Issues have arisen concerning the scoring and use of information obtained from these tests 1. reliability of an individuals responses at two different times 2. impact on person's score due to total number of responses, especially on tests with free response 3. scoring of information, especially for free response 4. few HR specialists trained in these tests

Simulation tests

Selection devices that assess applicants by testing situations that resemble actual parts of the job being considered are known as simulation tests, or performance tests Limitations: *Difficult to construct simulation tests that are representative of job activities *Simulations are developed assuming that applicants already have the knowledge, ability and skills necessary; if not, must train applicant; test longer, more difficult *Simulations are much more expensive than other selection devices Types of Simulation Tests Described by content: *Motor (behavioral; requires the physical manipulation of things or equipment or physical movement of applicant *Verbal (describes primarily language- or people-oriented exercises like simulating an interrogation, editing a manuscript for grammatical errors, etc. Described by fidelity: The degree to which the simulation matches or replicates the demands and activities of the job Under AC Simulation Tests (cont.) Case Analysis - each participant is given a long description of an organizational problem that changes according to job being considered; case frequently describes events in a firm, gives relevant financial data, marketing strategy, organizational structure; participant asked to solve dilemma with specific recommendations, presenting supporting data and detailing any changes in company strategy

Signs (under consistency of behavior)

Selection tests that are indirect indicators of an individual's predisposition to behave in certain ways. Examples: application forms asking for degrees, former job titles, work experience; some unstructured interview questions

Samples

Selection tests that gather information about behaviors that are consistent with the job behaviors being predicted. Types of Samples Information to determine whether applicant has ever demonstrated the necessary behaviors Work sample tests and simulations to determine whether the job behavior of interest can be completed Samples: two kinds: Instruments that gather information about specific, important job behaviors Simulation exercises that require applicant to complete a set of actions that replicate job behaviors Step 1: Perform Job Analysis Step 2: Identify Important Job Tasks to be Tested Step 3: Develop Testing Procedures Select Tasks Specify Testing Procedures Establish Independent Test Sections Eliminate Contaminating Factors Select the Number of Test Problems Step 4: Develop Scoring Procedures Standards Rules for Scoring Step 5: Train Judges Studies that examine validity have been consistently positive Authors also found only a small difference in validity between objective and subjective criteria measures Work-sample tests show much smaller differences between black and white applicants Other Results of Work-Sample Tests No complaints of inappropriateness of test

Situational judgment test (SJT)

Situational Judgment Tests (SJTs) are verbal simulations, regarded as low-fidelity because they are almost exclusively descriptions of work situations rather than actual replications Ask the respondent to choose among multiple-choice alternatives Item Stem #1. It is Friday afternoon and your immediate manager has just told you to stay until the office is finished compiling its weekly reports—until about 8:30 P.M. However, you have tickets for you and your date for an 8:00 P.M. concert. What would you do? 4 scenarios -2, -1, 1, 2 Developing SJTs 1. Collect stories from job incumbents or supervisors about situations encountered on the job that are important for successful performance. 2. Review the situations that are described to identify item stems. 3. Edit the situations into item stems. 4. Drop in appropriate situations or those that may raise legal concerns 5. Perform a job analysis including gathering ratings of the importance of and time spent on specific tasks. 6. Assemble the chosen items into a survey. 7. Administer survey to sample of SME. 8. The test developer reviews the offered responses to each situation and prepares an edited list of potential responses to each situation. 9. The list of situations and alternative responses is circulated to the same, or equivalent, group who provided alternative responses to the item. 10. If possible, the test developer should perform an empirical validity study for the SJT.

Process criteria

Standard --> Process Criteria --> Product Criteria Quality --> Accuracy --> Conformance to specifications Error rate --> Dimensions or other measures Choice of tools and/or materials --> Spacing Position Efficiency of steps taken --> Strength Suitability for use --> General appearance Quantity --> Time to complete --> Quantity of output Learning time --> Number of steps for which Improvement in meeting quantity standards guidance is needed --> Improvement in meeting quality standards Cost --> Amount of material used--> Number of rejects Safety --> Handling of tools --> Safety of completed product

Product criteria

Standard --> Process Criteria --> Product Criteria Quality --> Accuracy --> Conformance to specifications Error rate --> Dimensions or other measures Choice of tools and/or materials --> Spacing Position Efficiency of steps taken --> Strength Suitability for use --> General appearance Quantity --> Time to complete --> Quantity of output Learning time --> Number of steps for which Improvement in meeting quantity standards guidance is needed --> Improvement in meeting quality standards Cost --> Amount of material used--> Number of rejects Safety --> Handling of tools --> Safety of completed product

Standard testing procedures

Standard --> Process Criteria --> Product Criteria Quality --> Accuracy --> Conformance to specifications Error rate --> Dimensions or other measures Choice of tools and/or materials --> Spacing Position Efficiency of steps taken --> Strength Suitability for use --> General appearance Quantity --> Time to complete --> Quantity of output Learning time --> Number of steps for which Improvement in meeting quantity standards guidance is needed --> Improvement in meeting quality standards Cost --> Amount of material used--> Number of rejects Safety --> Handling of tools --> Safety of completed product

Performance tests

Steps in Development of Performance test: Perform job analysis Identify important tasks to be tested Develop testing procedures Develop scoring procedures Train judges Standard --> Process Criteria --> Product Criteria Quality --> Accuracy --> Conformance to specifications Error rate --> Dimensions or other measures Choice of tools and/or materials --> Spacing Position Efficiency of steps taken --> Strength Suitability for use --> General appearance Quantity --> Time to complete --> Quantity of output Learning time --> Number of steps for which Improvement in meeting quantity standards guidance is needed --> Improvement in meeting quality standards Cost --> Amount of material used--> Number of rejects Safety --> Handling of tools --> Safety of completed product

Faking

Test less transparent; reduces faking Faking may be especially possible for overt tests that directly ask about attitudes and conduct, but faking does not seem to make a difference in the validity of the test But this may not indicate the full effect of distortion on selection devices

Self-report questionnaires

The Validity of Self-Report Inventories A large meta-analysis, focused on predicting job performance, concluded: 1. Conscientiousness & emotional stability were valid predictors of overall work performance over all studies; these are universal predictors across all jobs 2. Extraversion was a valid predictor for some occupational groups such as managers, and specific criteria, such as training performance 3. Agreeableness and openness to experience demonstrated modest validity overall; but each trait was related to a specific criterion: Agreeableness was related to teamwork Openness was associated with training performance Other meta-analysis, examining unique criteria concluded: 1. When selecting effective leaders from ineffective ones, candidates high in extraversion, openness to experience and emotional stability experience the greatest success on the job 2. Conscientiousness, emotional stability and agreeableness useful to predict teamwork and performance 3. All Five-Factor Model traits except openness to experience predicted success as an expatriate or an entrepreneur; extraversion is a particularly important predictor 4. Deviant behavior less likely from those scoring high in conscientiousness and agreeableness 5. These studies investigated the process through which personality affects job performance; the primary means through which personality operates is motivation and interpersonal competence; conscientiousness and emotional stability related to 3 different measures of performance motivation

Recommendations/ Auditing (not kw)

The only way to improve your selection decision making is to learn from what you have done in the past Validation studies identify those factors that will predict job success and those that do not Most managers do not think about their success and failure rate when it comes to making selection decisions, but should Even a simple box-score tally of success and failures helps 1. Use standardized, reliable, and valid selection procedures for collection information on job applicants whenever possible 2. Encourage decision makers to participate in the data-collection process, but discourage them from combining scores with their gut feelings 3. When combining selection procedure scores, use a mechanical means for doing so, for example, multiple regression, unit weighting 4. Train managers & others making selection decisions to make systematic decisions, p 4. Train managers & others making selectin decisions to make systematic decisions, preferably using one of the objective (mechanical) strategies described 5. Although difficult, small businesses can specify, in advance, (a) the weight of standards to be used in evaluating candidates, (b) the procedures used for judging whether applicants meet those standards, (c) the procedures for combining the standardized weights multiplied by the ratings given in order to arrive at an overall score 6. Decide whether a compensatory method for selecting job applicants will be used. The relative value an organization places on minority hiring versus performance will likely affect the choice of selection method 7. For organizations using cutoff scores, the modified Angoff procedure is acceptable 8. Assuming a selection procedure predicts work performance, and that all applicants who are offered employment accept, maximum work performance will occur when top-down selection is used. But, disparate impact against racial minorities is likely if the procedure is highly correlated with general mental ability 9. Banding of selection procedure scores has been support in the courts. However, using minority status alone as a basis for selecting within bands probably not legal 10. For jobs in which selection has taken place, decide on a standard for defining a successful hire (ex: on the job 6 months or more), and an unsuccessful hire (ex: persons' dollar sales fall below minimum standard, or supervisor ratings low). Then have managers keep track of hits and misses 11. Periodically audit selection decisions throughout the organization to identify areas or individuals needing improvement

Methods for Combining Predictor Scores (not kw)

There are many different ways of mechanically combining predictor scores We describe and give an example of two methods, each with a long history of use in human resource selection and other fields: (a) multiple regression (b) unit weighting (c) multiple cutoffs (d)multiple hurdles

Paper-and-pencil integrity tests

These have been used for 65 years These are legal under federal law, but a few states have prohibited them or restricted use Integrity tests aimed toward identifying job applicants predisposed to engage in counterproductive behaviors; two forms: Overt integrity tests directly asking for information Personality-oriented measures, inventories measuring traits linked to certain CWBs

Verbal tests

Types of Simulation Tests *Verbal (describes primarily language- or people-oriented exercises like simulating an interrogation, editing a manuscript for grammatical errors, etc.

Motor tests

Types of Simulation Tests Described by content: *Motor (behavioral; requires the physical manipulation of things or equipment or physical movement of applicant Test Job Motor: Machine operator - Lathe , Drill press, Tool dexterity ,Screw board, test Packaging Clerical worker - Shorthand , StenographicTyping, Mechanic - Blueprint reading , Tool identificationInstalling beltsRepair of gearboxInstalling a motorVehicle repair Tracing trouble in a complex circuit Electronics technicianInspection of electronic defectsElectronics test

Contaminating factors

Under samples Step 1: Perform Job Analysis Step 2: Identify Important Job Tasks to be Tested Step 3: Develop Testing Procedures Select Tasks Specify Testing Procedures Establish Independent Test Sections Eliminate Contaminating Factors Select the Number of Test Problems Step 4: Develop Scoring Procedures Standards Rules for Scoring Step 5: Train Judges

Independent test sections

Under samples Step 1: Perform Job Analysis Step 2: Identify Important Job Tasks to be Tested Step 3: Develop Testing Procedures Select Tasks Specify Testing Procedures Establish Independent Test Sections Eliminate Contaminating Factors Select the Number of Test Problems Step 4: Develop Scoring Procedures Standards Rules for Scoring Step 5: Train Judges

"Crown Prince Syndrome"

Under the effectiveness of assessment centers If publicizing the AC evaluation does affect the selection decision, then the rating creates a self-fulfilling prophecy

Training Assessors

Understanding the behavioral dimensions Observing the behavior of participants Categorizing participant behavior into appropriate behavioral dimensions Judging the quality of participant behavior Determining the rating of participants on each behavioral dimension across the exercises Determining the overall evaluation of participants across all behavioral dimensions

Urine tests

Urine Tests Used most often, first used in 1960s, and continues to present day Positive workplace tests for cocaine and marijuana have fallen, while abuse of prescription drugs (pain killers) has risen Urine tests are: (a) Relatively inexpensive (b) results returned quickly (24 hrs) (c) testing convenient for applicant Urine Tests Require screening and confirmation tests Results are dependent on threshold level set

Judgmental information collection

We describe 6 methods for collecting and combining predictor information These describe how selectin decision makers collect & combine information mechanically, judgmentally, or both Table 15.3 lists these methods and provides an example of each Some methods are more effective than others A recent review concluded that the major issue is how data are combined, rather than how data are collected Conclusion: mechanical methods of combining predictor data improved the ability to predict work performance by more than 50% The rate of identifying acceptable hires was reduced by more than 25% when judgmental data combination methods were used

Assessment center (AC)

What Is an Assessment Center (AC)? A procedure for measuring KSAs or dimensions, usually administered in groups, that uses a series of devices (exercises), many of which are verbal performance tests. "A variety of testing techniques designed to allow candidates to demonstrate, under standardized conditions, the skills and abilities that are most essential for success in a given job" (Coleman, 1987). Uses of ACs Selection—identifying participants who demonstrate behaviors necessary for the position considered. Career development—determining those behaviors each participant does well and those in which each is deficient. ACs are measured by dimensions. Benefits of ACs for Managers/Assessors Development of proficiency in interviewing skills Development of better information communication skills A reduction of halo errors in assessing subordinates Criticisms: Cost: An AC is usually very expensive to develop and maintain and, if selection among applicants is its only use, there might be alternative methods that are much less expensive. Construct Validity: ACs have failed to demonstrate the patterns of correlations (both convergent and discriminant validity) among dimension ratings that they were designed to produce. (see table example)

Universal predictors

1. Conscientiousness & emotional stability were valid predictors of overall work performance over all studies; these are universal predictors across all jobs

Integrity Tests

1. Paper-and- pencil integrity tests 2. Overt integrity tests 3. Personality-based measures What We Know About Integrity Tests (3) Relationship Between Overt and Personality-Oriented Tests Do these measure the same constructs? Research shows: The integrity tests showed a great deal of similarity in what they measured Overt tests correlated more highly with honesty and supervision attitudes; personality tests correlated more highly with self/impulse control, home life/upbringing, risk taking, diligence and emotional stability What we know about them... Validity Determining the validity of integrity tests is difficult; which criterion variable should we correlate? Many CWBs not reported in employee records Self-reporting may not be an accurate measure Conclusions from a number of studies: Integrity tests valid for prediction of theft, general CWBs and work performance Usefulness of Integrity Tests (2) False Positives - 40 to 70% of applicants commonly fail integrity tests, and a large number of those may be incorrectly classified It may be in the interest of the publisher to avoid false negatives, so may set low cutoff score It may be in the interests of the employer to avoid false positives, so may set high cutoff score Because of implications of false positives, use of a personality inventory better than integrity test Usefulness of Integrity Tests (3) Faking Faking may be especially possible for overt tests that directly ask about attitudes and conduct, but faking does not seem to make a difference in the validity of the test But this may not indicate the full effect of distortion on selection devices Legal issues: Under ADA: personality inventories designed to detect mental health issues are a prohibited medical exam (Ex: MMPI) Tests may not include questions about previous drug or alcohol use Other Stuff Recent attempts to change format of integrity tests to biodata questionnaires, forced-choice response, etc.; hard to see an advantage

Guidelines for Drug Testing

1. limit testing to positions which have major safety implications; testing may be required for those 2. Private employers have more latitude unless they are subject to federal regulation 3. Private employers have more flexibility with job applicants than with existing employees 4. a combination of screening and confirmatory tests are necessary 5. obtain written consent of the job applicant prior to testing; if positive result, allow person to explain; physician's verification may be required; offer test by a lab of their choosing 6. use standardized procedures, applied to all job applicants; do not exempt anyone 7. program should be designed and reviewed periodically to ensure privacy for those tested; results must be kept confidential, especially from supervisors of existing employees, unless a defined need to know

Emotional intelligence (EI)

A broad construct composed of attributes beyond those of personality, and includes nearly all non-cognitive predictors, and some that appear to be cognitive measures. Measures the candidate's self-awareness and self-regulation, as well as social awareness and relationship management.

Polygraph

A polygraph (lie detector) is a machine that measures physiological responses of examinees that accompany their verbal responses to direct questions This data, and opinion of polygraph operator, used to evaluate truthfulness regarding the respondent's past or present behaviors Can only validate the truth Employee Polygraph Protection Act of 1988 restricts use of the polygraph for selection to police work, security, and the like Can't be used in court Used for theft Permissible Uses: By Specific Employers Private employers providing security services. Manufacturers, distributors, or dispensers of controlled substances. Federal, state, and local government employers. Under Specific Testing Conditions Workplace theft or other incident has occurred that resulted in an economic loss to the company. Employee must have had access to the property that is the subject of the investigation. "Reasonable suspicion" that the employee was involved in the incident. Employee has been given specific written information about the incident being investigated and the reasons for the testing. Procedures A pretest discussion with examinee that covers all questions in the test to ensure understanding Polygraph is attached, exam conducted; questions may be repeated Three types of questions: Irrelevant, non-emotional questions Emotional control questions Questions about behavior of interest Limitations Reactions other than guilt can trigger an emotional response There are a variety of countermeasures examinees can use to avoid detection: Tensing muscles, biting tongue, flexing toes, etc. Mental countermeasures like relaxation Major drawback to polygraph is frequency of false-positive results (Example: 68%) There are also false negatives

Combination method

A recent review concluded that the major issue is how data are combined, rather than how data are collected Conclusion: mechanical methods of combining predictor data improved the ability to predict work performance by more than 50% The rate of identifying acceptable hires was reduced by more than 25% when judgmental data combination methods were used Summary General Mental Ability is the single best predictor of job performance General Mental Ability and Integrity Tests are the best combination of two measures Mechanical combination increases the accuracy of prediction when weighting predictor scores.

Integrity scale

A subset of items are identified as an integrity scale, and these are used to measure the test-taker's integrity Susceptible to false positives

Trait

A trait is a continuous dimension on which individual differences may be measured Ex: sociability, independence, etc. People may differ sharply in their reactions to the same situation Use of personality data in selection needs: Specification of job tasks Identification of traits linked to those tasks Executive Conscientiousness, emotional stability, extraversion, ambition (especially) Supervisor Persistence, endurance, emotional stability, nurturance Salesperson Conscientiousness, achievement (especially), ambition, extraversion Secretary Conscientiousness, dependability (especially), emotional stability, agreeableness Computer Programmer Conscientiousness, original thinking, openness to new experiences Insurance Agent Conscientiousness, extraversion, original thinking Newspaper Writer Conscientiousness, emotional stability, openness to new experiences Carpenter Conscientiousness, dependability (especially), emotional stability Recent studies have challenged the notion that more of a trait is always better for work performance It is possible to have too much of a good thing Validity may be highest in the middle range of scores

Hogan Personality Instrument Scales (not a key word)

Adjustment - Remains calm under pressure Ambition - Leader-like, competitive and results focused Sociability - Needs social interaction, approachable Interpersonal Sensitivity - Perceptive, tactful, friendly Prudence - Planful, conforming, dependable Inquisitive - Creative and a resource for ideas and problem solving Learning Approach - Enjoys learning, achieving, and staying current