I/O Psychology Exam 2

Watson v. Fort Worth Bank and Trust

subjective procedures can be subject to adverse impact analysis

Bakke v. University of California

"reverse discrimination case"; school violated equal protection clause of 14th amendment to U.S. Constitution; admit white male Bakke to medical school

Meta-analysis

"study of studies"

Adverse Impact Scenario

(e.g., plaintiff brings prima facie case first using statistical evidence; the employer defends test via business necessity; plaintiff demonstrates there is equally valid alternative with less adverse impact)

Know Disparate treatment scenario

(e.g., plaintiff is qualified member of protected class and subject to adverse employment decision; employer gives legitimate reason for decision; plaintiff shows reason is a pretext for intentional discrimination)

Work Samples

"High-fidelity simulations" Ø Validity is excellent Ø They are limited because employees must be skilled, not as effective in assessing "people" vs. "things" jobs

Know 4 Goals of affirmative action

1. correct present inequities 2. compensate past inequities 3. provide role models 4. promote diversity

Testing in Retrospect

"The tyranny of testing"? - Tests are often "high stakes" Ø Tests are not mysterious; they are tools to make decisions that must be made and are probably better with them than without them (however, they are not perfect); abandoning tests is like "shooting the messenger" There seems to be an increased frequency of cheating on employment tests; some people take tests for the purpose of recalling questions Single-validity coefficients greater than .50 are not common; however multiple predictors are typically used to provide incremental predictive accuracy

Letters of Recommendation

-- Common, but often least valid ► Validity is around .13 (low) because of range restriction (almost all letters are positive)

Predictor Development (can be classified along 2 dimensions)

1. Measure the underlying construct in question (construct/behavioral sampling) 2. Measure something about the person's past or currently (and inferences about future behavior)

Assessment Centers

A technique for assessing job candidates in a specific location using a series of structured, standardized, group-oriented exercises that are evaluated by raters ► Multiple methods, multiple raters (usually a process, not a place) ► Individuals are assessed on a number of performance dimensions (e.g., leadership, decision-making) ► Wide variety of methods are used; each exercise does not tap into all the performance dimensions ► Often used with managerial candidates ► Some argue that are predictive because assessors and supervisors hold common stereotypes of the successful employee; biased assessments correlated with biased performance evaluations

Martin v. PGA Tour (2001)

ADA case; must provide reasonable accommodation - golf cart for disabled player

Validity

Accuracy of inferences based on test scores; the test's appropriateness for predicting or drawing inferences about criteria; a test may be valid for one purpose, but not the other

Drug Testing

People who test positive are absent much more frequently and had higher involuntary turnover.

Sources of Info about Testing

As a HR professional or an I/O psychologist you would be able to utilize the Mental Measurements Yearbook (or MMY) with Tests in Print to evaluate the reliability and valid of many published tests; the MMY is a comprehensive review of psychological tests (it is available on the Ritter Library Website - click on the Psychology tab - it is listed as an item)

(SHORT ANSWER) 1. What is a BFOQ? Provide an example of a BFOQ (e.g., Hooters example). Why is there the BFOQ exception?

Bona Fide Occupational Qualification; Hooters only hiring women

Reliability

Consistency or stability of a measure; generally .70 is the minimally acceptable level; can be lower for test-retest (.50); reliability limits validity

Inter-Rater Reliability

Correlation between raters; reflects degree of agreement; also called "conspect reliability"

EEOC v. Abercrombie & Fitch (2015)

Court ruled that woman wearing a hijab to job interview was discriminated against on religious grounds; employers must provide reasonable accommodation for religious practice and observance (this was already the case - the court reinforced it)

Interviews

Degree of Structure (Unstructured vs. Structured) Ø Most interviews fall along a continuum; there is higher validity for structured interviews; interviews are a social exchange Ø Personality traits are most often assessed Ø Highly structured and highly unstructured interviews do not measure the same constructs Ø People can be coached and engage in impression management Ø Quality of handshake related to hiring decisions Ø Most frequently used method of personnel selection Ø Validity is about .30

Integrity Tests

Designed to identify those who will steal or engage in other counterproductive behavior Ø Overt integrity versus personality Ø These tests are valid Ø Testing of this nature is sensitive; organizations (and consultants) do not want to make data available; getting caught is a rare event Ø Some argue that these tests are subject to faking

Albemarle Paper Co. v. Moody

EEOC Guidelines have deference of law; do professional job analysis/validity study

Speed vs. Power Tests

Easy and timed versus difficult (and more time) questions

Ricci v. DeStefano (2009)

Failure to certify test violates Title VII because is based on race (white,Hispanic)

Test-Retest

Give a test to same group at two points in time and correlate the measures; coefficient of stability

Internal-Consistency Reliability

Give one test and calculate a statistic of homogeneity; use either split-half or Cronbach's alpha: conceptually each test item is treated as a mini-test; the response to each item is correlated with the response on every other item; a matrix of inter-item correlations is formed whose average is related to the homogeneity of the test

Equivalent-Form Reliability

Give two forms of the test to the same group and correlate the scores; coefficient of equivalence

Mechanical Aptitude Tests

Highly predictive in manufacturing jobs; there is often a male/female differential

Graphology (handwriting analysis)

It's not valid.

Griggs v. Duke Power (1971)

Know the Supreme Court ruling; the "Grigg's burden"; burden of proof is on the employer to show business necessity; job-relatedness of a selection procedure

Americans with Disabilities Act (1990)

Know the major concepts from the Pretest (e.g., definition of disability - the 3 pronged test; reasonable accommodation; undue hardship); Know Martin v. PGA Tour (2001) court case

Personnel Selection from a Human Perspective

Know the major issues involved in using tests for selection and how they impact organizations and people

Polygraph or Lie Detection

Often used post hoc in criminal investigations ► Countermeasures can distort polygraph ► FBI study found did identify majority of guilty and innocent ► Polygraph Protection Act 1988 - forbids use for pre-employment screening for private employers

Conditional Reasoning Tests

On the surface test measures cognition, but really measures personality; they are often used to assess social aggression; indicate which justifications for a situation is most likely

Individual vs. Group Tests

One person at a time versus several simultaneously (cost is a factor)

Biographical Information (Biodata)

Past behavior is a good predictor of future behavior ► Often recorded on an application blank ► Are historical rather than behavioral intentions ► Should not invade privacy and also be legal ► Faking can occur

(SHORT ANSWER) 1. What is the diversity-validity dilemma (define it)? What are at least five (5) ways/strategies of reducing adverse impact? What are the "pros" and "cons" of each predictor method/strategy of reducing adverse impact (e.g., Ployhart & Holtz); Provide ONE "pro" and ONE "con" for each method or strategy. You do not need to include specific d-values (but you can if you want to), but you should have some indication of whether or not a specific predictor has higher or lower d-values. Diversity-validity dilemma information could also be utilized in the applied question on evaluating a selection system (#11).

Sample Answer: One predictor frequently used in selection involves using cognitive ability tests. PRO: These tests have the most validity (.40-.60) and therefore the most utility and are applicable to many jobs; CON - these tests have high d-values for B-W comparisons and are likely to produce adverse impact; these tests may not tap specific second-stratum cognitive abilities which may be more job relevant. Sample Answer: One method of reducing adverse impact involves retesting. PRO: Applicants get a chance at a "re-do" and can improve their scores. CON: Sometimes high scorers tend to get even higher scorers and if rank-ordering is used there still may be adverse impact.

Physical Abilities Testing

Strength and endurance are tapped into Ø Women score lower than men (leads to adverse impact)

Test Utility and Organizational Efficiency)

Test Utility refers to the dollar value of testing; the economic impact of the use of reliable and valid testing programs; More valid tests = greater utility $$$

Physical Ability Testing

Test for physical abilities in jobs that are physically demanding; in general men have greater static and explosive strength than women; physical ability testing should be job relevant

The Trinitarian View of Validity versus the Unitarian View

The model used to be various "types" of validity, but now validity is understood as a unitary concept

Face Validity

This is concerned with the appearance of the test items; do they look appropriate for the test? Estimates of face validity are make by test takers; estimates of content validity are made by test makers; a test item can be content valid, but not face valid and vice versa; there are more likely to be legal challenges when tests are not face valid

Paper-and-Pencil vs. Performance Tests

Whether or not physical manipulation of objects is involved

Guardians v. Civil Service

acceptable to use content validity to rank order; set cutoff scores based on Guidelines standard

Grutter v. Bollinger

all applicants who could succeed were admitted; diversity defined broadly; no quotas, separate tracks, or additional points; program upheld

Oncale v. Sundowner Offshore Services (1998)

applied Title VII protections to a gay man who was sexually harassed at work ("hostile environment")

Wal-Mart v. Dukes

class action suit; plaintiff couldn't aggregate across different locations

McDonnell Douglas v. Green

disparate treatment; plaintiff can provide indirect as well as direct evidence of discrimination

Situational specificity hypothesis

empirical evidence demonstrated lack of generalizability of criterion-related validity study results; different results for same job

Civil Rights Act of 1991

forbids score adjustments based on race or sex; ("within-group scoring"; "race norming")

Selection ratio

number of job openings/number of applicants; if SR = 1 or > any selection device has little value (labor shortage); SR between 0 and 1 has most value; smaller the SR, the greater the value of the predictor

(SHORT ANSWER) 1. What are two major arguments made by I/O psychologists regarding the debate concerning revising or eliminating the EEOC Uniform Guidelines and the concept of adverse impact? List two (2) main points (there are actually 4 possibilities; you need only list/describe 2).

page 137

International Teamsters v. U.S.

pattern or practice scenario; practice of using seniority system kept out minorities from desirable jobs

"Law of small numbers"

psychologists used small sample sizes which resulted in erroneous inferences about validity; great deal of variability in validity coefficients

Social Validity

quality of a selection process that makes it acceptable to job applicants; reactions of applicants to recruitment assessment process can be emotional; recruitment and selection are mutual processes

Recruitment

some organizations prefer recent college graduates because they believe they are more willing to learn new things; experienced applicants have a better work ethic and more realistic expectations

Price Waterhouse v. Hopkins -mixed motive case

use of sex stereotyping psychology research to support plaintiff's case; there is both a valid reason and a discriminatory reason for an employment decision; this case is becoming increasingly important in cases involving discrimination against transgendered/LGBTQ people (e.g., EEOC v. RG & GR Harris Funeral Homes)

Wards Cove

use qualified labor pool (not everyone) in statistical analysis to establish discrimination; must identify specific problematic practice; modified plaintiff's and employer's responsibilities in litigation pertaining to burden of proof

Base Rate

what constitutes successful job performance; base rate of .50 is optimal; with extreme base rates the value of the predictor decreases; hard to change the actual number of successful workers if most people are either very good or very poor performers

Validity Coefficient

when predictor scores are correlated with criterion data

Situational Judgement Tests

Ø A problem is given to the test taker and the test taker is required to rate various solutions Ø McDaniel et al.'s meta-analysis found that that behavioral tendency instructions correlated more strongly with personality, while knowledge instructions correlated more highly with general mental ability

Adverse Impact

Ø Adverse impact (disparate impact) versus disparate treatment (unequal treatment; intentional discrimination); 80 percent (4/5ths rule) Ø Be able to calculate adverse impact

Gratz v. Bollinger

Ø Affirmative action does not allow adding 20 points based on race; strict scrutiny; compelling state interest; narrowly tailored

Criterion-Related Validity

Ø Based on a correlation between a predictor and a criterion measure (a standard against which the predictor is evaluated) Ø Validity coefficient; typical validity coefficients are in the .30-.50 range; square the correlation coefficient to determine amount of variance accounted for by predictor (e.g., r = .50; accounts for 25% of variance) Ø Moderate validity coefficients attest to the complexity of human behavior Ø Two types: 1. Concurrent 2. Predictive

Intelligence Tests

Ø Cognitive ability or general mental ability ("g factor") Ø g predicts across wide range of jobs Ø Validity of g is impressive (.40-.60) Ø "g" is to psychology what carbon is to chemistry" Ø Some would say specific abilities are also important Ø Sternberg would say that practical and creative intelligence are not measured by conventional tests (tacit knowledge); people differ in thinking styles Ø This relates to conclusions from validity generalization studies which find that "g" is the best predictor overall. Ø Expand criteria to include various manifestations of intelligence; real-life problems need to be assessed; these tests complement rather than replace intelligence as "g"; the construct of intelligence is highly complex Ø "g" tests can be valid and fair

Concurrent Validity

Ø Concurrent validity involves collecting predictor and criterion data at same time

Selection Decision

Ø Criterion cutoff set by management; distinguishes successful from unsuccessful employee Ø Predictor cutoff set by selection ratio, cost, law (70% rule) Ø Need to determine whether or not false negatives or false positives have most serious consequences for organization; extreme base rate problem (e.g., predicting workplace violence) Ø False negatives may be as important as false positives (e.g., lawsuits)

Testing on the Internet

Ø Faster and cheaper means of testing Ø Test security and test content may be compromised unless there is human proctoring Ø Reduces time from assessment to feedback to candidate Ø Candidates cannot flip through pages which may be a disadvantage

Personality Inventories

Ø Inventories do not have "right" or "wrong" answers; items are scored according to a key such that responding one way or another to an item results in a higher or lower score on a respective scale Ø Myers-Briggs (MBTI) is a popular test Ø "Big Five" personality characteristics (Costa & McCrae) - OCEAN Ø Extraversion a valid predictor for social jobs (sales) - outgoing Ø Conscientiousness shows consistent correlation with job performance for all occupations and across different cultures Ø Validity coefficients may be modest (.20) Ø Are cross-culturally valid and may have less adverse impact Ø Issue of social desirability or faking does occur; should we correct for it or eliminate fakers? Ø Personality tests should be used in conjunction with other measures Ø "Can do" (Cognitive ability) versus "Will do" (personality); intelligence and personality can be predictive of job performance in their own way Ø Some argue that there is a "p" factor (just like the "g" factor of general intelligence) - "p" factor is general personality measure reflecting the ability to cope Ø Some researchers believe a structured interview is more accurate measure of personality (to control faking) Ø Check personality test against telephone reference check

Civil Rights Act of 1964

Ø Know purpose of law: Forbids discrimination based on race, color, religion, sex, or national origin (protected groups); law protects individuals, not groups, though; legislates behavior; Title VII specifically forbids discrimination against the protected groups in employment; CRA includes all personnel functions; Sexual harassment is a violation of the CRA (two types: quid pro quo and hostile environment); Equal Pay Act of 1963 prohibits unequal pay based on sex

Situational Interviews

Ø May focus on future, hypothetical contexts or past behavior Ø May assess experience or situational responses Ø Validity is around .39 Ø Are interviews valid or is there the illusion of validity?

Predictive Validity

Ø Predictive validity involves collecting criterion data at some future point in time

Content Validity

Ø Predictor is a representative sample of a content "domain" or behavior Ø Depends heavily on expert judgement Ø We do NOT compute a correlation coefficient Ø Face validity concerns whether or not the test "looks" to be valid Ø Job analysis is extremely important since it links test content to important KSAOs required for the job.

Computerized Adaptive Testing

Ø Questions are pre-calibrated in terms of difficulty and the examinee's responses to one question determines the selection of the next question - the test is "tailored" to the examinee Ø Widely used in the military and educational settings.

Validity Generalization (Powerpoints; text)

Ø Refers to a predictor's validity spreading or generalizing to other jobs or contexts beyond the one it was originally validated in Ø Based on the statistical technique of META-ANALYSIS - a "study of studies"

Selection

Ø Separate selected from rejected Ø Predictor validity, selection ratio, base rate determine quality of selected employees Ø Predictor validity - greater validity = greater value in increasing average criterion performance for accepted group/total group Ø Selection ratio - number of job openings/number of applicants; if SR = 1 or > any selection device has little value (labor shortage); SR between 0 and 1 has most value; smaller the SR, the greater the value of the predictor Ø Base rate - what constitutes successful job performance; base rate of .50 is optimal; with extreme base rates the value of the predictor decreases; hard to change the actual number of successful workers if most people are either very good or very poor performers

Ethical Standards in Testing

Ø Standards for Educational and Psychological Testing; degree of qualification to give and interpret tests depends on the circumstances Ø Issues of invasion of privacy and confidentiality Ø How long should records be maintained?

Validity Generalization

Ø The idea is to come up with one "true validity" based on large samples from various studies controlling for sample size (and other artifacts); sample size is a major contributor to variance in validities Ø Schmidt and Hunter are major proponents of VG Ø Local criterion-related validity studies are not always feasible; Small organizations could "borrow" validities or do transfer studies Ø "One test for all jobs" (General cognitive ability or the "g" factor) is the idea that all jobs require cognitive ability and therefore it is valid across all jobs Ø There are still jobs that do require specific abilities Ø VG is one of most important methodological advances in personnel selection research in past 20 years, but courts may not accept VG as evidence of validity (EEOC v. Atlas Paper)

Construct Validity

Ø The test is measuring a hypothetical, abstract variable; to what extent does our operational (empirically observable) definition match the theoretical concept? Ø Convergent validity refers to the fact that measures of the same "thing" correlate highly Ø Divergent (discriminant) validity refers to the fact that measures of different "things" should NOT correlated highly Ø Binning and Barrett (1989) suggest that construct validation is the process of demonstrating evidence for 5 linkages or inferences; the linkage between the test and the predictor is the only one we can measure directly; we should also be interested in the degree to which our measures reflect our theoretical concepts of interest; researchers need to assess linkages 2 and 4 (between theoretical concepts and actual measures)

Selection Decisions Outcomes

Ø True positives (test says they will succeed and they do) Ø True negatives (test says they will fail and do) Ø False negatives ("good ones that got away") Ø False positives ("bad ones we let in")

Situational Tests

Ø Two examples are leaderless group discussion and in-basket Ø The white-collar equivalent of work samples Ø Validity is low to moderate Ø Criterion of success as a manager is more difficult to define Ø Two examples are leaderless group discussion and in-basket Ø The white-collar equivalent of work samples Ø Validity is low to moderate Ø Criterion of success as a manager is more difficult to define Situational exercises are "low-fidelity simulations"

Determination of the Cutoff Score

Ø Why the 70 percent rule? Ø Cutoffs set to be "reasonable and consistent with the expectations of acceptable job proficiency in the workplace" Ø Cutoff set so that person can learn job and perform it safely Ø Cutoff set to minimize false positives/false negatives Ø We can empirically set cutoff if we have criterion-related validity data Ø Cutoff for content valid test rests of judgment Ø No single uniform cutoff score for all situations Ø Job analysis is important in determining a cutoff score Ø Take into account potential adverse impact when setting a cutoff score; BIG MISTAKE to set cutoff score so low that minimum job performance standards are not met; why even administer the test? Ø ****The BANDING debate: rank ordering (traditional method) versus treating individuals in a band as equivalent and select based on gender or race (issue: how wide are the bands?; more reliable tests produce wider bands)

Uniform Guidelines of 1978

Ø given deference of law Defines adverse impact Criterion-related validity, construct, and content methods to establish job-relatedness Can't test on knowledge and skills learned in training Cutoff scores on tests = consistent with normal expectations of proficiency bottom-line concept overturned (Connecticut v. Teal)

Emotional Intelligence

► Golman's (1985) popular book spurred an interest in emotional intelligence ►Controversial construct in I/O psychology ►The construct is ill-defined

A test's validity

►A test's validity can be described in terms of content relevance, criterion relatedness, and construct meaning. ►Either/or type of thinking about test validity is not correct ►Tests manifest varying degrees of validity ►It can be thought of as the overall weight of evidence that is brought before a jury in a legal trial (an analogy) ►The overall weight of evidence showing that it measures the construct it purports to measure ►The concept of validity is continually being refined

Overview and Evaluation of Predictors

►Predictors can be evaluated on the basis of validity, fairness, applicability, and cost ►There are frequently tradeoffs ►Table 4-4 provides a good general summary ►Hiring organizations try to understand how past experiences are translated into future job performance