PSY 370 Exam 3

Test user qualifications

-At minimum APA says test users should know: ~basic psychometrics *descriptive statistics *types of scales and scale scores, using norms *reliability and measurement error *validity -How to select appropriate tests -How to administer tests and interpret the results -Issues in testing people from diverse backgrounds -Issues in testing people with disabilities -These are for any test user..not just the experts -Ultimately the test user is responsible for making sure the test is used ethically ~the test author or publisher can only control so much

Factor analytic approach to intelligence

-by studying the relationships among all the various tests we use to measure intelligence we can figure out how man dimensions (or kinds) of intelligence there are

The basic projective approach

1. Present the test-taker with a series of stimuli that are ambiguous or incomplete insome way ~vague shapes or pictures, partial stories, words, sounds 2. Ask the test-taker to decribe the stimuli 3. Record as much information as possible about responses ~many tests include inerviewing the test-taker about each response-why do you say that? ~not just the response itself- reaction time, nonverbal behavior 4. Analyze and interpret the responses based on some kind of scoring system ~usually looking for themes or patterns ~often emphasis on symbolism ~may compare to normative data-usual vs unusual responses ~scoring systems vary a lot-some are informal and very flexible, others are quite complex

Other uses of testing in education

Counseling and guidance -tests of vocational interests Assessing study skills and attitudes Policy decisions -how well is a school or program working -what curriculum or teaching method is most effective -where are resources most needed -No child left behind, Colorado teacher accountability law

Testing current employees

Organizations also use testing for.. Performance appraisal ~productivity, motivation Training and development _usually, knowledge tests or skill tests Attitude and culture surveys

Integrity

Organizations want to avoid hiring people who will lie, cheat, steal, or engage in other counterproductive behaviors -but honesty is hard to measure Polygraph-"lie detectors" measure physiological responses and make inferences about honesty ~they don't work. Period ~they are illegal in most employment testing settings So what can we do instead

Example of projective test: Rorschach inkblot test

Thematic apperception test -similar to Rorschah in that is uses pictures and requires the test taker to explain what is going on in them -different in that the pictures are of people in "classical human situations" -Like the Rorschach both responses and extra test behavior are considered in scoring

Clinical vs Counseling perspectives

Theoretically -counseling psychologist focus on those experiencing normal development problems -clinical psychologist focus on the mentally ill and those who exhibit abnormal behavior In practice this distinction is blurry Clinical psychologists are more likely to use assessment as an information gathering tool ~make a diagnosis and facilitate communication about that diagnosis ~identify appropriate treatment Counselors are more likely to use assessment as part of therapy ~give client new information about themselves, this new information guides change

Critical clinical evaluations

Addiction and substance abuse Forensic assessment ~competency to stand trial or make major life changes ~emotional and psychological damages ~danger to self or others ~readiness for parole, etc. Custody evaluations ~evaluation of parents and child Child abuse and neglect

Cultural issues

-Culture= "the socially transmitted behavior patterns, beliefs, and products of work of a particular population, community, or group of people ~complex-nationality, ethnicity, religion, language, region -Culture affects a LOT of behavior ~in testing, are we measuring the person or their culture ~culture affects responses-do they mean what we thing they mean

Culture in the history of testing

-Early tests were often developed withing just one cultural group ~other cultures didn't score well-often interpreted negatively -Ex: Binet intelligence scale in the early 1900s -This led to a belief that tests were culturally specific and should be used only with people from the culture they were developed in -We started trying to develop "culture-specific" tests ~such as the Black Intelligence Test of Cultural Homogeneity ~what potential problems do you see with this approach

Assessment in clinical and counseling settings Clinical assessment

Clinical assessmnet=information-gathering and interpretive skills used by the professional counselor-therapist ~remember that assessment means an integration of multiple pieces of information, including tests ~we never make a diagnosis based on one test with no other context Assessment opens up access to the science of pyschology, as well as the art of interpersonal social support ~assessment allows us to do more to understand the problem, reduce the impact of our own biases, and approach the sitation in a systematic way

Clinical vs statistical prediction

How should a clinician integrate the results of various measures within a psychological assessment ~based on a holistic, qualitative evaluation of themes and patterns ~based on quanitative combinations of test scores and statistical information about probable diagnoses and outcomes Ex; mary's interview showed no signs of depression and she scores in the normal range on the beck depression inventory but a strong theme of depression emerged in her TAT responses

Another use of pictures

Instead of responding to a picture, the test-taker creates one The test has no standardized stimuli-just an instruction to draw, usually a person ~where the test part comes in is in the scoring or coding manual Usually assumed that the figure drawn is a symbolic self-portrait ~exaggerated or minimized features ~size and placement ~clothing or lack therof Includes discussion about the picture-"tell me about this person..." Several variations: House-Tree-Person

Scientific theories of intelligence

David Wechsler -intelligence= aggregate or global capacity..to act purposefully...think rationally and...deal effectively with the environment (problem solving) -consists of multiple abilities and we can separate them Originally 2 main factors-verbal and performance -Currently 4 factors-verbal comprehension, working memory, perceptual organization (mental rotation), and processing speed ~theoretically intelligence also includes personality, motivation, aesthetic and moral values, but these are not measured in the test Measured in ways that allow separating the factors -multiple subtests (mutlidimensional)

Intelligence testing

Lay theories of intelligence -ordinary people often mention things like: -book smarts ~verbal ability -common sense practical problem solving ability -people smarts ~social competence

Case Law and testing

Legislation is necessarily broad -details of how to implement the law are often worked out over time -When there is dispute about how to implement a law, it often ends up in court A judge's ruling sets precedent that other cases must follow unless the ruling is overturned -within the same court level-state, appeals, supreme This means that the judge's ruling essentially tells us what will be considered legal or illegal in the future

interviews

ONe of the most common selection techniques -alone or in combination with other tests As with clinical, can be structured or unstructured -but here, unstructured is really unstructured A good structured interview has three characteristics -the same questions are asked to every applicant -the questions are based on a job analysis-directly relevant to the job -there is a consistent system for scoring answers

Personality measures

Once thought to be a poor predictor -recently this as changed-better measures, better studies Conscientiousness predicts performance for virtually all jobs Extraversion and emotional stability are good predictors for many jobs ~different jobs have different personality requirements Predictive validity coefficients are lower than for some other tests... ..but personality tests are not usually used alone ~they add predictive power when combined with other tests -also show very little adverse impact

Controversy about projective methods

Projective techniques require some big assumptions ~there is such a thing as the unconcsious ~projective techiques reveal it-every response has meaning *distortions in figure drawing are symbolic, not the result of lack of artistic ability ~psychologists can accurately interpret this symbolism ~many more specific assumptions Most of these assumptions are essentially untestable The examiner affects the process -Nonverbal behavior and other subtle cues -Examiner personality factors may affect interpretation Psychometric principles don't always apply ~responses and interpretations are so unique ~how can you separate true score and error Are projective methods a science or an art

Performance appraisal

Purposes: give feedback, improve performance, document evidence for organizational decisions Tests -Evaluating attributes of people -assigning numbers -systematic Like any test, need to be reliable and valid

Faking

Quite a bit of controversy about how big a problem faking might be _can people fake _do they fake _how many people fake Can people fake -in the lab absolutely -but this is probably an exaggerated effect, everyone is trying to fake and in real life there will be differences in motivation -there are also individual differences in the ability to successfully fake

Using cognitive ability tests fairly

Race norming used to be suggested as a solution -use different norms, cutoffs, or prediction equations for each racial group ...but many people felt this was discriminatory in itself ~giving an unfair advantage ~made illegal in 1991 Now it is OK to use a test that shows adverse impact if the test is a valid predictor for all groups -idea: you are selecting based on predicted performance-if a minority applicant who would be a good performer applies, they should get the job -need a good, well-documented validity study to show this

Mental status examination

Similar to a general physical exam-basic screening of essential mental health information Focuses on basic intellectual, emotional and neurological functioning ~knowing who, when and where you are ~state of consciousness ~appropriate or inappropriate emotional expression ~comprehension of simple information ~insight Important first step in most assessments ~clinical intake ~competency evaluations

Early theories of intelligence

Sir Francis Galton -Intelligence=sensory ability -the more we percieve, the more we have to work with Measured with sensorimotor and perceptual test -can you distinguish very subtle difference in the weight of these cylinders Alfred Binet -intelligence=reasoning, judgment, memory, abstraction -these process interact when we solve problems-we can't separate them Develo[ed the first widely use intelligence test (educational test) -intended to asses whether french children with intellectual disabilities could be integrated into typical classrooms -Louis Terman and Maud Merrill at Stanford University adopted and then adapted the test-extensively into a broad general intelligence measure the Stanford-Binet Intelligence scale

Factor analysis and the big five

Something about the number 5 -several different researchers found 5-factor solutions -and the factors were pretty similar Costa and McCrae are currently the biggest proponenets of the Big Five ~find essentially the same five factors (OCEAN) across all kinds of samples:ages, languages, cultures ~argue that these five traits are universally important ~predict lots of different outcomes-work, life satisfaction, divorce, even logevity Other researchers still find different numbers of factors, but the big five are the most popular

Creative methods of assessing personality

The big five, MMPI, and similar measures are all objective measures of personality ~not because they are judgment free or perfectly accurate ~but because the scoring of each individual test doe not require much judgement Objective measures usually rely on self-reports or easily observable behavior ~as we've discussed, there are limitations to both of these How do we get information that people would rather keep to themselves?

Case history

Ususally looking to information beyond the interview to gain a comprehensive picture of the client -biography/life story -interviews with family and friends ~educational and medical records ~employement history Provides context to interpret findings from interviews and test conducted at this particular point in time ~previous experiences give insight into etiology and possible causal factors

How does culture affect assessment

Verbal communication -language -vocabulary level -translation Nonverbal communication -nonverbal behaviors have different meanings in different cultures -pace of life differs across cultures Cultural differences in standards -"normal" depends on culture

Achievement vs Aptitude

We can select based on achievement or aptitude Achievement-what you have learned already ~have you mastered the information you need to know you start this calss Aptitude-much harder to define -potential -informal learning -ability to solve problems you haven't formally been exposed to Achievement tests usually focus on past learning, while aptitude tests are used to predict future learning ~distinction is fuzzy...the same test may be an achievement or aptitude test, depending on the context

Testing for selection and placement in education

When do we need to select students? -school administration-colleges, grad schools, private schools, magnet schools -special programs -levels of a course Why select? -when resources are limited-can't offer to all -when only certain students will beneit ~e.g. you won't get much out of calculus if you can't do algebra These tests are usually created by testing specialists ~more often standardized

Nature or Nurture

Where does intelligence comes from? Intelligence and heredity -twin and genetic studies show more similarity among twins and siblings that we can explain by chance alone -the effect varies depending on how we measure intelligence Nurture-family environment -family and home life variables affect the expression and development of intelligence -siblings-even twins-don't necessarily have identical family environment Interactionist perspective ( both genes and environment are important)

Tests to diagnose disabilites

-A single test score should never be used as the sole basis for diagnosing a disability -Integrated assessment, combine test information with interviews, direct observation, and other methods -Test can be used to screen for disabilities-is further investigation needed?

Scholarly theories of intelligence

-Acquired knowledge -Ability to learn and acquire knowledge -Ability to perceive patterns -Ability to apply information to solve practical problems _lots of theories about what intelligence is Different theories have different implications for how we measure intelligence ....and the conclusions we make based on those tests

Federal laws affecting testing-Discrimination

-Civil rights act 1964-prohibits discrimination on the basis of race, color, religion, national origin, and gender ~applied to a wide variety of decisions-employment, education, housing -Created the Equal Employment Opportunity Commission-among other things, sets guidelines for fair and appropriate employment testing -Concerned with the results of testing-adverse impact against minority test-takers is viewed as discrimination ~unless the test user can show that the test is valid and appropriate(and that there are no better alternatives) -Very very controversial ~if minority or majority group members get different scores, is the problem with the test? -or is there a real difference with another cause

Federal laws affecting testing-Disability

-Education for all handicapped children, 1975-requires appropriate, professional testing of children suspected to have a disability ~Goal: to determine child's educational needs and meet them -Individuals with Disabilities Education Act, 1997-required appropriate testing to establish disability status and educational progress ~Educational progress tests must accommodate students with disabilities as far as is reasonably possible -Americans with Disabilities Act, 1990-prohibits discrimination on the basis of disabilities ~tests must accommodate persons with disabilities as far as is reasonably possible, unless the testing method is essential for the job

Testing people with disabilities

-Federal law prohibits discriminating against people with disabilities ~in the workplace(ADA, 1990) ~in education(IDEA, 1997) ~and other areas, but these two are most relevant to testing These laws specifically state that test must accommodate persons with disabilities as far as is reasonable possible

Key Innovation in the stanford-binet

-First test to include detailed instructions for consistent administration and scoring ~individually administered by a trained clinician -First test to create alternate forms First adaptive testing design -Begins with short routing test -Examiner chooses the next set of items based on test-takers responses so that items are at the level that will give the most information ~so that not everyone has to take the entire test if they do not need to it, if the are too easy

Personality traits

-Guilford: a trait is any distinguishable, relatively enduring way in which one individual varies from another ~a psychological trait usually involves behavior *possibly beliefs, feelings, etc *physical characteristics like height aren't usually considered psychological traits -We often talk about traits as tendencies to a particular kind of behavior Key points: -individual differences -relatively enduring -Traits are historically controversial ~people behave differently in different situations ~some have argued that this means traits don't exits-the situation determines behavior ~most researchers now concede that both traits and situations are important *neither alone can fully predict or explain behavior

What can we do about it

-Ignore it ~if the test predicts as it should, then it may reflect a real difference -Adjust score? ~this is illegal in employment decisions in the U.S. ~may make more sense for clinical or research purposes but caution is needed -Consider culture when interpreting results ~subgroup norms-appropriate comparison standard ~again, not legal for making employment decisions -Write better tests? ~identify factors that cause (irrelevant) differences and work to get rid of them

Culture-fair testing

-Items can be more or less culturally loaded-needing knowledge about a particular culture to answer correctly ~for intelligence tests: verbally-oriented items, specific facts, written instructions and responses, and many other factors tend to increase culture load -Can we write "culture-fair" or "culture-tree" tests that avoid cultural difference? ~performance or nonverbal items, oral instructions, and responses more abstract content -Neither culture-specific nor culture-fair tests tend to predict the same criteria as traditional tests ~suggesting they measure somewhat different constructs

Job knowledge and work samples

-Job knowledge or certification tests -portfolios -work sample ~applicant performs a task they would do on the job, under standard conditions ~evaluated as to outcome and process-may include follow-up questions These are excellent predictors if developed well

Its complicated

-Mean differences do no necessarily mean that: ~the test is invalid ~the test is intentionally biased -Differences can result from: ~true differences on the construct ~failure to consider culture when developing the test ~other factors (e.g. stereotype threat) -Especially important when tests are used to make decisions ~does the test keep members of one group from having access to a resource

Accommodations cont.

-Need to be very familiar with the purpose and intended use of the test ~need to understand the potential impact of any modification -Some tests have been written specifically for people with disabilities ~can you substitute one of these for another test? Maybe -As far as possible, need to compare to appropriate norms -Goal: ensure that the test score reflects the person and is not due to the disability -Keys: be aware and be informed ~if you don't know, ask ~if you have a disability, self-declare to ensure you are properly accommodated ~seek help from people who are knowledgeable about the disability and can identify appropriate accommodations

Federal laws affecting testing-Education

-No Child Left Behind, 2001-holds schools accountable for student improvement by assessing progress on standardized tests -Test-takers rights ~Family Education Rights and Privacy Act, 1974-parents and students have the right to access school records, including tests *May request a hearing to challenge test scores -Health Insurance Portability and Accountability Act, 1996-individuals have the right to know and control who has access to their medical information, including psychological test results

If not self-reports, who

-The alternative to self-reports is other-reports ~usually, people who know you well -May predict outcomes a little bit better ~depending on what kind of personality construct you're measuring... -But other reports have their own issues -Such as... ~raters often see a person in only on context(school, home, work, etc.) ~different raters may not agree *this is not necessarily a bad thing ~raters are not perfectly accurate *they have their own biases...also their own response tendencies

Test in translation

-Translating tests introduces all kinds of potential for error ~translation choices can change the meaning of an item(or even a whole test) ~different cultures may have different response styles ~constructs themselves may mean different things in different cultures *e.g. social skills...even mental health -Need to examine measurement equivalence ~do people from different groups understand the constructs in the same way? ~do people from different groups understand the items in the same way ~do these items measure equally well in both groups ~factor analytic and IRT methods to test this

Reasonable accomodations

-Vary depending on the person, the test, the situation, and the disability -May be common sense ~good lighting and large print for people with visual impairments ~amplification of spoken instructions for people with hearing impairments -But still require careful thought... ~if assistance is needed, who should give it? ~written instruction does not equal verbal instructions

Discovering personality-factor analysis

-Where did the big five model of personality come from -Descended from a long line of research based on something called the lexical hypothesis ~if a difference between people is important, society will develop words to describe that difference ~the more important the differences, the more words we will have for it and the more often they will be used ~by analyzing our language, we can identify the most important personality traits -Started by factor analyzing long, long lists of adjectives ~if two items are correlated, they have something in common ~eventually developed several large clusters of similar adjectives

The big issue in personality measurement

-Who is the best source of information about personality -The most common: self-reports -Pros: ~put some information you just can't get any other way ~you spend more time with yourself than anybody else...across all situations, across time -Cons: ~how do you know people are being honest *faking, social desirability ~even if people are being honest..do they really know themselves? *self-deception, self-serving bias Some focus on self-report measures as self-concept-not necessarily accurate, but its useful to know how you view yourself

Implications

-at least some people do fake.. but they're subtle about it -quite a bit of variability-not everyone faked -faking did change outcomes ~of the top 10 applicants, only 4 would have been in the top 10 based on their honest score Suggests faking is a legitamate concern in high-stakes situations

Integrity tests

-both types do predict outcomes But they are widely criticized -test publishers often oversell benefits And the are susceptible to faking -recommended procedure is use these as a screen out not screen in procedure -be careful in your conclusions

Why measure personality?

-job selection -mental health-personality disorders -recommending treatment

Testing, group differences, and bias

-many tests show mean differences between social groups (e.g. races, cultures, genders) -If the test is used to select people for something, on average, fewer members of Group A will be chosen than members of Group B

Ability testing

A little different from intelligence testing in other contexts -goal is usually to keep it short and simple, but maximize prediction -indivdually administerd tests like the SB-5 and WAIS are used only rarely Tests of genera cognitive ability are among the best predictors of perormance for most jobs -particularly for complex jobs But cognitive ability is a controversial predictor

Projective methods

the projective hypothesis: the way we create structure for unstructured stimuli is based on our personality ~can reveal needs, fears, impulses, conflicts, etc. -even when the person we are assessing is not consciously aware of these Projective methods ask the test-taker to complete some task that is unstructured or ambiguous -a trained test administrator can identify patterns in these responses that indicate various aspects of personality

APA-General Ethical Principles

1. Beneficence and Nonmaleficence -Do good, avoid doing harm -Consider the effects of your choices on others -In testing, test with a purpose, be mindful of the consequences of test results 2. Fidelity and Responsibility -Understand that you are in a position of trust -Be accountable, address conflicts of interest -In testing, use information carefully, be mindful that people are trusting you with information about sensitive things 3. Integrity -Encourage accuracy, honesty and truthfulness -When its necessary to be deceptive, do so very carefully ~In testing we're not talking about research deception-we're talking about what we tell our test-takers about what we are doing and why ~Deception by omission counts too 4. Justice -Ensure that all people have equal access to services and equal quality of service -In testing work to make your tests fair and appropriate for all people. Understand where bias might affect test results and weight them appropriately 5. Respect for People's Rights and Dignity -Acknowledge the dignity and worth of all people -Respect all people's rights to privacy, confidentiality, and self-determination -Respect cultural, individual, and role differences

4 Basic Rights of test-takers

1. Privacy/confidentiality -People have a basic legal right to choose what they share about themselves and who they share with it -Confidentiality=information should not be shared with someone other than the client without the client's permission ~not the same ass anonymity ~there are many situations in which it is necessary to share test results-but the test-taker needs to know who will be given what information before they are tested(or before the information is released) ~test results must be kept secure 2. Informed consent -Does the test-taker know what he or she is getting into? ~applies to all testing situations, not just research -Before they take the test, test takers must be told ~why they are being tested -how the data will be used -who will have access to the information -This information needs to be presented in a way that is understandable to the test-taker -Amount of detail varies by purpose of the test 3. Knowing and Understanding Results -Test takers have a right to know the results of the test ~pass/Fail? Diagnosis? Score and how it compares to others ~honestly...not just telling the test-taker what they want to hear ~need to be communicated in an appropriate and sensitive manner -What decisions or recommendations will be made based on the test -Will the test results be thrown out -Again amount of detail varies by purpose 4. The least stigmatizing label -Test results often include or imply labels ~extravert ~neurotic ~below average -Research shows that labels have consequences ~reputation, self-fulfilling prophecy ~does resistance to testing come from a fear of labels -APA recommends avoiding labels as much as possible ~neurotic vs a person high in neuroticism -When we must categorize, try to avoid labels that carry stigma

Diagnosis

Assessment provides information about symptoms Symptoms can then be compared to a classification system ~DSM-V ~Or something more informal, depending on your purpose The DSM-V ~modeled after medical diagnostic models ~descriptive lists of symptoms-classify the client based on symptom patterns, use the additional information to know what to expect Diagnosis is used to identify treatments or interventions ~clinician does this on his/her own-not included in the DSM

Using tests in the classroom

At the beginning of instruction -placement tests, pre-tests During instruction -formative assessment ~what are students learning and not learning -diagnostic assessment ~assessing an individual, not a class ~checking for learning disabilities or other learning problems ~very thorough and specific-done by a psychologist End of instruction -summative assessment ~have students mastered the material? what grade does each deserve

Approached to achievement testing

Norm-references ~test taker is compared to others in the same norm group ~better than 65% of students in the same grade Criterion referenced -test-taker is compared to an absolute standard -knows 75% of the material Or authentic assessment=assessment based on a complex behavior ~can the student apply knowledge in a real world setting -portfolios: multiple samples of a student's work completed over a specific time period -more examples, essay, oral reports

Authentic assessment

Advantages -does capture real behavior -less anxiety-provoking for many students Disadvantages -seldom have well-defined scoring criteria -reliability and validity unknown-very subjective -hard to administer on a large scale

The tat

Also codes for themes -usually assumed that the test-taker identifies with the protagonist of the story they tell-that charters feelings, thoughts and behavior give clues about the test takers Considers content as well as some structural information ~does the test-taker focus on the whole scene? do they explain all of the details? Focuses on Murray's concept of needs or implicit motives-what does the test-taker really want? what drives him or her?

Applications of neuropsychological assessment

Anxiety -Anxiety is a symptom of many serious disorders (OCD, PTSD) -Specific brain areas associated with anxiety-helps researchers see connections Depression ~impacts several neurophysiological tests Schizophrenia ~now linked to certain abnormal brain functions

Faking and cheating

Both imply that the test taker is consciously doing something to obtain a test score that is not consistent with his or her true score Cheating usually refers to cognitive measured, faking usually refers to non-cognitive ones Different processes involved -cheating requires getting the right answers -faking required identifying the right answers

Ethics

How do we know we are "doing the right thing" -Not legally binding, but do carry professional consequences American Psychological Association -Psychologists use tests-members need guidance on how to use tests appropriately Other groups develop ethical guidelines too -American Educational Research Associations, Association of test publishers, Society for Industrial and Organizational Psychology Aim to balance the needs and rights of: -Person being assessed -Person doing the assessing -People who will use the results -Person who developed the test

Cheating

Common solution: test security -restrict access to the test items, proctor the exams -but sometimes this is not feasible, or conflicts with other priorities Adaptive testing is another solution -use a lot of pre-knowledge about the items to tailor items to a test-takers estimated ability level -so different test takers get different items-item pre-knowledge doesn't help so much We can sometimes use advanced methods to detect aberrant response patterns -i.e getting hard questions right but missing easy questions -this only works when we have a lot of prior data about the items and the test

Evaluating interviews as tests

Concern #1-interrater reliability ~unstructured interviews often have poor interrater reliability *in other words-two clinicans may come to very different diagnoses ~adding structure tends to improve reliability ~is test-retest reliability relevant to interviews Concern #2- what are we agreeing on? ~reliability and validity are compromised when diagnostic criteria are not well-developed -Older DSM definitions tended to be vague Concern #3-criterion related validity ~sometimes a more formal test is the criterion but whos right ~varies quite a bit depending on the interview and the criterion ~could you count on a criterion validity coefficient published in an interview manual

Why use projective techniques

Difficult to fake-doesn't require test-taker to have self-insight Not nearly as verbally loaded as paper and pencil tests ~easier to assess people with limited language skills Get information you simply can't get any other way-the unconscious Acknowledges the uniqueness of each individual-allows the administrator to take a variety of information into account

Clinical interviews

Discussion between clinician and client ~can be structured, unstructured, or semistructured Unstructured interviews: ~are flexible and allow the clinician to be responsive ~but may be susceptible to biases *hypothesis confirmation bias *self-fulfilling prophecy *ethnocentrism -structured interviews can't remove these biases entirely, but can reduce them

So what can we do about faking?

Don't use fakable measures alone -use as a screen-out not a screen-in technique (trust in low scores, but not high scores) -Get additional data ~use in combination with other, less fakable predictors -verify with other-reports (not a perfect comparison but useful and often good predictors in their own right)

What can we do about it

Drop people whose scores are just too good? Develop a measure to identify fakers? -social desirability -unlikely virtues -lie scales

Neuropsychological assessment High tech

Electrophysiological techniques -Electrocephalogram(EEG)-written record of electrical activity in the brain -Event-related potentials (ERP)-electrical activity in response to a specific event Neuroimaging -PET scans-measure blood flow to different regions of the brain -MRI-brain tissue, also electrical activity in specific areas of the brain

Cheating only

ONe big current issue: unproctored internet testing -allowing test-takers to take a test from any location Current solution -use the unproctored test as a screen -as with personality-low scores can probably be trusted, but be skeptical of high ones -invite those who pass to take a proctored confirmation test ~using item response theory and adpative testing principle, the new test is tailored to their estimated ability level based on the first test -So the confimation test is much shorter, more efficient

Approaches to performance appraisal

Forced ranking Forced distribution Rating -graphic rating scales -behaviorally anchored rating scales -behavioral checklist Rating errors -distributional error -halo error -rater training is very important

Testing to help students learn: diagnostic assessment

Identify exactly where gaps in achievement or learning difficulties are occuring Break down skills or knowledge domains into very specific components EX: woodcock reading mastery tests ~letter identification ~word indentification ~word attack (phonics) -word comprehension -Passage comprehension (missing words) Psychoeducational test batteries -combine ability and achievement tests to form a comprehensive picture of the student-plan interventions

Can we trust our test score

If we are relying on people to provide us information about themselves, there are many reasons it may not be advisable to take that information at face value -faking -cheating -lack of self-knowledge -self-presentation -lack of motivation

Which is better (unstructured, structured)

In selection settings, structured interviews are much, much better than unstructured -much more reliable -much more valid -much more legally acceptable -period there is not it depends here

More challenges in assessing intelligence

Is intelligence fixed, or does it change -depends on how you're measuring it -some kinds of cognitive abilities decline with age ~usually after age 75 ~physical health and mental activity plays a role here too. The flynn effect -average scores (not individual) on intelligence tests appear to rise over time -but outcomes don't seem to change...not an increase in "true intelligence" Cultural issues in intelligence -if intelligence is tied to fundamental abilities like processing speed and pattern recognition we should be able to measure those things in the same way across cultures ~nonverbal measured show far fewer cultural differences but not verbal If intelligence is the ability to solve practical problems, we can"t separate that from culture

Social desirability scales

Items that theoretically no one can honestly strongly agree with Idea: high scores on these measures indicate a tendency to exaggerate or deliberately manipulate your responses to five a more positive impression of yourself ~which means you are also more likely to exaggerate or manipulate responses on other scales

Alternatives to G

Many people dislike the idea of general intelligence factor Cattell-fluid(patterns) vs crystallized(knowledge) intelligence ~not much more complex ~two different ways to be intelligent Thurstone-proposed 7 independent mental abilities ~but found they were all correlated Gardner-multiple intelligence ~not well supported-widely criticized that they were distinct Emotional intelligence -interesting...but we aren't very good at measuring it

Putting it all together

Most clinicians being with a clinical interview ~build rapport, start to develop hypotheses about diagnoses or other issues ~those who use projective methods may apply them here as well Then apply a broad assessment (MMPI) that covers a wide range of possible disorders and characteristics ~even if you know what the client is there for-why Standard battery includes at least: -one intelligence test -one personality test -one neurological test ~these test overlap and should be interpreted as a set After that add test that focus on a particular suspected problem area, such as depression, substance abuse, marital dissatisfaction ~single construct tests ~Beck Depression Inventory -helps pinpoint more specific diagnoses, level of severity

Integrated clinical assessment

Neuropsychological assessment-Low tech -focuses on very basis functions ~e.g. repeating digits, reassembling pictures, card sorting ~very simple tasks, not a test of intelligence Goal: assess brain damage of dysfunction -these kinds of tasks have been shown to be associated with particular brain regions ~if a person has difficulty with a task, we suspect brain injury or dysfunction

Summing up personality

Personality is hard to measure ~we are trying to see into peoples heads ~very little agreement on what we will find there Big philosophical divide between clinical and actuarial approached ~clinical: interpretive, individual-uncovering what is hidden from clients and others -actuarial=empircal, scientific-relying on what is observable and relatively objective *Goal: be confident that our conclusions mean what we thing they mean Hard to resolves this questions...evidence that one group relies on is dismissed by the other

Testing for employee selection

Probably the most common appliation of testing in organizational settings...but not the only Goals are a little bit different from some other testing contexts -predictive -comparative Tests can be used for -screening ~does this person meet essential minimum requirements -selection ~which applicants will we offer a job -classification/placement ~what is the best role in the organization for this person

Scope and context

Scope=how broad a construct are we measuring ~some tests attempt to measure all of personality-multiple dimensions ~some tests focus on one of a few important traits Context -Frame of reference=instructions to the respondent about what kind of behaviors or situations to focus on ~how you are in general ~how you would like to be in the future ~how you behave at work ~how you behave when you are comfortable and relaxed -why does this matter

Empirical keying

Sometimes, good prediction may be seen as more important than having a good theory Empirical keying=selecting items on the basis of whether they predict a criterion or not -start with a large pool of items -administer to two groups of people *those who definitely fit the criterion *a randomly selected sample -find items that differentiate the two groups-keep those Items may have no apparent relationship to each other or the construct -but if they predict, it doesn't matter

Spearman's theory

Spearman found that all intelligence tests were correlated with one another to some extent ~thus, he argued that there is a general common factory g, underlying all intelligence tests Tests are more or less saturated with g -tests with a lot of g measure overall intelligence -tests with less g measure more specific abilities -specific abilities may also form clusters (1 general mental ability to many specific abilities) Later researcher expanded on spearmans theory Ex: g math abilities algebra Intermediate level clusters called group factors Identifying specific abilities can go on and on ~how detailed you are depends as usual on your purpose

Achievement testing

Specifically focused on acquired skills or information-what have you learned ~in reference to a specific learning experience Can be very broad ~what have you learned in four years of high school Or very specific -have you mastered the material from yesterdays reading assignment Useful at the beginning, middle, and end of learning Often combined into test batteries ~multiple subtests-not necessarily believed to measure a common construct

Structured clinical interviews

Standard (published) set of questions/prompts ~with some built-in flexibility-semi structured Ex: Structured clinical interview for DSM disorders Scoring guides/trainings available ~compare your ratings to those of others How do structured clinical interviews address our 3 concerns

Research says

Statistical prediction is a little more accurate -if so why do so many people still use clinical predicition Key seems to be consistency: ~when clinicians make judgments in a systematic was, statistical prediction has a smaller advantage

Simulation based assessments

Stimulation exercised ~a kind of work sample, but often more interpersonal -put the applicant in a stimulated work-related situation and see what they do ~scored by trained observers and assessors A combination of multiple simulation exercises and other tests=an assessment center These types of assessments are popular with applicants -high face validity -reduced adverse impact -good predictors too

Specialized interviews

Stress interviews ~questions, context, interviewer behavior combine to place pressure on the interviewee ~why would you do this? Hypnotic and cognitive interviews ~used to aid recall-forensic psychology, trauma recovery, etc. ~goal of both is to put the interviewee in a focused state of mind to remember events clearly ~hypnosis is controversial-cognitive interviews try to create the same focus and relaxation with hypnotizing Collaborative interview -involving the interview directly in the assessment process ~what do you want to accomplish here

Adverse impact

Tests are potentially discriminatory when they cause adverse impact Adverse impact means that applicants from the majority group are selected at the disproportionate rate compared to minority applicants -key: proportion ~you do not have to hire equal numbers of majority or minority applicants ~but minority applicants should have the same odds of being hired -The 80% rule: the selection ratio for a minority group needs to be at least 80% of the selection ratio for the most selected group -not 80% of the selected numbere-80% of the ratio The catch: minority group members tend to score lower on cognitive ability tests than majority group members

Benefits of testing students

Tests increase motivation ~you will learn and remember information better when you know you will be tested on it ~no matter how intrinsically motivated you are Tests improve retention and transfer of learning ~practice explaining, applying, interpreting class material Tests help students self-assess ~feedback about what you know and what you'd don't Tests give information about teaching effectiveness ~if most of the class performs poorly on a test or item

Issues with social desirability scales

They are themselves quite remarkably easy to fake They are not related to other kinds of faking behavior ~social desirability scales do not predict who will fake in lab settings Social desirability scores are positively related to quite a few important criteria (including job performance) THis means they don;t work: -statistically controlling for social desirability often reduces the correlation between your real predictor and your criterion So social

How do we develop personality measures

Three major approaches: -theoretical-top-down -factor analytic("data reduction") -empirical or criterion keying Theoretical approach is what it sounds like -start with a theory about the structure of personality or an important trait, write items to reflect that trait, construct validate -a very common approach -what might be some disadvantages to this approach

Clinical interviews cont.

Three phases to a typical unstructured interview: Inital: -build rapport -get client's perception of the problem -observe the client Middle: -social and medical history -observe coping behaviors -form and test hypotheses about diagnosis -prognosis-will treatment work and how long will it take Termination: -ask if the client has questions -summarize the interview -explain and schedule further assessments or treatments

Integrity testing

Two approaches: -Overt integrity test ~measures attitudes about honestly-"i have used my employers office supplies for personal purposes -personality-based -measures personality traits that have been shown to be related to honesty-e.g. risk taking, impulse control Overt tests are direct while personality based tests are indirect

Words

Word association or free association ~emphasis is on speed of reaction-what connections are automatics ~trying to minimize self-presentation or self-editiing ~looking for unusal responses ~might use emotionall loaded or neutral words ~not much evidence linking these to diagnosis or outcomes Sentence completion ~asking fairly directly about needs, worries, values, etc. -Again, coded for themes and departures from good adjustment -Rotter Incomplete Sentences Blank shows good inter-scorer reliability and some validity data ~but transparent-potentially fakable