Psychological Testing Midterm

Representativeness Heuristics

"...judging the relationship between variables on the basis of similarity alone disregards other potentially significant factors..." Clinicians base their diagnoses on the degree to which an individual is thought to resemble those making up a diagnostic category Stereotypes (How similar is person X to the typical person in diagnostic category Y?) Prototypes (How similar is person X to the person showing all characteristics associated with diagnosis Y?) Exemplars (How similar is person X to those that the clinician has seen in their personal work?)

Hindsight Bias

"ad hoc fallacy" or tendency to explain an event after it occurs, unaware a biased prediction has occurred. "knew it all along" Foster unrealistic sense of confidence

Scaling

"the process by which a measuring device is designed and calibrated, and the way numbers - scale values - are assigned to different amounts of the trait, attribute, or characteristic being measured"

Test

A measuring device or procedure in which a sample of behavior is obtained, evaluated, and scored.

Test Retest Reliability

A reliable test should yield similar scores over time Used to gauge consistency of scores over time for the same person (via correlation) Test given to same group on two occasions, scores on first and second exam correlated

Psychological Test Definition

A standardized procedure for sampling behavior and describing it with categories or scores. A measurement tool or technique that requires a person to perform one or more behaviors in order to make inferences about human attributes, traits, or characteristics or predict future outcomes. By inference, we mean using evidence to reach a conclusion.

Non random/systematic measurement error

A test consistently measures something other than the target Occurs when source of error always increase or decrease a true score Does not lower reliability of a test since the test is reliably inaccurate by the same amount each time

Indirect Behavioral Assessment

ADHD assessment has scales designed to be completed by parents or teachers at more than one setting

Affective Assessment

Affective Assessment assesses all noncognitive features of an individual, including temperament, clinical disposition, personality, attitudes, values, and interests. Structured Inventories for diagnostic purposes, hypothesis testing, treatment planning, and progress evaluation (Minnesota Multiphasic Personality Inventory-II (MMPI-2) Strong Interest Inventory Unstructured assessment involves the use of projective techniques and qualitative methods Based on psychoanalytic theory, present the client with unstructured, ambiguous stimuli, allowing the client to "project" thoughts and feelings onto the stimulus Inkblots, pictures, incomplete sentences... yield insights into a client's motivation, personality, values, etc.

Late 19th century: Intelligence Tests

Alfred Binet and the Binet-Simon Scale Lewis Terman and the Stanford-Binet Intelligence Scales David Weschler and the Weschler-Bellevue Intelligence Scale and the Weschler Adult Intelligence Scale

Psychological Test Differences

Behavior performed Construct measured and outcome predicted Content Administration and format Scoring and interpretation Psychometric quality

Bootstrap approach

Combination of the two; sequential method First write items based on theory, next validate items based on using samples and statistically analyzing the findings.

Clinician Bias in relation to patient characteristics

Cultural Identities of the pt. influenced the diagnostic, therapeutic, and prognostic decisions made by clinical psychologists. E.g., increased organic dos diagnoses with increase in pt. age E.g., increased Borderline personality dos (i.e., emotional dysregulation, fear of abandon, rejection hypersensitivity...) among female pts. E.g., Increasing age along with poor health ------ less optimistic psychologist predictions regarding treatment and prognosis ("ageism" and "healthism" in everyday clinical practice

Limitations of Psychological Tests

Decisions about peoples' lives should not be made on the basis of a single high-stakes test score. Tests are biased and unfair to minorities and women Tests create anxiety and stress Tests label and categorize Test developers dictate what students must know or learn "Teaching to the test" inflates scores Multiple-choice questions punish creative thinkers; trivialize the complexities of the learning process

Validity

Does your test actually measure what it is designed to measure? Truthfullness

Reliability

Does your test yield consistent results? Consistency

Measurement Error and Reliability

Error reduces the reliability or repeatability of psychological test results A crucial assumption of classical theory is that unsystematic measurement errors act as random influences Main features of classical theory Measurement errors are random Mean error of measurement = 0 True scores and errors are uncorrelated: rTE = 0 Errors on different tests are uncorrelated: r12 = 0

Correlation Coefficient [but comparing the same person]

Expresses the degree of linear relationship between two sets of scores obtained from the same person Rxx = True score variance (T)/Total Variance of test scores (O) O = T + E If measurement error is very small, close to zero, R = If measurement error is very large, R =

Types of Validity

Face Validity Content-Related Validity Construct Validity Criterion-Related Validity

Possible Sources of Measurement error

GROUP 1 - TEST ITSELF GROUP 2 - TEST ADMINISTRATION GROUP 3 - TEST SCORING GROUP 4 - TEST TAKERS

Types of Tests

Group vs. individual Intelligence tests Aptitude tests Achievement tests Creativity tests Personality tests Interest inventories Behavioral procedures Neuropsychological tests

Psychological Test Similarities

Limited sample of behavior (an observable and measurable action) Standardized procedure Behavior used to make inferences about some psychological construct (an underlying, unobservable personal attribute, trait, or characteristic of an individual that is thought to be important in describing or understanding human behavior)

central tendency

MEAN MEDIAN MODE

Neuropsychological Tests

Measure cognitive, sensory, perceptual and motor performance to determine the extent, locus, and behavioral consequences of brain damage

Response Bias from Examinees

Motivation Fake good or fake bad (malingering) Integration of well-developed validity scales, review both self- and other-report data, use of objective mental-status examination findings...

Creativity Tests

Novel, original thinking

Confidence Interval Calculations

O=X+E 95% CI = X +/- 2(SEM) 99.7%CI = X +/- 3(SEM)

Behavioral Procedures

Objectively describe and count the frequency of a behavior//Identify the antecedents and consequences of the behavior

Availability Heuristic

Pertains to the situation where information used for prediction or decision making is that which is most easily accessed or recalled Illusory Correlation (forming test sign-symptom correlations without empirical evidence) Correlations based on clinicians' personal associations and projections than on data Recall/memory availability and vividness can limit judgement accuracy Clinician's memory capacity Only remembered one piece of information

Interest Inventories

Preference for certain activities or topics//Occupational/career interest

Psychological Test assumptions

Psychological tests measure what they purport to measure or predict what they are intended to predict An individual's behavior, and therefore test scores, will typically remain stable over time Individuals understand test items the same way Individuals will report accurately about themselves Individuals will report honestly about their thoughts and feelings The test score an individual receives is equal to his or her true score plus some error

Measures of variability

Range Standard Deviation

Misrepresentation of change

Regression to the Mean - extreme observations, scores, or performances on one occasion will likely be followed by less extreme results on future occasions E.g., "His depression scale score is lower than the one he scored two months ago. He is definitely better! The treatment definitely worked!"

Split Half Reliability

Reliability gauged by splitting a test into two parts and comparing an individual's scores on both halves If a test if split into two (odd vs. even questions), the two halves should yield similar scores for a given individual

Group One Test Itself Error Factors

Representation of the items, wording of the items, culturally biased items, linguistically biased items, double-barreled items

early 1900s: personality tests

Robert Woodworth and the Personal Data Sheet Carl Jung and the Rorschach Inkblot Test Henry Murray and C. D. Morgan and the Thematic Apperception Test

Standard Error of measurement

SD * square root of 1-r (reliability)

Empirical approach to test construction

Sampled large samples on random items - identify items that relate to the construct attempted to measure

Content Validity

Shows evidence that the test items adequately reflects the test domain - via literature review, consulting subject matter experts To ensure that you comprehensively cover/capture the test domain of interest

Criterion Related Validity

Shows that a test is able to predict the behavior that it is designed to predict (criterion/outcome) To show that your test scores actually lead to predicted behavioral outcomes either now (concurrent) or in the future (predictive)

Assessment

Systematic procedures for making inferences about characteristics of people. Broader and more comprehensive than testing.

Adjustment and Anchoring

Tendency for final judgments to be biased in the direction of initially reviewed data Judgement overly influenced by the first page of the clinical material reviewed Potential for reviewers to reach different opinions regarding the same evidence in the event that this evidence is reviewed in different sequences

Test vs. Assessment

Test Assessment

Construct Validity

The extent to which the variables being studied represent the constructs they are purported to measure. (Are we comparing oranges to apples?) Shows evidence that the test items adequately capture the concept (or construct) that it is designed to capture To show that your test correlates positively (convergent) with similar test and negatively (discriminant) with dissimilar tests

Confirmation Bias

The tendency to selectively attend to information that is in line with one's viewpoint while minimizing or disregarding data that may disconfirm this position. e.g., "selective" data review

Rational Approach to test construction

Theoretical approach

Personality Tests

Traits, qualities or behaviors that determine a person's individuality//Checklists, inventories, and projective techniques

Observed Exam Score =

True Exam Score + error

early mid 1900s vocational tests

U.S. Employment Service and the General Aptitude Test Battery

Inter Rater Reliability

Used to gauge the consistency of ratings across multiple raters (e.g., figure skating judges, diving judges) In order for a scoring method to be reliable, independent ratings by multiple judges should be highly similar

Parallel Forms Reliability

Used to gauge the equivalence of two or more different versions of an assessment that measure the same concept Different versions of the same test should yield highly similar scores for a given individual Test developer creates two forms of the test Assesses equivalence of two parallel forms scores on both tests correlated

Internal Consistency Reliability

Used to see if responses to a set of similar items are uniform (or consistent) Measured using Coefficient Alpha a way to compare individuals' scores on all possible ways of splitting the test in halves (instead of just one random split). Logic: A reliable test should contain only those questions that measure the same concept Heterogeneous test (or homogenous subsets) is split in half and scores on first half compared with scores on second half Assesses how related items or groups of items are to one another scores on both halves correlated

Psychometrician

a specialist in psychology or education who develops and evaluates psychological tests.

Intelligence Tests

ability in global areas

Ratio Variables

an interval scale, but with a true zero point. temperature, length, number of children, income...

Group Tests

are designed for administration to groups of participants simultaneously. Advantage - speed and efficiency Limitations in the type of test formats available (e.g., paper-and-pencil, computer-based). A major drawback is the inability to observe all examinees and control relevant individual factor (e.g., client motivation, mood).

Individual Tests

are often used for diagnostic decision making and generally require some interaction b/t the examiner and examinee. Allow the two to establish rapport, reduce anxiety Often the administrator require special training Can provide information about the client's presentation, affect, attitudes, verbal and nonverbal behaviors, etc.

Speed and Power Tests

bg

Aptitude Tests

capability in a specific task

Diagnostic Overshadowing Bias

client's problem receives inadequate treatment b/c attention is diverted to an more salient characteristics E.g., Gay or lesbian pt. ------ a clinician might perceive the presenting problem as related to conflicts over sexual orientation and fail to address other critical issues E.g., Individuals with AIDS were less likely to be referred to treatment for emotional symptoms (depression) than patients with other medical problems.

Heuristics

decisional simplification strategies - Mental Shortcuts Representativeness Availability Adjustment and Anchoring

Achievement Tests

degree of learning, success, or accomplishment in a subject or task

Behavioral Observations

direct indirect

Nonstandardized tests

do not do the above.

Random/Unsystematic Measurement Error (reduce reliability)

effects are unpredictable and inconsistent Random in nature Will increase and decrease a person's score by exactly the same amount with infinite testing Cancels itself out Lowers reliability of a test

Standardized vs nonstandardized tests

f

Group vs. Individual Tests

ff

Objective vs subjective scored tests

ff

Cognitive vs affective tests

gg

Norm vs criterion referenced tests

gr

Standardized Tests

have specific conditions for administration, timing, and scoring. To ensures that no matter who the examiner or examinee, the test will be administered under strict, replicable conditions. Allow comparability of scores and interpretations across time/situation Conform to rigorous test construction guidelines

Speed Test

is to measure how many of the simple items a person can complete within a certain amount of time; the score is simply the number of (correct) items completed within the time limit. Coding tests of the WAIS-IV

Objective Tests

leave no doubt as to the correctness of a given answer: correct answers are predetermined and require no judgement on the part of the examiner. Multiple-choice, true-false items Help control subjective bias in scoring (interscorer reliability)

Test Administration Error

level of noise, room temperature, lighting, inconsistent way of giving instructions and/or answering questions

Cognitive Tests

memory, perceptual, processing and reasoning capacities Intelligence Tests: measure a person's ability to learn, solve problems, and understand increasingly complex or abstract information Wechsler ADult Intelligence Scale - Fourth Edition (WAIS-IV) Aptitude Tests: predict a person's capacity to perform some skill or task in the future (e.g., college) SAT (actually, can only predict the freshman yr of college) Achievement Tests: measure knowledge students have acquired through instruction or training up to a certain point in their academic career.

Test Takers Error

motivation, anxiety, attention, and fatigue level

Norm Referenced Tests

often Standardized tests, are administered to a representative sample of participants (called a standardization sample), to determine average performances for various subgroups of interest (called a norm group). A client's score can then be compared to the average of the standaridization sample (i.e., Average, Above Average, Below Average) Commonly used to assess intelligence, achievement, personality, cognitive functioning... The raw score is transformed into some type of standard score or percentile rank Example, SAT

Nominal Variables

qualitative system; if numbers are used they are arbitrary. sex, ethnicity, college major, political affiliation...

Ordinal Variables

ranking according to characteristics, with intervals not necessarily being consistent between ranks. tallest to shortest, Olympic medalists...

Interval Variables

ranking with equal space between units but with no true zero. many exams/psych tests (at least aspire!) to interval...

Subjective Tests

require the examiner to make a judgment on the quality of the response in scoring an item Essay, open-ended questions Can elicit important and rich client information

Test Scoring Error

scorers' levels of skills and qualifications, criteria of scoring, subjectivity

Criterion Referenced Tests

tests compare a person's score to a predetermined standard or level of performance - a criterion. Either "passed" or "failed;" a cutoff score For example, a depression screening test with a total score of 20 and above - indicate further evaluation DSM-V diagnostic checklists (3 or more of the six listed symptom criteria make a clinical diagnosis)

Face Validity

the degree to which a procedure, especially a psychological test or assessment, appears effective in terms of its stated aims.

Direct Behavioral Assessment

the examiner physically present in the same environment with the client and uses a data collection procedure to assess the frequency, duration, and/or magnitude of one or more target behaviors A school counselor may observe a 2nd-grade student referred for overactivity in the classroom using a time-on-task observation system Has a natural control group

Power Test

the score is an indicator of the skills or abilities possessed by the examinee, without the pressure of time limits. Items vary in difficulty, and examinees eventually miss or could not complete many items in a row (reach the ceiling level), and the administration ceases. For example, Matrix Reasoning of the WAIS-IV

Past Behavior Heuristic

use of previous behavior to predict future behavioral outcomes Patient A's past substance use experience ---Patient A is an addict today!