Chapter 6 - Validity

Definition of a valid test, a test is valid when?

A judgment or estimate of how well a test measures what it purports to measure in a particular context

Rating error

A judgment resulting from the intentional or unintentional misuse of a rating scale can include leniency and severity

Generosity error

Less than accurate rating or evaluation by a rater due to that rater's general tendency to be lenient or insufficiently critical; also referred to as leniency error; contrast with severity error

How can you ensure content validity?

Study guide, book, lecture

Given terms, are they constructs?

conscientiousness, intelligence, political power, self-esteem, and group culture

Severity error

is when a teacher grades too harshly

What is a false positive? and false negative?

test says you're pregnant but you're really not Says you don't have cancer but you actually do

Criterion-related validity

A judgment of how adequately a test score can be used to infer an individual's most probable standing on some measure of interest (i.e., the criterion)

Content validity

A judgment of how adequately a test samples behavior representative of the universe of behavior that the test was designed to sample Do the test items adequately represent the content that should be included in the test?

Which way is not a method of evaluating the validity of a test?

Face validity

Definition of face validity

A judgment concerning how relevant the test items appear to be If a test appears to measure what it purports to measure "on the face of it," it could be said to be high in face validity Rorchack test are NOT face valid Keyboarding test is face valid

What is the definition of a criterion

A criterion is the standard against which a test or a test score is evaluated

Definition of a test bias?

A factor inherent in a test that systematically prevents accurate, impartial measurement Bias implies systematic variation in test scores Prevention during test development is the best cure for test bias

Halo effect

A tendency to give a particular person a higher rating than he or she objectively deserves because of a favorable overall impression

Concurrent validity

An index of the degree to which a test score is related to some criterion measure obtained at the same time (concurrently) maybe your test is better

Predictive validity

An index of the degree to which a test score predicts some criterion, or outcome, measure in the future; tests are evaluated as to their predictive validity (SAT compared with GPA in college)

Which of the following is best viewed as varieties of criterion-related validity?

Concurrent and predictive validity

The form of criterion related validity that reflects the degree to which a test score is correlated with a criterion measure obtained at the same time is known as what kind of validity?

Concurrent validity

What type of validity evidence best sheds light on how a shorter and less expensive test compare with a longer and more expensive test?

Concurrent validity

Lists of tests and which one is most face valid

Content validity most** Criterion-Related Validity Construct Validity

The form of criterion related validity that reflects the degree to which a test score correlates with a criterion measure obtained at some time subsequent to the test is known as what kind of validity?

Predictive validity

What types of validity evidence best sheds light on whether a college admission test is valid for selecting students who will complete the program in 4 years?

Predictive validity

Definition and classification of a predictive and concurrent validity

Predictive validity: An index of the degree to which a test score predicts some criterion, or outcome, measure in the future; tests are evaluated as to their predictive validity Concurrent validity: An index of the degree to which a test score is related to some criterion measure obtained at the same time (concurrently)

Hit rate

Proportion of people a test actually identifies as possessing or exhibiting a particular trait, behavior, characteristic or attribute (MRI showing the exact amount of a disease)

Miss rate

Proportion of people the test fails to identify as having or not having a particular trait, behavior, characteristic, or attribute false positive and false negative

What is true of rating error?

Raters may be either too lenient, too severe, or reluctant to give ratings at the extremes (central tendency error) judgment resulting from the intentional or unintentional misuse of a rating scale

What is factor analysis?

Reduction/mathematical technique Answer (D)

What is a quota?

Remedy for previously unfair practices

Base rate

The extent to which a particular trait behavior, characteristic, or attribute exists in a population (on DSM IV, base rate OCD is 1-3% during lifetime of population, # is the extent to which is prevalent

What is an example of criterion?

achievement test scores, success in being able to repair a defective toaster, and student ratings of teaching effectiveness (all of these.)

Test scores maybe affected in pre and post testing by_______?

anything, all of the above

What is true of test bias as compared with test fairness?

bias prevents accurate and impartial measurements fairness is the extent to which a test is used in an impartial, just, and equitable way

Definition of a construct

building blocks of theories, helping to explain how and why certain phenomena behave the way that they do.