Chapter 6 - Validity
Definition of a valid test, a test is valid when?
A judgment or estimate of how well a test measures what it purports to measure in a particular context
Rating error
A judgment resulting from the intentional or unintentional misuse of a rating scale can include leniency and severity
Generosity error
Less than accurate rating or evaluation by a rater due to that rater's general tendency to be lenient or insufficiently critical; also referred to as leniency error; contrast with severity error
How can you ensure content validity?
Study guide, book, lecture
Given terms, are they constructs?
conscientiousness, intelligence, political power, self-esteem, and group culture
Severity error
is when a teacher grades too harshly
What is a false positive? and false negative?
test says you're pregnant but you're really not Says you don't have cancer but you actually do
Criterion-related validity
A judgment of how adequately a test score can be used to infer an individual's most probable standing on some measure of interest (i.e., the criterion)
Content validity
A judgment of how adequately a test samples behavior representative of the universe of behavior that the test was designed to sample Do the test items adequately represent the content that should be included in the test?
Which way is not a method of evaluating the validity of a test?
Face validity
Definition of face validity
A judgment concerning how relevant the test items appear to be If a test appears to measure what it purports to measure "on the face of it," it could be said to be high in face validity Rorchack test are NOT face valid Keyboarding test is face valid
What is the definition of a criterion
A criterion is the standard against which a test or a test score is evaluated
Definition of a test bias?
A factor inherent in a test that systematically prevents accurate, impartial measurement Bias implies systematic variation in test scores Prevention during test development is the best cure for test bias
Halo effect
A tendency to give a particular person a higher rating than he or she objectively deserves because of a favorable overall impression
Concurrent validity
An index of the degree to which a test score is related to some criterion measure obtained at the same time (concurrently) maybe your test is better
Predictive validity
An index of the degree to which a test score predicts some criterion, or outcome, measure in the future; tests are evaluated as to their predictive validity (SAT compared with GPA in college)
Which of the following is best viewed as varieties of criterion-related validity?
Concurrent and predictive validity
The form of criterion related validity that reflects the degree to which a test score is correlated with a criterion measure obtained at the same time is known as what kind of validity?
Concurrent validity
What type of validity evidence best sheds light on how a shorter and less expensive test compare with a longer and more expensive test?
Concurrent validity
Lists of tests and which one is most face valid
Content validity most** Criterion-Related Validity Construct Validity
The form of criterion related validity that reflects the degree to which a test score correlates with a criterion measure obtained at some time subsequent to the test is known as what kind of validity?
Predictive validity
What types of validity evidence best sheds light on whether a college admission test is valid for selecting students who will complete the program in 4 years?
Predictive validity
Definition and classification of a predictive and concurrent validity
Predictive validity: An index of the degree to which a test score predicts some criterion, or outcome, measure in the future; tests are evaluated as to their predictive validity Concurrent validity: An index of the degree to which a test score is related to some criterion measure obtained at the same time (concurrently)
Hit rate
Proportion of people a test actually identifies as possessing or exhibiting a particular trait, behavior, characteristic or attribute (MRI showing the exact amount of a disease)
Miss rate
Proportion of people the test fails to identify as having or not having a particular trait, behavior, characteristic, or attribute false positive and false negative
What is true of rating error?
Raters may be either too lenient, too severe, or reluctant to give ratings at the extremes (central tendency error) judgment resulting from the intentional or unintentional misuse of a rating scale
What is factor analysis?
Reduction/mathematical technique Answer (D)
What is a quota?
Remedy for previously unfair practices
Base rate
The extent to which a particular trait behavior, characteristic, or attribute exists in a population (on DSM IV, base rate OCD is 1-3% during lifetime of population, # is the extent to which is prevalent
What is an example of criterion?
achievement test scores, success in being able to repair a defective toaster, and student ratings of teaching effectiveness (all of these.)
Test scores maybe affected in pre and post testing by_______?
anything, all of the above
What is true of test bias as compared with test fairness?
bias prevents accurate and impartial measurements fairness is the extent to which a test is used in an impartial, just, and equitable way
Definition of a construct
building blocks of theories, helping to explain how and why certain phenomena behave the way that they do.