Chapter 6: Validity

Pataasin ang iyong marka sa homework at exams ngayon gamit ang Quizwiz!

Construct

(latent variables) are scientifically developed concepts, ideas, or hypotheses that are used to describe or explain behavior

table of specifications

(test blueprint) is a two-dimensional chart that guides instrument development by listing the content areas that will be covered on a test, as well as the number (proportion) of tasks or items to be allocated to each content area

manifest (or observed) variables

A construct is defined or inferred by a group of interrelated variables or dimensions that can be directly measured or observed, which are known as ___________________________.

uncontaminated

A criterion measure should be ______________________, meaning that the measure should not be influenced by any external factors that are unrelated or irrelevant to the criterion.

confirmatory factor analysis

A procedure in which researchers, using factor analysis, consider the theory associated with a test and propose a set of underlying factors that they expect the test to contain; they then conduct a factor analysis to see whether the factors they proposed do indeed exist.

fluid intelligence and crystallized intelligence

According to Horn and Cattell, intelligence encompasses two general types, and these are:

the predictor variable, the criterion, the criterion measure

An aptitude test (_____) could be designed to predict job performance (_____) as measured by supervisor ratings (_____).

measuring the wrong thing or may be used in inappropriate ways.

Assessment results may be highly reliable, but may be

reliable

Because an instrument is judged based on its relationship to a criterion, the criterion itself needs to be measured accurately; it must be ___________________.

directly or observed

Constructs cannot be measured _______________ or _________________.

response processes

Evidence based on _______________________, sometimes viewed as part of content validity, provides validity support that examinees used intended cognitive and psychological processes during test taking.

validity coefficients

Evidence of relations in other variables is evaluated by _____________________, which indicate the relationship between test results and a criterion measure.

-factors within the test: ambiguous, inappropriate, or poorly constructed test items; too few test items; improper arrangement of items; identifiable patterns of answers -administration and scoring procedures: failure to follow standard directions and time limits, giving test takers unfair aid, scoring errors -test-taker characteristics: emotional disturbances, test anxiety, lack of motivation -inappropriate test group: giving the test to individuals who differ from the validation group

Other factors that can impact the validity of the interpretation of test results:

test content, response process, internal structure, relations to other variables, and testing consequences

Sources of information upon which an argument for the validity of an instrument can be made

(1) Clearly delineate the construct or the content domain to be measured. (2) Develop a table of specifications (3) The actual test items are written (4) Outside consultants are recruited to review the test items to determine if they represent the content domain

Steps in the content validation process

test validation

The process of __________________________ involves gathering and evaluating multiple sources of validity evidence that support the interpretations and inferences made from test scores.

relevant

To determine if a criterion measure is __________________ to the intended purpose of the test, one must ascertain whether it is a valid representation of the behavior or outcome in question.

construct underrepresentation and construct-irrelevant variance

Two main threats to validity

1. Validity refers to the appropriateness of the use and interpretation of test results, not the test itself. 2. Validity is a matter of degree; it does not exist on an all-or-none basis. 3. Validity is a unitary concept. It is a single, unified concept that centers on construct validity, which is supported by sources of validity evidence. 4. Validity is specific to some particular group, use, or purpose of the test score interpretations--no test is valid for all purposes or all groups.

When using the term validity in relation to tests and measurement, consider:

criterion

a direct and independent measure that the test is designed to predict or be correlated with

exploratory factor analysis

a method of factor analysis in which researchers do not propose a formal hypothesis but instead use the procedure to broadly identify underlying components

Factor analysis

a statistical procedure that analyzes the relationships among test components to determine whether a test is unidimensional

predictor variable

a variable whose scores are used to estimate scores of a criterion variable

true negative

correctly predicting failure

true positive

correctly predicting success

Age differentiation studies

focus on demonstrating validity by showing the degree to which test scores increase or decrease with age

Construct underrepresentation

implies that a test is too narrow and fails to include important dimensions or aspects of the identified construct

Criterion measures

indexes designed to assess or measure the particular criterion

Criterion-related evidence

involves examining the relationships between test results and external variables that are thought to be a direct measure of the construct

discriminant validity evidence

is based on consistently low correlations between the test and other tests that are supposed to measure different constructs

false negative error

is falsely predicting that a negative outcome would occur and it does not

false positive error

is falsely predicting that a positive outcome would occur and it does not

evidence of convergent validity

is obtained by correlating instrument results with the results of other instruments that assess the same construct

Construct-irrelevant variance

means that the instrument is too broad and contains too many variables, many of which are irrelevant to the construct

false negative

predicting failure but having a positive or successive outcome

false positive

predicting success but having a negative or failing outcome

Crystallized intelligence

refers to acquired knowledge and ability obtained through education and personal experience continues to increase through middle age and maintains during old age

Content validity evidence

refers to evidence of the relationship between a test's content and the construct it is intended to measure.

evidence based on consequences of testing

refers to examining the intended and unintended consequences of the use of test results

test homogeneity

refers to how uniform the items and components of a test are in measuring a single concept this is one means of providing evidence of the internal structure of a test

response processes

refers to the actions, thought processes, and emotional traits that the test taker invokes in responding to a test

Validity

refers to the adequacy and appropriateness of the uses of assessment results

Concurrent validity

refers to the degree to which a predictor variable is related to some criterion at the same time (concurrently)

Fluid intelligence

the ability to solve problems and adapt to new situations improves through childhood and adolescence, then begins to decline as early as late adolescence and into adulthood

predictive validity

the degree to which a test score estimates some future level of performance

Validity

the degree to which all the accumulated evidence supports the intended interpretation of test scores for the proposed purpose

construct validity

the extent to which test results can be interpreted in terms of a particular construct

a false positive error or a false negative error

two types of prediction errors

exploratory and confirmatory

two ways to conduct factor analysis

Group differentiation (contrasted group) studies

used to analyze whether an instrument differentiates between different groups of individuals in ways predicted by theory


Kaugnay na mga set ng pag-aaral

Smartbook: Chapter 7 Accounting Information Systems

View Set

Health Psych Chapter 4 Adherence

View Set

Chapter 08-QUIZ-Consideration of Internal Control in an Information Technology Environment

View Set

Differentiate factors that determine readiness to learn from those that determine ability to learn.

View Set

Lesson 5 English Synoyms, Antonyms, and Analogies

View Set

International Business Chapter 12

View Set