PSYC 421: Ch. 5 Questions

Sharon is computing the coefficient of split-half reliability of a test. She first divides the test into two equivalent halves. Identify the next step in the computation process. Multiple choice question.

She must calculate a Pearson r between scores on the two halves of the test.

If X is used to represent an observed score, T to represent a true score, and E to represent error, then X=_____.

T + E

Identify a true statement about an index of inter-item consistency.

A measure of inter-item consistency is calculated from a single administration of a single form of a test.

Identify a true statement about test-retest reliability estimates.

This measure is appropriate when evaluating the reliability of a test that purports to measure something that is relatively stable over time.

Identify a true statement about the Spearman-Brown formula.

It allows a test user to estimate internal consistency reliability from a correlation of two halves of a test formula.

The _____ is also referred to as the true score model of measurement.

classical test theory

The degree of the relationship between various forms of a test can be evaluated by means of an alternate-forms or parallel-forms coefficient of reliability known as the _____.

coefficient of equivalence

The simplest way to determine the degree of consistency among scorers in the scoring of a test is by calculating a coefficient of correlation known as _____.

coefficient of inter-scorer reliability

The influence of particular facets on a test score is represented by _____.

coefficients of generalizability

The variation among items within a test as well as variation among items between tests is known as _____.

content sampling

Developers examine the usefulness of test scores in helping the test user make decisions in the

decision study

Questions that can be answered with only one of two alternative responses are known as _____.

dichotomous test items

An acceptable way to split a test is to _____.

divide the test by content so that each half contains items equivalent with respect to content and difficulty

A test's reliability is conceived of as an objective measure of how precisely a test score assesses the domain from which the test draws a sample in _____.

domain sampling theory

Identify the component of an observed test score that has nothing to do with a testtaker's ability.

error

Identify a potential source of error variance during test administration.

examiner-related variables

True or false: The decision study is designed to tell a test user how much of an impact different facets of the universe have on a test score.

false

The degree of correlation among all the items on a scale is known as _____.

inter-item consistency

The degree of agreement or consistency between two or more raters with regard to a particular measure is known as _____.

inter-scorer reliability

A feature of the Spearman-Brown formula is that it _____.

is a specific application of a more general formula to estimate the reliability of a test that is lengthened or shortened by any number of items

The procedures of _____ provide a way to model the probability that a person with X ability will be able to perform at a level of Y.

item response theory

From the perspective of a test creator, a challenge in test development is to _____.

maximize the proportion of the total variance that is true variance and to minimize the proportion of the total variance that is error variance

A university administered a geometry test to a new batch of exchange students. All of the students received failing grades because they were unfamiliar with the language, English, that was used to administer the test. The test was designed to evaluate the students' knowledge of geometry but it reflected their knowledge and proficiency in English instead. This scenario is an example of _____.

measurement error

The term _____ refers to the inherent uncertainty associated with any measurement, even after care has been taken to minimize preventable mistakes.

measurement error

Identify an example of a polytomous test item.

multianswer questions

Which of the following forms of a test exist when the means and the variances of observed test scores, for each form of the test, are equal?

parallel forms

An estimate of the extent to which item sampling and other errors have affected test scores on versions of the same test when, for each form of the test, the means and variances of observed test scores are equal is known as _____.

parallel forms reliability

Questions with three or more alternative responses, where only one is scored correct or scored as being consistent with a targeted trait or other construct are known as _____.

polytomous test items

The reliability of a test is greater when the _____.

proportion of the total variance attributed to true variance is large

Identify the acceptable ways to split a test. (Check all that apply.)

randomly assigning items to one or the other half of the test assigning odd-numbered items to one half of the test and even-numbered items to the other half

In an observed distribution of test scores, the proportion of the total variance attributed to true variance refers to _____.

reliability

Identify an estimate of reliability that is obtained by correlating two pairs of scores obtained from equivalent halves of a single test administered once.

split-half reliability

Which of the following is a useful measure of reliability when it is impractical or undesirable to assess reliability with two tests or to administer a test twice?

split-half reliability

Variance is expressed as the _____.

standard deviation squared

If the variance of either variable in a correlational analysis is inflated by the sampling procedure used, then the resulting correlation coefficient _____.

tends to be higher

An estimate of reliability obtained by correlating pairs of scores from the same people on two different administrations of the same test is known as _____.

test-retest reliability

A student attempting an entrance examination is unable to concentrate due to certain emotional problems. In this scenario, the potential source of error variance during test administration is the _____.

testtaker variable

In the context of sources of error variance during test administration, identify an example of a testtaker variable.

the effect of medication on a respondent

True or false: A generalizability study examines how generalizable scores from a particular test are if the test is administered in different situations

true

True or false: In the language of psychometrics, reliability refers to consistency in measurement.

true

True or false: The alternate-forms and parallel-forms reliability of a test are an indication of whether two forms of a test are really equivalent.

true

True or false: The level of professionalism exhibited by examiners is a source of error variance.

true

According to classical test theory, _____ is a value that genuinely reflects an individual's ability (or trait) level as measured by a particular test.

true score

According to the generalizability theory, given the exact same conditions of all the facets in the universe, the exact same test score should be obtained which known as the _____.

universe score

In the generalizability theory, a(n) _____ replaces that of a true score.

universe score

A statistic useful in describing sources of test score variability is the variance ____ (σ2).

variance

If σ2 represents the total variance, σ2thσth2 represents the true variance, and σ2eσe2 represents the error variance, then the relationship of the variances in an observed distribution of test scores can be expressed as _____.

σ2= σ2th+σ2e