TES

194.) An orthogonal rotation of factors identified in a factor analysis produces a communality of .40 for one of the tests included in the analysis. This means that ____% of variability in the scores on the test is explained by the factor analysis.

o a factor loading is interpreted by squaring it to determine the amount of variability in test scores that is explained by ONE factor, o While a communality is interpreted directly as the amount of variability explained by ALL of the identified factors: § When a test's communality is .40, this means that the identified factors explain 40% of the variability in test scores.

Consensual observer drift __________ a measure's inter-rater reliability.

tends to artificially increase Consensual observer drift occurs when two or more raters communicate with each other while they're assigning ratings. It causes increased consistency (but often decreased accuracy) of their ratings and overestimates a measure's actual inter-rater reliability.

34.) In the context of factor analysis, "orthogonal" means:

A. statistically significant. B. statistically insignificant. C. uncorrelated. D. correlated. EPPP-P3-TES-Test Validity - Content and Construct Validity-07 Answer C is correct. The factors extracted (identified) in a factor analysis can be either orthogonal or oblique. Orthogonal factors are uncorrelated, and oblique factors are correlated.

3.) An assumption of classical test theory is that measurement error: A. is random. B. is systematic. C. is random and systematic. D. cannot be estimated.

EPPP-P3-TES-Item Analysis and Test Reliability-02 Answer A is correct. Classical test theory is based on the assumption that an examinee's obtained test score is due to a combination of "truth" and measurement error, with truth referring to the "true" amount of the characteristic measured by the test that the examinee has and measurement error being random (unsystematic).

An orthogonal rotation of factors identified in a factor analysis produces a communality of .40 for one of the tests included in the analysis. This means that ____% of variability in the scores on the test is explained by the factor analysis.

40 For the exam, keep in mind that o a factor loading is interpreted by squaring it to determine the amount of variability in test scores that is explained by ONE factor o While a communality is interpreted directly as the amount of variability explained by ALL of the identified factors: § When a test's communality is .40, this means that the identified factors explain 40% of the variability in test scores.

157.) Which of the following is used to estimate the effect of increasing a predictor's reliability on its criterion-related validity coefficient?

A. Spearman-Brown formula B. correction for attenuation formula C. coefficient of determination D. Cronbach's alpha EPPP-P3-TES-Test Validity - Criterion-Related Validity-10 Answer B is correct. Inadequate reliability is one of the factors that reduces a predictor's criterion-related validity coefficient, and the correction for attenuation formula is used to estimate the effect of increasing the reliability of the predictor and/or criterion on the predictor's criterion-related validity coefficient

26.) A test's __________ is the proportion of people who have a disorder and are correctly identified by the test as having the disorder.

A. sensitivity B. specificity C. positive predictive value D. negative predictive value EPPP-P3-TES-Test Validity - Criterion-Related Validity-09 Answer A is correct. Determining a test's sensitivity, specificity, positive predictive value, and negative predictive value are one way of evaluating its validity. A test's sensitivity refers to the test's ability to accurately identify people who have the disease or other attribute the test was designed to identify. It's calculated by dividing the number of true positives by the number of true positives plus false negatives.

157.) Which of the following is used to estimate the effect of increasing a predictor's reliability on its criterion-related validity coefficient?

B. correction for attenuation formula C. coefficient of determination D. Cronbach's alpha EPPP-P3-TES-Test Validity - Criterion-Related Validity-10 Answer B is correct. Inadequate reliability is one of the factors that reduces a predictor's criterion-related validity coefficient, and the correction for attenuation formula is used to estimate the effect of increasing the reliability of the predictor and/or criterion on the predictor's criterion-related validity coefficient

76.) To increase the difficulty level of a statistics test, you would add items that have an average item difficulty index (p) of: A. -1.0 B. -.50 C. .20 D. .90

EPPP-F1-TES-Item Analysis and Test Reliability-01 Answer C is correct. The item difficulty index (p) ranges from 0 to 1.0, with 0 indicating a very difficult item (none of the examinees in the tryout sample answered it correctly) and 1.0 indicating a very easy item (all examinees in the tryout sample answered it correctly). Therefore, to increase the difficulty of a test, you'd want to add items that have p values close to 0 - e.g., .20.

For a test that consists of 50 true/false questions, the optimal average item difficulty level (p) is: A. 1.0 B. .75 C. .50 D. .25

EPPP-P3-TES-Item Analysis and Test Reliability-01 Answer B is correct. The optimal difficulty level for test questions depends on several factors including the chance that examinees can choose the correct answers just by guessing. With regard to this factor, the optimal difficulty level falls halfway between 100% and the probability of choosing the correct answer by guessing: For true/false questions, the probability of guessing correctly is 50%, so the optimal difficulty level is halfway between 1.0 and .50, which is .75.

100.) All other things being equal, which of the following tests is likely to have the lowest reliability coefficient? A. a three-alternative multiple-choice test B. a four-alternative multiple-choice test C. a true/false test D. a fill-in-the-blanks test

EPPP-P3-TES-Item Analysis and Test Reliability-04 Answer C is correct. A test's reliability is affected by a number of factors including the chance that test questions can be answered correctly by guessing: As the probability of choosing the correct answer by guessing increases, the reliability of the test decreases. For this reason, true/false tests are less reliable than three-item and four-item multiple-choice tests which, in turn are less reliable than a fill-in-the-blanks test.

120.) A normal distribution of raw scores has a mean of 75 and standard deviation of 5. In this distribution, a raw score of 65 is equivalent to a: A. T-score of 45. B. stanine score of 3. C. z-score of -2.0. D. percentile rank of 16.

EPPP-P3-TES-Test Score Interpretation-12 Answer C is correct. In a normal distribution of scores that has a mean of 75 and standard deviation of 5, a score of 65 is two standard deviations below the mean and is equivalent to a T-score of 30, a stanine score of 1, a z-score of -2.0, and a percentile rank of 2.

11.) When using the multitrait-multimethod matrix to assess a test's construct validity, a large heterotrait-monomethod coefficient indicates which of the following? · A. adequate convergent validity · B. inadequate convergent validity · C. adequate divergent validity · D. inadequate divergent validity

EPPP-P4-TES-Test Validity - Content and Construct Validity-06 Answer D is correct. The heterotrait-monomethod coefficient indicates the correlation between the test being evaluated and a measure of a different trait (heterotrait) using the same method of measurement (monomethod). For example, if the multitrait-multimethod matrix is being used to assess the construct validity of a self-report measure of assertiveness, a heterotrait-monomethod coefficient might indicate the correlation between the self-report measure of assertiveness and a self-report measure of seriousness. When this coefficient is small, it provides evidence of the test's divergent validity; when it's large, it indicates that the test has inadequate divergent validity. (A measure's construct validity is demonstrated when it has adequate levels of both divergent and convergent validity.)

Sensitivity

Sensitivity is the proportion of people with the disorder who are identified by the test as having the disorder. It's calculated by dividing the true positives by the true positives plus the false negatives (TP/TP + FN).

A test's __________ refers to its ability to correctly identify individuals who are true negatives

Specificity Determining a test's sensitivity, specificity, positive predictive value, and negative predictive value are one way of evaluating its validity. A test's specificity refers to its ability to accurately identify people who do not have disorder or other attribute the test was designed to identify. It's calculated by dividing the number of true negatives by the number of true negatives plus false positives.

Specificity

Specificity is the proportion of people without the disorder who are identified by the test as not having the disorder. It's calculated by dividing the true negatives by the true negatives plus the false positives

129.) The item discrimination index (D) ranges from:

The value of D ranges from -1.0 to +1.0. When D is +1.0, this indicates that all examinees in the high-scoring group answered the item correctly and all examinees in the low-scoring group answered it incorrectly. Conversely, when D is -1.0, this indicates that all examinees in the low-scoring group answered the item correctly and all examinees in the high-scoring group answered it incorrectly.