Ch. 11 Measuring Research Variables

If a thermometer measured the temperature in an oven as 400 degrees give (5) days in a row when the temperature was actually 337 degrees, this measuring instrument would be considered

Reliable but not valid

The type of scale that uses items such as "Worthless-Valuable" is called a

Semantic Differential Scale

The use of intraclass correlation to estimate reliability in motor performance instead of Pearson r has the advantage(s) of

Estimating reliability for more than two trials

Use two (2) examples to explain the statement, "if the results of a test are valid, they will also be reliable, but if the results are reliable, they may or may not be valid."

1. Weighing yourself repeatedly on a broken scale would give reliable results, but not valid ones 2. Using the wrong formula/equation to calculate something. You did the math correctly, but used the wrong equation, so got the wrong answer

Validity is determined by finding the correlation between scores on

A test and some independent criterion

A predictive validity techniques involving the construction of a two-way grid of percentages of students at different levels of an aptitude test who achieved different grades is called

An Expectancy Table

Which of the following is a technique for providing information about the reliability of an instrument?

Coefficient Alpha

A physical education teacher develops a skill test in volleyball. After administering the test to 50 students, she asks the volleyball coach to rate the students on volleyball skills. She then correlates the students' test scores with the coach's rating. This an example of what type of validity?

Criterion Validity

Explain the difference between interrater and intrarater reliability. What evidence could you provide to demonstrate that your measure had interrater reliability?

Interrater (Intertester) reliability is the degree to which different testers can obtain the same scores on the same participants also called Objectivity

For scores from a test to have good content validity, the following statement(s) must be true:

The percentage of points for each topic area reflects the amount of emphasis given that topic

The maximum value of item discrimination in norm-referenced test items occurs when the item difficulty is

.50

When a tester divides the number of students who answered a test item correctly by the number of students who attempted the item, the result is called

Index of Difficulty

A teacher wishes to determine the reliability of three (3) trials on a performance test. He uses ANOVA to obtain the reliability coefficient. This technique is called

Intraclass Correlation

Discuss how the concept of standard error of measurement relates to one's interpretation of a student's score on a test

Standard error of measurement is directly related to a test's reliability. The larger the standard error of measurement, the lower the test's reliability

A threat to the validity of rating scales is that the rater may tend to let her previous knowledge of the student in another capacity (e.g., another course) influence the rating. The threat is called the

Halo Effect

In an item analysis, 14 out of the 20 high-ability students answered an item correctly, whereas only 8 out of the 20 low-ability students did. What is the discrimination index for this item?

.30

Reliability of a measure may be established by

1. Correlating scores from alternate forms of the same test 2. Giving the test to the same people on two (2) different occasions and correlating the two (2) sets of scores

The author of a test of anxiety proceeds on the premise, based on the literature, that the performance of people with high anxiety suffers when they are under stress. In an experiment, the author finds that people who scored as highly anxious on the test performed more poorly on a stressful task. The test author maintained that this finding was evidence of what kind of validity?

Construct

The accuracy with which a 12-min run estimates maximal oxygen consumption in a group of male high school seniors represents

Construct Validity

Comparing tests items with the course objectives (course topics) checks which type of validity?

Content