Chapter 8 quant

147. The stability of questionnaires can be assessed by using what technique? a. equivalent forms b. split-halves c. test-retest d. internal consistency

C

45. What can be said about a test item that discriminates negatively? a. It is not reliable. b. It is too difficult. c. The discrimination index is small. d. It is not valid

D

80. What is gained by careful test writing, planning, and administrating? a. time b. increased reliability c. increased validity d. two of these

D

86. What is the most important aspect of the respondents to your questionnaire? a. the number of respondents b. the type of respondents c. when the responses get returned d. the representativeness of the population

D

18. Generally, how do difficulty and discrimination values compare for mastery and discrimination tests? a. Mastery tests have higher difficulty indexes and lower discrimination indexes than discrimination tests. b. Mastery tests have higher discrimination indexes and lower difficulty indexes than discrimination tests. c. Difficulty and discrimination indexes are similar with mastery and discrimination items. d. Mastery tests usually have difficulty values approaching zero. e. Comparisons about the difficulty and discrimination indexes of mastery and discrimination tests cannot be made.

A

10. One hundred students are in each of the upper and lower groups. Of those, 50 students in the upper group and 80 students in the lower group got a question correct. What is the index of discrimination of this question? a. .25 b -.30 c. .65 d. -.65

B

102. Into what types can test items be categorized? a. objective and semiobjective b. objective, semiobjective, and essay c. semiobjective and essay d. objective and essay

B

1. Which type of reliability estimate would least likely be used with a written test? a. internal consistency b. split-halves reliability c. KR20 reliability d. interclass reliability

D

73. What is a major threat to the validity of subjective test items? a. bias b. the time required to complete the test c. the students' handwriting abilities d. the time required to prepare the test

A

12. What major advantage do multiple-choice questions have over true-false items? a. You can answer more of them in one hour. b. Guessing is less of a factor. c. Knowledge of trivia is tested with true-false questions. d. Objectivity is increased with multiple-choice items. e. The intraclass reliability of multiple-choice items generally exceeds zero.

B

33. What is the major difference between objective and essay items? a. Objective items are more reliable. b. Objective items are more susceptible to chance than essay items. c. Essay items are more valid than objective items. d. Essay items encourage study for understanding more than objective items do.

A

109. Matching questions are impossible to construct for measuring complex mental processes. a. true b. false

B

100. What format is the most efficient method of presenting an achievement test? a. oral presentation b. video projection c. separate test copies d. take-home test

C

101. What might be reduced by including a few relatively simple questions at the beginning of an exam? a. test score variance b. test score mean c. students' anxiety d. students' performance

C

22. If you could obtain only one piece of information about a test, what would you like to see? a. Pearson product-moment correlation coefficient b. reliability coefficient c. objectivity coefficient d. criterion coefficient e. validity coefficient

E

110. Instructions for matching questions should include whether responses in the second column can be used more than once. a. true b. false

A

115. With multiple-choice items, what is true about the keyed response? a. It is the best answer. b. It is incorrect. c. It is called a distracter. d. It is called a foil.

A

118. What is true about extrinsic ambiguity? a. It is desirable. b. It is undesirable. c. It is tolerated under certain circumstances. d. It is unavoidable.

A

120. Before giving a test, the tester should prepare the examinees for the test and proofread the test. a. true b. false

A

123. When preparing to take a test, what activity would be best to follow? a. relax and get a good night's rest b. cram the day before test day c. eat just before the test d. avoid stimulants and drink lots of fluids

A

125. When taking a test, sit in a comfortable place and look through the whole test before starting. a. true b. false

A

127. What term indicates a possible way to check the validity of questionnaire responses? a. cross-checking b. internal consistency c. item correlation d. split-halves

A

130. How does an increase in the amount of error affect the reliability of a test? a. The reliability decreases. b. The reliability increases. c. The reliability can either increase or decrease. d. The reliability is not affected.

A

137. When calculating an index of difficulty, the number of examinees answering the question correctly is put in the numerator of the ratio. a. true b. false

A

140. The Net D index of discrimination is calculated by dividing the difference in correct responses between the upper and lower groups by the total number of examinees in either the lower or the upper group. a. true b. false

A

145. What is the advantage of open-ended items in questionnaires? a. They allow for creative answers. b. They allow for standard answers. c. They require less time to answer. d. They are easy to code

A

15. What is the most serious problem if a test contains too many items that discriminate poorly? a. unreliability b. invalidity c. too many high scores d. too many low scores

A

4. What type of validity is enhanced by having a fellow teacher review the items on the test? a. content validity b. concurrent validity c. construct validity d. predictive validity

A

40. Which of the following considerations in constructing a written test should be fairly well established before the course begins? a. the emphasis to be placed on various topics b. the format for presenting the test items c. determination of the correct scoring method d. determination of the pass-fail level

A

108. What is the most serious limitation of true-false test items? a. finishing the exam too early b. guessing correctly c. misinterpretation d. taking too long to administer

B

112. Matching questions are most effective if there are the same number of answers as there are questions. a. true b. false

B

114. What is the least important advantage of multiple-choice test items? a. They can be scored quickly, efficiently, and reliably. b. They can be used to measure simple cognitive processes. c. They can measure all types of educational objectives. d. They can be analyzed to determine their contribution to the test's reliability and validity.

B

121. During a test, the tester should answer all examinees' questions publicly for fairness. a. true b. false

B

126. What activity would be best to avoid confusion when taking a matching test? a. do not attempt practice questions b. find out if answers can be used more than once c. stay up late the night before the exam memorizing definitions d. study as if an essay test were being given

B

129. If a test is perfectly reliable, what is true about each observed score? a. they would be valid b. they would contain very little error c. they would be objective d. they would be relevant

B

13. Which of the following is most important for an achievement test? a. the difficulty index b. the discrimination index c. how many items are on the test d. It depends on the purpose of the test.

B

131. Error in written tests can occur only because of mental and physical conditions of an examinee. a. true b. false

B

133. The KR21 can be used instead of KR20 when it's reasonable to assume that all items on a test have what characteristic? a They vary in difficulty. b. They are equally difficult. c. They vary in discrimination. d. They are equal in discrimination

B

136. The process of conducting an item analysis involves separating the test scores into how many subgroups? a. two b. three c. four d. N/2

B

14. Ideally, how difficult should the items be on written achievement tests? a. Most items should be rather difficult. b. Most items should be of middle difficulty. c. The items should run a continuous range from quite easy to quite difficult. d. About half should be rather easy; the other half should be rather difficult.

B

143. What value of the index of discrimination for an item is considered very good? a. 10 b. 40 c. 50 d. 70 e. 80

B

144. What is the most serious disadvantage of a mailed questionnaire? a. low response rates b. inability to clarify items to participants c. high cost d. unrepresentative sampling

B

2. Why is the test-retest method for obtaining a measure of reliability seldom appropriate for written tests? a. The resulting reliability measure will be lower than that obtained using the split-halves method. b. The first test administration will affect the results of the second test administration. c. The extra time involved is not usually worth the effort. d. Using the same students will result in a very homogeneous group and thus lower the reliability.

B

20. What characteristic is displayed in an item that is answered correctly by a larger number of good students than poor students? a. positive difficulty b. positive discrimination c. negative discrimination d. negative difficulty

B

29. Regarding completion of an item analysis, which of the following is always true? a. A test item with the highest possible difficulty index discriminates well. b. A test item with the highest possible discrimination index has a difficulty index of .50. c. A test item with a difficulty index of zero (0) is incorrectly keyed. d. A test item with an index of difficulty and an index of discrimination of 1.00 is most valid.

B

32. What increases when additional items are added to a written test? a. objectivity b. reliability c. validity d. discrimination

B

36. What type of objective test item is most seriously affected by blind guessing? a. completion b. true-false c. matching d. multiple choice e. short answer

B

55. Which of the following is not considered to be a semiobjective type of question? a. mathematical problem b. matching c. short answer d. completion

B

64. What technique is suggested to determine how well new questionnaire items function? a. item analysis b. pilot studies c. equivalent forms d. rereading them a week after writing them

B

67. What best describes a dichotomous categorical survey question? a. a written answer b. a yes-no or true-false answer c. a range of predetermined answers on a scale d. matching items

B

71. If you interviewed some questionnaire respondents and asked them essentially the same questions that appeared on the questionnaire, what would you be measuring? a. validity b. reliability c. relevance d. fairness

B

81. What advantage does KR21 have over KR20? a. It is more accurate. b. It is easier to calculate. c. It increases validity. d. It increases objectivity.

B

91. What type of test is a mastery test? a. norm-referenced test b. criterion-referenced test c. standardized test d. achievement test

B

98. How does increasing the number of items on a test affect its reliability? a. decreases it b. increases it c. has no affect on it d. there is no way to determine this

B

111. What word best describes good matching questions? a. diverse b. heterogeneous c. homogenous d. random

C

117. What changes if items are added to a written test? a. Objectivity increases. b. Validity increases. c. Reliability increases. d. Difficulty indexes increase.

C

119. What phrase best describes the situation when an answer to a multiple-choice item is unclear to a well-prepared test taker? a. adequately ambiguous b. extrinsically ambiguous c. intrinsically ambiguous d. too easy

C

134. When constructing a test, what is most helpful in ensuring its content validity? a. a correlation model b. a rubric c. a table of specifications d. descriptive statistics

C

139. How well an item separates those with different abilities can be assessed most appropriately with what index? a. index of determination b. index of difficulty c. index of discrimination d. both a and b

C

141. What best describes an item on a test which the students who scored well on the exam answered correctly and students who scored poorly on the exam answered incorrectly? a. It is a difficult item. b. It is an easy item. c. It is a good discriminating item. d. It has a good difficulty rating.

C

142. Positive discrimination occurs when items are being answered correctly by the _____________ group and items are being answered incorrectly by the _____________ group. a. lower; middle b. lower; upper c. upper; lower d. lower-middle; upper-middle

C

146. What would be used to estimate reliability of affective and cognitive domain subscales on a questionnaire? a. descriptive statistics b. index of difficulty c. alpha coefficient d. ANOVA

C

17. If 6 of 8 pupils answered a question correctly, what is the index of difficulty of the question? a. 25% b. 40% c. 75% d. 80%

C

24. Other things being equal, increasing the number of items on a test also increases a. objectivity b. difficulty c. reliability d. validity

C

26. What characteristic is measured by the Kuder-Richardson formulas? a. objectivity b. feasibility c. reliability d. validity

C

3. Which of the following is the major concern when you have an essay examination to grade? a. relevance b. reliability c. objectivity

C

34. What is the best way to make a multiple-choice question less difficult? a. Use novel situations. b. Use brief responses. c. Use heterogeneous responses. d. Use questions rather than incomplete sentences for item stems.

C

37. Which of the following should not be done when grading essay exams? a. Mark one item on all papers before marking any of the papers on the next item. b. Establish the content of an ideal answer. c. Sort the papers into three to five rough piles according to a quick inspection of the overall paper. d. Encourage students not to put their names on their papers.

C

38. A test-wise student would be most likely to do which of the following to enhance his or her score on an objective test? a. write neatly b. reread answers c. spot specific determiners d. watch the time

C

47. If a table of specifications calls for 20% of the 50 questions on a test to measure recall (knowledge) and 30% of them to measure strategy, how many of the test items should deal with recall (knowledge) of strategy? a. 1 b. 2 c. 3 d. 6

C

51. What is presented to the respondent by an open-ended question? a. a yes-no question b. a choice from among several possible answers c. the opportunity to write an answer d. a rank-order question

C

53. Which of Bloom's categories of educational objectives is most like Ebel's category of factual information? a. application b. comprehension c. knowledge d. evaluation

C

85. What advantage do closed-ended questionnaire items have over open-ended questionnaire items? a. ease of writing b. ease of interpreting c. ease of scoring d. detail of information obtained

C

9. One hundred students are in each of the upper and lower groups. Of those, 50 students in the upper group and 80 students in the lower group got a question correct. What is the index of difficulty of this question? a. .25 b. .30 c. .65 d. .80

C

93. What type of test is an achievement test? a. mastery test b. criterion-referenced test c. norm-referenced test d. both a and c

C

94. What is required for the development of a content-valid test? a. textbook on the subject matter b. journal articles on the subject matter c. table of specifications d. both a and b

C

95. A table of specifications includes what two types of objectives? a. formative; summative b. affective; cognitive c. content; educational d. reliability; validity

C

96. What aspect of a table of specifications determines how many questions are included to assess the course objectives? a. grade distribution b. content validity c. relative importance d. the total length of the test

C

105. Reliability of a test can be raised by increasing the a. number of items b. variability on the test c. neither a nor b d. both a and b

D

107. What type of question allows for the most assessments per unit of time? a. essay b. matching c. multiple-choice d. true-false

D

11. What characteristic of a written test would be most enhanced if every student were equally adept at test taking skills? a. difficulty b. objectivity c. reliability d. validity

D

116. With construction of multiple-choice items, what are the responses that are incorrect called? a. keyed b. distracters c. foils d. both b and c

D

122. After administering a test, what is the least important procedure to follow? a. correct the tests quickly b. report the results quickly c. report test scores anonymously d. publically congratulate the highest scorer

D

128. What is the order of importance (from most to least) of the attributes of a written test? a. objectivity, reliability, validity b. reliability, objectivity, validity c. validity, objectivity, reliability, d. validity, reliability, objectivity

D

132. The reliability of a test corrected as right or wrong (0,1) can be obtained with a(n) a. alpha coefficient b. interclass correlation coefficient c. KR20 d. both a and c

D

135. What procedure requires determining the difficulty level of test questions? a. correlation analysis b. descriptive analysis c. inferential analysis d. item analysis

D

16. If the same number of students in the upper 27% and the lower 27% choose the correct answer to a multiple-choice question, what is the index of difficulty for the question? a. 100% b. 50% c. 0% d. cannot determine from the information given

D

23. What major advantage do multiple-choice questions have over essay items? a. You have to think more with multiple-choice items. b. Multiple-choice items are more content valid. c. Multiple-choice items don't ask simple facts. d. Objectivity is greater with multiple-choice items.

D

30. How do you make a multiple-choice item more difficult? a. include tricky options b. increase the difficulty index c. include more options d. make the options more homogeneous

D

35. Which of the following leads to ambiguous true-false questions? a. including specific determiners in a question b. leaving out colorful and imaginative adjectives c. making a question out of an introductory sentence d. taking statements out of context

D

41. Generally, how many questions could be included on a test so that about 90% of a class of high school or college students would be expected to complete the test in 1 hour? a. 30 multiple choice or 30 true-false b. 30 multiple choice or 60 true-false c. 60 multiple choice or 90 true-false d. 60 multiple choice or 120 true-false

D

42. Using which of the following testing formats is most likely to lead to the use of good test construction techniques? a. take-home tests b. a separate test booklet for each student c. open-book tests d. objective tests

D

46. With what type of test is item analysis typically used? a. written tests only b. performance tests only c. the affective, cognitive, and psychomotor domains d. depends on the nature of the test

D

54. To help reduce test anxiety, a question with which index of difficulty might appear as the first item on the test? a .05 b. .25 c. .50 d. .85

D

60. If the difficulty index for a test item is .60 and there are 50 students in the high group and 50 students in the low group, what is the maximum discrimination index this item could achieve? a. can't be calculated b. .60 c. .70 d. .80

D

65. What is the most important ancillary device in helping to obtain the highest response rate for a questionnaire? a. including an inducement such as a dollar b. a follow-up letter c. clear instructions d. a well-written cover letter

D

68. Which of the following is not an advantage of open-ended items on a questionnaire? a. They allow freedom of expression. b. They eliminate the possibility of leaving out a category. c. They assess a complex issue more efficiently. d. They reduce ambiguity.

D

7. For a 50-item test given to 140 students, approximately how many of the answer sheets would be used in an item analysis? a. 28 b. 50 c. 54 d. 76

D

76. What does it mean when a test item reflects extrinsic ambiguity? a. It is a bad item and should not be used. b. Better students had difficulty understanding the item. c. The difficulty level was nearly 0.00 (0%). d. Poorer students had difficulty answering the item.

D

77. What effect do intrinsically ambiguous items have on a test's reliability? a. They have no effect on reliability. b. It depends on who gets the item right. c. It depends on the test's variability. d. They reduce the test's reliability.

D

99. What is the least important consideration when determining the length of a test? a. the amount of time you have to test b. the type of questions used c. the attention span of the students d. the number of students

D

103. Multiple-choice test items are categorized as what type of question? a. objective b. semiobjective c. essay d. probabilistic

A

104. Semiobjective test items are categorized as what types of test question? a. short answer b. matching c. true-false d. essay

A

74. What major advantage do forced-choice test items have over essay test items? a. objectivity b. the time required to take the test c. the students' handwriting abilities d. the time required to prepare the test items

A

106. Objective test items are useful because they a. are easy to prepare b. reduce the subjectivity involved in scoring c. are difficult to grade d. measure the integration of knowledge

B

113. What is the most effective objective test item type? a. matching b. multiple choice c. true-false d. short answer

B

75. What type of ambiguity would you like a test to reflect? a. no ambiguity b. extrinsic ambiguity c. intrinsic ambiguity d. either extrinsic or intrinsic ambiguity

B

78. What should you do when constructing multiple-choice test items? a. have five possible responses b. make the stem in the form of a question c. make the stem as short as possible d. write the answers before writing the stem

B

79. What is the advantage of analytically scoring rather than globally scoring an essay test? a. It saves time when scoring the test. b. It reduces potential bias. c. It increases the time needed to score the test. d. It increases potential bias.

B

124. What activity would most likely lead to the best performance on an essay exam? a. cram for all possible questions b. avoid help from other classmates to avoid confusion c. make up and answer practice questions in advance d. eat lots of food an hour before

C

138. With only what index of difficulty value can maximum discrimination occur? a. .10 b. .25 c. .50 d. .75

C

44. Which is most likely to increase as the variance of the test scores increases on a written test? a. difficulty of the test b. objectivity of the test c. reliability of the test d. relevance of the test

C

27. What would most likely be the average difficulty index on a mastery test? a. 1% b. 50% c. 60% d. 90%

D

39. A test constructed by which of the following is probably most valid and reasonably reliable? a. the department chairman, because he or she proposes the objectives of the course b. an evaluation service that employs skilled people for this purpose c. the textbook authors, because they are experts in the area and know what the book covers d. the classroom teacher, if familiar with test construction techniques

D

31. What changes when additional items are added to a written test? a. Objectivity increases. b. Validity increases. c. Reliability decreases. d. Difficulty indexes increase. e. Reliability increases

E

88. The following information was obtained on a recent item analysis. The correct option to this sample item was C. Option A: difficulty = .14 and discrimination = −.10. Option B: difficulty = .20 and discrimination = −.24. Option C: difficulty = .48 and discrimination = .48. Option D: difficulty = .18 and discrimination = −.01. Estimate the quality of the item. a. difficulty needs revision b. discrimination needs revision c. poor item d. fair item e. good item

E

72. Developing a table of specifications helps ensure what type of validity? a. content b. construct c. concurrent d. predictive

A

19. Which of the following factors will help ensure that a test is objective? a. a well-defined scoring method for the test b. several instructors scoring the test c. administering the test on two different days d. measuring with accurate instruments

A

21. A teacher has a test with an average difficulty of .65. What can be said with certainty about this test? a. It is easier than average (i.e., .50). b. It is a good test. c. Test validity is within desired levels. d. Item discrimination is high.

A

25. Which of the following is an important advantage of the multiple-choice test? a. wide sampling b. ease of validation c. emphasis on recall rather than recognition d. lack of subjectivity

A

49. When is an item considered to have good discrimination power? a. when the people who scored well on the test got the item correct b. when the people who scored poorly on the test got the item correct c. when the people who took the test missed the item d. when the people who took the test got the item correct

A

52. What does a table of specifications for a written test help to determine? a. what to test for b. when to test c. how many questions to include d. what format to use

A

62. With what domain of human experience are questionnaires most closely associated? a. affective b. cognitive c. eminent d. psychomotor

A

66. Which of the following is least likely to ensure that a questionnaire is valid? a. keeping it as short as possible b. having it examined by experts c. running pilot studies d. ensuring confidentiality

A

70. What is saved by setting up a "postage-due" arrangement with the post office for returned questionnaires? a. cost b. time c. paper d. space

A

5. Which of the following statements about testing is false? a. Before constructing test items, one must know the sequence of motor performance thoroughly. b. Once the content validity of a test is established, the test may be used at several skill levels. c. Adding more skill items to a test should increase the reliability of the test. d. One skill test can satisfy both formative and summative evaluation needs.

B

57. What is the least important information to include in the instructions for a matching-test item? a. the basis for matching the items in the two lists b. the point value of the question c. how to record the answers d. whether a response may be used more than once

B

59. What index of discrimination would be most likely for an intrinsically ambiguous question? a. .40 b. .00 c. .50 d. It depends on the item difficulty.

B

61. What assumption reduces the usefulness of the KR21 formula for estimating the reliability of a written test? a. All items have the same number of responses. b. All items are of equal difficulty. c. All items have a correct answer, not just a best answer. d. No items have a negative discrimination index.

B

83. What would you like to know if you could learn only one thing about an item on an achievement test? a. difficulty b. discrimination c. number of possible options d. type of item

B

92. What type of test is a certification examination for a personal fitness instructor? a. norm-referenced test b. mastery test c. standardized test d. achievement test

B

97. According to Bloom's taxonomy, what is the highest-level cognitive objective? a. knowledge b. evaluation c. application d. analysis

B

43. A teacher is scheduled to give an examination to four classes on December 1. The teacher begins coaching duties on December 1. If the four classes have been covering the same material in class and average 25 pupils each, what type of test should the teacher administer? a. essay; it is more efficient with small classes b. multiple choice; it is a better comparison of students because they are all covering the same material c. multiple choice; there is more time to construct the test than to grade it d. multiple choice; it is easier to grade for this class size

C

48. If 20 of 40 in the upper group and 20 of 40 in the lower group choose the correct answer to a multiple-choice question with five possible responses, what would the index of discrimination be for the item? a. .50 b. .20 c. .0 d. -.20

C

50. What type of survey would yield very consistent but inaccurate results? a. a survey with low bias and high precision b. a survey with low bias and low precision c. a survey with high bias and high precision d. a survey with high bias and low precision

C

56. Which of the following words would least likely appear in a true-false test item that was keyed as false? a. entirely b. inevitable c. generally d. none

C

58. Use of novel situations in the construction of written test questions is associated most closely with which of Bloom's educational objectives? a. knowledge b. comprehension c. application d. prediction

C

6. Why would you typically use KR21 and not KR20 when determining the reliability of your written test? a. KR20 is not appropriate for use with written tests. b. KR21 is a better estimate of the test reliability. c. KR21 is easier to calculate than KR20. d. KR20 cannot be used when items are scored right or wrong.

C

63. An open-ended item on a questionnaire is most like which type of written test item? a. classification b. completion c. essay d. multiple choice

C

69. When would an open-ended item be more appropriate than a closed-ended item on a questionnaire? a. when seeking factual information b. when seeking sensitive information c. when categorization is difficult d. when the time for analyzing results is limited

C

8. What do item characteristics' difficulty (P) and discrimination (r or D) affect? a. reliability b. validity c. both a and b d. neither a nor b

C

82. A test should consist of items of what level of difficulty? a. quite difficult b. quite easy c. average difficulty d. a range of difficulties

C

84. Open-ended questionnaire items have what potential problem? a. item difficulty b. item discrimination c. bias d. difficulty in interpreting the response

C

28. You have a written test; 150 students have taken the test. Approximately how many answer sheets should you use to conduct an item analysis? a. 30 b. 40 c. 60 d. 80

D

87. What is the most important aspect of a questionnaire? a. reliability b. objectivity c. representativeness d. validity

D

89. A test can be ________and still not ________. a. reliable; stable b. stable; reliable c. valid; reliable d. reliable; valid

D

90. Which of the following is not a requirement for constructing effective written tests? a. knowledge of proper test construction techniques b. knowledge of subject area to be tested c. adequate writing skills d. an advanced degree in education e. awareness of level of students' understanding

D