Chapter 6 & 7
Consider the following illustrative three-option multiple-choice item. An anonymously completed, self-report item regarding a student's values —anitem that has no clearly correct answer—is best suited for use in an: a. cognitive examination b. affective inventory c. psychomotor skills test Which of the following statements best characterizes the illustrative item?
The illustrative item violates a general item-writing guideline by providing a blatant grammatical clue to the correct answer.
Consider the following illustrative binary-choice item. For this next True/False item, indicate whether the item's statement is true or false by circling the T or F following the item. Validation is the joint responsibility of the test developer and the test user, but the accumulation of reliability/precision evidence is the exclusive responsibility of the test user. (Circle one: T or F) Which of the following statements best describes the illustrative item?
The illustrative True/False item violates one of the item-category guidelines by including two substantial concepts in a single item.
Consider the following illustrative binary-choice item. Please decide whether the following statement regarding the reliability of educational tests is True or False. Please place a check behind the True or False to indicate your answer. True ___ False___ When determining a test's classification consistency, there is no need to consider the cut score employed nor that cut score's location in the score distribution. Which of the following statements best describes the illustrative item?
This illustrative item violates none of the chapter's general item-writing guidelines or the specific guidelines for writing binary-choice items.
These illustrative short-answer items were created for use in a twelfth-grade English course and are intended to be used in the course's midterm exam. Please complete the short-answer items below by filling in the blank you will find in each item. • __________ is the case to be employed with all modifiers of gerund—definitely including pronouns. • A __________ infinitive that, in former times, was regarded as a grammatical error is now acceptably encountered in all kinds of writing. Which of the following assertions best reflects how these two short-answer items conform to the chapter's item-writing guidelines for such items.
Although several of the chapter's item-writing guidelines have been properly followed, there is the same, rather obvious, violation of an item-writing guideline in both items
This illustrative item is intended for use in a middle-school American history course. Directions: Remembering the class discussions of America's current immigration issues, please provide a brief essay on each of the issues cited below. You will have a full 50-minute class period to complete this examination, and you should divide your essay-writing efforts equally between the two topics. In grading your twin essays, equal weight will be given to each essay. Remember, compose two clear essays—onefor each issue. Your Two Essay Topics 1. Why would some form of "amnesty" for illegal aliens be a helpful solution to at least part of today's U.S. immigration problems? 2. Why would some form of "amnesty" for illegal aliens be a disastrous solution to today's U.S. immigration problems? Which of the following statements most accurately describes the match between the illustrative item and the Chapter 7 guidelines for creating essay items?
At least one of the chapter's guidelines has been explicitly followed in the illustrative item.
This illustrative essay item was written for sixth graders. Thinking back over the mathematics lessons and homework assignments that you received during the past 12 weeks, what mathematical conclusions can you draw? Describe those conclusions in no more than 300 words, written by hand on the test-booklets provided or as a printed copy of your conclusions composed on one of our classroom computers. Select the statement that most accurately appraises this essay item for sixth-grade students.
Despite its adherence to one of the chapter's item-writing guidelines for essay items, the shoddy depiction of a student's task renders the item dysfunctional.
This illustrative short-answer item was written for a third-grade class. The purpose is to help both the teacher and the students determine how well those students had achieved mastery of a recent state-approved language arts curriculum goal. Please write your answer legibly. _____________ is a good one-word description for commas, periods, question marks, and colons. Which one of the following statements most accurately describes the illustrative item?
For young students such as these third graders, direct questions should be used instead of incomplete statements—so the illustrative item violates an item-writing guideline for short-answer items.
This excerpt from a teacher's memo includes faculty-created rules for scoring their students' responses to essay items. The following rules for scoring students' responses to essay items were created last year by our faculty and were approved by a near unanimous vote of the faculty. Please review what those rules recommend prior to our taking this year's "confirmatory" faculty vote on these rules. RULES FOR SCORING RESPONSES TO ESSAY ITEMS When teachers in this school score their students' responses to essay items, those teachers should always (1) make a preliminary judgment about how much importance should be assigned to the conventions of writing, such as spelling, (2) decide whether to score holistically or analytically, (3) prepare a tentative scoring key prior to actually scoring students' responses, (4) try to score students' responses anonymously without knowing which student supplied which response, and (5) score a given student's responses to all essay items on a test and then move on to the next student's responses. Please select the most accurate assertion regarding these rules.
Only one of the faculty-approved rules is basically opposed to the Chapter 7 guidelines for scoring students' responses to essay items.
This illustrative item is destined for use in a high-school speech course that, in recent weeks, has been focused on debate preparation. Directions: To conclude our unit on how to prepare successfully for a debate, please consider carefully the following preparation-focused topics. After doing so, choose one that you regard as most important—to you—and then write a 300-400word essay describing how best to prepare for whatever topic you chose. Be sure to identify which of the potential topics you have selected. You will have 40 minutes to prepare your essay. Potential Essay Topics Introducing your position and defending it Use of evidence during the body of the debate Preparing for your opponents' rebuttal Please choose the statement that most accurately reflects the illustrative item's congruence with Chapter 7's guidelines for writing essay items.
The illustrative item is structured in direct opposition to one of the chapter's guidelines for writing essay items.
This illustrative short-answer item was constructed for tenth-grade students. Following World War Two, an international organization intended to maintain world peace was established, namely, the United Nations. Similarly, after World War One a peace-oriented international organization was established. What was the name of that earlier organization? _____________________ Which of the following statements best mirrors the degree to which the illustrative item is in accord with Chapter 7's guidelines for writing short-answer items?
The illustrative item violates none of the chapter's guidelines for writing short-answer items.
Consider the following illustrative four-alternative multiple-choice item and, then, indicate the degree to which the item adheres to the chapter's item-construction guidelines. Which one of the following kinds of validity evidence represents a different category of evidence than the other three kinds of validity evidence identified? a. Convergent evidence, that is, positive relationships between test scores and other measures intended to measure the same or similar constructs b. Discriminant evidence, that is, positive relationships between test scores and other measures purportedly assessing different constructs c. Alignment evidence d. Test-criterion relationship evidence representing the degree to which a test score predicts a relevant variable that is operationally distinct from the test Which of the following statements best describes the illustrative item?
The illustrative item violates one of the chapter's general item-writing guidelines by presenting a blatant cue regarding which answer is correct.
Here's an illustrative short-response item intended for use with ninth-graders in a high-school government course: Please accurately fill in the blanks you find in the statement given below regarding "How a bill becomes a law." In _______, _______ and _______ explored what ultimately became the _______ section of the northwestern United States with the assistance of a native-American guide known as _______. (Prod. These blank lines MUST be equal in length.) Select the most accurate of the following statements regarding this illustrative short-answer item.
The item satisfies the guideline regarding linear equality, yet violates the number-of-blanks guideline.
For following item, select the option that best illustrates the degree to which the item adheres to the chapter's general item-writing guidelines or the guidelines for specific categories of items. Note that following item deal with assessment-related content and thus might be regarded as a rudimentary form of "assessment enrichment." Consider whether the following binary-choice item adheres to the item-writing guidelines presented in the text. Presented below is a binary-choice item. Please indicate—by circling the R or W—whether the statement given in the item is right (R) or wrong (W). R or W Absence-of-bias determinations are typically made as a function of judgmental scrutiny and, when possible, empirical analysis. Which of the following statements best describes the illustrative item?
The item violates none of the chapter's guidelines, either the five general guidelines or the specific guidelines for binary-choice items.
Consider the following illustrative binary-choice item. Please consider the following binary-choice item and then indicate whether it is Accurate (A) or Inaccurate (I). A or I ___ If a teacher wishes to create assessments that truly tap students' mastery of higher order cognitive challenges, the teacher will not be working within the affective domain. Which of the following statements best describes the illustrative item?
The item violates the item-category guideline discouraging the use of negatives in such items.
Here is an illustrative response-scoring plan devised by a high-school Latin teacher. Please review how the teacher plans to evaluate students' Latin compositions, then select the option that most accurately describes the teacher's scoring intentions. A Latin teacher in an urban high school (that has a long and oft-honored history of preparing students for college) frequently expresses during faculty meetings her complete disdain for what she calls "multiple-guess exams." As part of her annual teacher-evaluation evidence, she has been asked by her school's principal to present a written description of how she plans to evaluate students' responses to her constructed-response items. Please consider the following description supplied by the teacher, then select from four alternatives the most accurate comment regarding this teacher's scoring plans. "I plan to score my students' essay responses holistically, not analytically, because I invariably ask students to generate brief essays in which they must incorporate at least half of the new vocabulary terms encountered during the previous week. I supply students with a set of explicit evaluative criteria that I will incorporate in arriving at a single, overall judgment of an essay's quality. Actually, I always pre-weight each of these evaluative criteria and post those weights for students in advance of their tackling this task. Because this is a course emphasizing the writing of Latin (rather than oral Latin), I make it clear to my students—well in advance—that grammar and the other mechanics of writing are very important. When I score students' essays, if there is more than one essay per test, I score all of Essay One before moving on to Essay Two. Because I want these students to become, in a sense, Latin "journalists," I require that they clearly identify themselves with a byline at the outset of each essay. This scoring system, based on nearly 20 years of my teaching Latin to hundreds of our school's students, really works!"
The teacher's approach violates one of the chapter's essay-scoring guidelines.
This illustrative essay item was written for eleventh-grade students taking an English course. In the space provided in your test booklet, please compose a brief editorial (of 250 words or less) in favor of the school district's after-school tutorial program. The intended audience for your position statement consists of those people who routinely read this town's weekly newspaper. Because you will have the entire class period to complete this task, you may wish to write a draft editorial using the scratch paper provided so that you can then revise the draft before copying your final version into the test booklet. Your grade on this task will contribute 40 percent toward the grade for the Six-Week Persuasive Writing Unit. Which of the following statement best characterizes this item?
This illustrative item contains no serious violation of any of the chapter's guidelines for writing essay items.
Consider the following multiple binary-choice item with its four separate sub-items and then decide how well the item adhered to the chapter's item-writing guidelines. Directions: For each statement in the following cluster of four statements, please indicate whether the statement is true (T) or false (F) by circling the appropriate letter. In an elaborate effort to ascertain the reliability of a new high-stakes test developed in their district, central-office administrators have calculated the following types of evidence based on a tryout of the test with nearly 2,300 students: • Internal consistency r = .83 • Test-retest r = .78 • Standard error of measurement = 4.3 T or F (1) The three types of reliability evidence calculated by the central-office staff are essentially interchangeable. T or F (2) The trivial difference between the test-retest coefficient and the internal consistency coefficient constitutes no cause for alarm. T or F (3) The test-retest r should never be smaller than a test's internal consistency estimate of reliability. T or F (4) The standard error measurement (4.3 in this instance) is derived more from validity evidence than from reliability evidence. Choose the most accurate of the following statements regarding the illustrative multiple binary-choice item as a whole.
This illustrative item seems to violate none of the chapter's guidelines for constructing such items, that is, the general guidelines, the guidelines for multiple binary-choice guidelines, and the guidelines for binary-choice items
Consider the following illustrative matching item. Choose the best match between the item categories in List X and the strengths/weaknesses in List Y. List X: (1). matching (2) binary choice (3) multiple binary choices List Y: a. Can cover much content b.Can test high-order cognition c. May elicit only low-level knowledge d. Cannot assess creative responses Which of the following statements best describes the quality of the illustrative item?
This illustrative matching item contains several departures from Chapter Six's item-writing guidelines for matching items.
Consider the following illustrative binary-choice item. It deals with a reliability/precision concept treated in the Standards for Educational and Psychological Testing (2014). Directions: Please indicate whether the statement below regarding the reliability/precision of educational tests is Accurate (Circle the A) or Inaccurate (Circle the I). A or I Because the standard error of measurement can be employed to generate confidence intervals around reported scores, it is typically more informative than a reliability coefficient. Which of the following statements best describes the illustrative item?
This illustrative binary-choice item violates none of the general or item-category guidelines for this type of selected-response item.
Consider the following illustrative five-option multiple-choice item. It addresses content presented in the Standards for Educational and Psychological Testing (2014) related to the fundamental notion of assessment validity. When we encounter a test whose scores are affected by processes that are quite extraneous to the test's intended purpose, we assert that the test displays which one of the following? a. Construct underrepresentation b. Construct deficiency c. Construct corruption d. Construct-irrelevant variance e. All of the above Which of the following statements best describes the illustrative item?
This illustrative item, because it includes an "all of the above" alternative, violates an important ite-writing guideline.
