Assessment Midterm Chapters 1-7, 12
Which of the following represents the most appropriate strategy by which to support the validity of score-based interpretations for specific uses?
Generation of an evidence-laden validity argument in support of a particular usage-specified score interpretation
This illustrative short-answer item was written for a third-grade class. The purpose is to help both the teacher and the students determine how well those students had achieved mastery of a recent state-approved language arts curriculum goal. Please write your answer legibly. _____________ is a good one-word description for commas, periods, question marks, and colons. Which statement most accurately describes the illustrative item?
A. For young students such as these third graders, direct questions should be used instead of incomplete statements so the illustrative item violates an item-writing guideline for short-answer items.
Which of the following is not a step in the four steps for creating a learning progression?
A. Form a basic and introductory understanding of a target curricular aim.
Which of the following is the most useful indicator of the consistency of an individual student's test performance?
A standard error of measurement
Please select the one answer that most accurately identifies the particular item's "quoted" reason for teachers to know about assessment. "Teachers need to give classroom assessments in order to assign grades to students indicating how well each student has attained the learning outcomes set for them." This is:
A. A traditional reason for teachers to know about assessment
Considering the knowledge you've gained regarding formative assessment in this chapter, which of the following characteristics of a class discussion could yield formative assessment information?
A. Class discussions that show what students are thinking.
Which of the following best captures the testing comparison between NCLB and ESSA?
A. ESSA permits greater state-determinations of testing than NCLB.
Classroom assessment in public schools is often a function of federal legislation. Which of the following pieces of legislation is historically considered to have had the greatest impact on public school testing policies?
A. Elementary and Secondary Education Act (ESEA) of 1965
Which of the following questions is not an element in a research-supported conception of formative assessment?
A. Formative assessment should be used only by teachers to adjust their ongoing instructional activities.
Decisions linked to classroom assessments should be made _____________.
A. In advance
Which term best describes the type of measurement that would yield the following feedback: Jonathan scored within the 92nd percentile on the SAT?
A. Norm-referenced measurement
Which of the following non-profit organizations played a significant role in the rapid adoption of the Common Core State Standards?
A. The Bill and Melinda Gates Foundation
Which of the five general item-writing commandments is most likely violated in the following question: When you scrupulously measure students' performance, you are assessing them. a. True b. False
A. Thou shall not use vocabulary that is more advanced than required.
The Elementary and Secondary Education Act contains various subsections referred to as "titles." Which of the following "titles" gets the most attention (and funding) from policymakers?
A. Title I
Please assume you are a middle-school English teacher who, despite this chapter's urging that you rarely, if ever, collect reliability evidence for your own tests, stubbornly decides to do so for all of your midterm and final exams. Although you wish to determine the reliability of your tests for the group of students in each of your classes, you only wish to administer the tests destined for such reliability analyses on one occasion, not two or more. Given this constraint, which of the following coefficients would be most suitable for your reliability-determination purposes?
An internal-consistency reliability coefficient
How does "classification consistency" differ conceptually from more traditional indicators of test reliability?
Classification-consistency approaches are focused more on capturing the degree of students' consistent categorizations rather than supplying only numerical indices of students' score consistency.
Which of the following terms best describes the means teachers employ in their attempt to promote students' achievement of the curricular ends being sought?
D. Instruction
Which of the five general item-writing commandments is most likely violated in the following question: Any time a teacher measures the performance of students, the teacher tests the students. a. true b. false
D. None of the five general item-writing commandments have been violated.
Which of the following is a consequence of a collaborative effort by the National Governor's Association (NGA) and the Council of Chief State School Officers (CCSSO)?
D. The Common Core State Standards (CCSS)
Which of the following is generally conceded to be a key component of formative assessment?
D. The framework provided by a learning progression's building blocks
Which of the following conceptions of assessment validity is most constant with the Standards for Educational and Psychological Testing released in 2014 by AERA, APA, and NCME?
The accuracy of score-based interpretations for intended test uses
Which of the following is not one of the three types of reliability evidence?
Validity
Which of the following descriptions of validity is most accurate?
Validity refers to the accuracy of score-based interpretations for specific purposes.
If educators wish to accurately estimate the likelihood of consistent decisions about students who score at or near a high-stakes test's previously determined cut-score, which of the following indicators would be most useful for this purpose?
A conditional standard error of measurement (near the cut-score)
Which one of the following statements could be technically correct?
"The test-based inference is valid."
Reliability coefficients range from ________________.
-1.00 to +1.00
These illustrative short-answer items were created for use in a 12th-grade English course and are intended to be used in the course's midterm exam. Please complete the short-answer items below by filling in the blank you will find in each item. o __________ is the case to be employed with all modifiers of gerundsdefinitely including pronouns. o A __________ infinitive that, in former times, was regarded as a grammatical error is now acceptably encountered in all kinds of writing. Which of the following assertions best reflects how these two short-answer items conform to the chapter's item-writing guidelines for such items.
A. there is the same violation of an item-writing guideline in both items.
Which of the following conclusions regarding multiple binary-choice items has not been supported by available research?
C. These items are a bit less difficult for students than multiple-choice items.
Which statement best characterized this nation's current use of formative assessment?
B. Although research-supported, formative assessment is not widely used.
One of the following rules for the construction of essay items is accurate. The other three rules are not. Which is the correct rule?
B. Construct all essay items so the student's task for each item is unambiguously described.
Which term best describes the type of measurement that would yield the following feedback: Jonathan mastered 92 percent of the tested content?
B. Criterion-referenced measurement
Which of the following is not a general item-writing rule for classroom assessments?
B. Do not inform students about how much the items on a test will be weighted.
Which of the following answer choices depicts the chronologically accurate development of assessment legislation in the United States?
B. ESEA, NCLB, ESSA
Which of the following rules is often recommended for the generation of matching items?
B. Employ relatively brief lists, placing the shorter words or phrases at the right.
One of the important rules to be followed in creating multiple binary-choice items is that:
B. Item clusters should be strikingly separated from one another.
Please select the one answer that most accurately identifies the particular item's "quoted" reason for teachers to know about assessment. "Just as physicians need to know about patients' blood pressure and what it indicates, teachers need to know about educational testing. It is simply part of what a solid educational professional needs to understand." This is:
B. Neither a traditional reason nor one of today's reasons for teachers to know about assessment
This excerpt from a teacher's memo includes faculty-created rules for scoring their students' responses to essay items. The following rules for scoring students' responses to essay items were created last year by our faculty and were approved by a near unanimous vote of the faculty. Please review what those rules recommend prior to our taking this year's "confirmatory" faculty vote on these rules. RULES FOR SCORING RESPONSES TO ESSAY ITEMS When teachers in this school score their students' responses to essay items, those teachers should always (1) make a preliminary judgment about how much importance should be assigned to the conventions of writing, such as spelling, (2) decide whether to score holistically or analytically, (3) prepare a tentative scoring key prior to actually scoring students' responses, (4) try to score students' responses anonymously without knowing which student supplied which response, and (5) score a given student's responses to all essay items on a test and then move on to the next student's responses. Please select the most accurate assertion regarding these rules.
B. Only one of the faculty-approved rules is basically opposed to the Chapter 7 guidelines for scoring students' responses to essay items.
Select the one accurate guideline below for teachers who are scoring students' responses to essay items.
B. Prepare at least a tentative scoring key in advance of judging students' responses to any item.
Consider the following illustrative binary-choice item. For this next True/False item, indicate whether the item's statement is true or false by circling the T or F following the item. Matching items should employ homogenous lists, but should seek to achieve relative brevity. (Circle one: T or F) Which statement best describes the illustrative item?
B. The illustrative True/False item violates one of the item-category guidelines by including two substantial concepts in a single item.
Here's an illustrative short-response item intended for use with ninth-graders in a high-school government course: Please accurately fill in the blanks you find in the statement given below regarding "Western Exploration and Expansion." In _______, _______ and _______ explored what ultimately became the _______ section of the northwestern United States with the assistance of a native-American guide known as _______. Select the most accurate of the following statements regarding this illustrative short-answer item.
B. The item satisfies the guideline regarding linear equality, yet violates the number-of-blanks guideline.
Consider whether the following binary-choice item adheres to the item-writing guidelines presented in the text. Presented below is a binary-choice item. Please indicate - by circling the R or W - whether the statement given in the item is right (R) or wrong (W). R or W: Absence-of-bias determinations are typically made as a function of judgmental scrutiny and, when possible, empirical analysis. Which statement best describes the illustrative item?
B. The item violates none of the chapter's guidelines, either the five general guidelines or the specific guidelines for binary-choice items.
Consider the illustrative binary-choice item.Please decide whether the following statement regarding the reliability of educational tests is True or False. Please place a check behind the True or False to indicate your answer. True ___ False___ When determining a test's classification consistency, there is no need to consider the cut score employed nor that cut score's location in the score distribution. Which statement best describes the illustrative item?
B. This illustrative item violates the item-specific guideline regarding the use of negative statements in a binary-choice item.
The National Assessment of Educational Progress (NAEP):
B. assesses national samples of U.S. students at three grade levels every few years in certain academic subjects.
When educators collect test-based evidence to inform decisions about already completed instructional activities they are engaging in a basic form of _____________________.
B. summative assessment
Please select the one answer that most accurately identifies the particular item's "quoted" reason for teachers to know about assessment. "Wishing that students will make progress does not guarantee that students actually will do so. And this is why I believe teachers have a fundamental responsibility to monitor their students' progress throughout the school year. I try to administer informal progress-monitoring quizzes every few weeks to make sure my instruction is "taking." If my instruction is not working as well as I want it to work, then I can make modifications in my upcoming teaching plans. Assessment-based monitoring of students' progress is so very sensible that it's hard for me to understand why it is not more widely used. This is:
C. A traditional reason for teachers to know about assessment
Which of the following terms refers to the degree to which there is a meaningful agreement between two or more of the following: curriculum, instruction, and assessment?
C. Alignment
Consider the following test item. Your primary concern in selecting techniques to assess a learning objective or objectives should be classroom practicality and efficiency. a. True b. False
C. Binary-choice
Which category of test items best describes the following item: According to educators, one of the major advantages of the Every Student Succeeds Act is that it forces schools and teachers to focus only on the important material included in the state test. a. True b. False
C. Binary-choice
These standards, released by the Council of Chief State School Officers and the National Governors Association for Best Practices were an attempt to establish continuity and consistency across varying state curricular aims.
C. Common Core State Standards
One of the most commonly misused terms in educational jargon is the word "standards." In reality, there is no singular, all-encompassing concept of a standard, but rather more specific subtypes of educational standards. Which of the following subtypes of standards could best be described as "the knowledge or skills that educators want students to learn"?
C. Content standard
Which of the following pieces of federal legislation attempts to install greater degrees of flexibility so that states and districts can particularize their programs for implementing chief provisions of the current successor to ESEA?
C. ESSA
Which of the following is a generally recommended item-writing rule for matching items?
C. In the test's directions, describe the basis for matching and the number of times a response can be used.
Consider the following test-item. Which of the following decisions requires educators to use quality assessment information? a. Choosing who should get into this college b. Deciding what reading group a student should be placed in c. Determining whether a student is legally disabled d. All of the above e. Only (a) and (b) Which category best describes this item?
C. Multiple choice
Which of the following pieces of federal legislation had notably been an attempt at reversing the growing achievement gap that left poor and minority students in failing schools while requiring a "show us" approach to student evaluation?
C. No Child Left Behind
Please select the one answer that most accurately identifies the particular item's "quoted" reason for teachers to know about assessment. "Just a year ago, the voters in our school district voted favorably in a huge school-levy election that brought in substantial tax dollars for our schools. Most of the district's teachers are convinced that this positive support for the schools was based on our schools' consistently high rankings on the state's annual accountability tests." This is:
C. One of today's reasons for teachers to know about assessment
Which of the rules given below accurately reflects one of the guidelines generally given to classroom teachers who must score students' responses to essay items?
C. Prepare a tentative scoring key prior to judging any student's response-being ready to modify it if this seems warranted during the scoring.
Which of the following is not an item-writing rule for the creation of binary-choice items?
C. Rarely use statements containing double negatives, although single or triple negatives are acceptable.
Which of the following should be most influential in guiding those who create educational tests?
C. The AERA-APA-NCME Standards for Educational and Psychological Testing
This illustrative short-answer item was constructed for 10th-grade students. Following World War II, an international organization intended to maintain world peace was established, namely, the United Nations. Similarly, after World War I a peace-oriented international organization was established. What was the name of that earlier organization? Which of the following statements best mirrors the degree to which the illustrative item is in accord with Chapter 7's guidelines for writing short-answer items?
C. The illustrative item violates none of the chapter's guidelines for writing short-answer items.
Here is an illustrative response-scoring plan devised by a high-school Latin teacher. Please review how the teacher plans to evaluate students' Latin compositions, then select the option that most accurately describes the teacher's scoring intentions. A Latin teacher in an urban high school (that has a long and oft-honored history of preparing students for college) frequently expresses during faculty meetings her complete disdain for what she calls "multiple-guess exams." As part of her annual teacher-evaluation evidence, she has been asked by her school's principal to present a written description of how she plans to evaluate students' responses to her constructed-response items. Please consider the following description supplied by the teacher, then select from four alternatives the most accurate comment regarding this teacher's scoring plans. "I plan to score my students' essay responses holistically, not analytically, because I invariably ask students to generate brief essays in which they must incorporate at least half of the new vocabulary terms encountered during the previous week. I supply students with a set of explicit evaluative criteria that I will incorporate in arriving at a single, overall judgment of an essay's quality. Actually, I always pre-weight each of these evaluative criteria and post those weights for students in advance of their tackling this task. Because this is a course emphasizing the writing of Latin (rather than oral Latin), I make it clear to my studentswell in advancethat grammar and the other mechanics of writing are very important. When I score students' essays, if there is more than one essay per test, I score all of Essay One before moving on to Essay Two. Because I want these students to become, in a sense, Latin "journalists," I require that they clearly identify themselves with a byline at the outset of each essay. This scoring system, based on nearly 20 years of my teaching Latin to hundreds of our school's students, really works." Select the statement that most accurately depicts this teacher's scoring plans.
C. The teacher's approach violates one of the chapter's essay-scoring guidelines.
This illustrative essay item was written for 11th-grade students taking an English course. In the space provided in your test booklet, please compose a brief editorial (of 250 words or less) in favor of the school district's after-school tutorial program. The intended audience for your position statement consists of those people who routinely read this town's weekly newspaper. Because you will have the entire class period to complete this task, you may wish to write a draft editorial using the scratch paper provided so that you can then revise the draft before copying your final version into the test booklet. Your grade on this task will contribute 40 percent toward the grade for the six-week persuasive writing unit. Which statement best characterizes this item?
C. This illustrative item contains no serious violation of any of the chapter's guidelines for writing essay items.
Consider the following illustrative five-option multiple-choice item. It addresses content presented in the Standards for Educational and Psychological Testing (2014) related to the fundamental notion of assessment validity. When we encounter a test whose scores are affected by processes that are quite extraneous to the test's intended purpose, we assert that the test displays which one of the following? a. Construct underrepresentation b. Construct deficiency c. Construct corruption d. Construct-irrelevant variance e. All of the above Which statement best describes the illustrative item?
C. This illustrative item, because it includes an "all of the above" alternative, violates an important item-writing guideline.
Consider the following test item. To what purpose, and for what end-goal, is diagnostic assessment done? a. To ascertain the achievement level of the students. b. To ascertain the specific strengths and weaknesses of the students. c. To ascertain the effectiveness of teaching.
C. Thou shall not employ complex syntax in your assessment items.
Which of the five general item-writing commandments is violated in the following test item? Complete the following sentence. In multiple choice-items, the first part of the item, given drawing its name from plant biology, is called the _______. a. stem b. cluster c. binary-choice d. test-item
C. Thou shall not provide students with unintentional clues regarding appropriate responses.
Which of the following is not a traditional reason that teachers assess students?
C. To clarify instructional intentions
Of the following four statements, one is not a guideline to be followed when constructing multiple-choice items. Which statement is it?
C. To keep stems brief, place most words in an item's alternatives.
Presented below are four item-writing rules. Which one is a guideline often recommended for the construction of short-answer items?
C. Typically employ direct questions rather than incomplete statements, especially for young students.
Suppose that the developers of a new science achievement test had inadvertently laden their test's items with gender-based stereotypes regarding the role of women in science and, when the new test was given, the test scores of girls were markedly lower than the test scores of boys. Which of the following deficits most likely accounts for this gender disparity in students' scores?
Construct-irrelevant variance
Ms. Brown attempted to design an assessment that assessed the reading comprehension of her students in regard to their ability to comprehend informative text. After reviewing her drafted assessment, she realized she had relied on other kinds of text and had not included an adequate amount of informative text. Which phrase best describes Mrs. Brown's mistake?
Content underrepresentation
When students' test scores are-as predicted-correlated positively with those students' scores on a test aimed at a similar measurement mission, of what is this an example?
Convergent validity evidence
Which is the most appropriate description of a learning progression?
D. A learning progression is an ordered sequence of the stuff a student must learn so as to achieve a significant curricular outcome.
Which of the following types of assessment targets a student's attitudes, interests, and values?
D. Affective assessment
Which of the following is a reasonable explanation for why the word assessment may be used over the word testing?
D. Assessment is a broader descriptor of the type of measurement practices in which teachers engage.
This illustrative item is intended for use in a middle-school American history course. Directions: Remembering the class discussions of America's current immigration issues, please provide a brief essay on each of the issues cited below. You will have a full 50-minute class period to complete this examination, and you should divide your essay-writing efforts equally between the two topics. In grading your twin essays, equal weight will be given to each essay. Remember, compose two clear essays - one for each issue. Your Two Essay Topics 1. Why would some form of "amnesty" for illegal aliens be a helpful solution to at least part of today's U.S. immigration problems? 2. Why would some form of "amnesty" for illegal aliens be a disastrous solution to today's U.S. immigration problems? Which of the following statements most accurately describes the match between the illustrative item and the Chapter 7 guidelines for creating essay items?
D. At least one of the chapter's guidelines has been explicitly followed in the illustrative item.
Please select the one answer that most accurately identifies the particular item's "quoted" reason for teachers to know about assessment. "I was quite surprised when our state's department of education insisted that each of the state's teachers collect accurate evidence of their students' growth because such evidence was to be used in evaluating all of the state's teachers. I have, for my entire career, collected pretest and posttest evidence of my students' achievement status because this helps meirrespective of what the state wants me to dodetermine which changes, if any, are needed during next year's instruction." This is:
D. Both a traditional reason and one of today's reasons for teachers to know about assessment
This illustrative essay item was written for sixth graders. Thinking back over the mathematics lessons and homework assignments that you received during the past 12 weeks, what mathematical conclusions can you draw? Describe those conclusions in no more than 300 words, written by hand on the test-booklets provided or as a printed copy of your conclusions composed on one of our classroom computers. Select the statement that most accurately appraises this essay item for sixth-grade students.
D. Despite its adherence to one of the chapter's item-writing guidelines for essay items, the depiction of a student's task renders the item dysfunctional.
Which of the following is an item-writing guideline for the construction of multiple-choice items?
D. Don't ever use "all-of-the-above" alternatives, but use a "none-of-the-above" alternative to increase an item's difficulty.
Which of the following is a recommended item-writing rule for the construction of binary-choice items?
D. Employ a roughly equal number of statements representing the two categories being tested.
The Common Core State Standards were an attempt to outline what students should know at each grade level in which of the following subjects?
D. English Language Arts and Mathematics
Which represents the most appropriate definition of formative assessment?
D. Formative assessment is a planned process in which assessment-elicited evidence of students' status is used by teachers to adjust their ongoing instructional procedures or by students to adjust their current learning tactics.
This illustrative item is destined for use in a high-school speech course that, in recent weeks, has been focused on debate preparation. Directions: To conclude our unit on how to prepare successfully for a debate, please consider carefully the following preparation-focused topics. After doing so, choose one that you regard as most important - to you - and then write a 300-400 word essay describing how best to prepare for whatever topic you chose. Be sure to identify which of the potential topics you have selected. You will have 40 minutes to prepare your essay. Potential Essay Topics Introducing your position and defending it Use of evidence during the body of the debate Preparing for your opponents' rebuttal Please choose the statement that most accurately reflects the illustrative item's congruence with Chapter 7's guidelines for writing essay items.
D. The illustrative item is structured in direct opposition to one of the chapter's guidelines for writing essay items.
Consider the illustrative three-option multiple-choice item: An anonymously completed, self-report item regarding a student's values - an item that has no clearly correct answer - is best suited for use in an: a. cognitive examination b. affective inventory c. psychomotor skills test Which statement best characterizes the illustrative item?
D. The illustrative item violates a general item-writing guideline by providing a blatant grammatical clue to the correct answer.
Consider the illustrative three-option multiple-choice item. Schools often purchase shorter-duration tests in an attempt to assist classroom teachers in adjusting their instructional activities to the progress of their students. These tests are an example of an ___________: a. interim assessment b. computer-based assessment c. constructed response Which of the five general item-writing principles has been violated?
D. Thou shall not provide students with unintentional clues regarding appropriate responses.
Proper formative assessment is conducted ___________________.
D. during the instructional process
Which of the following was a major shift in the 2014 AERA-APA-NCME Standards for Educational and Psychological Testing?
Defining assessment validity as inference-accuracy for a specific purpose.
Cronbach's coefficient alpha and the Kuder-Richardson reliability formulae are examples of:
Internal-consistency coefficients
If Mr. Higgins, a fourth-grade teacher, tries to evaluate his major exams by ascertaining the degree to which his test's items are functioning in a similar manner, what kind of test-evaluative evidence is this?
Internal-consistency reliability evidence
The standard error of measurement is focused chiefly on:
None of the above
What kind of evidence is most eagerly sought by the commercial testing firms that develop academic aptitude tests?
Relations to other variables evidence of validity
Based on the 2014 edition of the Standards for Educational and Psychological Testing, and on common sense, which one of the following statements about students' test results represents a potentially appropriate phrasing that's descriptive of a set of students' test performances?
Students' scores on the test permit valid interpretations for this test's use.
Which of the following sources of validity evidence are teachers most likely to collect?
Test content
Classroom teachers are most apt to focus on which of the following?
Test content evidence of validity
A history teacher, Mrs. Scroggins, tries to determine the consistency of her tests by occasionally re-administering them to her students, then seeing how much similarity there was in the way her students performed. What kind of reliability evidence is Mrs. Scroggins attempting to collect?
Test-retest
If a multistate assessment consortium has generated a new performance test of students' oral communication skills and wishes to verify that students' scores on the performance test remain relatively similar regardless of the time during the school year when the test was completed, which of the following kinds of consistency evidence would be most appropriate?
Test-retest evidence of reliability
Stability data regarding assessment consistency is an instance of:
Test-retest reliability
Validity evidence can be collected from a number of sources. For instance, suppose that a mathematics test has been built by a school district's officials to help identify those middle-school students who are unlikely to pass a statewide 11th-grade high-school diploma test. The new test will routinely be given to the district's seventh-grade students. To secure evidence supporting the validity of this kind of predictive application, the new test will be administered to current seventh-graders, and the seventh-grade tests will also be given to the district's current eleventh-graders. This will permit the eleventh-graders' two sets of test results to be compared. Which best describes this source of validity evidence?
The relationship of 11th-graders' performances on the two tests
Which of the following statements best describes the relationship among the three sanctioned forms of reliability evidence?
The three forms of evidence represent fundamentally different ways of representing a test's consistency.
What is the chief function of validity evidence when employed to confirm the accuracy of score-based interpretations about test-takers' status in relation to specific uses of an educational test?
To support relevant propositions in a validity argument that's marshaled to determine the defensibility of certain score-based interpretations
Which of the following statements most accurately reflects the relationship between students' aptitude and their achievement?
Whereas aptitude tends to reflect potential, achievement tends to reflect prior learning.
Reliability refers to the ___________________.
consistency of the test scores