Assessment Quiz Ch. 7-9

Essay

- Response is of a paragraph or more in length - Measure student's ability to synthesize, evaluate, and compose - Typically used to measure complex learning outcomes

creating a skill focused rubric

-Rule 1: Make sure the skill to be assessed is significant -Rule 2: Make certain all the rubric's evaluative criteria can be addressed instructionally ▪Students can be taught to master every potential criterion -Rule 3: Employ as few evaluative criteria as possible -Rule 4: Provide a succinct label for each evaluative criterion ▪Oral communication example: delivery, organization, content, and language -Helps you and your students focus on what is important -Rule 5: Match the length of the rubric to your own tolerance for detail ▪Published rubrics should be built to match the detail preferences of the teachers involved ▪Provide a long and a short version

Item-Writing Guidelines for Essay Items

1. Convey to students a clear idea regarding the extensiveness of the response desired. 2. Construct items so the student's task is explicitly described. 3. Provide students with the approximate time to be expended on each item as well as each item's value. 4. Do not employ optional items. 5. Precursively judge an item's quality by composing, mentally or in writing, a possible response.

Guidelines for Scoring Responses to Essay Items

1. Score responses holistically and/or analytically. 2. Prepare a tentative scoring key in advance of judging students' responses. 3. Make decisions regarding the importance of the mechanics of writing prior to scoring. 4. Score all responses to one item before scoring responses to the next item. 5. Insofar as possible, evaluate responses anonymously.

Item-Writing Guidelines for Short-Answer Items

1. Usually employ direct questions rather than incomplete statements, particularly for young students. 2. Structure the item so that a response should be concise. 3. Place blanks in the margin for direct questions or near the end of incomplete statements. 4. For incomplete statements, use only one or, at most, two blanks. 5. Make sure blanks for all items are equal in length.

key ingredients

1.Make sure students own their portfolios -Must be perceived as collections of their work, not receptacles for products you will grade 2.Decide what kinds of work samples to collect -Substantial variety is preferred over a limited range of work products 3.Collect and store work samples -Students decide, with teacher input, what to collect -Students collect and store them 4.Select criteria by which to evaluate work -Best if determined by teacher & student -Must be clearly described for self-evaluation to be successful 5.Students should evaluate products continually -Can evaluate holistically and/or analytically -Evaluations are dated and attached to the product 6.Schedule and conduct portfolio conferences -Essential 7.Involve parents in the process -Communicate the process to parents early -Encourage parents to review their children's work samples periodically ▪Enhances the message to students that the portfolio activity is worthwhile

Constructed response

A response in which a student generates a response or answer to an item -Focus is on traditional paper-and-pencil items in this chapter

can a portfolio be used for all three purposes

Can one portfolio achieve all three purposes? -Yes, but probably not well ▪The 3 purposes are fundamentally different ▪Build your portfolio assessment to satisfy the purpose you determine to be primary

challenges to differentiation

Challenges Require more time Bring with them to the classroom great reservoirs of knowledge that other students do not bring Need to move around more than others Seem to have given up on school - or themselves or adults - and are angry or lethargic much of the time Have difficulty concentrating during whole-class discussions Are poor test takers but actually understand far more than they show Will not engage with learning if they fail to see the point of it

Why we use constructed response

Closer to real world situations, mult. choice doesn't happen in real life. Measures higher-level knowledge and skills

truths about differentiation

Differentiation is a philosophy - a way of thinking about teaching and learning. It is, in fact, a set of principles. Learning to differentiate instruction well requires rethinking one's classroom practice and results from an ongoing process of trial, reflection, and adjustment in the classroom itself. Most teachers who remain in a classroom for longer than a day do pay attention to student variation and respond to it in some way- especially with students who threaten order in the classroom. However, very few teachers proactively plan instruction to consistently address differences. Although differentiation is an instructional approach, effective DI is inseparable from a positive learning environment, high-quality curriculum, assessment to inform teacher decision making, and flexible classroom management. To the degree that any one of those elements is weak, the others are also diminished.

Classroom applications

Portfolio assessment -Ideally the central focus of instruction ▪Students appraise their work with teacher input -Contrast earlier versions with later versions -Strengthens relationship between instruction and assessment ▪Has immense diagnostic value ▪Effective instruction should = discernable improvement •Classroom portfolio example (1 alternative) -Elementary students keep 3 portfolios, one for each subject -Students place early and revised work in the portfolio and date them -Meet with the teacher 3 times a semester for ~15 min. to evaluate their work -End of the year, students select products that represent their best versions and how the versions were created

Portfolio Assessment vs Standardized Testing

Portfolio- Represents the range of reading and writing in which students are engaged ST- Assesses students across a limited range of reading and writing assignments P- Engages students in assessing their progress and establishing ongoing learning goals ST- Mechanically scored or scored by teachers who have little input P- Measures each student's achievement while allowing for individual differences ST-Assesses all students on the same dimensions P- Represents a collaborative approach to assessment ST-Assessment process is not collaborative P- Student self-assessment is a goal ST-Student self-assessment is not a goal P- Addresses improvement, effort, and achievement ST- Addresses achievement only P- Links assessment and teaching to learning ST- Separates learning, testing, and teaching

Performance assessment must have

Some argue must have three features: 1.Multiple evaluative criteria ▪Response must be judged on more than one criteria (e.g., Student's ability to speak Spanish judged on the basis of the student's accent, syntax, and vocabulary) 2.Pre-specified quality standard ▪Evaluative criteria clearly explicated in advance 3.Judgmental appraisal ▪Depend on human judgments

Short answer

Student responds using a word, phrase or sentence •Can be a response to either a direct question or incomplete statement •Typically measure relatively simple learning outcomes

extended-response item

an essay item that gives students few constraints about how he or she is to respond especially with respect to the length of his or her responses

restricted-response item

an essay item that markedly limits the form and content of a students response to the item

Differentiation

comprehensive and flexible process that includes the planning, preparation and delivery of instruction to address the diversity of students' learning needs within the classroom. Through DI, teachers take into account who they teach, what they teach, where they teach and how they teach. "shaking up" what goes on in the classroom so that students have multiple options for taking in information, making sense of ideas, and expressing what they learn.

pros and cons of essay

pros- Measure complex learning outcomes and more than one outcome cons- Difficult to write the item properly and score reliably

pros and cons of short answer

pros: because responses are produced by the student, partial knowledge is not sufficient cons: Scoring can be difficult Scoring may result in inaccurate representations of students' abilities

Inside DI

readiness- a student's current proximity to specific knowledge, understanding, and skills. interests- That which engages the attention, curiosity, and involvement of a student learning profiles- preference for taking in, exploring, or expressing content. content- The knowledge, understanding, and skills we want students to learn process- How students come to understand or make sense of the content affect- How students' emotions and feelings impact their learning product- How students demonstrate what they have come to know, understand, and re able to do after an extended period of learning

halo effect

the error of allowing a test-scorer's overall impression of a student to influence any criterion-by-criterion evaluations of the student's response to a test

Rubric criteria

•A few truly important criteria are better to a ton of trifling criteria -Go for the big ones -Ask colleagues for ideas -Isolate a small number of instructionally addressable evaluative criteria •The greatest payoff from a well-formed scoring rubric is its contribution to improved instruction

portfolio assessment

•A systematic collection of students' work •Relatively new to education •Must be updated as a person's achievements and skills grow •An alternative to standardized testing

Pros and cons of performance assessment

•Advantages -Can measure higher cognitive skills -Can measure real-life skills -Can motivate students -Help students recognize the skills •Disadvantages -Take time to construct -Take time to create exemplary rubrics -Take time to score

Rubrics

•Defined as the scoring procedures for judging student performance -Specify the evaluative criteria-the standards on which the score or judgment will be based ▪E . g., Writing composition could be scored on the basis of organization, word choice, and clarity or alternately on the basis of spelling, punctuation, and grammar •Rubric must contain -Evaluative criteria ▪Defined as the factors that determine quality ▪Most important for developers to establish ▪Isolate 3 or 4 criteria per rubric -Descriptions of the qualitative differences for the evaluative criteria ▪Each criteria must be described in words to specify the qualities of the students' response

portfolio for showcasing

•Elements in a portfolio for showcasing: -A letter of introduction to reviewers -A table of contents -List of skills & knowledge demonstrated -Representative sample of best work -Dates on all entries -Evaluative criteria or rubric used -Student's self-reflection on all entries

Evaluating Performance Test Tasks

•Generalizability -Will performance generalize to similar tasks? •Authenticity -Is the task similar to a real-world task? •Multiple foci -Does the task measure multiple outcomes? •Teachability -Is the task one that students can gain proficiency as the result of instruction? Fairness -Is the task fair to all? •Feasibility -Can the task be realistically implemented when the cost, space, time, and equipment requirements are considered? •Scorability -Can the task responses be reliably and accurately scored?

Holistic or analytical rubric

•Need to decide if performance will be scored holistically or on a criterion-by-criterion basis -Holistic scoring fails to communicate to students, especially low-performing students, however it's quicker to do -Analytic scoring can pin-point areas of weakness and provide meaningful feedback

Generalizability

•Often more difficult to generalize students' knowledge and skills because there are fewer tasks -E.g., 20 double-digit multiplication problems vs. deriving an original solution to a problem and describing it in writing -Choose tasks that optimize the likelihood of accurately generalizing about your students' capabilities

Teacher time

•One additional factor to consider -Significance of the skill being assessed -If performance assessments aren't based on genuinely demanding skills, you will stop using them ▪The skills they assess need to be worth the trouble

ratings and observations

•Performance assessments that yield products are easier to score -Can be scored at your leisure •Assessments that yield a behavior must be observed -Usually scored during the observation •Systematic observations may be set up so you can make immediate and/or delayed allocations of points

Pros and Cons of Portfolio Assessment

•Pros -Tailored to individual needs, abilities, and interests -Documents and evaluates student growth in a meaningful way -Fosters student self-evaluation -Increases personal-ownership •Cons -Takes loads of time to carry out effectively -Teachers need sufficient training -Difficult to evaluate because the responses vary from student to student ▪Scoring guides can be useful, but are often too general

Sources of error

•Scoring-instrument flaws -Due to lack of descriptive rigor -Scoring criteria are ambiguous -Result in unreliable ratings •Procedural flaws -Due to rating on too many criteria -Overwhelmed scorers are ineffectual •Teachers' personal-bias errors -Unintentional bias in their scoring ▪Generosity error ▪Severity error ▪Central-tendency error ▪Halo effect

Large Scale applications

•Several states and large school districts have installed portfolios as a key component of their accountability programs -Results are not encouraging ▪Portfolios judged by teachers often result in bias, or if by trained scorers can be expensive -Vermont program is an example ▪Independent evaluators have found evidence of considerable unreliability

Performance assessment

•Students construct an original response •Teachers, or others, observe the process of the construction (performance) •Judgment is made of the performance Authentic ▪Real-world situations used as tasks ▪Alternative to traditional paper-and-pencil tests •Some want the tasks to be demanding -Higher cognitive difficulty level -Process is assessed as well as product

Suitable tasks

•Tasks generally few in number -Each task is significant and complex -Great care must be taken in creating and selecting the tasks •Tasks should provide evidence for inferences to be made from key curricular aims

Work-Sample Selection

•Think divergently about the kinds of entries that constitute the portfolio's chief concerns -But don't search for work samples just for the sake of variety -Include samples that allow valid inferences about students' skills and knowledge to be drawn

Appraising Portfolios

•Use a scoring rubric •Select the evaluative criteria •Students must have access to & fully understand the rubric used •Should contribute to your interpretations regarding students' knowledge or skills

Analytical scoring

▪Assigns points to each factor ▪Identifies students' strengths and weaknesses ▪Ignores overall quality of response

Documentation of student progress

▪Called working portfolios ▪Provide evidence of student growth or lack of it ▪Provide meaningful opportunities for self-evaluation

Skill focused rubric

▪Conceptualized around the skill that is: -Being measured by the assessment -Being pursued instructionally by the teacher ▪Provides clear instructional support ▪Most recommended type of rubric ▪Compose a narrative essay (400-600 words) to describe the class visit by local firefighters who described home fire escape plans. ▪Criterion for judging the organization of students' essays: -Superior •Contains an introduction, body, and conclusion •Content is sequenced logically, chronologically, or in order of importance

Hypergeneral rubric

▪Evaluative criteria described in very general terms ▪Lack clear descriptions of criteria for evaluating students' performances ▪Provide teachers with no genuine benefits for instructional planning ▪Compose a narrative essay (400-600 words) to describe the class visit by local firefighters who described home fire escape plans. ▪Criterion for judging the organization of students' essays: -Superior = content has been arranged in a genuinely excellent manner -Inferior = content not adequately organized -Adequate = in between superior and inferior

Task specific

▪Evaluative criteria linked to particular task ▪Does little to illuminate instructional decision-making ▪Students lose out if the performance task is too specific ▪Compose a narrative essay (400-600 words) to describe the class visit by local firefighters who described home fire-escape plans. ▪Criterion for judging as superior: 1.Commence with a recounting of the rationale for home fire-escape plans 2.Follow with a description of 6 elements of a plan in the order presented 3.Conclude with at least 3 fire-safety statistics

Holistic Scoring

▪Focuses on the entire response as a whole ▪Reflects overall performance

Showcasing student accomplishment

▪Selection of best work as a celebration -Especially appropriate for early grades ▪Students reflect on the quality of the work -Helps student to learn better -Provides insight into how the student learns

Evaluation of student status

▪Used to determine whether students have met previously specified quality level ▪Greater standardization is recommended if using portfolios for this purpose ▪Less need for student self-evaluation