Assessment & Evaluation

Cons of essay questions

-Very few items can be completed during a typical test session or class period -Takes a long time to grade and offer feedback -Can be less valid and reliable than selected response (human judgement)

benefits of using checklists

-reusable and can be adapted as your evaluation needs change -can be customized to meet your specific objectives of the evaluation (targeted evaluation) -quick to use since you only need to check if the child exhibits the behavior -reliable tool due to having targets specifically spelled out so different people can administer the evaluation with similar results

pros of analytic rubrics

-specific focus on each element of the prompt -specific feedback in the matrix -emphasis is on the details of instruction

cons of analytic rubric

-time consuming to make -language must be clear with little freedom for student interpretation

Level 3 Learning & Verbs

Applying -Use, diagram, make a chart, draw, apply, solve, calculate

Level 1 Learning & Verbs

Remembering -list, find, name, identify, locate, describe, memorize, define

How can you increase the usefulness of T/F questions?

-Correct the false statements -Justify both the true and the false statements -Elaborate on each question describing what makes it true or what make it false

Advantages to T/F questions

-Easy to write, score, and more reliable if teacher has to make a judgement decision -Can test large amounts of content in a short amount of time -Students can answer 3-4 questions per minute -Can have a high level of reliability -Helpful in testing misconceptions -Less dependent on students reading ability than some other question types

Advantages to ordering questions

-Evaluates students ability to make decisions about the sequence of items -All grade levels are able to learn a sequence to complete a task -Valuable when testing procedures, processes, steps in a sequence, etc

Pros of essay questions

-Evaluates students' ability to organize their thoughts and communicate effectively in writing -Takes little time to create the prompts -Using multiple essay questions can increase the reliability of an essay test

Advantages to Matching Questions

-Maximum content coverage in a minimum amount of space/prep time -Valuable for content that students need to memorize, remember, and use often -Valuable for testing content knowledge for novice learners

Disadvantages to T/F questions

-Questions are easy to answer -Difficult to discriminate between students that know and don't know the material -Students have a 50/50 chance of getting it right by guessing -You need a large number of items for high reliability (50-75 if whole test is T/F)

Tips for writing short answer/completion questions

-State the question in such a way that only one specific word, phrase, or solution can be correct -Omit only the big concept, the significant word in the sentence -Keep the line (missing answer) toward the end of the sentence -Completion items should contain enough info so that a person who has mastered the material can tell precisely what is being said -For problems requiring numerical answers, specify the degree of precision required -Sometimes the verb you use in your objective can be the task you choose for your short answer question. Be specific with explaining what that direction word means.

Multiple Choice Questions - Creating the Stem

-State the stem as a direct question -Have just one question with one answer -Eliminate unnecessary wordiness -Include in the stem any words that would be repeated in each alternative -Avoid grammatical clues between the stem and response (a/an) -State each item positively. Do not use negatively stated stems

Disadvantages of matching questions

-Time consuming for students to answer -Can be a lot of reading -May not be appropriate for higher levels of learning

Matching Questions Procedures

-Use a line and number on the LEFT HAND SIDE (stem or question) -Use a capital letter on the RIGHT HAND SIDE (options or answers) -Avoid asking students to draw lines to make the match if over 5 items -The left hand side is the stem or question; it is the longer side -The options or answers are on the right hand side; it is the shorter side -Use items from the same category of information for the 5-8 questions and options -Use at least one more option/answer than stems or questions to avoid guessing -Include explicit directions: what to do, where to write, how the terms are related, and the number of uses for the answers

MC questions - creating the distractors

-Use capital letters for responses -Use misconceptions as distractors -Do not use all or non of the above -Whenever possible, present distractors in some logical order (chronological, low to high, etc) -Put each distractor on a separate line -Keep distractors in a similar format (phrases or sentences) -Use similar length distractors -True statements that do not answer the question are good distractors -Avoid absolute words like always, never, or all

Tips for creating fill in the blank questions

-Write a true statement that is always true -Identify the key concept and replace it with a blank -Rewrite the statement so the blank is towards the end of the sentence -Include one blank per sentence whenever possible; two blanks max if you must have two

Procedures for writing T/F questions

-Write absolute true statements, then change the key concept to make it false -Write in your own words -Use a clear and simple sentence -Use one idea per test item -Use misconceptions as false statements -State each item positively -Avoid extreme modifiers: all, always, none, never, etc -Include explicit directions

Tips for writing essay questions

-Write the question extremely clearly so all students know exactly what is expected (format, number of examples, length, etc) -Make each part of an essay question measurable -Include why or how for greater student success and increase difficulty -Give students guidelines on time limits and the amount of information expected -Several questions each having a relatively narrow focus are better than one broad questions -To draw attention to different aspects of the question prompt you could separate the questions from one another. Leave space between parts or label the parts a b c etc (optional)

Characteristics of peer review

-a class partner, or peer from another section can conduct a review -can occur online, in person, or assigned as homework -sharing work before the due date with the intention of receiving comments to improve work -include an opportunity to edit and improve the work before it is graded -questions should be directly related to the grading criteria -limit the number of responses per question (1 or 2 max) -the same student does not have to review the entire project. rotate to respond to each question -hold reviewers accountable for the level of completeness of their comments

Advantages of MC questions

-almost any well-defined objective can be tested fairly in a MC format -considered one of the most versatile of all item types -minimum of writing for students -guessing is reduced -can cover a broad range of content

Additional uses for multiple choice questions

-as a pre-test before a unit or topic is taught -as an outline for a class discussion (why are the distractors wrong?) -checks for understanding, formative assessments -partners discuss the questions then answer using a student response system (iClickers) -partner quizzing and/or discussion -students can create questions to check their level of understanding -as a self-test before an exam or quiz

Using and scoring essay questions

-avoid the use of optional essays -before giving the test, develop a list of the main points that should be included in each answer and develop a scoring system -before scoring essays, review the material students were expected to learn -inform students how you will deal with factors that are independent from the learning outcome being measured, such as format, time, length, page numbers, academic integrity -score essay questions without knowing which student produced which responses -score all student responses to one item before scoring responses to another item -avoid giving students a choice of questions to answer -ask students to write more than one essay to increase validity and reliability -write out the correct answer yourself -decide on the facts or ideas a student must mention for full credit

Tips for creating rubrics

-create your own rubrics for your students and what you taught, avoid canned rubrics -create succinct, abbreviated rubrics that capture the key evaluative criteria needed to judge students' responses -lengthy rubrics and those that include jargon or are written negatively should be avoided -limit the number of criteria. be careful not to combine independent criteria -use specific terms and phrases -use measurable criteria. include characteristics of what quality or the absence of quality looks like. be sure that the description is not the criteria/element and vice versa.

analytic rubric

-defined as a matrix in which all the elements of performance are defined, independently valued and scored -use three or four quality levels -complete the high and low ends first, then work inward -list skills and traits consistently across quality levels

Disadvantages of MC questions

-difficult and time consuming to construct -difficult to come up with plausible distractors -may appear to be tricky when misconceptions are used

cons of holistic rubrics

-doesn't identify specific areas to improve -student projects or performances may exhibit traits at two or more levels at the same time

4 parts of a rubric

-elements/criteria -levels of quality -descriptions -score

location of 4 parts in an analytic rubric

-elements/criteria: down the left hand side. include 1 criteria or element per box (more if the element is logistic), elements come from the key criteria from the prompt, and can include weighting in the element box -levels of quality: located across the top. examples include numbers or words like beginning, developing, accomplished, met, not met, etc. -descriptions: middle boxes in the matrix. terms, phrases, or a sentence that describes the level of quality for each element -score: in a column or other place above or below the rubric

Steps to developing your own rubric

-identify your objective; make it measurable -create the prompt for the project -use the prompt to decide on the 3-5 most important criteria/elements to be assessed -make a list of concrete descriptors for each criterion/element -choose levels of quality -decide which type of rubric to use -begin with the structure of the table; fill it in beginning with the elements/criteria

pros of holistic rubrics

-less time to create -determines if the performance as a whole meets the target -efficient for large group scoring -takes less time to assess

single point rubric

-like an analytic rubric, it breaks down the components of an assignment into different criteria -only describes the criteria for proficiency, level 3 or 4 -it does not attempt to list all the ways a student could exceed or fall short of expectations -if you have an analytic rubric, take column 3 and move it to the center for a single point rubric -add a column titled "concerns" on the left and "exceeded" on the right. this is where the instructor writes comments

Drawbacks to using checklists

-limits the assessment scope -offers an incomplete picture of a child's behavior -leaves no gray area for kids who might exhibit a behavior only in certain situations or only partially demonstrates the trait -doesn't take into consideration the amount of time it takes to exhibit the behavior -lacks descriptive evidence or supporting details -a second evaluator will get an idea of what the child can do, but will not get a sense of what lead to the behaviors or actions

Benefits of peer review

-more people can see your work -students receive feedback from a variety of perspectives -get detailed feedback on the strengths and weaknesses of students work -students learn how to critically analyze their own work by analyzing others work -revising work after feedback is received is a great learning tool -builds classroom community -promotes respect and understanding of other cultures and ways of thinking -models the interpersonal, interactive, and group problem-solving nature of most workplaces -students submit better, well-rounded work to be scored

Clarity of your instructions for conducting a peer review exercise

-similar to the essay, write directions that indicate how many responses you expect (reasons, examples, points of consideration) -include what to do: circle, mark, write in complete sentences, etc -include where to do the task -ask questions specific to the assignment for high quality feedback -peer comments need to directly address the questions asked

Develop a feedback form

-start with the criteria/elements in the rubric or prompt -decide which characteristics you are aiming for in your assignment (what you will be grading/offering feedback on?) -convert your list of characteristics into a peer feedback form. types include criteria grid with reader responses, rubric, open-ended question format, or combining several types of questions into a unique form

advantages to single point rubric

-there is less language in the rubric so students are more likely to read and use it -takes less time to create and still offers a good amount of detail about expectations -doesn't attempt to identify all the ways a student can exceed or fall short of the goal -enhances creativity for students

holistic rubrics

-views product or performance as a whole, describes the characteristics of the product or performance according to each element/criteria -levels of quality are listed down the left hand side of the matrix -one column is used for the descriptions -each description includes all the elements/criteria in the same box, for each level

What levels of learning can you cover with matching questions?

1 or 2

Appropriate tasks for short answer

1) List: list format, brief statement with specific info 2) Describe: stating the most noticeable qualities or features of something 3) Name: using the appropriate discipline-specific term upon request 4) Compare: identifying characteristics or qualities that resemble each other 5) Contrast: identifying the differences 6) Explain: clarify, interpret elaborate on, give reasons why, and analyze causes or reasons 7) Discuss: pointing out the important features and expressing critical judgement

What can be observed using a checklist?

1) behaviors and classroom routines: turning work in, coming to class prepared, following structured activities, free play, safety, etc 2) social skills: listening, speaking, respect, responsibility, friendship, inclusive language, inclusive behavior, some aspects of group work 3) student success skills: reading, note taking, writing, organization, time management, etc 4) performance assessment strategies: presentation skills, demonstration, debates, skits, etc

Appropriate tasks for essays

1) comparing: identify the similarities and differences between 2) relating cause and effect: what are the major causes of...? what would be the effects of...? 3) justifying: explain why you agree or disagree with the following statement 4) generalizing: state a set of principles that can explain the following event 5) inferring: how would character X react to the following? 6) creating: what would happen if...? 7) applying: describe a situation that illustrates the principle of... 8) analyzing: find and correct the reasoning errors in the following passage 9) evaluating: assess the strengths and weaknesses of...

Types of peer review questions

1) completeness 2) evaluation: judging how well the student completed a part of the project 3) identification: identify the content included in the project 4) opinion: indicating what you think about a part of the project 5) improvement: recommend a way to improve or enhance the product 6) alternatives: suggest another way to do something other than the way it is presented

Levels of Blooms Taxonomy covered

1-5 (remember, understand, apply, analyze, and evaluate)

Assessment

A process by which information is obtained relative to some known objective or goal

Who is the coolest classmate in 415 because they made this awesome Quizlet set?

Camille (also its her birthday on May 18th so if you want to bring her candy to the final as a combo birthday/thank you for making this Quizlet set that would be encouraged but not necessary)

Level 5 Learning & Verbs

Evaluating -Judge, test, critique, defend, criticize

Objectives

Statements that define the expected goal of a curriculum, course, lesson, or activity in terms of skills or knowledge that will be acquired by a student as a result of instruction

Validity

The test measures what it claims to measure

Level 4 Learning & Verbs

Analyzing -Categorize, examine, compare/contrast, organize

Level 6 Learning & Verbs

Creating -Design, build, construct, plan, produce, devise, invent

Summative Assessment

High stakes assessment used to measure student learning at the end of an instructional unit comparing it to a standard or benchmark

Levels of Blooms covered in short answer/completion questions

Levels 1 or 2

Formative Assessment

Low stakes assessment used to monitor student learning and provide ongoing feedback

Evalution

Putting "value" on something. We make a judgement about a given situation based on our measurements and our assessments

Reliability

The consistency of a test , if replicated you will get similar results

Grading

The process of applying standardized measurements of varying levels of achievement in a course

Measurement

The process of using a standard measure and a standard instrument

Level 2 Learning & Verbs

Understanding -Interpret, summarize, explain, infer, paraphrase, discuss

Who can be observed with a checklist?

individual students, partners, small groups, whole class

disadvantage of single point rubric

more writing for the teacher