Module 15 - Assessment item writing: short answer and essay

Rules for scoring essay answers

1 Evaluate answer to essay questions in terms of the learning outcomes being measured 2 Score restricted-response answers by the point method, using a model answer as a guide. 3 Grade extended-response answers by the rating method, using defined criteria as a guide 4 Evaluate all of the students' answers to one question before proceeding to the next question 5 Evaluate answer to essay questions without knowing the identity of the writer 6 Whenever possible, have two or more persons grade each answer

What are the rules for writing short-answer items? (6)

1 State the items so that only a single, brief answer is possible. 2 Start with a direct question and switch to an incomplete statement only when greater conciseness is possible by doing so. 3 It is best to leave only one blank, and it should relate to the main point of the statement. 4 Place the banks at the end of the statement. 5 Avoid extraneous clues to the answer. 6 For numerical answers, indicate the degree of precision expected and the units n which they are to be expressed.

What are the rules for writing essay questions?

1 Use essay question s to measure complex learning outcomes only. 2 Relate the questions as directly as possible to the learning outcomes being measured. 3 Formulate questions that present a clear task to be performed. 4 Do not permit a choice of questions unless the learning outcome requires it. 5 Provide ample time for answering and suggest a time limit on each question.

8 point inspection checklist for evaluating short-answer items:

1. Is this type of item appropriate for measuring the intended learning outcome? 2. Does the item task match the learning task to be measured? 3. Does the item call for a single, brief answer? 4. Has the item been written as a direct question or a well-stated incomplete sentence? 5. Does the desired response relate to the main point of the item? 6. Is the blank placed at the end of the statement? 7. Have clues to the answer been avoided (e.g., "a" or "an," length of the blank)? 8. Are the units and degree of precision indicated for numerical answers?

8 point inspection checklist for evaluating essay questions.

1. Is this type of item appropriate for measuring the intended learning outcome? 2. Does the item task match the learning task to be measured? 3. Is the question designed to measure complex learning outcomes? 4. Does the question make clear what is being measured and how the answer will be evaluated? 5. Has terminology been used that clarifies and limits the task (e.g., "describe" not "discuss")? 6. Are all students required to answer the same questions? 7. Has an ample time limit been indicated for each question? 8. Have adequate provisions been made for scoring answers (e.g., model answers or criteria for evaluating)?

What are some related terms for helping create a complex outcome for essay questions? For the outcome of: Applying Analyzing Creating

Applying arrange, compute, describe, demonstrate, illustrate, rearrange, relate, summarize Analyzing break down, describe, diagram, differentiate, divide, list, outline, separate Creating compose, design, devise, draw, formulate, make up, present, propose

What are some related terms for helping create a complex outcome for essay questions? For the outcome of comparing? Interpreting?

Comparing compare, classify, describe, distinguish between, explain, outline, summarize Interpreting convert, draw, estimates, illustrate, interpret, restate, summarize, translate

What are some tips to assist with short-answer items with regards to rule #5 ' Avoid extraneous clues to the answer'?

Keep the blanks all the same size. No short blanks for short answers and long blanks for long answers. The indefinite article "a" or "an" just before the blank can offer clues. - in these cases use the plural leading into it or use a(an). example: The supply-type item used to measure the ability to organize and integrate material is called an (essay item). fix: Supply-type items used to measure the ability to organize and integrate material are called (essay items).

Summary comparison of Selection-type items vs Essay Questions: Probable Effect on Learning

Selection-type items: Encourages students to remember, interpret, and use the ideas of others. Essay questions: Encourages students to organize, integrate, and express their own ideas.

Summary comparison of Selection-type items vs Essay Questions: Learning Outcomes Measured

Selection-type items: Good for measuring the recall of knowledge, understanding, and application levels of learning; inadequate for organizing and expressing ideas. Essay questions: Inefficient for measuring the recall of knowledge; best for ability to organize, integrate, and express ideas.

Summary comparison of Selection-type items vs Essay Questions: Scoring

Selection-type items: Objective, simple, and highly reliable. Essay questions: Subjective, difficult, and less reliable.

Summary comparison of Selection-type items vs Essay Questions: Preparation of Items

Selection-type items: Preparation of good items is difficult and time consuming. Essay questions: Preparation of good items is difficult but easier than selection-type items.

Summary comparison of Selection-type items vs Essay Questions: Factors Distorting Scores

Selection-type items: Reading ability and guessing. Essay questions: Writing ability and bluffing.

Summary comparison of Selection-type items vs Essay Questions: Sampling of Content

Selection-type items: The use of a large number of items results in broad coverage, which makes representative sampling of content feasible. Essay questions: The use of a small number of items limits coverage, which makes representative sampling of content infeasible.

What are some strengths of Essay Questions? (3) Limitations(4)

Strengths: 1 The highest level learning outcomes (analyzing, evaluating, creating) can be measured. 2 Preparation time is less than that for selection-type items. 3 The integration and application of ideas is emphasized. Limitations: 1 There is an inadequate sampling of achievement due to time needed for answer each question. 2 It is difficult to relate to intended learning outcomes because of freedom to select, organize, and express ideas. 3 Scores are raised by writing skill and bluffing and lowered by poor handwriting, misspelling, and grammatical errors. 4 Scoring is time consuming and subjective, and it tends to be unreliable.

What are some strengths of Short-Answer items? Limitations?

Strengths: 1 it is easy to write test items 2 Guessing is less likely than in selection-type items 3 This item type is well suited to computational problems an other learning outcomes where supplying the answer is important 4 A broad range of knowledge outcomes can be measured. Limitations: 1 It is difficult to phrase statements so that only one answer is correct 2 Scoring is contaminated by spelling ability when responses are nonverbal 3 Scoring is tedious and time conuming 4 This item type is not very adaptable to measuring complex learning outcomes.

What are some related terms for helping create a complex outcome for essay questions? For the outcome of: Synthesizing Generalizing Evaluating

Synthesizing arrange, combine, construct, design, rearrange, regroup, relate, write Generalizing construct, develop, explain, formulate, generate, make, propose, state Evaluating appraise, criticize, defend, describe, evaluate, explain, judge, write

In a restricted-response question what types of limits are placed and what are some examples.

These types of questions generally have strict limits on the answers. Some ways in which it is conducted is by stating to: "list", "define", or "give reasons" And sometimes it is also limited further by the use of introductory material or special directions.

What is the short-answer item primarily used for? (2)

computational problems simple recall of knowledge

Supply-type items are typically dividing into what three groups?

short-answer items restricted-response essay extended-response essay

What is the essay question especially useful for measuring? (3)

the ability to organize, integrate, and express ideas.

What levels on the Bloom's taxonomy are extended-response items especially useful for measuring learning outcomes at?

the evaluating and creating levels

What levels on the Bloom's taxonomy are restricted-response items especially useful for measuring learning outcomes at?

the understanding, application, and analysis levels of learning.

Due to some of the weaknesses of when should a short-answer item be used?

when the intent is to have students 'recall' the information where computational problems are used where a selection-type item would make the answer obvious

What are some related terms that should not be generally used for creating a complex outcome for essay questions due to the outcome generally going to the recall of knowledge?

who," "what," "when," "where," "name," and "list." These terms tend to limit the response to the recall of knowledge outcomes