Educational Assessment - DRC1
Code of Fair Testing Practices in Education
-Guidelines primarily for use with commercially available standardized tests -Two sections states major obligations to test takers by professionals who develop or use education tests.
assessments
1. placement assessment: pre-test- measure student performance at the beginning of instruction 2. formative assessment: end of lesson quiz- happens during instruction 3. diagnostic assessment: diagnose learning difficulties during instruction 4. summative assessment: state testing: end of course achievement.
Absolute Grading
A common type of absolute grading is the use of letter grades defined by a 100-point system. *High subjectivity ex: A=90-100 B=80-89 C=70-79 D=60-69 F=under 60 Strengths 1. Grades can be described directly in terms of student performance, without reference to the performance of others. 2. All students can obtain high grades if mastery outcomes are stressed and instruction is effective. Limitations 1. Performance standards are set in an arbitrary manner and are difficult to specify and justify. 2. Performance standards tend to vary unintentionally due to variations in test difficulty, assignments, student ability, and instructional effectiveness. 3. Grades can be assigned without clear reference to what has been achieved (but, of course, they should not be).
normal curve
A symmetrical, bell-shape that describes the distribution of many types of data; most scores fall near the mean (68 percent fall within one standard deviation of it) and fewer and fewer near the extremes.
Stanine
A test interpretation that ranks scores on a scale of 1-9, with 5 being the average score a score that helps parents, students, and others to quickly and easily see where a student's score falls
Alternative Assessment
A title for performance assessments that emphasizes that these assessment methods provide an alternative to traditional paper-and-pencil testing. *test other than paper-pencil ex: As an educator, you are tired of standard paper and pencil exams. Instead, you have elected to experiment with a simulation where students demonstrate their abilities to complete an experiment.
Authentic Assessment
A title for performance assessments that stresses the importance of focusing on the application of understandings and skills to real problems in "real-world" contextual settings. ex: The final assessment includes students demonstrating the skills of ISO, exposure, white balance, and aperture control, resulting in a properly exposed photograph.
table of specifications
A two-way chart that specifies the number or proportion of test items (or assessment tasks) to be designed for each area of content and each type of intended learning outcome when planning a test or other assessment procedure. (1) selecting the learning outcomes to be tested (2) outlining the subject matter (3) making a two-way chart. The two-way chart describes the sample of items to be included in the test.
Reliability
Ability of a test to yield very similar scores for the same individual over repeated testings *lowers it if Scoring is subjective MUST BE CONSISTENT ways to test if it is reliable: 1. Test-retest method: The stability of test scores over a given period of time. 2. Equivalent-forms method: The consistency of the test scores over different forms of the test (that is, different samples of items). 3. Test-retest with equivalent forms: The consistency of test scores over both a time interval and different forms of the test. 4. Internal-consistency methods: The consistency of test scores over different parts of the test.
Placement Assessment
Covers the intended learning outcomes of the planned instruction This type of assessment is used prior to instruction
ESEA
Elementary and Secondary Education Act 1994 establishes equal opportunity to all students **states are required to set challenging standards, teaching & assessing high standards develop and administer assessments to measure student progress towards those standards
IDEA Act of 2004
Est that special ed teachers become highly qualified if they teach core academic content to students with disabilities. **moved students out of special ed into general classrooms Students required to test to the state academics with accommodations if necessary. If they cannot participate with accommodations, then alternate assessments will be used.
ESSA
Every Student Succeeds Act- required setting challenging standards for student achievement and developing and administering assessments to measure student progress toward those standards? Determination of sanctions Accountability for student academic achievement **included career & college readiness emphasis
Construct Irrelevance
Extent to which test scores are influenced by factors (ex: mode of presentation or response) that are irrelevant (not related) to the construct that the test is intended to measure. affects bias
evaluate
Justify a stand or decision Argue, Judges, criticizes, compares, justifies, concludes, discriminates, detects, defend, appraise, select, support, value, critique, weigh level 5 blooms taxonomy- higher level thinking
NCME
National Council on Measurement of Education
NDEA
National Defense Education Act provided money for education and training in science, math, and foreign languages **began the change to objective assessments with the use of Mult. Choice tests. 1958
Learning Outcomes
Outcomes that will be tested through assessments. 1st step for the table of specifications. Outcomes for a particular course will depend on the specific nature of the course, the objectives attained in previous courses, the philosophy of the school, the special needs of the students, & other. Include the following areas: (1) recall of knowledge (2) intellectual abilities and skills (3) general skills (labs, performance, communication, work-study) (4) attitudes, interests, and appreciations. best related to the first 2 categories in blooms taxonomy. Remember & Understand.
Differentiated Instruction
Practice of individualizing instructional methods, and possibly also individualizing specific content and instructional goals, to align with each student's existing knowledge, skills, and needs. Content: The knowledge & skills students need to master Process: The activities they use to master the content Product: The methods they use to demonstrate learning Accommodations: Do NOT change: * the expectations for learning * reduce the requirement of the task Modifications: DO change: * expectations for learning
create
Produce new or original work Combines, creates, formulates, designs, composes, constructs, rearranges, revises, assemble, conjecture, develop, author, investigate level 6 blooms taxonomy- highest level thinking
Analytic Scoring
Scoring a student's performance on an assessment by evaluating various aspects of it separately. WGU performance tasks: breaks down scoring into seperate scores for organization, style mechanics etc. GRADES SPECIFIC CRITERIA- INDIVIDUAL TREES assessing individual trees within the forest, not the whole forest (holistic)
attitude
Self image & social acceptance Determines what individuals WILL DO under natural conditions
Two-Way Chart
Step 3 in the table of specifications for assessments. Completes the table of specifications because everything is visually laid out for the assessment in a chart.
conceptual knowledge
The interrelationships among the basic elements within a larger structure that enable them to function together 2nd in the revised 4 categories of blooms taxonomies ex: able to describe the rock cycle
apply
Use information in new situations Demonstrates, computes, solves, modifies, arranges, operates, relates, uses, execute, implement, interpret, schedule, sketch level 3 blooms taxonomy- middle level thinking
standard deviation
a measure of variability that describes an average distance of every score from the mean useful in determining how a specific score relates to the mean score for an assessment.
learning outcome
a term that is included in instructional objective and it describes the specific results of instruction. evidence from student expected results student level FRUIT ON THE TREE ****include in notes
percentile band
an interpretation of a test score which takes into account the measurement error involved used in order to account for potential inconsistencies or errors in measurement.
classroom data
gives you specific, current information about how your students are doing at any given time. formative assessments observations projects/ essays / exams
alingment
how well an item conforms to an objective
standard error of measurement
hypothetical estimate of variation in scores if testing were repeated used to create error bands, or percentile bands, that are used when interpreting test scores.
achievment
knowledge and skills learned Degree of success of some past learning activity
content and process referenced test
more refined than domain referenced. ex: multiplication ( not math)
developmental objective
objectives are concerned with higher-level learning outcomes (e.g., understanding, application, evaluation). Students should be able to demonstrate that they have learned the basic knowledge and skills in order to pass. This is an example of absolute grading. However, it may not be possible to demonstrate full mastery of developmental objectives. Instead, relative grading should be used to show where a student falls within a range of achievement, or progress towards mastery.
Improving America's Schools Act
reauthorize a previous federal law
A Nation at Risk
recommended teaching and assessing high standards
Teachers' Standards for Student Assessment
1. Choosing assessment methods appropriate for instructional decisions 2. Developing assessment methods appropriate for instructional decisions 3. Administering, scoring, and interpreting the results of both externally produced and teacher produced assessment methods 4. Using assessment results when making decisions about individual students, planning teaching, developing curriculum, and school improvement 5. Developing valid pupil grading procedures that use pupil assessments 6. Communicating assessment results to students, parents, other lay audiences, and other educators 7. Recognizing unethical, illegal, and otherwise inappropriate assessment methods and uses of assessment information
Effective Assessment
1. Designed to assess a clearly defined set of intended learning outcomes. 2. The nature and function of the assessments are congruent with the outcomes to be assessed. 3. The assessments are designed to fit the relevant student characteristics and are fair to everyone. 4. Assessments provide information that is meaningful, dependable, and relevant. 5. Provision is made for giving the students early feedback of assessment results. 6. Specific learning weaknesses are revealed by the assessment results. 7. Assessment results provide information useful for evaluating the appropriateness of the objectives, the methods, and the materials of instruction. can contribute to: *the evaluation of instructional effectiveness *transfer of learning *retention *student motivation
Effective Instruction
1. Directed toward a clearly defined set of intended learning outcomes. 2. The methods and materials of instruction are congruent with the outcomes to be achieved. 3. The instruction is designed to fit the characteristics and needs of the students. 4. Instructional decisions are based on information that is meaningful, dependable, and relevant. 5. Students are periodically informed concerning their learning progress. 6. Remediation is provided for students not achieving the intended learning. 7. Instructional effectiveness is periodically reviewed and the intended learning outcomes and instruction modified as needed.
Item Writing
1. Select the type of test item that measures the intended learning outcome most directly. 2. Write the test item so that the performance it elicits matches the performance in the learning task. 3. Write the test item so that the test task is clear and definite. 4. Write the test item so that it is free from nonfunctional material. 5. Write the test item so that irrelevant factors do not prevent an informed student from responding correctly. 6. Write the test item so that irrelevant clues do not enable the uninformed student to respond correctly. 7. Write the test item so that the difficulty level matches the intent of the learning outcome, the age group to be tested, and the use to be made of the results. 8. Write the test item so that there is no disagreement concerning the answer. 9. Write the test items far enough in advance that they can be later reviewed and modified as needed. 10. Write more test items than called for by the test plan. arrangement in test: 1. For instructional purposes, it is usually desirable to group together items that measure the same outcome. 2. Where possible, all items of the same type should be grouped together. 3. The items should be arranged in terms of increasing difficulty.
effective student assessment
1. requires clear conception of all INTENDED LEARNING OUTCOMES 2. requires that a VARIETY OF ASSESSMENT PROCEDURES can be used. 3. requires that the INSTRUCTIONAL RELEVANCE of the procedures be considered. 4. requires an ADEQUATE SAMPLE OF STUDENT PERFORMANCE. 5. the be PROCEDURES BE FAIR TO EVERYONE. 6. requires the SPECIFICATIONS OF CRITERIA for judging successful performance. 7. requires FEEDBACK TO STUDENTS that emphasizes strengths of performance and weakness to be corrected. 8. must be supported by a COMPREHENSIVE GRADING AND REPORTING SYSTEM.
Holistic Scoring
A process for sorting or ranking students' written pieces on the basis of an overall impression of each piece. GRADES WHOLE PERFORMANCE- OVERALL IMPRESSION OF PERFORMACE- WHOLE FOREST assessing the whole forest, not individual trees (analytical) within the forest.
assessment procedures
An effective classroom assessment program will include a variety of procedures. Some will be based on a teacher's informal observations during instruction, while others provide a more systematic and structured means of obtaining evidence of achievement. 1. Informal observation during instruction. 2. Classroom achievement tests. 3. Performance assessment. 4. Product assessment. 5. Portfolio assessment.
Electronic Portfolio
An organized collection of projects, photos, and school work in a multimedia format that represents accomplishments. best practices: The student helps to select the entries, then reflects on their quality and evaluates the portfolio as a whole. The portfolio should contain samples of the student's best work, or work that shows growth. Having criteria in place ahead of time helps in determining what should be placed in the portfolio, and how it will be assessed. Portfolios can be used both by the teacher alone and in parent-teacher conferences, to show parents what students have been learning. Student growth or improvement should be measured in a portfolio for any content area. Never do: The teacher selects the work that is placed in the portfolio. You should include the student for ownership. When scoring portfolios, never use one or more checklists to evaluate the portfolio in a variety of
raw score
Assessment score based solely on the number or point value of correctly answered items. usually the number of items answered correctly. In order to compare results of difference tests, the raw score is converted to a derived score, such as a percentile or grade equivalent.
Checklist
Assessment tool with which a teacher evaluates student performance by indicating whether specific behaviors or qualities are present or absent. basically a list of measurable dimensions of a performance or product, with a place to record a simple "yes" or "no" judgment. ex: the steps to be followed might be placed in sequential order on the form; the observer would then simply check whether each action was taken or not taken.
Formative Assessment
Assessment used throughout teaching of a lesson and/or unit to gauge students' understanding and inform and guide teaching stick to these standards: * Feedback that is provided during instruction. * Challenging items that elicit understanding. * Collaboration between the teacher and the student. * Basing the assessment on relevant standards.
Performance Assessment
Assessments requiring students to demonstrate their achievement of understandings and skills by actually performing a task or set of tasks (e.g., writing a story, giving a speech, conducting an experiment, operating a machine). also called: authentic assessment alternative assessment ex: Students must write an original story including all necessary components of plot composition. evaluate performance by rubric, portfolios, self-assessments grade performance by: achievement, growth, habits assessing: procedure: giving the speech would be assessed, because the focus is on the delivery of the content and the use of persuasive techniques. Product: The product of the commercial would be assessed, because the focus is on the end product, which has qualities that can be identified and judged.
The Knowledge Dimension
Blooms taxonomy categories ranging from concrete to abstract with 4 categories. at each square where these meet a cognitive dimension- it equals a learning objective!! Factual, conceptual, procedural & metacognitive A. Factual Knowledge—The basic elements students most know to be acquainted with a discipline or solve problem in it. B. Conceptual Knowledge—The interrelationships among the basic elements within a larger structure that enable them to function together C. Procedural Knowledge—How to do something, methods of inquiry, and criteria for using skills, algorithms, techniques, and methods D. Metacognitive Knowledge—Knowledge of cognition in general as well as awareness and knowledge of one's own cognition
classroom assessment techniques
CAT- Tests that should be designed for the specific purposes that they are intended to serve and for the population that they will measure. Generated considerable controversy & inconsistency as states & districts attempt to measure student attainment of high standards.
aptitude
Capacity for learning; natural ability Determines what the student CAN DO when performing at their best
accommodations
Change HOW students are being taught and assessed. Students learn the same objective as peers students work together for a standard diploma Differentiation Do NOT change: * the expectations for learning * reduce the requirement of the task ex: more time different setting teacher reads aloud scribes Sing Language Pencil Grips Larger Font Colored printer paper text-to-speech Presentation: access visual information in other ways Response: allow students to respond answers differently- pen & paper, vocal Setting: change in environment is allowed Timing: extra time allowed
modification
Change WHAT students are being taught or assessed on. Students learn different objectives from non disabled peers Students work towards a special diploma Changing the content, material, or delivery of instruction DO change: * expectations for learning ex: reduce number of items alternate assignments lower level reading assignments reduction in competencies simplified language extra time (same as accommodation)
understand
Explain ideas or concepts Classifies, explains, summarizes, converts, predicts, interprets, illustrates, describe, discuss, identify, locate, recognize, support, select, translate level 2 blooms taxonomy- low level thinking
Diagnostic Assessment
Highly specialized, comprehensive and detailed procedures used to uncover persistent or recurring learning difficulties that require specially prepared diagnostic tests as well as various observational techniques. can be used when student's learning problems are so persistent that they cannot be resolved by the corrective help of the formative assessment. Thus, diagnostic assessment focuses on the common sources of error encountered by students, so that the learning difficulties can be pinpointed and remedied.
procedural knowledge
How to do something, methods of inquiry, and criteria for using skills, algorithms, techniques, and methods 3rd in the revised 4 categories of blooms taxonomies ex: explain the processes within the rock cycle that are responsible for 3 different types of rocks
metacognitive knowledge
Knowledge of cognition in general as well as awareness and knowledge of one's own cognition 4th in the revised 4 categories of blooms taxonomies ex: students find the rocks so interesting that they decide to research more about the rocks on their own.
NCLB
No Child Left Behind - 2001 Pres. Bush designed to promote "standards-based education reform" via assessments that measure progress; results often affect funding and administration control **sanctioned (punished) schools for failure to make adequate yearly progress re-authorization of ESEA. requires schools and states to meet "adequate yearly progress" by test scores. If schools do not meet, they will have graduated sanction goals.
Selection type items
Presents students with a set of possible responses, the student chooses the best answer 1. Multiple choice 2. True-false 3. Matching 4. Interpretive exercise BROAD SAMPLES SCORING IS OBJECTIVE DOES NOT ELIMINATE GUESSING DOES NOT ENCOURAGE ORIGINALITY Pros 1. identify specific learning errors 2. can test a large group of students because of large test items. 3. quick, objective scoring Cons: 1. guessing can be correct 2. harder to construct
Bloom's Taxonomy
Remember, Understand, Apply, Analyze, Evaluate, Create Selected Response: multiple choice, T/F, matching. Low levels * Remember * Understand Supply Response: Short answer. Low levels * Understand Constructed Response: Essays, Interactive. Higher levels * Apply * Analyze Performance Assessments: Higher order- Highest * Evaluate * Create
Supply type items
Require students to create their own answers 1. Short answer- *BROAD SAMPLES *ELIMINATES BLIND GUESSING *SCORING IN NOT OBJECTIVE *DOES NOT ENCOURAGE ORIGINALITY 2. Essay (restricted response) 3. Essay (extended response) *NO BROAD SAMPLES *ELIMINATES BLING GUESSING *SCORING IN NOT OBJECTIVE *ENCOURAGES ORIGINALITY Pros: 1. easier to construct 2. can measure complex learning outcomes Cons: 1. good writers can influence scores 2. bluffing can influence writing 3. time consuming to score 4. limited sample of learning tasks
outlining the subject matter
Step 2 of the table of specifications for assessments. A. Role of testing in the instructional process 1. Instructional decisions and test types 2. Influence of tests on learning and instruction B. Principles of achievement testing 1. Relation to instructional objectives 2. Representative sampling 3. Relevance of items to outcomes 4. Relevance of test to use of results 5. Reliability of results 6. Improvement of learning C. Norm-referenced versus criterion-referenced testing D. Planning the test 1. Determining the purpose of the test 2. Identifying the intended learning outcomes 3. Preparing the test specifications 4. Constructing relevant test items
grade equivalent score
Test score indicating the grade level of students to whom a test taker performed most similarly. a student's raw score on the test is converted to the grade level at which the score matches the average raw score of students in the norm group. As with other derived scores, tables in the test manual present parallel columns of raw scores and grade equivalents. ex: Reading= 4.5 = 4th grade halfway through Math 7.8= 7th grade etc. However, just because they score above grade level does NOT mean they are actually at that level. It can mean they answer questions faster and lucked out by getting more right.
objective-referenced tests
Tests that measure whether students have mastered a designated body of knowledge rather than how they compare with other students in a norm group. ex: student will multiply 2 digits by 3 digits
Validity
The ability of a test to measure what it is intended to measure MUST BE UNBIASED & FAIR 1. Validity is inferred from available evidence (not measured). 2. Validity depends on many different types of evidence. 3. Validity is expressed by degree (high, moderate, low). 4. Validity is specific to a particular use.
factual knowledge
The basic elements students most know to be acquainted with a discipline or solve problem in it. 1st in the revised 4 categories of blooms taxonomies ex: recognize the 3 basic types of rocks
ELL
Use formative or informal assessements like Performance based & Portfolio assessments for these kinds of students. must have these 4 elements for language learners: * Make criteria for evaluation clear to the students. * Require meaningful tasks. * Show students how to monitor their own work. * Designed for the students' proficiency level. ** Do not alter the CONSTRUCT of materials being assessed **should receive as much support as necessary. performance-based assessments to assess their language proficiency & academic achievement through oral reports, presentations, demonstrations, written assignments, and portfolios. ex: Reading with partners Retelling stories Role playing Giving descriptions or instructions using visual or written prompts Oral reporting to the whole class Telling a story by using a sequence of three or more pictures Completing dialogue or conversation through written prompts Debating, either one-on-one or taking turns in small groups Brainstorming Completing incomplete stories Playing games Portfolio collect descriptive records of a variety of student work over time that reflects growth toward the achievement of specific curricular objectives. These are a better way to crosscheck student progress than just one measure alone. ex: Samples of written student work: stories, completed forms, Drawings representing student content knowledge & proficiencies Tapes of oral work, such as role-playing, presentations Student accomplishments, such as performance on oral tasks Formal test data, checklists, and rating sheets Use the following: Scaffolding assessments allow ELLs to demonstrate their content knowledge through exhibits or projects, drawings, and graphic organizers. Consider giving ELLs extra time to complete these tasks, or to give short responses. Differentiated scoring scores content knowledge separately from language proficiency. To score content knowledge, look at how well ELLs understand key concepts, how accurate their responses are, and how well they demonstrate the processes they use to come up with responses.
criterion referenced
a test that measures certain criteria (knowledge about a specific set of facts or skills). --if a student knows the material, they will pass the test. Results reported in percentage. this is a PERCENTAGE-CORRECT score How accurately does performance on the assessment (e.g., test) predict future performance (predictive study) or estimate present performance (concurrent study) on some other valued measure called a criterion? 2 types to remember: The first type of study is concerned with the use of test performance to predict future performance on some other valued measure called a criterion. For example, we might use scholastic aptitude test scores to predict course grades (the criterion). For obvious reasons, this is called a predictive study. The second type of study is concerned with the use of test performance to estimate current performance on some criterion. For instance, we might want to use a test of study skills to estimate what the outcome would be of a careful observation of students in an actual study situation (the criterion).
rating scale
assessment in which a numerical value is assigned to specific behavior that is listed in the scale similar to the checklist and serves somewhat the same purpose in judging procedures and products. The main difference is that the rating scale provides an opportunity to mark the degree to which an element is present instead of using the simple "present-absent" judgment. 1-5 scale: *frequency with which an action is performed (e.g., always, sometimes, never), *the general quality of a performance (e.g., outstanding, above average, average, below average), *set of descriptive phrases that indicates degrees of acceptable performance (e.g., completes task quickly, slow in completing task, cannot complete task without help).
Domain-referenced tests
attempt to measure less-defined properties (e.g. intelligence), check for reliability and validity ex: math or science (not multiplication)
cognitive process dimension
blooms taxonomy categories of remember, understand, apply, analyze, evaluate, create
interpretative exercises
blooms taxonomy level varies with the exercise. could be many options including: multiple choice & alternative response items. Complex learning outcomes can frequently be more effectively measured by basing a series of test items on a common selection of introductory material. Pros: * efficient in measuring interpretation * more meaningful complex learning outcomes * scoring is easy, objective & reliable Cons: * difficult to construct effective items * highly dependent on reading level * highly subject to extraneous clues * does not measure ability to originate, organize & express ideas Rules: 1. Select introductory material that is relevant to the learning outcomes to be measured. 2. Select introductory material that is new to the examinees. 3. Keep the introductory material brief and readable. 4. Construct test items that call forth the type of performance specified in the learning outcome. 5. Follow the rules of effective item writing that pertain to the type of objective item used.
achievment assessment
broad category that includes all of the various methods for determining the extent to which students are achieving the intended learning outcomes of instruction. both pen & paper test and performance assessments.
standard
broad statement that follows a standard and describes what students should learn. provide the framework for for curriculum development broad statement state level TREE TRUNK ****include in notes
benchmark
clarifies in broad terms what a standard means. broadly clarify what students should know and be able to do at the end of a specified period of time clarify broad statement district level TREE BRANCHES ex: The content standard for an art class is to identify major artists from the 20th century. That is followed by a series of instructional statements, one of which is "Students will be able to identify Jackson Pollock's work." ****include in notes
Cumulative File
data that contains The records maintained by the local school district for any child enrolled in school. The file may contain evaluations and information about a child's disability and placement. It also contains grades and the results of standardized assessments. Parents have the right to inspect these files at any time. students long term history ex: housing conditions medical conditions changes in home life- divorce IEP
state tests
data that will give you insights into a student's current level. The data will tell you the level on which a student is performing. The data can also be used from a whole-class perspective to help you decide if the students need, for example, more challenge or more support.
instructional objectives
describe the intended learning outcomes & contain measurable performance terms. 1. list the general objectives as intended outcomes. 2. next list the specific type of student performances that will demonstrate that the learning outcome has been achieved. provide focus for : student learning assessments instruction Instructional objectives help to keep instruction, student learning, and assessment in close harmony. describe student performance at the end of the learning experience, as evidence of learning. class level
analyze
draw connections among ideas Differentiates, diagrams, estimates, separates, orders, subdivides, distinguishes between, organize, relate, compare, contrast, examine, experiment, question, test level 4 blooms taxonomy- higher level thinking
Relative Grading
grading on a curve, relative to other students. the students are typically ranked in order of performance (based on a set of test scores or combined assessment results), and the students ranking highest receive a letter grade of A, the next highest receive a B, and so on. ex: A= 15- top 10-20% of student scores B= 20- 20-30% of scores C= 45- 40-50% D= 10- 10-20% F= 5- 0-10% Strengths 1. Grades can be easily described and interpreted in terms of rank in a group. 2. Grades distinguish among levels of student performance that are useful in making prediction and selection decisions. Limitations 1. The percent of students receiving each grade is arbitrarily set. 2. The meaning of a grade varies with the ability of the student group. 3. Grades can be assigned without clear reference to what has been achieved (but, of course, they should not be).
Short Answer
high level of blooms taxonomy. requires the examinee to supply the appropriate words, numbers, or symbols to answer a question or complete a statement. used primarily to measure simple recall of knowledge. Pros: 1. It is easy to write test items. 2. Guessing is less likely than in selection-type items. 3. This item type is well suited to computational problems and other learning outcomes where supplying the answer is important. 4. A broad range of knowledge outcomes can be measured. Cons: 1. It is difficult to phrase statements so that only 1 answer is correct. 2. Scoring is contaminated by spelling ability when responses are nonverbal. 3. Scoring is tedious and time consuming. 4. This item type is not very adaptable to measuring complex learning outcomes. Rules: 1. State the item so that only a single, brief answer is possible. 2. Start with a direct question and switch to an incomplete statement only when greater conciseness is possible by doing so. 3. It is best to leave only one blank, and it should relate to the main point of the statement. 4. Place the blanks at the end of the statement. 5. Avoid extraneous clues to the answer. 6. For numerical answers, indicate the degree of precision expected and the units in which they are to be expressed.
essay
highest level of blooms taxonomy- especially useful for measuring the ability to organize, integrate, and express ideas. Students can write to the best of their ability and include their unique writing styles in this kind of question. Pros: 1. The highest level learning outcomes (analyzing, evaluating, creating) can be measured. 2. Preparation time is less than that for selection-type items. 3. The integration and application of ideas is emphasized. Cons: 1. There is an inadequate sampling of achievement due to time needed for answering each question. 2. It is difficult to relate to intended learning outcomes because of freedom to select, organize, and express ideas. 3. Scores are raised by writing skill and bluffing and lowered by poor handwriting, misspelling, and grammatical errors. 4. Scoring is time consuming and subjective, and it tends to be unreliable. Rules: 1. Use essay questions to measure complex learning outcomes only. 2. Relate the questions as directly as possible to the learning outcomes being measured. 3. Formulate questions that present a clear task to be performed. 4. Do not permit a choice of questions unless the learning outcome requires it. 5. Provide ample time for answering and suggest a time limit on each question. Grading rules: Always: *Score restricted-response answers using a point method, with a model answer as a guide. *When possible, have two or more persons grade each answer. *Evaluate answers to essay questions in terms of the learning outcomes being measured. Never: *Evaluate all of one student's essay answers before going on to the next student's responses. * Keep each student's strengths and weaknesses in mind when grading.
Bias
in tests and testing this refers to construct-irrelevant components that result in systemically higher or lower for identifiable groups of examinees. required for validity of an assessment. 9 types: 1. stereotyping 2. gender bias 3. regional or geographical bias 4. ethnic or cultural bias 5. socioeconomic or class bias 6. religious bias 7. ageism 8. bias against persons with disabilities 9. experiential bias revisions ex: Mrs. Jones was at home washing dishes. She washed 2 dishes every 4 minutes. How many dishes did Mrs. Jones wash in 1 hour? instead: Two siblings were washing dishes. etc. bias is the presence of some characteristic of an item and/or test that results in two individuals of the same ability but from different subgroups performing differently on the item and/or test.
Matching
low level blooms taxonomy- draw lines to possible outcomes or fill in the appropriate letter response. ex: 1-a, 2-j format consists of a series of stems, called premises, and a series of alternative answers, called responses. These are arranged in columns with directions that set the rules for matching. Pros: * compact and efficient * reading & response time is short * easily constructed * scoring is easy, objective & reliable Cons: * largely restricted to simple knowledge * limited homogeneous responses. * susceptible to irrelevant clues Rules: 1. Include only homogeneous material in each matching item. 2. Keep the lists of items short and place the brief responses on the right. 3. Use a larger, or smaller, number of responses than premises, and permit the responses to be used more than once. 4. Place the responses in alphabetical or numerical order. 5. Specify in the directions the basis for matching and indicate that each response may be used once, more than once, or not at all. 6. Put all of the matching item on the same page.
True/False
low level blooms taxonomy. answer is either true or false or yes on no. Pros: * good if only 2 possible options * less demand placed on reading ability * large number of answers can be answered in shorter time * complex outcomes can be measure with interpretative exercises * scoring is easy, objective & reliable Cons: * items might be ambiguous * could guess correctly * no diagnostic information from wrong answers * scores are more influenced by guessing Rules: 1. Include only one central idea in each statement. 2. Keep the statement short and use simple vocabulary and sentence structure. 3. Word the statement so precisely that it can unequivocally be judged true or false. 4. Use negative statements sparingly and avoid double negatives. 5. Statements of opinion should be attributed to some source unless used to distinguish facts from opinion. 6. When cause-effect relationships are being measured, use only true propositions. 7. Avoid extraneous clues to the answer. 8. Base items on introductory material to measure more complex learning outcomes.
Multiple Choice
many levels on blooms taxonomy. choose the correct choice from the list of options. most used type of assessment Stem- problem presented or question Distractor- plausible but wrong answers types: * incomplete statement: more concise, question form is easier to write and forces the test maker to pose a clear problem but tends to result in a longer stem * best-answer: alternatives are all partially correct but 1 is clearly better than the others. Used for more complex achievement. **Create a task in the stem that is clearly worded and can be understood without looking at the alternatives **Measure complex achievement by asking questions that apply content to new situations Always: *Phrase the stem to avoid equivocal answers *Use clear, simple language *Emphasize negative wording when it is used *Alternatives are grammatically consistent with the stem *Make the distractors plausible *Use the same level of detail for all distractors *Vary the length & position (never make each answer the same length or in the same ABCD order) *information given in one item should not help the student answer another item Never: *Limit the wording in the stem *State the stem in negative form when possible *Use always & never to emphasize *Include 2 responses with the same meaning *State the correct answer in textbook language *use "All of the Above" *be cautious if you use "non of the above"
minimal objective
objectives with concerns with basic knowledge, skill, and other lower-level learning outcomes.
restricted performance
performance assessment highly structured & limited in scope. Teacher is RESTRICTING the response from the students. gives students a LIMITED and HIGHLY structured task. ex: asking the students to write a paragraph that compares & contrasts earth and one other planet.
extended performance
performance assessment typically LESS structured & BROAD in scope, & more comprehensive. Students integrate and apply knowledge and skills. Teacher is EXTENDING the response from the students. ex: assigning students to write a short story about a visit to the planet of their choice.
objective
provides guidelines (blueprint) for planning instructional activities and preparing assessment instruction . TREE LEAVES must align with content and standards (grade level) ex: A teacher writes a unit on minerals for fourth-grade students. While writing the unit, the teacher refers to the National Science Standards, then writes the following goal for one of the lessons: The students will explain the process used to differentiate between talc and quartz after evaluating them using Moh's scale of hardness.
remember
recall facts and basic concepts Identifies, names, defines, describes, lists, matches, selects, outlines, duplicate, memorize, repeat, slate Level 1 Blooms taxonomy- low level thinking
Traditional Assessment
refers to the types of assessments generally found in classrooms: multiple choice, true-false, or matching objective exams or fill-in-the-blank, short-answer, or essay exams * limited in scope * paper-and-pencil * test knowledge, skill and content ony
supply Response
respond with a word, short phrase, or complete essay answer Students are required to SUPPLY the correct response: short answer or short essay items.
Selected Response
response test that is limited to presented options, common on standardized achievement tests Students are required to SELECT the correct answer: multiple choice, TF, matching, interperative provide items from which students can choose. questions might include multiple choice, matching, and/or true and false.
Constructed Response
response test that might include short answer or essay questions, or other open-ended questions and ask students to develop or create a response performance based
System of assessments
simple methods to gauge what new knowledge students have absorbed, understand & applied. series of "informal research" tools & practices designed to give teachers accurate information about the quality of student learning.
instructional objective
specific statements that describe how students will demonstrate achievement. describe intended learning outcomes clarify what standards and benchmarks mean specific statement class level ****include in notes
fairness
the bias, in some circumstances, to value cooperation and fair play over rational self-interest
cognitive demand
the level of thought process used to measure knowledge or skills. Competencies and evidence statements are assigned a cognitive demand based on a 3-level system: 1. recall - low 2. Application- medium 3. Analysis- High
anecdotal record
the simplest form of direct observation, is a brief narrative account of specific incident typically includes the observed behavior, the setting in which it occurred, and a separate interpretation of the event. * certain types of behavior (e.g., safety) * needing the most help (e.g., slow, careless). (1) they focus on meaningful incidents (2) they are recorded soon after the incident (3) they contain enough information to be understandable later (4) the observed incident and its interpretation are kept separate.
Code of Professional Responsibilities in educational measurement
to inform and remind those that are involved in educational assessment of their obligation to uphold the integrity of the manner in which assessments are developed, used, evaluated and marketed.
