Assessment Final

What are the levels of the Instructional Hierarchy and how do you define them?

(I'm Heard - A Fat Guy Applauded: Instructional Hierarchy - Acquisition; Fluency; Generalization; Adaption/Application) Acquisition- developing accuracy Fluency (proficiency)- developing fluency or speed Generalization- novel situation Adaption/Application- adapted The first step in learning a new skill is acquiring a skills and getting it right (accurately reading words). Need lots of practice and support. Next, we have to develop proficiency and use less cognitive resources (fluency). Then, generalization to novel situations. Finally, application and adaptation.

Name and define the five "big ideas" of reading in the NRP (Armbruster) report?

1. Phonemic awareness: Understanding of the smallest units of sounds that make up the language. Knowledge that words can be broken up into sounds. 2. Phonics: Letters represent sounds. The skill to link and blend sounds, like sounding out a new word 3. Reading Fluency and accuracy: with practice, the easy, automatic reading of connected text that facilitates comprehension 4. Vocabulary- words we must know to communicate effectively. Oral vocabulary is what we use to speak or listen and reading vocabulary are words we recognize in print. 5. Comprehension: Knowing the meaning of individual words. Understanding the meaning of written material Instructional Hierarchy Acquisition Slow and inaccurate Fluency Accurate but slow Generalization Novel situation Adoption/Application Problem solving

inference

A tentative conclusion or assumption that lacks explicit support from available data.

Adaptive reasoning (5 strands of math proficiency)

Adaptive reasoning - capacity for logical thought, reflection, explanation, and justification

Principles of CBA-ID

Aligns assessment with what is taught in the classroom Starts with knowns and progresses to unknowns Prioritizes instructional match Promotes mastery learning

If a problem is common for more than 20% of learners ...

Analyze at the system level

Area Under the Curve (AUC)

Quantitative measure of the accuracy of a test in discriminating between students at-risk and not at-risk across all decision thresholds. Helps determine cut points. More area under the curve means more diagnostic accuracy.

What can we do to improve reports?

Reduce jargon Focus on referral question(s) Individualize content Focus on strengths Make concrete recommendations

Reliability

Reliability-consistency of score over time (measurement of the amount of noise your measurement is measuring). Also, proportion of observed score variance that is true score variance.

Low inference assessment

Relies on direct observation of explicit behaviors; Easy to see defined behaviors; Straightforward to test hypotheses

High inference assessment

Relies on theoretical constructs like personality, aptitude profiles; difficult to observe constructs; requires many assoptions

Place value and grouping

Represent a numeral by choosing the correct number of base-ten blocks, and writing numerals according to a given number of base ten blocks

systematic way of quantifying and observing behavior in real time

SDO

Specificity

Specificity is the probability of the student testing negative if the difficulty does not exist. (TN /TN+FP) The proportion who did well on screener out of all students who did well on criterion measure.

Stage v. Phase Theory of Development

Stage - each stage has a prerequisite requirement Phase - does NOT.

Strategic competence (5 strands of math proficiency)

Strategic competence - ability to formulate, represent, and solve mathematical problems

Summative v. Formative uses of Assessment

Summative - summary of assessment activities; often part of a SPED eval; discussed at one team meeting; placed in student's file after completion Formative: Includes assessment and intervention results; written throughout intervention with a student; discussed regularly throughout problem solving; ideally helps drive instruction , referenced often.

Number Sequence

The ability to identify numbers and link numbers 0-31 (kindergarten) or 0-120 (first grade) with names.

Match Quantity

The ability to make a connection (or match) between a quantity 1 to 10 and the numeric symbol representing that quantity.

How does Deno (Ch. 2) define a problem?

The discrepancy between perceived reality and expectation

True negative

True negative- there is not a problem and they pass.

True positive

True positive- there is a problem and they do not pass.

CBMmath

Two types of assessments (process and fluency) that measure different computational skills from grades 1-6: Fluency skills: Automatized; include rapid recall of math facts (multiplication times tables) Process skills: The student may have to solve multiple steps to reach a solution (items are not memorized); more difficult than fluency skills

Number Sequence

Understanding of the mental number line, counting forward and backward, number before, number after, number between. K uses numbers 0-20; 1 uses numbers 0-120

Learning to read

Unitization Seeing words in chunks rather than single letters Information Processing Automaticity Reading words without having to think about it It happens involuntarily It happens quickly It uses few cognitive resources Reading fluency Automaticity + prosody Reading Development Decoding + Comprehending = Reading Phases vs. stages Stage theory "requires each stage to be a prerequisite for the next stage;" phase theory does not.

CBM vs. CBA-ID

Use CBM--> at risk for not meeting benchmarks (more broad), CBA--> challenges in current curriculum (start with CMB, lead into CBA; "drilling down," problem analysis)

Define the following acronyms: WW CLS WSC CWS (Hint: CBM - Writing)

WW- number of words written CLS- correct letter sequences- any two adjacent letters that are correct according to the spelling of the word. WSC- words spelled correctly CWS- correct word sequences- any two adjacent, correctly spelled words that are acceptable within the context of the sample.

Instructional Level (Gickling and Armstrong, 1978)

When you struggle, you disengage. When you are challenged with support, you engage. If you are not challenged at all, you get things right but stop paying attention. This idea applies to all students, not just those struggling. CBA-ID assumes curriculum is at the level of most student's instructional level. Highest on task behavior while reading at instructional level (though some overlap) Time on task slightly lower on independent level than frustration level Comprehension scores higher overall at instructional and independent levels than frustration level Supports the instructional level proposed in original work by Gickling and Armstrong

What is a norm, and how does it differ from a benchmark or criterion?

normal/average performance based on a comparison of the people in that sample. A norm is always a comparison. some outside criterion/target we want people to hit- not based on how others are performing.

Validity evidence

singular concept, types of evidence that support our interpretations

Phoneme

smallest, distinct units of sound

Orthography

spelling

Curriculum-based measurement

standardized, general outcome measurement, standardized and specific measurement procedures that can quantify student performance in basic academic skill areas

Syllable

a word part that contains a vowel or, in spoken language, a vowel sound (e-vent; news-pa-per; ver-y).

Test

administer a set of tasks, questions, or items that are designed to quantify some skill or trait.

NRP area of earlyReading for: concepts of print onset sounds letter names letter sounds rhyming word blending word segmenting sight words 50 sigh words 150 decodable words nonsense words sentence reading oral repetition

concepts of print: Doesn't have an NRP area onset sounds: Phonological awareness letter names: Phonics letter sounds: Phonics rhyming: Phonological awareness word blending: Phonological awareness word segmenting: Phonological awareness sight words 50: Phono, Phonics, Fluency, Vocab sigh words 150: Phono, Phonics, Fluency, Vocab decodable words: Phonics nonsense words: phonics sentence reading: Fluency oral repetition: fluency

Regression equation

dependentvariabile(y) = y-intercept (bo) + slopecoefficient (b1x) + randomerrorresidual(e)

What does IDEAL stand for?

general problem solving model- simple, clear and practical I Identify the problem to be solved D Defining the problem E Exploring alternative solutions A Applying the selected intervention L Looking at the effects

Technical Adequacy

high reliability/validity. Evidence that supports your use of a test Important for accountability assessments - certainty that what you are measuring is truly what you are measuring - eg teacher success by EOY performance

What are the four phases of reading development according to Ehri?

1. Pre-alphabetic: use cues not related to alphabetic system, non readers. Ex- "stop sign says stop" Do not form letter-sound connections to read words. If they read words at all they do it by remembering selected visual features. For example, they might remember look by the two eyeballs in the middle, or dog by the tail at the end, or camel by the humps in the middle 2. Partial alphabetic phase: learn names/sounds of letters and use these to remember how to read words. They use just the beginning and ending sounds Soap and stop may be said the same 3. Full alphabetic phase: learn sight words by forming complete connections between letters in spelling and phonemes in pronunciation 4. Consolidated phase- increasingly retain more sight words in memory, memorized common patterns of letters (Email Read - Preschool Parties For Connie; Ehri Reading - Pre-, Partial, Full, Consolidated)

Percent of students in: Tier 1, Tier 2, Tier 3

80 (core instruction problem at Tier 1) - 15 - 5

What is ALT and why is it important to consider?

ALT- Academic learning time- The amount of time a student is actively engaging in coursework. Sometimes an outcome measure, sometimes data you can gather.

Composing and Decomposing

Ability to compose and decompose numbers to and from 5 and 10.

Story problems

Ability to connect a real world story to a number expression

Verbal Addition & Subtraction

Ability to fluently perform basic arithmetic

Criterion-related validity evidence

Criterion-related: (split into predictive and concurrent) taking one test and correlating it to another (comparison test) Predictive-related: criterion but give one now and give one later (high stakes test, can predict who will do well in spring on it in the fall), extent to which scores predict future performance in same area

Aligning an aptitude, or patterns of aptitudes, with instructional or treatment methodology

Aptitude by Treatment Interactions

earlyMath

Assess early numeracy skills for student K and 1st grade. EarlyMath was developed to enable efficient screening and progress monitoring across three domains of early numeracy (Number, Relations, and Operations)

Content-related validity evidence

Content-related: evidence to which a test measures a particular domain, test development will provide this evidence, appropriately sampling items from the domain area

Say you do universal screening and 30% of students are at risk. What do you think is happening here?

Core curriculum problem - more than 30% of students have been identified as being at risk

Quantity discrimination

Choosing the biggest or smallest numeral from a set of 4; a reliable and valid indicator of student performance

Mathematical proficiency has 5 strands, what are they?

Conceptual understanding Procedural fluency Strategic competence Adaptive reasoning Productive disposition

Conceptual understanding (5 strands of math proficiency)

Conceptual understanding - comprehension of mathematical concepts, operations, and relations

Diagnostic accuracy

Diagnostic accuracy- how accurate your diagnoses is, sensitivity and specificity

Discriminate validity evidence

Discriminate: how well it distinguishes between two groups

GOM v. mastery measure

GOM-General outcome measure, broad global skills, long term (takes longer time to accomplish) (CBM)--> broad , ex. Reading ability Mastery measure- individual steps toward different mastery levels, short term (takes shorter time to accomplish) CBA-ID

General outcome measurement

General outcome measure, broad global skills, long term (takes longer time to accomplish) (CBM)--> broad , ex. Reading ability

False negative

False negative- screener says there is not a problem and they don't pass.

False positive

False positive- screener says there is a problem but they pass.

Equal Partitioning

Identifying if quantities are equivalent and demonstrate the ability to equally partition a set of objects.

Idiographic v. nomothetic frameworks

Idiographic- within one person (progress monitoring). Looking at one within-student issue. Nomothetic- looking at group.

Counting Objects

Initial Counting Schema: ability to count a small set of objects with 1-to-1 correspondence (developed by age 4-5) Principle of Cardinality: "How many?"

RIOT/ICEL

Method of Assessment: Review / Interview / Observe / Test Factors that Affect Learning: Instruction / Curriculum / Environment / Leaner

Negative Predictive Power

Negative Predictive Power is the proportion of truly non-struggling students among all those with negative test results. (TN / TN + FN)

Three domains of early numeracy

Number: the general understanding of rules and processes of the counting sequence Relations: Knowledge of how two or more numbers or objects are related, and understanding of the mental number line Operations: Understanding how groups of numbers, objects, or combination of the two are composed and decomposed.

Final Report Tips

Past tense for the report including conclusions and Future tense for recommendations Assessment results do not PROVE anything "These results suggested...." Spell check and proofread!! Referral questions (as clearly defined problems) should drive assessment choice AND report writing

Positive Predictive Power

Positive Predictive Power is the proportion of truly struggling students among those with a positive test result (TP / TP + FP)

Standard protocol v. problem-solving model (paradigm shift)

Pre-determined interventions Going through the problem solving model.

Procedural fluency (5 strands of math proficiency)

Procedural fluency - skill in carrying out procedures flexibly, accurately, efficiently, and appropriately

Procedural fluency vs. Conceptual knowledge (5 strands of math proficiency)

Procedural: knowledge of procedures, when and how to use them accurately and efficiently especially needed to support conceptual understanding of place value and the meanings of rational numbers. It also supports the analysis of similarities and differences between methods of calculating. Conceptual: more than facts and methods, but an understanding and connection of mathematical ideas (e.g., place value)

If education is an intervention, what is the desired outcome of the intervention?

Produce specific developmental outcomes This is a value judgement Education should provide students with the skills necessary to fill future goals (college, career, life skills)- Very value and culture-laden.

Productive disposition (5 strands of math proficiency)

Productive disposition - habitual inclination to see mathematics as sensible, useful, and worthwhile, coupled with a belief in diligence and one's own efficacy

3 stages of research on CBM (Fuchs)

Technical features of the static score (aka level/ screening) A single point in time, does it do what we think it does? Technical features of slope Trend/ progress monitoring Instructional Utility Aka usefulness

Key developmental markers of math proficiency

World of Formal symbols, which encompasses: World of real quantities & World of Counting Numbers

If a problem is common for more than 5% of learners ...

analyze at a group

If a problem is common for an Individual learner...

analyze at the individual level

Curriculum based assessment (the broad term)

any set of measurement activities that uses direct observation and recording of a student's performance in the local curriculum as a basis for gathering information to make instructional decisions, anything that is pulled from the curriculum

Subskill mastery measurement

individual steps toward different mastery levels, short term (takes shorter time to accomplish) CBA-ID

Subitizing

instantly recognizing a quantity between 1 and 3

Evaluation

making a judgement based on the information that has been gathered.

Which of the following is an ecological factor that impacts educational outcomes

number of words exposed to before entering school

Measurement

quantifying a skill or trait

Onset and rime

the onset consists of the initial consonant or consonant blend, the rime consists of the vowel and any final consonants.

Sensitivity

the probability of student testing positive if the given difficulty exists. (TP / TP + FN). The proportion of students who didn't do well on criterion measure who did not do well on screener.

Assessment

the process of gathering data to inform decision making.

Array identification

using mental counting to identify quantities between 4 and 7

Grapheme

written symbol that represents a phoneme