Assessment Final
What are the levels of the Instructional Hierarchy and how do you define them?
(I'm Heard - A Fat Guy Applauded: Instructional Hierarchy - Acquisition; Fluency; Generalization; Adaption/Application) Acquisition- developing accuracy Fluency (proficiency)- developing fluency or speed Generalization- novel situation Adaption/Application- adapted The first step in learning a new skill is acquiring a skills and getting it right (accurately reading words). Need lots of practice and support. Next, we have to develop proficiency and use less cognitive resources (fluency). Then, generalization to novel situations. Finally, application and adaptation.
Name and define the five "big ideas" of reading in the NRP (Armbruster) report?
1. Phonemic awareness: Understanding of the smallest units of sounds that make up the language. Knowledge that words can be broken up into sounds. 2. Phonics: Letters represent sounds. The skill to link and blend sounds, like sounding out a new word 3. Reading Fluency and accuracy: with practice, the easy, automatic reading of connected text that facilitates comprehension 4. Vocabulary- words we must know to communicate effectively. Oral vocabulary is what we use to speak or listen and reading vocabulary are words we recognize in print. 5. Comprehension: Knowing the meaning of individual words. Understanding the meaning of written material Instructional Hierarchy Acquisition Slow and inaccurate Fluency Accurate but slow Generalization Novel situation Adoption/Application Problem solving
inference
A tentative conclusion or assumption that lacks explicit support from available data.
Adaptive reasoning (5 strands of math proficiency)
Adaptive reasoning - capacity for logical thought, reflection, explanation, and justification
Principles of CBA-ID
Aligns assessment with what is taught in the classroom Starts with knowns and progresses to unknowns Prioritizes instructional match Promotes mastery learning
If a problem is common for more than 20% of learners ...
Analyze at the system level
Area Under the Curve (AUC)
Quantitative measure of the accuracy of a test in discriminating between students at-risk and not at-risk across all decision thresholds. Helps determine cut points. More area under the curve means more diagnostic accuracy.
What can we do to improve reports?
Reduce jargon Focus on referral question(s) Individualize content Focus on strengths Make concrete recommendations
Reliability
Reliability-consistency of score over time (measurement of the amount of noise your measurement is measuring). Also, proportion of observed score variance that is true score variance.
Low inference assessment
Relies on direct observation of explicit behaviors; Easy to see defined behaviors; Straightforward to test hypotheses
High inference assessment
Relies on theoretical constructs like personality, aptitude profiles; difficult to observe constructs; requires many assoptions
Place value and grouping
Represent a numeral by choosing the correct number of base-ten blocks, and writing numerals according to a given number of base ten blocks
systematic way of quantifying and observing behavior in real time
SDO
Specificity
Specificity is the probability of the student testing negative if the difficulty does not exist. (TN /TN+FP) The proportion who did well on screener out of all students who did well on criterion measure.
Stage v. Phase Theory of Development
Stage - each stage has a prerequisite requirement Phase - does NOT.
Strategic competence (5 strands of math proficiency)
Strategic competence - ability to formulate, represent, and solve mathematical problems
Summative v. Formative uses of Assessment
Summative - summary of assessment activities; often part of a SPED eval; discussed at one team meeting; placed in student's file after completion Formative: Includes assessment and intervention results; written throughout intervention with a student; discussed regularly throughout problem solving; ideally helps drive instruction , referenced often.
Number Sequence
The ability to identify numbers and link numbers 0-31 (kindergarten) or 0-120 (first grade) with names.
Match Quantity
The ability to make a connection (or match) between a quantity 1 to 10 and the numeric symbol representing that quantity.
How does Deno (Ch. 2) define a problem?
The discrepancy between perceived reality and expectation
True negative
True negative- there is not a problem and they pass.
True positive
True positive- there is a problem and they do not pass.
CBMmath
Two types of assessments (process and fluency) that measure different computational skills from grades 1-6: Fluency skills: Automatized; include rapid recall of math facts (multiplication times tables) Process skills: The student may have to solve multiple steps to reach a solution (items are not memorized); more difficult than fluency skills
Number Sequence
Understanding of the mental number line, counting forward and backward, number before, number after, number between. K uses numbers 0-20; 1 uses numbers 0-120
Learning to read
Unitization Seeing words in chunks rather than single letters Information Processing Automaticity Reading words without having to think about it It happens involuntarily It happens quickly It uses few cognitive resources Reading fluency Automaticity + prosody Reading Development Decoding + Comprehending = Reading Phases vs. stages Stage theory "requires each stage to be a prerequisite for the next stage;" phase theory does not.
CBM vs. CBA-ID
Use CBM--> at risk for not meeting benchmarks (more broad), CBA--> challenges in current curriculum (start with CMB, lead into CBA; "drilling down," problem analysis)
Define the following acronyms: WW CLS WSC CWS (Hint: CBM - Writing)
WW- number of words written CLS- correct letter sequences- any two adjacent letters that are correct according to the spelling of the word. WSC- words spelled correctly CWS- correct word sequences- any two adjacent, correctly spelled words that are acceptable within the context of the sample.
Instructional Level (Gickling and Armstrong, 1978)
When you struggle, you disengage. When you are challenged with support, you engage. If you are not challenged at all, you get things right but stop paying attention. This idea applies to all students, not just those struggling. CBA-ID assumes curriculum is at the level of most student's instructional level. Highest on task behavior while reading at instructional level (though some overlap) Time on task slightly lower on independent level than frustration level Comprehension scores higher overall at instructional and independent levels than frustration level Supports the instructional level proposed in original work by Gickling and Armstrong
What is a norm, and how does it differ from a benchmark or criterion?
normal/average performance based on a comparison of the people in that sample. A norm is always a comparison. some outside criterion/target we want people to hit- not based on how others are performing.
Validity evidence
singular concept, types of evidence that support our interpretations
Phoneme
smallest, distinct units of sound
Orthography
spelling
Curriculum-based measurement
standardized, general outcome measurement, standardized and specific measurement procedures that can quantify student performance in basic academic skill areas
Syllable
a word part that contains a vowel or, in spoken language, a vowel sound (e-vent; news-pa-per; ver-y).
Test
administer a set of tasks, questions, or items that are designed to quantify some skill or trait.
NRP area of earlyReading for: concepts of print onset sounds letter names letter sounds rhyming word blending word segmenting sight words 50 sigh words 150 decodable words nonsense words sentence reading oral repetition
concepts of print: Doesn't have an NRP area onset sounds: Phonological awareness letter names: Phonics letter sounds: Phonics rhyming: Phonological awareness word blending: Phonological awareness word segmenting: Phonological awareness sight words 50: Phono, Phonics, Fluency, Vocab sigh words 150: Phono, Phonics, Fluency, Vocab decodable words: Phonics nonsense words: phonics sentence reading: Fluency oral repetition: fluency
Regression equation
dependentvariabile(y) = y-intercept (bo) + slopecoefficient (b1x) + randomerrorresidual(e)
What does IDEAL stand for?
general problem solving model- simple, clear and practical I Identify the problem to be solved D Defining the problem E Exploring alternative solutions A Applying the selected intervention L Looking at the effects
Technical Adequacy
high reliability/validity. Evidence that supports your use of a test Important for accountability assessments - certainty that what you are measuring is truly what you are measuring - eg teacher success by EOY performance
What are the four phases of reading development according to Ehri?
1. Pre-alphabetic: use cues not related to alphabetic system, non readers. Ex- "stop sign says stop" Do not form letter-sound connections to read words. If they read words at all they do it by remembering selected visual features. For example, they might remember look by the two eyeballs in the middle, or dog by the tail at the end, or camel by the humps in the middle 2. Partial alphabetic phase: learn names/sounds of letters and use these to remember how to read words. They use just the beginning and ending sounds Soap and stop may be said the same 3. Full alphabetic phase: learn sight words by forming complete connections between letters in spelling and phonemes in pronunciation 4. Consolidated phase- increasingly retain more sight words in memory, memorized common patterns of letters (Email Read - Preschool Parties For Connie; Ehri Reading - Pre-, Partial, Full, Consolidated)
Percent of students in: Tier 1, Tier 2, Tier 3
80 (core instruction problem at Tier 1) - 15 - 5
What is ALT and why is it important to consider?
ALT- Academic learning time- The amount of time a student is actively engaging in coursework. Sometimes an outcome measure, sometimes data you can gather.
Composing and Decomposing
Ability to compose and decompose numbers to and from 5 and 10.
Story problems
Ability to connect a real world story to a number expression
Verbal Addition & Subtraction
Ability to fluently perform basic arithmetic
Criterion-related validity evidence
Criterion-related: (split into predictive and concurrent) taking one test and correlating it to another (comparison test) Predictive-related: criterion but give one now and give one later (high stakes test, can predict who will do well in spring on it in the fall), extent to which scores predict future performance in same area
Aligning an aptitude, or patterns of aptitudes, with instructional or treatment methodology
Aptitude by Treatment Interactions
earlyMath
Assess early numeracy skills for student K and 1st grade. EarlyMath was developed to enable efficient screening and progress monitoring across three domains of early numeracy (Number, Relations, and Operations)
Content-related validity evidence
Content-related: evidence to which a test measures a particular domain, test development will provide this evidence, appropriately sampling items from the domain area
Say you do universal screening and 30% of students are at risk. What do you think is happening here?
Core curriculum problem - more than 30% of students have been identified as being at risk
Quantity discrimination
Choosing the biggest or smallest numeral from a set of 4; a reliable and valid indicator of student performance
Mathematical proficiency has 5 strands, what are they?
Conceptual understanding Procedural fluency Strategic competence Adaptive reasoning Productive disposition
Conceptual understanding (5 strands of math proficiency)
Conceptual understanding - comprehension of mathematical concepts, operations, and relations
Diagnostic accuracy
Diagnostic accuracy- how accurate your diagnoses is, sensitivity and specificity
Discriminate validity evidence
Discriminate: how well it distinguishes between two groups
GOM v. mastery measure
GOM-General outcome measure, broad global skills, long term (takes longer time to accomplish) (CBM)--> broad , ex. Reading ability Mastery measure- individual steps toward different mastery levels, short term (takes shorter time to accomplish) CBA-ID
General outcome measurement
General outcome measure, broad global skills, long term (takes longer time to accomplish) (CBM)--> broad , ex. Reading ability
False negative
False negative- screener says there is not a problem and they don't pass.
False positive
False positive- screener says there is a problem but they pass.
Equal Partitioning
Identifying if quantities are equivalent and demonstrate the ability to equally partition a set of objects.
Idiographic v. nomothetic frameworks
Idiographic- within one person (progress monitoring). Looking at one within-student issue. Nomothetic- looking at group.
Counting Objects
Initial Counting Schema: ability to count a small set of objects with 1-to-1 correspondence (developed by age 4-5) Principle of Cardinality: "How many?"
RIOT/ICEL
Method of Assessment: Review / Interview / Observe / Test Factors that Affect Learning: Instruction / Curriculum / Environment / Leaner
Negative Predictive Power
Negative Predictive Power is the proportion of truly non-struggling students among all those with negative test results. (TN / TN + FN)
Three domains of early numeracy
Number: the general understanding of rules and processes of the counting sequence Relations: Knowledge of how two or more numbers or objects are related, and understanding of the mental number line Operations: Understanding how groups of numbers, objects, or combination of the two are composed and decomposed.
Final Report Tips
Past tense for the report including conclusions and Future tense for recommendations Assessment results do not PROVE anything "These results suggested...." Spell check and proofread!! Referral questions (as clearly defined problems) should drive assessment choice AND report writing
Positive Predictive Power
Positive Predictive Power is the proportion of truly struggling students among those with a positive test result (TP / TP + FP)
Standard protocol v. problem-solving model (paradigm shift)
Pre-determined interventions Going through the problem solving model.
Procedural fluency (5 strands of math proficiency)
Procedural fluency - skill in carrying out procedures flexibly, accurately, efficiently, and appropriately
Procedural fluency vs. Conceptual knowledge (5 strands of math proficiency)
Procedural: knowledge of procedures, when and how to use them accurately and efficiently especially needed to support conceptual understanding of place value and the meanings of rational numbers. It also supports the analysis of similarities and differences between methods of calculating. Conceptual: more than facts and methods, but an understanding and connection of mathematical ideas (e.g., place value)
If education is an intervention, what is the desired outcome of the intervention?
Produce specific developmental outcomes This is a value judgement Education should provide students with the skills necessary to fill future goals (college, career, life skills)- Very value and culture-laden.
Productive disposition (5 strands of math proficiency)
Productive disposition - habitual inclination to see mathematics as sensible, useful, and worthwhile, coupled with a belief in diligence and one's own efficacy
3 stages of research on CBM (Fuchs)
Technical features of the static score (aka level/ screening) A single point in time, does it do what we think it does? Technical features of slope Trend/ progress monitoring Instructional Utility Aka usefulness
Key developmental markers of math proficiency
World of Formal symbols, which encompasses: World of real quantities & World of Counting Numbers
If a problem is common for more than 5% of learners ...
analyze at a group
If a problem is common for an Individual learner...
analyze at the individual level
Curriculum based assessment (the broad term)
any set of measurement activities that uses direct observation and recording of a student's performance in the local curriculum as a basis for gathering information to make instructional decisions, anything that is pulled from the curriculum
Subskill mastery measurement
individual steps toward different mastery levels, short term (takes shorter time to accomplish) CBA-ID
Subitizing
instantly recognizing a quantity between 1 and 3
Evaluation
making a judgement based on the information that has been gathered.
Which of the following is an ecological factor that impacts educational outcomes
number of words exposed to before entering school
Measurement
quantifying a skill or trait
Onset and rime
the onset consists of the initial consonant or consonant blend, the rime consists of the vowel and any final consonants.
Sensitivity
the probability of student testing positive if the given difficulty exists. (TP / TP + FN). The proportion of students who didn't do well on criterion measure who did not do well on screener.
Assessment
the process of gathering data to inform decision making.
Array identification
using mental counting to identify quantities between 4 and 7
Grapheme
written symbol that represents a phoneme