COMD 4450 Alternative/Integrative Assessment Lecture

What is a Portfolio Assessment?

-- a systematic, purposeful, and meaningful collection of students work in one or more subject areas --examples of students work are included --assessment should be based on specific educational needs and look at the "whole" child --requires a multidisciplinary team--teacher, aide, parents, psychologist, SLP , others

Portfolio Assessment: Questions to answer when looking at Portfolio Assessment

-- can help to identify whether current treatment is successful and to plan the next step --what is the communication problem? --why is the learner experiencing this problem? --what can be done to remediate the problem? (as a team)

Client specific assessments:

-- developed for a given client --takes into account what needs to be assessed and how --Usually uses 15-20 exemplars if doing client specific assessment in a given area

What is Graduated Prompting?

-- using predetermined heirarchy of prompts designed to facilitate the child's responses during the assessment --clinician analyzes the number and level of prompts needed to evoke a target response and to facilitate generalization of learning to a new task -supportive cues for children with SLI --increase from least supportive to most supportive: 1) general statement (least supportive) 2)questions designed to elicit specific response 3) sentence completion tasks 4)indirect models 5)direct models (most supportive)

When do we use Alternative and Integrated Assessment Approaches?

--When standard assessment isn't enough --when we need to look at a child individually --when we don't need to compare them to norms **child may not be appropriate for the norms that were established by the standardized test

What are Authentic Assessments? "When students are expected to perform, produce, or otherwise demonstrate skills that represent realistic learning demands -- the contexts of the assessment are real life settings in and out of the classroom without contrived and standardized conditions"

--allow you to complete assessments in child's classroom ( or home) as opposed to pulling him out and testing him in therapy room or other location --rely heavily on speech samples taken in familiar environments (classroom/home) -language sample analysis may be subjective and is very time consuming

Dynamic Assessments: Testing the Limits or Task Variability:

--clinician modifies standardized testing procedures by providing elaborate feed back to client on his performance on test items. (may include notifying of correct response, explaining why response was correct or incorrect, explaining principles of a task) --task variability can include modifications in the way a test is administered: such as using a more naturalistic environment or letting child demonstrate a skill -- these strategies can help reduce child's "test anxiety" --can be more effective than standardized tests in estimating cognitive and academic ability of children from diverse ethnic groups **children who do not respond to modifications may indicate greater need for intervention ++++typically standardized test is not modified to suit needs of client

Authentic Assessments: Contrastive Analysis

--goal is to separate dialectic speech and language differences from clinically significant speech and language errors --clinician becomes familiar with dialect spoken by client --speech/language sample is collected --sample is evaluated to identify any differences from MAE --based on knowledge of clients dialect , clinician determines if differences are dialectal variations, or "true errors" **dialectal differences DO NOT indicate disorder

Authentic Assessments: Minimal Competency Core

--the least amount of linguistic skill or knowledge that a typical speaker would display for a given age and context -- a checklist of things to look for in a language sample (morphology, syntax, semantics, etc.) --helps to identify children for a language disorder

Criterion-Referenced and Client Specific Assessment: Criterion-Referenced Assessment:

--uses existing non-standard assessment tools -- interpret data in terms of whether the measured skills meet certain mastery criterion. -- May also use standardized items w/cautions **only compare results to child himself --very client specific

Disadvantages of processing-dependent measures

-are they true measures of social communication skill? (if they can repeat numbers back to you, how does that measure social skills?) language is intertwined w/ knowledge and experience -measures used may not directly reflect language skills tough to develop treatment plan (have no real language data) -eliminating language measures to avoid bias in assess. language is paradoxic (if we aren't assessing language, how do we have language data?) -if a child obtains a low language score, in spite of normal processing-dependent scores, what do we do?

What are Advantages of Dynamic Assessments?

-assists the clinician in assessing the child's learning process and to determine type and intensity of treatment -indicates the child's modifiability and ability to carryover new learning -may not see as many children due to modifiability (lowers caseload) -use standardized tests in flexible manner (can use these for actual instruction) --works well for children from varied Ethnical and SES backgrounds

What are advantages of Criterion referenced and Client specific Assessment?

-avoid pitfalls of standardized tests -behaviors to be assessed are sampled more adequately (15-20 exemplars) -results are uniquely relevant to child -results directly lead to treatment planning **(plan treatment based on results) -provide more reliable and valid data than standardized test

BOTH criterion referenced and client specific assessment instructions

-clinician develops stimulus materials that are relevant for the child **(customize assessment to child) - clinician uses case history info to understand the cultural background, bilingual status, education, occupation, SES of the client and family **use knowledge of child to create criterion referenced assessment -stimulus items preferably selected from child's home environment --familiar items to child, also culturally appropriate, since they are selected from child's environment -results from these assessments are interpreted to be informative of the child's current skill level **does not allow you to compare the child to normative data

Advantages of portfolio assessments:

-documentation of improvement in skills over time (can compare writing samples taken every year etc) -facilitation of interdisciplinary collaboration -assessment of child's production of targeted skills in various contexts -evaluation of communication in holistic and educational relevant manner -assistance in developing treatment goals and strategies **can cover more than just your area, using help from other specialists and teachers

What are disadvantages of Criterion Referenced and Client Specific Assessments?

-don't allow a comparison of individual child's performance w/ performance of a normative sample (standardized tests do) -don't allow for diagnosis of a disorder made strictly on the basis of normative comparison of an individual child's performance (sometimes school districts require you to perform standardized tests, so you can compare child to normative sample) -initially more time consuming due to material preparation ** (once materials are prepared you can use them over and over for the same population) +++you can complete these 2 assessments in addition to standardized testing, gives a more complete picture of the child

Supportive cues for a child w/ SLI

-general statement (least supportive) -questions designed to elicit a specific response -sentence completion tasks -indirect models -direct models (most supportive)

Dynamic Assessments How to teach/stimulate the selected skills?

-graduated prompting -testing the limits or task variability -test-teach-retest

Processing-Dependent measures

-helps identify valid measures that are not affected by subjects' prior knowledge or experience -tasks included: memory(ex: give child numbers like 3,6,9 have them repeat numbers back to you, etc) competing stimuli tasks, perceptual tasks (discriminating of tones, and sequencing tones) -research suggests that this is successful in differentiating children w/ language disorders from those w/ normal language while reducing bias inherent in knowledge-dependent measures **area needs more research

Items included in Portfolio Assessment:

-initial referral form -case history -language samples -story retell samples -referential communication samples -observation notes -work samples -teacher, parents, peer interviews -audio/videotapes -writing samples -peer evaluations -testing data -case conference notes

Disadvantages of portfolio assessments:

-lack of quantitative info -time consuming -difficult in managing and coordination the contents of each portfolio (can become unruly) -problems related to analysis of the contents and communicating results to team members -limited storage facilities available to clinician (easier now with digital storage) --can be hard to keep all team members on the same page

What is Dynamic Assessment?

-main objective is to understand how the child would perform on treatment tasks **sometimes described as a test of "modifiability" --clinicians go beyond traditional task of assessing client's existing skills --provide brief periods of intervention to see if a child can improve on certain tasks --because of brief treatment included in dynamic assessments, dynamic assessment better leads to treatment targets and strategies than traditional assessment (treatment plans are based on dynamic assessment results) -used to differentiate between cultural or dialectical language difference and language disorders **takes a look at "how" a child learns and "what works" or doesn't work for the individual child vs, just comparing to child to standardized norms -helps assess modifiability (what a child can do with the max. amount of help) of children's language differences ** may help in differentiating those children who will require therapeutic intervention from those who will not

Advantages of processing-dependent measures

-method of screening for language disorder that is not culturally/linguistically biased -quick and easy to administer -validly help distinguish minority children whose linguistic deficit is real and not related to their limited experience or knowledge --good to use if you don't speak the language of the children you are working with

Disadvantages of Authentic assessments:

-takes more time -results may be more subjective -may not meet workplace requirements

Advantages of Authentic Assessments:

-valid diagnosis w/ fewer false positives -samples collected in familiar, natural context -no need for standardized tests -used for children of different cultural backgrounds

What are Disadvantages of dynamic assessments?

-variability within approach creates reliability issues (providing scripts to follow, will help improve reliability) -extraordinary time due to assessment that requires treatment which may not be authorized ** however RTI is starting to become more common place in schools (this is similar to RTI) -difficult to interpret standardized test results if you are helping client through test -children may be denied needed services **definite need for more research in this area

Dynamic Assessments: Test-Teach-Retest

1. administer pretest to identify deficient skills 2. provide intervention designed to modify the client's level of functioning in a given skill 3. administer a post-test to assess modifiability of skill level as a function of intervention (LI kids typically didn't improve with this method)