OCT 1141H: Standardized Assessment and Outcome Measurement - Sept 27 Lecture
What are three different ways in which we might measure?
- generic assessments - condition-specific assessments - individualized assessments
According to the COPM a change of how many points is considered statistically significant change?
2 points
What is the COPM?
A semi-structure interview and rating component for the most important OPIs a person has - embedded in client-centred practice: consistent with CMOP-E
generic assessment
Can work with any type of client, doesn't matter what diagnosis they have. E.g. assessing QOL
CMOP-E
Canadian Model of Occupational Performance and Engagement - shows the dynamic relationship between the person, their occupations and their environment
True or false: COPM is an outcome measure for all clients of all ages and all disability groups
True, expect not on children younger than 8 years of age
Where is the domain of OT according to the CMOP-E?
Where the person, the occupation and the env all come together
true or false: all outcome measures are standardized
false
true or false: OT's have generally made measurement a priority
false. 5-10% use outcome measurements.
True or false: the client completes their own rating scales in the OPHI-II
false. The OT/interviewer does
habituation
life patterns
How does the COPM score performance?
on a scale of 1 to 10 where 1= not able to do it at all, 10 = able to do it extremely well
How does the COPM score satisfaction?
on a scale of 1 to 10 where 1=not satisfied at all, 10=extremely satisfied
A common semi-structured interview tool used by OTs is the: a) Canadian Model of Occupational Performance b) Canadian Occupational Performance Measure c) Competent Occupational Measure of Performance d) None of the above
option b)
The third and fourth steps of administering the COPM is: a) rating the importance - by client b) rating and socring performance and satisfaction - by client c) identification of occupational performance issues d) reassessment
option b)
The COPM is based on what model?
the Canadian Model of Occupational Performance (CMOP)
Availability
the affordability, is it in a different country, different language, may have to agree to send them your data
According to MOHO, volition, habituation and performance capacity are encompassed in what?
the environment and leads a person to be occupationally competent, have an occupation identity > if that is working well you have occupation adaptation (occupational performanc > the person can do all the things that they want to)
Moberg pick up test
timed test involving picking up, holding, manipulating, and identifying small objects - used with children and cognitively impaired adults - tests stereognosis, ability to discriminate different objects through touch, not through visiual information - timed, quick and inexpensive - 10 standad objects with the same temperature - do it with eyes open and then with eyes occluded, repeat sequence x3
True or false: TH COPM can be used to describe, evaluate, and predict occupational performance
true
True or false: the COPM enables clients to participate in a meaningful way in the OT process
true
True or false: the Canadian Occupational Performance Measure (COPM) is a semi-structured interview tool in OT assessment
true
true or false: The Occupational performance History Interview-Version II (OPHI-II) is a Semi-structured interview tool in OT assessment
true
true or false: there are 1000's of measures of interest to rehabilitation
true
For an evaluative outcome measure, what is construction concerned with?
whether there are responsive items
The OPHI-II is structured by what 5 concepts?
- activity/occupational choices - critical life events - daily routines - occupational roles - occupational behaviour settings (environment) NOTE: in comparison to the COPM which was structured by self-care, productivity and leisure
Elements of the person in the CMOP-E
- affective, cognitive, physical, spirituality - person exists within the blue circle which is a series of occupations
Descriptive measure
- an assessment instrument meant to describe the status (e.g. current occupational strenghts or limitations, characteristics, behaviours) or a person at one moment in time - broad range of items - information collected can identify problems and to evaluate the need/plan for intervention
Outcome measurement
- any measurement SYSTEM used to uncover or identify the health outcome of treatment (Geigle, 1990) - a PROCESS by which changes on an outcome measure, over two or more points in time are credited to some event (Mayo, 1994) > what you are going to do with your outcome measure - need to consider how the outcome measure can be integrated into care - what you are going to do with your outcome measure
Characteristics of Standardized Assessments
- assessment manual - instructions - equipment, question - data on test construction, reliability, validity - normative data (if applicable) - can vary depending on type of measure
What factors influence clinical utility?
- clinical applicability - specificity - availability - time/training demands (qualifications) - acceptability to clients - cost
Examining Fit with Values of OT and theoretical Assumptions
- consider what the assessment tool measure in terms of occupational performance and the theoretical assumptions of the theory (theories) guiding your OT practice - many measures are derived from a theoretical perspective - and we will see those principles embedded in the measure - e.g. social support measures based on theories that support the client's perspective may ask to rank how supported they feel, where other measures based on theories that quantify how many support persons they have will ask how many/frequency type questions
Items under Occupation Identity scale
- has personal goals and projects - identifies a desired occupational lifestyle - expects success - accepts responsibility - appraises abilities and limitations - has commitments and values - recognizes identity and obligations - has interests - felt effective - found meanign and satisfaction in lifestyle (past) - made occupational choices (past)
Why do we measure?
- helps determine status at the start of intervention (baseline) - helps determine if someone is actually improving during intervention, and at the end of intervention - improves clinical decision-making, care and client outcome (makes you a better OT!)
items under behaviour settings scale
- home-life occupational forms - major productive role occupational forms - leisure occupational forms - home-life social group - major productive social group - leisure social group - home-life physical spaces, objects and resources - major productive role physical spaces, objects and resources - lesiure physical spaces, objects and resources
Where is there area for variability in the Moberg pick up test?
- how the vision is occluded - size and position of the box - what you start and stop the stop watch - height of the table - size of the objects - no script
How does the COPM enable participates to participate in a meaningful way in the OT process
- identifying occupational performance problems - evaluating performance and satisfaction with their performance in those problem areas - measuring change in client's perception of occupational performance
Why semi-structure the interview?
- keeps focus and helps ensure key issues are not missed - relates assessment more explicitly to theoretical frameworks or models - shared reporting format, aid with odcumentation - increased validity and realiability of responses - might be easier to follow up and see if there has been change - skills at both structure and un-structured are important
items under occupational competence scale
- maintains satisfying lifestyle - fulfills role expectations - works toward goals - meets personal performance standards - organize time for responsibilities - participates in interests - fulfilled roles (past) - maintained habits (past) -achieved satisfaction (past)
MOHO
- occupational therapy based occupation oriented model of practice - asserts that what a person does in work, play, and self-care is a function of motivational factors (volition), life patterns (habituation, performance capacity (what they can do) and environmental influences - volition, habituation and performance capacity lead to whether they can participate, whether they can perform, and the sklls that they have
How does the COPM define productivity?
- paid/unpaid work e.g. finding/keeping a job, volunteering - household management e.g. cleaning, laundry, cooking - play/school e.g. play skills, homework
Purpose of standardiztion
- permits comparison of scores from one time to the next (individual or group scores) and between a client (or group) and a sample of individuals with similar characteristics
How does the COPM define self-care
- personal care e.g dressing, bathing, feeding, hygiene - functional mobility e.g. transfers, indoor, outdoor - community management e.g. transportation, shopping, finances
How does the COPM define leisure?
- quiet recreation e.g. hobbies, crafts, reading - active recreation e.g. sports, outings, travel - socialization e.g. visiting, phone calls, parties, correspondence
Step 2 of the COPM process
- rating importance - using the scoring cards provided, ask the client to rate, on a scale of 1 to 10, the importance of each activity - place the ratings in the corresponding boxes
Step 5 of the COPM process
- reassessment - at this point you have the issues, you have the ratings, you carry on with your OT treatment - re-evaluate each problem at a suitable interval in terms of performance and satisfaction. Calculate the new weighted scores and see if they have changed over time - possibly identify new occupational performance problems and follow through the 5 steps again
Step 3 and 4 of the COPM
- score performance and satisfaction - have the client rate for each of those important ones (based on the importance rating) and have the client score again on a scale of 1-10 how well they think they perform that particular issues RIGHT NOW and how satisfied they are with that performance - using the scoring cards, have the client rate each on performance and satisfaction on a scale of 1-10 - calculate the weighted scores - there are two ratings in total: one for performance, one for satisfaction
What does the OPHI-II format consist of?
- semi-structured interview - scales for rating the information obtained in the interview - a format for recording qualitative narrative data
Purpose of the OPHI-II
- semi-structured interview that explores a client's life history (vague, broad) in the areas of work, play and self-care performance
What are the 5 step processes of the COPM?
1) A semi-structured interview to identify the OP issues that that person has the most - problem identification 2) Rating importance of those issues (by client) 3 & 4) scoring performance and satisfaction (by client) 5) repeating 3 &4 at reassessment and as necessary
what are the 2 criteria for outcome measures?
1) clinically useful 2) standardized
What three evaluation criteria must be met under each type of measure (descriptive, predictive, evaluative)?
1) construction 2) reliability 3) validity
What are three types of measures?
1) descriptive 2) predictive 3) evaluative
What are the 3 scales for rating interview information in the OPHI-II?
1) occupational identity (11 items): what person believes, perceives, feels 2) occupational competence (9 items): what the person does 3) occupational behaviour setting (9 items): environment support - you rate each item on a scale of 1 to 4, where 4= exceptionally competence occupation functioning,, 3= appropriate, satisfactory occupation function, 2 = some occupational dysfunction, 1= extremely occuaptionally dysfunctional - each level of rating has set criteria
How long does the COPM take to complete?
30-40 minutes
outcome measure
An instrument that has been shown to measure desirable traits, accurately. There are varied forms, standardized or non-standardized. e.g. a ruler is an outcome measure of the length of something, a tally sheet shows how many times something happens, rating scales
Standardized test
Designed so that the test questions, method and conditions for the administrating, scoring and interpretation of the results are consistent (or fixed)
How does the COPM demonstrate client-centred practice?
Helps to engage clients right from the beginning of OT experience and increases client control of their own therapeutic process - it leads to validation of clients issues and establishes a baseline of occupational performance against which to measure outcomes
Instrument evaluation (Law, 1987)
Helps you pick out an outcome measure 1) Clinically useful? No - select another instrument 2) Standardized? No - select another instrument 3) what is the instrument's purpose? Descriptive, predictive, evaluative?
Clinical utility
Hos useful is it for your setting? Is it clinically useful i.e. ease and usefulness in a specific setting
Step 1 of the COPM Process
Identify occupational performance issues, interview the client using the questions as guidelines (self-care, productivity, and leisure)
Why is it often that in time two their ratings of satisfaction will have increased despite the fact that their performance ratings have not increased?
Might be because in the beginning it was important to the client to perform better on their OPI (e.g. self-care) but later on they realized it wasn't that important. Maybe their spouse will help them. Not changing in their performance but it no longer becomes an issue for them so their satisfaction goes higher.
How far can I stray from the manual?
Not at all. The more you stray from the manual, the less you can believe your results
condition-specific assessments
Specific to a certain population, e.g. examining QOL symptoms, what someone can and can't do
individualized assessments
The elements you assess are chosen by the client e.g. COPM. It's not a list of things you are assessing, it's what they are deciding what the issues are
outcome
The end result of clinical activity (Law, 1996). What you're hoping to achieve. e.g. less pain, more independence, better QOL. What you want to see change.
What might we measure?
Want to measure what we're hoping to effect and what we want to change: - range of motion - pain - swelling - pressure sore healing - cognition - risk of falling - depression - ADL's - participation in the community - successful return to work - occupational balance, - QOL - client satisfaction
Can I do the standardized assessment over the phone?
Yes, if it has been tested to show similar reliability and validity over the phone to in person. If not, have to be careful of how you interpret the results.
Is the Moberg Pick up test standardized?
Yes, it is semi-standardized. It is very reliable and valid > there is room for variability. If you personally are doing it exactly the same way (being consistent), you can see a change in the client
Can I use part of a standardized outcome measure?
You can use parts of an outcome manual if the manual tells you there are sub-components that you can make individual inferences from
True or false: for each category within the OPHI-II, how many pages of interview questions are there?
about 5-6 pages. Lengthy process of talking to someone about their owkr and what they're doing, whether they like it. You don't have to ask every single question
Acceptability to clients
age appropriate, culturally appropriate, language requirements, time
How do you choose between the COPM and the OPHI-II?
areas to consider: - theoretical constructs and models that the measures were based on (CMOP-E vs MOHO) - time (COPM is much shorter than OPHI-II) - degree of client-centredness, who does the rating? (COPM client does the rating, OPHI-II the therapist does the rating) - purpose - descriptive, evaluative, predictive (both measures address all 3, however they might predict different things) - client's abilities and preferences and the context of practice you are in matter a lot - psychometric properties
For a descriptive outcome measure, what is validity concerned with?
content construct
For an evaluative outcome measure, what is validity concerned with?
content construct
For a predictive outcome measure, what is validity concerned with?
content criterion
Which type of measure can often be used to classify an individual via comparison with norms?
descriptive measure
Which type of measure can discriminate between groups?
descriptive measure. E.g. when you measure cognition, you want to determine who has cognitive impairment and who does not
True or false: Jenny is 6 years old, so the COPM would be an appropriate outcome measure for her.
false
True or false: Performance and satisfaction on the COPM is rated on a scale from 1-5
false
For a descriptive outcome measure, what is reliability concerned with?
internal consistency observer
Any deviation from standardized procedures may result in what?
invalid conclusions about an examinee's test performance
volition
motivational factors
Who was the developer of the Model of Human Occupation upon which the OPHI-II is based? a) Baptiste b) Polatajko c) Keilhofner d) Law
option c)
Elements of the environment in the CMOP-E
physical, institutional, cultural, social. All elements of the person and occupation exist within an environment
Elements of the occupations in the CMOP-E
self-care, productivity, leisure
For a predictive outcome measure, what is reliability concerned with?
test-retest observer
For an evaluative outcome measure, what is reliability concerned with?
test-retest observer
According to OPHI-II to understand any person's occupational status in the present, it was considered necessary to understand what?
the historical shaping forces that led to that present status. All the forces that shaped that person over the course of their life.
What model is the OPHI-II based on?
the model of human occupation (MOHO)
clinical applicability
transferable and meaningful is the resuling information to intervention planning and communication. i.e. it helps you communicate the results and helps you in your intervention plan
True or false: the OPHI-IIis a describe, predict, and evaluate occupational performance
true. Describes service provision/ program planning. Predicts using scales that help determine client's potential for rehab success or succeeding in community adaptation after rehab. Evaluative: quality assurance and program evaluation
True or false: the COPM is focused on occupational performance, not the components of occupational performance
true. Focused on activity I might do as appopsed to physical and cognitive abilities.
true or false: measurement is a component of evidence-based practice
true. Objectively measure at least one of the most critical outcomes of what you are trying to achieve with your clients. Hard to measure everything, need to identify key outcomes.
true or false: the COPM is based on self-report
true. The client is giving thier own rating
specificity
what population was it designed for? Will it work for your clients? How close are your clients to the population it was designed for?
performance capacity
what they can do
For a descriptive outcome measure, what is construction concerned with?
whether there are descriptive items
For a predictive outcome measure, what is construction concerned with?
whether there are predictive items
Once you have gathered all the relevant information around the 5 categories, what do you do next?
you rate them according to three scales