OCT 1141H: Standardized Assessment and Outcome Measurement - Sept 27 Lecture

What are three different ways in which we might measure?

- generic assessments - condition-specific assessments - individualized assessments

According to the COPM a change of how many points is considered statistically significant change?

2 points

What is the COPM?

A semi-structure interview and rating component for the most important OPIs a person has - embedded in client-centred practice: consistent with CMOP-E

generic assessment

Can work with any type of client, doesn't matter what diagnosis they have. E.g. assessing QOL

CMOP-E

Canadian Model of Occupational Performance and Engagement - shows the dynamic relationship between the person, their occupations and their environment

True or false: COPM is an outcome measure for all clients of all ages and all disability groups

True, expect not on children younger than 8 years of age

Where is the domain of OT according to the CMOP-E?

Where the person, the occupation and the env all come together

true or false: all outcome measures are standardized

false

true or false: OT's have generally made measurement a priority

false. 5-10% use outcome measurements.

True or false: the client completes their own rating scales in the OPHI-II

false. The OT/interviewer does

habituation

life patterns

How does the COPM score performance?

on a scale of 1 to 10 where 1= not able to do it at all, 10 = able to do it extremely well

How does the COPM score satisfaction?

on a scale of 1 to 10 where 1=not satisfied at all, 10=extremely satisfied

A common semi-structured interview tool used by OTs is the: a) Canadian Model of Occupational Performance b) Canadian Occupational Performance Measure c) Competent Occupational Measure of Performance d) None of the above

option b)

The third and fourth steps of administering the COPM is: a) rating the importance - by client b) rating and socring performance and satisfaction - by client c) identification of occupational performance issues d) reassessment

option b)

The COPM is based on what model?

the Canadian Model of Occupational Performance (CMOP)

Availability

the affordability, is it in a different country, different language, may have to agree to send them your data

According to MOHO, volition, habituation and performance capacity are encompassed in what?

the environment and leads a person to be occupationally competent, have an occupation identity > if that is working well you have occupation adaptation (occupational performanc > the person can do all the things that they want to)

Moberg pick up test

timed test involving picking up, holding, manipulating, and identifying small objects - used with children and cognitively impaired adults - tests stereognosis, ability to discriminate different objects through touch, not through visiual information - timed, quick and inexpensive - 10 standad objects with the same temperature - do it with eyes open and then with eyes occluded, repeat sequence x3

True or false: TH COPM can be used to describe, evaluate, and predict occupational performance

true

True or false: the COPM enables clients to participate in a meaningful way in the OT process

true

True or false: the Canadian Occupational Performance Measure (COPM) is a semi-structured interview tool in OT assessment

true

true or false: The Occupational performance History Interview-Version II (OPHI-II) is a Semi-structured interview tool in OT assessment

true

true or false: there are 1000's of measures of interest to rehabilitation

true

For an evaluative outcome measure, what is construction concerned with?

whether there are responsive items

The OPHI-II is structured by what 5 concepts?

- activity/occupational choices - critical life events - daily routines - occupational roles - occupational behaviour settings (environment) NOTE: in comparison to the COPM which was structured by self-care, productivity and leisure

Elements of the person in the CMOP-E

- affective, cognitive, physical, spirituality - person exists within the blue circle which is a series of occupations

Descriptive measure

- an assessment instrument meant to describe the status (e.g. current occupational strenghts or limitations, characteristics, behaviours) or a person at one moment in time - broad range of items - information collected can identify problems and to evaluate the need/plan for intervention

Outcome measurement

- any measurement SYSTEM used to uncover or identify the health outcome of treatment (Geigle, 1990) - a PROCESS by which changes on an outcome measure, over two or more points in time are credited to some event (Mayo, 1994) > what you are going to do with your outcome measure - need to consider how the outcome measure can be integrated into care - what you are going to do with your outcome measure

Characteristics of Standardized Assessments

- assessment manual - instructions - equipment, question - data on test construction, reliability, validity - normative data (if applicable) - can vary depending on type of measure

What factors influence clinical utility?

- clinical applicability - specificity - availability - time/training demands (qualifications) - acceptability to clients - cost

Examining Fit with Values of OT and theoretical Assumptions

- consider what the assessment tool measure in terms of occupational performance and the theoretical assumptions of the theory (theories) guiding your OT practice - many measures are derived from a theoretical perspective - and we will see those principles embedded in the measure - e.g. social support measures based on theories that support the client's perspective may ask to rank how supported they feel, where other measures based on theories that quantify how many support persons they have will ask how many/frequency type questions

Items under Occupation Identity scale

- has personal goals and projects - identifies a desired occupational lifestyle - expects success - accepts responsibility - appraises abilities and limitations - has commitments and values - recognizes identity and obligations - has interests - felt effective - found meanign and satisfaction in lifestyle (past) - made occupational choices (past)

Why do we measure?

- helps determine status at the start of intervention (baseline) - helps determine if someone is actually improving during intervention, and at the end of intervention - improves clinical decision-making, care and client outcome (makes you a better OT!)

items under behaviour settings scale

- home-life occupational forms - major productive role occupational forms - leisure occupational forms - home-life social group - major productive social group - leisure social group - home-life physical spaces, objects and resources - major productive role physical spaces, objects and resources - lesiure physical spaces, objects and resources

Where is there area for variability in the Moberg pick up test?

- how the vision is occluded - size and position of the box - what you start and stop the stop watch - height of the table - size of the objects - no script

How does the COPM enable participates to participate in a meaningful way in the OT process

- identifying occupational performance problems - evaluating performance and satisfaction with their performance in those problem areas - measuring change in client's perception of occupational performance

Why semi-structure the interview?

- keeps focus and helps ensure key issues are not missed - relates assessment more explicitly to theoretical frameworks or models - shared reporting format, aid with odcumentation - increased validity and realiability of responses - might be easier to follow up and see if there has been change - skills at both structure and un-structured are important

items under occupational competence scale

- maintains satisfying lifestyle - fulfills role expectations - works toward goals - meets personal performance standards - organize time for responsibilities - participates in interests - fulfilled roles (past) - maintained habits (past) -achieved satisfaction (past)

MOHO

- occupational therapy based occupation oriented model of practice - asserts that what a person does in work, play, and self-care is a function of motivational factors (volition), life patterns (habituation, performance capacity (what they can do) and environmental influences - volition, habituation and performance capacity lead to whether they can participate, whether they can perform, and the sklls that they have

How does the COPM define productivity?

- paid/unpaid work e.g. finding/keeping a job, volunteering - household management e.g. cleaning, laundry, cooking - play/school e.g. play skills, homework

Purpose of standardiztion

- permits comparison of scores from one time to the next (individual or group scores) and between a client (or group) and a sample of individuals with similar characteristics

How does the COPM define self-care

- personal care e.g dressing, bathing, feeding, hygiene - functional mobility e.g. transfers, indoor, outdoor - community management e.g. transportation, shopping, finances

How does the COPM define leisure?

- quiet recreation e.g. hobbies, crafts, reading - active recreation e.g. sports, outings, travel - socialization e.g. visiting, phone calls, parties, correspondence

Step 2 of the COPM process

- rating importance - using the scoring cards provided, ask the client to rate, on a scale of 1 to 10, the importance of each activity - place the ratings in the corresponding boxes

Step 5 of the COPM process

- reassessment - at this point you have the issues, you have the ratings, you carry on with your OT treatment - re-evaluate each problem at a suitable interval in terms of performance and satisfaction. Calculate the new weighted scores and see if they have changed over time - possibly identify new occupational performance problems and follow through the 5 steps again

Step 3 and 4 of the COPM

- score performance and satisfaction - have the client rate for each of those important ones (based on the importance rating) and have the client score again on a scale of 1-10 how well they think they perform that particular issues RIGHT NOW and how satisfied they are with that performance - using the scoring cards, have the client rate each on performance and satisfaction on a scale of 1-10 - calculate the weighted scores - there are two ratings in total: one for performance, one for satisfaction

What does the OPHI-II format consist of?

- semi-structured interview - scales for rating the information obtained in the interview - a format for recording qualitative narrative data

Purpose of the OPHI-II

- semi-structured interview that explores a client's life history (vague, broad) in the areas of work, play and self-care performance

What are the 5 step processes of the COPM?

1) A semi-structured interview to identify the OP issues that that person has the most - problem identification 2) Rating importance of those issues (by client) 3 & 4) scoring performance and satisfaction (by client) 5) repeating 3 &4 at reassessment and as necessary

what are the 2 criteria for outcome measures?

1) clinically useful 2) standardized

What three evaluation criteria must be met under each type of measure (descriptive, predictive, evaluative)?

1) construction 2) reliability 3) validity

What are three types of measures?

1) descriptive 2) predictive 3) evaluative

What are the 3 scales for rating interview information in the OPHI-II?

1) occupational identity (11 items): what person believes, perceives, feels 2) occupational competence (9 items): what the person does 3) occupational behaviour setting (9 items): environment support - you rate each item on a scale of 1 to 4, where 4= exceptionally competence occupation functioning,, 3= appropriate, satisfactory occupation function, 2 = some occupational dysfunction, 1= extremely occuaptionally dysfunctional - each level of rating has set criteria

How long does the COPM take to complete?

30-40 minutes

outcome measure

An instrument that has been shown to measure desirable traits, accurately. There are varied forms, standardized or non-standardized. e.g. a ruler is an outcome measure of the length of something, a tally sheet shows how many times something happens, rating scales

Standardized test

Designed so that the test questions, method and conditions for the administrating, scoring and interpretation of the results are consistent (or fixed)

How does the COPM demonstrate client-centred practice?

Helps to engage clients right from the beginning of OT experience and increases client control of their own therapeutic process - it leads to validation of clients issues and establishes a baseline of occupational performance against which to measure outcomes

Instrument evaluation (Law, 1987)

Helps you pick out an outcome measure 1) Clinically useful? No - select another instrument 2) Standardized? No - select another instrument 3) what is the instrument's purpose? Descriptive, predictive, evaluative?

Clinical utility

Hos useful is it for your setting? Is it clinically useful i.e. ease and usefulness in a specific setting

Step 1 of the COPM Process

Identify occupational performance issues, interview the client using the questions as guidelines (self-care, productivity, and leisure)

Why is it often that in time two their ratings of satisfaction will have increased despite the fact that their performance ratings have not increased?

Might be because in the beginning it was important to the client to perform better on their OPI (e.g. self-care) but later on they realized it wasn't that important. Maybe their spouse will help them. Not changing in their performance but it no longer becomes an issue for them so their satisfaction goes higher.

How far can I stray from the manual?

Not at all. The more you stray from the manual, the less you can believe your results

condition-specific assessments

Specific to a certain population, e.g. examining QOL symptoms, what someone can and can't do

individualized assessments

The elements you assess are chosen by the client e.g. COPM. It's not a list of things you are assessing, it's what they are deciding what the issues are

outcome

The end result of clinical activity (Law, 1996). What you're hoping to achieve. e.g. less pain, more independence, better QOL. What you want to see change.

What might we measure?

Want to measure what we're hoping to effect and what we want to change: - range of motion - pain - swelling - pressure sore healing - cognition - risk of falling - depression - ADL's - participation in the community - successful return to work - occupational balance, - QOL - client satisfaction

Can I do the standardized assessment over the phone?

Yes, if it has been tested to show similar reliability and validity over the phone to in person. If not, have to be careful of how you interpret the results.

Is the Moberg Pick up test standardized?

Yes, it is semi-standardized. It is very reliable and valid > there is room for variability. If you personally are doing it exactly the same way (being consistent), you can see a change in the client

Can I use part of a standardized outcome measure?

You can use parts of an outcome manual if the manual tells you there are sub-components that you can make individual inferences from

True or false: for each category within the OPHI-II, how many pages of interview questions are there?

about 5-6 pages. Lengthy process of talking to someone about their owkr and what they're doing, whether they like it. You don't have to ask every single question

Acceptability to clients

age appropriate, culturally appropriate, language requirements, time

How do you choose between the COPM and the OPHI-II?

areas to consider: - theoretical constructs and models that the measures were based on (CMOP-E vs MOHO) - time (COPM is much shorter than OPHI-II) - degree of client-centredness, who does the rating? (COPM client does the rating, OPHI-II the therapist does the rating) - purpose - descriptive, evaluative, predictive (both measures address all 3, however they might predict different things) - client's abilities and preferences and the context of practice you are in matter a lot - psychometric properties

For a descriptive outcome measure, what is validity concerned with?

content construct

For an evaluative outcome measure, what is validity concerned with?

content construct

For a predictive outcome measure, what is validity concerned with?

content criterion

Which type of measure can often be used to classify an individual via comparison with norms?

descriptive measure

Which type of measure can discriminate between groups?

descriptive measure. E.g. when you measure cognition, you want to determine who has cognitive impairment and who does not

True or false: Jenny is 6 years old, so the COPM would be an appropriate outcome measure for her.

false

True or false: Performance and satisfaction on the COPM is rated on a scale from 1-5

false

For a descriptive outcome measure, what is reliability concerned with?

internal consistency observer

Any deviation from standardized procedures may result in what?

invalid conclusions about an examinee's test performance

volition

motivational factors

Who was the developer of the Model of Human Occupation upon which the OPHI-II is based? a) Baptiste b) Polatajko c) Keilhofner d) Law

option c)

Elements of the environment in the CMOP-E

physical, institutional, cultural, social. All elements of the person and occupation exist within an environment

Elements of the occupations in the CMOP-E

self-care, productivity, leisure

For a predictive outcome measure, what is reliability concerned with?

test-retest observer

For an evaluative outcome measure, what is reliability concerned with?

test-retest observer

According to OPHI-II to understand any person's occupational status in the present, it was considered necessary to understand what?

the historical shaping forces that led to that present status. All the forces that shaped that person over the course of their life.

What model is the OPHI-II based on?

the model of human occupation (MOHO)

clinical applicability

transferable and meaningful is the resuling information to intervention planning and communication. i.e. it helps you communicate the results and helps you in your intervention plan

True or false: the OPHI-IIis a describe, predict, and evaluate occupational performance

true. Describes service provision/ program planning. Predicts using scales that help determine client's potential for rehab success or succeeding in community adaptation after rehab. Evaluative: quality assurance and program evaluation

True or false: the COPM is focused on occupational performance, not the components of occupational performance

true. Focused on activity I might do as appopsed to physical and cognitive abilities.

true or false: measurement is a component of evidence-based practice

true. Objectively measure at least one of the most critical outcomes of what you are trying to achieve with your clients. Hard to measure everything, need to identify key outcomes.

true or false: the COPM is based on self-report

true. The client is giving thier own rating

specificity

what population was it designed for? Will it work for your clients? How close are your clients to the population it was designed for?

performance capacity

what they can do

For a descriptive outcome measure, what is construction concerned with?

whether there are descriptive items

For a predictive outcome measure, what is construction concerned with?

whether there are predictive items

Once you have gathered all the relevant information around the 5 categories, what do you do next?

you rate them according to three scales