Ch. 7 Assessment

How to prepare someone for assessment: (2)

1) RC should communicate to consumers the importance of their input throughout the assessment process 2) once a mutual decision is made for the assessment process, the consumer must be carefully prepared for what to anticipate. The professional needs to explain the purpose of the assessment and the ways the results can assist the consumer with achieving realistic rehabilitation goals.

How to promote consumer empowerment in the assessment process: (5)

1) establish a working relationship 2) understand the needs, expectations, values, and goals of the consumer 3) encourage active consumer involvement 4) allow the consumer to make informed choices and express self-determination when selecting evaluation approaches 5) finally, the professional should monitor the consumer to ascertain that she/he is still actively participating in the planned, identified tasks of assessment.

Name the issues related to test selection and development:

1) this process should be carefully planned and tailored to the needs of the consumer, so as to avoid overuse or indiscriminant use of testing 2) voc. assessment may be used as a descriptive appraisal of current levels of a consumer's functioning and as an indicator of potential. 3) make sure the test is appropriate in regards to reliability, validity, and normative populations 4) RCs should be aware of assessment measures available to choose from and the proper use of testing, and each test used in an assessment should be defined according to what it measures and its use, and the process of test development should be clearly outlined. 5) when selecting a test battery, adequate interpretive strategies should be available.

Test-Retest Reliability

A measure of consistency over time. Just as it sounds a test is administered, and then a week or two later it is readministered to the same people. The scores of both test administrations are correlated and then a correlation coefficient is obtained.

Split-Half Reliability

A measure of internal consistency. This indicates consistency of scores obtained by individuals within the same group on two different parts of the same test. Ex. A person takes the whole test, but the odd and even items represent each half of the test. Once the person is done, the two halves are correlated for consistency.

Parallel (Alternate) Form Reliability

A parallel form correlation indicates the consistency of scores of individuals within the same group on two alternate but equivalent forms of the same test taken at the same time. These parallel forms are administered to the same subjects within a 1-2 week interval.

Cronbach's Alpha

An internal consistency statistic calculated from the pairwise correlations between items in a group.

Test Interpretation

Be sure to communicate results at the consumer's level of understanding. Be sure to take into account how the test-taking process may influence the consumer's disability, and vice versa.

Nominal Measurement Scale

Classifies, assigns numerals but does not distinguish size, amount. Ex. any categorical variable, such as ethnicity or gender.

Interval Measurement Scale

Equal intervals on the scale Ex. Celsius temperature scale

What are the 4 ways to estimate a test's validity?

Face Content Criterion or Predictive Construct

Interest Inventories

Geist Picture Interest Inventory-R (R-FVII-2) Reading-Free vocational Interest Inventory-2 Strong Interest Inventory Kuder Occupational Interest Survey Self-Directed Search (SDS) Career Assessment Inventory (CAI) Occupational Aptitude Survey and Interest Schedule-3 (OASIS-3) Ohio Vocational Interset Survey-2 (OVIS-II) Career Decision-Making System-R (CDM-R) Campbell Interest and Skill Survey (CISS)

Aptitude Tests

General Aptitude Test Battery (GATB) Differential Aptitude Tests-5 (DAT) Armed Services Vocational Aptitude Battery (ASVAB) Bennett Mechanical Comprehension Test (BMCT) Minnesota Paper Form Board-R (MPFB-R) Career Ability Placement Survey (CAPS) Hand-Tool Dexterity Test Ability Explorer (AE) APTICOM Ball Aptitude Batter (BAB) Career Scope Career Planning Survey Occupational Aptitude Survey & Interest Schedule-3 (OASIS 3) PESCO 2001 Detroit Tests of Learning Aptitude Occupational Aptitude Survey Minnesota Clerical Test

Ordinal Measurement Scale

Indication of ordering, but no indication of distances between objects on the scale Ex. placing first, second, or third

Work Samples

McCarron-Dial Evaluation System (MDS) VALPAR Microcomputer Evaluation of Careers and Academics (MECA) Vocational Information and Evaluation Work Samples (VIEWS) Vocational Interst, Temperament, & Aptitude System (VITAS) Wide Range Employment Sample Test

Personality Tests

Minnesota Multiphasic Personality Inventory-2 (MMPI-2) Myers-Briggs Type Indicator (MBTI) Sixteen Personality Factor Questionnaire (16PF) California Psychological Inventory (CPI) Psychological Screening Inventory (PSI) Tennessee Self-Concept Scale (TSCS) Adult Personality Inventory (API) Jackson Personality Inventory-R Hogan Personality Inventory NEO Personality Inventory Rorschach Thematic Apperceptions Test

Measurement Scales (4)

Nominal , Ordinal, Interval, and Ratio

Ratio Measurement Scale

Possesses a non-arbitrary zero point Ex. measures weight

Intelligence Tests

Weschsler Adult Intelligence Scale-IV (WAIS-IV) Stanford-Binet-V Slosson Intelligence Test-R Peabody Picture Vocabulary Test-3rd Ed. Beta III Raven's Progressive Matrices Quick-Test Kaufman Brief Intelligence Test (KBIT) Luria-Nebraska Neuropsychological Battery Halstaed-Reitan Neuropsychological Test Battery Test of Nonverbal Intelligence (TONI) Culture Fair Intelligence Test Wonderlic Personnel Test Haptic Intelligence Test for the Blind Woodcock-Johnson Test of Cognitive Abilities

Test biases

When a test may show favor to a particular group and therefore disregard another group/person by containing items that favor one group over another, by using test results if the critieria used for selection and prediction varies greatly among different groups, and by not considering such test-related factors as motivation, anxiety, and the test sophistication of those taking the assessment tool.

Achievement Tests

Wide Range Achievement Test (WRAT-4) Adult Basic Learning Examination-2 (ABLE-2) Peabody Individual Achievement Test (PIAT-R) California's Achievement Test SRA Arithmetic Index and Reading Index Woodcock-Johnson Test of Achievement Wechsler Individual Achievement Test (WIAT)

What is Reliability?

a measure of consistency. This refers to a tests dependability, consistency, and precision of an assessment procedure. A reliable procedure is one that produces similar results when repeated. (Is the test consistent, dependable, and precise?)

Situational Assessments

a work assessment approach where a consumer's job performance and work behaviors are systemically observed in a realistic and controlled working environment, such as in a rehabilitation facility or sheltered workshop. Community-based assessment, job tryouts, on-the-job evaluation, and supported employment evaluation are often used interchangeably. These assessments can help employability behavior. It focuses on the consumers work potential in regards to factors such as ability to accept supervision, get along with coworkers, stay on task, sustain productivity for 8 hours, and tolerate frustration.

What are the 4 ways to estimate a test's reliability?

a) Test-retest b) split-half c) parallel forms d) cronbach's alpha

The Medical Evaluation provides information on...

a) clarifying the consumer's general health at present b) describing of the extent, stability, and prognosis of the present disability as well as any recommended treatment c) assessing present and future implications of the disability and its potential effects on performance of essential job functions d) reporting the presence of any residual medical conditions that could impact the individual during the rehabilitation process.

Medical Evaluation is used to...

a) establish the presence and extent of the disability b)provide info on the physical functioning of the consumer c) determine the types of activities precluded by the disability d) identify any additional medical evaluation necessary for achieving the first 3 purposes.

Test-Taker Rights in Assessment

a) know in advance of testing why they are being tested, how the testing will benefit them, how the test data will be used, and how much time and money the testing will cost them b) participate in planning and scheduling the testing c) be free of unnecessary and outdated tests d) be tested in an atmosphere that is free from distractions and conducive to positive test performance e) be assessed via the most appropriate instruments and techniques available f) have a complete, comprehensive, clear, and honest explanation, analysis, and application of test results g) discuss their test results with people competent to interpret their test protocols, related test data to other available data, and answer any questions h) have their confidentiality protected i) further counseling or assessment, if indicated.

List the categories of standardized tests

achievement, aptitude, intelligence, interest, and personality

Face Validity

an appraisal of what a test seems to measure. It is concerned with the extent to which the instrument measures, or 'looks like' it measures, what it is intended to measure. If the test looks valid to a non-expert. Ex. A test that measures anger, should have items that seem to measure anger.

Work Sample Assessment

an assessment approach whereby the consumer is observed performing a simulated or actual work activity, usually in a rehabilitation center or vocational evaluation unit. These are designed to see whether the individual follows the procedures and uses the tools and materials involved in actual jobs appropriately.

Test

an objective and standardized measure of a sample of behavior.

Assessment

any systematic method of obtaining information from tests and other sources used to draw inferences about people, objects, or programs.

Disability Considerations in Assessment

because standardized test instruments may not necessarily be the best way to assess factors such as motivation, work tolerance, or interest, observations of the consumer in various settings or trial work experiences may prove better methods for understanding rehabilitation potential. It is important that the RC be aware of the potential for disability effects to impact the voc. assessment process. If test accommodations need to be made be sure to document them. Observational information is of great value, you can learn just as much about someone by observing an individual in one day as you can in administering assessments.

Standard scores

can be sometimes yielded as results and can be used to make norm-referenced interpretations. This score is a transformed score with a specified mean and standard deviation in relation to the comparison group.

Vocational Evaluation in Supported Employment

can occur after the person has assumed a particular job. This assessment is to predict what a person can do in a specific job setting with the necessary social and vocational support system in place. This assessment focuses on the need for worksite accommodations and supports the level of the person's independent living skills, and the types of continuing services the person will need

Assessment of Personality

designed to measure an individual's emotional, interpersonal, motivational, and attitudinal characteristics. The purpose to use this assessment in RC is to identify traits of a person that might impact job acquisition and retention.

Test Fairness

during administration, you must consider the consumer's needs and reactions to the assessment process (ex. environmental influences and consumer traits). tests should be administered under the same conditions that were established in their standardization. When there is a need to modify tests to accommodate certain disability groups, attention should be given to ensure appropriate assessment methods so as to not compromise the validity and reliability of the test. Types of accommodations include testing medium, time limits, and test content.

What is included in vocational assessment?

exploring a person's strengths, weaknesses, and preferences and discovering how the individual's potential for vocational adjustment can be enhanced.

Assessment procedures may include...

interviews, standardized tests, inventories, observations, job tryouts, simulated tasks, and medical examinations.

Raw score

is performance on a test (percent of items correct). The raw score is meaningless without additional knowledge concerning the instrument.

What is the purpose of assessment?

is to integrate information from multiple sources to plan a course of action. Also, to identify a specific and potential problem relevant to achieving career goals, the development of career goals, and the planning of strategies to resolve problems and attain established objectives.

Assessment of Achievement

is used to provide an evaluation of the specific information that individuals have learned throughout their education and life experience to date. RCs are generally interested in assessment of the verbal and numerical skills of the consumer because these skills are important areas related to job efficiency

Criterion-referenced interpretation

is when a score is compared with an absolute standard, external frame of reference Ex. meeting a proficiency standard

Self-referenced interpretation

is when a score is compared with an internal frame of reference Ex. changes in an individual's performance on a test over time

Norm-referenced interpretation

is when a score is compared with scores obtained by other individuals, external frame of reference Ex. percentile comparison to norm group

Validity

provides an estimate of how well a test measures what it claims to measure. The extent to which meaningful and appropriate inferences can be made from the instrument.

Content Validity

refers to the extent to which a measure represents all facets of a given construct. focuses on the issue that while the test is being constructed, the test items are an adequate sample of a specific domain of content that defines what it is being measured. Evaluation by subject matter experts of test items representativeness of the construct being measured. Most likely to be reported in achievement tests. Ex. when the duties and tasks of a work sample or situational assessment are highly similar to those of the job itself. Ex. a depression scale may lack content validity if it only assesses the affective dimension of depression but fails to take into account the behavioral dimension.

Assessment of Aptitude

regarded as relatively stable abilities that are innate and developed over a long period of time. They are used to assess individuals' skills and abilities, measure more specific or focused areas that predict the likelihood to learn and master knowledge or skills needed for success in a specific vocation, and predict how successful an individual will likely be at learning different aspects of a formal training program.

Cultural Competence in Assessment

sensitivity and awareness of values, attitudes, and beliefs are essential components of this. Training for this includes a) determining the acculturation status of consumer prior to the standard assessment approaches b) applying methodology and research findings throughout the assessment process, c) using or appropriately adapting standard assessment instruments d) developing an awareness of attitudes, beliefs, knowledge, and skills in working with culturally diverse groups

Assessment of Intelligence

the ability to solve problems and to learn and retain new information.

Measurement

the assignment of numbers to attributes of persons according to rules stated explicitly

Criterion (Predictive) Validity

the extent to which a measure is related to an outcome. It is predictive, it tests and predicts how you will do in the future.

Construct Validity

the extent to which the measure actually measures the theoretical construct.

What is the purpose of Vocational Evaluation?

the purpose is to provide reliable and valid data to a) generate info about the consumer's current vocationally relevant levels of social, educational, psychological, and physiological functioning b) estimate the consumer's potential for behavior change and skill acquisition c) determine the consumer's most effective learning style d) identify possible jobs the consumer can perform without additional vocational services e) identify education or special training programs that might increase the vocational potential f)identify potentially feasible jobs for the consumer with further vocational services g) identify the community support services that might augment job retention following a successful consumer placement. This step focuses on assessing the relationship between the person's skills, abilities, personality characteristics, and physical tolerance to perform the required tasks associated with potential jobs.

On-the-job Evaluations

these assess the functioning of individuals with disabilities in actual work settings where they are involved in activities presumed to be compatible with their vocational interests and skills. This type of Eval. can occur within workstations in institutions, rehab facilities, or business and industries focusing on a variety of variables including personality, attitudes, aptitudes, work trats, work skills, and physical capacities. The time period can range from one day to a month or longer.

Assessment of Vocational Intersts

these help the consumers identify jobs in which the consumer is likely to experience greater job satisfaction. Knowledge of the match between the consumer's aptitdues and abilities and the skill demands of a job will enhance the prediction of the likelihood of a consumer's satisfaction with a given job.

Percentiles

this is a rank order score indicating the percentage of persons in the comparison group who attained a lower score. This is not a percentage

Intake Interview

this is the initial information collection process. This process generates a social-vocational history, which is useful in formulating the rehabilitation plan and in determining whether subsequent evaluations are needed based on questions that the consumer can answer directly. The focus should be on 1) determining the person's reason for rehabilitation services 2) providing the individual with necessary information about the role and function of the agency 3) developing adequate rapport 4) initiating the diagnostic process 5) informing the consumer of any medical, vocational, or psychological evaluations that must be completed and the purposed of such evaluations.

Ecological Assessments

this type of assessment occurs in the natural setting, such as actual worksites where the individual could potentially be a long-term employee. The goal is to evaluate the individual's capacity to meet the productivity demands of that setting at present or in the near future through the provision of training or on-the-job supports.

Psychological Evaluation

this yields info regarding consumers' intelligence, aptitudes, achievement, personality, interests, and adjustment related to vocational functioning. It can also be used to establish the presence of a cognitive or psychiatric disability. These results help to determine the appropriateness of long-term vocational training, the need for adjustment services, and the need to confront the consumer regarding unrealistic vocational choices.

What is the intent of psychological measurement?

to convert characteristics of people to quantifiable data or numbers. An ongoing concern is assessment and score interpretation is the extent to which test scores are meaningful.these measurement issues can be addressed through scaling techniques.