ASSESSMENT AND TESTING

intelligence testing

A subset of intellectual and cognitive functioning that assesses a broad range of cognitive capabilities that generally results in an I

mean

ADD SCORES AND DIVIDE BY NUMBER OF SCORES

measures of central tendency describe

ATTEMPT TO DESCRIBE A DISTRIBUTION BASED ONTYPICAL OR AVERAGE PERFORMANCE OR SCOREs

Alternate forms reliability

Alternate forms reliability is a way of assessing stability using different, but equivalent, versions of a test. To create alternate forms of a test, individual items are selected from a pool of test questions designed to assess a given construct. These forms should have similar score means, variance, item difficulty, and correlations with other measures. example: counselors are required to take a to become nationally certified counselors

CORRELATION COEFFICIENTS (R)THIS IS A NUMBER

BETWEEN -1 AND +1 THAT INDICATESDIRECTION AND STRENGTH OF THE RELATIONSHIPAS "R" APPROACHES +1, STRENGTH INCREASES IN A DIRECT ANDPOSITIVE WAYAS "R" APPROACHES -1, STRENGTH INCREASES IN AN INVERSE ANDNEGATIVE WAYAS "R" APPROACHES 0, THE RELATIONSHIP IS WEAK OR NONEXISTENT (AT ZERO

Range

COMPUTED BY SUBTRACTING THE LOWEST SCORE FROM THEHIGHEST SCOres

Assessment procedures can be

Formal: Rigor in test development (e.g., good, valid, and reliable)Informal: Rigor has not been demonstrated in the test development

Purpose of Assessment

Four general purposes: 1) screening 2) Identification and diagnosis 3) intervention planning 4) Progress and outcome evaluation

Describing scores measures of central tendency

HELPS TO PUT MORE MEANING TO SCORES• TELLS YOU SOMETHING ABOUT THE "CENTER" OF A SERIES OF SCORES • THREE COMMON MEASURES OF CENTRAL TENDENCY ARE: • MEAN • MEDIAN • MODE

The normal curve

IT IS UNIMODAL, WHICH MEANS THAT IT HAS A SINGLE POINT OF MAXIMUM FREQUENCY OR MAXIMUM HEIGHT • 100% OF THE SCORES FALL BETWEEN -3 AND +3 STANDARD DEVIATION FROM THE MEAN WITH APPROXIMATELY: • 68% OF THE SCORES FALLING BETWEEN -1 AND +1 STANDARD DEVIATIONS • 95% OF SCORES FALLING BETWEEN -2 AND +2 STANDARD DEVIATION • 99.5% OF SCORES FALLING BETWEEN -3 AND +3 STANDARD Deviation

Internal consistency

In internal consistency, you are looking at individual items to see if those items are consistent with one another and thus represent a singular construct or trait that is being assessed by the test. This method is used to determine whether errors associated with content sampling are present

Increasing reliability

Increase the number of test items Write understandable, unambiguous items Use selected-response (multiple choice)instead of constructed-response (essay)items Ensure items are not too difficult or too easy Provide clearly stated administration and scoring procedures Require training before individuals can administer, grade, and interpret a test

Select and implement assessment methods

Interviews Background information including family history, work and education background, social history, environmental factors, etc. Tests Evaluation of a person's cognitive functioning, knowledge, skills, abilities, or personal traits. Observation May be used to record or monitor a client's behavior in a particular setting

Variance

THE AVERAGE AMOUNT OF VARIABILITY IN A GROUP'S SCORES. ITIS COMPUTED AS THE AVERAGE SQUARED DEVIATIONS OF VALUES FROM THE MEAN

measures of variability refer to

THE DEGREE THAT SCORES ARE SPREAD OUT AND DIFFER FROM ONE ANOTHER•

What is a tes

Tests are a subset of assessment yielding scores based on collective data. Example: Finding the sum of correct items on multiple-choice exam

Test

Tests are instruments designed to measure specific attributes of an individual including knowledge or skill level, intellectual functioning, aptitude, interests, psychological symptoms, etc.

standards for scoring errors

The 2000 Guidelines of the International Test Commission (ICT) state that test users should apply quality control procedures to the scoring, analysis, and reporting process and that test users should maintain high standards • Scoring of assessments should be conducted properly and efficiently so that the results are reported accurately and in timely manner

Brief history of intelligence testing

The First test pioneered by Binet and Simon• Revised by Lewis Terman • Over the years, many new models of intelligence were defined • Many new intelligence tests were created based on the models

KR Formulas and coefficient Alpha

The Kuder-Richardson formulas were developed to serve as an alternative to the split-half method of assessing internal consistency. Like in the split-half approach, a single test is administered in a single session. The difference is that rather than arbitrarily dividing the test into two equivalent halves, a statistical process is applied to determine the the split-half reliability of all possible combinations of test halves that can be created by separating the items.

Standard deviation

The average distance of test scores from the mean

Factors that can impact the validity of score interpretations: Contextual factors

When interpreting results, test users should consider the relationship of the test to the instructional program, the opportunity to learn, quality of the educational program, work and home environment, and other factors. For example, if the test does not align to curriculum standards and how those standards are taught in the classroom, then the test results may not provide useful information

Z-scores

Z Scores convert test scores into a standard deviation value, ranging from -3.0 to +3.0.The mean for the distribution of z scores is 0and the standard deviation is 1.0

An effective assessment report is

a document that outlines the presenting problem that led the client to seek counseling, summarizes the services provided, and addresses the outcomes of those services that were provided

what is intelligence

a measure of your ability to acquire and apply knowledge

skilled counselors begin the process of screening their client when?

almost immediately nonverbal communication is an interesting phenomenon - appreance -motor skills (movements) -motivation -who accompanies the clients -voluntary or involuntary presentation

Case Conceptualization

can best be defined as "a method and clinical strategy for obtaining and organizing information about a client, understanding and explaining the client's situation and maladaptive patterns, guiding and focusing treatment, anticipating challenges, and road blocks, and preparing for successful termination

screening and initial assessments intake interview:

demographic information Referral reasons Current situation Previous counseling experience Birth and developmental history Family history Medical history Educational and/or vocational background intake and Assessment Role-Play Part 1 - Referral and Presenting Problems - YouTube

Frequency distribution

distribution is simply a set of scores. A frequency distribution orders a set of scores from high to low and lists the corresponding frequency of each score. Frequency distributions are useful in allowing examiners to see 1) the entire set of scores at a glance 2) whether scores are generally high or generally low 3) whether scores are concentrated in one area spread out

Selecting assessment instruments and strategies

does the instrument's manual provide clear and detailed instructions about administration procedures? does the manual provide sufficient information about scoring interpreting and reporting results? is the instrument biased? what level of competency is needed to use the instrument? A, B, or C?

Identify the problem

does this student have a learning disability? If so, does he/she qualify for special education or related services this child ready to begin kindergarten? Does this adult have PTSD? Is this individual suicidal? Does this parent have a mental disorder that might interfere with parenting? What are this individual's vocational interests

Five steps in selecting an assessment instrument: step five

evaluate and select an assessment instrument or strategy - When selecting which assessment instrument and strategies to use, counselors must evaluate instruments on the basis of several factors, including purpose, test scores, reliability, validity, etc.

Five steps in selecting an assessment instrument: step one

identify the type of information needed - Counselors determined what information to collect based on the purpose of the assessment. If you were looking to screen a client tos ee if there was a need for acute psychiatric care at the inpatient level, what information would be needed to help you make this decision? Information on presenting signs and symptoms, the presence of any psychotic features, and any verbalization of threats to harm self others would be supportive of your division to admit this client to an inpatient care facility

The Methods for collecting assessment information

interviews, tests, observations

observation

is an assessment method that involves watching and recording the behavior of an individual in a particular environment. It is away of seeing what a person actually does

Interrater

is used when we want to assess the level of agreement between two or more raters in their ecauation of a particular outcome. the most effective way to assess inter-rater reliability is to correlate the scores obtainned independently by each of the raters

What practical issues should be considered for this instrument? - It includes:

the time required for administration cost of the instrument format readability administration procedures scoring procedures interpretation

true score

the true reflection of ones ability, skills or knowledge

Time sampling error

time sampling errors result from repeated administrations of a test to the same individual. When an individualities the same test multiple times, the scores obtained will most likely vary. Fatigue

Presenting assessment reports, three kinds

to the adult client to minors debriefing

activities during administration

• The counselor may begin with a final check to see that all is in order• Examiners need to check on materials, lighting, ventilation, work space, etc. • Counselors need to deliver verbatim instructions given in the test manual• Counselors need to establish rapport with the examinee• The examiner should record any test behavior or other critical incidents that may increase or reduce an individual's opportunity to perform to capacity

purpose of intelligence testing

• To assist in determining giftedness• To assess mental retardation • To identify certain types of learning disabilities • To assess intellectual ability following an accident, the onset of dementia, substance abuse, disease processes, and trauma to the brain • Admissions process to certain private schools • As part of a personality assessment battery to aid in understanding the whole

The areas counselors assess are in diagnosis

• the history of the client's presenting problem • the developmental history of the problem • any precursors and consequences related to the problem • and the existence of any strengths or weaknesses the client possesses that may need to be highlighted or address use: the DSM

Charles Spearman's g theory

(1863-1945) two-factor approach: • Spearman postulated that performance on intelligence tests is based on• a general ability factor (g) and• one or more specific factors (s) • The g factor represented a measure of general intelligence that underlies performance on a wide variety of tasks while the s factors were specific learned skills that can influence intelligence performance

The assessment process is into four parts

1) identify the problem 2) Select and Implement Assessment Methods 3) Evaluate the Assessment Information 4) Report Assessment results and male recommendations

Scoring performance assessment components

1. One or more dimensions or attributes on which performance is rated 2. Descriptors of examples that illustrate each dimension or attribute being measured• Poor, Fair, Good, Excellent 3. A rating scale for each dimension• Unsatisfactory, Below satisfactory, satisfactory, and exemplary

CORRELATION COEFFICIENTS (R)

CORRELATION IS THE STATISTICAL EXPRESSION OF THE RELATIONSHIP BETWEEN TWO SETS OF SCORES (OR VARIABLES)• POSITIVE CORRELATION: AN INCREASE IN ONE VARIABLE ACCOMPANIED BY AN INCREASE IN ANOTHER "DIRECT" RELATIONSHIP • NEGATIVE CORRELATION: INCREASE IN ONE VARIABLE ACCOMPANIED BY DECREASE IN OTHER"INVERSE" RELATIONSHIPS

Content sampling error

Content sampling errors are related to the development and construction of tests. In some cases, it would be near impossible to create a test with enough items to sufficiently assess every aspect of a dimension or construct being assessed. E.g., IQ test

activities after administration

Counselors may need to collect materials according to a predetermined order, counting the test booklets and answer sheets and arranging them all face-up

Five steps in selecting an assessment instrument: step three

Determine the methods for obtaining information - Counselors may use interviews, tests,and observations when they established appropriate competency. However, the counselor should pay attention to the external variables, including setting, timing, financing, and legal involvement

Diagnosis

Diagnosis refers to the process of learning more about the client and his or her problem

Evaluate the assessment information

Evaluation involves scoring, interpreting, and integrating information obtained from all assessment methods and sources to answer referral questions' using basic skills such as, statistical concepts, psychometric principles, and procedures, counselors can organize collected data in the following way: Document any significant findings that clearly identify problem areas • Identify convergent findings across methods and sources • Identify and explain discrepancies in information across methods and sources • Arrive at a tentative formulation or hypothesis of the individual's problem • Determine the information to include in the assessment report

percentiles

Express the examinee's relative position on a norm-referenced test. Percentile ranks range from 0 to 100 and indicate the percentage of scores that were lower than the examinee. Percentiles are not equal-interval measurements

Factor analysis

FACTOR ANALYSIS ALSO ANALYZES THE RELATIONSHIP AMONGVARIABLES TO SIMPLIFY THE DESCRIPTION OF DATA BYREDUCING THE NUMBER OF NECESSARY VARIABLES

Factors that can impact the validity of score interpretations: Psychometric Factors

Factors such as the reliability, norms, standard error of measurement, and validity of instrument can impact an individual's scores and the interpretation of the test results

Selecting assessment instruments and strategies level A test

Level A tests are those that are designated for general use. They do not require advanced training or education. To administer a Level A test, users would need to familiarize themselves with the administration, scoring, and result-reporting protocols for the instrument. This information is typically found in an accompanying test manual. An example of a Level A test would be the Self-Directed Search(Holland, 1994).

Level B test selecting instruments and stratagies

Level B tests require users to possess technical knowledge related to the practice of assessment. This includes an understanding of instrument development, psychometric issues (reliability and validity), test score properties, and appropriate test usage. Individuals interested in using Level B tests should have completed graduate course workin assessment as part of their master's degree in counseling, psychology, or a related field. Most test publishers also require those interested in using these tests to document that they have appropriate licensing and credentialing in their field. Examples of Level B assessments are the Myers-Briggs Type Indicator and the 16-Personality Factor Questionnaire. When you begin your career as a counselor following the completion of your master's degree, these are the level tests you most likely will be using

level c test when selecting an assessment instrument and strategy

Level C is the highest level of qualification. Level C tests require advanced education and training. An earned doctorate in counseling, psychology, education, or a related field is required. In addition to the general assessment coursework required for Level B tests, potential users must also have had coursework in either a specific instrument (e.g., Wechsler Adult Intelligence Scale-Fourth Edition) or a class of instruments (intelligence testing). An example of a Level C test would be the Minnesota Multiphasic Personality Inventory-Second Edition(MMPI-2). Counselors interested in using Level tests in their practice should check to see if there are any state laws or regulations that may limit their use of certain assessments or tests from this level. In some states, the use of testing is protected and only members of certain professions(counselors, psychologist, social workers, can use specified tests

Thurstone's multifactor approach

Louis L. Thurstone, a British psychologist who lived from 1887 to1955, did not believe that g was the only factor that constitutes intelligence, nor did he support the idea that a single IQ fully and comprehensively assessed intelligence. • Seven primary mental abilities:• Verbal comprehension - ability to understand ideas expressed in word form• Number ability - ability to perform basic mathematic processes accurately and rapidly • Word fluency - ability to speak and write fluently • Perception speed - ability to perceive things quickly, such as visual details and similarities and differences among pictured objects • Spatial ability - ability to visualize and form relationships in three dimensions • Reasoning- abilities to derive rules and solve problems inductively Memory- the ability to recognize and recall information such as number letters and words

Median

MIDDLE SCORE • ODD NUMBER OF SCORES = EXACT MIDDLE SCORE • EVEN NUMBER = AVERAGE OF TWO MIDDLE SCORES

Five steps in selecting an assessment instrument: Step four

Mental MeasurementYearbook, tests in print, publisher's website or catalogs, manuals, research literature, internet resources, etc.

Scoring errors

Occur frequently, regardless of who is scoring the instrument or the scorer's level of experience with testing • Include assignment of incorrect score values to individual responses, incorrectly converting raw scores to derived scores, and making calculation errors

CORRELATION COEFFICIENTS (R)

PLOTTING TWO SETS OF SCORES FROM THE PREVIOUS EXAMPLES ON A GRAPH • PLACE PERSON A'S SAT SCORE ON THE X-AXIS, AND HIS/HERGPA ON THE Y-AXIS• CONTINUE THIS FOR PERSON B,C, D ETC. • THIS PROCESS FORMS A SCATTERPLOT

Treatment plan

Problems goals objectives interventions

qualification to use standardized test depend on at least four factors

Purpose of Testing Characteristics of tests Settings and conditions of test use Roles of test selectors, administrators, scorers, and interpreter

Regression

REGRESSION IS A STATISTICAL METHOD RELATED TOCORRELATION BUT IS PRIMARILY USED FOR PREDICTION • REGRESSION IS THE ANALYSIS OF RELATIONSHIPS AMONG VARIABLES FOR THE PURPOSE OF UNDERSTANDING HOW ONEVARIABLE MAY PREDIC T ANOTHER

Three commonly used measures of variability are

Range variance standard deviation

Standard scores

Refers to scores that have been converted to an interpretable scale that has a set mean and standard deviation. Standard scores allow individual test scores to be interpreted in terms of the normal curve

Report assessment results and make recommendations

Reporting results and making recommendations involve the following steps: • Describing the individual being assessed and his/her situation • Reporting general hypotheses about the individual • Supporting these hypotheses with assessment information • Proposing recommendations related to the original reason for referral

Stanine

Scores range from 1 to 9. Mean of 5 with a standard deviation of 2

Administering Assessment Instrument

Self-Administered • Individually Administered • Group Administered • Computer Administered • Video Administration• Audio Administration • American Sign Language • Nonverbal

Types of Norm-Referenced Scores

Standard Scores Normal curve Z-scores T -scores Stanine Percentile

T-scores

T scores use a fixed mean of 50 and a fixed standard deviation of 10. T scores eliminate the need for decimals and negative values. T-scores are calculated using the Z score as abase

Five steps in selecting an assessment instrument: step two

The existing information often provides useful ,relevant informationn about a client that may be used in the current assessment process intake questionnaires, biographical data, preliminary diagnosis, family information educational history, grades, etc.). Once you are able to quantify what you already know, you can look too select a test that helps you fill in the gaps and address those areas where you have an incomplete picture of the client or presenting situation.

Multiple Methods and Multiple Sources

The interview is a face-to-face meeting of the assessment professional and the client

Level of acceptable reliability

The level of acceptable reliability depends on: the construct being measured the way the test scores will be used the method used for estimating reliability

What is Assessment

The term assessment refers to any systematic procedure for collecting information that is used to make inferences or decisions about the characteristics of a person.

Classical test theory

The test scores are never completely consistent and there will always be at least some degree of variance.• To understand better the concept of measurement error, we have to understand the Classical test theory (CTT). • CTT often referred to as the true score model, describes a set of psychometric procedures that can be used to test the reliability, difficulty, and discriminatory properties of test items and scales

Factors that can impact the validity of score interpretations: Test-taker factors

The test taker's group membership (Gender, age, ethnicity, race, SES, relationship status) is a critical factor in the interpretation of test results. Test users should evaluate how the test taker's group membership can affect his or her test results

Methods of estimating reliability

There are four primary methods through which reliability can be assessed. These methods are: (a) test-retest, (b) alternate forms, (c) internal consistency, and (d) inter-rater reliability

test-retest

This approach is used when you are interested in assessing how reliable or stable scores on an instrument are overtime. The process of assessing test-retest reliability is straightforward. A set of participants is tested using the same test on two separate occasions. The observed scores of the participants on each test administration are then compared and a reliability coefficient is calculated. In this design, instability over time can be viewed as the primary source of measurement error

sources of measurement

Time-Sampling Error Content-Sampling Error Interrater Differences Other Sources of Error Quality of Test Items Test Length Task-Taker Variables Test Administration

Standard error of measurement (SEM)

measures an individual's test score fluctuations(due to error) if he/she took the same test repeatedly. The SEM is not a measure of reliability but can be used to create confidence intervals around specific observed scores. Confidence intervals establish the upper and lower limit in which a test taker's true score falls.

Multiple Linear Regression

multiple IVs and one DV

Scales of measurement

nominal ordinal interval ratio

Classical test theory consist of

observed score true score overserved score = true score + error (x=T+E)

Simple linear regression

one IV and one DV

Reliability

refers to the degree to which test scores are dependable, consistent, and stable upon additional tests. Multiple factors can impact reliability. Reliability refers to the results obtained; not the instrument

INTEGRATING ASSESSMENT INTO COUNSELING Practice 6 concepts

screening and initial assessments diagnosis case conceptualization developing client treatment plans writing effective assessment reports presenting assessment reports

Models of intelligence

spearman's g theory Cattell's fluid and crystallized intelligence Thurstone's primary mental abilities Vernon's hierarchical model of intelligence Sternberg's triarchic theory of intelligence Piaget cognitive developmental theory Gardner's multiple intelligences emotional intelligence information processing view

the are various sources of measurement error that can affect the reliability of test scores

systematic error unsystematic error (random error)

Observed score

the acutal score on a test