Assessment Chapter 4 (Of tests and testing)
Criterion-referenced testing and assessment
A method of evaluation and a way of deriving meaning from test scores by evaluating an individual's score with reference to a set standard. (Domain or content-referenced testing and assessment)
Subgroup norms
A normative sample can be segmented by any of the criteria initially used in selecting subjects for the sample. Norms for any defined group within a larger group. Are created when narrowly defined groups are sampled (i.e., socioeconomic status, education level, age).
Criterion
A standard on which judgment or decision may be based.
Percentile
An expression of the percentage of people whose score on a test or measure falls below a particular raw score. For example, if a student scores in the 85th percentile on a standardized test, then 85% of those taking the test had lower scores.
National norms
Norms derived from a standardization sample that was nationally representative of the population. For example, a national norm may be measured for all students in the United States who take the SATs in the Fall of their senior year. SAT officials will add all the scores for all the thousands of students who took the exam, and divide that large number by the total number of students who took the exam. The number that results from calculating the average is the SAT national norm for the Fall of that year.
Anchoring
Permits conversion of raw scores on the new version of the test into fixed reference group scores.
Local norms
Provide normative information with respect to the local population's performance on some test. Normative information about some limited population, frequency of specific interest to the test users. Are derived from the local population's performance on a measure. Typically created locally (i.e., by guidance counselor, personnel director, etc.)
Norming
Refer to the process of deriving norms. May be modified to describe a particular type of norm derivation.
Psychometric soundness of tests
Reliable, valid.
Cumulative scoring
The assumption that the more the testtaker responds in a particular direction as keyed by the test manual as correct or consistent with a particular trait, the higher the testtaker is presumed to be on the targeted ability or trait.
Reliability
The consistency of the measuring tool: the precision with which the test measures and the extent to which error is present in measurement.
Norm-referenced testing and assessment
method of evaluation and a way of deriving meaning from test scores by evaluating an individual testtaker's score and comparing it to scores of a group of testtakers.
Incidental sample (convenience sample)
one that is convenient or available for use.
error variance
the component of a test score attributable to sources other than the trait or ability measured.
Assumptions about psychological testing and assessment
1. Psychological traits and states exist 2. Psychological traits and states can be quantified and measured. 3. Test-related behavior predicts non-test related behavior. 4. Tests and other measurement techniques have strengths and weaknesses. 5. Various sources of error are part of the assessment process. 6. Testing and assessment can be conducted in a fair and unbiased manner. 7. Testing and assessment benefit society.
Construct
An informed, scientific concept developed or constructed to describe or explain behavior.
Overt behavior
An observable action or the product of an observable action, including test-or assessment-related responses.
Trait
Any distinguishable, relatively enduring way in which one individual various from another.
Grade norms
Developed by administering the test to representative samples of children over a range of consecutive grade levels. are norms specifically designed as a reference in the context of the grade of the test taker who achieved a particular score. For example, a high school may administer a standardized test for all senior students in the Fall of this year. When the results are in, half of the students scored below a 65 percent and half of the students score above a 65 percent. The median, or grade norm, in that case for seniors at that high school in the fall of this year is 65 percent.
States
Distinguish one person from another but are relatively less enduring.
Norm-referenced
Evaluate the test score in relation to other scores on the same test.
Validity
If a test does, in fact, measure what it purports to measure.
Stratified-random sampling
If every member of the population had the same chance of being included.
Purposive sample
If we arbitrarily select some sample because we believe it to be representative of the population.
Assumption 6: Testing and assessment can be conducted in fair and unbiased manner
More controversial than the other 6 assumptions. One source of fairness-related problems is the test user who attempts to use a particular test with people whose background and experience are different from the background and experience of people for whom the test was intended.
National anchor norms
Provide some stability to test scores by anchoring them to other test scores. An equivalency table for scores on two nationally standardized tests designed to measure the same thing. Used to consider two tests that were normed by using the same sample (i.e., each member of the sample took both tests).
Standardized test
Tests that have clearly specified procedures for administration, scoring, and interpretation in addition to norms.
Normative sample
That group of people whose performance on a particular test is analyzed for reference in evaluating the performance of individual testtakers.
Standard
That which others are compared to or evaluated against.
Age-equivalent scores (age norms)
The average of performance of different samples of testtakers who were at various ages at the time that the test was administered.
Race norming
The controversial practice of norming on the basis of race or ethnic background.
Percentage correct
The distribution of raw scores. The number of items that were answered correctly multiplied by 100 and divided by the number of items.
Fixed reference group scoring system
The distribution of scores obtained on the test from one group of testtakers- referred to as the fixed reference group - use as the basis for the calculation of test scores for future administrations of the test. (e.g., SAT)
Standardization or test standardization
The process of administering a test to a representative sample testtakers for the purpose of establishing norms.
Sampling
The process of selecting the portion of the universe deemed to be representative of the whole population.
Whether a trait manifests itself in observable behavior, and to what degree it manifests, is presumed to depend on
The strength of the trait in the individual but also the nature of the situation.
Psychological trait
covers a wide range of possible characteristics. Relate to intelligence, specific intellectual abilities, cognitive styles, adjustment interests, attitudes, sexual orientation and preferences, psychopathology, personality in general and specific personality traits.
User norms (program norms)
descriptive statistics based on a group of testtakers in a given period of time rather than norms obtained by formal sampling methods.
Samples of behavior may be obtained by:
direct observation, or pencil-to-paper answers.
Age norms
norms specifically designed for use as a reference in the context of age of the test taker who achieved a particular score. Commonly defined as social rules for age-appropriate behavior, including everyday actions and/or the timing and sequencing of major life events (i.e., marriage, parenthood)
Stratified sampling
such sampling would help prevent sampling bias and ultimately aid in the interpretation of the findings. The most accurate way of developing norm group. Common demographics to stratify: age, gender, socioeconomic status, geographic region.
Classical test theory (CTT, true score theory)
the assumption is made that each testtaker has a true score on a test that would be obtained but for the action of measurement error.
Equipercentile method
the equivalency of scores on different tests is calculated with reference to corresponding percentile scores.
Norms
the test performance data of a particular group of testtakers that are designed for use as a reference when evaluating or interpreting individual test scores.
