Exam 2: Chapters 3, 4, 5
A new academic achievement test assesses elementary-age students' math ability. The test developers found, however, that students in the research group who took the test two times had scores that were quite different upon the second test administration, which was conducted 2 weeks after the initial administration. It was determined that the test did not have acceptable _____________.
test-retest reliability
interrater reliability
the amount of agreement in the observations of different raters who witness the same behavior
Range
the difference between the highest and lowest scores in a distribution
Validity
the extent to which a test measures or predicts what it is supposed to
content validity
the extent to which a test samples the behavior that is of interest
concurrent validity
the extent to which two measures of the same trait or ability agree
construct validity
the extent to which variables measure what they are supposed to measure
No matter how many times a student may be tested, the student's _____________ may never be determined.
true score
A test instrument may have good reliability; however, that does not guarantee that the test has _____________.
validity
Approximately how many people would be expected to fall in the IQ range represented by the z scores of +3.0 to +4.0?
.13%
Equivalent forms reliability
2 forms of the same instrument are used. Items are matched for difficulty
Approximately what percentile rank would be assigned for the IQ score of 70?
3rd percentile rank
What percentile rank would be assigned to the z score of 0?
50th percentile rank
Chronological age of examinee
6 years, 10 months, 25 days
Raw score
7
What percentage of the scores would fall between the z scores of -2.0 and +2.0?
95%
What percentile rank would represent a person who scored at the z score of +3.0?
99th percentile rank
positive correlation
A correlation where as one variable increases, the other also increases, or as one decreases so does the other. Both variables move in the same direction.
bimodal distribution
A distribution (of opinions) that shows two responses being chosen about as frequently as each other.
Why is it important to understand the average performance of students in a particular grade level or age level?
So that you have something to compare other students to
The larger the standard error of measurement, the less _____________ the test.
reliable
Normal distribution
A function that represents the distribution of variables as a symmetrical bell-shaped graph.
Scattergram
A type of graph that represents the strength and direction of a relationship between co-variables in a correlational analysis.
age equivalent
A very general score that is used to compare the performance of children at the same age with one another.
measures of central tendency
A way to organize data to see hot the data cluster, or are distributed around a numerical representation of the average score mean, median, mode
The smaller the standard error of measurement, the more _____________ the test.
reliable
Lorenzo also requires the use of a stylus for writing or answers questions orally. He will also require changes in ____.
response mode
In the following set of data, what central tendency is represented by the number 77? 65,66,82,95,77
Median
Why is the following set of data interesting? 22,47,88,62,65,22,63,89,55,74,88,99,44,65,100
Multimodal
The following set of data has what type of distribution? 88, 33, 78, 56, 44, 37, 90, 99, 76, 78, 77, 62, 90
Negatively skewed
What term describes a set of data in which the mean is less than the most frequently occurring scores?
Negatively skewed
In a university, all students are given a new student identification number upon registration. These numbers are on what scale?
Nominal scale
If the heights of all fifth-grade elementary students in one large city were measured, in what manner would the resulting data be displayed?
Normal Distribution
Mean difference
Taking into account the diversity within a population
Reliability and Validity
Techniques for evaluating the relationship between measured and conceptual variables
PASA
The Pennsylvania Alternate System of Assessment is a statewide alternate assessment designed for students with the most significant of cognitive disabilities 1% per school district (a district could have more than 1%, school can allow the students to take the assessment but only count 1% of the students who take the test)
measures of dispersion
range, variance, standard deviation
To better understand a student's true abilities, the concept of _____________ must be understood and applied to obtained scores.
standard error of measurement
A student score that is reported to be exactly average with a score of 5 is reporting using ____ scores.
stanines
Information regarding how a test was developed is usually contained in the
test manual
Domain scores
5, 1, 1, 0
Chronological age
Age as measured in years from date of birth
Norm-referenced assessment
An assessment that compares a student's performance to a sample of that student's peers.
How is a norm-referenced test assembled?
An item pool is created Items are arranged and sequenced according to difficulty A developmental version is field tested with a small sample Professionals critique the assessment Revisions are made Field test with a larger sample
Explain how outliers in a data set can influence the mean.
An outlier is a value in a data set that is very different from the other values. That is, outliers are values unusually far from the middle. In most cases, outliers have influence on mean , but not on the median , or mode . Therefore, the outliers are important in their effect on the mean.
What are the issues and concerns of the statewide assessment of students with disabilities?
Concerns about the inconsistency of definitions, federal law requirements, variability among states and districts, differences in standards of expectations for students with disabilities, lack of participation of students with disabilities in test development and standardization of instruments, and lack of consistency. 2. Conceptual understanding of the purpose and nature of the assessment. 3. Mandatory statewide assessments have resulted in damaging the American education system for all students, and alternate assessments may not be the best way to measure academic progress. 4. Some teachers reported that high-stakes assessment helped teachers target and individualize instruction, and that their students who disliked reading or had difficulties with academics felt more in control of their own learning. 5. Performance task-based reading alternate tests can be scaled to statewide assessments, although determining their validity and reliability may be difficult. 6. Development of alternate assessments is difficult, and states require more time to develop appropriate measures. 7.Practicesonsomecampusesmightresultinspecificgroupsofstudentsbeingencouragednotto attend school on the days of the assessment so that campus data might be more favorable. 8. Test items were not comparable across assessments when the modified assessments used for children with various disabilities were analyzed. Moreover, the items varied by disability category.
Why is important to graph test data?
It is important that our tests are assessing what we want (if too many students are failing)
Basal item
Item 2, 3, 4
Ceiling item
Items 12, 13, 14
Discuss 3 ways a test administrator could decrease test bias.
Make sure that the way of responding is easily accessible for all students. Provide multiple opportunities for the student to respond to test questions. Make sure that all materials and methods of the exam are appropriate (appropriate for the age and grade level). Get to know and understand individuals unique background before they take the assessment. Create content that is relevant and necessary for the assessment. Build a relationship with the individual prior to testing. Explain instructions on students' developmental level. Have the student practice a variety of methods of recording responses.
Basals and ceilings
Many standardized tests, particularly language and intelligence tests, require the examiner to establish a basal level and a ceiling level. Tests will define both the levels in terms of correct and incorrect scores. A basal score is the entry level and the ceiling score is the test-terminating score.
Use the following set of data to determine the mean, median, mode, range, variance, standard deviation, standard error of measurement, and possible range for each score, assuming 68% confidence. The reliability coefficient is .85. Data: 50, 75, 31, 77, 65, 81, 90, 92, 76, 74, 88 Mean ______ Median ______ Mode ______ Range ______ Variance ______ Standard deviation ______ Standard error of measurement ______
Mean = 72.64 (simple average) Median = 76 (middle score) Mode = there is none Range = 61 (92-31) Variance = 297.45572364 Standard deviation = 17.24690 Standard error of measurement = 6.68
The mean, median and mode make up what?
Measures of central tendency
Describe the administration process of a norm assessment.
The test manual contains information that the examiner must understand thoroughly before administering the test. The first few attempts of practice administration should be supervised by someone who has had experience with the instrument. The protocol of a standardized test is the form used during the test administration and for scoring and interpreting results. Beginning the test: get to know the student and become familiar with them before the test, before starting the test, spend some time talking with them in friendly conversation, explain to the student why we are testing them on this particular level so they understand why the test is important but do not feel threatened by it and give a brief introduction to the test, begin testing in a calm manner.
Calculating range
To calculate range, subtract the smallest number from the largest number
Positively skewed
When a distribution includes an extreme score that is very high.
standard deviation
a computed measure of how much scores vary around the mean score
Negatively skewed
a distribution of scores in which scores are concentrated in the high end of the distribution
criterion-related validity
a measure of validity based on showing a substantial correlation between test scores and job performance scores
Students with disabilities who have IEPs and students who are served under Section 504 may need ____ for statewide assessments.
accommodations
When calculating the range of possible scores, it is best to use the appropriate standard error of measurement for the student's _____________ provided in the test manual.
age or grade
negative correlation
as one variable increases, the other decreases
Gregory is a student with a specific reading disability. In his general education classroom, Gregory's teacher and the classroom aide must read all directions to him and often must read questions and multisyllabic words to him. On his IEP, the team has included a statement of accommodation of ____.
assessment format
Lorenzo, who participates in the general curriculum, requires Braille for all reading material. He will require changes in ____.
assessment format
Variance
standard deviation squared
A closer estimation of the student's best performance can be calculated by using the _____________.
standard error of measurement
This represents the level of items that the student would most probably answer correctly, although they may not all be administered to the student.
basal
A student's standard score that compares the student with age peers is found by using the student's raw score, the student's ____, and the norm tables.
chronological age
A range of possible scores can then be determined by using the _____________ for the specific test.
confidence interval
In studying the relationship between the scores of the administration of one test administration with the second administration of the test, the number .89 represents the _____________.
correlation coefficient
The score obtained during the assessment of a student may not be the true score, because all testing situations are subject to chance _____________.
error
This step is completed using the developmental version to determine what changes are needed prior to the completion of the published test.
field test
multimodal distribution
frequency distribution with two or more high frequencies separated by a lower frequency; a bimodal distribution is the special case of two high frequencies
A student's actual skill level is not represented by the ____.
grade equivalent
grade equivalent
grade score assigned to a mean raw score of a group during the norming process
Standard error of measurement (SEM)
hypothetical estimate of variation in scores if testing were repeated
On a teacher-made test of math, the following items were included: two single-digit addition problems, one single-digit subtraction problem, four problems of multiplication of fractions, and one pre-algebra problem. This test does not appear to have good _____________.
internal consistency
Discuss a few examples of mistakes that are easily made by professionals in the process of identifying students with special needs.
Discrimination A bad test A test that is too easy
All fourth-grade students in Little City School were asked to participate in a reading contest to see which student could read the most books in a 3 month period. At the end of the 3 months, the winners were determined. The 10 students who read the most books were awarded the prizes. On the final day of the contest, the students anxiously looked at the list where the names were in ___________, from the highest number of books read to the lowest.
Rank order
Interpolating data
Reading between data points plotted on the graph.
How are measures of central tendency different than measures of dispersion?
Measures of central tendency: A way to organize data to see hot the data cluster, or are distributed around a numerical representation of the average score mean, median, mode Measures of dispersion: range, variance, standard deviation
A set of data has a symmetrical distribution of scores with the mean, median, and mode represented by the number 82. This set of data represents a _____________.
Normal distribution
PSSA
Pennsylvania System of School Assessment
High stakes testing
Practice of using students' performance on a single assessment instrument to make major decisions about students or school personnel.
Identify the 4 primary modes by which an assessment may be accommodated and provide an example of each.
Presentation accommodations Response accommodations Setting accommodations Timing and scheduling accommodations
A 7th grade pre-algebra class completed the first test of the new school year. Here are the data resulting from the first test: 100, 99, 95, 90, 89, 85, 84, 82, 81, 80, 79, 78, 77, 76, 70, 68, 65, 62, 60, 59, 55. In this set of data, what does the number 45 represent?
Range
predictive validity
The success with which a test predicts the behavior it is designed to predict; it is assessed by computing the correlation between test scores and the criterion behavior.
test-retest reliability
a method for determining the reliability of a test by comparing a test taker's scores on the same test taken on separate occasions
Ratio scale
a quantitative scale of measurement in which the numerals have equal intervals and the value of zero truly means "nothing" Example: GPA
Interval Scale
a scale of measurement in which the intervals between numbers on the scale are all equal in size Example: IQ test result
Ordinal Scale
a scale of measurement in which the measurement categories form a rank order along a continuum Example: Grades
One would expect that the number of classes a college student attends in a specific course and the final exam grade in that course would have a _____________.
positive correlation
A college student failed the first test of the new semester. The student hoped that the first test did not have strong _____________ about performance on the final exam.
predictive validity
When a teacher scores a classroom test including 10 items and determines that a student correctly answered 7, the number 7 represents a____.
raw score
A teacher discovers that although only second-, third-, and fifth-graders were included in the norm sample of a test, scores were presented for fourth grade. The fourth-grade scores were ____.
interpolated
A new test was designed to measure the self-concept of students of middle-school age. The test required students to use essay-type responses to answer three questions regarding their feelings about their own self-concept. Two assessment professionals were comparing the students' responses and how these responses were scored by the professionals. On this type of instrument, it is important that the _____________ is acceptable.
interrater reliability
Nominal scale
measurement in which numbers are assigned to objects or classes of objects solely for the purpose of identification Example: Gender or demographic
The number of times a student moves during elementary school may likely have a _____________ with the student's achievement scores in elementary school.
negative correlation
When a test is being developed, the test developer attempts to have this represent the population for whom the test is designed.
norm group
Both individual and group assessments may be ____ that compare students with age or grade expectations.
norm-referenced tests
