test fairness and test bias
what do participants do in the knox cube imitation test
, the administrator holds a cube and taps out a pattern on a set of 4 cubes. The person being tested imitates that pattern. pattterns increase in difficulty
what do participants do in the RPM
- participants view a series (or matrix) of geometric figures and select from several options the one that most accurately completes the series (or matrix).
what is intercept bias most commonly associated with
the notion of test fairness or unfairness
what is the intercept of a regression line
the point at which the regression line intercepts the y axis- about the constant
what is an example of a test that has been developed to take into account the fact that indigenous populations may not have experience with standardised testing
the queensland test- a test of general cognitive ability designed for use for aboriginal australians who have had little contact with westernised urbanised culture
what does intercept bias examine
the relationship between group means on the test and on the criterion
the higher the correlation=
the steeper the line as the slope equals the correlation between test scores (x) and the criterion scores (y)
what does steeper slope for one group indicate
the test has a better validity for predicting criterion scores for one group than another- there is differential validity different outcomes for each group are predicted from the same test
apporpriate tests of any cultural group must involve
the use of culturally fair assessment tools for which performance scores are interpret with respond to normative data established from the same group
what are the issues of test bias and fairness related to
the validity of test scores- if a test doesnt provide the same score for two individuals who have the same level of ability because the test is biased or unfair, the test scores arent an accurate representation of the persons level of ability
when are the predictive characteristics of test scores less likely to vary among cultural groups
then the test is intrinsically relevant to criterion performance
what happens if group a is higher on both test and criterion than group b
there is no bias as group a has a higher ability- reflected by higher scores on the criterion
what is a different way of testing intelligence
through examining other abilities e.g. fluid intelligence
what are many tests of cognitive ability designed for
use in westernised samples; norms are based on westernised samples
what does slope bias refer to
validity coeffficients
when does test makes systematic errors in measure
when bias exists- bias designates cnstant or systematic error as opposed to change error
what is slope bias
when the slope of a regression line for one group is significantly different from another group
what is intercept bias
when the slope of the regression lines for two group is equal but the intercept of a regression line for one group is significantly different from another group
when are the issues of test bias and fairness particularly relevant
when using tests of ability e.g. intelligence tests
who else is the knox cubes imitation test widely used with
with participants who are deaf
is the test equally valid for the two groups in intercept bias
yes but they have different mean test scores- systematically under or overpredicts performance for a specific group if we apply the wrong regression line
when two groups have parrallel lines do they have equal mean criterion ability
yes they have the same mean on the criterion score and thus have equivalent ability
does using appropriate norms correct intercept bias
yes through using appropriate regression line for each group will correct this bias
how are participants proposed to learn how to do the knox imitation test
by imitation tester has to keep repeating the practice item until the subject can do it correctly
what does a difference in means between 2 groups in identical regressions reflect
a real difference between groups in abilities
what is test bias
a statistical term used to designate whether a test or procedure systematically predicts one group in favour of another
what does RPM require regarding knowledge
familiarity with testing producer instructions are pretty standard for those familiar with testing but test-taker who has never sat a formal exam before may find the instructions overwhelming
what is slope bias to do with
how relevant a test is for a particular group
how can cognitive and SEWB assessments contribute to the misdiagnosis of impairment or mental illness
if they are based on foreign concepts and/or symptomatology
what happens if there is a difference in test scores but not criterion scores
intercept bias
what does it mean when two groups have parrallel slopes but different intercepts
intercept bias: the correlation between the test and criterion scores is equal for the two groups however for any given test score there are two different predicted criterion scores depending on which group you belong to
what can influence individuals performance on a test
interpretation of test scores, attitudinal and culturl variables in the home that affect not only test performance but also academic and occupational achievement
example of a subtest on the queensland test
knox cube imitation test
what happens to lines if two groups have the same validity
lines will be parallel- dont have slope bias
impact of testing indigenous individuals using tests with westernised norms
may underestimate their ability- factors other than ability may produce low scores
in the case of the weaker correlation in slope bias, what will there always be more of in predicting criterion score
measurement error
what is identical regressions
no evidence of slope or intercept bias for the two groups- regardless of which group an individual belongs to, you would predict the same score on y (the criterion) given their score of x (the test)
when two groups have parrallel lines do they have equal test scores
no there is a wide disparity in test score means- test is underestimating the ability of the group which has a lower test score mean
with slope bias does using appropriate norms based on the group an individual is from correct the bias
no, as the correlation between test and criterion is different for two group
Testers using the Queensland Test are given instructions on how to behave when testing. For instance, they are advised:
not to touch or pat subjects to watch if subjects are hunting for answers by looking at the tester's facial expressions to start testing the children of the tribe first so that the adults may follow
when the majority grup has higher intercept than minority group, what would use of the majority regression line and cuttoff score for both groups do
overpredict the criterion performance of minority group member sand hence discriminates in favour of the minority group
what is an example of a well-known fluid intelligence test
ravens standard progressive matrices (RPM)
what is regression
the line of best fit through all scores on a bivariate distribution- regression equation is the expression of the line of best fit based on the correlation between the two variables
how many sets of test items does the RPM have
5 sets of test items, each with 12 items sets become progressively mroe difficult
what influences scores on a general knowledge/ crystallised intelligence test
A person's experiences, interests, and cultural background are likely to influence their performance on this test more so than their cognitive ability or intelligence. facts are specialised and so performance of test is not a good indicator or ability but rather how interested the person is in the topics the general facts are testing
nature of the procedure to determine test bias
an objective/ data driven procedure
why are tests of mental functioning problematic for indigenous cultures
as such individuals tend to have poor english literacy, lack of formal education, as well as different concepts of numbers, time and space
why is the RPM considered a culturally reduced test
because it doesnt rely on knowledge of a particular culture
what do test scores in intercept bias correspond to
different criyerion scores, same test score has different predictive meaning for the two groups perform equally well on the criterion but have different mean test scores
what is slope bias also known as
differential validity: if there are different correlations between the test and the criterion for two groups, the test is not equally valid for use in the two groups
which subtest is the knox cube imitation test analogous to
digit span tests that are seen in standard tests of intelligence which require individuals to listen to increasingly long lists of digits and then recall them in
problems with current tests of mental functioning
rely heavily on use of english language require written responses- might be inappropriate for a culture that has no traditional written language and in which rates of english illiteracy can be high resemble formal educational processes: might be uncommon and objectionable to indigenous australians
what can be used to determine bias
scientific methods
what are the two slopes of test bias
slope bias and intercept bias
what does single group validity refer to
slope bias: when validity coefficient reaches statistical signfiicance in one group but fails to do so in another
what does intercept bias relate to
test unfairness
what does slope bias usually mean
that there are different intercepts but if the slopes are very different, the more important form of bias is slope bias as this indicates the test has differential validity
what is test bias usually determined on
the basis of a test's predictive validity- when a test is used to predict a certain criterion or outcome
what does it means if slopes are the same for both groups in regards to error
the correlation between scores on the test and on the criterion are the same for both groups= same amount of inherent error in our prediction- equally valid predictions
how is slope bias indicated
the correlation coefficient being different, as evident by different strengths of slope