test fairness and test bias

what do participants do in the knox cube imitation test

, the administrator holds a cube and taps out a pattern on a set of 4 cubes. The person being tested imitates that pattern. pattterns increase in difficulty

what do participants do in the RPM

- participants view a series (or matrix) of geometric figures and select from several options the one that most accurately completes the series (or matrix).

what is intercept bias most commonly associated with

the notion of test fairness or unfairness

what is the intercept of a regression line

the point at which the regression line intercepts the y axis- about the constant

what is an example of a test that has been developed to take into account the fact that indigenous populations may not have experience with standardised testing

the queensland test- a test of general cognitive ability designed for use for aboriginal australians who have had little contact with westernised urbanised culture

what does intercept bias examine

the relationship between group means on the test and on the criterion

the higher the correlation=

the steeper the line as the slope equals the correlation between test scores (x) and the criterion scores (y)

what does steeper slope for one group indicate

the test has a better validity for predicting criterion scores for one group than another- there is differential validity different outcomes for each group are predicted from the same test

apporpriate tests of any cultural group must involve

the use of culturally fair assessment tools for which performance scores are interpret with respond to normative data established from the same group

what are the issues of test bias and fairness related to

the validity of test scores- if a test doesnt provide the same score for two individuals who have the same level of ability because the test is biased or unfair, the test scores arent an accurate representation of the persons level of ability

when are the predictive characteristics of test scores less likely to vary among cultural groups

then the test is intrinsically relevant to criterion performance

what happens if group a is higher on both test and criterion than group b

there is no bias as group a has a higher ability- reflected by higher scores on the criterion

what is a different way of testing intelligence

through examining other abilities e.g. fluid intelligence

what are many tests of cognitive ability designed for

use in westernised samples; norms are based on westernised samples

what does slope bias refer to

validity coeffficients

when does test makes systematic errors in measure

when bias exists- bias designates cnstant or systematic error as opposed to change error

what is slope bias

when the slope of a regression line for one group is significantly different from another group

what is intercept bias

when the slope of the regression lines for two group is equal but the intercept of a regression line for one group is significantly different from another group

when are the issues of test bias and fairness particularly relevant

when using tests of ability e.g. intelligence tests

who else is the knox cubes imitation test widely used with

with participants who are deaf

is the test equally valid for the two groups in intercept bias

yes but they have different mean test scores- systematically under or overpredicts performance for a specific group if we apply the wrong regression line

when two groups have parrallel lines do they have equal mean criterion ability

yes they have the same mean on the criterion score and thus have equivalent ability

does using appropriate norms correct intercept bias

yes through using appropriate regression line for each group will correct this bias

how are participants proposed to learn how to do the knox imitation test

by imitation tester has to keep repeating the practice item until the subject can do it correctly

what does a difference in means between 2 groups in identical regressions reflect

a real difference between groups in abilities

what is test bias

a statistical term used to designate whether a test or procedure systematically predicts one group in favour of another

what does RPM require regarding knowledge

familiarity with testing producer instructions are pretty standard for those familiar with testing but test-taker who has never sat a formal exam before may find the instructions overwhelming

what is slope bias to do with

how relevant a test is for a particular group

how can cognitive and SEWB assessments contribute to the misdiagnosis of impairment or mental illness

if they are based on foreign concepts and/or symptomatology

what happens if there is a difference in test scores but not criterion scores

intercept bias

what does it mean when two groups have parrallel slopes but different intercepts

intercept bias: the correlation between the test and criterion scores is equal for the two groups however for any given test score there are two different predicted criterion scores depending on which group you belong to

what can influence individuals performance on a test

interpretation of test scores, attitudinal and culturl variables in the home that affect not only test performance but also academic and occupational achievement

example of a subtest on the queensland test

knox cube imitation test

what happens to lines if two groups have the same validity

lines will be parallel- dont have slope bias

impact of testing indigenous individuals using tests with westernised norms

may underestimate their ability- factors other than ability may produce low scores

in the case of the weaker correlation in slope bias, what will there always be more of in predicting criterion score

measurement error

what is identical regressions

no evidence of slope or intercept bias for the two groups- regardless of which group an individual belongs to, you would predict the same score on y (the criterion) given their score of x (the test)

when two groups have parrallel lines do they have equal test scores

no there is a wide disparity in test score means- test is underestimating the ability of the group which has a lower test score mean

with slope bias does using appropriate norms based on the group an individual is from correct the bias

no, as the correlation between test and criterion is different for two group

Testers using the Queensland Test are given instructions on how to behave when testing. For instance, they are advised:

not to touch or pat subjects to watch if subjects are hunting for answers by looking at the tester's facial expressions to start testing the children of the tribe first so that the adults may follow

when the majority grup has higher intercept than minority group, what would use of the majority regression line and cuttoff score for both groups do

overpredict the criterion performance of minority group member sand hence discriminates in favour of the minority group

what is an example of a well-known fluid intelligence test

ravens standard progressive matrices (RPM)

what is regression

the line of best fit through all scores on a bivariate distribution- regression equation is the expression of the line of best fit based on the correlation between the two variables

how many sets of test items does the RPM have

5 sets of test items, each with 12 items sets become progressively mroe difficult

what influences scores on a general knowledge/ crystallised intelligence test

A person's experiences, interests, and cultural background are likely to influence their performance on this test more so than their cognitive ability or intelligence. facts are specialised and so performance of test is not a good indicator or ability but rather how interested the person is in the topics the general facts are testing

nature of the procedure to determine test bias

an objective/ data driven procedure

why are tests of mental functioning problematic for indigenous cultures

as such individuals tend to have poor english literacy, lack of formal education, as well as different concepts of numbers, time and space

why is the RPM considered a culturally reduced test

because it doesnt rely on knowledge of a particular culture

what do test scores in intercept bias correspond to

different criyerion scores, same test score has different predictive meaning for the two groups perform equally well on the criterion but have different mean test scores

what is slope bias also known as

differential validity: if there are different correlations between the test and the criterion for two groups, the test is not equally valid for use in the two groups

which subtest is the knox cube imitation test analogous to

digit span tests that are seen in standard tests of intelligence which require individuals to listen to increasingly long lists of digits and then recall them in

problems with current tests of mental functioning

rely heavily on use of english language require written responses- might be inappropriate for a culture that has no traditional written language and in which rates of english illiteracy can be high resemble formal educational processes: might be uncommon and objectionable to indigenous australians

what can be used to determine bias

scientific methods

what are the two slopes of test bias

slope bias and intercept bias

what does single group validity refer to

slope bias: when validity coefficient reaches statistical signfiicance in one group but fails to do so in another

what does intercept bias relate to

test unfairness

what does slope bias usually mean

that there are different intercepts but if the slopes are very different, the more important form of bias is slope bias as this indicates the test has differential validity

what is test bias usually determined on

the basis of a test's predictive validity- when a test is used to predict a certain criterion or outcome

what does it means if slopes are the same for both groups in regards to error

the correlation between scores on the test and on the criterion are the same for both groups= same amount of inherent error in our prediction- equally valid predictions

how is slope bias indicated

the correlation coefficient being different, as evident by different strengths of slope