Chapter 3: Assessment, Effect Size and Ethics

Objective Test

A personality test that consists of a list of questions to be answered by the subject as True or False, Yes or No, or along a numeric scale (e.g., 1 to 7).

In the context of psychological research, what does the word "significant" mean?

A significant result is one that is unlikely to occur by chance. A "significant" result, in the research parlance, is one that is unlikely to result if everything were due to chance.

factor analysis

A statistical technique for finding clusters of related traits, tests, or items.

What is the basic assumption of the empirical method of test creation?

Certain kinds of people have distinctive ways of answering certain questions on personality inventories. In an empirically derived personality test, groups of participants are independently divided into the groups that psychologists wish to study

The Binomial Effect Size Display (BESD) is a way to think about the size of

Correlation coefficients The BESD provides another way to think about the size of correlation coefficients. It is a concrete display of what a correlation means in terms of specific outcomes.

Type II Error

In research, the mistake of thinking that one variable does not have an effect on or relationship with another, when really it does.

Type I Error

In research, the mistake of thinking that one variable has an effect on, or relationship with, another variable, when really it does not.

p-level

In statistical data analysis, the probability that the obtained correlation or difference between experimental conditions would be expected by chance.

Why do objective tests typically have so many test items or questions?

One way to make a test more reliable is simply to make it longer. Single answers tend to be unreliable (reliability being taken in the statistical sense). If a group of similar questions is asked, then aggregating their answers ought to be much more stable, or reliable, because random fluctuations tend to cancel each other out.

The Thematic Apperception Test (TAT) is an example of which type of test?

Projective The TAT asks clients to tell stories about a set of drawings of people and ambiguous events.

Most personality tests provide what kind of data?

S data Most tests ask you about what you are like, so the score you receive amounts to a summary of how you have described yourself. Most tests yield S data, although many others yield B data.

Three of the following four sentences about effect size are true. Which is FALSE?

The BESD shows how much of an outcome we can predict from an individual measurement of difference. The BESD provides an alternative way to examine effect size. Squaring the correlation coefficient provides a tenuous and unreliable measure of effect size.

Projective tests face the same drawbacks as other B-data tests, including that

The answer and the validity of the answer depend critically on the test interpreter. The fundamental difficulty with projective tests is that a psychologist cannot really be sure what a participant's answers mean; two different interpreters of the same response might come to different conclusions about it. (An additional drawback to projective tests is that, like other B-data tests, gathering data is relatively expensive.) Answer a is a disadvantage of I-data testing, and answers b and c are each disadvantages of S-data tests.

Content Validity

The degree to which an assessment instrument, such as a questionnaire, includes content obviously relevant to what it is intended to predict.

Correlation coefficients are measured between a range of +1 and −1. Which of the following is true regarding correlation coefficients?

The further the number is from zero in either direction, the stronger is the effect. If two variables are unrelated, the correlation between them will be close to zero. If two variables are positively associated, then the correlation coefficient will be greater than zero. If two variables are negatively associated, then the correlation coefficient will be less than zero. The further the number is from zero in either direction, the stronger is the effect.

For any rationally constructed, S-data personality test to work, four conditions must hold. Which of the answers below is NOT one of these vital conditions?

The tests must be applicable to participants of any gender, race, age, or linguistic group. For a rationally constructed test to measure accurately an attribute of personality, all of the conditions in answers must be met, but in fact, most rationally constructed tests fail to meet one or more of these criteria.

Often a scale is used to predict behavior, diagnosis, or category membership in new samples of participants. This process is referred to as

cross-validation If cross-validation of a scale succeeds, the scale is deemed ready to use.

If you wish to compare how two groups of people answer differently on a personality test, which method of test construction would be ideal?

empirical The basic assumption behind the empirical approach is that certain kinds of people have distinctive ways of answering certain questions on personality inventories.

What are the Big Five personality traits that were derived from the factor analytic method?

extraversion, neuroticism, conscientiousness, agreeableness, and openness The Big Five personality traits are extraversion, neuroticism, conscientiousness, agreeableness, and openness. These represent the traits that most psychologists agree are the fundamental traits of the human personality.

A researcher shows you various cards with pictures on them and asks you to tell a story about the scene and people in the pictures. What kind of test is this?

projective personality test Personality tests in which a participant is presented with ambiguous stimuli, designed to let her answer freely with open-ended responses, are projection tests.

The factor analytic method of test construction is an example of a psychological tool based on

statistics. The factor analytic method of test construction is an example of a psychological tool based on statistics.

What are the "factors" that the factor analytic method is designed to identify?

the properties that make groups of things seem to be alike Factor analysis is designed to identify groups of things, such as test items, that seem to be alike. The property that makes these things alike is called a factor.

What is the purpose of institutional review boards (IRBs) at research universities?

to review the procedures of all research experiments and ensure that they comply with ethical guidelines set by the federal government or the APA The American Psychological Association has developed a detailed set of ethical guidelines that psychological researchers are supposed to follow. Research universities also all have institutional review boards (IRBs) that review the ethics of procedures conducted with human participants. Ask yourself if Stanley Milgram's experiments would be allowed at research institutions today.

Which of the following sentences describes a Type I error in significance testing?

A Type I error involves deciding that one variable has a relationship with another, when really it does not. A Type I error involves deciding that one variable has a relationship with another when really it does not. A Type II error involves deciding that one variable does not have an effect on another variable when really it does.

Which of the following sentences describes a Type II error in significance testing?

A Type II error involves deciding that one variable does not have a relationship with another when really it does. A Type II error involves deciding that one variable does not have a relationship with another when really it does. A Type I error involves deciding that one variable has a relationship with another, when really it does not.

Binomial Effect Size Display (BESD)

A method for displaying and understanding more clearly the magnitude of an effect reported as a correlation, by translating the value of r into a 2 3 2 table comparing predicted with obtained results.

correlation coefficient

A number between 21 and 11 that reflects the degree to which one variable, traditionally called y, is a linear function of another, traditionally called x. A negative correlation means that as x goes up, y goes down; a positive correlation means that as x goes up, so does y; a zero correlation means that x and y are unrelated.

effect size

A number that reflects the degree to which one variable affects, or is related to, another variable.

If a test consists of a list of questions that can be answered yes or no, true or false, or on a numeric scale, and especially if the test uses a computer-scored answer sheet, then it is what kind of test?

An objective test The term "objective test" comes from the idea that the questions making up the test seem more objective and less open to interpretation than the pictures and blots used in projective tests; it is not clear, however, if the term "objective" is really justified.