Research Methods Chapter 5
known-groups paradigm
whether scores on the measure can distinguish among a set of groups whose behavior is already confirmed
quantitative variables
variables coded with meaningful numbers
Of the following validities, what would the researcher likely rank as most important? •Convergent validity •Criterion validity •Face validity •Discriminant validity
Criterion validity•
A researcher finds that BDI (Beck Depression Inventory) scores (higher score = more symptoms of depression) and a survey of psychological well-being (higher score = higher well-being) have a correlation coefficient of -.65. What is the conclusion?•Evidence of good convergent validity• Evidence of good discriminant validity •Evidence of poor convergent validity •Evidence of poor internal reliability
Evidence of good convergent validity•
interrater reliability
consistent scores are obtained no matter who measures or observes
test-retest reliability
consistent scores every time use the measure (self report, observational, and physiological)
self-report measure
operationalize by recording people's answers to questions about themselves in questionnaires or interviews
Georgina graduated as valedictorian of her high school class because of her class ranking. What type of scale is used for the quantitative variable of class ranking? nominal scale interval scale ratio scale ordinal scale
ordinal scale
what are the types of quantitative variables?
ordinal scale, interval scale, and ratio scale
what are the three types of reliability?
test-retest, interrater, and internal
ordinal scale
ranked order ex. rank in a race
Using Correlation Coefficient (r) to Evaluate Reliability
1. Test-retest reliability -0.5 or higher -Only relevant to variables that do not change over time 2. Interrater reliability -0.7 or higher 3. Internal reliability -Cronbach's alpha: correlation-based statistic to evaluate internal reliability
correlation coefficient (r)
A single number describing how close the dots on a scatterplot are to the line drawn through them
RESEARCH STUDY 5.1: Dr. Rodriquez is considering conducting a study examining whether narcissistic people have poorer romantic relationships than those who are not narcissistic. One of her first tasks is to determine which of her participants are narcissistic and which are not. She decides to use the scale created by a colleague, the Donalscale. Question 1 reads, "I tend not to think about other people as much as I think about myself." Question 2 reads, "I do not have a high opinion of myself." Question 3 reads, "I think other people think I am really special."•Before using the measure in her study, Dr. Rodriquez gives the measure to a group of participants on the first day of the semester and again on the last day of the semester. Dr. Rodriquez is examining the scatterplot of the data she collected on the first day of the semester and the last day of the semester. On the scatterplot, she sees that the dots are very close to forming a diagonal line. This indicates which of the following?• a.A strong relationship• b.A nonrelationship• c.A valid finding• d.A negative finding
A strong relationship
RESEARCH STUDY 5.1: Dr. Rodriquez is considering conducting a study examining whether narcissistic people have poorer romantic relationships than those who are not narcissistic. One of her first tasks is to determine which of her participants are narcissistic and which are not. She decides to use the scale created by a colleague, the Donalscale. Question 1 reads, "I tend not to think about other people as much as I think about myself." Question 2 reads, "I do not have a high opinion of myself." Question 3 reads, "I think other people think I am really special."•Dr. Rodriquez is concerned about the validity of the measure of narcissism recommended by her colleague. She sends a copy of the measure to the faculty members in her psychology department to look at, and they all tell her it looks like it will measure narcissism. She now has evidence of which of the following?• a.Content validity •b.Face validity• c.Discriminant validity• d.Concurrent validity
Face validity•
For her research methods class, Serena plans to interview several teachers about their attitude toward teaching children who have attention-deficit/hyperactivity disorder (ADHD). This is an example of what type of measurement?• a.Self-report measurement• b.Observational measurement• c.Physiological measurement• d.Archival measurement
Self-report measurement•
RESEARCH STUDY 5.1: Dr. Rodriquez is considering conducting a study examining whether narcissistic people have poorer romantic relationships than those who are not narcissistic. One of her first tasks is to determine which of her participants are narcissistic and which are not. She decides to use the scale created by a colleague, the Donalscale. Question 1 reads, "I tend not to think about other people as much as I think about myself." Question 2 reads, "I do not have a high opinion of myself." Question 3 reads, "I think other people think I am really special."•Dr. Rodriquez is concerned whether her measure will really measure narcissism or if it will measure some other related concept. She is concerned about the scale's .• a.operational definition •b.validity •c.reliability •d.convenience
b.validity•
Which scale of measurement would eye color require? •A. categorical• B. ordinal• C. interval• D. ratio
categorical
Josiane has found an online test that claims to measure IQ. It consists of choosing the correct definitions for a series of words. She is concerned that it doesn't include tests of other things that are part of IQ, such as problem solving or visual-spatial ability. Which type of validity is she questioning? content validity face validity criterion validity discriminant validity
content validity
Some colleges no longer require the SAT I or the ACT tests; instead, they base their admissions on other factors, such as high school GPA. A major reason they have done this is they have found a low correlation between the scores on the tests and the students' freshman-year GPA. In other words, they were concerned that college entrance exams lacked which type of validity? face validity criterion validity discriminant validity content validity
criterion validity
Dr. Nolan gives his new anxiety measure to a group of his colleagues who are anxiety experts. They agree that the questions on the measure appear to assess anxiety symptoms. This suggests that Dr. Nolan's measure has which of the following types of measurement validity? content validity face validity criterion validity discriminant validity
face validity
Reliability
is it necessary, but not sufficient for validity
Dr. Kamran studies domestic violence and has designed a self-report scale that is meant to assess men's negative attitudes toward women. To validate her scale, she administers it to two groups of recently incarcerated male prisoners: prisoners convicted of domestic violence and prisoners convicted of other crimes. Dr. Kamran finds a statistically significant difference in the mean scores of the two groups. Which technique is Dr. Kamran using to validate her scale? test-retest technique physiological measurements known-groups paradigm interrater reliability test
known-groups paradigm
categorical variables
levels that are qualitatively distinct categories
In a study of aggression in children, a researcher has his undergraduate research assistants watch a group of children on the playground, and record the number of instances of physical or verbal attacks. Which category of measured variable is this researcher using? self-report measures observational measures physiological measures neuropsychological measures
observational measures
physiological measure
operationalize by recording biological data ex. record muscle movement using EMG
observational measure
operationalize by recording observable behaviors ex. recording the number of times someone smiles
Lorenzo is studying aggression in children. He administers a questionnaire to the children that asks them about their feelings of aggression. Which type of measure is the questionnaire? ordinal scale observational physiological self-report
self-report
Dr. Johnson wants to investigate whether the physiological measure, heart rate variability, varies over time or whether it is a trait that stays stable within the same person over time. He records participants' heart rate variability once at the beginning of the semester and once at the end of the semester. He finds a high positive correlation (r = .65) between the first and second time points. Which type of reliability is he examining? test-retest interrater internal construct
test-retest
The Department of Motor Vehicles (DMV) receives a complaint that some of their employees who administer the road test pass a much higher percentage of test takers than other employees. In this example, what aspect of the road test is being questioned? the interrater reliability of the road test the internal reliability of the road test the test-retest reliability of the road test the measurement validity of the road test
the interrater reliability of the road test
content validity
the measure contains all parts that your theory says it should contain ( subjective) ex. intelligence includes reason and plan so it should assess both
RESEARCH STUDY 5.1: Dr. Rodriquez is considering conducting a study examining whether narcissistic people have poorer romantic relationships than those who are not narcissistic. One of her first tasks is to determine which of her participants are narcissistic and which are not. She decides to use the scale created by a colleague, the Donalscale. Question 1 reads, "I tend not to think about other people as much as I think about myself." Question 2 reads, "I do not have a high opinion of myself." Question 3 reads, "I think other people think I am really special."•Before using the measure in her study, Dr. Rodriquez gives the measure to a group of participants on the first day of the semester and again on the last day of the semester. She then compares the scores between the two time points. This is a test of which of the following?• a.convergent validity• b.Internal reliability• c.Test-retest reliability• d.Construct reliability
•c.Test-retest reliability
ratio scale
numerals represent equal intervals and there is a true zero ex. exam scores, you can say "answered twice as many problems correctly"
what are the two scales of measurement?
categorical variables and quantitative variables
face validity
it looks like what you want to measure (subjective) ex. vocab size
RESEARCH STUDY 5.1: Dr. Rodriquez is considering conducting a study examining whether narcissistic people have poorer romantic relationships than those who are not narcissistic. One of her first tasks is to determine which of her participants are narcissistic and which are not. She decides to use the scale created by a colleague, the Donalscale. Question 1 reads, "I tend not to think about other people as much as I think about myself." Question 2 reads, "I do not have a high opinion of myself." Question 3 reads, "I think other people think I am really special."•Before using the measure in her study, Dr. Rodriquez analyzes the data she gets from her students. She looks at the relationship between each of the individual questions. She sees that participants who agree with Question 1 also agree with Question 3 and disagree with Question 2. This is a test of which of the following? •a.Interrater reliability• b.Internal reliability• c.convergent validity •d.discriminant validity
Internal reliability
convergent validity
a self report measure should correlate more strongly with self-report of similar constructs (not actual behaviors so not criterion validity)
RESEARCH STUDY 5.1: Dr. Rodriquez is considering conducting a study examining whether narcissistic people have poorer romantic relationships than those who are not narcissistic. One of her first tasks is to determine which of her participants are narcissistic and which are not. She decides to use the scale created by a colleague, the Donalscale. Question 1 reads, "I tend not to think about other people as much as I think about myself." Question 2 reads, "I do not have a high opinion of myself." Question 3 reads, "I think other people think I am really special."•Dr. Rodriquez decides to test the internal reliability of her measure. Which of the following results would make her happy? •a.Cronbach's alpha = 0.10 •b.Cronbach's alpha = −0.03 •c.Cronbach's alpha= 0.85• d.Cronbach's alpha= −0.98
c.Cronbach's alpha= 0.85
correlations evidence for criterion validity
measure is associated with a concrete behavioral outcome that it should be associated with
Hosea is studying the relationship between caffeine consumption and problem-solving ability. Which of the following is a categorical way to operationalize caffeine consumption? •a.The number of cups of coffee consumed in a day •b.The number of milligrams of caffeine consumed during the study •c.The frequency of buying energy drinks• d.Whether the participant drank a soda in the 24 hours prior to the study
whether the participant drank a soda in the 24 hours prior to the study
discriminant validity (divergent validity)
measure should correlate less strongly with other measures of different constructs
interval scale
numerals represent equal distances between levels and there is no true zero ex. iq , you cannot say "twice as intelligent"
Which scale of measurement would degree of pain felt, as rated on a 10-point scale require? •A. categorical• B. ordinal• C. interval •D. ratio
ordinal and. interval•
internal reliability
participant provides consistent pattern of responses, regardless of how the researcher has phrased the question (only for self-report scales)
Dr. Johnson wants to investigate whether the physiological measure, heart rate variability, varies over time or whether it is a trait that stays stable within the same person over time. He records participants' heart rate variability once at the beginning of the semester and once at the end of the semester. He finds a high positive correlation (r = .55) between the first and second time points. Which of the following would a scatterplot of these results (heart rate variability at the beginning of the semester on the x-axis, heart rate variability at the end of the semester on the y-axis) look like? The cloud of points would have no slope at all. The cloud of points would slope upward from left to right. The cloud of points would slope downward from left to right. There isn't enough information given to determine this.
The cloud of points would slope upward from left to right.
