PSYC 344 - CH. 5

RESEARCH STUDY 5.1: Dr. Rodriquez is considering conducting a study examining whether narcissistic people have poorer romantic relationships than those who are not narcissistic. One of her first tasks is to determine which of her participants are narcissistic and which are not. She decides to use the scale created by a colleague, the Donal scale. Question 1 reads, "I tend not to think about other people as much as I think about myself." Question 2 reads, "I do not have a high opinion of myself." Question 3 reads, "I think other people think I am really special." Dr. Rodriquez decides to test the internal reliability of her measure. Which of the following results would make her happy?

=0.95

Which of the following is NOT possible?

A measure is valid but not reliable.

Establishing construct validity would probably be most important for which of the following?

A measure of spirituality

When using correlation coefficients to evaluate reliability, which of the following is undesirable?

A negative correlation coefficient

In looking at a scatterplot of interrater reliability, why would a researcher want to see all the dots close to the line of agreement?

Because it indicates that the researcher's two research assistants/raters are making similar measurements

RESEARCH STUDY 5.2: Dr. Sheffield is a clinical psychologist who specializes in treating pathological gambling. Pathological gambling is defined as being unable to resist impulses to gamble. Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition, he creates a new measure of pathological gambling. The measure has 15 questions, and it takes 20 minutes to complete. Dr. Sheffield decides to test the criterion validity of his measure. Dr. Sheffield gives his measure to a group of people that includes suspected problem gamblers and non-gamblers. Which of the following options below could he also do to get evidence for criterion validity?

Correlate the measure with a behavior, such as amount of money lost in a casino during the past year

According to its conceptual definition, a variable should be related to a particular behavior. If a researcher is able to demonstrate that his measure of the variable is related to the behavior, then he has established which of the following?

Criterion validity

Which of the following is NOT an example of physiological measurement?

Number of panic attacks a patient reports

RESEARCH STUDY 5.2: Dr. Sheffield is a clinical psychologist who specializes in treating pathological gambling. Pathological gambling is defined as being unable to resist impulses to gamble. Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition, he creates a new measure of pathological gambling. The measure has 15 questions, and it takes 20 minutes to complete. Dr. Sheffield wants to establish the discriminant validity of his pathological gambling measure. He gives his measure and three others to a group of 100 people. Which of the following provides the best evidence for discriminant validity?

That his measure is not strongly correlated with a measure of impulsivity

Asking many similar questions when trying to measure a concept is done to:

cancel out measurement error.

Todd is studying the effect of popularity on academic success for his research methods project. He decides to measure popularity by asking each elementary school student to tell him how many friends he or she has. He assumes that more friends means the student is more popular. Which of the following best describes this variable?

A ratio scale of measurement

What is the difference between a ratio scale of measurement and an interval scale of measurement?

A ratio scale of measurement has a zero value that actually means "nothing" or "the absence of something," but an interval scale does not.

In order to use the known groups paradigm to establish criterion validity, which of the following is necessary?

After testing, the groups should have significantly different scores on the measure.

Establishing construct validity is most important for which of the following?

An abstract concept

Naomi is studying the effect of popularity on academic success for her research methods project. To do this, she has elementary school students rate how popular each member of their class is. She then uses this information to rank the students on popularity (e.g., John is the most popular, Vanessa is the second-most popular). Which of the following best describes this variable?

An ordinal scale of measurement

Why are convergent and discriminant validity often evaluated together?

Both involve collecting information from a lot of psychological measures of theoretical interest.

What does it mean that "reliability is necessary but not sufficient for validity"?

If a measure is valid, it is also reliable.

Which types of reliability can be analyzed with scatterplots?

Interrater reliability and test-retest reliability

For his research methods class, Felipe plans to watch how students treat other children in their classrooms who have attention-deficit/hyperactivity disorder (ADHD). He will evaluate how positively or negatively the children are treated by their classmates. This is an example of what type of measurement?

Observational measurement

Two researchers tell you they study the same thing. However, when you look at their research papers, they do not use similar methodologies or measurements. How is this possible?

The researchers have the same conceptual definitions.

Which of the following is true of operational definitions?

The specification of operational definitions is one of the creative aspects of the research process.

A correlation coefficient and a scatterplot both provide which of the following pieces of information?

The strength and direction of the relationship between two measurements

Hosea is studying the relationship between caffeine consumption and problem-solving ability. Which of the following is a quantitative way to operationalize problem-solving ability?

The time spent solving a math problem

RESEARCH STUDY 5.1: Dr. Rodriquez is considering conducting a study examining whether narcissistic people have poorer romantic relationships than those who are not narcissistic. One of her first tasks is to determine which of her participants are narcissistic and which are not. She decides to use the scale created by a colleague, the Donal scale. Question 1 reads, "I tend not to think about other people as much as I think about myself." Question 2 reads, "I do not have a high opinion of myself." Question 3 reads, "I think other people think I am really special." Dr. Rodriquez calculates a correlation coefficient (r) to examine the relationship between Question 1 and Question 2 and between Question 1 and Question 3. She finds a correlation coefficient of r = −0.73 between Questions 1 and 2 and a correlation coefficient of r = 0.74 between Questions 1 and 3. Which of the following is true of her findings?

There appears to be good internal reliability in the scale.

RESEARCH STUDY 5.2: Dr. Sheffield is a clinical psychologist who specializes in treating pathological gambling. Pathological gambling is defined as being unable to resist impulses to gamble. Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition, he creates a new measure of pathological gambling. The measure has 15 questions, and it takes 20 minutes to complete. To test his measure, Dr. Sheffield gives his measure to a group of people in Gamblers Anonymous (GA) and another group in Alcoholics Anonymous (AA). He finds that people in the GA group have higher scores on his new measure than people in the AA group. Why did Dr. Sheffield do this?

To obtain evidence for criterion validity

Hosea is studying the relationship between caffeine consumption and problem-solving ability. Which of the following is a categorical way to operationalize caffeine consumption?

Whether the participant drank a soda in the 24 hours prior to the study

RESEARCH STUDY 5.2: Dr. Sheffield is a clinical psychologist who specializes in treating pathological gambling. Pathological gambling is defined as being unable to resist impulses to gamble. Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition, he creates a new measure of pathological gambling. The measure has 15 questions, and it takes 20 minutes to complete. Dr. Sheffield gives his measure to his supervisor, who is also an expert in pathological gambling. His supervisor says that his measure appears to test all the components of pathological gambling, including feeling restless when attempting to stop gambling, jeopardizing jobs in order to keep gambling, and using gambling to escape from problems and a bad mood. Given this information, Dr. Sheffield's measure has evidence of which of the following?

content validity

Your friend Dominic is complaining about having to take the Graduate Record Examination (GRE), a test that is required to go to graduate school and is similar to the ACT and SAT. He complains, "Tests like the GRE don't really measure how well people actually do in graduate school." Dominic is questioning the __________________ of the test.

criterion validity

Another word for discriminant validity is ___________________ validity.

divergent

RESEARCH STUDY 5.2: Dr. Sheffield is a clinical psychologist who specializes in treating pathological gambling. Pathological gambling is defined as being unable to resist impulses to gamble. Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition, he creates a new measure of pathological gambling. The measure has 15 questions, and it takes 20 minutes to complete.To test his measure, Dr. Sheffield gives his measure to a group of people in Gamblers Anonymous (GA) and another group of people in Alcoholics Anonymous (AA). He finds that people in the GA group have higher scores on his new measure than people in the AA group. This procedure is known as a:

known-groups paradigm.

How many subcategories of quantitative variables exist?

three

In which of the following ways are content and face validity similar?

Both involve subjective judgments.

Your friend Alanna says that when examining validity, you always want to see positive correlations. Why is she wrong?

Both the strength and the direction of a correlation matter when examining validity.

What is the term for a researcher's definition of the variable in question at a theoretical level?

Conceptual definition

RESEARCH STUDY 5.2: Dr. Sheffield is a clinical psychologist who specializes in treating pathological gambling. Pathological gambling is defined as being unable to resist impulses to gamble. Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition, he creates a new measure of pathological gambling. The measure has 15 questions, and it takes 20 minutes to complete. To test his measure, Dr. Sheffield gives his measure to a group of his clients and at the same time measures how many times they have been gambling in the past month. He predicts that clients who score higher on his measure will also report gambling more times in the past month. This procedure is meant to provide evidence for which of the following?

Criterion validity

A correlation-based statistic called _________________________ is commonly used to determine internal reliability.

Cronbach's alpha

In interrogating the construct validity of a measure, which question should a researcher ask?

Is there enough evidence that this measure is valid?

RESEARCH STUDY 5.1: Dr. Rodriquez is considering conducting a study examining whether narcissistic people have poorer romantic relationships than those who are not narcissistic. One of her first tasks is to determine which of her participants are narcissistic and which are not. She decides to use the scale created by a colleague, the Donal scale. Question 1 reads, "I tend not to think about other people as much as I think about myself." Question 2 reads, "I do not have a high opinion of myself." Question 3 reads, "I think other people think I am really special." Before using the measure in her study, Dr. Rodriquez gives the measure to a group of participants on the first day of the semester and again on the last day of the semester. She then compares the scores between the two time points. This is a test of which of the following?

Test-retest reliability

RESEARCH STUDY 5.1: Dr. Rodriquez is considering conducting a study examining whether narcissistic people have poorer romantic relationships than those who are not narcissistic. One of her first tasks is to determine which of her participants are narcissistic and which are not. She decides to use the scale created by a colleague, the Donal scale. Question 1 reads, "I tend not to think about other people as much as I think about myself." Question 2 reads, "I do not have a high opinion of myself." Question 3 reads, "I think other people think I am really special." Before using the measure in her study, Dr. Rodriquez gives the measure to a group of participants on the first day of the semester and again on the last day of the semester. Dr. Rodriquez is examining the scatterplot of the data she collected on the first day of the semester and the last day of the semester. On the scatterplot, she sees that the dots are very close to forming a diagonal line. This indicates which of the following?

a strong relationship

Asking an expert or experts to evaluate a measure is used to establish ________ validity.

face

An educational psychologist is testing the discriminant validity of a new measure of numerical learning difficulties. He gives his measure to a group of students along with another measure of verbal learning difficulties, which he predicts should not be strongly related to numerical learning difficulties. Which of the following correlations would the psychologist hope to find in order to establish discriminant validity?

r = −0.18

A helpful tool for visualizing test-retest reliability and interrater reliability is a:

scatterplot

RESEARCH STUDY 5.2: Dr. Sheffield is a clinical psychologist who specializes in treating pathological gambling. Pathological gambling is defined as being unable to resist impulses to gamble. Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition, he creates a new measure of pathological gambling. The measure has 15 questions, and it takes 20 minutes to complete. Dr. Sheffield has decided to test the discriminant validity of his new measure. He has a group of first-time Gamblers Anonymous (GA) attendants complete his measure and finds that they score higher than a group of people who do not attend the group. Which of the following results would provide evidence for discriminant validity?

He finds that the measure of gambling is not correlated with a measure of life satisfaction in the same two groups of people.

For her research methods class, Serena plans to interview several teachers about their attitude toward teaching children who have attention-deficit/hyperactivity disorder (ADHD). This is an example of what type of measurement?

Self-report measurement

RESEARCH STUDY 5.2: Dr. Sheffield is a clinical psychologist who specializes in treating pathological gambling. Pathological gambling is defined as being unable to resist impulses to gamble. Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition, he creates a new measure of pathological gambling. The measure has 15 questions, and it takes 20 minutes to complete. Dr. Sheffield has now decided that he wants to test his measure on some university students (who some estimates say have a 6% prevalence rate of compulsive gambling). He has a group of 100 university students complete his measure. He also has them complete two other measures (one that measures addictive behavior in general and one that measures general attitudes toward gambling). He finds that his new measure is positively associated with each of these other measures. This procedure has provided evidence for the ____________________________ of Dr. Sheffield's measure.

convergent validity

RESEARCH STUDY 5.1: Dr. Rodriquez is considering conducting a study examining whether narcissistic people have poorer romantic relationships than those who are not narcissistic. One of her first tasks is to determine which of her participants are narcissistic and which are not. She decides to use the scale created by a colleague, the Donal scale. Question 1 reads, "I tend not to think about other people as much as I think about myself." Question 2 reads, "I do not have a high opinion of myself." Question 3 reads, "I think other people think I am really special." Dr. Rodriquez is concerned about the validity of the measure of narcissism recommended by her colleague. She sends a copy of the measure to the faculty members in her psychology department to look at, and they all tell her it looks like it will measure narcissism. She now has evidence of which of the following?

face validity

If a measurement looks like it is a plausible operationalization of a conceptual variable, then it has:

face validity.

RESEARCH STUDY 5.1: Dr. Rodriquez is considering conducting a study examining whether narcissistic people have poorer romantic relationships than those who are not narcissistic. One of her first tasks is to determine which of her participants are narcissistic and which are not. She decides to use the scale created by a colleague, the Donal scale. Question 1 reads, "I tend not to think about other people as much as I think about myself." Question 2 reads, "I do not have a high opinion of myself." Question 3 reads, "I think other people think I am really special." Before using the measure in her study, Dr. Rodriquez analyzes the data she gets from her students. She looks at the relationship between each of the individual questions. She sees that participants who agree with Question 1 also agree with Question 3 and disagree with Question 2. This is a test of which of the following?

internal reliability

RESEARCH STUDY 5.1: Dr. Rodriquez is considering conducting a study examining whether narcissistic people have poorer romantic relationships than those who are not narcissistic. One of her first tasks is to determine which of her participants are narcissistic and which are not. She decides to use the scale created by a colleague, the Donal scale. Question 1 reads, "I tend not to think about other people as much as I think about myself." Question 2 reads, "I do not have a high opinion of myself." Question 3 reads, "I think other people think I am really special." Dr. Rodriquez is concerned whether her measure will really measure narcissism or if it will measure some other related concept. She is concerned about the scale's .

validity

RESEARCH STUDY 5.2: Dr. Sheffield is a clinical psychologist who specializes in treating pathological gambling. Pathological gambling is defined as being unable to resist impulses to gamble. Bothered by not having a good measure that he can give to clients to determine whether they are suffering from this condition, he creates a new measure of pathological gambling. The measure has 15 questions, and it takes 20 minutes to complete. If Dr. Sheffield's measure does not actually measure pathological gambling, his measure is said to lack which of the following?

validity