Research Methods Exam 2
For her research methods class, Serena plans to interview several teachers about their attitude toward teaching children who have attention-deficit/hyperactivity disorder (ADHD). This is an example of what type of measurement? Self-report measurement physiological measurement archival measurement observational measurement
self-report measurement
if a study uses an unrepresentative sample, which of the following questions should you ask when assessing its external validity "are the characteristics that make the sample biased actually relevant to what is being measure" "is the study making frequency, association, or causal claim?" "is the sample size sufficiently large?" "could the study have used a representative sample instead?"
"are the characteristics that make the sample biased actually relevant to what is being measure"
which of the following is a good reason a researcher may give for using observational methods as opposed to self-report methods? "i do not want to have to worry about ethics" "I do not want to have to worry about the construct validity of my conceptual variable" "I want to measure something that people may not know how often they do it" " I want to make a casual claim"
"I want to measure something that people may not know how often they do it"
"a strong relationship was found. Children with more positive relationships with their parents had higher academic performance than children with less positive relationships with their parents." from this information, you know that the R in the study was closest to +.10 .50 .10 +.50
+.50
another word for observer effects is expectancy effects unobtrusive observation interrater reliability observer bias
expectancy effects
in order to use the known groups paradigm to establish criterion validity, which of the following is necessary? after testing, the groups should have significantly different scores on the measure at least three groups must be used prior to testing, similarity between the groups must be demonstrated the groups must be composed of experts of the field of psychology
after testing, the groups should have significantly different scores on the measure
Which of the following correctly explains the relationship between a moderator and a spurious association? They are different: a moderator indicates the association is not spurious they are different: a moderator only involves one variable, while a spurious association involves both variables they are different: a moderator indicates that the same association exists within subgroups, while a spurious association indicates that the association differ for the subgroups they are the same: a moderator indicates that the association is spurious
They are different: a moderator indicates the association is not spurious
Which of the following is correct about sample size? a sample of 1,000 has the optimal balance of external validity and statistical accuracy a large sample size still needs to be a random sample in order to be generalizable a sample size of 1,000 allows for the most statistically accurate conclusions A large sample size is more representative of the population than a small sample
a large sample size still needs to be random sample in order to be generalizable
Which of the following is NOT possible? A measure is neither reliable nor valid a measure is reliable but not valid a measure is valid by not reliable a measure is both valid and reliable
a measure is valid but not reliable
establishing construct validity is most important for which of the following? abstract concept physical measurements a behavior that is directly observable concrete construct
abstract concept
when examining an association in which one variable is categorical and one is quantitative, which of the following is most likely to be used? curvilinear association an outlier scatterplot bar graph
bar graph
research articles that use terms such as unbiased samples, random samples, or representative sample allow for readers to make a frequency claim reject the conclusions made by the researchers skip interrogating statistical validity be confident in study's external validity
be confident in study's external validity
Dr. Pierre is interested in studying levels of anxiety in children who have lived through major natural disasters. As she plans her study, she is considering various operational definitions of anxiety. Which of the following is an example of an operational definition for anxiety that she could consider? Level of apprehension feeling worried changes in heart rate while viewing images of the aftereffect of a natural disaster state of being uneasy
changes in heart rate while viewing images of the aftereffect of a natural disaster
A correlation-based statistic called ________ is commonly used to determine internal reliability. a scatterplot pearson's r cronbach's alpha kappa
cronbach's alpha
why might a researcher choose purposive sampling over systematic sampling external validity is not vital to the researchers study only purposive sampling allows the researcher to study a particular type of participant the researcher does not have to specify a population of interest ahead of time purposive sampling is always cheaper
external validity is not vital to the researchers study
Fatima is trying to measure gender role stereotypes using gender role survey. she believes that her participants will be able to tell that she is measuring gender role stereotypes because the survey looks like it is measuring stereotypes. Fatima believes that her scale has what type of validity? face criterion discriminant convergent
face
asking an expert or experts to evaluate a measure is used to establish ______ validity criterion content face divergent
face
Forced-choice question formats are especially good at dealing with which of the following issues? fence sitting response sets people self-reporting more than they can know faking good
fence sitting
which of the following determines the construct validity of a survey question? how well it is worded how many response options it has how short it is how many people answer it
how well it is worded
which of the following is true of behavioral observation? it requires a research assistant to be with the participant at all times it requires recording technology, such as video cameras it is a more reliable and valid method than self-report methodology it may tell a different story than the data collected by self-report questions
it may tell a different story than the data collected by self-report questions
which of the following is true of the relationship between effect size and statistical significance? effect size and statistical significance are synonymous terms an association effect size has no effect on statistical significance statistical significance alone is sufficient to indicate effect size larger effect sizes are advantageous for statistical significance
larger effect sizes are advantageous for statistical significance
Martin has found a correlation of r = .18 between the two variables of prescription stimulants and frontal lobe activity. This correlation is more likely to be statistically significant if martin measured frontal lobe activity extremely accurately martin's measure of prescription stimulant use is categorical martin used a larger number of subjects the study can be applied to the real world
martin used a larger number of subjects
what is the relationship between moderators and external validity moderators suggest that associations may not generalize to all subgroups of people moderators suggest that association may be spurious moderators are necessary for external validity to be established moderators suggest that an association between two variables will extend to another variable
moderators suggest that associations may not generlize to all subgroups of people
the difference between a cluster sample and a multistage sample is cluster samples are probability samples, multistage samples are not there is no difference between cluster samples and multistage samples cluster samples rely on clusters of participants, multistage samples collect data from participants at different stages multistage samples sample both clusters and participants; clusters samples just sample clusters
multistage samples sample both clusters and participants; clusters samples just sample clusters
Which of the following statements are correct about the questions format of survey and polls? political polls are typically use Likert scales to ask people which candidate they would vote for researchers compute and average to score a survey with forced-choice questions open-ended questions usually include rich and spontaneous information the format of a question has a larger impact on construct validity than wording
open-ended questions usually include rich and spontaneous information
Zariah placed five identical pairs of black socks on a table and asked passersby to rate which pair of socks were their favorites. Although the socks were exactly the same, people rated the last pair of socks as having the highest quality. How should Zariah interpret this result? people are not always able to accurately explain their responses people were giving socially desirable responses and not being honest Zariah needs to consider the accuracy of flashbulb memories Zariah was mistaken, and the socks actually did differ in quality
people are not always able to accurately explain their responses
Why are convergent and discriminant validity often evaluated together? both terms refer to the same type of validity both types of validity are subjective in that they determined by experts in the field researchers have to look at the patterns of correlations for both types of validity researchers have to determine whether convergent validity predicts discriminant validity
researchers have to look at the patterns of correlations for both types of validity
which of the following is true of question wording? researchers may alter the wording of a question to determine if it does have an affect on the results no research has scientifically demonstrated that question wording affects the answers participants give differences in how questions are worded always lead to different results it has no effect on the results of the survey poll
researchers may alter the wording of a question to determine if it does have an affect on the results
statistical significance depends on which of the following sample size and effect size direction of the association and strength of the association sample size and number of variables analyzed number of outliers and direction of the association
sample size and effect size
faking good is also known as non-differentiation fence sitting socially desirable responding acquiescence
socially desirable responding
in developing a measure of "need for cognition" (the degree to which people like thinking and problem-solving), Dr. Jonason asks his participants to rate their agreement with the following statement. "I frequently solve and enjoy solving crossword puzzles." What should Dr. Jonason do to improve the construct validity of this question? phrase the question in a neutral way avoid using negative phrasing in the question avoid suggesting a particular viewpoint in the question split up the question into two separate questions
split up the question into two separate questions
oversampling is a variant used in which of the following sampling techniques? stratified random sampling cluster sampling convenience sampling simple random sampling
stratified random sampling
a study finds a correlation coefficient r= .52. This number gives you information about which of the following statistical significance and effect size statistical validity and external validity type of relationship and importance strength and direction of the relationship
strength and direction of the relationship
which of the following is an unbiased sample? snowball sample convenience sample purposive sample systematic sample
systematic sample
Which of the following indicates that a study used a bivariate correlational design? the inclusion of quantitative variables the presence of two measured variables the depiction of a bar graph the use of correlational statistics
the presence of two measured variables
Dr. Paul is concerned about a fence sitting response set when he conducts his survey. Which of the following might you recommend to decrease fence sitting using reverse worded questions providing a "no opinion" option using scales with an even number of responses options using a likert scale
using scales with an even number of responses options
how might a researcher deal with reactivity? wear a name tag identifying them as a researcher measure behavior by having each participant wear a camera on their head wait to begin observation use multiple observers
wait to being observation
When using correlation coefficients to evaluate reliability, which of the following is undesirable? a strong correlation coefficient a correlation coefficient close to 1 it depends on the type of reliability being evaluated a negative correlation coefficent
a negative correlation coefficient
which of the following is a poll likely to measure a persons thoughts about whether they prefer advil or tylenol a persons attitude toward their doctor a person opinions about healthcare law a person feelings about people diagnosed with cancer
a person opinions about healthcare law
How are quota sampling and stratified random sampling similar? both identify subgroups that need to be studied both result in representative samples both result in non-representative samples both randomly sample subgroups to be studied
both identify subgroups that need to be studied
in which of the following ways are content and face validity similar? both are very difficult to establish both are preferred by psychologists as ideal measures of validity both involve subjective judgements both involve asking participants for their opinions about the measurement
both involve subjective judgements
Asking many similar questions when trying to measure a concept is done to: make sure participants are not lying cancel out measurement errors allow participants to skip questions ensure validity
cancel out measurement errors
when bivariate association claims do not meet the criteria of temporal precedence and internal validity, this means that _____ cannot be _____ covariance; established construct validity; interrogated hypotheses; tested causal inference; made
causal inference; made
which of the following hypotheses would be best tested with a survey? Children who are exposed to more words per day have faster brain development drinking coffee makes people walk faster sharing with others increases dopamine levels college students with part time jobs have high self-esteem
college students with part time jobs have high self-esteem
what is a term for a researcher's definition of the variable in question at the theoretical level? construct validity conceptual definition operational defintion measurement validity
conceptual definition
if a researchers measure every member of a population, they have collected a sample increased internal validity conducted a census biased the study
conducted a census
the temporal precedence crtierion is also known as the ______ problem covariance association third variable directionality
directionality
when you are interrogating the external validity of a sample, which is the most important question to ask? how many people are in the sample? how many people re in the population? how was the sample collected? how were the participants measured?
how was the sample collected
what information should you consider to examine the external validity of the claim? "two-thirds of American College students sleep less than six hours a day" if the study used probability sampling if the sample includes non-American participants if the sample size is large enough if random assignment was used in the study
if the study used probability sampling
online surveys commonly suffer form which of the following? poor reliability probability sampling over sampling self-selection
self-selection
Jiyun has just conducted a study examining the association between a child's level of impulsivity and the amount of corporal punishment used by their parents. She used a questionnaire about discipline strategies to assess corporal punishment. She calculated the scale's internal reliability as .85 and concluded that her scale had good internal reliability. Jiyun most likely calculated reliability using what statistic? slope of a line the average inter-item correlation (AIC) kappa Cronbach's alpha
Cronbach's alpha
what is the difference between a ratio scale of measurement and an interval scale of measurement? a ratio scale of measurement has a zero value that actually means "nothing" or "the absence of something" but and interval scale does not an interval scale has equal intervals, but a ratio scale does not a ratio scale of measurement cannot be sued to compare people's scores, but interval scales can an interval scale of measurement is.a type of measurement used for categorical measurements, but a ratio scale is used for quantitative measurements
a ratio scale of measurement has a zero value that actually means "nothing" or "the absence of something" but and interval scale does not
a study finds a correlation coefficient of r = .52 and reports 95% CI [.37, .67] the 95% CI is a coefficient estimate confidence estimate confidence interval coefficient interval
confidence interval
if a person is asking whether the variables in an association claim are measured appropriately, what is being interrogated? construct validity external validity internal validity statistical validity
construct validity
In his measure of "need for cognition" (the degree to which people like thinking and problem-solving), Dr. Jonason asks his participants to rate their agreement with the following statement: "I have never not enjoyed thinking." What is the problem with this question? It is a double-barreled questions it has a double negative it is a forced choice question it is a leading question
it has a double negative
For his research methods class, Felipe plans to watch how teachers treat children in their classrooms who have ADHD. He will evaluate how positively or negatively the children are treated. This is an example of what type of measurement? physiological measurement observational measurement self-report measurement archival measurement
observational measurement
why might question order affect how people respond to a survey or poll people may try to appear consistent people are easily confused people are lazy people cannot understand multiple quesitons
people may try to appear consistent
Which of the following is the most direct way to control for question order effects? order effects cannot be controlled for combine multiple questions into single questions give the survey questions to different groups of people prepare different versions of the survey, varying the order of the questions
prepare different versions of the survey, varying the order of the questions
another term for probability sampling is convenience sampling random sampling cluster sampling purposive sampling
random sampling
the construct validity of observations can be threatened by leading questions acquiescence reactivity socially desirable responding
reactivity
which of the following increases accurate responding? nay-saying response sets reverse-worded questions acquiescence fence sitting
reverse worded questions
in a scatterplot, the direction of the relationship can be seen by the slope of the line the presence of a line drawn through the dots spread of the dots number of dots
slope of the line
Dr. Tung is investigating the association between smartphone use and mindless eating. In his first study with a sample size of 25 college students, he finds r= .32. If Dr. Tung hopes to find that college major (education, history) is a moderator, which of the following results should he find? smartphone use is positively associated with mindless eating for education majors, but not for history majors smartphone use is positively linked to mindless eating for both education and history majors education majors engage in mindless eating more than history majors because they use their smartphones more education majors use their smartphones more and mindlessly eat more, compared to history major
smartphone use is positively associated with mindless eating for education majors, but not for history majors
which of the following is true about research using surveys and polls? surveys and polls can efficiently measure people's subjective feelings surveys and polls can support only frequency claims surveys and polls are an accurate way to measure people's actual behaviors surveys and polls utilize only one type of question format
surveys and polls can efficiently measure people's subjective feelings
which of the following is true of moderators they help establish a cause and effect relationship they weaken statistical significance they decrease effect size they can inform external validity
they can inform external validity
which of the following is true about outliers? they have the biggest effect when dealing with small sample sizes they usually affect a majority of the data points they are only problematic when they affect one variable and not the other they can affect the strength , but not the direction, of an association
they have the biggest effect when dealing with small sample sizes
Dr. Jewell is interested in measuring people's attitudes toward proposed tax cuts. One of his items reads, "people who support cutting taxes are not well informed about how the government works" which of the following items should Dr. Jewell use instead ot improve construct validity? "do you agree that tax cuts are an important issue and there should be tax cuts? yes or no?" "most Americans believe that there should not be tax cuts. Please rate your opinions about tax cuts on a ! to 5 scale" "please rate your opinions about tax cuts on a 1 (strongly disagree) to 7 (strong agree) scale" "the government should never raise taxes. Please rate your agreement on a 1 to 7 scale"
"please rate your opinions about tax cuts on a 1 (strongly disagree) to 7 (strong agree) scale"
in which of the following cases would a large sample especially be needed? a study of people who have been to the doctor in the past year a study of high school students a study of teenagers whose parents are both deployed overseas in the military a study of first time homeowners
a study of teenagers whose parents are both deployed overseas in the military
your friend Alanna says that when examining validity, you always want to see positive correlations. Why is she wrong? correlations are not used to examine validity the strength of a correlation matters, but the direction of a correlation does not matter negative correlations are desirable when examining validity both the strength and the direction of a correlation matter when examining validity
both the strength and the direction of a correlation matter when examining validity
Naomi is studying the effect of popularity on academic success for her research methods project. To do this, she has elementary school students rate how popular each member of their class is. She then uses this information to rank the students on popularity (e.g., John is the most popular, Vanessa is the second-most popular). Which of the following best describes this variable? a self-report measurement an ordinal scale measurement an interval scale measurement a categorical measurement
ordinal scale measurement
If an association study did not select people for the study by using random sampling, which of the following statements is true? the study must be done again using the same participants the findings should be replicated in another population the association should be rejected as inconclusive the effect size should be considered, but tests of statistical significance should not
the findings should be replicated in another population
in which of the following studies is self-report the best data collection option? a study examining the intensity of pain during natural child birth a study examining the health of children born via natural child birth a study examining discussions about natural childbirth between mothers and doctors a study examining the breathing styles used during natural childbirth
a study examining the intensity of pain during natural child birth
observer bias relates mainly to _________, whereas observer effects stem from _______ outcomes; expectations researchers: participants faking good; faking bad validity; reliability
researchers; participants
a correlation coefficient and a scatterplot both provide which of the following pieces of information? the strength and direction of the relationship between two measurements the outliers present in the two measurements the validity and reliability of the two measurements the path and significance of the relationship between two measurements
the strength and direction of the relationship between two measurements
Hosea is studying the relationship between caffeine consumption and problem-solving ability. Which of the following is a quantitative way to operationalize problem-solving ability? The type of puzzle solved whether the participants used insight or trial-and-error techniques to solve the problem the time spent solving a math problem the report of a teacher about whether a student is a good or bad problem solver
the time spent solving a math problem
When evaluating the external validity of an association claim, which of the following is the most important issue to consider? the size of the original population the number of subgroups the size of the sample the way the sample was selected from the population
the way the sample was selected from the population
Dr. Sanchez conducts a simple random sample of 500 men who became fathers for the first time in the past year. He finds that 23% of them report being unsure of their ability to be good fathers, plus or minus 4%. If Dr. Sanchez increased his sample size to 1,000, which of the following would happen external validity would become less important the margin of error would become smaller the true estimate would increase statistical validity would become negatively affected
true estimate would increase???????
while reading about a research study, which of the following would tell you that an association claim is being made two measured variables two manipulated variables two categorical variables one manipulated and measured variable
two measured variables
Having a representative sample is most important in which of the following example claims? "receiving weekly feedback from your supervisor increases work productivity" "forty-three percent of psychology majors report being frustrated by people asking them if they are psychoanalyzing them" "people who report knowing someone who has been diagnosed with skin cancer also report having greater sunscreen" "having a dark triad personality is associated with having greater relationship problems"
"forty-three percent of psychology majors report being frustrated by people asking them if they are psychoanalyzing them"
a researcher wants to know what people really do, not what they think they do. Which method would you advise him to use? neither behavioral observations nor self-report questions in desirable for finding out what people really do both behavioral observation and self-report questions are equally good for finding out what people really do behavioral observation self-report questions
behavioral observation
when interrogating the external validity of a sample, which is the most important question to ask? how many people are in the population how many people are in the sample how were the participants measured how was the sample collected
how was the sample collected
shoppers often leave reviews for products that they purchased online. Which of the following best describe the external validity of product reviews on an online shopping site? the external validity is high because this is a randomly selected sample of online shoppers the external validity is low because this is a self-selected sample of online shoppers the external validity is high because online shoppers are more likely to be honest about their opinions the external validity is low because online shoppers are more likely to change their mind
the external validity is low because this is a self-selected sample of online shoppers
Two researchers tell you they study the same thing. However, when you look at their research papers, they do not use similar methods or measurements. How is this possible? the researchers have the same conceptual definitions and operational definitions the researchers have the same operational definitions the researchers have the same conceptual definitions the researchers do not have the same conceptual or operational definition
the researchers have the same conceptual definitions
For a third variable to be plausible as the explanation in an established association, which of the following must also be true? third variable must be related to both of the measured variables in the original association third variable must be measured on the same scale as the original measured variables third variable must have a positive relationship with the two measured variables in the original association third variable must be a categorical variable
third variable must be related to both of the measured variables in the original association
Studies that use non-probability samples have _____ external validity zero guaranteed enhanced unknown
unknown
if a sample is biased, then it is_______ the population of interest unrepresentative of unrelated to unfairly applied to incorrectly compared to
unrepresentative of
a biased sample consists of too many _______ samples ideal unusual basic complicated
unusual
which of the following may lead to a biased sample? using people who agree to participate using people who are readily available to the researcher using people who accept compensation to participate using people who have participated in other research studies
using people who are readily available to the researcher
the study that found that two-thirds of American experienced at least one adverse childhood experience (ACE) did not use random sample. What conclusion should you make about the results of the study? we are uncertain about the generalizability of the results we can generalize the results of the study only to Americans with adverse childhood experiences We can generalize the results of the study to all Americans The results are incorrect because the study did not use a random sample
we are uncertain about the generalizability of the results
Hosea is studying the relationship between caffeine consumption and problem-solving ability. Which of the following is a categorical way to operationalize caffeine consumption? Whether the participant drank coffee in the 24 hours prior to the study the number of milligrams of caffeine consumed during the study the frequency of buying energy drinks the number of cups of coffee consumed in a day
whether the participants drank coffee in the 24 hours prior to the study
which of the following is true of a non-representative sample in a research claim you should ask whether more participants are necessary you should automatically accept the claim you should ask whether it is relevant to what the researchers are measuring you should automatically disregard the claim
you should ask whether it is relevant to what the researchers are measuring
