Inquizative 16 EXAM 4

Identify the true statements about replicating studies.

- A study that has been replicated is more credible than a study that has not been replicated.

The Müller-Lyer illusion is a visual illusion in which lines with arrows angling out (B) look longer than lines with arrows angling in (A). Researchers examined the Müller-Lyer illusion across several different cultural contexts. Which of the following statements are lessons we can learn from this research? *two lines with arrows pointing inwards or outwards*

- Cross-cultural studies provide strong evidence for external validity. - People living in areas without buildings are less susceptible to the illusion.

Identify the false statements about replicating studies.

- If a study gets significant results, it does not need to be replicated. - Conceptual replications are less useful than direct replications in assuring that a finding is reliable. - A direct replication reproduces every single detail of the original study.

Horselenberg and colleagues (2003) conducted an experiment based on Kassin and Kieche (1996) in which participants were falsely accused of striking a computer key that caused important data to be deleted. The first study showed that 69% of the participants were willing to sign a confession that they had struck the key. In the later study, researchers changed the procedure so that in addition to signing the confession, participants were asked to give up part of their compensation for participating in the study. Identify the *false* statements about what the researchers could have done to achieve certain goals in their study.

- If they wanted to strengthen external validity, the researchers could have increased the number of participants in the study. - If they wanted to strengthen construct validity, the researchers could have conducted several different replication studies.

Identify the false statements about popular media and replicability.

- Popular media articles usually indicate whether a study is replicable.

Identify the true statements about transparent and questionable research practices.

- Questionable research practices go against Merton's norms of communality and disinterestedness. - Researchers should refrain from conducting statistical tests that they did not originally plan to conduct.

Which of the following scenarios are examples of questionable research practices?

- Researchers create new hypotheses after collecting the data. - Researchers add more participants to their study after discovering a result that is not significant.

Not questionable research practices

- Researchers freely share the data and materials from their study online. - Researchers conduct a study with a large sample.

Identify the true statements about direct replication studies.

- Researchers often value other types of replication studies as supplements to direct replication studies. - Direct replication studies cannot duplicate the original study in every single detail.

Identify the true statements about popular media and replicability.

- Responsible journalists should include a summary of the entire body of literature on a topic. - Journalists typically focus on the results of a single new study.

Identify the false statements about transparent and questionable research practices.

- Responsible researchers modify their hypotheses before publication if they notice the results are not what they predicted. - Reporting every single analysis conducted on every single variable can be misleading.

Horselenberg and colleagues (2003) conducted an experiment based on Kassin and Kieche (1996) in which participants were falsely accused of striking a computer key that caused important data to be deleted. The first study showed that 69% of the participants were willing to sign a confession that they had struck the key. In the later study, researchers changed the procedure so that in addition to signing the confession, participants were asked to give up part of their compensation for participating in the study. Identify the *true* statements about what the researchers could have done to achieve certain goals in their study.

- The original study was conducted in the United States. If they wanted to strengthen external validity, the researchers could have conducted their replication study in the Netherlands. - If they wanted to increase experimental realism, the researchers could have made the lab situation closely parallel false confessions of a crime.

Zareen reads a psychology journal article with these badges. Based on the badges, identify the conclusions that can be drawn about the article. - open materials / preregistered

- The researchers conducted only the analyses that they had originally planned to conduct. - The article reports all of the variables in the study, even those that did not produce significant results.

Identify the false statements about direct replication studies.

- Threats to internal validity from the original study do not apply to direct replications. - Direct replication studies use the same variables as the original study but operationalize them in different ways.

preregistered

HARKING - Preregistration, in which researchers post their hypotheses and plans for a study prior to collecting data, can address HARKing. Because their hypotheses are posted online with a time-stamp, researchers cannot change them after they obtain their results.

Sora is examining whether more hours of sleep and higher quality of sleep decrease stress in college students. She asks her participants to report their level of stress, the number of hours of sleep the night before, quality of sleep the night before, and other demographic variables. Match each questionable research practice to the corresponding scenario that might occur in Sora's study. underreporting null findings / p-hacking / HARKing - After looking at her results, Sora decides to restate her hypothesis as "more sleep increases stress."

HARKing

Anita is conducting a study on sleep and video games. When she begins her study, she is not sure whether playing video games causes people to lose sleep, or whether people often turn to video games when they cannot sleep. Once she analyzes her data, she discovers that it is more likely that participants were losing sleep because they were staying up playing video games. After realizing this, Anita removes any mention of the idea that people who can't sleep turn to video games from her paper before submitting it for publication. What questionable research practice is Anita engaging in?

HARKing - Anita removes one hypothesis and emphasizes another based on her results. This can majorly skew how others interpret the results. Anita should have reported her first hypothesis and said that it was not supported, while still discussing her new ideas.

Researchers conducted several replication studies surrounding a paradigm initially developed by Deese (1959). With this paradigm, when participants are given a list of words to recall that has to do with a particular word but does not include it, they tend to falsely recall this word, even though it was not on the list. For example, if participants are given a list that includes the words sandal, towel, sunscreen, and waves, then they would falsely remember the word beach even though it was not on the list. Identify the type of replication that was conducted in each study - direct replication / replication-plus-extension / conceptual replication - The researchers used a different list of words from the original study.

conceptual replication

direct replication / replication-plus-extension / conceptual replication - A scientist attempts to replicate the better-than-average effect with niche talents such as playing musical instruments, rather than in the classroom.

conceptual replication

Horselenberg and colleagues (2003) conducted an experiment based on Kassin and Kieche (1996) in which participants were falsely accused of striking a computer key that caused important data to be deleted. The first study showed that 69% of the participants were willing to sign a confession that they had struck the key. In the later study, researchers changed the procedure so that in addition to signing the confession, participants were asked to give up part of their compensation for participating in the study. What type of replication is the second study? - direct replication - meta-analysis - replication-plus-extension - conceptual replication

conceptual replication - Horselenberg and colleagues examined the same research question as the original study, but they used different procedures.

Researchers conducted several replication studies surrounding a paradigm initially developed by Deese (1959). With this paradigm, when participants are given a list of words to recall that has to do with a particular word but does not include it, they tend to falsely recall this word, even though it was not on the list. For example, if participants are given a list that includes the words sandal, towel, sunscreen, and waves, then they would falsely remember the word beach even though it was not on the list. Identify the type of replication that was conducted in each study - direct replication / replication-plus-extension / conceptual replication - The researchers used the same list of words and the same procedure as the original study.

direct replication

direct replication / replication-plus-extension / conceptual replication - A team of researchers recreates a previous experiment on persuasion exactly as it was written in the original publication.

direct replication

If a researcher wants to reproduce a previous study, she can do so in multiple ways. A(n) _______ involves re-creating previous work as closely as possible to the original. Scientists can also conduct a(n) _______, in which a study uses different procedures to re-create a similar finding to the original. Finally, a(n) ______ involves copying the original experiment and including additional variables to answer new questions.

direct replication / conceptual replication / replication-plus-extension

Some studies are conducted in a real-world situation, or a(n) ____. These studies are often high in _____ validity: they are likely to be encountered in everyday life. Even though some researchers believe real-world settings are superior, many others claim that laboratory studies can have _____ --- in other words, even though the study is taking place in a controlled environment, it still evokes behavior and emotions that are realistic.

field setting / ecological / experimental realism

Theory-testing mode or generalization mode? A researcher is recreating a previous study from liberal arts colleges at large public universities.

generalization mode

Theory-testing mode or generalization mode? Two researchers are attempting to replicate a finding from the United States in Japan.

generalization mode

open data

p hacking - Open data, in which researchers share their full datasets, can address p-hacking. If everyone can see a dataset, they can rerun analyses on it to confirm results.

Sora is examining whether more hours of sleep and higher quality of sleep decrease stress in college students. She asks her participants to report their level of stress, the number of hours of sleep the night before, quality of sleep the night before, and other demographic variables. Match each questionable research practice to the corresponding scenario that might occur in Sora's study. underreporting null findings / p-hacking / HARKing - After looking at her results, Sora decides to control for several new variables, including gender.

p-hacking

Sora is examining whether more hours of sleep and higher quality of sleep decrease stress in college students. She asks her participants to report their level of stress, the number of hours of sleep the night before, quality of sleep the night before, and other demographic variables. Match each questionable research practice to the corresponding scenario that might occur in Sora's study. underreporting null findings / p-hacking / HARKing -Sora's results are almost significant, so she removes two participants who slept only 2 hours.

p-hacking

Researchers conducted several replication studies surrounding a paradigm initially developed by Deese (1959). With this paradigm, when participants are given a list of words to recall that has to do with a particular word but does not include it, they tend to falsely recall this word, even though it was not on the list. For example, if participants are given a list that includes the words sandal, towel, sunscreen, and waves, then they would falsely remember the word beach even though it was not on the list. Identify the type of replication that was conducted in each study - direct replication / replication-plus-extension / conceptual replication - The researchers administered the words one at a time to some participants, while administering them all at once to others.

replication-plus-extension

direct replication / replication-plus-extension / conceptual replication - A researcher is recreating an experiment on study habits and adding in ethnicity as a new factor to see if there are any differences.

replication-plus-extension

Besides replication, another way to increase the strength or significance of a particular finding is to review the ____ about that finding and conduct a study that compiles the results of similar studies into one statistic. This type of study is called a(n) _____, and the statistic typically calculated in it is a(n) ____.

scientific literature / meta analysis / effect size

Dr. Okafor thinks most people are generally trusting of strangers. She conducts a study at her university in the United States to test this idea, and the results confirm her hypothesis. In this case, Dr. Okafor is conducting her study in ____ mode. Later, Dr. Okafor is discussing the results of her study with Dr. Zhu, a psychologist in China. Dr. Zhu thinks people in East Asian contexts, such as China, are less trusting of strangers than are people in North American contexts, such as the United States. Dr. Okafor and Dr. Zhu decide to run a second study in which they have a group of Chinese participants and a group of American participants complete the task from Dr. Okafor's original study. In this second study, Dr. Okafor and Dr. Zhu are working in _______ mode. The second study is a _____ study in the area of _____ psychology.

theory testing / generalization / replication-plus-extension / cultural

Theory-testing mode or generalization mode? A lab is conducting a series of studies to understand the effects of emotion on the written word.

theory-testing mode

Theory-testing mode or generalization mode? Researchers design a study to understand what, specifically, compels people to help others in emergency situations.

theory-testing mode

When a researcher attempts to test specific, nuanced aspects of a hypothesis or theory, this process is called _______ , and the researcher typically prioritizes _______ validity. When a researcher prioritizes ________ validity in an attempt to apply her findings to a larger population, this process is called ___________.

theory-testing mode /internal/ external / generalization mode --Researchers operating in theory-testing mode prioritize internal validity in order to rule out alternative explanations as they investigate their theory. Researchers operating in generalization mode prioritize external validity in order to apply their findings to other populations and settings.

open materials

underreporting null effects - Openly sharing all the data and materials from a study can address underreporting null effects. If other people can see the full study design and materials, they can check that all conditions and variables—not just the significant ones—were reported.

Sora is examining whether more hours of sleep and higher quality of sleep decrease stress in college students. She asks her participants to report their level of stress, the number of hours of sleep the night before, quality of sleep the night before, and other demographic variables. Match each questionable research practice to the corresponding scenario that might occur in Sora's study. underreporting null findings / p-hacking / HARKing - Sora finds that stress is not associated with quality of sleep, so she does not include it in her results.

underreporting null findings