Chapter 2-Research Methods
If data collected in a study do not confirm a hypothesis, which of the following is likely to happen?
The researchers will change the theory a little bit, and test the revised theory in a new study
Institutional Review Board (IRB)
A committee at each institution where research is conducted to review every experiment for ethics and methodology. must approve the plan
p-hacking
A family of questionable data analysis techniques, such as adding participants after the results are initially analyzed, looking for outliers, or trying new analyses in order to obtain a p value of just under .05, which can lead to non replicable results
Statistical significance
A process of inference that applies rules of logic and probability to estimate whether the results obtained in a study's sample are the same in a larger population low p-value: doesn't mean hypothesis is true just means the data are unlikely to occur under the null hypothesis high p-value: does not mean hypothesis is false, it means the data is fairly likely to occur until the null hypothesis
measured variables
Asking people how happy they are on a scale, 1 (extremely unhappy) to 10 (extremely happy) scale and record their answers Gender is always a measured variable also depression, ethnicity, or disease status is almost always measured
You read a headline on your newsfeed that states, "Study finds that women are better drivers than men." You find the original study and read that the study was conducted on a sample of 16-year-olds who just earned their drivers' licenses and is based on how high they scored on the written and driving portions of their exams. Which of the following questions would be related to your concern about the study's external validity? How big is the difference between the scores or male and female drivers? Can the results of the study say something about all drivers? Can you really measure "good driving" based on how well they did on these two tests? Could the sex difference in god driving be related to whether girls are more likely to study and prepare for these exams?
Can the results of the study say something about all drivers
A developmental psychologist is conducting a study on children's interest in attending college. Which of the following variables could a researcher measure but not manipulate? Grade point average, Participation in an afterschool program, # of books read this week, Exposure to college posters in the classroom
GPA
3 questions to access validity
How well did the researchers operationalize the variables? Are the people they studied representative of the population of interest? Can we rule out the most plausible alternative explanations?
Saying that a finding is "statistically significant" means that it Is unlikely to have happened by chance if the null hypothesis is true, had been replicated, represents a large effect, is the result of a meta-analysis
Is unlikely to have happened by chance if the null hypothesis is true
Preregistration is an important aspect of high-quality science because
It allows researchers to publicize their hypotheses before collecting data
Which of the following explains why conclusions provided by psychological science are better than those reached by people's intuition or personal experience?
Psychological science systematically tests ideas using controlled methods and multiple people Psychological science collects information systematically and from multiple people, overcoming any one person's bias or overconfidence.
Reliability
consistency of measurement ex: put flour on a scale, if you get the same weight each time with the same act of flour it is reliable
Variability statistic
described by the spread of its sampling distribution which the scores in a batch differ from one another
three types of research to test theories
descriptive, correlational, experimental
Sensitivity
does your measure detect meaningful differences ex: can you measure a fake smile vs a real smile
internal validity
extent to which we can draw cause-and-effect inferences from a study and rule our alternative explanations also 3rd criterion for deciding whether a study can support a casual claim ex: alternative could be students who took notes by hands were already better students than those who took notes on laptops--if better students also tended to take notes by hands, their greater abilities would be an alternative explanation for why they seem to learn more rule out alternatives and internal validity by performing a well designed experiment
external validity
extent to which we can generalize findings to real-world settings major issue for a lot of research ex: if you find results regarding mice can you generalize them to humans
manipulated variable
factor in an experiment that a scientist purposely changes; also known as independent variable
controlled group
group in an experiment that is exposed to the same conditions as the experimental group except for one independent variable
When looking at graphs presented in the popular press, which of the toolkit questions might you need to especially focus on? How strongs is the result? What am I being asked to believe? Are they making a causal claim? Has the study been replicated?
how strong is the result
harking
hypothesizing after results are known
random assignment
in which a random method is used to decide which participants will be in which group. (ex. Picking out of a hat) only used in experiments
Ruling out alternative explanations, or confounds, enhances which aspect of a study?
internal validity
self-report
people's answers to questions about the topic of interest themselves usually through a survey or interview
After discussing the original correlational study examining social media usage and life satisfaction, you and your roommate have decided to create an experiment to determine if there is a causal link between the two variables. You have chosen to manipulate how many minutes people spend using social media by creating three groups (0 minutes, 15 minutes, and 30 minutes on social media per day) and then measure people's life satisfaction. Which of the following is necessary to ensure that the three groups are equally satisfied with life at the beginning of the study? Multiple experimental groups, Making the experiment "blind" to participants and researchers, a Control group, Random Assignment
random assignment
random sampling or assignment or neither The American Automobile Association (AAA) conducted a survey using a random-digit phone-dialing system to call a national sample of 2,000 U.S. drivers. Drivers were asked whether they had ever talked on a cell phone while driving. Did this study use random sampling or random assignment?
random sampling
using the 3rs for using animals for research
replacement: find alternatives if possible refinement: modify experimental procedures and other aspects of animal care to minimize or eliminate animal distress reduction: adopt designs and procedures that require the fewest animal subject
d statistic
represents differences between groups, useful for comparing research results across multiple studies. .5=avg response of the 2 groups different by half a standard deviation .2=small .5=medium .8=large
what are important when drawing conclusions rather on relying on experience or intuition especially if life or death is on the line
scientific methods
the stronger the correlation is ....
the better the predictions will be
Validity
the extent to which a test measures or predicts what it is supposed to the appropriateness or accuracy of some claim or conclusion
construct validity
the extent to which variables measure what they are supposed to measure
Operationalization and state 3 types
the process of assigning a precise method for measuring a term being examined for use in a particular study assigning a method to a variable you are trying to measure self-report, direct observation, technology
descriptive statistics
these statistics summarize participants' differing responses in terms of what was most typical and how much people's responses varied from the average
placebo effect
In an experiment, a group or condition in which people expect to receive a treatment but are exposed only to an inert version, like a sugar pill
experimental group
In an experiment, the group that is exposed to the treatment, that is, to one version of the independent variable.
sample
Smaller group of participants in the research (sample is selected from the larger group known as the population of interest)
Variable
Something of interest that can vary from person to person or situation to situation can be measured or manipulated
Scientific Method Steps
Theory-data cycle. theory, hypothesis, design, data, compare
underreporting nonsignificant effects
This practice becomes a problem if, in writing about the research, the researcher only reports the variables that showed strong effects, and does not mention the others
double-blind procedure
This technique, in which both participants and the researcher who has contact with them are unaware of condition assignment → having participants blind to their condition assignment ensure they act spontaneously
Technology
Used to operationalize variables that are not easily observable by researchers or participants themselves ex: recording a persons BAC
psychologists strive to _______
isolate other causes ex: separating the effect of talking on the phone from the factors that could influence driving
Measured or Manipulated In a study on phobias, a researcher hooks people up to a skin conductance device to record their stress levels while they view photos. Some participants see photos of spiders, some see snakes, and some see puppies. The type of photo is
manipulated
Measured or Manipulated Researchers have participants solve anagrams. Some participants solve difficult anagrams, some solve easy anagrams, and some solve medium-difficulty anagrams. Anagram difficulty is
manipulated
Measured or Manipulated The researcher dyes some people's hair black and other people's hair blond. Hair color is
manipulated
independent variable
manipulated variable
dependent variable
measured variable
Measured or Manipulated The researcher stands at an intersection and records whether or not drivers are using cell phones. Driver behavior is
measured
random sampling or assignment or neither A research team in Utah placed trained observers at a local intersection with a four-way stop sign. The observers recorded two pieces of information about each driver: whether the driver stopped legally behind the stop sign, and whether the driver was using a hand-held cell phone while driving. Did this study use random sampling or random assignment?
neither
which operationalizations are better
observational and physiological because self-report may not be completely accurate
naturalistic observation
observing and recording behavior in naturally occurring situations without trying to manipulate and control the situation
A researcher decides to measure the variable of how self-focused people are by counting the number of times a person says "I" or "me" in a 10-minute recorded conversation. This specification of how this variable will be measured is known as which of the following? Observational measurement, Measured variable, Quantification, Operational definition
operational definition
Inferential Statistics
use sample results to infer what is true about the broader population A set of procedures used to estimate whether a pattern of results represents a true relationship or difference in the population ex: If we see a difference between two conditions for example is it in fact a reliable difference, is it real, or is it due to chance We use inferential statistics to differentiate patterns that are likely due to chance to that that are likely to occur
experience
using experience to draw conclusions about behavior of others ex: you have drove while talking on the phone and never got into an accident ex: dad has bluetooth and never got into an accident problem: our personal experiences do not show us all possible events, can be more than one explanation...cant base our answer just on this
descriptive research
what do people do? focus on one measured variable at a time with the goal of describing what is typical measuring how people typically think, feel, or behave cannot test relationships among variables can measure many variables
correlational research
what kinds of people do this? the study of the naturally occurring relationships among variables ex:The republicans are engaging in the self-reporting less than the Democrats based on their beliefs -This is an association (correlation) between political beliefs and how much you're doing something (engaging in protective behaviors) measuring how variables are related to one another cannot identify casual direction of the relationship can measure many variables
in order to conclude one variable causes another you have to satisfy what 3 criteria
1. The two variables have to be correlated or associated with each other 2. We must know for certain which variable came first in time -third variable: something that causes them to be related but not one of the two main variables 3. no reasonable alternative explanation for the pattern
meta-analysis
A process in which researchers locate all of the studies that have tested the same variables and mathematically average them to estimate the effect size of the entire body of studies
preregistration
A researcher's public statement of a study's expected outcome before collecting any data Pre Registration allows others to verify that findings are not the results of Harking or p-hacking and that all variables are reported
A cognitive psychologist hypothesizes that people who exercise more will also have better memories (e.g., be able to recall more words). Which of the following scatterplot patterns would provide support for her hypothesis?
Dots that slope upward from left to right (more exercise = better memory)
In a field experiment, Middlemis, Knowles, and Matter (1976) wanted to test how an invasion of personal space might affect people's physiological responses. They set up an experiment on a college campus in a men's public restroom with three urinals. When men entered the restroom to use a urinal, there was a confederate in the restroom who, by random assignment, used the urinal either directly next to the man who entered or the one two urinals away (they used an "out of order" sign on the remaining urinal). The researchers hypothesized that men would take longer to begin urinating when someone was standing directly next to them. To measure the delay in urination, another researcher hid in a bathroom stall and, using a mirrored periscope focused on the target's midsection, recorded how long the man took to start urinating. The observer did not see the faces of the participants. None of the 60 users of the restroom were aware they had been participants in an experiment. As hypothesized, when the confederate was standing in the neighboring urinal, men took longer to begin to urinate, compared with when the confederate was standing two urinals away. Does this reflect beneficence?
Having someone stand next to them in the bathroom certainly seemed to make the participants uncomfortable, but there is a reasonable chance of this occurring regardless of the experiment.
Researchers wanted to know whether seeing positive news online makes people feel more positive emotions. They manipulated 700,000 Facebook accounts by withholding certain positive news posts from some users' feeds for a few hours and withholding certain negative posts from others' feeds. Later, they counted how many positive and negative emotion words people used in their status messages. They found that people used fewer positive emotion words when their positive news had been temporarily withheld and used fewer negative emotion words when negative posts had been temporarily withheld (Kramer, Guillory, & Hancock, 2014). Facebook users consent to having their accounts analyzed when they sign up for an account, so the researchers did not ask people for consent to participate in this study. Does this study reflect the ethical principle of autonomy?
In this study, participants did not consent to participate, so one could argue that this violated their autonomy. However, because users on Facebook agreed to the terms of service agreement, and this study was conducted by Facebook itself, others say the company had no obligation to obtain informed consent.
Researchers wanted to know whether seeing positive news online makes people feel more positive emotions. They manipulated 700,000 Facebook accounts by withholding certain positive news posts from some users' feeds for a few hours and withholding certain negative posts from others' feeds. Later, they counted how many positive and negative emotion words people used in their status messages. They found that people used fewer positive emotion words when their positive news had been temporarily withheld and used fewer negative emotion words when negative posts had been temporarily withheld (Kramer, Guillory, & Hancock, 2014). Facebook users consent to having their accounts analyzed when they sign up for an account, so the researchers did not ask people for consent to participate in this study. Does this study reflect the ethical principle of beneficence?
It is possible that increased negative emotions in someone's daily life may have serious negative consequences. However, the knowledge gained here about the emotional effects of social media is extremely valuable to the scientific field.
Population of Interest
Larger set of individuals that the researcher is trying to understand
You and your roommate are discussing the results of a self-report study that measured the number of minutes spent on social media during a person's first week of college and their satisfaction with life during their fourth week of college. Which of the following changes could be made to the study to change it from a correlational study to an experimental study? Manipulating how many minutes people spend on social media during the first week of college, Measuring both social media usage and life satisfaction at both time points, Collecting the social media usage data with observational methods rather than self-report methods, Adding other measured/dependent variables (ex. Satisfaction with college)
Manipulating how many minutes people spend on social media during the first week of college
random sampling
Randomly selecting people to be their sample tested can be used in any study--a study, observational study, correlational study, or an experiment
In a field experiment, Middlemis, Knowles, and Matter (1976) wanted to test how an invasion of personal space might affect people's physiological responses. They set up an experiment on a college campus in a men's public restroom with three urinals. When men entered the restroom to use a urinal, there was a confederate in the restroom who, by random assignment, used the urinal either directly next to the man who entered or the one two urinals away (they used an "out of order" sign on the remaining urinal). The researchers hypothesized that men would take longer to begin urinating when someone was standing directly next to them. To measure the delay in urination, another researcher hid in a bathroom stall and, using a mirrored periscope focused on the target's midsection, recorded how long the man took to start urinating. The observer did not see the faces of the participants. None of the 60 users of the restroom were aware they had been participants in an experiment. As hypothesized, when the confederate was standing in the neighboring urinal, men took longer to begin to urinate, compared with when the confederate was standing two urinals away. Does this reflect the ethical principle of autonomy?
Participants did not consent to participation in the study, so their autonomy seems to have been violated. Arguably, however, this study could not have been conducted accurately if informed consent was granted.
Which type of data would be better collected by observational methods than by self-reported methods? The number of times a person has donated blood this year, A person's attitude toward helping strangers, The emotion a person feels when seeing a person in distress, The amount of time it takes to help a person pick up dropped papers
The amount of time it takes to help a person pick up dropped papers
confound
The experiment is said to have a confound when the experimental groups accidentally differ on more than just the independent variable Ex. A group drinks alc and orange juice and the other is water, the confound = not know whether the higher aggression was caused by the alcohol or by the taste or ingredients in the orange juice
You read a headline on your newsfeed that states, "Study finds that women are better drivers than men." To consider this study's construct validity, you would want information about which of the following? The # of males and females in the study, The operational definition of "good driver", the size of the difference between men and women's "good driving", and whether the study's author was male or female
The operational definition of "good driver"
Which of the following relationships between a sample and a population of interest would allow us to have confidence in the results of a survey?
The people in the sample closely resemble the people in the population
open science
The practice of sharing one's data, materialis, analysis plans, and published articles freely so others can collaborate, use, verify, and learn about the results
You and your roommate are discussing the results of a study that found a negative relationship between the number of minutes people spend on social media during a person's first week of college and their satisfaction with life during their fourth week of college. Which criterion for causation prevents you from concluding that social media usage causes lower life satisfaction? It is unclear which variable comes first, There may be an alternate explanations for the relationship between social media usage and life satisfaction, There is not a correlation between the social media usage and life satisfaction, and It is unclear who the people in the original study were
There may be an alternate explanations for the relationship between social media usage and life satisfaction
standard deviation
a computed measure of how much scores vary around the mean score
Scatterplot
a graphed cluster of dots, each of which represents the values of two variables shows correlational studies
false positive
a statistically significant finding that does not reflect a real effect A study with a larger sample will have greater power to detect an effect that actually exists and reduces the risk of finding false positives that do not replicate in later studies
frequency distribution
an arrangement of data that indicates how often a particular score or observation occurs
intuition
an effortless, immediate, automatic feeling or thought, as contrasted with explicit, conscious reasoning use logic, a subjective feeling about what makes sense ex: hands free is the way to go because if you have your hands on the wheels your eyes will be on the road and you are fine problem: ideas "feel right" due to us being overconfidence in what we know and how well we think we understand events...but these feelings are not always accurate easy to see other faults in intuition but difficult to accept fault in our own
case study
an observation technique in which one person is studied in depth of a persons experience, abilities, and behavior in the hope of revealing universal principles
direct observation
assessment in which the professional observes the client engaged in ordinary, day-to-day behavior in either a clinical or natural setting
random sampling or assignment or neither A research team invited college students who were taking an introductory psychology class into the laboratory. Students were asked to drive a standard, 15-minute course on a professional-grade driving simulator while having a conversation with a member of the research team. Half of the students were talking via cell phone, and the other half were talking to their conversation partner in the passenger's seat. The researchers flipped a coin to assign students to one of the two conditions. Did this study use random sampling or random assignment?
assignment
Providing informed consent is done to address which of the following ethical principles?
autonomy
random sampling or assignment or neither The American Automobile Association (AAA) conducted a survey using a random-digit phone-dialing system to call a national sample of 2,000 U.S. drivers. The target question was whether or not drivers thought that driving while talking on a cell phone should be illegal. However, before asking this target question, the surveyor told half of the participants that hands-free cell phones are associated with higher accident rates. The other half of the participants were not given this information. The computer used a random method to decide which participants were and were not given the information. The AAA wanted to know if hearing about research on cell-phone-related accidents would prompt people to change their responses to the target question. Did this study use random sampling or random assignment?
both
In what way are means and standard deviations similar to one another? They are both examples of measures of frequency, They are both examples of measures of variability, They are both examples of descriptive statistics, They are both examples of central tendency measures
both descriptive stats bc they summarize participants different responses
how are r and d statistics similar They are both used in experiments, They are both measures of the size of an effect, They are both descriptive statistics, They are both useful for establishing a study's construct validity
both measures the size of an effect
Correlation does not equal
causation
Measured or Manipulated Researchers place volunteers in an fMRI machine, which detects brain activity in the occipital cortex (the back of the brain). The brain activity in the occipital cortex is
measured
effect size
describes the magnitude of the relationship between manipulated or measured variables strength of the correlation or the degree of difference between groups in an experiment
autonomy
did the participants have the chance to review the procedure of the study and decide whether or not they wanted to participate where vulnerable groups protected in this study (children, disabled, prisoners) were they debriefed about the study at the end
Justice
did the sample of participants used in the study resemble the types of people who will benefit from it Only those who participate in research bear the burdens of potential discomfort, pain, or inconvenience, yet the research studies can benefit an entire community of people
Beneficence
did they take care to minimize risks to participants do the fingering of the study offer potential benefit to the community
the coefficient r
direction and strength of a correlation the value of r can range from -1: perfect neg relationship 0: zero relationship 1: perfect positive relationship
reasons a study may not be replicable
false positive, harking, p-hacking, underreports nonsignificant effects
Measured or Manipulated The researcher has participants operate a driving simulator. She assigns some participants to drive while talking on a hands-free phone and others to drive while talking to a "passenger" sitting nearby. Whether drivers talked on a phone or to a passenger is
manipulated
Measured or Manipulated A researcher asks participants how difficult they found a set of math problems to be. Their difficulty ratings are
measured
Measured or Manipulated In a study on phobias, a researcher hooks people up to a skin conductance device to record their stress levels while they view photos of spiders. The skin conductance stress levels are
measured
experimental research
why do people do this? make you more confident that the hypothesis is a good one or a bad one and gives more confidence in the theory testing whether one variable causes another can only examine a few variables in one study, not all variables can be manipulated can establish a casual direction of a relationship
