experimental exam 2
APA ethics code 5 priciples
A: beneficence and malifencence--maximize benefits and minimize risks B: fidelity and responsibility: be responsible and professional in interactions with people (if researcher does not show up on time_ C: integrity: dont lie, cheat, steal, commit fraud D; Justice--fairness and equity E: respect for peoples rights and dignity--respect for persons informed consent
simple effects
-IF the interaction is statistically significant, then we examine simple effects (called "post-hoc" tests -if the interaction is NOT statistically significant, then we do not examine simple effects
minimal risk
-IRB places some attention around this -standard psychological measures: -voice recordings not involving danger to participants -studies of cognition/perception not involving stress -fully informed consent generally not required, debriefing/ehtical concerns are important
what does a p value tell us
-if the null if actually true--there is a ..% chance -the p value is the probability of observed results this extreme or more extreme, if the null hypothesis is true -must be lower than chosen alpha to indicate a statistically significant difference -for example: p=.01 indicates a low probability that the observed results would occur if the null is true -low p-value is evidence against the null-- - If p - value is < or = to 5% means there is strong evidence against the null hypothesis, so you reject the null - If p - value is > than 5% means there is weak evidence against the null hypothesis, so you fail to reject the null -ex: p= .07 IF the null is true--the groups are equal in the population--there is a 7% chance that we would observe these results (or a difference more extreme)
discussion: compare and contrast to existing literature
-if your results are similar to prior studies, compare in methodology to say how your study provides evidence of generalization -if your results are dissimilar to prior studies, compare differences in methodology to try to explain why your findings are in contrast with established findings--esp if there is a lot of existing evidence that your findings go against -this often involves restating some of what you said in the intro
broken system in psychology
-in order to be successful, you have to have your results published--but some of these results are not even right!
who decides what is ethical and how?
-individual researchers -IRB: institutional review board--required at every institution that receives funding -US department of health and human services (HHS)--office for human research protection (OHRP)
confidentiality
-is an issue when the researcher has assured subjects that the collected data are accessible only to people with permission, generally only the researcher -important when studying sensitive topics -researchers will attempt to avoid confidentiality problems by making sure that the responses are completely anonymous
2 methods to determine statistical significance: method 2
-is the p-value calculated from sample data lower than the chosen alpha? -if yes, then reject the null (H0) -this is the method we use with jasp
2 methods to determine statistical significance: method 1
-is the test statistic more extreme than the critical cutoff value? -use alpha to look up th critical cutoff value in the table -comput the test statistic -if the test stat is more extreme than the critical value, then reject the null (H0)
main problem with facebook study
-lack of consent and lack of debriefing are hugely problematic -the effects were also super tiny participants were not: -informed of the study -warned of risks -provided a point of contact for questions -given the opportunity to decline participant
risks in behavior research: physical harm
-many medical procedures: administering a drug like all or caffeine -physical stressors: such as loud noise, extreme hot or cold temps, deprivation of sleep for a long time -the research would have to offer clear benefits that would outweigh the risks in order for this to be possible
beneficence
-maximize benefits (studying something that could cure cancer) and minimize risks of research (small manipulation od someones diet)--bad balance would be putting people in an intense state of anxiety to see if they would forget information -when we submit our rearch to IRB we decide whether we think it is no risk, minimal risk, or greater than minimal risk
alpha level
-the probability required for significance -the probability of rejecting the null when the null is true -usually .05 -the otiose of the study is considered significant when there is a .05 or less probability of obtaining the results -that is there are only 5 chances out of 100 that the results were due to random error in one sample from the population
no risk
-things that people do in daily like -if this is approved by IRB, then you dont need informed consent from participants -studying normal education al practices -cognitive aptitude/achievement measures -anonymous surveys -observations of nonsensitive public behaviors where participants cannot be identified
sample size
-total number of observations -as your sample size increases, you are more confident that your outcome is actually different from the null hypothesis expectation
inferential statistics
-used to determine whether the results match what would happen if were to conduct the experiment again and again with multiple samples -allows researchers to make inferences about the true difference in the population on the basis of the sample data -gives the probability that the difference between means reflects error rather than a real difference
F test/analysis of variance
A statistical significance test for determining whether two or more means are significantly different. F is the ratio of systematic variance to error variance. F =systematic variance/error variance -used when comparing more than two groups--3 or more levels of one variable (factorial design) -the larger thhe F variable is, the more likely that the results are significant --true difference -can be used in the most simple experimental design, and the most complex design calculate F (and p values) for each effect of interest ex: in a 2x2 factorial design: -interaction -main effect for factor A -main effect of factor B -simple effects** but only if the interaction is significant
type 1 error
Rejecting null hypothesis when it is true -saying that the population means are not equal, when they actually are equal -type 1 error rate = alpha -occurs when simply by chance we obtain a large value of t or F -incorrectly decide that an independent variable had an effect -propability of this error is determined by the choice of significance or alpha level*** -if we increase alpha, there is a greater chance of a type 1 error (bc increasing the critical values range) =.1 vs .05--making an error 10% of the time vs 5% of the time -when the significance level for deciding whether to reject the null is .05, the potability of a type 1 error is 5% -the probability of making a type 1 error can be changed by either decreasing or increasing the significance level
2 tailed test
Uses a non-directional hypothesis.
Belmont report
current ethical guidlines for behavioral and medical researchers: respect for persons, beneficence, justice
intro:1. state the problem (and hook the reader)
goals: -introduce the topic of your work -establish importance avoid descriptions of the state of the research literature as the topics of your sentences and paragraphs state the problem: use one of these strategies for the first paragraph -begin with an example that readers will understand and identify with -begim with a rhetorical question -begin with an interesting stat **avoid overstating the importance of your topic "choice overload is the most important phenomenon ever"
probability
likelihood of the ovvurance of some event or outcome -want to specify the probability that an event (a difference between means in the sample) will occur if there is no difference in the population
components of science:
science is: -based on evidence (empiricism) -progressive -cumulative -NOT free from errors -self correcting
confidence interval
the range of values within which a population parameter is estimated to lie
degrees of freedom
the total number of participants in the groups minus the number of groups -the number of scores free to vary once the means are known
if the probability is high that the null is true...
then we fail to reject the null hypothesis "there is not a statistically significant difference..."
discussion: limiting conditions
threats to internal validity: is this study a fair test of the hypothesis -operational definitions of variables -confounding variables threats to external validity: would the results of the study generalize? -participant characterisitixs -stimuli characterisitics -setting is the way data were collected sufficient to test the hypothesis?: -sample size -source of sample
violating the rules of science (Stapel)
-highly successful social psychologist in netherlands -fabricated over 50 peer review journal articles -he would gather data and change values to make sure his predictions were correct
key question
-how likely/unlikely is it that this observed difference occurred by chance, if there is truly no effect in the population -tool to answer: NHST
cohens d
# of standard deviations means are from each other -effect size in terms of SD (Mcondition1-Mcondition2)/population SD
what affects statistical significance
-sample size: a larger sample increases the ability to detect differences; have a greater probability of representing the true population parameters (less random error) -alpha chosen to set threshold--larger alpha makes it easier to reject the null if there is an effect -effect size--bigger effects are easier to detect -measurement accuracy: poor measurement can increase error variance, which can reduce the ability to detect an effect that exists
any numerical difference we see in our samples could reveal
-a difference that occurred just due to chance (no real difference in the poulation) -or an actual difference
abstracts exclude..
-a long intro -definitions -citations -descriptive phrasing
other ethically quesirnoable things
-a man is abducted on way home from work--it was a violation of ethical guidelines but it was morally justified in this case since it was hitler -dozens of people are asked to shock another human being..most comply--study about obedience -were these actions morally justified? -did they violate ethical guidelines?
choosing a sample size: power analysis
-a more formal approach to determining a sample size is on the basis of a desired probability of correctly rejecting the null -this probability is called the power of the statistical test -it is related to the probability of a type two error -higher desired power demands a greater sample size; this is bc you want a more certain guarantee that your results will be statistically significant
nuremberg code
-a set of 10 rules of research conduct that would help prevent future research atrocities -set of principles without any enforcement structure or endorsement by professional organization -not generally seen as applicable to general research settings -sooo world medical association developed a code that is known as the declaration of Helsinki--broader application of the nuremberg side that was produced by the equal community and included a requirement that journal editors ensure that published research conform to the principles of the declaration
statistical significance
-a significant result is one that has a very low probability of occurring if the population means are equal -there is a low probability that ther difference between the obatained sample means was due to random error
a good abstract is
-accurate and non evaluative : accurately reflects the content of the manuscript; report, not evaluate -coherent and concise: clear and concise lang; use active voice
pretests
-allows researchers to be sure that groups are equaivalent -also enables the reader to assess mortality effects when it is likely that some participants will withdraw from the experiment -you can determine if the people who withdrew are different from those who completed the study -but, pretesting may limit the ability to generalize to populations that did not receive a pretest--taking the pretest may cause subjects to behave differently than they would without the pretest --Solomon 4 group design sizes this: half participants are given the pretest and half receive the poshest only
meta-analysis
-another technique for comparing a large number of studies in an area -the researcher combines the actual results of a number of studies -they involve examining the effect sizes and significance levels obtainied -**they are important for determining the reliability of a finding by merging the results from many different studies
replication
-attempt to repeat the results of an experiment by repeating an original study -replicatino of research is a way of overcoming any problems of generalization that occur in a single study -exact replication -conceptual replication
many labs project
-attempted to replicated 13 findings -36 groups of researchers (global effort) -6,344 participants from 12 countries -10 out of 13 successful replicated--2 of them failed completely (both social priming studies); 1 of them half failed
privacy
-becaomes an issue when, without the subjects permission, the researcher collects information under circumstances that the subject believes are private--free from unwanted observations by others -in some studies, researchers make observations aof behavior in public places without informing the people being observed -th einternet has posed other issues of privacy--messages can potentially be used as data
3 ethical principels-
-benficence -autonomy -justice
discussion: summarize key findings
-breifly restate: the hypothesis. preditionc, rationale -summarize the key findings (but not with stats)--interaction (and simple effects if necessary) AND main effects -state whether the predictions were confirmed or not don't write: "we rejected (or failed to) the null hypothesis" dont overstate any aspect of your claims -dont: "obviously we have rejected the null" -better: "the evidence presented supports that tests during learning can eliminate..."
how to chose alpha
-by convention, a=.05
social contagion
-can one persons emotional state be transferred to another persons via social interactions -data gathered naturalistically have suggested that the answer is yes..but these are correlational findings, so it is hard to state causal relationships facebook experiment
failure to replicate social priming studies
-changes in goals, motivations, choice of action -ex: reading elderly related words make people walk more slowly as they leave the lab -ton of conceptual replication, but no exact replication of original experiment
open science collaboration
-conducted replications of 100 experimental and correlational studies published in 2008 in 3 journals -found: out of the 100 studies they tried to replicate, fewer than 40 help up -no evidence or fraud or data manipulation in the OG evidence -problem was worse in social psychology than in cognitive psychology **what does this OSC mean: -some people said the replication effort was flawed -some say it was a good start
discussion: conclusino
-connect your results to broade issues within the general area of your research that you question is part of -tie back to the hook you used earluer
systematic variance
-deviation of the groups of means from the grand mean, or the mean score of all individuals in all groups -is small when the difference between group means is small and increases as the group mean differences increase (between groups variance)
error variance
-deviation of the individual scores in each group from their respective group means (within group variance)
discussion: interpret results (go beyond null NHST)
-discuss other aspects of the data the help with interpretation (effect size and power) explain why we did get the predicted interaction: -this is a real effect and we detected it: double check other expected effects (testing effect) -this is not a real effect, but we detected it anyways: maybe type 1 error, check effect size of interaction, something wrong with our experiment? if we get the predicted Null effect: -there is truly no effect and we accurately observed this null effect: double check other expected effects -there actually is a real effect, but we failed to detect it: type 2 error, check effect size of interaction
facebook experiment
-does the emotional content in a personas facebook new feed have the potential to alter the persons emotional stsate -manipulated the amount of emotional content in users news feed -measured emotional content of users posts
effect size: cohens d (for comparing two groups)
-effect size estimates standardize the scores so we can compare across studies -cohens d: # of standard deviations means are from each other (Mcondition1-Mcondition2)/population SD -can go over 1.00 (unlike r, which ranges from -1.00 to 1.00) -note: for Cohens d report without a sign heuristics: small: 0.2 medium: 0.5 large: 0.8
justice
-ensure that equity is not violated when selecting participants --deicions to include or exclude must be made on scientific grounds
ethically questionable: little albert
-expose baby to things and see reaction -made huge loud noises when Albert saw a rat--created classical conditioning to the point where eventually the rat caused fear -ethically sketchy: why not try to condition a positive conditioning??? and why not de condition the fear afterwards?
generalizability
-fundamental results from a study arise across a variety of situations: -varied contexts (social, environmental); different groups of people; different points in time; different materials; different operational definitions
why is replication important
-generalize results (conceptual replicatoins0 -weed out false effects (exact replications)--type 1 errors, fraud detections BUILDS CONFIDENCE IN THE TRUTH OF SCIENTIFIC EVIDENCE
milgram study
-one of the many that played an important role in the development of ethical standards that guide our ethical decision making
discussion: future directions
-outline an idea for additional research that you could help address the main hypothesis of your paper -should give general predictions and ideas of how your new idea would inform the hypothesis of your current work
replication crisis in science--things that were happening that began the questioning of science
-paper on p-hacking launches discussion of questionable research practices -false positive psychology--study that made the claim that it is super easy to claim that things are statistically significant -Bem: evidence for precognition -Stapel (fraud) -failures to replicate social priming research
how do we avoid fraud
-peer review is not catching fraud -exact replicating can help, but not perfect--journals dont usually present work that was deemed insignificant, so even if things were proven wrong it was probably not payed attention to -transparency in research -need to create a method to determine if data is fabricated--humas are bad at being as random as what the environment actually Is, so these methods can find ways I which variables dont seem real
was the face book study ethical?
-people were angry that facebook tried to manipulate peoples emotional states -scientists said that the experiment is scandalous and violated accepted research ethics-
research hypothesis
-poulation means are in fact not equal -null hypothesis is false
generalization as a statistical interaction
-problem of generalization can be though of as an interaction in a factorial design -an interaction occurs when a relationship between variables exists under one condition but not the other --if there is an interaction , the results can not be generalized across subjects
risks in behavioral research: stress and distress
-psychological stress is more common than physical stress -like participants in Milram study -telling someone they scored very high or low on a subject -asking peoplee about traumatic experiences in their life
open framework
-pushes science to be more transparent form -preregistration: state ahead of time the things you are going to do: all methods, hypothesis, etc so that there is no chance that you will go back on what you were planning oto do -brain noseick\ -change standard p value? -badges--make it harder for people to have fake data
exact replication
-repeat experiments under the same conditions, with as few changes as possible -used to ensure that the initial findings are not the case of discovering an effect that is not real (type 1 error) -when an alpha is set at .05 this means that 1 in 20 significant effects is a false finding (if there is truly no effect to begin with)
is it worse to report a false effect (type 1) or miss a real effect (type 2)
-reporting a false effect can get people to believe in "truths" that are not real--bad for cumulative nature of science -missing a real effect can cause us to lose opportunities to advance our understanding and or to fix real problems
greater than minimal risk
-research involving phsycia stress, psychological stress, invasion of privacy, measures of sensitive info where participants may be identified -full IRB review required, and special ethical procedures may be imposed
literature reviews
-researchers draw conclusions about the external validity of research findings by conducting literature reveiws -a reviewer reads a number of studies that address particular topic and then writes a peer review that summarizes and evaluates the literature provides info that 1. summarizes what has been found 2. tells the reader which findings are strongly supported and which are only weakly supported 3. points out inconsistent findings and areas in which research is lacking 4. discusses future directinos for research
autonomy
-respect for peoples informed consent -ciritcal info that allows the potential participant to make an informed deacon of whether or not to participate -coercion is a threat to autonomy informed consent must include: -statement that participants are being asked to participate in a research study -desccription of any reasonably foreseeable risks or discomforts and safeguard s to minimize the risks -if an incentive is offered, a description of the incentive and requirement to obtain it: also, a description of the impact of a decision to discontinue participation -statement that participation is voluntary, refusal to participate will involve no penalty or loss of nbenefits to which the subject is other wise entities, and the subject may discontinue to participate at any time
Do our results replicate in science--is the truth that we know actually true? Problem:
-sign that results in psychology are not reproducible--not everything replciates
effect size: comparing two groups
-simplest effect size when comparing two groups: Mcondition1-Mconditin 2 -but raw mean differences are not always useful; ex: 4.5-3.2= 1.3; 94.5-93.2 = 1.3--same mean, VERY different contexts -so, to better interpret mean differences, we need standardized effect size measures
methods of catching/correcting errors in science
-solid training in methodology an statistics -fixing peer review to be more rigorous -open discourse: talking about replication used to be taboo--want to chang this -retractions: pulls public work out of the literacture--reduces fraud -promoting replication (within and across labs) -promoting fraud detection -open framework
effect size vs p value
-sometimes a statistically significant effect can be small -sometimes a large effect can fail to reach statical significance -soo.. significance testing does not tell us how big an effect is -statistically significant does not necessarily mean practically important
one tailed
-specificed a direction of differencce between the groups
Null Hypothesis significance testing (NHST)
-start by assuming no different between populations of interest : H0--there is no difference ex: retrieval practice and restudy practice DO NOT differ in how they affect memory -state the research (alternative) hypothesis: -H1: there is a difference; retrieval practice and restudy practice do differ in how they affect memory -choose a cutoff value that determines how extreme the observations must be--relative to the distribution representing the null hypothesis--to reject the null -take a sample of observations
generalizing from laboratory settings
-studies done in labs have high internal validity, but the articificality of the lab setting limits the ability to generalize what is observed in the lab to real life setting -to fix this, do field experiements--researcher manipulates the IV in a natural setting
discussion
-summarize key findings -interpret results--go beyond NHST -limiting conditions -compare/contrast to existing lit -future directions -conclusion
t test vs anova (F test)
-t test: comparing two groups -anova: comparing more than 2 groups/conditions
conceptual replicaiton
-test same variables, but with different methods -keep the same operational definitions of variables, but use participants with different characteristics -the same IV is operationalized in a different way and the dependent variable might be measured in a different way too -or you can change the operational definiotns of varibales -used for generalization ; tests the truth of the underlying hypothesis; discover boundary conditions -without them science is full of false findings
t test
-used to examine whether two groups are significantly different from each other t= group difference/within-group variability = between-condition difference/within-condition variability -larger t value indicates a true difference -does the mean of the no model group differs from the mean of the model group - need to calculate a value of t from the obtained data and evaluate the obtained t in terms of the sampling distribution of t that is based on the null -thif the obtained t has a low probability of occurrence, then the null is rejected -the t value is a ratio of: the group means and the variability within groups -group difference--the difference between your obtained means of the two groups --under the null you expect this difference to be zero; the value of t increases as the difference between you sample means increases (expected value of t under the null is zero) -withing grow variability is the amount of variability of scores about the mean--the indicator of the amount of random error in your sample
if the probability is low that the null is true,,,
-we reject the null -there is a difference/effect -"there is a statistically significant difference..'
3primary reasons for a decrease in the type of elaborate deception seen in Milgram study
1. researchers have become more interested in cognitive variables rather than emotions--using methods similar to those used by memory and cognitive psychologists 2. the level of awareness of ethical issues has led researches to conduct studies in other ways 3. ethics committees at universities now review proposed research more carefully
intro: 3 parts
1. state the problem (and hook the reader) 2. describe relevant scholarship 3. state the hypothesis and how it was evaluated in this study
informed consent
a -it should not be written in the first person--should be written as if the researcher was having a conversation with the participantn ethical principle that research participants be told enough to enable them to choose whether they wish to participate -should cover the purpose of the research, procedures that will be used including time involved, risks and benefits, nay compensation, confidentially, assurance of voluntary participation and permission to withdraw, and contact info for questions
what is an abstract and why is it important?
a brief, comprehensive summary of the contents of the article -usually 120-250 words -components: background, method, results, conclusion/implications important: -summarizes researhc -helps other researchers with literature review -depending on the abstract, readers may or may not read the entire article
deception
active misrepresentation of information about the nature of a study
within group variability
divide the variance of each group by the number of subjects in that group -add these together -then take the root
fraud
fabrication of data -most serious in two areas: science and journalism bc they are both fields in which written reports are assumed to be accurate descriptions of actual events -there are not dependent accounting agencies to check on the activities of scientists and journalists
type 2 error
failing to reject a false null hypothesis -when the population means are not equal but the results of the experiment do not least to a decision to reject the null related to 3 factors: 1.the significance level (alpha)--if we set a low significance level to decrease chances of a type 1 error, we increase chances of type 2 error 2. sample size--true differences are more likely to be detected if the same size is large 3. effect size--if effect size is large, a type 2 error is unlikely
sampling distribution
the distribution of values taken by the statistic in all possible samples of the same size from the same population -has a mean of 0 and a standard deviation of 1 -it reflects all the possible outcomes we could expect if we compare the means of two groups and the null hypothesis is correct
effect size
the magnitude of a relationship between two or more variables --a value that tells us about the magnitude of an effect -makes it possible to compare effects across studies -helps us interpret our findings -a value that can range from 0 to 1 -effect size indicates the size of a difference -cohens d and effect size are not effected by sample size -on a graph you can see the effect of sample size via: overlap and variance--the more they overlap, the smaller the effect is; the less variance=the less effect
nullhypothesis
the population means are equal--the observed difference is due oto random error -the independent variable has no effect
debreifing
the post-experimental explanation of a study, including its purpose and any deceptions, to its participants
power
the probability of correctly rejecting the null hypothesis -related to the probability of type 2 error -increase power to decrease chance of type 2 error
