Research methods 1 Exam #1
barnum question**
"Have you ever eaten fast food?"
What are some of the issues surrounding whether and to what extent psychology is a science?
- Science is: *Empirical: learned through observation *Objective: we think that we are free from bias *Systematic: procedures follow carefully planned steps *Controlled: eliminate other extra factors
What is observer bias and how can blind observers and multiple observers help mitigate the effects of this?
- observers selectively attend to what they expect to hope or see - Blind observers don't know the expectations. Multiple observers can increase objectivity.
What is a confounding variable? How is it a threat to internal validity?
-A variable that is not the focus of the research study, but affects the variables of interest in the study. -
What is a confounding variable? Which type of validity do they hurt if you have them in your study?
-A variable that is not the focus of the research study, but affects the variables of interest in the study. -Hurts internal validity
Scholarly works
-advances knowledge, written by experts for other experts
What is a pilot study and a catch trial? How is each helpful when designing a questionnaire?*
Pilot study: help determine the best way to ask Catch trial: test consistency, honesty, arbitrary answers -They are helpful because multiple questions create reliability
What is the difference between qualitative and quantitative variables?
Qualitative: non-numeric,based on type or characteristic ex. college major Quantitative: numeric, based on amount of variable ex. score on IQ test, number of errors
Difference between a structured and a semi-structured interview
Structured: more objective, data easier to deal with Semi-structured: wider range of responses, can discover new variables
two-group pretest-protest
administering a pretest and a posttest to both groups of participants and compare the change in scores between the groups
snowball sampling
identify one person, then another (from them) *least used
Hawthorne effect
improvement in performance due to being studied
content validity
inclusion of all aspects of a construct by items on a scale or measure
observational research
involves observing and recording the behavior of humans or animals
one-group pretest-posttest
non experimental design in which all participants are tested prior to exposure to a variable of interest and again after exposure
nonparticipant vs. participant
nonparticipant: the researcher or observer is not directly involved in the situation participant: the researcher or observer becomes actively involved in the situation
experimenter expectancy
subtle cues can influence participants
alternate forms reliability
the relationship between scores on two different forms of a scale
convenience sampling
those who are convenient
ordinal
vary in amount (quantitative), but each interval isn't necessarily the same, rankings ex. rank in army, educational degree, runners finishing race
two-group posttest only
we could omit the pretest and compare the posttest scores between two groups
construct validity
whether a measure mirrors the characteristics of a hypothetical construct, can be assessed in multiple ways
secondary source
authors review research but do not report results of original study *review article, scholarly book
What are two biomedical and two psychological experiments that were thought to be conducted unethically and the resulting codes of ethics that came about as results?
Biomedical: 1) Nuremberg code (1947) : in response to physicians conducting medical studies on prisoners in Nazi concentration camps 2) Belmont Report (1978): in response to Tuskegee syphilis studies conducted from 1932-1972. Psychological: 1) Milgram's Shocking Experiment (1963) 2) Zimbardo's Stanford Prison Experiment (1972)
Why does correlation not equal causation?
Causality is a function of a research design, not the type of statistics you use to analyze the design.
Why does random assignment in experiments allow us to be able to control for more potentially confounding factors? Which one(s)?
Change will occur due to the variables and not individual differences or characteristics.
Difference between confidentiality and anonymity and why they are important
Confidentiality: responses kept private, but researcher may be able to link participant w/ responses Anonymity: no one other than participant can link the participant to his/her responses They are important because they both require vigilance on the part of the researcher.
Difference between correlational and experimental designs and the strength of their potential conclusions
Correlational: Does _____ change as a function of ____? -height/weight -drug use/intelligence *Researchers do not attempt to alter variables- they measure them! Experimental: Does _______ influence/(cause) _______? cause: alcohol effect: memory *independent var. *dependent var. * There is nothing inherently independent or dependent about a variable, it depends how it is being used.
Requirements for a good hypothesis:
Hypothesis must be: 1) testable 2) falsifiable (if wrong) 3) precise 4) rational 5) parsimonious
difference between stratified random assignment and simple random assignment with regard to independent groups
In stratified random assignment, randomization occurs after selection of the sample and is designed to even out key characteristics across IV variables. In simple random assignment, it isn't guaranteed that participant characteristics will be evened our across the IV conditions and procedure is often ineffective w/ small heterogeneous samples.
What is the difference between internal validity and external validity and how do they usually relate within the context of an experiment? How is ecological validity related to one or both of these?
Internal: the extent to which you can demonstrate a casual relationship between your IV and DV External: the extent to which the results of a study can be generalized to other samples, settings, or procedures -ecological validity relates to external validity: situation common? natural?
Where does a control group fit in a research design?
It is a group of an independent variable that doesn't get any of the variable. -ex. 2 groups- one doesn't get assigned drug while other group does
What does it mean for a measure to be valid?
It produces accurate results
Why is peer review important for psychology as a science?
It provides some insight into the process and progress of science.
What does it mean for a study (or the relationship between variables) to be reliable?
It refers to the expectation that we will find similar results when we repeat a study.
What is the difference between a literal and a conceptual replication?
Literal replication: conducting the same study with new participants Conceptual replication: conducting a study examining the same patterns or relationships but with different methods
difference between matched random assignment and random assignment to order of conditions within a dependent-groups design
Matched random: each participant only receives one level of the IV to impact DV Random assignment to order of conditions: participants experience all levels of IV to impact DV
What is the difference between an open-ended and a closed-ended response format in questionnaires?
Open-ended: allows respondents to provide their own answers, non-numerical=type of qualitative measure. Closed-ended: typically quantitative measures that provide options that the respondents select from and can vary from dichotomous options.
What are some positives and negatives for conducting interviews rather than questionnaires?
Positives: obtain individual's perspective, allow a researcher to ask follow-up questions and obtain detailed responses Negatives: potential social desirability bias, time-consuming, heightened demand characteristics, have potential for interviewer bias
What is the motivation behind and elements of an informed consent?
Potential participants must be informed of the topic, procedures, risks, and benefits of participation prior to consenting to participate. -Elements: 1) purpose/topic 2) generally what they will do/how long it will take 3)benefits (incentives) and risks (pain, discomfort, confidentiality) 4) how confidentiality will be protected 5) right to decline or withdraw w/o negative effects 6) names/contact info. for researchers
What are some potential problems with offering incentives to participants and the solutions that have become part of our code of research on human subjects?
Potential problems: 1) What if someone is uncomfortable but they still want the thing? 2) What if people lie to get the thing? Solutions: 1) Researchers should carefully consider who their potential participants are and not offer incentives that they would have a difficult time refusing. 2) The incentive should not be contingent on the participant completing the study.
What is an extraneous variable? Why does it matter if it is systematically changing or not?
-An extraneous variable is any factor or variable that causes an effect (or potential affects) other than the variable being studied - It matters because there could be potential confounds
What is debriefing and when is it particularly important?
-Clearing up any misconceptions that the participant might have and addressing any negative effects of the study. -Important when: 1) participation causes distress 2) you are employing deception
Types of IV manipulations
-Environmental: systematic changes to physical or social environment -Scenario: systematic changes to a scenario -Instructional: systematic changes to instructions, educational information, or feedback -Physiological: systematic changes to participants' or subjects' physical functioning
types of methods psychologists use to collect data
Questionnaires: -scales -questions: MC, dichotomous (yes/no), likert (rating), numerical responses, open-ended, close-ended Observational and unobtrusive measures
Tips on how to construct questions for a questionnaire
Questions should: 1) be clear to the participant 2)generate interpretable results 3) reflect typical behavior
What are demand characteristics? How could they affect the results of a questionnaire or interview?
-Extraneous cue that guides or biases a participant's behavior with or without awareness. - social desirability bias, interviewer bias-body language/cues
experience/environment factors that cause unwanted changes to your variable of interest
-History: event or environmental conditions caused the change -Maturation:natural changes over time caused the change -Testing: changes were due to previous exposure to a measure. -Instrumentation: changes were due to inconsistency in the measurement, instrument, observers
participant factors that cause unwanted changes to your variable of interest
-Regression to the mean:changes were b/c extreme scores usually become less extreme -Attrition:change was due to participants withdrawing from the study *can be a problem if people who leave are different from the people who stay -Selection: changes were due to preexisting differences between groups -Selection interactions: changes were due to an interaction between preexisting differences and other
Why doesn't science prove things to be true?
-Science can rule out alternative accounts, but there can always be another. -Science is about strength of evidence
Difference between true experiments and quasi-experiments
-True: investigate cause/effect, people are randomly assigned to conditions -Quasi: try to investigate cause/effect, people are not randomly assigned to conditions ex. sex, age, major, etc
What is restricted range and how can it be avoided when constructing a response scale?
-You cannot predict beyond the range of data you have. -It can be avoided by having 7 points along the scale, having a help and question text, and anchoring w/o extreme words to minimize demand characteristics
leading question
-a question that prompts or encourages the desired answer. -"Do you support the murder of innocent children through partial- birth abortion?"
double-barreled question
-a single question in which asks about more than one issue, but only allows for one answer -"Do you think the Twilight series is poorly written and unoriginal?"
Independent variable and its levels
-a variable that is manipulated in an experiment -Ex. of levels: Reduce high stress Increase high stress
What is a hypothetical construct? Examples?
-abstract concepts that cannot be directly observed or measured. -ex. happiness, love, learning, emotional intelligence
Primary (empirical) source
-authors report results of original research study that they conducted *data
What are some resources available for you to find scholarly articles about psychology?
-conference papers/posters -unpublished manuscripts -scholarly books -theses and dissertations -undergraduate research -abstracts
Types of non probability
-convenience sampling -quota sampling -maximum variation sampling -snowball sampling
Popular works
-entertain or educate people not in the field -ex. Wikipedia, NY Times
Threats to validity
-internal: confounds -external: bad samples, artificial situation
What makes a good sample?
-representative in large population
descriptive research design
-research design in which the primary goal is to describe the variables, but not examine the relationships among variables. -simply describes a sample or population *one variable
correlational design
-research design in which the relationship among two or more variables is examined, but causality cannot be determined. -which behaviors, events, and feelings co-occur with other behaviors, events, and feelings?
Types of probability
-simple random sampling -stratified random sampling -cluster sampling
What does it mean to generalize?
-taking info from one case and applying it to other cases
What is Cronbach's alpha and how is it used in combination with a split-half procedure to determine the internal consistency of a measure?
-test used to assess the internal consistency of a scale by computing the intercorrelations among responses to scale items. - determining level of overlap between responses
What is inter-rater reliability and why is it important?
-the measure of agreement between different raters' scores -It is important because the raters make independent judgements, meaning that they do not know each other's codes or scores.
What is a manipulation check and why is it important?
-the process of verifying that the participants attended to the manipulation -It is important b/c it is possible that the participants did not read carefully enough to pick up on the manipulation.
requirements for causality
1) Does variable A influence/(cause) variable B? 2) Correlation: there must be a relationship between A and B. 3) Sequencing: the change in variable A must come before the change in variable B. 4) Ruling out alternative explanations
Steps for research
1) Identify topic 2) Find, read, evaluate past literature 3) Develop good hypothesis about a variable/a relationship between variables- anything that can take on different values 4) Choose a research design 5)Plan and conduct your study 6) Analyze data 7) Communicate results 8) Repeat
Arguments against deception:
1) May harm participants (embarrassed, uncomfortable) 2) May harm the field b/c makes people suspicious
What types of things do researchers typically record on a code sheet?
1) Narrative about experience/what you observed 2) checklist for key variables 3) duration of behavior 4) timing: task completion, latency 5) rating scale
APA rules for deception:
1) Necessary and justifiable given potential benefits 2) No physical/emotional pain expected 3) Debriefing occurs ASAP
Four measurement scales
1) Nominal 2) Ordinal 3) Interval 4) Ratio
Demand characteristics that are specific to experimental design
1) Reactivity bias: Bias in responses due to being observed 2) Hawthorne effect: Improvement in performance due to being studied 3) single-blind study: make participant unaware of manipulations/predictions 4) experimenter expectancy: subtle cues can influence participants 5) double-blind study: make participant unaware of manipulations/predictions and researcher unaware of condition 6) deception: disguising a manipulation or measure *If you deceive, you must debrief! 7) placebo: a treatment or substance that in and of itself has no therapeutic affect, such as a sugar pill
Process involved in peer review
1) Scientists study something 2) Scientists write about their results 3) Journal editor receives an article and sends it out for peer review. 4) Peer reviewers read the article and provide feedback to the editor. 5) Editor may send reviewer comments to the scientists who may then revise and submit the article for further review. If an article does not maintain sufficiently high scientific standards, it may be rejected at this point 6) If an article finally meets editorial and peer review standards, it is published in a journal.
What are the parts of an APA empirical paper?
1) Title page 2) Abstract 3) Introduction 4)Method 5) Results 6) Discussion 7)References 8) Tables and figures
What types of questions should you avoid putting in a questionnaire and why?
1) Too vague questions: "How long have you studied Spanish?" 2) Double barreled question 3) Barnum question 4) Leading question
Arguments for deception:
1)Essential in creating rare occurrence 2) essential in eliciting genuine responses
What does it mean for a study (or the relationship between variables) to be valid?
The degree to which the result of the study reflects what it is intended to reflect
What does it mean for a measure to be reliable?
A measure is said to have a high reliability if it produces similar results under consistent conditions
test-retest reliability
a measure of the stability of scores on a scale over time
placebo
a treatment or substance that in and of itself has no therapeutic affect, such as a sugar pill
simple random sampling
a type of probability sampling in which every single member of the population has an equal chance of being selected for the sample.
cluster sampling
a type of probability sampling in which groups, or clusters, are randomly selected instead of individuals
stratified random sampling
a type of probability sampling that results in the sample representing key subpopulations based on characteristics such as age, gender, and ethnicity
Dependent variable
a variable that is measured in an experiment and is expected to vary or change based on the independent variable manipulation
difference between independent groups (between-subjects) experiment and dependent groups (within-subjects) experiment
between-subjects: experiment in which each participant experiences only one level of the IV. within-subjects: experiment in which the groups are related, in that participants were matched prior to exposure to the IV or in that the participants experience all levels of the IV.
reactivity bias
bias in responses due to being observed
quota sampling
convenience sampling w/ an eye for quota
covert vs. overt
covert: observations are made w/o the participant's awareness overt: no attempts are made to hide the observation
deception
disguising a manipulation or measure *If you deceive, you must debrief!
single-blind study
make participant unaware of manipulations/predictions
double-blind study
make participant unaware of manipulations/predictions and researcher unaware of condition
naturalistic vs. contrived
naturalistic: observations that occur in natural environments or situations and do not involve interference by anyone involved in the research contrived: the researcher sets up the situation and observes how participants or subjects respond.
divergent validity
negative or no relationship between two scales measuring different constructs
social desirability bias
participants may respond based on how they want to be perceived or what is socially acceptable
concurrent validity
positive correlation between scale scores and a current behavior that is related to the construct assessed by the scale
predictive validity
positive relationship between scale scores and a future behavior that is related to the construct assessed by the scale
convergent validity
positive relationship between two scales measuring the same or similar constructs
nominal
qualitative differences, categories, types, no ordering. ex. hair type, dog or cat person
interval
quantitative, equally spaced, intervals, does not have true zero ex. likert-type answers, IQ, temperature
ratio
quantitative, has true zero ex. length of something, reaction time, # of years in college
experimental design
research design that attempts to determine a casual relationship by manipulating one variable , randomly assigning participants or subjects to different levels of that manipulated variable, and measuring the effect of that manipulation on another variable.
quasi-experimental design
research design that includes a key characteristic of an experiment, namely, the manipulation of a variable. -no random assignment -no causation
maximum variation sampling
researcher seeks full range of extremes in population
Difference between a sample and a population
sample: a subset of the population from which data are collected. population: the group of people, animals, or archives that you are interested in examining.
What are two types of archival research?
secondary data and records/documents
variable
something that varies in that it has at least two possible values
What does it mean to operationalize a variable?
specific way you are measuring hypothetical construct
How do you make sure your dependent measure is sensitive enough? Why does it matter?
• Quantitative > Qualitative • Avoid restricted range -Widen! - Ceiling/floor effects
