Research Methods Exam 1
Measuring reliability
Test re-test reliability (getting consistent scores every time you use a measure); Interrater reliability (consistent scores obtained no matter who measures/observes); Internal consistency (each question is a measurement device, are all of the questions measuring the same thing?, degree to which items on a scale are related to each other)
Study
The entire research project
The Research Process
Unrefined research idea/question > review literature > Hypotheses > design study > Conduct study > Data analysis > Conclusions > report results; conclusions uncover new questions, which leads to more research
variables involved in a t-test
2 groups; categorical predictor & continuous outcome
Experiment
A study in which one (independent) variable is manipulated and the other (dependent) variable is measured
Snowball sampling
A type of non-probability sampling, asking participants to spread the word about your study, non-representative because people are recruited via social networks which aren't random
Measuring Internal Consistency
Actual Score= True Score + Random Error
APA style Results section
Always begins right after Method section; Use tables to present multiple values (means, correlations, inferential tests); use figures to present strongest results
Item-Total Correlation
How much does item x correlate with the rest of the scale?, negative correlation means you probably forgot to recode an item, having multiple questions allows you to cancel random error out
Construct
Hypothetical factor that cannot be observed directly, but inferred from other observations; Label that organizes and summarizes the essence of some behavior, event or object; has both conceptual definitions (ex: happiness) and operational definitions (ex: counting how much someone smiles to measure happiness); pretty much everything we study in Psychology
Operational Definition
Specific description of exactly how a construct will be measured or defined, Dimensions may include Frequency, Intensity (How much is this like you?), Duration, State vs. Trait; Modalities=Self-Report, Behavioral/Observational, Physiological
Testability
Specific predictions and falsifiability
Cluster sampling
randomly select clusters of participants within the population of interest, and then collect data from all individuals in each cluster
Convenience sampling
relatively easy to obtain, not representative but can be a good place to start, requires getting a representative sample later, most psych research=convenience samples
Systematic Random sampling
researcher counts off members of a population to achieve a sample, using a randomly chosen interval (ex: every "nth" person, where n is a randomly selected number)
Quota sampling
researcher identifies subsets of the population of interest, then sets a target number for each category in the sample (ex: 80 Latinos, 80 Asians), then samples from the population of interest non-randomly until the quotas are filled; differs from stratified random sampling in that participants are not selected randomly
Simple Random Sampling
sample is chosen completely at random from the population of interest (ex: drawing names out of a hat); if it's a random sample, it will have the same demographics as the population
Stratified random sampling
separating population into strata (layers) based on demographics and then randomly selecting individuals within each strata
APA style Method section
should enable reader to conduct an exact replication study; use subsections like "Design", "Participants", "Material", "Procedure"
ANOVAs and correlations
significant ANOVA=correlation
APA style abstract
summarizes research question, methods used, primary results, and interpretation of results in terms of a theory
Parsimony
the degree to which a theory provides the simplest explanation of a phenomenon; best explanation that requires making the fewest exceptions or qualifications
Confirmatory Hypothesis testing
the tendency to ask only the questions that will lead to the expected answer
Pop-up principle/Availability Heuristic
the tendency to rely on evidence that easily comes to mind
Self-Selected sample
when a sample is known to contain only people who volunteer to participate, common issue with response rate (often seen in online polls), can cause serious problems for internal validity
comparison groups
you always have these if you're doing an experiment
Dealing with Self-Report disadvantages
Include special survey items identifying socially desirable responders (ex: I never hesitate to go out of my way to help someone in trouble), if people agree with these items then that data can be discarded under suspicion that they are exaggerating on the other items
constructing effective surveys
Informal Avoid double-negatives Avoid double-barreled questions Avoid forced-choice items (when you can only pick between one or the other) Avoid questions that do not yield any variance Avoid loaded questions Make sure questions are relevant to everyone in the study Mix it up (measure multiple constructs in different ways) Ease into socially sensitive questions Ask sensitive questions sensitively Guarantee anonymity
Validity
Is it accurately measuring what it claims to measure?, extent to which a result corresponds to the actual state of the world; free from systemic error
Determining Error Variance
Is the variation in their answer related to something real? Or just random?
Scale/questionnaire
Measuring a construct using multiple items; if every question is required then the research is involuntarily and thus unethical
Types of Variables
Nominal (Categorical) - levels are categories Quantitative- can be recorded numerically Ordinal- levels represent a ranked order, unclear if distance between levels is equal Interval- distance between levels is equal Ratio- equal intervals, 0 means nothing Continuous- infinite number of possible values Confounds- third variables that create systematic error in experimental designs
Cronbach's Coefficient Alpha
Represents how tightly correlated the variables are; Below .65= items probably aren't correlated; CANNOT BE NEGATIVE, True claim means alpha close to 1; Alpha if item deleted=good way to isolate which variable is different; in psychology we always take into account both what the numerical data is showing us & what we're seeing; From lab: if it is high it doesn't matter if there's a negatively re-correlated item (don't have to remove this item); you don't want to delete any items because then you can't compare your results to any study that used the same scale
Non-Probability sampling
purposive, deliberately only includes certain people to get a particular demographic
APA style Discussion section
Beginning: summarize results and interpret how they fit in with the theory & past literature; Middle: evaluate the study and explain how it fulfills the 4 big validities; End: suggestions for future research
Random Error
Chance fluctuation in measurement around a true score due to uncontrolled, unidentified variables; errors cancel each other out in the long run, threatens reliability
Claims
Claim=argument someone is trying to make, Types: Frequency (44% of Americans struggle to stay happy), Association (argues that one level of a variable is likely to be associated with a particular level of another variable; ex: positive, negative, zero, curvilinear), Causal (argues that one variable changes the other, ex: covariance(correlation), temporal precedence(one came before the other), internal validity)
Likert Scale
Common survey question format; a rating scale containing multiple response options that are anchored by the terms "strongly agree", "agree", "neither agree nor disagree", "disagree", and "strongly disagree"; a scale that does not follow this format exactly is called a "Likert-Type Scale"
Types of Validities
Construct Validity (quality of measurement, "How did you phrase the question?"), External ("is it generalizable to the larger population?"), Internal (No third variable problem, "are there any alternative explanations?"), Statistical (probability of error, "Was it statistically significant?")
Population
Everyone that you want to make a claim about, most participants are W.E.I.R.D because they are easily accessible (why most psych studies are biased), also called "population of interest"
Reliability
Extent to which outcome is found consistently; reliable measures are free from random error
Major types of Construct Validity (DON'T CONFUSE WITH "Types of Validities")
Face Validity: just by looking at it, can you tell that it's a measure of the variable? Content Validity: measure contains all the parts that your theory says it should contain Convergent Validity: measure is more strongly associated with measures of similar constructs; involves finding a scale & seeing if your construct is compatible with it; From lab: significant positive correlation between two scales of two different constructs=weak evidence for convergent validity Discriminant Validity: measure distinguishes itself from things it shouldn't be related to; to find this, look for things it should be related to but isn't and measure it
Cherry-picking evidence
When you only pick certain information that would support your claim and nothing that would counter it
Self-Report
Whenever the participant gives verbal information; usually a questionnaire or interview; advantages: fast, easy, direct; disadvantages: self-deception/lack of insight, reactive to factors like social desirability & cooperative responding
Reactivity
a change in behavior of study participants because they are aware they are being watched
Embodied Cognition
a construct; The way we feel will affect how we think in ways we can't always notice; ex: In Lewendowski study, it's the visceral perception of temperature change
Semantic Differential format
a response scale whose numbers are anchored with contrasting adjectives
Oversampling
a variation of stratified random sampling in which the researcher intentionally overrepresents one or more groups
Non-Reactive measures
behavioral, observational, psychological; Disadvantages: cost, time, training & bridging the gap
APA style title
communicates purpose of research in 12 words or less; does not use the words "method", "results," "a study of," or "an experimental investigation of," centered and not boldfaced
Construct Validity
operational definition matches the concept you set out to assess
control group
do not receive treatment
issue with generalizability
external validity error; to prevent this, researchers need to ensure that the participants in the sample adequately represent all of the population of interest
APA style intro
first paragraph: describes general area of research, middle paragraphs: summarize & interpret past research studies (gets at the gap), last paragraphs: describe the method & primary variables & state hypothesis or research question
Systematic error (bias)
fluctuation in measurement that systematically increases or decreases measurement of the true score due to identifiable variables (confounds); something that you should have controlled for; error does not cancel out, threatens validity;
The Importance of Theory
foundation of hypotheses and outcome of studies
Sampling
getting a subset of the population of interest; sampling technique matters more than size; Small sample ok if it's unbiased, large samples can be flawed
Present Present Bias
heuristic, tendency of people to give stronger weight to payoffs that are closer to the present time
Multistage sampling
involves at least two stages: a random sample of clusters followed by a random sample of people within the selected clusters
APA style references
listed in alphabetical order by author's last name; within entries, the author matters (first author contributed most, last author contributed least); starts on new page after Discussion
Observational measures
measuring a variable by recording observable behaviors or physical traits of behaviors, aka behavioral measures
Observer bias
occurs when observers' expectations influence their interpretation of the participants' behaviors or the study's outcome