Research Methods Exam 1

Measuring reliability

Test re-test reliability (getting consistent scores every time you use a measure); Interrater reliability (consistent scores obtained no matter who measures/observes); Internal consistency (each question is a measurement device, are all of the questions measuring the same thing?, degree to which items on a scale are related to each other)

Study

The entire research project

The Research Process

Unrefined research idea/question > review literature > Hypotheses > design study > Conduct study > Data analysis > Conclusions > report results; conclusions uncover new questions, which leads to more research

variables involved in a t-test

2 groups; categorical predictor & continuous outcome

Experiment

A study in which one (independent) variable is manipulated and the other (dependent) variable is measured

Snowball sampling

A type of non-probability sampling, asking participants to spread the word about your study, non-representative because people are recruited via social networks which aren't random

Measuring Internal Consistency

Actual Score= True Score + Random Error

APA style Results section

Always begins right after Method section; Use tables to present multiple values (means, correlations, inferential tests); use figures to present strongest results

Item-Total Correlation

How much does item x correlate with the rest of the scale?, negative correlation means you probably forgot to recode an item, having multiple questions allows you to cancel random error out

Construct

Hypothetical factor that cannot be observed directly, but inferred from other observations; Label that organizes and summarizes the essence of some behavior, event or object; has both conceptual definitions (ex: happiness) and operational definitions (ex: counting how much someone smiles to measure happiness); pretty much everything we study in Psychology

Operational Definition

Specific description of exactly how a construct will be measured or defined, Dimensions may include Frequency, Intensity (How much is this like you?), Duration, State vs. Trait; Modalities=Self-Report, Behavioral/Observational, Physiological

Testability

Specific predictions and falsifiability

Cluster sampling

randomly select clusters of participants within the population of interest, and then collect data from all individuals in each cluster

Convenience sampling

relatively easy to obtain, not representative but can be a good place to start, requires getting a representative sample later, most psych research=convenience samples

Systematic Random sampling

researcher counts off members of a population to achieve a sample, using a randomly chosen interval (ex: every "nth" person, where n is a randomly selected number)

Quota sampling

researcher identifies subsets of the population of interest, then sets a target number for each category in the sample (ex: 80 Latinos, 80 Asians), then samples from the population of interest non-randomly until the quotas are filled; differs from stratified random sampling in that participants are not selected randomly

Simple Random Sampling

sample is chosen completely at random from the population of interest (ex: drawing names out of a hat); if it's a random sample, it will have the same demographics as the population

Stratified random sampling

separating population into strata (layers) based on demographics and then randomly selecting individuals within each strata

APA style Method section

should enable reader to conduct an exact replication study; use subsections like "Design", "Participants", "Material", "Procedure"

ANOVAs and correlations

significant ANOVA=correlation

APA style abstract

summarizes research question, methods used, primary results, and interpretation of results in terms of a theory

Parsimony

the degree to which a theory provides the simplest explanation of a phenomenon; best explanation that requires making the fewest exceptions or qualifications

Confirmatory Hypothesis testing

the tendency to ask only the questions that will lead to the expected answer

Pop-up principle/Availability Heuristic

the tendency to rely on evidence that easily comes to mind

Self-Selected sample

when a sample is known to contain only people who volunteer to participate, common issue with response rate (often seen in online polls), can cause serious problems for internal validity

comparison groups

you always have these if you're doing an experiment

Dealing with Self-Report disadvantages

Include special survey items identifying socially desirable responders (ex: I never hesitate to go out of my way to help someone in trouble), if people agree with these items then that data can be discarded under suspicion that they are exaggerating on the other items

constructing effective surveys

Informal Avoid double-negatives Avoid double-barreled questions Avoid forced-choice items (when you can only pick between one or the other) Avoid questions that do not yield any variance Avoid loaded questions Make sure questions are relevant to everyone in the study Mix it up (measure multiple constructs in different ways) Ease into socially sensitive questions Ask sensitive questions sensitively Guarantee anonymity

Validity

Is it accurately measuring what it claims to measure?, extent to which a result corresponds to the actual state of the world; free from systemic error

Determining Error Variance

Is the variation in their answer related to something real? Or just random?

Scale/questionnaire

Measuring a construct using multiple items; if every question is required then the research is involuntarily and thus unethical

Types of Variables

Nominal (Categorical) - levels are categories Quantitative- can be recorded numerically Ordinal- levels represent a ranked order, unclear if distance between levels is equal Interval- distance between levels is equal Ratio- equal intervals, 0 means nothing Continuous- infinite number of possible values Confounds- third variables that create systematic error in experimental designs

Cronbach's Coefficient Alpha

Represents how tightly correlated the variables are; Below .65= items probably aren't correlated; CANNOT BE NEGATIVE, True claim means alpha close to 1; Alpha if item deleted=good way to isolate which variable is different; in psychology we always take into account both what the numerical data is showing us & what we're seeing; From lab: if it is high it doesn't matter if there's a negatively re-correlated item (don't have to remove this item); you don't want to delete any items because then you can't compare your results to any study that used the same scale

Non-Probability sampling

purposive, deliberately only includes certain people to get a particular demographic

APA style Discussion section

Beginning: summarize results and interpret how they fit in with the theory & past literature; Middle: evaluate the study and explain how it fulfills the 4 big validities; End: suggestions for future research

Random Error

Chance fluctuation in measurement around a true score due to uncontrolled, unidentified variables; errors cancel each other out in the long run, threatens reliability

Claims

Claim=argument someone is trying to make, Types: Frequency (44% of Americans struggle to stay happy), Association (argues that one level of a variable is likely to be associated with a particular level of another variable; ex: positive, negative, zero, curvilinear), Causal (argues that one variable changes the other, ex: covariance(correlation), temporal precedence(one came before the other), internal validity)

Likert Scale

Common survey question format; a rating scale containing multiple response options that are anchored by the terms "strongly agree", "agree", "neither agree nor disagree", "disagree", and "strongly disagree"; a scale that does not follow this format exactly is called a "Likert-Type Scale"

Types of Validities

Construct Validity (quality of measurement, "How did you phrase the question?"), External ("is it generalizable to the larger population?"), Internal (No third variable problem, "are there any alternative explanations?"), Statistical (probability of error, "Was it statistically significant?")

Population

Everyone that you want to make a claim about, most participants are W.E.I.R.D because they are easily accessible (why most psych studies are biased), also called "population of interest"

Reliability

Extent to which outcome is found consistently; reliable measures are free from random error

Major types of Construct Validity (DON'T CONFUSE WITH "Types of Validities")

Face Validity: just by looking at it, can you tell that it's a measure of the variable? Content Validity: measure contains all the parts that your theory says it should contain Convergent Validity: measure is more strongly associated with measures of similar constructs; involves finding a scale & seeing if your construct is compatible with it; From lab: significant positive correlation between two scales of two different constructs=weak evidence for convergent validity Discriminant Validity: measure distinguishes itself from things it shouldn't be related to; to find this, look for things it should be related to but isn't and measure it

Cherry-picking evidence

When you only pick certain information that would support your claim and nothing that would counter it

Self-Report

Whenever the participant gives verbal information; usually a questionnaire or interview; advantages: fast, easy, direct; disadvantages: self-deception/lack of insight, reactive to factors like social desirability & cooperative responding

Reactivity

a change in behavior of study participants because they are aware they are being watched

Embodied Cognition

a construct; The way we feel will affect how we think in ways we can't always notice; ex: In Lewendowski study, it's the visceral perception of temperature change

Semantic Differential format

a response scale whose numbers are anchored with contrasting adjectives

Oversampling

a variation of stratified random sampling in which the researcher intentionally overrepresents one or more groups

Non-Reactive measures

behavioral, observational, psychological; Disadvantages: cost, time, training & bridging the gap

APA style title

communicates purpose of research in 12 words or less; does not use the words "method", "results," "a study of," or "an experimental investigation of," centered and not boldfaced

Construct Validity

operational definition matches the concept you set out to assess

control group

do not receive treatment

issue with generalizability

external validity error; to prevent this, researchers need to ensure that the participants in the sample adequately represent all of the population of interest

APA style intro

first paragraph: describes general area of research, middle paragraphs: summarize & interpret past research studies (gets at the gap), last paragraphs: describe the method & primary variables & state hypothesis or research question

Systematic error (bias)

fluctuation in measurement that systematically increases or decreases measurement of the true score due to identifiable variables (confounds); something that you should have controlled for; error does not cancel out, threatens validity;

The Importance of Theory

foundation of hypotheses and outcome of studies

Sampling

getting a subset of the population of interest; sampling technique matters more than size; Small sample ok if it's unbiased, large samples can be flawed

Present Present Bias

heuristic, tendency of people to give stronger weight to payoffs that are closer to the present time

Multistage sampling

involves at least two stages: a random sample of clusters followed by a random sample of people within the selected clusters

APA style references

listed in alphabetical order by author's last name; within entries, the author matters (first author contributed most, last author contributed least); starts on new page after Discussion

Observational measures

measuring a variable by recording observable behaviors or physical traits of behaviors, aka behavioral measures

Observer bias

occurs when observers' expectations influence their interpretation of the participants' behaviors or the study's outcome