PSY 301- University of Oregon

Internal Validity

In a relationship between one variable (A) and another (B), the extent to which A, rather than some other variable (C), is responsible for the effect on B.

double-barreld questions

asks two questions in one; have poor construct validity because people might only be responding to the first half of the question, the second half, or both

observer effects

observers change the behavior of those they are observing, such that participant's behavior changed to match observer expectations; can occur even in seemingly objective observations;

observer bias

occurs when observer's expectations influence their interpretation of the participants' behaviors or the outcome of the study; rate behaviors according to their expectations or hypotheses

reactivity

occurs when people change their behavior (react) in some way when they know another person is watching; do not display their typical behavior

leading questions

avoid them! one questions wording affects the way they answer the next; ex one question has more negative context vs the other has more positive changing the responses

Why is precision important to scientists?

scientists use theories to guide hypotheses; they state these hypotheses using operational definitions

Internal reliability

the extent to which multiple measures or items are all answered the same by the same set of people

What is the Empirical Method?

The use of verifiable evidence as the basis for conclusions; collecting data systematically and using it to develop, support, or challenge a theory. Base one's conclusions on systematic observations.

Parsimony

"all other things being equal, the simplest solution is the best" if two theories explain the data equally well but one is simpler, most scientists will opt for the simpler theory

Type I Error

"false positive"

What was Milgram's obedience study?

"teacher" vs "learner" ; as teacher you have to punish the learner when they respond incorrectly and with each error there is an increase in the level of shock; the experimenter tells you to continues administering the shock even though the learner is screaming, the participants did not know the learner wasn't actually being shocked; caused a lot of stress on the teacher

What is the typical anatomy of an empirical article, and what goes into each section?

1. Abstract-summary 2. Intro- intro to the research and hypothesis 3.Methods- how it is tested; who participated 4. Results- main findings 5.Discussion- why does it matter?

Theory Data Cycle

1. Theory- how variables relate to each other 2. Research questions- specific to the theory 3.Research design- 4. Hypothesis- prediction; way of stating the specific outcome the researcher expects to see 5. data- set of observations 6. supporting data strengthens theory.. no supporting data leads to revise theory or make a new design

What was the Tuskegee syphilis study?

600 african american men with syphillis were lied to about treatment for their illness and some were not even aware they had the STD and when there was an appropriate cure they did not provide it; VERY UNETHICAL

correlation method

does your measure correlate with the behavior or outcome of interest?

What makes a sample representative? how can it be achieved? when does it matter most or least?

A sample is representative when it gives all people of the population a chance of being chosen. Can be achieved through simple random sampling, cluster sampling, multiple stage sampling, stratified random sampling, oversampling, systematic sampling, etc. Matters when wanting to generalize to population

Falsifiability

A theory must lead hypotheses that, when tested, could actually fail to support the theory.

Interrogating Association Claims

Construct Validity: How well did the researchers measure each variable? External Validity: Can it generalize to other populations, as well as to other contexts, times, or places? How representative is the sample? To what other problems might the association be generalized? Statistical Validity: What is the effect size? how strong is the association? is the association statistically significant? false positive? Internal Validity: not relevant

Interrogating Frequency Claims

Construct Validity: how well did the researchers measure their variables? How accurately did they operationalize the variable? External Validity: How did the researcher choose the study's participants and how well do those participants represent the intended population? Statistical Validity: How well do the numbers support the claim? (margin of error) Internal Validity: not relevant

Interrogating Casual Claims

Covariance Temporal Precedence Internal Validity: Was the study an experiment? does the study achieve temporal precedence? does the study control for alternative explanations by randomly assigning participants to groups? Does the study avoid the threats to internal validity? Construct Validity: How well was the researcher measured or manipulated the variables in the study? Statistical Validity: What is the effect size? Is there a difference between groups, and how large it is? Is the difference statistically significant? External Validity: To what populations, settings, and times can we generalize this causal claim? How representative is the sample? How representative are the manipulations and measures?

Why is systematic research usually preferable to simple experience?

Experience has no comparison group; research asked the critical question "compared to what?" which allows researcher to compare what happens with or without the thing they are interested in

Semantic Differential

Fats are Unhealthy 1 2 3 4 5 Healthy (numeric system that is anchored with adjectives)

Why is skepticism important to scientists?

Must treat conclusions with caution

Population vs Sample vs. Census

Population- the entire set of people or products in which you are interested Sample-smaller set taken from that population Census-testing the whole population

Representative vs Bias sample

Representative- all members of a population have an equal chance of being included in the sample; allow us to make inferences about the population Bias- some members of the population

Theory vs. Hypothesis

Theory: a statement or set of statements that describes general principles about how variables relate to one another Hypothesis: a statement of the specific relationship between a study's variables that the researcher expects to observe if a theory is accurate. -different than everyday definition of a theory because these theories spark questions that are actually tested...everyday definition is more of a guess

Covariance

X and Y are correlated

Temporal Precedence

X comes before Y

Internal Validity

Z does not cause X and Y; no alternate explanations

What is an Institutional Review Board?

a committee responsible for interpreting ethical principles and ensuring that research using human participants is conducted ethically

reliability vs measurement validity

a measure can be reliable but it might not be valid for its intended use; it can be less valid than reliable but not less reliable than valid

Content validity

a measure must capture all parts of a defined construct; must assess all components of variable like intelligence

Surveys as self report measures

a way of posing questions to people on the phone, in personal interviews, on written questionnaires or online.

open-ended questions

allows respondents to answer any way they like; provides researchers with spontaneous, rich information. Drawback is responses must be coded and categorized, which is difficult and time consuming

Confounds

alternative explanations; occurs, essentially, when you think one thing caused an outcome but in fact other things changed too so you are confused about what the cause really was

Cronbach's alpha

an average of all the possible inter-item correlations. the closer to one the better the reliability. if internal reliability is good researchers can average all the items together

fence-sitting

answering in the middle of the scale; to prevent take away the neutral option or give forced-choice with only 2 options

Ordinal scale

applies when numerals of quantitative variables represents a ranked order (ex top 10 selling books)

Association Claim

argues that one level of a variable is likely to be associated with a particular level of another variable; related; must involve at least two variables that are measured not manipulated; positive, negative, or zero association

Causal Claim

argues that one of the variables is responsible for changing the other; has three criteria: covariance, temporal precedence, and internal validity

order of survey questions

can affect responses to a survey; the earlier questions can change the way respondents understand and answer the later questions

negatively worded questions

can cause confusion therefore affecting the construct validity of a survey; makes questions unnecessarily complicated; ex abortion should never be restricted

blind (or masked) design

can prevent observer bias/effects; observers are blind to which conditions participants have been assigned and are unaware of what the study is about

Categorical variables

categories (sex,species,etc)

Quantitive variables

coded with meaningful numbers (height and weight)

Meta-analysis

combines the results of many studies and gives a number that summarizes magnitude of a relationship

Belmont Report (Respect for Persons, Beneficence, Justice)

conference in which a short document was made that outlines three main principles for guiding ethical decisions making -Respect for Persons: individuals participating in research should be treated as autonomous agents(free to make up their own minds about whether they wish to participate) entitled to informed consent; people who have less autonomy are entitled to special protection (ex children, disabled,prisoners) -Beneficence: researchers must take precautions to protect research participants from harm and ensure their well being; must carefully asses risks and benefits; must consider who could benefit or who could be harmed -Justice: calls for a fair balance between the kinds of people who participate in research and the kinds of people who benefit from it; people must be representative of the people who would benefit from its results

Frequency Claim

describe a particular rate or degree of a single variable. Claim how frequent or common something is. mentions a percentage of a variable, the number of people who engage in a certain activity, or a certain group's level

Empirical article vs. Review article

empirical- report on the results of an empirical research study and contain details on the study's method, the statistical tests used, and the numerical results review- provides a summary of all the published studies that have been does in one research area

Comparison groups

enables us to compare what would happen with and without the thing we are interested in

Criterion validity

evaluates whether the measure under consideration is related to a concrete outcome, such as behavior, that it should be related to, according to the theory being tested.ex previously using IQ test to measure sales aptitude but want to develop a better test...do the new test scores actually correlate with the key behavior (good sales)

What constitutes data to scientists?

good data is large in quantity and variety; a theory must change to accommodate data

External Validity

how well the results of a study generalizes to, or represent, people or contexts besides those in the study itself

What is the cherry-picking problem?

ignoring inconvenient data; only acknowledge certain data that aligns with what we want to see

Statistical Validity

is the extent to which a study's statistical conclusions are accurate and reasonable

Why is openness important to scientists?

it allows for replication, checks and balances, and existing in literature

Survey question wording

it is crucial that each question be clear and straightforward to answer

What are the components of measurement validity?

it is established with subjective judgments or with empirical data; establishes construct validity; face validity, content,criterion,convergent, and discriminant

Probabilistic

its (behavioral research) findings are not expected to explain all cases all of the time. Instead the conclusions of research are meant to explain a certain proportion of the possible cases.

Independent variable

manipulated variable

Dependent variable

measured variable

Type II Error

might mistakenly conclude there is no association when there really is; "miss"

Response set

non differentiation; shortcut when answering questions; people answer the same to all questions

ratio scale

numerals of a quantitative variable that equal intervals and when the value of 0 truly means "nothing"

interval scale

numerals that represent equal intervals (distances) between levels and there is no "true zero"(a person can get a score of 0, but zero doesn't mean "nothing")

unobtrusive observations and data

observations: make yourself less noticeable (to avoid observer effects) data: avoid reactivity; measure the traces that behavior leaves behind instead of measuring behavior; ex measuring the amount of empty liquor bottles to see how much alcohol is being consumed rather than observing how much someone drinks

Experiment

one variable is manipulated and the other is measured

Physiological measures

operationalizes a variable by recording biological data such as brain activity, hormone levels, or heart rate.

Self-report measure

operationalizes a variable by recording people's answers to questions about themselves in a questionnaire or interview

Confirmation bias

paying extra scrutiny or extremely skeptical of research that contradicts your own idea and less critical of work that fits with your idea

Likert Scale

people are presented with a statement and are asked to answer using a rating scale (strongly agree, agree, neither, disagree, strongly disagree) or could be likest-like scale and be (1-5)

What is the problem with "common sense" stories?

people do not generally step forward with non-examples

forced-choice questions

people give their opinion by picking the best of two or more options; researcher adds up the number of time people chose responses

Construct Validity

refers to how well a conceptual variable operationalized

Validity

refers to the appropriateness of a conclusion or decision; in general a valid claim is reasonable, accurate, and justifiable

self-reporting more than you know or remember

researcher cannot assume the reasons people give for their behavior are their actual reasons. people may not be able to accurately explain why they acted.

known groups paradigm

researchers see whether scores on the measure can discriminate among a set of groups whose behavior is already well understood. ex salivary levels to measure stress.. measure for speaker and for audience since speaking is a known stressor

Is deception ever used?

researchers withheld some details of the study from participants-deception through omission deception through commission- they actively lie to them; used in some cases but the principle of beneficence must be applied: what are the ethical costs and benefits of doing the study with deception?

nominal scale

same as categorical

Confirmatory Hypothesis Testing

selected questions that would lead to a particular,expected answer; conducted in a way that is decidedly not scientific

Discriminant validity

should correlate less strongly with measures of different constructs

Correlation Coefficient (r)

slope direction: postive=up; negative=down; zero=none strength of relationship: r falls between -1.0 and 1.0 and the closer r is to zero the weaker it is (sign indicates relationship)

waiting it out

solution for reactivity; let participants get used to the observers presence so they will display their typical behavior

Observational measures

sometimes called a behavioral measure; operationalizes a variable by recording observable behaviors or physical traces of behavior. ex a researcher could operationalize happiness by observing how many times a person smiles

inter-rater reliability

the degree to which two or more coders or observers give consistent rating of a set of targets

Face validity

the extent that it appears to experts to be a plausible measure of the variable in question; subjective judgment; if it looks good it has face validity

Effect size

the magnitude of a relationship between two or more variables

Convergent Validity

the measure should correlate more strongly with other measures of the same constructs

Test-retest reliability

the researcher gets consistent scores every time they use the measure. applies whether operationalization is self-report, observational, or physiological. Primarily relevant when researchers are measuring constructs that they expect to be relatively stable in most people

informed consent

the researchers obligation to explain the study to potential participants in everyday language and give them a chance to decide whether to participate. in most settings it is obtained by providing a written document that outlines the procedures, risks, and benefits. sometime not necessary when there is no potential harm(like in a educational setting)

What is Random Assignment?

the use of a random method (e.g. flipping a coin) to assign participants into different experimental groups

Give an example of an availability heuristic?

things that pop up easily in our mind then to guide our thinking; can lead us to overestimate frequency example: death by fire or falling? - falling occurs more frequently but we say fire because it is more memorable and imagined to be more fatal

inter-rater reliability

two or more independent observers will come up with consistent (or very similar findings). most relevant for observational measures

Give an example of Bias Blind Spot

we are biased to being bias. most of us think we are less biased than others. example: people who like the president may think anybody who disagrees with him must be unintelligent, failing to recognize their own bias in the president's favor

Present bias

we often fail to looks for absences and it is easy to notice what is present; it is related to the need for comparison groups.

Observational data

when a researcher watches people or animals and systematically records how they behave or what they are doing; basis for frequency claims; usually more trust worthy than self-report

acquiescence ("yea-saying")

when people say yes or strongly agree to everything; to prevent/catch throw in reverse wording

Debriefing

when researchers us deception they must spend time after the study debriefing each participant in a structured conversation; describe the nature of the deception and explain why it is necessary and to restore an honest relationship with the participants

Socially desirable responding (faking good)

when survey respondents give answers that make them look better than they really are. respondents are embarrassed, shy, or worried about giving an unpopular opinion; solution: anonymity