Research Methods

repeated measures ANOVA

- a statistic used to test a hypothesis from a within-subjects design with three or more conditions

simple random sampling

- probability - each member of population as equal change of being selected for the same - selection is independent of one another - each

general sampling methods

- probability - non-probability

cluster sampling

- probability - used when naturally occurring pockets of population are geographically dispersed - randomly select from pockets

Predictor Variable

What factors may be related to the behavior of interest? (independent v) examples: Recess time, Public v private school, proportion of boys in the class

Sampling Non-human elements

What helps *maximize generalization* by looking at *surrounding* elements?

Increase effect

What increases power by *upping* the intensity of tx or the *independent variable*?

Correlational

What involves *measuring relationship* between different variables whether *direct* or *indirect*?

File drawer problem

What is another name for *publication bias* which is where only the significant results are more likely to be published?

Research hypothesis

What is another name for an *alternative hypothesis (HA)* in which you predict that a *relationship exists*?

Less confounds

What is the *strength* of a *lab* based experiment?

major types of bias (5)

investigator, selection, performance, attrition, detection

median

middle score

A factorial design with at least one independent-group factor, and at least one within-groups factor.

mixed design

A variable which changes the relationship between an independent variable and a dependent variable.

moderator

Chi-square

nominal or ordinal data addresses whether independent variables differ in their relative distributions on the dependent variable ex: whether the outcome is contingent on group membership

Mode is used when:

nominal scales and discrete variables

4 scales of measurement

nominal, ordinal, interval, ratio

Mortality

number of deaths

NNH definition

number of patients needed to be treated for one to be harmed

prospective timeline

present to future

zero correlation

no relationship between variables

internal validity

the degree to which the effects observed in an experiment are due to the intervention

Main effect

the effect each variable has by itself.

all professional activities of psychologists.

the ethics code covers _________.

uncertainty

the exact value of any biological property is difficult to know for certain

Skewed distribution

the extent to which a distribution leans to one side of the mean instead of being evenly distributed based on where tail is

True Experimental Design

utilizes both random assignment and comparison group

interaction effect

whether the effect of the original independent variable depends on the level of another independent variable

explain (goal of psych/res)

why did this behavior occur? which factors influenced this outcome

If it would cause them harm to know

why would you never debrief the participant?

assent

willingness to participate in a study

tenacity

wisdom from previous generations, "American dream"

Negative Correlation

x and y changes in *opposite* direction - more you exercise and less you weigh, more income and less likely to get diseases

Positive Correlation

x and y changes in same direction - more shy you are, the better you are at reading facial expressions

systematic sampling

- probability sampling - every "nth" subject on sampling frame is selected - starting point is chosen via random number table/generator

naturalistic observation

-subjects watched in a natural setting -disturbed as little as possible ex: Goodall watching chimps in Africa for 50 years

experimental manipulation

-true experiment

Number of variables

-univariate: 1 variable at a time -bivariate: more than 1

descriptive stats

-what should we describe? -center: what is common -spread: how far apart data is spread

p value cutoff

.05

Weak (correlation strength)

.10/-.10

Strong (correlation strength)

.50/-.50

What is the minimal number of members on a research ethics board (REB)?

5

Ethics in Research

5 General ethical Principles: 1. Beneficence & nonmaleficence 2. Fidelity & responsibility 3. Integrity 4. Justice 5. Respect for rights & responsibilities

Empiricism

5 senses, taking in information through observation

Marginal means are useful for seeing Simple main effects Two-way interactions Three-way interactions

A

Conceptual replication

A researcher has the same research question as another study but uses different procedures. This improves external validity.

Single-N design

A study in which researchers obtain a large amount of information from a single animal or person.

OR of 3.0

200% increase in odds

OR of 0.75

25% reduction of odds

Factorials displays

2X2 chart, graph

Negative skew

What type of frequency distribution is skewed to the *right*?

parallel RCT

subject randomized to treatment or control only

double blinding

subjects and investigators do not know randomization schedule

block randomization

subjects divided into blocks which are then randomized

single blinding

subjects do not know treatment group

laboratory observation

subjects watched in the laboratory ex: bobo doll experiment

Internal Validity

"Are there confounding variables?"

External Validity

"Can we generalize the results to the population?"

Face Validity

"Does it *look* like they are measuring what they think they are measuring?"

Statistical Validity

"What do the mathematical results tell me?"

Correlations

"r". Tells us direction, positive/negative, ranges from -1 to +1. the closer to -1 or +1 the stronger the correlation. The closer to 0 the weaker the correlation.

The Rational Method

- An example of seeking answers by logical reasoning - In this method the first two sentences are premise statements while the last is a logical conclusion based on the premises

Which critical evaluation question should we ask ourselves when evaluating lists of items cited?

- Is the references list current and complete?

Environmental Changes

- It is difficult to attain the ideal of identical circumstances - Example: Raining and students have an indoor recess

The Empirical Method (Empiricism)

- Answering questions by direct observation or personal experience - Based on the philosophy that all knowledge is acquired through senses

What are three steps to getting approved for human subjects?

- Application to REB - Research Proposal - Consent Forms

What are the 7 dimensions of ABA? *Remember GET A CAB*

- Applied - Behavioural - Analytic - Technological - Conceptual Systems - Effective - Generality

What are the three important principles to the scientific method?

- It is empirical - It is public - It is objective

ERIC Database

(Education Resource Information Center) provides access to education literature and resources

Institutional Review Board;5

(IRB) stands for ____________. They must have a minimum of _____ members.

Pro/Con of Close Ended

*Pro:* No coding required. May provide answers that they wouldn't have recalled. *Con:* May not include all options. Answers may be influenced by response options given (framing effect).

Pro/Con of Open Ended

*Pro:* Rich Info *Con:* Usually require coding, length/detail is not always clear, performance on recall worse than recognition for memory question

Every observation and survey will always have some degree of error T/F

*TRUE*

Experience has no comparison group; experience is confounded T/F

*TRUE*

Correlation & Strength of Association

+/- .10 - .29 = Weak +/- .30 - .49 = Moderate +/- .50 - .69 = Strong +/- .70 and above = Very Strong

independent groups

(aka between-subjects): Different groups of participants take part in different conditions.

dependent groups

(aka within-subjects) Only one group of participants. Each participant takes part in every condition.

What is the minimal number of replications needed?

3

Method of Faith

- Is a variation of method of authority - People have unquestioning faith in authority figure - Accept without questioning, challenge, or doubt

*Introduction*

- *A general introduction to the topic of the paper* - *A review of the relevant literature* - *A statement of the problem or purpose of the study with relevant variables clearly defined* - *A description of the research strategy that used to evaluate the hypothesis or to obtain an answer to the research question*

*When in APA reports are numbers used?*

- *Quantities above 10* - *Date* - *Age* - *Measurements* - *Units of time*

*Methods*

- A relatively detailed description of exactly how the variables were defined and measured and how the research study was conducted - *Participants, Materials, Process*

Three types of construct validity

1. Convergent 2. Discriminant 3. Criterion-related

OR of 1.5

50% increase in odds

typical power level

80%

Field settings

A real-world setting for a research study.

Who is the SLC research officer?

Anthony Wright

Parsimony

As succinct as possible

Respect of Persons

Autonomy, Informed Ongoing Consent

Introduction

Background, research question(s)

Beneficence/Concern for Welfare

Benefits outweigh the risks, unnecessary risks, confidentiality, informed

observer effects

By using a masked design, the researchers in the "Seeing Red" study minimized the risk of: Measurement error Observer effects Reactivity Confounds

4. A 3 × 3 × 2 factorial design includes how many levels in the second independent variable? A) 1 B) 2 C) 3 D) 4

C - 3

7. A(n) _____ is an experimental design that combines within-subjects and between-subjects methods of data collection. A) A-B design B) A-B-A design C) mixed design D) single-subjects design

C - mixed design

Mode of Delivery

Can affect responses to questions that may be affected by social desirability (e.g., online surveys individuals may be less swayed to give "desirable" answers

Temporal Precedence (3 criteria for causal claims)

Can you establish that x came before y?

What organization is in charge of animal care in Canada?

Canadian Council on Animal Care

Experimental Design

Cause & effect quantitative design

Causal Relationship

Cause and effect

Nonlinear Relationship

Change in one entity does not correspond to change in the other entity

Variables

Characteristics or conditions that change or have different values for different individuals

Splitplot

Combination of easily manipulated and difficult (impossible; ex. race, gender) to manipulate independent variables are both assessed for effect on dependent.

Benefit of Mixed design

Combines between-subjects and within-subjects designs. Strengths of each design offset the weaknesses of the other.

Active Deception

Commission

Autocorrelation

Correlation between *1 variable with itself*, measured at *two different time points*

Demand characteristics

Cues that inform the subject how he or she is expected to behave.

Review Article

Summary of published studies done in a research area (e.g., a meta-analysis)

Covariance Temporal precedence Internal validity

What are the three criteria for a causal claim?

ordinal scale of measurement

Data are assigned to categories that can be ranked.

Objective-Oriented

Determine extent to which program objectives have been achieved. Ex. Midterm & Final - identify discrepancies between stated objectives & desired outcomes

True

Ethics & research: The principle of doing no harm means that researchers should never subject research participants to physical danger, cause them psychological harm, or expose them to legal liability.T/F

causal claims

Experiments are conducted to make ________

Internal Validity

Extent to which independent variable truly effect changes in dependent.

Confirmation Bias

Focusing on the evidence that fits our beliefs a tendency to search for information that confirms one's preconceptions

marginal means

The arithmetic means for each level of an independent variable, averaging over the levels of another independent variable.

Varieties of Psychological Research

From observations, and serendipitous events

Participant-Oriented

Gain insight through the participants in the program. What do they think/need of the program? Different perspective than those who are implementing the program or funding it.

Empirical Approach

Gathering information using your senses. Relying on systematic observations! Unbiased Objective Publicly verifiable events

Covert Observer

In field research, refers to a researcher who is secretly studying a group without the group members knowing they are being studied (neo-nazi group)

Overt Observer

In field research, refers to a researcher who is studying the group members with their full knowledge

experimental design

Good ________________________ is important to doing successful research.

Healthy adherer effect

Greater adherence to health-promoting behaviours, such as medication adherence, is indicative of overall healthy behaviour

Continuous Variable

Have a large number of levels. -More than 10 levels Examples: -response time, cigarettes per day

variance

The average of the squared differences from the mean.

Placebos

Inactive substances, not real medications, that are used to test the effectiveness of drugs.

Method

How did they conduct the study?

Between Group difference

How do you know how much *difference* is expected *due to chance* between groups?

IRB stands for

Institutional Review Board

Null Hypothesis

Hypothesis where there is no significant difference between specified populations, always based on data already established. ex. Each bag has 17 M&M's.

FDA has to approve all protocols for _____

INDs

example of experiment

Indep: treatment -> level: 1- cognitive behavioral therapy 2- Family Therapy 3- wait list Dep: scores on Beck depression scale

Methods of Intuition

Information that is accepted on the basis of a hunch

Provide two examples of constructs

Intelligence and Hunger

Basic Research

Intended to solve theoretical issues

when the effect of one IV depends on the level of another IV.

Interaction effect

Questionnaire instructions

It is important that the directions for completing the questionnaire are very clear.

Internal validity

The certainty with which results of an experiment can be attributed to the manipulation of the independent variable rather than to some other, confounding variable.

Sample

The collection of subjects selected for observation in an empirical study.

Item-total correlation

The correlation between scores on individual items with the total score on all items of a measure.

Population

The defined group of individuals from which a sample is drawn.

Measurement error

The degree to which a measurement deviates from the true score value.

Reliability

The degree to which a measure is consistent.

Scatterplot Direction

Linear relationships can have a positive or negative direction

Nominal

Looking at categories (sex, species, race)

Limitations of Tenacity

Lots of potential inaccuracies and no method for correcting erroneous ideas

Give two examples of intangible and abstract attributes to operationally define

Motivation and Self-Esteem

- Effect Size - Sample Size

What are the two components of a significance test that when increased are the most effective way to *increase power*?

Parametric Statistic

Make assumptions about population fitting normal curve.

Nonparametric Statistic

Make no assumptions about the population fitting normal distribution.

Type 1 error

Making a claim of association or cause when there isn't one Rejecting the null when it is true

Type 2 error

Making a claim of no association or cause when there is one Not rejecting the null when it is false

Measurement error

The degree to which the measurement, used to assess a participant in a study, is different from the true measurement value. When these are random they are capable of inflating or deflating the true scores. But if they are systematic they may result in a biased measurement.

What two things must you be able to do with variables that are being studied?

Measure and Define them

What does the REB take into consideration?

Mitigation of risks, benefits outweigh risks, fair, informed consent

Type B

More relaxed and easy-going personality

Forced Choice Question

Multiple Choice, True/False, Yes/No questions are examples of...

number needed to harm

NNH = 1/attributable risk

Type II Error

Not claiming a result as significant when you should have. Fail to reject the null hypothesis when there is a difference. Missed a finding that is actually there. False negative. ex. Telling a 9 month pregnant woman that she is not pregnant.

Limitations of the Empirical Method

Observations can be misinterpreted, sensory experience can be swayed by beliefs, time consuming, can be dangerous relying on one person's opinion

Association Claims

One level of a variable is likely to be associated with a particular level of another variable. Also known as correlations or covariations

When selecting a mentor in the field, what is it important to consider?

One should be in the field for years prior to being considered a mentor

Causal Claims

One variable is responsible for changing another variable

Criterion Variable

Outcome variable being measured (dependent variable)

Psychology

The scientific study of behaviour and mental processes. It provides detailed and accurate knowledge about psychological problems and establishes whether treatments are effective.

Reduce Social Desirability Bias

Phrasing questions in ways that make norm violations appear less objectionable and that present a wider range of behavior as acceptable (e.g., face-saving alternatives)

Journalism

Popular press for general public

Exposure therapy is often used for what psychological disorder?

Post Traumatic Stress Disorder

Debriefing

Post-experimental explanation of the study to the participant (especially if deception is used)

Discriminant validity

The extent to which scores on the measure are not related to scores on conceptually unrelated measures. I.e. scores on a test designed to assess aggressiveness should not be positively correlated with scores from tests designed to assess intelligence.

Interestingness

The extent to which the answer is in doubt, whether it fills a gap in the research literature, and whether it has important practical implications

increasing the sample size using stronger manipulations eliminating unwanted sources of variance (noise)

Power is increased by:

A

Practice effects and carryover effects are examples of _________ effects. order scientific between-person causal

Basic Research

Primary goal is to gain knowledge

What are the pros and cons of journals and journalism?

Pros: easy to read, more likely to influence lives Cons: can be warped and misinterpreted

a study where the levels of the independent variable are naturally occurring, rather than manipulated by the researcher.

Quasi-experiment

What other REBs is it possible for SLC students to need to go through?

Queens/Various Hospitals

strongest type of study design in clinical research

RCT

Ordinal

Rank order, but not equally spaced Categorical data Frequency data and Spearman rho Nonparametric measures

Observer bias

Rather than rating behaviors objectively, researchers might (unknowingly) rate behaviors according to their expectations or hypotheses. - Ex: All participants saw the same taping of a football game between Dartmouth and Princeton. However, the number of observed rule infractions appeared to vary based on which school a person attended

Which method obtains answers from reasoning and logical conclusions?

Rationalism

Disadvantages of experimental design

Realism; Mundane vs. Experimental

Face-to-face interview

Require that the interviewer and respondent meet to conduct the interview

Graphic rating scale

Requires a mark along a continuous 100-millimeter line that is anchored with descriptions at each end.

coefficient of variation

SD/mean; useful for comparison of variability between groups where comparisons are difficult because of differences in scale or size of the subjects being compared

Probability Sampling

Sampling strategy that gives every subject in the population the same likelihood of being selected. Purpose: save time; cost; accuracy of generalization

Historical Control

What compares data to *known populations* or *previous sampling*?

Family Therapy is often used for what psychological disorder?

Schizophrenia

Secondary Sources

Secondhand reports in which the authors discuss someone else's observations

Bias Blind Spot

Seeing bias in others but not in ourselves

Alpha Level and meaning related to error

Set that likelihood at what is called an alpha level (usually .05) which means there are only 5 opportunities in 100 that a Type I error will exist.

Interaction

Situation in which the effect of one independent variable on the dependent variable changes, depending on the level of another independent variable.

Spearman Rho Correlation Coefficient (r sub s)

Special form of Pearson r used when one variable is ordinal and the other is continuous. Ex. Class rank (ordinal) and PCA score (continuous)

Internal Consistency - Split-Half Reliability

Splits the test in half computing a separate score for each half, and then calculating the degree of consistency between the two scores for a group of participants

STROBE

Strengthening The Reporting of OBservational studies in Epidemiology

Longitudinal Design

Studies in which the same variables are measured in the same people at different points in time *helps establish temporal precedence*

Double-Blind Study

Study in which neither the experimenter nor the subjects know if the subjects are in the experimental or control groups.

Characteristics of Good Scientific Theories

Supported by data Falsifiable Parsimonious (Occam's Razor)

Pseudoscience

System of ideas often presented as science

Coding Data

Systematically reorganizing raw numerical data into a format that is easy to analyze using computers For example, with nominal data, researcher codes single people as 1, married as 2, common-law/cohabitating as 3, divorced as 4, separated as 5, and widowed people as 6

The APA Ethics Code

Ten ethical categories (150 standards) to provide guidance for psychologists

Nuremberg Code (1947)

Ten guidelines for the ethical treatment of human subjects in research

Availability Heuristic

Things that are easily brought to mind guide our thinking, Especially problematic when trying to determine how often something occurs.

Repeated Measure design

What type of design is anytime there is *more than one measure*?

Measured Score

True Score + Error

Nonexperimental method

Use of measurement of variables to determine whether variables are related to one another. Also called correlational method.

Phi Coefficient

Used when both sets of data are nominal, dichotomous measures. Ex. Sex (male/female) and Passing the NCE (pass/fail)

How do we demonstrate consistency of a measure?

Two scores (one score from the new measure and one score from a well-established measure) are represented for each person in a scatter plot

Nonprobability sampling

Type of sampling procedure in which one cannot specify the probability that any member of the population will be included in the sample.

Operational Definitions

Types of variables involved in research

Understanding

Understanding is a precondition for greater depth, not an end in itself. Learn to think and act within a member's perspective. Empathy, or adopting another's perspective

Netnography

Understanding social interaction on the internet (digital media)

Concurrent Validity

Validated measure and behavioral outcome measure are administered at the same time

Predictive Validity

Validated measure is administered weeks, months, or years before behavioral outcome measure is taken

Interval

Values are ordered and equally spaced Continuous data Likert scales Parametric measures

Ratio

Values are ordered and equally spaced with a meaningful 0.

Humanistic view of research participant

View where the participant is a colleague who cooperates in providing data. Tends to be more sloppy.

- Increase sample size - Increase external validity

What are the *strengths* of *Multi-Center studies*?

Observational design

What type of design is simply *observing* relationships as they *exist*?

descriptive statistics

summarize data

- History - Maturation - Testing - Instrumentation - Selection - Regression to the mean - Diffusion or imitation of tx - Compensatory rivalry - Compensatory equalization

What are the *threats* to *internal validity*?

Faulty Thinking

We believe a theory that makes intuitive "sense"

*Why* we cannot make causal claims (from correlational data)

We don't know if x caused y Or if y caused x Or if a third variable (z) caused x AND y

Present/Present Bias

We tend to not look for absences—while it is easy to notice when something is present. It is a failure to notice appropriate comparison groups

Multiple measures

What *increases power* by including multiple ways to measure the dependent variables?

More comfounds

What is the *weakness* of *field* based experiments?

Masked design

When the observer does not know which group received the treatment.

observer bias

When the observer's expectations influence observations.

- Participant - Researcher

Who are "blind" in a *double blind study*?

Because if participants are randomly assigned to groups, the groups are unlikely to have systematic differences.

Why does random assignment reduce the chance of confounds?

can you be reliable without being valid?

YES

Nuremberg Trials

______ _______ led to the need for ethical controls in scientific research

The IRB

________ ensures that psychological studies present as little risk to subjects and participants as possible and have scientific merit.

Quantitative

_____________ research measures variables numerically and assumes that the social world can be understood objectively.

Perception-Based Needs Assessment

utilize surveys to identify the subjective needs of various client populations.

Acquiescence (Yea-saying)

agreeing or answering in a positive manner for all questions. The opposite strategy is sometime called nay-saying. - Try reverse-wording some questions. be careful about double negatives

variable

any measurable property that can take different values either within a subject, among different subjects, or between groups of subjects

Positive Correlations

as A increases, B increases

observational studies are more _____ than RCTs

ethical

study for rarer outcomes

case-control

coercion

compelling or influencing a decision to participate in a research study

highest level of validity

construct validity

Utility

do results provide users with practical, easily used information.

outliers

data points that are highly improbable, although not impossible

invalid data

data points that fall outside the defined range for that variable of data

safety monitoring

documenting and reviewing of adverse events

inferential statistics

draw conclusions that can be applied to broader population

Measurement Quality

generalizability, validity, reliability,. -a study can be reliable but not valid or vice versa. -Validity: you measured what you wanted to -Reliability: this can be repeated and you will get the same answer every time

theory to data

hypothesis deduced: studies designed and completed, outcomes provide or far to provide inductive support, falsification is possible

Good theories have...

parsimony (simpler explanation), productivity, and falsification

Sample

part of a population

Ways to Handle Missing Data: Pairwise Deletion

participant dropped in analysis for only missing variable. participants answer all items on one scale, but not another, kept in analysis for only completed items. Used in smaller data sets where less power is observed.

Concurrent measure design

participants are exposed to all levels of an IV roughly at the same time and a single measurement of the DV is recorded

odds definition

probability of one outcome divided by the probability of another

strongest type of observational design

prospective cohort

discrete variable

represent discrete categories or counted whole numbers

types of randomization

simple, block, stratified, adaptive

Mixed Methods: Single Phase

simultaneously collect both qualitative and quantitative data. One set of questions does not rely on the others. ex. survey with open-ended questions

Mixed Method Design Types: Triangulation

single phase approach that simultaneously collect both qualitative and quantitative data, merge the data and use results to understand a research problem. Qualitative, quantitative and previous research data carry equal weight.

types of blinding (4)

single, double, triple, open label

carryover effects

some form of contamination carries over from one condition to the next how much does one influence on the next (very similar to priming) (e.g., survey order)

Single-blind procedure

special type of within-subjects design using one participant (human or animal) or perhaps one group to assess changes within that individual or group participants are unaware of, or blind to, the type of treatment they are receiving, but the administrator knows

inclusion criteria

specific characteristics the investigator is interested in studying

science

synthesis of rationalism and empiricism / helps to correct own bias

content validity

the extent to which a test samples the behavior that is of interest. Specifically asking if all necessary constructs were captured

Computer-assisted telephone interviewing (CATI)

this technology provides the script the telephone interviewer is to follow. It also allows the telephone interviewer to record the responses into a database as they are given.

Levels

values that a variable can take on.

Conceptual Variable

variable of interest stated in agreed upon terminology. abstract concepts such as "shyness" or "intelligence" Also called a construct, conceptual definition

Mixed-mode surveys

—combining methods of conducting survey research, such as interview or mail, with online based survey methods has shown promise as a means of getting high response rates.

Electronic instrument design

—effective online surveys do not mirror a questionnaire exactly, but instead make specific considerations for question arrangement and layout.

control group

a level of an independent variable that is intended to represent "no treatment" or a neutral condition

statistics

a logic and methodology for the measurement of uncertainty and for an examination of the consequences of that uncertainty in the planning and interpretation of experimentation and observation

practice effects

a long sequence might lead participants to get better at the task, or to get tired or bored toward the end over time get better (e.g., IAT)

measures of central tendency

a measure of the average score in a distribution, such as the mean, median, mode

measures of variability

a measure of the degree of difference between scores in a distribution: range, variance, standard deviation

pilot study

a study completed before (or sometimes after) the study of primary interest, usually to test the effectiveness or characteristics of the manipulations

factorial design

a study in which there are two or more independent variables, levels, or factors allow a researcher to simultaneously examine the impact of 2 or more independent variables on the dependent variable. an experimental design that has more than one independent variable 2X2 factorial design

Internal reliability

a study participant gives a consistent pattern of answers, no matter how the researcher has phrased the question do the different questions/parts of the measure produce similar results?

Hawthorne effect

a subject changes their behavior because they know they are being monitored

missing data code

a symbol, such as an asterisk (*), that is entered in a cell that has no data

Experiments must

establish internal validity systematic variability unsystematic variability

Confound

an unanticipated variable not accounted for in a study that could be causing observed changes

Factorial Design

evaluate two or more independent variables, including interaction of the variables in relation to the dependent variable. Help account for extraneous variables; allows assessment of interaction.

factorial design

evaluates multiple interventions in a single experiment

complete counterbalancing

everyone gets a different order within participants ABBA between AB for some, BA for others

Panel Studies

exact same individuals surveyed at each time of data collection.

seeing red experiment

example of posttest only experiment

interference effects

exhaustion (e.g., memory)

placebo effect

experimental results caused by expectations alone; any effect on behavior caused by the administration of an inert substance or condition, which the recipient assumes is an active agent.

design confound

experimenter makes a mistake in designing the IV in which a second variable varies systematically with the intended IV poor internal validity cannot support causal claim

9.05 Debriefing

- Behavior analysts inform the participant that debriefing will occur at the conclusion of the participant's involvement in the research

Predictive Validity

- Scores obtained from a measure accurately predict behavior according to a theory - Example: Functional Analysis

Construct Validity

- Scores obtained from a measurement behave exactly the same as the variable itself - Example: Temperature going high and aggression increasing

Concurrent Validity

- Scores obtained from a new measure - Directly related to scores obtained from a more established measure of the same variable

error of central tendency

- a response bias where a participant tends to avoid using the extreme response alternatives on a scale - solution: use more alternatives (5-9 is optimal)

non-probability sampling

- chosen on some basis other than random selection - snowball - connivence - purposive

non-directional hypothesis

- a hypothesis that does not make a specific prediction as to how two variables are related - ex: someone will win the game

directioanl hypothesis

- a hypothesis that makes a specific prediction as to the exact nature of the relationship between two variables - ex: my team will win the game

manipulation check

- a measure that helps determine whether the manipulation effectively changed or varied the independent variable across conditions

survey

- a quantitative research strategy for systematically collecting information from a group of individuals - information is then generalized to a larger group of interest

correlational study

- a research approach that focuses on how variables relate to one another - helps identify associations but not causations

Correlation

A measure of the extent to which two factors vary together, and thus of how well either factor predicts the other.

Semantic differential scale

A measure of the meaning of concepts. I.e. Respondents are asked to rate any concept—persons, objects, behaviors, ideas—on a series of bipolar adjectives using 7-point scales

Correlation

A measure of the relationship between two variables

Range

A measure of variation computed as the difference between the highest and lowest scores in a distribution. For example, the range for the bus stop in front of the bar is from 25 to 35, or 35 - 25 = 10 years For ordinal-, interval-, and ratio-level data

interquartile range

A measure of variation in a set of numerical data, the interquartile range is the distance between the first and third quartiles of the data set.

Percentile

A measure of variation to tell the score at a specific place within the distribution. Example: median is 50th. For ordinal-, interval-, and ratio-level data

Standard Deviation

A measure of variation; it is also the most comprehensive and widely used. The "average difference" between all scores and the mean. Requires an interval or ratio level of measurement. The bigger it is, the bigger the average difference. The smaller it is, the more similar the values in a distribution are to one another

Inductive Reasoning

A small set of specific observations is the basis for forming a general statement about a larger set of observations

Stable-basline design

A small-N study where a researcher observes the participant's behavior prior to the start of treatment, in order to get a baseline. If the behavior is consistent during the baseline, the researcher is able to be more certain that the treatment is effective. This offers protection against maturation and regression.

Multiple baseline designs

A small-N study where researchers manipulate the independent variable/treatment over various amount of time, contexts, or situations. The dependent variable is then repeatedly recorded in order to establish a stable baseline between the intervention stages. When there is a clear association between the timing of the independent variable manipulations and the timing of any changes in the dependent variable, it offers evidence for internal validity.

non-probability sample

any sample in which little or no attempt is made to get a representative cross section of the population. e.g purposive - picking who to talk to, talking to mayors of towns

control variable

any variable that an experimenter holds constant on purpose

quantitative technology

application of mathematical techniques to describe an analysis of data

Statistical Control

applying the proper statistical analysis when evaluating outcome data to minimize effect of potential confounding variables. Control extraneous variables.

case-control study

compares a group of people with a disease to a group of people without a disease

chi-square (x^2)

compares observed frequencies to expected frequencies. Has to be nominal dichotomous data only. How to read: The higher the value, the more statistically significant difference exists. If you have cells with less than 5, look at Fisher test. If not, use Pearson. ex. There is a statistically significant difference in the relative frequency of individuals from different tracks and eating one side of Twix first.

Inductive Approach

An approach to inquiry or social theory in which one begins with concrete empirical details, then works toward abstract ideas or general principles

Matched Assignment

An assignment method in which subjects are first divided into subgroups based on a specific characteristic such as age, gender, and so forth; members of each subgroup are then randomly assigned to each group in the study to balance the characteristics across the groups. Alternative to random assignment, but difficult to assign similar characteristics.

Positive Relationship

An association between two variables such that as values on one increase, values on the other also increase

Negative Relationship

An association between two variables such that as values on one variable increase, values on the other variable fall or decrease

Census

An attempt to count everyone in a target population. Every 5 years (2021)

A case study

describes a survey conducted on college students.

Descriptive research

describes social trends and phenomena.

Expectancy effects (experimenter bias)

Any intentional or unintentional influence that the experimenter exerts on subjects to confirm the hypothesis under investigation.

Situation noise

Any kind of external distraction that is able to cause variability within-groups that complicate between-groups differences. This can be minimized by making sure the experimental surroundings are controlled for.

Extraneous Variables

Any variables other than the independent variable that seem likely to influence the dependent variable in a specific study.

Extraneous Variable

Any variables that you are not intentionally studying in the experiment.

Technological

Applied interventions are described well enough that they can be implemented by anyone with training and resources

Generality

Applied interventions are designed from the outset to operate in new environments and continue after the formal treatments have ended

Conceptual

Applied interventions arise from a specific and identifiable theoretical base rather than being a set of packages or tricks

Behavioural

Applied interventions deal with measurable behaviour (or reports if they can be validated)

Applied

Applied interventions deal with problems of demonstrated social importance

Effective

Applied interventions produce strong, socially important effects

Analytic

Applied interventions require an objective demonstration that the procedures caused the effect

Operationalization

Approximating the true variables of interests with one that can be measured or tabulated -single measure, e.g GPA -composite measure

Which method obtains answers from an expert?

Authority

Observer bias Demand characteristics Placebo effects

However, even with a comparison group there can still be other issues:

participants;subjects

Humans in research are referred to as ___________, and animals are referred to as ___________.

Methods of Tenacity

Information that is accepted as true because it has historically been believed or superstition supports it

Informed Consent

Informing the research participants of the nature of the study and obtaining their explicit agreement. Ensures that researchers do not violate the privacy of participants and that participants have enough info to decide whether they want to participate

objective measures; Counterbalancing

Instrument threats can be prevented by using ____________ at pretest and posttest. Or, by using a posttest only design. __________ also helps.

Interviewer bias

Intentional or unintentional infl uence exerted by an interviewer in such a way that the actual or interpreted behavior of respondents is consistent with the interviewer's expectations

In which two aspects should research questions be evaluated in?

Interestingness and Feasibility

Which method obtains answers from a hunch or feeling?

Intuition

Theory-testing mode

Investigating the support for a theory.

Scales of measurement

Nominal Ordinal Interval Ratio

Alternative explanations

Part of causal inference; a potential alternative cause of an observed relationship between variables

Placebo effects

Participant group improves when they think they are given the treatment for the experiment but are actually not given the treatment.

within-groups repeated measures counterbalance

Participants are exposed to different conditions at different times. Researchers need to _________ the order of presentation.

concurrent measures contact/comfort with monkeys

Participants are exposed to different conditions at the same time. example?

Informed Consent

Participants are informed in advance of all aspects of the research that might influence their decision to participate

Justice

Participants are not exploited and treated fairly

posttest only

Participants are randomly assigned to groups, and tested on a dependent variable once. It's a ________ because the DV is measured after the IV.

Debriefing

Participants are told about the nature of the study once it has been completed

Testing threat

Participants change as a result of the dependent variable/test being given to the subjects more than once. This can be stopped by including a comparison group or only using a posttest-only design experiment.

Reactivity

Participants may simply act differently when they know they are being observed. Possible solutions include: • Unobtrusive observations (blend in). • Wait it out (allow acclimation to one's presence). • Measure the results of a behavior (indirect measures of constructs), rather than the behavior itself. - remember to observe ethically

Self-report Measurement

Participants provide responses to verbal questions about themselves (or others) through surveys or interviews

Observer effects (expectancy effects)

Participants' behavior changes to better match the observer's expectations. - reactivity type

Assumption

Parts of social theories that are not tested, but act as starting points or basic beliefs about the world. They are necessary to make other theoretical statements and to build social theory

Two Forms of Deception

Passive Deception Active Deception

True

Peer review is a collaborative process whereby researchers who are knowledgeable about a particular subject are asked to review and comment on another researcher's work and recommend whether it should be published. T/F

Journals

Peer-reviewed articles for scientists

Ethical Survey

People can conduct surveys in ethical and unethical ways -Major issue is the invasion of privacy -Treat all respondents with dignity and reduce anxiety -Responsible for protecting the confidentiality of data. -Voluntary participation -Some people use surveys to mislead others and produce desired results -Currently, there are no quality-control standards to regulate the opinion polls or surveys reported in the Canadian media.

If data were not public, what may occur?

People could rearrange their data so it looks better

random assignment if you have a large sample... Or, with matched groups if dealing with a smaller sample.

Selection effects can be avoided by

Scatterplot Precision

The amount of spread in the points on the graph. Close together means highly related, further apart less closely related

Mean

The arithmetic average of a distribution, obtained by adding the scores and then dividing by the number of scores. Used only with interval or ratio data. Compute by adding up all scores, then divide by the number of scores -Strongly affected by changes in extreme values -Used in most statistics

Mechanical Method

The assignment is automatic - not based on any personal preference or specific cases

exhaustiveness

The capacity for a list of attributes to contain all reasonably possible levels of a variable is referred to as:

Predictive validity

The construct validity of a measure is assessed by examining the ability of the measure to predict a future behavior.

Validity

The degree in which the measurement process measures the variable it claims to measure

Reliability

The degree of stability or consistency of measurement

Face validity

The degree to which a measurement device appears to accurately measure a variable.

Construct validity

The degree to which a test measures what it claims to be measuring.

Marginal means (how they are represented & calculated)

The means for each level of an independent variable averaging over the levels of another independent variable in a factorial designs study.

Generalization mode

The researcher's intent to generalize their findings from their study to other populations or contexts.

Systematic

The scientific method has been developed as a set of strategies researchers can use making research what?

Statistical correlation

What is a type of *significance test* that does not tell about random assignment?

Symmetrical distribution

What is also known as *normal distribution* when the frequency *distribution* from both sides of the *median mirror* the other?

Normal distribution

What is also known as *symmetrical distribution* when the frequency *distribution* from *both sides* of the *median mirror* the other?

Procrustean Data Torturing

What is an *analysis and publication bias* where *data* is *manipulated* to support a *hypothesis*?

Opportunistic Data Torturing

What is an *analysis and publication bias* where a researcher analyzes to find anything significant, then develops a *hypothesis based off* the *data*?

Publication Bias

What is an *analysis and publication bias* where only the *significant results* are more likely to be *published*?

Research questions

What is an *answerable inquiry* into a specific concern or issue that is the *initial step* in a research project?

Variable

What is an element of a hypothesis that involves *how* a *construct's* *operationalization* is *measured*?

Operationalization

What is an element of a hypothesis that is *how a construct* is defined within one specific study?

Construct

What is an element of a hypothesis that is an abstract representation including both *independent* and *dependent variable*?

Double blind study

What is an experiment in which neither the subjects nor those who administer the tx know who is in the experimental group or control group that is and *limits experimenter expectancy bias*?

Random assignment

What is an experimental technique for assigning participants or animal subjects to different groups in an experiment using randomization and lends strength to *internal validity*?

Confound

What is any *other explanation* for the *results* other than the hypothesized explanation?

Dispersion

What is the *distribution* of *values around* some *central value*, such as an *average* which observes how typical that value is?

Significance test

What is the *effect size* x the *sample size*?

Alternative hypothesis (HA)

What is the *hypothesis* in which you predict that a *relationship exists*, also known as *research hypothesis*?

Null hypothesis (Ho)

What is the *hypothesis* that there is *no relationship* or that nothing happens in which *you want to reject*?

Mode

What is the *most frequent value* that occurs?

Frequency

What is the *number of times* a value occurs?

Confidence interval

What is the *range of values* within which a population parameter is estimated to lie, which is the mean difference on test scores which speaks to validity?

Extreme values are, by definition, unusual. Therefore, the next measurement will most likely be less extreme (closer to the mean).

explain the concept of regression to the mean

Ethical Issues in Research Analysis: Data Dredging

exploring data post hoc for significant findings in the absence of significant results for the stated hypothesis. Playing with data looking for significance after analysis.

withing subjects

expose participants to all levels of treatment - randomly assign a sequence of treatment conditions - + power -internal and external validity

between subjects

expose participants to one level of treatment - randomly assign participants to one condition- + internal and external validity -power

OR > 1

exposure associated with higher odds of outcome

OR < 1

exposure associated with lower odds of outcome

Face validity

extent to which respondents can tell what the items are measuring

external validity

extent to which we can generalize findings to practice

Anatomy of a two way Factorial ANOVA - 4 groups (2X2)

f(#,#) = #.##, p= ##, eta^2 = .## f = f test symbol (first number) is between subject degrees of freedom and within subjects degrees of freedom #.## = f score p ## = significance level eta ##= calculated effect size

Types of measurement validity

face validity, content validity, convergent validity, discriminant validity

population-based case-control

has a defined base population

in retrospective studies, outcomes have/have not occurred

have

in prospective studies, outcomes have/have not occurred

have not

Control Variable

held constant on purpose

Internal consistency reliability

the assessment of reliability using responses at only one point in time.

strengthening internal validity (2)

include control group, randomization

Positive linear relationship

increases in the values of one variable are accompanied by increases in the values of the second variable

A research design where there are different participants at each level of the independent variable.

independent group design Example: one group gets a treatment, the other gets a placebo.

predictor variable

independent variable

Quasi-Experimental Design

independent variable cannot be truly manipulated. Comparison group, but no random assignment. Use of male vs. female, ethnicity, SES.

Factorial within-groups

independent variables are manipulated as within-groups. - example: 2x2 study, there is only one group of participants experiencing all levels/conditions

open cohort

individuals can join or drop out of study over time

placebo

inert treatment

accuracy

infer the population parameters with as little uncertainty as possible; unbiased

I-Data

informant report

unobservable confounders (2)

instrumental variable methods, Heckman selection model

helps to minimize attrition bias

intention-to-treat analysis

Reactivity

participants can change their behavior (react) when they know another person is watching - even if observer didn't give any cues about expectations Solution: Unobtrusive Observation (hide) Let people habituate (wait it out) Measure behavior's results

retrospective timeline

past to present

cases

people afflicted with a disease

SAMPLING: population

people we want to know about

P-data

physiological (Heart Rate, Skin detection, etc)

efficacy

physiology, survival, and quality of life in a near-perfect setting

types of control groups (4)

placebo, active control, historical control, non-inferiority trials

confound

potential threats to internal validity. Means to confuse

regression

predicting the value of one variable from another based on their correlation

ratio scales of measurement

order plus equal intervals plus true zero, score of 0 = no amount of variable being measured (true 0)

interval scales of measurement

order plus equal intervals, intelligence scales & personality scales

UNDERSTANDING DATA: dependent variable

outcome, what you're interested in, how it varies across certain population (effect)

margin means

overall means - the means for two groups or cells Main effects of the two IV (pick up lines, scent) the average of all participants on one level of the independent variable ignoring the other independent variable

cohort studies _____ clinical trials

parallel

repeated measures design

participants are measured on the dependent variable more than once after than have been exposed to each level (or condition) of the independent variable

Power

probability that a study will show a statistically significant result when an independent variable truly has an effect in the population.

Matching

process of selecting individuals to represent treatment and comparison groups based on specific similar criteria or characteristics. Similar traits evenly matched. People.

Hypothesis

proposed explanation made on the basis of limited evidence as a starting point for further investigation

Mixed Methods

provides more robust data, brings out unique variances and ideas and can help reduce bias. Must be proficient at both types of research design (qualitative and quantitative). Research question order determines the priority of the methods used, must relate question to type of design, and ensure questions are specific and measurable relative to type of design employed.

Double-Barreled Questions

questions that *attempt to get at multiple issues at once*, and tend to receive incomplete or confusing answers

simple randomization

random number generator

cohort studies differ from RCTs as cohort studies do not _____

randomize exposure

Interquartile range

range from the 25th percentile to the 75th percentile

measures of variability

range, variance, standard deviation, IQR

Ordinal

rank, order. Meaningful but unequal intervals (1st, 2nd, 3rd)

minimal risk; nine

research must represent ________ _______ and fall into one of _______ categories in order to be expedited.

STROBE checklist (7)

research question, study design, study population, exposure and outcome, data analysis and confounding, presentation of results and interpretation, practice and policy implications

Manipulated Variables

researcher sets up conditions to produce it.

Experiment

researchers manipulated at least one variable and measured another

Alternative Hypothesis (H1)

researchers prediction but cannot be absolutely proven

Ethical Responsibility

responsibility to individual subjects, to discipline of science, and to society as a whole

similar to stable-baseline, but treatment is taken away after improvement to see if progress reverses.

reversal design: Example: how will depressive/coping behavior change in response to treatment and nontreatment periods?

risk difference equation

risk exposed - risk unexposed

risk ratio equation

risk exposed divided by risk unexposed

factors that threaten external validity (5)

selection, pretesting, research setting, history, multiple treatments

S-data

self-report

3 types of operationalizations

self-report, observational, physiological

Ethics

set of principles for behaving in a way that is morally correct

Familiarity with questionnaire

the interviewer should be very familiar with the overall format of the survey and every item.—

median

the middlemost score in a distribution

mode

the most common score in a frequency distribution

animal rights

the notion that animals have the same sort of rights as people, including legal rights; not generally accepted

Single-blind experiment

the participant is unaware of whether a placebo or the actual drug is being administered.

Observer Effects

the participant picks up on observer's unintentional cues and responds the way they think they should Solution: Develop good scoring rubrics Use more than one observer

double-blind procedure

the participants and administrators of treatment are unaware, or blind, to the types of treatment they are administering to reduce the influence of expectancies they know the condition might have in order to reduce the likelihood that expectancies or knowledge of condition will influence the results, both the participants and the administrators of treatment are unaware of, or blind to, the types of treatment being provided

treatment group

the participants in an experiment who are exposed to the level of the independent variable that involves a medication, therapy, or intervention

Sensitivity

the percentage of cases with a diagnosis or condition who are correctly identified

specificity

the percentage of cases without the diagnosis or condition who are correctly identified ability to avoid false positives

Time horizon (in decision analytic modeling)

the period of time during which the outcomes are predicted by a decision analytic model

the investigator

the person who is in overall charge of the research and has the greatest responsibility to see that ethical principles are followed

Placebo effect

the phenomenon in which some people experience some benefit after the administration of a placebo.

Median - NOT how to calculate it, just what it is

the point that divides the distribution into two parts, such that an equal number of values falls above and below that point

Power

the probability of rejecting a false null hypothesis the probability of finding statistical significance when the result is significant

debriefing

the process of informing participant after the session of the experiment's true purpose to increase their understanding and to remove possible harmful effects.

data reduction

the process of transcribing data from individual data sheets to a summary form

Confidence interval

the range of values within which a population value is likely to fall, estimated at a certain level of probability

Null hypothesis

the relationship between or among the study variables is not as hypothesized by the investigator an idea that competes with the working hypothesis

negative correlation

the relationship between two variables in which one variable increases as the other variable decreases

Standard Deviation

the square root of the variance

standard deviation

the square root of the variance

standard deviation

the square root of the variance; a measure of variability in the same units as the scores being described

Correlation

the strength of the relationship between two variables

independent variable

the variable that you have chosen to manipulate in the experiment in which you are setting up

dependent variable

the variable you will measure in your experiment, that will change in accordance to the independent variable

constructs

theoretical concepts formulated to serve as casual or descriptive explanations: Milgram's construct was obedience

Research Question

to be mixed method design, both qualitative and quantitative research questions must be included.

IRB mission

to protect all human subjects from physical and psychological harm caused by experimental research

number

to remove bias and to protect the identity of the participant give them a _________ to be recognized by.

operationalize

to turn a concept of interest into a measured or manipulated variable

authority

trusting those who are "above" you

the commitment to expand knowledge and the cost to research participants

two values that create conflict in the decision to conduct research

What specific individuals and number of those people should be on a research ethics board (REB)?

two with expertise, one the ethics, one with no affiliation

mortality rate

usually annual number of deaths in a year, divided by the population in that year

Evaluation research

utilizes a wide range of methods to assess the efficiency or effectiveness of a program, policy, or process

Dependent variable

variable that is being measured

Dependent Variable

variable that reflects the impact of the manipulated variable

extrinsic source of variation

variation caused by imprecision of measurement (human error) or sampling variation (not biologically real)

Ethical Issues in Research Analysis: Coarsened Data

various techniques such as: rounding, grouping or transforming missing data changes or reduces amount of specificity of the data to reach significance.

Perspective (in decision analytic modeling)

viewpoint from which a decision analytic model is conducted

when all participants plan to receive treatment, researchers may compare those who receive early treatments to those who are still waiting (a strategy to reduce selection effects)

wait-list design

Propriety

was evaluation conducted in an ethically appropriate manner.

systematic variability

way to remove a confound in statistical analysis

can support an association (but NOT a causal claim)

we haven't measured every variable, haven't accounted for 3rd variable

change or influence/control (goal of psych/res)

what can we do to change or modify this behavior?

independent

what is making the outcome happen; explanatory variable (cause)

interval/ratio variables

what you report will depend on shape of graph -the bigger the SD, the more spread out the data is -SD of 0 means all data is the same #, there is no spread ** -Symmetric distribution: report mean, SD -Skewed: negative or positive skew, better to report median, min/max or range (Pos is if data is to the left, neg is to the right)

experimenter expectancy effect

when a bias causes a researcher to unconsciously influence the participants of an experiment (also known as expectancy bias or experimenter effect)

Formats for respondents

when asking a question or item with multiple possible responses, it is important that the responses are clearly distinguished from one another.

regression threat

when difference between pretest and posttest is caused by regression to the mean.

carryover effect

when exposure to one condition changes the response to another.

order effects

when exposure to one level of the independent variable influences responses to the next level of the independent variable

maturation threat

when participants change on their own over time.

experiment

when researchers manipulate at least one variable, and measure others.

history threat

when something happens between the pretest and posttest.

3-way interactions

when the difference of differences are different - example: a crossing and a spreading interaction

crossover interaction

when the influence of one independent variable on the other reverses across levels of the other variable

selection effects

when the kinds of participants in one level of the independent variable are systematically different from those in the other

instrument threat

when the measuring instrument changes over time. Example: When teachers become more lenient (or stricter) with their grading over time.

order effects practice, fatigue, carryover

when the order in which participants perform conditions in within-groups experiment affects the results. examples:

selection effect

when the traits of participants influence which condition they are assigned to, creating a confound.

null effect

when there is no significant covariance between the variables of interest.

Factorial independent-groups

where both independent variables are studied as independent-groups - example: 2x2 study, four different groups of participants

sampling variation

where multiple samples from the same population vary in composition and therefore vary in sample statistics

derived statistic

where one value for data is expressed in terms of another value

Coordination and control

—interviewers should be trained before they begin collecting data to identify any problems they might have administering the questionnaire. It is also important to monitor the interviewers during the course of the survey.

Follow question wording exactly

—it is important that the interviewer not deviate from the wording of the questions, which could bias the results.

Probing for responses

—oftentimes the interviewer will ask the respondent to give more information, which is common for open-ended questions.

Response rates in interview surveys

—response rates in telephone interviews have declined as respondent are increasingly likely to screen their calls.

Record responses exactly

—similarly, the responses should be recorded exactly as they are given.

Appearance and demeanor

—the interviewer should dress similar to the respondents and be polite.

Design your own 2X2 factorial study on the topic by doing this:

Determine the two independent variables involved in testing this topic. Determine two levels for each of the independent variables. Determine and develop the dependent variable. Develop a hypothesis.

Strength of Relationship

Determined by numeric value of the correlation coefficient. Range from -1.00 to +1.00. -1.00 is a perfect negative correlation. +1.00 is a positive correlation. 0.00 indicates no correlation

Reliability

Did the authors measure their constructs/variables well?

Converging evidence

Different measures provide outcomes pointing to the same conclusion

Counterbalancing

Different subsets of participants complete conditions in different orders - *how to avoid order effects confounds*

Experimental method

Direct manipulation and control of variables

Behavioral measures

Direct observations of behavior.

Applied Research

Directed towards solving practical problems

Coverage

Disadvantage of online survey. Involves sampling and unequal access or use. Older, less-educated, lower-income, and more rural people are less likely to have good internet access

Complexity of Questionnaire Design

Disadvantage of online survey. It is best to provide screen-by-screen questions and make an entire question visible on the screen at one time in a consistent format with drop-down boxes for answer choices

Privacy and Confidentiality

Disadvantage of online survey. Researchers should encrypt all data, use secure websites

Bimodal

Distributions with two modes

Random Assignment

Dividing subjects into groups at the beginning of experimental research using an unbiased, random process, so the experimenter can treat the groups as equivalent. Random in the statistic or mechanical sense.

What do all areas of ethics surround?

Do no harm

evolution

Ethics are in a rapid state of ________.

Risk-benefit analysis

Evaluation of the ethical implications of a research procedure by considering the benefits of the research in relation to the potential risks to participants.

Conceptual Definition

Everyday vernacular use of a term. Used by journalists when reporting.

Empiricism

Evidence is acquired from systematic, objective (unbiased) OBSERVATION, independently verifiable by other observers that provides a tentative answer.

Type A

Exaggerated sense of time urgency, competitiveness, and drive

Statistical Relationship

Exists between two variables when the average score on one differs systematically across levels of the others

Quasi-experimental Design

Experimental designs that are stronger than pre-experimental designs. They are variations on the classical experimental design that an experimenter uses in special situations or when an experimenter has limited control over the independent variable

Pre-experimental Design

Experimental designs that lack random assignment or use shortcuts and are much weaker than the classical experimental design. They may be substituted in situations where an experimenter cannot use all the features of a classical experimental design, but they have weaker internal validity

Each question be clear to answer so it doesn't confuse respondents (or influence their answers)

Explain how carefully prepared questions improve the construct validity of a poll or survey.

Debriefing

Explanation of the purposes of the research that is given to participants following their participation in the research.

External Validity

Extent to which study can be generalized to other settings and populations.

______ ________ is a major challenge for small-N studies.

External validity

Empirical Article

First time report of result from research Ask: What are the arguments? What is the evidence?

Discrete Variable

Have a small number of levels. -Less than 10 levels Examples: -gender, smoker status

Empiricism - Objective

Have to be able to come to a consensus

Name two examples of guidelines that may be requested in order to participate at a conference

Having an abstract/website

Give two examples of well defined, easily observed, and measured variables to operationally define

Height and Weight

Anatomy of a factorial design

How a factorial design is described conveys the number of independent variables in the designs and how many levels exist for each independent variable. The number of numbers indicates how many independent variables the design includes. The numbers themselves indicate how many levels or conditions each independent variable has.

- Remove noncompliant pts - Intention to tx stat adjustment - Tx attrition as a dependent variable - Dropouts significantly different

How do researchers deal with *attrition bias* where one group is more likely to *dropout* where the dropping out may even be a dependent variable?

Within Group difference

How do you know how much *difference* is expected *due to chance* within a group?

Fail reject Null Hypothesis (p >0.05)

How do you know if you have *no significant* results in your study?

Reject Null Hypothesis (p <0.05)

How do you know if you have *significant* results in your study?

Ordering items in a questionnaire

How items are ordered on a questionnaire can affect the answers respondents give. Certain early items may influence later responses, as might the ordering of general versus specific questions on a given topic.

Effect Size

How much evidence is present that the results represent a general society

Empiricism

Objective observation that is independent of prejudices

Systematic observation

Observations of one or more specific variables, usually made in a precisely defined setting.

Observer Bias

Observer records what they expect (or want) to see - only looking for behavior they want Solution: Develop good scoring rubrics Use more than one observer

Masked (blind) design

Observers are unaware of the conditions to which participants have been assigned and are unaware of the purpose of the study. - can't provide cues when reach correct answer

Exposure Therapy with Response Prevention is often used for what psychological disorder?

Obsessive Compulsive Disorder

Cofounding variable

Occurs when two variables are linked in a way that makes it difficult to sort out their specific effects.

negative predictive value

Of cases who test negative, the percentage who are actual negatives (don't have the disease)

What are the main problems with headlines?

Often -oversimplified -overconfident Sometimes -misrepresent findings

Passive Deception

Omission

Intuition

One source of information "Common sense" Example: -It makes sense to me that speaking two languages from a young age would make it harder to learn each because the terms from one language interfere with another.

Authority

One source of information Authority figures or convention Example: -Dr. Mitch Smith, a dentist, says that he always recommends that his patients chew Orbit brand gum.

Experience

One source of information Can be: -Personal -Anecdotes or testimonials (from others) Example: -Exercise makes me feel calm - after my Cycle bar classes, I can always sleep well.

Causal

One variable is responsible for changing another variable. Involves at least two variables Implies that the reason for change in one variable is due to the other variable Example: Music lessons enhance IQ

Types of Pre-experimental Design

One-shot case study design One-group pretest post-test design Static-group comparison

Confirmation Bias/Cherry-picking

Only picking out what supports your theory (usually people with an agenda) *This is the biggest bias*

foolish or stupid

When debriefing, do not make the participant feel ________.

Random sample

When each member of the population has an equal chance of being selected to represent the whole.

A

When interrogating experiments, on which of the big validities should a person focus? Internal validity External validity Construct validity Statistical validity

- Unethical randomly assign - Biologically impossible to randomly assign - Before investing for full experiment

When is it good to conduct a *correlation design*?

Selection-attrition threat

When only one group in the experiment experiences a decrease in the number of participants.

Demand characteristics

When participants determine what the study is about and change their behavior based on what they think the expected behavior should be.This can be prevented by using a masked design or double-blind design.

demand characteristics

When participants guess the hypothesis of a study, and change their behavior.

Practice Effects Confound (+)(order effects)

When prior participation in a level of an independent variable or measurement *[[provides]] participants with additional skills that influence their performance* on subsequent measurements

Fatigue Effects Confound (-)(order effects)

When prior participation in a level of an independent variable or measurement *[[tires]] the participant and influences their performance* on subsequent measures

attrition threat

When some types of participants are more likely to drop out of the study than others

Coercion to Participate

When someone in authority applies pressure or attempts to influence individuals into participating. Individuals believe that not participating will have negative consequences. Ex.) Offering exceptionally high or excessive rewards for participating.

Pseudo Survey

When someone who has little or no real interest in learning information from a respondent uses the survey format to try to persuade someone to do something

supported or does not support

When stating your conclusion, you must decide if the data collected _____________________________ the hypothesis.

Selection Confound

When the *kinds of participants in one level of the independent variable are systematically different* from those in the other level of the independent variable Procedure 1: 20% chem majors Procedure 2: 80% chem majors

Covariation of cause and effect

When the cause is present, the effect occurs; when the cause is not present, the effect does not occur.

synergistic effect

When the combination of two variables results in something greater than simply the sum of the individual variables. an effect where two variable combine to produce an outcome that is bigger than what each individual variable contributes a type of interaction effect

suppression effect

When the effect of combining two variables results in something less than simply the sum of the individual variables an effect where two variables combine to produce an outcome that is smaller than what each individual variable contributes a type of interaction effect

placebo group

a control group that is exposed to an inert treatment

A research design where the same participants are tested at each level of the independent variable

repeated measures design

precision

repeated samples that are collected

Ways to Handle Missing Data: Imputation

replace missing values with substitutes such as mean value. Put something in its place. If item is nominal or categorical this option is not used.

Evidence Based Practice Model

represents practitioners' intent to utilize the best available research evidence to inform treatment planning and practice. Includes: Program description Utilize outcome research Intervention evaluation

assent

required from children, a basic yes or no

Consent

required from participants

examples of repeated measures quasi-experiment tests?

- Interrupted time-series design - Nonequivalent groups interrupted time-series design

Construct Validity

"*Are they actually* measuring what they think they are measuring?"

masked design

(researchers were unaware of which condition participants had been assigned to)

Independent Variable

*2 or more levels* chosen by the researcher - manipulated variable - comes first - *causing the change*

*Which APA Titles are not bolded?*

*Abstract, Title of paper, References, Appendices, Authors note, Footnotes*

*Level 1 Heading*

*Center boldface, upper and lower case*

Three Criteria for Causation

*Covariance* The measured variable must change as the manipulated variable changes (correlations; correlations satisfy 1 of the 3 criteria---covariance). *Temporal precedence* The manipulation must precede the change in behavior. A should always happen before B, never B before A! *Internal validity* Alternative causal explanations must be ruled out

*Level 2 Heading*

*Flush left, boldface, upper and lower case*

*Level 3 Heading*

*Indented, boldface, Capital on first letter only, Period at end.*

3 characteristics of good questions (understandable)

*Only ask one question at a time* - Avoid double-barreled questions *Use simple wording* - Avoid negatives - Avoid uncommon words/jargon - Avoid lengthy/complex statements *Use precise wording* - Avoid ambiguity

*Title Page*

- *Running head: (abbreviated title) and page number (1)* - *Title of the paper* - *Author name(s) (byline) and affiliations* - *Author note*

*When in APA reports are printed numbers used?*

- *When the number is at the start of a sentence* - *Quantities below 10* - *Anything in a title*

Likert Scale (lick-urt)

- A scale in which each statement is accompanied by five response options: strongly disagree, disagree, neither agree nor disagree, agree, and strongly agree - minor variations are often called Likest-Type scales - very common in psychology

Results

- A summary of the data and the statistical analyses - Begins immediately following the method section—not starting on a new page - Provides a complete and unbiased reporting of the findings, just the facts, with no discussion of the findings

What are the limitations of operational definitions?

- An operational definition is not the same as the construct itself - Concerns about the quality of operational definitions and measurements produced - It is easy for operational definitions to leave out important components of a construct - Operational definitions often include extra components that are not part of the construct being measured

9.02 Characteristics of Responsible Research

- Approval by REB, due concern for the dignity and welfare of the participants, minimize misleading results, trained, responsible for assistants, experience, maximize benefit and minimize risk, minimize financial, social, organizational, or political factors, avoid conflicts of interest when conducting research, minimize interference - This is a guideline that was not followed by Milgram

Which critical evaluation question should we ask ourselves when evaluating alternative explanations?

- Are alternative conclusions/explanations possible?

Which critical evaluation question should we ask ourselves when evaluating a justified conclusion?

- Are the conclusions justified by the results?

Which critical evaluation questions should we ask ourselves when evaluating a procedure?

- Are there alternative ways to define and measure the variables? - Could alternative procedures be used?

How can existing publication lead to ideas for new research studies?

- Authors generally make explicit statements regarding unanswered questions - Authors sometimes point out limitations or problems in their own studies - Create a new idea from two existing results - Frame a question based on differing results

What considerations should be applied when selecting references?

- Be conservative - Choose references directly relevant to the study you are presenting - Select only those references that are useful and contribute to your arguments

Survey Questions

- Behaviour. How frequently do you go to art galleries? - Attitudes/beliefs/opinions. What kind of job do you think the mayor is doing? Do you think other people say many negative things about you when you are not there? - Characteristics. Are you married, cohabiting, single, divorced, separated, or widowed? - Expectations. Do you plan to enrol in a degree program in the next 12 months? How much schooling do you think your child will get? - Self-classification. Do you consider yourself to be liberal, moderate, or conservative? Into which social class would you put your family? - Knowledge. Who was elected prime minister in the last election?

*Abstract*

- Centred and not bolded - No indent - *Statement of problem* - *Describe participants* - *Describe method* - *Report results* - Conclusions or Implications

Considerations when Designing Mixed Methods

- Determine the order of implementations of the qualitative and quantitative aspects of the study. Concurrently or in a sequence, illustrates intent of the study: broad to narrow, narrow to broad. - Determine the weight given to qualitative and quantitative data (research question order). Extent to which both will be incorporated into the study, primary focus of the study. - Determine how the two types of data will be mixed or combined. Interpretation might involve transforming qualitative themes into quantitative numbers. Interpretation might involve comparing numbers to lived experiences. - Determine theory (both qual and quan) to guide study. Forms the questions, analysis and interpretation of the data.

9.04 Using Confidential Information for Didactic or Instructive Purposes

- Do not disclose personally identifiable information concerning their individual or organizational research participants unless written consent - Disguise confidential information concerning participants so that they are not individually identifiable to others

Which critical evaluation questions should we ask ourselves when evaluating limits to generalization?

- Do the results have real-world applications? - Is there reason to suspect that the same results would not occur outside the lab? - Would the same results be expected with different participants or under different circumstances?

Which critical evaluation question should we ask ourselves when evaluating results related to hypothesis?

- Do the results really support (or refute) the hypothesis?

Which critical evaluation questions should we ask ourselves when evaluating a specific prediction from hypothesis?

- Does the predicted outcome logically follow from the hypothesis? - Can other specific predictions be made?

American Psychological Association (APA) Guidelines

- Ethical Guidelines for the Use and Treatment of Human Participants in Research - First set of guidelines for protecting human subjects in research (1973) - Updated and expanded many times, and periodically reviewed - Is considered the gold standard of guidelines that are out there

Safeguards to Prevent Fraud

- Experimental research studies that are designed to be replicated - Research articles that are peer reviewed by experts in the field - The consequences of being found guilty of fraud - a big deterrent

Oral Presentations/Symposiums

- Five minute talk about research, allow a few minutes for questions from audience - In preparing a talk, presenters should memorize key elements such as why you conducted the study, what your research question was, how you tested it and what you discovered. - The presenter should look at the audience members and speak to them in a conversational tone that is less formal than APA-style writing but more formal than a conversation with a friend.

Behaviour Analysts Ethical Guidelines

- Guideline 9 (The Behaviour Analyst and Research; 2017) - Complex set of requirements with the BACB Guidelines - Behaviour analysts design, conduct, and report research in accordance with recognized standards of scientific competence and ethical research.

Which critical evaluation questions should we ask ourselves when evaluating participants?

- If participants were restricted (e.g., males only), is it justified? - Would different participants produce different results?

9.03 Informed Consent

- Inform participants or their guardian or surrogate in understandable language about the nature of the research - Not coerced - Process to decline to participate, or to withdraw without penalty; - Available to answer any other questions participants may have about the research - If you want to do research informed consent is a MUST! - Make the potential participants aware that even if they do not choose to participate, treatment is still available

When giving an oral presentation at a conference, what procedure should be followed?

- Intro and Opening remarks: - Develop your problem - Outline research - Develop your point or solution (use lots of visuals) - Summary - Thank Audience

The Scientific Method

- Is an approach to acquiring knowledge - Involves formulating specific questions and then systematically finding answers - Combines several different methods of acquiring knowledge

Which critical evaluation questions should we ask ourselves when evaluating a hypothesis/purpose for study?

- Is the hypothesis clearly stated? - Is the hypothesis directly related to the reviewed literature?

Which critical evaluation questions should we ask ourselves when evaluating a literature review?

- Is the review up to date and complete? - Are relevant or related topics not covered?

References

- Listing of complete references for all sources of information cited in the report o Organized alphabetically by the last name of the first author o Begins on a new page o Title is not bolded - Includes DOI is available

How should you prepare for a literature search?

- Narrow down your general idea to a specific research question o Find all published information to document and support that question - A good starting point: recently published secondary sources o A textbook, chapter headings, and subheadings will help focus your search on a more narrowly defined area

Name three important reasons to attended conferences

- Networking - Keeping up with regional laws - Keeping up with latest literature

9.09 Accuracy and Use of Data

- Not fabricate data or falsify results - If errors are discovered in their published data, they take steps to correct such errors (retraction, erratum) - Do not omit findings that might alter interpretations - Do not withhold the data on published work

Presenting at Conferences

- Organizers will outline written guidelines to apply to speak at a conference - Incomplete or incomprehensible submissions will be rejected - Must apply before a deadline - It will be evaluated by a panel and you will receive notice of rejection or acceptance

Name five potential sources for research ideas

- Personal interests and curiosities - Casual observation - Reports of others' observations - Practical problems or questions - Behavioral theories

What are five things to keep in mind when finding inspiration for a research topic?

- Pick a topic in which you are interested - Do your homework: collect background information - Keep an open mind - Focus, focus, focus - Take it one step at a time

9.01 Conforming with Laws and Regulations

- Plan and conduct research in a manner consistent with all applicable laws and regulations, as well as professional standards governing the conduct of research. - Comply with other applicable laws and regulations relating to mandated-reporting requirements.

Poster Presentations

- Posters are typically presented during a one- to two-hour poster session that takes place in a large room at the conference site. - Presenters set up their posters around the room and stand near them. - Other researchers then circulate through the room, read the posters, and talk to the presenter - Application guidelines - Format guidelines - Organization should inform if they have poster board, bring pins - Eye catching - Easy to read - Minimal content - Have copies of poster, have business cards

Appendices

- Presents detailed information - Useful, but would interrupt the flow of information if presented in the body of the paper

What is the goal of a research report?

- Provide three kinds of information about the research study o What was done o What was found o How the study is related to other knowledge in the area

Fundamental Characteristics that Allow for Cause & Effect to be Claimed

- Random Assignment: cornerstone of design as it allows equal chance to participate - Manipulation of intervention (IV) - Measurement of outcomes (DV) - Control of extraneous variables: factors that influence the outcome of the study by not being associated with the intervention - Making comparisons between groups: treatment vs. control group

Methods of Authority

- Relies on information or answers from an expert in the field - Is a good starting point and often quickest and easiest to obtain answers

Unavoidable Ethical Conflict

- Researchers must think through the ethical issues raised by their research, mitigate risks o Weigh the risks against the benefits, o Be able to explain their ethical decisions, o Seek feedback about these decisions o Take responsibility for them

Core Principles of Ethics

- Respect of Persons - Beneficence / Concern for Welfare - Justice

What is research ethics?

- Responsibility of researchers to be honest and respectful to all individuals who are affected by their research studies or their reports of the studies' results - Guided by the American (and Canadian!) Psychological Association's, BCBA ethical codes and Tri-Council Policy Statement

Discussion

- Restates the hypothesis, summarizes the results, and then presents a discussion of the interpretation, implications, and possible applications of the results - Begins immediately after the results section—not starting on a new page

What are the purposes of a literature review?

- Show relationships between previous studies or theories - Find out what information already exists in your field of research - Identify gaps in literature - Find other people working in your field - Identify major seminal works - Identify main methodologies and research techniques - Identify main idea, conclusions, and theories and establish similarities and differences - Provide a context for your own research

Divergent Validity

- Showing little or no relationship between the measurements of two different constructs - *Example: Measuring aggression and nothing shows

Face Validity

- Simplest and least scientific - Whether a measure superficially appears to measure what it claims to measure

What are the first two steps to the research process?

- Step 1: find a research idea - Step 2: form a hypothesis

Convergent Validity

- Strong relationship between the scores obtained from two (or more) different methods of measuring the same construct - Example: Many measures used and they all show the same type of data

Informed Consent

- Subjects must be informed of all available information about the study so they can make a rational decision to participate (or not) - Consent should state the they are free to not participate and there will be no ramifications - Consent should state that participants can remove consent at any time

Figures/Tables

- Supplement the text—they should not duplicate information that has already been presented in text form o Should be mentioned in the text by number o The text should point out some of the more important aspects - Each is on a new page

What is science?

- Systematic Empiricism - Empirical Questions - Creates Public Knowledge

Mean is used when:

- The data are normally distributed AND - There are no outliers

Participant Changes

- The participant can change between measurements (e.g., degree of focus) - Example: The student did not eat breakfast before coming to school

Assent

- There is no age of consent in Ontario. (understand the information & ramifications) - Assent (written or verbal) should be secured - The form should include simple declarative statements that describe concretely the main features of the study procedures and the voluntary nature of research participation. - Needs to be written or stated at the child's level

Why is it important to know about research methods?

- To learn how to apply science to the study of human behavior - To understand and interpret others' research - To evaluate claims in everyday media - To discover how to acquire knowledge and make decisions by applying the scientific method

Key Note Speakers

- Try to attend these during bigger conferences - Often there are concurrent sessions

During a literature search, in which ways can irrelevant materials be weeded out?

- Use the title as your first basis for screening - Use the abstract as your second screening device - Look for a link to a full-text version - If the article still looks relevant, then read it carefully - Use references from articles you have already found to expand your literature search

Name four keys to a good presentation

- Visuals - Practice - Don't Read - Watch your clock

Principles of Tri-Council Policy Statement

- Weighing risks against benefits - Acting responsibly and with integrity - Seeking justice - Respecting people's rights and dignity

Which critical evaluation questions should we ask ourselves when evaluating statistics (descriptive and inferential)?

- Were the appropriate statistics used? - Exactly what is significant and what is not? - Are the effects large enough to be meaningful?

Give 4 examples of bases for empirical research questions

- What are some possible causes of the behaviour or characteristic? - What are some possible effects of the behaviour or characteristic? - What types of people might exhibit more or less of the behaviour or characteristic? - What types of situations might elicit more or less of the behaviour or characteristic?

Which critical evaluation question should we ask ourselves when evaluating applications?

- Would other variables affect the results?

matched pair sampling

- a design in which one creates a set of two participants who are highly similar on a key trait and then randomly assigns individuals in the pair to different groups

alternative form reliability

- a form of reliability that evaluates how well a measure correlates with a similar, but different, measure of the same variable - correlation between different version of the scale - statistics: correlation between version 1 and version 2

empty control

- a group that does not receive any form of the treatment and just completes the dependent variable

placebo control

- a group where participants believe they are getting the treatment, but in reality they are not

Likert scale

- a scale where a participant evaluates a series of statements using a set of predetermined responses - responses are summed to represent the overall measurement for the variable - can measure level of agreement, frequency of occurrence, or overall evaluation of something

forced choice scale

- a scale where a person must choose between only two response alternatives for each item - limits sensitivity and encourages people to skip the item if they do not have a strong opinion - basically a yes or no scale, but two alternatives may not be sufficient - solution: add more varied responses

reverse coding

- a scoring strategy where more negative response alternatives are assigned higher numerical values and more positive response alternatives are assigned lower numerical values - used to minimize potential for an acquiescent response set - change the direction of your scale (if 1 is strongly agree and 5 is strongly disagree, 1 would be strongly disagree and 5 would be strongly agree)

Cronbach's alpha

- a statistic used to evaluate internal consistency reliability of a scale - can range from 0 to 1.0

Chi-square

- a statistical test in which both variables are categorical - generally tells us if the distribution of participants across categories is different from what would happen if there were no difference between the groups

one way ANOVA

- a statistical test that determines whether responses from the different conditions are essentially the same or whether the responses from at least one of the conditions differ from the others - compares the means across two or more groups - tells you there's a difference somewhere, but does not specify which groups are different

history

- a threat to internal validity due to an external event potentially influencing participants' behavior during the study - ex: fire drill causes evacuation during the study

instrumentation

- a threat to internal validity due to changes in how a variable is measured during the course of the study - ex: raters may become better or more refined in their observations leading to changes in ratings, differences in how tests are administered (such as paper vs online)

order effect

- a threat to internal validity in a within-subjects design resulting from influence that the sequence of experimental conditions can have on the dependent variable

maturation

- a threat to internal validity stemming from either long-term or short-term physiological changes occurring within the participants that may influence the dependent variable ex: participants may become tired, bored, or hungry as the study progresses

testing effect

- a threat to internal validity where participants' scores may change on subsequent measurements simply because of their increased familiarity with the measurement - ex: if you continue asking the same question participants may begin to guess the purpose, participants may remember earlier answers which could influence current answers

confounding variables

- a variable that the researcher unintentionally varies along with the manipulation

Clear and detailed codebooks

- allows observers to make accurate records - allows you to mediate disagreeing observation - allow others to assess the quality of operationalizations

two-group design

- an experimental design that compares two groups or conditions and is the most basic way to establish cause and effect - also known as simple experiment

random assignment

- any method of placing participants in groups that is unsystematic and unbiased - ensures each participant has an equal chance of being in any group

wording mistake: double-barreled

- asks participants to respond to more than one item using a single number - fix: break down into multiple questions so each one is only asking about a single item

practice effect

- changes in a participant's responses or behavior due to increased experience with the measurement instrument, not the variable under investigation

sensitization effect

- continued exposure to experimental conditions in a within-subjects study increasing the likelihood of hypothesis-guessing, potentially influencing participants' responses in later experimental conditions

fatigue effect

- deterioration in measurements due to participants becoming tired, less attentive, or careless during the course of the study

consequences of intrinsic variation for biological systems

- evolution by natural selection and adaptation - medical treatment/management - ecological management - biodiversity and diversity in ecological function - production in farming and other industries

carryover effect

- exposure to earlier experimental conditions influencing responses to subsequent conditions

extraneous variables

- factor other than the intended treatment that might change the outcome variable - eliminate or control these to ensure the IV is what is actually causing changes in the DV

socially desirable responding

- faking good and alternatively faking bad - giving survey responses that make one appear better (or worse) than is actually the case - Guarantee anonymity and ensure that people know that their responses are anonymous. - past deception in research many problems. If participants don't trust the researcher, then they many not believe that their responses really are anonymous. • Include items to identify socially desirable responding: - My table manners at home are as good as when I eat at a restaurant. - I don't find it particularly difficult to get along with loud- mouthed obnoxious people. - I never hesitate to go out of my way to help someone in trouble. • Perhaps observations or ratings from friends will be more accurate. • Use implicit measures.

How big should a sample be? It depends on 3 things.

- how accurate and precise do you need to be in estimation - in a H test, how small or subtle a relationship you are trying to determine - amount of inherent variation

how to get a precies and accurate representative sample

- identify populations of interest to be sampled - use of unbiased sample selection procedure (random sampling) - sample size

Why does increasing sample size matter?

- improves sample accuracy - improves sample precision

sources or causes of measured variation in biological systems

- intrinsic sources of variation - extrinsic sources of variation

snowball sampling

- non-probability - recruit subjects who then recruit more

convenience sampling

- non-probability - use of readily available subjects

purposive sampling

- non-probability - uses handpicked subjects that meet the researcher's needs

between groups advantages

-Generalizable (more participants are more representative) -collect more data at a given level -shorter time for each participant

stratified sampling

- probability - Ensure certain subgroups are represented or preserve proportions of subgroups in the population within the sample - requires definition of non-overlapping stratification criteria - randomly select in each stratum

Every multi-item scale has an order. What should we do?

- produce multiple versions, then compare results. If there is no difference between versions, then we can have greater confidence in the results. If there is a difference between versions, then we should report both outcome - try to identify a set of questions. (Alternatively, find questions that are not framed inappropriately by earlier questions) - if there is a one "most-important" question on your survey, you may wish to ask that question first

probability sampling methods

- randomization occurs at some point - simple random sampling - systematic sampling - stratified sampling - cluster sampling

Control of extraneous variables

-Have to be known to all participants -everyone gets same experience

Semantic differential format

- scaling for rating a target along several dimensions

posthoc test (Bonferroni correction)

- statistical tests that examine all of the possible combinations of conditions in a way that statistically accounts for the fact that we did not predict them ahead of time - if there is an overall difference, you can use posthoc analyses to compare groups to each other

internal consistency reliability

- the degree to which the individual items on a scale are interrelated or correlated - ask yourself: do the questions measure the same general topic? - statistics: Cronbach's alpha

content validity

- the degree to which the items on a scale reflect the range of material that should be included in a measurement of the target variable - ask yourself: does it assess the topic in a "wholesome" or comprehensive way?

criterion validity

- the extent to which a measurement relates to a particular outcome or behavior - established by evaluating the concurrent and predictive validity of the measurement - ask yourself: does it relate to actual behavior?

construct validity

- the extent to which the scale actually measures the desired construct - established by evaluating the convergent and discriminant validity of the measurement - ask yourself: does it actually measure our construct? - if the scale converges with measurements of similar variables and diverges from measurements of dissimilar variables, it has high construct validity

test-retest reliability

- the temporal stability of a measure - the ability to produce similar measurements across different time periods - statistics: correlation between Time 1 and Time 2 scores

methodological pluralism

- the use of multiple methods or strategies to answer a research question - sometimes it's better to use more than one

wording mistake: jargon

- using jargon that is specific to something that participants may be unfamiliar with (ex: rushing a fraternity) fix: avoid using words or phrases participants may not be familiar with

wording mistake: vocabulary

- vocabulary is complicated or complex - fix: keep it simple

acquiescent response bias

- when a participant tends to agree with most, if not all, of the items on a scale, regardless of what they are asking - participant anticipates questions and answers without reading - solution: reverse code some questions

covariation

- when changes in one variable are associated with changes in another variable - the two variables change together in a systematic way - correlation does not equal causality

temporal precedence

- when changes in the suspected cause (treatment) occur before changes in the effect (outcome) - accomplished by manipulating the IV before measuring the DV

wording mistake: not specific

- wording is not specific and too general - fix: make statements clear and direct about what we are asking

Question Errors

-Accepting a middle choice response when respondents hold a non-neutral opinion -Forcing respondents to choose a position when respondents hold non-neutral opinion -Issue of non-attitudes can be approached by distinguishing among three kinds of attitude questions: standard-format, quasi-filter, and full-filter questions

Mail and Self-Administered Questionnaires

-Advantages: Cheap and can be conducted by a single researcher Anonymity Avoid interviewer bias. -Disadvantages: Low response rates. Cannot control the conditions under which a mail questionnaire is completed People can answer dishonestly -Mail questionnaires are ill-suited for the illiterate

Online Surveys

-Advantages: Very fast and inexpensive. Allow flexible designs Can use visual images and/or audio or video -Disadvantages: Coverage Privacy Verification Design issues

Telephone

-Advantages: 95% of the population can be reached. Interviewers control the sequence of questions and can use some probes. All questions answered. -Disadvantages: More expensive than mail or online. The use of an interviewer reduces anonymity and introduces potential interviewer bias. No visual aids, difficult to use open-ended questions

Face-to-face

-Advantages: Highest response rates and permit the longest questionnaires. The interviewer can observe surroundings and can use nonverbal communication and visual aids. Can ask a range of questions -Disadvantages: The training, travel, supervision, and personnel costs can be high. Interviewer bias is high.

Advantages of experimental design

-Can account for Individual Differences -Eliminate other Confounds -Observe the invisible -Minimize noise

Tri-Council Policy Statement

-Canadian Version of ethics incorporating three different agencies -Published in 1998 - It replaced all previous guidelines that had been developed by individual agencies and institutions. - In 2010 a second edition (TCPS 2) was published that consolidated the core principles

key issues of Between Groups Design

-Creating equivalent groups -> same in every way except the IV -Random Assignment -Matching

Skewed Distribution

-If most cases have lower scores with a few extreme high scores, the mean will be the highest, the median in the middle, and the mode the lowest. -If most cases have higher scores with a few extreme low scores, the mean will be the lowest, the median in the middle, and the mode the highest.

Questions when reading journalism...

-Is original research evidence cited? -Has the research been published in a peer-reviewed journal yet? -How does coverage of the same journal article differ across news stories?

Questionnaire design issues

-Length of Survey or Questionnaire: Length depends on the survey format and on the respondent's characteristics -Question Order and Sequence: organization of the overall questionnaire, question order effects, and context effects. -Organization: A questionnaire has opening, middle, and ending questions. After an introduction explaining the survey, it is best to make opening questions pleasant, interesting, and easy to answer to help a respondent feel comfortable about the questionnaire. -Order Effects: An effect in survey research in which respondents hear some specific questions before others and the earlier questions affect their answers to later questions.

Qualitative questions

-Less structured, nondirective, in-depth interviews - differ from survey research interviews -Insights, feelings, and cooperation are essential parts of a discussion process that reveals subjective meanings

factorial design

-More than one IV -Main Effects and Interactions

Tentative Answers

-Not absolute answers (not proven) -Consider "weight of the evidence": Draw conclusions from entire scientific literature Proportion of studies consistent with theory

Problems with Headlines (Journalism)

-Often oversimplified overconfident -Sometimes misrepresent finding

one-way design

-One independent variable --->Two-group design --->Multiple groups design

key issues of within groups design

-Sequence Effects - scores on the DV are affected by the order in which levels of the IV are presented. --->Carry-over effects --->Practice effects --->Interference effects

Research questions are:

-Specific (is my dog happier after I walk him or after I give him a hug?) -Testable

Empiricism - Systematic

-Specified exact action you're looking to record, and how -Systematic observation typically involves specification of the EXACT actions, attributes, or other variables that are to be recorded and precisely how they are to be recorded.

Median is used when:

-The data are NOT normally distributed OR -There are any outliers (even 1!)

problem with counterbalancing

-Time interval between trials --->Mortality, loss of participants, or changes in participant

Measurement

-Unit of analysis: the individual, a state, etc -Variables: any quality that can be different between analysis e.g time spent reading, athletic involvement, age, race

Qualitative Interviewing

-Unstructured, semi-structured, in depth, ethnographic, open ended, informal, and long -Informal and non directive -A mutual sharing of experiences -Explicit purpose—to learn about the informant and setting (differs from friendly conversation) -Rarely have hypothesis being tested Advantages: 1. Allows the "voice" of the interviewee to be heard 2. Data rich with descriptions and explanations 3. Development of new theories 4. Development of new avenues of research Disadvantages 1. Sample sized are usually small in comparison to those used in quantitative work - question whether the results are actually applicable to a wider population 2. Very rare to archive qualitative data. More and more researchers are sharing so that the data can be scrutinized by others

Aiding Respondent Recall

-When survey involve the recall of past events researchers should provide respondents with special instructions and extra thinking time -Past events: should also provide aids such as a fixed time frame or location references

mixed design

-When there is more than one independent variable --->One or more variables can be within --->One or more variables can be between E.g.: Men and women each take a personality test for the big 5. Gender is the between subjects variable and the five different personality scales are within.

different people

-between groups design -only go through one condition

random assignment

-control for individual differences -creating equivalent groups

Quantitative data:

-data: 1 piece of data -data sets: lots of data connected, e.g homes, prices, size, and years (has codebook) -types of data sets: existing, build your own

purpose of analysis

-descriptive: describe variable -inferential: make prediction based on sample

two types of experimental groups

-different people -same people

within groups advantages

-fewer participants needed -Important for special groups -statistically more powerful (less variability)

between groups disadvantages

-may not be random -unequal N -potential confounds -requires more participants

describing center

-mean (average) -median (middle #) -mode (# that appears most)

two ways to create groups

-naturally occuring -experimental manipulation

within groups disadvantages

-not naïve after first trials -order effects --->practice and fatigue --->non-symmetric or differential transfer

normal distribution curve

-one peak -perfectly symmetrical -median and mean in same spot (center) -50% below avg, 50% above -follows 68 - 95 - 99.7 rule -close to symmetric: enough to fit normal curve

Two types of design

-post-test only -Pretest-posttest

Naturally occurring groups

-quasi-experimental Ex: men and women specific career groups

same people

-within groups design -all participants go through all conditions

Moderate (correlation strength)

.30/-.30

How are samples used for inferences?

1) estimation of values of population parameters from sample summary statistics 2) hypothesis (H) testing

Experiential Avoidance Article: What is the sample? What is the variables? What is the overall results? What is experiential avoidance?

1. 202 undergraduate/ 19.5 years of age/ mainly white females 2. 3. These findings suggest that experiential avoidance is significantly associated with lower delay of gratification both directly and indirectly, through heightened depression and anger 4. the tendency to negatively evaluate and avoid emotion and cognition, tend to report higher levels of distress.

Ways to ensure confidentiality:

1. Anonymous data 2. Code data 3. Obscuring details of a particular case

Things to avoid when writing survey questions

1. Avoid jargon, slang, and abbreviations (e.g., NATO) 2. Avoid ambiguity (What is your income? - does this mean annually, monthly? Family or personally) 3. Avoid double-barrelled questions 4. Avoid leading (or loaded) questions 5. Avoid double negatives

Goals of Debriefing

1. Clarifies the nature of the study 2. Allows the participant to express their feelings about being deceived 3. Removes negative consequence 4. Allows participants to reveal reactions or other comments about the study 5. Allows researcher to convey their appreciation

What types of validity do you need for causal claims?

1. Construct 2. Statistical 3. Three criteria for cause --temporal precedence --covariation --internal

What types of validity do you need for frequency and association claims?

1. Construct 2. Statistical 3. External

What are the first three steps to research?

1. Find an unanswered question that will serve as a research idea 2. Form a research question/hypothesis 3. Use the hypothesis to develop an empirical (observable) research study that will either support or refute the hypothesis

Concurrent validity

A measure of how well a particular test correlates with a previously validated measure.

Two Forms of Statistical Relationships

1. Differences between groups 2. Correlations between quantitative variables

Steps in the Research Process

1. Find a research idea: select a topic and search the literature to find an unanswered question 2. Form a hypothesis 3. Determine how you will define and measure your variables 4. Identify the participants or subjects for the study, decide how they will be selected, and plan for their ethical treatment 5. Select a research strategy 6. Select a research design 7. Conduct the study 8. Evaluate the data 9. Report the results 10. Refine or reformulate your research idea 11. Return to step 2 and continue

Three Types of Claims:

1. Frequency 2. Association 3. Causal

Common Characteristics of Pseudoscience

1. High "success" rates are claimed. 2. Rapid effects are promised. 3. The therapy is said to be effective for many symptoms or disorders. 4. The "theory" behind the therapy contradicts objective knowledge (and sometimes, common sense). 5. The therapy is said to be easy to administer, requiring little training or expertise. 6. Other, proven treatments are said to be unnecessary, inferior, or harmful. 7. Promoters of the therapy are working outside their area of expertise. 8. Promoters benefit financially or otherwise from adoption of the therapy. 9. Testimonials, anecdotes, or personal accounts are offered in support of claims about the therapy's effectiveness, but little or no objective evidence is provided. 10. Catchy, emotionally appealing slogans are used in marketing the therapy. 11. Belief and faith are said to be necessary for the therapy to "work." 12. Skepticism and critical evaluation are said to make the therapy's effects evaporate. 13. Promoters resist objective evaluation and scrutiny of the therapy by others. 14. Negative findings from scientific studies are ignored or dismissed.

Threats to Internal Validity and what each of them mean.

1. History- Any event that occurs between the first and second measurements but is not part of the manipulation. 2. Maturation- People change over time. 3. Testing- Simply taking the pretest changes the participant's behavior 4. Instrument Decay- The basic characteristics of the measuring instrument change over time 5. Regression Toward the Mean- When participants are selected because they score extremely high or extremely low.

Smoking Study Article: What are the IVs? What is the experimental design? What is the overall results?

1. IVs are smokers vs. nonsmokers 2. experimental design is 2x2 mixed (2 levels & 2 IVs) 3. Overall results are smokers were more generous

Kvale

1. Introducing questions 2. Follow-up questions 3. Probing questions 4. Specifying questions 5. Direct questions 6. Indirect questions 7. Structuring questions 8. Interpreting questions 9. Silence

What three things should you take into consideration to form a good research topic?

1. It's a topic that strongly interests you 2. It's creative 3. It is not to broad

Steps to conduct a survey

1. Keep it clear 2. Keep it simple 3. Keep the respondent's perspective in mind A researcher would not know if differences in respondents accounted for different answers. A good questionnaire forms an integrated whole (e.g., instructions for clarification and measures each variable with one or more survey questions)

Seven Concerns of the IRB:

1. Lack of Debriefing 2. Breach of Confidentiality 3. Lack of Informed Consent 4. Invasion of Privacy 5. Coercion 6. Deception 7. Harm to the Participants

Justifications for Deception

1. MUST INCLUDE A DEBRIEFING SESSION 2. Study cannot be completed otherwise 3. Cannot deceive participants about the study that might sway their willingness to participate 4. Cannot deceive about risks or possible discomfort

Principle of Respect for Persons

1. Must treat everyone as autonomous beings (can make their own decisions --informed consent --cannot mislead about risks --cannot coerce 2. Those who are not autonomous must have special protections --children, disabled, prisoners

4 levels of measurement

1. Nominal: name/label, no rank or value. e.g your major at college (categorical variables) 2. Ordinal: name/ label, has rank or value. e.g MU status, 1st year student (categorical variables) 3. Interval: numbers have rank, value. no real 0 value. e.g height is 6'2 4. Ratio: have true 0 value, 0 has meaning. e.g voted 0 times

Steps of the Scientific Method

1. Observe behaviour or other phenomena 2. Form a tentative hypothesis to generate a testable prediction 3. Use your hypothesis to generate a testable prediction 4. Evaluate the prediction by making systematic and planned observations 5. Use the observations to support, refute, or refine the original hypothesis

Principle of Beneficience

1. Protect the participants from harm 2. Cannot withhold beneficial treatment --control groups must eventually receive treatment --must implement new, innovative treatments even if your study has already begun --must inform about risks of waiting if in placebo

Who are the three categories of people who should be considered when making ethical decisions?

1. Research Participants 2. Scientific Community 3. Society

What are the three key ethical principles?

1. Respect for persons 2. Beneficence 3. Justice

What are the steps to submitting a manuscript for publication?

1. Select a journal that is appropriate for the topic of your research report 2. Consult the journal's Instructions to Authors for specific submission requirements 3. Enclose a cover letter to the journal editor along with the manuscript

What are two ways you can test for inter-item reliability?

1. Split-half reliability 2. Item total correlation

How do you increase your reliability?

1. Standardize the administration of the measure 2. Clarify instructions and questions 3. Train observers 4. Minimize coding errors

What are the two phases for participating in a conference presentation?

1. Submit a written summary or abstract to the conference organizers for approval 2. Give an oral presentation: typically involves a PowerPoint presentation (10 to 20 minutes) or Present a poster: guidelines are provided by the organization for style and requirements

3 Conditions for Causality

1. Temporal Order 2. Association 3. No Alternative Explanation

Sources of Ethical Directives:

1. The Belmont Report 2. APA Ethical Principles 3. APA Ethical Standards

Sources of Measurement Error

1. Transient States of the Participant 2. Stable Attributes of the Participant 3. Situational Factors 4. Characteristics of the Measurement 5. Experimenter Error

7 Parts of True Experiment

1. Treatment of independent variable 2. Dependent variable 3. Pretest 4. Post-test 5. Experimental group 6. Control group 7. Random assignment Note: not all experiments have all 7 parts

An informed consent form must include:

1. Why is the researcher doing this study? 2. What will the participant be doing? 3. Are there any risks involved? 4. Consequences of refusal/withdrawal 5. Confidentiality assurance 6. Does the participant have any questions? 7. How does the participant contact the researcher afterwards? 8. Signature lines

4 methods of survey administration

1. face to face 2. written 3. computerized 4. telephone

Three main principles (The Belmont Report)

1. principle of respect for persons 2. principle of beneficience 3 principle of justice

Researchers have two obligations:

1. protect the rights of the participants 2. to further scientific research

4 steps to calculate variance

1. subtract the mean from each score 2. square each of the deviation scores 3. add the squared deviation scores 4. divide the sum of the squared deviations by the number of scores

Marital Ages Article: What is operational sex ratio? What is female bias OSR? What is male bias OSR? What is OSR of 100? What is the results? How was the data obtained?

1. the number of sexually active males per 100 sexually receptive females 2. Where the OSR is female biased, males tend to pursue short-term mating as long as possible 3. When the OSR is male biased, available men outnumber available women and the greater degree of female choice will increase the male social status and resource potential necessary for securing female partners. 4. if the OSR is of 100, the population is balanced between the sexes 5. As predicted, when women are scarce they marry earlier on average and the variance in male marital age increased, however there was no significant relationship with male marital ages , though the trend was for earlier marriages. 6. 50 largest Metropolitan Statistical Areas (MSAs) in the United States with year 2000 Census data

What constitutes deception?

1. using confederates 2. providing false feedback 3. presenting related studies as unrelated 4. lying about stimulus materials

a trial finds a 6.7% cure rate for drug A and a 4.7% cure rate for drug b. what is the number needed to treat with drug a instead of drug b to produce one additional cure?

1/(0.067-0.047)= 50

How long must data be kept from a study?

10 years OR 10 years following the 18th birthday of the participant

What is an acceptable reliability score?

>.70

pilot study

: before conducting the "real" experiment, a researcher may test the procedure on a small sample to confirm the effectiveness of manipulations, make sure everything runs smoothly, etc

According to the textbook, what is the problem with WEIRD samples? They are not very representative of the world's population. They are not good for theory-testing mode. They are not used frequently enough. They are not very easy to measure.

A

Dr. Gavin is conducting a 2 x 4 independent-groups factorial design. How many interactions will Dr. Gavin need to examine? 1 2 3 4

A

In popular media articles, why does the inclusion of participant variable information often denote a factorial design? Participant variables are often used as moderators. Participant variables are only found in factorial designs. Studies with participant variables always find significant interactions. Studies with participant variables are easier to write about.

A

Seeing stability in a stable-baseline design can help rule out which of the following threats to internal validity? Regression to the mean Placebo effects Attrition Observer bias

A

The degree to which a quasi-experiment supports a causal claim depends on which of the following? Its design and its results Its duration and its sample size Its importance and its external validity Its statistical significance and its practical significance

A

The mathematical way to describe an interaction is: A difference in differences A caveat A qualified main effect A patterned pattern

A

The text states that the first step in establishing a study's importance is to establish its _________. replicability external validity internal validity acceptance by scientists

A

Which of the following is a difference between true experiments and quasi-experiments? Quasi-experiments do not use random assignment. Quasi-experiments do not involve manipulated variables. Quasi-experiments cannot have comparison groups. Quasi-experiments cannot have pretest measures.

A

Which of the following is a within-groups quasi-experimental design? Interrupted time-series design Nonequivalent control group design Matched group factorial design Multiple regression design

A

Which of the following is another term for ecological validity? Mundane realism Everyday realism Experimental realism Cultural realism

A

Constant

A "thing" that does not vary. It only has one level in the current context but it could potentially vary.

Constant

A "thing" that does not vary. It only has one level. Usually something that you are NOT interested in either measuring or manipulating

Variable

A "thing" that varies. It must have at least two levels.

Variables

A "thing" that varies. It must have at least two levels. Usually what you are interested in measuring or manipulating

3. _____ refers to the average of the dependent variable for participants with a specific combination of the levels of the independent variables A) Cell mean B) Marginal mean C) Main effect D) Crossover interaction

A - cell mean

1. is a prediction that focuses on one independent variable A) A main effect hypothesis B) An interaction effect hypothesis C) A suppression effect D) A crossover interaction

A - main effect hypothesis

Verstehen

A German word that translates as understanding; specifically, it means an empathetic understanding of another's worldview

theory introduction

A _______ is a statement, or set of statements, that attempts to explain the social world.

pretest-posttest

A ____________ design can show how people change over time as a result of some manipulation.

Proposition

A basic statement in social theory that two ideas or variables are related to one another. It can be true or false, conditional, and/or causal

Social Desirability Bias

A bias in survey research in which respondents give a normative response or a socially acceptable answer rather than giving a truthful answer

Cultural psychology

A branch of psychology that mainly focuses on the way cultural contexts shape the way a person feels, thinks, acts, and behaves.

Maturation (spontaneous remission) threat

A change in behavior that changes more or less over time. This can be prevented by including a comparison group- conducting a true experiment

Parameter

A characteristic of the entire population that is estimated from a sample ie. The percentage of city residents who smoke cigarettes

Association

A co-occurrence of two events, factors, characteristics, or activities, such that when one happens, the other is likely to occur as well. Many statistics measure this.

Concept Cluster

A collection of interrelated ideas that share common assumptions, belong to the same larger social theory, and refer to one another

Cells

A condition in an experiment that represents one of the possible combinations of two independent variables.

Strength of Correlation

A correlation above .70 is very strong A correlation above .40 is strong Lower than .20 is no correlation

Consistent Positive Relationship

A correlation near +1.00

Consistent Negative Relationship

A correlation near -1.00

Inconsistent Relationship

A correlation near zero

positive correlation

A correlation where as one variable increases, the other also increases, or as one decreases so does the other. Both variables move in the same direction.

Blame Analysis

A counterfeit argument presented as if it were a theoretical explanation that substitutes attributing for a causal explanation and implies an intention or negligence, or responsibility for an event or situation

Case study

A descriptive account of the behavior, past history, and other relevant factors concerning a specific individual.

naturalistic observation

A descriptive research method in which the researcher engages in careful observation of behavior.

case study

A descriptive technique in which one individual or group is studied in depth in the hope of revealing universal principles.

Mixed factorial design

A design that includes both independent groups (between-subjects) and repeated measures (within-subjects) variables.

Scatterplot

A diagram to display the statistical relationship between two variables based on plotting each other's values for both of the variables. -Used for variables measured at the interval or ratio level, rarely for ordinal variables, and never if either variable is nominal. Usually, the independent variable (symbolized by the letter X) goes on the horizontal axis and the dependent variable (symbolized by Y) goes on the vertical axis. -Determines: form, direction, and precision

Interaction effect

A difference of differences. A result in a factorial design in which the difference in one level of the independent variable changes depending on the other level of the independent variable. In a factorial design with two independent variables, the first two results obtained are the main effects for each independent variable. The third result is the interaction effect. Whereas the main effects are simple differences, the interaction effect is the difference in differences.

Pie Chart

A display of numerical information on one variable that divides a circle into fractions by lines representing the proportion of cases in the variable's attributes.

Bar Chart

A display of quantitative data for one variable in the form of rectangles where longer rectangles indicate more cases in the variable category. Usually, it is used discrete data, and there is a small space between rectangles.

Sampling Distribution

A distribution created by drawing many random samples from the same population

Sampling Distribution of Sample Means

A distribution of sample means created by drawing many random samples from the same population ie. Age - a continuous variable • Random sample of 30 - calculate the average age of students • Sample 1: average age = 21.63, Sample 2: Average age=20.38 • Repeat procedure until 12 different samples of 30 students have been drawn

Codebook

A document describing the coding procedure and the location of data for variables in a format that computers can use

IV x PV design

A factorial design that includes both an experimental independent variable (IV) and a nonexperimental participant variable (PV).

Pseudoscience

A fake or false science that makes claims based on little or no scientific evidence.

hypothesis

A formal statement that predicts how a change in one or more variables might cause a change in another variable is called a(n):

Normal Distribution

A function that represents the distribution of variables as a symmetrical bell-shaped graph.

Normal distribution

A function that represents the distribution of variables as a symmetrical bell-shaped graph.

Paradigm

A general organizing framework for social theory and empirical research. It includes basic assumptions, major questions to be answered, models of good research practice and theory, and methods for finding the answers to questions

Deductive Reasoning

A general statement is the basis for reaching a conclusion about specific examples

Descriptive Statistics

A general type of simple statistics used by researchers to describe basic patterns in the (numerical) data. They can be categorized by the number of variables involved: univariate, or multivariate

True

A hypothesis is a statement that predicts the answer to our research question. T/F

Identify the object or event that is to be observed, and identify any factors (variables) that might affect the observation

A hypothesis is expressed as a statement and must be capable of being tested. Before writing your hypothesis you must do what 2 things?

Central Limit Theorem

A law-like mathematical relationship stating that whenever many random samples are drawn from a population and plotted, a normal distribution is formed, and the centre of such a distribution for a variable is equal to its population parameter

Sampling Frame

A list of cases in a population, or the best approximation of it. Crucial to good sampling, size less crucial than whether it accurately represents population

Random Number Table

A list of numbers that has no pattern in it and is used to create a random process for selecting cases and other randomization purposes

Interview Guide

A list of questions a researcher wishes to address in the course of a qualitative interview

Manipulation check

A measure used to determine whether the manipulation of the independent variable has had its intended effect on a subject.

Matched pairs design

A method of assigning subjects to groups in which pairs of subjects are first matched on some characteristic and then individually assigned randomly to groups.

Counterbalancing

A method of controlling for order effects in a repeated measures design by either including all orders of treatment presentation or randomly determining the order for each subject.

Survey research

A method of collecting information by asking questions.

Curvilinear Correlation

A non-linear relationship between co-variables

Correlation Coefficient

A numerical index of the degree of relationship between two variables.

Response set

A pattern of response to questions on a self report measure that is not related to the content of the questions.

Confederate

A person posing as a participant in an experiment who is actually part of the experiment.

typology

A police officer develops a way to classify her colleagues in terms of their professionalism on the job. The categories are career officer, work-a-day officer, and slacker officer. This measure is best described as a:

Stratified random sampling

A probability sampling method in which a population is divided into subpopulation groups called strata; individuals are then randomly sampled from each of the strata.

Cluster sampling

A probability sampling method in which existing groups or geographic areas, called clusters, are identified. Clusters are randomly sampled and then everyone in the selected clusters participates in the study.

Reactivity

A problem of measurement in which the measure changes the behavior being observed.

Carryover effect

A problem that may occur in repeated measures designs if the effects of one treatment are still present when the next treatment is given.

Random Assignment

A procedure in which all subjects in a study have an equal chance of being assigned to any group or condition.

Double-blind experiment

A procedure wherein both the experimenter and the participant are unaware of whether the participant is in the experimental (treatment) or the control condition

Ideal Type

A pure model about an idea, process, or event. One develops it to think about it more clearly and systematically. It is used both as a method of qualitative data analysis and in social theory building

Field Research

A qualitative style in which a researcher directly observes and (usually) participates in small-scale social settings (distinct from pure qualitative) (ex. barber shop, daycare, etc.) -Used when other methods are not feasible

Categorical Variable

A quality, such as chosen major, and is typically measured by assigning a category label to each individual

Survey Research

A quantitative social research technique in which one systematically asks many people the same questions, then records and analyzes their answers

Variables

A quantity or quality which varies across people or situations

Quantitative Variable

A quantity, such as height, that is typically measured by assigning a number

Matched groups

A quasi-experimental design technique used to group participants into sets based on the similarity of a measured variable. Those coordinated sets are then randomly assigned to different conditions. This is used to control for selection effects.

Empirical Generalization

A quasi-theoretical statement that summarizes findings or regularities in empirical evidence. It uses few if any abstract concepts and only makes a statement about a recurring pattern that researchers observe

Leading (or loaded) questions

A question that leads the respondent to choose one response over another by its wording

Confidence Interval

A range of values, usually a little higher and lower than a specific value found in a sample, within which a researcher has a specified and high degree of confidence that the population parameter lies. High level of confidence (e.g., 95%) that the true population parameter lies with a certain range

Negative linear relationship

A relationship in which increases in the values of the first variable are accompanied by decreases in the values of the second variable.

Linear Relationship

A relationship of direct proportionality that, when plotted on a graph, traces a straight line

Split-half reliability

A reliability coefficient determined by the correlation between scores on half of the items on a measure with scores on the other half of a measure.

Test-retest reliability

A reliability coefficient determined by the correlation between scores on a measure given at one time with scores on the same measure given at a later time.

Experiment

A research method in which an investigator manipulates one or more factors to observe the effect on some behavior or mental process.

Direct replication

A researcher closely repeats an original study to see if the same effect shows up in the new data

Disclosing

A researcher decides how much to reveal about himself or herself and the research project. Disclosure ranges on a continuum

stratified random sampling

A researcher divides the residents of a low-income housing project into two groups: those age 30 and under, and those over age 30. Ten people from each group are randomly selected for interviews. This probability sampling technique is best defined as:

simple random sampling

A researcher draws names out of a hat to see which police officers in a particular precinct will be interviewed for a study on workplace satisfaction. This probability sampling technique is best defined as:

Only the children who participated in every phase of the study should be analyzed.

A researcher is conducting a longitudinal study involving repeated measurements of children as they grow up. Some of the children drop out of the study before it is completed. What should the researcher do about this?

random sampling error

A researcher is interested in whether participants in a drug treatment program are successful in their rehabilitation. He asks for volunteers to participate in a long-term study of post-treatment success. Twenty of the 130 participants volunteer to participate in the four-year follow up study. The bias in this sampling plan is best described as:

Observational/behavioral Measurement

A researcher records observable behaviors (including behavioral indicators of internal states) Observe what they ACTUALLY do Watch someone all day and count number of drinks Observe actual accuracy or speed on tasks Give a stats test and see how many they get right

one-group, pretest/posttest design

A researcher takes one group of participants, measures them before and after the treatment is given.

Yea-saying and nay-saying

A respondent may employ a response set to agree or disagree with all the question

Simple random sampling

A sampling procedure in which each member of the population has an equal probability of being included in the sample.

Quota sampling

A sampling procedure in which the sample is chosen to reflect the numerical composition of various subgroups in the population. A haphazard sampling technique is used to obtain the sample.

Interval scale

A scale of measurement in which the intervals between numbers on the scale are all equal in size.

Ordinal scale

A scale of measurement in which the measurement categories form a rank order along a continuum.

Ratio scale

A scale of measurement in which there is an absolute zero point, indicating an absence of the variable being measured.

Nominal scale

A scale of measurement with two or more categories that have no numerical (less than, greater than) properties

benchmarking.

A scientific technique that allows a researcher to learn something about a population by studying a few members of that population is called:

Scientific literature

A series of related studies that are conducted by various researchers and have tested similar variables.

Statistic

A set of collected numbers (e.g., numbers telling how many people live in a city) as well as a branch of applied mathematics used to manipulate and summarize the features of numbers

Coding System

A set of rules used to categorize observations

Theory

A set of statements or principles devised to explain a group of facts or phenomena

Reversal designs

A small-N study where the researcher observes behavior both before and during the administration of the independent variable/treatment, then takes away the treatment for a period of time to see if the behavior returns to the baseline. If the participant's behavior changes back, the independent variable/treatment was effective to the dependent variable. When this occurs, the researcher is able to test for internal validity and make a causal statement. This design is used to examine treatments that are not suspected to have long-term impact.

Pilot study

A small-scale study conducted prior to conducting an actual experiment; designed to test and refine procedures.

Sample

A smaller set of cases a researcher selects from a larger pool and generalizes to the population

Focus Group

A special qualitative research technique in which people are informally "interviewed" in a group-discussion setting -Popular way for market researchers to test their products -Topics might include public attitudes (e.g., race relations, work- place equality), personal behaviours (e.g., avoiding sexually transmitted infections), a new product (e.g., breakfast cereal), a political candidate -Includes moderator

single subject design

A special type of within-subjects designs using one participant (human or animal) or perhaps one group to assess changes within that individual or group. examples: A-B Design - baseline, intervention A-B-A Design - baseline, intervention, no intervention A-B-A-B Design - baseline, intervention, no intervention, intervention

Prediction

A statement about something that is likely to occur in the future

Causal Explanation

A statement in social theory about why events occur that is expressed in therms of causes and effects. They correspond to associations in the empirical world

Hypothesis

A statement that describes or explains a relationship between variables that can lead to several different observable and measurable predictions

Small-N design

A study in which researchers obtain a large amount of information from a few small cases. Each participant is treated as a separate experiment that is almost always a repeated-measures design.

Factorial designs

A study in which there are two or more independent variables. This is the best way to study how combined variables interact. Increases the number of comparison groups

Replicability

A study's potential to have its results replicated.

Ecological validity

A study's similarity to real-world contexts

Frequency Distribution

A summary chart, showing how frequently each of the various scores in a set of data occurs. • It can be used with nominal, ordinal, interval, or ratio-level data and takes many forms: Raw Count, Bar Chart, Grouped Data Frequency, and Frequency Polygon

Theory

A system of interrelated ideas used to explain a set of observations.

Frequency Table

A table for organizing a set of data that shows the number of times each item or number appears. and then find percentage

Participant observation

A technique of observing a situation wherein the observer takes an active role in the situation.

Attitude of Strangeness

A technique in field research in which researchers study a field site by mentally adjusting to "see" it for the first time or as an outsider. E.g., gift giving customs

Latin square

A technique to control for order effects without having all possible orders or to counterbalance.

Theoretical Saturation

A term associated with the grounded theory approach that refers to the point at which no new themes emerge from the data and sampling is considered complete

Science of the Sophomore

A term used to refer to the potentially limited external validity of studies based on undergraduate samples, usually using the experimental method

Hypothesis

A testable prediction, often implied by a theory.

History Effects

A threat to internal validity due to something that occurs and affects the dependent variable during an experiment, but that is unplanned and outside the control of the experimenter

Mortality

A threat to internal validity due to subjects failing to participate through the entire experiment

Maturation

A threat to internal validity in experimental research due to natural processes of growth, boredom, and so on that occur to subjects during the experiment and affect the dependent variable

Experimenter Expectancy

A threat to internal validity that occurs when the treatment "spills over" from the experimental group, and control group subjects modify their behaviour because they learn of the treatment

Diffusion of Treatment/Contamination

A threat to internal validity that occurs when the treatment "spills over" from the experimental group, and control group subjects modify their behaviour because they learn of the treatment. (ie. chatting during break)

Selection Bias

A threat to internal validity when groups in an experiment are not equivalent at the beginning of the experiment

Questionnaire

A tool used in survey research consisting of a list of questions asked to a respondent

Posttest-only design

A true experimental design in which the dependent variable (posttest) is measured only once, after manipulation of the independent variable.

Bivariate

A two-variable analysis, such as a crosstabulation of disease incidence by age or a comparison of mean health care costs by sex

Psychobiography

A type of case study in which the life of an individual is analyzed using psychological theory.

Purposive sampling

A type of haphazard sample conducted to obtain predetermined types of individuals for the sample

Histogram

A type of bar chart used to visually display the distribution of a continuous variable

Third-variable problem

A type of confounding variable in which a third variable leads to a mistaken causal relationship between two others.

Pearson product-moment correlation

A type of correlation coefficient used with interval and ratio scale data. In addition to providing information on the strength of relationship between two variables, it indicates the direction (positive or negative) of the relationship.

Budgetary impact analysis

A type of decision analytic model that estimates the impact of a change in treatment or policy on overall expenditures

Sequential Sampling

A type of non-random sample in which a researcher tries to find as many relevant cases as possible until there is no new information or diversity from the cases. Similar to purposive with one difference: in this sampling, a researcher continues to gather cases until the amount of new information or diversity of cases is filled, whereas in purposive the goal is to get every possible case until resources are exhausted. ie. Interviews with 60 widows over 70 years of age who have been living without a spouse for more than 10 years

Snowball Sampling

A type of non-random sample in which the researcher begins with one case then, based on information about interrelationships from that case, identifies other cases, and then repeats the process again and again. Multi-stage technique ie. heroin users

Quota Sampling

A type of non-random sample in which the researcher first identifies general categories into which cases or people will be selected, then he or she selects a predetermined number of cases in each category. ie. male and female, under the age of 30, 30 to 60 etc.

Haphazard Sampling

A type of non-random sample in which the researcher selects anyone he or she happens to come across. Cheap, quick, but ineffective and unrepresentative ie. street interview or show asking caller to vote online

Purposive Sampling

A type of non-random sample in which the researcher uses a wide range of methods to locate all possible cases of a highly specific and difficult-to-reach population Appropriate for 3 situations: 1. Unique cases (specific trendy women's magazine) 2. Members of a difficult to reach, specialized population (sex workers) 3. Researcher wants to identify particular type of case for in-depth investigation

Deviant Case Sampling

A type of non-random sample, especially used by qualitative researchers, in which a qualitative researcher selects unusual or nonconforming cases purposely as a way to provide greater insight into social processes or a setting. ie. studying high school dropouts who are from stable two-parent, upper-middle- income families who are geographically stable and well educated

Simple Random Sampling

A type of random sample in which a researcher creates a sampling frame and uses a pure random process to select cases. Each sampling element in the population will have an equal probability of being selected

Systematic Sampling

A type of random sample in which a researcher selects every kth (e.g., 12th) case in the sampling frame using a sampling interval. Simple random sampling with a short-cut for random selection. Often yields similar results. ie. Sample 300 names from 900. After a random starting point, you select every third name of the 900 and get a sample of 300. Sampling interval is 3

Stratified Sampling

A type of random sample in which the researcher first identifies a set of mutually exclusive and exhaustive categories, then uses a random selection method to select cases for each category. More representative. Researcher controls relative size.

Cluster Sampling

A type of random sample that uses multiple stages and is often used to cover wide geographic areas in which aggregated units are randomly selected; samples are then drawn from the sampled aggregated units, or clusters. Involves multiple stages and clusters. Less expensive than random sampling, but less accurate (each stage has sampling errors). Researcher first samples clusters, and then draws a second sample from within the clusters selected • Advantage when dealing with geographically dispersed populations • A design with more clusters is better because elements within clusters (e.g., people living on the same block) tend to be similar to each other (e.g., more alike than those on different blocks)

Random Sample

A type of sample in which the researcher uses a random- number table or similar mathematical random process so that each sampling element in the population will have an equal probability of being selected

Non-random Sample

A type of sample in which the sampling elements are selected using something other than a mathematically random process

Functional Theory

A type of social theory based on biological analogies, in which the social world or its parts are seen as systems, with its parts serving the needs of the system

Multivariate

A type of statistical analysis in which the effects of multiple variables are assessed simultaneously, such as an analysis of health care cost controlling for age, sex, comorbidities, and inclusion in the intervention versus comparison group of a study

Full-filter Question

A type of survey research question in which respondents are first asked whether they have an opinion or know about a topic, then only the respondents with an opinion or knowledge are asked a specific question on the topic (ie. Do you have an opinion on that? If yes, do you agree or disagree?)

Open-ended Questions

A type of survey research question in which respondents are free to offer any answer they wish to the question

Threatening Questions

A type of survey research question in which respondents are likely to cover up or lie about their true behaviour or beliefs because they fear a loss of self-image or may appear to be engaging in undesirable or deviant behaviour -Use only after a warm-up when an interviewer has developed a rapport and trust with respondents

Closed-ended Questions

A type of survey research question in which respondents must choose from a fixed set of answers -Large-scale surveys use these because they are quicker and easier for both the respondents and researchers -Help to reduce respondent fatigue -Sensitive topics (e.g., liquor consumption) may be more accurately measured

Standard-format Question

A type of survey research question in which the answer categories fail to include "no opinion" or "don't know"

Quasi-filter Question

A type of survey research question including the answer choice "no opinion" or "don't know"

True

A unit of analysis can be an individual, a group, a community, a state, or even an entire nation. T/F

Cluster

A unit that contains final sampling elements

dependent variable

A variable (often denoted by y ) whose value depends on that of another.

Manipulated Variable

A variable a researcher controls, usually by assigning participants to a particular level of the variable

independent variable

A variable that causes change to another variable is called a(n):

dependent variable

A variable that changes as a result of a change in another variable is called a(n):

intervening variable

A variable that occurs between an independent and dependent variable and affects the relationship between them is called a(n):

Measured Variable

A variable whose levels are simply observed and recorded. Typically accomplished using scales, inventories, rulers, or devices

Participant variables

A variable with levels that are selected and measured. (ex: age, gender, ethnicity)

traditional view of research participant

A view where participant contributes behavior to the experiment like urine is contributed to a medical experiment. Very objective.

Meta-analysis

A way to average the results of all of the studies that tested for the same variables to come to a conclusion that the whole body of evidence supports.

Nonequivalent control group interrupted time-series design

A within-groups quasi-experimental design that has two or more groups in which participants have not been randomly assigned (treatment group and comparison group), and are repeatedly measured on a dependent variable before, during, and after the interruption caused by some event- the timing of this event often differs among the groups.

Interrupted time-series design

A within-groups quasi-experimental design that repeatedly measures participants (in the singular treatment group), on a dependent variable before, during, and after the interruption caused by some event.

10. A double-blind procedure is used in an effort to reduce the likelihood that _____ of condition will influence the results; therefore, both the participants and the administrators of treatment are unaware of which treatments are being provided. A) the expectancies B) the inexperience C) the unfamiliarity D) the foreignness

A- the expectancies

8. _____ is an empty control group often used in clinical research. A) A waiting-list control group B) A placebo control group C) An intervention group D) A research participant pool

A- waiting list

Deductive Approach

An approach to inquiry or social theory in which one begins with abstract ideas and principles then works toward concrete, empirical evidence to test the ideas

OR equation

AD/BC

Which format should research reports be written in?

APA

Scale sensitivity

Ability of the scale to detect differences. 3 numbers lack sensitivity; more than 7 provides meaningless sensitivity.

Strength of Experimental Method

Ability to demonstrate causation by observation. In other methods causality is implied

Sections of a Journal Article

Abstract Introduction Methods Results Discussion References

What Each Section Contains or Does for the Reader

Abstract - provides essence of research Method - who, what and how section Results - report important findings and statistics/themes. Present what is being assessed, demographics, hypotheses. Discussion - interprets results for reader. Does not restate results in detail, only if needed.

Quasi-Experimental Designs

Addresses the need to study the effect of an independent variable in settings in which the control features of true experimental designs cannot be achieved.

Primary Criteria to Determine Suitability

Adequacy Utility Feasibility Propriety

How to Avoid Type I Error

Adjust alpha level to a more stringent level. Use two-tailed instead of a one-tailed. Bonferroni adjustment: dividing the alpha level (.05) by the number of statistical tests performed to create a new and more stringent alpha level.

graph

After all trials have been carried out and data has been recorded in the Data Chart, making a _________ will further organize data and help you to see trends and patterns.

research

After deciding on the problem, you must do _______________ to find out what is already known about your topic.

Simultaneous Measurements - Inter-Rater Reliability

Agreement between two observers who simultaneously record measurements of the behaviors

Management-Oriented

Aid leaders in making decisions about a program. Often looks at context, input (resources), process & product.

Behavioural Couples Therapy is often used for what psychological disorders?

Alcoholism and Substance Abuse

variables

All factors affecting observations and results in an experiment are ___________________.

constants

All other factors (except the two above) that could possibly affect the outcome of the experiment and must be kept the same are called _____________.

What is the purpose of a methods section?

Allows individuals reading the paper to replicate the study as well as to identify flaws and potential areas of improvement for future research

Name three things that should be avoided when writing in APA format

Ambiguity, Changes in topic, and Colloquial expressions

Idiographic

An approach that focuses on creating detailed descriptions of specific events in particular time periods and settings. It rarely goes beyond empirical generalizations to abstract social theory or causal laws

Ethnography

An approach to field research that emphasizes providing a very detailed description of a different culture from the viewpoint of an insider in that culture to permit a greater understanding of it -Considered a methodology rather than a method

Context Effect

An effect in survey research when an overall tone or set of topics heard by a respondent affects how he or she interprets the meaning of subsequent questions

Hawthorne Effect

An effect of reactivity named after a famous case in which subjects reacted to the fact that they were in an experiment more than they reacted to the treatment

Hypothetical Construct

An entity that we know exists in each individual, but is *intangible* (cannot be overtly observed). Must define the construct using some tangible metric

Margin of Error

An estimate about the amount of sampling error that exists in a survey's results

Field experiment

An experiment that is conducted in a natural setting rather than in a laboratory setting.

Experimental realism

An experiment in a laboratory is made to create a setting in which people are made to feel authentic emotions, motivations, and behaviors.

Field Experiment

An experiment set up in the real world, usually with participants who are not aware that they are in a study of any kind. -Less control over experimental condition -Greater external validity, lower internal validity

Solomon four-group design

An experimental design in which subjects are randomly assigned to two control groups and two experimental groups. Only one experimental group and one control group receive a pretest. All four groups receive a post-test. Researcher believes that the pretest has an influence on the treatment

Classical Experimental Design

An experimental design that has random assignment, a control group, an experimental group, and a pretest and post-test for each group

Double-blind placebo control strudy

An experimental design where the participants nor the observers for the study know which group is receiving the treatment or the placebo treatment.

One-shot case study design

An experimental design with only an experimental group and a post-test, no pretest Disadvantages: does not take into account external factors, findings may not generalize, may falsely assume an effect when none exists

Static Group Comparison

An experimental design with two groups, no random assignment, and only a post-test. Also called the post-test-only nonequivalent group design

demand characterisitc

An experimenter is interested in whether people will rate food as tastier when it's served on matching plates and utensils, compared to mismatched utensils. However, it's a repeated-measures design, and some participants realize that they are "expected" to like the food better in the matching condition. What kind of confound is this?

manipulation check

An extra dependent variable researchers can include to determine how well an experimental manipulation worked.

Praxis

An idea in critical social science that social theory and everyday practice interact or work together, mutually affecting one another. This interaction can promote social change

A

An independent-groups design is also known as a _________ design. between-subjects matched-groups within-groups mixed

nonequivalent control group pretest/posttest design

An independent-groups, quasi-experimental, pretest and posttest design that has at least one treatment group and one comparison group, but participants have not been randomly assigned to the two groups.

nonequivalent control group design

An independent-groups, quasi-experimental, pretest only design that has at least one treatment group and one comparison group, but participants that have not been randomly assigned to the two groups.

Correlation coefficient

An index of how strongly two variables are related to each other.

Cronbach's alpha

An indicator of internal consistency reliability assessed by examining the average correlation of each item (question) in a measure with every other question

Interrater reliability

An indicator of reliability that examines the agreement of observations made by two or more raters (judges).

Confidence interval

An interval of values within which there is a given level of confidence (e.g., 95%) where the population value lies.

Focus group

An interview with a group of about 6 to 10 individuals brought together for a period of usually 2-3 hours.

The role of the survey interviewer

An interviewer can increase the response rate by doing the "work" involved in the survey. They can also observe the respondent and the setting within which the interview takes place (typically the respondent's house). Of course, the presence of an interviewer may bias the respondent's answers on particularly sensitive questions.

Theoretical Sampling

An iterative sampling technique associated with the grounded theory approach in which the sample size is determined when the data reach theoretical saturation -A researcher does not know in advance how many individuals he or she needs to interview

confound

An unaccounted-for variable in an experiment that provides an alternative explanation for an effect.

Intention to treat versus per protocol

Analytical approach in which the outcomes for all those who are assigned to the treatment or intervention of interests are included in the analysis in contrast with a per protocol approach in which only those who successfully complete the trail are analyzed

statistical power you don't need as many participants in a within-groups study to find a significant effect.

Another advantage of within-groups design is increased This means:

Variable

Any event, situation, behavior, or individual characteristic that varies—that is, has at least two values.

Covariance (3 criteria for causal claims)

Are x and y related?

Alternate forms reliability

Assessment of reliability by administering two different forms of the same measure to the same individuals at two points in time.

Limitations to the Rational Method

Assumes that the premise statements are true, sometimes the argument does not logically support the conclusion

What is considered minimal risk?

At no greater risk for harm than you would be doing an everyday task

If a researcher is concerned about external validity, which of the following would you recommend with regard to conducting small-N designs? Do not conduct small-N designs if you are concerned about external validity. Compare the results of a small-N design with other studies. Conduct only reversal designs. Use only one's own clients/patients/students.

B

Imagine that you are reading a journal article and you see the following sentence: "The study used a 2 x 2 x 4 design." Based on this sentence alone, you would know which of the following pieces of information? The number of participants in the study The number of main effects that need to be examined The statistical significance of the interaction The number of participant variables

B

The number of main effects that need to be examined is _________ the number of independent variables. more important than equal to unrelated to independent of

B

Which of the following is a reason that psychologists especially value meta-analyses? Meta-analyses eliminate the need for replications. Meta-analyses allow researchers to examine the strength of a relationship. Meta-analyses are immune to threats to internal validity. Meta-analyses always take less time to conduct.

B

Which of the following is an advantage of using quasi-experimental designs? They allow researchers to capitalize on random assignment. They allow researchers to enhance external validity. They allow researchers to disregard internal validity. They are better suited to detect significant effects.

B

Which of the following types of study support external validity? A study using non-randomly selected participants A study that adds a new independent variable A study that is a direct replication A failed replication study

B

Why do quasi-experiments tend to have very good construct validity for the independent variable? The manipulations have been previously validated in the lab. They use real-world manipulations/experiences. They tend to use more participants. They also have good construct validity for the dependent variable

B

5 types of data

B-Data, P-Data, L-Data, I-Data, and S-Data

Guide Lines for asking Questions

B. Make items clear—the respondent should be able to easily understand the items on the survey. C. Avoid double-barreled questions—this refers to questions with multiple components, and thus, it is impossible to determine which part the respondent is answering. D. Respondents must be competent to answer—the respondent should have enough information to answer the question accurately. E. Respondents must be willing to answer—oftentimes respondents are reticent to respond to questions, particularly on sensitive topics. F. Questions should be relevant—the items on the survey should be meaningful to the respondents. G. Short items are best—the longer and more complicated the item, the more difficultly the respondent has in answering the question. H. Avoid negative items—respondents often overlook negative items in a question or statement, and give an inaccurate response. I. Avoid biased items and terms—certain terms and phrases encourage certain responses; the items should employ neutral language.

What questions do reliability and validity answer about a measure?

BOTH have to do with how good a measure is

Classification Concept

Complex, multidimensional concepts that have sub-types. They are parts of social theories between one simple concept and a full theoretical explanation

Qualitative Research

Based on making observations that are summarized and interpreted in a narrative report

Statistical Relationships

Based on two ideas: correlation and independence

What are the 4 criteria for empiricism?

Be acquired from systematic observation Be acquired from objective (unbiased) observation Be independently verifiable by other observer Provide a tentative answer.

variables

Before deciding on the independent variable, you must first determine all possible ______________.

data chart

Before starting the experiment, you must decide how you will record your data by creating an appropriate ____________________.

Availability Heuristic

Being persuaded by what easily comes to mind

Good Story Bias

Being swayed by a good story "It just makes sense"

Experiment Qualities

Best for narrow scope, micro-level concerns. Isolate and target impact of few causal variables. Not viable for research of diverse variables

Dr. Gavin is conducting a 2 x 4 independent-groups factorial design. Assuming he wants 25 people in each cell, how many participants does Dr. Gavin need to recruit? 100 150 200 250

C

In the case of a factorial design, another term for independent variable is: Main effect Cell Factor Moderator

C

Which of the following studies is most likely to be conducted in generalization mode? Examining whether dog ownership decreases stress levels Exploring the relationship between weight and number of fast-food restaurants in one's neighborhood Determining how frequently people report fighting with their bosses Investigating whether there is a causal link between receiving cognitive therapy and increased coping skills

C

9. _____is a statistical analysis that tests for differences between two or more categorical independent variables, when one is a between-subjects variable and another is a within- subjects variable. A) A one-way analysis of variance B) A repeated-measure analysis of variance C) A mixed design analysis of variance D) A two-way analysis of variance

C - mixed design of variance

2. ____ is when the influence of one independent variable on the other reverses across the levels of the other variable. A) A main effect B) A correlation C) A crossover interaction D) A marginal effect

C- crossover interaction

What are the problems with claims based on intuition?

COGNITIVE BIASES: thinking the easy way Being swayed by a good story - good story bias Being persuaded by what easily comes to mind - availability heuristic MOTIVATIONAL BIASES: thinking what we want Focusing on the evidence that fits our beliefs - confirmation bias Being biased about being biased - bias blind spot

CONSORT

CONsolidated Standards Of Reporting Trials

Scientific research involves evidence being obtained in which ways?

Careful, systematic, and objective observations

testing threat

Change in participants as a result of taking a test more than once. Did performance improve because of the manipulation, or because participants are already familiar with the test?

Carryover Effects Confound (order effects)

Changes in the *scores observed in one level of an independent variable are caused by the lingering effects* of a level of an independent variable previously experienced

Type I Error

Claiming a result was significant when you should have not done so. Reject the null when you should not have done so. False positive. Tend to be more problematic because false knowledge is distributed. ex. Telling a man that he is pregnant.

Conceptualization

Clarifying what we mean by a concept - develop the concept. (variable of interest)

Open-ended and closed-ended questions

Closed ended questions include all possible responses to the question, while open-ended questions place no constraints on the respondent's answer; they are free to respond however they choose.

student

Coercion is most common when using _______ participants.

Between-subjects design

Comparisions are made between different groups of participants

Self Questionnaire- Compensation for respondents

Compensation in survey research is often controversial. It often consists of gift certificates, prize drawings, and lotteries, and when prepaid, are more successful.

Pearson Product Moment Correlation Coefficient: Pearson r

Computed when data for both measured variables are quantitative in nature (interval or ratio) and linear in relationship - both variables are continuous. Also used as a prediction for correlation of determination. ex. GPA, years of experience, NCE score.

Conceptual Variable and Operational Definition Examples

Conceptual Variable -Binge Drinking -Learning -Anxiety -Height Operational Definition -4 to 5 drinks in two hours -Score on an exam -Score on anxiety inventory -Inches? Centimeters?

Applied research

Conducted to address issues in which there are practical problems and potential solutions.

Applied Research

Conducted to find solutions for specific "real world" problems To improve the human condition

Motivation

Confirmation bias/Cherry-picking Asking biased questions The bias blind spot

Self-reports and accuracy

Consequently (of flashbulb memories and confidence) a, retrospective self-reports may be inaccurate (i.e., may have poor construct validity as a measure of past behaviors). • Possible solution: observe directly, rather than rely one self-reports.

What is the best method for determining how a variable should be measured?

Consult previous research involving the same variable

Randomization

Controlling for the effects of extraneous variables by ensuring that the variables operate in a manner determined entirely by chance.

Pattern and Parsimony

Converging evidence from correlational studies helps us get closer to causal claims

Cross-Sectional Correlation

Correlation between *2 variables*, measured at the *same time*

Cross-Lag Correlation

Correlation between an *earlier measure of one* variable and a *later measure of another* variable - remains across time - temporal precedence

payer perspective

Costs represent the charges for health care products and services allowed or reimbursed by the payer

True

Cross-sectional research involves data collection at one point in time. (The findings from this research cannot be used to draw conclusions about change over time, but they can provide a snapshot of a social condition or population at that point in time). T/F

Control

Crucial in experimental research, important to isolate the effects of the treatment to eliminate alternative explanations. Deception allows the experimenter to control subjects' perception of the situation

A small-N design that involves providing treatment and then removing treatment is known as a(n): interrupted time-series design. stable-baseline design. multiple-baseline design. reversal design.

D

An author might use all of the following methods to indicate statistical significance EXCEPT: Using the word significant Including an asterisk in a table Using the notation p < 0.05 The inclusion of a graph

D

Dr. Gavin decides that instead of conducting a 2 x 4 independent-groups factorial design, he is going to conduct a 2 x 3 x 4 mixed factorial design. Which of the following things will NOT have to change? The number of main effects that need to be examined The number of interactions that need to be examined The number of participants needed The number of researchers needed

D

Dr. Gavin is conducting a 2 x 4 independent-groups factorial design. How many cells are in his design? 2 4 6 8

D

In a previous correlational study, Dr. Lafayette has found that owning dogs is associated with lower levels of daily stress in a sample of returning war veterans. Which of the following would constitute a shift from theory-testing mode to generalization mode? Conducting a study on the same sample of war veterans, this time using a different measure of daily stress Conducting a study examining the stress reduction mechanism Conducting an experimental study in which he assigns people to own a dog or not Conducting a study on dog ownership using a sample of elderly participants from a local retirement home

D

In small-N designs, each participant is treated: with multiple interventions. by a clinical psychologist. as a data point. as a separate experiment.

D

Which of the following is true of a scientific literature? It comprises studies conducted by a single researcher. It comprises studies that have tested the exact same variable. It comprises studies conducted all in the same year. It comprises studies conducted with different methods.

D

Which of the following is true of cultural psychology? Its researchers function only in theory-testing mode. Its main focus is to disprove theories. It typically finds support for a theory across different cultural contexts. It starts with the assumption that theories apply to all humans, regardless of culture.

D

Which of the following is true of small-N experiments? Small-N designs determine whether a finding is replicable by doing a test of statistical significance. Data from small-N designs are presented as averages. Data from small-N designs are grouped together. Each person in a small-N design is treated as a separate experiment

D

Which of the following types of replications investigate the same research question but use different procedures? Statistical replication Direct replication Replication-plus-extension Conceptual replication

D

6. _____ is a single-subject design in which researchers take a baseline measurement and then introduce the intervention and measure the same variable again. A) An A-B-A design B) A single-blind procedure C) A mixed design D) An A-B design

D - A-B design

Fatigue effect

Deterioration in participant performance with repeated testing.

5. _____ is an effect in which two variables combine to produce an outcome that is smaller than what each individual variable contributes. A) Carryover effect B) Synergistic effect C) Main effect D) Suppression effect

D- suppression effect

Cross-sectional

Data collected only one point in time to understand needs, attitudes, make comparisons or evaluate program.

Longitudinal

Data collected over time with various pre-established points for administration.

Nonparametric

Data for which the probability distribution is unknown or known not to be normal skewed examples: Mann-Whitney and Spearman's test

Limitation

Data only of what a person or organization says, may differ from what he or she actually does

check if the the participant realized the deception

Debriefing is a good time for manipulation checks, which are?

Operational definition

Definition of a concept that specifies the method used to measure or manipulate the concept.

Likert Scale

Degree of Agreement Strongly Disagree 1 - Strongly Agree 5

Experimental Validity

Degree to which the results of the study can be attributed to independent variable truly effected changes in dependent variable and results can be generalized to target population.

Behavioural Therapy is often used for what psychological disorder?

Depression

CBT is often used for what psychological disorders?

Depression, panic disorder, bulimia nervosa, and post traumatic stress disorder

Frequency Claim

Describe a particular rate or degree of a SINGLE variable

Frequency Claims

Describe a particular rate or degree of a SINGLE variable. Examples: -1 in 5 people have a dental filling -64% of Americans report being very happy with their partner

When they can avoid three problems: observer bias, observer effects, and reactivity

Describe how researchers can make observations with good construct validity.

Association Claims

Describe the relationship between two variables. Also known as correlations or covariation. Relationship between x and y. Involves at least two variables Cannot make causal claims Example: -Children of divorced parents are more likely to suffer from an anxiety disorder

Naturalistic observation

Descriptive method in which observations are made in a natural social setting. Also called field observation.

no

Do you accept or reject the hypothesis?

America the Beautiful

Documentary in which women said they did not feel beautiful, but the majority do

Non-probability Sampling

Does not involve random selection. Used by qualitative researchers for detailed findings

Discriminant Validity

Does our measure only weakly relate (or have no relationship) to other measures that have little/nothing to do with our construct?

Convergent Validity

Does our measure positively correlate with other measures that are assessing similar constructs? AND/OR Does our measure negatively correlate with other measures that are assessing opposite constructs?

Criterion-Related Validity

Does the measure allow you to distinguish people on the basis of a particular criterion or behavioral outcome? Researchers identify behavioral outcomes that the measure should be related to and then correlate their measure with the occurrence of those outcomes

Reliability

Does the measure produce consistent results?

C

Dr. Alfonse, a developmental psychologist, conducts a study to determine whether children prefer books with drawn illustrations or with photographs. A group of 30 preschoolers are shown two copies of a book (Ferdinand the Bull) at the same time. Although the story is the same, one book is illustrated with drawings and the other is illustrated with photos. Students are then asked to indicate which book they prefer. This is an example of which of the following designs? Longitudinal design Repeated-measures design Concurrent-measures design Posttest-only design

C

Dr. Deveraux has conducted a study that has resulted in a null effect. Nonetheless, she suspects that there truly is a causal relationship between her independent and dependent variables. Which of the following is UNLIKELY to be to blame? A weak manipulation An insensitive measure Too many participants A reverse confound

C

Dr. Hoff is curious as to whether children in a kindergarten classroom will be friendlier after talking about friendship. Immediately before and after the discussion, she has several undergraduate research assistants code the "friendliness" of a group of kindergarteners. Which of the following threats should Dr. Hoff be most concerned about? History Maturation Instrumentation Testing

D

Dr. Kline, an environmental psychologist, conducts a study to examine whether visiting zoos causes people to have more positive attitudes toward environmental conservation. He asks a group of 45 people attending the zoo on a Saturday morning about their attitudes. He finds that 69% of the people report having a positive attitude after their visit. Which of the following is true? His control group is people who did not visit the zoo. He does not have a dependent variable. He can make a strong causal claim about the effect of zoo visits on environmental attitudes. His study does not qualify as an experiment.

B

Dr. Sanderson is curious as to whether exposing people to violent video games causes them to be more aggressive. She assigns half her participants to play a violent video game for 5 minutes and the other half to play the same game for 25 minutes. Afterward, she has them play a board game and has a well-trained coder determine whether they are very aggressive in their playing style, barely aggressive, or not at all aggressive. She finds that a vast majority of her participants, regardless of group assignment, are rated as very aggressive. This outcome would be known as a(n): weak manipulation. ceiling effect. floor effect. outlier.

Same conclusion

During the Rosenthal Study, it was observed that *meta-analysis literature reviews* had what sort of conclusions?

Varied conclusion

During the Rosenthal study, it was observed that *narrative literature reviews* had what sort of conclusions?

Accumulation of knowledge

Each new generation of researchers don't have to reinvent the wheel because research is what?

Crossover

Effect of two interventions observed across a single group with assessment before, during and after interventions. May use multiple groups to adjust/compare order of intervention given.

Latin Square

Effects of three interventions assessed. Randomly assigned groups with different sequential order of intervention.

Experimental Control

Efforts to minimize effects of extraneous variables through design, development and implantation of research: - Random Assignment - Comparison Group - Counterbalancing - Matching

What does it mean to be Scientific?

Empirical Approach Theory-data cycle Make findings public

Which method obtains answers from direct sensory observation?

Empiricism

Planning

Entering and gaining access to a field site is a process that depends on common sense judgment and social skills

Input Evaluation

Environmental assessment of available resources in work environment.

Interval

Equal intervals between units but *no meaningful zero* (IQ test)

Ratio

Equal intervals, a *meaningful zero* zero means "none" (calories consumed, weight, brain activity)

How to avoid design confounds

Equate everything that you can other than the independent variable (hold everything constant)

According to the concept of reliability, inconsistency in measurement comes from what?

Error

Benefits of factorial design

Establish Cause and Effect Conduct Multiple Experiments at Once Examine How a Combination of IVs Affects the DV (Interaction)

Experimental control

Extraneous variables are kept constant.

anatomy of a mixed design ANOVA

F (2, 87) = #, p=#, eta2 = # f= f test symbol 2= between subjects DF 87 = within subjects DF p = significance eta= calculated effect size

Factorial design

Factorial designs are designs with more than one independent variable (or factor).

How to Avoid Type II Error

Fail to reject the null. Increase the number of participants. Review threats to reliability of instruments. Consider alternate test statistics.

Floor effect

Failure of a measure to detect a diff erence because it was too difficult

Ceiling effect

Failure of a measure to detect a difference because it was too easy

True or False: Correlation always means that a causation is present

False

True or False: In order for a measure to be reliable, it must be valid

False

Hybrid

Field experiments - contrived situations in the real world

Process Evaluation (Formative)

Finding answers to questions that pertain to the implementation of a program and opportunities to make revisions; periodically to see how they are doing.

Primary Sources

Firsthand reports in which the authors describe their own observations

Periodic Probe

Follow-up questions (helpful for respondent fatigue)

Solomon four group

Four groupings used to mitigate testing effects of pretest while assuring equal groups are used: - Control group that receives pretest and posttest. - Control group that receives only posttest. - Treatment group that receives both pretest and posttest. - Treatment group that only receives posttest.

most famous cohort study in the US

Framingham Heart Study

Operational Definition

How the researcher decides to measured or manipulate the conceptual variable. Also called an operational variable, measure

pretest-posttest

Give dependent variable, then manipulate, then give dependent variable again. Ensures equivalent groups Gives change and whether significant Gives power and can see confound and remove statistically

Laboratory Experiment

Greater internal validity but lower external validity. They are logically tighter and better controlled but less able to be generalized

intuition

Gut feeling, emotion, "follow your heart"

Hybrid Design

Hybrid designs have at least one factor that we cannot actually manipulate (e.g., gender). In factorial designs, you can test the impact of an experimental variable and a nonexperimental variable at the same time. any factorial design that has at least one quasi-independent variable examples: race, age, gender, (something you cant change about participants

RCTs are required in phase _____ clinical trials as a part of an NDA

III

all protocols are reviewed by the _____

IRB

Product Evaluation (Summative)

Identified extent to which goals and objectives were achieved or if improvement of discontinuations are needed; improving for next time, not the current product.

Subject Words

Identify and describe the variables in the study and the characteristics of the participants

Why is a stable-baseline design more convincing than a simple pretest/posttest?

If the baseline is stable over time, you can rule out regression to the mean and maturation threats.

Alpha (aka critical P value)

If the probability is less than the alpha (critical P value, usually 0.05), null hypothesis is rejected

hypothesis

If your conclusion leads you to new questions, you should create a new ______________________.

Practice effect (also learning effect)

Improvement in participant performance with repeated testing.

Nothing

In a *prospective* research study, you need more information since it can be correlative or experiment, what can you assume?

No causal inference

In a *retrospective* research study, what can you assume?

Order Effects Confound

In a [[within-subjects design]], when a participant *experiences a series of different levels of an independent variable and a series of measurements*, their performance at any point may be influenced by an experience that occurred earlier

Simple main effect

In a factorial design, the effect of one independent variable at a particular level of another independent variable.

main effect

In a factorial design, the overall effect of one independent variable on the dependent variable, averaging over the levels of the other independent variable. a HYPOTHESIS (prediction) that focuses on one independent variable at a time, ignoring all other independent variables

Order effect

In a repeated measures design, the effect that the order of introducing treatment has on the dependent variable.

Stem

In a survey, the statement, question or prompt to which the respondent replies.

Contingency questions

In certain instances, questions may not be relevant to all respondents. Typically, the researcher will ask a question to determine who should answer the subset of questions and who should ignore them. The questions that are only answered by a subset of the respondents are referred to as contingency questions.

How to identify factorial designs in reading

In empirical journal articles, the method section will usually state "this was a factorial design, with ___ and ___ as independent variables." and the results section will describe the statistical tests for main effects and interactions (these are ANOVAs, or F tests). In popular press articles look for key phrases such as "it depends" and look for participant variables

Subjects

In experimental research, the cases or people used in research projects and on whom variables are measured

manipulation; measurement

In experiments, the ____________ occurs first, then a __________ is made.

Complete Participant

In field research, when a researcher fully participates in all aspects of the study group's activities as though a member of the group (too much sympathy can be an issue)

Complete Observer

In field research, when a researcher only observes the study group without participating in their activities

Devil's Advocate

In focus group research, refers to a person whose role is to argue against a dominant idea and who could introduce new questions and new ways of thinking into the group so as to prevent the tendency toward group conformity

Groupthink

In focus group research, refers to people's natural desire to avoid conflict and lean toward group consensus, even when the opinion of the group does not reflect their own personal opinions

quantitatively

In order to make a graph, the data collected must be ____________________ testable.

Significance (statistical)

Is achieved when there is a low probability that the results of an experiment occurred by chance alone. If the probability of their occurrence by chance is equal to or less than 5 per cent or 0.05

Limitations of Authority

Is not always accurate, often accepts expert's statements as facts, not all experts are experts

What questions are helpful to ask while reading journals and journalism?

Is original research evidence cited? Has the research been published in a peer-reviewed journal yet? How does coverage of the same journal article differ across news stories?

Convergent validity

Is the measure strongly related to other established measures of the same (or a similar) constructs?

Discriminant validity

Is the measure weakly (or not at all) related to established measures of a different construct? Is the measure weakly correlated with something it shouldn't be correlated with? (this is good, do not want high correlation with something measure should not be correlated with)

Self Questionnaire- Monitoring returns

It is important to keep track of returns so that the researcher identifies who need to be contacted again and the optimal time for sending follow-up (reminder) mailings.

Deontological Objection

It is never okay to lie even if it is used for a study and people may not be at risk for discomfort, etc. Could lead to the development of mistrust and suspicion in future studies

Filler items

Items included in a questionnaire measure to help disguise the true purpose of the measure.

Reverse Scoring

Items stated opposite of the other items; must be reversed in analysis.

Problems with Experience

Lack of COMPARISON: Typically no fair control condition CONFOUNDED Claim: Other possible explanation for your findings -Example: You feel calm after you exercise, you think its the exercise but it could be the temperature of the gym or the music you had playing

What are two problems with claims based on experience?

Lack of COMPARISON: Typically no fair control condition CONFOUNDED Claim: Other possible explanation for your findings -Example: You feel calm after you exercise, you think its the exercise but it could be the temperature of the gym or the music you had playing

1. State the Question or Problem 2. Hypothesis 3. Plan the Investigation 4. Gather Data 5. Organize Data 6. Analyze Data 7. Draw a Conclusion

List the 7 steps of the Scientific Method that we will use.

Hour Glass Writing of Research

Lit review is broad and narrows to a specific research question. methods and results sections are more narrow and specific. The discussion starts narrow tied to the meaning of results then broadens to importance for the field.

Justice

Making sure the subjects are not exploited

National Research Act (1974)

Mandated regulations for protection of subjects

Correlation

Means that things go together or are associated (e.g., people with higher values on the income variable are likely to have higher values on the life expectancy variable)

Physiological

Measure of a biological state Examples: Hormone levels, blood flow in brain, heart rate, pupil dilation Conceptual variable: Arousal Operational variable: Pupil dilation

Coefficient of Determination (r^2) & how to calculate

Measure of the amount of variance in one variable can be predicted from the other variable. Calculated by squaring the value of r. Ex. If correlation of variable x to y is r = .70 then r^2 = .49. .49 x 100= 49% of the criterion value is explained by the predictor value.

How is validity established?

Measurement procedure must accurately capture the variable that it is supposed to measure

Face validity

Measures whether a test looks like it tests what it is supposed to test Does the measure look like a plausible measure of the construct at face value?

The following is an example of which method: Black cats are bad.

Methods of Tenacity

Deception

Misinformation that a participant receives during experiment

In a Negatively Skewed Distribution...

Mode is the highest mean is the lowest median is in between the two

In a Positively Skewed Distribution...

Mode is the lowest Mean is the highest Median is in between the two

We cannot make causal claims from correlational data

Most important thing to remember about correlations

Survey

Most widely used data-gathering technique in social research. Sppropriate for research questions about self-reported beliefs and behaviours. -Started from positivist approach -Can ask about many things at one time and test several hypotheses

Empiricism - Independently Verifiable

Need to be able to distribute data and make it so another scientist can follow the methods you used and get the same answer

Nominal

No inherent order Categorical data Often open-ended, and demographic questions Frequency data and chi-square Nonparametric measures

Internal Validity (3 criteria for causal claims)

No other possible explanations for why x and y are related

Why use deception?

No other way to conduct your study

Zero Correlation

No real association - line of best fit straight across, no trend

Can you prove or disprove something in research?

No, you can only support

examples of independent groups quasi-experiment tests?

Nonequivalent control group design (posttest-only) - Nonequivalent control group design (pretest/posttest).

quasi-experimental study with at least two groups, measured one time.

Nonequivalent control group posttest only Example: The Head Start Study

quasi experimental study with at least two groups; measured before and after an intervention.

Nonequivalent control group pretest/posttest design: Example: psychological effects of cosmetic surgery

Scientific Journalism

Not considered science per se. Is scientific news and commentary written by another party (usually a reporter, not a scientist) and posted to a wide readership (usually a non-expert audience).

Survival

Number of people alive at a certain point (often 1 year or 5 year) after diagnosis

survival rate

Number of people alive divided by population

U.S. Surgeon General

Orders federally-funded research proposals from the Public Health Service to be reviewed for ethical treatment of subjects

Quantitative/continuous measurements

Ordinal, interval, ratio

Confound

Other possible explanations for an outcome

pretest/posttest

Other times, researchers want to measure a dependent variable before and after a manipulation.

Which conference will the professor of the BPSYC Research Methods class be presenting at this year?

Persuasive Technology

Pretesting the questionnaire

Piloting the questionnaire is critical for catching any errors or confusing items.

the people who were present for the pretest AND the posttest. (don't count the drop-outs).

Preventing attrition threats is easy. Only analyze

True

Research questions should be actual questions that can be answered rather than statements that make some claim.T/F

Consumer-Oriented

Provide consumers with complete picture on effects of a program. Consumers often other counselors, prospective clients, administrators.

Expertise-Oriented

Provide professional judgements about quality of a program. Ex. Accreditation standards for CACREP counseling.

Declaration of Helsinki (1946)

Provided international ethics guidelines for medical research (written protocol, detailed description, reviewed by council)

Quantitative Research

Provides numerical scores and is submitted to statistical analysis for summary and interpretation

MEDLINE Database

Provides the authoritative medical information on medicine, nursing, dentistry, veterinary medicine, the health care system, and pre-clinical sciences

IQR (interquartile range)

Q3-Q1 (split data in half and then find the median of both halves)

Pre/post-test

Quantitative measure of dependent variable given before and after the intervention. Can compare similarities between groups before onset of study.

Post-test

Quantitative measure used to determine the effects of the intervention between the treatment and control group.

Order Effects (2 sources of question bias)

Question order Solution: Avoid open-ended after forced-choice questions Multiple response options Solution: Order in more than one way

Questions and statements

Questions elicit specific information from a respondent; statements ask a respondent their reaction to a particular claim.

Empirically Testable Research Questions

Questions expressed in terms of a single variable or relationship between variables

CONSORT is used to evaluate _____

RCTs

Based on the video watched in class, what is the quickest method to extract information from a research article?

Read through the article in three passes

Discussion

Recap results, limitations, future directions

Physiological measure

Recordings of responses of the body.

Why Questions

Research warn against using these for a survey. Only appropriate if a researcher wants to discover a respondent's subjective understanding or informal theory

Moderator

Refers to a trained facilitator used in focus group research who guides the focus group discussion

Matrix questions

Refers to instances when a series of items share the same set of responses. While efficient, it can produce a response set, where respondents answer all of the items in the same way.

Semi-participant

Refers to the role of the researcher in field research when he or she participates to some degree in group activities, but not as much as a full member.

prevelance

Refers to the total number of cases of a condition at a given time. whether new or previously diagnosed

B

Regression is especially problematic in which of the following situations? When the experimental and comparison groups are equal at posttest When one group has an extremely high score at pretest When the experimental and comparison groups are equal at pretest When one group has an extremely low score at posttest

Type 1 error

Rejecting null hypothesis when it is true

Type 1 error

Rejecting null hypothesis when it is true (false positive)

Scatterplot Form

Relationships can take three forms: independent, linear, and curvilinear

What is the relationship between reliability and validity?

Reliability is a prerequisite for validity

Minimal risk research

Research in which participants are exposed to risks that are no greater than those encountered in daily life or in routine physical or psychological tests.

True

Research can be found in academic journals, books, newspapers, magazines, legislative actions, policy decisions, and databases. Some of this information is more objective and accurate than others.T/F

Alternative Hypothesis

Research hypothesis contradicting the null. ex. Each bag does not have 17 M&M's.

Panel study

Research in which the same sample of subjects are surveyed at two or more points in time, usually to assess changes that occur over time.

Exempt research

Research in which there is no risk is exempt from review by the IRB. Ex. Questionnaires, surveys, naturalistic observations, etc.

Surveys

Research methods in which researchers use questionaires or interviews.

tentative

Research of ethics is _________ and not chiseled in stone.

1) Literature Review 2) Hypothesis 3) Develop Methodology 4) Data Collection 5) Analyze Results 6) Interpret & Research Conclusion

Research process

Factor Analysis

Researcher determines the effect of two or more independent variables (factors), both singularly and interacting with each other on a dependent variable. Purpose of data reduction & summarization. Two values: main effect & interaction effect

Charm and Trust

Researcher needs social skills and personal charm to build rapport

Observer bias

Researchers expectations influence how they interpret the results of a study

voluntarily, meaning that the research subjects must be completely free to participate or not participate without threat and informed, meaning that the researcher must inform the research subjects about the research and any risks associated with participation in the research.

Researchers must obtain consent from research subjects prior to collecting information from them. This consent must be:

Principle of Justice

Researchers must show that the people participating in the study are REPRESENTATIVE of the people that will be helped by any positive results

Central Tendency

Researchers often want to summarize the information about one variable into a single number: mean, median, or mode

Beneficence

Researchers should try not to have any negative impact on the wellbeing of the people who participate in their studies

True

Researchers use three causal rules to determine whether a causal relationship exists between two variables. Each of these three rules must be met before a researcher can prove that one variable is the cause of another. T/F

Belmont Report (1979)

Risk and benefits fairly distributed

SD v. standard error

SD: -sample statistic -known -spread of observation -normal curve applies -relating all known into to each other Standard error: -tool for predicting -calculated based on sample info, sample size -predict what you might find in population using 1 observation *both take normal curve approach -get standard error to be smaller by taking a larger sample -varied population will produce large standard error

State the Question or Problem

Scientific Method Step #1

Hypothesis

Scientific Method Step #2

Plan the Investigation

Scientific Method Step #3

Gather Data

Scientific Method Step #4

Organize Data

Scientific Method Step #5

Analyze Data

Scientific Method Step #6

Draw a Conclusion

Scientific Method Step #7

Haphazard (convenience) sampling

Selecting subjects in a haphazard manner, usually on the basis of availability, and not with regard to having a representative sample of the population; a type of nonprobability sampling.

Why is it important to ensure the methods section of a report is accurate?

So that others can replicate it

clear, step by step, and numbered

So that others can replicate your exact experiment, you must write a __________________________________________ procedure.

Negotiating

Social relations are negotiated and formed throughout the process of fieldwork. Occurs with each new member until a stable relationship develops to gain access, develop trust and reduce hostile reactions

Qualitative Sampling

Social scientists typically select interview participants through nonprobability sampling (e.g., snowball and purposive sampling are often used)

Micro-Level Theory

Social theories and explanations about the concrete, small-scale, and narrow level of reality, such as face-to-face interaction in small groups during a two-month period

Macro-Level Theory

Social theories and explanations about the more abstract, large scale and broad scope aspects of social reality such as social change in major institutions (eg. the family, education) in a whole nation across several decades

Grounded Theory

Social theory that is rooted in observations of specific, concrete details

Measurement error Individual differences Situation noise

Some common sources of unsystematic within-group variance (noise)

Gatekeeper

Someone with the formal or informal authority to control access to a site

ceiling effect, floor effect

Sometimes an IV has no effect because there is little room for improvement (_________), or little room for doing worse (__________).

Weak manipulations Insensitive measures Ceiling and floor effects Design confounds

Sometimes real differences between groups are hard to detect. There can be multiple causes for this:

Point Biserial Correlation Coefficient (r sub pb)

Special form of Person r used when one variable is nominal and the other is continuous. Nominal must be dichotomous (yes/no, true/false, male/female). Ex. Passing the NCE (dichotomous) and grade point average (continuous).

What are 2 criteria for a research question?

Specific and Testable

Operational Definition

Specify precisely how the concept/term is measured or manipulated in that particular study. *Must operationally define hypothetical constructs!!*

bias

Sponsorship ______occurs when the sponsor of a research project attempts to influence the study design or interpretation of data for its own benefit.

C

Spontaneous remission in clinical studies is an example of which of the following threats to internal validity? Regression Attrition Maturation Placebo effects

True

Spuriousness refers to a false causal finding. It occurs when a researcher alleges a causal relationship between two variables but fails to confirm at least one of the three causal rules. T/F

Staged manipulation

Staging events during the experiment in order to manipulate the independent variable successfully.

Alternative Hypothesis

States that there is a relationship between the two variables being studied (as opposed to the null hypothesis)

Correlation coefficient

Statistic showing strength of the relationship (r)

p < .05

Statistical Significance

Random Error

Statistical fluctuations that are caused by unknown & unpredictable changes in the experiment

Univariate Statistics

Statistical measures that deal with one variable only Easiest way to describe: frequency distribution

Bivariate Statistics

Statistical measures that involve two variables only

Example of Mixed Design

Study was looking at how long children stare at angry or happy faces based on whether or not they had been previously abused. Between subjects: Child status was naturally occurring- quasi-experimental within subjects: face stimuli- angry or happy

When conducting a literature review which two key elements should you make note of?

Subject Words and Authors Names

Respect for Persons

Subjects have to give informed consent

Types of Measures of Reliability

Successive Measurements Simultaneous Measurements Internal Consistency

Belmont Report

Summarizes ethical principles and guidelines for research involving human subjects. It has three basic ethical principles.

Abstract

Summary

Which method obtains answers from habit or superstition?

Tenacity

Fence Sitting (Response Sets + Solution)

Tendency to be neutral about everything Solution: Use even number of responses. But sometimes people really have no opinion!

Social Desirability (+ Solution)

Tendency to give answers to look better or conform to social norms ("faking good" or "faking bad") Solution: Make survey anonymous Include filler questions to mask purpose Include 'too-good-to-be-true' questions

Observed P value

Test calculates probability of obtaining the observed results if the null hypothesis is true

Successive Measurements

Test-retest reliability compares scores of two successive measurements of the same individuals and correlates the scores

Behavioural Theories

Testing predictions that are part of a theory

If an ethical dilemma is not mentioned in the BCBA guidelines, what should be referenced?

The APA Guidelines

"Ethic Principles of Psychologists and Code of Conduct"

The APA has developed an extensive document known as the "_____________" in an effort to keep pace with the challenging changes of scientific ethics.

PsycINFO Database

The American Psychological Association's (APA) definitive resource for abstracts of scholarly journal articles, book chapters, books, and dissertations devoted to peer-reviewed literature in behavioural science and mental health

The Horizontal-Vertical Illusion is an example of which method?

The Empirical Method

Who enforces ethical guidelines?

The Institutional Review Board

The following is an example of which method: All 3 year old children are afraid of the dark. Amy is a 3 year old girl. Therefore, Amy is afraid of the dark.

The Rational Method

scientists

The Scientific Method is used by ________________ to find out about the natural world.

expedited

The ____________ review does not require approval from the full IRB committee, but only a few members, and is completed more quickly than a full review.

materials

The _____________ list is not numbered.

literature review

The ______________ involves locating and understanding what previous researchers have learned about a topic.

data chart

The ______________________ is used to collect information during the experiment in an organized way.

independent variable

The _________________________ is the variable you choose to manipulate or change in the experiment.

dependent variable

The ___________________________ is the variable that you will measure in your experiment and will change in response to the independent variable.

Sensitivity

The ability of a measure to detect differences between groups.

Validity

The ability of a test to measure what it is intended to measure, are you hitting the bullseye

Internal Validity

The ability to eliminate alternative explanations of the dependent variable Rule out variables other than the treatment by controlling experimental conditions. Threats: -Selection Bias -History Effects -Maturation -Mortality -Diffusion of Treatment or Contamination -Experimenter Expectancy

External Validity

The ability to generalize experimental findings to events and settings outside the experiment itself

External validity

The degree to which the results of an experiment may be generalized.

General questionnaire format

The design of the questionnaire should be straightforward and uncomplicated. Items should be clearly distinguished from one another so they are not missed.

Non-statistical Definition of Sampling Error

The deviation between sample results and a population parameter caused by random processes

attributes

The different characteristics or values that a variable can take on are called that variable's:

Sampling error

The error that arises as a result of taking a sample from a population rather than using the whole population.

Content validity

The estimate of how much a measure represents every single element of a construct.

What is important to consider when writing a research proposal in terms of the ethicist on the REB?

The ethist may not have an ABA background, therefore it is important when writing a proposal to write it at a level that this individual will understand

independent variable

The experimental factor that is manipulated; the variable whose effect is being studied.

reliability capicty

The extent to which a measure consistently measures a social phenomenon is referred to as the measure's:

Convergent validity

The extent to which the scores on one measure are related to scores collected from a similar or different measure.

ANOVA

The farther away from *1.0* the *more* statistically *significant* the result in what test for *multiple related groups*?

True

The first causal rule, temporal order, requires that the cause must precede the effect. In other words, the variable that is alleged to be the cause of another variable must happen first. T/F

-refine their research question -identify a theory related to their research question -propose a hypothesis that predicts the answer to their research question.

The literature review process helps researchers:

Mortality (also attrition)

The loss of subjects who decide to leave an experiment.

Attrition (also mortality)

The loss of subjects who decide to leave an experiment.

before

The freedom to refuse or to discontinue the research study must be made clear to the participant ____________ the experiment.

Reactivity

The general threat to external validity that arises because subjects are aware they are in an experiment and being studied -Hawthorne Effect

Control Group

The group that does not receive the treatment in an experiment

Experimental Group

The group that receives the treatment in an experiment

your prior knowledge and research

The hypothesis is based on ___________________________________________.

tested

The hypothesis must be expressed as a statement and must be capable of being _____________.

personal pronouns or iffy words

The hypothesis should never contain

File drawer problem

The idea that meta-analysis could be overestimating the support for a theory because studies that find null effects are less likely to be published therefore are less likely to be included in such reviews.

Journal Article Impact

The impact factor (IF) is a measure of the frequency with which the average article in a journal has been cited in a particular year. It is used to measure the importance or rank of a journal by calculating the times it's articles are cited.

What information can you use to make a judgement about a journal article's impact?

The impact factor (IF) is a measure of the frequency with which the average article in a journal has been cited in a particular year. It is used to measure the importance or rank of a journal by calculating the times it's articles are cited.

Observer Error

The individual who makes the measurements can introduce simple human error

Sampling frame

The individuals or clusters of individuals in a population who might actually be selected for inclusion in the sample.

Framing Effects

The influence, on the respondent, of how a polling question is asked; changes in wording can significantly alter many people's answers

Power

The likelihood that a study will have results that are statistically significant when the independent variable causes a change in the dependent variable. The more power a study has, the more likely we are able to detect true differences. In order for experimenters to increase the power, they must have large samples, accurate measurements, or strong manipulations to escalate the size of the effect.

Literature

The mass of published information worldwide

level of precision

The measure of a sample's representativeness and therefore its ability to accurately predict conditions with the population from which it was collected is referred to as its:

Dependent Variable

The measured variable; the outcome - expect to change

Post-test

The measurement of the dependent variable of the experiment after the treatment

Pretest

The measurement of the dependent variable of the experiment prior to the treatment

Median

The middle score in a distribution; half the scores are above it and half are below it. 50th percentile. First organize the scores from highest to lowest, then count to the middle -Best for skewed distributions

within; between groups

The more unsystematic variance (noise) there is __________ groups, the harder it is to see differences __________ groups (b/c there's more overlap)

Mode

The most common, frequently occurring number. The easiest to use and can be used with nominal, ordinal, interval, or ratio data. Distribution can have more than one.

Population

The much larger collection of animals or people that researchers want to generalize about.

Sampling Element

The name for a case or single unit to be selected. It can be a person, a group, an organization, a social action. ie. an arrest, a kiss

Population

The name for the large general group of many cases from which a researcher draws a sample and which is usually stated in theoretical terms. Researcher begins with this and then defines it more precisely. Abstract concept that requires an operational definition. ie. All persons aged 16 or older living in Regina on December 2, 2016, who were not incarcerated in prison, asylums, and similar institutions All business establishments employing more than 100 persons in Ontario that operated in the month of July 2015 All admissions to public or private hospitals in the province of Alberta between August 1, 2010, and August 1, 2015

Target Population

The name for the large general group of many cases from which a sample is drawn and which is specified in very concrete terms

Interview Schedule

The name of a survey research questionnaire when a telephone or face-to-face interview is used

-answer research questions -to resolve disagreements among researchers -to fill gaps within the body of knowledge -to produce more research questions.

The objectives of research are to

Field Site

The one or more natural locations where a researcher conducts field research

Independence

The opposite of correlation. It means there is no association or relationship between variables. If two variables are independent, cases with certain values on one variable do not have any particular value on the other variable

main effect

The overall effect of one independent variable on the dependent variable, averaging over the levels of the other independent variable. Overall effect

Response rate

The percentage of people selected for a sample who actually completed a survey.

Subjects

The persons or animals whose behavior is systematically observed in a study.

Falsifiability

The principle that a good scientific idea or theory should be capable of being shown to be false when tested using scientific methods..

Replication

The principle that researchers must be able to repeat scientific findings in multiple studies to have a high level of confidence that the findings are true

power

The probability that an experiment will find a significant effect, assuming the effect actually exists.

question ending with a question mark

The problem must be written as a _________________________________________________________________.

Matching

The procedure whereby pairs of subjects are matched on the basis of their similarities on one or more variables, and one member of the pair is assigned to the experimental group and the other to the control group.

Sampling

The process of choosing members of a population to be included in a sample.

Odds and odds ratio

The ratio of probability of an event occurring to the probability of the event not occurring that is p divided by (1-p) where p=probability and The amount by which the odds are multiplied for one group compared with another. Odds for men relative to women are 1.5/0.54=2.77 - we say that this is the odds ratio for males, which means that the odds for men are 2.77 times those for women.

Sampling Ratio

The ratio of the size of the sample to the size of the target population ie. If the population is 500 and the researcher samples 100, ratio is 100/500 = 0.20, or 20 percent

True score

The real score on the variable

correlation

The second causal rule, ___________, requires that the variables in a causal relationship be related to one another, or change together. A change in one variable must be associated with a change in another variable. Correlations can be positive or negative.

How to Evaluate a Hypothesis regarding p and alpha levels

The smaller the p-value, the stronger the evidence is against the null hypothesis. If the p is low, the Ho must go. If p is less than or equal to alpha (.05), the observed effect is statistically significant. The null hypothesis is rejected in favor of the alternative hypothesis. If p is greater than alpha (.05), there is no statistically significant difference, fail to reject the null hypothesis.

counterbalancing.

The solution to order effects is

How many articles is enough during a literature search?

The study (or studies) you find provides some justification for new research

Control Group

The subjects who do not receive the special treatment given to the experimental group.

Experimental group

The subjects who receive some special treatment in regard to the independent variable.

matrix

The summary data sheet should contain all the data in a _______ format and be a single sheet.

Respondent Fatigue

The tendency for respondents to lose interest in participating in research if they perceive their participation to be taking too long or requiring too much effort

Confirmation Bias

The tendency to seek out and interpret evidence consistent with what we believe and to deny, dismiss, and distort evidence that contradicts what we believe

True

The third causal rule, lack of plausible alternative explanations, requires the researcher to eliminate all other reasonable causes before concluding that one variable causes another. T/F

How does new research grow out of old? What visual is used to describe this?

The tree-like structure emphasizes the notion that current research (the tips of the branches) is always based in previous research.

Positive Relationship

The two measurements change together in the same direction

Negative Relationship

The two measures change in opposite directions

3 Assumptions Needed to Test whether There is a Difference between Population Means

The two populations have the same variance: The assumption of homogeneity of variance. The populations are normally distributed. Each value is sampled independently from each other value.

True

The unit of analysis is the 'what' or 'whom' about which researchers gather information during a research project. T/F

Archival research

The use of existing sources of information for research. Sources include statistical records, survey archives, and written records.

Error Variance

The variability or change in a person's behavior that is NOT related to the other variable of interest, but instead remains unexplained

Systematic Variance

The variability or change in a person's behavior that is related to the other variable of interest

Independent variable

The variable that is manipulated to observe its effect on the dependent variable.

interval ratio

The variable age (in years) is measured at what level of measurement?

ordinal nominal

The variable education (high school diploma or equivalent, Bachelors degree, Masters degree or higher) is measured at what level of measurement?

Theory-Data Cycle

Theory a set of propositions that attempts to specify the interrelationships among a set of concepts (aka how the world works). How and why things are related. Hypothesis A specific prediction derived from a theory and a specific research design

Trend studies collect the same data over time from different samples of the same population. Panel studies collect the same data over time from the same sample. Cohort studies collect data over time from samples of individuals who share common characteristics.

There are three types of longitudinal research designs.

Confidentiality

There are two types of privacy in research. (1) __________ means that the researcher knows who the research subjects are but does not disclose their identity to others. (2)Anonymity means that nobody, not even the researcher, knows the identity of the research subjects.

What does a 95% confidence interval of 100 to 110 mean?

There is a 95% probability that the interval from 100 to 110 contains the true population value.

What is it important to consider when using secondary sources?

There is a chance that the source you obtained it from may have misinterpreted the primary source

Limitations of Intuition

There is no method for separating accurate from inaccurate knowledge

Provide two examples of a research paper

Thesis and journals such as the Journal of Applied Behaviour Analysis

One-group pretest post-test design

This design has one group, a pretest, a treatment, and a post-test. NO control group and NO random assignment

Self Questionnaire- Follow-up mailings

This entails sending either a reminder or another survey to respondents who have not already returned the survey.

Main effects (what they are, what they mean, how they are found)

This is the overall effect that one independent variable has on the dependent variable over all of the levels of the other independent variable. It is basically the simple difference/ effect a single variable. -For each independent variable, there is one main effect.

Self Questionnaire- Response rates

This refers to the proportion of the sample that returned the survey. The higher the response rate, the more representative the survey is of the population.

Covariance, Temporal Precedence, Internal Validity

Three criteria for causal claims

Belmont Report

Three key ethical principles to guide all human research

Feasibility

Time, money, technical knowledge and skill, and access to special equipment and research participants

Wait-list design

To control for selection effects, all participants in a quasi-experimental study plan to receive treatment, but are assigned to receive the treatment at different times. This is reassuring to the researchers because they are confident that they have similar types of people in each group

One-group Pretest-Postest Design

To measure participants before the manipulation (a pretest) and again afterward (a posttest).

True

To research means to follow a logical process that uses concepts, principles, and techniques to produce knowledge. T/F

Probability sampling

Type of sampling procedure in which one is able to specify the probability that any member of the population will be included in the sample.

Self Questionnaire- Mail distribution and return

Typically self-administered surveys are distributed through the mail, and the responses are subsequently mailed back to the researcher. Researchers make every effort to ensure that returning the survey is as easy as possible.

Noise/error variance/unsystematic variability

Undifferentiating variability among the members of a group in an experiment. This gets in the way of detecting the true difference between groups. The more unsystematic variability there is, the more the groups' scores overlap, which causes the difference between the two groups to become indistinguishable.

Number needed to treat

Used in assessing the practical interpretation of the differences between two or more groups Multiplicative inverse of the estimated benefit of the treatment. For example, if the treatment reduces an outcome by two percentage points, the NNT is 1/0.02=50.

Independent Variable

Variable that is manipulated in an experiment

Negative Correlation

Variables tend to move in opposite direction. As one variable goes up, the other goes down. As one variable goes down, the other goes up. Denoted by - before the correlation coefficient.

Positive Correlation

Variables tend to move in the same direction. As one variable goes up, the other goes up. As one variable goes down. The other goes down. Denoted by + before the correlations coefficient value.

Constructs

Variables that cannot be observed or measured directly

Counterbalancing

Vary order of treatment to distribute or measure order effects randomized blocks

Mean

What *central tendency* calculates the *sum* of the scores in a distribution of a population or collection of things *divided* by the *number of scores*?

Median

What *central tendency* is the *midpoint* or *mid-score* in a distribution that presents with 50% falling below and 50% falling above that line?

Decrease variance

What *increases power* by *reducing* the *within group differences* make observing sample groups that are more *homogeneous* which removes potential outliers?

Increase reliability of Measures

What *increases power* by reducing error, wobble, and variance by using the *most recently developed tests* for measurement?

- Diffusion - Imitation - Rivalry - Equalization

What *threats* to *internal validity cannot* be ruled out by *random assignment*?

- History - Maturation - Instrumentation - Selection - Regression to Mean

What *threats* to *internal validity* are ruled out by *random assignment*?

Standard deviation

What accounts for how much *dispersion* there is *around* the *mean*?

Systemic error

What are *differences* that can be explained *due* to a *confound*?

Between subjects

What are *groups* that are *completely independent* of one another?

Within Subjects

What are *groups* that are some how *related* to one another?

- Convenience sample - Quota sample - Snowball sample

What are *non-probability sample* methodologies?

- Mean - Median - Mode

What are the components of *central tendency*?

- Nominal - Ordinal - Interval - Ratio

What are the different types *descriptive statics* of data format?

Random sampling

What is a subset of a statistical population in which each member of the subset has an equal probability of being chosen that lends strength to *external validity*?

Sample

What is a subset of people selected out of the population for inclusion in the study?

- Experimenter Expectancy - Volunteer-Non-response bias - Withdrawal - Contamination - Attrition - Mortality

What are the type of *biases* that weaken *internal validity*?

- Opportunistic Data Torturing - Procrustean Data Torturing - Publication Bias

What are the types of *bias in analysis* and publication?

- Increase *Effect Size* - Increase *Sample Size* - Increase effect - Increase reliability of Measures - Decrease variance - Multiple measures

What can researchers do to increase power of their findings?

Statistical significance

What can you ascertain if there is less than 5% likelihood that a result is due to chance?

Inclusion criteria

What comes *firsts*, decides who will be *included in a study*, and helps determine whom the results *can* be *generalized* to?

Exclusion criteria

What comes *second*, decides who will be *excluded in a study*, and whom the results *cannot* be *generalized* to?

Comparison Group

What compares *multiple tx* options?

Results

What did the author find?

Covariance

What do all correlational studies buy you?

Experimenter Expectancy bias

What involves an *experimentors hypothesis* and *expectations* affecting the result or *outcome* of an experiment that is limited by a *double blind study*?

Comparing Means

What involves looking at *two groups* and comparing the *means* of each group to make *inferential statistics*?

Peer review

What involves sending a journal article to *four peers* in the field and they critic it?

Diffusion/Imitation

What is *threat* to *internal validity* that involves the *control* group being *affected* by *tx* from the *tx group*?

Type II: beta error

What is a *failure* to *reject* the *null hypothesis*, but in *reality* the medication is effective, giving a *false negative*?

Convenience sample

What is a *non-probability sampling* methodology that includes anyone who is convenient?

Snowball sampling

What is a *non-probability sampling* methodology that is future sampling guided by results of earlier data collection?

Quota sample

What is a *non-probability sampling* methodology that sets *quotas* for proportions of people to be included with different characteristics?

Hypothesis

What is a *prediction* about how two or more variables are related that has to be *testable*?

Stratified random sample

What is a *probability sampling type* first stratifies the population into *categories* then randomly selects from the categories?

Pure random selection

What is a *probability sampling type* involves each person in the population has an equal chance of being in the sample?

Hierarchical random sample

What is a *probability sampling type* randomly selects from *progressively smaller units*?

Type I: alpha-error

What is a *rejection* of the *null hypothesis*, but in *reality* the medication is not effective, giving a *false positive*?

Central tendency

What is a *statistical index* of a typical set of scores that comes from the *center* of the *distribution* of scores with the three most common being: *mean*, *median*, and *mode*?

Ceiling

What is a *threat* to *internal validity* from *instrumentation* where *can not go any higher* with instrumentation?

Floor effect

What is a *threat* to *internal validity* from *instrumentation* where *can't go any lower* with the specific instrumentation used?

Maturation

What is a *threat* to *internal validity* that involves the *mere passage of time*?

History

What is a *threat* to *internal validity* that is an *event* or *events* that deferentially *affects* the *tx* and *control* groups?

Instrumentation

What is a *threat* to *internal validity* that is due to *changes* or *limitation* in *instruments*?

Testing

What is a *threat* to *internal validity* that is due to *repeated exposure* to a *independent variable*?

Theory

What is a prediction about *why* variables are related with an actual story behind the prediction?

A statement explaining why you think that your hypothesis is correct.

What is a rationale?

Multi-Center Studies

What is a research trial is a clinical trial that uses the *same protocol* at *multiple sites* conducted at more than one medical center or clinic?

Effect size

What is the *strength of relationship between control* and *tx group*, where the farther from zero in negative or positive direction, the stronger the correlation relationship?

Less artificial

What is the *strength* of *field* based experiments?

Regression to the mean

What is the *threat* to *internal validity* that is just *due to chance*, where *extreme* scores tend to *regress* towards the *mean* in *later* measurements?

Compensatory rivalry

What is the *threat* to *internal validity* where the *control* group is *creative* or *motivated* to *out perform* the *tx group*?

Compensatory equalization

What is the *threat* to *internal validity* where the *control* group receives *similar tx* through *alternate means*?

Selection

What is the *threat* to *internal validity* which is *bias* in terms of *who ends up* in the *experimental* or *control group* and is ruled out by *random assignment*?

Artificial

What is the *weakness* of a *lab* based experiments?

Interquartile range

What is the amount of *dispersion* around the *mean* regarding how much *variability* there is *around the sample*?

Criterion Variable

What is the behavior of interest that is changing? (dependent v) example: Behavior problems in the classroom

- Random Assignment - Control group - Double-blind

What is the best case scenario for an experimental design?

Independent variables

What is the construct which is an *actor* variable that is manipulated by the researcher?

Dependent variables

What is the construct which is an *affected* variable that relies on the independent variable?

Interaction effect

What is the effect of one independent variable *depending* on the effect of another independent variable?

Main effect

What is the effect of one independent variable *ignoring* other independent variables?

External Validity

What is the extent to which one can *generalize results* that is strengthened by *random sampling*?

Internal validity

What is the extent to which one can make *causal inference* that is strengthened by *random assignment*?

5%

What is the likelihood of a *type I: alpha-error* where there is a rejection of the null hypothesis, but in reality the medication is not effective, giving a *false positive*?

20%

What is the likelihood of a *type II: beta-error* where there is a failure to reject the null hypothesis, but in reality the medication is effective, giving a *false negative*?

Power

What is the likelihood one can detect a *true effect*?

make causal claims

What is the main reason for conducting experiments? a) to have a more valid study b) to generalize to more people c) to make causal claims d) to collect evidence in the real world

-1.0

What is the perfect *negative correlation* r value?

+1.0

What is the perfect *positive correlation* r value?

C

What is the primary difference between pretest/posttest designs and within-groups designs? The number of participants used The number of times the dependent variable is measured The number of levels of the independent variable participants are exposed to There is no difference between the two designs.

Control Group

What is the pure, *no tx* or placebo group?

Over Sampling

What is the strategic *over-presentation* of population characteristics?

Research

What is the systematic accumulation of knowledge?

Compare Apples-to-Oranges

What is the top criticism of *meta analysis*?

Population

What is the total group of ALL people with defined characteristics?

Frequency distribution

What is usually a graph or table representing all values from least to greatest value with a *frequency count* of each *value*?

Random Error

What is when the *differences between participants* are *random*?

Open Ended Question

What learning strategies do you use when preparing for an exam?

Dropouts significantly different

What method of dealing with *attrition bias* accounts for significant different between dropouts and those who did not in terms of certain *characteristics*?

Tx attrition as dependent variable

What method of dealing with *attrition bias* considers that attrition may happen because of the *nature of the study* or *intervention*?

Intention to tx statistical adjustment

What method of dealing with *attrition bias* involves creating a *projection* of what would happen if the participants completed the study?

Remove noncompliant

What method of dealing with *attrition bias* involves removing pts who *do not* complete or follow the *proper protocol*?

- More competent - Different metrics - Different protocol

What needs to be considered in *multi-center studies* that can change the interactions?

Tx-by-Center interactions

What needs to be considered in *multi-center studies* where the tx may be more or less effective at different locations?

Ecological Validity

What results reflect *real world phenomena* that makes field research more *externally valid*?

Maintaining Competence

What section of code is this? "psychologists undertake ongoing efforts to develop or maintain their knowledge in a field"

Informed consent prior to recording voices or images in research

What section of code is this? Obtain informed consent prior to recording voices or images for date unless it consists of naturalistic observation in public places.

deception in research

What section of code is this? Only use deception if it is necessary. Do not deceive when their is physical pain or severe emotional distress. Explain all of the deception that was integral no later than the end of the experiment and allow the participant to withdraw their data.

informed consent

What section of code is this? Participants must always give informed consent and if they cannot physically give it, then it must be provided by a legal guardian. This must be documented.

Boundaries of Competence

What section of code is this? Psychologists must be competent in the fields in which they work.

Client/Patient, Student, and Subordinate Research Participants

What section of code is this? When research participation is a course requirement or an opportunity for extra credit, the prospective participant is given the choice of equitable alternative activities.

informed consent to research

What section of code is this? what the psychologist plans to do must be told truthfully to the participants unless deception is involved.

Offering Inducements for Research Participation

What section of code is this?Make reasonable efforts to avoid offering excessive or inappropriate financial or other inducements for participation that may be coercive, if offering professional services, must tell of risks, obligation, and limitations

Institutional approval

What section of code is this?When approval is required for a study, psychologists will provide accurate information about their research proposals prior to conducting the research.

Volunteer or non-response bias

What source of *bias* involves pts either volunteer or *refuse to participate*?

Mortality

What source of *bias* involves pts in one group being *more likely to die* where the dying may even be a *dependent variable*?

Attrition

What source of *bias* involves pts in one group is more likely to *dropout* where the dropping out may even be a *dependent variable*?

Contamination bias

What source of bias involves *pts* in the study *receiving* more than one *intervention* or tx in *conjunction* with the *study*?

Withdrawal bias

What source of bias involves *researcher withdraw pts* to avoid undesirable outcomes?

Inferential statistics

What statistical method *draws tentative conclusions* about a *population* based on observations of a *sample* selected from that population and furthermore make a probability statement about those conclusions to aid in their evaluation?

Rosenthal study

What study observed that *narrative reviews* provided *varied conclusions* while *meta-analysis* provided the *same conclusions*?

Parametric

What test is good for *statistical power* that requires *interval* or *ratio data*, variance *groups is similar*, and the *assumption* of a *normally distributed* population?

Nonparametric

What test makes *less specific assumptions* that does not directly incorporate estimates pertaining to population resulting in less power and less likelihood to detect a true effect?

Treatment

What the independent variable in experimental research is called

Nominal

What type of *data format* involves *categorical data*, categories like ethnicity, where numbers don't really have a meaning?

Ordinal

What type of *data format* involves greater numbers meaning *"more,"* but difference between units is not uniform, the difference between gold, silver, and bronze winners in the olympics?

Interval

What type of *data format* is the difference between units in uniform, but there is *no absolute zero* like temperature?

Ratio

What type of *data format* is where the difference between units is uniform, and there is an *absolute zero* like with a weight scale?

Crossover design

What type of *experimental design* involves *more than one* type of *tx* administered to each subject where the tx are *provided sequentially*, *rather* than *congruently*, and *comparisons* are made of the effects of the *different tx* on the *same subject*?

Narrative Literature Review

What type of *literature review* involves making the researchers own conclusions about what the literature says leading to bias in articles selected and how the researches a conclusion?

Systematic literature review

What type of *literature review* involves specific rules about how literature is selected for the analysis so anyone else could replicate how selected literature?

Meta-analysis

What type of *literature review* is a quantitative summary of a body of literature that attempts to get ride of bias of conclusion?

Randomized Block Design

What type of design *randomly assigns* at least one variable, but *can't randomly assign all variables*?

Longitudinal design

What type of design follows pts *over* a *significant time period*?

Correlation Design

What type of design has *no random assignment* and doesn't give inference to statistical tests?

Quasi-experimental design

What type of design has *no random assignment* but gets closer to the *causal inference* with *other methodological elements* with the goal of eliminating the top rival hypothesis?

Cross-sectional design

What type of design has *no random assignment*, but still *compare groups*?

Experimental design

What type of design has *random assignment*?

Positive Skew

What type of frequency distribution is skewed to the *left*?

Prospective research

What type of research involves *looking forward* and planing ahead to collect data and *cannot assume anything* since it can be *correlative* or *experimental*?

Retrospective research

What type of research involves looking back at existing data and therefore *cannot have random assignment* and therefore *cannot assume causal inference*?

- Pure random sample - Stratified random sample - Hierarchical random sample

What type of sampling are *probability sample* methodologies?

Mediators

What type of theory involves the relationship *between* two variables that can be explained by a third variable?

Moderators

What type of theory involves the relationship of one variable *on* another variable depending on a third variable?

Sample size

What, the bigger it is the better the *external* and *internal validity*?

Replication-plus-extension

When a researcher replicates their own study but includes additional variables to test other questions.

Measurement Bias

When a test or test item is not equally valid for individuals from a particular gender, ethnicity, or social class, or other group

Design Confound

When a variable other than the independent variable happens to vary systematically with the independent variable *due to an experimenter's mistake*

History threat

When an external/historical affects the majority of members of the treatment group when given the treatment. This can be prevented by including a comparison group.

Regression to the mean threat

When an extremely high or low-performance score recorded for the pretest that is likely to be not as extreme (closer to average) for the posttest. These extreme scores can give off the vibe that treatment motivated the effect of regression on the participant. This can be controlled for by using comparison groups as well to determine whether or not the treatment had the same, extreme effect as the comparison group who did not receive the treatment.

Selection-history threat

When an outside factor systematically affects one level of the independent variable of those participating in the study.

True

When conducting a research experiment, nearly every university, some criminal justice agencies, and many research organizations maintain Institutional Review Boards (IRBs) and require researchers to seek approval from these boards prior to doing research on human subjects.T/F

Social desirability

When the individual tries to answer questions in the most socially acceptable way.

Attrition threat

When the number of participants decreases from pretest to posttest (participants drop out of the study being conducted). In order to avoid this, researchers should remove the pretest scores of the subjects who did not complete the posttest- or at least not include the scores for the pretest average.

Double-blind designs

When the observer nor the participant does not have knowledge as to which group they are in.

Justification to waive informed consent:

When the research involves no more than minimal risk to participants. The waiver of informed consent will not adversely affect participants. The research could not be carried out if informed consent was required.

Null effects/null results

When the results of a study show that the independent variable made no significant change to the dependent variable.

Instrumental threat

When the tool used to measure the dependent variable becomes unreliable and changes over time (application, usage, different interpretation). Prevention can be conducted by calibrating the tool used for measurement to ensure that the pretest and posttest measures are equivalent. This can be done by solidifying codebooks before the experiment is conducted to ensure the observers are using the same measurement. Or researchers can conduct a posttest-only experiment.

Ceiling effects

When there is a problem in the experimental design where the independent variable groups score similar to the dependent variable, such that the scores fall to the higher end of the distribution.

Floor effects

When there is a problem in the experimental design where the independent variable groups score similar to the dependent variable, such that the scores fall to the lower end of the distribution.

Simple Regression

When using single independent variable to predict a single dependent variable.

Multiple Regression

When using two or more independent variables to predict a single dependent variable. All together; combined effect, which one will stand out more?

Negative Correlation

When variables vary in *opposite direction*?

Positive Correlation

When variables vary in the *same direction* it is called what?

References

Where did they get their information from?

Close Ended Question

Which do you do more often, restudy or self-test?

A

Which of the following can help prevent testing effects? Using a comparison group Establishing reliability of the measure Using a clear coding manual Employing a pretest-only design

A

Which of the following is NOT true of control variables? Control variables help define the control group. Control variables help establish internal validity. Control variables are essential in experimental designs. Control variables are kept the same for all participants.

C

Which of the following is NOT true of selection effects? Selection effects are a type of confound. Selection effects can occur when experimenters allow participants to choose their own treatment group. Selection effects are unimportant for interrogating internal validity. Selection effects can occur when researchers assign one type of person to one treatment group and another type of person to another treatment group.

D

Which of the following is a threat to internal validity found in within-groups designs but not in independent-groups designs? Selection effects Design confounds Demand characteristics Practice effects

D

Which of the following is an advantage of within-groups designs? These designs avoid order effects. These designs avoid demand characteristics. These designs are always possible. These designs rely on fewer participants.

C

Which of the following is true of instrumentation threats? They are the same as testing threats. They occur only when using mechanical instruments (e.g., blood pressure cuffs, scales). They can be avoided with counterbalancing. They are only problematic in observational research.

ratio

Which of the following levels of measurement is the most precise? ordinal interval ratio

B

Which of the following research designs is used to address possible selection effects? Posttest-only designs Matched-groups designs Pretest/posttest designs Correlational designs

B

Which of the following studies would NOT have a possible threat of observer bias? A study looking at the effect of eating fruits and vegetables and general physical fitness A study looking at the effect of tutoring on SAT scores A study looking at whether increasing the number of hours listening to music increases singing ability A study looking at the effect of increasing the rate of text messaging in a day on writing/English composition skil

D

Which of the following threats to internal validity can apply even when a control group is used? Attrition History threats Instrumentation threats Demand characteristics

The data you have gathered and analyzed accepts or rejects the hypothesis. The data either proves your hypothesis right or wrong.

Why or why not do you not accept or reject the hypothesis?

Tuskegee Study (1932-1972)

Withheld penicillin from participants with syphilis even after it became the standard treatment - and without their knowledge

Are Observations better than self-reports?

Without training, observers may often exhibit bias. • However, self-reports are, in some sense, just reports from untrained observers. - Poor operationalization can also increase the likelihood of bias for both self-report measures and trained observers. • If there is no clear operational definition for the behavior that is being rated, reliability may be low. If reliability is low, we can't have good construct validity.

Double-barreled questions

Wording that occurs when two ideas are combined into one question and it is unclear whether the answer is for the combination of both or one or the other question

Applied

_______ research is a type of applied research that involves practitioners in the research design and then the implementation and evaluation of a resulting policy or programmatic change. Its purpose is to improve some aspect of practice (e.g. combating gang violence). In action research, a problem is diagnosed; an intervention or strategy is devised, and data is collected to fine tune the intervention strategy. This type of research is an integral part of the problem-oriented policing philosophy.

Pure

__________ research is conducted for the purpose of expanding the body of knowledge. University professors conduct research primarily to expand the body of knowledge about a particular subject or to develop theories about social behavior. Typically, this research is published in scholarly journals and read by other scholars interested in the same topics.

Qualitative

____________ research probes the meaning of variables and assumes that the social world is best understood subjectively, often from the perspective of the research subject.

The Scientific Method

__________________________ uses process skills and follows a logical, organized, empirical, yet flexible approach to problem solving.

The Scientific Method

______________________________ is a systematic way of testing an idea, hypothesis, prediction or inference.

_____________________________________

____________________________________________

Applied

_________research is intended to collect the information necessary to solve a specific problem. Researchers, like practitioners, consultants, and some academics, conduct research to address and solve problems. This research is often done for a client, who intends to use the knowledge gained from the research to address a specific issue. Typically, this research is published in reports consumed internally within organizations. Although applied researchers often produce knowledge, their primary intent is to address a specific issue or solve a current problem. We will discuss applied research in more detail in a later chapter.

normal curve

a bell-shaped curve described by a certain mathematical function

nested case-control study

a case-control study nested within a cohort study

Within group

a change in the dependent variable in a group is measured across time: - Crossover - Latin Square - Splitplot

vignette

a description of a hypothetical situation, event, or scenario to which participants react

table

a display of data in a matrix format

skewed

a distribution that is not symmetrical

frequency polygon

a frequency distribution in which the frequencies are connected by straight lines

histogram

a frequency distribution in which the frequencies are represented bu contiguous bars

cumulative frequency distribution

a frequency distribution that shows the number of scores that fall at or below a certain score

time-series graph

a graph in which the abscissa (x-axis) represents time

scattergram

a graph showing the responses of a number of individuals on two variables; visual display of correlational data

frequency distribution

a graph that shows the number of scores that fall into specific bins, or divisions of the variable

Scatterplots

a graphed cluster of dots, each of which represents the values of two variables. The slope of the points suggests the direction of the relationship between the two variables. The amount of scatter suggests the strength of the correlation (little scatter, close to line of best fit, indicates high correlation). x-axis= Test 1, y-axis= Test 2

test-retest reliability

a method for determining the reliability of a test by comparing a test taker's scores on the same test taken on separate occasions. Results the second time should be close to results the first time

Binomial

a nominal value with only TWO possible values ex: yes vs. no, candidate a vs candidate b

frequency distribution

a numerical and graphical means of illustrating variation in values of a data set

Journal

a periodical that publishes technical and scholarly material, usually in a narrowly defined area of inquiry.

societal perspective

a perspective that takes account of all costs and benefits, no matter to whom they accrue time/burden of caregiver, transportation, etc.

Experimenter Bias

a phenomenon that occurs when a researcher's expectations or preferences about the outcome of a study influence the results obtained

Negatively worded questions

a question in a survey or poll that contains negatively phrased statements, making its wording complicated or confusing and potentially weakening its construct validity - Avoid negatively worded questions when the negative does not serve a useful purpose. Try not to use double negatives. • Example: "Try not to use double negatives in your questions." • VS. "Do not fail to discard double negatives from your questions" • Goal: simplicity. Even one negative can increase complexity. Two negatives are worse. • Book example (Morling, 2017, p.158): Abortion should never be restricted.

empirical questions

a question that can be answered by making objective observations

Leading questions

a question that encourages or implies a particular answer - Ex: what did you think of that amazing concert? - avoid. use neutral phrasing to try to capture person's "true" opinion

The coding guide

a record that specifies the variables of a study, the columns they occupy in the date file, and their possible values

positive correlation

a relationship between two variables in which both variables either increase or decrease together

Curvilinear relationship

a relationship in which changes in the values of the first variable are accompanied by both increases and decreases in the values of another variable

graph

a representation of data by spatial relationships in a diagram

interaction effect

a result from a factorial design, in which the difference in the levels of one independent variable changes causing the DV to change, depending on the level of the other independent variable; a difference in differences a HYPOTHESIS (prediction) about how the levels of one independent variable combine with another independent to influence the dependent variable, in a way that extends beyond the sum of the two separate main effects.

convenience sample

a sample collected from subjects that are available

percentile

a score below which a certain percentage of the cases in a distribution fall; a percentile is a score, not a percentage

central tendency

a single number that best represents a group/condition

A-B-A design

a single-subject design in which researchers establish a baseline, introduce the intervention and measure the same variable again, then remove the intervention and take another measurement

A-B-A-B

a single-subject design in which researchers establish a baseline, introduce the intervention, remove the intervention, and then reintroduce the intervention, measuring the dependent variable each time.

A-B design

a single-subject design in which researchers takes a baseline measurement and then introduce the intervention or experimental treatment and measure the same variable again

manipulation check

a small test to make sure that a manipulation is working.

operational definition

a statement of the procedures used to define research variables

null hypothesis

a statement or idea that can be falsified, or proved wrong no difference/effect

Mixed Design Analysis of Variance (mixed design ANOVA)

a statistical analysis that tests for differences between two or more categorical independent variables, while one is a between-subjects variable and another is a within-subjects variable

meta-analysis

a statistical strategy for comparing and integrating the results from multiple studies on the same research question to summarize what they say collectively on that topic

Two-Way Analysis of Variance (ANOVA)

a statistical test that allows us to simultaneously test how two separate nominal or categorical independent variables or (factors) influence the dependent variable, and how those independent variables interact to influence the dependent variables. AKA two way ANOVA or factorial ANOVA

Edited Books

a type of book that does not consist of one comprehensive study but instead a number of studies written by various authors

box-and-whisker plots

a type of graph based on median and percentiles rather than mean and standard deviation

response sets (nondifferentiation)

a type of shortcut respondents can take when answering survey questions. Rather than reporting sincere, thoughtful, and question-specific responses, a person may develop a pattern for responding

cross-sectional study

all individuals of a sample at the same point in time

risk difference

actual risk attributed to exposure

not a widely accepted form of randomization

adaptive

period for information bias

after enrollment and group assignment

Yay-Saying (and Nay-Saying--Response Sets + Solution)

agreeing with everything Solution: Reverse-word questions

group or cluster randomization

all members of a group/cluster are randomized together

variables

all of the factors that affect the observation results in an experiment

constants

all of the other variables that could possibly affect your experiment that must be kept the same in every trial in order to keep your experiment valid

statistical population

all subjects to whom the study might apply

Open-ended questions

allow any sort of response - Example: "What is your favorite style of music?" (without any response options provided) • Allows for rich information and unanticipated responses • Requires careful analysis and effortful (time-intensive) coding of information

Common SPSS Commands: Compute Function

allows creation of new variable from previous existing variables through arithmetic operations.

Multiple Coders

allows you to assess interrupter reliability

Third Variable (z)

alternative explanation for the relationship between x and y

Drawing Histogram

always include, title and labels y-axis is always frequency

Nomothetic

an approach based on laws or one that operates according to a system of laws

waiting-list control group

an empty control group often used in clinical research; participants in this group do not receive treatment or intervention until after the completion of the study

sampling error

an error that occurs when a sample somehow does not represent the target population. (you can't get the whole population, this error happens just from taking survey etc)

Measurement error

an error that occurs when there is a difference between the information desired by the researcher and the information provided by the measurement process

A composite illustration

an example from the General Social Survey.

Independent groups design

an experimental design in which different groups of participants are exposed to different levels of the independent variable, such that each participant experiences only one level of the independent variable

Mixed design

an experimental design that combines within-subjects and between-subjects methods of data collection

Feasibility

are the results and recommendations realistic and achievable.

Negative Correlations

as A increases, B decreases OR as A decreases, B increases

double-barreled questions

ask two questions at once. - Avoid this • Example: "Wasn't that guitar riff amazing and weren't the song lyrics clever?" • If someone says "yes" what does that mean? Are they referring to the guitar solo, the song lyrics, or both? • Goal: simplicity • One thought at a time.

Program evaluation

assesses the social reforms and innovations that occur in government, education, the criminal justice system, industry, health care, and mental health institutions.

adaptive randomization

assigning patients to a treatment group based on the previous success of the treatment as the trial progresses

Explanatory research

attempts to explain or find a cause for social trends and phenomena.

plagiarism

attempts to present another person's work as your own

Mean

average sum of values divided by N of cases

standard deviation

average distance from the mean

mean

average, variance, st. deviation

Predict (goal of psych/res)

based on what we know, what will future behavior, mental processes or outcomes

closed cohort

begins and ends with set group of individuals

Deductive reasoning

begins with a theory that explains social behavior and uses observations to test the theory.

Inductive reasoning

begins with observations of social behavior and uses those observations to develop a theory that explains the behavior.

B-data

behavioral observations

intrinsic sources of variation

biologically real; true genetic and phenotypic variation among subjects

helps to minimize investigator bias

blinding

helps to minimize performance bias

blinding

Theories

cannot be proven and lead to hypothesis

power

capacity to detect a difference between study groups

crossover RCTs take longer because of _____ and _____

carryover effect, washout period

hospital-/clinic-based case-control

cases have disease, controls received care from same institution

Nominal scale of measurement

categorical data often reported as frequencies and percentage

ordinal variables

center: mode, median spread: frequency dist.

Participant variables

characteristics such as age, gender, and intelligence that vary from one individual to another

Fence sitting

choosing middle response options (possibly to avoid reporting controversial opinions. - removing neutral or middle response options may help. But, what happens when people really are neutral or ambivalent?

Pearson's r

coefficient is represented by r coefficient that represents correlation coefficient range from -1 to 1+

ECHO is used to evaluate _____

cohort studies

baseline measurement

collected prior to randomization

Reason Rationalism

coming up with solutions based on well crafted thought

Two-Tailed (Non-Directional) Hypothesis

concerned with both regions of rejection about distribution. Just wants to know that there is a difference, does not care about how much of the specifics. ex. The mean of all bags doesn't have 17 M&M's.

One-Tailed (Directional) Hypothesis

concerned with the region of rejection only one tail of the sampling distribution. Directional difference: more than/less than, better/worse grade. ex. The mean of all M&M bags is less (or greater) than 17 M&M's.

Residual confounding

confounding that remains even after many confounding variables have been controlled

design confounds

confounds caused by a problem in the design of the independent variable.

Establish Internal Validity

considering confound- alternative explanation for change in dependent variable

Reliability

consistency of measurement, are you hitting the same spot every time?

Ethical Issues in Research Analysis: p-hacking

continuing to test data and post hoc analysis to produce significant results. Altering p value found in analysis report. ex. analysis gives alpha .06 and researcher reports .05.

face validity

correct on the surface

content validity

correct to experts

criterion validity

correlates to other measures or events

demand characteristics

cues in an experiment that tell the participant what behavior is expected

active control

current standard of care

4 goals of psychological research

describe, explain, predict, control

fatigue effect

decline in performance over time as a participant gets tired or bored.

effect size

degree of difference between treatment groups that is clinically important

Ways to Handle Missing Data: Listwise Deletion

delete case (or participant) completely where missing cases found. Only participants with all answers included are left in data set. Used when power is reached/exceeded

Program Evaluation

demonstrate the value or worth of an intervention or program within a specific context. Provide basis for making judgements about the value of a program, focused on describing elements, outcomes and value of programs. Performed as a team effort.

examples of derived statistics

density, percentages, adjusted scores, rates, ratios, any summary statistic (averages, variation)

criterion variable

dependent variable

Which numbers to report?

depends on level of measurement -describe center: use mode -describe spred: use frequency distribution

summary statistic

derived numerical value that summarizes a data set (central location, stats expressing variation

Levene's Test for Equality of Variances

determines if the two conditions have about same or different amounts of variability. See Notes for How to Read SPSS Output

Attributes of Scientific inquiry

determinism (events have causes), discoverability, objectivity (answer can be found, reputable and replicable), data driven, tentative conclusions, answerable questions, falsifiable theories, scientists are creative "skeptical optimists"

Program Accountability Model

developed to be more responsive to counselor's immediate need for information on effectiveness of program, to specify focus and determine modifications needed. Cyclical pattern: - Needs assessment, strategic program planning (action plan), program implementation, program assessment, program feedback.

range

difference between the highest and lowest scores in a distribution

ordinal interaction

differences are in same direction, but differ in magnitude when one group's predicted means is always greater than another group's predicted means. For example, the predicted male means are always greater than predicted female means, yet the differences between males and females varies by SES, therefore an ordinal interaction results

conditions

different levels of an independent variable in an experiment.

Adequacy

do the results provide an accurate description of the program and show something.

Test-retest reliability

do you get similar results if you administer the test more than once?

Interrater reliability

do you get the similar results from two different observers?

intention-to-treat analysis

dropouts analyzed as if they completed the study

probability sample

equal chance for population to get into the sample, e.g simple random sample

stratified randomization

equal subgroups based on certain factors

describing spread

e.g data goes from 3-6 -frequency: % distribution -minimum/maximum -range: low subtracted from high -quartiles -variance -SD

Within-Subjects Design

each participant experiences all levels of the IV (CONDITIONS)

Random Assignment

each participant is equally likely to be assigned to any level of the independent variable - *how to avoid selection confounds*

Between-Subjects Design

each participant only experiences 1 level of the IV (GROUPS) con: requires more people to test

Question order

earlier questions can change the way respondents understand and answer later questions - Preceding questions can frame subsequent interpretations and encourage particular response patterns on later questions. -Imagine that I asked a series of questions relating to oil or natural gas scarcity and then I asked how people felt about a new drilling project. • Do you think I would obtain different results if I asked a series of questions about damage to various ecosystems, and then I asked how people felt about the new drilling project?

ECHO

economic, clinical, humanistic, outcomes

example of between groups design

effect of appetitive stimuli on choice of impulse purchases One group given stimuli then choice, one group given impulse choice first

Between Group

effect of intervention on outcome is based on differences between groups: - Post-test - Pre-test/post-test - Solomon four group - Factorial design

case-control strengths (6)

efficient for rare diseases, relatively easy, fewer subjects needed, reduced costs, may study several risk factors for one disease, can investigate chronic diseases with long latency periods

single-blind study

either the participant, or the experimenter doesn't know which condition participants are assigned to.

missing data

empty cells in a data matrix

comparison group

enables us to compare what would happen both with and without the thing we are interested in

A study with two (or more) independent variables Each IV has at least two levels. IVs are crossed with each other, creating all possible combinations of the levels. IVs can be participant variables (e.g. gender) or manipulated variables. IVs can be within-groups variables or independent groups variables.

factorial design

Confounding variable

factors other than the independent variable that may cause a result.

exclusion criteria

factors that could confound or impair a study's results, safety factors

Type 2 error

fail to reject null hypothesis when it is false (false negative)

type 2 error

failing to reject a false null hypothesis

Type 2 error

failing to reject the null hypothesis when the null hypothesis is actually false this can be calculated by taking: 100%-power of test

bar graph

graphical representation of categorical data in which the heights of separated bars, or columns, show the various relationships between variable

frequency distribution

graphs that shows how many scores falls into particular bins, or divisions of the variable

historical control

group observed at different time or setting

Multiple Regression Analyses

help reduce the third-variable problem

we want the NNH to be as _____ as possible

high

Describe (goal of psych/res)

how do people think, feel and act in various situations or on average

z score

how far something is from mean (observation-mean)/SD *conditions for using Z score: -can only use for nominal dist, or approximately normal -have the mean and SD -possible values of Z: 0(the average), positive infinity, negative infinity z score outside of -3 to 3 is unusual

Variance

how spread out the scores are around the mean

Using a within-groups design (top), or a matched-pairs design (bottom) can help. A larger sample also helps.

how to account for individual differences:

sampling frame

how to go from the population to sample, e.g passing surveys out at the student center -the method is the survey *the sampling frame matters Results are influenced by: -who you ask: sampling frame -how you ask: phone v. online, type of questions

Ethical Issues in Research Analysis: HARKing

hypothesizing after results are known. Statistical analysis abuses often presented (ex. using a one-tailed rather than a two-tailed), type I errors more common.

when are reversal designs appropriate?

if the benefits of treatment aren't expected to last

Leading Questions (2 sources of question bias)

imply a "right answer" Solution: Use neutral wording

practice effects

improvement on a task over time due to practice.

error bars

in a graph, vertical lines that indicate plus or minus one standard deviation of the data or, less frequently, the standard error of the mean

Independent variable: bowl size Conditions: medium and large.

in the eating pasta experiment, what were the independent and condition variables?

a quasi-experiment with repeated measures.

interrupted time-series design Example: Does a food break result in judges making more favorable parole decisions?

6 ways of knowing about the world

intuition, tenacity/tradition, authority, reason/rationalism, empiricism, science

Assessment Confound (design confound)

involves *assessments that are different for one level* of an independent variable versus another Procedure 1: take an AP chem test Procedure 2: recall anything remembered

Material Confound (design confound)

involves *materials that are different for one level* of an independent variable versus another Procedure 1: read textbook chapter Procedure 2: watch a video lecture

Procedure Confound (design confound)

involves *participants doing things differently in one level* of an independent variable versus another Procedure 1: study for 30 min Procedure 2: study for 1 hr

Exploratory research

is often necessary in order to learn about social trends and phenomena that we know very little about.

prevalent cases

known existing disease

Range

largest value - smallest value

consent

legally given permission to participate in a study

advantages of retrospective cohort studies (2)

less time, less expense

L-data

life outcomes and records

Forced-choice formats

limit type of responses • Requires selecting from a pre-specified set of responses Example: • Narcissistic Personality Inventory (Raskin & Terry, 1988): Two opposing statements are provided and the respondent is asked to choose one. _____I really like to be the center of attention. _____It makes me uncomfortable to be the center of attention.

partial counterbalancing

limited amount of different orders.

t-test (t)

looks at differences between two groups on some variable of interest. The independent variable is categorical (male/female, undergrad/grad) and the dependent is continuous. See Notes for How to Read SPSS Output

we want the NNT to be as _____ as possible

low

the overall effect of one independent variable on the dependent variable.

main effect

flashbulb memories

major events - vivid but no more accurate - likewise, confidence is poorly calibrated with accuracy

hypothesis testing

make and test an educated guess about a problem/solution

independent variable

manipulate or shift controlled by experimenter Must have two or more "levels" or conditions

Straightforward manipulation

manipulating an independent variable with relative simplicity by presenting written, verbal, or visual material to the participants.

Experiments can support causal claims

manipulating the causal variable means potentially controlling for alternative explanations - use random assignment to ensure that the different groups are as similar as possible

Identify main effects from _______ _____ (compute the average for each level of the IV of interest, while averaging over the levels of the other IV).

marginal means

minimize confounding

match cases with controls

construct validity

matches up with theoretical constructs

comparison group

maturation threat isn't a problem if you have:

if left skewed...

mean < median

If right skewed..

mean > median

In a Normal Distribution....

mean, median, and mode are all the same

measure of central tendencies

mean, median, mode

EXCEL: HOW TO CALCULATE MEAN, MEDIAN, SD

mean- =average( highlight column ) enter median- =median( highlight ) enter SD- stdev( highlight ) enter

Post-test only

measure dependent variable after the manipulation

Concurrent validity

measure is administered and behavior is measured at the same time. the degree to which the measures gathered from one tool agree with the measures gathered from other assessment techniques

Predictive validity

measure is administered weeks, months, or years before behavior is measured predictive validity is the extent to which a score on a scale or test predicts scores on some criterion measure. Do the results accurately predict a future outcome?

biomarkers

measure of disease progression

semi-interquartile range

measure of variability defined as half the interquartile range

dependent variable

measured, result outcome variable take the form of records of behavior or attitudes, such as self-reports, behavioral observations, or physiological measures

continuous variable

measurements made on a scale that can be infinitely subdivided

data

measurements or statistics

effectiveness

measures how well an intervention performs in a clinical setting (real world)

physical trace observation

method examines evidence left behind ex: foot prints, fossils, going through people's trash

archival research

method of research using past records or data sets to answer various research questions, or to search for interesting patterns or relationships ex: prostitution in Gutaemala

margin of error

more common to see in reports -already includes confidence level using critical value -only have to subtract/add once and that's your range

mode

most frequently occurring score

Mixed Method Design Type: Exploratory

multiphase (often two) design where qualitative results are obtained first followed by quantitative. Used when: no measurements or instruments available to guide the study, variables are unknown or study has no guiding framework. Begins with qualitative to develop rich detail, develop theoretical framework or identification of variables, then quantitative used to determine association of variables within the study.

Mixed Method Design Type: Explanatory

multiphase (usually two) design in which qualitative data helps to explain initial quantitative data. Quantitative data are collected and analyzed prior to the collection and analysis of qualitative data. The quantitative phase allows the researcher to discover variables relationship with one another, the data is then used to inform/develop questions for qualitative piece.

similar to stable-baseline, but with multiple interventions staggered across time.

multiple-baseline design Example: Interventions throughout the school year to reduce several problem behaviors in a developmentally disabled student.

controls

must represent general population

Measured Variables

naturally occurring, observed, and collected. You cannot (or choose not to) manipulate.

confidence statement

need: -sample proportion/mean -range of possible values in population -confidence level

Making estimates for sample

need: -sample stats (mean or proportion) -standard error - based on SD but not the same -estimate of certainty

double-blind study

neither the participant nor the experimenter knows which condition participants are assigned to.

non-inferiority trial

new treatment not worse than active control

incident cases

newly diagnosed

OR = 1

no association

open label

no blinding

Null Hypothesis (H0)

no difference expected (variable x has no affect on variable y)

OR of 1.0

no difference in odds

Preexperimental Design

no random assignment and no experiment group. All participants are assigned the intervention.

unsystematic variability

noise- random fluctuation in data, that can't pinpoint a cause

scales of measurement

nominal, ordinal, interval, ratio

value in active control studies

non-maleficence

helps to minimize detection bias

non-study personnel assesses outcomes

quasi experiment combining an independent-groups variable with a repeated-measures variable.

nonequivalent control group interrupted time-series design example: tv access and larceny rate over time

If confidence interval crosses 1, what happens to the odds ratio?

nonsignificant

t-test

normal distribution interval level data, two independent groups ex: male vs female, health plan A vs health plan B result affected not only by size of the mean differences but also by the variability of each group, highly affected by outliers

case-control weaknesses (6)

not representative of whole population, incidence/prevalence unknown, inefficient when exposure is rare, selection bias, only one outcome at a time, information bias

lab notebook

notebook for recording information important to a study, such as design, procedure, and the planned analysis

NNT definition

number of patients needed to be treated for one to benefit

*What are the manuscript settings used in APA?*

o *Double-spaced* o One-inch margins o Left-aligned without hyphenation o Indent the first line of each paragraph o *Typeface: 12-point Times New Roman*

Empirical

o Answers are obtained by making structured or systematic observations o Results will provide clear support for or will clearly refute the hypothesis

Casual Observation

o Behaviour of people or animals may generate a general research topic o Example: Going to the mall and observing people's behaviour

What are the guidelines for avoiding biased language in APA writing?

o Describe people with a level of specificity that is accurate o Be sensitive to labels: call people what they prefer to be called o Acknowledge people's participation in your study

How do you know that your literature search is complete?

o Feel comfortable with your knowledge about the topic area o Have found a few recent research studies that are particularly relevant to your own interests

What are the main jobs you must complete and consider when conducting a literature review?

o Finding, reading, summarizing published research relevant to your question o Literature can help you form your research question, or determine if it has been solved o Help evaluate interestingness of research idea and how it fits into current literature o Help you with ideas on how to conduct your study

The Stanford Prison Experiment

o How being placed in different roles affected by individuals acted o Participants were either given the role of prisoner or guard o Each set of participants were given corresponding outfits and the guards were told to do whatever they had to do in order to maintain order o Study was called off after 6 days as the conditions increased to highly o Prisoners were found to become submissive and guards were found to become aggressive and abusive

Laboratory Observation

observe and record behavior in a controlled environment - More control over who you observe and what/how often conditions

Naturalistic Observation

observe and record behavior in its natural environment without intrusion - Get to see behavior "in the wild". - Less likely to change behavior.

participant observation

observer becomes part of the group being observed ex: Doomsday call

Little Albert

o John Watson and Classical Conditioning o Pairing an unconditioned stimulus with a conditioned stimulus to elicit a conditioned response o Albert was conditioned to fear fuzzy white rat o Albert was never unconditioned o It is unknown if Albert's mother gave informed consent

Name some well known conferences

o OBACPN(Ontario Behaviour Analytic Community of Practice Network, divided into regions) o ONTABA (Ontario Association of Behaviour Analysis) o OADD (Ontario Association of Developmental Disabilities) o ABAI (Applied Behaviour Analysis International) o APBA (Associative Professional Behaviour Analysis) o Persuasive Technology

Public

o Observations are available for evaluation by others

Objective

o Outcome is not skewed by bias

Pass 2

o Pay closer attention to the beginning and ending of each major section o Look up your highlighted words from the previous pass o Try to answer any questions you may have written down

The Bystander Effect

o Placed college students alone in rooms and gave them headphones o Subjects told they would be communicating with other students via intercom o Other students were actually recordings, one of which mentioned early on in conversation that they had occasional seizures o Later on that voice began to have trouble speaking and asked for help o Researchers measured how long it took the participants to obtain help o It was found that the more people the participant believed was on the call, the longer it took for them to get help

Pass 1

o Read through the framework of the paper This includes the abstract, introduction, conclusion, section headings, and any tables of diagrams including their captions o Underline any words or phrases that are unfamiliar to you o Ask yourself questions What is the point or thesis of this paper? What are the main arguments of the paper? Why is this paper important? How does it contribute to my field of study? o Write down any other questions you may have o Try to summarize the paper in a sentence of two in your own words

Milgram's Obedience Study (1963)

o Sought to discover how far subjects would go in order to obey an authority figure o Participants felt a sense of responsibility that they had killed someone o This study is an example of one which would not pass in ethics today

Practical Problems or Questions

o Stemming from daily life (e.g., placement of audio controls for less distracted driving) o Example: Legalization of marijuana and the effects on impaired driving

The Monster Study

o Studied how positive and negative feedback affected the way children learned language o Feedback was provided to children from an orphanage regarding speech stutters o Children in group 1 were told they did not have a speech impairment and were told they should ignore anyone who criticized the way they spoke o Children in group 2 were told they did have a speech impairment and were told they should never speak unless they could do it right o Feedback impacted the self-esteem of the children

Milgram Experiment

o Studied the psychology of obedience and how subjects would react if researchers encouraged them to engage in activity that went against their morals o Three roles: The experimenter, The teacher, and The learner o Teacher and learner were in separate rooms o Teacher gave the learner word pairings and if they responded with an incorrect answer the teacher was instructed by the experimenter to deliver an electric shock to the learner who was acting and not actually receiving the shocks o It was found that when instructed the majority of participants delivered the highest shock possible and when the learner pretended to have passed out the teachers believed that they may have killed the learner

Methods Subsection

o Subjects subsection o Procedure subsection o Apparatus subsection o Materials subsection

Pass 3

o Tie it all together so that you have a good idea of what the paper is trying to say at this point o With a critical eye this is the time for reflection and analysis o Take notes as you read looking at arguments, evidence, and conclusions o Ask yourself questions Did the authors do what they set out to do? Are the methods they used sound? Are their methods fluid and logical? What assumptions did they make?

In which order should the manuscript elements appear in APA?

o Title page o Abstract o Text (body) o References o Tables o Figures o Appendices (if any)

What are three things that an APA style report describes?

o Why the research was done o How the research was done o The results it produced

Reductionism

occurs when a researcher makes a prediction about how a group might behave based on data collected at the individual level of analysis.

Ecological fallacy

occurs when a researcher makes a prediction about how an individual might behave based on data collected at the group level of analysis.

systematic error

occurs when sample statistics are routinely unrepresentative of population parameters and is usually due to a faulty sampling procedure

SOURCES OF ERROR: measurement error

occurs when the measurement we obtain is not accurate portrayal of what we tried to measure. asked wrong question, collected wrong information

used in retrospective studies

odds ratio

Positive predictive value

of cases who test positive, the percentage who are actual positives People who actually have the disease

outliers

one extreme #, pulls up average, but not good indicator of avg

Factorial mixed

one independent variable is manipulated as independent-groups and the other is manipulated as within-groups - example: study wants 50 participants in each cell, meaning 100 participants are needed in total

interaction

one independent variable's influence on the dependent variable changes, depending on the level of other of the other independent variable

Mixed Method Design Type: Embedded

one type of data is in a lead role with other data supporting. Single phase. Used when researchers need to answer questions from a different perspective and their research has primarily focused on one type of data. Often insert a qualitative component into a quantitative design. ex. survey with open-ended questions.

experiment

one variable is manipulated (indep) one variable is measured (dep)

Online Devices

online surveys have always been associated with desktop computers but the expansion of devices such as smart phones or tablets add an additional consideration for design.

per-protocol analysis

only those who completed the study exactly are analyzed

rate ratios are used in _____

open cohorts

difference between opened ended questions and restricted questions

open ended questions people can answer in their own words while restricted questions answers are limited

Interval vs. Ratio scale

ration = observable interval = unobservable qualities

Ordinal (scale of measurement)

refers to categories that are ordered but not necessarily equidistant: very, somewhat, not at all strongly agree, agree somewhat, disagree somewhat, disagree strongly

Interval (scale of measurement)

refers to equidistant categories number of days, age in years

Nominal (scale of measurement)

refers to non-ordered categories (no order of importance. examples: male/female, racial group)

sampling variation

refers to normal variation (dispersion) in statistics measured on repeated samples from a population

sample bias

refers to systematic error in measurements

Incidence

refers to the number (or rate) of new cases of a disease/condition in a population during a given period

The principle that extreme observations tend to be followed by less extreme observations.

regression to the mean

Type 1 error and cumulative type 1 error

rejecting the null hypothesis when the null hypothesis is actually true false-positive the probability of this happening is = to alpha

used in prospective studies

relative risk

Data-Driven Needs Assessment

relies primarily on existing data as a means for objectively identifying needs of client populations.

empirical science

relying, or based on, direct observation, experiment of experience

sample bias

samples that have been poorly collected or assembled; measured samples could systematically misrepresent a population parameter even if they are precise

Purposeful Sampling

sampling specifically chosen population based on certain element that will highlight and inform a particular aspect of the study. Gives you useful data when you use this method of sampling for mixed methods.

How do you evaluate reliability?

scatterplots and correlations

discriminant validity

scores on the measure are not related to other measures that are theoretically different. Should correlate less strongly with measures of different constructs

Convergent validity

scores on the measure are related to other measures of the same construct. Should correlate more strongly with measures of the same/similar constructs

Mixed Methods: Multiphase

second part of the study depends on and elaborates the first. Two approvals needed from IRB. ex. interview, create theory from interview, generate survey to test theory.

Correlational Research (what makes it correlational)

seeks to figure out if two or more variables are related and, if so, in what way *both variables are measured*

factors that threaten internal validity (7)

selection, history, maturation, mortality/attrition, testing, instrumentation, statistical regression

Cronbach's alpha coefficient

splits the data all sorts of way then finds the average --> that is your correlation coefficient

2-way interactions: what they look like ("a difference in differences")

spreading interaction, crossing interaction

a study where the researcher observes behavior for an extended period before and after a treatment.

stable-baseline design Example: evaluating the effectiveness of a memory improvement strategy in an Alzheimer's patient.

research protocol

standardized document that covers all aspects of a study

Standard deviation (what it is, and how variability in the data affect it, but not how to calculate it)

standardized measure of variability, calculated based on square distances from points to to the mean

correlational coefficient

statistic that indicates the strength of the relationship between two variables

Correlation

statistical technique that measures degree of relationship between two or more variables (no manipulation of variables).

inferential statistics

statistics that help us to draw conclusions about populations

descriptive statistics

statistics that summarize a set of data

observable confounders (2)

stratification, statistical adjustment

helps to minimize selection bias

strict adherence to randomization schedule

case study

study of one individual in great detail

informed consent requirements (7)

study purpose and expected benefits, methods to be employed, anticipated inconveniences and discomforts, potential temporary and permanent risks, contingent compensation, all reasonable alternatives, right to withdraw

what is evaluated in RCTs? (7)

study sample, randomization and blinding, intervention and control group, clinical endpoints, trial findings, limitations and implications, application to clinical practice

informed consent

subject is presented with information about the study and willingly volunteers to participate

crossover RCT

subject receives treatment and control at different times

triple blinding

subjects, investigators, and those collecting data do not know randomization schedule

sample

subset of people we will collect data on

sample

subset of population of interest

sum of squares

sum of squared deviations from the mean

Cohort Studies

survey representatives of a group of individuals over time to follow a group's progress. Sampling of each, same group over time. Ex. Research Class

bias

systematic deviation of a study's result from the truth

Counterbalancing

systematically ordering treatments or measurements within a protocol to eliminate effect of order in which they are presented to participants. Each group switch out order to rotate the intervention. Intervention.

advantages of prospective cohort studies (4)

tailoring of study to research questions, purposefully capture data on important risk factors, using reliable and validated techniques to assess exposure and outcome measures, measuring exposure before disease occurs

non sampling error

target off bc/ of how sample was chosen. e.g being off because you did an email survey

good subject tendency

tendency of experimental participants to act according to what they think the experimenter wants

Speciesism

term used by analogy with racism and sexism by those who claim that it is unethical to treat animals differently from humans, particularly in research

What are the two types of reliability

test-retest and interrater

ANOVA (F)

tests the significance of group differences between two or more groups. The independent variable has two or more categories, the dependent is continuous. only determines if there is a difference between groups, not which is different. Ex. Low-, middle-, and high-income residents. See Notes for How to Read SPSS Output

cell mean

the average score of the participants in a single cell or a single group the mean on the dependent variable for participants with a specific combination of the levels of the independent variables

Bias Blindspot

the belief that we are unlikely to fall prey to cognitive biases Being biased about being biased

Decision analytic model

the calculation of expected outcomes of an intervention or treatment based on a set of input assumptions, usually performed using specialized software and techniques, such as Markov chain modeling or decision tree analysis

mean

the common average

face validity

the degree to which a scale appears, on the surface, to measure the intended variable - should be obvious enough to make sense but not too obvious that it introduces demand characteristics - ask yourself: does it appear to test what it claims?

Interrater reliability

the degree to which different observers agree on their observations. Two or more "independent" observers, looking at the exact same thing

Range

the difference between the highest and lowest scores in a distribution

Criterion Validity

the extent to which a measure is related to an outcome Does the measure allow you to distinguish people on the basis of a behavioral outcome? Two types: -Concurrent validity -Predictive validity

Construct Validity

the extent to which variables measure what they are supposed to measure

construct validity

the extent to which variables measure what they are supposed to measure

Construct validity

the extent to which variables measure what they are supposed to measure Is the measure ACTUALLY measuring what the researcher intends to be measuring?

relative frequency

the fraction or percent of the time that an event occurs in an experiment (frequency/total number of measurements)

animal welfare

the generally accepted term for concerns about the care and use of animals

Hazard ratio

the hazard of an event (e.g death) for one group relative to another

alternative hypothesis

the hypothesis that a proposed result is true for the population researcher predicts its going to happened

confounding

unaccounted for factor associated with both exposure and outcome

EXAMPLE OF VARIABLES: NBA DATASET

unit of analysis: NBA teams -team: nominal -total attendance: ratio (0 people has value) -did they make playoffs?: nominal -place in conference: ordinal, ranking

period for selection bias

up to and including enrollment and group assignment

Inferential Statistics

use samples to make informed guesses about the characteristics of the population from which the sample is drawn. Estimate likelihood or probabilities (p) that represent a true situation.

Common SPSS Commands: Recode Variable

used for collapsing or merging variables and for reverse coding.

interim analysis

used in cases of safety or ethical concerns

Predictor Variable

used to estimate the criterion (independent variable)

inferential statistics

used to inductively infer from known data (sample data) to the probably values of incompletely known quantitative properties; used to quantify degree of uncertainty around these inferences

descriptive statistics

used to quantitatively describe and summarize known data such as data obtained from samples

Measure of central tendency

used to represent a "typical" result or value around which results cluster

Trend Studies

used when interested in understanding how a variable of interest changes over time within some general population. Same general population, different people, same questions over time. Ex. surveying different individuals over multiple years from the same population, 5th grade students or incoming clients

Common SPSS Commands: Select Cases

useful when wanting to include only a portion or subset of data (ex. 1st year counseling student population specifically).

Improving response rates

—with rates comparable to mail surveys, online surveys benefit due to their lower cost. Research is still ongoing as to what is best to increase response rates, but email reminders and setting deadlines has shown promise.

Saying more than we can know

• Nisbett & Wilson (1977): People will report opinions and the reasons on which those opinions are based, even though they're effectively making it up as they go along. • For identical nylon stockings: people generally preferred the right- most one, and claimed their preference was based on quality. - Empirically, there may be many forces influencing our behavior. However, we may often be unaware of them. - When asked, we attribute reasons to our actions. Those inferred reasons might be biased or simply wrong.

Field Research Concerns

◦ Selecting a site and gaining access to it, ◦ Entering the field, ◦ Learning the ropes, and ◦ Developing rapport with members in the field.