Research Methods
repeated measures ANOVA
- a statistic used to test a hypothesis from a within-subjects design with three or more conditions
simple random sampling
- probability - each member of population as equal change of being selected for the same - selection is independent of one another - each
general sampling methods
- probability - non-probability
cluster sampling
- probability - used when naturally occurring pockets of population are geographically dispersed - randomly select from pockets
Predictor Variable
What factors may be related to the behavior of interest? (independent v) examples: Recess time, Public v private school, proportion of boys in the class
Sampling Non-human elements
What helps *maximize generalization* by looking at *surrounding* elements?
Increase effect
What increases power by *upping* the intensity of tx or the *independent variable*?
Correlational
What involves *measuring relationship* between different variables whether *direct* or *indirect*?
File drawer problem
What is another name for *publication bias* which is where only the significant results are more likely to be published?
Research hypothesis
What is another name for an *alternative hypothesis (HA)* in which you predict that a *relationship exists*?
Less confounds
What is the *strength* of a *lab* based experiment?
major types of bias (5)
investigator, selection, performance, attrition, detection
median
middle score
A factorial design with at least one independent-group factor, and at least one within-groups factor.
mixed design
A variable which changes the relationship between an independent variable and a dependent variable.
moderator
Chi-square
nominal or ordinal data addresses whether independent variables differ in their relative distributions on the dependent variable ex: whether the outcome is contingent on group membership
Mode is used when:
nominal scales and discrete variables
4 scales of measurement
nominal, ordinal, interval, ratio
Mortality
number of deaths
NNH definition
number of patients needed to be treated for one to be harmed
prospective timeline
present to future
zero correlation
no relationship between variables
internal validity
the degree to which the effects observed in an experiment are due to the intervention
Main effect
the effect each variable has by itself.
all professional activities of psychologists.
the ethics code covers _________.
uncertainty
the exact value of any biological property is difficult to know for certain
Skewed distribution
the extent to which a distribution leans to one side of the mean instead of being evenly distributed based on where tail is
True Experimental Design
utilizes both random assignment and comparison group
interaction effect
whether the effect of the original independent variable depends on the level of another independent variable
explain (goal of psych/res)
why did this behavior occur? which factors influenced this outcome
If it would cause them harm to know
why would you never debrief the participant?
assent
willingness to participate in a study
tenacity
wisdom from previous generations, "American dream"
Negative Correlation
x and y changes in *opposite* direction - more you exercise and less you weigh, more income and less likely to get diseases
Positive Correlation
x and y changes in same direction - more shy you are, the better you are at reading facial expressions
systematic sampling
- probability sampling - every "nth" subject on sampling frame is selected - starting point is chosen via random number table/generator
naturalistic observation
-subjects watched in a natural setting -disturbed as little as possible ex: Goodall watching chimps in Africa for 50 years
experimental manipulation
-true experiment
Number of variables
-univariate: 1 variable at a time -bivariate: more than 1
descriptive stats
-what should we describe? -center: what is common -spread: how far apart data is spread
p value cutoff
.05
Weak (correlation strength)
.10/-.10
Strong (correlation strength)
.50/-.50
What is the minimal number of members on a research ethics board (REB)?
5
Ethics in Research
5 General ethical Principles: 1. Beneficence & nonmaleficence 2. Fidelity & responsibility 3. Integrity 4. Justice 5. Respect for rights & responsibilities
Empiricism
5 senses, taking in information through observation
Marginal means are useful for seeing Simple main effects Two-way interactions Three-way interactions
A
Conceptual replication
A researcher has the same research question as another study but uses different procedures. This improves external validity.
Single-N design
A study in which researchers obtain a large amount of information from a single animal or person.
OR of 3.0
200% increase in odds
OR of 0.75
25% reduction of odds
Factorials displays
2X2 chart, graph
Negative skew
What type of frequency distribution is skewed to the *right*?
parallel RCT
subject randomized to treatment or control only
double blinding
subjects and investigators do not know randomization schedule
block randomization
subjects divided into blocks which are then randomized
single blinding
subjects do not know treatment group
laboratory observation
subjects watched in the laboratory ex: bobo doll experiment
Internal Validity
"Are there confounding variables?"
External Validity
"Can we generalize the results to the population?"
Face Validity
"Does it *look* like they are measuring what they think they are measuring?"
Statistical Validity
"What do the mathematical results tell me?"
Correlations
"r". Tells us direction, positive/negative, ranges from -1 to +1. the closer to -1 or +1 the stronger the correlation. The closer to 0 the weaker the correlation.
The Rational Method
- An example of seeking answers by logical reasoning - In this method the first two sentences are premise statements while the last is a logical conclusion based on the premises
Which critical evaluation question should we ask ourselves when evaluating lists of items cited?
- Is the references list current and complete?
Environmental Changes
- It is difficult to attain the ideal of identical circumstances - Example: Raining and students have an indoor recess
The Empirical Method (Empiricism)
- Answering questions by direct observation or personal experience - Based on the philosophy that all knowledge is acquired through senses
What are three steps to getting approved for human subjects?
- Application to REB - Research Proposal - Consent Forms
What are the 7 dimensions of ABA? *Remember GET A CAB*
- Applied - Behavioural - Analytic - Technological - Conceptual Systems - Effective - Generality
What are the three important principles to the scientific method?
- It is empirical - It is public - It is objective
ERIC Database
(Education Resource Information Center) provides access to education literature and resources
Institutional Review Board;5
(IRB) stands for ____________. They must have a minimum of _____ members.
Pro/Con of Close Ended
*Pro:* No coding required. May provide answers that they wouldn't have recalled. *Con:* May not include all options. Answers may be influenced by response options given (framing effect).
Pro/Con of Open Ended
*Pro:* Rich Info *Con:* Usually require coding, length/detail is not always clear, performance on recall worse than recognition for memory question
Every observation and survey will always have some degree of error T/F
*TRUE*
Experience has no comparison group; experience is confounded T/F
*TRUE*
Correlation & Strength of Association
+/- .10 - .29 = Weak +/- .30 - .49 = Moderate +/- .50 - .69 = Strong +/- .70 and above = Very Strong
independent groups
(aka between-subjects): Different groups of participants take part in different conditions.
dependent groups
(aka within-subjects) Only one group of participants. Each participant takes part in every condition.
What is the minimal number of replications needed?
3
Method of Faith
- Is a variation of method of authority - People have unquestioning faith in authority figure - Accept without questioning, challenge, or doubt
*Introduction*
- *A general introduction to the topic of the paper* - *A review of the relevant literature* - *A statement of the problem or purpose of the study with relevant variables clearly defined* - *A description of the research strategy that used to evaluate the hypothesis or to obtain an answer to the research question*
*When in APA reports are numbers used?*
- *Quantities above 10* - *Date* - *Age* - *Measurements* - *Units of time*
*Methods*
- A relatively detailed description of exactly how the variables were defined and measured and how the research study was conducted - *Participants, Materials, Process*
Three types of construct validity
1. Convergent 2. Discriminant 3. Criterion-related
OR of 1.5
50% increase in odds
typical power level
80%
Field settings
A real-world setting for a research study.
Who is the SLC research officer?
Anthony Wright
Parsimony
As succinct as possible
Respect of Persons
Autonomy, Informed Ongoing Consent
Introduction
Background, research question(s)
Beneficence/Concern for Welfare
Benefits outweigh the risks, unnecessary risks, confidentiality, informed
observer effects
By using a masked design, the researchers in the "Seeing Red" study minimized the risk of: Measurement error Observer effects Reactivity Confounds
4. A 3 × 3 × 2 factorial design includes how many levels in the second independent variable? A) 1 B) 2 C) 3 D) 4
C - 3
7. A(n) _____ is an experimental design that combines within-subjects and between-subjects methods of data collection. A) A-B design B) A-B-A design C) mixed design D) single-subjects design
C - mixed design
Mode of Delivery
Can affect responses to questions that may be affected by social desirability (e.g., online surveys individuals may be less swayed to give "desirable" answers
Temporal Precedence (3 criteria for causal claims)
Can you establish that x came before y?
What organization is in charge of animal care in Canada?
Canadian Council on Animal Care
Experimental Design
Cause & effect quantitative design
Causal Relationship
Cause and effect
Nonlinear Relationship
Change in one entity does not correspond to change in the other entity
Variables
Characteristics or conditions that change or have different values for different individuals
Splitplot
Combination of easily manipulated and difficult (impossible; ex. race, gender) to manipulate independent variables are both assessed for effect on dependent.
Benefit of Mixed design
Combines between-subjects and within-subjects designs. Strengths of each design offset the weaknesses of the other.
Active Deception
Commission
Autocorrelation
Correlation between *1 variable with itself*, measured at *two different time points*
Demand characteristics
Cues that inform the subject how he or she is expected to behave.
Review Article
Summary of published studies done in a research area (e.g., a meta-analysis)
Covariance Temporal precedence Internal validity
What are the three criteria for a causal claim?
ordinal scale of measurement
Data are assigned to categories that can be ranked.
Objective-Oriented
Determine extent to which program objectives have been achieved. Ex. Midterm & Final - identify discrepancies between stated objectives & desired outcomes
True
Ethics & research: The principle of doing no harm means that researchers should never subject research participants to physical danger, cause them psychological harm, or expose them to legal liability.T/F
causal claims
Experiments are conducted to make ________
Internal Validity
Extent to which independent variable truly effect changes in dependent.
Confirmation Bias
Focusing on the evidence that fits our beliefs a tendency to search for information that confirms one's preconceptions
marginal means
The arithmetic means for each level of an independent variable, averaging over the levels of another independent variable.
Varieties of Psychological Research
From observations, and serendipitous events
Participant-Oriented
Gain insight through the participants in the program. What do they think/need of the program? Different perspective than those who are implementing the program or funding it.
Empirical Approach
Gathering information using your senses. Relying on systematic observations! Unbiased Objective Publicly verifiable events
Covert Observer
In field research, refers to a researcher who is secretly studying a group without the group members knowing they are being studied (neo-nazi group)
Overt Observer
In field research, refers to a researcher who is studying the group members with their full knowledge
experimental design
Good ________________________ is important to doing successful research.
Healthy adherer effect
Greater adherence to health-promoting behaviours, such as medication adherence, is indicative of overall healthy behaviour
Continuous Variable
Have a large number of levels. -More than 10 levels Examples: -response time, cigarettes per day
variance
The average of the squared differences from the mean.
Placebos
Inactive substances, not real medications, that are used to test the effectiveness of drugs.
Method
How did they conduct the study?
Between Group difference
How do you know how much *difference* is expected *due to chance* between groups?
IRB stands for
Institutional Review Board
Null Hypothesis
Hypothesis where there is no significant difference between specified populations, always based on data already established. ex. Each bag has 17 M&M's.
FDA has to approve all protocols for _____
INDs
example of experiment
Indep: treatment -> level: 1- cognitive behavioral therapy 2- Family Therapy 3- wait list Dep: scores on Beck depression scale
Methods of Intuition
Information that is accepted on the basis of a hunch
Provide two examples of constructs
Intelligence and Hunger
Basic Research
Intended to solve theoretical issues
when the effect of one IV depends on the level of another IV.
Interaction effect
Questionnaire instructions
It is important that the directions for completing the questionnaire are very clear.
Internal validity
The certainty with which results of an experiment can be attributed to the manipulation of the independent variable rather than to some other, confounding variable.
Sample
The collection of subjects selected for observation in an empirical study.
Item-total correlation
The correlation between scores on individual items with the total score on all items of a measure.
Population
The defined group of individuals from which a sample is drawn.
Measurement error
The degree to which a measurement deviates from the true score value.
Reliability
The degree to which a measure is consistent.
Scatterplot Direction
Linear relationships can have a positive or negative direction
Nominal
Looking at categories (sex, species, race)
Limitations of Tenacity
Lots of potential inaccuracies and no method for correcting erroneous ideas
Give two examples of intangible and abstract attributes to operationally define
Motivation and Self-Esteem
- Effect Size - Sample Size
What are the two components of a significance test that when increased are the most effective way to *increase power*?
Parametric Statistic
Make assumptions about population fitting normal curve.
Nonparametric Statistic
Make no assumptions about the population fitting normal distribution.
Type 1 error
Making a claim of association or cause when there isn't one Rejecting the null when it is true
Type 2 error
Making a claim of no association or cause when there is one Not rejecting the null when it is false
Measurement error
The degree to which the measurement, used to assess a participant in a study, is different from the true measurement value. When these are random they are capable of inflating or deflating the true scores. But if they are systematic they may result in a biased measurement.
What two things must you be able to do with variables that are being studied?
Measure and Define them
What does the REB take into consideration?
Mitigation of risks, benefits outweigh risks, fair, informed consent
Type B
More relaxed and easy-going personality
Forced Choice Question
Multiple Choice, True/False, Yes/No questions are examples of...
number needed to harm
NNH = 1/attributable risk
Type II Error
Not claiming a result as significant when you should have. Fail to reject the null hypothesis when there is a difference. Missed a finding that is actually there. False negative. ex. Telling a 9 month pregnant woman that she is not pregnant.
Limitations of the Empirical Method
Observations can be misinterpreted, sensory experience can be swayed by beliefs, time consuming, can be dangerous relying on one person's opinion
Association Claims
One level of a variable is likely to be associated with a particular level of another variable. Also known as correlations or covariations
When selecting a mentor in the field, what is it important to consider?
One should be in the field for years prior to being considered a mentor
Causal Claims
One variable is responsible for changing another variable
Criterion Variable
Outcome variable being measured (dependent variable)
Psychology
The scientific study of behaviour and mental processes. It provides detailed and accurate knowledge about psychological problems and establishes whether treatments are effective.
Reduce Social Desirability Bias
Phrasing questions in ways that make norm violations appear less objectionable and that present a wider range of behavior as acceptable (e.g., face-saving alternatives)
Journalism
Popular press for general public
Exposure therapy is often used for what psychological disorder?
Post Traumatic Stress Disorder
Debriefing
Post-experimental explanation of the study to the participant (especially if deception is used)
Discriminant validity
The extent to which scores on the measure are not related to scores on conceptually unrelated measures. I.e. scores on a test designed to assess aggressiveness should not be positively correlated with scores from tests designed to assess intelligence.
Interestingness
The extent to which the answer is in doubt, whether it fills a gap in the research literature, and whether it has important practical implications
increasing the sample size using stronger manipulations eliminating unwanted sources of variance (noise)
Power is increased by:
A
Practice effects and carryover effects are examples of _________ effects. order scientific between-person causal
Basic Research
Primary goal is to gain knowledge
What are the pros and cons of journals and journalism?
Pros: easy to read, more likely to influence lives Cons: can be warped and misinterpreted
a study where the levels of the independent variable are naturally occurring, rather than manipulated by the researcher.
Quasi-experiment
What other REBs is it possible for SLC students to need to go through?
Queens/Various Hospitals
strongest type of study design in clinical research
RCT
Ordinal
Rank order, but not equally spaced Categorical data Frequency data and Spearman rho Nonparametric measures
Observer bias
Rather than rating behaviors objectively, researchers might (unknowingly) rate behaviors according to their expectations or hypotheses. - Ex: All participants saw the same taping of a football game between Dartmouth and Princeton. However, the number of observed rule infractions appeared to vary based on which school a person attended
Which method obtains answers from reasoning and logical conclusions?
Rationalism
Disadvantages of experimental design
Realism; Mundane vs. Experimental
Face-to-face interview
Require that the interviewer and respondent meet to conduct the interview
Graphic rating scale
Requires a mark along a continuous 100-millimeter line that is anchored with descriptions at each end.
coefficient of variation
SD/mean; useful for comparison of variability between groups where comparisons are difficult because of differences in scale or size of the subjects being compared
Probability Sampling
Sampling strategy that gives every subject in the population the same likelihood of being selected. Purpose: save time; cost; accuracy of generalization
Historical Control
What compares data to *known populations* or *previous sampling*?
Family Therapy is often used for what psychological disorder?
Schizophrenia
Secondary Sources
Secondhand reports in which the authors discuss someone else's observations
Bias Blind Spot
Seeing bias in others but not in ourselves
Alpha Level and meaning related to error
Set that likelihood at what is called an alpha level (usually .05) which means there are only 5 opportunities in 100 that a Type I error will exist.
Interaction
Situation in which the effect of one independent variable on the dependent variable changes, depending on the level of another independent variable.
Spearman Rho Correlation Coefficient (r sub s)
Special form of Pearson r used when one variable is ordinal and the other is continuous. Ex. Class rank (ordinal) and PCA score (continuous)
Internal Consistency - Split-Half Reliability
Splits the test in half computing a separate score for each half, and then calculating the degree of consistency between the two scores for a group of participants
STROBE
Strengthening The Reporting of OBservational studies in Epidemiology
Longitudinal Design
Studies in which the same variables are measured in the same people at different points in time *helps establish temporal precedence*
Double-Blind Study
Study in which neither the experimenter nor the subjects know if the subjects are in the experimental or control groups.
Characteristics of Good Scientific Theories
Supported by data Falsifiable Parsimonious (Occam's Razor)
Pseudoscience
System of ideas often presented as science
Coding Data
Systematically reorganizing raw numerical data into a format that is easy to analyze using computers For example, with nominal data, researcher codes single people as 1, married as 2, common-law/cohabitating as 3, divorced as 4, separated as 5, and widowed people as 6
The APA Ethics Code
Ten ethical categories (150 standards) to provide guidance for psychologists
Nuremberg Code (1947)
Ten guidelines for the ethical treatment of human subjects in research
Availability Heuristic
Things that are easily brought to mind guide our thinking, Especially problematic when trying to determine how often something occurs.
Repeated Measure design
What type of design is anytime there is *more than one measure*?
Measured Score
True Score + Error
Nonexperimental method
Use of measurement of variables to determine whether variables are related to one another. Also called correlational method.
Phi Coefficient
Used when both sets of data are nominal, dichotomous measures. Ex. Sex (male/female) and Passing the NCE (pass/fail)
How do we demonstrate consistency of a measure?
Two scores (one score from the new measure and one score from a well-established measure) are represented for each person in a scatter plot
Nonprobability sampling
Type of sampling procedure in which one cannot specify the probability that any member of the population will be included in the sample.
Operational Definitions
Types of variables involved in research
Understanding
Understanding is a precondition for greater depth, not an end in itself. Learn to think and act within a member's perspective. Empathy, or adopting another's perspective
Netnography
Understanding social interaction on the internet (digital media)
Concurrent Validity
Validated measure and behavioral outcome measure are administered at the same time
Predictive Validity
Validated measure is administered weeks, months, or years before behavioral outcome measure is taken
Interval
Values are ordered and equally spaced Continuous data Likert scales Parametric measures
Ratio
Values are ordered and equally spaced with a meaningful 0.
Humanistic view of research participant
View where the participant is a colleague who cooperates in providing data. Tends to be more sloppy.
- Increase sample size - Increase external validity
What are the *strengths* of *Multi-Center studies*?
Observational design
What type of design is simply *observing* relationships as they *exist*?
descriptive statistics
summarize data
- History - Maturation - Testing - Instrumentation - Selection - Regression to the mean - Diffusion or imitation of tx - Compensatory rivalry - Compensatory equalization
What are the *threats* to *internal validity*?
Faulty Thinking
We believe a theory that makes intuitive "sense"
*Why* we cannot make causal claims (from correlational data)
We don't know if x caused y Or if y caused x Or if a third variable (z) caused x AND y
Present/Present Bias
We tend to not look for absences—while it is easy to notice when something is present. It is a failure to notice appropriate comparison groups
Multiple measures
What *increases power* by including multiple ways to measure the dependent variables?
More comfounds
What is the *weakness* of *field* based experiments?
Masked design
When the observer does not know which group received the treatment.
observer bias
When the observer's expectations influence observations.
- Participant - Researcher
Who are "blind" in a *double blind study*?
Because if participants are randomly assigned to groups, the groups are unlikely to have systematic differences.
Why does random assignment reduce the chance of confounds?
can you be reliable without being valid?
YES
Nuremberg Trials
______ _______ led to the need for ethical controls in scientific research
The IRB
________ ensures that psychological studies present as little risk to subjects and participants as possible and have scientific merit.
Quantitative
_____________ research measures variables numerically and assumes that the social world can be understood objectively.
Perception-Based Needs Assessment
utilize surveys to identify the subjective needs of various client populations.
Acquiescence (Yea-saying)
agreeing or answering in a positive manner for all questions. The opposite strategy is sometime called nay-saying. - Try reverse-wording some questions. be careful about double negatives
variable
any measurable property that can take different values either within a subject, among different subjects, or between groups of subjects
Positive Correlations
as A increases, B increases
observational studies are more _____ than RCTs
ethical
study for rarer outcomes
case-control
coercion
compelling or influencing a decision to participate in a research study
highest level of validity
construct validity
Utility
do results provide users with practical, easily used information.
outliers
data points that are highly improbable, although not impossible
invalid data
data points that fall outside the defined range for that variable of data
safety monitoring
documenting and reviewing of adverse events
inferential statistics
draw conclusions that can be applied to broader population
Measurement Quality
generalizability, validity, reliability,. -a study can be reliable but not valid or vice versa. -Validity: you measured what you wanted to -Reliability: this can be repeated and you will get the same answer every time
theory to data
hypothesis deduced: studies designed and completed, outcomes provide or far to provide inductive support, falsification is possible
Good theories have...
parsimony (simpler explanation), productivity, and falsification
Sample
part of a population
Ways to Handle Missing Data: Pairwise Deletion
participant dropped in analysis for only missing variable. participants answer all items on one scale, but not another, kept in analysis for only completed items. Used in smaller data sets where less power is observed.
Concurrent measure design
participants are exposed to all levels of an IV roughly at the same time and a single measurement of the DV is recorded
odds definition
probability of one outcome divided by the probability of another
strongest type of observational design
prospective cohort
discrete variable
represent discrete categories or counted whole numbers
types of randomization
simple, block, stratified, adaptive
Mixed Methods: Single Phase
simultaneously collect both qualitative and quantitative data. One set of questions does not rely on the others. ex. survey with open-ended questions
Mixed Method Design Types: Triangulation
single phase approach that simultaneously collect both qualitative and quantitative data, merge the data and use results to understand a research problem. Qualitative, quantitative and previous research data carry equal weight.
types of blinding (4)
single, double, triple, open label
carryover effects
some form of contamination carries over from one condition to the next how much does one influence on the next (very similar to priming) (e.g., survey order)
Single-blind procedure
special type of within-subjects design using one participant (human or animal) or perhaps one group to assess changes within that individual or group participants are unaware of, or blind to, the type of treatment they are receiving, but the administrator knows
inclusion criteria
specific characteristics the investigator is interested in studying
science
synthesis of rationalism and empiricism / helps to correct own bias
content validity
the extent to which a test samples the behavior that is of interest. Specifically asking if all necessary constructs were captured
Computer-assisted telephone interviewing (CATI)
this technology provides the script the telephone interviewer is to follow. It also allows the telephone interviewer to record the responses into a database as they are given.
Levels
values that a variable can take on.
Conceptual Variable
variable of interest stated in agreed upon terminology. abstract concepts such as "shyness" or "intelligence" Also called a construct, conceptual definition
Mixed-mode surveys
—combining methods of conducting survey research, such as interview or mail, with online based survey methods has shown promise as a means of getting high response rates.
Electronic instrument design
—effective online surveys do not mirror a questionnaire exactly, but instead make specific considerations for question arrangement and layout.
control group
a level of an independent variable that is intended to represent "no treatment" or a neutral condition
statistics
a logic and methodology for the measurement of uncertainty and for an examination of the consequences of that uncertainty in the planning and interpretation of experimentation and observation
practice effects
a long sequence might lead participants to get better at the task, or to get tired or bored toward the end over time get better (e.g., IAT)
measures of central tendency
a measure of the average score in a distribution, such as the mean, median, mode
measures of variability
a measure of the degree of difference between scores in a distribution: range, variance, standard deviation
pilot study
a study completed before (or sometimes after) the study of primary interest, usually to test the effectiveness or characteristics of the manipulations
factorial design
a study in which there are two or more independent variables, levels, or factors allow a researcher to simultaneously examine the impact of 2 or more independent variables on the dependent variable. an experimental design that has more than one independent variable 2X2 factorial design
Internal reliability
a study participant gives a consistent pattern of answers, no matter how the researcher has phrased the question do the different questions/parts of the measure produce similar results?
Hawthorne effect
a subject changes their behavior because they know they are being monitored
missing data code
a symbol, such as an asterisk (*), that is entered in a cell that has no data
Experiments must
establish internal validity systematic variability unsystematic variability
Confound
an unanticipated variable not accounted for in a study that could be causing observed changes
Factorial Design
evaluate two or more independent variables, including interaction of the variables in relation to the dependent variable. Help account for extraneous variables; allows assessment of interaction.
factorial design
evaluates multiple interventions in a single experiment
complete counterbalancing
everyone gets a different order within participants ABBA between AB for some, BA for others
Panel Studies
exact same individuals surveyed at each time of data collection.
seeing red experiment
example of posttest only experiment
interference effects
exhaustion (e.g., memory)
placebo effect
experimental results caused by expectations alone; any effect on behavior caused by the administration of an inert substance or condition, which the recipient assumes is an active agent.
design confound
experimenter makes a mistake in designing the IV in which a second variable varies systematically with the intended IV poor internal validity cannot support causal claim
9.05 Debriefing
- Behavior analysts inform the participant that debriefing will occur at the conclusion of the participant's involvement in the research
Predictive Validity
- Scores obtained from a measure accurately predict behavior according to a theory - Example: Functional Analysis
Construct Validity
- Scores obtained from a measurement behave exactly the same as the variable itself - Example: Temperature going high and aggression increasing
Concurrent Validity
- Scores obtained from a new measure - Directly related to scores obtained from a more established measure of the same variable
error of central tendency
- a response bias where a participant tends to avoid using the extreme response alternatives on a scale - solution: use more alternatives (5-9 is optimal)
non-probability sampling
- chosen on some basis other than random selection - snowball - connivence - purposive
non-directional hypothesis
- a hypothesis that does not make a specific prediction as to how two variables are related - ex: someone will win the game
directioanl hypothesis
- a hypothesis that makes a specific prediction as to the exact nature of the relationship between two variables - ex: my team will win the game
manipulation check
- a measure that helps determine whether the manipulation effectively changed or varied the independent variable across conditions
survey
- a quantitative research strategy for systematically collecting information from a group of individuals - information is then generalized to a larger group of interest
correlational study
- a research approach that focuses on how variables relate to one another - helps identify associations but not causations
Correlation
A measure of the extent to which two factors vary together, and thus of how well either factor predicts the other.
Semantic differential scale
A measure of the meaning of concepts. I.e. Respondents are asked to rate any concept—persons, objects, behaviors, ideas—on a series of bipolar adjectives using 7-point scales
Correlation
A measure of the relationship between two variables
Range
A measure of variation computed as the difference between the highest and lowest scores in a distribution. For example, the range for the bus stop in front of the bar is from 25 to 35, or 35 - 25 = 10 years For ordinal-, interval-, and ratio-level data
interquartile range
A measure of variation in a set of numerical data, the interquartile range is the distance between the first and third quartiles of the data set.
Percentile
A measure of variation to tell the score at a specific place within the distribution. Example: median is 50th. For ordinal-, interval-, and ratio-level data
Standard Deviation
A measure of variation; it is also the most comprehensive and widely used. The "average difference" between all scores and the mean. Requires an interval or ratio level of measurement. The bigger it is, the bigger the average difference. The smaller it is, the more similar the values in a distribution are to one another
Inductive Reasoning
A small set of specific observations is the basis for forming a general statement about a larger set of observations
Stable-basline design
A small-N study where a researcher observes the participant's behavior prior to the start of treatment, in order to get a baseline. If the behavior is consistent during the baseline, the researcher is able to be more certain that the treatment is effective. This offers protection against maturation and regression.
Multiple baseline designs
A small-N study where researchers manipulate the independent variable/treatment over various amount of time, contexts, or situations. The dependent variable is then repeatedly recorded in order to establish a stable baseline between the intervention stages. When there is a clear association between the timing of the independent variable manipulations and the timing of any changes in the dependent variable, it offers evidence for internal validity.
non-probability sample
any sample in which little or no attempt is made to get a representative cross section of the population. e.g purposive - picking who to talk to, talking to mayors of towns
control variable
any variable that an experimenter holds constant on purpose
quantitative technology
application of mathematical techniques to describe an analysis of data
Statistical Control
applying the proper statistical analysis when evaluating outcome data to minimize effect of potential confounding variables. Control extraneous variables.
case-control study
compares a group of people with a disease to a group of people without a disease
chi-square (x^2)
compares observed frequencies to expected frequencies. Has to be nominal dichotomous data only. How to read: The higher the value, the more statistically significant difference exists. If you have cells with less than 5, look at Fisher test. If not, use Pearson. ex. There is a statistically significant difference in the relative frequency of individuals from different tracks and eating one side of Twix first.
Inductive Approach
An approach to inquiry or social theory in which one begins with concrete empirical details, then works toward abstract ideas or general principles
Matched Assignment
An assignment method in which subjects are first divided into subgroups based on a specific characteristic such as age, gender, and so forth; members of each subgroup are then randomly assigned to each group in the study to balance the characteristics across the groups. Alternative to random assignment, but difficult to assign similar characteristics.
Positive Relationship
An association between two variables such that as values on one increase, values on the other also increase
Negative Relationship
An association between two variables such that as values on one variable increase, values on the other variable fall or decrease
Census
An attempt to count everyone in a target population. Every 5 years (2021)
A case study
describes a survey conducted on college students.
Descriptive research
describes social trends and phenomena.
Expectancy effects (experimenter bias)
Any intentional or unintentional influence that the experimenter exerts on subjects to confirm the hypothesis under investigation.
Situation noise
Any kind of external distraction that is able to cause variability within-groups that complicate between-groups differences. This can be minimized by making sure the experimental surroundings are controlled for.
Extraneous Variables
Any variables other than the independent variable that seem likely to influence the dependent variable in a specific study.
Extraneous Variable
Any variables that you are not intentionally studying in the experiment.
Technological
Applied interventions are described well enough that they can be implemented by anyone with training and resources
Generality
Applied interventions are designed from the outset to operate in new environments and continue after the formal treatments have ended
Conceptual
Applied interventions arise from a specific and identifiable theoretical base rather than being a set of packages or tricks
Behavioural
Applied interventions deal with measurable behaviour (or reports if they can be validated)
Applied
Applied interventions deal with problems of demonstrated social importance
Effective
Applied interventions produce strong, socially important effects
Analytic
Applied interventions require an objective demonstration that the procedures caused the effect
Operationalization
Approximating the true variables of interests with one that can be measured or tabulated -single measure, e.g GPA -composite measure
Which method obtains answers from an expert?
Authority
Observer bias Demand characteristics Placebo effects
However, even with a comparison group there can still be other issues:
participants;subjects
Humans in research are referred to as ___________, and animals are referred to as ___________.
Methods of Tenacity
Information that is accepted as true because it has historically been believed or superstition supports it
Informed Consent
Informing the research participants of the nature of the study and obtaining their explicit agreement. Ensures that researchers do not violate the privacy of participants and that participants have enough info to decide whether they want to participate
objective measures; Counterbalancing
Instrument threats can be prevented by using ____________ at pretest and posttest. Or, by using a posttest only design. __________ also helps.
Interviewer bias
Intentional or unintentional infl uence exerted by an interviewer in such a way that the actual or interpreted behavior of respondents is consistent with the interviewer's expectations
In which two aspects should research questions be evaluated in?
Interestingness and Feasibility
Which method obtains answers from a hunch or feeling?
Intuition
Theory-testing mode
Investigating the support for a theory.
Scales of measurement
Nominal Ordinal Interval Ratio
Alternative explanations
Part of causal inference; a potential alternative cause of an observed relationship between variables
Placebo effects
Participant group improves when they think they are given the treatment for the experiment but are actually not given the treatment.
within-groups repeated measures counterbalance
Participants are exposed to different conditions at different times. Researchers need to _________ the order of presentation.
concurrent measures contact/comfort with monkeys
Participants are exposed to different conditions at the same time. example?
Informed Consent
Participants are informed in advance of all aspects of the research that might influence their decision to participate
Justice
Participants are not exploited and treated fairly
posttest only
Participants are randomly assigned to groups, and tested on a dependent variable once. It's a ________ because the DV is measured after the IV.
Debriefing
Participants are told about the nature of the study once it has been completed
Testing threat
Participants change as a result of the dependent variable/test being given to the subjects more than once. This can be stopped by including a comparison group or only using a posttest-only design experiment.
Reactivity
Participants may simply act differently when they know they are being observed. Possible solutions include: • Unobtrusive observations (blend in). • Wait it out (allow acclimation to one's presence). • Measure the results of a behavior (indirect measures of constructs), rather than the behavior itself. - remember to observe ethically
Self-report Measurement
Participants provide responses to verbal questions about themselves (or others) through surveys or interviews
Observer effects (expectancy effects)
Participants' behavior changes to better match the observer's expectations. - reactivity type
Assumption
Parts of social theories that are not tested, but act as starting points or basic beliefs about the world. They are necessary to make other theoretical statements and to build social theory
Two Forms of Deception
Passive Deception Active Deception
True
Peer review is a collaborative process whereby researchers who are knowledgeable about a particular subject are asked to review and comment on another researcher's work and recommend whether it should be published. T/F
Journals
Peer-reviewed articles for scientists
Ethical Survey
People can conduct surveys in ethical and unethical ways -Major issue is the invasion of privacy -Treat all respondents with dignity and reduce anxiety -Responsible for protecting the confidentiality of data. -Voluntary participation -Some people use surveys to mislead others and produce desired results -Currently, there are no quality-control standards to regulate the opinion polls or surveys reported in the Canadian media.
If data were not public, what may occur?
People could rearrange their data so it looks better
random assignment if you have a large sample... Or, with matched groups if dealing with a smaller sample.
Selection effects can be avoided by
Scatterplot Precision
The amount of spread in the points on the graph. Close together means highly related, further apart less closely related
Mean
The arithmetic average of a distribution, obtained by adding the scores and then dividing by the number of scores. Used only with interval or ratio data. Compute by adding up all scores, then divide by the number of scores -Strongly affected by changes in extreme values -Used in most statistics
Mechanical Method
The assignment is automatic - not based on any personal preference or specific cases
exhaustiveness
The capacity for a list of attributes to contain all reasonably possible levels of a variable is referred to as:
Predictive validity
The construct validity of a measure is assessed by examining the ability of the measure to predict a future behavior.
Validity
The degree in which the measurement process measures the variable it claims to measure
Reliability
The degree of stability or consistency of measurement
Face validity
The degree to which a measurement device appears to accurately measure a variable.
Construct validity
The degree to which a test measures what it claims to be measuring.
Marginal means (how they are represented & calculated)
The means for each level of an independent variable averaging over the levels of another independent variable in a factorial designs study.
Generalization mode
The researcher's intent to generalize their findings from their study to other populations or contexts.
Systematic
The scientific method has been developed as a set of strategies researchers can use making research what?
Statistical correlation
What is a type of *significance test* that does not tell about random assignment?
Symmetrical distribution
What is also known as *normal distribution* when the frequency *distribution* from both sides of the *median mirror* the other?
Normal distribution
What is also known as *symmetrical distribution* when the frequency *distribution* from *both sides* of the *median mirror* the other?
Procrustean Data Torturing
What is an *analysis and publication bias* where *data* is *manipulated* to support a *hypothesis*?
Opportunistic Data Torturing
What is an *analysis and publication bias* where a researcher analyzes to find anything significant, then develops a *hypothesis based off* the *data*?
Publication Bias
What is an *analysis and publication bias* where only the *significant results* are more likely to be *published*?
Research questions
What is an *answerable inquiry* into a specific concern or issue that is the *initial step* in a research project?
Variable
What is an element of a hypothesis that involves *how* a *construct's* *operationalization* is *measured*?
Operationalization
What is an element of a hypothesis that is *how a construct* is defined within one specific study?
Construct
What is an element of a hypothesis that is an abstract representation including both *independent* and *dependent variable*?
Double blind study
What is an experiment in which neither the subjects nor those who administer the tx know who is in the experimental group or control group that is and *limits experimenter expectancy bias*?
Random assignment
What is an experimental technique for assigning participants or animal subjects to different groups in an experiment using randomization and lends strength to *internal validity*?
Confound
What is any *other explanation* for the *results* other than the hypothesized explanation?
Dispersion
What is the *distribution* of *values around* some *central value*, such as an *average* which observes how typical that value is?
Significance test
What is the *effect size* x the *sample size*?
Alternative hypothesis (HA)
What is the *hypothesis* in which you predict that a *relationship exists*, also known as *research hypothesis*?
Null hypothesis (Ho)
What is the *hypothesis* that there is *no relationship* or that nothing happens in which *you want to reject*?
Mode
What is the *most frequent value* that occurs?
Frequency
What is the *number of times* a value occurs?
Confidence interval
What is the *range of values* within which a population parameter is estimated to lie, which is the mean difference on test scores which speaks to validity?
Extreme values are, by definition, unusual. Therefore, the next measurement will most likely be less extreme (closer to the mean).
explain the concept of regression to the mean
Ethical Issues in Research Analysis: Data Dredging
exploring data post hoc for significant findings in the absence of significant results for the stated hypothesis. Playing with data looking for significance after analysis.
withing subjects
expose participants to all levels of treatment - randomly assign a sequence of treatment conditions - + power -internal and external validity
between subjects
expose participants to one level of treatment - randomly assign participants to one condition- + internal and external validity -power
OR > 1
exposure associated with higher odds of outcome
OR < 1
exposure associated with lower odds of outcome
Face validity
extent to which respondents can tell what the items are measuring
external validity
extent to which we can generalize findings to practice
Anatomy of a two way Factorial ANOVA - 4 groups (2X2)
f(#,#) = #.##, p= ##, eta^2 = .## f = f test symbol (first number) is between subject degrees of freedom and within subjects degrees of freedom #.## = f score p ## = significance level eta ##= calculated effect size
Types of measurement validity
face validity, content validity, convergent validity, discriminant validity
population-based case-control
has a defined base population
in retrospective studies, outcomes have/have not occurred
have
in prospective studies, outcomes have/have not occurred
have not
Control Variable
held constant on purpose
Internal consistency reliability
the assessment of reliability using responses at only one point in time.
strengthening internal validity (2)
include control group, randomization
Positive linear relationship
increases in the values of one variable are accompanied by increases in the values of the second variable
A research design where there are different participants at each level of the independent variable.
independent group design Example: one group gets a treatment, the other gets a placebo.
predictor variable
independent variable
Quasi-Experimental Design
independent variable cannot be truly manipulated. Comparison group, but no random assignment. Use of male vs. female, ethnicity, SES.
Factorial within-groups
independent variables are manipulated as within-groups. - example: 2x2 study, there is only one group of participants experiencing all levels/conditions
open cohort
individuals can join or drop out of study over time
placebo
inert treatment
accuracy
infer the population parameters with as little uncertainty as possible; unbiased
I-Data
informant report
unobservable confounders (2)
instrumental variable methods, Heckman selection model
helps to minimize attrition bias
intention-to-treat analysis
Reactivity
participants can change their behavior (react) when they know another person is watching - even if observer didn't give any cues about expectations Solution: Unobtrusive Observation (hide) Let people habituate (wait it out) Measure behavior's results
retrospective timeline
past to present
cases
people afflicted with a disease
SAMPLING: population
people we want to know about
P-data
physiological (Heart Rate, Skin detection, etc)
efficacy
physiology, survival, and quality of life in a near-perfect setting
types of control groups (4)
placebo, active control, historical control, non-inferiority trials
confound
potential threats to internal validity. Means to confuse
regression
predicting the value of one variable from another based on their correlation
ratio scales of measurement
order plus equal intervals plus true zero, score of 0 = no amount of variable being measured (true 0)
interval scales of measurement
order plus equal intervals, intelligence scales & personality scales
UNDERSTANDING DATA: dependent variable
outcome, what you're interested in, how it varies across certain population (effect)
margin means
overall means - the means for two groups or cells Main effects of the two IV (pick up lines, scent) the average of all participants on one level of the independent variable ignoring the other independent variable
cohort studies _____ clinical trials
parallel
repeated measures design
participants are measured on the dependent variable more than once after than have been exposed to each level (or condition) of the independent variable
Power
probability that a study will show a statistically significant result when an independent variable truly has an effect in the population.
Matching
process of selecting individuals to represent treatment and comparison groups based on specific similar criteria or characteristics. Similar traits evenly matched. People.
Hypothesis
proposed explanation made on the basis of limited evidence as a starting point for further investigation
Mixed Methods
provides more robust data, brings out unique variances and ideas and can help reduce bias. Must be proficient at both types of research design (qualitative and quantitative). Research question order determines the priority of the methods used, must relate question to type of design, and ensure questions are specific and measurable relative to type of design employed.
Double-Barreled Questions
questions that *attempt to get at multiple issues at once*, and tend to receive incomplete or confusing answers
simple randomization
random number generator
cohort studies differ from RCTs as cohort studies do not _____
randomize exposure
Interquartile range
range from the 25th percentile to the 75th percentile
measures of variability
range, variance, standard deviation, IQR
Ordinal
rank, order. Meaningful but unequal intervals (1st, 2nd, 3rd)
minimal risk; nine
research must represent ________ _______ and fall into one of _______ categories in order to be expedited.
STROBE checklist (7)
research question, study design, study population, exposure and outcome, data analysis and confounding, presentation of results and interpretation, practice and policy implications
Manipulated Variables
researcher sets up conditions to produce it.
Experiment
researchers manipulated at least one variable and measured another
Alternative Hypothesis (H1)
researchers prediction but cannot be absolutely proven
Ethical Responsibility
responsibility to individual subjects, to discipline of science, and to society as a whole
similar to stable-baseline, but treatment is taken away after improvement to see if progress reverses.
reversal design: Example: how will depressive/coping behavior change in response to treatment and nontreatment periods?
risk difference equation
risk exposed - risk unexposed
risk ratio equation
risk exposed divided by risk unexposed
factors that threaten external validity (5)
selection, pretesting, research setting, history, multiple treatments
S-data
self-report
3 types of operationalizations
self-report, observational, physiological
Ethics
set of principles for behaving in a way that is morally correct
Familiarity with questionnaire
the interviewer should be very familiar with the overall format of the survey and every item.—
median
the middlemost score in a distribution
mode
the most common score in a frequency distribution
animal rights
the notion that animals have the same sort of rights as people, including legal rights; not generally accepted
Single-blind experiment
the participant is unaware of whether a placebo or the actual drug is being administered.
Observer Effects
the participant picks up on observer's unintentional cues and responds the way they think they should Solution: Develop good scoring rubrics Use more than one observer
double-blind procedure
the participants and administrators of treatment are unaware, or blind, to the types of treatment they are administering to reduce the influence of expectancies they know the condition might have in order to reduce the likelihood that expectancies or knowledge of condition will influence the results, both the participants and the administrators of treatment are unaware of, or blind to, the types of treatment being provided
treatment group
the participants in an experiment who are exposed to the level of the independent variable that involves a medication, therapy, or intervention
Sensitivity
the percentage of cases with a diagnosis or condition who are correctly identified
specificity
the percentage of cases without the diagnosis or condition who are correctly identified ability to avoid false positives
Time horizon (in decision analytic modeling)
the period of time during which the outcomes are predicted by a decision analytic model
the investigator
the person who is in overall charge of the research and has the greatest responsibility to see that ethical principles are followed
Placebo effect
the phenomenon in which some people experience some benefit after the administration of a placebo.
Median - NOT how to calculate it, just what it is
the point that divides the distribution into two parts, such that an equal number of values falls above and below that point
Power
the probability of rejecting a false null hypothesis the probability of finding statistical significance when the result is significant
debriefing
the process of informing participant after the session of the experiment's true purpose to increase their understanding and to remove possible harmful effects.
data reduction
the process of transcribing data from individual data sheets to a summary form
Confidence interval
the range of values within which a population value is likely to fall, estimated at a certain level of probability
Null hypothesis
the relationship between or among the study variables is not as hypothesized by the investigator an idea that competes with the working hypothesis
negative correlation
the relationship between two variables in which one variable increases as the other variable decreases
Standard Deviation
the square root of the variance
standard deviation
the square root of the variance
standard deviation
the square root of the variance; a measure of variability in the same units as the scores being described
Correlation
the strength of the relationship between two variables
independent variable
the variable that you have chosen to manipulate in the experiment in which you are setting up
dependent variable
the variable you will measure in your experiment, that will change in accordance to the independent variable
constructs
theoretical concepts formulated to serve as casual or descriptive explanations: Milgram's construct was obedience
Research Question
to be mixed method design, both qualitative and quantitative research questions must be included.
IRB mission
to protect all human subjects from physical and psychological harm caused by experimental research
number
to remove bias and to protect the identity of the participant give them a _________ to be recognized by.
operationalize
to turn a concept of interest into a measured or manipulated variable
authority
trusting those who are "above" you
the commitment to expand knowledge and the cost to research participants
two values that create conflict in the decision to conduct research
What specific individuals and number of those people should be on a research ethics board (REB)?
two with expertise, one the ethics, one with no affiliation
mortality rate
usually annual number of deaths in a year, divided by the population in that year
Evaluation research
utilizes a wide range of methods to assess the efficiency or effectiveness of a program, policy, or process
Dependent variable
variable that is being measured
Dependent Variable
variable that reflects the impact of the manipulated variable
extrinsic source of variation
variation caused by imprecision of measurement (human error) or sampling variation (not biologically real)
Ethical Issues in Research Analysis: Coarsened Data
various techniques such as: rounding, grouping or transforming missing data changes or reduces amount of specificity of the data to reach significance.
Perspective (in decision analytic modeling)
viewpoint from which a decision analytic model is conducted
when all participants plan to receive treatment, researchers may compare those who receive early treatments to those who are still waiting (a strategy to reduce selection effects)
wait-list design
Propriety
was evaluation conducted in an ethically appropriate manner.
systematic variability
way to remove a confound in statistical analysis
can support an association (but NOT a causal claim)
we haven't measured every variable, haven't accounted for 3rd variable
change or influence/control (goal of psych/res)
what can we do to change or modify this behavior?
independent
what is making the outcome happen; explanatory variable (cause)
interval/ratio variables
what you report will depend on shape of graph -the bigger the SD, the more spread out the data is -SD of 0 means all data is the same #, there is no spread ** -Symmetric distribution: report mean, SD -Skewed: negative or positive skew, better to report median, min/max or range (Pos is if data is to the left, neg is to the right)
experimenter expectancy effect
when a bias causes a researcher to unconsciously influence the participants of an experiment (also known as expectancy bias or experimenter effect)
Formats for respondents
when asking a question or item with multiple possible responses, it is important that the responses are clearly distinguished from one another.
regression threat
when difference between pretest and posttest is caused by regression to the mean.
carryover effect
when exposure to one condition changes the response to another.
order effects
when exposure to one level of the independent variable influences responses to the next level of the independent variable
maturation threat
when participants change on their own over time.
experiment
when researchers manipulate at least one variable, and measure others.
history threat
when something happens between the pretest and posttest.
3-way interactions
when the difference of differences are different - example: a crossing and a spreading interaction
crossover interaction
when the influence of one independent variable on the other reverses across levels of the other variable
selection effects
when the kinds of participants in one level of the independent variable are systematically different from those in the other
instrument threat
when the measuring instrument changes over time. Example: When teachers become more lenient (or stricter) with their grading over time.
order effects practice, fatigue, carryover
when the order in which participants perform conditions in within-groups experiment affects the results. examples:
selection effect
when the traits of participants influence which condition they are assigned to, creating a confound.
null effect
when there is no significant covariance between the variables of interest.
Factorial independent-groups
where both independent variables are studied as independent-groups - example: 2x2 study, four different groups of participants
sampling variation
where multiple samples from the same population vary in composition and therefore vary in sample statistics
derived statistic
where one value for data is expressed in terms of another value
Coordination and control
—interviewers should be trained before they begin collecting data to identify any problems they might have administering the questionnaire. It is also important to monitor the interviewers during the course of the survey.
Follow question wording exactly
—it is important that the interviewer not deviate from the wording of the questions, which could bias the results.
Probing for responses
—oftentimes the interviewer will ask the respondent to give more information, which is common for open-ended questions.
Response rates in interview surveys
—response rates in telephone interviews have declined as respondent are increasingly likely to screen their calls.
Record responses exactly
—similarly, the responses should be recorded exactly as they are given.
Appearance and demeanor
—the interviewer should dress similar to the respondents and be polite.
Design your own 2X2 factorial study on the topic by doing this:
Determine the two independent variables involved in testing this topic. Determine two levels for each of the independent variables. Determine and develop the dependent variable. Develop a hypothesis.
Strength of Relationship
Determined by numeric value of the correlation coefficient. Range from -1.00 to +1.00. -1.00 is a perfect negative correlation. +1.00 is a positive correlation. 0.00 indicates no correlation
Reliability
Did the authors measure their constructs/variables well?
Converging evidence
Different measures provide outcomes pointing to the same conclusion
Counterbalancing
Different subsets of participants complete conditions in different orders - *how to avoid order effects confounds*
Experimental method
Direct manipulation and control of variables
Behavioral measures
Direct observations of behavior.
Applied Research
Directed towards solving practical problems
Coverage
Disadvantage of online survey. Involves sampling and unequal access or use. Older, less-educated, lower-income, and more rural people are less likely to have good internet access
Complexity of Questionnaire Design
Disadvantage of online survey. It is best to provide screen-by-screen questions and make an entire question visible on the screen at one time in a consistent format with drop-down boxes for answer choices
Privacy and Confidentiality
Disadvantage of online survey. Researchers should encrypt all data, use secure websites
Bimodal
Distributions with two modes
Random Assignment
Dividing subjects into groups at the beginning of experimental research using an unbiased, random process, so the experimenter can treat the groups as equivalent. Random in the statistic or mechanical sense.
What do all areas of ethics surround?
Do no harm
evolution
Ethics are in a rapid state of ________.
Risk-benefit analysis
Evaluation of the ethical implications of a research procedure by considering the benefits of the research in relation to the potential risks to participants.
Conceptual Definition
Everyday vernacular use of a term. Used by journalists when reporting.
Empiricism
Evidence is acquired from systematic, objective (unbiased) OBSERVATION, independently verifiable by other observers that provides a tentative answer.
Type A
Exaggerated sense of time urgency, competitiveness, and drive
Statistical Relationship
Exists between two variables when the average score on one differs systematically across levels of the others
Quasi-experimental Design
Experimental designs that are stronger than pre-experimental designs. They are variations on the classical experimental design that an experimenter uses in special situations or when an experimenter has limited control over the independent variable
Pre-experimental Design
Experimental designs that lack random assignment or use shortcuts and are much weaker than the classical experimental design. They may be substituted in situations where an experimenter cannot use all the features of a classical experimental design, but they have weaker internal validity
Each question be clear to answer so it doesn't confuse respondents (or influence their answers)
Explain how carefully prepared questions improve the construct validity of a poll or survey.
Debriefing
Explanation of the purposes of the research that is given to participants following their participation in the research.
External Validity
Extent to which study can be generalized to other settings and populations.
______ ________ is a major challenge for small-N studies.
External validity
Empirical Article
First time report of result from research Ask: What are the arguments? What is the evidence?
Discrete Variable
Have a small number of levels. -Less than 10 levels Examples: -gender, smoker status
Empiricism - Objective
Have to be able to come to a consensus
Name two examples of guidelines that may be requested in order to participate at a conference
Having an abstract/website
Give two examples of well defined, easily observed, and measured variables to operationally define
Height and Weight
Anatomy of a factorial design
How a factorial design is described conveys the number of independent variables in the designs and how many levels exist for each independent variable. The number of numbers indicates how many independent variables the design includes. The numbers themselves indicate how many levels or conditions each independent variable has.
- Remove noncompliant pts - Intention to tx stat adjustment - Tx attrition as a dependent variable - Dropouts significantly different
How do researchers deal with *attrition bias* where one group is more likely to *dropout* where the dropping out may even be a dependent variable?
Within Group difference
How do you know how much *difference* is expected *due to chance* within a group?
Fail reject Null Hypothesis (p >0.05)
How do you know if you have *no significant* results in your study?
Reject Null Hypothesis (p <0.05)
How do you know if you have *significant* results in your study?
Ordering items in a questionnaire
How items are ordered on a questionnaire can affect the answers respondents give. Certain early items may influence later responses, as might the ordering of general versus specific questions on a given topic.
Effect Size
How much evidence is present that the results represent a general society
Empiricism
Objective observation that is independent of prejudices
Systematic observation
Observations of one or more specific variables, usually made in a precisely defined setting.
Observer Bias
Observer records what they expect (or want) to see - only looking for behavior they want Solution: Develop good scoring rubrics Use more than one observer
Masked (blind) design
Observers are unaware of the conditions to which participants have been assigned and are unaware of the purpose of the study. - can't provide cues when reach correct answer
Exposure Therapy with Response Prevention is often used for what psychological disorder?
Obsessive Compulsive Disorder
Cofounding variable
Occurs when two variables are linked in a way that makes it difficult to sort out their specific effects.
negative predictive value
Of cases who test negative, the percentage who are actual negatives (don't have the disease)
What are the main problems with headlines?
Often -oversimplified -overconfident Sometimes -misrepresent findings
Passive Deception
Omission
Intuition
One source of information "Common sense" Example: -It makes sense to me that speaking two languages from a young age would make it harder to learn each because the terms from one language interfere with another.
Authority
One source of information Authority figures or convention Example: -Dr. Mitch Smith, a dentist, says that he always recommends that his patients chew Orbit brand gum.
Experience
One source of information Can be: -Personal -Anecdotes or testimonials (from others) Example: -Exercise makes me feel calm - after my Cycle bar classes, I can always sleep well.
Causal
One variable is responsible for changing another variable. Involves at least two variables Implies that the reason for change in one variable is due to the other variable Example: Music lessons enhance IQ
Types of Pre-experimental Design
One-shot case study design One-group pretest post-test design Static-group comparison
Confirmation Bias/Cherry-picking
Only picking out what supports your theory (usually people with an agenda) *This is the biggest bias*
foolish or stupid
When debriefing, do not make the participant feel ________.
Random sample
When each member of the population has an equal chance of being selected to represent the whole.
A
When interrogating experiments, on which of the big validities should a person focus? Internal validity External validity Construct validity Statistical validity
- Unethical randomly assign - Biologically impossible to randomly assign - Before investing for full experiment
When is it good to conduct a *correlation design*?
Selection-attrition threat
When only one group in the experiment experiences a decrease in the number of participants.
Demand characteristics
When participants determine what the study is about and change their behavior based on what they think the expected behavior should be.This can be prevented by using a masked design or double-blind design.
demand characteristics
When participants guess the hypothesis of a study, and change their behavior.
Practice Effects Confound (+)(order effects)
When prior participation in a level of an independent variable or measurement *[[provides]] participants with additional skills that influence their performance* on subsequent measurements
Fatigue Effects Confound (-)(order effects)
When prior participation in a level of an independent variable or measurement *[[tires]] the participant and influences their performance* on subsequent measures
attrition threat
When some types of participants are more likely to drop out of the study than others
Coercion to Participate
When someone in authority applies pressure or attempts to influence individuals into participating. Individuals believe that not participating will have negative consequences. Ex.) Offering exceptionally high or excessive rewards for participating.
Pseudo Survey
When someone who has little or no real interest in learning information from a respondent uses the survey format to try to persuade someone to do something
supported or does not support
When stating your conclusion, you must decide if the data collected _____________________________ the hypothesis.
Selection Confound
When the *kinds of participants in one level of the independent variable are systematically different* from those in the other level of the independent variable Procedure 1: 20% chem majors Procedure 2: 80% chem majors
Covariation of cause and effect
When the cause is present, the effect occurs; when the cause is not present, the effect does not occur.
synergistic effect
When the combination of two variables results in something greater than simply the sum of the individual variables. an effect where two variable combine to produce an outcome that is bigger than what each individual variable contributes a type of interaction effect
suppression effect
When the effect of combining two variables results in something less than simply the sum of the individual variables an effect where two variables combine to produce an outcome that is smaller than what each individual variable contributes a type of interaction effect
placebo group
a control group that is exposed to an inert treatment
A research design where the same participants are tested at each level of the independent variable
repeated measures design
precision
repeated samples that are collected
Ways to Handle Missing Data: Imputation
replace missing values with substitutes such as mean value. Put something in its place. If item is nominal or categorical this option is not used.
Evidence Based Practice Model
represents practitioners' intent to utilize the best available research evidence to inform treatment planning and practice. Includes: Program description Utilize outcome research Intervention evaluation
assent
required from children, a basic yes or no
Consent
required from participants
examples of repeated measures quasi-experiment tests?
- Interrupted time-series design - Nonequivalent groups interrupted time-series design
Construct Validity
"*Are they actually* measuring what they think they are measuring?"
masked design
(researchers were unaware of which condition participants had been assigned to)
Independent Variable
*2 or more levels* chosen by the researcher - manipulated variable - comes first - *causing the change*
*Which APA Titles are not bolded?*
*Abstract, Title of paper, References, Appendices, Authors note, Footnotes*
*Level 1 Heading*
*Center boldface, upper and lower case*
Three Criteria for Causation
*Covariance* The measured variable must change as the manipulated variable changes (correlations; correlations satisfy 1 of the 3 criteria---covariance). *Temporal precedence* The manipulation must precede the change in behavior. A should always happen before B, never B before A! *Internal validity* Alternative causal explanations must be ruled out
*Level 2 Heading*
*Flush left, boldface, upper and lower case*
*Level 3 Heading*
*Indented, boldface, Capital on first letter only, Period at end.*
3 characteristics of good questions (understandable)
*Only ask one question at a time* - Avoid double-barreled questions *Use simple wording* - Avoid negatives - Avoid uncommon words/jargon - Avoid lengthy/complex statements *Use precise wording* - Avoid ambiguity
*Title Page*
- *Running head: (abbreviated title) and page number (1)* - *Title of the paper* - *Author name(s) (byline) and affiliations* - *Author note*
*When in APA reports are printed numbers used?*
- *When the number is at the start of a sentence* - *Quantities below 10* - *Anything in a title*
Likert Scale (lick-urt)
- A scale in which each statement is accompanied by five response options: strongly disagree, disagree, neither agree nor disagree, agree, and strongly agree - minor variations are often called Likest-Type scales - very common in psychology
Results
- A summary of the data and the statistical analyses - Begins immediately following the method section—not starting on a new page - Provides a complete and unbiased reporting of the findings, just the facts, with no discussion of the findings
What are the limitations of operational definitions?
- An operational definition is not the same as the construct itself - Concerns about the quality of operational definitions and measurements produced - It is easy for operational definitions to leave out important components of a construct - Operational definitions often include extra components that are not part of the construct being measured
9.02 Characteristics of Responsible Research
- Approval by REB, due concern for the dignity and welfare of the participants, minimize misleading results, trained, responsible for assistants, experience, maximize benefit and minimize risk, minimize financial, social, organizational, or political factors, avoid conflicts of interest when conducting research, minimize interference - This is a guideline that was not followed by Milgram
Which critical evaluation question should we ask ourselves when evaluating alternative explanations?
- Are alternative conclusions/explanations possible?
Which critical evaluation question should we ask ourselves when evaluating a justified conclusion?
- Are the conclusions justified by the results?
Which critical evaluation questions should we ask ourselves when evaluating a procedure?
- Are there alternative ways to define and measure the variables? - Could alternative procedures be used?
How can existing publication lead to ideas for new research studies?
- Authors generally make explicit statements regarding unanswered questions - Authors sometimes point out limitations or problems in their own studies - Create a new idea from two existing results - Frame a question based on differing results
What considerations should be applied when selecting references?
- Be conservative - Choose references directly relevant to the study you are presenting - Select only those references that are useful and contribute to your arguments
Survey Questions
- Behaviour. How frequently do you go to art galleries? - Attitudes/beliefs/opinions. What kind of job do you think the mayor is doing? Do you think other people say many negative things about you when you are not there? - Characteristics. Are you married, cohabiting, single, divorced, separated, or widowed? - Expectations. Do you plan to enrol in a degree program in the next 12 months? How much schooling do you think your child will get? - Self-classification. Do you consider yourself to be liberal, moderate, or conservative? Into which social class would you put your family? - Knowledge. Who was elected prime minister in the last election?
*Abstract*
- Centred and not bolded - No indent - *Statement of problem* - *Describe participants* - *Describe method* - *Report results* - Conclusions or Implications
Considerations when Designing Mixed Methods
- Determine the order of implementations of the qualitative and quantitative aspects of the study. Concurrently or in a sequence, illustrates intent of the study: broad to narrow, narrow to broad. - Determine the weight given to qualitative and quantitative data (research question order). Extent to which both will be incorporated into the study, primary focus of the study. - Determine how the two types of data will be mixed or combined. Interpretation might involve transforming qualitative themes into quantitative numbers. Interpretation might involve comparing numbers to lived experiences. - Determine theory (both qual and quan) to guide study. Forms the questions, analysis and interpretation of the data.
9.04 Using Confidential Information for Didactic or Instructive Purposes
- Do not disclose personally identifiable information concerning their individual or organizational research participants unless written consent - Disguise confidential information concerning participants so that they are not individually identifiable to others
Which critical evaluation questions should we ask ourselves when evaluating limits to generalization?
- Do the results have real-world applications? - Is there reason to suspect that the same results would not occur outside the lab? - Would the same results be expected with different participants or under different circumstances?
Which critical evaluation question should we ask ourselves when evaluating results related to hypothesis?
- Do the results really support (or refute) the hypothesis?
Which critical evaluation questions should we ask ourselves when evaluating a specific prediction from hypothesis?
- Does the predicted outcome logically follow from the hypothesis? - Can other specific predictions be made?
American Psychological Association (APA) Guidelines
- Ethical Guidelines for the Use and Treatment of Human Participants in Research - First set of guidelines for protecting human subjects in research (1973) - Updated and expanded many times, and periodically reviewed - Is considered the gold standard of guidelines that are out there
Safeguards to Prevent Fraud
- Experimental research studies that are designed to be replicated - Research articles that are peer reviewed by experts in the field - The consequences of being found guilty of fraud - a big deterrent
Oral Presentations/Symposiums
- Five minute talk about research, allow a few minutes for questions from audience - In preparing a talk, presenters should memorize key elements such as why you conducted the study, what your research question was, how you tested it and what you discovered. - The presenter should look at the audience members and speak to them in a conversational tone that is less formal than APA-style writing but more formal than a conversation with a friend.
Behaviour Analysts Ethical Guidelines
- Guideline 9 (The Behaviour Analyst and Research; 2017) - Complex set of requirements with the BACB Guidelines - Behaviour analysts design, conduct, and report research in accordance with recognized standards of scientific competence and ethical research.
Which critical evaluation questions should we ask ourselves when evaluating participants?
- If participants were restricted (e.g., males only), is it justified? - Would different participants produce different results?
9.03 Informed Consent
- Inform participants or their guardian or surrogate in understandable language about the nature of the research - Not coerced - Process to decline to participate, or to withdraw without penalty; - Available to answer any other questions participants may have about the research - If you want to do research informed consent is a MUST! - Make the potential participants aware that even if they do not choose to participate, treatment is still available
When giving an oral presentation at a conference, what procedure should be followed?
- Intro and Opening remarks: - Develop your problem - Outline research - Develop your point or solution (use lots of visuals) - Summary - Thank Audience
The Scientific Method
- Is an approach to acquiring knowledge - Involves formulating specific questions and then systematically finding answers - Combines several different methods of acquiring knowledge
Which critical evaluation questions should we ask ourselves when evaluating a hypothesis/purpose for study?
- Is the hypothesis clearly stated? - Is the hypothesis directly related to the reviewed literature?
Which critical evaluation questions should we ask ourselves when evaluating a literature review?
- Is the review up to date and complete? - Are relevant or related topics not covered?
References
- Listing of complete references for all sources of information cited in the report o Organized alphabetically by the last name of the first author o Begins on a new page o Title is not bolded - Includes DOI is available
How should you prepare for a literature search?
- Narrow down your general idea to a specific research question o Find all published information to document and support that question - A good starting point: recently published secondary sources o A textbook, chapter headings, and subheadings will help focus your search on a more narrowly defined area
Name three important reasons to attended conferences
- Networking - Keeping up with regional laws - Keeping up with latest literature
9.09 Accuracy and Use of Data
- Not fabricate data or falsify results - If errors are discovered in their published data, they take steps to correct such errors (retraction, erratum) - Do not omit findings that might alter interpretations - Do not withhold the data on published work
Presenting at Conferences
- Organizers will outline written guidelines to apply to speak at a conference - Incomplete or incomprehensible submissions will be rejected - Must apply before a deadline - It will be evaluated by a panel and you will receive notice of rejection or acceptance
Name five potential sources for research ideas
- Personal interests and curiosities - Casual observation - Reports of others' observations - Practical problems or questions - Behavioral theories
What are five things to keep in mind when finding inspiration for a research topic?
- Pick a topic in which you are interested - Do your homework: collect background information - Keep an open mind - Focus, focus, focus - Take it one step at a time
9.01 Conforming with Laws and Regulations
- Plan and conduct research in a manner consistent with all applicable laws and regulations, as well as professional standards governing the conduct of research. - Comply with other applicable laws and regulations relating to mandated-reporting requirements.
Poster Presentations
- Posters are typically presented during a one- to two-hour poster session that takes place in a large room at the conference site. - Presenters set up their posters around the room and stand near them. - Other researchers then circulate through the room, read the posters, and talk to the presenter - Application guidelines - Format guidelines - Organization should inform if they have poster board, bring pins - Eye catching - Easy to read - Minimal content - Have copies of poster, have business cards
Appendices
- Presents detailed information - Useful, but would interrupt the flow of information if presented in the body of the paper
What is the goal of a research report?
- Provide three kinds of information about the research study o What was done o What was found o How the study is related to other knowledge in the area
Fundamental Characteristics that Allow for Cause & Effect to be Claimed
- Random Assignment: cornerstone of design as it allows equal chance to participate - Manipulation of intervention (IV) - Measurement of outcomes (DV) - Control of extraneous variables: factors that influence the outcome of the study by not being associated with the intervention - Making comparisons between groups: treatment vs. control group
Methods of Authority
- Relies on information or answers from an expert in the field - Is a good starting point and often quickest and easiest to obtain answers
Unavoidable Ethical Conflict
- Researchers must think through the ethical issues raised by their research, mitigate risks o Weigh the risks against the benefits, o Be able to explain their ethical decisions, o Seek feedback about these decisions o Take responsibility for them
Core Principles of Ethics
- Respect of Persons - Beneficence / Concern for Welfare - Justice
What is research ethics?
- Responsibility of researchers to be honest and respectful to all individuals who are affected by their research studies or their reports of the studies' results - Guided by the American (and Canadian!) Psychological Association's, BCBA ethical codes and Tri-Council Policy Statement
Discussion
- Restates the hypothesis, summarizes the results, and then presents a discussion of the interpretation, implications, and possible applications of the results - Begins immediately after the results section—not starting on a new page
What are the purposes of a literature review?
- Show relationships between previous studies or theories - Find out what information already exists in your field of research - Identify gaps in literature - Find other people working in your field - Identify major seminal works - Identify main methodologies and research techniques - Identify main idea, conclusions, and theories and establish similarities and differences - Provide a context for your own research
Divergent Validity
- Showing little or no relationship between the measurements of two different constructs - *Example: Measuring aggression and nothing shows
Face Validity
- Simplest and least scientific - Whether a measure superficially appears to measure what it claims to measure
What are the first two steps to the research process?
- Step 1: find a research idea - Step 2: form a hypothesis
Convergent Validity
- Strong relationship between the scores obtained from two (or more) different methods of measuring the same construct - Example: Many measures used and they all show the same type of data
Informed Consent
- Subjects must be informed of all available information about the study so they can make a rational decision to participate (or not) - Consent should state the they are free to not participate and there will be no ramifications - Consent should state that participants can remove consent at any time
Figures/Tables
- Supplement the text—they should not duplicate information that has already been presented in text form o Should be mentioned in the text by number o The text should point out some of the more important aspects - Each is on a new page
What is science?
- Systematic Empiricism - Empirical Questions - Creates Public Knowledge
Mean is used when:
- The data are normally distributed AND - There are no outliers
Participant Changes
- The participant can change between measurements (e.g., degree of focus) - Example: The student did not eat breakfast before coming to school
Assent
- There is no age of consent in Ontario. (understand the information & ramifications) - Assent (written or verbal) should be secured - The form should include simple declarative statements that describe concretely the main features of the study procedures and the voluntary nature of research participation. - Needs to be written or stated at the child's level
Why is it important to know about research methods?
- To learn how to apply science to the study of human behavior - To understand and interpret others' research - To evaluate claims in everyday media - To discover how to acquire knowledge and make decisions by applying the scientific method
Key Note Speakers
- Try to attend these during bigger conferences - Often there are concurrent sessions
During a literature search, in which ways can irrelevant materials be weeded out?
- Use the title as your first basis for screening - Use the abstract as your second screening device - Look for a link to a full-text version - If the article still looks relevant, then read it carefully - Use references from articles you have already found to expand your literature search
Name four keys to a good presentation
- Visuals - Practice - Don't Read - Watch your clock
Principles of Tri-Council Policy Statement
- Weighing risks against benefits - Acting responsibly and with integrity - Seeking justice - Respecting people's rights and dignity
Which critical evaluation questions should we ask ourselves when evaluating statistics (descriptive and inferential)?
- Were the appropriate statistics used? - Exactly what is significant and what is not? - Are the effects large enough to be meaningful?
Give 4 examples of bases for empirical research questions
- What are some possible causes of the behaviour or characteristic? - What are some possible effects of the behaviour or characteristic? - What types of people might exhibit more or less of the behaviour or characteristic? - What types of situations might elicit more or less of the behaviour or characteristic?
Which critical evaluation question should we ask ourselves when evaluating applications?
- Would other variables affect the results?
matched pair sampling
- a design in which one creates a set of two participants who are highly similar on a key trait and then randomly assigns individuals in the pair to different groups
alternative form reliability
- a form of reliability that evaluates how well a measure correlates with a similar, but different, measure of the same variable - correlation between different version of the scale - statistics: correlation between version 1 and version 2
empty control
- a group that does not receive any form of the treatment and just completes the dependent variable
placebo control
- a group where participants believe they are getting the treatment, but in reality they are not
Likert scale
- a scale where a participant evaluates a series of statements using a set of predetermined responses - responses are summed to represent the overall measurement for the variable - can measure level of agreement, frequency of occurrence, or overall evaluation of something
forced choice scale
- a scale where a person must choose between only two response alternatives for each item - limits sensitivity and encourages people to skip the item if they do not have a strong opinion - basically a yes or no scale, but two alternatives may not be sufficient - solution: add more varied responses
reverse coding
- a scoring strategy where more negative response alternatives are assigned higher numerical values and more positive response alternatives are assigned lower numerical values - used to minimize potential for an acquiescent response set - change the direction of your scale (if 1 is strongly agree and 5 is strongly disagree, 1 would be strongly disagree and 5 would be strongly agree)
Cronbach's alpha
- a statistic used to evaluate internal consistency reliability of a scale - can range from 0 to 1.0
Chi-square
- a statistical test in which both variables are categorical - generally tells us if the distribution of participants across categories is different from what would happen if there were no difference between the groups
one way ANOVA
- a statistical test that determines whether responses from the different conditions are essentially the same or whether the responses from at least one of the conditions differ from the others - compares the means across two or more groups - tells you there's a difference somewhere, but does not specify which groups are different
history
- a threat to internal validity due to an external event potentially influencing participants' behavior during the study - ex: fire drill causes evacuation during the study
instrumentation
- a threat to internal validity due to changes in how a variable is measured during the course of the study - ex: raters may become better or more refined in their observations leading to changes in ratings, differences in how tests are administered (such as paper vs online)
order effect
- a threat to internal validity in a within-subjects design resulting from influence that the sequence of experimental conditions can have on the dependent variable
maturation
- a threat to internal validity stemming from either long-term or short-term physiological changes occurring within the participants that may influence the dependent variable ex: participants may become tired, bored, or hungry as the study progresses
testing effect
- a threat to internal validity where participants' scores may change on subsequent measurements simply because of their increased familiarity with the measurement - ex: if you continue asking the same question participants may begin to guess the purpose, participants may remember earlier answers which could influence current answers
confounding variables
- a variable that the researcher unintentionally varies along with the manipulation
Clear and detailed codebooks
- allows observers to make accurate records - allows you to mediate disagreeing observation - allow others to assess the quality of operationalizations
two-group design
- an experimental design that compares two groups or conditions and is the most basic way to establish cause and effect - also known as simple experiment
random assignment
- any method of placing participants in groups that is unsystematic and unbiased - ensures each participant has an equal chance of being in any group
wording mistake: double-barreled
- asks participants to respond to more than one item using a single number - fix: break down into multiple questions so each one is only asking about a single item
practice effect
- changes in a participant's responses or behavior due to increased experience with the measurement instrument, not the variable under investigation
sensitization effect
- continued exposure to experimental conditions in a within-subjects study increasing the likelihood of hypothesis-guessing, potentially influencing participants' responses in later experimental conditions
fatigue effect
- deterioration in measurements due to participants becoming tired, less attentive, or careless during the course of the study
consequences of intrinsic variation for biological systems
- evolution by natural selection and adaptation - medical treatment/management - ecological management - biodiversity and diversity in ecological function - production in farming and other industries
carryover effect
- exposure to earlier experimental conditions influencing responses to subsequent conditions
extraneous variables
- factor other than the intended treatment that might change the outcome variable - eliminate or control these to ensure the IV is what is actually causing changes in the DV
socially desirable responding
- faking good and alternatively faking bad - giving survey responses that make one appear better (or worse) than is actually the case - Guarantee anonymity and ensure that people know that their responses are anonymous. - past deception in research many problems. If participants don't trust the researcher, then they many not believe that their responses really are anonymous. • Include items to identify socially desirable responding: - My table manners at home are as good as when I eat at a restaurant. - I don't find it particularly difficult to get along with loud- mouthed obnoxious people. - I never hesitate to go out of my way to help someone in trouble. • Perhaps observations or ratings from friends will be more accurate. • Use implicit measures.
How big should a sample be? It depends on 3 things.
- how accurate and precise do you need to be in estimation - in a H test, how small or subtle a relationship you are trying to determine - amount of inherent variation
how to get a precies and accurate representative sample
- identify populations of interest to be sampled - use of unbiased sample selection procedure (random sampling) - sample size
Why does increasing sample size matter?
- improves sample accuracy - improves sample precision
sources or causes of measured variation in biological systems
- intrinsic sources of variation - extrinsic sources of variation
snowball sampling
- non-probability - recruit subjects who then recruit more
convenience sampling
- non-probability - use of readily available subjects
purposive sampling
- non-probability - uses handpicked subjects that meet the researcher's needs
between groups advantages
-Generalizable (more participants are more representative) -collect more data at a given level -shorter time for each participant
stratified sampling
- probability - Ensure certain subgroups are represented or preserve proportions of subgroups in the population within the sample - requires definition of non-overlapping stratification criteria - randomly select in each stratum
Every multi-item scale has an order. What should we do?
- produce multiple versions, then compare results. If there is no difference between versions, then we can have greater confidence in the results. If there is a difference between versions, then we should report both outcome - try to identify a set of questions. (Alternatively, find questions that are not framed inappropriately by earlier questions) - if there is a one "most-important" question on your survey, you may wish to ask that question first
probability sampling methods
- randomization occurs at some point - simple random sampling - systematic sampling - stratified sampling - cluster sampling
Control of extraneous variables
-Have to be known to all participants -everyone gets same experience
Semantic differential format
- scaling for rating a target along several dimensions
posthoc test (Bonferroni correction)
- statistical tests that examine all of the possible combinations of conditions in a way that statistically accounts for the fact that we did not predict them ahead of time - if there is an overall difference, you can use posthoc analyses to compare groups to each other
internal consistency reliability
- the degree to which the individual items on a scale are interrelated or correlated - ask yourself: do the questions measure the same general topic? - statistics: Cronbach's alpha
content validity
- the degree to which the items on a scale reflect the range of material that should be included in a measurement of the target variable - ask yourself: does it assess the topic in a "wholesome" or comprehensive way?
criterion validity
- the extent to which a measurement relates to a particular outcome or behavior - established by evaluating the concurrent and predictive validity of the measurement - ask yourself: does it relate to actual behavior?
construct validity
- the extent to which the scale actually measures the desired construct - established by evaluating the convergent and discriminant validity of the measurement - ask yourself: does it actually measure our construct? - if the scale converges with measurements of similar variables and diverges from measurements of dissimilar variables, it has high construct validity
test-retest reliability
- the temporal stability of a measure - the ability to produce similar measurements across different time periods - statistics: correlation between Time 1 and Time 2 scores
methodological pluralism
- the use of multiple methods or strategies to answer a research question - sometimes it's better to use more than one
wording mistake: jargon
- using jargon that is specific to something that participants may be unfamiliar with (ex: rushing a fraternity) fix: avoid using words or phrases participants may not be familiar with
wording mistake: vocabulary
- vocabulary is complicated or complex - fix: keep it simple
acquiescent response bias
- when a participant tends to agree with most, if not all, of the items on a scale, regardless of what they are asking - participant anticipates questions and answers without reading - solution: reverse code some questions
covariation
- when changes in one variable are associated with changes in another variable - the two variables change together in a systematic way - correlation does not equal causality
temporal precedence
- when changes in the suspected cause (treatment) occur before changes in the effect (outcome) - accomplished by manipulating the IV before measuring the DV
wording mistake: not specific
- wording is not specific and too general - fix: make statements clear and direct about what we are asking
Question Errors
-Accepting a middle choice response when respondents hold a non-neutral opinion -Forcing respondents to choose a position when respondents hold non-neutral opinion -Issue of non-attitudes can be approached by distinguishing among three kinds of attitude questions: standard-format, quasi-filter, and full-filter questions
Mail and Self-Administered Questionnaires
-Advantages: Cheap and can be conducted by a single researcher Anonymity Avoid interviewer bias. -Disadvantages: Low response rates. Cannot control the conditions under which a mail questionnaire is completed People can answer dishonestly -Mail questionnaires are ill-suited for the illiterate
Online Surveys
-Advantages: Very fast and inexpensive. Allow flexible designs Can use visual images and/or audio or video -Disadvantages: Coverage Privacy Verification Design issues
Telephone
-Advantages: 95% of the population can be reached. Interviewers control the sequence of questions and can use some probes. All questions answered. -Disadvantages: More expensive than mail or online. The use of an interviewer reduces anonymity and introduces potential interviewer bias. No visual aids, difficult to use open-ended questions
Face-to-face
-Advantages: Highest response rates and permit the longest questionnaires. The interviewer can observe surroundings and can use nonverbal communication and visual aids. Can ask a range of questions -Disadvantages: The training, travel, supervision, and personnel costs can be high. Interviewer bias is high.
Advantages of experimental design
-Can account for Individual Differences -Eliminate other Confounds -Observe the invisible -Minimize noise
Tri-Council Policy Statement
-Canadian Version of ethics incorporating three different agencies -Published in 1998 - It replaced all previous guidelines that had been developed by individual agencies and institutions. - In 2010 a second edition (TCPS 2) was published that consolidated the core principles
key issues of Between Groups Design
-Creating equivalent groups -> same in every way except the IV -Random Assignment -Matching
Skewed Distribution
-If most cases have lower scores with a few extreme high scores, the mean will be the highest, the median in the middle, and the mode the lowest. -If most cases have higher scores with a few extreme low scores, the mean will be the lowest, the median in the middle, and the mode the highest.
Questions when reading journalism...
-Is original research evidence cited? -Has the research been published in a peer-reviewed journal yet? -How does coverage of the same journal article differ across news stories?
Questionnaire design issues
-Length of Survey or Questionnaire: Length depends on the survey format and on the respondent's characteristics -Question Order and Sequence: organization of the overall questionnaire, question order effects, and context effects. -Organization: A questionnaire has opening, middle, and ending questions. After an introduction explaining the survey, it is best to make opening questions pleasant, interesting, and easy to answer to help a respondent feel comfortable about the questionnaire. -Order Effects: An effect in survey research in which respondents hear some specific questions before others and the earlier questions affect their answers to later questions.
Qualitative questions
-Less structured, nondirective, in-depth interviews - differ from survey research interviews -Insights, feelings, and cooperation are essential parts of a discussion process that reveals subjective meanings
factorial design
-More than one IV -Main Effects and Interactions
Tentative Answers
-Not absolute answers (not proven) -Consider "weight of the evidence": Draw conclusions from entire scientific literature Proportion of studies consistent with theory
Problems with Headlines (Journalism)
-Often oversimplified overconfident -Sometimes misrepresent finding
one-way design
-One independent variable --->Two-group design --->Multiple groups design
key issues of within groups design
-Sequence Effects - scores on the DV are affected by the order in which levels of the IV are presented. --->Carry-over effects --->Practice effects --->Interference effects
Research questions are:
-Specific (is my dog happier after I walk him or after I give him a hug?) -Testable
Empiricism - Systematic
-Specified exact action you're looking to record, and how -Systematic observation typically involves specification of the EXACT actions, attributes, or other variables that are to be recorded and precisely how they are to be recorded.
Median is used when:
-The data are NOT normally distributed OR -There are any outliers (even 1!)
problem with counterbalancing
-Time interval between trials --->Mortality, loss of participants, or changes in participant
Measurement
-Unit of analysis: the individual, a state, etc -Variables: any quality that can be different between analysis e.g time spent reading, athletic involvement, age, race
Qualitative Interviewing
-Unstructured, semi-structured, in depth, ethnographic, open ended, informal, and long -Informal and non directive -A mutual sharing of experiences -Explicit purpose—to learn about the informant and setting (differs from friendly conversation) -Rarely have hypothesis being tested Advantages: 1. Allows the "voice" of the interviewee to be heard 2. Data rich with descriptions and explanations 3. Development of new theories 4. Development of new avenues of research Disadvantages 1. Sample sized are usually small in comparison to those used in quantitative work - question whether the results are actually applicable to a wider population 2. Very rare to archive qualitative data. More and more researchers are sharing so that the data can be scrutinized by others
Aiding Respondent Recall
-When survey involve the recall of past events researchers should provide respondents with special instructions and extra thinking time -Past events: should also provide aids such as a fixed time frame or location references
mixed design
-When there is more than one independent variable --->One or more variables can be within --->One or more variables can be between E.g.: Men and women each take a personality test for the big 5. Gender is the between subjects variable and the five different personality scales are within.
different people
-between groups design -only go through one condition
random assignment
-control for individual differences -creating equivalent groups
Quantitative data:
-data: 1 piece of data -data sets: lots of data connected, e.g homes, prices, size, and years (has codebook) -types of data sets: existing, build your own
purpose of analysis
-descriptive: describe variable -inferential: make prediction based on sample
two types of experimental groups
-different people -same people
within groups advantages
-fewer participants needed -Important for special groups -statistically more powerful (less variability)
between groups disadvantages
-may not be random -unequal N -potential confounds -requires more participants
describing center
-mean (average) -median (middle #) -mode (# that appears most)
two ways to create groups
-naturally occuring -experimental manipulation
within groups disadvantages
-not naïve after first trials -order effects --->practice and fatigue --->non-symmetric or differential transfer
normal distribution curve
-one peak -perfectly symmetrical -median and mean in same spot (center) -50% below avg, 50% above -follows 68 - 95 - 99.7 rule -close to symmetric: enough to fit normal curve
Two types of design
-post-test only -Pretest-posttest
Naturally occurring groups
-quasi-experimental Ex: men and women specific career groups
same people
-within groups design -all participants go through all conditions
Moderate (correlation strength)
.30/-.30
How are samples used for inferences?
1) estimation of values of population parameters from sample summary statistics 2) hypothesis (H) testing
Experiential Avoidance Article: What is the sample? What is the variables? What is the overall results? What is experiential avoidance?
1. 202 undergraduate/ 19.5 years of age/ mainly white females 2. 3. These findings suggest that experiential avoidance is significantly associated with lower delay of gratification both directly and indirectly, through heightened depression and anger 4. the tendency to negatively evaluate and avoid emotion and cognition, tend to report higher levels of distress.
Ways to ensure confidentiality:
1. Anonymous data 2. Code data 3. Obscuring details of a particular case
Things to avoid when writing survey questions
1. Avoid jargon, slang, and abbreviations (e.g., NATO) 2. Avoid ambiguity (What is your income? - does this mean annually, monthly? Family or personally) 3. Avoid double-barrelled questions 4. Avoid leading (or loaded) questions 5. Avoid double negatives
Goals of Debriefing
1. Clarifies the nature of the study 2. Allows the participant to express their feelings about being deceived 3. Removes negative consequence 4. Allows participants to reveal reactions or other comments about the study 5. Allows researcher to convey their appreciation
What types of validity do you need for causal claims?
1. Construct 2. Statistical 3. Three criteria for cause --temporal precedence --covariation --internal
What types of validity do you need for frequency and association claims?
1. Construct 2. Statistical 3. External
What are the first three steps to research?
1. Find an unanswered question that will serve as a research idea 2. Form a research question/hypothesis 3. Use the hypothesis to develop an empirical (observable) research study that will either support or refute the hypothesis
Concurrent validity
A measure of how well a particular test correlates with a previously validated measure.
Two Forms of Statistical Relationships
1. Differences between groups 2. Correlations between quantitative variables
Steps in the Research Process
1. Find a research idea: select a topic and search the literature to find an unanswered question 2. Form a hypothesis 3. Determine how you will define and measure your variables 4. Identify the participants or subjects for the study, decide how they will be selected, and plan for their ethical treatment 5. Select a research strategy 6. Select a research design 7. Conduct the study 8. Evaluate the data 9. Report the results 10. Refine or reformulate your research idea 11. Return to step 2 and continue
Three Types of Claims:
1. Frequency 2. Association 3. Causal
Common Characteristics of Pseudoscience
1. High "success" rates are claimed. 2. Rapid effects are promised. 3. The therapy is said to be effective for many symptoms or disorders. 4. The "theory" behind the therapy contradicts objective knowledge (and sometimes, common sense). 5. The therapy is said to be easy to administer, requiring little training or expertise. 6. Other, proven treatments are said to be unnecessary, inferior, or harmful. 7. Promoters of the therapy are working outside their area of expertise. 8. Promoters benefit financially or otherwise from adoption of the therapy. 9. Testimonials, anecdotes, or personal accounts are offered in support of claims about the therapy's effectiveness, but little or no objective evidence is provided. 10. Catchy, emotionally appealing slogans are used in marketing the therapy. 11. Belief and faith are said to be necessary for the therapy to "work." 12. Skepticism and critical evaluation are said to make the therapy's effects evaporate. 13. Promoters resist objective evaluation and scrutiny of the therapy by others. 14. Negative findings from scientific studies are ignored or dismissed.
Threats to Internal Validity and what each of them mean.
1. History- Any event that occurs between the first and second measurements but is not part of the manipulation. 2. Maturation- People change over time. 3. Testing- Simply taking the pretest changes the participant's behavior 4. Instrument Decay- The basic characteristics of the measuring instrument change over time 5. Regression Toward the Mean- When participants are selected because they score extremely high or extremely low.
Smoking Study Article: What are the IVs? What is the experimental design? What is the overall results?
1. IVs are smokers vs. nonsmokers 2. experimental design is 2x2 mixed (2 levels & 2 IVs) 3. Overall results are smokers were more generous
Kvale
1. Introducing questions 2. Follow-up questions 3. Probing questions 4. Specifying questions 5. Direct questions 6. Indirect questions 7. Structuring questions 8. Interpreting questions 9. Silence
What three things should you take into consideration to form a good research topic?
1. It's a topic that strongly interests you 2. It's creative 3. It is not to broad
Steps to conduct a survey
1. Keep it clear 2. Keep it simple 3. Keep the respondent's perspective in mind A researcher would not know if differences in respondents accounted for different answers. A good questionnaire forms an integrated whole (e.g., instructions for clarification and measures each variable with one or more survey questions)
Seven Concerns of the IRB:
1. Lack of Debriefing 2. Breach of Confidentiality 3. Lack of Informed Consent 4. Invasion of Privacy 5. Coercion 6. Deception 7. Harm to the Participants
Justifications for Deception
1. MUST INCLUDE A DEBRIEFING SESSION 2. Study cannot be completed otherwise 3. Cannot deceive participants about the study that might sway their willingness to participate 4. Cannot deceive about risks or possible discomfort
Principle of Respect for Persons
1. Must treat everyone as autonomous beings (can make their own decisions --informed consent --cannot mislead about risks --cannot coerce 2. Those who are not autonomous must have special protections --children, disabled, prisoners
4 levels of measurement
1. Nominal: name/label, no rank or value. e.g your major at college (categorical variables) 2. Ordinal: name/ label, has rank or value. e.g MU status, 1st year student (categorical variables) 3. Interval: numbers have rank, value. no real 0 value. e.g height is 6'2 4. Ratio: have true 0 value, 0 has meaning. e.g voted 0 times
Steps of the Scientific Method
1. Observe behaviour or other phenomena 2. Form a tentative hypothesis to generate a testable prediction 3. Use your hypothesis to generate a testable prediction 4. Evaluate the prediction by making systematic and planned observations 5. Use the observations to support, refute, or refine the original hypothesis
Principle of Beneficience
1. Protect the participants from harm 2. Cannot withhold beneficial treatment --control groups must eventually receive treatment --must implement new, innovative treatments even if your study has already begun --must inform about risks of waiting if in placebo
Who are the three categories of people who should be considered when making ethical decisions?
1. Research Participants 2. Scientific Community 3. Society
What are the three key ethical principles?
1. Respect for persons 2. Beneficence 3. Justice
What are the steps to submitting a manuscript for publication?
1. Select a journal that is appropriate for the topic of your research report 2. Consult the journal's Instructions to Authors for specific submission requirements 3. Enclose a cover letter to the journal editor along with the manuscript
What are two ways you can test for inter-item reliability?
1. Split-half reliability 2. Item total correlation
How do you increase your reliability?
1. Standardize the administration of the measure 2. Clarify instructions and questions 3. Train observers 4. Minimize coding errors
What are the two phases for participating in a conference presentation?
1. Submit a written summary or abstract to the conference organizers for approval 2. Give an oral presentation: typically involves a PowerPoint presentation (10 to 20 minutes) or Present a poster: guidelines are provided by the organization for style and requirements
3 Conditions for Causality
1. Temporal Order 2. Association 3. No Alternative Explanation
Sources of Ethical Directives:
1. The Belmont Report 2. APA Ethical Principles 3. APA Ethical Standards
Sources of Measurement Error
1. Transient States of the Participant 2. Stable Attributes of the Participant 3. Situational Factors 4. Characteristics of the Measurement 5. Experimenter Error
7 Parts of True Experiment
1. Treatment of independent variable 2. Dependent variable 3. Pretest 4. Post-test 5. Experimental group 6. Control group 7. Random assignment Note: not all experiments have all 7 parts
An informed consent form must include:
1. Why is the researcher doing this study? 2. What will the participant be doing? 3. Are there any risks involved? 4. Consequences of refusal/withdrawal 5. Confidentiality assurance 6. Does the participant have any questions? 7. How does the participant contact the researcher afterwards? 8. Signature lines
4 methods of survey administration
1. face to face 2. written 3. computerized 4. telephone
Three main principles (The Belmont Report)
1. principle of respect for persons 2. principle of beneficience 3 principle of justice
Researchers have two obligations:
1. protect the rights of the participants 2. to further scientific research
4 steps to calculate variance
1. subtract the mean from each score 2. square each of the deviation scores 3. add the squared deviation scores 4. divide the sum of the squared deviations by the number of scores
Marital Ages Article: What is operational sex ratio? What is female bias OSR? What is male bias OSR? What is OSR of 100? What is the results? How was the data obtained?
1. the number of sexually active males per 100 sexually receptive females 2. Where the OSR is female biased, males tend to pursue short-term mating as long as possible 3. When the OSR is male biased, available men outnumber available women and the greater degree of female choice will increase the male social status and resource potential necessary for securing female partners. 4. if the OSR is of 100, the population is balanced between the sexes 5. As predicted, when women are scarce they marry earlier on average and the variance in male marital age increased, however there was no significant relationship with male marital ages , though the trend was for earlier marriages. 6. 50 largest Metropolitan Statistical Areas (MSAs) in the United States with year 2000 Census data
What constitutes deception?
1. using confederates 2. providing false feedback 3. presenting related studies as unrelated 4. lying about stimulus materials
a trial finds a 6.7% cure rate for drug A and a 4.7% cure rate for drug b. what is the number needed to treat with drug a instead of drug b to produce one additional cure?
1/(0.067-0.047)= 50
How long must data be kept from a study?
10 years OR 10 years following the 18th birthday of the participant
What is an acceptable reliability score?
>.70
pilot study
: before conducting the "real" experiment, a researcher may test the procedure on a small sample to confirm the effectiveness of manipulations, make sure everything runs smoothly, etc
According to the textbook, what is the problem with WEIRD samples? They are not very representative of the world's population. They are not good for theory-testing mode. They are not used frequently enough. They are not very easy to measure.
A
Dr. Gavin is conducting a 2 x 4 independent-groups factorial design. How many interactions will Dr. Gavin need to examine? 1 2 3 4
A
In popular media articles, why does the inclusion of participant variable information often denote a factorial design? Participant variables are often used as moderators. Participant variables are only found in factorial designs. Studies with participant variables always find significant interactions. Studies with participant variables are easier to write about.
A
Seeing stability in a stable-baseline design can help rule out which of the following threats to internal validity? Regression to the mean Placebo effects Attrition Observer bias
A
The degree to which a quasi-experiment supports a causal claim depends on which of the following? Its design and its results Its duration and its sample size Its importance and its external validity Its statistical significance and its practical significance
A
The mathematical way to describe an interaction is: A difference in differences A caveat A qualified main effect A patterned pattern
A
The text states that the first step in establishing a study's importance is to establish its _________. replicability external validity internal validity acceptance by scientists
A
Which of the following is a difference between true experiments and quasi-experiments? Quasi-experiments do not use random assignment. Quasi-experiments do not involve manipulated variables. Quasi-experiments cannot have comparison groups. Quasi-experiments cannot have pretest measures.
A
Which of the following is a within-groups quasi-experimental design? Interrupted time-series design Nonequivalent control group design Matched group factorial design Multiple regression design
A
Which of the following is another term for ecological validity? Mundane realism Everyday realism Experimental realism Cultural realism
A
Constant
A "thing" that does not vary. It only has one level in the current context but it could potentially vary.
Constant
A "thing" that does not vary. It only has one level. Usually something that you are NOT interested in either measuring or manipulating
Variable
A "thing" that varies. It must have at least two levels.
Variables
A "thing" that varies. It must have at least two levels. Usually what you are interested in measuring or manipulating
3. _____ refers to the average of the dependent variable for participants with a specific combination of the levels of the independent variables A) Cell mean B) Marginal mean C) Main effect D) Crossover interaction
A - cell mean
1. is a prediction that focuses on one independent variable A) A main effect hypothesis B) An interaction effect hypothesis C) A suppression effect D) A crossover interaction
A - main effect hypothesis
Verstehen
A German word that translates as understanding; specifically, it means an empathetic understanding of another's worldview
theory introduction
A _______ is a statement, or set of statements, that attempts to explain the social world.
pretest-posttest
A ____________ design can show how people change over time as a result of some manipulation.
Proposition
A basic statement in social theory that two ideas or variables are related to one another. It can be true or false, conditional, and/or causal
Social Desirability Bias
A bias in survey research in which respondents give a normative response or a socially acceptable answer rather than giving a truthful answer
Cultural psychology
A branch of psychology that mainly focuses on the way cultural contexts shape the way a person feels, thinks, acts, and behaves.
Maturation (spontaneous remission) threat
A change in behavior that changes more or less over time. This can be prevented by including a comparison group- conducting a true experiment
Parameter
A characteristic of the entire population that is estimated from a sample ie. The percentage of city residents who smoke cigarettes
Association
A co-occurrence of two events, factors, characteristics, or activities, such that when one happens, the other is likely to occur as well. Many statistics measure this.
Concept Cluster
A collection of interrelated ideas that share common assumptions, belong to the same larger social theory, and refer to one another
Cells
A condition in an experiment that represents one of the possible combinations of two independent variables.
Strength of Correlation
A correlation above .70 is very strong A correlation above .40 is strong Lower than .20 is no correlation
Consistent Positive Relationship
A correlation near +1.00
Consistent Negative Relationship
A correlation near -1.00
Inconsistent Relationship
A correlation near zero
positive correlation
A correlation where as one variable increases, the other also increases, or as one decreases so does the other. Both variables move in the same direction.
Blame Analysis
A counterfeit argument presented as if it were a theoretical explanation that substitutes attributing for a causal explanation and implies an intention or negligence, or responsibility for an event or situation
Case study
A descriptive account of the behavior, past history, and other relevant factors concerning a specific individual.
naturalistic observation
A descriptive research method in which the researcher engages in careful observation of behavior.
case study
A descriptive technique in which one individual or group is studied in depth in the hope of revealing universal principles.
Mixed factorial design
A design that includes both independent groups (between-subjects) and repeated measures (within-subjects) variables.
Scatterplot
A diagram to display the statistical relationship between two variables based on plotting each other's values for both of the variables. -Used for variables measured at the interval or ratio level, rarely for ordinal variables, and never if either variable is nominal. Usually, the independent variable (symbolized by the letter X) goes on the horizontal axis and the dependent variable (symbolized by Y) goes on the vertical axis. -Determines: form, direction, and precision
Interaction effect
A difference of differences. A result in a factorial design in which the difference in one level of the independent variable changes depending on the other level of the independent variable. In a factorial design with two independent variables, the first two results obtained are the main effects for each independent variable. The third result is the interaction effect. Whereas the main effects are simple differences, the interaction effect is the difference in differences.
Pie Chart
A display of numerical information on one variable that divides a circle into fractions by lines representing the proportion of cases in the variable's attributes.
Bar Chart
A display of quantitative data for one variable in the form of rectangles where longer rectangles indicate more cases in the variable category. Usually, it is used discrete data, and there is a small space between rectangles.
Sampling Distribution
A distribution created by drawing many random samples from the same population
Sampling Distribution of Sample Means
A distribution of sample means created by drawing many random samples from the same population ie. Age - a continuous variable • Random sample of 30 - calculate the average age of students • Sample 1: average age = 21.63, Sample 2: Average age=20.38 • Repeat procedure until 12 different samples of 30 students have been drawn
Codebook
A document describing the coding procedure and the location of data for variables in a format that computers can use
IV x PV design
A factorial design that includes both an experimental independent variable (IV) and a nonexperimental participant variable (PV).
Pseudoscience
A fake or false science that makes claims based on little or no scientific evidence.
hypothesis
A formal statement that predicts how a change in one or more variables might cause a change in another variable is called a(n):
Normal Distribution
A function that represents the distribution of variables as a symmetrical bell-shaped graph.
Normal distribution
A function that represents the distribution of variables as a symmetrical bell-shaped graph.
Paradigm
A general organizing framework for social theory and empirical research. It includes basic assumptions, major questions to be answered, models of good research practice and theory, and methods for finding the answers to questions
Deductive Reasoning
A general statement is the basis for reaching a conclusion about specific examples
Descriptive Statistics
A general type of simple statistics used by researchers to describe basic patterns in the (numerical) data. They can be categorized by the number of variables involved: univariate, or multivariate
True
A hypothesis is a statement that predicts the answer to our research question. T/F
Identify the object or event that is to be observed, and identify any factors (variables) that might affect the observation
A hypothesis is expressed as a statement and must be capable of being tested. Before writing your hypothesis you must do what 2 things?
Central Limit Theorem
A law-like mathematical relationship stating that whenever many random samples are drawn from a population and plotted, a normal distribution is formed, and the centre of such a distribution for a variable is equal to its population parameter
Sampling Frame
A list of cases in a population, or the best approximation of it. Crucial to good sampling, size less crucial than whether it accurately represents population
Random Number Table
A list of numbers that has no pattern in it and is used to create a random process for selecting cases and other randomization purposes
Interview Guide
A list of questions a researcher wishes to address in the course of a qualitative interview
Manipulation check
A measure used to determine whether the manipulation of the independent variable has had its intended effect on a subject.
Matched pairs design
A method of assigning subjects to groups in which pairs of subjects are first matched on some characteristic and then individually assigned randomly to groups.
Counterbalancing
A method of controlling for order effects in a repeated measures design by either including all orders of treatment presentation or randomly determining the order for each subject.
Survey research
A method of collecting information by asking questions.
Curvilinear Correlation
A non-linear relationship between co-variables
Correlation Coefficient
A numerical index of the degree of relationship between two variables.
Response set
A pattern of response to questions on a self report measure that is not related to the content of the questions.
Confederate
A person posing as a participant in an experiment who is actually part of the experiment.
typology
A police officer develops a way to classify her colleagues in terms of their professionalism on the job. The categories are career officer, work-a-day officer, and slacker officer. This measure is best described as a:
Stratified random sampling
A probability sampling method in which a population is divided into subpopulation groups called strata; individuals are then randomly sampled from each of the strata.
Cluster sampling
A probability sampling method in which existing groups or geographic areas, called clusters, are identified. Clusters are randomly sampled and then everyone in the selected clusters participates in the study.
Reactivity
A problem of measurement in which the measure changes the behavior being observed.
Carryover effect
A problem that may occur in repeated measures designs if the effects of one treatment are still present when the next treatment is given.
Random Assignment
A procedure in which all subjects in a study have an equal chance of being assigned to any group or condition.
Double-blind experiment
A procedure wherein both the experimenter and the participant are unaware of whether the participant is in the experimental (treatment) or the control condition
Ideal Type
A pure model about an idea, process, or event. One develops it to think about it more clearly and systematically. It is used both as a method of qualitative data analysis and in social theory building
Field Research
A qualitative style in which a researcher directly observes and (usually) participates in small-scale social settings (distinct from pure qualitative) (ex. barber shop, daycare, etc.) -Used when other methods are not feasible
Categorical Variable
A quality, such as chosen major, and is typically measured by assigning a category label to each individual
Survey Research
A quantitative social research technique in which one systematically asks many people the same questions, then records and analyzes their answers
Variables
A quantity or quality which varies across people or situations
Quantitative Variable
A quantity, such as height, that is typically measured by assigning a number
Matched groups
A quasi-experimental design technique used to group participants into sets based on the similarity of a measured variable. Those coordinated sets are then randomly assigned to different conditions. This is used to control for selection effects.
Empirical Generalization
A quasi-theoretical statement that summarizes findings or regularities in empirical evidence. It uses few if any abstract concepts and only makes a statement about a recurring pattern that researchers observe
Leading (or loaded) questions
A question that leads the respondent to choose one response over another by its wording
Confidence Interval
A range of values, usually a little higher and lower than a specific value found in a sample, within which a researcher has a specified and high degree of confidence that the population parameter lies. High level of confidence (e.g., 95%) that the true population parameter lies with a certain range
Negative linear relationship
A relationship in which increases in the values of the first variable are accompanied by decreases in the values of the second variable.
Linear Relationship
A relationship of direct proportionality that, when plotted on a graph, traces a straight line
Split-half reliability
A reliability coefficient determined by the correlation between scores on half of the items on a measure with scores on the other half of a measure.
Test-retest reliability
A reliability coefficient determined by the correlation between scores on a measure given at one time with scores on the same measure given at a later time.
Experiment
A research method in which an investigator manipulates one or more factors to observe the effect on some behavior or mental process.
Direct replication
A researcher closely repeats an original study to see if the same effect shows up in the new data
Disclosing
A researcher decides how much to reveal about himself or herself and the research project. Disclosure ranges on a continuum
stratified random sampling
A researcher divides the residents of a low-income housing project into two groups: those age 30 and under, and those over age 30. Ten people from each group are randomly selected for interviews. This probability sampling technique is best defined as:
simple random sampling
A researcher draws names out of a hat to see which police officers in a particular precinct will be interviewed for a study on workplace satisfaction. This probability sampling technique is best defined as:
Only the children who participated in every phase of the study should be analyzed.
A researcher is conducting a longitudinal study involving repeated measurements of children as they grow up. Some of the children drop out of the study before it is completed. What should the researcher do about this?
random sampling error
A researcher is interested in whether participants in a drug treatment program are successful in their rehabilitation. He asks for volunteers to participate in a long-term study of post-treatment success. Twenty of the 130 participants volunteer to participate in the four-year follow up study. The bias in this sampling plan is best described as:
Observational/behavioral Measurement
A researcher records observable behaviors (including behavioral indicators of internal states) Observe what they ACTUALLY do Watch someone all day and count number of drinks Observe actual accuracy or speed on tasks Give a stats test and see how many they get right
one-group, pretest/posttest design
A researcher takes one group of participants, measures them before and after the treatment is given.
Yea-saying and nay-saying
A respondent may employ a response set to agree or disagree with all the question
Simple random sampling
A sampling procedure in which each member of the population has an equal probability of being included in the sample.
Quota sampling
A sampling procedure in which the sample is chosen to reflect the numerical composition of various subgroups in the population. A haphazard sampling technique is used to obtain the sample.
Interval scale
A scale of measurement in which the intervals between numbers on the scale are all equal in size.
Ordinal scale
A scale of measurement in which the measurement categories form a rank order along a continuum.
Ratio scale
A scale of measurement in which there is an absolute zero point, indicating an absence of the variable being measured.
Nominal scale
A scale of measurement with two or more categories that have no numerical (less than, greater than) properties
benchmarking.
A scientific technique that allows a researcher to learn something about a population by studying a few members of that population is called:
Scientific literature
A series of related studies that are conducted by various researchers and have tested similar variables.
Statistic
A set of collected numbers (e.g., numbers telling how many people live in a city) as well as a branch of applied mathematics used to manipulate and summarize the features of numbers
Coding System
A set of rules used to categorize observations
Theory
A set of statements or principles devised to explain a group of facts or phenomena
Reversal designs
A small-N study where the researcher observes behavior both before and during the administration of the independent variable/treatment, then takes away the treatment for a period of time to see if the behavior returns to the baseline. If the participant's behavior changes back, the independent variable/treatment was effective to the dependent variable. When this occurs, the researcher is able to test for internal validity and make a causal statement. This design is used to examine treatments that are not suspected to have long-term impact.
Pilot study
A small-scale study conducted prior to conducting an actual experiment; designed to test and refine procedures.
Sample
A smaller set of cases a researcher selects from a larger pool and generalizes to the population
Focus Group
A special qualitative research technique in which people are informally "interviewed" in a group-discussion setting -Popular way for market researchers to test their products -Topics might include public attitudes (e.g., race relations, work- place equality), personal behaviours (e.g., avoiding sexually transmitted infections), a new product (e.g., breakfast cereal), a political candidate -Includes moderator
single subject design
A special type of within-subjects designs using one participant (human or animal) or perhaps one group to assess changes within that individual or group. examples: A-B Design - baseline, intervention A-B-A Design - baseline, intervention, no intervention A-B-A-B Design - baseline, intervention, no intervention, intervention
Prediction
A statement about something that is likely to occur in the future
Causal Explanation
A statement in social theory about why events occur that is expressed in therms of causes and effects. They correspond to associations in the empirical world
Hypothesis
A statement that describes or explains a relationship between variables that can lead to several different observable and measurable predictions
Small-N design
A study in which researchers obtain a large amount of information from a few small cases. Each participant is treated as a separate experiment that is almost always a repeated-measures design.
Factorial designs
A study in which there are two or more independent variables. This is the best way to study how combined variables interact. Increases the number of comparison groups
Replicability
A study's potential to have its results replicated.
Ecological validity
A study's similarity to real-world contexts
Frequency Distribution
A summary chart, showing how frequently each of the various scores in a set of data occurs. • It can be used with nominal, ordinal, interval, or ratio-level data and takes many forms: Raw Count, Bar Chart, Grouped Data Frequency, and Frequency Polygon
Theory
A system of interrelated ideas used to explain a set of observations.
Frequency Table
A table for organizing a set of data that shows the number of times each item or number appears. and then find percentage
Participant observation
A technique of observing a situation wherein the observer takes an active role in the situation.
Attitude of Strangeness
A technique in field research in which researchers study a field site by mentally adjusting to "see" it for the first time or as an outsider. E.g., gift giving customs
Latin square
A technique to control for order effects without having all possible orders or to counterbalance.
Theoretical Saturation
A term associated with the grounded theory approach that refers to the point at which no new themes emerge from the data and sampling is considered complete
Science of the Sophomore
A term used to refer to the potentially limited external validity of studies based on undergraduate samples, usually using the experimental method
Hypothesis
A testable prediction, often implied by a theory.
History Effects
A threat to internal validity due to something that occurs and affects the dependent variable during an experiment, but that is unplanned and outside the control of the experimenter
Mortality
A threat to internal validity due to subjects failing to participate through the entire experiment
Maturation
A threat to internal validity in experimental research due to natural processes of growth, boredom, and so on that occur to subjects during the experiment and affect the dependent variable
Experimenter Expectancy
A threat to internal validity that occurs when the treatment "spills over" from the experimental group, and control group subjects modify their behaviour because they learn of the treatment
Diffusion of Treatment/Contamination
A threat to internal validity that occurs when the treatment "spills over" from the experimental group, and control group subjects modify their behaviour because they learn of the treatment. (ie. chatting during break)
Selection Bias
A threat to internal validity when groups in an experiment are not equivalent at the beginning of the experiment
Questionnaire
A tool used in survey research consisting of a list of questions asked to a respondent
Posttest-only design
A true experimental design in which the dependent variable (posttest) is measured only once, after manipulation of the independent variable.
Bivariate
A two-variable analysis, such as a crosstabulation of disease incidence by age or a comparison of mean health care costs by sex
Psychobiography
A type of case study in which the life of an individual is analyzed using psychological theory.
Purposive sampling
A type of haphazard sample conducted to obtain predetermined types of individuals for the sample
Histogram
A type of bar chart used to visually display the distribution of a continuous variable
Third-variable problem
A type of confounding variable in which a third variable leads to a mistaken causal relationship between two others.
Pearson product-moment correlation
A type of correlation coefficient used with interval and ratio scale data. In addition to providing information on the strength of relationship between two variables, it indicates the direction (positive or negative) of the relationship.
Budgetary impact analysis
A type of decision analytic model that estimates the impact of a change in treatment or policy on overall expenditures
Sequential Sampling
A type of non-random sample in which a researcher tries to find as many relevant cases as possible until there is no new information or diversity from the cases. Similar to purposive with one difference: in this sampling, a researcher continues to gather cases until the amount of new information or diversity of cases is filled, whereas in purposive the goal is to get every possible case until resources are exhausted. ie. Interviews with 60 widows over 70 years of age who have been living without a spouse for more than 10 years
Snowball Sampling
A type of non-random sample in which the researcher begins with one case then, based on information about interrelationships from that case, identifies other cases, and then repeats the process again and again. Multi-stage technique ie. heroin users
Quota Sampling
A type of non-random sample in which the researcher first identifies general categories into which cases or people will be selected, then he or she selects a predetermined number of cases in each category. ie. male and female, under the age of 30, 30 to 60 etc.
Haphazard Sampling
A type of non-random sample in which the researcher selects anyone he or she happens to come across. Cheap, quick, but ineffective and unrepresentative ie. street interview or show asking caller to vote online
Purposive Sampling
A type of non-random sample in which the researcher uses a wide range of methods to locate all possible cases of a highly specific and difficult-to-reach population Appropriate for 3 situations: 1. Unique cases (specific trendy women's magazine) 2. Members of a difficult to reach, specialized population (sex workers) 3. Researcher wants to identify particular type of case for in-depth investigation
Deviant Case Sampling
A type of non-random sample, especially used by qualitative researchers, in which a qualitative researcher selects unusual or nonconforming cases purposely as a way to provide greater insight into social processes or a setting. ie. studying high school dropouts who are from stable two-parent, upper-middle- income families who are geographically stable and well educated
Simple Random Sampling
A type of random sample in which a researcher creates a sampling frame and uses a pure random process to select cases. Each sampling element in the population will have an equal probability of being selected
Systematic Sampling
A type of random sample in which a researcher selects every kth (e.g., 12th) case in the sampling frame using a sampling interval. Simple random sampling with a short-cut for random selection. Often yields similar results. ie. Sample 300 names from 900. After a random starting point, you select every third name of the 900 and get a sample of 300. Sampling interval is 3
Stratified Sampling
A type of random sample in which the researcher first identifies a set of mutually exclusive and exhaustive categories, then uses a random selection method to select cases for each category. More representative. Researcher controls relative size.
Cluster Sampling
A type of random sample that uses multiple stages and is often used to cover wide geographic areas in which aggregated units are randomly selected; samples are then drawn from the sampled aggregated units, or clusters. Involves multiple stages and clusters. Less expensive than random sampling, but less accurate (each stage has sampling errors). Researcher first samples clusters, and then draws a second sample from within the clusters selected • Advantage when dealing with geographically dispersed populations • A design with more clusters is better because elements within clusters (e.g., people living on the same block) tend to be similar to each other (e.g., more alike than those on different blocks)
Random Sample
A type of sample in which the researcher uses a random- number table or similar mathematical random process so that each sampling element in the population will have an equal probability of being selected
Non-random Sample
A type of sample in which the sampling elements are selected using something other than a mathematically random process
Functional Theory
A type of social theory based on biological analogies, in which the social world or its parts are seen as systems, with its parts serving the needs of the system
Multivariate
A type of statistical analysis in which the effects of multiple variables are assessed simultaneously, such as an analysis of health care cost controlling for age, sex, comorbidities, and inclusion in the intervention versus comparison group of a study
Full-filter Question
A type of survey research question in which respondents are first asked whether they have an opinion or know about a topic, then only the respondents with an opinion or knowledge are asked a specific question on the topic (ie. Do you have an opinion on that? If yes, do you agree or disagree?)
Open-ended Questions
A type of survey research question in which respondents are free to offer any answer they wish to the question
Threatening Questions
A type of survey research question in which respondents are likely to cover up or lie about their true behaviour or beliefs because they fear a loss of self-image or may appear to be engaging in undesirable or deviant behaviour -Use only after a warm-up when an interviewer has developed a rapport and trust with respondents
Closed-ended Questions
A type of survey research question in which respondents must choose from a fixed set of answers -Large-scale surveys use these because they are quicker and easier for both the respondents and researchers -Help to reduce respondent fatigue -Sensitive topics (e.g., liquor consumption) may be more accurately measured
Standard-format Question
A type of survey research question in which the answer categories fail to include "no opinion" or "don't know"
Quasi-filter Question
A type of survey research question including the answer choice "no opinion" or "don't know"
True
A unit of analysis can be an individual, a group, a community, a state, or even an entire nation. T/F
Cluster
A unit that contains final sampling elements
dependent variable
A variable (often denoted by y ) whose value depends on that of another.
Manipulated Variable
A variable a researcher controls, usually by assigning participants to a particular level of the variable
independent variable
A variable that causes change to another variable is called a(n):
dependent variable
A variable that changes as a result of a change in another variable is called a(n):
intervening variable
A variable that occurs between an independent and dependent variable and affects the relationship between them is called a(n):
Measured Variable
A variable whose levels are simply observed and recorded. Typically accomplished using scales, inventories, rulers, or devices
Participant variables
A variable with levels that are selected and measured. (ex: age, gender, ethnicity)
traditional view of research participant
A view where participant contributes behavior to the experiment like urine is contributed to a medical experiment. Very objective.
Meta-analysis
A way to average the results of all of the studies that tested for the same variables to come to a conclusion that the whole body of evidence supports.
Nonequivalent control group interrupted time-series design
A within-groups quasi-experimental design that has two or more groups in which participants have not been randomly assigned (treatment group and comparison group), and are repeatedly measured on a dependent variable before, during, and after the interruption caused by some event- the timing of this event often differs among the groups.
Interrupted time-series design
A within-groups quasi-experimental design that repeatedly measures participants (in the singular treatment group), on a dependent variable before, during, and after the interruption caused by some event.
10. A double-blind procedure is used in an effort to reduce the likelihood that _____ of condition will influence the results; therefore, both the participants and the administrators of treatment are unaware of which treatments are being provided. A) the expectancies B) the inexperience C) the unfamiliarity D) the foreignness
A- the expectancies
8. _____ is an empty control group often used in clinical research. A) A waiting-list control group B) A placebo control group C) An intervention group D) A research participant pool
A- waiting list
Deductive Approach
An approach to inquiry or social theory in which one begins with abstract ideas and principles then works toward concrete, empirical evidence to test the ideas
OR equation
AD/BC
Which format should research reports be written in?
APA
Scale sensitivity
Ability of the scale to detect differences. 3 numbers lack sensitivity; more than 7 provides meaningless sensitivity.
Strength of Experimental Method
Ability to demonstrate causation by observation. In other methods causality is implied
Sections of a Journal Article
Abstract Introduction Methods Results Discussion References
What Each Section Contains or Does for the Reader
Abstract - provides essence of research Method - who, what and how section Results - report important findings and statistics/themes. Present what is being assessed, demographics, hypotheses. Discussion - interprets results for reader. Does not restate results in detail, only if needed.
Quasi-Experimental Designs
Addresses the need to study the effect of an independent variable in settings in which the control features of true experimental designs cannot be achieved.
Primary Criteria to Determine Suitability
Adequacy Utility Feasibility Propriety
How to Avoid Type I Error
Adjust alpha level to a more stringent level. Use two-tailed instead of a one-tailed. Bonferroni adjustment: dividing the alpha level (.05) by the number of statistical tests performed to create a new and more stringent alpha level.
graph
After all trials have been carried out and data has been recorded in the Data Chart, making a _________ will further organize data and help you to see trends and patterns.
research
After deciding on the problem, you must do _______________ to find out what is already known about your topic.
Simultaneous Measurements - Inter-Rater Reliability
Agreement between two observers who simultaneously record measurements of the behaviors
Management-Oriented
Aid leaders in making decisions about a program. Often looks at context, input (resources), process & product.
Behavioural Couples Therapy is often used for what psychological disorders?
Alcoholism and Substance Abuse
variables
All factors affecting observations and results in an experiment are ___________________.
constants
All other factors (except the two above) that could possibly affect the outcome of the experiment and must be kept the same are called _____________.
What is the purpose of a methods section?
Allows individuals reading the paper to replicate the study as well as to identify flaws and potential areas of improvement for future research
Name three things that should be avoided when writing in APA format
Ambiguity, Changes in topic, and Colloquial expressions
Idiographic
An approach that focuses on creating detailed descriptions of specific events in particular time periods and settings. It rarely goes beyond empirical generalizations to abstract social theory or causal laws
Ethnography
An approach to field research that emphasizes providing a very detailed description of a different culture from the viewpoint of an insider in that culture to permit a greater understanding of it -Considered a methodology rather than a method
Context Effect
An effect in survey research when an overall tone or set of topics heard by a respondent affects how he or she interprets the meaning of subsequent questions
Hawthorne Effect
An effect of reactivity named after a famous case in which subjects reacted to the fact that they were in an experiment more than they reacted to the treatment
Hypothetical Construct
An entity that we know exists in each individual, but is *intangible* (cannot be overtly observed). Must define the construct using some tangible metric
Margin of Error
An estimate about the amount of sampling error that exists in a survey's results
Field experiment
An experiment that is conducted in a natural setting rather than in a laboratory setting.
Experimental realism
An experiment in a laboratory is made to create a setting in which people are made to feel authentic emotions, motivations, and behaviors.
Field Experiment
An experiment set up in the real world, usually with participants who are not aware that they are in a study of any kind. -Less control over experimental condition -Greater external validity, lower internal validity
Solomon four-group design
An experimental design in which subjects are randomly assigned to two control groups and two experimental groups. Only one experimental group and one control group receive a pretest. All four groups receive a post-test. Researcher believes that the pretest has an influence on the treatment
Classical Experimental Design
An experimental design that has random assignment, a control group, an experimental group, and a pretest and post-test for each group
Double-blind placebo control strudy
An experimental design where the participants nor the observers for the study know which group is receiving the treatment or the placebo treatment.
One-shot case study design
An experimental design with only an experimental group and a post-test, no pretest Disadvantages: does not take into account external factors, findings may not generalize, may falsely assume an effect when none exists
Static Group Comparison
An experimental design with two groups, no random assignment, and only a post-test. Also called the post-test-only nonequivalent group design
demand characterisitc
An experimenter is interested in whether people will rate food as tastier when it's served on matching plates and utensils, compared to mismatched utensils. However, it's a repeated-measures design, and some participants realize that they are "expected" to like the food better in the matching condition. What kind of confound is this?
manipulation check
An extra dependent variable researchers can include to determine how well an experimental manipulation worked.
Praxis
An idea in critical social science that social theory and everyday practice interact or work together, mutually affecting one another. This interaction can promote social change
A
An independent-groups design is also known as a _________ design. between-subjects matched-groups within-groups mixed
nonequivalent control group pretest/posttest design
An independent-groups, quasi-experimental, pretest and posttest design that has at least one treatment group and one comparison group, but participants have not been randomly assigned to the two groups.
nonequivalent control group design
An independent-groups, quasi-experimental, pretest only design that has at least one treatment group and one comparison group, but participants that have not been randomly assigned to the two groups.
Correlation coefficient
An index of how strongly two variables are related to each other.
Cronbach's alpha
An indicator of internal consistency reliability assessed by examining the average correlation of each item (question) in a measure with every other question
Interrater reliability
An indicator of reliability that examines the agreement of observations made by two or more raters (judges).
Confidence interval
An interval of values within which there is a given level of confidence (e.g., 95%) where the population value lies.
Focus group
An interview with a group of about 6 to 10 individuals brought together for a period of usually 2-3 hours.
The role of the survey interviewer
An interviewer can increase the response rate by doing the "work" involved in the survey. They can also observe the respondent and the setting within which the interview takes place (typically the respondent's house). Of course, the presence of an interviewer may bias the respondent's answers on particularly sensitive questions.
Theoretical Sampling
An iterative sampling technique associated with the grounded theory approach in which the sample size is determined when the data reach theoretical saturation -A researcher does not know in advance how many individuals he or she needs to interview
confound
An unaccounted-for variable in an experiment that provides an alternative explanation for an effect.
Intention to treat versus per protocol
Analytical approach in which the outcomes for all those who are assigned to the treatment or intervention of interests are included in the analysis in contrast with a per protocol approach in which only those who successfully complete the trail are analyzed
statistical power you don't need as many participants in a within-groups study to find a significant effect.
Another advantage of within-groups design is increased This means:
Variable
Any event, situation, behavior, or individual characteristic that varies—that is, has at least two values.
Covariance (3 criteria for causal claims)
Are x and y related?
Alternate forms reliability
Assessment of reliability by administering two different forms of the same measure to the same individuals at two points in time.
Limitations to the Rational Method
Assumes that the premise statements are true, sometimes the argument does not logically support the conclusion
What is considered minimal risk?
At no greater risk for harm than you would be doing an everyday task
If a researcher is concerned about external validity, which of the following would you recommend with regard to conducting small-N designs? Do not conduct small-N designs if you are concerned about external validity. Compare the results of a small-N design with other studies. Conduct only reversal designs. Use only one's own clients/patients/students.
B
Imagine that you are reading a journal article and you see the following sentence: "The study used a 2 x 2 x 4 design." Based on this sentence alone, you would know which of the following pieces of information? The number of participants in the study The number of main effects that need to be examined The statistical significance of the interaction The number of participant variables
B
The number of main effects that need to be examined is _________ the number of independent variables. more important than equal to unrelated to independent of
B
Which of the following is a reason that psychologists especially value meta-analyses? Meta-analyses eliminate the need for replications. Meta-analyses allow researchers to examine the strength of a relationship. Meta-analyses are immune to threats to internal validity. Meta-analyses always take less time to conduct.
B
Which of the following is an advantage of using quasi-experimental designs? They allow researchers to capitalize on random assignment. They allow researchers to enhance external validity. They allow researchers to disregard internal validity. They are better suited to detect significant effects.
B
Which of the following types of study support external validity? A study using non-randomly selected participants A study that adds a new independent variable A study that is a direct replication A failed replication study
B
Why do quasi-experiments tend to have very good construct validity for the independent variable? The manipulations have been previously validated in the lab. They use real-world manipulations/experiences. They tend to use more participants. They also have good construct validity for the dependent variable
B
5 types of data
B-Data, P-Data, L-Data, I-Data, and S-Data
Guide Lines for asking Questions
B. Make items clear—the respondent should be able to easily understand the items on the survey. C. Avoid double-barreled questions—this refers to questions with multiple components, and thus, it is impossible to determine which part the respondent is answering. D. Respondents must be competent to answer—the respondent should have enough information to answer the question accurately. E. Respondents must be willing to answer—oftentimes respondents are reticent to respond to questions, particularly on sensitive topics. F. Questions should be relevant—the items on the survey should be meaningful to the respondents. G. Short items are best—the longer and more complicated the item, the more difficultly the respondent has in answering the question. H. Avoid negative items—respondents often overlook negative items in a question or statement, and give an inaccurate response. I. Avoid biased items and terms—certain terms and phrases encourage certain responses; the items should employ neutral language.
What questions do reliability and validity answer about a measure?
BOTH have to do with how good a measure is
Classification Concept
Complex, multidimensional concepts that have sub-types. They are parts of social theories between one simple concept and a full theoretical explanation
Qualitative Research
Based on making observations that are summarized and interpreted in a narrative report
Statistical Relationships
Based on two ideas: correlation and independence
What are the 4 criteria for empiricism?
Be acquired from systematic observation Be acquired from objective (unbiased) observation Be independently verifiable by other observer Provide a tentative answer.
variables
Before deciding on the independent variable, you must first determine all possible ______________.
data chart
Before starting the experiment, you must decide how you will record your data by creating an appropriate ____________________.
Availability Heuristic
Being persuaded by what easily comes to mind
Good Story Bias
Being swayed by a good story "It just makes sense"
Experiment Qualities
Best for narrow scope, micro-level concerns. Isolate and target impact of few causal variables. Not viable for research of diverse variables
Dr. Gavin is conducting a 2 x 4 independent-groups factorial design. Assuming he wants 25 people in each cell, how many participants does Dr. Gavin need to recruit? 100 150 200 250
C
In the case of a factorial design, another term for independent variable is: Main effect Cell Factor Moderator
C
Which of the following studies is most likely to be conducted in generalization mode? Examining whether dog ownership decreases stress levels Exploring the relationship between weight and number of fast-food restaurants in one's neighborhood Determining how frequently people report fighting with their bosses Investigating whether there is a causal link between receiving cognitive therapy and increased coping skills
C
9. _____is a statistical analysis that tests for differences between two or more categorical independent variables, when one is a between-subjects variable and another is a within- subjects variable. A) A one-way analysis of variance B) A repeated-measure analysis of variance C) A mixed design analysis of variance D) A two-way analysis of variance
C - mixed design of variance
2. ____ is when the influence of one independent variable on the other reverses across the levels of the other variable. A) A main effect B) A correlation C) A crossover interaction D) A marginal effect
C- crossover interaction
What are the problems with claims based on intuition?
COGNITIVE BIASES: thinking the easy way Being swayed by a good story - good story bias Being persuaded by what easily comes to mind - availability heuristic MOTIVATIONAL BIASES: thinking what we want Focusing on the evidence that fits our beliefs - confirmation bias Being biased about being biased - bias blind spot
CONSORT
CONsolidated Standards Of Reporting Trials
Scientific research involves evidence being obtained in which ways?
Careful, systematic, and objective observations
testing threat
Change in participants as a result of taking a test more than once. Did performance improve because of the manipulation, or because participants are already familiar with the test?
Carryover Effects Confound (order effects)
Changes in the *scores observed in one level of an independent variable are caused by the lingering effects* of a level of an independent variable previously experienced
Type I Error
Claiming a result was significant when you should have not done so. Reject the null when you should not have done so. False positive. Tend to be more problematic because false knowledge is distributed. ex. Telling a man that he is pregnant.
Conceptualization
Clarifying what we mean by a concept - develop the concept. (variable of interest)
Open-ended and closed-ended questions
Closed ended questions include all possible responses to the question, while open-ended questions place no constraints on the respondent's answer; they are free to respond however they choose.
student
Coercion is most common when using _______ participants.
Between-subjects design
Comparisions are made between different groups of participants
Self Questionnaire- Compensation for respondents
Compensation in survey research is often controversial. It often consists of gift certificates, prize drawings, and lotteries, and when prepaid, are more successful.
Pearson Product Moment Correlation Coefficient: Pearson r
Computed when data for both measured variables are quantitative in nature (interval or ratio) and linear in relationship - both variables are continuous. Also used as a prediction for correlation of determination. ex. GPA, years of experience, NCE score.
Conceptual Variable and Operational Definition Examples
Conceptual Variable -Binge Drinking -Learning -Anxiety -Height Operational Definition -4 to 5 drinks in two hours -Score on an exam -Score on anxiety inventory -Inches? Centimeters?
Applied research
Conducted to address issues in which there are practical problems and potential solutions.
Applied Research
Conducted to find solutions for specific "real world" problems To improve the human condition
Motivation
Confirmation bias/Cherry-picking Asking biased questions The bias blind spot
Self-reports and accuracy
Consequently (of flashbulb memories and confidence) a, retrospective self-reports may be inaccurate (i.e., may have poor construct validity as a measure of past behaviors). • Possible solution: observe directly, rather than rely one self-reports.
What is the best method for determining how a variable should be measured?
Consult previous research involving the same variable
Randomization
Controlling for the effects of extraneous variables by ensuring that the variables operate in a manner determined entirely by chance.
Pattern and Parsimony
Converging evidence from correlational studies helps us get closer to causal claims
Cross-Sectional Correlation
Correlation between *2 variables*, measured at the *same time*
Cross-Lag Correlation
Correlation between an *earlier measure of one* variable and a *later measure of another* variable - remains across time - temporal precedence
payer perspective
Costs represent the charges for health care products and services allowed or reimbursed by the payer
True
Cross-sectional research involves data collection at one point in time. (The findings from this research cannot be used to draw conclusions about change over time, but they can provide a snapshot of a social condition or population at that point in time). T/F
Control
Crucial in experimental research, important to isolate the effects of the treatment to eliminate alternative explanations. Deception allows the experimenter to control subjects' perception of the situation
A small-N design that involves providing treatment and then removing treatment is known as a(n): interrupted time-series design. stable-baseline design. multiple-baseline design. reversal design.
D
An author might use all of the following methods to indicate statistical significance EXCEPT: Using the word significant Including an asterisk in a table Using the notation p < 0.05 The inclusion of a graph
D
Dr. Gavin decides that instead of conducting a 2 x 4 independent-groups factorial design, he is going to conduct a 2 x 3 x 4 mixed factorial design. Which of the following things will NOT have to change? The number of main effects that need to be examined The number of interactions that need to be examined The number of participants needed The number of researchers needed
D
Dr. Gavin is conducting a 2 x 4 independent-groups factorial design. How many cells are in his design? 2 4 6 8
D
In a previous correlational study, Dr. Lafayette has found that owning dogs is associated with lower levels of daily stress in a sample of returning war veterans. Which of the following would constitute a shift from theory-testing mode to generalization mode? Conducting a study on the same sample of war veterans, this time using a different measure of daily stress Conducting a study examining the stress reduction mechanism Conducting an experimental study in which he assigns people to own a dog or not Conducting a study on dog ownership using a sample of elderly participants from a local retirement home
D
In small-N designs, each participant is treated: with multiple interventions. by a clinical psychologist. as a data point. as a separate experiment.
D
Which of the following is true of a scientific literature? It comprises studies conducted by a single researcher. It comprises studies that have tested the exact same variable. It comprises studies conducted all in the same year. It comprises studies conducted with different methods.
D
Which of the following is true of cultural psychology? Its researchers function only in theory-testing mode. Its main focus is to disprove theories. It typically finds support for a theory across different cultural contexts. It starts with the assumption that theories apply to all humans, regardless of culture.
D
Which of the following is true of small-N experiments? Small-N designs determine whether a finding is replicable by doing a test of statistical significance. Data from small-N designs are presented as averages. Data from small-N designs are grouped together. Each person in a small-N design is treated as a separate experiment
D
Which of the following types of replications investigate the same research question but use different procedures? Statistical replication Direct replication Replication-plus-extension Conceptual replication
D
6. _____ is a single-subject design in which researchers take a baseline measurement and then introduce the intervention and measure the same variable again. A) An A-B-A design B) A single-blind procedure C) A mixed design D) An A-B design
D - A-B design
Fatigue effect
Deterioration in participant performance with repeated testing.
5. _____ is an effect in which two variables combine to produce an outcome that is smaller than what each individual variable contributes. A) Carryover effect B) Synergistic effect C) Main effect D) Suppression effect
D- suppression effect
Cross-sectional
Data collected only one point in time to understand needs, attitudes, make comparisons or evaluate program.
Longitudinal
Data collected over time with various pre-established points for administration.
Nonparametric
Data for which the probability distribution is unknown or known not to be normal skewed examples: Mann-Whitney and Spearman's test
Limitation
Data only of what a person or organization says, may differ from what he or she actually does
check if the the participant realized the deception
Debriefing is a good time for manipulation checks, which are?
Operational definition
Definition of a concept that specifies the method used to measure or manipulate the concept.
Likert Scale
Degree of Agreement Strongly Disagree 1 - Strongly Agree 5
Experimental Validity
Degree to which the results of the study can be attributed to independent variable truly effected changes in dependent variable and results can be generalized to target population.
Behavioural Therapy is often used for what psychological disorder?
Depression
CBT is often used for what psychological disorders?
Depression, panic disorder, bulimia nervosa, and post traumatic stress disorder
Frequency Claim
Describe a particular rate or degree of a SINGLE variable
Frequency Claims
Describe a particular rate or degree of a SINGLE variable. Examples: -1 in 5 people have a dental filling -64% of Americans report being very happy with their partner
When they can avoid three problems: observer bias, observer effects, and reactivity
Describe how researchers can make observations with good construct validity.
Association Claims
Describe the relationship between two variables. Also known as correlations or covariation. Relationship between x and y. Involves at least two variables Cannot make causal claims Example: -Children of divorced parents are more likely to suffer from an anxiety disorder
Naturalistic observation
Descriptive method in which observations are made in a natural social setting. Also called field observation.
no
Do you accept or reject the hypothesis?
America the Beautiful
Documentary in which women said they did not feel beautiful, but the majority do
Non-probability Sampling
Does not involve random selection. Used by qualitative researchers for detailed findings
Discriminant Validity
Does our measure only weakly relate (or have no relationship) to other measures that have little/nothing to do with our construct?
Convergent Validity
Does our measure positively correlate with other measures that are assessing similar constructs? AND/OR Does our measure negatively correlate with other measures that are assessing opposite constructs?
Criterion-Related Validity
Does the measure allow you to distinguish people on the basis of a particular criterion or behavioral outcome? Researchers identify behavioral outcomes that the measure should be related to and then correlate their measure with the occurrence of those outcomes
Reliability
Does the measure produce consistent results?
C
Dr. Alfonse, a developmental psychologist, conducts a study to determine whether children prefer books with drawn illustrations or with photographs. A group of 30 preschoolers are shown two copies of a book (Ferdinand the Bull) at the same time. Although the story is the same, one book is illustrated with drawings and the other is illustrated with photos. Students are then asked to indicate which book they prefer. This is an example of which of the following designs? Longitudinal design Repeated-measures design Concurrent-measures design Posttest-only design
C
Dr. Deveraux has conducted a study that has resulted in a null effect. Nonetheless, she suspects that there truly is a causal relationship between her independent and dependent variables. Which of the following is UNLIKELY to be to blame? A weak manipulation An insensitive measure Too many participants A reverse confound
C
Dr. Hoff is curious as to whether children in a kindergarten classroom will be friendlier after talking about friendship. Immediately before and after the discussion, she has several undergraduate research assistants code the "friendliness" of a group of kindergarteners. Which of the following threats should Dr. Hoff be most concerned about? History Maturation Instrumentation Testing
D
Dr. Kline, an environmental psychologist, conducts a study to examine whether visiting zoos causes people to have more positive attitudes toward environmental conservation. He asks a group of 45 people attending the zoo on a Saturday morning about their attitudes. He finds that 69% of the people report having a positive attitude after their visit. Which of the following is true? His control group is people who did not visit the zoo. He does not have a dependent variable. He can make a strong causal claim about the effect of zoo visits on environmental attitudes. His study does not qualify as an experiment.
B
Dr. Sanderson is curious as to whether exposing people to violent video games causes them to be more aggressive. She assigns half her participants to play a violent video game for 5 minutes and the other half to play the same game for 25 minutes. Afterward, she has them play a board game and has a well-trained coder determine whether they are very aggressive in their playing style, barely aggressive, or not at all aggressive. She finds that a vast majority of her participants, regardless of group assignment, are rated as very aggressive. This outcome would be known as a(n): weak manipulation. ceiling effect. floor effect. outlier.
Same conclusion
During the Rosenthal Study, it was observed that *meta-analysis literature reviews* had what sort of conclusions?
Varied conclusion
During the Rosenthal study, it was observed that *narrative literature reviews* had what sort of conclusions?
Accumulation of knowledge
Each new generation of researchers don't have to reinvent the wheel because research is what?
Crossover
Effect of two interventions observed across a single group with assessment before, during and after interventions. May use multiple groups to adjust/compare order of intervention given.
Latin Square
Effects of three interventions assessed. Randomly assigned groups with different sequential order of intervention.
Experimental Control
Efforts to minimize effects of extraneous variables through design, development and implantation of research: - Random Assignment - Comparison Group - Counterbalancing - Matching
What does it mean to be Scientific?
Empirical Approach Theory-data cycle Make findings public
Which method obtains answers from direct sensory observation?
Empiricism
Planning
Entering and gaining access to a field site is a process that depends on common sense judgment and social skills
Input Evaluation
Environmental assessment of available resources in work environment.
Interval
Equal intervals between units but *no meaningful zero* (IQ test)
Ratio
Equal intervals, a *meaningful zero* zero means "none" (calories consumed, weight, brain activity)
How to avoid design confounds
Equate everything that you can other than the independent variable (hold everything constant)
According to the concept of reliability, inconsistency in measurement comes from what?
Error
Benefits of factorial design
Establish Cause and Effect Conduct Multiple Experiments at Once Examine How a Combination of IVs Affects the DV (Interaction)
Experimental control
Extraneous variables are kept constant.
anatomy of a mixed design ANOVA
F (2, 87) = #, p=#, eta2 = # f= f test symbol 2= between subjects DF 87 = within subjects DF p = significance eta= calculated effect size
Factorial design
Factorial designs are designs with more than one independent variable (or factor).
How to Avoid Type II Error
Fail to reject the null. Increase the number of participants. Review threats to reliability of instruments. Consider alternate test statistics.
Floor effect
Failure of a measure to detect a diff erence because it was too difficult
Ceiling effect
Failure of a measure to detect a difference because it was too easy
True or False: Correlation always means that a causation is present
False
True or False: In order for a measure to be reliable, it must be valid
False
Hybrid
Field experiments - contrived situations in the real world
Process Evaluation (Formative)
Finding answers to questions that pertain to the implementation of a program and opportunities to make revisions; periodically to see how they are doing.
Primary Sources
Firsthand reports in which the authors describe their own observations
Periodic Probe
Follow-up questions (helpful for respondent fatigue)
Solomon four group
Four groupings used to mitigate testing effects of pretest while assuring equal groups are used: - Control group that receives pretest and posttest. - Control group that receives only posttest. - Treatment group that receives both pretest and posttest. - Treatment group that only receives posttest.
most famous cohort study in the US
Framingham Heart Study
Operational Definition
How the researcher decides to measured or manipulate the conceptual variable. Also called an operational variable, measure
pretest-posttest
Give dependent variable, then manipulate, then give dependent variable again. Ensures equivalent groups Gives change and whether significant Gives power and can see confound and remove statistically
Laboratory Experiment
Greater internal validity but lower external validity. They are logically tighter and better controlled but less able to be generalized
intuition
Gut feeling, emotion, "follow your heart"
Hybrid Design
Hybrid designs have at least one factor that we cannot actually manipulate (e.g., gender). In factorial designs, you can test the impact of an experimental variable and a nonexperimental variable at the same time. any factorial design that has at least one quasi-independent variable examples: race, age, gender, (something you cant change about participants
RCTs are required in phase _____ clinical trials as a part of an NDA
III
all protocols are reviewed by the _____
IRB
Product Evaluation (Summative)
Identified extent to which goals and objectives were achieved or if improvement of discontinuations are needed; improving for next time, not the current product.
Subject Words
Identify and describe the variables in the study and the characteristics of the participants
Why is a stable-baseline design more convincing than a simple pretest/posttest?
If the baseline is stable over time, you can rule out regression to the mean and maturation threats.
Alpha (aka critical P value)
If the probability is less than the alpha (critical P value, usually 0.05), null hypothesis is rejected
hypothesis
If your conclusion leads you to new questions, you should create a new ______________________.
Practice effect (also learning effect)
Improvement in participant performance with repeated testing.
Nothing
In a *prospective* research study, you need more information since it can be correlative or experiment, what can you assume?
No causal inference
In a *retrospective* research study, what can you assume?
Order Effects Confound
In a [[within-subjects design]], when a participant *experiences a series of different levels of an independent variable and a series of measurements*, their performance at any point may be influenced by an experience that occurred earlier
Simple main effect
In a factorial design, the effect of one independent variable at a particular level of another independent variable.
main effect
In a factorial design, the overall effect of one independent variable on the dependent variable, averaging over the levels of the other independent variable. a HYPOTHESIS (prediction) that focuses on one independent variable at a time, ignoring all other independent variables
Order effect
In a repeated measures design, the effect that the order of introducing treatment has on the dependent variable.
Stem
In a survey, the statement, question or prompt to which the respondent replies.
Contingency questions
In certain instances, questions may not be relevant to all respondents. Typically, the researcher will ask a question to determine who should answer the subset of questions and who should ignore them. The questions that are only answered by a subset of the respondents are referred to as contingency questions.
How to identify factorial designs in reading
In empirical journal articles, the method section will usually state "this was a factorial design, with ___ and ___ as independent variables." and the results section will describe the statistical tests for main effects and interactions (these are ANOVAs, or F tests). In popular press articles look for key phrases such as "it depends" and look for participant variables
Subjects
In experimental research, the cases or people used in research projects and on whom variables are measured
manipulation; measurement
In experiments, the ____________ occurs first, then a __________ is made.
Complete Participant
In field research, when a researcher fully participates in all aspects of the study group's activities as though a member of the group (too much sympathy can be an issue)
Complete Observer
In field research, when a researcher only observes the study group without participating in their activities
Devil's Advocate
In focus group research, refers to a person whose role is to argue against a dominant idea and who could introduce new questions and new ways of thinking into the group so as to prevent the tendency toward group conformity
Groupthink
In focus group research, refers to people's natural desire to avoid conflict and lean toward group consensus, even when the opinion of the group does not reflect their own personal opinions
quantitatively
In order to make a graph, the data collected must be ____________________ testable.
Significance (statistical)
Is achieved when there is a low probability that the results of an experiment occurred by chance alone. If the probability of their occurrence by chance is equal to or less than 5 per cent or 0.05
Limitations of Authority
Is not always accurate, often accepts expert's statements as facts, not all experts are experts
What questions are helpful to ask while reading journals and journalism?
Is original research evidence cited? Has the research been published in a peer-reviewed journal yet? How does coverage of the same journal article differ across news stories?
Convergent validity
Is the measure strongly related to other established measures of the same (or a similar) constructs?
Discriminant validity
Is the measure weakly (or not at all) related to established measures of a different construct? Is the measure weakly correlated with something it shouldn't be correlated with? (this is good, do not want high correlation with something measure should not be correlated with)
Self Questionnaire- Monitoring returns
It is important to keep track of returns so that the researcher identifies who need to be contacted again and the optimal time for sending follow-up (reminder) mailings.
Deontological Objection
It is never okay to lie even if it is used for a study and people may not be at risk for discomfort, etc. Could lead to the development of mistrust and suspicion in future studies
Filler items
Items included in a questionnaire measure to help disguise the true purpose of the measure.
Reverse Scoring
Items stated opposite of the other items; must be reversed in analysis.
Problems with Experience
Lack of COMPARISON: Typically no fair control condition CONFOUNDED Claim: Other possible explanation for your findings -Example: You feel calm after you exercise, you think its the exercise but it could be the temperature of the gym or the music you had playing
What are two problems with claims based on experience?
Lack of COMPARISON: Typically no fair control condition CONFOUNDED Claim: Other possible explanation for your findings -Example: You feel calm after you exercise, you think its the exercise but it could be the temperature of the gym or the music you had playing
1. State the Question or Problem 2. Hypothesis 3. Plan the Investigation 4. Gather Data 5. Organize Data 6. Analyze Data 7. Draw a Conclusion
List the 7 steps of the Scientific Method that we will use.
Hour Glass Writing of Research
Lit review is broad and narrows to a specific research question. methods and results sections are more narrow and specific. The discussion starts narrow tied to the meaning of results then broadens to importance for the field.
Justice
Making sure the subjects are not exploited
National Research Act (1974)
Mandated regulations for protection of subjects
Correlation
Means that things go together or are associated (e.g., people with higher values on the income variable are likely to have higher values on the life expectancy variable)
Physiological
Measure of a biological state Examples: Hormone levels, blood flow in brain, heart rate, pupil dilation Conceptual variable: Arousal Operational variable: Pupil dilation
Coefficient of Determination (r^2) & how to calculate
Measure of the amount of variance in one variable can be predicted from the other variable. Calculated by squaring the value of r. Ex. If correlation of variable x to y is r = .70 then r^2 = .49. .49 x 100= 49% of the criterion value is explained by the predictor value.
How is validity established?
Measurement procedure must accurately capture the variable that it is supposed to measure
Face validity
Measures whether a test looks like it tests what it is supposed to test Does the measure look like a plausible measure of the construct at face value?
The following is an example of which method: Black cats are bad.
Methods of Tenacity
Deception
Misinformation that a participant receives during experiment
In a Negatively Skewed Distribution...
Mode is the highest mean is the lowest median is in between the two
In a Positively Skewed Distribution...
Mode is the lowest Mean is the highest Median is in between the two
We cannot make causal claims from correlational data
Most important thing to remember about correlations
Survey
Most widely used data-gathering technique in social research. Sppropriate for research questions about self-reported beliefs and behaviours. -Started from positivist approach -Can ask about many things at one time and test several hypotheses
Empiricism - Independently Verifiable
Need to be able to distribute data and make it so another scientist can follow the methods you used and get the same answer
Nominal
No inherent order Categorical data Often open-ended, and demographic questions Frequency data and chi-square Nonparametric measures
Internal Validity (3 criteria for causal claims)
No other possible explanations for why x and y are related
Why use deception?
No other way to conduct your study
Zero Correlation
No real association - line of best fit straight across, no trend
Can you prove or disprove something in research?
No, you can only support
examples of independent groups quasi-experiment tests?
Nonequivalent control group design (posttest-only) - Nonequivalent control group design (pretest/posttest).
quasi-experimental study with at least two groups, measured one time.
Nonequivalent control group posttest only Example: The Head Start Study
quasi experimental study with at least two groups; measured before and after an intervention.
Nonequivalent control group pretest/posttest design: Example: psychological effects of cosmetic surgery
Scientific Journalism
Not considered science per se. Is scientific news and commentary written by another party (usually a reporter, not a scientist) and posted to a wide readership (usually a non-expert audience).
Survival
Number of people alive at a certain point (often 1 year or 5 year) after diagnosis
survival rate
Number of people alive divided by population
U.S. Surgeon General
Orders federally-funded research proposals from the Public Health Service to be reviewed for ethical treatment of subjects
Quantitative/continuous measurements
Ordinal, interval, ratio
Confound
Other possible explanations for an outcome
pretest/posttest
Other times, researchers want to measure a dependent variable before and after a manipulation.
Which conference will the professor of the BPSYC Research Methods class be presenting at this year?
Persuasive Technology
Pretesting the questionnaire
Piloting the questionnaire is critical for catching any errors or confusing items.
the people who were present for the pretest AND the posttest. (don't count the drop-outs).
Preventing attrition threats is easy. Only analyze
True
Research questions should be actual questions that can be answered rather than statements that make some claim.T/F
Consumer-Oriented
Provide consumers with complete picture on effects of a program. Consumers often other counselors, prospective clients, administrators.
Expertise-Oriented
Provide professional judgements about quality of a program. Ex. Accreditation standards for CACREP counseling.
Declaration of Helsinki (1946)
Provided international ethics guidelines for medical research (written protocol, detailed description, reviewed by council)
Quantitative Research
Provides numerical scores and is submitted to statistical analysis for summary and interpretation
MEDLINE Database
Provides the authoritative medical information on medicine, nursing, dentistry, veterinary medicine, the health care system, and pre-clinical sciences
IQR (interquartile range)
Q3-Q1 (split data in half and then find the median of both halves)
Pre/post-test
Quantitative measure of dependent variable given before and after the intervention. Can compare similarities between groups before onset of study.
Post-test
Quantitative measure used to determine the effects of the intervention between the treatment and control group.
Order Effects (2 sources of question bias)
Question order Solution: Avoid open-ended after forced-choice questions Multiple response options Solution: Order in more than one way
Questions and statements
Questions elicit specific information from a respondent; statements ask a respondent their reaction to a particular claim.
Empirically Testable Research Questions
Questions expressed in terms of a single variable or relationship between variables
CONSORT is used to evaluate _____
RCTs
Based on the video watched in class, what is the quickest method to extract information from a research article?
Read through the article in three passes
Discussion
Recap results, limitations, future directions
Physiological measure
Recordings of responses of the body.
Why Questions
Research warn against using these for a survey. Only appropriate if a researcher wants to discover a respondent's subjective understanding or informal theory
Moderator
Refers to a trained facilitator used in focus group research who guides the focus group discussion
Matrix questions
Refers to instances when a series of items share the same set of responses. While efficient, it can produce a response set, where respondents answer all of the items in the same way.
Semi-participant
Refers to the role of the researcher in field research when he or she participates to some degree in group activities, but not as much as a full member.
prevelance
Refers to the total number of cases of a condition at a given time. whether new or previously diagnosed
B
Regression is especially problematic in which of the following situations? When the experimental and comparison groups are equal at posttest When one group has an extremely high score at pretest When the experimental and comparison groups are equal at pretest When one group has an extremely low score at posttest
Type 1 error
Rejecting null hypothesis when it is true
Type 1 error
Rejecting null hypothesis when it is true (false positive)
Scatterplot Form
Relationships can take three forms: independent, linear, and curvilinear
What is the relationship between reliability and validity?
Reliability is a prerequisite for validity
Minimal risk research
Research in which participants are exposed to risks that are no greater than those encountered in daily life or in routine physical or psychological tests.
True
Research can be found in academic journals, books, newspapers, magazines, legislative actions, policy decisions, and databases. Some of this information is more objective and accurate than others.T/F
Alternative Hypothesis
Research hypothesis contradicting the null. ex. Each bag does not have 17 M&M's.
Panel study
Research in which the same sample of subjects are surveyed at two or more points in time, usually to assess changes that occur over time.
Exempt research
Research in which there is no risk is exempt from review by the IRB. Ex. Questionnaires, surveys, naturalistic observations, etc.
Surveys
Research methods in which researchers use questionaires or interviews.
tentative
Research of ethics is _________ and not chiseled in stone.
1) Literature Review 2) Hypothesis 3) Develop Methodology 4) Data Collection 5) Analyze Results 6) Interpret & Research Conclusion
Research process
Factor Analysis
Researcher determines the effect of two or more independent variables (factors), both singularly and interacting with each other on a dependent variable. Purpose of data reduction & summarization. Two values: main effect & interaction effect
Charm and Trust
Researcher needs social skills and personal charm to build rapport
Observer bias
Researchers expectations influence how they interpret the results of a study
voluntarily, meaning that the research subjects must be completely free to participate or not participate without threat and informed, meaning that the researcher must inform the research subjects about the research and any risks associated with participation in the research.
Researchers must obtain consent from research subjects prior to collecting information from them. This consent must be:
Principle of Justice
Researchers must show that the people participating in the study are REPRESENTATIVE of the people that will be helped by any positive results
Central Tendency
Researchers often want to summarize the information about one variable into a single number: mean, median, or mode
Beneficence
Researchers should try not to have any negative impact on the wellbeing of the people who participate in their studies
True
Researchers use three causal rules to determine whether a causal relationship exists between two variables. Each of these three rules must be met before a researcher can prove that one variable is the cause of another. T/F
Belmont Report (1979)
Risk and benefits fairly distributed
SD v. standard error
SD: -sample statistic -known -spread of observation -normal curve applies -relating all known into to each other Standard error: -tool for predicting -calculated based on sample info, sample size -predict what you might find in population using 1 observation *both take normal curve approach -get standard error to be smaller by taking a larger sample -varied population will produce large standard error
State the Question or Problem
Scientific Method Step #1
Hypothesis
Scientific Method Step #2
Plan the Investigation
Scientific Method Step #3
Gather Data
Scientific Method Step #4
Organize Data
Scientific Method Step #5
Analyze Data
Scientific Method Step #6
Draw a Conclusion
Scientific Method Step #7
Haphazard (convenience) sampling
Selecting subjects in a haphazard manner, usually on the basis of availability, and not with regard to having a representative sample of the population; a type of nonprobability sampling.
Why is it important to ensure the methods section of a report is accurate?
So that others can replicate it
clear, step by step, and numbered
So that others can replicate your exact experiment, you must write a __________________________________________ procedure.
Negotiating
Social relations are negotiated and formed throughout the process of fieldwork. Occurs with each new member until a stable relationship develops to gain access, develop trust and reduce hostile reactions
Qualitative Sampling
Social scientists typically select interview participants through nonprobability sampling (e.g., snowball and purposive sampling are often used)
Micro-Level Theory
Social theories and explanations about the concrete, small-scale, and narrow level of reality, such as face-to-face interaction in small groups during a two-month period
Macro-Level Theory
Social theories and explanations about the more abstract, large scale and broad scope aspects of social reality such as social change in major institutions (eg. the family, education) in a whole nation across several decades
Grounded Theory
Social theory that is rooted in observations of specific, concrete details
Measurement error Individual differences Situation noise
Some common sources of unsystematic within-group variance (noise)
Gatekeeper
Someone with the formal or informal authority to control access to a site
ceiling effect, floor effect
Sometimes an IV has no effect because there is little room for improvement (_________), or little room for doing worse (__________).
Weak manipulations Insensitive measures Ceiling and floor effects Design confounds
Sometimes real differences between groups are hard to detect. There can be multiple causes for this:
Point Biserial Correlation Coefficient (r sub pb)
Special form of Person r used when one variable is nominal and the other is continuous. Nominal must be dichotomous (yes/no, true/false, male/female). Ex. Passing the NCE (dichotomous) and grade point average (continuous).
What are 2 criteria for a research question?
Specific and Testable
Operational Definition
Specify precisely how the concept/term is measured or manipulated in that particular study. *Must operationally define hypothetical constructs!!*
bias
Sponsorship ______occurs when the sponsor of a research project attempts to influence the study design or interpretation of data for its own benefit.
C
Spontaneous remission in clinical studies is an example of which of the following threats to internal validity? Regression Attrition Maturation Placebo effects
True
Spuriousness refers to a false causal finding. It occurs when a researcher alleges a causal relationship between two variables but fails to confirm at least one of the three causal rules. T/F
Staged manipulation
Staging events during the experiment in order to manipulate the independent variable successfully.
Alternative Hypothesis
States that there is a relationship between the two variables being studied (as opposed to the null hypothesis)
Correlation coefficient
Statistic showing strength of the relationship (r)
p < .05
Statistical Significance
Random Error
Statistical fluctuations that are caused by unknown & unpredictable changes in the experiment
Univariate Statistics
Statistical measures that deal with one variable only Easiest way to describe: frequency distribution
Bivariate Statistics
Statistical measures that involve two variables only
Example of Mixed Design
Study was looking at how long children stare at angry or happy faces based on whether or not they had been previously abused. Between subjects: Child status was naturally occurring- quasi-experimental within subjects: face stimuli- angry or happy
When conducting a literature review which two key elements should you make note of?
Subject Words and Authors Names
Respect for Persons
Subjects have to give informed consent
Types of Measures of Reliability
Successive Measurements Simultaneous Measurements Internal Consistency
Belmont Report
Summarizes ethical principles and guidelines for research involving human subjects. It has three basic ethical principles.
Abstract
Summary
Which method obtains answers from habit or superstition?
Tenacity
Fence Sitting (Response Sets + Solution)
Tendency to be neutral about everything Solution: Use even number of responses. But sometimes people really have no opinion!
Social Desirability (+ Solution)
Tendency to give answers to look better or conform to social norms ("faking good" or "faking bad") Solution: Make survey anonymous Include filler questions to mask purpose Include 'too-good-to-be-true' questions
Observed P value
Test calculates probability of obtaining the observed results if the null hypothesis is true
Successive Measurements
Test-retest reliability compares scores of two successive measurements of the same individuals and correlates the scores
Behavioural Theories
Testing predictions that are part of a theory
If an ethical dilemma is not mentioned in the BCBA guidelines, what should be referenced?
The APA Guidelines
"Ethic Principles of Psychologists and Code of Conduct"
The APA has developed an extensive document known as the "_____________" in an effort to keep pace with the challenging changes of scientific ethics.
PsycINFO Database
The American Psychological Association's (APA) definitive resource for abstracts of scholarly journal articles, book chapters, books, and dissertations devoted to peer-reviewed literature in behavioural science and mental health
The Horizontal-Vertical Illusion is an example of which method?
The Empirical Method
Who enforces ethical guidelines?
The Institutional Review Board
The following is an example of which method: All 3 year old children are afraid of the dark. Amy is a 3 year old girl. Therefore, Amy is afraid of the dark.
The Rational Method
scientists
The Scientific Method is used by ________________ to find out about the natural world.
expedited
The ____________ review does not require approval from the full IRB committee, but only a few members, and is completed more quickly than a full review.
materials
The _____________ list is not numbered.
literature review
The ______________ involves locating and understanding what previous researchers have learned about a topic.
data chart
The ______________________ is used to collect information during the experiment in an organized way.
independent variable
The _________________________ is the variable you choose to manipulate or change in the experiment.
dependent variable
The ___________________________ is the variable that you will measure in your experiment and will change in response to the independent variable.
Sensitivity
The ability of a measure to detect differences between groups.
Validity
The ability of a test to measure what it is intended to measure, are you hitting the bullseye
Internal Validity
The ability to eliminate alternative explanations of the dependent variable Rule out variables other than the treatment by controlling experimental conditions. Threats: -Selection Bias -History Effects -Maturation -Mortality -Diffusion of Treatment or Contamination -Experimenter Expectancy
External Validity
The ability to generalize experimental findings to events and settings outside the experiment itself
External validity
The degree to which the results of an experiment may be generalized.
General questionnaire format
The design of the questionnaire should be straightforward and uncomplicated. Items should be clearly distinguished from one another so they are not missed.
Non-statistical Definition of Sampling Error
The deviation between sample results and a population parameter caused by random processes
attributes
The different characteristics or values that a variable can take on are called that variable's:
Sampling error
The error that arises as a result of taking a sample from a population rather than using the whole population.
Content validity
The estimate of how much a measure represents every single element of a construct.
What is important to consider when writing a research proposal in terms of the ethicist on the REB?
The ethist may not have an ABA background, therefore it is important when writing a proposal to write it at a level that this individual will understand
independent variable
The experimental factor that is manipulated; the variable whose effect is being studied.
reliability capicty
The extent to which a measure consistently measures a social phenomenon is referred to as the measure's:
Convergent validity
The extent to which the scores on one measure are related to scores collected from a similar or different measure.
ANOVA
The farther away from *1.0* the *more* statistically *significant* the result in what test for *multiple related groups*?
True
The first causal rule, temporal order, requires that the cause must precede the effect. In other words, the variable that is alleged to be the cause of another variable must happen first. T/F
-refine their research question -identify a theory related to their research question -propose a hypothesis that predicts the answer to their research question.
The literature review process helps researchers:
Mortality (also attrition)
The loss of subjects who decide to leave an experiment.
Attrition (also mortality)
The loss of subjects who decide to leave an experiment.
before
The freedom to refuse or to discontinue the research study must be made clear to the participant ____________ the experiment.
Reactivity
The general threat to external validity that arises because subjects are aware they are in an experiment and being studied -Hawthorne Effect
Control Group
The group that does not receive the treatment in an experiment
Experimental Group
The group that receives the treatment in an experiment
your prior knowledge and research
The hypothesis is based on ___________________________________________.
tested
The hypothesis must be expressed as a statement and must be capable of being _____________.
personal pronouns or iffy words
The hypothesis should never contain
File drawer problem
The idea that meta-analysis could be overestimating the support for a theory because studies that find null effects are less likely to be published therefore are less likely to be included in such reviews.
Journal Article Impact
The impact factor (IF) is a measure of the frequency with which the average article in a journal has been cited in a particular year. It is used to measure the importance or rank of a journal by calculating the times it's articles are cited.
What information can you use to make a judgement about a journal article's impact?
The impact factor (IF) is a measure of the frequency with which the average article in a journal has been cited in a particular year. It is used to measure the importance or rank of a journal by calculating the times it's articles are cited.
Observer Error
The individual who makes the measurements can introduce simple human error
Sampling frame
The individuals or clusters of individuals in a population who might actually be selected for inclusion in the sample.
Framing Effects
The influence, on the respondent, of how a polling question is asked; changes in wording can significantly alter many people's answers
Power
The likelihood that a study will have results that are statistically significant when the independent variable causes a change in the dependent variable. The more power a study has, the more likely we are able to detect true differences. In order for experimenters to increase the power, they must have large samples, accurate measurements, or strong manipulations to escalate the size of the effect.
Literature
The mass of published information worldwide
level of precision
The measure of a sample's representativeness and therefore its ability to accurately predict conditions with the population from which it was collected is referred to as its:
Dependent Variable
The measured variable; the outcome - expect to change
Post-test
The measurement of the dependent variable of the experiment after the treatment
Pretest
The measurement of the dependent variable of the experiment prior to the treatment
Median
The middle score in a distribution; half the scores are above it and half are below it. 50th percentile. First organize the scores from highest to lowest, then count to the middle -Best for skewed distributions
within; between groups
The more unsystematic variance (noise) there is __________ groups, the harder it is to see differences __________ groups (b/c there's more overlap)
Mode
The most common, frequently occurring number. The easiest to use and can be used with nominal, ordinal, interval, or ratio data. Distribution can have more than one.
Population
The much larger collection of animals or people that researchers want to generalize about.
Sampling Element
The name for a case or single unit to be selected. It can be a person, a group, an organization, a social action. ie. an arrest, a kiss
Population
The name for the large general group of many cases from which a researcher draws a sample and which is usually stated in theoretical terms. Researcher begins with this and then defines it more precisely. Abstract concept that requires an operational definition. ie. All persons aged 16 or older living in Regina on December 2, 2016, who were not incarcerated in prison, asylums, and similar institutions All business establishments employing more than 100 persons in Ontario that operated in the month of July 2015 All admissions to public or private hospitals in the province of Alberta between August 1, 2010, and August 1, 2015
Target Population
The name for the large general group of many cases from which a sample is drawn and which is specified in very concrete terms
Interview Schedule
The name of a survey research questionnaire when a telephone or face-to-face interview is used
-answer research questions -to resolve disagreements among researchers -to fill gaps within the body of knowledge -to produce more research questions.
The objectives of research are to
Field Site
The one or more natural locations where a researcher conducts field research
Independence
The opposite of correlation. It means there is no association or relationship between variables. If two variables are independent, cases with certain values on one variable do not have any particular value on the other variable
main effect
The overall effect of one independent variable on the dependent variable, averaging over the levels of the other independent variable. Overall effect
Response rate
The percentage of people selected for a sample who actually completed a survey.
Subjects
The persons or animals whose behavior is systematically observed in a study.
Falsifiability
The principle that a good scientific idea or theory should be capable of being shown to be false when tested using scientific methods..
Replication
The principle that researchers must be able to repeat scientific findings in multiple studies to have a high level of confidence that the findings are true
power
The probability that an experiment will find a significant effect, assuming the effect actually exists.
question ending with a question mark
The problem must be written as a _________________________________________________________________.
Matching
The procedure whereby pairs of subjects are matched on the basis of their similarities on one or more variables, and one member of the pair is assigned to the experimental group and the other to the control group.
Sampling
The process of choosing members of a population to be included in a sample.
Odds and odds ratio
The ratio of probability of an event occurring to the probability of the event not occurring that is p divided by (1-p) where p=probability and The amount by which the odds are multiplied for one group compared with another. Odds for men relative to women are 1.5/0.54=2.77 - we say that this is the odds ratio for males, which means that the odds for men are 2.77 times those for women.
Sampling Ratio
The ratio of the size of the sample to the size of the target population ie. If the population is 500 and the researcher samples 100, ratio is 100/500 = 0.20, or 20 percent
True score
The real score on the variable
correlation
The second causal rule, ___________, requires that the variables in a causal relationship be related to one another, or change together. A change in one variable must be associated with a change in another variable. Correlations can be positive or negative.
How to Evaluate a Hypothesis regarding p and alpha levels
The smaller the p-value, the stronger the evidence is against the null hypothesis. If the p is low, the Ho must go. If p is less than or equal to alpha (.05), the observed effect is statistically significant. The null hypothesis is rejected in favor of the alternative hypothesis. If p is greater than alpha (.05), there is no statistically significant difference, fail to reject the null hypothesis.
counterbalancing.
The solution to order effects is
How many articles is enough during a literature search?
The study (or studies) you find provides some justification for new research
Control Group
The subjects who do not receive the special treatment given to the experimental group.
Experimental group
The subjects who receive some special treatment in regard to the independent variable.
matrix
The summary data sheet should contain all the data in a _______ format and be a single sheet.
Respondent Fatigue
The tendency for respondents to lose interest in participating in research if they perceive their participation to be taking too long or requiring too much effort
Confirmation Bias
The tendency to seek out and interpret evidence consistent with what we believe and to deny, dismiss, and distort evidence that contradicts what we believe
True
The third causal rule, lack of plausible alternative explanations, requires the researcher to eliminate all other reasonable causes before concluding that one variable causes another. T/F
How does new research grow out of old? What visual is used to describe this?
The tree-like structure emphasizes the notion that current research (the tips of the branches) is always based in previous research.
Positive Relationship
The two measurements change together in the same direction
Negative Relationship
The two measures change in opposite directions
3 Assumptions Needed to Test whether There is a Difference between Population Means
The two populations have the same variance: The assumption of homogeneity of variance. The populations are normally distributed. Each value is sampled independently from each other value.
True
The unit of analysis is the 'what' or 'whom' about which researchers gather information during a research project. T/F
Archival research
The use of existing sources of information for research. Sources include statistical records, survey archives, and written records.
Error Variance
The variability or change in a person's behavior that is NOT related to the other variable of interest, but instead remains unexplained
Systematic Variance
The variability or change in a person's behavior that is related to the other variable of interest
Independent variable
The variable that is manipulated to observe its effect on the dependent variable.
interval ratio
The variable age (in years) is measured at what level of measurement?
ordinal nominal
The variable education (high school diploma or equivalent, Bachelors degree, Masters degree or higher) is measured at what level of measurement?
Theory-Data Cycle
Theory a set of propositions that attempts to specify the interrelationships among a set of concepts (aka how the world works). How and why things are related. Hypothesis A specific prediction derived from a theory and a specific research design
Trend studies collect the same data over time from different samples of the same population. Panel studies collect the same data over time from the same sample. Cohort studies collect data over time from samples of individuals who share common characteristics.
There are three types of longitudinal research designs.
Confidentiality
There are two types of privacy in research. (1) __________ means that the researcher knows who the research subjects are but does not disclose their identity to others. (2)Anonymity means that nobody, not even the researcher, knows the identity of the research subjects.
What does a 95% confidence interval of 100 to 110 mean?
There is a 95% probability that the interval from 100 to 110 contains the true population value.
What is it important to consider when using secondary sources?
There is a chance that the source you obtained it from may have misinterpreted the primary source
Limitations of Intuition
There is no method for separating accurate from inaccurate knowledge
Provide two examples of a research paper
Thesis and journals such as the Journal of Applied Behaviour Analysis
One-group pretest post-test design
This design has one group, a pretest, a treatment, and a post-test. NO control group and NO random assignment
Self Questionnaire- Follow-up mailings
This entails sending either a reminder or another survey to respondents who have not already returned the survey.
Main effects (what they are, what they mean, how they are found)
This is the overall effect that one independent variable has on the dependent variable over all of the levels of the other independent variable. It is basically the simple difference/ effect a single variable. -For each independent variable, there is one main effect.
Self Questionnaire- Response rates
This refers to the proportion of the sample that returned the survey. The higher the response rate, the more representative the survey is of the population.
Covariance, Temporal Precedence, Internal Validity
Three criteria for causal claims
Belmont Report
Three key ethical principles to guide all human research
Feasibility
Time, money, technical knowledge and skill, and access to special equipment and research participants
Wait-list design
To control for selection effects, all participants in a quasi-experimental study plan to receive treatment, but are assigned to receive the treatment at different times. This is reassuring to the researchers because they are confident that they have similar types of people in each group
One-group Pretest-Postest Design
To measure participants before the manipulation (a pretest) and again afterward (a posttest).
True
To research means to follow a logical process that uses concepts, principles, and techniques to produce knowledge. T/F
Probability sampling
Type of sampling procedure in which one is able to specify the probability that any member of the population will be included in the sample.
Self Questionnaire- Mail distribution and return
Typically self-administered surveys are distributed through the mail, and the responses are subsequently mailed back to the researcher. Researchers make every effort to ensure that returning the survey is as easy as possible.
Noise/error variance/unsystematic variability
Undifferentiating variability among the members of a group in an experiment. This gets in the way of detecting the true difference between groups. The more unsystematic variability there is, the more the groups' scores overlap, which causes the difference between the two groups to become indistinguishable.
Number needed to treat
Used in assessing the practical interpretation of the differences between two or more groups Multiplicative inverse of the estimated benefit of the treatment. For example, if the treatment reduces an outcome by two percentage points, the NNT is 1/0.02=50.
Independent Variable
Variable that is manipulated in an experiment
Negative Correlation
Variables tend to move in opposite direction. As one variable goes up, the other goes down. As one variable goes down, the other goes up. Denoted by - before the correlation coefficient.
Positive Correlation
Variables tend to move in the same direction. As one variable goes up, the other goes up. As one variable goes down. The other goes down. Denoted by + before the correlations coefficient value.
Constructs
Variables that cannot be observed or measured directly
Counterbalancing
Vary order of treatment to distribute or measure order effects randomized blocks
Mean
What *central tendency* calculates the *sum* of the scores in a distribution of a population or collection of things *divided* by the *number of scores*?
Median
What *central tendency* is the *midpoint* or *mid-score* in a distribution that presents with 50% falling below and 50% falling above that line?
Decrease variance
What *increases power* by *reducing* the *within group differences* make observing sample groups that are more *homogeneous* which removes potential outliers?
Increase reliability of Measures
What *increases power* by reducing error, wobble, and variance by using the *most recently developed tests* for measurement?
- Diffusion - Imitation - Rivalry - Equalization
What *threats* to *internal validity cannot* be ruled out by *random assignment*?
- History - Maturation - Instrumentation - Selection - Regression to Mean
What *threats* to *internal validity* are ruled out by *random assignment*?
Standard deviation
What accounts for how much *dispersion* there is *around* the *mean*?
Systemic error
What are *differences* that can be explained *due* to a *confound*?
Between subjects
What are *groups* that are *completely independent* of one another?
Within Subjects
What are *groups* that are some how *related* to one another?
- Convenience sample - Quota sample - Snowball sample
What are *non-probability sample* methodologies?
- Mean - Median - Mode
What are the components of *central tendency*?
- Nominal - Ordinal - Interval - Ratio
What are the different types *descriptive statics* of data format?
Random sampling
What is a subset of a statistical population in which each member of the subset has an equal probability of being chosen that lends strength to *external validity*?
Sample
What is a subset of people selected out of the population for inclusion in the study?
- Experimenter Expectancy - Volunteer-Non-response bias - Withdrawal - Contamination - Attrition - Mortality
What are the type of *biases* that weaken *internal validity*?
- Opportunistic Data Torturing - Procrustean Data Torturing - Publication Bias
What are the types of *bias in analysis* and publication?
- Increase *Effect Size* - Increase *Sample Size* - Increase effect - Increase reliability of Measures - Decrease variance - Multiple measures
What can researchers do to increase power of their findings?
Statistical significance
What can you ascertain if there is less than 5% likelihood that a result is due to chance?
Inclusion criteria
What comes *firsts*, decides who will be *included in a study*, and helps determine whom the results *can* be *generalized* to?
Exclusion criteria
What comes *second*, decides who will be *excluded in a study*, and whom the results *cannot* be *generalized* to?
Comparison Group
What compares *multiple tx* options?
Results
What did the author find?
Covariance
What do all correlational studies buy you?
Experimenter Expectancy bias
What involves an *experimentors hypothesis* and *expectations* affecting the result or *outcome* of an experiment that is limited by a *double blind study*?
Comparing Means
What involves looking at *two groups* and comparing the *means* of each group to make *inferential statistics*?
Peer review
What involves sending a journal article to *four peers* in the field and they critic it?
Diffusion/Imitation
What is *threat* to *internal validity* that involves the *control* group being *affected* by *tx* from the *tx group*?
Type II: beta error
What is a *failure* to *reject* the *null hypothesis*, but in *reality* the medication is effective, giving a *false negative*?
Convenience sample
What is a *non-probability sampling* methodology that includes anyone who is convenient?
Snowball sampling
What is a *non-probability sampling* methodology that is future sampling guided by results of earlier data collection?
Quota sample
What is a *non-probability sampling* methodology that sets *quotas* for proportions of people to be included with different characteristics?
Hypothesis
What is a *prediction* about how two or more variables are related that has to be *testable*?
Stratified random sample
What is a *probability sampling type* first stratifies the population into *categories* then randomly selects from the categories?
Pure random selection
What is a *probability sampling type* involves each person in the population has an equal chance of being in the sample?
Hierarchical random sample
What is a *probability sampling type* randomly selects from *progressively smaller units*?
Type I: alpha-error
What is a *rejection* of the *null hypothesis*, but in *reality* the medication is not effective, giving a *false positive*?
Central tendency
What is a *statistical index* of a typical set of scores that comes from the *center* of the *distribution* of scores with the three most common being: *mean*, *median*, and *mode*?
Ceiling
What is a *threat* to *internal validity* from *instrumentation* where *can not go any higher* with instrumentation?
Floor effect
What is a *threat* to *internal validity* from *instrumentation* where *can't go any lower* with the specific instrumentation used?
Maturation
What is a *threat* to *internal validity* that involves the *mere passage of time*?
History
What is a *threat* to *internal validity* that is an *event* or *events* that deferentially *affects* the *tx* and *control* groups?
Instrumentation
What is a *threat* to *internal validity* that is due to *changes* or *limitation* in *instruments*?
Testing
What is a *threat* to *internal validity* that is due to *repeated exposure* to a *independent variable*?
Theory
What is a prediction about *why* variables are related with an actual story behind the prediction?
A statement explaining why you think that your hypothesis is correct.
What is a rationale?
Multi-Center Studies
What is a research trial is a clinical trial that uses the *same protocol* at *multiple sites* conducted at more than one medical center or clinic?
Effect size
What is the *strength of relationship between control* and *tx group*, where the farther from zero in negative or positive direction, the stronger the correlation relationship?
Less artificial
What is the *strength* of *field* based experiments?
Regression to the mean
What is the *threat* to *internal validity* that is just *due to chance*, where *extreme* scores tend to *regress* towards the *mean* in *later* measurements?
Compensatory rivalry
What is the *threat* to *internal validity* where the *control* group is *creative* or *motivated* to *out perform* the *tx group*?
Compensatory equalization
What is the *threat* to *internal validity* where the *control* group receives *similar tx* through *alternate means*?
Selection
What is the *threat* to *internal validity* which is *bias* in terms of *who ends up* in the *experimental* or *control group* and is ruled out by *random assignment*?
Artificial
What is the *weakness* of a *lab* based experiments?
Interquartile range
What is the amount of *dispersion* around the *mean* regarding how much *variability* there is *around the sample*?
Criterion Variable
What is the behavior of interest that is changing? (dependent v) example: Behavior problems in the classroom
- Random Assignment - Control group - Double-blind
What is the best case scenario for an experimental design?
Independent variables
What is the construct which is an *actor* variable that is manipulated by the researcher?
Dependent variables
What is the construct which is an *affected* variable that relies on the independent variable?
Interaction effect
What is the effect of one independent variable *depending* on the effect of another independent variable?
Main effect
What is the effect of one independent variable *ignoring* other independent variables?
External Validity
What is the extent to which one can *generalize results* that is strengthened by *random sampling*?
Internal validity
What is the extent to which one can make *causal inference* that is strengthened by *random assignment*?
5%
What is the likelihood of a *type I: alpha-error* where there is a rejection of the null hypothesis, but in reality the medication is not effective, giving a *false positive*?
20%
What is the likelihood of a *type II: beta-error* where there is a failure to reject the null hypothesis, but in reality the medication is effective, giving a *false negative*?
Power
What is the likelihood one can detect a *true effect*?
make causal claims
What is the main reason for conducting experiments? a) to have a more valid study b) to generalize to more people c) to make causal claims d) to collect evidence in the real world
-1.0
What is the perfect *negative correlation* r value?
+1.0
What is the perfect *positive correlation* r value?
C
What is the primary difference between pretest/posttest designs and within-groups designs? The number of participants used The number of times the dependent variable is measured The number of levels of the independent variable participants are exposed to There is no difference between the two designs.
Control Group
What is the pure, *no tx* or placebo group?
Over Sampling
What is the strategic *over-presentation* of population characteristics?
Research
What is the systematic accumulation of knowledge?
Compare Apples-to-Oranges
What is the top criticism of *meta analysis*?
Population
What is the total group of ALL people with defined characteristics?
Frequency distribution
What is usually a graph or table representing all values from least to greatest value with a *frequency count* of each *value*?
Random Error
What is when the *differences between participants* are *random*?
Open Ended Question
What learning strategies do you use when preparing for an exam?
Dropouts significantly different
What method of dealing with *attrition bias* accounts for significant different between dropouts and those who did not in terms of certain *characteristics*?
Tx attrition as dependent variable
What method of dealing with *attrition bias* considers that attrition may happen because of the *nature of the study* or *intervention*?
Intention to tx statistical adjustment
What method of dealing with *attrition bias* involves creating a *projection* of what would happen if the participants completed the study?
Remove noncompliant
What method of dealing with *attrition bias* involves removing pts who *do not* complete or follow the *proper protocol*?
- More competent - Different metrics - Different protocol
What needs to be considered in *multi-center studies* that can change the interactions?
Tx-by-Center interactions
What needs to be considered in *multi-center studies* where the tx may be more or less effective at different locations?
Ecological Validity
What results reflect *real world phenomena* that makes field research more *externally valid*?
Maintaining Competence
What section of code is this? "psychologists undertake ongoing efforts to develop or maintain their knowledge in a field"
Informed consent prior to recording voices or images in research
What section of code is this? Obtain informed consent prior to recording voices or images for date unless it consists of naturalistic observation in public places.
deception in research
What section of code is this? Only use deception if it is necessary. Do not deceive when their is physical pain or severe emotional distress. Explain all of the deception that was integral no later than the end of the experiment and allow the participant to withdraw their data.
informed consent
What section of code is this? Participants must always give informed consent and if they cannot physically give it, then it must be provided by a legal guardian. This must be documented.
Boundaries of Competence
What section of code is this? Psychologists must be competent in the fields in which they work.
Client/Patient, Student, and Subordinate Research Participants
What section of code is this? When research participation is a course requirement or an opportunity for extra credit, the prospective participant is given the choice of equitable alternative activities.
informed consent to research
What section of code is this? what the psychologist plans to do must be told truthfully to the participants unless deception is involved.
Offering Inducements for Research Participation
What section of code is this?Make reasonable efforts to avoid offering excessive or inappropriate financial or other inducements for participation that may be coercive, if offering professional services, must tell of risks, obligation, and limitations
Institutional approval
What section of code is this?When approval is required for a study, psychologists will provide accurate information about their research proposals prior to conducting the research.
Volunteer or non-response bias
What source of *bias* involves pts either volunteer or *refuse to participate*?
Mortality
What source of *bias* involves pts in one group being *more likely to die* where the dying may even be a *dependent variable*?
Attrition
What source of *bias* involves pts in one group is more likely to *dropout* where the dropping out may even be a *dependent variable*?
Contamination bias
What source of bias involves *pts* in the study *receiving* more than one *intervention* or tx in *conjunction* with the *study*?
Withdrawal bias
What source of bias involves *researcher withdraw pts* to avoid undesirable outcomes?
Inferential statistics
What statistical method *draws tentative conclusions* about a *population* based on observations of a *sample* selected from that population and furthermore make a probability statement about those conclusions to aid in their evaluation?
Rosenthal study
What study observed that *narrative reviews* provided *varied conclusions* while *meta-analysis* provided the *same conclusions*?
Parametric
What test is good for *statistical power* that requires *interval* or *ratio data*, variance *groups is similar*, and the *assumption* of a *normally distributed* population?
Nonparametric
What test makes *less specific assumptions* that does not directly incorporate estimates pertaining to population resulting in less power and less likelihood to detect a true effect?
Treatment
What the independent variable in experimental research is called
Nominal
What type of *data format* involves *categorical data*, categories like ethnicity, where numbers don't really have a meaning?
Ordinal
What type of *data format* involves greater numbers meaning *"more,"* but difference between units is not uniform, the difference between gold, silver, and bronze winners in the olympics?
Interval
What type of *data format* is the difference between units in uniform, but there is *no absolute zero* like temperature?
Ratio
What type of *data format* is where the difference between units is uniform, and there is an *absolute zero* like with a weight scale?
Crossover design
What type of *experimental design* involves *more than one* type of *tx* administered to each subject where the tx are *provided sequentially*, *rather* than *congruently*, and *comparisons* are made of the effects of the *different tx* on the *same subject*?
Narrative Literature Review
What type of *literature review* involves making the researchers own conclusions about what the literature says leading to bias in articles selected and how the researches a conclusion?
Systematic literature review
What type of *literature review* involves specific rules about how literature is selected for the analysis so anyone else could replicate how selected literature?
Meta-analysis
What type of *literature review* is a quantitative summary of a body of literature that attempts to get ride of bias of conclusion?
Randomized Block Design
What type of design *randomly assigns* at least one variable, but *can't randomly assign all variables*?
Longitudinal design
What type of design follows pts *over* a *significant time period*?
Correlation Design
What type of design has *no random assignment* and doesn't give inference to statistical tests?
Quasi-experimental design
What type of design has *no random assignment* but gets closer to the *causal inference* with *other methodological elements* with the goal of eliminating the top rival hypothesis?
Cross-sectional design
What type of design has *no random assignment*, but still *compare groups*?
Experimental design
What type of design has *random assignment*?
Positive Skew
What type of frequency distribution is skewed to the *left*?
Prospective research
What type of research involves *looking forward* and planing ahead to collect data and *cannot assume anything* since it can be *correlative* or *experimental*?
Retrospective research
What type of research involves looking back at existing data and therefore *cannot have random assignment* and therefore *cannot assume causal inference*?
- Pure random sample - Stratified random sample - Hierarchical random sample
What type of sampling are *probability sample* methodologies?
Mediators
What type of theory involves the relationship *between* two variables that can be explained by a third variable?
Moderators
What type of theory involves the relationship of one variable *on* another variable depending on a third variable?
Sample size
What, the bigger it is the better the *external* and *internal validity*?
Replication-plus-extension
When a researcher replicates their own study but includes additional variables to test other questions.
Measurement Bias
When a test or test item is not equally valid for individuals from a particular gender, ethnicity, or social class, or other group
Design Confound
When a variable other than the independent variable happens to vary systematically with the independent variable *due to an experimenter's mistake*
History threat
When an external/historical affects the majority of members of the treatment group when given the treatment. This can be prevented by including a comparison group.
Regression to the mean threat
When an extremely high or low-performance score recorded for the pretest that is likely to be not as extreme (closer to average) for the posttest. These extreme scores can give off the vibe that treatment motivated the effect of regression on the participant. This can be controlled for by using comparison groups as well to determine whether or not the treatment had the same, extreme effect as the comparison group who did not receive the treatment.
Selection-history threat
When an outside factor systematically affects one level of the independent variable of those participating in the study.
True
When conducting a research experiment, nearly every university, some criminal justice agencies, and many research organizations maintain Institutional Review Boards (IRBs) and require researchers to seek approval from these boards prior to doing research on human subjects.T/F
Social desirability
When the individual tries to answer questions in the most socially acceptable way.
Attrition threat
When the number of participants decreases from pretest to posttest (participants drop out of the study being conducted). In order to avoid this, researchers should remove the pretest scores of the subjects who did not complete the posttest- or at least not include the scores for the pretest average.
Double-blind designs
When the observer nor the participant does not have knowledge as to which group they are in.
Justification to waive informed consent:
When the research involves no more than minimal risk to participants. The waiver of informed consent will not adversely affect participants. The research could not be carried out if informed consent was required.
Null effects/null results
When the results of a study show that the independent variable made no significant change to the dependent variable.
Instrumental threat
When the tool used to measure the dependent variable becomes unreliable and changes over time (application, usage, different interpretation). Prevention can be conducted by calibrating the tool used for measurement to ensure that the pretest and posttest measures are equivalent. This can be done by solidifying codebooks before the experiment is conducted to ensure the observers are using the same measurement. Or researchers can conduct a posttest-only experiment.
Ceiling effects
When there is a problem in the experimental design where the independent variable groups score similar to the dependent variable, such that the scores fall to the higher end of the distribution.
Floor effects
When there is a problem in the experimental design where the independent variable groups score similar to the dependent variable, such that the scores fall to the lower end of the distribution.
Simple Regression
When using single independent variable to predict a single dependent variable.
Multiple Regression
When using two or more independent variables to predict a single dependent variable. All together; combined effect, which one will stand out more?
Negative Correlation
When variables vary in *opposite direction*?
Positive Correlation
When variables vary in the *same direction* it is called what?
References
Where did they get their information from?
Close Ended Question
Which do you do more often, restudy or self-test?
A
Which of the following can help prevent testing effects? Using a comparison group Establishing reliability of the measure Using a clear coding manual Employing a pretest-only design
A
Which of the following is NOT true of control variables? Control variables help define the control group. Control variables help establish internal validity. Control variables are essential in experimental designs. Control variables are kept the same for all participants.
C
Which of the following is NOT true of selection effects? Selection effects are a type of confound. Selection effects can occur when experimenters allow participants to choose their own treatment group. Selection effects are unimportant for interrogating internal validity. Selection effects can occur when researchers assign one type of person to one treatment group and another type of person to another treatment group.
D
Which of the following is a threat to internal validity found in within-groups designs but not in independent-groups designs? Selection effects Design confounds Demand characteristics Practice effects
D
Which of the following is an advantage of within-groups designs? These designs avoid order effects. These designs avoid demand characteristics. These designs are always possible. These designs rely on fewer participants.
C
Which of the following is true of instrumentation threats? They are the same as testing threats. They occur only when using mechanical instruments (e.g., blood pressure cuffs, scales). They can be avoided with counterbalancing. They are only problematic in observational research.
ratio
Which of the following levels of measurement is the most precise? ordinal interval ratio
B
Which of the following research designs is used to address possible selection effects? Posttest-only designs Matched-groups designs Pretest/posttest designs Correlational designs
B
Which of the following studies would NOT have a possible threat of observer bias? A study looking at the effect of eating fruits and vegetables and general physical fitness A study looking at the effect of tutoring on SAT scores A study looking at whether increasing the number of hours listening to music increases singing ability A study looking at the effect of increasing the rate of text messaging in a day on writing/English composition skil
D
Which of the following threats to internal validity can apply even when a control group is used? Attrition History threats Instrumentation threats Demand characteristics
The data you have gathered and analyzed accepts or rejects the hypothesis. The data either proves your hypothesis right or wrong.
Why or why not do you not accept or reject the hypothesis?
Tuskegee Study (1932-1972)
Withheld penicillin from participants with syphilis even after it became the standard treatment - and without their knowledge
Are Observations better than self-reports?
Without training, observers may often exhibit bias. • However, self-reports are, in some sense, just reports from untrained observers. - Poor operationalization can also increase the likelihood of bias for both self-report measures and trained observers. • If there is no clear operational definition for the behavior that is being rated, reliability may be low. If reliability is low, we can't have good construct validity.
Double-barreled questions
Wording that occurs when two ideas are combined into one question and it is unclear whether the answer is for the combination of both or one or the other question
Applied
_______ research is a type of applied research that involves practitioners in the research design and then the implementation and evaluation of a resulting policy or programmatic change. Its purpose is to improve some aspect of practice (e.g. combating gang violence). In action research, a problem is diagnosed; an intervention or strategy is devised, and data is collected to fine tune the intervention strategy. This type of research is an integral part of the problem-oriented policing philosophy.
Pure
__________ research is conducted for the purpose of expanding the body of knowledge. University professors conduct research primarily to expand the body of knowledge about a particular subject or to develop theories about social behavior. Typically, this research is published in scholarly journals and read by other scholars interested in the same topics.
Qualitative
____________ research probes the meaning of variables and assumes that the social world is best understood subjectively, often from the perspective of the research subject.
The Scientific Method
__________________________ uses process skills and follows a logical, organized, empirical, yet flexible approach to problem solving.
The Scientific Method
______________________________ is a systematic way of testing an idea, hypothesis, prediction or inference.
_____________________________________
____________________________________________
Applied
_________research is intended to collect the information necessary to solve a specific problem. Researchers, like practitioners, consultants, and some academics, conduct research to address and solve problems. This research is often done for a client, who intends to use the knowledge gained from the research to address a specific issue. Typically, this research is published in reports consumed internally within organizations. Although applied researchers often produce knowledge, their primary intent is to address a specific issue or solve a current problem. We will discuss applied research in more detail in a later chapter.
normal curve
a bell-shaped curve described by a certain mathematical function
nested case-control study
a case-control study nested within a cohort study
Within group
a change in the dependent variable in a group is measured across time: - Crossover - Latin Square - Splitplot
vignette
a description of a hypothetical situation, event, or scenario to which participants react
table
a display of data in a matrix format
skewed
a distribution that is not symmetrical
frequency polygon
a frequency distribution in which the frequencies are connected by straight lines
histogram
a frequency distribution in which the frequencies are represented bu contiguous bars
cumulative frequency distribution
a frequency distribution that shows the number of scores that fall at or below a certain score
time-series graph
a graph in which the abscissa (x-axis) represents time
scattergram
a graph showing the responses of a number of individuals on two variables; visual display of correlational data
frequency distribution
a graph that shows the number of scores that fall into specific bins, or divisions of the variable
Scatterplots
a graphed cluster of dots, each of which represents the values of two variables. The slope of the points suggests the direction of the relationship between the two variables. The amount of scatter suggests the strength of the correlation (little scatter, close to line of best fit, indicates high correlation). x-axis= Test 1, y-axis= Test 2
test-retest reliability
a method for determining the reliability of a test by comparing a test taker's scores on the same test taken on separate occasions. Results the second time should be close to results the first time
Binomial
a nominal value with only TWO possible values ex: yes vs. no, candidate a vs candidate b
frequency distribution
a numerical and graphical means of illustrating variation in values of a data set
Journal
a periodical that publishes technical and scholarly material, usually in a narrowly defined area of inquiry.
societal perspective
a perspective that takes account of all costs and benefits, no matter to whom they accrue time/burden of caregiver, transportation, etc.
Experimenter Bias
a phenomenon that occurs when a researcher's expectations or preferences about the outcome of a study influence the results obtained
Negatively worded questions
a question in a survey or poll that contains negatively phrased statements, making its wording complicated or confusing and potentially weakening its construct validity - Avoid negatively worded questions when the negative does not serve a useful purpose. Try not to use double negatives. • Example: "Try not to use double negatives in your questions." • VS. "Do not fail to discard double negatives from your questions" • Goal: simplicity. Even one negative can increase complexity. Two negatives are worse. • Book example (Morling, 2017, p.158): Abortion should never be restricted.
empirical questions
a question that can be answered by making objective observations
Leading questions
a question that encourages or implies a particular answer - Ex: what did you think of that amazing concert? - avoid. use neutral phrasing to try to capture person's "true" opinion
The coding guide
a record that specifies the variables of a study, the columns they occupy in the date file, and their possible values
positive correlation
a relationship between two variables in which both variables either increase or decrease together
Curvilinear relationship
a relationship in which changes in the values of the first variable are accompanied by both increases and decreases in the values of another variable
graph
a representation of data by spatial relationships in a diagram
interaction effect
a result from a factorial design, in which the difference in the levels of one independent variable changes causing the DV to change, depending on the level of the other independent variable; a difference in differences a HYPOTHESIS (prediction) about how the levels of one independent variable combine with another independent to influence the dependent variable, in a way that extends beyond the sum of the two separate main effects.
convenience sample
a sample collected from subjects that are available
percentile
a score below which a certain percentage of the cases in a distribution fall; a percentile is a score, not a percentage
central tendency
a single number that best represents a group/condition
A-B-A design
a single-subject design in which researchers establish a baseline, introduce the intervention and measure the same variable again, then remove the intervention and take another measurement
A-B-A-B
a single-subject design in which researchers establish a baseline, introduce the intervention, remove the intervention, and then reintroduce the intervention, measuring the dependent variable each time.
A-B design
a single-subject design in which researchers takes a baseline measurement and then introduce the intervention or experimental treatment and measure the same variable again
manipulation check
a small test to make sure that a manipulation is working.
operational definition
a statement of the procedures used to define research variables
null hypothesis
a statement or idea that can be falsified, or proved wrong no difference/effect
Mixed Design Analysis of Variance (mixed design ANOVA)
a statistical analysis that tests for differences between two or more categorical independent variables, while one is a between-subjects variable and another is a within-subjects variable
meta-analysis
a statistical strategy for comparing and integrating the results from multiple studies on the same research question to summarize what they say collectively on that topic
Two-Way Analysis of Variance (ANOVA)
a statistical test that allows us to simultaneously test how two separate nominal or categorical independent variables or (factors) influence the dependent variable, and how those independent variables interact to influence the dependent variables. AKA two way ANOVA or factorial ANOVA
Edited Books
a type of book that does not consist of one comprehensive study but instead a number of studies written by various authors
box-and-whisker plots
a type of graph based on median and percentiles rather than mean and standard deviation
response sets (nondifferentiation)
a type of shortcut respondents can take when answering survey questions. Rather than reporting sincere, thoughtful, and question-specific responses, a person may develop a pattern for responding
cross-sectional study
all individuals of a sample at the same point in time
risk difference
actual risk attributed to exposure
not a widely accepted form of randomization
adaptive
period for information bias
after enrollment and group assignment
Yay-Saying (and Nay-Saying--Response Sets + Solution)
agreeing with everything Solution: Reverse-word questions
group or cluster randomization
all members of a group/cluster are randomized together
variables
all of the factors that affect the observation results in an experiment
constants
all of the other variables that could possibly affect your experiment that must be kept the same in every trial in order to keep your experiment valid
statistical population
all subjects to whom the study might apply
Open-ended questions
allow any sort of response - Example: "What is your favorite style of music?" (without any response options provided) • Allows for rich information and unanticipated responses • Requires careful analysis and effortful (time-intensive) coding of information
Common SPSS Commands: Compute Function
allows creation of new variable from previous existing variables through arithmetic operations.
Multiple Coders
allows you to assess interrupter reliability
Third Variable (z)
alternative explanation for the relationship between x and y
Drawing Histogram
always include, title and labels y-axis is always frequency
Nomothetic
an approach based on laws or one that operates according to a system of laws
waiting-list control group
an empty control group often used in clinical research; participants in this group do not receive treatment or intervention until after the completion of the study
sampling error
an error that occurs when a sample somehow does not represent the target population. (you can't get the whole population, this error happens just from taking survey etc)
Measurement error
an error that occurs when there is a difference between the information desired by the researcher and the information provided by the measurement process
A composite illustration
an example from the General Social Survey.
Independent groups design
an experimental design in which different groups of participants are exposed to different levels of the independent variable, such that each participant experiences only one level of the independent variable
Mixed design
an experimental design that combines within-subjects and between-subjects methods of data collection
Feasibility
are the results and recommendations realistic and achievable.
Negative Correlations
as A increases, B decreases OR as A decreases, B increases
double-barreled questions
ask two questions at once. - Avoid this • Example: "Wasn't that guitar riff amazing and weren't the song lyrics clever?" • If someone says "yes" what does that mean? Are they referring to the guitar solo, the song lyrics, or both? • Goal: simplicity • One thought at a time.
Program evaluation
assesses the social reforms and innovations that occur in government, education, the criminal justice system, industry, health care, and mental health institutions.
adaptive randomization
assigning patients to a treatment group based on the previous success of the treatment as the trial progresses
Explanatory research
attempts to explain or find a cause for social trends and phenomena.
plagiarism
attempts to present another person's work as your own
Mean
average sum of values divided by N of cases
standard deviation
average distance from the mean
mean
average, variance, st. deviation
Predict (goal of psych/res)
based on what we know, what will future behavior, mental processes or outcomes
closed cohort
begins and ends with set group of individuals
Deductive reasoning
begins with a theory that explains social behavior and uses observations to test the theory.
Inductive reasoning
begins with observations of social behavior and uses those observations to develop a theory that explains the behavior.
B-data
behavioral observations
intrinsic sources of variation
biologically real; true genetic and phenotypic variation among subjects
helps to minimize investigator bias
blinding
helps to minimize performance bias
blinding
Theories
cannot be proven and lead to hypothesis
power
capacity to detect a difference between study groups
crossover RCTs take longer because of _____ and _____
carryover effect, washout period
hospital-/clinic-based case-control
cases have disease, controls received care from same institution
Nominal scale of measurement
categorical data often reported as frequencies and percentage
ordinal variables
center: mode, median spread: frequency dist.
Participant variables
characteristics such as age, gender, and intelligence that vary from one individual to another
Fence sitting
choosing middle response options (possibly to avoid reporting controversial opinions. - removing neutral or middle response options may help. But, what happens when people really are neutral or ambivalent?
Pearson's r
coefficient is represented by r coefficient that represents correlation coefficient range from -1 to 1+
ECHO is used to evaluate _____
cohort studies
baseline measurement
collected prior to randomization
Reason Rationalism
coming up with solutions based on well crafted thought
Two-Tailed (Non-Directional) Hypothesis
concerned with both regions of rejection about distribution. Just wants to know that there is a difference, does not care about how much of the specifics. ex. The mean of all bags doesn't have 17 M&M's.
One-Tailed (Directional) Hypothesis
concerned with the region of rejection only one tail of the sampling distribution. Directional difference: more than/less than, better/worse grade. ex. The mean of all M&M bags is less (or greater) than 17 M&M's.
Residual confounding
confounding that remains even after many confounding variables have been controlled
design confounds
confounds caused by a problem in the design of the independent variable.
Establish Internal Validity
considering confound- alternative explanation for change in dependent variable
Reliability
consistency of measurement, are you hitting the same spot every time?
Ethical Issues in Research Analysis: p-hacking
continuing to test data and post hoc analysis to produce significant results. Altering p value found in analysis report. ex. analysis gives alpha .06 and researcher reports .05.
face validity
correct on the surface
content validity
correct to experts
criterion validity
correlates to other measures or events
demand characteristics
cues in an experiment that tell the participant what behavior is expected
active control
current standard of care
4 goals of psychological research
describe, explain, predict, control
fatigue effect
decline in performance over time as a participant gets tired or bored.
effect size
degree of difference between treatment groups that is clinically important
Ways to Handle Missing Data: Listwise Deletion
delete case (or participant) completely where missing cases found. Only participants with all answers included are left in data set. Used when power is reached/exceeded
Program Evaluation
demonstrate the value or worth of an intervention or program within a specific context. Provide basis for making judgements about the value of a program, focused on describing elements, outcomes and value of programs. Performed as a team effort.
examples of derived statistics
density, percentages, adjusted scores, rates, ratios, any summary statistic (averages, variation)
criterion variable
dependent variable
Which numbers to report?
depends on level of measurement -describe center: use mode -describe spred: use frequency distribution
summary statistic
derived numerical value that summarizes a data set (central location, stats expressing variation
Levene's Test for Equality of Variances
determines if the two conditions have about same or different amounts of variability. See Notes for How to Read SPSS Output
Attributes of Scientific inquiry
determinism (events have causes), discoverability, objectivity (answer can be found, reputable and replicable), data driven, tentative conclusions, answerable questions, falsifiable theories, scientists are creative "skeptical optimists"
Program Accountability Model
developed to be more responsive to counselor's immediate need for information on effectiveness of program, to specify focus and determine modifications needed. Cyclical pattern: - Needs assessment, strategic program planning (action plan), program implementation, program assessment, program feedback.
range
difference between the highest and lowest scores in a distribution
ordinal interaction
differences are in same direction, but differ in magnitude when one group's predicted means is always greater than another group's predicted means. For example, the predicted male means are always greater than predicted female means, yet the differences between males and females varies by SES, therefore an ordinal interaction results
conditions
different levels of an independent variable in an experiment.
Adequacy
do the results provide an accurate description of the program and show something.
Test-retest reliability
do you get similar results if you administer the test more than once?
Interrater reliability
do you get the similar results from two different observers?
intention-to-treat analysis
dropouts analyzed as if they completed the study
probability sample
equal chance for population to get into the sample, e.g simple random sample
stratified randomization
equal subgroups based on certain factors
describing spread
e.g data goes from 3-6 -frequency: % distribution -minimum/maximum -range: low subtracted from high -quartiles -variance -SD
Within-Subjects Design
each participant experiences all levels of the IV (CONDITIONS)
Random Assignment
each participant is equally likely to be assigned to any level of the independent variable - *how to avoid selection confounds*
Between-Subjects Design
each participant only experiences 1 level of the IV (GROUPS) con: requires more people to test
Question order
earlier questions can change the way respondents understand and answer later questions - Preceding questions can frame subsequent interpretations and encourage particular response patterns on later questions. -Imagine that I asked a series of questions relating to oil or natural gas scarcity and then I asked how people felt about a new drilling project. • Do you think I would obtain different results if I asked a series of questions about damage to various ecosystems, and then I asked how people felt about the new drilling project?
ECHO
economic, clinical, humanistic, outcomes
example of between groups design
effect of appetitive stimuli on choice of impulse purchases One group given stimuli then choice, one group given impulse choice first
Between Group
effect of intervention on outcome is based on differences between groups: - Post-test - Pre-test/post-test - Solomon four group - Factorial design
case-control strengths (6)
efficient for rare diseases, relatively easy, fewer subjects needed, reduced costs, may study several risk factors for one disease, can investigate chronic diseases with long latency periods
single-blind study
either the participant, or the experimenter doesn't know which condition participants are assigned to.
missing data
empty cells in a data matrix
comparison group
enables us to compare what would happen both with and without the thing we are interested in
A study with two (or more) independent variables Each IV has at least two levels. IVs are crossed with each other, creating all possible combinations of the levels. IVs can be participant variables (e.g. gender) or manipulated variables. IVs can be within-groups variables or independent groups variables.
factorial design
Confounding variable
factors other than the independent variable that may cause a result.
exclusion criteria
factors that could confound or impair a study's results, safety factors
Type 2 error
fail to reject null hypothesis when it is false (false negative)
type 2 error
failing to reject a false null hypothesis
Type 2 error
failing to reject the null hypothesis when the null hypothesis is actually false this can be calculated by taking: 100%-power of test
bar graph
graphical representation of categorical data in which the heights of separated bars, or columns, show the various relationships between variable
frequency distribution
graphs that shows how many scores falls into particular bins, or divisions of the variable
historical control
group observed at different time or setting
Multiple Regression Analyses
help reduce the third-variable problem
we want the NNH to be as _____ as possible
high
Describe (goal of psych/res)
how do people think, feel and act in various situations or on average
z score
how far something is from mean (observation-mean)/SD *conditions for using Z score: -can only use for nominal dist, or approximately normal -have the mean and SD -possible values of Z: 0(the average), positive infinity, negative infinity z score outside of -3 to 3 is unusual
Variance
how spread out the scores are around the mean
Using a within-groups design (top), or a matched-pairs design (bottom) can help. A larger sample also helps.
how to account for individual differences:
sampling frame
how to go from the population to sample, e.g passing surveys out at the student center -the method is the survey *the sampling frame matters Results are influenced by: -who you ask: sampling frame -how you ask: phone v. online, type of questions
Ethical Issues in Research Analysis: HARKing
hypothesizing after results are known. Statistical analysis abuses often presented (ex. using a one-tailed rather than a two-tailed), type I errors more common.
when are reversal designs appropriate?
if the benefits of treatment aren't expected to last
Leading Questions (2 sources of question bias)
imply a "right answer" Solution: Use neutral wording
practice effects
improvement on a task over time due to practice.
error bars
in a graph, vertical lines that indicate plus or minus one standard deviation of the data or, less frequently, the standard error of the mean
Independent variable: bowl size Conditions: medium and large.
in the eating pasta experiment, what were the independent and condition variables?
a quasi-experiment with repeated measures.
interrupted time-series design Example: Does a food break result in judges making more favorable parole decisions?
6 ways of knowing about the world
intuition, tenacity/tradition, authority, reason/rationalism, empiricism, science
Assessment Confound (design confound)
involves *assessments that are different for one level* of an independent variable versus another Procedure 1: take an AP chem test Procedure 2: recall anything remembered
Material Confound (design confound)
involves *materials that are different for one level* of an independent variable versus another Procedure 1: read textbook chapter Procedure 2: watch a video lecture
Procedure Confound (design confound)
involves *participants doing things differently in one level* of an independent variable versus another Procedure 1: study for 30 min Procedure 2: study for 1 hr
Exploratory research
is often necessary in order to learn about social trends and phenomena that we know very little about.
prevalent cases
known existing disease
Range
largest value - smallest value
consent
legally given permission to participate in a study
advantages of retrospective cohort studies (2)
less time, less expense
L-data
life outcomes and records
Forced-choice formats
limit type of responses • Requires selecting from a pre-specified set of responses Example: • Narcissistic Personality Inventory (Raskin & Terry, 1988): Two opposing statements are provided and the respondent is asked to choose one. _____I really like to be the center of attention. _____It makes me uncomfortable to be the center of attention.
partial counterbalancing
limited amount of different orders.
t-test (t)
looks at differences between two groups on some variable of interest. The independent variable is categorical (male/female, undergrad/grad) and the dependent is continuous. See Notes for How to Read SPSS Output
we want the NNT to be as _____ as possible
low
the overall effect of one independent variable on the dependent variable.
main effect
flashbulb memories
major events - vivid but no more accurate - likewise, confidence is poorly calibrated with accuracy
hypothesis testing
make and test an educated guess about a problem/solution
independent variable
manipulate or shift controlled by experimenter Must have two or more "levels" or conditions
Straightforward manipulation
manipulating an independent variable with relative simplicity by presenting written, verbal, or visual material to the participants.
Experiments can support causal claims
manipulating the causal variable means potentially controlling for alternative explanations - use random assignment to ensure that the different groups are as similar as possible
Identify main effects from _______ _____ (compute the average for each level of the IV of interest, while averaging over the levels of the other IV).
marginal means
minimize confounding
match cases with controls
construct validity
matches up with theoretical constructs
comparison group
maturation threat isn't a problem if you have:
if left skewed...
mean < median
If right skewed..
mean > median
In a Normal Distribution....
mean, median, and mode are all the same
measure of central tendencies
mean, median, mode
EXCEL: HOW TO CALCULATE MEAN, MEDIAN, SD
mean- =average( highlight column ) enter median- =median( highlight ) enter SD- stdev( highlight ) enter
Post-test only
measure dependent variable after the manipulation
Concurrent validity
measure is administered and behavior is measured at the same time. the degree to which the measures gathered from one tool agree with the measures gathered from other assessment techniques
Predictive validity
measure is administered weeks, months, or years before behavior is measured predictive validity is the extent to which a score on a scale or test predicts scores on some criterion measure. Do the results accurately predict a future outcome?
biomarkers
measure of disease progression
semi-interquartile range
measure of variability defined as half the interquartile range
dependent variable
measured, result outcome variable take the form of records of behavior or attitudes, such as self-reports, behavioral observations, or physiological measures
continuous variable
measurements made on a scale that can be infinitely subdivided
data
measurements or statistics
effectiveness
measures how well an intervention performs in a clinical setting (real world)
physical trace observation
method examines evidence left behind ex: foot prints, fossils, going through people's trash
archival research
method of research using past records or data sets to answer various research questions, or to search for interesting patterns or relationships ex: prostitution in Gutaemala
margin of error
more common to see in reports -already includes confidence level using critical value -only have to subtract/add once and that's your range
mode
most frequently occurring score
Mixed Method Design Type: Exploratory
multiphase (often two) design where qualitative results are obtained first followed by quantitative. Used when: no measurements or instruments available to guide the study, variables are unknown or study has no guiding framework. Begins with qualitative to develop rich detail, develop theoretical framework or identification of variables, then quantitative used to determine association of variables within the study.
Mixed Method Design Type: Explanatory
multiphase (usually two) design in which qualitative data helps to explain initial quantitative data. Quantitative data are collected and analyzed prior to the collection and analysis of qualitative data. The quantitative phase allows the researcher to discover variables relationship with one another, the data is then used to inform/develop questions for qualitative piece.
similar to stable-baseline, but with multiple interventions staggered across time.
multiple-baseline design Example: Interventions throughout the school year to reduce several problem behaviors in a developmentally disabled student.
controls
must represent general population
Measured Variables
naturally occurring, observed, and collected. You cannot (or choose not to) manipulate.
confidence statement
need: -sample proportion/mean -range of possible values in population -confidence level
Making estimates for sample
need: -sample stats (mean or proportion) -standard error - based on SD but not the same -estimate of certainty
double-blind study
neither the participant nor the experimenter knows which condition participants are assigned to.
non-inferiority trial
new treatment not worse than active control
incident cases
newly diagnosed
OR = 1
no association
open label
no blinding
Null Hypothesis (H0)
no difference expected (variable x has no affect on variable y)
OR of 1.0
no difference in odds
Preexperimental Design
no random assignment and no experiment group. All participants are assigned the intervention.
unsystematic variability
noise- random fluctuation in data, that can't pinpoint a cause
scales of measurement
nominal, ordinal, interval, ratio
value in active control studies
non-maleficence
helps to minimize detection bias
non-study personnel assesses outcomes
quasi experiment combining an independent-groups variable with a repeated-measures variable.
nonequivalent control group interrupted time-series design example: tv access and larceny rate over time
If confidence interval crosses 1, what happens to the odds ratio?
nonsignificant
t-test
normal distribution interval level data, two independent groups ex: male vs female, health plan A vs health plan B result affected not only by size of the mean differences but also by the variability of each group, highly affected by outliers
case-control weaknesses (6)
not representative of whole population, incidence/prevalence unknown, inefficient when exposure is rare, selection bias, only one outcome at a time, information bias
lab notebook
notebook for recording information important to a study, such as design, procedure, and the planned analysis
NNT definition
number of patients needed to be treated for one to benefit
*What are the manuscript settings used in APA?*
o *Double-spaced* o One-inch margins o Left-aligned without hyphenation o Indent the first line of each paragraph o *Typeface: 12-point Times New Roman*
Empirical
o Answers are obtained by making structured or systematic observations o Results will provide clear support for or will clearly refute the hypothesis
Casual Observation
o Behaviour of people or animals may generate a general research topic o Example: Going to the mall and observing people's behaviour
What are the guidelines for avoiding biased language in APA writing?
o Describe people with a level of specificity that is accurate o Be sensitive to labels: call people what they prefer to be called o Acknowledge people's participation in your study
How do you know that your literature search is complete?
o Feel comfortable with your knowledge about the topic area o Have found a few recent research studies that are particularly relevant to your own interests
What are the main jobs you must complete and consider when conducting a literature review?
o Finding, reading, summarizing published research relevant to your question o Literature can help you form your research question, or determine if it has been solved o Help evaluate interestingness of research idea and how it fits into current literature o Help you with ideas on how to conduct your study
The Stanford Prison Experiment
o How being placed in different roles affected by individuals acted o Participants were either given the role of prisoner or guard o Each set of participants were given corresponding outfits and the guards were told to do whatever they had to do in order to maintain order o Study was called off after 6 days as the conditions increased to highly o Prisoners were found to become submissive and guards were found to become aggressive and abusive
Laboratory Observation
observe and record behavior in a controlled environment - More control over who you observe and what/how often conditions
Naturalistic Observation
observe and record behavior in its natural environment without intrusion - Get to see behavior "in the wild". - Less likely to change behavior.
participant observation
observer becomes part of the group being observed ex: Doomsday call
Little Albert
o John Watson and Classical Conditioning o Pairing an unconditioned stimulus with a conditioned stimulus to elicit a conditioned response o Albert was conditioned to fear fuzzy white rat o Albert was never unconditioned o It is unknown if Albert's mother gave informed consent
Name some well known conferences
o OBACPN(Ontario Behaviour Analytic Community of Practice Network, divided into regions) o ONTABA (Ontario Association of Behaviour Analysis) o OADD (Ontario Association of Developmental Disabilities) o ABAI (Applied Behaviour Analysis International) o APBA (Associative Professional Behaviour Analysis) o Persuasive Technology
Public
o Observations are available for evaluation by others
Objective
o Outcome is not skewed by bias
Pass 2
o Pay closer attention to the beginning and ending of each major section o Look up your highlighted words from the previous pass o Try to answer any questions you may have written down
The Bystander Effect
o Placed college students alone in rooms and gave them headphones o Subjects told they would be communicating with other students via intercom o Other students were actually recordings, one of which mentioned early on in conversation that they had occasional seizures o Later on that voice began to have trouble speaking and asked for help o Researchers measured how long it took the participants to obtain help o It was found that the more people the participant believed was on the call, the longer it took for them to get help
Pass 1
o Read through the framework of the paper This includes the abstract, introduction, conclusion, section headings, and any tables of diagrams including their captions o Underline any words or phrases that are unfamiliar to you o Ask yourself questions What is the point or thesis of this paper? What are the main arguments of the paper? Why is this paper important? How does it contribute to my field of study? o Write down any other questions you may have o Try to summarize the paper in a sentence of two in your own words
Milgram's Obedience Study (1963)
o Sought to discover how far subjects would go in order to obey an authority figure o Participants felt a sense of responsibility that they had killed someone o This study is an example of one which would not pass in ethics today
Practical Problems or Questions
o Stemming from daily life (e.g., placement of audio controls for less distracted driving) o Example: Legalization of marijuana and the effects on impaired driving
The Monster Study
o Studied how positive and negative feedback affected the way children learned language o Feedback was provided to children from an orphanage regarding speech stutters o Children in group 1 were told they did not have a speech impairment and were told they should ignore anyone who criticized the way they spoke o Children in group 2 were told they did have a speech impairment and were told they should never speak unless they could do it right o Feedback impacted the self-esteem of the children
Milgram Experiment
o Studied the psychology of obedience and how subjects would react if researchers encouraged them to engage in activity that went against their morals o Three roles: The experimenter, The teacher, and The learner o Teacher and learner were in separate rooms o Teacher gave the learner word pairings and if they responded with an incorrect answer the teacher was instructed by the experimenter to deliver an electric shock to the learner who was acting and not actually receiving the shocks o It was found that when instructed the majority of participants delivered the highest shock possible and when the learner pretended to have passed out the teachers believed that they may have killed the learner
Methods Subsection
o Subjects subsection o Procedure subsection o Apparatus subsection o Materials subsection
Pass 3
o Tie it all together so that you have a good idea of what the paper is trying to say at this point o With a critical eye this is the time for reflection and analysis o Take notes as you read looking at arguments, evidence, and conclusions o Ask yourself questions Did the authors do what they set out to do? Are the methods they used sound? Are their methods fluid and logical? What assumptions did they make?
In which order should the manuscript elements appear in APA?
o Title page o Abstract o Text (body) o References o Tables o Figures o Appendices (if any)
What are three things that an APA style report describes?
o Why the research was done o How the research was done o The results it produced
Reductionism
occurs when a researcher makes a prediction about how a group might behave based on data collected at the individual level of analysis.
Ecological fallacy
occurs when a researcher makes a prediction about how an individual might behave based on data collected at the group level of analysis.
systematic error
occurs when sample statistics are routinely unrepresentative of population parameters and is usually due to a faulty sampling procedure
SOURCES OF ERROR: measurement error
occurs when the measurement we obtain is not accurate portrayal of what we tried to measure. asked wrong question, collected wrong information
used in retrospective studies
odds ratio
Positive predictive value
of cases who test positive, the percentage who are actual positives People who actually have the disease
outliers
one extreme #, pulls up average, but not good indicator of avg
Factorial mixed
one independent variable is manipulated as independent-groups and the other is manipulated as within-groups - example: study wants 50 participants in each cell, meaning 100 participants are needed in total
interaction
one independent variable's influence on the dependent variable changes, depending on the level of other of the other independent variable
Mixed Method Design Type: Embedded
one type of data is in a lead role with other data supporting. Single phase. Used when researchers need to answer questions from a different perspective and their research has primarily focused on one type of data. Often insert a qualitative component into a quantitative design. ex. survey with open-ended questions.
experiment
one variable is manipulated (indep) one variable is measured (dep)
Online Devices
online surveys have always been associated with desktop computers but the expansion of devices such as smart phones or tablets add an additional consideration for design.
per-protocol analysis
only those who completed the study exactly are analyzed
rate ratios are used in _____
open cohorts
difference between opened ended questions and restricted questions
open ended questions people can answer in their own words while restricted questions answers are limited
Interval vs. Ratio scale
ration = observable interval = unobservable qualities
Ordinal (scale of measurement)
refers to categories that are ordered but not necessarily equidistant: very, somewhat, not at all strongly agree, agree somewhat, disagree somewhat, disagree strongly
Interval (scale of measurement)
refers to equidistant categories number of days, age in years
Nominal (scale of measurement)
refers to non-ordered categories (no order of importance. examples: male/female, racial group)
sampling variation
refers to normal variation (dispersion) in statistics measured on repeated samples from a population
sample bias
refers to systematic error in measurements
Incidence
refers to the number (or rate) of new cases of a disease/condition in a population during a given period
The principle that extreme observations tend to be followed by less extreme observations.
regression to the mean
Type 1 error and cumulative type 1 error
rejecting the null hypothesis when the null hypothesis is actually true false-positive the probability of this happening is = to alpha
used in prospective studies
relative risk
Data-Driven Needs Assessment
relies primarily on existing data as a means for objectively identifying needs of client populations.
empirical science
relying, or based on, direct observation, experiment of experience
sample bias
samples that have been poorly collected or assembled; measured samples could systematically misrepresent a population parameter even if they are precise
Purposeful Sampling
sampling specifically chosen population based on certain element that will highlight and inform a particular aspect of the study. Gives you useful data when you use this method of sampling for mixed methods.
How do you evaluate reliability?
scatterplots and correlations
discriminant validity
scores on the measure are not related to other measures that are theoretically different. Should correlate less strongly with measures of different constructs
Convergent validity
scores on the measure are related to other measures of the same construct. Should correlate more strongly with measures of the same/similar constructs
Mixed Methods: Multiphase
second part of the study depends on and elaborates the first. Two approvals needed from IRB. ex. interview, create theory from interview, generate survey to test theory.
Correlational Research (what makes it correlational)
seeks to figure out if two or more variables are related and, if so, in what way *both variables are measured*
factors that threaten internal validity (7)
selection, history, maturation, mortality/attrition, testing, instrumentation, statistical regression
Cronbach's alpha coefficient
splits the data all sorts of way then finds the average --> that is your correlation coefficient
2-way interactions: what they look like ("a difference in differences")
spreading interaction, crossing interaction
a study where the researcher observes behavior for an extended period before and after a treatment.
stable-baseline design Example: evaluating the effectiveness of a memory improvement strategy in an Alzheimer's patient.
research protocol
standardized document that covers all aspects of a study
Standard deviation (what it is, and how variability in the data affect it, but not how to calculate it)
standardized measure of variability, calculated based on square distances from points to to the mean
correlational coefficient
statistic that indicates the strength of the relationship between two variables
Correlation
statistical technique that measures degree of relationship between two or more variables (no manipulation of variables).
inferential statistics
statistics that help us to draw conclusions about populations
descriptive statistics
statistics that summarize a set of data
observable confounders (2)
stratification, statistical adjustment
helps to minimize selection bias
strict adherence to randomization schedule
case study
study of one individual in great detail
informed consent requirements (7)
study purpose and expected benefits, methods to be employed, anticipated inconveniences and discomforts, potential temporary and permanent risks, contingent compensation, all reasonable alternatives, right to withdraw
what is evaluated in RCTs? (7)
study sample, randomization and blinding, intervention and control group, clinical endpoints, trial findings, limitations and implications, application to clinical practice
informed consent
subject is presented with information about the study and willingly volunteers to participate
crossover RCT
subject receives treatment and control at different times
triple blinding
subjects, investigators, and those collecting data do not know randomization schedule
sample
subset of people we will collect data on
sample
subset of population of interest
sum of squares
sum of squared deviations from the mean
Cohort Studies
survey representatives of a group of individuals over time to follow a group's progress. Sampling of each, same group over time. Ex. Research Class
bias
systematic deviation of a study's result from the truth
Counterbalancing
systematically ordering treatments or measurements within a protocol to eliminate effect of order in which they are presented to participants. Each group switch out order to rotate the intervention. Intervention.
advantages of prospective cohort studies (4)
tailoring of study to research questions, purposefully capture data on important risk factors, using reliable and validated techniques to assess exposure and outcome measures, measuring exposure before disease occurs
non sampling error
target off bc/ of how sample was chosen. e.g being off because you did an email survey
good subject tendency
tendency of experimental participants to act according to what they think the experimenter wants
Speciesism
term used by analogy with racism and sexism by those who claim that it is unethical to treat animals differently from humans, particularly in research
What are the two types of reliability
test-retest and interrater
ANOVA (F)
tests the significance of group differences between two or more groups. The independent variable has two or more categories, the dependent is continuous. only determines if there is a difference between groups, not which is different. Ex. Low-, middle-, and high-income residents. See Notes for How to Read SPSS Output
cell mean
the average score of the participants in a single cell or a single group the mean on the dependent variable for participants with a specific combination of the levels of the independent variables
Bias Blindspot
the belief that we are unlikely to fall prey to cognitive biases Being biased about being biased
Decision analytic model
the calculation of expected outcomes of an intervention or treatment based on a set of input assumptions, usually performed using specialized software and techniques, such as Markov chain modeling or decision tree analysis
mean
the common average
face validity
the degree to which a scale appears, on the surface, to measure the intended variable - should be obvious enough to make sense but not too obvious that it introduces demand characteristics - ask yourself: does it appear to test what it claims?
Interrater reliability
the degree to which different observers agree on their observations. Two or more "independent" observers, looking at the exact same thing
Range
the difference between the highest and lowest scores in a distribution
Criterion Validity
the extent to which a measure is related to an outcome Does the measure allow you to distinguish people on the basis of a behavioral outcome? Two types: -Concurrent validity -Predictive validity
Construct Validity
the extent to which variables measure what they are supposed to measure
construct validity
the extent to which variables measure what they are supposed to measure
Construct validity
the extent to which variables measure what they are supposed to measure Is the measure ACTUALLY measuring what the researcher intends to be measuring?
relative frequency
the fraction or percent of the time that an event occurs in an experiment (frequency/total number of measurements)
animal welfare
the generally accepted term for concerns about the care and use of animals
Hazard ratio
the hazard of an event (e.g death) for one group relative to another
alternative hypothesis
the hypothesis that a proposed result is true for the population researcher predicts its going to happened
confounding
unaccounted for factor associated with both exposure and outcome
EXAMPLE OF VARIABLES: NBA DATASET
unit of analysis: NBA teams -team: nominal -total attendance: ratio (0 people has value) -did they make playoffs?: nominal -place in conference: ordinal, ranking
period for selection bias
up to and including enrollment and group assignment
Inferential Statistics
use samples to make informed guesses about the characteristics of the population from which the sample is drawn. Estimate likelihood or probabilities (p) that represent a true situation.
Common SPSS Commands: Recode Variable
used for collapsing or merging variables and for reverse coding.
interim analysis
used in cases of safety or ethical concerns
Predictor Variable
used to estimate the criterion (independent variable)
inferential statistics
used to inductively infer from known data (sample data) to the probably values of incompletely known quantitative properties; used to quantify degree of uncertainty around these inferences
descriptive statistics
used to quantitatively describe and summarize known data such as data obtained from samples
Measure of central tendency
used to represent a "typical" result or value around which results cluster
Trend Studies
used when interested in understanding how a variable of interest changes over time within some general population. Same general population, different people, same questions over time. Ex. surveying different individuals over multiple years from the same population, 5th grade students or incoming clients
Common SPSS Commands: Select Cases
useful when wanting to include only a portion or subset of data (ex. 1st year counseling student population specifically).
Improving response rates
—with rates comparable to mail surveys, online surveys benefit due to their lower cost. Research is still ongoing as to what is best to increase response rates, but email reminders and setting deadlines has shown promise.
Saying more than we can know
• Nisbett & Wilson (1977): People will report opinions and the reasons on which those opinions are based, even though they're effectively making it up as they go along. • For identical nylon stockings: people generally preferred the right- most one, and claimed their preference was based on quality. - Empirically, there may be many forces influencing our behavior. However, we may often be unaware of them. - When asked, we attribute reasons to our actions. Those inferred reasons might be biased or simply wrong.
Field Research Concerns
◦ Selecting a site and gaining access to it, ◦ Entering the field, ◦ Learning the ropes, and ◦ Developing rapport with members in the field.