Research Methods Review
median
the middlemost score in a distribution
Ordinal
rank, order. Meaningful but unequal intervals (1st, 2nd, 3rd)
Predictive Validity
- Scores obtained from a measure accurately predict behavior according to a theory - Example: Functional Analysis
Construct Validity
- Scores obtained from a measurement behave exactly the same as the variable itself - Example: Temperature going high and aggression increasing
What are the first two steps to the research process?
- Step 1: find a research idea - Step 2: form a hypothesis
Name four keys to a good presentation
- Visuals - Practice - Don't Read - Watch your clock
Principles of Tri-Council Policy Statement
- Weighing risks against benefits - Acting responsibly and with integrity - Seeking justice - Respecting people's rights and dignity
matched pair sampling
- a design in which one creates a set of two participants who are highly similar on a key trait and then randomly assigns individuals in the pair to different groups
placebo control
- a group where participants believe they are getting the treatment, but in reality they are not
directioanl hypothesis
- a hypothesis that makes a specific prediction as to the exact nature of the relationship between two variables - ex: my team will win the game
manipulation check
- a measure that helps determine whether the manipulation effectively changed or varied the independent variable across conditions
Strength of Experimental Method
Ability to demonstrate causation by observation. In other methods causality is implied
variables
All factors affecting observations and results in an experiment are ___________________.
manipulation check
An extra dependent variable researchers can include to determine how well an experimental manipulation worked.
Qualitative Research
Based on making observations that are summarized and interpreted in a narrative report
What constitutes deception?
1. using confederates 2. providing false feedback 3. presenting related studies as unrelated 4. lying about stimulus materials
Budgetary impact analysis
A type of decision analytic model that estimates the impact of a change in treatment or policy on overall expenditures
What is an acceptable reliability score?
>.70
Sampling error
The error that arises as a result of taking a sample from a population rather than using the whole population.
Positive Correlation
Variables tend to move in the same direction. As one variable goes up, the other goes up. As one variable goes down. The other goes down. Denoted by + before the correlations coefficient value.
Constructs
Variables that cannot be observed or measured directly
Fatigue effect
Deterioration in participant performance with repeated testing.
all protocols are reviewed by the _____
IRB
No causal inference
In a *retrospective* research study, what can you assume?
strongest type of study design in clinical research
RCT
Scales of measurement
Nominal Ordinal Interval Ratio
predictor variable
independent variable
prevalent cases
known existing disease
Range
largest value - smallest value
consent
legally given permission to participate in a study
we want the NNT to be as _____ as possible
low
In a Normal Distribution....
mean, median, and mode are all the same
used in retrospective studies
odds ratio
retrospective timeline
past to present
confounding
unaccounted for factor associated with both exposure and outcome
bias
systematic deviation of a study's result from the truth
sample
subset of population of interest
Construct Validity
"*Are they actually* measuring what they think they are measuring?"
Psychobiography
A type of case study in which the life of an individual is analyzed using psychological theory.
Between-subjects design
Comparisions are made between different groups of participants
What do all areas of ethics surround?
Do no harm
Empirical Article
First time report of result from research Ask: What are the arguments? What is the evidence?
Primary Sources
Firsthand reports in which the authors describe their own observations
Method
How did they conduct the study?
Survey
Most widely used data-gathering technique in social research. Sppropriate for research questions about self-reported beliefs and behaviours. -Started from positivist approach -Can ask about many things at one time and test several hypotheses
Internal Validity (3 criteria for causal claims)
No other possible explanations for why x and y are related
Why use deception?
No other way to conduct your study
Zero Correlation
No real association - line of best fit straight across, no trend
pretest/posttest
Other times, researchers want to measure a dependent variable before and after a manipulation.
Independent Variable
Variable that is manipulated in an experiment
Characteristics of Good Scientific Theories
Supported by data Falsifiable Parsimonious (Occam's Razor)
If an ethical dilemma is not mentioned in the BCBA guidelines, what should be referenced?
The APA Guidelines
The Horizontal-Vertical Illusion is an example of which method?
The Empirical Method
The following is an example of which method: All 3 year old children are afraid of the dark. Amy is a 3 year old girl. Therefore, Amy is afraid of the dark.
The Rational Method
Covariance, Temporal Precedence, Internal Validity
Three criteria for causal claims
Belmont Report
Three key ethical principles to guide all human research
References
Where did they get their information from?
Close Ended Question
Which do you do more often, restudy or self-test?
assent
willingness to participate in a study
Categorical Variable
A quality, such as chosen major, and is typically measured by assigning a category label to each individual
OR equation
AD/BC
Simultaneous Measurements - Inter-Rater Reliability
Agreement between two observers who simultaneously record measurements of the behaviors
A
An independent-groups design is also known as a _________ design. between-subjects matched-groups within-groups mixed
Confidence interval
An interval of values within which there is a given level of confidence (e.g., 95%) where the population value lies.
confound
An unaccounted-for variable in an experiment that provides an alternative explanation for an effect.
If data were not public, what may occur?
People could rearrange their data so it looks better
Pseudoscience
System of ideas often presented as science
before
The freedom to refuse or to discontinue the research study must be made clear to the participant ____________ the experiment.
How many articles is enough during a literature search?
The study (or studies) you find provides some justification for new research
Experimental group
The subjects who receive some special treatment in regard to the independent variable.
*Why* we cannot make causal claims (from correlational data)
We don't know if x caused y Or if y caused x Or if a third variable (z) caused x AND y
Peer review
What involves sending a journal article to *four peers* in the field and they critic it?
Statistical correlation
What is a type of *significance test* that does not tell about random assignment?
Double blind study
What is an experiment in which neither the subjects nor those who administer the tx know who is in the experimental group or control group that is and *limits experimenter expectancy bias*?
Criterion Variable
What is the behavior of interest that is changing? (dependent v) example: Behavior problems in the classroom
Experimental design
What type of design has *random assignment*?
Mean
average sum of values divided by N of cases
Benefits of factorial design
Establish Cause and Effect Conduct Multiple Experiments at Once Examine How a Combination of IVs Affects the DV (Interaction)
RCTs are required in phase _____ clinical trials as a part of an NDA
III
types of control groups (4)
placebo, active control, historical control, non-inferiority trials
case study
study of one individual in great detail
triple blinding
subjects, investigators, and those collecting data do not know randomization schedule
content validity
the extent to which a test samples the behavior that is of interest. Specifically asking if all necessary constructs were captured
coercion
compelling or influencing a decision to participate in a research study
Two-Tailed (Non-Directional) Hypothesis
concerned with both regions of rejection about distribution. Just wants to know that there is a difference, does not care about how much of the specifics. ex. The mean of all bags doesn't have 17 M&M's.
missing data
empty cells in a data matrix
a quasi-experiment with repeated measures.
interrupted time-series design Example: Does a food break result in judges making more favorable parole decisions?
Measured Variables
naturally occurring, observed, and collected. You cannot (or choose not to) manipulate.
scales of measurement
nominal, ordinal, interval, ratio
Empirical
o Answers are obtained by making structured or systematic observations o Results will provide clear support for or will clearly refute the hypothesis
ratio scales of measurement
order plus equal intervals plus true zero, score of 0 = no amount of variable being measured (true 0)
Reactivity
participants can change their behavior (react) when they know another person is watching - even if observer didn't give any cues about expectations Solution: Unobtrusive Observation (hide) Let people habituate (wait it out) Measure behavior's results
Predictor Variable
used to estimate the criterion (independent variable)
Dependent variable
variable that is being measured
Ethical Issues in Research Analysis: Coarsened Data
various techniques such as: rounding, grouping or transforming missing data changes or reduces amount of specificity of the data to reach significance.
experimenter expectancy effect
when a bias causes a researcher to unconsciously influence the participants of an experiment (also known as expectancy bias or experimenter effect)
Results
What did the author find?
Parsimony
As succinct as possible
Quantitative/continuous measurements
Ordinal, interval, ratio
counterbalancing.
The solution to order effects is
face validity
correct on the surface
Mortality
number of deaths
Ways to ensure confidentiality:
1. Anonymous data 2. Code data 3. Obscuring details of a particular case
What are the three key ethical principles?
1. Respect for persons 2. Beneficence 3. Justice
OR of 3.0
200% increase in odds
Ethics in Research
5 General ethical Principles: 1. Beneficence & nonmaleficence 2. Fidelity & responsibility 3. Integrity 4. Justice 5. Respect for rights & responsibilities
Empiricism
5 senses, taking in information through observation
OR of 1.5
50% increase in odds
Ethnography
An approach to field research that emphasizes providing a very detailed description of a different culture from the viewpoint of an insider in that culture to permit a greater understanding of it -Considered a methodology rather than a method
Matched Assignment
An assignment method in which subjects are first divided into subgroups based on a specific characteristic such as age, gender, and so forth; members of each subgroup are then randomly assigned to each group in the study to balance the characteristics across the groups. Alternative to random assignment, but difficult to assign similar characteristics.
data chart
Before starting the experiment, you must decide how you will record your data by creating an appropriate ____________________.
Factorial design
Factorial designs are designs with more than one independent variable (or factor).
How to Avoid Type II Error
Fail to reject the null. Increase the number of participants. Review threats to reliability of instruments. Consider alternate test statistics.
Justice
Participants are not exploited and treated fairly
Which conference will the professor of the BPSYC Research Methods class be presenting at this year?
Persuasive Technology
Pretesting the questionnaire
Piloting the questionnaire is critical for catching any errors or confusing items.
Scatterplot Form
Relationships can take three forms: independent, linear, and curvilinear
What is the relationship between reliability and validity?
Reliability is a prerequisite for validity
Face-to-face interview
Require that the interviewer and respondent meet to conduct the interview
Staged manipulation
Staging events during the experiment in order to manipulate the independent variable successfully.
Alternative Hypothesis
States that there is a relationship between the two variables being studied (as opposed to the null hypothesis)
Abstract
Summary
Falsifiability
The principle that a good scientific idea or theory should be capable of being shown to be false when tested using scientific methods..
power
The probability that an experiment will find a significant effect, assuming the effect actually exists.
question ending with a question mark
The problem must be written as a _________________________________________________________________.
Systematic
The scientific method has been developed as a set of strategies researchers can use making research what?
Availability Heuristic
Things that are easily brought to mind guide our thinking, Especially problematic when trying to determine how often something occurs.
One-group pretest post-test design
This design has one group, a pretest, a treatment, and a post-test. NO control group and NO random assignment
Netnography
Understanding social interaction on the internet (digital media)
History threat
When an external/historical affects the majority of members of the treatment group when given the treatment. This can be prevented by including a comparison group.
test-retest reliability
a method for determining the reliability of a test by comparing a test taker's scores on the same test taken on separate occasions. Results the second time should be close to results the first time
Hawthorne effect
a subject changes their behavior because they know they are being monitored
missing data code
a symbol, such as an asterisk (*), that is entered in a cell that has no data
Edited Books
a type of book that does not consist of one comprehensive study but instead a number of studies written by various authors
box-and-whisker plots
a type of graph based on median and percentiles rather than mean and standard deviation
risk difference
actual risk attributed to exposure
A composite illustration
an example from the General Social Survey.
Predict (goal of psych/res)
based on what we know, what will future behavior, mental processes or outcomes
B-data
behavioral observations
helps to minimize investigator bias
blinding
helps to minimize performance bias
blinding
crossover RCTs take longer because of _____ and _____
carryover effect, washout period
Reason Rationalism
coming up with solutions based on well crafted thought
Reliability
consistency of measurement, are you hitting the same spot every time?
highest level of validity
construct validity
content validity
correct to experts
demand characteristics
cues in an experiment that tell the participant what behavior is expected
effect size
degree of difference between treatment groups that is clinically important
Ways to Handle Missing Data: Listwise Deletion
delete case (or participant) completely where missing cases found. Only participants with all answers included are left in data set. Used when power is reached/exceeded
Utility
do results provide users with practical, easily used information.
Adequacy
do the results provide an accurate description of the program and show something.
Interrater reliability
do you get the similar results from two different observers?
Face validity
extent to which respondents can tell what the items are measuring
external validity
extent to which we can generalize findings to practice
Types of measurement validity
face validity, content validity, convergent validity, discriminant validity
Control Variable
held constant on purpose
we want the NNH to be as _____ as possible
high
Describe (goal of psych/res)
how do people think, feel and act in various situations or on average
Variance
how spread out the scores are around the mean
open cohort
individuals can join or drop out of study over time
placebo
inert treatment
advantages of retrospective cohort studies (2)
less time, less expense
L-data
life outcomes and records
partial counterbalancing
limited amount of different orders.
NNT definition
number of patients needed to be treated for one to benefit
Online Devices
online surveys have always been associated with desktop computers but the expansion of devices such as smart phones or tablets add an additional consideration for design.
difference between opened ended questions and restricted questions
open ended questions people can answer in their own words while restricted questions answers are limited
assent
required from children, a basic yes or no
Correlation
statistical technique that measures degree of relationship between two or more variables (no manipulation of variables).
descriptive statistics
statistics that summarize a set of data
Correlation & Strength of Association
+/- .10 - .29 = Weak +/- .30 - .49 = Moderate +/- .50 - .69 = Strong +/- .70 and above = Very Strong
snowball sampling
- non-probability - recruit subjects who then recruit more
convenience sampling
- non-probability - use of readily available subjects
purposive sampling
- non-probability - uses handpicked subjects that meet the researcher's needs
Who are the three categories of people who should be considered when making ethical decisions?
1. Research Participants 2. Scientific Community 3. Society
4 methods of survey administration
1. face to face 2. written 3. computerized 4. telephone
Consistent Positive Relationship
A correlation near +1.00
Pie Chart
A display of numerical information on one variable that divides a circle into fractions by lines representing the proportion of cases in the variable's attributes.
Stratified random sampling
A probability sampling method in which a population is divided into subpopulation groups called strata; individuals are then randomly sampled from each of the strata.
ordinal scale of measurement
Data are assigned to categories that can be ranked.
How to avoid design confounds
Equate everything that you can other than the independent variable (hold everything constant)
External validity
The degree to which the results of an experiment may be generalized.
Product Evaluation (Summative)
Identified extent to which goals and objectives were achieved or if improvement of discontinuations are needed; improving for next time, not the current product.
Debriefing
Participants are told about the nature of the study once it has been completed
Testing threat
Participants change as a result of the dependent variable/test being given to the subjects more than once. This can be stopped by including a comparison group or only using a posttest-only design experiment.
matrix
The summary data sheet should contain all the data in a _______ format and be a single sheet.
Interval
Equal intervals between units but *no meaningful zero* (IQ test)
interval ratio
The variable age (in years) is measured at what level of measurement?
Concurrent Validity
Validated measure and behavioral outcome measure are administered at the same time
Negative Correlation
Variables tend to move in opposite direction. As one variable goes up, the other goes down. As one variable goes down, the other goes up. Denoted by - before the correlation coefficient.
Mean
What *central tendency* calculates the *sum* of the scores in a distribution of a population or collection of things *divided* by the *number of scores*?
*Level 1 Heading*
*Center boldface, upper and lower case*
*Introduction*
- *A general introduction to the topic of the paper* - *A review of the relevant literature* - *A statement of the problem or purpose of the study with relevant variables clearly defined* - *A description of the research strategy that used to evaluate the hypothesis or to obtain an answer to the research question*
Weak (correlation strength)
.10/-.10
9.01 Conforming with Laws and Regulations
- Plan and conduct research in a manner consistent with all applicable laws and regulations, as well as professional standards governing the conduct of research. - Comply with other applicable laws and regulations relating to mandated-reporting requirements.
Figures/Tables
- Supplement the text—they should not duplicate information that has already been presented in text form o Should be mentioned in the text by number o The text should point out some of the more important aspects - Each is on a new page
general sampling methods
- probability - non-probability
different people
-between groups design -only go through one condition
How are samples used for inferences?
1) estimation of values of population parameters from sample summary statistics 2) hypothesis (H) testing
Goals of Debriefing
1. Clarifies the nature of the study 2. Allows the participant to express their feelings about being deceived 3. Removes negative consequence 4. Allows participants to reveal reactions or other comments about the study 5. Allows researcher to convey their appreciation
What are the first three steps to research?
1. Find an unanswered question that will serve as a research idea 2. Form a research question/hypothesis 3. Use the hypothesis to develop an empirical (observable) research study that will either support or refute the hypothesis
What three things should you take into consideration to form a good research topic?
1. It's a topic that strongly interests you 2. It's creative 3. It is not to broad
What is the minimal number of members on a research ethics board (REB)?
5
typical power level
80%
In popular media articles, why does the inclusion of participant variable information often denote a factorial design? Participant variables are often used as moderators. Participant variables are only found in factorial designs. Studies with participant variables always find significant interactions. Studies with participant variables are easier to write about.
A
True
A hypothesis is a statement that predicts the answer to our research question. T/F
Curvilinear Correlation
A non-linear relationship between co-variables
Field settings
A real-world setting for a research study.
Linear Relationship
A relationship of direct proportionality that, when plotted on a graph, traces a straight line
Quasi-filter Question
A type of survey research question including the answer choice "no opinion" or "don't know"
Context Effect
An effect in survey research when an overall tone or set of topics heard by a respondent affects how he or she interprets the meaning of subsequent questions
Hawthorne Effect
An effect of reactivity named after a famous case in which subjects reacted to the fact that they were in an experiment more than they reacted to the treatment
Hypothetical Construct
An entity that we know exists in each individual, but is *intangible* (cannot be overtly observed). Must define the construct using some tangible metric
nonequivalent control group pretest/posttest design
An independent-groups, quasi-experimental, pretest and posttest design that has at least one treatment group and one comparison group, but participants have not been randomly assigned to the two groups.
Beneficence/Concern for Welfare
Benefits outweigh the risks, unnecessary risks, confidentiality, informed
Experiment Qualities
Best for narrow scope, micro-level concerns. Isolate and target impact of few causal variables. Not viable for research of diverse variables
no
Do you accept or reject the hypothesis?
experimental design
Good ________________________ is important to doing successful research.
Why is a stable-baseline design more convincing than a simple pretest/posttest?
If the baseline is stable over time, you can rule out regression to the mean and maturation threats.
example of experiment
Indep: treatment -> level: 1- cognitive behavioral therapy 2- Family Therapy 3- wait list Dep: scores on Beck depression scale
Methods of Tenacity
Information that is accepted as true because it has historically been believed or superstition supports it
IRB stands for
Institutional Review Board
Basic Research
Intended to solve theoretical issues
Justice
Making sure the subjects are not exploited
Deception
Misinformation that a participant receives during experiment
Give two examples of intangible and abstract attributes to operationally define
Motivation and Self-Esteem
Nominal
No inherent order Categorical data Often open-ended, and demographic questions Frequency data and chi-square Nonparametric measures
Can you prove or disprove something in research?
No, you can only support
Causal Claims
One variable is responsible for changing another variable
Confound
Other possible explanations for an outcome
Journalism
Popular press for general public
Exposure therapy is often used for what psychological disorder?
Post Traumatic Stress Disorder
Debriefing
Post-experimental explanation of the study to the participant (especially if deception is used)
Basic Research
Primary goal is to gain knowledge
IQR (interquartile range)
Q3-Q1 (split data in half and then find the median of both halves)
Pre/post-test
Quantitative measure of dependent variable given before and after the intervention. Can compare similarities between groups before onset of study.
What other REBs is it possible for SLC students to need to go through?
Queens/Various Hospitals
Ordinal
Rank order, but not equally spaced Categorical data Frequency data and Spearman rho Nonparametric measures
Based on the video watched in class, what is the quickest method to extract information from a research article?
Read through the article in three passes
Physiological measure
Recordings of responses of the body.
Moderator
Refers to a trained facilitator used in focus group research who guides the focus group discussion
prevelance
Refers to the total number of cases of a condition at a given time. whether new or previously diagnosed
Exempt research
Research in which there is no risk is exempt from review by the IRB. Ex. Questionnaires, surveys, naturalistic observations, etc.
Belmont Report (1979)
Risk and benefits fairly distributed
Plan the Investigation
Scientific Method Step #3
Gather Data
Scientific Method Step #4
Organize Data
Scientific Method Step #5
Analyze Data
Scientific Method Step #6
Measurement error Individual differences Situation noise
Some common sources of unsystematic within-group variance (noise)
What are 2 criteria for a research question?
Specific and Testable
Non-statistical Definition of Sampling Error
The deviation between sample results and a population parameter caused by random processes
attributes
The different characteristics or values that a variable can take on are called that variable's:
Content validity
The estimate of how much a measure represents every single element of a construct.
Field Site
The one or more natural locations where a researcher conducts field research
Sampling Ratio
The ratio of the size of the sample to the size of the target population ie. If the population is 500 and the researcher samples 100, ratio is 100/500 = 0.20, or 20 percent
Respondent Fatigue
The tendency for respondents to lose interest in participating in research if they perceive their participation to be taking too long or requiring too much effort
Nonexperimental method
Use of measurement of variables to determine whether variables are related to one another. Also called correlational method.
Sample size
What, the bigger it is the better the *external* and *internal validity*?
A
Which of the following can help prevent testing effects? Using a comparison group Establishing reliability of the measure Using a clear coding manual Employing a pretest-only design
A
Which of the following is NOT true of control variables? Control variables help define the control group. Control variables help establish internal validity. Control variables are essential in experimental designs. Control variables are kept the same for all participants.
manipulation check
a small test to make sure that a manipulation is working.
statistical population
all subjects to whom the study might apply
variable
any measurable property that can take different values either within a subject, among different subjects, or between groups of subjects
control variable
any variable that an experimenter holds constant on purpose
Establish Internal Validity
considering confound- alternative explanation for change in dependent variable
Sample
part of a population
Mixed Methods: Multiphase
second part of the study depends on and elaborates the first. Two approvals needed from IRB. ex. interview, create theory from interview, generate survey to test theory.
period for selection bias
up to and including enrollment and group assignment
when all participants plan to receive treatment, researchers may compare those who receive early treatments to those who are still waiting (a strategy to reduce selection effects)
wait-list design
sampling variation
where multiple samples from the same population vary in composition and therefore vary in sample statistics
Semantic differential format
- scaling for rating a target along several dimensions
Online Surveys
-Advantages: Very fast and inexpensive. Allow flexible designs Can use visual images and/or audio or video -Disadvantages: Coverage Privacy Verification Design issues
The degree to which a quasi-experiment supports a causal claim depends on which of the following? Its design and its results Its duration and its sample size Its importance and its external validity Its statistical significance and its practical significance
A
Variables
A quantity or quality which varies across people or situations
Coding System
A set of rules used to categorize observations
Extraneous Variables
Any variables other than the independent variable that seem likely to influence the dependent variable in a specific study.
Which method obtains answers from an expert?
Authority
most famous cohort study in the US
Framingham Heart Study
Provide two examples of constructs
Intelligence and Hunger
Nominal
Looking at categories (sex, species, race)
Limitations of Tenacity
Lots of potential inaccuracies and no method for correcting erroneous ideas
U.S. Surgeon General
Orders federally-funded research proposals from the Public Health Service to be reviewed for ethical treatment of subjects
A
Practice effects and carryover effects are examples of _________ effects. order scientific between-person causal
ceiling effect, floor effect
Sometimes an IV has no effect because there is little room for improvement (_________), or little room for doing worse (__________).
level of precision
The measure of a sample's representativeness and therefore its ability to accurately predict conditions with the population from which it was collected is referred to as its:
Measured Score
True Score + Error
criterion validity
correlates to other measures or events
A research design where there are different participants at each level of the independent variable.
independent group design Example: one group gets a treatment, the other gets a placebo.
cohort studies differ from RCTs as cohort studies do not _____
randomize exposure
Ethics
set of principles for behaving in a way that is morally correct
mode
the most common score in a frequency distribution
descriptive statistics
used to quantitatively describe and summarize known data such as data obtained from samples
Scatterplot Direction
Linear relationships can have a positive or negative direction
Alternative explanations
Part of causal inference; a potential alternative cause of an observed relationship between variables
Placebo effects
Participant group improves when they think they are given the treatment for the experiment but are actually not given the treatment.
Post-test
Quantitative measure used to determine the effects of the intervention between the treatment and control group.
Discussion
Recap results, limitations, future directions
data chart
The ______________________ is used to collect information during the experiment in an organized way.
independent variable
The _________________________ is the variable you choose to manipulate or change in the experiment.
Internal Validity
The ability to eliminate alternative explanations of the dependent variable Rule out variables other than the treatment by controlling experimental conditions. Threats: -Selection Bias -History Effects -Maturation -Mortality -Diffusion of Treatment or Contamination -Experimenter Expectancy
External Validity
The ability to generalize experimental findings to events and settings outside the experiment itself
Statistical Validity
"What do the mathematical results tell me?"
American Psychological Association (APA) Guidelines
- Ethical Guidelines for the Use and Treatment of Human Participants in Research - First set of guidelines for protecting human subjects in research (1973) - Updated and expanded many times, and periodically reviewed - Is considered the gold standard of guidelines that are out there
What is science?
- Systematic Empiricism - Empirical Questions - Creates Public Knowledge
stratified sampling
- probability - Ensure certain subgroups are represented or preserve proportions of subgroups in the population within the sample - requires definition of non-overlapping stratification criteria - randomly select in each stratum
simple random sampling
- probability - each member of population as equal change of being selected for the same - selection is independent of one another - each
cluster sampling
- probability - used when naturally occurring pockets of population are geographically dispersed - randomly select from pockets
Selection Bias
A threat to internal validity when groups in an experiment are not equivalent at the beginning of the experiment
Bivariate
A two-variable analysis, such as a crosstabulation of disease incidence by age or a comparison of mean health care costs by sex
random assignment
-control for individual differences -creating equivalent groups
two types of experimental groups
-different people -same people
experimental manipulation
-true experiment
Hypothesis
proposed explanation made on the basis of limited evidence as a starting point for further investigation
4 steps to calculate variance
1. subtract the mean from each score 2. square each of the deviation scores 3. add the squared deviation scores 4. divide the sum of the squared deviations by the number of scores
Variable
Any event, situation, behavior, or individual characteristic that varies—that is, has at least two values.
Expectancy effects (experimenter bias)
Any intentional or unintentional influence that the experimenter exerts on subjects to confirm the hypothesis under investigation.
Extraneous Variable
Any variables that you are not intentionally studying in the experiment.
What is considered minimal risk?
At no greater risk for harm than you would be doing an everyday task
In the case of a factorial design, another term for independent variable is: Main effect Cell Factor Moderator
C
What is the best method for determining how a variable should be measured?
Consult previous research involving the same variable
Nonparametric Statistic
Make no assumptions about the population fitting normal distribution.
The following is an example of which method: Black cats are bad.
Methods of Tenacity
What does the REB take into consideration?
Mitigation of risks, benefits outweigh risks, fair, informed consent
Secondary Sources
Secondhand reports in which the authors discuss someone else's observations
random assignment if you have a large sample... Or, with matched groups if dealing with a smaller sample.
Selection effects can be avoided by
Measurement error
The degree to which the measurement, used to assess a participant in a study, is different from the true measurement value. When these are random they are capable of inflating or deflating the true scores. But if they are systematic they may result in a biased measurement.
Independent Variable
*2 or more levels* chosen by the researcher - manipulated variable - comes first - *causing the change*
posthoc test (Bonferroni correction)
- statistical tests that examine all of the possible combinations of conditions in a way that statistically accounts for the fact that we did not predict them ahead of time - if there is an overall difference, you can use posthoc analyses to compare groups to each other
internal consistency reliability
- the degree to which the individual items on a scale are interrelated or correlated - ask yourself: do the questions measure the same general topic? - statistics: Cronbach's alpha
wording mistake: not specific
- wording is not specific and too general - fix: make statements clear and direct about what we are asking
Cells
A condition in an experiment that represents one of the possible combinations of two independent variables.
Random Number Table
A list of numbers that has no pattern in it and is used to create a random process for selecting cases and other randomization purposes
Negative linear relationship
A relationship in which increases in the values of the first variable are accompanied by decreases in the values of the second variable.
Conceptual replication
A researcher has the same research question as another study but uses different procedures. This improves external validity.
Contamination bias
What source of bias involves *pts* in the study *receiving* more than one *intervention* or tx in *conjunction* with the *study*?
Inferential statistics
What statistical method *draws tentative conclusions* about a *population* based on observations of a *sample* selected from that population and furthermore make a probability statement about those conclusions to aid in their evaluation?
Nominal
What type of *data format* involves *categorical data*, categories like ethnicity, where numbers don't really have a meaning?
Positive Skew
What type of frequency distribution is skewed to the *left*?
1. State the Question or Problem 2. Hypothesis 3. Plan the Investigation 4. Gather Data 5. Organize Data 6. Analyze Data 7. Draw a Conclusion
List the 7 steps of the Scientific Method that we will use.
Physiological
Measure of a biological state Examples: Hormone levels, blood flow in brain, heart rate, pupil dilation Conceptual variable: Arousal Operational variable: Pupil dilation
Coefficient of Determination (r^2) & how to calculate
Measure of the amount of variance in one variable can be predicted from the other variable. Calculated by squaring the value of r. Ex. If correlation of variable x to y is r = .70 then r^2 = .49. .49 x 100= 49% of the criterion value is explained by the predictor value.
How is validity established?
Measurement procedure must accurately capture the variable that it is supposed to measure
Type II Error
Not claiming a result as significant when you should have. Fail to reject the null hypothesis when there is a difference. Missed a finding that is actually there. False negative. ex. Telling a 9 month pregnant woman that she is not pregnant.
Systematic observation
Observations of one or more specific variables, usually made in a precisely defined setting.
Observer Bias
Observer records what they expect (or want) to see - only looking for behavior they want Solution: Develop good scoring rubrics Use more than one observer
Masked (blind) design
Observers are unaware of the conditions to which participants have been assigned and are unaware of the purpose of the study. - can't provide cues when reach correct answer
Exposure Therapy with Response Prevention is often used for what psychological disorder?
Obsessive Compulsive Disorder
Cofounding variable
Occurs when two variables are linked in a way that makes it difficult to sort out their specific effects.
negative predictive value
Of cases who test negative, the percentage who are actual negatives (don't have the disease)
What are the main problems with headlines?
Often -oversimplified -overconfident Sometimes -misrepresent findings
Types of Pre-experimental Design
One-shot case study design One-group pretest post-test design Static-group comparison
Confirmation Bias/Cherry-picking
Only picking out what supports your theory (usually people with an agenda) *This is the biggest bias*
Panel study
Research in which the same sample of subjects are surveyed at two or more points in time, usually to assess changes that occur over time.
Surveys
Research methods in which researchers use questionaires or interviews.
tentative
Research of ethics is _________ and not chiseled in stone.
Why Questions
Research warn against using these for a survey. Only appropriate if a researcher wants to discover a respondent's subjective understanding or informal theory
bias
Sponsorship ______occurs when the sponsor of a research project attempts to influence the study design or interpretation of data for its own benefit.
Correlation coefficient
Statistic showing strength of the relationship (r)
STROBE
Strengthening The Reporting of OBservational studies in Epidemiology
When conducting a literature review which two key elements should you make note of?
Subject Words and Authors Names
Types of Measures of Reliability
Successive Measurements Simultaneous Measurements Internal Consistency
Review Article
Summary of published studies done in a research area (e.g., a meta-analysis)
Which method obtains answers from habit or superstition?
Tenacity
Fence Sitting (Response Sets + Solution)
Tendency to be neutral about everything Solution: Use even number of responses. But sometimes people really have no opinion!
Observed P value
Test calculates probability of obtaining the observed results if the null hypothesis is true
Behavioural Theories
Testing predictions that are part of a theory
Institutional Review Board;5
(IRB) stands for ____________. They must have a minimum of _____ members.
Core Principles of Ethics
- Respect of Persons - Beneficence / Concern for Welfare - Justice
empty control
- a group that does not receive any form of the treatment and just completes the dependent variable
covariation
- when changes in one variable are associated with changes in another variable - the two variables change together in a systematic way - correlation does not equal causality
between groups advantages
-Generalizable (more participants are more representative) -collect more data at a given level -shorter time for each participant
Sample
The collection of subjects selected for observation in an empirical study.
same people
-within groups design -all participants go through all conditions
What types of validity do you need for causal claims?
1. Construct 2. Statistical 3. Three criteria for cause --temporal precedence --covariation --internal
IV x PV design
A factorial design that includes both an experimental independent variable (IV) and a nonexperimental participant variable (PV).
Correlation Coefficient
A numerical index of the degree of relationship between two variables.
Split-half reliability
A reliability coefficient determined by the correlation between scores on half of the items on a measure with scores on the other half of a measure.
Test-retest reliability
A reliability coefficient determined by the correlation between scores on a measure given at one time with scores on the same measure given at a later time.
Disclosing
A researcher decides how much to reveal about himself or herself and the research project. Disclosure ranges on a continuum
Interval scale
A scale of measurement in which the intervals between numbers on the scale are all equal in size.
Scientific literature
A series of related studies that are conducted by various researchers and have tested similar variables.
Participant observation
A technique of observing a situation wherein the observer takes an active role in the situation.
Attitude of Strangeness
A technique in field research in which researchers study a field site by mentally adjusting to "see" it for the first time or as an outsider. E.g., gift giving customs
Cluster Sampling
A type of random sample that uses multiple stages and is often used to cover wide geographic areas in which aggregated units are randomly selected; samples are then drawn from the sampled aggregated units, or clusters. Involves multiple stages and clusters. Less expensive than random sampling, but less accurate (each stage has sampling errors). Researcher first samples clusters, and then draws a second sample from within the clusters selected • Advantage when dealing with geographically dispersed populations • A design with more clusters is better because elements within clusters (e.g., people living on the same block) tend to be similar to each other (e.g., more alike than those on different blocks)
Non-random Sample
A type of sample in which the sampling elements are selected using something other than a mathematically random process
Standard-format Question
A type of survey research question in which the answer categories fail to include "no opinion" or "don't know"
What Each Section Contains or Does for the Reader
Abstract - provides essence of research Method - who, what and how section Results - report important findings and statistics/themes. Present what is being assessed, demographics, hypotheses. Discussion - interprets results for reader. Does not restate results in detail, only if needed.
Face validity
The degree to which a measurement device appears to accurately measure a variable.
Quasi-Experimental Designs
Addresses the need to study the effect of an independent variable in settings in which the control features of true experimental designs cannot be achieved.
Primary Criteria to Determine Suitability
Adequacy Utility Feasibility Propriety
Reliability
The degree to which a measure is consistent.
Census
An attempt to count everyone in a target population. Every 5 years (2021)
Double-blind placebo control strudy
An experimental design where the participants nor the observers for the study know which group is receiving the treatment or the placebo treatment.
One-shot case study design
An experimental design with only an experimental group and a post-test, no pretest Disadvantages: does not take into account external factors, findings may not generalize, may falsely assume an effect when none exists
Static Group Comparison
An experimental design with two groups, no random assignment, and only a post-test. Also called the post-test-only nonequivalent group design
Conceptual
Applied interventions arise from a specific and identifiable theoretical base rather than being a set of packages or tricks
Behavioural
Applied interventions deal with measurable behaviour (or reports if they can be validated)
Applied
Applied interventions deal with problems of demonstrated social importance
Effective
Applied interventions produce strong, socially important effects
Analytic
Applied interventions require an objective demonstration that the procedures caused the effect
Covariance (3 criteria for causal claims)
Are x and y related?
reliability capicty
The extent to which a measure consistently measures a social phenomenon is referred to as the measure's:
Interestingness
The extent to which the answer is in doubt, whether it fills a gap in the research literature, and whether it has important practical implications
Convergent validity
The extent to which the scores on one measure are related to scores collected from a similar or different measure.
ANOVA
The farther away from *1.0* the *more* statistically *significant* the result in what test for *multiple related groups*?
True
The first causal rule, temporal order, requires that the cause must precede the effect. In other words, the variable that is alleged to be the cause of another variable must happen first. T/F
Experimental Group
The group that receives the treatment in an experiment
Journal Article Impact
The impact factor (IF) is a measure of the frequency with which the average article in a journal has been cited in a particular year. It is used to measure the importance or rank of a journal by calculating the times it's articles are cited.
correlation
The second causal rule, ___________, requires that the variables in a causal relationship be related to one another, or change together. A change in one variable must be associated with a change in another variable. Correlations can be positive or negative.
Trend studies collect the same data over time from different samples of the same population. Panel studies collect the same data over time from the same sample. Cohort studies collect data over time from samples of individuals who share common characteristics.
There are three types of longitudinal research designs.
What are the problems with claims based on intuition?
COGNITIVE BIASES: thinking the easy way Being swayed by a good story - good story bias Being persuaded by what easily comes to mind - availability heuristic MOTIVATIONAL BIASES: thinking what we want Focusing on the evidence that fits our beliefs - confirmation bias Being biased about being biased - bias blind spot
Active Deception
Commission
Self Questionnaire- Compensation for respondents
Compensation in survey research is often controversial. It often consists of gift certificates, prize drawings, and lotteries, and when prepaid, are more successful.
Pearson Product Moment Correlation Coefficient: Pearson r
Computed when data for both measured variables are quantitative in nature (interval or ratio) and linear in relationship - both variables are continuous. Also used as a prediction for correlation of determination. ex. GPA, years of experience, NCE score.
Dr. Gavin is conducting a 2 x 4 independent-groups factorial design. How many cells are in his design? 2 4 6 8
D
Objective-Oriented
Determine extent to which program objectives have been achieved. Ex. Midterm & Final - identify discrepancies between stated objectives & desired outcomes
Bimodal
Distributions with two modes
Accumulation of knowledge
Each new generation of researchers don't have to reinvent the wheel because research is what?
Floor effect
Failure of a measure to detect a diff erence because it was too difficult
Confirmation Bias
Focusing on the evidence that fits our beliefs a tendency to search for information that confirms one's preconceptions
Name two examples of guidelines that may be requested in order to participate at a conference
Having an abstract/website
Between Group difference
How do you know how much *difference* is expected *due to chance* between groups?
Stem
In a survey, the statement, question or prompt to which the respondent replies.
Contingency questions
In certain instances, questions may not be relevant to all respondents. Typically, the researcher will ask a question to determine who should answer the subset of questions and who should ignore them. The questions that are only answered by a subset of the respondents are referred to as contingency questions.
Subjects
In experimental research, the cases or people used in research projects and on whom variables are measured
What questions are helpful to ask while reading journals and journalism?
Is original research evidence cited? Has the research been published in a peer-reviewed journal yet? How does coverage of the same journal article differ across news stories?
Deontological Objection
It is never okay to lie even if it is used for a study and people may not be at risk for discomfort, etc. Could lead to the development of mistrust and suspicion in future studies
Self Questionnaire- Mail distribution and return
Typically self-administered surveys are distributed through the mail, and the responses are subsequently mailed back to the researcher. Researchers make every effort to ensure that returning the survey is as easy as possible.
Understanding
Understanding is a precondition for greater depth, not an end in itself. Learn to think and act within a member's perspective. Empathy, or adopting another's perspective
Present/Present Bias
We tend to not look for absences—while it is easy to notice when something is present. It is a failure to notice appropriate comparison groups
Median
What *central tendency* is the *midpoint* or *mid-score* in a distribution that presents with 50% falling below and 50% falling above that line?
Decrease variance
What *increases power* by *reducing* the *within group differences* make observing sample groups that are more *homogeneous* which removes potential outliers?
- Diffusion - Imitation - Rivalry - Equalization
What *threats* to *internal validity cannot* be ruled out by *random assignment*?
Covariance
What do all correlational studies buy you?
Predictor Variable
What factors may be related to the behavior of interest? (independent v) examples: Recess time, Public v private school, proportion of boys in the class
Sampling Non-human elements
What helps *maximize generalization* by looking at *surrounding* elements?
Increase effect
What increases power by *upping* the intensity of tx or the *independent variable*?
Correlational
What involves *measuring relationship* between different variables whether *direct* or *indirect*?
Type II: beta error
What is a *failure* to *reject* the *null hypothesis*, but in *reality* the medication is effective, giving a *false negative*?
Convenience sample
What is a *non-probability sampling* methodology that includes anyone who is convenient?
Quota sample
What is a *non-probability sampling* methodology that sets *quotas* for proportions of people to be included with different characteristics?
Hypothesis
What is a *prediction* about how two or more variables are related that has to be *testable*?
Stratified random sample
What is a *probability sampling type* first stratifies the population into *categories* then randomly selects from the categories?
Dispersion
What is the *distribution* of *values around* some *central value*, such as an *average* which observes how typical that value is?
Alternative hypothesis (HA)
What is the *hypothesis* in which you predict that a *relationship exists*, also known as *research hypothesis*?
Null hypothesis (Ho)
What is the *hypothesis* that there is *no relationship* or that nothing happens in which *you want to reject*?
Mode
What is the *most frequent value* that occurs?
Frequency
What is the *number of times* a value occurs?
Less artificial
What is the *strength* of *field* based experiments?
Less confounds
What is the *strength* of a *lab* based experiment?
Regression to the mean
What is the *threat* to *internal validity* that is just *due to chance*, where *extreme* scores tend to *regress* towards the *mean* in *later* measurements?
Compensatory rivalry
What is the *threat* to *internal validity* where the *control* group is *creative* or *motivated* to *out perform* the *tx group*?
Interaction effect
What is the effect of one independent variable *depending* on the effect of another independent variable?
Main effect
What is the effect of one independent variable *ignoring* other independent variables?
External Validity
What is the extent to which one can *generalize results* that is strengthened by *random sampling*?
-1.0
What is the perfect *negative correlation* r value?
+1.0
What is the perfect *positive correlation* r value?
Ecological Validity
What results reflect *real world phenomena* that makes field research more *externally valid*?
Maintaining Competence
What section of code is this? "psychologists undertake ongoing efforts to develop or maintain their knowledge in a field"
Ratio
What type of *data format* is where the difference between units is uniform, and there is an *absolute zero* like with a weight scale?
Longitudinal design
What type of design follows pts *over* a *significant time period*?
Retrospective research
What type of research involves looking back at existing data and therefore *cannot have random assignment* and therefore *cannot assume causal inference*?
Mediators
What type of theory involves the relationship *between* two variables that can be explained by a third variable?
Moderators
What type of theory involves the relationship of one variable *on* another variable depending on a third variable?
Replication-plus-extension
When a researcher replicates their own study but includes additional variables to test other questions.
True
When conducting a research experiment, nearly every university, some criminal justice agencies, and many research organizations maintain Institutional Review Boards (IRBs) and require researchers to seek approval from these boards prior to doing research on human subjects.T/F
Random sample
When each member of the population has an equal chance of being selected to represent the whole.
attrition threat
When some types of participants are more likely to drop out of the study than others
Pseudo Survey
When someone who has little or no real interest in learning information from a respondent uses the survey format to try to persuade someone to do something
supported or does not support
When stating your conclusion, you must decide if the data collected _____________________________ the hypothesis.
synergistic effect
When the combination of two variables results in something greater than simply the sum of the individual variables. an effect where two variable combine to produce an outcome that is bigger than what each individual variable contributes a type of interaction effect
Double-blind designs
When the observer nor the participant does not have knowledge as to which group they are in.
D
Which of the following is a threat to internal validity found in within-groups designs but not in independent-groups designs? Selection effects Design confounds Demand characteristics Practice effects
D
Which of the following is an advantage of within-groups designs? These designs avoid order effects. These designs avoid demand characteristics. These designs are always possible. These designs rely on fewer participants.
ratio
Which of the following levels of measurement is the most precise? ordinal interval ratio
Qualitative
____________ research probes the meaning of variables and assumes that the social world is best understood subjectively, often from the perspective of the research subject.
Quantitative
_____________ research measures variables numerically and assumes that the social world can be understood objectively.
table
a display of data in a matrix format
skewed
a distribution that is not symmetrical
cumulative frequency distribution
a frequency distribution that shows the number of scores that fall at or below a certain score
time-series graph
a graph in which the abscissa (x-axis) represents time
scattergram
a graph showing the responses of a number of individuals on two variables; visual display of correlational data
Binomial
a nominal value with only TWO possible values ex: yes vs. no, candidate a vs candidate b
frequency distribution
a numerical and graphical means of illustrating variation in values of a data set
positive correlation
a relationship between two variables in which both variables either increase or decrease together
interaction effect
a result from a factorial design, in which the difference in the levels of one independent variable changes causing the DV to change, depending on the level of the other independent variable; a difference in differences a HYPOTHESIS (prediction) about how the levels of one independent variable combine with another independent to influence the dependent variable, in a way that extends beyond the sum of the two separate main effects.
operational definition
a statement of the procedures used to define research variables
Drawing Histogram
always include, title and labels y-axis is always frequency
Mixed design
an experimental design that combines within-subjects and between-subjects methods of data collection
adaptive randomization
assigning patients to a treatment group based on the previous success of the treatment as the trial progresses
seeing red experiment
example of posttest only experiment
interference effects
exhaustion (e.g., memory)
between subjects
expose participants to one level of treatment - randomly assign participants to one condition- + internal and external validity -power
OR > 1
exposure associated with higher odds of outcome
OR < 1
exposure associated with lower odds of outcome
Anatomy of a two way Factorial ANOVA - 4 groups (2X2)
f(#,#) = #.##, p= ##, eta^2 = .## f = f test symbol (first number) is between subject degrees of freedom and within subjects degrees of freedom #.## = f score p ## = significance level eta ##= calculated effect size
A study with two (or more) independent variables Each IV has at least two levels. IVs are crossed with each other, creating all possible combinations of the levels. IVs can be participant variables (e.g. gender) or manipulated variables. IVs can be within-groups variables or independent groups variables.
factorial design
Confounding variable
factors other than the independent variable that may cause a result.
Type 2 error
fail to reject null hypothesis when it is false (false negative)
type 2 error
failing to reject a false null hypothesis
Type 2 error
failing to reject the null hypothesis when the null hypothesis is actually false this can be calculated by taking: 100%-power of test
when are reversal designs appropriate?
if the benefits of treatment aren't expected to last
Leading Questions (2 sources of question bias)
imply a "right answer" Solution: Use neutral wording
unobservable confounders (2)
instrumental variable methods, Heckman selection model
helps to minimize attrition bias
intention-to-treat analysis
Material Confound (design confound)
involves *materials that are different for one level* of an independent variable versus another Procedure 1: read textbook chapter Procedure 2: watch a video lecture
t-test (t)
looks at differences between two groups on some variable of interest. The independent variable is categorical (male/female, undergrad/grad) and the dependent is continuous. See Notes for How to Read SPSS Output
the overall effect of one independent variable on the dependent variable.
main effect
flashbulb memories
major events - vivid but no more accurate - likewise, confidence is poorly calibrated with accuracy
minimize confounding
match cases with controls
median
middle score
A factorial design with at least one independent-group factor, and at least one within-groups factor.
mixed design
Mixed Method Design Type: Exploratory
multiphase (often two) design where qualitative results are obtained first followed by quantitative. Used when: no measurements or instruments available to guide the study, variables are unknown or study has no guiding framework. Begins with qualitative to develop rich detail, develop theoretical framework or identification of variables, then quantitative used to determine association of variables within the study.
double-blind study
neither the participant nor the experimenter knows which condition participants are assigned to.
OR = 1
no association
open label
no blinding
Null Hypothesis (H0)
no difference expected (variable x has no affect on variable y)
OR of 1.0
no difference in odds
Preexperimental Design
no random assignment and no experiment group. All participants are assigned the intervention.
NNH definition
number of patients needed to be treated for one to be harmed
The Bystander Effect
o Placed college students alone in rooms and gave them headphones o Subjects told they would be communicating with other students via intercom o Other students were actually recordings, one of which mentioned early on in conversation that they had occasional seizures o Later on that voice began to have trouble speaking and asked for help o Researchers measured how long it took the participants to obtain help o It was found that the more people the participant believed was on the call, the longer it took for them to get help
Milgram's Obedience Study (1963)
o Sought to discover how far subjects would go in order to obey an authority figure o Participants felt a sense of responsibility that they had killed someone o This study is an example of one which would not pass in ethics today
Methods Subsection
o Subjects subsection o Procedure subsection o Apparatus subsection o Materials subsection
Positive predictive value
of cases who test positive, the percentage who are actual positives People who actually have the disease
Factorial mixed
one independent variable is manipulated as independent-groups and the other is manipulated as within-groups - example: study wants 50 participants in each cell, meaning 100 participants are needed in total
experiment
one variable is manipulated (indep) one variable is measured (dep)
cohort studies _____ clinical trials
parallel
Concurrent measure design
participants are exposed to all levels of an IV roughly at the same time and a single measurement of the DV is recorded
repeated measures design
participants are measured on the dependent variable more than once after than have been exposed to each level (or condition) of the independent variable
cases
people afflicted with a disease
P-data
physiological (Heart Rate, Skin detection, etc)
Power
probability that a study will show a statistically significant result when an independent variable truly has an effect in the population.
used in prospective studies
relative risk
Data-Driven Needs Assessment
relies primarily on existing data as a means for objectively identifying needs of client populations.
A research design where the same participants are tested at each level of the independent variable
repeated measures design
precision
repeated samples that are collected
Ethical Responsibility
responsibility to individual subjects, to discipline of science, and to society as a whole
similar to stable-baseline, but treatment is taken away after improvement to see if progress reverses.
reversal design: Example: how will depressive/coping behavior change in response to treatment and nontreatment periods?
risk difference equation
risk exposed - risk unexposed
risk ratio equation
risk exposed divided by risk unexposed
sample bias
samples that have been poorly collected or assembled; measured samples could systematically misrepresent a population parameter even if they are precise
factors that threaten internal validity (7)
selection, history, maturation, mortality/attrition, testing, instrumentation, statistical regression
research protocol
standardized document that covers all aspects of a study
Standard deviation (what it is, and how variability in the data affect it, but not how to calculate it)
standardized measure of variability, calculated based on square distances from points to to the mean
informed consent requirements (7)
study purpose and expected benefits, methods to be employed, anticipated inconveniences and discomforts, potential temporary and permanent risks, contingent compensation, all reasonable alternatives, right to withdraw
what is evaluated in RCTs? (7)
study sample, randomization and blinding, intervention and control group, clinical endpoints, trial findings, limitations and implications, application to clinical practice
face validity
the degree to which a scale appears, on the surface, to measure the intended variable - should be obvious enough to make sense but not too obvious that it introduces demand characteristics - ask yourself: does it appear to test what it claims?
all professional activities of psychologists.
the ethics code covers _________.
uncertainty
the exact value of any biological property is difficult to know for certain
Skewed distribution
the extent to which a distribution leans to one side of the mean instead of being evenly distributed based on where tail is
Construct Validity
the extent to which variables measure what they are supposed to measure
Single-blind experiment
the participant is unaware of whether a placebo or the actual drug is being administered.
Observer Effects
the participant picks up on observer's unintentional cues and responds the way they think they should Solution: Develop good scoring rubrics Use more than one observer
Placebo effect
the phenomenon in which some people experience some benefit after the administration of a placebo.
What specific individuals and number of those people should be on a research ethics board (REB)?
two with expertise, one the ethics, one with no affiliation
Inferential Statistics
use samples to make informed guesses about the characteristics of the population from which the sample is drawn. Estimate likelihood or probabilities (p) that represent a true situation.
inferential statistics
used to inductively infer from known data (sample data) to the probably values of incompletely known quantitative properties; used to quantify degree of uncertainty around these inferences
null effect
when there is no significant covariance between the variables of interest.
Mixed Method Design Types: Triangulation
single phase approach that simultaneously collect both qualitative and quantitative data, merge the data and use results to understand a research problem. Qualitative, quantitative and previous research data carry equal weight.
Speciesism
term used by analogy with racism and sexism by those who claim that it is unethical to treat animals differently from humans, particularly in research
Range
the difference between the highest and lowest scores in a distribution
Main effect
the effect each variable has by itself.
animal welfare
the generally accepted term for concerns about the care and use of animals
Hazard ratio
the hazard of an event (e.g death) for one group relative to another
the investigator
the person who is in overall charge of the research and has the greatest responsibility to see that ethical principles are followed
negative correlation
the relationship between two variables in which one variable increases as the other variable decreases
frequency distribution
a graph that shows the number of scores that fall into specific bins, or divisions of the variable
variables
all of the factors that affect the observation results in an experiment
Statistical Control
applying the proper statistical analysis when evaluating outcome data to minimize effect of potential confounding variables. Control extraneous variables.
double-barreled questions
ask two questions at once. - Avoid this • Example: "Wasn't that guitar riff amazing and weren't the song lyrics clever?" • If someone says "yes" what does that mean? Are they referring to the guitar solo, the song lyrics, or both? • Goal: simplicity • One thought at a time.
Experiments must
establish internal validity systematic variability unsystematic variability
Factorial Design
evaluate two or more independent variables, including interaction of the variables in relation to the dependent variable. Help account for extraneous variables; allows assessment of interaction.
factorial design
evaluates multiple interventions in a single experiment
exclusion criteria
factors that could confound or impair a study's results, safety factors
Trend Studies
used when interested in understanding how a variable of interest changes over time within some general population. Same general population, different people, same questions over time. Ex. surveying different individuals over multiple years from the same population, 5th grade students or incoming clients
Common SPSS Commands: Select Cases
useful when wanting to include only a portion or subset of data (ex. 1st year counseling student population specifically).
Every observation and survey will always have some degree of error T/F
*TRUE*
Key Note Speakers
- Try to attend these during bigger conferences - Often there are concurrent sessions
forced choice scale
- a scale where a person must choose between only two response alternatives for each item - limits sensitivity and encourages people to skip the item if they do not have a strong opinion - basically a yes or no scale, but two alternatives may not be sufficient - solution: add more varied responses
Cronbach's alpha
- a statistic used to evaluate internal consistency reliability of a scale - can range from 0 to 1.0
maturation
- a threat to internal validity stemming from either long-term or short-term physiological changes occurring within the participants that may influence the dependent variable ex: participants may become tired, bored, or hungry as the study progresses
testing effect
- a threat to internal validity where participants' scores may change on subsequent measurements simply because of their increased familiarity with the measurement - ex: if you continue asking the same question participants may begin to guess the purpose, participants may remember earlier answers which could influence current answers
confounding variables
- a variable that the researcher unintentionally varies along with the manipulation
Which of the following is a difference between true experiments and quasi-experiments? Quasi-experiments do not use random assignment. Quasi-experiments do not involve manipulated variables. Quasi-experiments cannot have comparison groups. Quasi-experiments cannot have pretest measures.
A
Which of the following is a within-groups quasi-experimental design? Interrupted time-series design Nonequivalent control group design Matched group factorial design Multiple regression design
A
Which of the following is another term for ecological validity? Mundane realism Everyday realism Experimental realism Cultural realism
A
theory introduction
A _______ is a statement, or set of statements, that attempts to explain the social world.
Parameter
A characteristic of the entire population that is estimated from a sample ie. The percentage of city residents who smoke cigarettes
Inconsistent Relationship
A correlation near zero
Bar Chart
A display of quantitative data for one variable in the form of rectangles where longer rectangles indicate more cases in the variable category. Usually, it is used discrete data, and there is a small space between rectangles.
Sampling Distribution
A distribution created by drawing many random samples from the same population
Normal Distribution
A function that represents the distribution of variables as a symmetrical bell-shaped graph.
Normal distribution
A function that represents the distribution of variables as a symmetrical bell-shaped graph.
Systematic Sampling
A type of random sample in which a researcher selects every kth (e.g., 12th) case in the sampling frame using a sampling interval. Simple random sampling with a short-cut for random selection. Often yields similar results. ie. Sample 300 names from 900. After a random starting point, you select every third name of the 900 and get a sample of 300. Sampling interval is 3
Stratified Sampling
A type of random sample in which the researcher first identifies a set of mutually exclusive and exhaustive categories, then uses a random selection method to select cases for each category. More representative. Researcher controls relative size.
Full-filter Question
A type of survey research question in which respondents are first asked whether they have an opinion or know about a topic, then only the respondents with an opinion or knowledge are asked a specific question on the topic (ie. Do you have an opinion on that? If yes, do you agree or disagree?)
Threatening Questions
A type of survey research question in which respondents are likely to cover up or lie about their true behaviour or beliefs because they fear a loss of self-image or may appear to be engaging in undesirable or deviant behaviour -Use only after a warm-up when an interviewer has developed a rapport and trust with respondents
True
A unit of analysis can be an individual, a group, a community, a state, or even an entire nation. T/F
Cluster
A unit that contains final sampling elements
dependent variable
A variable (often denoted by y ) whose value depends on that of another.
Manipulated Variable
A variable a researcher controls, usually by assigning participants to a particular level of the variable
independent variable
A variable that causes change to another variable is called a(n):
dependent variable
A variable that changes as a result of a change in another variable is called a(n):
intervening variable
A variable that occurs between an independent and dependent variable and affects the relationship between them is called a(n):
Measured Variable
A variable whose levels are simply observed and recorded. Typically accomplished using scales, inventories, rulers, or devices
Interrupted time-series design
A within-groups quasi-experimental design that repeatedly measures participants (in the singular treatment group), on a dependent variable before, during, and after the interruption caused by some event.
If a researcher is concerned about external validity, which of the following would you recommend with regard to conducting small-N designs? Do not conduct small-N designs if you are concerned about external validity. Compare the results of a small-N design with other studies. Conduct only reversal designs. Use only one's own clients/patients/students.
B
variables
Before deciding on the independent variable, you must first determine all possible ______________.
4. A 3 × 3 × 2 factorial design includes how many levels in the second independent variable? A) 1 B) 2 C) 3 D) 4
C - 3
7. A(n) _____ is an experimental design that combines within-subjects and between-subjects methods of data collection. A) A-B design B) A-B-A design C) mixed design D) single-subjects design
C - mixed design
2. ____ is when the influence of one independent variable on the other reverses across the levels of the other variable. A) A main effect B) A correlation C) A crossover interaction D) A marginal effect
C- crossover interaction
testing threat
Change in participants as a result of taking a test more than once. Did performance improve because of the manipulation, or because participants are already familiar with the test?
Variables
Characteristics or conditions that change or have different values for different individuals
Splitplot
Combination of easily manipulated and difficult (impossible; ex. race, gender) to manipulate independent variables are both assessed for effect on dependent.
Self-reports and accuracy
Consequently (of flashbulb memories and confidence) a, retrospective self-reports may be inaccurate (i.e., may have poor construct validity as a measure of past behaviors). • Possible solution: observe directly, rather than rely one self-reports.
Pattern and Parsimony
Converging evidence from correlational studies helps us get closer to causal claims
Autocorrelation
Correlation between *1 variable with itself*, measured at *two different time points*
Cross-Sectional Correlation
Correlation between *2 variables*, measured at the *same time*
payer perspective
Costs represent the charges for health care products and services allowed or reimbursed by the payer
A small-N design that involves providing treatment and then removing treatment is known as a(n): interrupted time-series design. stable-baseline design. multiple-baseline design. reversal design.
D
Dr. Gavin decides that instead of conducting a 2 x 4 independent-groups factorial design, he is going to conduct a 2 x 3 x 4 mixed factorial design. Which of the following things will NOT have to change? The number of main effects that need to be examined The number of interactions that need to be examined The number of participants needed The number of researchers needed
D
Which of the following is true of a scientific literature? It comprises studies conducted by a single researcher. It comprises studies that have tested the exact same variable. It comprises studies conducted all in the same year. It comprises studies conducted with different methods.
D
Which of the following is true of cultural psychology? Its researchers function only in theory-testing mode. Its main focus is to disprove theories. It typically finds support for a theory across different cultural contexts. It starts with the assumption that theories apply to all humans, regardless of culture.
D
6. _____ is a single-subject design in which researchers take a baseline measurement and then introduce the intervention and measure the same variable again. A) An A-B-A design B) A single-blind procedure C) A mixed design D) An A-B design
D - A-B design
Frequency Claims
Describe a particular rate or degree of a SINGLE variable. Examples: -1 in 5 people have a dental filling -64% of Americans report being very happy with their partner
Reliability
Did the authors measure their constructs/variables well?
Discriminant Validity
Does our measure only weakly relate (or have no relationship) to other measures that have little/nothing to do with our construct?
Convergent Validity
Does our measure positively correlate with other measures that are assessing similar constructs? AND/OR Does our measure negatively correlate with other measures that are assessing opposite constructs?
Criterion-Related Validity
Does the measure allow you to distinguish people on the basis of a particular criterion or behavioral outcome? Researchers identify behavioral outcomes that the measure should be related to and then correlate their measure with the occurrence of those outcomes
Reliability
Does the measure produce consistent results?
C
Dr. Deveraux has conducted a study that has resulted in a null effect. Nonetheless, she suspects that there truly is a causal relationship between her independent and dependent variables. Which of the following is UNLIKELY to be to blame? A weak manipulation An insensitive measure Too many participants A reverse confound
C
Dr. Hoff is curious as to whether children in a kindergarten classroom will be friendlier after talking about friendship. Immediately before and after the discussion, she has several undergraduate research assistants code the "friendliness" of a group of kindergarteners. Which of the following threats should Dr. Hoff be most concerned about? History Maturation Instrumentation Testing
B
Dr. Sanderson is curious as to whether exposing people to violent video games causes them to be more aggressive. She assigns half her participants to play a violent video game for 5 minutes and the other half to play the same game for 25 minutes. Afterward, she has them play a board game and has a well-trained coder determine whether they are very aggressive in their playing style, barely aggressive, or not at all aggressive. She finds that a vast majority of her participants, regardless of group assignment, are rated as very aggressive. This outcome would be known as a(n): weak manipulation. ceiling effect. floor effect. outlier.
External Validity
Extent to which study can be generalized to other settings and populations.
Varieties of Psychological Research
From observations, and serendipitous events
Empirical Approach
Gathering information using your senses. Relying on systematic observations! Unbiased Objective Publicly verifiable events
intuition
Gut feeling, emotion, "follow your heart"
Give two examples of well defined, easily observed, and measured variables to operationally define
Height and Weight
Ordering items in a questionnaire
How items are ordered on a questionnaire can affect the answers respondents give. Certain early items may influence later responses, as might the ordering of general versus specific questions on a given topic.
Methods of Intuition
Information that is accepted on the basis of a hunch
Filler items
Items included in a questionnaire measure to help disguise the true purpose of the measure.
Which method obtains answers from reasoning and logical conclusions?
Rationalism
Univariate Statistics
Statistical measures that deal with one variable only Easiest way to describe: frequency distribution
variance
The average of the squared differences from the mean.
True
The unit of analysis is the 'what' or 'whom' about which researchers gather information during a research project. T/F
Self Questionnaire- Follow-up mailings
This entails sending either a reminder or another survey to respondents who have not already returned the survey.
How do we demonstrate consistency of a measure?
Two scores (one score from the new measure and one score from a well-established measure) are represented for each person in a scatter plot
Hierarchical random sample
What is a *probability sampling type* randomly selects from *progressively smaller units*?
Type I: alpha-error
What is a *rejection* of the *null hypothesis*, but in *reality* the medication is not effective, giving a *false positive*?
Ceiling
What is a *threat* to *internal validity* from *instrumentation* where *can not go any higher* with instrumentation?
Symmetrical distribution
What is also known as *normal distribution* when the frequency *distribution* from both sides of the *median mirror* the other?
Normal distribution
What is also known as *symmetrical distribution* when the frequency *distribution* from *both sides* of the *median mirror* the other?
Procrustean Data Torturing
What is an *analysis and publication bias* where *data* is *manipulated* to support a *hypothesis*?
Random assignment
What is an experimental technique for assigning participants or animal subjects to different groups in an experiment using randomization and lends strength to *internal validity*?
More comfounds
What is the *weakness* of *field* based experiments?
Artificial
What is the *weakness* of a *lab* based experiments?
Over Sampling
What is the strategic *over-presentation* of population characteristics?
Informed consent prior to recording voices or images in research
What section of code is this? Obtain informed consent prior to recording voices or images for date unless it consists of naturalistic observation in public places.
deception in research
What section of code is this? Only use deception if it is necessary. Do not deceive when their is physical pain or severe emotional distress. Explain all of the deception that was integral no later than the end of the experiment and allow the participant to withdraw their data.
informed consent
What section of code is this? Participants must always give informed consent and if they cannot physically give it, then it must be provided by a legal guardian. This must be documented.
Offering Inducements for Research Participation
What section of code is this?Make reasonable efforts to avoid offering excessive or inappropriate financial or other inducements for participation that may be coercive, if offering professional services, must tell of risks, obligation, and limitations
Institutional approval
What section of code is this?When approval is required for a study, psychologists will provide accurate information about their research proposals prior to conducting the research.
Volunteer or non-response bias
What source of *bias* involves pts either volunteer or *refuse to participate*?
Mortality
What source of *bias* involves pts in one group being *more likely to die* where the dying may even be a *dependent variable*?
Attrition
What source of *bias* involves pts in one group is more likely to *dropout* where the dropping out may even be a *dependent variable*?
Withdrawal bias
What source of bias involves *researcher withdraw pts* to avoid undesirable outcomes?
Parametric
What test is good for *statistical power* that requires *interval* or *ratio data*, variance *groups is similar*, and the *assumption* of a *normally distributed* population?
Nonparametric
What test makes *less specific assumptions* that does not directly incorporate estimates pertaining to population resulting in less power and less likelihood to detect a true effect?
Ordinal
What type of *data format* involves greater numbers meaning *"more,"* but difference between units is not uniform, the difference between gold, silver, and bronze winners in the olympics?
Meta-analysis
What type of *literature review* is a quantitative summary of a body of literature that attempts to get ride of bias of conclusion?
Floor effects
When there is a problem in the experimental design where the independent variable groups score similar to the dependent variable, such that the scores fall to the lower end of the distribution.
Because if participants are randomly assigned to groups, the groups are unlikely to have systematic differences.
Why does random assignment reduce the chance of confounds?
The data you have gathered and analyzed accepts or rejects the hypothesis. The data either proves your hypothesis right or wrong.
Why or why not do you not accept or reject the hypothesis?
placebo group
a control group that is exposed to an inert treatment
frequency polygon
a frequency distribution in which the frequencies are connected by straight lines
Confound
an unanticipated variable not accounted for in a study that could be causing observed changes
Negative Correlations
as A increases, B decreases OR as A decreases, B increases
Program evaluation
assesses the social reforms and innovations that occur in government, education, the criminal justice system, industry, health care, and mental health institutions.
Explanatory research
attempts to explain or find a cause for social trends and phenomena.
plagiarism
attempts to present another person's work as your own
closed cohort
begins and ends with set group of individuals
Deductive reasoning
begins with a theory that explains social behavior and uses observations to test the theory.
Inductive reasoning
begins with observations of social behavior and uses those observations to develop a theory that explains the behavior.
hospital-/clinic-based case-control
cases have disease, controls received care from same institution
Nominal scale of measurement
categorical data often reported as frequencies and percentage
Fence sitting
choosing middle response options (possibly to avoid reporting controversial opinions. - removing neutral or middle response options may help. But, what happens when people really are neutral or ambivalent?
outliers
data points that are highly improbable, although not impossible
invalid data
data points that fall outside the defined range for that variable of data
strengthening internal validity (2)
include control group, randomization
Positive linear relationship
increases in the values of one variable are accompanied by increases in the values of the second variable
Quasi-Experimental Design
independent variable cannot be truly manipulated. Comparison group, but no random assignment. Use of male vs. female, ethnicity, SES.
value in active control studies
non-maleficence
quasi experiment combining an independent-groups variable with a repeated-measures variable.
nonequivalent control group interrupted time-series design example: tv access and larceny rate over time
Ways to Handle Missing Data: Pairwise Deletion
participant dropped in analysis for only missing variable. participants answer all items on one scale, but not another, kept in analysis for only completed items. Used in smaller data sets where less power is observed.
efficacy
physiology, survival, and quality of life in a near-perfect setting
treatment group
the participants in an experiment who are exposed to the level of the independent variable that involves a medication, therapy, or intervention
Sensitivity
the percentage of cases with a diagnosis or condition who are correctly identified
specificity
the percentage of cases without the diagnosis or condition who are correctly identified ability to avoid false positives
Time horizon (in decision analytic modeling)
the period of time during which the outcomes are predicted by a decision analytic model
Median - NOT how to calculate it, just what it is
the point that divides the distribution into two parts, such that an equal number of values falls above and below that point
debriefing
the process of informing participant after the session of the experiment's true purpose to increase their understanding and to remove possible harmful effects.
data reduction
the process of transcribing data from individual data sheets to a summary form
Standard Deviation
the square root of the variance
change or influence/control (goal of psych/res)
what can we do to change or modify this behavior?
carryover effect
when exposure to one condition changes the response to another.
order effects
when exposure to one level of the independent variable influences responses to the next level of the independent variable
experiment
when researchers manipulate at least one variable, and measure others.
history threat
when something happens between the pretest and posttest.
tenacity
wisdom from previous generations, "American dream"
3 characteristics of good questions (understandable)
*Only ask one question at a time* - Avoid double-barreled questions *Use simple wording* - Avoid negatives - Avoid uncommon words/jargon - Avoid lengthy/complex statements *Use precise wording* - Avoid ambiguity
Likert Scale (lick-urt)
- A scale in which each statement is accompanied by five response options: strongly disagree, disagree, neither agree nor disagree, agree, and strongly agree - minor variations are often called Likest-Type scales - very common in psychology
Why is it important to know about research methods?
- To learn how to apply science to the study of human behavior - To understand and interpret others' research - To evaluate claims in everyday media - To discover how to acquire knowledge and make decisions by applying the scientific method
correlational study
- a research approach that focuses on how variables relate to one another - helps identify associations but not causations
Likert scale
- a scale where a participant evaluates a series of statements using a set of predetermined responses - responses are summed to represent the overall measurement for the variable - can measure level of agreement, frequency of occurrence, or overall evaluation of something
wording mistake: double-barreled
- asks participants to respond to more than one item using a single number - fix: break down into multiple questions so each one is only asking about a single item
one-group, pretest/posttest design
A researcher takes one group of participants, measures them before and after the treatment is given.
Formats for respondents
when asking a question or item with multiple possible responses, it is important that the responses are clearly distinguished from one another.
What are three steps to getting approved for human subjects?
- Application to REB - Research Proposal - Consent Forms
Three Types of Claims:
1. Frequency 2. Association 3. Causal
OR of 0.75
25% reduction of odds
pilot study
: before conducting the "real" experiment, a researcher may test the procedure on a small sample to confirm the effectiveness of manipulations, make sure everything runs smoothly, etc
According to the textbook, what is the problem with WEIRD samples? They are not very representative of the world's population. They are not good for theory-testing mode. They are not used frequently enough. They are not very easy to measure.
A
Dr. Gavin is conducting a 2 x 4 independent-groups factorial design. How many interactions will Dr. Gavin need to examine? 1 2 3 4
A
Marginal means are useful for seeing Simple main effects Two-way interactions Three-way interactions
A
Seeing stability in a stable-baseline design can help rule out which of the following threats to internal validity? Regression to the mean Placebo effects Attrition Observer bias
A
Closed-ended Questions
A type of survey research question in which respondents must choose from a fixed set of answers -Large-scale surveys use these because they are quicker and easier for both the respondents and researchers -Help to reduce respondent fatigue -Sensitive topics (e.g., liquor consumption) may be more accurately measured
Which format should research reports be written in?
APA
graph
After all trials have been carried out and data has been recorded in the Data Chart, making a _________ will further organize data and help you to see trends and patterns.
5 types of data
B-Data, P-Data, L-Data, I-Data, and S-Data
What questions do reliability and validity answer about a measure?
BOTH have to do with how good a measure is
Availability Heuristic
Being persuaded by what easily comes to mind
CONSORT
CONsolidated Standards Of Reporting Trials
Limitation
Data only of what a person or organization says, may differ from what he or she actually does
Hybrid
Field experiments - contrived situations in the real world
Observer bias Demand characteristics Placebo effects
However, even with a comparison group there can still be other issues:
participants;subjects
Humans in research are referred to as ___________, and animals are referred to as ___________.
objective measures; Counterbalancing
Instrument threats can be prevented by using ____________ at pretest and posttest. Or, by using a posttest only design. __________ also helps.
Interviewer bias
Intentional or unintentional infl uence exerted by an interviewer in such a way that the actual or interpreted behavior of respondents is consistent with the interviewer's expectations
when the effect of one IV depends on the level of another IV.
Interaction effect
Which method obtains answers from a hunch or feeling?
Intuition
Theory-testing mode
Investigating the support for a theory.
Passive Deception
Omission
True
Peer review is a collaborative process whereby researchers who are knowledgeable about a particular subject are asked to review and comment on another researcher's work and recommend whether it should be published. T/F
Declaration of Helsinki (1946)
Provided international ethics guidelines for medical research (written protocol, detailed description, reviewed by council)
Type 1 error
Rejecting null hypothesis when it is true (false positive)
Minimal risk research
Research in which participants are exposed to risks that are no greater than those encountered in daily life or in routine physical or psychological tests.
1) Literature Review 2) Hypothesis 3) Develop Methodology 4) Data Collection 5) Analyze Results 6) Interpret & Research Conclusion
Research process
Framing Effects
The influence, on the respondent, of how a polling question is asked; changes in wording can significantly alter many people's answers
Pretest
The measurement of the dependent variable of the experiment prior to the treatment
Theory-Data Cycle
Theory a set of propositions that attempts to specify the interrelationships among a set of concepts (aka how the world works). How and why things are related. Hypothesis A specific prediction derived from a theory and a specific research design
What does a 95% confidence interval of 100 to 110 mean?
There is a 95% probability that the interval from 100 to 110 contains the true population value.
C
Which of the following is NOT true of selection effects? Selection effects are a type of confound. Selection effects can occur when experimenters allow participants to choose their own treatment group. Selection effects are unimportant for interrogating internal validity. Selection effects can occur when researchers assign one type of person to one treatment group and another type of person to another treatment group.
B
Which of the following research designs is used to address possible selection effects? Posttest-only designs Matched-groups designs Pretest/posttest designs Correlational designs
data
measurements or statistics
Consent
required from participants
ANOVA (F)
tests the significance of group differences between two or more groups. The independent variable has two or more categories, the dependent is continuous. only determines if there is a difference between groups, not which is different. Ex. Low-, middle-, and high-income residents. See Notes for How to Read SPSS Output
Internal consistency reliability
the assessment of reliability using responses at only one point in time.
cell mean
the average score of the participants in a single cell or a single group the mean on the dependent variable for participants with a specific combination of the levels of the independent variables
Bias Blindspot
the belief that we are unlikely to fall prey to cognitive biases Being biased about being biased
Decision analytic model
the calculation of expected outcomes of an intervention or treatment based on a set of input assumptions, usually performed using specialized software and techniques, such as Markov chain modeling or decision tree analysis
maturation threat
when participants change on their own over time.
Statistical Relationship
Exists between two variables when the average score on one differs systematically across levels of the others
Two Forms of Deception
Passive Deception Active Deception
Journals
Peer-reviewed articles for scientists
Boundaries of Competence
What section of code is this? Psychologists must be competent in the fields in which they work.
derived statistic
where one value for data is expressed in terms of another value
interaction effect
whether the effect of the original independent variable depends on the level of another independent variable
explain (goal of psych/res)
why did this behavior occur? which factors influenced this outcome
If it would cause them harm to know
why would you never debrief the participant?
Multi-Center Studies
What is a research trial is a clinical trial that uses the *same protocol* at *multiple sites* conducted at more than one medical center or clinic?
Random sampling
What is a subset of a statistical population in which each member of the subset has an equal probability of being chosen that lends strength to *external validity*?
Sample
What is a subset of people selected out of the population for inclusion in the study?
Incidence
refers to the number (or rate) of new cases of a disease/condition in a population during a given period
3-way interactions
when the difference of differences are different - example: a crossing and a spreading interaction
______ ________ is a major challenge for small-N studies.
External validity
Experimental control
Extraneous variables are kept constant.
Benefit of Mixed design
Combines between-subjects and within-subjects designs. Strengths of each design offset the weaknesses of the other.
Applied Research
Conducted to find solutions for specific "real world" problems To improve the human condition
Process Evaluation (Formative)
Finding answers to questions that pertain to the implementation of a program and opportunities to make revisions; periodically to see how they are doing.
Internal Consistency - Split-Half Reliability
Splits the test in half computing a separate score for each half, and then calculating the degree of consistency between the two scores for a group of participants
What considerations should be applied when selecting references?
- Be conservative - Choose references directly relevant to the study you are presenting - Select only those references that are useful and contribute to your arguments
Correlations
"r". Tells us direction, positive/negative, ranges from -1 to +1. the closer to -1 or +1 the stronger the correlation. The closer to 0 the weaker the correlation.
masked design
(researchers were unaware of which condition participants had been assigned to)
survey
- a quantitative research strategy for systematically collecting information from a group of individuals - information is then generalized to a larger group of interest
Results
- A summary of the data and the statistical analyses - Begins immediately following the method section—not starting on a new page - Provides a complete and unbiased reporting of the findings, just the facts, with no discussion of the findings
The Rational Method
- An example of seeking answers by logical reasoning - In this method the first two sentences are premise statements while the last is a logical conclusion based on the premises
The Empirical Method (Empiricism)
- Answering questions by direct observation or personal experience - Based on the philosophy that all knowledge is acquired through senses
How should you prepare for a literature search?
- Narrow down your general idea to a specific research question o Find all published information to document and support that question - A good starting point: recently published secondary sources o A textbook, chapter headings, and subheadings will help focus your search on a more narrowly defined area
Name five potential sources for research ideas
- Personal interests and curiosities - Casual observation - Reports of others' observations - Practical problems or questions - Behavioral theories
Mail and Self-Administered Questionnaires
-Advantages: Cheap and can be conducted by a single researcher Anonymity Avoid interviewer bias. -Disadvantages: Low response rates. Cannot control the conditions under which a mail questionnaire is completed People can answer dishonestly -Mail questionnaires are ill-suited for the illiterate
Tri-Council Policy Statement
-Canadian Version of ethics incorporating three different agencies -Published in 1998 - It replaced all previous guidelines that had been developed by individual agencies and institutions. - In 2010 a second edition (TCPS 2) was published that consolidated the core principles
Questions when reading journalism...
-Is original research evidence cited? -Has the research been published in a peer-reviewed journal yet? -How does coverage of the same journal article differ across news stories?
key issues of within groups design
-Sequence Effects - scores on the DV are affected by the order in which levels of the IV are presented. --->Carry-over effects --->Practice effects --->Interference effects
Research questions are:
-Specific (is my dog happier after I walk him or after I give him a hug?) -Testable
Median is used when:
-The data are NOT normally distributed OR -There are any outliers (even 1!)
Moderate (correlation strength)
.30/-.30
Strong (correlation strength)
.50/-.50
Threats to Internal Validity and what each of them mean.
1. History- Any event that occurs between the first and second measurements but is not part of the manipulation. 2. Maturation- People change over time. 3. Testing- Simply taking the pretest changes the participant's behavior 4. Instrument Decay- The basic characteristics of the measuring instrument change over time 5. Regression Toward the Mean- When participants are selected because they score extremely high or extremely low.
Sources of Measurement Error
1. Transient States of the Participant 2. Stable Attributes of the Participant 3. Situational Factors 4. Characteristics of the Measurement 5. Experimenter Error
Three main principles (The Belmont Report)
1. principle of respect for persons 2. principle of beneficience 3 principle of justice
Maturation (spontaneous remission) threat
A change in behavior that changes more or less over time. This can be prevented by including a comparison group- conducting a true experiment
Strength of Correlation
A correlation above .70 is very strong A correlation above .40 is strong Lower than .20 is no correlation
Consistent Negative Relationship
A correlation near -1.00
Sampling Distribution of Sample Means
A distribution of sample means created by drawing many random samples from the same population ie. Age - a continuous variable • Random sample of 30 - calculate the average age of students • Sample 1: average age = 21.63, Sample 2: Average age=20.38 • Repeat procedure until 12 different samples of 30 students have been drawn
Codebook
A document describing the coding procedure and the location of data for variables in a format that computers can use
Response set
A pattern of response to questions on a self report measure that is not related to the content of the questions.
Confederate
A person posing as a participant in an experiment who is actually part of the experiment.
typology
A police officer develops a way to classify her colleagues in terms of their professionalism on the job. The categories are career officer, work-a-day officer, and slacker officer. This measure is best described as a:
Matched groups
A quasi-experimental design technique used to group participants into sets based on the similarity of a measured variable. Those coordinated sets are then randomly assigned to different conditions. This is used to control for selection effects.
Leading (or loaded) questions
A question that leads the respondent to choose one response over another by its wording
Cluster sampling
A probability sampling method in which existing groups or geographic areas, called clusters, are identified. Clusters are randomly sampled and then everyone in the selected clusters participates in the study.
Reactivity
A problem of measurement in which the measure changes the behavior being observed.
Carryover effect
A problem that may occur in repeated measures designs if the effects of one treatment are still present when the next treatment is given.
Quantitative Variable
A quantity, such as height, that is typically measured by assigning a number
Inductive Reasoning
A small set of specific observations is the basis for forming a general statement about a larger set of observations
Stable-basline design
A small-N study where a researcher observes the participant's behavior prior to the start of treatment, in order to get a baseline. If the behavior is consistent during the baseline, the researcher is able to be more certain that the treatment is effective. This offers protection against maturation and regression.
Sample
A smaller set of cases a researcher selects from a larger pool and generalizes to the population
Small-N design
A study in which researchers obtain a large amount of information from a few small cases. Each participant is treated as a separate experiment that is almost always a repeated-measures design.
Single-N design
A study in which researchers obtain a large amount of information from a single animal or person.
Factorial designs
A study in which there are two or more independent variables. This is the best way to study how combined variables interact. Increases the number of comparison groups
Technological
Applied interventions are described well enough that they can be implemented by anyone with training and resources
Limitations to the Rational Method
Assumes that the premise statements are true, sometimes the argument does not logically support the conclusion
Which of the following is an advantage of using quasi-experimental designs? They allow researchers to capitalize on random assignment. They allow researchers to enhance external validity. They allow researchers to disregard internal validity. They are better suited to detect significant effects.
B
Why do quasi-experiments tend to have very good construct validity for the independent variable? The manipulations have been previously validated in the lab. They use real-world manipulations/experiences. They tend to use more participants. They also have good construct validity for the dependent variable
B
Likert Scale
Degree of Agreement Strongly Disagree 1 - Strongly Agree 5
Experimental Validity
Degree to which the results of the study can be attributed to independent variable truly effected changes in dependent variable and results can be generalized to target population.
Behavioural Therapy is often used for what psychological disorder?
Depression
CBT is often used for what psychological disorders?
Depression, panic disorder, bulimia nervosa, and post traumatic stress disorder
Frequency Claim
Describe a particular rate or degree of a SINGLE variable
Experimental method
Direct manipulation and control of variables
Varied conclusion
During the Rosenthal study, it was observed that *narrative literature reviews* had what sort of conclusions?
Crossover
Effect of two interventions observed across a single group with assessment before, during and after interventions. May use multiple groups to adjust/compare order of intervention given.
Anatomy of a factorial design
How a factorial design is described conveys the number of independent variables in the designs and how many levels exist for each independent variable. The number of numbers indicates how many independent variables the design includes. The numbers themselves indicate how many levels or conditions each independent variable has.
- Remove noncompliant pts - Intention to tx stat adjustment - Tx attrition as a dependent variable - Dropouts significantly different
How do researchers deal with *attrition bias* where one group is more likely to *dropout* where the dropping out may even be a dependent variable?
Reject Null Hypothesis (p <0.05)
How do you know if you have *significant* results in your study?
Subject Words
Identify and describe the variables in the study and the characteristics of the participants
Limitations of Intuition
There is no method for separating accurate from inaccurate knowledge
How to identify factorial designs in reading
In empirical journal articles, the method section will usually state "this was a factorial design, with ___ and ___ as independent variables." and the results section will describe the statistical tests for main effects and interactions (these are ANOVAs, or F tests). In popular press articles look for key phrases such as "it depends" and look for participant variables
manipulation; measurement
In experiments, the ____________ occurs first, then a __________ is made.
Covert Observer
In field research, refers to a researcher who is secretly studying a group without the group members knowing they are being studied (neo-nazi group)
Significance (statistical)
Is achieved when there is a low probability that the results of an experiment occurred by chance alone. If the probability of their occurrence by chance is equal to or less than 5 per cent or 0.05
Limitations of Authority
Is not always accurate, often accepts expert's statements as facts, not all experts are experts
Convergent validity
Is the measure strongly related to other established measures of the same (or a similar) constructs?
Hour Glass Writing of Research
Lit review is broad and narrows to a specific research question. methods and results sections are more narrow and specific. The discussion starts narrow tied to the meaning of results then broadens to importance for the field.
quasi-experimental study with at least two groups, measured one time.
Nonequivalent control group posttest only Example: The Head Start Study
quasi experimental study with at least two groups; measured before and after an intervention.
Nonequivalent control group pretest/posttest design: Example: psychological effects of cosmetic surgery
MEDLINE Database
Provides the authoritative medical information on medicine, nursing, dentistry, veterinary medicine, the health care system, and pre-clinical sciences
a study where the levels of the independent variable are naturally occurring, rather than manipulated by the researcher.
Quasi-experiment
Order Effects (2 sources of question bias)
Question order Solution: Avoid open-ended after forced-choice questions Multiple response options Solution: Order in more than one way
Matrix questions
Refers to instances when a series of items share the same set of responses. While efficient, it can produce a response set, where respondents answer all of the items in the same way.
B
Regression is especially problematic in which of the following situations? When the experimental and comparison groups are equal at posttest When one group has an extremely high score at pretest When the experimental and comparison groups are equal at pretest When one group has an extremely low score at posttest
State the Question or Problem
Scientific Method Step #1
True
Research questions should be actual questions that can be answered rather than statements that make some claim.T/F
Factor Analysis
Researcher determines the effect of two or more independent variables (factors), both singularly and interacting with each other on a dependent variable. Purpose of data reduction & summarization. Two values: main effect & interaction effect
Observer bias
Researchers expectations influence how they interpret the results of a study
voluntarily, meaning that the research subjects must be completely free to participate or not participate without threat and informed, meaning that the researcher must inform the research subjects about the research and any risks associated with participation in the research.
Researchers must obtain consent from research subjects prior to collecting information from them. This consent must be:
Principle of Justice
Researchers must show that the people participating in the study are REPRESENTATIVE of the people that will be helped by any positive results
Bias Blind Spot
Seeing bias in others but not in ourselves
Haphazard (convenience) sampling
Selecting subjects in a haphazard manner, usually on the basis of availability, and not with regard to having a representative sample of the population; a type of nonprobability sampling.
Why is it important to ensure the methods section of a report is accurate?
So that others can replicate it
clear, step by step, and numbered
So that others can replicate your exact experiment, you must write a __________________________________________ procedure.
Negotiating
Social relations are negotiated and formed throughout the process of fieldwork. Occurs with each new member until a stable relationship develops to gain access, develop trust and reduce hostile reactions
Coding Data
Systematically reorganizing raw numerical data into a format that is easy to analyze using computers For example, with nominal data, researcher codes single people as 1, married as 2, common-law/cohabitating as 3, divorced as 4, separated as 5, and widowed people as 6
literature review
The ______________ involves locating and understanding what previous researchers have learned about a topic.
dependent variable
The ___________________________ is the variable that you will measure in your experiment and will change in response to the independent variable.
Sensitivity
The ability of a measure to detect differences between groups.
Population
The defined group of individuals from which a sample is drawn.
Control Group
The group that does not receive the treatment in an experiment
your prior knowledge and research
The hypothesis is based on ___________________________________________.
tested
The hypothesis must be expressed as a statement and must be capable of being _____________.
Population
The name for the large general group of many cases from which a researcher draws a sample and which is usually stated in theoretical terms. Researcher begins with this and then defines it more precisely. Abstract concept that requires an operational definition. ie. All persons aged 16 or older living in Regina on December 2, 2016, who were not incarcerated in prison, asylums, and similar institutions All business establishments employing more than 100 persons in Ontario that operated in the month of July 2015 All admissions to public or private hospitals in the province of Alberta between August 1, 2010, and August 1, 2015
Target Population
The name for the large general group of many cases from which a sample is drawn and which is specified in very concrete terms
Interview Schedule
The name of a survey research questionnaire when a telephone or face-to-face interview is used
Independence
The opposite of correlation. It means there is no association or relationship between variables. If two variables are independent, cases with certain values on one variable do not have any particular value on the other variable
Response rate
The percentage of people selected for a sample who actually completed a survey.
Subjects
The persons or animals whose behavior is systematically observed in a study.
Matching
The procedure whereby pairs of subjects are matched on the basis of their similarities on one or more variables, and one member of the pair is assigned to the experimental group and the other to the control group.
Sampling
The process of choosing members of a population to be included in a sample.
Provide two examples of a research paper
Thesis and journals such as the Journal of Applied Behaviour Analysis
3 Assumptions Needed to Test whether There is a Difference between Population Means
The two populations have the same variance: The assumption of homogeneity of variance. The populations are normally distributed. Each value is sampled independently from each other value.
Archival research
The use of existing sources of information for research. Sources include statistical records, survey archives, and written records.
Error Variance
The variability or change in a person's behavior that is NOT related to the other variable of interest, but instead remains unexplained
Independent variable
The variable that is manipulated to observe its effect on the dependent variable.
ordinal nominal
The variable education (high school diploma or equivalent, Bachelors degree, Masters degree or higher) is measured at what level of measurement?
Confidentiality
There are two types of privacy in research. (1) __________ means that the researcher knows who the research subjects are but does not disclose their identity to others. (2)Anonymity means that nobody, not even the researcher, knows the identity of the research subjects.
Wait-list design
To control for selection effects, all participants in a quasi-experimental study plan to receive treatment, but are assigned to receive the treatment at different times. This is reassuring to the researchers because they are confident that they have similar types of people in each group
One-group Pretest-Postest Design
To measure participants before the manipulation (a pretest) and again afterward (a posttest).
True
To research means to follow a logical process that uses concepts, principles, and techniques to produce knowledge. T/F
Nonprobability sampling
Type of sampling procedure in which one cannot specify the probability that any member of the population will be included in the sample.
Operational Definitions
Types of variables involved in research
Predictive Validity
Validated measure is administered weeks, months, or years before behavioral outcome measure is taken
Interval
Values are ordered and equally spaced Continuous data Likert scales Parametric measures
Ratio
Values are ordered and equally spaced with a meaningful 0.
Counterbalancing
Vary order of treatment to distribute or measure order effects randomized blocks
Humanistic view of research participant
View where the participant is a colleague who cooperates in providing data. Tends to be more sloppy.
Interquartile range
What is the amount of *dispersion* around the *mean* regarding how much *variability* there is *around the sample*?
- Random Assignment - Control group - Double-blind
What is the best case scenario for an experimental design?
Client/Patient, Student, and Subordinate Research Participants
What section of code is this? When research participation is a course requirement or an opportunity for extra credit, the prospective participant is given the choice of equitable alternative activities.
- Mean - Median - Mode
What are the components of *central tendency*?
- Nominal - Ordinal - Interval - Ratio
What are the different types *descriptive statics* of data format?
Design Confound
When a variable other than the independent variable happens to vary systematically with the independent variable *due to an experimenter's mistake*
Regression to the mean threat
When an extremely high or low-performance score recorded for the pretest that is likely to be not as extreme (closer to average) for the posttest. These extreme scores can give off the vibe that treatment motivated the effect of regression on the participant. This can be controlled for by using comparison groups as well to determine whether or not the treatment had the same, extreme effect as the comparison group who did not receive the treatment.
Selection-history threat
When an outside factor systematically affects one level of the independent variable of those participating in the study.
- Unethical randomly assign - Biologically impossible to randomly assign - Before investing for full experiment
When is it good to conduct a *correlation design*?
Instrumental threat
When the tool used to measure the dependent variable becomes unreliable and changes over time (application, usage, different interpretation). Prevention can be conducted by calibrating the tool used for measurement to ensure that the pretest and posttest measures are equivalent. This can be done by solidifying codebooks before the experiment is conducted to ensure the observers are using the same measurement. Or researchers can conduct a posttest-only experiment.
Selection-attrition threat
When only one group in the experiment experiences a decrease in the number of participants.
Demand characteristics
When participants determine what the study is about and change their behavior based on what they think the expected behavior should be.This can be prevented by using a masked design or double-blind design.
demand characteristics
When participants guess the hypothesis of a study, and change their behavior.
Covariation of cause and effect
When the cause is present, the effect occurs; when the cause is not present, the effect does not occur.
Simple Regression
When using single independent variable to predict a single dependent variable.
The IRB
________ ensures that psychological studies present as little risk to subjects and participants as possible and have scientific merit.
Pure
__________ research is conducted for the purpose of expanding the body of knowledge. University professors conduct research primarily to expand the body of knowledge about a particular subject or to develop theories about social behavior. Typically, this research is published in scholarly journals and read by other scholars interested in the same topics.
vignette
a description of a hypothetical situation, event, or scenario to which participants react
Negatively worded questions
a question in a survey or poll that contains negatively phrased statements, making its wording complicated or confusing and potentially weakening its construct validity - Avoid negatively worded questions when the negative does not serve a useful purpose. Try not to use double negatives. • Example: "Try not to use double negatives in your questions." • VS. "Do not fail to discard double negatives from your questions" • Goal: simplicity. Even one negative can increase complexity. Two negatives are worse. • Book example (Morling, 2017, p.158): Abortion should never be restricted.
Leading questions
a question that encourages or implies a particular answer - Ex: what did you think of that amazing concert? - avoid. use neutral phrasing to try to capture person's "true" opinion
graph
a representation of data by spatial relationships in a diagram
percentile
a score below which a certain percentage of the cases in a distribution fall; a percentile is a score, not a percentage
A-B-A design
a single-subject design in which researchers establish a baseline, introduce the intervention and measure the same variable again, then remove the intervention and take another measurement
A-B-A-B
a single-subject design in which researchers establish a baseline, introduce the intervention, remove the intervention, and then reintroduce the intervention, measuring the dependent variable each time.
study for rarer outcomes
case-control
Participant variables
characteristics such as age, gender, and intelligence that vary from one individual to another
One-Tailed (Directional) Hypothesis
concerned with the region of rejection only one tail of the sampling distribution. Directional difference: more than/less than, better/worse grade. ex. The mean of all M&M bags is less (or greater) than 17 M&M's.
inferential statistics
draw conclusions that can be applied to broader population
Within-Subjects Design
each participant experiences all levels of the IV (CONDITIONS)
historical control
group observed at different time or setting
helps to minimize detection bias
non-study personnel assesses outcomes
t-test
normal distribution interval level data, two independent groups ex: male vs female, health plan A vs health plan B result affected not only by size of the mean differences but also by the variability of each group, highly affected by outliers
Public
o Observations are available for evaluation by others
confound
potential threats to internal validity. Means to confuse
Interval vs. Ratio scale
ration = observable interval = unobservable qualities
Cronbach's alpha coefficient
splits the data all sorts of way then finds the average --> that is your correlation coefficient
order effects practice, fatigue, carryover
when the order in which participants perform conditions in within-groups experiment affects the results. examples:
Perspective (in decision analytic modeling)
viewpoint from which a decision analytic model is conducted
Internal Validity
"Are there confounding variables?"
External Validity
"Can we generalize the results to the population?"
Face Validity
"Does it *look* like they are measuring what they think they are measuring?"
ERIC Database
(Education Resource Information Center) provides access to education literature and resources
independent groups
(aka between-subjects): Different groups of participants take part in different conditions.
dependent groups
(aka within-subjects) Only one group of participants. Each participant takes part in every condition.
*Which APA Titles are not bolded?*
*Abstract, Title of paper, References, Appendices, Authors note, Footnotes*
Three Criteria for Causation
*Covariance* The measured variable must change as the manipulated variable changes (correlations; correlations satisfy 1 of the 3 criteria---covariance). *Temporal precedence* The manipulation must precede the change in behavior. A should always happen before B, never B before A! *Internal validity* Alternative causal explanations must be ruled out
Pro/Con of Close Ended
*Pro:* No coding required. May provide answers that they wouldn't have recalled. *Con:* May not include all options. Answers may be influenced by response options given (framing effect).
Pro/Con of Open Ended
*Pro:* Rich Info *Con:* Usually require coding, length/detail is not always clear, performance on recall worse than recognition for memory question
Experience has no comparison group; experience is confounded T/F
*TRUE*
*When in APA reports are numbers used?*
- *Quantities above 10* - *Date* - *Age* - *Measurements* - *Units of time*
*Title Page*
- *Running head: (abbreviated title) and page number (1)* - *Title of the paper* - *Author name(s) (byline) and affiliations* - *Author note*
*When in APA reports are printed numbers used?*
- *When the number is at the start of a sentence* - *Quantities below 10* - *Anything in a title*
*Methods*
- A relatively detailed description of exactly how the variables were defined and measured and how the research study was conducted - *Participants, Materials, Process*
What are the limitations of operational definitions?
- An operational definition is not the same as the construct itself - Concerns about the quality of operational definitions and measurements produced - It is easy for operational definitions to leave out important components of a construct - Operational definitions often include extra components that are not part of the construct being measured
What are the 7 dimensions of ABA? *Remember GET A CAB*
- Applied - Behavioural - Analytic - Technological - Conceptual Systems - Effective - Generality
9.02 Characteristics of Responsible Research
- Approval by REB, due concern for the dignity and welfare of the participants, minimize misleading results, trained, responsible for assistants, experience, maximize benefit and minimize risk, minimize financial, social, organizational, or political factors, avoid conflicts of interest when conducting research, minimize interference - This is a guideline that was not followed by Milgram
Which critical evaluation question should we ask ourselves when evaluating alternative explanations?
- Are alternative conclusions/explanations possible?
Which critical evaluation question should we ask ourselves when evaluating a justified conclusion?
- Are the conclusions justified by the results?
Which critical evaluation questions should we ask ourselves when evaluating a procedure?
- Are there alternative ways to define and measure the variables? - Could alternative procedures be used?
How can existing publication lead to ideas for new research studies?
- Authors generally make explicit statements regarding unanswered questions - Authors sometimes point out limitations or problems in their own studies - Create a new idea from two existing results - Frame a question based on differing results
9.05 Debriefing
- Behavior analysts inform the participant that debriefing will occur at the conclusion of the participant's involvement in the research
*Level 2 Heading*
*Flush left, boldface, upper and lower case*
*Level 3 Heading*
*Indented, boldface, Capital on first letter only, Period at end.*
Appendices
- Presents detailed information - Useful, but would interrupt the flow of information if presented in the body of the paper
systematic sampling
- probability sampling - every "nth" subject on sampling frame is selected - starting point is chosen via random number table/generator
Steps in the Research Process
1. Find a research idea: select a topic and search the literature to find an unanswered question 2. Form a hypothesis 3. Determine how you will define and measure your variables 4. Identify the participants or subjects for the study, decide how they will be selected, and plan for their ethical treatment 5. Select a research strategy 6. Select a research design 7. Conduct the study 8. Evaluate the data 9. Report the results 10. Refine or reformulate your research idea 11. Return to step 2 and continue
Smoking Study Article: What are the IVs? What is the experimental design? What is the overall results?
1. IVs are smokers vs. nonsmokers 2. experimental design is 2x2 mixed (2 levels & 2 IVs) 3. Overall results are smokers were more generous
Factorials displays
2X2 chart, graph
Case study
A descriptive account of the behavior, past history, and other relevant factors concerning a specific individual.
Survey research
A method of collecting information by asking questions.
Good Story Bias
Being swayed by a good story "It just makes sense"
observer effects
By using a masked design, the researchers in the "Seeing Red" study minimized the risk of: Measurement error Observer effects Reactivity Confounds
Temporal Precedence (3 criteria for causal claims)
Can you establish that x came before y?
FDA has to approve all protocols for _____
INDs
Informed Consent
Informing the research participants of the nature of the study and obtaining their explicit agreement. Ensures that researchers do not violate the privacy of participants and that participants have enough info to decide whether they want to participate
Informed Consent
Participants are informed in advance of all aspects of the research that might influence their decision to participate
Observer effects (expectancy effects)
Participants' behavior changes to better match the observer's expectations. - reactivity type
Type 1 error
Rejecting null hypothesis when it is true
Alternative Hypothesis
Research hypothesis contradicting the null. ex. Each bag does not have 17 M&M's.
Random Error
Statistical fluctuations that are caused by unknown & unpredictable changes in the experiment
Mechanical Method
The assignment is automatic - not based on any personal preference or specific cases
Literature
The mass of published information worldwide
Number needed to treat
Used in assessing the practical interpretation of the differences between two or more groups Multiplicative inverse of the estimated benefit of the treatment. For example, if the treatment reduces an outcome by two percentage points, the NNT is 1/0.02=50.
Phi Coefficient
Used when both sets of data are nominal, dichotomous measures. Ex. Sex (male/female) and Passing the NCE (pass/fail)
Faulty Thinking
We believe a theory that makes intuitive "sense"
- Opportunistic Data Torturing - Procrustean Data Torturing - Publication Bias
What are the types of *bias in analysis* and publication?
Power
What is the likelihood one can detect a *true effect*?
C
What is the primary difference between pretest/posttest designs and within-groups designs? The number of participants used The number of times the dependent variable is measured The number of levels of the independent variable participants are exposed to There is no difference between the two designs.
Control Group
What is the pure, *no tx* or placebo group?
Research
What is the systematic accumulation of knowledge?
Compare Apples-to-Oranges
What is the top criticism of *meta analysis*?
measures of central tendency
a measure of the average score in a distribution, such as the mean, median, mode
Multiple Coders
allows you to assess interrupter reliability
Feasibility
are the results and recommendations realistic and achievable.
standard deviation
average distance from the mean
mean
average, variance, st. deviation
active control
current standard of care
safety monitoring
documenting and reviewing of adverse events
practice effects
improvement on a task over time due to practice.
error bars
in a graph, vertical lines that indicate plus or minus one standard deviation of the data or, less frequently, the standard error of the mean
Practical Problems or Questions
o Stemming from daily life (e.g., placement of audio controls for less distracted driving) o Example: Legalization of marijuana and the effects on impaired driving
rate ratios are used in _____
open cohorts
How do you evaluate reliability?
scatterplots and correlations
S-data
self-report
types of blinding (4)
single, double, triple, open label
science
synthesis of rationalism and empiricism / helps to correct own bias
standard deviation
the square root of the variance; a measure of variability in the same units as the scores being described
Correlation
the strength of the relationship between two variables
Research Question
to be mixed method design, both qualitative and quantitative research questions must be included.
Saying more than we can know
• Nisbett & Wilson (1977): People will report opinions and the reasons on which those opinions are based, even though they're effectively making it up as they go along. • For identical nylon stockings: people generally preferred the right- most one, and claimed their preference was based on quality. - Empirically, there may be many forces influencing our behavior. However, we may often be unaware of them. - When asked, we attribute reasons to our actions. Those inferred reasons might be biased or simply wrong.
Survey Questions
- Behaviour. How frequently do you go to art galleries? - Attitudes/beliefs/opinions. What kind of job do you think the mayor is doing? Do you think other people say many negative things about you when you are not there? - Characteristics. Are you married, cohabiting, single, divorced, separated, or widowed? - Expectations. Do you plan to enrol in a degree program in the next 12 months? How much schooling do you think your child will get? - Self-classification. Do you consider yourself to be liberal, moderate, or conservative? Into which social class would you put your family? - Knowledge. Who was elected prime minister in the last election?
*Abstract*
- Centred and not bolded - No indent - *Statement of problem* - *Describe participants* - *Describe method* - *Report results* - Conclusions or Implications
Considerations when Designing Mixed Methods
- Determine the order of implementations of the qualitative and quantitative aspects of the study. Concurrently or in a sequence, illustrates intent of the study: broad to narrow, narrow to broad. - Determine the weight given to qualitative and quantitative data (research question order). Extent to which both will be incorporated into the study, primary focus of the study. - Determine how the two types of data will be mixed or combined. Interpretation might involve transforming qualitative themes into quantitative numbers. Interpretation might involve comparing numbers to lived experiences. - Determine theory (both qual and quan) to guide study. Forms the questions, analysis and interpretation of the data.
9.04 Using Confidential Information for Didactic or Instructive Purposes
- Do not disclose personally identifiable information concerning their individual or organizational research participants unless written consent - Disguise confidential information concerning participants so that they are not individually identifiable to others
Which critical evaluation questions should we ask ourselves when evaluating limits to generalization?
- Do the results have real-world applications? - Is there reason to suspect that the same results would not occur outside the lab? - Would the same results be expected with different participants or under different circumstances?
Which critical evaluation question should we ask ourselves when evaluating results related to hypothesis?
- Do the results really support (or refute) the hypothesis?
Which critical evaluation questions should we ask ourselves when evaluating a specific prediction from hypothesis?
- Does the predicted outcome logically follow from the hypothesis? - Can other specific predictions be made?
Safeguards to Prevent Fraud
- Experimental research studies that are designed to be replicated - Research articles that are peer reviewed by experts in the field - The consequences of being found guilty of fraud - a big deterrent
Oral Presentations/Symposiums
- Five minute talk about research, allow a few minutes for questions from audience - In preparing a talk, presenters should memorize key elements such as why you conducted the study, what your research question was, how you tested it and what you discovered. - The presenter should look at the audience members and speak to them in a conversational tone that is less formal than APA-style writing but more formal than a conversation with a friend.
Behaviour Analysts Ethical Guidelines
- Guideline 9 (The Behaviour Analyst and Research; 2017) - Complex set of requirements with the BACB Guidelines - Behaviour analysts design, conduct, and report research in accordance with recognized standards of scientific competence and ethical research.
Which critical evaluation questions should we ask ourselves when evaluating participants?
- If participants were restricted (e.g., males only), is it justified? - Would different participants produce different results?
9.03 Informed Consent
- Inform participants or their guardian or surrogate in understandable language about the nature of the research - Not coerced - Process to decline to participate, or to withdraw without penalty; - Available to answer any other questions participants may have about the research - If you want to do research informed consent is a MUST! - Make the potential participants aware that even if they do not choose to participate, treatment is still available
examples of repeated measures quasi-experiment tests?
- Interrupted time-series design - Nonequivalent groups interrupted time-series design
When giving an oral presentation at a conference, what procedure should be followed?
- Intro and Opening remarks: - Develop your problem - Outline research - Develop your point or solution (use lots of visuals) - Summary - Thank Audience
Method of Faith
- Is a variation of method of authority - People have unquestioning faith in authority figure - Accept without questioning, challenge, or doubt
The Scientific Method
- Is an approach to acquiring knowledge - Involves formulating specific questions and then systematically finding answers - Combines several different methods of acquiring knowledge
Which critical evaluation questions should we ask ourselves when evaluating a hypothesis/purpose for study?
- Is the hypothesis clearly stated? - Is the hypothesis directly related to the reviewed literature?
Which critical evaluation question should we ask ourselves when evaluating lists of items cited?
- Is the references list current and complete?
Which critical evaluation questions should we ask ourselves when evaluating a literature review?
- Is the review up to date and complete? - Are relevant or related topics not covered?
Environmental Changes
- It is difficult to attain the ideal of identical circumstances - Example: Raining and students have an indoor recess
What are the three important principles to the scientific method?
- It is empirical - It is public - It is objective
References
- Listing of complete references for all sources of information cited in the report o Organized alphabetically by the last name of the first author o Begins on a new page o Title is not bolded - Includes DOI is available
Name three important reasons to attended conferences
- Networking - Keeping up with regional laws - Keeping up with latest literature
9.09 Accuracy and Use of Data
- Not fabricate data or falsify results - If errors are discovered in their published data, they take steps to correct such errors (retraction, erratum) - Do not omit findings that might alter interpretations - Do not withhold the data on published work
Presenting at Conferences
- Organizers will outline written guidelines to apply to speak at a conference - Incomplete or incomprehensible submissions will be rejected - Must apply before a deadline - It will be evaluated by a panel and you will receive notice of rejection or acceptance
What are five things to keep in mind when finding inspiration for a research topic?
- Pick a topic in which you are interested - Do your homework: collect background information - Keep an open mind - Focus, focus, focus - Take it one step at a time
Poster Presentations
- Posters are typically presented during a one- to two-hour poster session that takes place in a large room at the conference site. - Presenters set up their posters around the room and stand near them. - Other researchers then circulate through the room, read the posters, and talk to the presenter - Application guidelines - Format guidelines - Organization should inform if they have poster board, bring pins - Eye catching - Easy to read - Minimal content - Have copies of poster, have business cards
What is the goal of a research report?
- Provide three kinds of information about the research study o What was done o What was found o How the study is related to other knowledge in the area
Fundamental Characteristics that Allow for Cause & Effect to be Claimed
- Random Assignment: cornerstone of design as it allows equal chance to participate - Manipulation of intervention (IV) - Measurement of outcomes (DV) - Control of extraneous variables: factors that influence the outcome of the study by not being associated with the intervention - Making comparisons between groups: treatment vs. control group
Methods of Authority
- Relies on information or answers from an expert in the field - Is a good starting point and often quickest and easiest to obtain answers
Unavoidable Ethical Conflict
- Researchers must think through the ethical issues raised by their research, mitigate risks o Weigh the risks against the benefits, o Be able to explain their ethical decisions, o Seek feedback about these decisions o Take responsibility for them
What is research ethics?
- Responsibility of researchers to be honest and respectful to all individuals who are affected by their research studies or their reports of the studies' results - Guided by the American (and Canadian!) Psychological Association's, BCBA ethical codes and Tri-Council Policy Statement
Discussion
- Restates the hypothesis, summarizes the results, and then presents a discussion of the interpretation, implications, and possible applications of the results - Begins immediately after the results section—not starting on a new page
Concurrent Validity
- Scores obtained from a new measure - Directly related to scores obtained from a more established measure of the same variable
What are the purposes of a literature review?
- Show relationships between previous studies or theories - Find out what information already exists in your field of research - Identify gaps in literature - Find other people working in your field - Identify major seminal works - Identify main methodologies and research techniques - Identify main idea, conclusions, and theories and establish similarities and differences - Provide a context for your own research
Divergent Validity
- Showing little or no relationship between the measurements of two different constructs - *Example: Measuring aggression and nothing shows
Face Validity
- Simplest and least scientific - Whether a measure superficially appears to measure what it claims to measure
Convergent Validity
- Strong relationship between the scores obtained from two (or more) different methods of measuring the same construct - Example: Many measures used and they all show the same type of data
Informed Consent
- Subjects must be informed of all available information about the study so they can make a rational decision to participate (or not) - Consent should state the they are free to not participate and there will be no ramifications - Consent should state that participants can remove consent at any time
Mean is used when:
- The data are normally distributed AND - There are no outliers
Participant Changes
- The participant can change between measurements (e.g., degree of focus) - Example: The student did not eat breakfast before coming to school
Assent
- There is no age of consent in Ontario. (understand the information & ramifications) - Assent (written or verbal) should be secured - The form should include simple declarative statements that describe concretely the main features of the study procedures and the voluntary nature of research participation. - Needs to be written or stated at the child's level
During a literature search, in which ways can irrelevant materials be weeded out?
- Use the title as your first basis for screening - Use the abstract as your second screening device - Look for a link to a full-text version - If the article still looks relevant, then read it carefully - Use references from articles you have already found to expand your literature search
Which critical evaluation questions should we ask ourselves when evaluating statistics (descriptive and inferential)?
- Were the appropriate statistics used? - Exactly what is significant and what is not? - Are the effects large enough to be meaningful?
Give 4 examples of bases for empirical research questions
- What are some possible causes of the behaviour or characteristic? - What are some possible effects of the behaviour or characteristic? - What types of people might exhibit more or less of the behaviour or characteristic? - What types of situations might elicit more or less of the behaviour or characteristic?
Which critical evaluation question should we ask ourselves when evaluating applications?
- Would other variables affect the results?
alternative form reliability
- a form of reliability that evaluates how well a measure correlates with a similar, but different, measure of the same variable - correlation between different version of the scale - statistics: correlation between version 1 and version 2
non-directional hypothesis
- a hypothesis that does not make a specific prediction as to how two variables are related - ex: someone will win the game
error of central tendency
- a response bias where a participant tends to avoid using the extreme response alternatives on a scale - solution: use more alternatives (5-9 is optimal)
reverse coding
- a scoring strategy where more negative response alternatives are assigned higher numerical values and more positive response alternatives are assigned lower numerical values - used to minimize potential for an acquiescent response set - change the direction of your scale (if 1 is strongly agree and 5 is strongly disagree, 1 would be strongly disagree and 5 would be strongly agree)
repeated measures ANOVA
- a statistic used to test a hypothesis from a within-subjects design with three or more conditions
Chi-square
- a statistical test in which both variables are categorical - generally tells us if the distribution of participants across categories is different from what would happen if there were no difference between the groups
one way ANOVA
- a statistical test that determines whether responses from the different conditions are essentially the same or whether the responses from at least one of the conditions differ from the others - compares the means across two or more groups - tells you there's a difference somewhere, but does not specify which groups are different
history
- a threat to internal validity due to an external event potentially influencing participants' behavior during the study - ex: fire drill causes evacuation during the study
instrumentation
- a threat to internal validity due to changes in how a variable is measured during the course of the study - ex: raters may become better or more refined in their observations leading to changes in ratings, differences in how tests are administered (such as paper vs online)
order effect
- a threat to internal validity in a within-subjects design resulting from influence that the sequence of experimental conditions can have on the dependent variable
Clear and detailed codebooks
- allows observers to make accurate records - allows you to mediate disagreeing observation - allow others to assess the quality of operationalizations
two-group design
- an experimental design that compares two groups or conditions and is the most basic way to establish cause and effect - also known as simple experiment
random assignment
- any method of placing participants in groups that is unsystematic and unbiased - ensures each participant has an equal chance of being in any group
practice effect
- changes in a participant's responses or behavior due to increased experience with the measurement instrument, not the variable under investigation
non-probability sampling
- chosen on some basis other than random selection - snowball - connivence - purposive
sensitization effect
- continued exposure to experimental conditions in a within-subjects study increasing the likelihood of hypothesis-guessing, potentially influencing participants' responses in later experimental conditions
fatigue effect
- deterioration in measurements due to participants becoming tired, less attentive, or careless during the course of the study
consequences of intrinsic variation for biological systems
- evolution by natural selection and adaptation - medical treatment/management - ecological management - biodiversity and diversity in ecological function - production in farming and other industries
carryover effect
- exposure to earlier experimental conditions influencing responses to subsequent conditions
extraneous variables
- factor other than the intended treatment that might change the outcome variable - eliminate or control these to ensure the IV is what is actually causing changes in the DV
socially desirable responding
- faking good and alternatively faking bad - giving survey responses that make one appear better (or worse) than is actually the case - Guarantee anonymity and ensure that people know that their responses are anonymous. - past deception in research many problems. If participants don't trust the researcher, then they many not believe that their responses really are anonymous. • Include items to identify socially desirable responding: - My table manners at home are as good as when I eat at a restaurant. - I don't find it particularly difficult to get along with loud- mouthed obnoxious people. - I never hesitate to go out of my way to help someone in trouble. • Perhaps observations or ratings from friends will be more accurate. • Use implicit measures.
How big should a sample be? It depends on 3 things.
- how accurate and precise do you need to be in estimation - in a H test, how small or subtle a relationship you are trying to determine - amount of inherent variation
how to get a precies and accurate representative sample
- identify populations of interest to be sampled - use of unbiased sample selection procedure (random sampling) - sample size
Why does increasing sample size matter?
- improves sample accuracy - improves sample precision
sources or causes of measured variation in biological systems
- intrinsic sources of variation - extrinsic sources of variation
Every multi-item scale has an order. What should we do?
- produce multiple versions, then compare results. If there is no difference between versions, then we can have greater confidence in the results. If there is a difference between versions, then we should report both outcome - try to identify a set of questions. (Alternatively, find questions that are not framed inappropriately by earlier questions) - if there is a one "most-important" question on your survey, you may wish to ask that question first
probability sampling methods
- randomization occurs at some point - simple random sampling - systematic sampling - stratified sampling - cluster sampling
content validity
- the degree to which the items on a scale reflect the range of material that should be included in a measurement of the target variable - ask yourself: does it assess the topic in a "wholesome" or comprehensive way?
criterion validity
- the extent to which a measurement relates to a particular outcome or behavior - established by evaluating the concurrent and predictive validity of the measurement - ask yourself: does it relate to actual behavior?
construct validity
- the extent to which the scale actually measures the desired construct - established by evaluating the convergent and discriminant validity of the measurement - ask yourself: does it actually measure our construct? - if the scale converges with measurements of similar variables and diverges from measurements of dissimilar variables, it has high construct validity
test-retest reliability
- the temporal stability of a measure - the ability to produce similar measurements across different time periods - statistics: correlation between Time 1 and Time 2 scores
methodological pluralism
- the use of multiple methods or strategies to answer a research question - sometimes it's better to use more than one
wording mistake: jargon
- using jargon that is specific to something that participants may be unfamiliar with (ex: rushing a fraternity) fix: avoid using words or phrases participants may not be familiar with
wording mistake: vocabulary
- vocabulary is complicated or complex - fix: keep it simple
acquiescent response bias
- when a participant tends to agree with most, if not all, of the items on a scale, regardless of what they are asking - participant anticipates questions and answers without reading - solution: reverse code some questions
temporal precedence
- when changes in the suspected cause (treatment) occur before changes in the effect (outcome) - accomplished by manipulating the IV before measuring the DV
Question Errors
-Accepting a middle choice response when respondents hold a non-neutral opinion -Forcing respondents to choose a position when respondents hold non-neutral opinion -Issue of non-attitudes can be approached by distinguishing among three kinds of attitude questions: standard-format, quasi-filter, and full-filter questions
Telephone
-Advantages: 95% of the population can be reached. Interviewers control the sequence of questions and can use some probes. All questions answered. -Disadvantages: More expensive than mail or online. The use of an interviewer reduces anonymity and introduces potential interviewer bias. No visual aids, difficult to use open-ended questions
Face-to-face
-Advantages: Highest response rates and permit the longest questionnaires. The interviewer can observe surroundings and can use nonverbal communication and visual aids. Can ask a range of questions -Disadvantages: The training, travel, supervision, and personnel costs can be high. Interviewer bias is high.
Advantages of experimental design
-Can account for Individual Differences -Eliminate other Confounds -Observe the invisible -Minimize noise
key issues of Between Groups Design
-Creating equivalent groups -> same in every way except the IV -Random Assignment -Matching
Control of extraneous variables
-Have to be known to all participants -everyone gets same experience
Skewed Distribution
-If most cases have lower scores with a few extreme high scores, the mean will be the highest, the median in the middle, and the mode the lowest. -If most cases have higher scores with a few extreme low scores, the mean will be the lowest, the median in the middle, and the mode the highest.
Questionnaire design issues
-Length of Survey or Questionnaire: Length depends on the survey format and on the respondent's characteristics -Question Order and Sequence: organization of the overall questionnaire, question order effects, and context effects. -Organization: A questionnaire has opening, middle, and ending questions. After an introduction explaining the survey, it is best to make opening questions pleasant, interesting, and easy to answer to help a respondent feel comfortable about the questionnaire. -Order Effects: An effect in survey research in which respondents hear some specific questions before others and the earlier questions affect their answers to later questions.
Qualitative questions
-Less structured, nondirective, in-depth interviews - differ from survey research interviews -Insights, feelings, and cooperation are essential parts of a discussion process that reveals subjective meanings
factorial design
-More than one IV -Main Effects and Interactions
Tentative Answers
-Not absolute answers (not proven) -Consider "weight of the evidence": Draw conclusions from entire scientific literature Proportion of studies consistent with theory
Problems with Headlines (Journalism)
-Often oversimplified overconfident -Sometimes misrepresent finding
one-way design
-One independent variable --->Two-group design --->Multiple groups design
Empiricism - Systematic
-Specified exact action you're looking to record, and how -Systematic observation typically involves specification of the EXACT actions, attributes, or other variables that are to be recorded and precisely how they are to be recorded.
problem with counterbalancing
-Time interval between trials --->Mortality, loss of participants, or changes in participant
Qualitative Interviewing
-Unstructured, semi-structured, in depth, ethnographic, open ended, informal, and long -Informal and non directive -A mutual sharing of experiences -Explicit purpose—to learn about the informant and setting (differs from friendly conversation) -Rarely have hypothesis being tested Advantages: 1. Allows the "voice" of the interviewee to be heard 2. Data rich with descriptions and explanations 3. Development of new theories 4. Development of new avenues of research Disadvantages 1. Sample sized are usually small in comparison to those used in quantitative work - question whether the results are actually applicable to a wider population 2. Very rare to archive qualitative data. More and more researchers are sharing so that the data can be scrutinized by others
Aiding Respondent Recall
-When survey involve the recall of past events researchers should provide respondents with special instructions and extra thinking time -Past events: should also provide aids such as a fixed time frame or location references
mixed design
-When there is more than one independent variable --->One or more variables can be within --->One or more variables can be between E.g.: Men and women each take a personality test for the big 5. Gender is the between subjects variable and the five different personality scales are within.
within groups advantages
-fewer participants needed -Important for special groups -statistically more powerful (less variability)
between groups disadvantages
-may not be random -unequal N -potential confounds -requires more participants
two ways to create groups
-naturally occuring -experimental manipulation
within groups disadvantages
-not naïve after first trials -order effects --->practice and fatigue --->non-symmetric or differential transfer
Two types of design
-post-test only -Pretest-posttest
Naturally occurring groups
-quasi-experimental Ex: men and women specific career groups
naturalistic observation
-subjects watched in a natural setting -disturbed as little as possible ex: Goodall watching chimps in Africa for 50 years
p value cutoff
.05
Experiential Avoidance Article: What is the sample? What is the variables? What is the overall results? What is experiential avoidance?
1. 202 undergraduate/ 19.5 years of age/ mainly white females 2. 3. These findings suggest that experiential avoidance is significantly associated with lower delay of gratification both directly and indirectly, through heightened depression and anger 4. the tendency to negatively evaluate and avoid emotion and cognition, tend to report higher levels of distress.
Things to avoid when writing survey questions
1. Avoid jargon, slang, and abbreviations (e.g., NATO) 2. Avoid ambiguity (What is your income? - does this mean annually, monthly? Family or personally) 3. Avoid double-barrelled questions 4. Avoid leading (or loaded) questions 5. Avoid double negatives
What types of validity do you need for frequency and association claims?
1. Construct 2. Statistical 3. External
Three types of construct validity
1. Convergent 2. Discriminant 3. Criterion-related
Two Forms of Statistical Relationships
1. Differences between groups 2. Correlations between quantitative variables
Common Characteristics of Pseudoscience
1. High "success" rates are claimed. 2. Rapid effects are promised. 3. The therapy is said to be effective for many symptoms or disorders. 4. The "theory" behind the therapy contradicts objective knowledge (and sometimes, common sense). 5. The therapy is said to be easy to administer, requiring little training or expertise. 6. Other, proven treatments are said to be unnecessary, inferior, or harmful. 7. Promoters of the therapy are working outside their area of expertise. 8. Promoters benefit financially or otherwise from adoption of the therapy. 9. Testimonials, anecdotes, or personal accounts are offered in support of claims about the therapy's effectiveness, but little or no objective evidence is provided. 10. Catchy, emotionally appealing slogans are used in marketing the therapy. 11. Belief and faith are said to be necessary for the therapy to "work." 12. Skepticism and critical evaluation are said to make the therapy's effects evaporate. 13. Promoters resist objective evaluation and scrutiny of the therapy by others. 14. Negative findings from scientific studies are ignored or dismissed.
Kvale
1. Introducing questions 2. Follow-up questions 3. Probing questions 4. Specifying questions 5. Direct questions 6. Indirect questions 7. Structuring questions 8. Interpreting questions 9. Silence
Steps to conduct a survey
1. Keep it clear 2. Keep it simple 3. Keep the respondent's perspective in mind A researcher would not know if differences in respondents accounted for different answers. A good questionnaire forms an integrated whole (e.g., instructions for clarification and measures each variable with one or more survey questions)
Seven Concerns of the IRB:
1. Lack of Debriefing 2. Breach of Confidentiality 3. Lack of Informed Consent 4. Invasion of Privacy 5. Coercion 6. Deception 7. Harm to the Participants
Justifications for Deception
1. MUST INCLUDE A DEBRIEFING SESSION 2. Study cannot be completed otherwise 3. Cannot deceive participants about the study that might sway their willingness to participate 4. Cannot deceive about risks or possible discomfort
Principle of Respect for Persons
1. Must treat everyone as autonomous beings (can make their own decisions --informed consent --cannot mislead about risks --cannot coerce 2. Those who are not autonomous must have special protections --children, disabled, prisoners
Steps of the Scientific Method
1. Observe behaviour or other phenomena 2. Form a tentative hypothesis to generate a testable prediction 3. Use your hypothesis to generate a testable prediction 4. Evaluate the prediction by making systematic and planned observations 5. Use the observations to support, refute, or refine the original hypothesis
Principle of Beneficience
1. Protect the participants from harm 2. Cannot withhold beneficial treatment --control groups must eventually receive treatment --must implement new, innovative treatments even if your study has already begun --must inform about risks of waiting if in placebo
What are the steps to submitting a manuscript for publication?
1. Select a journal that is appropriate for the topic of your research report 2. Consult the journal's Instructions to Authors for specific submission requirements 3. Enclose a cover letter to the journal editor along with the manuscript
What are two ways you can test for inter-item reliability?
1. Split-half reliability 2. Item total correlation
What are the two phases for participating in a conference presentation?
1. Submit a written summary or abstract to the conference organizers for approval 2. Give an oral presentation: typically involves a PowerPoint presentation (10 to 20 minutes) or Present a poster: guidelines are provided by the organization for style and requirements
3 Conditions for Causality
1. Temporal Order 2. Association 3. No Alternative Explanation
Sources of Ethical Directives:
1. The Belmont Report 2. APA Ethical Principles 3. APA Ethical Standards
7 Parts of True Experiment
1. Treatment of independent variable 2. Dependent variable 3. Pretest 4. Post-test 5. Experimental group 6. Control group 7. Random assignment Note: not all experiments have all 7 parts
An informed consent form must include:
1. Why is the researcher doing this study? 2. What will the participant be doing? 3. Are there any risks involved? 4. Consequences of refusal/withdrawal 5. Confidentiality assurance 6. Does the participant have any questions? 7. How does the participant contact the researcher afterwards? 8. Signature lines
Researchers have two obligations:
1. protect the rights of the participants 2. to further scientific research
Marital Ages Article: What is operational sex ratio? What is female bias OSR? What is male bias OSR? What is OSR of 100? What is the results? How was the data obtained?
1. the number of sexually active males per 100 sexually receptive females 2. Where the OSR is female biased, males tend to pursue short-term mating as long as possible 3. When the OSR is male biased, available men outnumber available women and the greater degree of female choice will increase the male social status and resource potential necessary for securing female partners. 4. if the OSR is of 100, the population is balanced between the sexes 5. As predicted, when women are scarce they marry earlier on average and the variance in male marital age increased, however there was no significant relationship with male marital ages , though the trend was for earlier marriages. 6. 50 largest Metropolitan Statistical Areas (MSAs) in the United States with year 2000 Census data
a trial finds a 6.7% cure rate for drug A and a 4.7% cure rate for drug b. what is the number needed to treat with drug a instead of drug b to produce one additional cure?
1/(0.067-0.047)= 50
How long must data be kept from a study?
10 years OR 10 years following the 18th birthday of the participant
What is the minimal number of replications needed?
3
The mathematical way to describe an interaction is: A difference in differences A caveat A qualified main effect A patterned pattern
A
The text states that the first step in establishing a study's importance is to establish its _________. replicability external validity internal validity acceptance by scientists
A
Constant
A "thing" that does not vary. It only has one level in the current context but it could potentially vary.
Constant
A "thing" that does not vary. It only has one level. Usually something that you are NOT interested in either measuring or manipulating
Variable
A "thing" that varies. It must have at least two levels.
Variables
A "thing" that varies. It must have at least two levels. Usually what you are interested in measuring or manipulating
3. _____ refers to the average of the dependent variable for participants with a specific combination of the levels of the independent variables A) Cell mean B) Marginal mean C) Main effect D) Crossover interaction
A - cell mean
1. is a prediction that focuses on one independent variable A) A main effect hypothesis B) An interaction effect hypothesis C) A suppression effect D) A crossover interaction
A - main effect hypothesis
pretest-posttest
A ____________ design can show how people change over time as a result of some manipulation.
Social Desirability Bias
A bias in survey research in which respondents give a normative response or a socially acceptable answer rather than giving a truthful answer
Cultural psychology
A branch of psychology that mainly focuses on the way cultural contexts shape the way a person feels, thinks, acts, and behaves.
positive correlation
A correlation where as one variable increases, the other also increases, or as one decreases so does the other. Both variables move in the same direction.
naturalistic observation
A descriptive research method in which the researcher engages in careful observation of behavior.
case study
A descriptive technique in which one individual or group is studied in depth in the hope of revealing universal principles.
Mixed factorial design
A design that includes both independent groups (between-subjects) and repeated measures (within-subjects) variables.
Scatterplot
A diagram to display the statistical relationship between two variables based on plotting each other's values for both of the variables. -Used for variables measured at the interval or ratio level, rarely for ordinal variables, and never if either variable is nominal. Usually, the independent variable (symbolized by the letter X) goes on the horizontal axis and the dependent variable (symbolized by Y) goes on the vertical axis. -Determines: form, direction, and precision
Interaction effect
A difference of differences. A result in a factorial design in which the difference in one level of the independent variable changes depending on the other level of the independent variable. In a factorial design with two independent variables, the first two results obtained are the main effects for each independent variable. The third result is the interaction effect. Whereas the main effects are simple differences, the interaction effect is the difference in differences.
Pseudoscience
A fake or false science that makes claims based on little or no scientific evidence.
hypothesis
A formal statement that predicts how a change in one or more variables might cause a change in another variable is called a(n):
Deductive Reasoning
A general statement is the basis for reaching a conclusion about specific examples
Descriptive Statistics
A general type of simple statistics used by researchers to describe basic patterns in the (numerical) data. They can be categorized by the number of variables involved: univariate, or multivariate
Identify the object or event that is to be observed, and identify any factors (variables) that might affect the observation
A hypothesis is expressed as a statement and must be capable of being tested. Before writing your hypothesis you must do what 2 things?
Central Limit Theorem
A law-like mathematical relationship stating that whenever many random samples are drawn from a population and plotted, a normal distribution is formed, and the centre of such a distribution for a variable is equal to its population parameter
Sampling Frame
A list of cases in a population, or the best approximation of it. Crucial to good sampling, size less crucial than whether it accurately represents population
Interview Guide
A list of questions a researcher wishes to address in the course of a qualitative interview
Manipulation check
A measure used to determine whether the manipulation of the independent variable has had its intended effect on a subject.
Concurrent validity
A measure of how well a particular test correlates with a previously validated measure.
Correlation
A measure of the extent to which two factors vary together, and thus of how well either factor predicts the other.
Semantic differential scale
A measure of the meaning of concepts. I.e. Respondents are asked to rate any concept—persons, objects, behaviors, ideas—on a series of bipolar adjectives using 7-point scales
Correlation
A measure of the relationship between two variables
Range
A measure of variation computed as the difference between the highest and lowest scores in a distribution. For example, the range for the bus stop in front of the bar is from 25 to 35, or 35 - 25 = 10 years For ordinal-, interval-, and ratio-level data
interquartile range
A measure of variation in a set of numerical data, the interquartile range is the distance between the first and third quartiles of the data set.
Percentile
A measure of variation to tell the score at a specific place within the distribution. Example: median is 50th. For ordinal-, interval-, and ratio-level data
Standard Deviation
A measure of variation; it is also the most comprehensive and widely used. The "average difference" between all scores and the mean. Requires an interval or ratio level of measurement. The bigger it is, the bigger the average difference. The smaller it is, the more similar the values in a distribution are to one another
Matched pairs design
A method of assigning subjects to groups in which pairs of subjects are first matched on some characteristic and then individually assigned randomly to groups.
Counterbalancing
A method of controlling for order effects in a repeated measures design by either including all orders of treatment presentation or randomly determining the order for each subject.
Random Assignment
A procedure in which all subjects in a study have an equal chance of being assigned to any group or condition.
Double-blind experiment
A procedure wherein both the experimenter and the participant are unaware of whether the participant is in the experimental (treatment) or the control condition
Field Research
A qualitative style in which a researcher directly observes and (usually) participates in small-scale social settings (distinct from pure qualitative) (ex. barber shop, daycare, etc.) -Used when other methods are not feasible
Survey Research
A quantitative social research technique in which one systematically asks many people the same questions, then records and analyzes their answers
Confidence Interval
A range of values, usually a little higher and lower than a specific value found in a sample, within which a researcher has a specified and high degree of confidence that the population parameter lies. High level of confidence (e.g., 95%) that the true population parameter lies with a certain range
Experiment
A research method in which an investigator manipulates one or more factors to observe the effect on some behavior or mental process.
Direct replication
A researcher closely repeats an original study to see if the same effect shows up in the new data
stratified random sampling
A researcher divides the residents of a low-income housing project into two groups: those age 30 and under, and those over age 30. Ten people from each group are randomly selected for interviews. This probability sampling technique is best defined as:
simple random sampling
A researcher draws names out of a hat to see which police officers in a particular precinct will be interviewed for a study on workplace satisfaction. This probability sampling technique is best defined as:
Only the children who participated in every phase of the study should be analyzed.
A researcher is conducting a longitudinal study involving repeated measurements of children as they grow up. Some of the children drop out of the study before it is completed. What should the researcher do about this?
random sampling error
A researcher is interested in whether participants in a drug treatment program are successful in their rehabilitation. He asks for volunteers to participate in a long-term study of post-treatment success. Twenty of the 130 participants volunteer to participate in the four-year follow up study. The bias in this sampling plan is best described as:
Observational/behavioral Measurement
A researcher records observable behaviors (including behavioral indicators of internal states) Observe what they ACTUALLY do Watch someone all day and count number of drinks Observe actual accuracy or speed on tasks Give a stats test and see how many they get right
Yea-saying and nay-saying
A respondent may employ a response set to agree or disagree with all the question
Simple random sampling
A sampling procedure in which each member of the population has an equal probability of being included in the sample.
Quota sampling
A sampling procedure in which the sample is chosen to reflect the numerical composition of various subgroups in the population. A haphazard sampling technique is used to obtain the sample.
Ordinal scale
A scale of measurement in which the measurement categories form a rank order along a continuum.
Ratio scale
A scale of measurement in which there is an absolute zero point, indicating an absence of the variable being measured.
Nominal scale
A scale of measurement with two or more categories that have no numerical (less than, greater than) properties
benchmarking.
A scientific technique that allows a researcher to learn something about a population by studying a few members of that population is called:
Statistic
A set of collected numbers (e.g., numbers telling how many people live in a city) as well as a branch of applied mathematics used to manipulate and summarize the features of numbers
Theory
A set of statements or principles devised to explain a group of facts or phenomena
Multiple baseline designs
A small-N study where researchers manipulate the independent variable/treatment over various amount of time, contexts, or situations. The dependent variable is then repeatedly recorded in order to establish a stable baseline between the intervention stages. When there is a clear association between the timing of the independent variable manipulations and the timing of any changes in the dependent variable, it offers evidence for internal validity.
Reversal designs
A small-N study where the researcher observes behavior both before and during the administration of the independent variable/treatment, then takes away the treatment for a period of time to see if the behavior returns to the baseline. If the participant's behavior changes back, the independent variable/treatment was effective to the dependent variable. When this occurs, the researcher is able to test for internal validity and make a causal statement. This design is used to examine treatments that are not suspected to have long-term impact.
Pilot study
A small-scale study conducted prior to conducting an actual experiment; designed to test and refine procedures.
Focus Group
A special qualitative research technique in which people are informally "interviewed" in a group-discussion setting -Popular way for market researchers to test their products -Topics might include public attitudes (e.g., race relations, work- place equality), personal behaviours (e.g., avoiding sexually transmitted infections), a new product (e.g., breakfast cereal), a political candidate -Includes moderator
single subject design
A special type of within-subjects designs using one participant (human or animal) or perhaps one group to assess changes within that individual or group. examples: A-B Design - baseline, intervention A-B-A Design - baseline, intervention, no intervention A-B-A-B Design - baseline, intervention, no intervention, intervention
Hypothesis
A statement that describes or explains a relationship between variables that can lead to several different observable and measurable predictions
Replicability
A study's potential to have its results replicated.
Ecological validity
A study's similarity to real-world contexts
Frequency Distribution
A summary chart, showing how frequently each of the various scores in a set of data occurs. • It can be used with nominal, ordinal, interval, or ratio-level data and takes many forms: Raw Count, Bar Chart, Grouped Data Frequency, and Frequency Polygon
Theory
A system of interrelated ideas used to explain a set of observations.
Frequency Table
A table for organizing a set of data that shows the number of times each item or number appears. and then find percentage
Latin square
A technique to control for order effects without having all possible orders or to counterbalance.
Theoretical Saturation
A term associated with the grounded theory approach that refers to the point at which no new themes emerge from the data and sampling is considered complete
Science of the Sophomore
A term used to refer to the potentially limited external validity of studies based on undergraduate samples, usually using the experimental method
Hypothesis
A testable prediction, often implied by a theory.
History Effects
A threat to internal validity due to something that occurs and affects the dependent variable during an experiment, but that is unplanned and outside the control of the experimenter
Mortality
A threat to internal validity due to subjects failing to participate through the entire experiment
Maturation
A threat to internal validity in experimental research due to natural processes of growth, boredom, and so on that occur to subjects during the experiment and affect the dependent variable
Experimenter Expectancy
A threat to internal validity that occurs when the treatment "spills over" from the experimental group, and control group subjects modify their behaviour because they learn of the treatment
Diffusion of Treatment/Contamination
A threat to internal validity that occurs when the treatment "spills over" from the experimental group, and control group subjects modify their behaviour because they learn of the treatment. (ie. chatting during break)
Questionnaire
A tool used in survey research consisting of a list of questions asked to a respondent
Posttest-only design
A true experimental design in which the dependent variable (posttest) is measured only once, after manipulation of the independent variable.
Purposive sampling
A type of haphazard sample conducted to obtain predetermined types of individuals for the sample
Histogram
A type of bar chart used to visually display the distribution of a continuous variable
Third-variable problem
A type of confounding variable in which a third variable leads to a mistaken causal relationship between two others.
Pearson product-moment correlation
A type of correlation coefficient used with interval and ratio scale data. In addition to providing information on the strength of relationship between two variables, it indicates the direction (positive or negative) of the relationship.
Sequential Sampling
A type of non-random sample in which a researcher tries to find as many relevant cases as possible until there is no new information or diversity from the cases. Similar to purposive with one difference: in this sampling, a researcher continues to gather cases until the amount of new information or diversity of cases is filled, whereas in purposive the goal is to get every possible case until resources are exhausted. ie. Interviews with 60 widows over 70 years of age who have been living without a spouse for more than 10 years
Snowball Sampling
A type of non-random sample in which the researcher begins with one case then, based on information about interrelationships from that case, identifies other cases, and then repeats the process again and again. Multi-stage technique ie. heroin users
Quota Sampling
A type of non-random sample in which the researcher first identifies general categories into which cases or people will be selected, then he or she selects a predetermined number of cases in each category. ie. male and female, under the age of 30, 30 to 60 etc.
Haphazard Sampling
A type of non-random sample in which the researcher selects anyone he or she happens to come across. Cheap, quick, but ineffective and unrepresentative ie. street interview or show asking caller to vote online
Purposive Sampling
A type of non-random sample in which the researcher uses a wide range of methods to locate all possible cases of a highly specific and difficult-to-reach population Appropriate for 3 situations: 1. Unique cases (specific trendy women's magazine) 2. Members of a difficult to reach, specialized population (sex workers) 3. Researcher wants to identify particular type of case for in-depth investigation
Deviant Case Sampling
A type of non-random sample, especially used by qualitative researchers, in which a qualitative researcher selects unusual or nonconforming cases purposely as a way to provide greater insight into social processes or a setting. ie. studying high school dropouts who are from stable two-parent, upper-middle- income families who are geographically stable and well educated
Simple Random Sampling
A type of random sample in which a researcher creates a sampling frame and uses a pure random process to select cases. Each sampling element in the population will have an equal probability of being selected
Random Sample
A type of sample in which the researcher uses a random- number table or similar mathematical random process so that each sampling element in the population will have an equal probability of being selected
Multivariate
A type of statistical analysis in which the effects of multiple variables are assessed simultaneously, such as an analysis of health care cost controlling for age, sex, comorbidities, and inclusion in the intervention versus comparison group of a study
Open-ended Questions
A type of survey research question in which respondents are free to offer any answer they wish to the question
Participant variables
A variable with levels that are selected and measured. (ex: age, gender, ethnicity)
traditional view of research participant
A view where participant contributes behavior to the experiment like urine is contributed to a medical experiment. Very objective.
Meta-analysis
A way to average the results of all of the studies that tested for the same variables to come to a conclusion that the whole body of evidence supports.
Nonequivalent control group interrupted time-series design
A within-groups quasi-experimental design that has two or more groups in which participants have not been randomly assigned (treatment group and comparison group), and are repeatedly measured on a dependent variable before, during, and after the interruption caused by some event- the timing of this event often differs among the groups.
10. A double-blind procedure is used in an effort to reduce the likelihood that _____ of condition will influence the results; therefore, both the participants and the administrators of treatment are unaware of which treatments are being provided. A) the expectancies B) the inexperience C) the unfamiliarity D) the foreignness
A- the expectancies
8. _____ is an empty control group often used in clinical research. A) A waiting-list control group B) A placebo control group C) An intervention group D) A research participant pool
A- waiting list
Scale sensitivity
Ability of the scale to detect differences. 3 numbers lack sensitivity; more than 7 provides meaningless sensitivity.
Sections of a Journal Article
Abstract Introduction Methods Results Discussion References
How to Avoid Type I Error
Adjust alpha level to a more stringent level. Use two-tailed instead of a one-tailed. Bonferroni adjustment: dividing the alpha level (.05) by the number of statistical tests performed to create a new and more stringent alpha level.
research
After deciding on the problem, you must do _______________ to find out what is already known about your topic.
Management-Oriented
Aid leaders in making decisions about a program. Often looks at context, input (resources), process & product.
Behavioural Couples Therapy is often used for what psychological disorders?
Alcoholism and Substance Abuse
constants
All other factors (except the two above) that could possibly affect the outcome of the experiment and must be kept the same are called _____________.
What is the purpose of a methods section?
Allows individuals reading the paper to replicate the study as well as to identify flaws and potential areas of improvement for future research
Name three things that should be avoided when writing in APA format
Ambiguity, Changes in topic, and Colloquial expressions
Margin of Error
An estimate about the amount of sampling error that exists in a survey's results
Field experiment
An experiment that is conducted in a natural setting rather than in a laboratory setting.
Experimental realism
An experiment in a laboratory is made to create a setting in which people are made to feel authentic emotions, motivations, and behaviors.
Field Experiment
An experiment set up in the real world, usually with participants who are not aware that they are in a study of any kind. -Less control over experimental condition -Greater external validity, lower internal validity
Solomon four-group design
An experimental design in which subjects are randomly assigned to two control groups and two experimental groups. Only one experimental group and one control group receive a pretest. All four groups receive a post-test. Researcher believes that the pretest has an influence on the treatment
Classical Experimental Design
An experimental design that has random assignment, a control group, an experimental group, and a pretest and post-test for each group
demand characterisitc
An experimenter is interested in whether people will rate food as tastier when it's served on matching plates and utensils, compared to mismatched utensils. However, it's a repeated-measures design, and some participants realize that they are "expected" to like the food better in the matching condition. What kind of confound is this?
nonequivalent control group design
An independent-groups, quasi-experimental, pretest only design that has at least one treatment group and one comparison group, but participants that have not been randomly assigned to the two groups.
Correlation coefficient
An index of how strongly two variables are related to each other.
Cronbach's alpha
An indicator of internal consistency reliability assessed by examining the average correlation of each item (question) in a measure with every other question
Interrater reliability
An indicator of reliability that examines the agreement of observations made by two or more raters (judges).
Focus group
An interview with a group of about 6 to 10 individuals brought together for a period of usually 2-3 hours.
The role of the survey interviewer
An interviewer can increase the response rate by doing the "work" involved in the survey. They can also observe the respondent and the setting within which the interview takes place (typically the respondent's house). Of course, the presence of an interviewer may bias the respondent's answers on particularly sensitive questions.
Theoretical Sampling
An iterative sampling technique associated with the grounded theory approach in which the sample size is determined when the data reach theoretical saturation -A researcher does not know in advance how many individuals he or she needs to interview
Intention to treat versus per protocol
Analytical approach in which the outcomes for all those who are assigned to the treatment or intervention of interests are included in the analysis in contrast with a per protocol approach in which only those who successfully complete the trail are analyzed
statistical power you don't need as many participants in a within-groups study to find a significant effect.
Another advantage of within-groups design is increased This means:
Who is the SLC research officer?
Anthony Wright
Situation noise
Any kind of external distraction that is able to cause variability within-groups that complicate between-groups differences. This can be minimized by making sure the experimental surroundings are controlled for.
Generality
Applied interventions are designed from the outset to operate in new environments and continue after the formal treatments have ended
Alternate forms reliability
Assessment of reliability by administering two different forms of the same measure to the same individuals at two points in time.
Respect of Persons
Autonomy, Informed Ongoing Consent
Imagine that you are reading a journal article and you see the following sentence: "The study used a 2 x 2 x 4 design." Based on this sentence alone, you would know which of the following pieces of information? The number of participants in the study The number of main effects that need to be examined The statistical significance of the interaction The number of participant variables
B
The number of main effects that need to be examined is _________ the number of independent variables. more important than equal to unrelated to independent of
B
Which of the following is a reason that psychologists especially value meta-analyses? Meta-analyses eliminate the need for replications. Meta-analyses allow researchers to examine the strength of a relationship. Meta-analyses are immune to threats to internal validity. Meta-analyses always take less time to conduct.
B
Which of the following types of study support external validity? A study using non-randomly selected participants A study that adds a new independent variable A study that is a direct replication A failed replication study
B
Introduction
Background, research question(s)
Statistical Relationships
Based on two ideas: correlation and independence
What are the 4 criteria for empiricism?
Be acquired from systematic observation Be acquired from objective (unbiased) observation Be independently verifiable by other observer Provide a tentative answer.
Dr. Gavin is conducting a 2 x 4 independent-groups factorial design. Assuming he wants 25 people in each cell, how many participants does Dr. Gavin need to recruit? 100 150 200 250
C
Which of the following studies is most likely to be conducted in generalization mode? Examining whether dog ownership decreases stress levels Exploring the relationship between weight and number of fast-food restaurants in one's neighborhood Determining how frequently people report fighting with their bosses Investigating whether there is a causal link between receiving cognitive therapy and increased coping skills
C
9. _____is a statistical analysis that tests for differences between two or more categorical independent variables, when one is a between-subjects variable and another is a within- subjects variable. A) A one-way analysis of variance B) A repeated-measure analysis of variance C) A mixed design analysis of variance D) A two-way analysis of variance
C - mixed design of variance
Mode of Delivery
Can affect responses to questions that may be affected by social desirability (e.g., online surveys individuals may be less swayed to give "desirable" answers
What organization is in charge of animal care in Canada?
Canadian Council on Animal Care
Scientific research involves evidence being obtained in which ways?
Careful, systematic, and objective observations
Experimental Design
Cause & effect quantitative design
Causal Relationship
Cause and effect
Nonlinear Relationship
Change in one entity does not correspond to change in the other entity
Carryover Effects Confound (order effects)
Changes in the *scores observed in one level of an independent variable are caused by the lingering effects* of a level of an independent variable previously experienced
Type I Error
Claiming a result was significant when you should have not done so. Reject the null when you should not have done so. False positive. Tend to be more problematic because false knowledge is distributed. ex. Telling a man that he is pregnant.
student
Coercion is most common when using _______ participants.
Conceptual Variable and Operational Definition Examples
Conceptual Variable -Binge Drinking -Learning -Anxiety -Height Operational Definition -4 to 5 drinks in two hours -Score on an exam -Score on anxiety inventory -Inches? Centimeters?
Applied research
Conducted to address issues in which there are practical problems and potential solutions.
Motivation
Confirmation bias/Cherry-picking Asking biased questions The bias blind spot
Randomization
Controlling for the effects of extraneous variables by ensuring that the variables operate in a manner determined entirely by chance.
Cross-Lag Correlation
Correlation between an *earlier measure of one* variable and a *later measure of another* variable - remains across time - temporal precedence
True
Cross-sectional research involves data collection at one point in time. (The findings from this research cannot be used to draw conclusions about change over time, but they can provide a snapshot of a social condition or population at that point in time). T/F
Control
Crucial in experimental research, important to isolate the effects of the treatment to eliminate alternative explanations. Deception allows the experimenter to control subjects' perception of the situation
Demand characteristics
Cues that inform the subject how he or she is expected to behave.
An author might use all of the following methods to indicate statistical significance EXCEPT: Using the word significant Including an asterisk in a table Using the notation p < 0.05 The inclusion of a graph
D
In a previous correlational study, Dr. Lafayette has found that owning dogs is associated with lower levels of daily stress in a sample of returning war veterans. Which of the following would constitute a shift from theory-testing mode to generalization mode? Conducting a study on the same sample of war veterans, this time using a different measure of daily stress Conducting a study examining the stress reduction mechanism Conducting an experimental study in which he assigns people to own a dog or not Conducting a study on dog ownership using a sample of elderly participants from a local retirement home
D
In small-N designs, each participant is treated: with multiple interventions. by a clinical psychologist. as a data point. as a separate experiment.
D
Which of the following is true of small-N experiments? Small-N designs determine whether a finding is replicable by doing a test of statistical significance. Data from small-N designs are presented as averages. Data from small-N designs are grouped together. Each person in a small-N design is treated as a separate experiment
D
Which of the following types of replications investigate the same research question but use different procedures? Statistical replication Direct replication Replication-plus-extension Conceptual replication
D
5. _____ is an effect in which two variables combine to produce an outcome that is smaller than what each individual variable contributes. A) Carryover effect B) Synergistic effect C) Main effect D) Suppression effect
D- suppression effect
Cross-sectional
Data collected only one point in time to understand needs, attitudes, make comparisons or evaluate program.
Longitudinal
Data collected over time with various pre-established points for administration.
Nonparametric
Data for which the probability distribution is unknown or known not to be normal skewed examples: Mann-Whitney and Spearman's test
check if the the participant realized the deception
Debriefing is a good time for manipulation checks, which are?
Operational definition
Definition of a concept that specifies the method used to measure or manipulate the concept.
When they can avoid three problems: observer bias, observer effects, and reactivity
Describe how researchers can make observations with good construct validity.
Association Claims
Describe the relationship between two variables. Also known as correlations or covariation. Relationship between x and y. Involves at least two variables Cannot make causal claims Example: -Children of divorced parents are more likely to suffer from an anxiety disorder
Naturalistic observation
Descriptive method in which observations are made in a natural social setting. Also called field observation.
Design your own 2X2 factorial study on the topic by doing this:
Determine the two independent variables involved in testing this topic. Determine two levels for each of the independent variables. Determine and develop the dependent variable. Develop a hypothesis.
Strength of Relationship
Determined by numeric value of the correlation coefficient. Range from -1.00 to +1.00. -1.00 is a perfect negative correlation. +1.00 is a positive correlation. 0.00 indicates no correlation
Converging evidence
Different measures provide outcomes pointing to the same conclusion
Counterbalancing
Different subsets of participants complete conditions in different orders - *how to avoid order effects confounds*
Behavioral measures
Direct observations of behavior.
Applied Research
Directed towards solving practical problems
Coverage
Disadvantage of online survey. Involves sampling and unequal access or use. Older, less-educated, lower-income, and more rural people are less likely to have good internet access
Complexity of Questionnaire Design
Disadvantage of online survey. It is best to provide screen-by-screen questions and make an entire question visible on the screen at one time in a consistent format with drop-down boxes for answer choices
Privacy and Confidentiality
Disadvantage of online survey. Researchers should encrypt all data, use secure websites
Random Assignment
Dividing subjects into groups at the beginning of experimental research using an unbiased, random process, so the experimenter can treat the groups as equivalent. Random in the statistic or mechanical sense.
America the Beautiful
Documentary in which women said they did not feel beautiful, but the majority do
Non-probability Sampling
Does not involve random selection. Used by qualitative researchers for detailed findings
C
Dr. Alfonse, a developmental psychologist, conducts a study to determine whether children prefer books with drawn illustrations or with photographs. A group of 30 preschoolers are shown two copies of a book (Ferdinand the Bull) at the same time. Although the story is the same, one book is illustrated with drawings and the other is illustrated with photos. Students are then asked to indicate which book they prefer. This is an example of which of the following designs? Longitudinal design Repeated-measures design Concurrent-measures design Posttest-only design
D
Dr. Kline, an environmental psychologist, conducts a study to examine whether visiting zoos causes people to have more positive attitudes toward environmental conservation. He asks a group of 45 people attending the zoo on a Saturday morning about their attitudes. He finds that 69% of the people report having a positive attitude after their visit. Which of the following is true? His control group is people who did not visit the zoo. He does not have a dependent variable. He can make a strong causal claim about the effect of zoo visits on environmental attitudes. His study does not qualify as an experiment.
Same conclusion
During the Rosenthal Study, it was observed that *meta-analysis literature reviews* had what sort of conclusions?
Latin Square
Effects of three interventions assessed. Randomly assigned groups with different sequential order of intervention.
Experimental Control
Efforts to minimize effects of extraneous variables through design, development and implantation of research: - Random Assignment - Comparison Group - Counterbalancing - Matching
What does it mean to be Scientific?
Empirical Approach Theory-data cycle Make findings public
Which method obtains answers from direct sensory observation?
Empiricism
Planning
Entering and gaining access to a field site is a process that depends on common sense judgment and social skills
Input Evaluation
Environmental assessment of available resources in work environment.
Ratio
Equal intervals, a *meaningful zero* zero means "none" (calories consumed, weight, brain activity)
According to the concept of reliability, inconsistency in measurement comes from what?
Error
True
Ethics & research: The principle of doing no harm means that researchers should never subject research participants to physical danger, cause them psychological harm, or expose them to legal liability.T/F
evolution
Ethics are in a rapid state of ________.
Risk-benefit analysis
Evaluation of the ethical implications of a research procedure by considering the benefits of the research in relation to the potential risks to participants.
Conceptual Definition
Everyday vernacular use of a term. Used by journalists when reporting.
Empiricism
Evidence is acquired from systematic, objective (unbiased) OBSERVATION, independently verifiable by other observers that provides a tentative answer.
Type A
Exaggerated sense of time urgency, competitiveness, and drive
Quasi-experimental Design
Experimental designs that are stronger than pre-experimental designs. They are variations on the classical experimental design that an experimenter uses in special situations or when an experimenter has limited control over the independent variable
Pre-experimental Design
Experimental designs that lack random assignment or use shortcuts and are much weaker than the classical experimental design. They may be substituted in situations where an experimenter cannot use all the features of a classical experimental design, but they have weaker internal validity
causal claims
Experiments are conducted to make ________
Each question be clear to answer so it doesn't confuse respondents (or influence their answers)
Explain how carefully prepared questions improve the construct validity of a poll or survey.
Debriefing
Explanation of the purposes of the research that is given to participants following their participation in the research.
Internal Validity
Extent to which independent variable truly effect changes in dependent.
anatomy of a mixed design ANOVA
F (2, 87) = #, p=#, eta2 = # f= f test symbol 2= between subjects DF 87 = within subjects DF p = significance eta= calculated effect size
Ceiling effect
Failure of a measure to detect a difference because it was too easy
True or False: Correlation always means that a causation is present
False
True or False: In order for a measure to be reliable, it must be valid
False
Periodic Probe
Follow-up questions (helpful for respondent fatigue)
Solomon four group
Four groupings used to mitigate testing effects of pretest while assuring equal groups are used: - Control group that receives pretest and posttest. - Control group that receives only posttest. - Treatment group that receives both pretest and posttest. - Treatment group that only receives posttest.
Participant-Oriented
Gain insight through the participants in the program. What do they think/need of the program? Different perspective than those who are implementing the program or funding it.
pretest-posttest
Give dependent variable, then manipulate, then give dependent variable again. Ensures equivalent groups Gives change and whether significant Gives power and can see confound and remove statistically
Healthy adherer effect
Greater adherence to health-promoting behaviours, such as medication adherence, is indicative of overall healthy behaviour
Laboratory Experiment
Greater internal validity but lower external validity. They are logically tighter and better controlled but less able to be generalized
Continuous Variable
Have a large number of levels. -More than 10 levels Examples: -response time, cigarettes per day
Discrete Variable
Have a small number of levels. -Less than 10 levels Examples: -gender, smoker status
Empiricism - Objective
Have to be able to come to a consensus
Within Group difference
How do you know how much *difference* is expected *due to chance* within a group?
Fail reject Null Hypothesis (p >0.05)
How do you know if you have *no significant* results in your study?
Effect Size
How much evidence is present that the results represent a general society
Operational Definition
How the researcher decides to measured or manipulate the conceptual variable. Also called an operational variable, measure
Hybrid Design
Hybrid designs have at least one factor that we cannot actually manipulate (e.g., gender). In factorial designs, you can test the impact of an experimental variable and a nonexperimental variable at the same time. any factorial design that has at least one quasi-independent variable examples: race, age, gender, (something you cant change about participants
Null Hypothesis
Hypothesis where there is no significant difference between specified populations, always based on data already established. ex. Each bag has 17 M&M's.
Alpha (aka critical P value)
If the probability is less than the alpha (critical P value, usually 0.05), null hypothesis is rejected
hypothesis
If your conclusion leads you to new questions, you should create a new ______________________.
Practice effect (also learning effect)
Improvement in participant performance with repeated testing.
Nothing
In a *prospective* research study, you need more information since it can be correlative or experiment, what can you assume?
Order Effects Confound
In a [[within-subjects design]], when a participant *experiences a series of different levels of an independent variable and a series of measurements*, their performance at any point may be influenced by an experience that occurred earlier
Simple main effect
In a factorial design, the effect of one independent variable at a particular level of another independent variable.
main effect
In a factorial design, the overall effect of one independent variable on the dependent variable, averaging over the levels of the other independent variable. a HYPOTHESIS (prediction) that focuses on one independent variable at a time, ignoring all other independent variables
Order effect
In a repeated measures design, the effect that the order of introducing treatment has on the dependent variable.
Overt Observer
In field research, refers to a researcher who is studying the group members with their full knowledge
Complete Participant
In field research, when a researcher fully participates in all aspects of the study group's activities as though a member of the group (too much sympathy can be an issue)
Complete Observer
In field research, when a researcher only observes the study group without participating in their activities
Devil's Advocate
In focus group research, refers to a person whose role is to argue against a dominant idea and who could introduce new questions and new ways of thinking into the group so as to prevent the tendency toward group conformity
Groupthink
In focus group research, refers to people's natural desire to avoid conflict and lean toward group consensus, even when the opinion of the group does not reflect their own personal opinions
quantitatively
In order to make a graph, the data collected must be ____________________ testable.
In which two aspects should research questions be evaluated in?
Interestingness and Feasibility
Discriminant validity
Is the measure weakly (or not at all) related to established measures of a different construct? Is the measure weakly correlated with something it shouldn't be correlated with? (this is good, do not want high correlation with something measure should not be correlated with)
Questionnaire instructions
It is important that the directions for completing the questionnaire are very clear.
Self Questionnaire- Monitoring returns
It is important to keep track of returns so that the researcher identifies who need to be contacted again and the optimal time for sending follow-up (reminder) mailings.
Reverse Scoring
Items stated opposite of the other items; must be reversed in analysis.
Problems with Experience
Lack of COMPARISON: Typically no fair control condition CONFOUNDED Claim: Other possible explanation for your findings -Example: You feel calm after you exercise, you think its the exercise but it could be the temperature of the gym or the music you had playing
What are two problems with claims based on experience?
Lack of COMPARISON: Typically no fair control condition CONFOUNDED Claim: Other possible explanation for your findings -Example: You feel calm after you exercise, you think its the exercise but it could be the temperature of the gym or the music you had playing
Parametric Statistic
Make assumptions about population fitting normal curve.
Type 1 error
Making a claim of association or cause when there isn't one Rejecting the null when it is true
Type 2 error
Making a claim of no association or cause when there is one Not rejecting the null when it is false
National Research Act (1974)
Mandated regulations for protection of subjects
Correlation
Means that things go together or are associated (e.g., people with higher values on the income variable are likely to have higher values on the life expectancy variable)
What two things must you be able to do with variables that are being studied?
Measure and Define them
Face validity
Measures whether a test looks like it tests what it is supposed to test Does the measure look like a plausible measure of the construct at face value?
In a Negatively Skewed Distribution...
Mode is the highest mean is the lowest median is in between the two
In a Positively Skewed Distribution...
Mode is the lowest Mean is the highest Median is in between the two
Type B
More relaxed and easy-going personality
We cannot make causal claims from correlational data
Most important thing to remember about correlations
Forced Choice Question
Multiple Choice, True/False, Yes/No questions are examples of...
number needed to harm
NNH = 1/attributable risk
Empiricism - Independently Verifiable
Need to be able to distribute data and make it so another scientist can follow the methods you used and get the same answer
examples of independent groups quasi-experiment tests?
Nonequivalent control group design (posttest-only) - Nonequivalent control group design (pretest/posttest).
Scientific Journalism
Not considered science per se. Is scientific news and commentary written by another party (usually a reporter, not a scientist) and posted to a wide readership (usually a non-expert audience).
Survival
Number of people alive at a certain point (often 1 year or 5 year) after diagnosis
survival rate
Number of people alive divided by population
Limitations of the Empirical Method
Observations can be misinterpreted, sensory experience can be swayed by beliefs, time consuming, can be dangerous relying on one person's opinion
Association Claims
One level of a variable is likely to be associated with a particular level of another variable. Also known as correlations or covariations
When selecting a mentor in the field, what is it important to consider?
One should be in the field for years prior to being considered a mentor
Intuition
One source of information "Common sense" Example: -It makes sense to me that speaking two languages from a young age would make it harder to learn each because the terms from one language interfere with another.
Authority
One source of information Authority figures or convention Example: -Dr. Mitch Smith, a dentist, says that he always recommends that his patients chew Orbit brand gum.
Experience
One source of information Can be: -Personal -Anecdotes or testimonials (from others) Example: -Exercise makes me feel calm - after my Cycle bar classes, I can always sleep well.
Causal
One variable is responsible for changing another variable. Involves at least two variables Implies that the reason for change in one variable is due to the other variable Example: Music lessons enhance IQ
Criterion Variable
Outcome variable being measured (dependent variable)
within-groups repeated measures counterbalance
Participants are exposed to different conditions at different times. Researchers need to _________ the order of presentation.
concurrent measures contact/comfort with monkeys
Participants are exposed to different conditions at the same time. example?
posttest only
Participants are randomly assigned to groups, and tested on a dependent variable once. It's a ________ because the DV is measured after the IV.
Reactivity
Participants may simply act differently when they know they are being observed. Possible solutions include: • Unobtrusive observations (blend in). • Wait it out (allow acclimation to one's presence). • Measure the results of a behavior (indirect measures of constructs), rather than the behavior itself. - remember to observe ethically
Self-report Measurement
Participants provide responses to verbal questions about themselves (or others) through surveys or interviews
Ethical Survey
People can conduct surveys in ethical and unethical ways -Major issue is the invasion of privacy -Treat all respondents with dignity and reduce anxiety -Responsible for protecting the confidentiality of data. -Voluntary participation -Some people use surveys to mislead others and produce desired results -Currently, there are no quality-control standards to regulate the opinion polls or surveys reported in the Canadian media.
Reduce Social Desirability Bias
Phrasing questions in ways that make norm violations appear less objectionable and that present a wider range of behavior as acceptable (e.g., face-saving alternatives)
increasing the sample size using stronger manipulations eliminating unwanted sources of variance (noise)
Power is increased by:
the people who were present for the pretest AND the posttest. (don't count the drop-outs).
Preventing attrition threats is easy. Only analyze
What are the pros and cons of journals and journalism?
Pros: easy to read, more likely to influence lives Cons: can be warped and misinterpreted
Consumer-Oriented
Provide consumers with complete picture on effects of a program. Consumers often other counselors, prospective clients, administrators.
Expertise-Oriented
Provide professional judgements about quality of a program. Ex. Accreditation standards for CACREP counseling.
Quantitative Research
Provides numerical scores and is submitted to statistical analysis for summary and interpretation
Empirically Testable Research Questions
Questions expressed in terms of a single variable or relationship between variables
CONSORT is used to evaluate _____
RCTs
Observer bias
Rather than rating behaviors objectively, researchers might (unknowingly) rate behaviors according to their expectations or hypotheses. - Ex: All participants saw the same taping of a football game between Dartmouth and Princeton. However, the number of observed rule infractions appeared to vary based on which school a person attended
Disadvantages of experimental design
Realism; Mundane vs. Experimental
Semi-participant
Refers to the role of the researcher in field research when he or she participates to some degree in group activities, but not as much as a full member.
Graphic rating scale
Requires a mark along a continuous 100-millimeter line that is anchored with descriptions at each end.
True
Research can be found in academic journals, books, newspapers, magazines, legislative actions, policy decisions, and databases. Some of this information is more objective and accurate than others.T/F
Charm and Trust
Researcher needs social skills and personal charm to build rapport
Central Tendency
Researchers often want to summarize the information about one variable into a single number: mean, median, or mode
Beneficence
Researchers should try not to have any negative impact on the wellbeing of the people who participate in their studies
True
Researchers use three causal rules to determine whether a causal relationship exists between two variables. Each of these three rules must be met before a researcher can prove that one variable is the cause of another. T/F
coefficient of variation
SD/mean; useful for comparison of variability between groups where comparisons are difficult because of differences in scale or size of the subjects being compared
Probability Sampling
Sampling strategy that gives every subject in the population the same likelihood of being selected. Purpose: save time; cost; accuracy of generalization
Family Therapy is often used for what psychological disorder?
Schizophrenia
Hypothesis
Scientific Method Step #2
Draw a Conclusion
Scientific Method Step #7
Alpha Level and meaning related to error
Set that likelihood at what is called an alpha level (usually .05) which means there are only 5 opportunities in 100 that a Type I error will exist.
Interaction
Situation in which the effect of one independent variable on the dependent variable changes, depending on the level of another independent variable.
Qualitative Sampling
Social scientists typically select interview participants through nonprobability sampling (e.g., snowball and purposive sampling are often used)
Gatekeeper
Someone with the formal or informal authority to control access to a site
Weak manipulations Insensitive measures Ceiling and floor effects Design confounds
Sometimes real differences between groups are hard to detect. There can be multiple causes for this:
Spearman Rho Correlation Coefficient (r sub s)
Special form of Pearson r used when one variable is ordinal and the other is continuous. Ex. Class rank (ordinal) and PCA score (continuous)
Point Biserial Correlation Coefficient (r sub pb)
Special form of Person r used when one variable is nominal and the other is continuous. Nominal must be dichotomous (yes/no, true/false, male/female). Ex. Passing the NCE (dichotomous) and grade point average (continuous).
Operational Definition
Specify precisely how the concept/term is measured or manipulated in that particular study. *Must operationally define hypothetical constructs!!*
C
Spontaneous remission in clinical studies is an example of which of the following threats to internal validity? Regression Attrition Maturation Placebo effects
True
Spuriousness refers to a false causal finding. It occurs when a researcher alleges a causal relationship between two variables but fails to confirm at least one of the three causal rules. T/F
p < .05
Statistical Significance
Bivariate Statistics
Statistical measures that involve two variables only
Longitudinal Design
Studies in which the same variables are measured in the same people at different points in time *helps establish temporal precedence*
Double-Blind Study
Study in which neither the experimenter nor the subjects know if the subjects are in the experimental or control groups.
Example of Mixed Design
Study was looking at how long children stare at angry or happy faces based on whether or not they had been previously abused. Between subjects: Child status was naturally occurring- quasi-experimental within subjects: face stimuli- angry or happy
Respect for Persons
Subjects have to give informed consent
Belmont Report
Summarizes ethical principles and guidelines for research involving human subjects. It has three basic ethical principles.
The APA Ethics Code
Ten ethical categories (150 standards) to provide guidance for psychologists
Nuremberg Code (1947)
Ten guidelines for the ethical treatment of human subjects in research
Social Desirability (+ Solution)
Tendency to give answers to look better or conform to social norms ("faking good" or "faking bad") Solution: Make survey anonymous Include filler questions to mask purpose Include 'too-good-to-be-true' questions
Successive Measurements
Test-retest reliability compares scores of two successive measurements of the same individuals and correlates the scores
PsycINFO Database
The American Psychological Association's (APA) definitive resource for abstracts of scholarly journal articles, book chapters, books, and dissertations devoted to peer-reviewed literature in behavioural science and mental health
Who enforces ethical guidelines?
The Institutional Review Board
expedited
The ____________ review does not require approval from the full IRB committee, but only a few members, and is completed more quickly than a full review.
materials
The _____________ list is not numbered.
Validity
The ability of a test to measure what it is intended to measure, are you hitting the bullseye
Scatterplot Precision
The amount of spread in the points on the graph. Close together means highly related, further apart less closely related
Mean
The arithmetic average of a distribution, obtained by adding the scores and then dividing by the number of scores. Used only with interval or ratio data. Compute by adding up all scores, then divide by the number of scores -Strongly affected by changes in extreme values -Used in most statistics
marginal means
The arithmetic means for each level of an independent variable, averaging over the levels of another independent variable.
exhaustiveness
The capacity for a list of attributes to contain all reasonably possible levels of a variable is referred to as:
Internal validity
The certainty with which results of an experiment can be attributed to the manipulation of the independent variable rather than to some other, confounding variable.
Predictive validity
The construct validity of a measure is assessed by examining the ability of the measure to predict a future behavior.
Item-total correlation
The correlation between scores on individual items with the total score on all items of a measure.
Measurement error
The degree to which a measurement deviates from the true score value.
Validity
The degree in which the measurement process measures the variable it claims to measure
Reliability
The degree of stability or consistency of measurement
Construct validity
The degree to which a test measures what it claims to be measuring.
What is important to consider when writing a research proposal in terms of the ethicist on the REB?
The ethist may not have an ABA background, therefore it is important when writing a proposal to write it at a level that this individual will understand
independent variable
The experimental factor that is manipulated; the variable whose effect is being studied.
Discriminant validity
The extent to which scores on the measure are not related to scores on conceptually unrelated measures. I.e. scores on a test designed to assess aggressiveness should not be positively correlated with scores from tests designed to assess intelligence.
Reactivity
The general threat to external validity that arises because subjects are aware they are in an experiment and being studied -Hawthorne Effect
personal pronouns or iffy words
The hypothesis should never contain
File drawer problem
The idea that meta-analysis could be overestimating the support for a theory because studies that find null effects are less likely to be published therefore are less likely to be included in such reviews.
What information can you use to make a judgement about a journal article's impact?
The impact factor (IF) is a measure of the frequency with which the average article in a journal has been cited in a particular year. It is used to measure the importance or rank of a journal by calculating the times it's articles are cited.
Observer Error
The individual who makes the measurements can introduce simple human error
Sampling frame
The individuals or clusters of individuals in a population who might actually be selected for inclusion in the sample.
Power
The likelihood that a study will have results that are statistically significant when the independent variable causes a change in the dependent variable. The more power a study has, the more likely we are able to detect true differences. In order for experimenters to increase the power, they must have large samples, accurate measurements, or strong manipulations to escalate the size of the effect.
-refine their research question -identify a theory related to their research question -propose a hypothesis that predicts the answer to their research question.
The literature review process helps researchers:
Mortality (also attrition)
The loss of subjects who decide to leave an experiment.
Attrition (also mortality)
The loss of subjects who decide to leave an experiment.
Marginal means (how they are represented & calculated)
The means for each level of an independent variable averaging over the levels of another independent variable in a factorial designs study.
Dependent Variable
The measured variable; the outcome - expect to change
Post-test
The measurement of the dependent variable of the experiment after the treatment
Median
The middle score in a distribution; half the scores are above it and half are below it. 50th percentile. First organize the scores from highest to lowest, then count to the middle -Best for skewed distributions
within; between groups
The more unsystematic variance (noise) there is __________ groups, the harder it is to see differences __________ groups (b/c there's more overlap)
Mode
The most common, frequently occurring number. The easiest to use and can be used with nominal, ordinal, interval, or ratio data. Distribution can have more than one.
Population
The much larger collection of animals or people that researchers want to generalize about.
Sampling Element
The name for a case or single unit to be selected. It can be a person, a group, an organization, a social action. ie. an arrest, a kiss
-answer research questions -to resolve disagreements among researchers -to fill gaps within the body of knowledge -to produce more research questions.
The objectives of research are to
main effect
The overall effect of one independent variable on the dependent variable, averaging over the levels of the other independent variable. Overall effect
Odds and odds ratio
The ratio of probability of an event occurring to the probability of the event not occurring that is p divided by (1-p) where p=probability and The amount by which the odds are multiplied for one group compared with another. Odds for men relative to women are 1.5/0.54=2.77 - we say that this is the odds ratio for males, which means that the odds for men are 2.77 times those for women.
True score
The real score on the variable
Generalization mode
The researcher's intent to generalize their findings from their study to other populations or contexts.
Psychology
The scientific study of behaviour and mental processes. It provides detailed and accurate knowledge about psychological problems and establishes whether treatments are effective.
How to Evaluate a Hypothesis regarding p and alpha levels
The smaller the p-value, the stronger the evidence is against the null hypothesis. If the p is low, the Ho must go. If p is less than or equal to alpha (.05), the observed effect is statistically significant. The null hypothesis is rejected in favor of the alternative hypothesis. If p is greater than alpha (.05), there is no statistically significant difference, fail to reject the null hypothesis.
Control Group
The subjects who do not receive the special treatment given to the experimental group.
Confirmation Bias
The tendency to seek out and interpret evidence consistent with what we believe and to deny, dismiss, and distort evidence that contradicts what we believe
True
The third causal rule, lack of plausible alternative explanations, requires the researcher to eliminate all other reasonable causes before concluding that one variable causes another. T/F
How does new research grow out of old? What visual is used to describe this?
The tree-like structure emphasizes the notion that current research (the tips of the branches) is always based in previous research.
Positive Relationship
The two measurements change together in the same direction
Negative Relationship
The two measures change in opposite directions
Systematic Variance
The variability or change in a person's behavior that is related to the other variable of interest
What is it important to consider when using secondary sources?
There is a chance that the source you obtained it from may have misinterpreted the primary source
Main effects (what they are, what they mean, how they are found)
This is the overall effect that one independent variable has on the dependent variable over all of the levels of the other independent variable. It is basically the simple difference/ effect a single variable. -For each independent variable, there is one main effect.
Self Questionnaire- Response rates
This refers to the proportion of the sample that returned the survey. The higher the response rate, the more representative the survey is of the population.
Feasibility
Time, money, technical knowledge and skill, and access to special equipment and research participants
Probability sampling
Type of sampling procedure in which one is able to specify the probability that any member of the population will be included in the sample.
Noise/error variance/unsystematic variability
Undifferentiating variability among the members of a group in an experiment. This gets in the way of detecting the true difference between groups. The more unsystematic variability there is, the more the groups' scores overlap, which causes the difference between the two groups to become indistinguishable.
Multiple measures
What *increases power* by including multiple ways to measure the dependent variables?
Increase reliability of Measures
What *increases power* by reducing error, wobble, and variance by using the *most recently developed tests* for measurement?
- History - Maturation - Instrumentation - Selection - Regression to Mean
What *threats* to *internal validity* are ruled out by *random assignment*?
Standard deviation
What accounts for how much *dispersion* there is *around* the *mean*?
Systemic error
What are *differences* that can be explained *due* to a *confound*?
Between subjects
What are *groups* that are *completely independent* of one another?
Within Subjects
What are *groups* that are some how *related* to one another?
- Convenience sample - Quota sample - Snowball sample
What are *non-probability sample* methodologies?
- Increase sample size - Increase external validity
What are the *strengths* of *Multi-Center studies*?
- History - Maturation - Testing - Instrumentation - Selection - Regression to the mean - Diffusion or imitation of tx - Compensatory rivalry - Compensatory equalization
What are the *threats* to *internal validity*?
Covariance Temporal precedence Internal validity
What are the three criteria for a causal claim?
- Effect Size - Sample Size
What are the two components of a significance test that when increased are the most effective way to *increase power*?
- Experimenter Expectancy - Volunteer-Non-response bias - Withdrawal - Contamination - Attrition - Mortality
What are the type of *biases* that weaken *internal validity*?
- Increase *Effect Size* - Increase *Sample Size* - Increase effect - Increase reliability of Measures - Decrease variance - Multiple measures
What can researchers do to increase power of their findings?
Statistical significance
What can you ascertain if there is less than 5% likelihood that a result is due to chance?
Inclusion criteria
What comes *firsts*, decides who will be *included in a study*, and helps determine whom the results *can* be *generalized* to?
Exclusion criteria
What comes *second*, decides who will be *excluded in a study*, and whom the results *cannot* be *generalized* to?
Comparison Group
What compares *multiple tx* options?
Historical Control
What compares data to *known populations* or *previous sampling*?
Experimenter Expectancy bias
What involves an *experimentors hypothesis* and *expectations* affecting the result or *outcome* of an experiment that is limited by a *double blind study*?
Comparing Means
What involves looking at *two groups* and comparing the *means* of each group to make *inferential statistics*?
Diffusion/Imitation
What is *threat* to *internal validity* that involves the *control* group being *affected* by *tx* from the *tx group*?
Snowball sampling
What is a *non-probability sampling* methodology that is future sampling guided by results of earlier data collection?
Pure random selection
What is a *probability sampling type* involves each person in the population has an equal chance of being in the sample?
Central tendency
What is a *statistical index* of a typical set of scores that comes from the *center* of the *distribution* of scores with the three most common being: *mean*, *median*, and *mode*?
Floor effect
What is a *threat* to *internal validity* from *instrumentation* where *can't go any lower* with the specific instrumentation used?
Maturation
What is a *threat* to *internal validity* that involves the *mere passage of time*?
History
What is a *threat* to *internal validity* that is an *event* or *events* that deferentially *affects* the *tx* and *control* groups?
Instrumentation
What is a *threat* to *internal validity* that is due to *changes* or *limitation* in *instruments*?
Testing
What is a *threat* to *internal validity* that is due to *repeated exposure* to a *independent variable*?
Theory
What is a prediction about *why* variables are related with an actual story behind the prediction?
A statement explaining why you think that your hypothesis is correct.
What is a rationale?
Opportunistic Data Torturing
What is an *analysis and publication bias* where a researcher analyzes to find anything significant, then develops a *hypothesis based off* the *data*?
Publication Bias
What is an *analysis and publication bias* where only the *significant results* are more likely to be *published*?
Research questions
What is an *answerable inquiry* into a specific concern or issue that is the *initial step* in a research project?
Variable
What is an element of a hypothesis that involves *how* a *construct's* *operationalization* is *measured*?
Operationalization
What is an element of a hypothesis that is *how a construct* is defined within one specific study?
Construct
What is an element of a hypothesis that is an abstract representation including both *independent* and *dependent variable*?
File drawer problem
What is another name for *publication bias* which is where only the significant results are more likely to be published?
Research hypothesis
What is another name for an *alternative hypothesis (HA)* in which you predict that a *relationship exists*?
Confound
What is any *other explanation* for the *results* other than the hypothesized explanation?
Significance test
What is the *effect size* x the *sample size*?
Confidence interval
What is the *range of values* within which a population parameter is estimated to lie, which is the mean difference on test scores which speaks to validity?
Effect size
What is the *strength of relationship between control* and *tx group*, where the farther from zero in negative or positive direction, the stronger the correlation relationship?
Compensatory equalization
What is the *threat* to *internal validity* where the *control* group receives *similar tx* through *alternate means*?
Selection
What is the *threat* to *internal validity* which is *bias* in terms of *who ends up* in the *experimental* or *control group* and is ruled out by *random assignment*?
Independent variables
What is the construct which is an *actor* variable that is manipulated by the researcher?
Dependent variables
What is the construct which is an *affected* variable that relies on the independent variable?
Internal validity
What is the extent to which one can make *causal inference* that is strengthened by *random assignment*?
5%
What is the likelihood of a *type I: alpha-error* where there is a rejection of the null hypothesis, but in reality the medication is not effective, giving a *false positive*?
20%
What is the likelihood of a *type II: beta-error* where there is a failure to reject the null hypothesis, but in reality the medication is effective, giving a *false negative*?
Population
What is the total group of ALL people with defined characteristics?
Frequency distribution
What is usually a graph or table representing all values from least to greatest value with a *frequency count* of each *value*?
Random Error
What is when the *differences between participants* are *random*?
Open Ended Question
What learning strategies do you use when preparing for an exam?
Dropouts significantly different
What method of dealing with *attrition bias* accounts for significant different between dropouts and those who did not in terms of certain *characteristics*?
Tx attrition as dependent variable
What method of dealing with *attrition bias* considers that attrition may happen because of the *nature of the study* or *intervention*?
Intention to tx statistical adjustment
What method of dealing with *attrition bias* involves creating a *projection* of what would happen if the participants completed the study?
Remove noncompliant
What method of dealing with *attrition bias* involves removing pts who *do not* complete or follow the *proper protocol*?
- More competent - Different metrics - Different protocol
What needs to be considered in *multi-center studies* that can change the interactions?
Tx-by-Center interactions
What needs to be considered in *multi-center studies* where the tx may be more or less effective at different locations?
informed consent to research
What section of code is this? what the psychologist plans to do must be told truthfully to the participants unless deception is involved.
Rosenthal study
What study observed that *narrative reviews* provided *varied conclusions* while *meta-analysis* provided the *same conclusions*?
Treatment
What the independent variable in experimental research is called
Interval
What type of *data format* is the difference between units in uniform, but there is *no absolute zero* like temperature?
Crossover design
What type of *experimental design* involves *more than one* type of *tx* administered to each subject where the tx are *provided sequentially*, *rather* than *congruently*, and *comparisons* are made of the effects of the *different tx* on the *same subject*?
Narrative Literature Review
What type of *literature review* involves making the researchers own conclusions about what the literature says leading to bias in articles selected and how the researches a conclusion?
Systematic literature review
What type of *literature review* involves specific rules about how literature is selected for the analysis so anyone else could replicate how selected literature?
Randomized Block Design
What type of design *randomly assigns* at least one variable, but *can't randomly assign all variables*?
Correlation Design
What type of design has *no random assignment* and doesn't give inference to statistical tests?
Quasi-experimental design
What type of design has *no random assignment* but gets closer to the *causal inference* with *other methodological elements* with the goal of eliminating the top rival hypothesis?
Cross-sectional design
What type of design has *no random assignment*, but still *compare groups*?
Repeated Measure design
What type of design is anytime there is *more than one measure*?
Observational design
What type of design is simply *observing* relationships as they *exist*?
Negative skew
What type of frequency distribution is skewed to the *right*?
Prospective research
What type of research involves *looking forward* and planing ahead to collect data and *cannot assume anything* since it can be *correlative* or *experimental*?
- Pure random sample - Stratified random sample - Hierarchical random sample
What type of sampling are *probability sample* methodologies?
Measurement Bias
When a test or test item is not equally valid for individuals from a particular gender, ethnicity, or social class, or other group
foolish or stupid
When debriefing, do not make the participant feel ________.
A
When interrogating experiments, on which of the big validities should a person focus? Internal validity External validity Construct validity Statistical validity
Practice Effects Confound (+)(order effects)
When prior participation in a level of an independent variable or measurement *[[provides]] participants with additional skills that influence their performance* on subsequent measurements
Fatigue Effects Confound (-)(order effects)
When prior participation in a level of an independent variable or measurement *[[tires]] the participant and influences their performance* on subsequent measures
Coercion to Participate
When someone in authority applies pressure or attempts to influence individuals into participating. Individuals believe that not participating will have negative consequences. Ex.) Offering exceptionally high or excessive rewards for participating.
Selection Confound
When the *kinds of participants in one level of the independent variable are systematically different* from those in the other level of the independent variable Procedure 1: 20% chem majors Procedure 2: 80% chem majors
suppression effect
When the effect of combining two variables results in something less than simply the sum of the individual variables an effect where two variables combine to produce an outcome that is smaller than what each individual variable contributes a type of interaction effect
Social desirability
When the individual tries to answer questions in the most socially acceptable way.
Masked design
When the observer does not know which group received the treatment.
observer bias
When the observer's expectations influence observations.
Justification to waive informed consent:
When the research involves no more than minimal risk to participants. The waiver of informed consent will not adversely affect participants. The research could not be carried out if informed consent was required.
Null effects/null results
When the results of a study show that the independent variable made no significant change to the dependent variable.
Ceiling effects
When there is a problem in the experimental design where the independent variable groups score similar to the dependent variable, such that the scores fall to the higher end of the distribution.
Multiple Regression
When using two or more independent variables to predict a single dependent variable. All together; combined effect, which one will stand out more?
Negative Correlation
When variables vary in *opposite direction*?
Positive Correlation
When variables vary in the *same direction* it is called what?
C
Which of the following is true of instrumentation threats? They are the same as testing threats. They occur only when using mechanical instruments (e.g., blood pressure cuffs, scales). They can be avoided with counterbalancing. They are only problematic in observational research.
B
Which of the following studies would NOT have a possible threat of observer bias? A study looking at the effect of eating fruits and vegetables and general physical fitness A study looking at the effect of tutoring on SAT scores A study looking at whether increasing the number of hours listening to music increases singing ability A study looking at the effect of increasing the rate of text messaging in a day on writing/English composition skil
D
Which of the following threats to internal validity can apply even when a control group is used? Attrition History threats Instrumentation threats Demand characteristics
- Participant - Researcher
Who are "blind" in a *double blind study*?
Tuskegee Study (1932-1972)
Withheld penicillin from participants with syphilis even after it became the standard treatment - and without their knowledge
Are Observations better than self-reports?
Without training, observers may often exhibit bias. • However, self-reports are, in some sense, just reports from untrained observers. - Poor operationalization can also increase the likelihood of bias for both self-report measures and trained observers. • If there is no clear operational definition for the behavior that is being rated, reliability may be low. If reliability is low, we can't have good construct validity.
Double-barreled questions
Wording that occurs when two ideas are combined into one question and it is unclear whether the answer is for the combination of both or one or the other question
can you be reliable without being valid?
YES
Nuremberg Trials
______ _______ led to the need for ethical controls in scientific research
Applied
_______ research is a type of applied research that involves practitioners in the research design and then the implementation and evaluation of a resulting policy or programmatic change. Its purpose is to improve some aspect of practice (e.g. combating gang violence). In action research, a problem is diagnosed; an intervention or strategy is devised, and data is collected to fine tune the intervention strategy. This type of research is an integral part of the problem-oriented policing philosophy.
_____________________________________
____________________________________________
normal curve
a bell-shaped curve described by a certain mathematical function
nested case-control study
a case-control study nested within a cohort study
Within group
a change in the dependent variable in a group is measured across time: - Crossover - Latin Square - Splitplot
histogram
a frequency distribution in which the frequencies are represented bu contiguous bars
Scatterplots
a graphed cluster of dots, each of which represents the values of two variables. The slope of the points suggests the direction of the relationship between the two variables. The amount of scatter suggests the strength of the correlation (little scatter, close to line of best fit, indicates high correlation). x-axis= Test 1, y-axis= Test 2
control group
a level of an independent variable that is intended to represent "no treatment" or a neutral condition
statistics
a logic and methodology for the measurement of uncertainty and for an examination of the consequences of that uncertainty in the planning and interpretation of experimentation and observation
practice effects
a long sequence might lead participants to get better at the task, or to get tired or bored toward the end over time get better (e.g., IAT)
measures of variability
a measure of the degree of difference between scores in a distribution: range, variance, standard deviation
Journal
a periodical that publishes technical and scholarly material, usually in a narrowly defined area of inquiry.
societal perspective
a perspective that takes account of all costs and benefits, no matter to whom they accrue time/burden of caregiver, transportation, etc.
Experimenter Bias
a phenomenon that occurs when a researcher's expectations or preferences about the outcome of a study influence the results obtained
empirical questions
a question that can be answered by making objective observations
Curvilinear relationship
a relationship in which changes in the values of the first variable are accompanied by both increases and decreases in the values of another variable
convenience sample
a sample collected from subjects that are available
central tendency
a single number that best represents a group/condition
A-B design
a single-subject design in which researchers takes a baseline measurement and then introduce the intervention or experimental treatment and measure the same variable again
null hypothesis
a statement or idea that can be falsified, or proved wrong no difference/effect
Mixed Design Analysis of Variance (mixed design ANOVA)
a statistical analysis that tests for differences between two or more categorical independent variables, while one is a between-subjects variable and another is a within-subjects variable
meta-analysis
a statistical strategy for comparing and integrating the results from multiple studies on the same research question to summarize what they say collectively on that topic
Two-Way Analysis of Variance (ANOVA)
a statistical test that allows us to simultaneously test how two separate nominal or categorical independent variables or (factors) influence the dependent variable, and how those independent variables interact to influence the dependent variables. AKA two way ANOVA or factorial ANOVA
pilot study
a study completed before (or sometimes after) the study of primary interest, usually to test the effectiveness or characteristics of the manipulations
factorial design
a study in which there are two or more independent variables, levels, or factors allow a researcher to simultaneously examine the impact of 2 or more independent variables on the dependent variable. an experimental design that has more than one independent variable 2X2 factorial design
Internal reliability
a study participant gives a consistent pattern of answers, no matter how the researcher has phrased the question do the different questions/parts of the measure produce similar results?
response sets (nondifferentiation)
a type of shortcut respondents can take when answering survey questions. Rather than reporting sincere, thoughtful, and question-specific responses, a person may develop a pattern for responding
not a widely accepted form of randomization
adaptive
period for information bias
after enrollment and group assignment
Acquiescence (Yea-saying)
agreeing or answering in a positive manner for all questions. The opposite strategy is sometime called nay-saying. - Try reverse-wording some questions. be careful about double negatives
Yay-Saying (and Nay-Saying--Response Sets + Solution)
agreeing with everything Solution: Reverse-word questions
cross-sectional study
all individuals of a sample at the same point in time
group or cluster randomization
all members of a group/cluster are randomized together
constants
all of the other variables that could possibly affect your experiment that must be kept the same in every trial in order to keep your experiment valid
Common SPSS Commands: Compute Function
allows creation of new variable from previous existing variables through arithmetic operations.
Third Variable (z)
alternative explanation for the relationship between x and y
waiting-list control group
an empty control group often used in clinical research; participants in this group do not receive treatment or intervention until after the completion of the study
Measurement error
an error that occurs when there is a difference between the information desired by the researcher and the information provided by the measurement process
Independent groups design
an experimental design in which different groups of participants are exposed to different levels of the independent variable, such that each participant experiences only one level of the independent variable
quantitative technology
application of mathematical techniques to describe an analysis of data
Positive Correlations
as A increases, B increases
intrinsic sources of variation
biologically real; true genetic and phenotypic variation among subjects
Theories
cannot be proven and lead to hypothesis
power
capacity to detect a difference between study groups
Pearson's r
coefficient is represented by r coefficient that represents correlation coefficient range from -1 to 1+
ECHO is used to evaluate _____
cohort studies
baseline measurement
collected prior to randomization
case-control study
compares a group of people with a disease to a group of people without a disease
chi-square (x^2)
compares observed frequencies to expected frequencies. Has to be nominal dichotomous data only. How to read: The higher the value, the more statistically significant difference exists. If you have cells with less than 5, look at Fisher test. If not, use Pearson. ex. There is a statistically significant difference in the relative frequency of individuals from different tracks and eating one side of Twix first.
Residual confounding
confounding that remains even after many confounding variables have been controlled
design confounds
confounds caused by a problem in the design of the independent variable.
Ethical Issues in Research Analysis: p-hacking
continuing to test data and post hoc analysis to produce significant results. Altering p value found in analysis report. ex. analysis gives alpha .06 and researcher reports .05.
fatigue effect
decline in performance over time as a participant gets tired or bored.
Program Evaluation
demonstrate the value or worth of an intervention or program within a specific context. Provide basis for making judgements about the value of a program, focused on describing elements, outcomes and value of programs. Performed as a team effort.
examples of derived statistics
density, percentages, adjusted scores, rates, ratios, any summary statistic (averages, variation)
criterion variable
dependent variable
summary statistic
derived numerical value that summarizes a data set (central location, stats expressing variation
4 goals of psychological research
describe, explain, predict, control
A case study
describes a survey conducted on college students.
Descriptive research
describes social trends and phenomena.
Levene's Test for Equality of Variances
determines if the two conditions have about same or different amounts of variability. See Notes for How to Read SPSS Output
Attributes of Scientific inquiry
determinism (events have causes), discoverability, objectivity (answer can be found, reputable and replicable), data driven, tentative conclusions, answerable questions, falsifiable theories, scientists are creative "skeptical optimists"
Program Accountability Model
developed to be more responsive to counselor's immediate need for information on effectiveness of program, to specify focus and determine modifications needed. Cyclical pattern: - Needs assessment, strategic program planning (action plan), program implementation, program assessment, program feedback.
range
difference between the highest and lowest scores in a distribution
ordinal interaction
differences are in same direction, but differ in magnitude when one group's predicted means is always greater than another group's predicted means. For example, the predicted male means are always greater than predicted female means, yet the differences between males and females varies by SES, therefore an ordinal interaction results
conditions
different levels of an independent variable in an experiment.
Test-retest reliability
do you get similar results if you administer the test more than once?
intention-to-treat analysis
dropouts analyzed as if they completed the study
Random Assignment
each participant is equally likely to be assigned to any level of the independent variable - *how to avoid selection confounds*
Between-Subjects Design
each participant only experiences 1 level of the IV (GROUPS) con: requires more people to test
Question order
earlier questions can change the way respondents understand and answer later questions - Preceding questions can frame subsequent interpretations and encourage particular response patterns on later questions. -Imagine that I asked a series of questions relating to oil or natural gas scarcity and then I asked how people felt about a new drilling project. • Do you think I would obtain different results if I asked a series of questions about damage to various ecosystems, and then I asked how people felt about the new drilling project?
ECHO
economic, clinical, humanistic, outcomes
example of between groups design
effect of appetitive stimuli on choice of impulse purchases One group given stimuli then choice, one group given impulse choice first
Between Group
effect of intervention on outcome is based on differences between groups: - Post-test - Pre-test/post-test - Solomon four group - Factorial design
case-control strengths (6)
efficient for rare diseases, relatively easy, fewer subjects needed, reduced costs, may study several risk factors for one disease, can investigate chronic diseases with long latency periods
single-blind study
either the participant, or the experimenter doesn't know which condition participants are assigned to.
comparison group
enables us to compare what would happen both with and without the thing we are interested in
stratified randomization
equal subgroups based on certain factors
observational studies are more _____ than RCTs
ethical
complete counterbalancing
everyone gets a different order within participants ABBA between AB for some, BA for others
Panel Studies
exact same individuals surveyed at each time of data collection.
placebo effect
experimental results caused by expectations alone; any effect on behavior caused by the administration of an inert substance or condition, which the recipient assumes is an active agent.
design confound
experimenter makes a mistake in designing the IV in which a second variable varies systematically with the intended IV poor internal validity cannot support causal claim
Extreme values are, by definition, unusual. Therefore, the next measurement will most likely be less extreme (closer to the mean).
explain the concept of regression to the mean
Ethical Issues in Research Analysis: Data Dredging
exploring data post hoc for significant findings in the absence of significant results for the stated hypothesis. Playing with data looking for significance after analysis.
withing subjects
expose participants to all levels of treatment - randomly assign a sequence of treatment conditions - + power -internal and external validity
bar graph
graphical representation of categorical data in which the heights of separated bars, or columns, show the various relationships between variable
frequency distribution
graphs that shows how many scores falls into particular bins, or divisions of the variable
population-based case-control
has a defined base population
in retrospective studies, outcomes have/have not occurred
have
in prospective studies, outcomes have/have not occurred
have not
Multiple Regression Analyses
help reduce the third-variable problem
Using a within-groups design (top), or a matched-pairs design (bottom) can help. A larger sample also helps.
how to account for individual differences:
theory to data
hypothesis deduced: studies designed and completed, outcomes provide or far to provide inductive support, falsification is possible
Ethical Issues in Research Analysis: HARKing
hypothesizing after results are known. Statistical analysis abuses often presented (ex. using a one-tailed rather than a two-tailed), type I errors more common.
Independent variable: bowl size Conditions: medium and large.
in the eating pasta experiment, what were the independent and condition variables?
Factorial within-groups
independent variables are manipulated as within-groups. - example: 2x2 study, there is only one group of participants experiencing all levels/conditions
accuracy
infer the population parameters with as little uncertainty as possible; unbiased
I-Data
informant report
6 ways of knowing about the world
intuition, tenacity/tradition, authority, reason/rationalism, empiricism, science
major types of bias (5)
investigator, selection, performance, attrition, detection
Assessment Confound (design confound)
involves *assessments that are different for one level* of an independent variable versus another Procedure 1: take an AP chem test Procedure 2: recall anything remembered
Procedure Confound (design confound)
involves *participants doing things differently in one level* of an independent variable versus another Procedure 1: study for 30 min Procedure 2: study for 1 hr
Exploratory research
is often necessary in order to learn about social trends and phenomena that we know very little about.
hypothesis testing
make and test an educated guess about a problem/solution
independent variable
manipulate or shift controlled by experimenter Must have two or more "levels" or conditions
Straightforward manipulation
manipulating an independent variable with relative simplicity by presenting written, verbal, or visual material to the participants.
Experiments can support causal claims
manipulating the causal variable means potentially controlling for alternative explanations - use random assignment to ensure that the different groups are as similar as possible
Identify main effects from _______ _____ (compute the average for each level of the IV of interest, while averaging over the levels of the other IV).
marginal means
construct validity
matches up with theoretical constructs
comparison group
maturation threat isn't a problem if you have:
if left skewed...
mean < median
If right skewed..
mean > median
measure of central tendencies
mean, median, mode
Post-test only
measure dependent variable after the manipulation
Concurrent validity
measure is administered and behavior is measured at the same time. the degree to which the measures gathered from one tool agree with the measures gathered from other assessment techniques
Predictive validity
measure is administered weeks, months, or years before behavior is measured predictive validity is the extent to which a score on a scale or test predicts scores on some criterion measure. Do the results accurately predict a future outcome?
biomarkers
measure of disease progression
semi-interquartile range
measure of variability defined as half the interquartile range
continuous variable
measurements made on a scale that can be infinitely subdivided
effectiveness
measures how well an intervention performs in a clinical setting (real world)
physical trace observation
method examines evidence left behind ex: foot prints, fossils, going through people's trash
archival research
method of research using past records or data sets to answer various research questions, or to search for interesting patterns or relationships ex: prostitution in Gutaemala
A variable which changes the relationship between an independent variable and a dependent variable.
moderator
Mixed Method Design Type: Explanatory
multiphase (usually two) design in which qualitative data helps to explain initial quantitative data. Quantitative data are collected and analyzed prior to the collection and analysis of qualitative data. The quantitative phase allows the researcher to discover variables relationship with one another, the data is then used to inform/develop questions for qualitative piece.
similar to stable-baseline, but with multiple interventions staggered across time.
multiple-baseline design Example: Interventions throughout the school year to reduce several problem behaviors in a developmentally disabled student.
non-inferiority trial
new treatment not worse than active control
incident cases
newly diagnosed
zero correlation
no relationship between variables
unsystematic variability
noise- random fluctuation in data, that can't pinpoint a cause
Chi-square
nominal or ordinal data addresses whether independent variables differ in their relative distributions on the dependent variable ex: whether the outcome is contingent on group membership
Mode is used when:
nominal scales and discrete variables
4 scales of measurement
nominal, ordinal, interval, ratio
If confidence interval crosses 1, what happens to the odds ratio?
nonsignificant
case-control weaknesses (6)
not representative of whole population, incidence/prevalence unknown, inefficient when exposure is rare, selection bias, only one outcome at a time, information bias
*What are the manuscript settings used in APA?*
o *Double-spaced* o One-inch margins o Left-aligned without hyphenation o Indent the first line of each paragraph o *Typeface: 12-point Times New Roman*
Casual Observation
o Behaviour of people or animals may generate a general research topic o Example: Going to the mall and observing people's behaviour
What are the guidelines for avoiding biased language in APA writing?
o Describe people with a level of specificity that is accurate o Be sensitive to labels: call people what they prefer to be called o Acknowledge people's participation in your study
How do you know that your literature search is complete?
o Feel comfortable with your knowledge about the topic area o Have found a few recent research studies that are particularly relevant to your own interests
What are the main jobs you must complete and consider when conducting a literature review?
o Finding, reading, summarizing published research relevant to your question o Literature can help you form your research question, or determine if it has been solved o Help evaluate interestingness of research idea and how it fits into current literature o Help you with ideas on how to conduct your study
The Stanford Prison Experiment
o How being placed in different roles affected by individuals acted o Participants were either given the role of prisoner or guard o Each set of participants were given corresponding outfits and the guards were told to do whatever they had to do in order to maintain order o Study was called off after 6 days as the conditions increased to highly o Prisoners were found to become submissive and guards were found to become aggressive and abusive
Little Albert
o John Watson and Classical Conditioning o Pairing an unconditioned stimulus with a conditioned stimulus to elicit a conditioned response o Albert was conditioned to fear fuzzy white rat o Albert was never unconditioned o It is unknown if Albert's mother gave informed consent
Name some well known conferences
o OBACPN(Ontario Behaviour Analytic Community of Practice Network, divided into regions) o ONTABA (Ontario Association of Behaviour Analysis) o OADD (Ontario Association of Developmental Disabilities) o ABAI (Applied Behaviour Analysis International) o APBA (Associative Professional Behaviour Analysis) o Persuasive Technology
Objective
o Outcome is not skewed by bias
Pass 2
o Pay closer attention to the beginning and ending of each major section o Look up your highlighted words from the previous pass o Try to answer any questions you may have written down
Pass 1
o Read through the framework of the paper This includes the abstract, introduction, conclusion, section headings, and any tables of diagrams including their captions o Underline any words or phrases that are unfamiliar to you o Ask yourself questions What is the point or thesis of this paper? What are the main arguments of the paper? Why is this paper important? How does it contribute to my field of study? o Write down any other questions you may have o Try to summarize the paper in a sentence of two in your own words
The Monster Study
o Studied how positive and negative feedback affected the way children learned language o Feedback was provided to children from an orphanage regarding speech stutters o Children in group 1 were told they did not have a speech impairment and were told they should ignore anyone who criticized the way they spoke o Children in group 2 were told they did have a speech impairment and were told they should never speak unless they could do it right o Feedback impacted the self-esteem of the children
Milgram Experiment
o Studied the psychology of obedience and how subjects would react if researchers encouraged them to engage in activity that went against their morals o Three roles: The experimenter, The teacher, and The learner o Teacher and learner were in separate rooms o Teacher gave the learner word pairings and if they responded with an incorrect answer the teacher was instructed by the experimenter to deliver an electric shock to the learner who was acting and not actually receiving the shocks o It was found that when instructed the majority of participants delivered the highest shock possible and when the learner pretended to have passed out the teachers believed that they may have killed the learner
Pass 3
o Tie it all together so that you have a good idea of what the paper is trying to say at this point o With a critical eye this is the time for reflection and analysis o Take notes as you read looking at arguments, evidence, and conclusions o Ask yourself questions Did the authors do what they set out to do? Are the methods they used sound? Are their methods fluid and logical? What assumptions did they make?
In which order should the manuscript elements appear in APA?
o Title page o Abstract o Text (body) o References o Tables o Figures o Appendices (if any)
What are three things that an APA style report describes?
o Why the research was done o How the research was done o The results it produced
Laboratory Observation
observe and record behavior in a controlled environment - More control over who you observe and what/how often conditions
Naturalistic Observation
observe and record behavior in its natural environment without intrusion - Get to see behavior "in the wild". - Less likely to change behavior.
participant observation
observer becomes part of the group being observed ex: Doomsday call
Reductionism
occurs when a researcher makes a prediction about how a group might behave based on data collected at the individual level of analysis.
Ecological fallacy
occurs when a researcher makes a prediction about how an individual might behave based on data collected at the group level of analysis.
systematic error
occurs when sample statistics are routinely unrepresentative of population parameters and is usually due to a faulty sampling procedure
interaction
one independent variable's influence on the dependent variable changes, depending on the level of other of the other independent variable
Mixed Method Design Type: Embedded
one type of data is in a lead role with other data supporting. Single phase. Used when researchers need to answer questions from a different perspective and their research has primarily focused on one type of data. Often insert a qualitative component into a quantitative design. ex. survey with open-ended questions.
per-protocol analysis
only those who completed the study exactly are analyzed
interval scales of measurement
order plus equal intervals, intelligence scales & personality scales
margin means
overall means - the means for two groups or cells Main effects of the two IV (pick up lines, scent) the average of all participants on one level of the independent variable ignoring the other independent variable
Good theories have...
parsimony (simpler explanation), productivity, and falsification
regression
predicting the value of one variable from another based on their correlation
prospective timeline
present to future
odds definition
probability of one outcome divided by the probability of another
Matching
process of selecting individuals to represent treatment and comparison groups based on specific similar criteria or characteristics. Similar traits evenly matched. People.
strongest type of observational design
prospective cohort
Mixed Methods
provides more robust data, brings out unique variances and ideas and can help reduce bias. Must be proficient at both types of research design (qualitative and quantitative). Research question order determines the priority of the methods used, must relate question to type of design, and ensure questions are specific and measurable relative to type of design employed.
Double-Barreled Questions
questions that *attempt to get at multiple issues at once*, and tend to receive incomplete or confusing answers
simple randomization
random number generator
Interquartile range
range from the 25th percentile to the 75th percentile
measures of variability
range, variance, standard deviation, IQR
Ordinal (scale of measurement)
refers to categories that are ordered but not necessarily equidistant: very, somewhat, not at all strongly agree, agree somewhat, disagree somewhat, disagree strongly
Interval (scale of measurement)
refers to equidistant categories number of days, age in years
Nominal (scale of measurement)
refers to non-ordered categories (no order of importance. examples: male/female, racial group)
sampling variation
refers to normal variation (dispersion) in statistics measured on repeated samples from a population
sample bias
refers to systematic error in measurements
The principle that extreme observations tend to be followed by less extreme observations.
regression to the mean
Type 1 error and cumulative type 1 error
rejecting the null hypothesis when the null hypothesis is actually true false-positive the probability of this happening is = to alpha
empirical science
relying, or based on, direct observation, experiment of experience
Ways to Handle Missing Data: Imputation
replace missing values with substitutes such as mean value. Put something in its place. If item is nominal or categorical this option is not used.
discrete variable
represent discrete categories or counted whole numbers
Evidence Based Practice Model
represents practitioners' intent to utilize the best available research evidence to inform treatment planning and practice. Includes: Program description Utilize outcome research Intervention evaluation
minimal risk; nine
research must represent ________ _______ and fall into one of _______ categories in order to be expedited.
STROBE checklist (7)
research question, study design, study population, exposure and outcome, data analysis and confounding, presentation of results and interpretation, practice and policy implications
Manipulated Variables
researcher sets up conditions to produce it.
Experiment
researchers manipulated at least one variable and measured another
Alternative Hypothesis (H1)
researchers prediction but cannot be absolutely proven
Purposeful Sampling
sampling specifically chosen population based on certain element that will highlight and inform a particular aspect of the study. Gives you useful data when you use this method of sampling for mixed methods.
discriminant validity
scores on the measure are not related to other measures that are theoretically different. Should correlate less strongly with measures of different constructs
Convergent validity
scores on the measure are related to other measures of the same construct. Should correlate more strongly with measures of the same/similar constructs
Correlational Research (what makes it correlational)
seeks to figure out if two or more variables are related and, if so, in what way *both variables are measured*
factors that threaten external validity (5)
selection, pretesting, research setting, history, multiple treatments
3 types of operationalizations
self-report, observational, physiological
types of randomization
simple, block, stratified, adaptive
Mixed Methods: Single Phase
simultaneously collect both qualitative and quantitative data. One set of questions does not rely on the others. ex. survey with open-ended questions
carryover effects
some form of contamination carries over from one condition to the next how much does one influence on the next (very similar to priming) (e.g., survey order)
Single-blind procedure
special type of within-subjects design using one participant (human or animal) or perhaps one group to assess changes within that individual or group participants are unaware of, or blind to, the type of treatment they are receiving, but the administrator knows
inclusion criteria
specific characteristics the investigator is interested in studying
2-way interactions: what they look like ("a difference in differences")
spreading interaction, crossing interaction
a study where the researcher observes behavior for an extended period before and after a treatment.
stable-baseline design Example: evaluating the effectiveness of a memory improvement strategy in an Alzheimer's patient.
correlational coefficient
statistic that indicates the strength of the relationship between two variables
inferential statistics
statistics that help us to draw conclusions about populations
observable confounders (2)
stratification, statistical adjustment
helps to minimize selection bias
strict adherence to randomization schedule
informed consent
subject is presented with information about the study and willingly volunteers to participate
parallel RCT
subject randomized to treatment or control only
crossover RCT
subject receives treatment and control at different times
double blinding
subjects and investigators do not know randomization schedule
block randomization
subjects divided into blocks which are then randomized
single blinding
subjects do not know treatment group
laboratory observation
subjects watched in the laboratory ex: bobo doll experiment
sum of squares
sum of squared deviations from the mean
descriptive statistics
summarize data
Cohort Studies
survey representatives of a group of individuals over time to follow a group's progress. Sampling of each, same group over time. Ex. Research Class
Counterbalancing
systematically ordering treatments or measurements within a protocol to eliminate effect of order in which they are presented to participants. Each group switch out order to rotate the intervention. Intervention.
advantages of prospective cohort studies (4)
tailoring of study to research questions, purposefully capture data on important risk factors, using reliable and validated techniques to assess exposure and outcome measures, measuring exposure before disease occurs
good subject tendency
tendency of experimental participants to act according to what they think the experimenter wants
What are the two types of reliability
test-retest and interrater
mean
the common average
Interrater reliability
the degree to which different observers agree on their observations. Two or more "independent" observers, looking at the exact same thing
internal validity
the degree to which the effects observed in an experiment are due to the intervention
Criterion Validity
the extent to which a measure is related to an outcome Does the measure allow you to distinguish people on the basis of a behavioral outcome? Two types: -Concurrent validity -Predictive validity
Construct validity
the extent to which variables measure what they are supposed to measure Is the measure ACTUALLY measuring what the researcher intends to be measuring?
relative frequency
the fraction or percent of the time that an event occurs in an experiment (frequency/total number of measurements)
alternative hypothesis
the hypothesis that a proposed result is true for the population researcher predicts its going to happened
Familiarity with questionnaire
the interviewer should be very familiar with the overall format of the survey and every item.—
animal rights
the notion that animals have the same sort of rights as people, including legal rights; not generally accepted
double-blind procedure
the participants and administrators of treatment are unaware, or blind, to the types of treatment they are administering to reduce the influence of expectancies they know the condition might have in order to reduce the likelihood that expectancies or knowledge of condition will influence the results, both the participants and the administrators of treatment are unaware of, or blind to, the types of treatment being provided
Power
the probability of rejecting a false null hypothesis the probability of finding statistical significance when the result is significant
Confidence interval
the range of values within which a population value is likely to fall, estimated at a certain level of probability
Null hypothesis
the relationship between or among the study variables is not as hypothesized by the investigator an idea that competes with the working hypothesis
standard deviation
the square root of the variance
independent variable
the variable that you have chosen to manipulate in the experiment in which you are setting up
dependent variable
the variable you will measure in your experiment, that will change in accordance to the independent variable
constructs
theoretical concepts formulated to serve as casual or descriptive explanations: Milgram's construct was obedience
Computer-assisted telephone interviewing (CATI)
this technology provides the script the telephone interviewer is to follow. It also allows the telephone interviewer to record the responses into a database as they are given.
IRB mission
to protect all human subjects from physical and psychological harm caused by experimental research
number
to remove bias and to protect the identity of the participant give them a _________ to be recognized by.
operationalize
to turn a concept of interest into a measured or manipulated variable
authority
trusting those who are "above" you
the commitment to expand knowledge and the cost to research participants
two values that create conflict in the decision to conduct research
Common SPSS Commands: Recode Variable
used for collapsing or merging variables and for reverse coding.
interim analysis
used in cases of safety or ethical concerns
Measure of central tendency
used to represent a "typical" result or value around which results cluster
mortality rate
usually annual number of deaths in a year, divided by the population in that year
Perception-Based Needs Assessment
utilize surveys to identify the subjective needs of various client populations.
Evaluation research
utilizes a wide range of methods to assess the efficiency or effectiveness of a program, policy, or process
True Experimental Design
utilizes both random assignment and comparison group
Levels
values that a variable can take on.
Conceptual Variable
variable of interest stated in agreed upon terminology. abstract concepts such as "shyness" or "intelligence" Also called a construct, conceptual definition
Dependent Variable
variable that reflects the impact of the manipulated variable
extrinsic source of variation
variation caused by imprecision of measurement (human error) or sampling variation (not biologically real)
Propriety
was evaluation conducted in an ethically appropriate manner.
systematic variability
way to remove a confound in statistical analysis
can support an association (but NOT a causal claim)
we haven't measured every variable, haven't accounted for 3rd variable
regression threat
when difference between pretest and posttest is caused by regression to the mean.
crossover interaction
when the influence of one independent variable on the other reverses across levels of the other variable
selection effects
when the kinds of participants in one level of the independent variable are systematically different from those in the other
instrument threat
when the measuring instrument changes over time. Example: When teachers become more lenient (or stricter) with their grading over time.
selection effect
when the traits of participants influence which condition they are assigned to, creating a confound.
Factorial independent-groups
where both independent variables are studied as independent-groups - example: 2x2 study, four different groups of participants
Negative Correlation
x and y changes in *opposite* direction - more you exercise and less you weigh, more income and less likely to get diseases
Positive Correlation
x and y changes in same direction - more shy you are, the better you are at reading facial expressions
Mixed-mode surveys
—combining methods of conducting survey research, such as interview or mail, with online based survey methods has shown promise as a means of getting high response rates.
Electronic instrument design
—effective online surveys do not mirror a questionnaire exactly, but instead make specific considerations for question arrangement and layout.
Coordination and control
—interviewers should be trained before they begin collecting data to identify any problems they might have administering the questionnaire. It is also important to monitor the interviewers during the course of the survey.
Follow question wording exactly
—it is important that the interviewer not deviate from the wording of the questions, which could bias the results.
Probing for responses
—oftentimes the interviewer will ask the respondent to give more information, which is common for open-ended questions.
Response rates in interview surveys
—response rates in telephone interviews have declined as respondent are increasingly likely to screen their calls.
Record responses exactly
—similarly, the responses should be recorded exactly as they are given.
Appearance and demeanor
—the interviewer should dress similar to the respondents and be polite.
Improving response rates
—with rates comparable to mail surveys, online surveys benefit due to their lower cost. Research is still ongoing as to what is best to increase response rates, but email reminders and setting deadlines has shown promise.
Field Research Concerns
◦ Selecting a site and gaining access to it, ◦ Entering the field, ◦ Learning the ropes, and ◦ Developing rapport with members in the field.
construct validity
the extent to which variables measure what they are supposed to measure
How do you increase your reliability?
1. Standardize the administration of the measure 2. Clarify instructions and questions 3. Train observers 4. Minimize coding errors
Guide Lines for asking Questions
B. Make items clear—the respondent should be able to easily understand the items on the survey. C. Avoid double-barreled questions—this refers to questions with multiple components, and thus, it is impossible to determine which part the respondent is answering. D. Respondents must be competent to answer—the respondent should have enough information to answer the question accurately. E. Respondents must be willing to answer—oftentimes respondents are reticent to respond to questions, particularly on sensitive topics. F. Questions should be relevant—the items on the survey should be meaningful to the respondents. G. Short items are best—the longer and more complicated the item, the more difficultly the respondent has in answering the question. H. Avoid negative items—respondents often overlook negative items in a question or statement, and give an inaccurate response. I. Avoid biased items and terms—certain terms and phrases encourage certain responses; the items should employ neutral language.
Open-ended and closed-ended questions
Closed ended questions include all possible responses to the question, while open-ended questions place no constraints on the respondent's answer; they are free to respond however they choose.
Placebos
Inactive substances, not real medications, that are used to test the effectiveness of drugs.
Empiricism
Objective observation that is independent of prejudices
Questions and statements
Questions elicit specific information from a respondent; statements ask a respondent their reaction to a particular claim.
"Ethic Principles of Psychologists and Code of Conduct"
The APA has developed an extensive document known as the "_____________" in an effort to keep pace with the challenging changes of scientific ethics.
scientists
The Scientific Method is used by ________________ to find out about the natural world.
General questionnaire format
The design of the questionnaire should be straightforward and uncomplicated. Items should be clearly distinguished from one another so they are not missed.
Attrition threat
When the number of participants decreases from pretest to posttest (participants drop out of the study being conducted). In order to avoid this, researchers should remove the pretest scores of the subjects who did not complete the posttest- or at least not include the scores for the pretest average.
make causal claims
What is the main reason for conducting experiments? a) to have a more valid study b) to generalize to more people c) to make causal claims d) to collect evidence in the real world
The Scientific Method
__________________________ uses process skills and follows a logical, organized, empirical, yet flexible approach to problem solving.
The Scientific Method
______________________________ is a systematic way of testing an idea, hypothesis, prediction or inference.
Applied
_________research is intended to collect the information necessary to solve a specific problem. Researchers, like practitioners, consultants, and some academics, conduct research to address and solve problems. This research is often done for a client, who intends to use the knowledge gained from the research to address a specific issue. Typically, this research is published in reports consumed internally within organizations. Although applied researchers often produce knowledge, their primary intent is to address a specific issue or solve a current problem. We will discuss applied research in more detail in a later chapter.
The coding guide
a record that specifies the variables of a study, the columns they occupy in the date file, and their possible values
Open-ended questions
allow any sort of response - Example: "What is your favorite style of music?" (without any response options provided) • Allows for rich information and unanticipated responses • Requires careful analysis and effortful (time-intensive) coding of information
Forced-choice formats
limit type of responses • Requires selecting from a pre-specified set of responses Example: • Narcissistic Personality Inventory (Raskin & Terry, 1988): Two opposing statements are provided and the respondent is asked to choose one. _____I really like to be the center of attention. _____It makes me uncomfortable to be the center of attention.
dependent variable
measured, result outcome variable take the form of records of behavior or attitudes, such as self-reports, behavioral observations, or physiological measures
mode
most frequently occurring score
controls
must represent general population
lab notebook
notebook for recording information important to a study, such as design, procedure, and the planned analysis
