Research Methods & Statistics Midterm #3
Degrees of freedom
# of values free to vary when a statistic is used to estimate a parameter So, for testing the mean of one sample with unknown population standard deviation, df=n-1
Chi-Square Test of Independence statistic
(row total)(column total)/N
Cohen's (1988) rules of thumb
- A "small" effect produces an n^2 of . 01-.05 - A "medium" effect produces an n^2 of . 06-.14 - A "large" effect produces an n^2 of .15 or greater
B.E.A.N.
- Beta Error (Power = 1 -Beta Error): Beta error (or Type II error) is the probability that a test of statistical significance will fail to reject the null hypothesis when it is false (e.g., when there really is an effect of training). - Effect Size: The effect size is the magnitude of the difference between the actual population mean and the null hypothesized mean (μ1 -μ0) relative to standard deviation of scores (σ). - Alpha error: Alpha error (or Type I error) is the probability that a statistical test will produce a statistically significant finding when the null hypothesis is true (e.g., there is no effect of training). - Sample Size: As the sample size increases, the variability of sample means decreases.
The hypothesis test is always made of the null hypothesis
- Essentially, we assume the null is true - Then, examine the data to see how reasonable or likely (probability)that the null hypothesis is true, given the evidence provided by the sample data
Characteristics of the F-distribution
- Non-normal - Positively skewed • Ratio of variances, which are always positive
Assumptions of the Chi-Square distribution
-Independence of observations•No observation can be in more than one category -Minimum expected frequencies•none of the expected frequencies should be equal to zero•most if not all of the expected frequencies should be greater than 5
Two types of standard error of the mean
-Population standard error of the mean 휎(#•Used when the population standard deviation (휎)is known -Standard error of the mean 푠(#•Used when the population standard deviation (휎)is unknown•Use the standard deviation (푠)to estimate the population standard deviation•Sometimes also called the estimatedstandard error of the mean
True 3 to 1 odds
3 out of 4 times you will lose, you have a 1 in 4 (0.25) chance of winning
True 3 to 1 odds on
3 out of 4 times you will win, you have a 3 in 4 (0.75) chance of winning
Chi-Square Test of Independence measure of effect size
= x^2/N
If A and B are correlated, then ...
A could cause B B could cause A Another variable could cause both A and B
What is a sampling distribution of means and why is it important for our hypothesis testing? (7 points)
A sampling distribution of means is (a) a hypothetical distribution of (b) the means of samples (c) of the same size repeatedly drawn from a population. It is important because it allows us to determine the proportion of samples that would have a particular mean or higher/lower than the sample we have drawn.
1 variable with more than two categories; 1 variable with many levels/ are the means of the categories different?
ANOVA
What does the search for themes in thematic analysis involve?
Actively exploring data. c. b. Organizing codes into a logical system of meaningful themes that help researchers present a coherent picture of 'what is going on' in the data. Translating the codes that emerged in the previous phase into discrete analytic themes. All of these
You are the teaching assistant for the course and are to explain to the class that a 2 X 2 factorial study is, in a sense, two separate studies combined but better. How would you do it?
Alternatively, you could have two separate studies each with one independent variable with two levels each. Conducting two separate studies, though, would each have their own risk of Type 1 error. (2 points) By conducting an analysis of variance we control for overall Type 1 error (set at .05 total) by testing both IVs in the same analysis. (2 points)
the criteria to establish causality
Association - is there an established relationship between welfare benefits and grades (1.5 points) Time order - which comes first - do people have lower grades and then receive welfare, or vice versa (1.5 points) Nonspuriousness - could there be a third variable associated with both welfare benefits and grades that is responsible for the relationship
Proof
Because hypothesis testing is based on probabilities, it cannot provide "proof"•Proof implies absolute certainty•It can provide support (or lack of support) for a hypothesis.
Why is ProbabilityImportant to Researchers?
Because research is typically conducted with samples rather than populations, researchers evaluate data using probability•Sampling error: • The difference between statistics calculated from a sample and those from the population • Samples are imperfect representations of the population
ANOVA degrees of freedom
Between-group: # of groups -1 Within-group: sum (# participants - 1)
Chi-Square Test of Independence degrees of freedom
Calculate the degrees of freedom (df) df= (# rows - 1)(# columns - 1) df = (# rows - 1)(# columns - 1)
Chi-Square Goodness of Fit Test
Calculate the degrees of freedom (df)df= # groups - 1
The Sampling Distribution ofthe Difference: Characteristics
Central Tendency −Mean of the distribution of difference is 0. −We've assumed in the null that there's no difference between the means. •Shape −Approximately normal. −Shape determined by sample size. •Variability −Standard error of the difference
two variables with two or more levels/ are the two categorical variables related
Chi squared test of independence
Chi-square test
Chi-square test: Used for a single categorical variable that has more than two levels
elements of Interpretative Phenomenological Analysis?
Clustering and ordering of themes b. The interview schedule d. The reconciliation of themes
two variables with many levels/ are scores on the two variables related?
Correlation coefficient
Direction of causality
Correlation coefficients say nothing about which variable causes the other to change
Immersion and transcription are involved in which phase of thematic analysis?
Data familiarization
Concerns with unplanned comparisons
Data-driven, not theory-driven • Results might be an artifact of these specific samples and might not appear in another sample.
Characteristics of the Chi squared distribution
Distributions differ depending on df -always positive -Positively skewed distribution
What is effect size, and why is it useful to include a measure of effect size when reporting the results of a statistical analysis such as the t-test?
Effect size is how big or how much of something we are interested in. It can be a measure like Cohen's d, or a mean or difference between means. It gives us info about the size of our results relative to error. Degree to which between groups variance makes up a significant portion of total variance. It is useful to include a measure of effect size because our statistical analyses tell us whether something is significantly different from something else but doesn't tell us about the magnitude of the difference
ANOVA stat
F = between-group variability/ within-group variability F = information/error
Nonparametric Tests
Features: -Not attempting to estimate parameters -No assumption of normality -"distribution free"
One-sample t:
How likely is it that a sample with these characterisics comes from an underlying distribuion with certain characterisics?•Do students (n=17; ) believe that they live longer than the naional average ?
Chi square
How likely is it that paricipants would be distributed across categories as we ind in this sample if they were evenly distributed? Are they distributed evenly?•Are the four personality types (alphas, betas, gammas, deltas) evenly distributed across persons? How likely is it that paricipants would be distributed across the combinaion of these two variables that we ind in this sample if the variables were unrelated?•Is recycling behavior related to shoppers' decisions about paper or plasic bags?
One-way analysis of variance
How likely is it that we would draw N number of samples with these characterisics if they all came from the sameunderlying distribuion with certain characterisics (if the null were true)? •Does the method of learning chess strategy (observaion, predicion, explanaion) afect how well students play chess?
Two-way analysis of variance
How likely is it that we would draw N number of samples with these characterisics on Z variables if they all came from the same underlying distribuion with certain characterisics on Z variables (if the null were true)? •Does the efect of a depression drug depend on how long people have been depressed (short term, long term) and the strength of the dose (high or low)?
Two sample t
How likely is it that we would draw two samples with these characterisics if they came from the same underlying distribuion with certain characterisics (if the null were true)? •Will people take diferent amounts of ime to leave a parking space when another driver is waiing compared to when no other driver is waiing?
Stating a decision rule
If the value of the statistic calculated for the sample lies beyond the critical values, reject the null hypothesis; otherwise, do not reject the null hypothesis.•Example: If the number of wins in 12 Super Bowls is fewer than 3 or greater than 9, reject H0; otherwise do not reject H0.
What is the standard error of the mean (list its component parts) and when do we use it? (7 points)
It is the measure of deviation for the sampling distribution of the mean (5 points). It includes the standard deviation of the population divided by the square root of sample size (4 points). Can use it to calculate value of Z for a sample, determine how many samples would have that score (or higher/lower) from the population.
1 variable with more than two categories; 1 variable with many levels/ do the categories have different central tendencies?
Kruskal-Wallis test
distribution free analogue for one way between subjects anova
Kruskal-Wallis test
Controlling familywise error
Lower the probability of Type I error across the set of comparisons by making it more difficult to reject the null for each individual comparison • Scheffé test Divide alpha level by the number of comparisons (t-tests) • Bonferroni adjustment • Protected alpha
One variable with two categories; one variable with many levels/ do the two categories have different central tendencies
Mann-Whitney test
r^2
Measure of common variance By squaring the value of r you get the proportion of variance in one variable shared by the other indicates the proportion of variance accounted for
Partial correlation
Measures the relationship between two variables, controlling for the effect that a third variable has on them both.
Alternative hypothesis
Proposes that the hypothesized change, difference, or relationship does exist.
null hypothesis
Proposes that the hypothesized change, difference, or relationship does not exist
When researchers display reflexivity what does this involve?
Reflection on the way in which they may contribute to the production of the particular patterns of behavior. Reflection on their own role in the research process.
Region of rejection
Represents the values of a statistic whose combined probability is low enough that we could reject H0
Region of non-rejection
Represents the values whose probability is not low enough to allow us to reject H0
Research hypotheses vs. statistical hypotheses
Research: focuses on verbal expression of concepts Statistical: focuses on numerical expressions of relationships between variables
Planned (a priori) comparisons
Researcher identifies comparisons prior to collecting data
Unplanned (post hoc) comparisons
Researcher makes group comparisons that were not designed into the study
Why is Type I error a concern?
Saying an effect exists when it doesn't communicates false information •Imagine a drug-trial in which there was no improvement in depression, but the researchers claimed there was.
Type I error: alpha
Stating that there is an effect when none exists (accepting an expirimental hypothesis when the null is true) reject a null hypothesis when you shouldn't because the null is true-Found the defendant guilty but he's really innocent
Type II error: Beta
Stating there is not an effect when one exists (failure to reject null hypothesis when it's false) Fail to reject a null hypothesis when you should because the null is false-Found the defendant innocent but he's really guilty
A researcher is interested in the phenomenology of shopping. What does this mean?
That they are interested in studying the phenomenon of shopping as it is experienced by those who do it.
Main Effects and Interactions
The effect of each of the independent variables on the dependent variable is the main effect of that variable The combined effect of two or more independent variables on the dependent variable (i.e., more than just a sum of the main effects) is an interaction Interactions: the effects of one IV on the other IV Main effects: the effect of one IV in isolation
Which of the following statements refer to the Central Limit Theorem?
The expected value of the mean of the distribution of sample means is b. The distribution of sample means is approximately normally distributed When a non-normal population is sampled, the distribution of sample means is still normally distributed, as long as the sample size is more than 20 to 30.
evaluating interactions
The interaction is best seen by graphing the results •The fact that the lines are not parallel suggests an interaction, which is confirmed by the ANOVA
Familywise error
The likelihood of making at least one Type I error across a number of comparisons. Experimentwise error
In which of the following ways is the standard normal distribution different from other normal distributions?
The mean of the distribution. c. The unit of measurement of the distribution. b. The standard deviation of the distribution.
Which of the following is not part of the contextual critique of quantitative approaches?
The objection that there is more to d. psychological phenomena than can be conveyed by mere numbers.
In reviewing themes as part of thematic analysis, what does the principle of meta-contrast relate to?
The question of whether there is sufficient similarity among the data that are illustrative of the same theme
Why does Type I error occur?
These errors occur due to chance factors and fluctuations in sampling
Describe this study design =
This is a 2x3 factorial study (1 point) IV 1: Alcohol condition (some, none) (1/2 point) IV 2: Amount of caffeine (0, 200, 400mg) (1/2 point) DV: Reaction time (in Msec) (1 point)
tandard error of the mean
This is the standard deviation of the sampling distribution of the mean
t-test for one mean
Used when population standard deviation is not known •Use the sample standard deviation (s) to estimate the population standard deviation •Uses the t-distribution
Critical value
Value of a statistic that separates the (region of rejection and region of non-rejection)
What data would convince us that H0 is false
We look for evidence that has a low probability of occurring if H0 is true
Correlation
What is the relaionship between variable X and variable Y? How likely is it that individuals would have these scores on variable X and variable Y if the two variables were unrelated?•Are anxiety and exam performance related?
mann whitney test
a distribution free test that is usually used to compare the central tendency of two independent groups distribution free analogue of a between subjects t-test
Loglinear analysis
a statistical procedure for analyzing data in complex contingency tables
Probability of making a Type I erro
a= .05, so a 5% chance of concluding an effect exists when it doesn't •95% chance of not making a Type I error
One way ANOVA null hypothesis
all means are equal
in a study using a within-subjects design, a likely confounding variable is
b. sequence effects (not regression to the mean)
Statistical odds
betting odds of 3 to 1 produce statistical odds of 0.333, odds of 3 to 1 on are expressed as 3
1 variable with 2 categories; 1 variable with many levels/ are the means of the two categories different?
between subjects t-test
One variable with two levels/ is one level of the variable more common than the other?
binomial test
Probability of making a Type II error
bis difficult to determine, because we frequently don't know true population values
Thematic analysis top-down vs. bottom-up
both are valid
Qualitative data...
can be gathered for research purposes or can exist independently of research. Qualitative researchers can be internal or external to their data. The more structured qualitative data are, the more likely they are to be handled in widely agreed ways after they have been collected.
Longitudinal design: multiple time points
can get association, time ordervery valuable but difficult to collect
One variable with more than two levels/ do the levels occur with the frequencies we would expect?
chi squared test of goodness of fit
CI
confidence interval = mean +or- t(S of mean)
Effect size
d = difference between means/standard deviation
A researcher wants to study bonding behavior in chimpanzees. Unfortunately, the research has no real knowledge about chimpanzees and there is no information about bonding in the literature. Which of the following would be the most appropriate way to begin to study the topic?
d. naturalistic observation
The binomial test
dealing with a single categorical variable with two levels
As the size of a sample (N) increases, the difference between the value of the sample standard deviation and the population standard deviation calculated for the same set of data
decreases
Odds ratios
dividing the larger odds by the smaller odds
Cross sectional: data at single time point
easy for association, difficult for time order can sometimes infer time order
Contingencies
events with uncertain outcomes that may represent potential liabilities events which are the combination of two or more categories of event. For example, the contingency of being a female with an eating disorder is the consequence of being both a female and having an eating disorder.
What components comprise the test statistic formulas?
for example •One Way ANOVA:•Mean Squares Between Groups: Sums of Squares Between and degrees of freedom•Mean Squares Within Groups: Sums of Squares Within and degrees of freedom
distribution free analogue for one way within subjects anova
friedman test
Arnold compares the means of three groups and calculates a value of -2.01 for the F-ratio. This implies
he has made an error in his calculations F distribution is positive and positively skewed
A good research hypothesis
implies that that predicted relationship can be tested empirically.
The third-variable problem
in any correlation, causality between two variables cannot be assumed because there may be other measured or unmeasured variables affecting the results.
two sample t
is sample different from another sample
single sample t test
is sample different from known pop
Covariance
it tells is by how much scores on two variables differ from their respective means
distribution free analogues for within subject t-tests
mcnemar test sign test wilcoxon matched-pairs signed-ranks test
1 variable with many levels/ does the mean differ from some value?
one sample t-test
n^2
percentage of variance associated with between-group differences
Grand mean
position that describes the whole set of scores with least uncertainty
For which of these research situations would you most likely calculate a correlation coefficient
quantitative and quantitative
In the multilevel, completely-randomized, between-subjects design, participants are
randomly assigned to three or more conditions.
marginal totals
row and column totals in a two-way table sum of contingencies in a contingency table
t=
sample statistics - parameter value proposed by null hypothesis
two variables with many levels/ is the central tendency of the two variables different?
sign test
Conditional odds
statements of the likelihood of one event occurring given that another has occurred
Parametric Tests
statistical tests that are designed to be used with data that are determined to be normally shaped and normally distributed Assumptions: -Tests being used to estimate a population parameter -Populations based on normal distributions -Samples are also normally distributed
descriptive uncertainty
statistical uncertainty from variation in observations around a measure or central tendency. Uncertainty in predicting individual scores from a central tendency measure.
inferential uncertainty
statistical undertainty due to random error or chance
Sum of squares
sum of squared deviations from the mean
The standardised version of Covariance
the Correlation coefficient
Power
the ability to detect an effect when it does exist in the population
Variance may be defined as:
the average squared deviation of a score from the mean
The classification of a variable (e.g., independent vs. dependent) depends on
the characteristics of the variable and the nature of the study.
For ethical reasons, researchers should compare a new treatment with
the currently available treatment
Sampling distribution of the difference
the distribution of all possible values of the difference between two sample means when an infinite number of pairs of samples of size Nare randomly selected from two populations. -a distribution of statistics for samples randomly drawn from populations. -used to determine the probability of obtaining any particular difference between two sample means.
Error term
the mathematical expression that estimates how much random error there is in the data.
Information term
the mathematical expression that tells us how much evidence there is that the results are not due to chance or random error.
For a one-way analysis of variance, the degrees of freedom BG indicates
the number of levels of the independent variable
Ina double-blind control procedure,
the participants and researcher are blind to assignment of participants.
Experimentwise error
the probability of making at least one Type I error across a set of comparisons • When you conduct multiple unplanned comparisons to determine the source of a significant F-ratio, you are increasing the probability of making at least one Type I error.
Alpha (a)
the probability of the statistic used to make a decision to reject H0
Total sum of squares
the sum of the squared deviations of each score from the grand mean
distribution free
they are free of restrictive assumptions about the shapes of distributions
distribution free vs non parametric
they do not make assumptions about the parameters particular distributions
ANOVA
three or more samples - at least one different?
Effect sizes for ANOVA
when ratio is close to 1, lots of between group variance when ratio is close to 0, lots of within group variance
2 variables with many levels/ are the means of the two variables different
within-subjects t-test
The null hypothesis should be accepted if
you should never accept the null hypothesis
Why does Type II error occur?
•Chance fluctuations •Small effects that are difficult to detect •Research design issues - We are making decisions on samples, not the whole population
Controlling Type I and Type II Error
•Controlling Type I error: Make it more difficult to reject H0.•Lower the value of afrom .05 to .01 •However, a more stringent aalso increases the probability of making a Type II error•The more difficult we make it to reject H0, the more likely we are to miss effects that actually exist. Controlling Type II error: Make it more reasonable to reject H0.•Increase sample size•Larger sample size yields lower critical values•Raise a•However, this increases the probability of Type I error •Controlling Type II error: Make it more reasonable to reject H0. •Use a directional alternative hypothesis (H1)•A directional hypothesis examines statistics in only one tail of the distribution, thereby consolidating ain one region •Increase between-group variability•Maximize the differences between groups by using effective experimental manipulations •Controlling Type II error: Make it more reasonable to reject H0.•Decrease within-group variability•Minimize the error variance within a group by increasing the sample size and standardizing testing procedures
Why is Type II error a concern?
•It may result in a non-communication of correct information •If a researcher believes no effect exists, she may be less likely to attempt to publish the research
Evaluating main effects involves
•Looking at all the people tested under each level a Factor A regardless of the level of Factor B •Doing the same for Factor B, as shown here
Introduction to the tDistribution
•Modality:-The mean of the t-distribution is 0 •Symmetry:-Approximately normal, but changes shape with different sample sizes •Variability:-Standard error of the mean: average deviation of the sample •When we use s to estimate the distribution of test statistics will not be quite normal -Will be somewhat flatter than normal distribution -The degree of "flatness" depends on how large the sample is •If extremely large sample, can get near-perfect estimate •If very small sample, will have more variability, not as good an estimate
Applying Probabilityto Normal Distributions
•Percentages of scores in the normal curve can be re-expressed as probabilities. •For example, •The percentage of scores between z = 0 and z = 2.24 is the same as the probability that a randomly selected will fall between z = 0 and z = 2.24 •This will allow us to test hypotheses about group scores
Factors that Affect the Decision
•Sample size:•Larger samples make it more likely that we can reject H0 •Alpha•The smaller the value of a, the more difficult it is to reject H0. •Directionality of the alternative hypothesis.−Non-directional hypotheses do not specify the direction of change, including a ≠ in the numeric statement.−Directional hypotheses specify the direction of change, including a > or < in the numeric statement. - Directional hypotheses have a region of rejection in only one extreme of the distribution, and are called one-tailed. - Non-directional hypotheses have regions of rejection in both extremes of the distribution, and are called two-tailed.
What is the process (sequence of steps) for null hypothesis testing?
•State the null and alternaive hypotheses •Make a decision about the null hypothesis• Calculate the degrees of freedom• Set alpha, idenify the criical value, and state a decision rule•Calculate the test staisic value•Make a decision whether to reject the null hypothesis or fail to reject the null hypothesis•Determine the level of significance •Draw a conclusion from the analysis •Relate the results to the research hypothesis
The relationship between main effects and interactions
•The presence or absence of a main effect does not tell us whether there is an interaction, or vice versa•They are independent and will be tested independently •The order in which main effects and interactions are interpreted•If interactions exist, this implies that one IV depends on the level of the other IV.•Therefore, the IVs cannot really be examined in isolation•If no interactions exist, the IVs can be examined separately.
z-test for one mean
•Used when population standard deviation is known •Uses the standard normal distribution
what does the confidence interval mean
•We are claiming that if we ran our study 100 times, using 100 different samples of subjects, 95 of the confidence intervals we constructed would include the population mean. •95% confident that our CI contains the population mean, saying that the values in the interval are plausibleas true values for the population mean, and that values outside the interval are relatively implausible -although not impossible.
Multiple F-tests for Two-way ANOVA
•Will have an F-statistic for each hypothesis •One hypothesis for the main effect of each independent variable •One hypothesis for the interaction effect •Make decision about whether to reject the null for each one
Cohen's d
●Effect size is essentially the difference between the means divided by the population standard deviation (calculated different ways for different tests)-Is a standardized, unit-free measurement just like Z ●Suggested effect size interpretationsd=.2 smalld=.5 mediumd=.8 large