Quantitative Methods II Exam 1

null hypothesis

(H0) states that, in the general population, there is no change, no difference, or is no relationship

alternative hypothesis

(H1) states that there is a change, a difference, or there is a relationship in the general population

Directional

(one-way): Predicts there will be an effect in a certain direction

Non-directional

(two-way): Predicts there will be an effect without stating the direction

Confidence intervals

- Alternative technique for describing effect size - Estimates μ from the sample mean (M) - Based on the reasonable assumption that M should be "near" μ - The interval constructed defines "near" based on the estimated standard error of the mean (sM) - Can confidently estimate that μ should be located in the interval

Influences on power Increased Power

- As effect size increases, power also increases - Larger sample sizes produce greater power - Using a one-tailed (directional) test increases power (relative to a two-tailed test)

Structurally similar to the other t statistics

- Essentially the same as the single-sample t - Based on difference scores (D) rather than raw scores (X)

Influences on power Decreased Power

- Reducing the alpha level (making the test more stringent) reduces power - Using two-tailed (non-directional) test decreases power (relative to a one-tailed test)

Type 2 error

- Researcher fails to reject a null hypothesis that is really false - Researcher has failed to detect a real treatment effect - ________ error probability is not easily identified

Type 1 error

- Researcher rejects a null hypothesis that is actually true - Researcher concludes that a treatment has an effect when it has none - Alpha level is the probability that a test will lead to a ______ error

Logic of hypothesis test

- State hypothesis about a population - Predict the expected characteristics of the sample based on the hypothesis - Obtain a random sample from the population - Compare the obtained sample data with the prediction made from the hypothesis

Assumptions for the Independent-measure t-test

- The observations within each sample must be independent - The two populations from which the samples are selected must be normal - The two populations from which the samples are selected must have equal variances -- Homogeneity of variance

Assumptions of the t-test

- Values in the sample are independent observations. - The population sampled must be normal. With large samples, this assumption can be violated without affecting the validity of the hypothesis test.

- Data ______ collected after hypotheses stated - Data ______ collected after establishing decision criteria •This sequence assures _______

- always - always - objectivity

z-scores problems

- requires move information than what researchers usually have - needs knowledge of population standard deviation o - usually only have the sample data available

Factors that influence the outcome of hypothesis test

- size difference between sample mean and original population - variability of the scores - number of scores in the sample

What happens when treatment has inconsistent effect?

-Difference scores are more scattered -Variability is high

What happens when treatment has consistent effect?

-Difference scores cluster together -Variability is low

What are disadvantages of repeated-measures design

-Factors besides treatment may cause subject's score to change during the time between measurements -Participation in first treatment may influence score in the second treatment (order effects)

What are the advantages of repeated-measures design

-Requires fewer subjects -Able to study changes over time -Reduces or eliminates influence of individual differences -Substantially less variability in scores

small effect r2

0.01

medium effect r2

0.09

large effect r2

0.25

Four Steps of Hypothesis Testing

1. state the hypothesis 2. set the criteria for a decision 3. compute the test statistic 4. make a decision

normal distribution

A function that represents the distribution of variables as a symmetrical bell-shaped graph.

population

A group of individuals that belong to the same species and live in the same area

Sample

A relatively small proportion of people who are chosen in a survey so as to be representative of the whole.

A sports coach is investigating the impact of a new training method. In words, what would the null hypothesis say? A) The new training program produces different results from the existing one B) The new training program produces results about like the existing one C) The new training program produces better results than the existing one D) There is no way to predict the results of the new training program

B) The new training program produces results about like the existing one

When n is small (less than 30), the t distribution ______ A) is almost identical in shape to the normal z distribution B) is flatter and more spread out than the normal z distribution C) is taller and narrower that the normal z distribution D) cannot be specified, making hypothesis tests impossible

B) is flatter and more spread out than the normal z distribution

A researcher uses a hypothesis test to evaluate H0: µ = 80. Which combination of factors is most likely to result in rejecting the null hypothesis? A) o= 5 and n= 25 B) o= 5 and n= 50 C) o= 10 and n= 25 D) o= 10 and n= 50

B) o= 5 and n= 50

The results of a hypothesis test are reported as follows: t(21) = 2.38, p < .05. What was the statistical decision and how big was the sample? A) the null hypothesis was rejected using a sample of n=21 B) the null hypothesis was rejected using a sample of n=22 C) the null hypothesis was not rejected using a sample of n=21 D) the null hypothesis was not rejected using a sample of n=22

B) the null hypothesis was rejected using a sample of n=22

A researcher is predicting that a treatment will decrease scores. If this treatment is evaluated using a directional hypothesis test, then the critical region for the test. A) would be entirely in the right-hand tail of the distribution B) would be entirely in the left-hand tail of the distribution C) would be divided equally between the two tails of the distribution D) would be divided equally between the two tails of the distribution

B) would be entirely in the left-hand tail of the distribution

For an independent-measures research study, the value of Cohen's d or r2 helps to describe ______ A) the risk of a Type I error B) the risk of a Type II error C) how much difference there is between the two treatments D) whether the difference between the two treatments is likely to have occurred by chance

C) how much difference there is between the two treatments

Assuming that the sample mean difference remains the same, which of the following sets of data is most likely to produce a significant t statistic? A) n = 15 and SS = 10 B) n = 15 and SS = 100 C) n = 30 and SS = 10 D) n = 30 and SS = 100

C) n = 30 and SS = 10

degrees of freedom

Computation of sample variance requires computation of the sample mean first.

For which of the following would a repeated-measures study be appropriate? A) A group of twins is tested for IQ B) Comparing boys and girls strength at age 3 C) Evaluating the difference in self-esteem between athletes and non-athletes D) Students' knowledge is tested in September and December

D) Students' knowledge is tested in September and December

Which combination of factors is most likely to produce a significant value for an independent-measures t statistic? A) a small mean difference and small sample variances B) .a large mean difference and large sample variances C) a small mean difference and large sample variances D) a large mean difference and small sample variances

D) a large mean difference and small sample variances

The power of a statistical test is the probability of _____ A) rejecting a true null hypothesis B) supporting a true null hypothesis C) rejecting a false null hypothesis D) supporting a false null hypothesis

D) rejecting a false null hypothesis

_________ _________ ________ (___) is used as in place of the real standard error when the value of σM is unknown

Estimated standard error (SM)

T/F Sample size has a great influence on measures of effect size

False Measures of effect size are not influenced to any great extent by sample size

T/F If both samples have n = 10, the independent-measures t statistic will have df = 19

False df = (n1-1) + (n2-1) = 9 + 9 = 18

T/F Increasing the sample size will also increase the effect size

False Sample size does not affect Cohen's d

T/F When the value of the t statistic is near 0, the null hypothesis should be rejected

False When the value of t is near 0, the difference between M and u is also near 0

Null hypotheses for independent measures

H0: u1-u2=0

Alternative hypotheses for independent measures

H1: u1-u2/=0

What is an effect size?

How big the effect is

Treatment effect may be significant when variability is _____, but not significant when variability is _____

Low High

Null, directional, or non-directional? There will be no relationship between learners' responses to recasts and their development of ESL question formation.

Null

______ subjects are used in both treatment conditions

Same

T/F If the alpha level is decreased, the size of the critical region decreases

True Alpha is the proportion of the area in the critical region(s)

T/F For an independent-measures t statistic, the estimated standard error measures how much difference is reasonable to expect between the sample means if there is no treatment effect

True This is an accurate interpretation

T/F Compared to independent-measures designs, repeated-measures studies reduce the variance by removing individual differences

True Using the same subjects in both treatments removes individual differences across treatments

T/F An effect that exists is more likely to be detected if n is large

True A larger sample produces a smaller standard error and larger z

T/F An effect that exists is less likely to be detected if σ is large

True A larger standard deviation increases the standard error and produces a smaller z

T/F Larger differences between the sample and population mean increase effect size

True The mean difference is in the numerator of Cohen's d

T/F The critical region defines unlikely values if the null hypothesis is true

True This is the definition of "unlikely"

Repeated-measures design

Two separate scores are obtained for each individual in the sample

z-score

a measure of how many standard deviations you are away from the norm (average or mean)

standard deviation

a measure of variability that describes an average distance of every score from the mean

hypothesis testing

a statistical method that uses sample data to evaluate the validity of a hypothesis about a population parameter

one- tailed test

allows rejecting H0 with relatively small difference provided the difference is in the predicted direction

Critical region(s)

consist of the extreme sample outcomes that are "very unlikely"

___________ is a way to control time-related or order effects

counterbalancing

Small effect

d=0.2

Medium effect

d=0.5

Large effect

d=0.8

Null, directional, or non-directional? Learners who have received recasts will show greater development than learners who have not.

directional

For an independent-measures t statistic, the estimated standard error measures _______ ______ ______ is reasonable to expect between the sample means if there is no treatment effect

how much difference

Observations within each treatment condition must be _____________

independent

Data from two completely different, independent participant groups (an ___________ ___________ or __________ _________

independent-measures between-subjects design

Larger discrepancies --> ______ z-scores

larger

More variability --> _________ standard error

larger

Hypothesis testing is one of the _____ commonly used inferential procedures

most

Null, directional, or non-directional? Learners' familiarity with the content of the task will affect the efficacy of recasts.

non-directional

If the interval contain zero, then it is ______ a significant effect

not

Boundaries of critical region(s) are determined by the __________ ______ by the alpha level

probability set

two-tailed test

requires relatively large difference regardless of the direction of the difference

What is the difference between sample mean and population mean

sample mean is the mean of the sample and population mean is the estimate of the population

Estimation can provide an indication of the ______ of the effect

significance

Alpha level

significance level, is a probability value used to define "very unlikely" outcomes

If the interval does NOT contain zero, the treatment effect was __________

significant

Estimation can provide an indication of the _____ of the treatment effect

size

Larger n --> _______ standard error

smaller

In general two-tailed tests should be used unless there is a __________ __________ for a directional prediction

strong justification

____ statistic is an alternative to z

t

standard error of the mean

the standard deviation of the sampling distribution of sample means

Most research studies compare _____ sets of data

two

t-statistic

used to test hypotheses about an unknown population mean μ when the SD value of σ is also unknown

When do we use t-statistics and t-tests?

when your comparing 2 means and don't have the information for the z-score

Data from the same or related participant group(s) (a __________ _______ or _________ ________)

within-subjects repeated-measures design

Repeated-measures design is also known as

within-subjects design

What are the factors that influence hypothesis test outcomes?

•Size of the sample mean difference (larger mean difference à larger numerator so increases t •Sample size (larger sample size à smaller standard error—denominator—so larger t) •Larger sample variance à larger standard error—denominator—so larger t)