Ch. Inferential Statistics

Demand characteristics

Subtle or often unconscious hints and cues that guide participants to act in accordance with the experimenter's wishes or expectations.

Null hypothesis significance testing (NHST)

A statistical method for testing whether or not an independent variable has an effect on the dependent variable.

Significance

A treatment effect in an experiment that is not likely due to chance. Findings are considered statistically significant if the probability that we are wrong (where we reject H0 and H0 is true) is less than .05. Significant findings indicate that the results of the experiment are real and not due to chance.

Insignificant

A value judgment such as deciding between good and evil or between worthless and valuable. It is often erroneously used instead of the correct statistical term not significant or nonsignificant.

Directional alternative hypothesis

Also called a one-tailed test of significance where the alternative hypothesis is specifically stated beforehand; for example, Group 1's mean is greater than Group 2's mean.

Nondirectional alternative hypothesis

Also called a two-tailed test of significance where the null hypothesis will be rejected if two groups' means differ in either direction.

Signal-to-noise ratio

Borrowed from signal detection theory, in which the effect of a treatment is considered the signal and random variation in the numbers is considered the noise.

Nonsignificant

Findings are considered statistically nonsignificant or not significant if the probability of being wrong is greater than .05. Nonsignificant findings indicate that the null hypothesis has been retained and the results of any effect of the independent variable on the dependent variable are attributed to chance.

Trend

Frequently reported when the data do not reach the conventional level of statistical significance (.05) but come close (e.g., .06, .10). The American Psychological Association's (2010) style manual no longer addresses the issue of publishing data with trends or marginal significance.

Research hypothesis

Most frequently what the experimenter thinks may be true or wishes to be true before he or she begins an experiment. It can also be considered the experimenter's hunch, and it is also labeled the alternative hypothesis

Alternative hypothesis (Ha)

Most frequently, what the experimenter thinks may be true or wishes to be true before beginning an experiment. Also called the research hypothesis. It can also be considered the experimenter's hunch.

Controlled experiment

The most powerful experimental design because it allows for the inference of causation. It is most often a two-group experiment, where the participants are randomly assigned to one of two (or more) groups: an experimental group that receives some treatment (independent variable) and a control group that receives a placebo. The parameter of statistical interest is the difference between the two groups' means on some dependent or response variable as a function of the independent variable.

p level

The probability of committing the Type I error, that is, rejecting H0 when H0 is true. In statistical testing, there is no way to ultimately know whether rejecting or retaining the null hypothesis is the correct decision; therefore, when rejecting the null hypothesis, all that can be stated is that the probability of being wrong is less than .05 or less than 5%.

Alpha (α)

The probability of committing the Type I error. To consider findings significant, the probability of alpha must be less than .05.

Beta (β)

The probability of committing the Type II error. A Type II error can occur only when the null hypothesis is false and the experimenter fails to reject the null hypothesis.

Replication

The repeated ability to duplicate the results of a scientific experiment. Replication usually consists of a series of experiments after an initial study where the series varies from the initial study in types of subjects, experimental conditions, and so on. Replication should be conducted not only by the initial study's author but also by other scientists who do not have a conflict of interest with the eventual outcome. The .05 significance level is actually derived from Fisher's original concept that one may be more certain of a finding if the same outcome can be found in 19 of 20 similar experiments.

Probability

The science of predicting future events or the likelihood of any given event occurring.

Null hypothesis (H0)

The starting point in scientific research where the experimenter assumes there is no effect of the treatment or no relationship between two variables.

Type I error

When an experimenter incorrectly rejects the null hypothesis when it is true.

Type II error

When an experimenter incorrectly retains the null hypothesis when it is false.