Psych Research Methods Unit 3

Within-subjects factorial design

observe same participants in each group and control for timing and order effects

Mixed-factorial design

participants are observed at each level of a between-subjects factor and also repeatedly observed across the levels of a within-subjects factor

Type I error

probability of rejecting a null hypothesis that is actually true. Researchers directly control for the probability of committing this error by stating the level of significance

Type II error

probability of retaining a null hypothesis that is actually false. This means the researcher is reporting no effect in the population, when in truth there is an effect

Nonparametric test

significance tests that are used to test hypotheses about data that can have any type of distribution and to analyze data on a nominal or ordinal scale of measurement

Parametric test

significance tests that are used to test hypotheses about parameters in a population in which the data in the population are normally distributed and measured on an interval or ration scale

Practice effect

similar to a testing effect Way to control: modify the measure (vary the measure), provide with training (measuring the skill and not how they learn how to do the task): trial run

Error variance: The less scores in each group overlap

smaller error variance

Partial counterbalancing

some, but not all, possible order sequences in which participants receive different treatments or participate in different groups are balance or offset

Interaction effect

source of variation associated with how the effects of one factor are influenced by, or depend on, the levels of a second factor

Main effect

source of variation associated with mean differences across the levels of a single factor Determine main effects for columns and rows

Experimental manipulation

the identification of an independent variable and the creation of two or more groups that constitute the levels of that variable

Statistical Power

the likelihood that a study will detect an effect when there is an effect there to be detected

Counterbalancing

the order in which participants receive different treatments or participate in different groups is balanced or offset in an experiment

Professional talks

the presenter is likely the only presenter or one of only a few presenters for the hour or so that the talk is given ; great way to reach an engage audience and promote your research, identify the scientific merits of you research, and even get people excited about your research

Order effects

threat to internal validity in which the order in which participants receive different treatment or participate in different groups causes changes in the dependent variable: potential confound

Participant fatigue

tired, bored, less enthusiastic Way to control: keep tasks brief, breaks, interesting, not as taxing

Individual differences

unique characteristics of participants in a sample that can differ from one participant to another

Describe how results are reported for a qualitative research design

"Analysis" provides a series of interpretations and contributes a new perspective or generates the possibility that many different perspectives can explain the observations made; written as a narrative

Describe how results are reported for a Quantitative research design

"Results" reports the statistical outcomes of the measured data

Three types of factorial design

1.) Between-subjects design 2.) Within-subjects design 3.) Mixed factorial design

Ways to control order effects

1.) Complete counterbalancing 2.) Partial counterbalancing

Control by matching

1.) Identification of variables to be matched across groups 2.) Measurement of the matching variable for each participant 3.) Assignment of participants to groups by means of a restricted random assignment that ensures balance between group

Four time-related factors and why they each are important to consider in a within-subjects experimental design

1.) Participant fatigue 2.) Practice effects 3.) Carryover effect 4.) Order effects

Three ways to control by holding constant

1.) Random Assignment 2.) Control by matching 3.) Control by holding constant

Three types of measures for a dependent variable?

1.) Self-report 2.) Behavioral measures 3.) Physiological measures

Describe the three categories of data structures and corresponding research strategies discussed in class

1.) Single group of participants with one score per participant 2.) Single group of participants with two variables measured for each participant 3.) Two or more groups of scores with each score a measurement of the same variable

Control by holding constant

Can eliminate the variable under question (e.g. use only female participants) Restrict range of values for a variable Limits external validity Attempting to improve internal validity by exercising control within a research study can threaten external validity

Be able to find the number of groups in a factorial design.

Identify any type of factorial design by the number of levels in each factor Multiply the levels of each factor 3x4 factorial design= 12 groups Assignment of participants to groups depends on whether manipulate levels of a within-subject factor or the levels of a between-subject factor

Why it is important to measure error variance in an experiment?

Individual differences can be measured numerically: error variance

Descriptive Statistics: Numerical scores from interval or ration scales

Mean and standard deviation Proportions or percentages to describe distribution across categories

Descriptive Statistics: Ordinal scores (ranks or ordered categories)

Median proportions or percentages to describe distribution across categories

Descriptive Statistics: Nominal scores (names categories)

Mode proportions or percentages to describe distribution across categories

Disadvantages of Between-Subjects Design

Need large sample size: particularly with multiple groups

The three categories of data structures and corresponding research strategies discussed in class: Single group of participants with one score per participant

One group of participants with one variable measured for each participant. These data are produced by studies using a descriptive research approach Purpose: describe individual variables as they exist naturally Data are a portion of results from a larger study examining several variables—look at one variable at a time Often report information like the mean, median, and mode

The three categories of data structures and corresponding research strategies discussed in class: Single group of participants with two variables measured for each participant

One group of participants with two (or more) variables measured for each participant. These data are produced by the correlational research strategy Purpose: Examine relationships between variables No attempt to manipulate or control the variables—simply observed and recorded as they exist naturally

Self-report measures

Participants respond to one or more questions/statements to indicate their perceived experiences, attitudes or opinions This could be subject to inaccuracies (social desirability)

Discuss the peer review process and why it is important.

Peer review is the primary mechanism used to judge quality of research and researchers Peer reviews are central to multiple aspects of research and professional development: o Grant reviews o Manuscript reviews o Personnel reviews (e.g. promotion and tenure) Responsible peer reviews must be timely, thorough, constructive, free from personal bias, and respectful of confidentiality guidelines

Disadvantages of Within-groups design

Potential for order effects: threaten internal validity Might not be practical or possible Demand characteristic: when participants see all levels of IV, they may change the way they normally act (demand characteristic)

Describe the statistical techniques that could be used for examining differences with related samples and one-way within-subjects comparisons

Related sample: a sample in which the same or matched participants are observed in each group 1.) Use a related samples t-test/paired-samples t-test to compare differences between two group means Repeated-measures design used to observe participants in more than two groups 2.) Use a one-way within-subjects ANOVA: when same participants are observed at each level of a factor

Behavioral measures

Researchers directly observe and record the behavior of subjects of participants Ex: frequency, rate, or duration of behaviors

Physiological measures

Researchers record physical responses of the brain and body in a human or an animal Ex: cortisol, blood, saliva, sweat, heart rate, blood pressure, body temperature, eye movement

Null hypothesis significance testing

To use NHST, we begin by stating a null hypothesis Null hypothesis, states as the null: a statement about a population parameter, such as the population mean, that is assumed to be true, but contradicts the research hypothesis In other words, we begin by assuming we are wrong After we state a null hypothesis, then we set a criterion upon which we will decide to retain or reject the null hypothesis The criterion is a probability value for the likelihood of obtaining the data in a sample if the null hypothesis were true for the population

Advantages of Within-groups design

True experiments because they involve manipulated variable and measured variable Can measure temporal precedence When you can control for order effects, you can establish internal validity

The three categories of data structures and corresponding research strategies discussed in class: Two or more groups of scores with each score a measurement of the same variable

Two or more groups of scores with each score a measurement of the same variable. These data are produced by the experimental, nonexperimental, and quasi-experimental research designs Evaluating different levels or manipulating something Use the categories of one variable to define groups and then measure a second variable to obtain a set of scores within each group Participant characteristics (age, ethnicity, etc), time, or treatment conditions

Experimental group

a condition in an experiment in which participants are treated or exposed to a manipulation, or level of the independent variable, that is believed to cause a change in the dependent variable

Comparison group

a condition in an experiment in which participants are treated the same as participants in an experimental group, except that the manipulation believed to cause a change in the dependent variable is omitted

Level of significance

a criterion of judgment upon which a decision is made regarding the value stated in a null hypothesis. The criterion is based on the probability of obtaining as statistic measured in a sample if the value stated in the null hypothesis were true

Skewed distribution

a distribution of scores that included outliers or scores that fall substantially above or below most other scores in a data set

Cronbach's alpha

a measure of internal consistency that estimates the average correlation for every possible way that a measure can be split in half Want it to be .7 or above

Cohen's kappa

a measure of interrater reliability that estimates the level of agreement between two raters, while taking into account the probability that the two raters agree by chance or error

Restricted random assignment

a method of controlling differences in participant characteristics between groups on known participant characteristics, then using a random procedure to assign participants to each group. Two strategies of ___ are control by matching and control by holding constant

Error variance/Error

a numeric measure of the variability in scores that can be attributed to or is caused by the individual differences of participants in each group

Complete Counterbalancing

a procedure in which all possible order sequences in which participants receive different treatments or participate in different groups are balanced or offset in an experiment

Factorial experimental design

a research design in which groups are created by manipulating the levels of two or more factors, then the same or different participants are observed in each group using experimental procedures of randomization (for a between-subjects factor) and using control for timing and order effects (for a within-subjects factor)

Factorial design

a research design in which participants are observed across the combination of levels of two or more factors

Between-subjects factorial design

a research design in which the levels of two or more between-subjects factors are combined to create groups, meaning that different participants are observed in each group

Within-subjects design (repeated measures design)

a research design in which the same participants are observed one time in each group of a research study

Error

a source of variance that cannot be attributed to having different groups or treatments

Pairwise comparison

a statistical comparison for the difference between two group means. A post hoc test evaluates all possible pairwise comparisons for an ANOVA with any number of groups

Post hoc test

a statistical procedure computed following a significant ANOVA to determine which pair or pairs of group means significantly differ. These tests are needed with more than two groups because multiple comparisons must be made

Two-way ANOVA

a statistical procedure used to analyze the variance in a dependent variable between groups created by combining the levels of two factors

Two-independent-sample t test (independent-sample t-test)

a statistical procedure used to test hypotheses concerning the difference in interval or ratio scale data between two group means, in which the variance in the population is unknown

Paired-samples t test

a statistical procedure used to test hypotheses concerning the differences in interval or ratio scale data for two related samples in which the variance in one population is unknown

Related-samples t test

a statistical procedure used to test hypotheses concerning the differences in interval or ratio scale data for two related samples in which the variance in one population is unknown

One-way between-subjects ANOVA

a statistical procedure used to test hypotheses for one factor with two or more levels concerning the variance among group means. This test is used when different participants are observed at each level of a factor and the variance in a given population is unknown

Normal distribution

a theoretical distribution with data that are symmetrically distributed around the mean, the median, and the mode

Discuss the goal of a literature review.

a written comprehensive report of findings from previously published works about a problem in the form of a synthesis of previous articles or as a meta-analysis The goal is to organize, integrate, and evaluate published works about a problem and to consider progress made toward clarifying that problem

Solomon four-group design

an experimental research design in which different participants are assigned to each of four groups in such a way that comparisons can be made to (1) determine if a treatment causes changes in posttest measure and (2) control for possible confounds or extraneous factors related to giving a pretest measure and observing participants over time

Within-subjects experimental design

an experimental research design in which the levels of a within-subjects factor are manipulated and then the same participants are observed in each group or at each level of the factor. To qualify as an experiment, the researcher must (1) manipulate the levels of the factor and include a comparison/control group, and (2) make added efforts to control for order and time-related factors

Discuss advantages of presenting results in poster format

concise findings; eye-catching and engaging

Chi-square test for independence

determining whether frequencies at the levels of two categorical variables are independent or related

Carryover effect

exposure to one treatment changes your reaction to the second treatment A--B Way to control: differentiate, debriefing/clear, increase the time

Random Assignment

helps ensure that each participant has the same likelihood of being selected to a given group DOES NOT guarantee groups will be equivalent (especially small groups) With random assignment, we assume individual differences are the same in each group Sometimes do not want to take the chance Take steps to control the assignment of participants to groups-restricted random assignment

Interrater reliability

how much does the raters agree about what they see

Internal consistency

how similar the measures are

Descriptive Statistics

important for demographic and measures

Error variance: The more scores in each group overlap

larger error variance

Between-subjects experimental design

levels of between-subjects factor are manipulated Different participants randomly assigned to each group/level Participants observed one time

Individual Differences

unique characteristics of participants in a sample that can differ from one participant to another o Individual differences can be confounding variables o Individual differences can produce high variability in scores difficult to know if treatment has an effect

Between-subjects factorial design

use random assignment to observe different participants in each group

Advantages of Between-Subjects Design

• Only design that can meet 3 requirements of an experiment (randomization, manipulation, control) Places less burden on participant and researcher because (typically) only one observation

Recognize the elements of an APA-style manuscript

• Title Page • Abstract • Main body • References • Footnotes • Tables • Figures • Appendices