Exam 3: Research designs
Pearson's r
a statistic that measures the direction and strength of the linear relation between two variables that have been measured on an interval or ratio scale
Tukey's HSD
A procedure for the multiple comparison of means after a significant F ratio has been obtained BETWEEN SUBJECTS DESIGN
f test
A procedure used to determine whether there is more variability in the scores of one sample than in the scores of another sample.
Spearman's rho
A test for correlation when data is at least ordinal level matched pairs The association must be monotonic (i.e., variables increase in value together, or one increases while the other decreases)
The meaning of measurement
Assigning numerals to objects or events according to some rule A numeral is a symbol. It does not have quantitative meaning unless we want it to represent/convey such information Numbers are numerals that have quantitative meaning.
ordinal measures
Ordinal measures: numbers convey information about identity and rank (order) can't tell you how much more pf a quality someone has than another 1st place, 2nd place, 3rd place 1 = hot, 2 = hotter, 3 = hottest Can establish equality as well as the direction of inequality A = B, A =/= B A < B, A > B can compute mean/median Ranking of favourites Academic grades percentile
What is the value of a factorial design?
Theoretical: explain processes more precisely Generalizability: finding an interaction focuses generalizability, finding no interaction expands generalizability Statistical power: taking a participant variable into account reduces within-group variability and increases F ratio
Our first example of research relating brain lesioning with performance deficits in problem solving told us that yes, there is an effect. Lesioning a portion of the hippocampus caused more errors on the problem solving task. But we now need to find out why. One possibility is that lesioning impairs access to prior learning; it disrupts retrieval from memory. A second possibility is that lesioning impairs the acquisition of new information or knowledge; it disrupts encoding into memory. Group 1: (No Lesion) completes ten learning trials of a complex maze (Learn-pre "treatment"), is housed for three weeks in a standard colony cage, and is then tested on the maze (M = 2.4 errors, SD = .41). Group 2: (No Lesion) is housed for three weeks in a standard colony cage, completes ten learning trials of a complex maze (Learn-post "treatment"), and is then tested on the maze (M = 2.1 errors, SD = .3). Group 3: completes ten learning trials of a complex maze (Learn-pre treatment), undergoes lesioning procedure (Lesion) and convalesces for three weeks in a standard colony cage, and is then tested on the maze (M = 2.6 errors, SD = .28). Group 4: undergoes lesioning procedure (Lesion) and convalesces for three weeks in a standard colony cage, completes ten learning trials of a complex maze (Learn-post treatment), and is then tested on the maze (M = 3.6 errors, SD = .14). What is (are) the independent variable(s)? How many levels does each independent variable have? What are they? What is (are) the dependent variable(s)? How might we organize the four values reported above in a table? What are the sources of between-group variability for this experiment? How many F-ratios do we need to calculate? How does this experimental design help us to determine which of the two possible hypotheses/explanations (lesioning disrupts encoding versus lesioning disrupts retrieval) for the lesioning effect is correct?
What is (are) the independent variable(s)? retrieval and encoding How many levels does each independent variable have? What are they? 2 levels each What is (are) the dependent variable(s)? number of mistakes made How might we organize the four values reported above in a table? What are the sources of between-group variability for this experiment? How many F-ratios do we need to calculate? time of treatment (pre vs post), type of treatment How does this experimental design help us to determine which of the two possible hypotheses/explanations (lesioning disrupts encoding versus lesioning disrupts retrieval) for the lesioning effect is correct?
Post hoc trend analysis:
What is the shape of the relationship?
Decisions about the number of levels to use
When IV levels differ quantitatively (i.e., in amount): curvilinear relationships (arousal and performance or physical beauty and attraction), threshold effects, ceiling and floor effects When IV levels differ qualitatively (i.e., in kind): identifying critical element (psychotherapy outcome research or educational intervention)
Post hoc t tests among group means:
Which groups differ?
multiple baselines design
look at changes corresponding to the introduction of an intervention rather than looking for reversals associated with the withdrawal of an intervention (IV) try to increase ability to make causal statements w/o being a true experiment Examples Across behaviors Across situations Across subjects Often used in operant conditioning studies Collect baselines for several behaviors Reinforce one of the behaviors. See if that behavior increases (and the others don't) Reinforce a second behavior
Analysis of Variance (ANOVA)
a statistical technique that determines whether three or more means are statistically different from one another F=1 means treatment didn't work, wasn't significant larger F = less likely to be due to chance
t test
a statistical test used to evaluate the size and significance of the difference between two means
reversal design
a way of obtaining comparison data and of confirming causal significance of an intervention give treatment then take it away and see if behavior returns Example: Tongue-thrusting of three-year old prevented her from enjoying and benefitting from interactions with adults and peers (social isolation) Value of baseline condition Inadequacy of "single baseline-lemon juice" sequence Use of reversal, i.e., the withdrawal of the lemon juice treatment, confirms efficacy of the lemon juice Example 2: Chronically high number of burglaries in a six-square mile residential area in Nashville, TN Value of baseline condition Inadequacy of "single baseline-helicopter patrol" sequence However, should assess displacement before endorsing use of helicopter patrols
two reasons to use inferential statistics
estimate certain parameters (population mean) determine whether the relationship between variables would hold in the population
Three hundred survey respondents were asked to give their political party preference: (1) Republican, (2) Democrat, or (3) Independent. What type of measurement is this?
nominal
n =
number in each group
introduction is where authors tell you
how they came up with hypothesis why hypothesis will be supported why testing hypothesis is important why authors' way of testing is the best way to test hypothesis
Value of empty control groups
in the absence of a control group, comparison of qualitatively different treatments can reveal which has the greatest effect but cannot reveal if the treatments helped or hurt compared with no treatment
how do you set up a situation in which the only systematic difference between the no treatment and the treatment group is the treatment
independent random assignment: each participant has an equal chance of being assigned to the treatment or control group allows for cause-effect statements
problem with range as a measure of spread
only looks at two scores, may be outliers and overestimated variability
College applicants completed a nationally standardized entrance exam. The percentile scores for the applicants were recorded on their admissions folders. what type of measurement is this?
ordinal
Five observers rated twenty preschool students on cooperation. Each observer rated each child on a nine-point scale from 0 (very uncooperative) to 8 (very cooperative). The average of the five judgments was used as the score for each child. what type of measurement is this?
ordinal
One-hundred people completed a thirty-item questionnaire designed to measure altruism. People received one point for each question they answered altruistically. Each person received a total score that ranged from 0 to 30. what type of measurement is this?
ordinal
The thirty members of a class were ranked according to height. The shortest person was assigned the number 1, the second shortest, the number 2, and so on. what type of measurement is this?
ordinal
mode
qualitative information (nominal)
how to ensure random assignment
random numbers table
measures of variability
range, standard deviation, variance, min, max
A nutritionist compared the effect of a drug said to promote weight loss to a placebo. After six months of treatment the number of pounds lost by each experimental and control participant was calculated. what type of measurement is this?
ratio
Participants completed a 20-item free association test. The experimenter recorded the amount of time each person took to respond to each word. The average time to respond was computed for each participant. what type of measurement is this?
ratio absolute zero
F ratio equation
ratio of the between-groups population variance estimate to the within-groups population variance estimate F = MSbetween/MSwithin estimated population variance = SS/df (N-1) estimated sample variance = SS/N
two ways to stop random error from overwhelming treatment effect
reduce the effects of random error increase the size of the treatment effect
N =
total population size
nuisance variable
unwanted variables that can cause the variability of scores within groups to increase (systematic variation within groups)
null hypothesis
varying the treatment has no effect any difference you observe between groups could be due to chance the failure to find a treatment effect doesn't mean that the treatment has no effect, you just failed to prove beyond a reasonable doubt that the IV had an effect
why you shouldn't test your control group in one session and your experimental group in a different session
when tested in groups they make become group members who influence each other responses group testing turns random differences between sessions into systematic effects
Within group degrees of freedom
within group design gets rid of systematic error
What kinds of control groups are there?
Empty control group: participants do nothing Placebo control group: participants do nothing effective Two control groups: empty and placebo
Disadvantages to experimental design
Ethics: cannot expose subjects to some stimuli of importance External validity or generalizability: events occurring under controlled conditions are not similar to or representative of events occurring under uncontrolled conditions
nominal measures
Nominal measures: using numerals to name/identify objects or events Person's jersey number Nominal measures allow one to designate identity Quantitatively, nominal measures enable one to establish equality or the lack thereof A = B, A =/= B cannot use means -- must use frequencies or percentages to represent data social security numbers Gender colour shape numbers that represent KINDS not amounts - qualities, types or categories
Level of Measurement --> Comparisons possible
Nominal: Equality, A = B Ordinal: Equality, Direction of difference, A = B, A < > B Interval: Equality, Direction of difference, Magnitude of difference, A = B, A < > B, C - A ≤ ≥ C - B Ratio: Equality, Direction of difference, Magnitude of difference, Proportional relationship, A = B, A < > B, C - A ≤ ≥ C - B, C/A ≤ ≥ C/B
type 2 error
Accepting null hypothesis when you should have rejected it
SSwithin
Sum of (X - Mean Score)^2
The effectiveness of three methods of instruction for vocabulary acquisition were compared to one another and to an uninstructed control condition. There were four groups, one for each instructional condition. A total of 100 participants were distributed randomly and equally among the four conditions. The three experimental methods were: Synonym: study a synonym of the to-be-learned word Definition: study a dictionary definition of the to-be-learned word Passage: study the to-be-learned word in the context of a literary passage The researcher expected all three experimental methods to be better than the uninstructed control group, but the passage method was expected to be the best of all. Participants completed a vocabulary usage test following the study period. They had to select one sentence from a set of three for which the vocabulary word was appropriate. What is the independent variable? How many levels of the independent variable are there? What are they? What is the dependent variable? Why is a between-subjects design appropriate for this investigation? Why are there more than two groups? What is the null hypothesis? What is the alternative/experimental hypothesis?
*single-factor between-subjects design* What is the independent variable? method of instruction How many levels of the independent variable are there? What are they? 4 levels (control, passage, definitions, synonym) What is the dependent variable? number of words correct Why is a between-subjects design appropriate for this investigation? carryover effects with within group design Why are there more than two groups?* What is the null hypothesis? there is no difference between the methods of instruction What is the alternative/experimental hypothesis? not H0
MSbetween
- Numerator - Measures size of mean differences SSbetween/dfbetween
Advantages of using more than two values of the independent variable
A. Comparing more than two kinds of treatments B. Comparing more than two levels (amounts) of an independent variable to increase external validity C. Using multiple levels to improve construct validity of experiments Better ability to estimate the effects of different amounts (levels) of a treatment* Better ability to rule out confounding variables
Time-series designs
A. Eliminate selection threats B. Assume that the effects of history, maturation, mortality, testing, instrumentation, and regression would be consistent or cyclical over time C. Problem: Effects are not always continuous, especially history effects D. One solution: Reversal designs E. Another solution: Combine the strengths of the time-series design with the strengths of the non-equivalent control group design by using a two-group time-series design F. Examples of two-group time-series designs (drinking age and different states and accidents)
Differences between within subjects designs and between subjects designs
Between: random assignment to experimental conditions, eliminate effects of participant differences by randomly assigning, requires plentiful participants, avoids order effects Within: counterbalance order of experimental conditions, eliminate effects of participant differences by comparing to themselves, requires relatively few participants
descriptive statistics appropriate for ordinal data
Central Tendency: median Variability: range
descriptive statistics appropriate for interval data
Central tendency: mean, median, mode Variability: range, variance, standard deviation
descriptive statistics appropriate for ratio data
Central tendency: mean, median, mode Variability: range, variance, standard deviation
descriptive statistics appropriate for nominal data
Central tendency: mode Variability: variety
Within-subjects design
Comparison of subjects with themselves eliminates concern about equivalence of groups Control over sequencing of conditions and treatment of subjects ensures that only exposure to varying levels of the independent variable differentiates one condition from another Any resulting differences can be attributed to different experiences with the independent variable
What makes an investigation a true experiment?
Complete control over all variables in the investigation Allows one to confidently assert that scores or measures obtained under various conditions differ only because of differing values on the independent variable
single subjects design advantage
Confirm effectiveness of a treatment for an individual subject Circumvent ethical concerns re. withholding treatment from a control group Helpful when only a small number of cases are available
Between-subjects design
Control over assignment of subjects ensures equivalence of groups at the outset of the investigation Control over treatment of subjects ensures that only exposure to varying levels of the independent variable differentiate subjects Any resulting differences can be attributed to different experiences with the independent variable
Should you include an empty control group?
Eliminate empty control groups No need for empty control group with IV that varies quantitatively (e.g., minutes of exercise, numbers of repetitions)
Mann-Whitney U test
Determines whether two uncorrelated means differ significantly when data are nonparametric nominal IV x ordinal DV USE SAMPLE SIZE FOR U TABLE NOT DF
Interrupted time series designs
Example: effect of newspaper crime reporting e.g., publishing court results of people convicted of shoplifting on subsequent crime rate because of the social stigma attached to shoplifting and the concern about such behavior becoming public knowledge Design eliminates maturation as a threat to internal validity; can observe rate and direction of change before and after event or treatment Design does not eliminate history as a threat to internal validity, with data on only one group cannot determine whether or not a second event coincided with the event or treatment of interest Design does not eliminate instrumentation changes in programs and policies often accompanied by changes in record keeping
(T5L5) Half the participants in a pain study received a drug (nalaxone) that inhibits the effect of morphine, the other half received a placebo. Half of the placebo group and half of the drug group received acupuncture. The others did not. The participants then rated the painfulness of a series of electric shocks on a scale ranging from 1 (not at all painful) to 10 (very painful). Does it look like there is a main effect for the drug? Does it look like there is a main effect for acupuncture? Does it look like the drug and acupuncture interact to influence pain ratings?
IV - 2 levels (placebo vs treatment) DV - perceived pain
A developmental psychologist interested in exploratory behavior in preschoolers knows that when presented with two objects, one which the child has seen before, and another which is new (novel), that the child will typically reach for the novel object. She wondered whether different types of reinforcers would affect this behavior, and decided to compare verbal and nonverbal reinforcers. She selected a sample of toddlers, and administered a test of "curiosity drive"; Children were scored as either "low-curious" or "high-curious." Then, half of the children in each group were randomly assigned to one of the reinforcement conditions. On each trial, a child was shown a pair of objects, one that had been seen previously, and one that was novel. For children in the verbal reinforcement group, each time they reached for the novel object, the experimenter said "very good"; For children in the nonverbal reinforcement group, each time they reached for the novel object, the experimenter smiled and clapped her hands. On each trial, the reaching behavior was recorded. The child's total score was the percent of times they reach for the novel object. 1. identify each independent and/or predictor variable (i.e., distinguish between manipulated variables and subject variables) and dependent variable 2. tell how many levels there are of each IV/PV and what those levels are 3. if the design is factorial, give the design description (e.g., 2 x 3) 4. identify which variable(s) is/are between subjects, and which is/are within subjects 5. state (based on your answers to #4) whether the design is between-subjects, within-subjects, or mixed
IV - reinforcer (2 levels) PV - curiosity drive (2 levels) DV - % that novel object was chosen Design - 2 (reinforcer) x 2 (curiosity) Nature of manipulation - reinforcer = between, curiosity = between Between subjects
Researchers were interested in the effects of daily stress on participants'; sleep latency (length of time it takes them to fall asleep). They were also interested in whether different types of training procedures could alleviate stress, and decrease sleep latency. They operationalized "stress"; as how participants spent their day, and compared several groups of volunteer participants: those who spent the full day at home, those who spent the full day outside their home but were unemployed, those employed in skill jobs, those employed in unskilled jobs, and those employed in professional jobs. Each participant came into the sleep-lab, and his/her average sleep latency (over a five-night period) was recorded. Next, each participant was trained in either deep-muscle relaxation or in transcendental meditation. After four weeks of using their newly learned skills, participants came back to the lab, and were tested again for average sleep latency. Next, they were trained in the method which they had not learned the first time, and asked to use it for the next four weeks, at the end of which time they returned to the lab again, and were tested for average sleep latency. 1. identify each independent and/or predictor variable (i.e., distinguish between manipulated variables and subject variables) and dependent variable 2. tell how many levels there are of each IV/PV and what those levels are 3. if the design is factorial, give the design description (e.g., 2 x 3) 4. identify which variable(s) is/are between subjects, and which is/are within subjects 5. state (based on your answers to #4) whether the design is between-subjects, within-subjects, or mixed
IV - type of training (3 levels - none, TM, deep muscle) PV - daily stress DV - sleep latency Design - 3 (training type) x 5 (stress level) Nature of manipulation - training = within, stress = between Mixed design
In a study of eyewitness testimony, Loftus, Miller, & Burns (1978) showed participants 30 slides depicting an automobile-pedestrian accident. After viewing the slides, participants completed a test of recognition of various events, which did or did not appear in the slides. For half of the participants, the questions asked on the test were consistent with the information presented in the slides; for the other half of the participants, the questions asked were inconsistent with the slide information. Each participants score was the proportion of items correctly answered on the test. A week later, the same participants were brought back to the lab, and, without showing the slides again, were asked to take the same recognition test, to see whether their memory would be affected by the length of time between seeing the accident and trying to remember what occurred. 1. identify each independent and/or predictor variable (i.e., distinguish between manipulated variables and subject variables) and dependent variable 2. tell how many levels there are of each IV/PV and what those levels are 3. if the design is factorial, give the design description (e.g., 2 x 3) 4. identify which variable(s) is/are between subjects, and which is/are within subjects 5. state (based on your answers to #4) whether the design is between-subjects, within-subjects, or mixed
IV- content of test (2 levels), time of test (2 levels) DV - number of items correctly answered Design - 2 (test content) x 2 (time of test) Nature of manipulation - content = between, time = within Mixed design
Researchers sought to determine whether it is a: characteristic of the child, characteristic of the parent, or an interaction of the two that produces problematic behavior in children. They observed 32 mothers as they interacted (by playing a game) with two different boys, aged 6 to 11 years. Half the mothers had conduct-disordered children (CDC), children who had been referred to a mental-health facility because of the severity of their disorder (e.g., aggressiveness). The others were mothers of "normal" children (NC), children who were matched for age and number of parents, with no serious behavior problems. Each mother interacted for 15 minutes with each of two boys: someone else's conduct-disordered child (CDC) and someone else's "normal" child (NC). The researchers counted the mothers': positive actions (e.g., praise, compliment), negative actions (e.g., criticism, complaint), number of requests, and the level of compliance among the boys during the sessions. They found that the two groups of mothers did not differ in their behavior. However, the mothers, as a group, engaged in: more positive behaviors with the "normal" child (NC), more negative behavior and made more requests of the conduct-disordered child (CDC). The researchers also found that the conduct-disordered children (CDC) were less compliant. Identify the independent variables and the levels of each. Which are implemented using a between-subjects design? Which are implemented using a within-subjects design? Identify the dependent variables. Format a table for the mean values that would be generated by the research described above.
Identify the independent variables and the levels of each. PV - nature of mom, IV - nature of child Which are implemented using a between-subjects design? nature of mom Which are implemented using a within-subjects design? nature of child Identify the dependent variables. Problematic behavior Format a table for the mean values that would be generated by the research described above.
Advantages to experimental design
Internal validity: no extraneous variables i.e., no other variable can explain findings
interval measures
Interval measures: Each interval re. the variable being measured is equal to every other interval re. that variable. Arbitrary zero (i.e., it has no special meaning) Numbers convey information about identity rank (order), and size of the difference (interval distance) temperature, F, C Discrete - Thoughts, behaviours, feelings, etc. on a Likert scale Metric
chi square
Involves categorical variables. Looks at 2 distributions of categorical data to see if they differ from each other. Male vs Female preference of sports drink nominal IV x nominal DV expected = (r x c) / g df = (r-1)(c-1) determine whether differences between sample percentages reflect difference in population percentages
In an investigation on the effects of repetition on memory among brain-injured adults, a group of sixteen participants, each with damage to the hippocampus, were asked to learn four lists (i.e., lists' A, B, C, and D) of consonant-vowel-consonant (CVC) trigrams. Each list contained ten trigrams. The trigrams were presented, one at a time, on a computer monitor. In the course of presenting a list, all the trigrams on that list were presented either two times, four times, six times, or eight times. Once a list of ten trigrams had been presented the requisite number of times, a test sheet containing both trigrams from the list and novel trigrams was presented to a participant. Participants identified whether the trigrams were familiar or novel. The number of recognition errors was recorded. CVC list and number of repetitions were counterbalanced. Is this a between-subjects or a within-subjects design? How do you know? What is the independent variable? How many levels does it have? Name them. What is the dependent variable? How is it measured? Why is this type of design (between- or within-subjects) appropriate for this experiment?
Is this a between-subjects or a within-subjects design? within subjects How do you know? compared against themselves What is the independent variable? repetition (2, 4, 6, 8 times) What is the dependent variable? How is it measured? memory - familiar or novel, recognition errors recorded Why is this type of design (between- or within-subjects) appropriate for this experiment? variability between subjects would be greater than variability within
A psychologist tested the effect of stimulus type and distraction on recall. Participants (N = 20) studied picture lists and word lists for one minute each while either distracted or not distracted. Each picture list included simple line drawings of 20 common objects (e.g., a chair, hammer, shirt). Each word list included the names of 20 common objects. All participants studied two picture lists and two word lists. For one picture list and one word list participants studied without distraction. For the other picture list and word list participants were distracted, as they had to press a button each time they heard a beep. A three minute waiting period followed each study session. Participants then recalled as many objects from the list of pictures and list of words as possible. Is this a between-subjects or within-subjects design? How do you know? What is (are) the independent variable(s)? How many levels does it have? Name them. What is the dependent variable? How is it measured? Why is this type of design (between- or within-subjects) appropriate for this experiment?
Is this a between-subjects or within-subjects design? How do you know? within subjects What is (are) the independent variable(s)? stimulus type and distraction How many levels does it have? Name them. 2 levels each (pics vs words and distracted vs not distracted) What is the dependent variable? How is it measured? recall - number of items recalled Why is this type of design (between- or within-subjects) appropriate for this experiment? everyone experiences all levels of treatment, don't have to worry about order/carryover effects --- individual differences create systematic differences so within group designs eliminate systematic error
Ten participants answered true or false when presented with simple arithmetic equations. Some equations were obviously true, for example, 3 + 4 = 7 or 5 x 2 = 10. Others were obviously false, for example, 6 ‑ 1 = 4 or 3 x 5 = 9. Others were false, but could be true if the arithmetic function was misinterpreted, for example 2 + 8 = 16 or 7 x 2 = 9. The equations were presented at a computer console and the participants' time to respond was measured. The participants responded to 60 equations, twenty for each of the three types of equations. The *average time to respond*, in milliseconds, across the 20 equations of each type was recorded and included in the analysis. Is this a between-subjects or within-subjects design? What is the independent variable? How many levels does it have? Name them. What is the dependent variable? How is it measured? Why is this type of design (between- or within-subjects) appropriate for this experiment? Report the results.
Is this a between-subjects or within-subjects design? between subjects What is the independent variable? type of arithmetic equation How many levels does it have? Name them. 3 levels - obviously true, obviously false and easily misinterpreted What is the dependent variable? How is it measured? time to respond - computer program Why is this type of design (between- or within-subjects) appropriate for this experiment? between - practice effects Report the results.
Five participants repeatedly said the alphabet backwards. As they were reciting the alphabet, a light was turned on, and then turned off. As soon as the light went off participants wrote down how long, in seconds, they estimated it had been on. During the presentation sequence the light was on six times at each of four illumination intervals: 12, 24, 36 or 48 seconds. Participants therefore estimated the duration of 24 illumination trials. The participants' average estimate, across the six presentations of each illumination interval, was recorded and included in the analysis. Is this a between-subjects or within-subjects design? What is the independent variable? How many levels does it have? Name them. What is the dependent variable? How is it measured? Why is this type of design (between- or within-subjects) appropriate for this experiment? Report the results.
Is this a between-subjects or within-subjects design? within - single factor What is the independent variable? time light was on How many levels does it have? Name them. 4 - 12, 24, 36, 48 seconds What is the dependent variable? How is it measured? estimated duration light was on Why is this type of design (between- or within-subjects) appropriate for this experiment? Report the results.
Ten participants answered true or false when presented with simple arithmetic equations. Some equations were obviously true, for example, 3 + 4 = 7 or 5 x 2 = 10. Others were obviously false, for example, 6 ‑ 1 = 4 or 3 x 5 = 9. Others were false, but could be true if the arithmetic function was misinterpreted, for example 2 + 8 = 16 or 7 x 2 = 9. During the presentation sequence the light was on six times at each of four illumination intervals: 12, 24, 36 or 48 seconds. Participants therefore estimated the duration of 24 illumination trials. The participants' average estimate, across the six presentations of each illumination interval, was recorded and included in the analysis. Is this a between-subjects or within-subjects design? What is the independent variable? How many levels does it have? Name them. What is the dependent variable? How is it measured? Why is this type of design (between- or within-subjects) appropriate for this experiment? Report the results.
Is this a between-subjects or within-subjects design? within - single factor What is the independent variable? type of arithmetic equation How many levels does it have? 3 levels - obviously true, obviously false and easily misinterpreted What is the dependent variable? How is it measured? time to respond Why is this type of design (between- or within-subjects) appropriate for this experiment? math ability could vary between groups Report the results.
Why worry about level of measurement?
Levels of measurement vary in precision. More precise measures provide greater statistical power. I need a short board. I need a shorter board. I need a 4 ft. board. One's level of measurement affects the types of statistical analyses possible.
Concerns with reversal designs
Methodological: failure to reverse Ethical: discontinuation of an effective treatment
What are the advantages of expanding a simple experiment to include more levels of the independent variable? Specifically, address the benefits in terms of - generalizing results to a range of levels of the independent variable - construct validity
Multi-level experiments have more external validity than simple experiments
Each of 96 introductory psychology students tried to learn 20 unfamiliar English vocabulary words. Participants were randomly assigned in equal numbers to each group. Participants studied the vocabulary words either along with: a familiar synonym (synonym) a dictionary definition (definition) a literary passage in which the word was used correctly (passage) or without any accompanying information (control) In addition, half the participants in each group wrote sentences using the new words during the study period (active processing) while the other half of the participants simply studied without writing (passive processing) What is/are the independent variable(s)? How many levels does each independent variable have? What are they? What is/are the dependent variable(s)? What are the sources of between-group variability for this experiment? Therefore, how many F ratios should we calculate? What would be a logical means for presenting the scores on the dependent variable? Why? Which mean values would you compare to examine the overall main effect of the different studying conditions on vocabulary learning? Which mean values would you compare to examine the overall main effect of the different processing types on vocabulary learning? Which differences would you compare to examine the interactive effect of studying and processing on vocabulary learning?
Multiple-factor between-subjects design What is/are the independent variable(s)? study strategy, active vs passive processing How many levels does each independent variable have? What are they? 4 levels: synonym, definition, passage, control What is/are the dependent variable(s)? number of words remembered What are the sources of between-group variability for this experiment? Therefore, how many F ratios should we calculate? study strategy, processing, interacts What would be a logical means for presenting the scores on the dependent variable? Why? column means, row means Which mean values would you compare to examine the overall main effect of the different studying conditions on vocabulary learning? Which mean values would you compare to examine the overall main effect of the different processing types on vocabulary learning? Which differences would you compare to examine the interactive effect of studying and processing on vocabulary learning?
Children in three age groups (1 year, 2 years, and 3 years) were brought into a laboratory, and placed in a play yard. After a certain time interval had passed (5 min., 10 min., 15 min., 20 min.), a "strange" experimenter entered the room, and picked the child up. Each child was tested at each of the four time intervals. Half of the children were picked up by a female experimenter, and the other half by a male experimenter. Each child was always picked up by the same experimenter. Each time the child was picked up, a hidden rater evaluated the amount of distress exhibited by the child, using a scale from 1 to 10, with 1 indicating the least distress, and 10 indicating the most distress. 1. identify each independent and/or predictor variable (i.e., distinguish between manipulated variables and subject variables) and dependent variable 2. tell how many levels there are of each IV/PV and what those levels are 3. if the design is factorial, give the design description (e.g., 2 x 3) 4. identify which variable(s) is/are between subjects, and which is/are within subjects 5. state (based on your answers to #4) whether the design is between-subjects, within-subjects, or mixed
PV - 3 age groups DV - amount of distress IV - time interval( 4 levels), gender of experimenter (2 levels) Design - 2 (gender) x 3 (age) x 4 (time) Nature of manipulation - time = within, experimenter = between, age = between Mixed design
In a recent replication of some of Schacters (1959) work, experimenters examined the question "does misery love company?"; Male and female students were tested for anxiety, and each gender group was divided into those who were "low-anxious," and those who were "high-anxious." The "low- anxious" participants in each group were told by a warm, friendly experimenter that they would receive mild shocks during the experiment, which was to occur shortly. The "high-anxious" participants in each group were told by a cold, unfriendly experimenter (who was the same person as the experimenter in the other group) that they would receive painful shocks during the experiment, which was to occur shortly. Participants in both groups were then told that while the experimental equipment was being set up they could choose to wait alone or with someone else. The response measure of interest was whether they would choose to wait alone or in the company of another, to test the hypothesis that anxiety is related to affiliation. In addition, the researchers were interested in knowing whether females and males would differ in their tendency to affiliate. 1. identify each independent and/or predictor variable (i.e., distinguish between manipulated variables and subject variables) and dependent variable 2. tell how many levels there are of each IV/PV and what those levels are 3. if the design is factorial, give the design description (e.g., 2 x 3) 4. identify which variable(s) is/are between subjects, and which is/are within subjects 5. state (based on your answers to #4) whether the design is between-subjects, within-subjects, or mixed
PV - gender (2 levels), anxiety level (2 levels) IV - situational anxiety (experimenter - 2 levels) DV - affiliation Design - 2 (gender) x 2 (dispositional anxiety) x 2 (situational anxiety) Nature of manipulation - gender = between, dispositional anxiety = between, situational = between Between subjects
When to use a between-subjects design
Participant/subject availability: when many are available Task type: long duration, not reversible, skill development, memory, surgery Sensitivity: when effect/difference is expected to be large (increased sensitivity = higher power)
ratio measures
Ratio measures: Scale begins at a true/real zero, (0°Kelvin is an absolute zero, absence of molecular motion). Numbers in a ratio scale allow one to specify identity, rank (order), interval distance or difference, and how many times greater or less than one item is than another Example: 2:00, 2:05, 2:10, 2:20 Age Weight Time Deg. Kelvin 20C is not 2x 10C because you don't start at an absolute zero
type 1 error
Rejecting null hypothesis when it is true
Issues related to the process of doing research
Risk Consent Deception Privacy Benefits Animal subjects
MSwithin
SSwithin/dfwithin measures the variability in the DV due only to error
Quasi-experimental designs
Semi- or partial-experimental design that attempts to maximize internal validity while markedly increasing external validity Quasi refers to amount of experimental control available to the researcher, and, as a result, the confidence with which one can assert causal relationships no random assignment
The value of within-subjects designs
Sensitivity and accounting for variability due to participants
Single-subject and quasi-experimental designs
Single subject designs: reversal design, multiple baseline design Quasi-experimental designs: interrupted time series design, multiple time series design
Modified Bonferroni
The Bonferroni correction is used to reduce the chances of obtaining false-positive results (type I errors) when multiple pair wise tests are performed on a single set of data. Put simply, the probability of identifying at least one significant result due to chance increases as more hypotheses are tested. WITHIN SUBJECTS DESIGN
Two professors of experimental psychology were interested in the effect of type of instruction and student ability on mastering multifactor between-subjects experimental designs. Students either attended lectures and completed daily homework assignments or attended lectures, completed daily homework assignments, and spent one additional hour solving illustrative problems under the guidance of the professor in small‑group tutoring sessions. Based on admissions records, the professors knew that ten of the students had superior mathematical ability, ten had average ability, and ten had below average ability. Half the students at each ability level were randomly assigned to each of the instructional methods. Mastery was measured by performance on a test given at the end of the unit. Given the research discussed in each example, why is a between-subjects design appropriate? How many independent variables are included in the example? Name them. Identify the number and nature of the levels of each independent variable. Describe the design of the experiment (i.e., single-factor analysis of variance, ? x ? factorial design). Justify/criticize the number of independent variables included in the study. Justify/criticize the number of levels chosen for each independent variable. Identify the dependent variable. Identify the sources of variability for each example. State the conclusions justified by the results. Discuss any limitations one must acknowledge given the nature of the study and its results.
Why between subjects? can't tell which one caused the effect, carryover effect How many IV's? What are they? What are the levels? type of instruction (2 levels) PV - student ability Design of experiment? between subjects Justification/criticize # of IV's and levels chosen: DV: mastery material Variability: State conclusions: Main effect of problems vs no problems and student ability Limitations:
A physiological psychology class compared the stress induced by various situations. The hormone corticosterone was used as a measure of stress, measured in milligrams per 100 milliliters of blood. Five students had their level of corticosterone assessed immediately upon returning from summer vacation (M = 4.0 mg/100 ml, s = .47). Five students had their level of corticosterone assessed after completing the second week of class (M = 8.0 mg/100 ml, s = .68). Five more students had their level of corticosterone assessed immediately before final exam week (M = 13.0 mg/100 ml, s = .38). All measures were taken at the same time of day. Given the research discussed in each example, why is a between-subjects design appropriate? How many independent variables are included in the example? Name them. Identify the number and nature of the levels of each independent variable. Describe the design of the experiment (i.e., single-factor analysis of variance, ? x ? factorial design). Justify/criticize the number of independent variables included in the study. Justify/criticize the number of levels chosen for each independent variable. Identify the dependent variable. Identify the sources of variability for each example. State the conclusions justified by the results. Discuss any limitations one must acknowledge given the nature of the study and its results.
Why between subjects? carryover effects How many PV's? What are they? What are the levels? PV - time of school year Design of experiment? correlational study Justification/criticize # of IV's and levels chosen: CV: stress level Variability: State conclusions: Limitations:
An exercise physiologist is interested in the effects of amount and timing of exercise on nighttime sleep. College student participants engaged in either light, moderate, or heavy exercise (jogging 0.25, 1, or 3 miles). They exercised either first thing in the morning (7:30 AM) or shortly before going to bed (7:00 PM). Thirty-six participants, all in good overall physical condition, were randomly and equally distributed among the groups. Participants maintained their schedules for two weeks. Each morning they judged the quality of the previous night's sleep using a ten-item questionnaire. Given the research discussed in each example, why is a between-subjects design appropriate? How many independent variables are included in the example? Name them. Identify the number and nature of the levels of each independent variable. Describe the design of the experiment (i.e., single-factor analysis of variance, ? x ? factorial design). Justify/criticize the number of independent variables included in the study. Justify/criticize the number of levels chosen for each independent variable. Identify the dependent variable. Identify the sources of variability for each example. State the conclusions justified by the results. Discuss any limitations one must acknowledge given the nature of the study and its results.
Why between subjects? fatigue effects How many IV's? What are they? What are the levels? amount of exercise (3 levels) and time of exercise (2 levels) Design of experiment? 3 (amount of exercise) x 2 (time length of exercise) factorial design Justification/criticize # of IV's and levels chosen: only two levels, there may be a different moderate time that shows a different graph, most people don't go to bed shortly after 7 DV: quality of sleep Variability: athleticism, insomnia State conclusions: Limitations:
To determine the best way to present an important topic to her class, a professor used a different teaching method with each of three groups of nine students. Students from the class were randomly assigned to the three groups. Scores on a test one week later were used as a measure of instructional effectiveness. One group listened to a lecture (M = 85.2, s = 3.1). Another group listened to the lecture and completed some supplemental readings (M = 91.0, s = 2.3). A third group watched a film and completed some supplemental readings (M = 81.6, s = 5.7). Given the research discussed in each example, why is a between-subjects design appropriate? How many independent variables are included in the example? Name them. Identify the number and nature of the levels of each independent variable. Describe the design of the experiment (i.e., single-factor analysis of variance, ? x ? factorial design). Justify/criticize the number of independent variables included in the study. Justify/criticize the number of levels chosen for each independent variable. Identify the dependent variable. Identify the sources of variability for each example. State the conclusions justified by the results. Discuss any limitations one must acknowledge given the nature of the study and its results.
Why between subjects? gave three methods to one person wouldn't be able to tell which was the cause, carryover effects How many IV's? What are they? What are the levels? teaching method (3 levels) Design of experiment? between subjects Justification/criticize # of IV's and levels chosen: DV: effectiveness Variability: State conclusions: Limitations:
A clinical psychologist at a college counseling center is interested in the effects of anxiety and test difficulty on test performance. She has identified three groups of 15 students, each group manifesting a different level of trait anxiety (slight, moderate, excessive), as measured by the Taylor Manifest Anxiety Scale. She has located three tests of cognitive functioning. The tests differ in level of difficulty (low, medium, high). One third of the students from each group of 15 are randomly assigned to complete each of the three tests. Given the research discussed in each example, why is a between-subjects design appropriate? How many independent variables are included in the example? Name them. Identify the number and nature of the levels of each independent variable. Describe the design of the experiment (i.e., single-factor analysis of variance, ? x ? factorial design). Justify/criticize the number of independent variables included in the study. Justify/criticize the number of levels chosen for each independent variable. Identify the dependent variable. Identify the sources of variability for each example. State the conclusions justified by the results. Discuss any limitations one must acknowledge given the nature of the study and its results.
Why between subjects? practice effects How many PV's? What are they? What are the levels? anxiety (3 levels), test difficulty Design of experiment? between subjects 3 (anxiety level) x 3 (test difficulty) Justification/criticize # of IV's and levels chosen: CV: stress level Variability: State conclusions: Limitations:
single subjects design disadvantages
Without a comparison group, do not know if measured value differs (and how) from untreated controls Do not know why the measured value differs from untreated controls, given possible alternative explanations
Four types of persuasive messages (30 and 60 second television public service announcements) were used to try to convince people to stop smoking. Facts: statistical information relating smoking to disease Fear: emotionally compelling first-person accounts of smoking-related disease Humor: humorous depictions of smoking and smoking related behavior Hope: emotionally compelling first-person accounts of quitting smoking Sixteen people were randomly assigned to each of the four groups and each group received one of the message types. The dependent variable was the number of cigarettes each participant reported smoking in the week following exposure to the persuasive messages. What is the independent variable? How many levels of the independent variable are there? What are they? What is the dependent variable? Why is a between‑subjects design appropriate for this investigation? Why are there more than two groups? What is the null hypothesis? What is the alternative/experimental hypothesis?
single-factor between-subjects design IV: type of persuasive message Levels: 4 - facts, fear, humor, hope DV: dependent variable was the number of cigarettes each participant reported smoking in the week following exposure to the persuasive messages Why between subjects? if you gave 1 person all videos we may not know what the cause, carryover effects Why are there more than two groups? qualitative IV Null hypothesis: type of persuasive video does not have an effect on number of cigarettes smokes (all means are equal) Alternative hypothesis: not H0 (type of video will have an effect)
SS between
subtract grand mean from each condition mean, square these differences, and multiply them by the sample size for that condition. Add all of these values together Variability due to everything AND the IV Sum of [n](Group mean - Grand mean)^2
SStotal
sum of squared deviations of each score from the overall mean of all scores, completely ignoring the group a score is in Sum of (X-Grand mean)^2
sum of squares equation
sum((yactual-ycalculated).^2)