Stat121 lesson 5

What are the 3 principals of good experimental design?

1. Control 2. Randomization 3. Replication

The basic principles of statistical design of experiments are (3 things)?

1. control (the effects of lurking variables on the response, most simply by comparing two or more treatments). 2. Randomize (use chance to assign subjects to treatments). 3. Use enough subjects (in each group to reduce chance variation in results).

What is replication in statistics and why do we need it?

1. replication is assigning of at least 2 individuals to a treatment group. 2. Having enough individuals in a group will allow one to measure for chance variation (the more in a group, the less chance can play on the group).

What is a block?

A group of individuals that have some characteristic in common. The similarity was known before the experiment begins.

What type of survey is most similar to a randomized block design?

A stratified sample

What is meant by lack of realism (3 things)?

A. Subjects may not be from the population we wish to study B. Treatments may not be those we actually want to study C. Experimental setting may not duplicate the situation we desire

Fencing in all cows wearing insecticide ear tags and allowing all cows wearing placebo ear tags to roam free creates a bad condition. What is the name of this condition?

Confounding. The explanatory variable, insecticide ear tag versus placebo ear tag, is confounded with the lurking variable, fenced-in or not.

True or false: Realism can always be achieved in an experiment.

False. Due to ethical reasons, realism may not be possible (can't use an actual pregnant woman to study the effect of smoking vs. non-smoking on a fetus. must use mice).

A study was conducted to test the effectiveness of using an antidepressant called imipramine in treating bulimia, an eating disorder. Twenty patients were randomly assigned to one of two groups with ten in each. One group received imipramine and the other received a placebo. The response measured was binge frequency. What type of study is this?

First of all, since the treatments (imipramine and placebo) were assigned to the subjects, this was an experiment. Second, since the randomization was done using all twenty subjects and allocating them to treatments, this was a completely randomized experiment.

Gas mileage was compared for Premium and Regular gas for twenty Toyota Prius's. (A Prius is a new hybrid car.) Each car was run on a tank of Premium gas and on a tank of Regular gas. Before the study, a coin was tossed for each car. If the coin was heads, the car was first tested with Premium gas; if the coin was tails, the car was first tested with Regular gas. At the end of the study, average miles per gallon for the two types of gas was compared. What type of study is this?

First of all, this is an experiment because the type of gas was assigned, not merely observed. Second, since the randomization is done for each car and there are two measurements on each car, this is a matched pairs experiment. Note: We say that we randomized within each pair (i.e., for each car.)

What problem is reduced when the doctor does not know who received the placebo and who received the active treatment?

Hidden bias is reduced. Doctors diagnosing the response may be influenced by knowing which subject receives the placebo and which subject received the active treatment. This may bias their diagnosis.

How is randomization carried out in a randomized block design?

Individuals are randomly allocated to treatments with each block.

What is a consequence of lack of realism?

Lack of realism results in bias.

When are results statistically significant?

Most people think that significant mean important. But in statistics, the term "significant" implies "too big" to be due to chance.

I a double blind experiment, the subject/those who interact with the subject/both/neither knows which treatment each subject is receiving (whether placebo or actual treatment).

Neither the subject or those who interact with the subject know if they are getting the actual treatment or the placebo.

What problem is reduced by giving a placebo to subjects in a control group?

Placebo effect. If subjects in an active treatment group received a "pill" and subjects in the control group did not, the subjects in the active treatment group may respond to the "pill" and not to the treatment. This results in a placebo effect.

Why do experiments need randomization?

Randomization, whether in a survey or in an experiment, reduces bias.

Which principle is implemented differently for a randomized block design than for a completely randomized design?

Randomization. In a randomized block design, randomization is carried out within each block whereas for a completely randomized design, randomization is carried out for all individuals available for the experiment.

Referring to question 5 (bandaid removal), what is the replication?

Since this is a matched pairs experiment and fifty subjects participated, we have 50 measurements for each type of removal. Thus, fifty is the replication.

Why is using litters as blocks a good idea in the rat/maze study? - think of what the rats looked like in the picture shown in the actual quiz

Size of rat is expected to affect time to complete the maze.

In double blind, the subjects do not know which treatment they receive. Who else is blinded in a "double-blind" study?

The doctor or nurse who diagnoses the response

Question 5: What is the response variable? Does ripping off a band-aid hurt less than slowly pulling it off? Researchers plan to compare the two methods by applying two medium sized band-aids to the underside of the right forearm of each of 50 volunteer subjects. One band-aid will be randomly selected to be pulled off with a quick yank while the other will be gently pealed off. The order in which the band-aids are removed will also be randomized. After each band-aid is removed, the subject will be asked, "On a scale of 0 to 10, how much did the removal hurt?--with zero for "no pain" and 10 for "the worst pain imaginable." What is the response variable?

The response measured on each subject after the treatment is applied. So, it is pain score

People who eat lots of fruits and vegetables have lower rates of colon cancer than those who eat little of these foods. Fruits and vegetables are rich in "antioxidants" such as vitamins A, C, and E. Will taking antioxidants help prevent colon cancer? A medical experiment studied this question with 864 people who were at risk of colon cancer. The subjects were divided into four groups: daily beta-carotene, daily vitamins C and E, all three vitamins (beta-carotene, vitamins C and E) every day, or daily placebo. After four years, the researchers were surprised to find no significant difference in colon cancer rates among the groups. What is the explanatory variable?

There are four treatments: daily beta-carotene, daily vitamins C and E, all three vitamins (beta-carotene, vitamins C and E) every day, or daily placebo. So the explanatory variable is which vitamins (if any) were received daily.

Referring to question 5 (bandaid), what type of study is this?

This is a matched pairs study because two measurements were taken on each subject—one pain score for the quick yank removal and the other for the gently peel removal. The order in which the treatments were applied was randomized. Subjects did not choose order of type of removal.

Referring to question 13 (vitamins and colon cancer), what type of study is this?

This is an experiment since treatments are assigned to the subjects rather than subjects self-selecting their treatments. Since all subjects are randomly allocated to the treatments, this is a completely randomized design. Randomization is carried out for all subjects and not within certain groups.

Why is the principle of control/comparison so important in an experiment?

To neutralize the effect of lurking variables and measure treatment differences.

True or false: A matched pairs design is a special case of a randomized block design.

True

True or false: In a randomized block design, treatments are randomly assigned to subjects within each block.

True

True or false: Principles of control/comparison and randomization balance out the effect of lurking variables.

True

True or False: a statistically significant association in data from a well designed experiment does imply causation.

True.

True or false: If subjects are randomly assigned to groups, then differences in average response can be attributed to either the effects of treatments or to chance.

True.

True or false: In a matched pairs experiment, control/comparison is implemented with comparing two treatments.

True. One of the treatments could be a control (placebo) treatment.

True or false: Hidden bias can be reduced by treating all subjects identically.

True. To eliminate bias always treat all subjects the same.

True or false: A randomized block design should be used when the subjects within groups (called blocks) are similar in ways that affect the response variable, but different from one block to the next.

True. When the subjects are similar within a block, but different from block to block, a randomized block design is recommended. This allows us to remove differences or variation associated with blocks from the experimental error. A completely randomized design should only be used when the subjects are as similar as possible in ways that affect the response variable.

What is the biggest advantage of a randomized block design (RBD) compared to a completely randomized design (CRD)?

Variation associated with the blocking variable is removed from chance variation, resulting in a more precise estimate of chance variation.

Why is the randomization carried out within each litter?

We recognize an experiment as being a randomized block design when the randomization is carried out within each group.

When can a completely randomized design be performed?

We should only perform a completely randomized design when the individuals are similar with respect to variables that might affect the response variable.

When is a randomized block design recommended?

When individuals are similar within a block, but very different from block to block.

When is a completely randomized design ok?

When the individuals are similar.

When should a randomized block design be performed?

When the individuals available for the study belong to groups before the study begins. And the individuals within each group are similar, but different from group to group.

How is randomization of pairs of individuals carried out in a matched pairs design?

When the pair consists of individuals, randomization must be carried out within the pair (block). So the individuals in each pair must be randomly allocated to the two treatments.

Referring to question 13 (vitamins and colon cancer), what is the response variable? Remember: the response variable is the measurement variable on the individual (not the group) at the end of the study.

Whether or not each subject contracted colon cancer.

How is control/comparison implemented in the rat/drug/maze experiment?

With the six litters. Control/comparison requires either a control versus an active treatment group OR two or more active treatment groups or both. In this experiment, five active treatments (five drugs) will be compared so it has comparison.

When litters are used as blocks, how should rats be assigned to drugs?

With this randomized block design, rats should be randomly assigned to drugs within each litter.

What is the purpose of a control in an experiment?

Without a control group, we cannot know whether the response is due to the active treatment or to chance or to something else.

Why do experiments need replication?

Without replication, we cannot measure chance variation.

30 rats. Subject=Newly weened. Explanatory variable=type of treatment given (drug a, b, c, d, or e). Response variable=time to complete maze. Rats are different sizes, lengths, and breeds. Will size of rat affect time to complete a maze? There are six different litters. These 6 litters make up the 6 blocks.

Yes.

In addition to hidden bias, placebo effect and lack of realism, could there be other problems in experiments?

Yes. Factors that the researcher cannot control such as bad weather, drop outs, death of animals, etc., can create problems in experiments.

What is a block in a matched pairs design?

a pair of measurements or a pair of individuals

Name some potential problems the study described in question 5 (bandaid removal).

a. Amount of hair a subject has may affect the degree of pain. c. Results may be biased because there is no blinding of subjects as to when they receive each treatment. d. Subjects may already prefer one method of removal over another so their pain score report may be biased. e. The study was not double blind. In fact, it wasn't even single blind. f. The person doing all of the quick yanks may not be able to do them exactly the same for all subjects.

What is statistical significance?

an observed effect so large that it would rarely occur by chance. It is too large to attribute plausibly to chance variation.

Which one of the following is a benefit of randomized block designs (RBD)? a. RBD removes bias. b. RBD reduces chance variation by removing variation associated with the blocking (lurking) variable. c. RBD eliminates the placebo effect. d. RBD eliminates all lurking variables.

benefit of RBD is it reduces chance variation by removing variation associated with the blocking (lurking) variable.

In an experiment, how do researchers form groups that are similar before treatments are applied?

by allocating individuals to groups using a random device. Random devices will create groups that are similar in all respects.

RBD removes ____________ or _____________ variables with response variables.

confounding or lurking

Two groups of cows were tagged with an ear tag containing insecticide or placebo. They were testing to see if insecticide on ear tag removed rates of stuff and hoopla. Yadda yadda yadda. In addition, one group was kept within a fence, and the other roamed free. What hidden bias is present?

cows in fenced in areas attract more insects.

A response variable is a variable that...?

it is a variable that measures an outcome of a study

Referring to question 5 (bandaid removal), what is the explanatory variable?

method of bandaid removal (explanatory describes all treatments)

What principle is fulfilled by randomly assigning rats to drugs within each litter?

randomization

What is observed effect?

the difference between what we see in the data and what we expect to see in the data.

How is replication implemented in a matched pairs design?

the number of pairs is the replication

What is a solution to remove hidden bias in the cow ear tag experiment?

treat all cows the same - either fence in all cows or allow all cows to roam free.

What is a good solution for lack of realism?

using common sense is the best alternative when important information is needed, but ethically experiments can actually be done on the true target subject (again, think pregnant woman and smoking vs. non-smoking).

How is replication fulfilled in this randomized block design? A. With the five drugs B. With the 30 rats C. With the six litters

with the 6 litters. Since there are six litters and one rat in each litter gets each drug, six rats receive each drug. So, the replication is the six rats receiving each drug.