AP Stats Experimental Design #1-30
An advantage to using surveys as opposed to experiments is that A. Surveys are generally cheaper to conduct. B. it is generally easier to conclude cause and effect from surveys. C. surveys are generally not subject to bias. D. Surveys involve use of randomization. E. surveys can make user of stratification.
A. Surveys are generally cheaper and quicker to conduct than experiments; however surveys are subject to bias, and it is very difficult to conclude cause and effect from surveys. Experiments also use randomization in the form of random assignment to treatments. Blocking in experimental design corresponds to stratification in sampling design.
A newspaper advice columnist asks her readers if they would have married their current spouse if they had it to do over again. Of the 25,000 or so responses, 80% said no. What does this show? A. The survey is meaningless because of voluntary response bias. B. No meaningful conclusion is possible without knowing something more about the characteristics of her readers. C. The survey would have been more meaningful if she had picked a random sample of the 25,000 readers who responded. D. The survey would have been more meaningful if she had used a control group. E. This was a legitimate sample, randomly drawn from her readers, and of sufficient size to allow the conclusion that most of her readers who are married would have second thoughts about marrying their current spouse.
A. This is an example of voluntary response bias, which often over-represents negative opinions. The people who chose to respond were most likely those who were unhappily married, and so there is very little chance that the 25,000 respondents were representative of the population. Knowing more about her readers, or taking a sample of the sample, would not have helped.
In general, for a survey to yield usable results: A. A sample size of n = 30 is usually sufficient. B. Researchers must be careful in the way questions are worded. C. Researchers must carefully choose people who they think are representative of the population. D. A census is the only truly accurate methodology. E. Sampling error must be avoided.
B. Care must be taken with regard to the wording of questions so as to avoid leading questions and thereby RESPONSE BIAS. Wording of questions must be neutral as subjects will give different answers depending on phrasing. Allowing researchers to choose people they think are representative is a prescription for disaster - random sampling of the population is best. A census is not worth the effort if wording leads to bias. Sampling error, also called sampling variation, is the natural variation between samples.
In a study of Parkinson's disease, 100 volunteers had incisions made through their skulls. The patients were randomly sorted into two groups, one of which had a new drug inserted in the brain. In the other group, the skulls were closed with no treatment given. The patients did not know who received the drug. In the weeks to follow all 100 volunteers showed improvement in physical condition. What is this an example of? A. The effect of a treatment unit B. The placebo effect C. The control group effect D. Sampling error E. Voluntary response bias
B. Patients believing they have received the new drug may be able to somehow alleviate their own symptoms. Such a medical effect based on the power of suggestion is called a placebo effect.
To conduct a survey on holiday shopping patterns, a researcher opens a telephone book to a random page, closes his eyes, puts his finger down on the page, and then reads off the next 100 names. Which of the following is not a true statement? A. The survey incorporates chance. B. The procedure results in a systematic sample. C. The procedure could easily result in selection bias. D. The procedure is not an SRS. E. The use of a phone book will result in undercoverage bias.
B. This is not a systematic sample because it does not pick every nth name on the list. There is a very real chance of selection bias. All possible groups of size 100 do not have the same chance of being picked, so it is not an SRS. Undercoverage is present because some people have only cell phones.
Two studies are run to compare the experiences of low-income families receiving food stamps to those receiving cash subsidies. The first study interviews 50 families who have been in each government program for at least 2 years, while the second randomly assigns 50 families to each program and interviews them after 2 years. Which of the following is a true statement? A Both studies are observational studies because of the time period involved. B. Both studies are observational studies because there are no control groups. C. The first study is an observational study; the second is an experiment. D. The first study is an experiment; the second is an observational study. E. Both studies are experiments, because in each, families are receiving treatments (food stamps or cash).
C. In the first study ,the families were already receiving assistance; in the second study, one of two treatments was applied to each family.
Which of the following is most important in minimizing the placebo effect? A. Replication and randomization B. Replication and blinding C. Randomization and blinding D. Randomization and a control E. Blinding and a control
E. Use of a control group and blinding as to which subjects are in the control group are the best tools to minimize the possibility of confounding due to the placebo effect. Replication and randomization are important marks of good experimental design, but they do not impact the placebo effect as does the use of a control and blinding.
Two anti depressants are to be compared in the treatment of elderly patients in a nursing home. Each patient has his or her own room, some with spectacular views of the ocean. The experimental design is to create homogeneous blocks with respect to window view. How should randomization be used for a randomized block design? A. Within each block, randomly pick half the patients to receive each anti-depressant. B. Randomly pick half of all patients to receive each antidepressant, but then analyze the results separately by blocks. C. Randomly choose which blocks will receive which antidepressant. D. Randomly choose half the blocks to receive each antidepressant for a given time period; then for the same time period switch the medication in each block and compare the results. E. For ethical reasons, allow patients to choose which medication they prefer taking, but then randomly assign patients to the blocks.
A. In a randomized block design, we assign subjects to treatments at random within each of the homogeneous blocks. In effect, to reduce variability we run parallel experiments on the blocks.
Two antidepressants are to be compared in the treatment of elderly patients in a nursing home. Each patient has his or her own room, some with spectacular views of the ocean. The experimental design is to create homogeneous blocks with respect to window view. How should randomization be used for a randomized block design? A. Within each block, randomly pick half the patients to receive each antidepressant. B. randomly pick half of all patients to receive each antidepressant, but then analyze the results separately by blocks. C. Randomly choose which blocks will receive which antidepressant. D. Randomly choose half the blocks to receive each antidepressant for a given time period; then for the same time period switch the medication in each block and compare the results. E. For ethical reasons, allow patients to choose which medication they prefer taking, but then randomly assign patients to the blocks.
A. In a randomized block design, we assign subjects to treatments at random within each of the homogeneous blocks. In effect, to reduce variability we run parallel experiments on the blocks.
A researcher planning a survey of heads of households in New York has census lists for each of the 62 counties in the state. The procedure will be to obtain an SRS of heads of households from each of the counties rather than grouping all the census lists together and obtaining a sample from the entire group. Which of the following is not a true statement about the resulting stratified sample? A. It is more susceptible to bias than would be an SRS. B. It is easier and more cost effective than an SRS. C. It gives comparative information that an SRS wouldn't give. D. It recognizes that opinions of heads of household in rural NY communities may differ from those in urban communities. E. All of the above are true statements.
A. Stratified samples are often easier and more cost effective to obtain and also make comparative data more available. In this case, responses can be compared in and between various counties. Rural and urban counties are less varied WITHIN each county than the population as a whole, and stratified sampling recognizes this. There is no reason to suspect any more bias in this stratified sample than would be present in an SRS.
A food judge is given an assignment to choose and sample the food at 52 (out of over 20,000) NYC restaurants. She has an assistant list all restaurants whose name begins with A, assigns each a number, and uses a random number generator to pick two of these numbers and thus two restaurants. She proceeds to use the same procedure for each letter of the alphabet and combines the results into a group of 52. Which of the following is a true statement? A. Her procedure makes use of chance. B. Her procedure results in a simple random sample. C. Each restaurant in NYC has an equal probability of being selected. D. The restaurant Moosewod probably has a higher probability of being selected than the restaurant Xander. E. This is an example of a systematic sample.
A. This will not result in a simple random sample because each possible set of 52 restaurants does not have the same chance of being selected. For example, a group of 52 restaurants whose names all started with A will not be chosen. Even though the food judge, does use chance, each restaurant would have the same chance of being selected only if the same number of restaurants have names starting with each letter of the alphabet. There are probably many more restaurants whose name begins with M than with X, and with only two chosen per letter, Moosewood has less chance to be selected. Systematic sampling involves a random starting point followed by picking every nth subject from an ordered list.
Which of the following statements is incorrect? A. Voluntary response samples often under-represent people with strong opinions. B. Convenience samples often lead to undercoverage bias. C. Questionnaires with nonneutral wording are likely to have response bias. D. There is no way to fix the results if a biased sampling method was employed. E. Nonresponse bias should be avoided because those who do not respond might have different views from those who do respond.
A. Voluntary response samples, like radio call-in surveys, are based on individuals who offer and they typically over-represent persons with strong opinions. Convenience samples, like shopping mall surveys, are based on choosing individuals who are easy to reach, and they typically miss a large segment of the population. Non-neutral wording can readily lead to response bias, and if surveyors want a particular result, they deliberately use certain wording. Always check carefully for bias before collecting data because there is no recovery from a biased sample.
A botanist is running an experiment on two fertilizers that require different amounts of watering. She has 40 test plots, half of which are in sunny locations, and half are in the shade. She randomly selects 10 sunny plots and 10 shady plots for which to use one fertilizer with its correct watering, while the remaining plots are for the other fertilizer with its correct watering. Of the following, which is the most important observation about this procedure? A. The variables, fertilizer and water, are confounded. B. The variables, fertilizer and sun, are confounded. C. The variables, water and sun, are confounded. D. No variables are confounded. E. There is a hidden lurking variable.
A. when there is uncertainty with regard to which variable is causing an effect, we say the variables are CONFOUNDED. In this experiment, it might be difficult to determine if the observed changes in plant growth are due to difference in fertilizers or difference in watering. With both sunny and shady plots for each fertilizer, fertilizer and sun are not confounded.
A company wishes to survey what people think about a new product it plans to market. They decide to randomly sample from their customer database as this includes phone numbers and addresses. This procedure is an example of which type of sampling? A. Cluster B. Convenience C. Simple random D. Stratified E. Systematic
B. Convenience samples are based on choosing individuals who are easy to reach. A typical example is sampling based on interviews at a shopping mall. Data obtained from convenience samples tends to be highly unrepresentative of the entire population. In this example, while using the company database is convenient, the resulting data tells nothing about what people outside the customer base think about the new product.
A sales representative wishes to survey her client base of 47 companies. She has 47 business cards, all of the identical size, from her contacts in the companies, and decides to drop them all in a small box, shake them up, and reach in to pick 5 cards for her sample. This procedure is an example of which type of sampling? A. Cluster B. Convenience C. Simple random D. Stratified E. Systematic
C. A simple random sample (SRS) is one in which every possible sample of the desired size has an equal chance of being selected. In this case, every possible sample of five companies has an equal chance of being selected. Note that even though it is also true that each company has an equal chance of being selected, this by itself would not ensure that we have an SRS.
To find out a town's average family size, a researcher interviews a random sample of parents arriving at a pediatrician's office. The average family size in the final 100-family sample is 3.48. Is this estimate probably too low or too high? A. Too low because of undercoverage bias B. Too low because convenience samples underestimate average results. C. Too high because of undercoverage bias. D. Too high because convenience samples overestimate average results. E. Too high because voluntary response samples overestimate average results.
C. The procedure misses all or nearly all childless families.
Fifty migraine patients are randomly selected from hospital records. Half the patients are told to drink ice water and sit in the dark when they next experience a migraine; the remaining patients are told to use neither of these possible remedies. Participants then report back as to relief, if any. Faults of this experimental design include all the following except: A. Lack of randomization B. Confounding variables C. Lack of blinding D. Unclear factor levels E. Measurement of response variable
C. There is no indication that randomization is being used to decide which patients have the treatment of ice water and darkness and which patients are in the control group. If the treatment of ice water and darkness helps, there is no way to tell whether ice water or darkness or the combination of both is responsible, and so the variables will be confounded. Lack of blinding is not a design fault here because blinding is impossible in this experiment. The instructions were unclear as to factor levels, that is, quantity of ice water and length of time to sit in the dark. It is unclear how the respons variable, "relief", is to be measured and thus compared between the treatments.
Sampling error occurs A. when interviewers make mistakes resulting in bias. B. when interviewers use judgment instead of random choice in picking the sample. C. When samples are too small. D. Because a sample statistic is used to estimate a population parameter. E. In all of the above cases.
D. Different samples give different sample statistics, all of which are estimates for the same population parameter, and so error, called sampling error, is naturally present.
Before taking an exam, students either went to bed at their normal times or were sleep deprived for 4 or 8 hours. Half of each group were given a caffeine pill before taking the exam. Determine the number of factors, levels for each, and number of treatments. A. 1 factor with 2 levels, 5 treatment. B. 2 factors, 1 with 1 and 1 with 2 levels, 3 treatments. C. 2 factors, 1 with 2 and one with 3 levels, 5 treatments. D. 2 factors, 1 with 2 and 1 with three levels, 6 treatments. E. 3 factors, each with 2 levels, 6 treatments.
D. Factors x levels = treatments. There are 2 factors: sleep deprivation (none, 4 or 8 hr) and caffeine (none or pill). With 3 x 2 = 6 treatments.
A telephone survey with regard to support of a bond issue resulted in: Age:21-30 For 45, Against 30 Age:31-40 For 32, Against 43 Age:41-50 For 28, Against 47 Age:51-60 For 25, Against 50 Age:61-70 For 15, Against 60 Age:71-80 For 8, Against 67 Total For: 153, Total Against: 297. Which of the following sampling strategies was most likely used? A. Cluster B. Proportional C. Simple random D. Stratified E. Systematic
D. Given the exact same number of people surveyed in each age group, stratified sampling was probably used. The age is homogeneous groups called strata, and random samples of people from each group are chosen. You could do further proportional sampling by matching the SIZE of the random samples to depend on the PROPORTION of the total population each represents. This is not clustering. Clustering divides the population into heterogeneous (different) groups called clusters, and then you take a random sample of cluster(s), one or more as needed for your sample size, from among all the clusters.
A bank wishes to survey its customers. The decision is made to randomly pick ten customers who just have checking accounts, ten customers who just have savings accounts, and ten customers who have both checking and savings accounts. This procedure is an example of which type of sampling? A. Cluster B. Convenience C. SRS D. Stratified E. Systematic
D. In stratified sampling, the population is divided into homogeneous groups called strata, and random samples of person from all strata are chosen. In this example, the bank stratified by type of account.
A consumer product agency tests kilowatts per hour for a sample of refrigerators, each one of three different sizes. Which of the following is true? A. There are 3 explanatory variables and 1 response variable. B. There is one explanatory variable with 3 levels of response. C. Kilowatts per hour is the only explanatory variable, but there are 3 response variables corresponding to the different sizes. D. There are 3 levels of a single explanatory variable. E. Each explanatory level has an associated level of response.
D. Size is the only explanatory variable, and it is being tested at 3 levels. Kilowatts per hour is the single response variable.
Which of the following is a true statement about sampling? A. Careful analysis of a given sample will indicate whether or not it is random. B. Sampling error implies an error, possibly very small but still an error, on the part of the surveyor. C. Data obtained when conducting a census are always more accurate than data obtained from a sample, no matter how careful the design of the sample. D. The clusters in cluster sampling all should look pretty much alike. E. The sampling frame is the population of interest.
D. To determine if a sample is random, one must analyze not the sample but the procedure by which it was obtained. Sampling error, also called variability, is natrual variation among samples, it is not a mistake. If a census is poorly run, it will provide less information and be less accurate than a well-designed survey. For example, having the principal ask every single student in the school whether or not he or she regularly cheats on exams produces less useful data than a carefully worded anonymous questionnaire filled out by a randomly selected sample of the student body. Clusters are heterogeneous and all should look pretty much like the population as a whole.
Which of the following is a true statement about blocking? A. Blocking is to experiment design as stratification is to sampling design. B. By controlling certain variables, blocking can make conclusions more specific. C. The paired (matched pairs) comparison design is a special case of blocking. D. Blocking is a useful procedure when there are certain attributes, not under study, which may affect the outcomes. E. All of the above are true statements about blocking.
E. Blocking in experiment design first divides the subjects into representative groups called blocks that share a similar characteristic, just as stratified sampling designs first divide into "strata" by a shared characteristic. This can control certain variables by bringing them directly into the picture, and thus conclusions are more specific. You have controlled for known variation. Matched pairs design is simply a special case of blocking by pairs, where each pair is considered its own block. When we can't control for certain attributes that we suspect affect the outcomes, then blocking may help us to more clearly see the true differences caused by treatments.
Which of the following is a true statement? A. If bias is present in a sampling procedure, it can be overcome by dramatically increasing the sample size. B. There is no such thing as a "bad sample." C. Sampling techniques that use probability techniques effectively eliminate bias. D. Sampling techniques that allow the surveyor to choose participants with care and precision go a long way to control bias. E. In choosing a sample size, actual sample size is more important than the fraction of the population that is surveyed.
E. If there is bias, taking a larger sample just magnifies the bias. If there is enough bias, the sample can be worthless. Even when the subjects are chosen randomly, there can be bias due, for example, to nonresponse or to the wording of the questions. Allowing surveyors to choose participants usually fails to result in representative samples because there are too many unknowns. Most important is not the fraction of the population, but rather the actual sample size.
Each of the 30 major league baseball teams carries a 40-person roster. A sample of 60 players (5 percent of all 1,200 players) is to be randomly selected to undergo drug tests. To do this, each team is instructed to put their 40 names in a hat and randomly draw two names. Will this method result in a simple random sample of the 1,200 baseball players? A. yes, because each player has the same chance of being selected. B. Yes, because each team is equally represented. C. Yes, because this is an example of stratified sampling, which is a special case of simple random sampling. D. No, because the teams are not chosen randomly. E. No, because not each group of 60 players has the same chance of being selected.
E. In a simple random sample, every possible group of the given size has to be equally likely to be selected, and this is not true here. For example, with this procedure, it will be impossible for all the Cubs to be together in the final sample. This procedure is an example of stratified sampling, but stratified sampling does not result in simple random samples.
What is bias in conducting surveys? A. An example of sampling error. B. Lack of a control group. C. Confounding variables. D. Difficulty in concluding cause and effect. E. A tendency to favor selection of certain members of a population.
E. Poorly designed sampling techniques result in bias, that is, in a tendency to favor selection of certain members of a population. For example, door-to-door surveys ignore the homeless, radio call-in programs give too much emphasis to persons with strong opinions, and interviews at shopping malls typically give the opinions of a very select sample of the population.
Which of the following is most useful in establishing cause-and-effect relationships? A. A complete census B. A least squares regression line showing high correlation C. A simple random sample (SRS) D. A well-designed, well-conducted survey incorporating chance to ensure a representative sample E. A controlled experiment
E. Regression lines show association, not causation. Surveys suggest relationships, which controlled experiments can help show to be cause and effect.
A human resources department plans to survey 100 of the 3,000 employees in the firm. An alphabetical list of the employees is available, a random number between 1 and 30 is picked, and the sample consists of the person that far down the list together with every 30th person after that. This is an example of sampling by: A. Cluster B. Convenience C. Simple random D. Stratified E. Systematic
E. Systematic sampling involves listing the population in some order, choosing a random place to start, and then picking every nth person by rule. This is reasonable sampling so long as the original order is not influencing the variable under consideration.
A television network conducts a weekly survey to determine the proportion of viewers who watch various programs. For the coming year, they decide to double the sample size. The main benefit of this is to: A. reduce undercoverage bias. B. reduce nonresponse bias. C. eliminate sampling error. D. decrease population variability. E. decrease the standard deviation of the sampling distribution.
E. The standard deviation of the sample proportions, δ = square root of (p(1-p)/n. This is reduced if the sample size n is increased. This also reduces the margin of error but has no effect on possible bias.
Two wordings for a questionnaire on independence for Puerto Rico: I. Would you vote for independence for Puerto Rico? II. Would you support an independent Puerto Rico separate from the U.S.? One of these showed 35% support while the other showed 45% support in favor of independence. Which produced which result and why? A. The 1st question showed 45% because of lack of randomization in choice of subjects as evidenced by wording. B. The 1st question showed 35% because of a placebo effect. C. the 1st question showed 45% due to lack of blocking. D. The 1st question showed 35% because of response bias due to the wording of the question. E. The 1st question showed 45% because of response bias due to the wording of the question.
E. The wording of the questions can lead to response bias. Here, "independence" is more positive, while "separate" can come across as a negative word.