Research Quiz 1

Novelty: An IV might work because it is new or novel and then once the novelty wears off, the difference between the experimental group and the control group fades. It is difficult to generalize the results since more then the IV impacted the findings Which of the following can be used to avoid novelty?

A.) Allow participants to adjust to investigators' presence of novel element (e.g., materials, camera) before introducing the IV B.) avoid novel items C.) survey participants about past experiences D.) none of the above A

Think about an experimental research study. Which of the following statements implies causality?

A.) Fifty percent of toys contain moving parts. B.) The room remained at 72 degrees throughout the intervention. C.) Standardized tests are most commonly used. D.) The average posttest score improved 25% when compared to the average pretest score. D

Differential Selection: Participants in the study are unlike in some way and therefore they will respond to the independent variable (IV) differently. This is a risk for quasi-experimental studies since random assignment was not used. Differences between groups are due to differential selection or assignment. This is a common threat when pre-existing groups are involved even if we demonstrate at baseline the groups are equivalent.

A.) The effects of two types of hearing aids (IV) on listening skills (DV) are compared. One group was in enrolled Classroom A and the other was enrolled in Classroom B. After listening instruction was provided, the Classroom B group outscored the Classroom A group. B.) Two pain-reducing treatments were investigated (IV). Children were told in advance which treatments would be used, and they were asked to determine which group they would like to be placed in: Treatment 1 or Treatment 2. After the treatment, there was no difference found between the two groups on self-reported pain scores (DV). C.) The effects of traditional speech sound therapy (IV) on speech intelligibility (DV) were investigated. Students with similar speech impairments were randomly assigned to two conditions: An experimental and control condition. After the intervention, there was no difference found between the two groups on speech intelligibility measures. C

Differential selection effects occur when groups are selected in a differential manner. The following is an example of this threat to internal validity:

A.) When researchers select preexisting groups to be involved in an experiment, and the participants are not similar to each other on a critical variable (e.g., severity of language impairment). B.) Participants in both conditions (experimental and control groups) are equivalent on a critical variable (e.g., all have average hearing). Unfortunately, they are not equivalent to each other on another critical variable that could also account for group differences (e.g., IQ). C.) Participants are allowed to select their group. Those who are interested in improving a skill are encouraged to select the experimental group. Those who are disinterested are placed in the control group. D.) All of the above. D

Interaction of time of measurement and treatment: Since most skills taught will deteriorate over time (if practice opportunities are not provided), the interaction of time of measurement and treatment effects must be considered. Posttests that are done immediately after the completion of the study may not show if the IV variable was effective over time. Need to consider maintenance of the IV's effects and generalization of skills across time. Which of the following can be used to avoid interaction of time of measurement and treatment?

A.) conduct one posttest B.) conduct one posttest 6 weeks after independent variable has ended C.) conduct multiple posttests after independent variable has ended D.) none of the above C

Experimenter: One researcher in the study acts differently than another person in the same study (e.g., gestures, mannerisms, rate of speech, appearance, gender). These differences impact DV. Which of the following can be used to avoid experimenter bias?

A.) inform participants about the purpose of the study B.) use statistical adjustments C.) ensure uniformity through fidelity checks, scripts, and training D.) none of the above C

Hawthorne: When the participants are aware they are being assessed or their behaviors are being monitored, their performance will change. Which of the following can be used to avoid the Hawthorne effect?

A.) test participants in a 1:1 session B.) use unobtrusive measures C.) tell the participant that you are testing them now D.) all of the above B

A researcher is interested in studying the effects of humming as an intervention for high school age children experiencing stress and anxiety. The participants are all receiving mental health services from a the same outpatient clinic in Akron. They all have mild to severe anxiety disorders. Eight participants are enrolled in the five-week long research study. In order to assign children to a condition, the researcher writes the letter "A" on four pieces of paper and the letter "B" on four pieces of paper and puts them in a hat. She goes through the list of participants and selects one piece of paper for each of them. The participants who get a paper with the letter "A" are put into the experimental group. The participants who get a paper with the letter "B" receive nothing, and they are considered in the control group. Which of the following describes this study?

An experimental study with random assignment, but no random selection.

A research study is designed to determine if the Visualizing and Verbalizing program can improve listening comprehension in elementary school-aged children. The study has 30 participants, and they are randomly assigned to either the experimental or control group. The experimental group receives instruction using the Visualizing and Verbalizing program. Weekly sessions are 60 minutes long, and the experiment lasts 12 weeks. The control group watches a Disney movie during that same time frame. All participants continue to receive phonics instruction using the Orton-Gillingham Reading Program. Which of the following describes the reading instruction that the sample receives?

Constant variable

A researcher designs a study in which 10 participants, divided into two groups, will receive two different interventions to order to see which one is more effective. Two graduate students will deliver the independent variables. In order to learn the interventions, the graduate students are given an article to read and a YouTube demonstration video to view. No script or training protocol is created, and the graduate students are free to deliver the intervention as they see fit. No fidelity checks are conducted by the lead researcher. Which threat to validity is most likely to have influenced the results of this study?

Experimenter effect

A true experimental research design must have a control group and random selection, but doesn't need to have random assignment. True or False

False

Attrition/Mortality: Participants in the study drop out due to moving, disinterest, death, etc. When this happens, the groups become unequivalent. Loss of participants. Using a control group is one way to avoid attrition/mortality. True or False

False

Determinism is when the experimenter and/or observers are unaware of the research questions or hypotheses. They are also do not know to which group the sample is assigned- experimental or control. True or False

False

History: Outside events that occur during the course of the experiment or between the repeated measures of the dependent variable (DV) influence the results. Increasing the sample size is one way to avoid threats of history. True or False

False

Statistical regression is a threat to validity that occurs whenever participants are observed unobtrusively and react to the observer's presence. True or False

False

Testing: The experience of taking a test may influence the results. An attitude may change because of the survey or the participant's responses to a test may change after repeated measures (e.g., annoyance, avoidance). Using random selection is one way to avoid threats of testing. True or False

False

The results of basic research will have immediate clinical value for practicing professionals. True or False

False

When a researcher has a choice, a pretest posttest with control group research design should always be used. All other true experimental research designs are inferior to it. True or False

False

A researcher uses a two group research design: One experimental group and one control group. In order to establish baseline levels, both groups are recorded at home playing with a parent. The researcher then scores the participants on a checklist after watching those recordings (baseline). The participants then come to a hospital clinic twice a week for four weeks to receive intervention (independent variable). During the final session, the researcher plays with the participants and scores them on the checklist while playing (posttest observation). Which threat to validity will most likely influence the results?

Hawthorne effect

What are some reasons why it can be challenging to find a sample that truly represents the population.

It is expensive to use random selection. Researchers might not know all members of the population. It takes a long time to locate all members in the population.

Which threats to validity can be controlled using a control group?

Maturation

A research study contains three groups: one that receives an intervention with a parent present, one that receives an intervention without a parent present, and one that is a control group. The participants are randomly selected from a tri-county area and randomly assigned to groups. Three experimenters rotate among the sites administering the treatment and collecting data. At the end of the intervention, the participants are given a posttest. The group that received the intervention without a parent present performed higher than the group that received the intervention with a parent present. The control group that received no intervention scored the lowest on the posttest. During administration of the posttest, one of the researchers discovers that participants in some of the conditions began a tutoring program during the course of the study. Which threat to validity is most likely to have influenced these results?

Multiple treatment interference

A true experimental design (select all that apply):

Must investigate causality Should have used a sampling method Must have random assignment to a control group or comparison group Should have a control group

Which of the following is an assumption of the Scientific Method?

Order, Determinism, Discoverability

A researcher owns a company that developed a new intervention tool called the Stress ReducerTM, which uses technology to decrease stress and anxiety. The researcher wants to study the effectiveness of the Stress ReducerTM, and he creates a pretest-posttest with control group study. The study lasts 1 week and participants are asked to refrain from yoga and other stress/anxiety relieving interventions. The sample of 9 participants are divided into 3 groups: 1) One group receives the Stress ReducerTM treatment for 5 minutes 2) one group receives the Stress ReducerTM treatment for 30 minutes; and 3) one group does not receive any intervention. Stress and anxiety were measured using a survey that was given before and after the intervention period. The main problem with this research design is:

Personal Bias

Which of the following can be used to control for the novelty effect?

Random assignment Use of a control group Limit study to one independent variable None of the above None of the above

How do researchers control for this threat: Generalization across participants?

Replicate the study

A researcher is interested in studying the effect of visual schedules on direction following. A norm-referenced test is administered by trained graduate students who are blinded. The sample is randomly assigned to two groups, an experimental and a control group, using a computer program. The participants are all children with severe disabilities who are educated in special education classrooms. The children all scored 2 standard deviations below the mean on the pretest. There was initial group equivalency. The intervention lasted four weeks, and a posttest was given to the children in both conditions. When the scores were analyzed, the children in both conditions scored within 1 standard deviation from the mean on the posttest. It is important to note, an alternative form of the pretest was used at posttest. Which threat to validity is most likely to have influenced these results?

Statistical regression to the mean

A researcher is interested in studying the impact visual schedules have on direction following. Direction following will measured using a standardized test. The number of directions followed correctly will be the

The dependent variable

The actual quiz will contain several questions about ways to control for certain threats to validity. What are some ways researchers try to minimize (control/remedy) the impact a threat to validity has on the study?

They add a control group and then randomly assign (put) participants to either the experimental or control group. Remember, you assign people to a group. They use blinding (e.g., someone in the study completes a task without knowledge of the research question or hypotheses). Remember, they are blind or can't see what you are trying to accomplish with your study. They use rigorous research designs like cause-effect experimental ones (e.g., randomized controlled trials). Remember, this class is about studies that try to show the IV causes the DV to change somehow. They use a more rigorous way of selecting the sample from the population (e.g., random selection). Remember, you Select the Sample to be in the study.

The Pretest is administered for the following reason:

To collect baseline data

The following is the goal of an experimental study:

To measure how the independent variable affects the dependent variable.

Differential selection: Participants in the study are unlike in some way and therefore they will respond to the independent variable (IV) differently. This is a risk for quasi-experimental studies since random assignment was not used. Differences between groups are due to differential selection or assignment. Random assignment is one way to avoid differential selection. True or False

True

Disruption: Treatment interrupts the participants' routine, & they fail to respond due to that interruption rather (not due to the failure of the IV to change the behavior. Researcher may have also done a poor job in administering new & unfamiliar IV, & this poor administration impacted the results. Minimizing distractions or disruptions to the normal routine is one way to avoid the threat of disruption. True or False

True

Generalization across participants: The only way to generalize results from the study participants to the accessible population and then to the target population is through several replications of the study. This type of validity is difficult to achieve since you can't ensure the accessible population is similar in all respects to the larger population (target). There are limits on the extent to which you can generalize to the target (total) population Replicating the study is one way to avoid the threat of generalization across participants True or False

True

Instrumentation: The reliability of an instrument may change in calibration or from a change in the evaluator's ability to measure the differences (e.g., fatigue). Changes in the way the DV was measured. Creating a standardized protocol for data collection is one way to avoid threats of instrumentation. True or False

True

Interaction of personological variables and treatment effects: Interaction of personological variables (e.g., high motivation for money, type of learner) may impact the treatment effects. The IV has a different effect on the participants based on their personal characteristics (e.g., temperament, learning style, motivations) Using random assignment is one way to avoid the threat of interaction. True or False

True

Maturation: Changes in the DV that are due to aging or development, rather than due to the IV. Changes can be biological or psychological. Using a control group is one way to avoid threats of maturation. True or False

True

Multiple-treatment interference: When multiple IVs interfere with results. Difficult to generalize the effects of one IV to the population when the sample was exposed to multiple IVs. Limiting the study to one IV is one way to avoid multiple-treatment interference. True or False

True

Nonclinical groups often serve as the control group in quasi-experimental studies. True or False

True

Statistical regression to the mean: There is a tendency that participants with high or low scores on the DV will regress toward the mean on the next administration of the test. This is especially true when the measure is not reliable and there is variation between repeated measures. When subjects are assigned to a group or condition based on extreme scores, the change in the DV can't be attributed to the treatment (IV). Need a control group to control this. Using a control group is one way to avoid statistical regression to the mean. True or False

True

The Solomon Four-Group Design is not often used, although it offers a great amount of experimental control. True or False

True

The dependent variable affects the independent variable. True or False

True

The effects of differential selection can be controlled by random assignment. True or False

True

The presence of an extraneous variable weakens the internal validity of a study. True or False

True

The sample is defined as

a subset of the population

Testing: The experience of taking a test may influence the results. The act of taking the pretest may provide the participants with practice taking the posttest, and therefore be the reason why the performance on the posttest was higher. An attitude may change because of the survey or the participant's responses to a test may change after repeated measures (e.g., annoyance, avoidance). Review the definition above. Which of the following scenarios does NOT demonstrate this specific threat?

a) 40 participants were randomly assigned to an experimental and control condition. The experimental condition received a pretest and a posttest (that was the same as the pretest). The control group only received a posttest. b) 40 participants were randomly assigned to an experimental and control condition. Both groups received a pretest and a posttest but different forms of the test were used at each administration (i.e., there was a form A for pretest and a form B for posttest). c) 3 individuals with autism were assigned to three different experimental conditions. The experiment lasted one year. A standardized language assessment (i.e., the pretest) was administered. One of the participants had a strong aversion to standardized testing, he remembered his pretest anxiety, and he rushed through the questions when the standardized posttest was given. B

Statistical Regression: There is a tendency that participants with high or low scores on the DV will regress toward the mean on the next administration of the test. This is especially true when the measure is not reliable and there is variation between repeated measures. When subjects are assigned to a group or condition based on extreme scores, the change in the DV can't be attributed to the treatment (IV). Review the definition above. Which of the following scenarios does NOT demonstrate this specific threat?

a) 15 children are placed in a study and their average direction following skills at baseline show a standard score of 55. After the independent variable is administered and a posttest is given, the group standard score increased to 65. b) There were two conditions: Experimental and control. Both groups had extremely low expressive language standard scores (i.e., 50) after a pretest was given. After the intervention, the experimental group outperformed the control group on the posttest. c) So you could practice testing administration, your parents were given a vocabulary test and they received scores above a standard score of 150. They took the same test but a different form, and their vocabulary skills were found to be 130. B

History: Outside events that occur during the course of the experiment or between the repeated measures of the dependent variable (DV) influence the results. Review the definition above. Which of the following scenarios does NOT demonstrate this specific threat?

a) A pretest posttest control study was used to determine if there were changes in physiological stress after therapeutic play intervention. The control group received no therapeutic play intervention. After both groups were pretested, it was learned that three people in the control group received medication to reduce stress and anxiety. b) We aimed to decrease distractibility (DV) in students with autism who are placed in a residential school. The students were randomly assigned to two conditions: An experimental group that learned visual cues (IV) and a control group. At the end of the one month study, the experimental group outperformed the control group. c) 15 adults with language impairment were randomly assigned into two groups: Experimental and control. The experimental condition received training on word finding strategies. Unbeknownst to the researcher, four participants in the experimental group received additional speech therapy. At the end of the study, the experimental group had less word finding difficulties than the control group. B

Threat: Interaction of Personological Variables and Treatment Effects Interaction of personological variables (e.g., high motivation for money, type of learner) may impact the treatment affects. The IV has a different affect on the participants based on their personal characteristics (e.g., temperament, learning style, motivations) Scenario: Convenience sampling was used (i.e., the researcher's caseload) and participants were placed into groups purposefully by the researcher. Participants in the experimental group 1 received $5 dollars each time they used their speech-generating device in the community. The experimental group 2 received verbal praise for the same task. Those individuals in group 1 used their device less, and the researcher concluded that money cannot be used as a viable reinforcer for users of speech-generating devices. Which of the following best identifies the problem(s) in this scenario?

a) It is unreasonable to compare results between a group that was paid and a group that received no financial compensation. b) Socioeconomic status and motivation were not considered as a control variables at the onset of the study. c) Participants were not paid enough money. d) There is no problem with this study methodology. B

Instrumentation: The reliability of an instrument may change in calibration or from a change in the evaluator's ability to measure the differences (e.g., fatigue). Occurs if there are changes in the way the DV was measured or how it was measured. Review the definition above. Which of the following scenarios does NOT demonstrate this specific threat?

a) A pretest posttest control study was used to determine if there were changes in physiological stress after therapeutic play intervention. The control group received no therapeutic play intervention. Observers were blinded to the purposes of the study and did not know which condition was which. Each observer received explicit training in measuring physiological stress and the same instruments were used among observers. b) 30 children were randomly assigned to two conditions: Experimental and control. At baseline, the dependent variable, self-reported pain score, was measured using a scale of 1-10 while sitting in bed. At the end of the study, the children were asked to rate their pain on a scale of 1-10 while walking. c) 30 students were randomly assigned to two conditions: Experimental and control. At baseline, the dependent variable, pitch, was measured by using the 1996 version of software. At the end of the study and after updated laptops, microphones, and software were purchased, the participants' pitch was measured using the new technology and the newest version of the software. A

Attrition/Mortality: Participants in the study drop out due to moving, disinterest, death, etc. When this happens, the groups become unequivalent. There is a differential loss of participants. Review the definition above. Which of the following scenarios does NOT demonstrate this specific threat?

a) A pretest posttest control study was used to determine if there were changes in vocal quality (DV) after voice therapy (IV). The control group received no therapy. After both groups were pretested, the researcher removed two participants from the control group because their voice quality was judged to be extremely poor, and they needed medical attention. b) 30 children in a local children's hospital were randomly assigned into two groups: Experimental and control. The experimental condition received therapeutic play interventions. Through the course of the study, 7 patients who exhibited the highest levels of stress were removed from the study due to worsening medical condition. At the end of the study, the experimental group had lower psychological and physiological stress than the control group. c) Two groups were involved in learning memory strategies. At pretest, both groups scored similarly on memory tasks. At posttest, condition one outperformed the other. It was also discovered that 10% of the total participants in both groups dropped out. Further information was collected on the drop-outs, and the drop-outs from both groups were found to be similar on important characteristics (e.g., diagnosis, severity, intelligence, age). C

Threat: Pretest / Posttest Sensitization The administration of the pretest may have impacted the DV (e.g., serves as a preview to what will happen next and reduce anxiety about the experiment). Or, A posttest may impact the DV if the posttest is a synthesis of previously learned material. Scenario: A group of 4 children with ADHD are given a pretest about direction following. One of the individuals had previous negative experiences with standardized testing. He is so certain the experiment will involve more standardized test administration that he can't concentrate on the tasks at hand and is highly anxious. At the end of the study, the posttest inadvertently summarized the strategies that they were supposed to learn in the study. He and the other members in the study were able to determine from the posttest what they should do when they encounter directions. They did well on the posttest because of what they were able to figure out from taking the posttest, not because of the intervention. Which of the following best identifies the problem(s) in this scenario?

a) Anxiety about standardized testing could have affected one participant's ability to participate fully in the study. b) Participants were reminded how to answer based on clues that were inadvertently placed in posttest. c) It is unreasonable to expect participants to complete written testing during any research study. d) A and B only. e) All of the above. D

Threat: Disruption Treatment interrupts the participants' routine, & they fail to respond due to that interruption (not due to the failure of the IV to change the behavior). Researcher may have also done a poor job in administering new & unfamiliar IV, & this poor administration impacted the results. Scenario: 20 children with autism spectrum disorders are placed in a study in order to assess the use of intermittent questioning on task completion. While they were working, the researcher interrupted their work and asked them questions about the task at hand. The researcher hoped that intermittent questions would serve as a reminder to the participants and redirect them back to the task at hand. Which of the following best identifies the problem(s) in this scenario?

a) Individuals with autism spectrum disorders may not respond well to their routine being disrupted. b) The failure to respond might be due to a break in routine, rather than to inability to answer the question. c) The independent variable might have worked if it was implemented before work or before each task began (but not during). d) All of the above. D

Threat: Measurement of the Dependent Variable The method for measuring the dependent variable may be what affected the dependent variable not the IV (standardized test vs. observing skill in real-life). Scenario: A group of 7 year olds with speech sound disorders were placed in a pretest posttest control study. The pretest consisted of a single word articulation test. After the independent variable was administered, a posttest was given. The posttest was a continuous speech sample. Which of the following best identifies the problem(s) in this scenario?

a) It is unclear if the change in the dependent variable was due to the IV since the pre-and posttests were different. b) Students perform differently on single word articulation tests than they do on conversational tasks when co-articulation is involved. c) The level of self-monitoring is more difficult on conversational tasks than on single-word tasks. d) All of the above. D

Interaction of history and treatment: The time frame a study was conducted or other events that occurred during the study can affect the generalization if the environment changed. Which of the following can be used to avoid interaction of history and treatment?

a) limit time between pre- and post-test, or the overall duration of the study b) use multiple post-tests c) survey participants about what happened in their life while the study progressed (e.g., did you get additional speech therapy at any point during the study) d) all of the above D

Measurement of the dependent variable: The method for measuring the dependent variable may be what affected the dependent variable not the IV (standardized test vs. observing skill in real-life). Which of the following can be used to avoid threats of measurement?

a) train evaluators b) conduct fidelity checks or random observations of data collection c) use blinding d) all of the above D

Threat: Interaction of History and Treatment The time frame a study was conducted or other events that occurred during the study can affect the generalization if the environment changed. Scenario: A group of children who were admitted to the children's hospital were placed in a study to determine the effect of breathing exercises on stress levels in multiple hospital settings. During the intervention phase of the study, two of the participants received medications to manage stress. These participants received the independent variable and medication throughout the course of the yearlong study. Which of the following best identifies the problem(s) in this scenario?

a) One year is too long for this type of study, so the results are not valid. b) The threat is minimal because only 2 participants received medication to manage stress, and the rest were only enrolled in the research study. c) There is no way to know if the results were based on the independent variable, the confounding variable (medication administration) or both. d) As this study was conducted in multiple settings, the results are not valid, due to changes in the environment during the study. C

Threat: Generalization across Participants The only way to generalize results from the study participants to the accessible population and then to the target population is through several replications of the study. This type of validity is difficult to achieve since you can't ensure the accessible population is similar in all respects to the larger population (target). There are limits on the extent to which you can generalize to the target (total) population. Scenario: A new pain reducing drug was tested on an experimental group of volunteers. The experimental group had less pain than the control group. The researcher concluded the drug works and can be sold at his private clinic to new clients. Which of the following best identifies the problem(s) in this scenario?

a) Random selection was not used. b) This is the only study of its kind c) The study has not been replicated. d) All of the above D

Threat: Experimenter One researcher in the study acts differently than another person in the same study (e.g., gestures, mannerisms, rate of speech, appearance, gender). These differences impact DV. Scenario: 100 children with autism spectrum disorders are placed into an experimental condition to study the use of visual schedules on task completion. Because the sample size is so big, the groups are divided up into four groups, but they all receive the same independent variable. The experimenter in groups 1 and 3 are very talkative, and they explain the visual schedule to each participant in their group and talk on their cell phones while the participants work. The other twp experimenters adhere strictly to the research plan. The head researcher never observes the implementation of the independent variable, and he is unaware about the talking and explanations. Which of the following best identifies the problem(s) in this scenario?

a) The participants in groups 1 and 3 likely responded to the IV differently based on the extra explanations and/or distractions. Their performance would reflect the introduction of the IV and the extraneous variable. b) There is no problem with this study methodology. The experimenters did the best they could. c) The experimenter for groups 1 and 3 is more energetic, so those groups perform better. The others should be more energetic. d) There should only be one experimenter for all participants. A

Threat: Multiple-Treatment Interference When multiple IVs interfere with results. Difficult to generalize the effects of one IV to the population when the sample was exposed to multiple IVs. Scenario: The researcher wants to determine if guided imagery intervention is more beneficial in reducing post-operative pain when taught pre-operatively or post-operatively. The participants are first given pre-operative guided imagery instruction. Then they are given post-operative guided imagery intervention. At the end of the study, the researcher concluded that post-operative guided imagery was superior. Which of the following best identifies the problem(s) in this scenario?

a) The researcher is correct in their conclusion, because post-operative guided imagery was the second intervention. b) The researcher cannot conclude that post-operative intervention is superior to pre-operative intervetion for reducing pain, because the participants only received each intervention for one week. c) It is not possible to know which independent variable caused the change in the dependent variable, as all participants received both interventions. The researcher cannot conclude that one is superior based on these results alone. d) There is no multiple-treatment interference because both independent variables target reducing post-operative pain. C

Threat: Interaction of Time of Measurement and Treatment Since most skills taught will deteriorate over time (if practice opportunities are not provided), the interaction of time of measurement and treatment effects must be considered. Posttests that are done immediately after the completion of the study may not show if the IV variable was effective over time. Need to consider maintenance of the IV's effects and generalization of skills across time. Scenario: A pretest posttest design was conducted and a group of preschoolers who demonstrated poor vocal behaviors, such as excessive yelling, was enrolled in the study. The researcher wanted to determine if vocal instruction could be conducted in order to reduce instances of poor vocal behaviors. After the month long training, a posttest was immediately conducted. The researcher determined that the intervention worked because the preschoolers demonstrated what they learned during the vocal instruction. Which of the following best identifies the problem(s) in this scenario?

a) The researcher should done a posttest only design. The pretest affected the results. b) The researcher is correct in concluding that the intervention worked, because a posttest was conducted and the data are proof. c) The researcher was premature in his conclusion. He did not take into account the likelihood that the students would forget what they learned and revert back to old behaviors. Ongoing booster trainings or additional posttests would be requied to ensure that the skills were maintained. d) There was a different between the pretest and posttest. Consequently, the intervention was totally effective. Researchers aren't concerned about skill maintenance. C

Threat: Hawthorne When the participants are aware they are being assessed or their behaviors are being monitored, their performance will change. Scenario: A group of children who were admitted to the children's hospital were placed in a study to determine the effect of breathing exercises on stress levels in multiple hospital settings. After the intervention phase of the study, their stress levls were posttested in a common area. 25% of the participants noticed the researcher hiding behind a couch and taking data. The presence of the researcher served as a reminder that they should try to stay calm and remember their breathing exercises. Which of the following best identifies the problem(s) in this scenario?

a) The researcher should not have tried to hide. b) This study cannot be replicated because it is unclear how well the researcher was hidden. c) They may have seen the researcher and become anxious, causing them to perform poorly. Additionally, the presence of the researcher caused 25% of the participants to recall the strategies. It wasn't clear if they'd remember what to do on their own. d) Participants would not be inclined to act differently even though they knew they were being watched. C

Threat: Verification of the Independent Variable/Describing the Independent Variable Explicitly Procedures must be described fully by the researcher in sufficient detail to be reduplicated in other settings. Need detailed descriptions in order to generalized results to different settings. Extent to which the implementation of the IV can be reduplicated. The IV may not have been implemented as described. Scenario: A detailed research plan was created by the researcher. The research team, four graduate students, implemented independent variable. Each graduate student did "their own thing" when it came time to implement the independent variable despite the detailed research plan. The lead researcher was on medical leave while the intervention phase of the study was implemented. Two of the conditions did extremely well while the other two conditions did not demonstrate improvements. Which of the following best identifies the problem(s) in this scenario?

a) The study cannot be replicated, because there is not sufficient information about exactly how each team member conducted intervention. b) The research team cannot conduct intervention when the lead researcher is not present. c) There is no problem with this study methodology. The study can easily be replicated. A

Threat: Novelty An IV might work because it is new or novel and then once the novelty wears off, the difference between the experimental group and the control group fades. It is difficult to generalize the results since more than the IV (i.e., the novelty) impacted the findings Scenario: A pretest posttest posttest posttest design is used. The researcher wants to determine if the expanding expressions technique can improve describing skills in kindergarteners from lower socioeconomic backgrounds. The participants are given multiple colored beads to assist in their skill development. At the first posttest, the students demonstrated a marked increase in describing skills. When subsequent posttests were given, their accuracy decreased. Which of the following best identifies the problem(s) in this scenario?

a) There is not a threat of novelty in this scenario. The participants' accuracy decreased over time because they were asked to describe more complex things. b) The participants were excited about the new technique because it incorporated what they though was a toy. Over time, the novelty wore off and the activity was no longer exciting for them. c) The researcher should have ended the study after one posttest, rather than trying to complete several more. d) There is not a threat of novelty in this scenario. The particpants' accuracy decreased over time because they forgot how to use the colored beads, and the researcher was not allowed to remind them. B

Maturation: Changes in the DV that are due to aging or development, rather than due to the IV. Changes can be biological or psychological. Maturation is also a concern when the measures are taken at different times for the two different groups. Review the definition above. Which of the following scenarios does NOT demonstrate this specific threat?

a) Three types of reading instruction (IV) were tested to see if there would be any improvements sounding out words (DV). The sample consisted of 30 six year olds with learning disabilities. One additional group served as the control group. The pretest was given in all conditions before instruction began and after breakfast. A month later, the posttest was given after lunch for all conditions. b) 4 year old child with communication impairments was enrolled in a study to investigate whether visual supports (IV) could be used to improve direction following skills (DV). At the end of the yearlong study, the child improved significantly on a direction following task over baseline. The researcher attributed the change to the introduction of the independent variable. c) A pretest posttest with control design was used. 30 participants in a study were placed in an experimental group and 30 participants were placed in a control group. The experimental group received graphic organizer instruction (IV) in order to improve listening comprehension (DV). The posttest was administered early in the morning after breakfast for the experimental group and right before lunch for the control group. A difference was observed at the end of this two day study. The experimental group outperformed the control group on listening comprehension tasks. A

Pretest sensitization: The administration of the pretest may have impacted the DV (e.g., serves as a preview to what will happen next and reduce anxiety about the experiment). Which of the following can be used to avoid or understand if pretest sensitization occurred?

a) avoid pretest -posttest research designs b) use Soloman Four research design c) Do 2 pretests before introducing the independent variable d) All of the above e) A and B E

Posttest sensitization: A posttest may impact the DV if the posttest is a synthesis of previously learned material. Which of the following can be used to avoid or understand if posttest sensitization occurred?

a) carefully select a test with different versions/forms b) avoid using posttests c) calibrate all equipment and instruments d) none of the above A

Verification of the independent variable/describing the independent variable explicitly: Procedures must be described fully by the researcher in sufficient detail to be reduplicated in other settings. Need detailed descriptions in order to generalized results to different settings. Extent to which the implementation of the IV can be reduplicated. The IV may not have been implemented as described. Which of the following can be used to avoid this threat?

a) explicitly describe the independent variable and methods to aid in replication studies b) train investigators on implementation of the IV c) monitor the IV to ensure fidelity of implementation and retrain investigators when necessary d) all of the above D

Mario studied how far room temperature water would spurt out of a plastic milk carton when 3mm holes were punched at different heights from the bottom of the carton. Identify the independent variable.

height of the holes in the carton

Tiffany was investigating how quickly Hayden reacted to different trumpet sounds. Identify the dependent variable.

how quickly Hayden reacted

Mrs. Crain wanted to see how different types of music affected students' pulse rates. She played 6 different types of music: heavy metal, R&B, alternative, pop, country, and classical. Identify the dependent variable.

pulse rate

Nick wanted to see how high an ice cube would float in different temperatures of water. Identify the independent variable.

temperature of water

Mrs. Crain wanted to see how different types of music affected students' pulse rates. She played 6 different types of music: heavy metal, R&B, alternative, pop, country, and classical. What constant should be considered when doing this experiment? Pick the best one.

the amount of time each type of music was played

Tiffany was investigating how quickly Hayden reacted to different trumpet sounds. Identify the independent variable.

the different sounds

Mario wanted to see how far room temperature water would spurt out of a plastic milk carton when 3mm holes were punched at different heighs from the bottom of the caron. Identify the dependent variable.

the distance the water traveled from the carton

Nick wanted to see how high an ice cube would float in different temperatures of water. Identify the dependent variable.

the height of the ice cube in water

Mario wanted to see how far room temperature water would spurt out of a plastic milk carton when 3mm holes were punched at different heighs from the bottom of the caron. Identify a constant.

the size of the holes (3mm)

Mrs. Crain wanted to see how different types of music affected students' pulse rates. She played 6 different types of music: heavy metal, R&B, alternative, pop, country, and classical. Identify the independent variable.

the type of music played