Chapter 14
The _____________________________ consists of three conditions: During the first, A condition, the skills to be treated (the dependent variables) are baserated During the second, B condition, treatment (independent variable) is applied to the skills In the final, A condition, the treatment is withdrawn and the skills are measured
ABA design
_________________________ is not a threat in SSDs that include one or more participants. Participants who drop out of those studies are typically replaced. Also, there are no statistically compared participant groups.
Attrition
This class of evidence is based on a randomized group experimental design study, often referred to as a randomized clinical trial; this is the best evidence supporting a procedure. To be classified as this class, the evidence must come from at least one larger clinical trial with experimental and control groups as described earlier.
Class I evidence
In medicine, evidence is classified into three major classes.
Class I evidence Class II evidence Class III evidence
This class of evidence is based on well designed studies that compare the performance of groups that are not randomly selected or assigned to different groups. Because of lack of randomization, the groups may or may not be equal to begin with. Therefore, there is no assurance that differences noted on posttests are due to treatment.
Class II evidence
This class of evidence is based on expert opinion and case studies. As noted, case studies can claim improvement, but not effectiveness. Case studies do not include control groups. This is the weakest of the levels of evidence. Nevertheless, evidence based on case studies is preferable to that based on expert opinions.
Class III evidence
____________ are the result of systematic observation, and in many cases, experimentation. To test their hypotheses, or to answer questions with no particular hypothesis, scientists gather _________. Scientists observe events and record some measured values of those events (in our earlier example, the actual number of dysfluencies when stress was increased). Scientific ________ are empirical, meaning that they are based upon actual events that resulted in some form of sensory contact.
Data
The two philosophical hallmarks of science are empiricism and determinism. _____________________ is the philosophical position that statements must be supported by experimental or observational evidence; this evidence is typically sensory experience (the touching, seeing, tasting, smelling, and hearing of phenomena) must be objectively verifiable. In other words, events must be experienced in such a way as to permit observation and measurement because sensory experience is the basis of scientific knowledge. ______________________________ means that events do not happen randomly or haphazardly; they are caused by other events. Scientific activity based on _____________________ is a search for causes of events. The goals of science are to:
Empiricism determinism determinism describe natural events or phenomena understand and explain natural phenomena, especially in terms of cause effect relationships predict occurrences of events control natural phenomena by understanding the causes of events and predicting their occurrence
_______________________ ___________________ refers to ______________________________ of research findings. It is the extent to which an investigator can generalize the study's results to other individuals and situations.
External validity generalizability
The ______________________________ ______________ is the extent to which a study's results are affected by participants' knowledge that they are taking part in an experiment or that they are being treated differently than usual.
Hawthorne effect
_____________________ validity is the degree to which data in a study reflect a true cause effect relationship. A study with strong ________________ validity is one in which the dependent variable was affected only by manipulation of the independent variable. No confounding variable was present. In a treatment study, for example, an _________________ valid study can claim that it was the treatment, and no other variable, that produced the positive changes in the clients who received it.
Internal internal internally
A critical examination of research evidence is the heart of evidence based practice. Clinicians are expected to systematically gather and integrates information on treatment procedures from such sources as treatment research evidence, client preferences, and prior knowledge to arrive at a decision. After examining the internal and extenral validity of treatment research evidence, clinciians may choose techniques that are supported by well designed and well executed treatment efficacy studies. Evidence to support a treatment procedure is often not categorical; it varies on a continuum.
K
Control groups in group designs and treatment reversal and other measures in SSDs adequately control for history.
K
Control groups in group designs and treatment reversal and staggered treatments in SSDs help control the effects of regression to mean.
K
For these levels of evidence and any updated information, see the ASHA's website)
K
Generally, relevant events that occur outside the experimental setting and thus outside the experimenter's control, have a greater chance of occurring in long term studies.
K
Group design studies conducted in any discipline use statistics to help organize, summarize, and analyze data. Dozens of statistical techniques are available. Commonly used techniques include measures of variability and measures of central tendency. Measurement scales are often involved, as they are closely related to statistical scaling techniques.
K
History includes the participant's life events that occur during the course of an experiment that may be partially or totally responsible for changes recorded in the dependent variable after the independent variable is introduced. For example, a child who receives threapy for a speech disorder and shows significant improvement may have received pressure equalizing tubes in her ears during the course of therapy. Any improvement noted in speech intelligibility may be partly or wholly due to the aural intervention, not speech therapy.
K
If distributions are relaively symmetrical, the three measures of central tendency should be the same, and results in a normal distribution or bell shaped curve
K
In clinical research, many clients seek treatment when their problem is at its worst. For example, clients who abuse their voices may seek treatment when they are the most hoarse after cheering at a ball game. However, the voice will eventually (in most cases) sound less hoarse (regression to the mean). Improvement in hoarseness shown by the participants in a voice treatment research may be confounded by such regression.
K
Interpretation of data may especially be confounded by the interaction between participant selection and some other factor, such as maturation.
K
It is the responsibility of the clinician to evaluate treatment techniques in terms of the level of evidence they have attained. Generally, it is unethical to select a technique with a lower level of evidence when one that is supported by a higher level of evidence is available. This is why it is important to understand the research methods used in generating scientific evidence.
K
One problem with the levels of evidence just described is that they ignore the SSDs that have produced significant evidence supporting procedures used to treat communication disorders. Also, the levels equate expert opinion with uncontrolled case studies. With all their limitaitons, the results of case studies still offer better guidelines fro treatment selection than are expert opinions. An alternative way of classifying evidence for clinical procedures accepts all valid research designs and is based on reserach that is uncontrolled, controlled, and replicated by the same or different investigators. This hieararchy of evidence moves from the least desirable to the most desriable evidence.
K
People who take part in research studies may know what is expected of them; this knowledge alone may influence how they respond under different study conditions. For instance, individuals who completed an experimental treatment for their stuttering may be asked to fill out a questionnaire to assess their satisfaction with the treatment. Because they know they are part of a research project, the participants might want to please the experimenters and state high satisfaction. The experimenters' conclusion that the treatment was highly satisfactory may be questionable. Such results cannot be generalized to people at large who stutter.
K
Qualitative data are verbal descriptions of attributes of events, whereas quantitative data are numerical descriptions of attributes of events. Qualitative data are stated in words; quantitative data are given in numbers. For example, a clinician might state, "The client has a severe articulation disorder characterized by mutliple omissions of phonemes" (qualititaive data). The clinician might further state, "In a 5 minute spontaneous speech sample, the client omitted word final phoneme in 75% of the contexts" (quantitative data).
K
Results of a study may be confounded by a combination of factors described so far. For example, a study involving the use of questionnaires to evaluate attitudes of spouses of people with aphasia might suffer from both testing and attrition.
K
Stasticial techniques help organize, analyze, and draw conclusions from research data.
K
The range can be deceptive, however, because extremely high and low scores can make the range appear greater or more variable than it really is. For example, let us assume 1 student received a score of 40 on the examination. The next highest score was 70, and 49 out of the 50 students received scores between 70 and 100. The 1 student scoring 40 caused the class to have a 60 point range in scores instead of a 30 point range.
K
There are several major purposes for statistics. The first purpose is to take large quantities of data and reduce them to managebale forms. For example, if there are 100 individual scores on an examination, it is easier to calculate an average than to look at and remember 100 individual scores.
K
Three measures help organize and summarize data; measures of central tendency, measures of variability, and measures of association. Measures of association (also called correlation) have been described already. Thus, measures of variability and measures of central tendency will be briefly discussed.
K
Threats from instruments is controlled by frequent calibration of mechanical equipment and adequate training of human observers.
K internal validity
There is no evidence supporting a treatment; the procedure is advocated by an expert. Such a procedure is good only for research, not for routine clinical practice.
Level 1 Expert advocacy
A case study with no control group or such single subject control conditions as baseline, withdrawal, reinstatement, or multiple baselines has shown the procedure to produce improvement, there is no assurance of effectiveness. The procedure is acceptable only if one with better evidence is not available.
Level 2 Uncontrolled unreplicated evidence
A case study without controls has been replicated by the same investigator in the same setting and has obtained the same or similar levels of improvement. While there is better evidence than that in the previous two levels, still there is no assurance of effectiveness (we do not know if it is better than no treatment).
Level 3 Uncontrolled directly replicated evidence
A case study has been replicated by another investigator in another setting with different clients and has obtained the same or similar levels of improvement reported by the original investigators. Although still uncontrolled, the evidence is getting stronger and is likely to show effectiveness in a controlled study.
Level 4 Uncontrolled systematically replicated evidence
The first level at which efficacy is substantiated for a treatment procedure. One of the group or single subject designs has been used to show that treatment is better than no treatment and that extraneous variables (such as maturation or family's work at home) are not responsibile for the positive changes observed. Not only improvement, but also effectiveness has been deomonstrated for the procedure.
Level 5 Controlled unreplicated evidence
The same investigator who demonstrated the effectiveness for the first time has replicated the study with new clients and has obtained similar results to document effectiveness. The technique is now known to reliably produce the effects, at least in the same setting.
Level 6 Controlled directly replicated evidence
This is the highest level of evidence. The effectiveness of a treatment technique has been replicated by other investigators, in different settings, with different clients, to show that the technique will produce effects under varied conditions. A technique that reaches this level may be recommended for general practice.
Level 7 controlled systematically replicated evidence
_____________________ is an alternative to random selection and assignment. In an experiemntal group control group design, researcher will first identify two participants who are similar to relevant variables and then assign one to the exerpimental group and the other to the control group. This ensures that those who receive treatment and those who do not are similar kinds of particpants. When individual to individual matching is impractical, groups may be matched on the basis of the statistical mean. For instance, the mean dysfluency rates in the experimental and control group in a treatment efficacy study may be comparable (e.g. 15% vs. 14.5%)
Matching
______________________ refers to biological and other kinds of changes within participants themselves; such changes can have an effect on the dependent variable. For example, a study on the effect of a language stimulation program on kindergarten children, conducted over a period of one year, runs the risk of _______________________ confoudning the results. Improved language skills in the children may be due to _______________________ to an unknown extent. Control groups in group designs and treatment reversal in SSDs help control the effects of ________________________.
Maturation maturation
________________ in its singular form refers to the field of study involved with the art and science of data analysis. __________________ deals with probability or the chance of something occurring. For example, if the probability of rain is .75, there is 75% chance that it will rain.
Statistics
______________________________ ___________________, also called ______________________ _______________________________ ____ ________ ________________, refers to a behavior that goes from an extreme high or low point to an average level.
Statistical regression statistical regression to the mean
___________________ refers to ______________________ ____________________, changes that occurs in a dependent variable simply because it has been measured more than once (e.g. the administration of pre and posttests). In such cases, the investigator may incorrectly conclude that the treatment variable was responsibiel for the change recorded. For example, attitude questinnaires participants fill out before and after treatment are highly reactive. The clients may show significant attitude changes when they fill out the posttreatment questionnaire, even though their attitudes have not changed much. The effects of testing may be minimized by measuring behaviors directly, instead of through questionnaires and interviews.
Testing reactive measures
__________________ refers to the dispersion or spread in a set of data. Common measures of _____________________ include the range, interquartile range, semi interquartile range, and standard deviation. For example, if an insructor administers an examination to 50 students and the highest score is 100 and the lowest score is 40, there is great _____________________ in that set of scores. But if the lowest grade is 90 and the highest grade is 100, the _______________________ in that set of scores is minimal.
Variability
Group designs, which depend heavily on analysis of results based on group averages, are very vulnearable to ______________________. __________________ is a serious problem when investigator uses statistical analysis based on group means and the ____________________ is differential in the experiemntal and control groups. For example, if more severely affected participants drop out of the experimental group and less severely affected participants drop out of the control group, the pre and posttest scores of the two groups may differ in favor of treatment, even if the treatment was ineffective.
attrition attrition attrition
______________________ __________________ is derived from an examination of the measuring instrument to determine if this instrument samples the full range of skills that is purports to measure. More relevant to standardized tests, ________________ _________________ is based on expert judgment that a particular test measures what it purports to measure.
content validity
The ______________________ ______________________ of a distribution or set of scores is an index of the average or typical score for that distribution. There are three measures of ___________________ _____________________: mean, median, and mode.
central tendency central tendency
There are two types of variables in some kinds of descriptive research. The ________________________ ________________ is analogous to the independent variable in experimental research, although unmanipulated, and the __________________________ _____________________ is analogous to the dependent variable in experimental research. For example, people with dementia might be compared to people without dementia on cognitive and linguistic measures. The ___________________ variable would be group status (dementia as opposed to without dementia) and the ___________________ variable would be the performance on the cognitive and linguistic measures.
classification variable criterion variable classification variable criterion
A difficulty with _____________________ research is that the similarities and differences found between groups of idnvidiuals might be due to variables other than the classification variable. For instance, in the above example, clients with head injury might perform differently than non head injured clients due to premorbid educational or socioeconomic differences, not the presence or absence of head injury (classification variable).
comparative
The purpose of ____________________________ research, also known as standard group comparison, is to measure the similarities and differences of groups of people with defined characteritics. Attention and memory skills of people with head injury, for example, may be compared and contrasted with those of individuals without head injury.
comparative
__________________________ ____________________, considered a form of criterion-related validity; is the degree to which a new measure correlates with an established measure of known validity, the concept applies most forcefully to standardized tests. For example, a new test of language skills might be correlated with a well established test of known validity to demonstrate the ___________________ validity of the new test. A moderate, positive correlation is good for the new test; if the correlation is too high, however, there may be a question of the need for the new test.
concurrent validity
________________________ ______________________ is the degree to which measures are consistent with theoretical constructs or concepts. For instance, a test of language development in children should meet the theoretical expectation that as children grow older, their language skills improve. Similarly, a study on speech sound development may meet certain theoretical expectations (e.g. systematic increase in sound learning across age groups).
construct validity
_____________________________ does not receive treatment and does not show significant improvement. The goal of having these two groups is to demonstrate that the treatment is better than no treatment- an essential condition to show that the treatment was effective.
control group
A ______________________________ is a statistical method of data analysis suggesting that two or more events are somehow associated or related. It suggests the direction (positive or negative) and the strength (high or low) of the relationship.
correlation
A negative _________________________ is found when high values of one variable are associated with low values of the other variable. When one event increases, the other event decreases. For example, increased pressure to communicate may decrease a speaker's fluency.
correlation
A positive ________________________ is found when high values of one variable predict high values of the other variable; when one event increases, the other event increases. For example, as the temperature rises on a hot July day, local grocery stores sell icnreased quantities of ice.
correlation
As previously stated, to express the strength and direction of _________________________ relationships, the Pearson product moment ____________________ coefficient is used. This is usually abbreviated Pearson r. Pearson r is a number ideally ranging from -1.0 to 1.0. Positive and negative correlation coefficient 1 in relation to 0, as shown below page 611
correlational correlation
________________________________ research tries to find relationships or associations between variables. Like descriptive research, _____________________ research cannot uncover cause effect relations. __________________________ suggests that events studied are related, but whether there is a cause effect relation would not be clear.
correlational correlation
In the _____________________________________, researchers select participants from various age levels and simultaneously or within a short duration of time sample the behaviors or characteristics of the groups formed on the basis of age. The ____________________________________ is cheaper, faster, and more practical than longitudinal research. A problem with _______________________ studies is that what is observed for a particular age group in the study may not hold good for others of the same age who were not in the study.
cross sectional method
The ____________________ method is an explain first and verify later approach. In this method, based on initial observations, the investigator explains an event by proposing a theory and then attempts to verify the theory by conducting experiments. Hypothesis testing is a _________________ method.
deductive
________________________ research is also useful in understanding a pheonemonen. For instance, ______________________ research helps us understand the nature of language disorders. It also helps us understand what are the typical behaviors of children on the autism spectrum, what behaviors are commonly manifested after a stroke, and so forth. In essence, ___________________ research helps us understand what already exists. ________________________ research permits us to actively manipulate conditions to change what exists.
descriptive experimental
The purpose of ____________________________ research is to measure changes in individuals over time as they mature or get older. In _____________________ research, the presumed independent variable is maturation, which implies that age is the cause of changes seen in people, especially developing children. _________________________ research has been used extensively to create developmental norms. Therefore, ________________________ research is also known as __________________________ research.
developmental developmental developmental developmental normative
The investigators conduct detailed observations, making copious notes and sometimes video and audio recordings of phenomena and people being studied. Without formulating hypotheses, invesitgators take an inductive approach in which observations eventually lead to conclusions. Unlike experimental research, _____________________ research is qualitative. Extensive verbal descriptions are generated, but quantitative analyses are uncommon.
ethnographic
In speech language pathology, ___________________________ research is advantageous for detailed study of clients, sitautions, and cultural groups when experimentation is not ideal or possible. Disadvantages of ___________________________ research are that it is time consuming, often expensive, yields data that are difficult to quantify, and lacks the objectivity of experimental research. _______________________ research is still relatively new in speech language pathology.
ethnographic ethnographic ethnographic f
___________________________________ _______________ involves observation and description of naturally occurring phenomena; thus, it is included under the aegis of descriptive research (albeit somewhat artificially). It is not an experimental type of research.
ethnographic research
___________________________ ____________________ was originally designed by anthropologists as a means of conducting in depth analytical descriptions of cultural scenes. The most common method of this research is for investigators to immerse themselves in the situation being studied to gain a deeper understanding of it. _____________________ researchers study existing pehnomena without altering them.
ethnographic research ethnographic
Investigations into potnential causes of disorders or diseases are typically ___________________. For example, one might investigate the potnetial causes of hearing loss in a child. In this case, both the causes and the effects have occurred in the past. A detailed history of the child may reveal that the mother, while carrying the child, had an infection that is suspectd to cause fetal ear damage. Potential causes of many diseases, including cancer or stroke, are investigated only through ___________________________ case studies.
ex post facto
True __________________________ designs are the most powerful of the group design strategies. They are useful in isolating cause effect relationships between variables.
experimental
Research involving a group design may be _______________________ or __________________________. If it is __________________________, it will consist of only one group; the research is then _______________________, or a case study if it is __________________. If it is ___________________________, it will consist of two or more groups.
experimental nonexperimental nonexperimental observational clinical experimental
___________________________________ _________________ using groups to rule out the influence of confounding (extraneous) variables through the use of experimental and control groups.
experimental designs
___________________________ _________________ receives treatment and thus shows significant changes in behaviors treated.
experimental group
Threats to _____________________ validity limit the degree to which well established cause effect relations, that is, internally valid results, can be generalized. Results of a single study-regardless of the design used- cannot be generalized to a wide variety of settings and people. For example, a study on the language skills of spanish speaking mexican children living in california cannot be generalized to spanish speaking puerto rican children living in New York or to spanish pseaking cuban children living in Florida.
external
_____________________ validity of a study can be threatened by several factors. These include the Hawthorne effect, multiple treatment interference, and reactive or interactive effects of pretesting.
external
____________________________ or _________________________ variables should be ruled out in experiments. Study conditions are carefully controlled to eliminate the influence of unwanted variables. To establish a cause effect relationship in clinical treatment research, the experimenter should rule out the influence of ____________________ variables that may also affect the disorder or the skill. For instance, in demonstrating that a child language treatment is effective, the investigator must rule out the influence of maturation and the teachers' or parents' work on the child's language skills.
extraneous or confounding extraneous
The closer the r is to 1.00, the _________________ the reliability of the test or measurment.
greater
A ______________________ is a proposed answer to a research question, but verifiable through additional research. A __________________ is concerned with a more specific prediction stemming from a theory. For example, a _____________________ might state, "When people who stutter are put in highly stressful speaking situations, their amount of stuttering will increase." In other words, the _____________________ is that stressful speaking situations will cause the effect of increased stuttering.
hypothesis hypothesis hypothesis hypothesis
An ____________________________________ is directly manipulated by the experimenter to produce changes in a dependent variable. In treatment research, for example, all treatments are ________________________.
independent variable independent variables
The second purpose of statistics is to aid the researcher in making inferences. An ______________________ is a conclusion one arrives at through reasoning. Statistical ___________________ are frequently made from samples to populations. That is, researchers make observations on a sample, or small group of people, to make a decision about a larger group. For example, investigators who have studied the effects of exercise on cholesterol levels in samples of participants have discovered that the individuals who increased their level of exercise had a decrease in their cholesterol levels. Thus, the investigators inferred that people not in the study, the larger population of people with high cholesterol, might benefit from increased levels of exercise.
inference inferences
_____________________________ refers to problems with such measuring devices as mechanical and electrical instruments, pencil and paper instruments (e.g. questionnaires and tests), and human observers.
instrumentation
This refers to the extent to which two or more observers agree in measuring an event. For example, if three judges independently rate the fluency of a speaker, there is high ____________________ reliability if there is good agreement between judges. Optimally, good agreement results in an ____________________ reliability coefficient of 0.90 or more.
interjudge or interobserver reliability
Observers used in a study also act as instruments. Like bad instruments, biased or inexperienced observers negatively affect the _______________ validity of a study. Individuals who observe glottal fry, for example, on pretests and posttests in a voice treatemnt study may render invalid scores because they lacked experience in measuring the behavior at the pretest and gained it by the posttest. Observers also may become bored during the posttest and fail to observe and score as vigilantly. Finally, criteria used by individual judges may become more or less stringent over the course of a study. Different judges may render vastly different judgements on the same phenomenon.
internal
mechanical instruments (e.g. audiometers) that are not properly calibrated may negatively affect the ________________ validity of a study in which the hearing loss is measured before and after the administration of a drug to improve hearing.
internal
Research studies can be evaluated according to the parameters of internal and external validity. __________________ validity is the degree to which the extraneous variables were ruled out and the cause effect relations revealed in an experiment are trustworthy. _________________ validity is the degree to which the results of an experiment may be extended to specific populations.
internal external
Certain factors can reduce the ___________________ validity of a study. These threats to _________________ validity include instrumentation, history, statistical regression, maturation, attrition, testing, subject selection biases, and interaction of factors.
internal internal
Standardized tests administered to inappropriate samples also affect ________________ validity of a study. For example, administering a test standardized on monolingual, English-speaking, white children to Vietnamese refugee children who speak English as a second language to distinguish language differences from language disorders would pose a serious threat to the _______________ validity of the study.
internal internal
To deal with such problems, the _____________________ range and _________________________________ range may be used. The _________________ range "cuts off" the highest and lowest 25% of the scores in a distribution. Thus, what is left is the middle 50% of the scores. The ___________________________ range is the interquartile range divided by 2. For example, if 100 students take an examination and the 25 highest scores range from 75 to 100 and the 25 lowest scores range from 0 to 25, the ______________________ range of those scores is 50, because the lowest 25 scores and teh highest 25 scores are disregarded. The ________________________________ range is 25:50 (the interquartile range divided by 2).
interquartile semi interquartile interquartile semi interquartile interquartile semi interquartile
An _____________________ _________________ of measurement is a numerical scale that can be arranged according to rank orders or levels; the numbers on the scale must be assigned in such a way that the intervals between them are equal with regard to the attribute being scaled. For example, the attribute of distance is equal between one number and another. 1 2 3 4 5 6 7 8 9 10
interval scale
This refers to the extent to which the same observer repeatedly measures the same event consistently. For example, if the same clinciian rates a child's intelligiblity over several sessions, those ratings should be consistent to assure acceptable level of ____________________ reliability (assuming, of course, the child's speech intelligiblity has not changed).
intraobserver or intrajudge reliability
Psychological and social scientists typically describe what they term ______________ _____ ________________________. Some of these ______________ _____ __________________________ are closely related to statistical scaling techniques. The four commonly used ________________ ______ ____________________ are represented by nominal, ordinal, interval, and ratio scales.
levels of measurement
In _____________________________ research, the same participants are studied over time to observe the changes that occur within them as they get older. An advantage of this type of research is that the investigator can directly observe changes in such behaviors as language skills or fluency in the same participants as they get older. Disadvantages are that it is (a) time consuming, (b) expensive, and (c) vulnerable to subject atrition. Thus, these studies often have small numbers of participants, limiting the generalizability of the results.
longitudinal research
The ______________ is the most commonly used measure of central tendancy. To calculate a _______________ , the investigator adds up the scores and divides the total by the number of scores that were added together. For instance, 60 plus 50 pluse 40 plus 80 - 230. Next, 230 divided by 4 is 57.5. Thus, the __________ for this set of scores is 57.5.
mean
The ____________________ is the score in the exact middle of the distribution. The ________________ divides the distirbution into two parts so that an equal number of scores fall above and below it, as shown: 38 42 51 57 63 69 78 85 88 90 97 The ________________ score is 69 because five scores fall below it and five scores fall above it.
median median
The ______________ is the most frequently occurring score in a distribution. This is illustrated as follows: 22 29 34 46 57 63 63 63 78 85 100 The ____________ is 63, because 63 is the most frequently occurring score in this distribution
mode
Also called _________________, _____________________ refers to the problem of losing participants as the course of an experiment and thus affecting the final results of the study.
mortality attrition
A ____________________ ______________________ __________________ _______________________ ______________________ involves several behaviors that are sequentially taught to show that only treated behaviors change, untreated behaviors show no change, and thus the treatment was effective. The researcher selects three or more target behaviors baserates those target behaviors teaches the first behavior to a training criterion (e.g. 80% accuracy over three sessions) repeats the baserates on the remaining untreated behaviors teaches the second behavior while repeating base rates on the remaining untreated behaviors continues to alternate baserates and treatment until all the behaviors are taught
multiple baseline across behaviors design
_____________________ ________________________________ __________________________ refers to the positive or negative effect of one treatment over the other when two or more treatments are evaluated in a single study and all participants receive all the treatments. For example, when people who stutter receive both counseling and time out for stuttering, administered in a certain order, the overall outcome may be a result of one treatment influencing the other and one treatment following the other (called the ______________ _____________). The results may be generalized only to those clients who receive both the treatments, administered in the same order, not to any one treatment administered by itself.
multiple treatment interference order effect
In a _________________ ________________, a category is present (e.g. hypernasality) or absent (normal nasality). Items or observations are classified into named groupings or discrete categories that do not have a numerical relationship to one another. For example, in a survey of clinical practices, respondents might be asked if they "never", "sometimes", or "always" engage in a certain clinical practice. Or, a researchers might label diagnostic categories in a nominal way as "aphasia", "specific language impairment", "dementia", "dysphagia", and others.
nominal scale
____________________________ research methods have been used extensively to establish age based norms on children's speech sound and language skills. The method has also been used to study the cognitive skills of older adults as they continue to age. Three methods are used in this type of research:
normative longitudinal cross sectional semilongitudinal
An ___________________ _________________ is a numerical scale that can be arranged according to rank orders or levels. ________________ ________________ use relative concepts, such as greater than and less than. The numbers in an _________________ _____________ and their corresponding categories do not have mathematical meaning. Also, the intervals between numbers of categories are unknown and probably not equal. Examples of ________________ __________________ of measurement are: 1= strongly agree 2- agree 3 = neutral 4 = disagree 5= strongly disagree 1 = little hoarseness 3 = moderate hoarseness 5= great hoarseness
ordinal scale Ordinal scales ordinal scale
A _____________________ is a large, defined group (e.g. patients scheduled for laryngectomy surgery, people who stutter) identified for the purpose of a study.
population
The closer a number is to 1.0, the stronger and more ___________________ the relationship. The closer a number is to -1.0, the stronger and more _____________________ the relationship. The closer a number is to 0, the _________________ the relationship. An r of 0.35 suggests a weaker relation between two events than an r of 0.95.
positive negative weaker
There are different kinds of validity:
predictive validity, criterion validity concurrent validity construct validity content validity
The __________________________________________ is the basic experimental group design to establish casuality. It has two groups: an experimental group and a control group. This design helps evaluate the effects of a single treatment. The participants should be randomly selected and assigned to groups, although in practice randomization is compromised at various levels. Each particpant in each group undergoes a pretest and a post test.
pretest-posttest control group design
In contrast to retrospective research, one might design a study that is _______________________. Smokers who do not have lung cancer may be followed up for a decade to see how many develop that type of cancer.
prospective
The _______________ is the difference between the highest and the lowest scores in a distribution or set of scores. For example, a test on which the highest score is 100 points and the lowest score is 40 points, the _______________ is 60 points. One hundred (highest score) minus 40 (lowest score) yields the _____________ in that set of scores.
range
A ____________ ___________________ has the same properties as an interval scale, but numerical values must be related to an absolute zero point. The zero suggests an absence of the property being measured. An example of a ___________ _________________ is one that involves frequency counts in stuttering; it is possible to have zero instances of stuttering in a speech sample. A ratio scale for the attribute of distance is shown below. 0 1 2 3 4 5 6 7 8 9 10
ratio scale
In the ___________________________________ , the total age range to be studied is divided into several overlapping age spans. The participants selected are those who are at the lower end of each age span, and they are followed until they reach the upper end of their age span. For example, researchers might select three groups of participants and study the language development of those groups for 1 year. The groups might be as follows: 3 year olds followed until the age of 4; 4 year olds followed until the age of 5; and 5 year olds followed until the age of 6. By the end of the study, the researcher would have made observations both between and within participants as time passed.
semi-longitudinal procedure
Both the pre and posttests might __________________ the study particiapnts so that the treatment effect is enhanced. Unlike testing that directly changes behaviors as discussed under internal validity, _________________________ makes the participants more receptive to treatment; the final outcome is partly determined by this _______________________. __________________________ itself does not change the beahvior, but makes it more likely to change with the treatment. For example, before participating in a voice therapy program to reduce vocally abusive behaviors, the participants might fill out a questionnaire that assessed how often they abused their vocal mechanism. This pretest may not have changed their vocally abusive beahviors (hence no testing effect), but they may have become more receptive (sensitized) to treatment. When they take a posttest, they may simply recall some of the suggestions on reducing vocally abusive beahviors that were offered during treatment. Study results may be generalized only to those who take similar pre and posttests.
sensitize sensitization sensitization sensitization
The ____________________________ _______________________ is the extent to which scores deviate from the mean or average score. ______________________ ________________________ reflects the variability of all measures or scores in a distribution. The larger the ______________________ ______________________, the more variable the scores, and vice versa. For example, if on an instructor's test, the highest score is 100 and the lowest score is 50, that 50 point difference in the highest and lowest scores results in a large _____________________ ___________________. If on another test, the highest score is 100 and the lowest score is 85, that 15 point difference between the highest and lowest scores results in a relatively small _______________________ ________________________.
standard deviation Standard deviation standard deviation
The singular ______________________ is a measure or a number calculated from a sample. This differs from a _______________________, which refers to a characteristic of a population. Examples of statistics, or summaries of samples, include percentages, means, variances, standard deviations, and other measures calculated from samples.
statistic parameter
When used in the plural form, ____________________ refers to specific pieces of data that will be or have been gathered. For example, baseball statistics might include a baseball player's number of home runs and strikeouts.
statistics
To control for ___________________ ___________________________ ____________, it is best to use randomly selected and assigned groups. Another solution is to use groups that have very carefully matched participants- that is, participants who are similar on important variables. As the SSDs do not use group comparisons, selection bias is not significant in them.
subject selection biases
____________________ _________________________ ___________________ are subjective factors taht influence the selection of who participates in a study. A key feature of internally valid experiments is that differences found between experimental and control groups on posttests are attributable only to the treatment. However, if the two groups were different to begin with, the initial differences in the participants themeslves may account for the differences in posttest measures.
subject selection biases
TV shows on dance or music competition that ask viewers to vote illustrate both the strengths and weaknesses of _______________ research. Thousands of votes are generated, but those who vote tend to be people who are very strongly biased toward one performer or another. People who have a neutral stance toward the performers and their success or failure generally do not cast votes.
survey
An advantage of __________________ is the wide range of data that can be obtained. For example, one can literally sample thousands of people through a _________________. The problems of ________________ are that (a) they cannot be used to illustrate cause effect relationships, and (b) their samples are often biased because those who return them may not be comparable to those who do not.
surveys
An r value of __________ indicates that there is no relationship between two measures. The highest possible positive value of r is _________. Conversely, the lowest possible negative value of r is ___________.
0.00 1.00 -1.00
A major limitation of group experimental designs is that it is not always possible to randomly draw participants from specific clinical populations to which clinicians do not always have access.
K
All professional practices are built upon a foundation of basic and applied research. The current emphasis on evidence based practice in speech language pathology requires an understanding of the philosophy of science, methods of basic and applied research, techniques of data analysis, and the constraints on drawing valid conclusions from data.
K
Another limitation of group experimental designs is that the results may be extended to groups but not individual clients. Most practictioners wish to extend treatment study results to their individual clients; SSDs fare better in this respect.
K
Because most readers will have completed coursework on research designs, statistics, and evidence based practice, the information presented in this chapter serves to refresh reader's knowledge and strengthen their understanding of critical concepts.
K
Because researchers cannot assess an entire population, they survey a randomly selected, representative sample of that population. EAch person in the sample is asked a set of questions of interest designed to evoke answers. (surveys)
K
Descriptive research is useful when it would be unethical to use experimental methods. For example, to find out the effects of maternal drinking on the baby's eventual language skills, researchers would not tell one group of pregnant women to drink alcohol daily and tell a second group of pregnant women not to drink alcohol. However, researchers can observe and measure the language skills of babies born to women who consumed notable quantitities of alcohol during pregnancy and describe those language skills. Children of a second group of women who never drink alcohol during their pregnancy may provide comparative data.
K
Experimental and control groups are equal when random selection and random assignment are done properly. The two levels of randomization reduce experimenter bias in selecting particpants and ensure that the sample is representative of the population.
K
Experimental research investigates cause effect relationships. Experimenters manipulate independent variables to assess the effect of these variables upon dependent variables. Group designs, the most popular type of experimental design in medicine and speech alnguage pathology, usually involve control and experimental groups. Single-subject experimental designs (SSDs) are often used in treatment evaluations in behavioral and clinical sciences, SSDs usually involve 6 to 10 participants and help establish cause effect relations based on individual performance as opposed to group averages.
K
First, clinicians can integrate research and clinical service by using the clients they serve as participants in experiments that attempt to answer significant clinical questions. Except for the ABA design, clients who participate in a study receive treatment; they are not denied treatment as clients in a control group might be if a group design is used.
K
Most measures of relability are expressed in terms of a _________________________ _______________________. The _____________________ _____________________ is a number or index that indicates the relationship between two or more independent measures. It is usually expressed through Pearson Product Moment r (often referred to as r).
correlational coefficient correlational coefficient
____________________ _______________________, also called ____________________ ____________________, is the accuracy with which a measure predicts future performance on a related task. The concept applies to both standardized tests and research study results (measures). For example, the performance on the Graduate Record Examination (GRE) should predict the GPA or other measures in graduate programs. Measures reported in a research study on language development in children should be able to predict the developmental stages in children not sampled in the study.
Predictive validity criterion validity
______________________ are participants' existing behaviors or skills measured before starting an experimental treatment or teaching program. ___________________ are measures of behaviors established after completing the treatment program. A comparison of these measures helps demonstrate the effects of treatment while ruling out the influence of confounding variables by showing that only the treated group changed and the control group remained at or close to the pretest. The participants are selected and assigned to the two groups randomly. Ideally, the _______________ measures of the two groups are similar, and if the treatment was effective, the ______________ measures will be significantly different.
Pretests Posttests pretest postteset
__________________________ is the degree to which an instrument measures what it purports to measure. For example, a ___________ child language test should measure language skills, not auditory memory. In research studies, the skills or behaviors measured should be relevant to the research question asked. For example, in an investigation of stuttering treatment, stuttering, not the self-concept of the participants, should be measured.
Validity valid
A ______________________________ or _______________ is the variable that is affected by manipulation of the ___________________________________. In treatment research, all disorders are _______________________________. Manipulation of treatment ((___________________________) changes the _____________________________ (disorder or a particular skill). _________________________________ must be defined very specifically so that they are measurable. Table 14.1 contains examples of independent and dependent variables.
dependent variable effect independent variable dependent variables independent variable dependent variable dependent variables
In ________________________________, researchers observe phenomena of interest and records their observations. The researchers do not want their presence to interfere with the natural phenomena that are being observed.
descriptive research
The most significant limitation of ___________________ research is that it can only suggest, but not confirm, causes of disorders and diseases.
ex post facto
Scientific data should meet two criteria of ____________________ and _______________________. ____________________ and ______________________ are critical aspects of any type of scientific measurement, including measurement through standardized tests. Measures in speech language pathology, whether they apply to research studies, clinical measurement of clients' skills, or standardized and nonstandardized tests, need to be _________ and ___________, as defined in the next section.
validity reliability validity and reliability valid and reliable
The _______________________ is the basic experimental SSD. The ____________________________ is an extension of the basic design. Both are useful in establishing treatment efficacy. In both designs, the first A condition refers to baselines, the next B condition refers to treatment, and the third A condition refers to treatment withdrawal. In the ______________________________, the fourth B condition refers to the reinstatement of treatment.
ABA design ABAB design ABAB design
The _________________________ by adding the final treatment reinstatement condition, ends with treatment; clients benefit from the experiment. The final phase may be continued as long as it is necessary to stablizie the newly established skills. By showing that the baserated stable and low level skills (A) increased markedly under the new treatment (B), but decreased when the treatment was withdrawn (A) and increased again when the treatment was reapplied (B), we demonstrate that treatment was indeed effective (see Figure 14.2 for a graphic representation of the results of a hypothetical study using the ABAB design.)
ABAB design
The _______________________ consists of four conditions: During the first A condition, the target skills are baserated without treatment During the second B condition in which the new treatment is offered During the third A condition, the treatment is withdrawn During the fourth and final B condition, the same treatment is reinstated
ABAB designs
For example, an investigator seeks to study the effects of enriched maternal speech on infant language development. She forms a control and an experimental group of infant mother pairs by randomization or matching, most likely matching. The investigator must match the mothers on such characteristics as age, socioeconomic status, language background, and educational level. The experimental group of mothers is taught specific strategies for enriching their infants' language; the control group of mothers is not taught those strategies. The vocalizations of babies in the experimental and control groups are measured before the experiment (pretest) and after the experiment (posttest). If vocaliations increased only in the babies of the experimental group then the enriched maternal speech is the cause of that increase, because in its absence, the vocalizations of babies in the control gorup did not increase.
K
If the study demonstrates that the baserated skills were stable in the first A condition, increased when the treatment was applied in the B condition, and decreased when the treatment was withdrawn in the second and final A condition, then it may be concluded that it was the treatment, and no other variable, that was responsible for the increase in skill elvels during B treatment condition (see Figure 14.1 for a graphic representation of hypothetical results from an ABA design study). Although the design helps establish treatment efficacy, it ends with no treatment, and hence no benefit to the participants. The next design avoids this problem.
K
In evaluating the relative effects of three treatments that are known to be effective, for example, the investigator reandomly selects a sample from the population and randomly assigns them to one of three groups. Each group receives a different treatment. If an optional control group is included, the investigator will know whether treatments are better than no treatment and whether one treatment is better than the other. If no control group is included, the investigator can only answer the question: Which treatment is better?
K
In the group designs, the average performance of one group is analyzed or those of two groups are compared.
K
Science developed out of curiosity and a need to understand the world around us. The beginnings of science were based in the philosophical inquiry into truth. Therefore, philosophy, especially empirical philosophy, is the basis of modern science.
K
Scienstists use objective, experimental methods to systematically investigate research questions. Being the means of answering such questions, the research methods involve observation, measurement, and manipulation of some variables to produce valid and reliable data from which objective conclusions may be drawn.
K
Speech language pathology and audiology are clinical sciences that build upon research evidence gathered by scientists in several disciplines. Today's clinicians are expected to be critical consumers of research so that they may select methods of assessment and intervention that are based on scienctifically replicated evidence for thier usefulness and effectiveness.
K
The results of a single or few SSD studies cannot be generalized to the population. However, the results may be extended to individual clients who have the same profile as those in the research study. However, generality may be achieved by replicating the experiment with different participants in different settings by different investigators.
K
There are several group experimental designs. Many advanced designs are variations of the basic designs. We will describe one basic design that helps evaluate the effects of a single treatment and the other basic design that helps evaluate more than one treatment.
K
Unlike experimental research, no variety of descriptive research can lead to cause-effect statements.
K
Well-conducted experimental group designs have strong internal validity (described later in this chapter, in the section dealing with evaluation), in that extraneous or confounding variables are ruled out. Thus, the experimenter can be confident that it was indeed the manipulation of the independent variable (treatment) that caused the change in the dependent variable (improvement._
K
While the goal of experimental research is to explain the effects by finding their causes, the goal of descriptive research is to describe phenomena. There is no manipulation of variables (e.g. offering treatment) in descriptive research; hence, it cannot demonstrate cause-effect relationships. Varieties of descriptive research include ex post facto research (retrospective research), survey research, comparative research (standard group comparisons), developmental (normative) research, correlational research, and ehtnographic research.
K
________________________ is the consistency with which the same event is measured repeatedly. Scores are reliable if they are consistent across repeated testing or measurment. A standardized test given to the same individual on two occasions should result in similar scores. A clinician who measures stuttering in a client on two consecutive days should record similar rates of dysfluencies. The number of misarticulations baserated by a researcher before administering an experimental treatment should be comparable. Measures will not be trustworthy without that kind of consitency across two or more measures.
Reliability
_____________________ is what scientists do as they practice science; it is science in action. Research is the process of asking and answering questions; it includes steps scientists take as they search for uniformity and order in nature. Science is conceptual and philosophical whereas resaerch is methodological.
Research
_______________________________ case studies also help study rare, individual cases of diseases and disorders in depth. For example, a child with Russell Silver syndrome ( a type of dwarfism) would be an ideal subject for a case study because the syndrome is rare and almost no literature exists that describes communication disorders in that population.
Retrospective
__________________ is a philosophy of events and nature that values evidence more than opinions. The scientist uses objective, experimental methods to systematically investigate research questions and produce valid and reliable results that help answer them.
Science
This is a measure of internal consistency of a test. REsponses to items on the first half of a test are correlated with responses given on the second half. Or the responses to even numbered items may be correlated with responses to odd numbered items. This reliability generally overestimates reliability because it does not measure stability of scores over time.
Split half reliability
_________________ assess some characteristics of a group of people or a particular society. ________________ attempt to discover how variables such as attitudes, opinions, or certain social practices are distributed in a population.
Surveys
This is based on the consistency of measures when two parallel forms of the same tests are administered to the same people.
alternate form or parallel form reliability
_____________________________________ research is after the fact research. Because the investigator makes a retrospective search for causes of events that have occurred, it is also called ___________________________ research. In most clinical disciplines, _______________________ studies also may be called ______________________________ studies. Researchers begin by defining the effect as it currently exists. They then look backward in an effort to describe potential cause or causes.
ex post facto research retrospective ex post facto casual comparative studies
An ______________________ is a means of establishing ___________________ relationships. __________________ test if then relationships. For example, if a man who stutters is put into a highly stressful situation, then will he stutter more? Does the highly stressful situation cause the effect of increased suttering?
experiment cause-effect experiments
_____________ designs and ____________________designs are the two types of experimental designs used in speech language pathology.
group signle subject
The __________________ method is an experiment first and explain later approach. The researcher first observes the phenomenon, conducts a series of experiments on it, and then proposes a theory based on the results of those experiments.
inductive
Scientists frequently conduct research with the goal of explaining events and effects. An explanation of an event specifies its causes, and an event is scientifically explained when the scientist experimentally demonstratees its cause(s). To explain events, scientists may use one of two approaches: the ________________ method or the _________________________ method.
inductive deductive
A _________________________ ____________________ ___________________ __________________ _______________ involves a single behavior being sequentially taught in different settings to demonstrate that the behavior changed only in a treated setting, and thus treatment was effective. The researcher baserates a target behavior in three or more settings (e.g. the hospital room, hospital lobby, parking lot, and home) teaches the behavior in one setting repeats the base rates in the remaining untreated settings teaches the behavior in another setting continues to alternate base rates and teaching in different settings until the behavior is taught in all settings
multiple baseline across settings design
A ____________________ _______________________ _______________ __________________ ___________________ involves several participants who are taught one or more behaviors sequentially (in a staggered fashion) to show that only the behaviors of treated participants change; those of untreated participants do not change. This outcome, too, demonstrates that the treatment was effective. The researcher selects a target behavior to be taught to three or more participants baserates the target behaviors in all participants before treatment is applied treats one participant while repeating the baserates on the untreated participants treats the second participant while repeating the base rates on the untreated participants alternates treatment and base rates until all participants receive treatment
multiple baseline across subjects design
A single subject design that avoids the disadvantage of treatment withdrawal is the ________________ ______________________ _________________ in which the effects of treatment are demonstrated by showing that untreated skills did not change and only the treated skills did. The three variations of the multiple baseline design are across subjects, across settings, and across behaviors.
multiple baseline design
Hypotheses may be _______ or __________________. _______ means zero, and a _____________ _____________________ states that two variables are not related. An __________________________ ________________________ states that the two variables are indeed related; perhaps one is the cause of the other.
null alternative null null hypothesis alternative hypothesis
In the example just given, the __________ ____________________ would be that stressful situations and stuttering are not related, that when people who stutter are placed in highly stressful speaking situations, stuttering does not increase. The ______________________ _____________________ is that there is indeed a cause effect relationship between stressful situations and stuttering. That is, highly stressful speaking sitautions do cause an increase in stuttering.
null hypothesis alternative hypothesis
The two most common types of survey research tools are ____________________________ and ________________, although the two may be combined. Personal _______________________ help obtain in depth answers from sruvey participants. _____________________________ filled out by survey participants may yield a greater quantitity of responses, but those responses may not contain the depth of those given in ____________________.
questionnaires interviews interviews questionnaires interviews
__________________________________ is one method of forming two or more groups for a study. The researcher first randomly selects a sample from the population. The selection is random when each potential participant in the population has an equal chance of being selected for the study.
randomization
Randomly selected sample participants are then _____________________________ to treatment and control groups. The assignment is random when each selected particpant has the same chance of being selected to one or the other group.
randomly assigned
The researcher hopes to ___________ the null hypothesis and ____________ the alternative hypothesis, because the researcher usually believes in the alternative hypothesis. The alternative hypothesis holds that there is a specific relationship between the variables specified.
reject accept
The multigroup pretest-posttest design helps evaluate the relative effects of two or more treatments. A question of _________________ _________________ asks: Of the two or more, which treatment is more effective? Each treatment requires an experimental group, as shown below. The illustrated design helps evaluate relative effects of three treatments. The participants are selected and assigned randomly. Each group receives a different treatment. The control group in this design is optional; hence, it is shwon in parentheses.
relative effects
Usefulness of ___________________________ studies is that they help investigate phenomena that are not amenable to experimental manipulation at the human level. For example, researchers would not ask some people to smoke and others not to and then observe over the years to see if those who smoked developed lung cancer. Thus, in many cases, ______________________ research is the only option that is ethically possible.
retrospective ex post facto
A _________________ is a smaller set that is representative of the population.
sample
The _______________________ experimental strategy demonstrates treatment effects by contrasting conditions of no treatment, treatment, withdrawal of treatment, and other control procedures. All of these conditions are aplied to all participants; there are no control participants who do not receive treatment.
single subject design
To extend the conclusion of _________________ studies, investigators need to replicate studies in different settings with different particpants. Replications are needed to extend conclusions to the population even when group designs are used, however.
single subject design
_______________________ studies are not efficient in predicting the behaviors of groups of individuals; they are good at intensively studying individuals.
single subject design
Most ___________________________ do not necessarily have a single particpant; most have multiple particpants. Each participant's data are analyzed separately and displayed graphically; no group averages are used.
single subject designs
There are many types of ______________________; three designs- the ABA design, the ABAB design, and the multiple baseline design- are used most often.
single subject designs
Unlike group designs, _________________________ allow extended and intensive study of individuals and do not involve comparison based on group performances.
single subject designs
________________________ help generalize from research studies to individual clients. Detailed descriptions of individual client characteristics and treatment effects can help establish generalizability of treatemnts across a wide variety of clients.
single subject designs
_________________________ are much more easily replicated than group design studies, because the difficulty of randomly selecting and assigning a large number of people is avoided.
single subject designs
_________________________________ designs compare the performance of the same participant under different conditions of an experiment.
single subject designs
Most ___________________________ are experimental. However, as in the group design strategy, there is a single subject design, called an AB design, which is not experimental and is similar to the _____________ _____________ that is used also in group design strategy. The routine clinical work in which the baselines are established (A), treatment is offered, (B), and the progress is summarized at the end is a _______________________. Based on a _____________________ results, the clinician can claim improvement but not effectiveness for the procedure.
single subject designs case study case study case study's
_______________________________________ are playing an increasing role in establishing efficacy of communication treatment procedures. These designs help establish cause effect relations based on __________________ performances under different conditions of an experiment.
single subject designs differential
Instead of the pre and posttests of group designs; _______________ measure the dependent variables continuously and repeatedly to establish their reliability. Participants are not necessarily randomly selected, and the results of the ____________________ are not always analyzed statistically.
single subject designs single subject designs
This refers to consistency of measures when the same test is administered to the same people twice. When the two sets of scores are positively correlated, the stability of the scores over time is assumed.
test retest reliability
There are several types of reliablity of a measure or a test:
test retest reliability alternate form reliability (parallel form reliability) split half reliability interobserver or interjudge reliability intraobserver or intrajudge reliablity
Scientists use inductive and deductive reasoning to build theories. A ______________ is a systematic body of information concerning a phenomenon, describing an event, explaining why the event occurs, and specifying how the _______________ can be verified. A _____________ specifies casual variables; a ______________ states taht X causes Y.
theory
