COSD 601 Single-Subject Design
assessment of internal validity guidelines:
1. determine if subjects have been adequately described 2. IVs and DVs must be operationally defined in enough detail to permit accurate replication of the study 3. DV must be appropriate for the communicative disorder under consideration - what type of measurement and respective criterion levels are most appropriate? 4. the effects of the IV need to be isolated from confounding variables 5. methodology must be consistent 6. statistical analyses must be appropriate and justified - tie it back to the 3 criteria for using inferential statistics 7. determine if reasonable explanations for and appropriate conclusions from the results are provided and suggestions for future research --> to build knowledge and progress the research
if using inferential statistics, there must be fulfillment of the following assumptions:
1. observations within a segment or phase are normally distributed 2. variances between or among segment(s) or phase(s) are equal 3. observations within a phase or segment are independent **the first two follow parametric statistical analysis criteria
4 key components to a single-subject design:
1. operational specificity 2. repeated measures 3. interobserver agreement 4. external validity
other designs include the classic _____________, which is better than the A-only design
A-B design (before-and-after)
with descriptive designs, we are looking for...
a shift in behavior, not for significant differences
descriptive designs - results:
are typically plotted on a graph with the behavior (DV) on the y-axis and time represented on the x-axis
direct measurement
assesses the actual performance
A=
baseline
multiple-baselines-across-settings design - method:
baseline and treatment are applied to one setting, and when performance reaches its criterion, the target behavior is measured sequentially during baseline and treatment in other settings (e.g., home, school, involving parents and teachers)
multiple-baselines-across-behaviors design - method:
baseline measures are obtained on both behaviors simultaneously until stability is attained, at which a time treatment is applied to behavior A while behavior measurement of baseline is continued on behavior B; when performance on behavior A has reached its criterion, treatment is initiated on behavior B and continues until a change occurs, demonstrating an effect on both behaviors
observing behavior over time without manipulating treatment or conditions is called a
baseline-only design (A-only); **also known as longitudinal diary study or case study
withdrawal and reversal design
baseline-treatment-baseline/withdrawal (A-B-A); simplest option that qualifies as experimental
repeated measurement involves assessing:
behaviors often and consistently over time --> taking multiple data points for a better indication of what the client does over time
multiple baseline design: can establish multiple baselines across
behaviors, settings, subjects, and/or groups
operational specificity
both IVs and DVs must be specifically defined so that others can replicate the study or correctly apply treatment protocol; IV (treatment protocol) must clearly define the: who, what, where, when, and why of a treatment (e.g., instructions to the patient, visual/tactile stimuli, response time provided, parameters of the response, and feedback given)
interval scoring
breaking an observation period into consecutive, equal segments of time and then counting occurrences of the target behavior during each portion
characteristics of single-subject design
can be descriptive or experimental
the use of statistics in single-subject research is somewhat
controversial; **some believe simple observation of trends is sufficient enough regarding treatment effects; others do not
A-only designs and A-B designs are ____________ in nature
descriptive
disadvantages of single-subject design:
difficulty in controlling for order/sequence effects; generalization to a population generally not possible; statistical procedures assessing reliability of data not well developed
can use a combination of
direct measurements
advantages of single-subject design:
ease of use by clinician-researchers; employment across various behaviors, subjects, and settings; flexibility in design before and during the experiment; identification of "functional relationships" between variables; determination of factors that do/do not affect performance; exploration of intra- and inter-subject variability; detection of individual differences; examination of individual responsiveness to treatment; application to participants with co-morbidities or unusual forms of a disorder
multiple baseline design: treatment does not need to be removed in order to
establish the effectiveness of a therapy regiment
multiple-baselines-across-settings design - purpose:
evaluate the effectiveness of a treatment for a target behavior in the same subject(s) across treatment settings
advantages of group designs:
execution of protocol no more than once per participant; ease in controlling sequence effects; statistical power for making inferences between cause and effect; statistical procedures for assessing reliability well developed; generalization from subject sample to population
behaviors measured during this "baseline" phase theoretically predict the _______________ of the behavior, assuming that no treatment is provided
future status
external validity
generalizability of results; dependent on the internal validity of the study
multiple-baseline-across-settings designs offer an opportunity to demonstrate
generalization of achievement across criterion levels of performance of the target behavior in a different environment, such as home, classroom, playground, and other relevant settings
statistical procedures: can use t-tests of ANOVAs to determine...
if the mean of the observations are significantly different between or among the different segments or phases of the study
multiple baseline design determines
if treatment can be generalized
treatment only (B-only) design is typically used in
intervention studies
experimental
involve observing patients' behavior before treatment, during treatment, and then after treatment and seek to establish causality between changes in the DV and treatment
descriptive
involve observing trends in patients' behavior over time without experimental manipulation
disadvantages of withdrawal and reversal design
many treatment effects for communicative disorders are longstanding, and the removal of treatment may not result in a return to baseline performance; the removal of treatment can be viewed as unethical (because patients must relapse in order to demonstrate effectiveness)
direct measurement - need criterion level of performance, which is an indication of...
mastery of a particular goal and must be achieved over the course of at least 2 data collection sessions
duration measures
measure how long a particular behavior lasts - this measure is rare in our field
interobserver agreement
must be consistent in the way that you take measurement; have someone else take data and score it - you want to see consistent results; high intra-observer + inter-observer agreement = high consistency
trial scoring
proportion of number of times a target behavior occurred per the number of opportunities is calculated as percent correct
rate measures
ratio of the number of times a target behavior occurred per period of time
disadvantages of group designs:
requirement of ten or more subjects; limitations of number of behaviors under consideration; application to excluded subjects or across service-delivery sites; requirement of riding experimental control in "ideal" conditions; lack of flexibility in design during experiment; generalizations made to a theoretical "average" member of a particular population
C=
second treatment that is different from the first
advantages of the withdrawal and reversal design
simplicity for determining the effectiveness of intervention
direct measurement and testing effects
space out the assessment
operational specificity: measurement of the DV (target behavior) must be...
specifically defined (direct or indirect measurement)
time sampling
sporadically observing target behaviors for specific periods of time during therapy, such as three one-minute intervals during a thirty-minute session
A-only designs provide information regarding _________ of behavior over time
stability; **e.g., increasing at different rates, changing in level, stable, highly variable, slightly variable
frequency measures
tally or number of times a behavior occurs during an observation period
multiple-baselines-across-behaviors design - purpose:
test the effectiveness of a treatment in changing two functionally independent behaviors within the same subject
we typically see A-B-A designs in medical settings, not in communication disorders. why?
the behaviors that we are treating should be maintained post discharge, the patient should not return to baseline
multiple-baselines-across-subjects design - assesses:
the effectiveness of an intervention on changing the same behavior(s) across a group of homogenous participants
indirect measurement
those not directly observable or physiological in nature and are measured in the laboratory setting
descriptive designs - primary objective:
to determine if treatment can influence behavior
B=
treatment
more complex designs involve adding a _________ and observing differences in behavior between the baseline and treatment phases
treatment (B)
direct measurement includes:
trial scoring; rate measures; frequency measures; interval scoring; time sampling; duration measures
experimental designs are used to assess:
whether the treatment results in improved performance relative to no treatment
multiple baseline design - important consideration:
whether two functionally independent behaviors, compatible settings, or homogenous participants can be identified for the study; do all participants experience the same results?
descriptive designs are ideal for exploring questions like:
will training on specific communication strategies improve the conversational fluency of an adult with sensorineural hearing loss?; can a relaxation technique reduce the number of disfluencies experienced by a stutterer when delivering a presentation?