Scientific Thinking Exam 2

Monitoring for Error Variance Criteria for Monitoring Error Variance Visual Inspection of Graphic Display for: Trend -Refers to ____ of ____ across data path from beginning to end of phase -Is there a ____ of ____ evident during baseline? During treatment? -A phase: Absence of trend → Behavior appears relatively stable (graph) -B phase: Increasing trend evident (graph) Level -Refers to the ____ ___ of data path within phase -Example: Imagine horizontal line balancing data points within phase. Where would you place that line for baseline? For treatment? -Differences in level indicate _____ of change

-pattern of change -pattern of change -average value -magnitude

Types of Single-Subject Designs Many types, but will focus only on basic prototypes: 1. _____ 2. _____ 3. _______ 4. ____ ____

1. A-B 2. A-B-A 3. A-B-A-B 4. Multiple Baseline

Within-subjects and Between-subjects Designs: Notation System VIDEO Group 1 R O1 X1 O2 Group 2 R O3 X2 O4 Group Designs -Like single-subject designs, group designs use a ____ ____ to describe their blueprint for arranging events then observing events -Instead of A's and B's, group designs use ___ and ___ Within-subjects and Between-subjects designs: Notation System -Within-subjects and Between-subjects are ____ designs -Several subjects are studied as a "group" rather than as ____ or ____ ____ -Group designs are diagrammed differently than single subject design

Group Designs -notation system -X's and O's Within-subjects and Between-subjects designs -group -individuals or single subjects

In toto, observational research -Though limited in ability to provide clear evidence of ____-___-____ relationships -Still provides important and relevant information that may guide our understanding of ____ ____ and how to best help those who seek our ____ ____

-cause-and-effect -communication disorders, professional guidance

In sum, science appreciates that -Expectations can influence ____ and ____ -Thus, procedures such as blinding or intentional ignorance are implemented to ______ their influence -And to better understand _____ _____ -Note: blinding may be implemented in other ____ when it makes sense to do so Example: Gillam et al. (2008). The efficacy of Fast ForWord Language Intervention in school-age children w lang impairment: a randomized controlled trial. Def of language impairment (LI) = difficulty learning/using lang., despite adequate hearing, nonverbal intelligence, and motor abilities, 7% of all school-age children Purpose: Is Fast Forward Language Intervention superior to other treatment approaches for treating LI in children? Reasons: Fast Forward widely publicized in media as groundbreaking, reports by developers showed dramatic lang improvements, and claims that it resulted in "neural reorganization" Will Fast Forward live up to claims of developers when investigated by independent researchers? Participants: 216 children w LI Age range = 6;0-8;11 years (avg. age = 7;6) Randomly assigned to following 4 conditions within each site: Fast Forward Lang. Intervention (FFW-L) - Experimental Tx Computer-based intervention Play games requiring participant, using headphones, to discriminate stimuli such as tones, speech sounds, words, sentences Stimuli acoustically modified by prolonging segments and amplifying certain frequencies Receive feedback for responses plus rewards such as points and jingles No verbal responses required Computer assisted language intervention (CALI) - Tx w subtraction Computer based intervention Very similar to FFW-L Except none of the stimuli are acoustically modified No verbal responses required Individualized language intervention (ILI) = standard practice Tx Face-to-face intervention w SLP Target lang skills w activities related to topic and content of children's books Includes interactive verbal exchanges b/w clinician and children Verbal responses required Academic enrichment (AE) - nonspecific treatment control Computer-based games to teach math, science, and geography No language skills targeted No verbal responses required Exs of games included: Magic school bus discovers flight Zurk's rainforest adventure Dinosaur 3D Method: Additional considerations Tx 6 weeks long summer program for all conditions for 3.5 hours a day w play, group activities, and snacks Treatment occurred for 1 hr. 40 min each day Attendance monitored All sessions videotaped Treatment fidelity established Main dependent variable: Comprehensive assessment of spoken language (CASL): Norm-referenced, standardized test of receptive and expressive language skills Well-established reliability and validity Also included auditory processing measure, but will ignore here Dependent variable assessed on four occasions: Pretreatment Posttreatment 3 month after treatment 6 months after treatment Triple-blind condition: Principal investigators (authors of study) Data collectors and analyzers Parents of children Only research coordinator at each research site knew treatment assignments Clinicians knew treatment they were administering, but unaware of study's specific purpose Design: Randomized treatment comparison design w blinding and nonspecific control Method: Unintended non-equivalent groups? No significant differences b/w subjects on CASL across conditions prior to treatment Attrition? 3 subjects dropped out 10 subjects remained but missed 1 or more assessments 203 subjects were assessed on all 4 occasions Results: Let's see if we can predict likely conclusion Children in all 4 conditions improved significantly on CASL FFWL outcomes no different than other three interventions Percent of children who moved into normal range on CASL (i.e., 83+): FFW-L = 59.3% CALI = 64.8% ILI = 75.9% AE = 70.4% Conclusion: FFWL outcomes no better than other interventions; thus failed to live up to developers' inflated claims But why did all children improve? What could be alternative explanation? No treatment control group and little kids sometimes get better on their own "maybe all four conditions were equally ineffective..." It is unclear if maturation or testing could be possible alternative explanations for all improvement But, for ethical reasons, this control group was excluded However, percent of children who moved to "normal" range suggests Tx most likely reason for change AND why did AE children improve? Don't know: Perhaps, nonspecific factors such as more social interaction in class AE computer games improved test-taking skills on CASL Or special attention from staff b/c children were involved in study Final consideration: -Beware treatment developers' claims of inflated treatment benefits -Think critically about claims that seem too good to be true -And if possible, wait for science to confirm or deny

-perceptions and behavior -minimize -clinical reality -designs

Within-subjects Design: Examples 1.____-____ 2.____ _____ 3. ______

1. Pretest-Posttest 2. Multi Treatment 3. Crossover

Qualitative Data: Techniques Examples of techniques 1.) Content Analysis: refers to analyzing written or spoken record for occurrence of ____ ____ or _____ -Usually ____-_____ approach -In other words, usually ______-driven -Operational definition of terms --> Clearly define what _____ means -Select unit of analysis --> ____, ____, ____ -Rater reliability --> Do raters ______ -Validity of results --> Are results consistent with a)what subjects _____ and/or b) other ___ ____ Content analysis technique Q. describe your stuttering for me. " I repeat and block out certain sounds. I can tell which words I am going to stutter on, so I substitute words I can say for words I can't say. Speaking with strangers always presents a problem for me." 1. Personal factors (PF) -Individuals' style of ______ with _____ 2. Activity limitations (AL) -Problems experienced in _____ ______ situations 3. Impairment (IM) -Difficulties in _____ ______ of speech "Repeat and block certain sounds" → impairment "Substitute words" → personal factor "Speaking with strangers always presents a problem for me" → activity limitation 2.) Thematic Analysis -Analyze written or spoken record for common _____ and _____ and label accordingly -____-____ approach --> In other words, ______ rather than hypothesis-driven Considerations -Select unit of analysis --> __, ____, _____, ____ of _____ -Operational definition of themes --> Clearly define what _____ mean as they emerge -Thematic reliability --> Do independent judges agree that ____ matches ______ -Validity of results --> Are results consistent with a) what ____ _____ B)other _____ _____

Content Analysis -specific categories or behavior -top-down -hypothesis -category -words, phrases, sentences -agree -believes, research findings 1. coping with problems 2. everyday communication 3. mechanical production Thematic Analysis -patterns and themes -Bottom-up, exploratory -Words, phrases, sentences, text of interview -themes -text matches theme -subject believes, research findings

Observational Research Design Cohort Design Characteristics -Also known as _______ or ______ -Follow ___ ____ of subjects (i.e., cohort) over time -Examine changes on ____ _____ during that time -Goal: Do changes over time correlate with changes in ____ ____ over time? Examples: -Speech and language milestones in typically developing children -Changes in hearing acuity from middle-age to geriatric years Designs Two basic variations include: 1. ____-_____ Longitudinal 2. ___ ____ Longitudinal 1. Multi-Cohort Longitudinal (also known as Cross-Sectional Design) Characteristics: -Obtain subjects as different ____ intervals -Assess all on _____ variable -Assess again when students from one age interval reach same age as ______ age interval Example: O1 → O2 → O3→ 04 2 yrs → 3 yrs → 4 yrs → 5 yrs 2. Birth Cohort Longitudinal Characteristics -Obtain subjects at ____ age (i.e., birth year) -Assess all on _____ variable -Assess again when subjects reach _____ age of _____ Examples: O1 --------------------------> O2 1: Birth 12 mos 2. 50 yrs 6o yrs Limitations 1. Subject mortality -Loss of participants over time limits how well they _____ general population 2. Testing -Changes over time on measures may be related to ____ ____ with measure rather than changes in ______ 3. Cross-generation effect -Conclusion drawn from one particular generation may not apply to ____ ____ In sum, cohort designs -Help identify relationships between _____ _____ that emerge across time and how these may be relevant to ____, ____, and/or _____ considerations -But should be interpreted with caution in terms of ____-____-_____ relationships

- Developmental and Longitudinal -same group -same variable(s) -dependent variable 1. Multi-Cohort 2. Birth Cohort Longitudinal 1. Multi-Cohort -age -dependent -subsequent 2. Birth Cohort Longitudinal -similar -dependent -subsequent age of interest - represent -increased experience with measure, phenomenon -another generation -subject characteristics -development, diagnosis, and/or treatment -cause-and-effect

What do we mean by ethics? 1. The discipline dealing with what is_____ and ____ and with ______ duty and obligation 2a: A set of moral principles or values b:a theory or system of moral values <the present-day materialistic ethic> c: the principles of _____ governing an ____ or _____ <professional ethics> d: a guiding philosophy What are some principles of conduct that might guide how scientists behave as members of the scientific profession? Ethics of Science -Ethical standards based on _____ of profession: -Search for ______ knowledge -Eliminate _______ -Solve _____ _____ Standards of conduct also have moral foundation: -The standards that apply to ____ _____ regardless of _______ -Thus, ethical conduct in science: should not violate ____ ____ _____ and should promote ______ of scientific goals Standards of Ethical Conduct in Science 1. ______ 2. _________ 3. _______ 4._____ for _____

- good and bad, moral duty and obligation -conduct, individual or a group -goals -objective -ignorance -practical problems -all people, profession -accepted moral standards, advancement 1. Honesty 2. Openness 3. Credit 4. Respect for Subjects

2. Openness -Public nature of science is essential -Share ideas, methods findings with others --> Example:_____, conventions -Accept appropriate _____ --> Don't take it personally -Be open-minded --> Consider _____ explanations and be wary of applying ____ ____ (e.g., confirmation bias) Openness: Peer Review -Research paper reviewed by ____ -Usually 2 or 3 experts and Editor and Editor-in-Chief -Consensus determines if paper is ___, ____, or ____ Consider: -Worthwhile idea? -Reasonably accurate and complete overview of relevant literature? -Trustworthy measures? -Quality of research design? -Reasonable accurate interpretation of results? Pros -____ control -Reduce ___ -Minimize _____ -Insure proper _____ and _____ Cons -Lack of _____ -Theoretical _____ -Personal _____/_____

- journals -criticism -alternative, double standard -experts -accepted, revised, or rejected -Quality -bias -mistakes -credit and recognition -consensus -bias -animosity/favoritism

Criteria for Differentiating Science from Pseudoscience? Warning signs of pseudoscientific claims -_____ boundaries separate scientific from pseudoscientific claims -Signs are probabilistic in nature, meaning the more that are evident, greater likelihood claim is ______ Several descriptions of warning signs applied across helping professions" 1. Overuse of after-the-fact loopholes to explain away negative evidence -When treatment fails to work, it's someone else's _____-- the client or sloppy practitioner-- and not a flaw in the treatment 2. Absence of self-correction -Because treatment is viewed as time-honored, flawless, and based on long-held wisdom, it doesn't need to ____ 3. Present confirming evidence only, ignore or reinterpret contradictory evidence -Only supportive evidence for treatment claim is acknowledged, whereas negative evidence is _____ or deemed ______ 4. Overreliance on testimonial or anecdotal evidence -Client and/or practitioner public statements provide positive praise that serve as ____ ____ for treatment claim Ex: NDF Tummy 5. Make expansive claims that exceed evidence -Level and quality of evidence is insufficient to adequately support ____-_____ or ____- ____ treatment claims 6. Rely on non-peer reviewed venues for disseminating information -Rely on ____ and ___ ___ -Target "peer-reviewed" journals with ____ ____ -Publish in "_______" journals 7. Unrelated to well-established scientific models -"____" viewpoints about treatment or disorder exist in a world of their own, and fail to account for already ____-_____ facts 8. Use terms or concepts that are difficult to specify or operationalize -Impressive sounding wording or highly technical jargon is difficult to ____ or _____ Ex: NDF Tummy → What does "improve" or "decrease" mean? What do "bugs" refer to? What specifically are "tummy issues"? 9. Lack well-defined conditions for their application or outcomes -Treatment benefit claimed for ____ _____ or its outcomes are defined in ____, ___-____ terms 10. Appeal to "whole is greater than the sum of its parts" -Insisting that "____ ____" must be taken into account, without specifying what those _____ are or how they _____

-Fuzzy -questionable 1. fault 2. change 3. unacknowledged or deemed unworthy 4. primary evidence 5. strongly-worded or wide-ranging 6.internet and social media -weak oversight -predatory 7. New, well-established 8. define or measure 9. multiple problems , fuzzy, feel-good terms 10. all factors, factors, interrelate

1. Honesty -Must be honest -Present results in ____ and _____ manner -Represent data as ______ Do not: -Make up results -->_______ -Dishonestly alter data --> ________ -Surreptitiously omit relevant information --> ________ -Most _____ standard-- without it, science's goals would be impossible to meet -Example: Schon at Bell Labs -More recent example: Diedrek Stapel Professor of social psychology at Tillburg University Fabricated and manipulated data across 55 publications Total fabrication of 30 studies Three graduate students noticed anomalies in data he provided them Two professors thought his findings "too good to be true" Tilburg University investigated claims Stapel dismissed from University

-truthful and accurate -obtained - Fabrication -Falsification -Misrepresentation -important

Basics Single-Subject Design Video Recall: Experimental Research includes three basic characteristics 1. Manipulate values of ____ ______ 2. Observe effects of manipulation on values of ____ _____ 3. At same time, keep ____ _____ under control or accounted for -Extraneous variables → variables that may impact value of ____ _____ such that they hide or mimic effects of independent variable Three types of experimental research 1. ___-_____ design 2. ____-_____ design 3. ____-______ design

1. independent variable 2. dependent variable 3. extraneous variables -dependent variable 1. Single-subject 2. Within-subjects 3. Between-subjects

Types of Single-Subject Designs: A-B Minimizing Concerns Minimizing threats to internal validity: Some considerations 1. Sudden and large change in ____ -Example: "never" to "many" or "many" to "never" 2. Sudden and rapid change in ____ 3. Knowledge of ____ ____ -Example: no other treatment was received 4. Knowledge of ____ or ____ under investigation -Example: Problem rarely improves spontaneously or is usually resistant to change 5. Replicate findings with ____ ____

1. level 2. trend 3. extraneous variable 4. behavior or disorder 5. additional subjects

Random Assignment -Occurs when all subjects qualified to participate in study have equal chance of being _____ to different ___ or ____ of independent variable -Result: Minimizes potential for ____ _____ of subject factors between groups that could obscure effects of independent variable -Note: Random assignment can be implemented with ___ ___ of ____ that requires more than _____ treatment condition (e.g., see crossover design) or experimental task Example: If study requires participants to complete two or more experimental tasks in a study, order in which they complete them can be randomly assigned, if it makes sense to do so Random Assignment: Challenges -Reduces selection bias as plausible alternative, but doesn't guarantee ____ ____ -Especially when sample sizes are ____, may still have nonequivalent groups due to chance -Ideally, minimum number of subjects for each group should be ____ -Matching: With small samples, may need to match subjects on ____ _____ first, such as severity, then randomly assign to groups

-assigned, conditions, levels -unequal distribution -any type of design, one -group equivalence -small -40+ -relevant characteristic

3. Credit -Credit for contributions to: Development, conduct, authorship of research -Acknowledge research and ideas of others Credit located in three places: -List of _____ -List of _____ -_________ Most egregious violation of credit is: -______--> Taking someone else's work, idea, etc., and passing it off as one's own (Oxford Dictionary) -Addendum: Who plagiarized whom? Baev (2004) vs. Willemse et al. (2006)? Authorship determined by: -Contribution to conceptualization of ____ _____/______ -_____ report or _____ it -Approved ____ version Credit plays two important roles -_______ scientists Credit → (+)Recognition → (+)Reputation → Rewards! -_____ _____ for unethical behavior Credit→ (-)Recognition → (-)Reputation → Punishment! Recall Shon and Stapel

-authors -references -acknowledgements -plagiarism -research question/design -Wrote, revised -final -motivates -allocate blame

Symbols and Graphic Elements A → ______ B → ______ Connected dots → ____ ____ -Dots don't connect from A to B Role of Graphic Display and Error Variance -Graphic display plays critical role in single-subject design -Graph each new data point as it is collected -Why? Problem of ____ ____ What is error variance? -Error variance: Refers to _____ or ____ of scores due to influence of variables other than ____ ____ There are two sources of error variance, and you are already familiar with them: 1. ____ _____: Refers to History, Maturation, and other threats to internal validity 2. ____ ____: Refers to inconsistencies in measurement procedures (e.g., problems with observer reliability) How does single-subject design control for error variance? -Error variance controlled or accounted for by ____ ____ of data based on ____ ____ of ____ by: 1. Obtaining repeated measurements of ____ _____ during Baseline (A phase) 2. _____ each data point and examining for unusual or unexpected ___, ___, and ____ in data path Example: Monitoring for error variance: -Consider this: Treatment goal is to increase frequency of behavior. Days 1-8 baseline reveals this data path → increase in frequency of behaviors. Is error variance a possible concern? Yes, because improvement is occurring in absence of treatment

-baseline -treatment -data line -error variance -variability or fluctuation, independent variable 1. Extraneous variables 2. Measurement Error -continuous monitoring, visual inspection of graph 1. dependent variable 2. Graphing, trend, level, or variability

Observational Research Design Prospective Design Characteristics -Similar to cohort design with important difference -Follow cohort over time to identify factors leading to outcome of interest (e.g., disorder) -Group assessed_____ outcome of interest has occurred (e.g., disorder has emerged) Example: -Follow group of language disordered children over time -None receive treatment -Some recover naturally, some do not -What factors appear to precipitate natural recovery? (e.g., age, sex, severity, family, history) Considerations -Prospective design satisfies 2 of 3 conditions of causality 1. ______ order of antecedent and outcome can be established -X and Y are _______ -X ______ Y in time BUT -Cannot rule out ______ factors -Remember Z Limitations -As a form of cohort design, prospective design has similar limitations such as ____ ____ and ____-_____ effects, but some that are unique including: -Depends on knowing ____ __ ____ those antecedents (i.e., "x") or predictive factors that are most likely to lead to disorder, otherwise may miss factors that are as, or more, important than those studied, and thus fail to get complete cause-and-effect picture -Even when assessing the most likely predictive factors that lead to disorder, it may still take an ____ ___ __ ____ before disorder emerges (e.g., how many years of cigarette smoking before lung cancer emerges?) In sum, prospective designs -Help identify important relationships between ____ ____ characteristics and how these may be relevant to future investigation of ____ or _____ considerations -But should be interpreted with caution in terms of ____-___-____ relationships

-before 1. Temporal -associated -precedes -extraneous -subject mortality, cross-generation - ahead of time -inordinate amount of time -clinical population, etiology or treatment -cause-and-effect relationships

Observational Research Designs Survey Design Characteristics -Ask questions about ____, ____, ___, and/or _______ -Purpose: ___, ____, or _____ behavior -Example: Do you use a clinician-directed, parent-administered treatment approach for preschool children who stutter? Considerations -Clearly ____ topic: -Includes _____ _____ of topic being studied -Example: Operational definition of "clinician-directed, parent-administered treatment" Writing questionnaire items that: 1. Are ____ and _____ 2. Are _____ and _____ 3. Consist of ______ wording Example: Which of the following may have biased wording? 1. Do you believe the indirect approach for managing stuttering allows for a more positive and encouraging home speaking environment than direct approach? 2. What treatment approach do you believe is most sensitive to home environment for managing stuttering in preschool children? The first! How will survey be _____? -Mail survey -Electronic-website (e.g., Qualtrics) -Telephone -Face-to-face interview Is survey reliable? -Produces ____ or _____ results on repeated administrations Is survey valid? -Measures what is it _____ to measure How will sample for survey be obtained? -_______ sample that matches population characteristics of interest Limitations -Findings are _____ only -____ _____ cannot be established Example: -Knowledge about low confidence in treating stuttering may allow us to predict clinicians are less likely to treat people who stutter -But does not allow us to say that low confidence causes clinicians' reluctance to treat stuttering -Lack of knowledge more likely the reasons for reluctance In sum, survey designs -Provide a useful "____ ___ ___" of behavior, attitudes, beliefs, and/or intentions in population -But should be interpreted with caution in terms of ____-___-____ relationships

-behavior, attitudes, beliefs, and/or intentions -Describe, explain, or predict -defined -operational definition 1. simple and understandable 2. relevant and concise 3. unbiased -administered -same or similar -intended -representative -descriptive -Causal relationships -snapshot in time -cause-and-effect

Observational Research Designs Case Study Characteristics Aka: ____ ___ design 1.) ____ ____ of one or more persons to support a conclusion 2.) wide variations in ____: -Some are narrative-like -Some are data-based -Others fall somewhere in between -Example: some range from narratives style (diaries) to data-based studies (A-B design) Considerations -Usually focus on _____ rather than _____ -And individuals may often serve as useful ____ ____ for understanding phenomenon -Example: Bloom examined preschool children speaking naturally in their homes to develop ideas about early language development -Describe new treatment _____ -Provides ____ _____ of treatment with one or two clients as introduction to approach -Example: freud psychoanalytic treatment approach introduced and based on case studies -Describe ____ phenomena or conditions -Provides detailed description of disorders ____ ____ that provide insight into ____ ____ -Example: sammler: demonstrated that right hemisphere speech perception essential for decoding speech prosody in patient with brain tumor before and after surgery -Describe ____ ____ to widely accepted beliefs or theories -Provides detailed description of rare phenomena that raise ____ about widely accepted scientific views -Example: 1970s, larynx widely believed to be anatomical foundation of stuttering; Tuck (1979) described case study of 56 year old male who stuttered all his life, and continued to stutter after removal or his larynx due to cancer -Often have _____ and _____ value -Detailed or personal aspects of some case studies capture scientists' attention, thus prompting ____ _____ Limitations -Difficult to control for ____ ____ -Limited, if any, control for confounding influence of ____ ____ -Difficult to _____ conclusions beyond study In sum, -Case study designs provide useful ______ of phenomena that can prompt _____ study or raise scientific ______ -But should be interpreted with caution in terms of _____ _____

-case history 1) detailed description 2) style Considerations -individual, group -starting point -procedure -detailed description -rare -rarely seen, typical function -counter examples -doubts -persuasive and motivational -additional research Limitations -investigator bias -extraneous variables generalize In Sum -descriptions, further, doubts -causal relationships.

Observational Research Case-Control Design Characteristics -Different values of a ______ variable are compared on the same ______ variable -Goal: Compare subjects who share characteristic on some ______ (i.e., cases) with those who do not and see how they are the same or different on some _____ Considerations Classification variable: -Value of variable based on _____ or _____ characteristic -Example: age, sex, type of disorder Criterion variable: -Variable believed to _____ with different values of _____ variable -Example: communicative abilities, social adjustment, neurophysiology Schematic: -Classification variable: Disorder or No disorder -Criterion variable: Disorder and No Disorder -A widely used descriptive research design in speech, language, and hearing sciences Limitations -Differences obtained from criterion measure may or may not be related to different ____of ____ variable -Why? Because ____ ____ may be acting at the same time with classification variable to influence criterion variable -Difficult to establish cause-effect (X → Y) ______ (e.g., what happened first) In sum, case-control designs -Help identify important relationships between ____ ____ _____ and how these may be relevant to future investigation of ____ or _____ considerations -But should be interpreted with caution in terms of ______-___-____ relationships

-classification, criterion -classification, criterion -attribute or identifying -covary, classification -values of classification -extraneous variables -timeline -clinical population characteristics, etiology or treatment -cause-and-effect

6. Placebo/Common Factors Treatment -Treatment's benefits are often influenced by "treatment" plus _____ ____ in its benefits -Treatment comparison is between "treatment" and ____ ____ in treatment when unaware that "treatment" is ____ -Thus, how much are treatment's benefits based on actual "treatment" relative to client's beliefs in treatment -Example: Recall Bentler et al. (2003) digital hearing aid labelling study-- see class notes Considerations: -Purpose of placebo is to expose participants to conditions that resemble ____ in all ways except ___ ____ is missing -In other words, how much does treatment benefit? Is the ___ ____ a significant contributing factor? Placebo Treatment: Examples Pharmaceutical treatments: -Experimental drug compared with ____ ____, where placebo drug may even be designed to mimic drug side effects, but missing ____ drug ingredients Medical procedures: -Experimental medical procedures compared with "____" medical procedure -Example: Real acupuncture compared with "fake" acupuncture (i.e., needles inserted but not in places that should matter for proposed benefit) Behavioral treatments: -_____ or ____ factors become main consideration -Nonspecific factors refer to conditions that _____ treatment, but are not treatment itself -Examples of nonspecific factors Can you think of any? Besides treatment itself, what other conditions are part of treatment/clinical experience that may be influential? Consider visiting your physician for an uncertain ailment, what aspects of that visit seemed to help you? 1. Attend _____ (e..g., "Yes, I'm going to get help") 2. Personal _____ with and ____ from clinician (e.g., "What a relief, I'm finally seeing the professional") 3. ____ ____ for problem (e.g., "Ah, now that I understand, I feel better already") 4. Belief in treatment's ______ (e.g., "I believe this treatment will make me better") 5. _____ aspects of treatment (e.g., daily visits to clinic, engage in nonspecific communication activities) Net Effect: "It looks and feels like I'm in treatment, therefore I believe I am being treated." Final note about placebos -What is the mechanism behind placebo benefit? -Is it psychological or does belief trigger brain response that results in physiological benefit? -Research suggests belief triggers ____ ____-- To be discussed in CMSD 4800/6800

-clients' belief -clients' belief, absent -treatment, active treatment -client's belief -placebo drug, active -sham -Nonspecific or common -accompany 1. treatment 2. contact, attention 3. Logical rationale 4. effectiveness 5. Procedural -brain response

In toto, single-subject designs are: -Adaptable to ____ ____ -Flexible for ____ ____ -Often demonstrate confidence in cause-effect relationships by: -Accounting for ____ to ___ ____ -And minimizing measurement error by -Demonstrating _____ ____, if included by investigators

-clinic context -unexpected situations -threats to internal validity -observer reliability

Qualitative Research Limitations -Findings are ______ only -Causal relationships ____ be ______ -But still may prove to be useful as ____ ____ for building theories, designing assessments, and shaping clinical practice In sum, Qualitative Designs -Provide important insights about how others_____ and _____ the world from their perspective -But should be interpreted with caution in terms of ____ ____

-descriptive -cannot, established -starting point -view and understand -causal relationships

More Notation R O1 X O2 R O3 _____ O4 -This diagram is the same as the previous one, except there is a blank space between O3 and O4 This is important because it tells us that: -Group 1 received treatment (X) -Whereas, Group 2 ___ ___ receive any treatment because there is a blank space between O3 and O4 Therefore, this diagram tells us the following: -___ ____ of subjects were investigated -Before beginning the study, subject were ____ ____ to either Group 1 or Group 2 -First row: First, Group 1 was measured on the dependent variable (O1). Next, Group 1 received Treatment (X). Once treatment was completed, Group 1 was measured again on the same dependent variable (02) -Second row: First, Group 2 was measured on the same dependent variable as Group 1 (O3). Next, Group 2 received no treatment, they continued living their lives as usual. Once treatment was completed for Group 1, Group 2 was measured again on the same dependent variable (O4) This is called a "____ ___ ____ ___" design

-did not -2 groups -randomly assigned -Randomization Treatment Control Group

Considerations: Two important considerations include: 1. What is the purpose of comparing treatments? 2. How might participants' knowledge of treatment condition influence outcome? How might participants' knowledge of treatment condition influence outcome? -If participants know treatment condition they are in, may act _____ than otherwise would; thus, _____ outcome Example: -Participants who know they are receiving "new and improved" treatment may have different expectations of treatment outcome and, thus, act different compared to those who receive "conventional" treatment -Again, recall Bentler et al. (2003) digital hearing aid labelling study -- see class notes How might we control for subjects' knowledge or awareness of experimental condition they are in? -_____ --> refers to when investigator tries to minimize impact of subject bias or experimenter bias by implementing conditions that create ____ ____ about _____ _____ Blinding Conditions Three basic types of blinding conditions: 1. Single-blind -Refers to when ____ are intentionally kept unaware of experimental conditions -In other words, they have no knowledge of their ____ ____ -Minimizes ____ ____ of experimental arrangements (See External Validity: Contextual characteristics) -Example: Subject's belief that experimental treatment will improve influences their behavior in positive manner 2. Double-blind -Refers to when both _____ and "_____" are intentionally kept unaware of experimental conditions -"Experimenters" refers to persons in ____ ____ with ____, such as clinicians in treatment study -Minimizes _____ ____ ____ that may influence treatment effects -Example: Experimenter's belief that experimental treatment will benefit has positive impact on how subject performs 3. Triple-blind -Refers to when _____, "_____", and _____ ____ or ____ data are intentionally kept unaware of experimental conditions -In other words, persons who administer assessments of dependent variable or analyze data are _____ of subject's ______ condition -Minimizes potential for ___ ____ or how data is analyzed -Example: Observer scores subject's behavior based on expectation that they should be improving when exposed to experimental treatment

-differently, biasing -blinding -deliberate ignorance, experimental conditions 1. Single Blind -subjects -assigned condition -reactive effects 2. Double-blind -subjects and "experimenters" -direct contact with subjects -experimenter expectancy effects 3. Triple-Blind -subjects, "experimenters," and persons collecting or analyzing data -unaware, assigned -observer bias

In sum, -Visual inspection of graphic display of baseline helps control or account for ____ ____ -When baseline trend, level, and variability are stable, it serves another important experimental purpose -What would happen in the ____ of _____? In other words, what is the likely _______?

-error variance -absence of intervention, counterfactual

In sum, -Scientists, just like professionals in other fields, must adhere to certain ____ _____ when conducting research -On one hand... failure to do so ____ the ____ of the entire scientific enterprise -On the other hand... careful consideration of ethical guidelines further supports science as the most______ way of acquiring knowledge

-ethical guidelines -threatens the goals - trustworthy

Variability -Refers to _____ in measures relative to ____ -Fluctuation = means to shift back and forth uncertainly, to ebb and flow -What values mark the variability in measures for: Baseline? Treatment? -In other words, what are the most _____ scores? -Graph shows the _____ of ______ in measures relative to ____ (difference between highest and lowest values for baseline; difference between highest and lowest values for treatment) -Variability is important because provides a more accurate reflection of the ____ of ____ between _____, than level does

-fluctuation, level -extreme -variability of fluctuation, level -magnitude of change, phases

In sum, A-B design provides ____ ability for inferring treatment effects -Therefore, it is not considered a ___ experimental design -But may still provide an important ___ ___ for determining a treatment's potential -Or potential ___ ____ between independent and dependent variables -And in some cases, introduce ____ about prevailing scientific views

-limited -true -first step -causal relationship -doubts

Within-Subjects Design Pretest Posttest In sum, -Despite their inherent flaws, sometimes pretest-posttest designs may be sufficient to signal "_____ ____" with more research or "___ ____ ____" because it may be waste of time

-move forward, go no further

Basics of Single-Subject Design Purpose of Baseline -What does Baseline (A phase) suggest would happen if treatment was ____ ____? -Thus, Baseline (A phase) suggests what would likely happen if treatment is _____ ____, at least within context of the study -- an example of _____ In toto, basics of single-subject design Provide foundation for understanding: -Elements of ___ ___ of study data -Basis for inspecting graphs for ___, ____, and ____ -And the role of graphic display for monitoring threats to ___ ____ and ____ ____

-not introduced, -not introduced, counterfactual -visual display -trend, level, and variability -internal validity and measurement error

2. A-B-A Withdrawal Design Minimizing Concerns related to A-B-A withdrawal of treatment -If treatment known to be effective, then withdrawal ____ ___ -If treatment not known to be effective, then withdrawal serves important functions: 1. Demonstrates ____ ____ ___ -Keep in mind good clinical practice means not using ineffective treatments 2. Client _____ value of treatment -"Hey, I liked that treatment, please bring it back" 3. Duration of treatment withdrawal can be relatively ____ if immediate deterioration is evident (See Reed and Godden, 1977) 4. Upon demonstration of valid treatment effect, obvious next step is to ____ ____ -Example: A-B-A-B

-not necessary 1. valid treatment effect 2. experiences 3. short 4. re-introduce treatment

Within-subjects and Between-subjects Designs: Notation System: Some Basic Notation: O1, X, O2 -O = _____ or measurement of the dependent variable -X = "______" or manipulation of an independent variable -Number = indicates how many times a group of subjects was ____ during the study -This diagrams tells us that there was _____ group of subjects because there is only ___ ___ of X's and O's -Read the diagram from left-to-right in terms of _____ of events -O1: First, a group of subjects was ____ on the measure of the _____ variable -X: Next, this group of subjects received a ______ or _____ of _____ _____ -O2: Once the treatment was completed, the group of subjects was _____ again on the same measure of the ____ ____ -This is called a "_____-_____" design Scenario: -At the beginning of the semester a group of subjects was measured on the dependent variable (O1) -Next, all subjects received the same treatment for a semester (X) -Finally, at the end of the semester, all subjects were measured again on the same dependent variable (O2) O1 → observe all subjects, Pretreatment Start of semester -Treatment, All subjects During semester Observe all subjects, -Posttreatment End of semester

-observation -Treatment -observed -only one, one row -sequence -observed, dependent -"treatment", manipulation of independent variable -observed, dependent variable -Pretest-Posttest

Qualitative Research Characteristics -Qualitative data based on written record of ____ ____ or _____ _____ that are analyzed qualitatively -Qualitative data: data in which values of variable differ in ____ (____) rather than amount Example: compare following 1.) living with a family member who has aphasia is a challenging experience (strongly disagree → strongly agree) 2.) describe how it feels to have a family member who has aphasia Which question is more likely to provide info related to a person's quality of experience? Question 2. Open-ended, and they tell their own experience. Considerations -To describe and understand ____ ____ -To describe meaning of ____ _____ -Examples: different cultural perspectives on nature and treatment of communication disorders -Client's perception of practical ____ of ____ Underlying concepts 1. Phenomenology: refers to understanding world from _____ ____ of _____ because it is _____ constructed -Example: person who stutters view of their disorder shaped by bullying and teasing during school years 2. Symbolic interaction: refers to view that objects, people, situations, and events do not possess own _____, rather it is spared by ______/______ experience -Example: phone is neutral object for most, yet is often feared by people who stutter

-observed behavior, subject interviews -kind (quality), amount -human experience -person's experience -benefits of treatment 1. subject's point of view, socially 2.meaning, personal/cultural

Between-subjects Designs: Examples Randomized Treatment Comparison -In sum, randomized treatment comparison designs are designed to address different _____ relevant to scientific understanding of _____ and their hypothesized _____

-questions, treatments, benefits

Some More Notation R O1 X1 O2 R O3 X2 O4 -R = _____ ____ -This diagram is the same as the previous one, except now there is an "R" at the beginning of each row of X's and O's -This is important because it tells us that: -When the subjects were assigned by the experimenter to Group 1 and Group 2, they were randomly assigned to each group -Whereas, in the previous example, there was no "r" and, thus, the subjects were not randomly assigned to each group -This is called a "____ ____ ____" design

-random assignment -Randomization Treatment Comparison

In sum, -Between-subjects designs control threats to internal validity (e.g., selection bias) by ____ ____ participants to comparison groups -In contrast, random selection is different procedure that determines how _____ participants are identified and selected for study, and may enhance ____ _____

-randomly assigning -qualified, external validity

False Promises to People Relying on us for Help Introduction -Era of Evidence-Based Practice (EBP) -EBP goal: focus decision making on _____, ____-_____ sources of evidence rather than less trustworthy sources such as ____, ____, and ____ ____ ____ -Why? To raise quality of clinical ____ ____ and improve ____ for people needing our help -EBP promoted in our professions for close to 20 years -And: recent survey shows it is beneficial and important to engage in EBP; though SLPs often report insufficient time to practice it Yet... -PS, ineffective, unsubstantiated, or questionable treatment approaches prevail across helping professions, including speech-language pathology -And some, such as facilitated comm, despite being soundly discredited, have continued to persist and thrive

-reliable, well-founded, intuition, authority, and unsystematic clinical experience -decision making, services

In toto, Between-Subject designs: -Address range of _____ ____ that can inform our clinical _____ -Can demonstrate confidence in ____-_____ relationships by: -Accounting for threats to _____ _____ -And minimizing various forms of ____ -Even weak forms of design may provide info that informs _____ of ______ around us

-scientific questions, knowledge -cause-effect relationships -internal validity -bias -understanding, world

Random Selection -Refers to all potential _____ in population who qualify for study having ____ chance of becoming participants in that study -In other words, how did investigators find subjects for study in the first place? -Were participants randomly selected or did they self-select (i.e., chose to respond to study recruitment flyer, advertisement, etc.) ? -Note: Random selection applies to any ___ ___ ___, not just between-subjects designs -Benefit: If random selection achieved, enhances ____ ____ of findings because more likely _____ of population of interest Example: Election surveys attempt to randomly select all eligible voters and determine their voting preference -Helped by fact that there are lists of registered voters Random Selection: Question for You How easy do you think it would be to find all children who have a w/r misarticulation in USA, then randomly select from that population? -Answer: If not impossible, then incredibly challenging and expensive to accomplish! Random Selection Considerations: 1. Impossible to sample from ___ ____ in population, including those who are deceased or not yet born, unless narrowly defined -But in doing so, will likely restrict _____ of findings 2. Also practical considerations in ____ all subjects in first place (e.g., everyone who has a specific disorder) Example 1: Random Selection -Consider locating all persons with specific communication disorder in Athens, Ga, then randomly selecting to participate in UGA study, assuming they want to participate! -Even if achieved, would participants necessarily be representative of people with the same disorder in Atlanta, Mexico City, Toronto, Beijing, New Delhi, etc.? Example 2: Random Selection Craig, A., Hancock, K., Tran, Y., and Craig, M. (2003). Anxiety levels in people who stutter: A randomized population study. Journal of Speech, Language, and Hearing Research, 46(5), 1197-1206. Purpose: To identify anxiety levels of randomly selected sample of adults who stutter from homes in New South Wales (NSW), Australia Reason: Adults who stutter that participate in treatment research may do so because more anxious than adults who stutter who do not participate Method: Based on phone directories for NSW, randomly contacted persons registered with those phone numbers Interview goals on phone included: 1. Identify any adult in household who stutters 2. Verify person has stuttering 3. Complete anxiety questionnaire Recontact 15% of sample for reliability (asked same questions) Results: Total of 4,689 families contacted consisting of 12,131 persons total Total 87 adults who stutter identified, 63 participated Main findings included: If severe stuttering, then moderately more anxious than the non-stuttering controls from past research, and more like persons who had sought treatment If less severe, then no more anxious than the non-stuttering controls, and less likely to seek treatment Conclusion: Adults who present for stuttering therapy are more likely to present with higher levels of stuttering and anxiety compared to those who never seek treatment Implication: Clinicians may need to be prepared to help manage anxiety as well as stuttered speech in adults who stutter

-subjects, equal -type of design -external validity, representative -all subjects , narrowly, generality -locating

What can we do about it? Most frequent recommendations include: -To educate professionals to ____ ____ -And learn to recognize most common ____ ______ -Learn to recognize ____ _____ that distinguish science from pseudoscience -And actually practice _____-_____ ______. (current high quality research, clinical expertise, and client preferences) to make well-informed treatment decisions Evidence suggests: -Many thinking strategies and values associated with critical thinking have been shown to minimize _____ _____, ,result in better ____ _____, and reduce ________ thinking Framework for debunking pseudoscientific claims -_____: Lead with the fact if it's clear, pithy, and sticky-make it simple, concrete, and plausible. It must fit with the story -_____ about the myth: Warn beforehand that a myth is coming...mention it once only -_____ fallacy: Explain how myth misleads -______: Finish by reinforcing the fact-multiple times if possible, make sure it provides an alternative causal relationship

-think critically -cognitive biases -warning signs -evidence-based practice -thinking errors, life decisions, pseudoscientific -Fact -Warn -Explain -Fact

What is Pseudoscience? -Science is a way of ____, a view of world that tries to face reality ___ ___, and takes little for ___ Scientific method -Strategies used for _____ world, solving ____ ____, and correcting ____ _____ -Goal: to protect scientists as much as possible from ____ ____ and ____ ____ -____ or _____ science -Branch of knowledge or a system of beliefs mistakenly regarded as based on ____ ___ or having status of ___ _____ -Study or research claimed as scientific but is not generally accepted as such Sample of questionable treatments: -Facilitated communication (rapid prompting method) -Essential oils for speech disorders -Non-speech oral motor therapy/exercises for speech sound production disorders -NDF Tummy -Dyslexia fonts -Candling for cerumen management -Herbal supplements for tinnitus treatment -Kinesio taping

-thinking, head on, granted -investigating, practical problems, erroneous beliefs -personal bias, wishful thinking -Spurious or pretended -scientific method, scientific truth

2. A-B-A Withdrawal Design -In sum, A-B-A withdrawal design is viewed as ____ experimental design -Because it allows for drawing valid inferences about ___ ___ ___ or influence of ____ ___ -Clinical considerations can be ____ under some circumstances

-true -possible treatment effects, independent variable -minimized

In sum, A-B-A-B design is considered a ___ experimental design -Because it allows valid inferences about ____ ____ or the influence of the ____ ___ -Includes built-in _____ of the ____ -And in some cases, may introduce concerns about ____ a ___ ___ if it doesn't work as expected

-true -treatment effects, independent variable -replication of the effect -pursuing a new treatment

How to diagram a study that has more than one group of subjects O1 X1 O2 O3 X2 O4 -This diagram tells that there were ____ groups of subjects that participated in this study because there are two ____ of X's and O's -First row = Group __ -Second row = Group __ -There were a total of ____ observations of the dependent variable -Notice that the X's and O's for Group 1 and Group 2 are aligned with each other. This means that in terms of the timeline for this study Group1 and Group 2 were both measured on the same dependent variable at the ____ ____ and received treatment at the ____ ____ -Thus, O1 and O3 were obtained essentially the ____ point in time, prior to receiving treatment -And X1 and X2 were treatments that were administered during the ____ time period -And O2 and O4 were obtained at essentially the ____ point in time, after completing treatment -Note that there are ___ treatments: X1 and X2 -The numbers mean not only that there were two treatments, but that they were two _____ treatments -Group 1 received Treatment 1 -Group 2 received Treatment 2 -Thus, this diagram tells us the following: __ ___ of subjects were investigated -First row: First, Group 1 was measured on the dependent variable (O1). Next, Group 1 received Treatment (X1). Once treatment was completed, Group 1 was measured again on the same dependent variable (O2) Second row: First, Group 2 was measured on the same dependent variable as Group 1 (O3). Next, Group 2 received Treatment 2 (X2). Once treatment was completed, Group 2 was measured again on the same dependent variable (O4) This is called a "____ ____" design

-two, rows -1 -2 -four -same time, same time -same -same -same -2 -different -2 groups -Treatment Comparison

Between-subjects Designs: Examples Non-equivalent groups Characteristics -Subjects appropriate for study already assigned to groups that investigators are ___ to ____ -Thus, non-equivalent b/c cannot account for ____ ____ -Otherwise, study design same as those described already Examples: -Most situations in schools, cities, states, and federal levels are unable to satisfy ____ ____ -Consider comparing: Scientific thinking skills of UGA CMSD students w UA CMSD students - can't randomly assign school you attend or your major Threats to Internal Validity -Selection bias is serious flaw to ____ ____ of design -Due to non-equivalence may introduce threats such as: 1. 2. 3. -Result: often referred to as ____-____ design Considerations -Despite shortcomings, non-equivalent designs can still provide useful info -Investigators can minimize threats if they can show groups are more _____ than ____ on most important subject characteristics -Examples: groups may be similar on variables such as severity, gender, race/ethnicity, age, assuming these matter -Or corroborating evidence from other studies suggests that hypothesized independent variable is ____ ____

-unable to change -selection bias -random assignment -internal validity 1. History 2. Maturation 3. Statistical regression -quasi-experimental -similar than different -active agent

Between-subjects Designs: Examples Non-equivalent groups -In sum, non-equivalent group designs may still provide ____ evidence, even when random assignment is not a realistic or practical option

-worthwhile

Why Do Pseudoscientific Claims Prevail and Why are We Susceptible to Them? 3 possible life cycles of pseudoscientific claims 1.) enthusiasm → discredited → fade away from use 2.) enthusiasm → supported → crossover to mainstream 3.) enthusiasm → discredited → continue to exist and thrive -Facilitated communication timeline Why are we susceptible to them? 1. Treatment package -Treatment claims provide ____ and ____ solutions -Provide _____ instead of probability -Confirm what we would want to believe -Provide _____, without consideration for kind of quality of evidence 2. Human fallibility -Being smart, good-intentioned isn't sufficient to prevent _____ decisions and holding ___ ____ -For instance, physicians, psychologists, wall street, politicians, scientists, and other professionals too -Why? In part because of ____ _____ -All of us are susceptible to thinking errors that can lead to false beliefs and poor decisions Three examples: 1.We sometimes _____ worlds around us -Assume world is precisely as we see it (naive realism) -We trust _____ is ____ -But fail to appreciate that we don't always ___ or ____ what we think we ____ or ____. And, unaware that expectation shapes ______ 2.We seek to ____ not question our beliefs -Refers to reflexive search for positive evidence, and when we find it, we _____! -Search for negative evidence. However, it requires ______ effort. -And when we do find it, may apply ____ ____ -Fail to consider negative evidence provides more ____ and ____ picture than positive evidence alone 3.We fail to appreciate role of _____ and _____ -natural to believe that everything happens for a ____ -Because often things do happen for a ______ -But, fail to appreciate that ____ and _______ are reasons, too

1. -sound and easy -certainty -evidence 2. -foolish -false beliefs -cognitive biases 1. misperceive -seeing is believing -see or hear, see or hear -perception 2. confirm -stop -deliberate -double standard -complete and balanced 3. chance and coincidence -reason -reason -chance and coincidence

Basics of Single-subject Design Characteristics and Advantages Single-subject design: Characteristics 1. ____ or ______ assessment of dependent variable over time 2. Subjects participate in ___ _____/______ of experiment 3. Experimental effects typically evaluated on basis of ____ ____ of ____ ____ of data Advantages 1. Adaptable to ______ context -Design suitable for collecting data on _____ clients rather than requiring ____ of clients 2. Flexible -Design can be ____ in-the-moment when unexpected circumstances occur or ____ with other single-subject designs 3. Power to draw ____ _____ -In most cases, design allows practitioner to determine if treatment is _____ or____ _____

1. Continuous or repeated 2. all conditions/phases 3. visual inspection of graphic display 1. clinical -individual, groups 2. Flexible - modified, combined 3. valid inferences - real, chance event

In sum, questionable treatment claims: 1. Explain away ____ evidence 2. Never _____ 3. Present ____ _____ only 4. Rely on ____ ____ 5. Sound ____ ____ to be true 6. Go straight to the ______ 7. Work outside of typical _____ ______ 8. Use ______ terms 9. Claim _____ is the limit 10. Need to consider ____ ____ In toto, -Pseudoscientific TXs can cause harm in a variety of ways, including ______ treatment and blocking the use of ______ -Important for us to provide counter-narrative to the false hope of pseudoscience, and replace it with real-world promise of _____-_____ _____

1. negative 2. change 3. supporting evidence 4. personal stories 5. too good 6. consumer 7. scientific frameworks 8. fancy 9. sky 10. all things -delaying, EBPs -science-based practice

Between-subjects Designs: Examples 1. ______ ____-____ ____ ______ 2. ______ _____ _____ 3. ______ _______

1. Randomized Pretest-Posttest Control group 2. Randomized Treatment Comparison 3. Nonequivalent Groups

Types of Single-Subject Designs: A-B Characteristics A Phase 1. _____ measures of ____ ____ under nontreatment conditions 2. Examine ___, _____, and ____ 3. If the baseline is acceptable, then introduce... ___ ____ 4. Repeated measures of ____ ____ Possible Outcomes Questions: What are 3 possible (treatment) outcomes of phase change? What are the implications of these outcomes? 1. Improvement -Change ____ with expected treatment effect Implications? 1. Suggests independent variable may be ____ ____ 2. Continue with treatment until ____ ___ 2. Deterioration -Change ____ of expected treatment effect Implications? 1. Independent variable ___ ____ have expected effect 2. Or there are potent ___ ____ at work 3. _____ treatment 3. No Improvement -Trend or level seen during ______ (___ phase) continues during _____ (___ Phase) Implications? 1. Independent variable is not ___ ____ or ____ 2. Wait and see if there is a ____ ____ OR... 3. Add or subtract _____ from _____ (tweaking what you have) -In other words, introduce a "____" Phase Illustrates flexibility of SSD OR... 4. Try totally different type of ____ or new ____ ____ -In other words, introduce a "___" Phase Also illustrates the flexibility of SSD Possible Concerns Question: What are possible threats to internal validity of A-B design? -Possible threats to internal validity: 1. 2. 3. 4. Basic Issue: -One phase change cannot rule out ____ or _____ as an alternative hypothesis -Although, repeated measures during baseline provide ___ ____ -Result: A-B design provides limited control for ____ ____ Thus, often referred to as -____-_______ -____ ___ ___

1. Repeated, dependent variables 2. trend, level, and variability 3. B Phase 4. dependent variable 1. Improvement -consistent 1. causal variable 2. successful conclusion 2. Deterioration -opposite 1. did not 2. counter factors 3. Withdraw 3. No Improvement -Baseline (A phase), Treatment (B Phase) 1. causal variable or is important 2. delayed effect 3. components, treatment -B1 4. treatment, independent variable, C Possible Concerns 1. History 2. Maturation 3. Testing 4. Statistical Regression -chance or coincidence -limited control -internal validity -Quasi-experimental -Case study design

2. Within-subjects Design Characteristics 1. ____ ____ of subjects observed over ____ 2. All subjects participate in ____ ____ of experiment -Thus, subjects serve as their own ____ Example: All subjects receive ___ ___ 3. ____ or ____ assessment of dependent variable over ____ -In contrast to single-subject design, assessment intervals are ____ and ____ before the ____ by the design Example: Once before and once after treatment What is error variance? Two sources of error variance: 1. ____ ___ 2. _____ _____ How do within-subjects designs attempt to control for error variance? Consider: -To determine effects of independent variable must keep factors _____ across all conditions of experiment -Thus, only variable that is different is presence or absence of _____ variable One way to reduce error variance: -Keep subject characteristics as ____ as possible across experimental conditions -Within-subjects design controls error variance by comparing subjects with _____ across all _____ of experiment -Example: Subject factors such as age and gender are constants throughout study -Thus, effects of independent variable may be more easily detected by comparing people with _____ rather than comparing _____ people with each other

1. Same group, time 2. all conditions, controls -same treatment 3. Continuous or repeated, time -specified and set, study 1. Extraneous factors 2. Measurement Errors -constant -independent -similar -themselves, conditions -themselves, different

2. A-B-A Withdrawal Design Characteristics A Phase: 1. Repeated measures of ____ ____ under ___-_____ conditions: 2. If baseline is acceptable, then introduce... ___ ____: 3. Repeated measures of dependent variable under ____ ____ (_____ variables) 4. If clear change emerges relative to baseline, then return to... ___ ____ 5. Cease ____ ____ variable -In other words, _____ treatment Possible Outcomes and Implications Questions: What are three possible outcomes of the withdrawal phase? What are the implications of these outcomes? 1. Deterioration -Change reverts to _____ -Replication of ___ _____ Implication? -Independent variable (treatment) was _____ for change in __ ____ 2. Continued Improvement -Change predicted by B phase _____ -What to do next? 1. What and see if there is a ___ ___ (e.g., temporary carryover) 2. Or let improvement continue to ___ ____ Implications? 1. Introduces _____ about experimental effect 2. Or an A-B with unexpected ____-___ ____ 3. Or repeat with ____ ____ to determine if ____ ____ or _____ 3. No Change -Level of change observed in B phase ____ during _____ ___ Implications? 1. Wait and see if there is a delayed ____ ____ 2. Re-implement ___ ____ -If clear level/trend emerges, then infer treatment was ______ for change in B phase -Illustrate flexibility of SSD 3. Introduce a totally different___ of ____, ____ Phase -In other words, the original treatment was not believed to be _____, so add ____ ___ to continue progress -Also illustrates flexibility of SSD Possible Concerns Questions: -What are possible threats to internal validity of the A-B-A design? Or to think of it another way: -Does additional phase change to A-B design allow investigators to claim that independent variable is most likely reason for study's finding and not extraneous factor? A-B-A Threats to Internal Validity: -Satisfying ____ of ____ accounts for threats to internal validity -In other words, demonstrating experimental control accounts for ____ ____ -Basic logic: Two phase changes minimizes ____ as an alternative hypothesis A-B-A Withdrawal Design: Some Considerations on Withdrawing Treatment 1. Some ______ variables, such as those associated with skills (e.g., language), may not be applicable to this design -Because once skill is achieved, withdrawal of treatment is unlikely to _____ that skill -Example: Once child learns rule for plurals, unlikely to reverse that skill 2. No problem in laboratory setting because demonstrating experimental control is paramount -But, in clinical setting, it can raise ____ _____ because treatment was removed However...

1. dependent variable, non-treatment 2. B Phase 3. treatment conditions, independent 4. A Phase: 5. manipulating independent -withdraw 1. Deteoration -baseline -A phase -responsible, B phase 2. Continued Improvement -continues 1. delayed effect 2. successful conclusion 1. uncertainty 2. follow-up phase 3. other subjects , chance event or real 3. No Change -stabilizes, withdrawal phase 1. withdrawal effect 2. B phase, responsible 3. type of treatment, C -sufficient, new treatment -logic of design -alternative explanations -chance -dependent, skill -reverse -ethical concerns

1. Pretest-Posttest Characteristics 1. Obtain ___ of ____ that meet study requirements 2. Assess all subjects on measure of ____ ____ 3. Then apply _____ 4. _____ all subjects on ____ measure of dependent variable -Compare scores on ____ (______ treatment) to scores on _____ (_____ treatment) Example: Typical Timelines for This Study Design in University Settings O1 Observe all subjects pretreatment → start of semester X Treatment all subjects → during semester O2 Observer all subjects posttreatment → end of semester Threats to Internal Validity What are threats to internal validity? Consider: Is there a Time 1-Time 2 concern? 1. 2. 3. 4. -Because Pretest-Posttest design fails to provide adequate controls for ____ ____, referred to as ____-______ design Considerations -Despite flaws, serves as useful and practical first step for evaluating ____ or __________ ______ to determine potential value for further ______ -Sometimes referred to as a "____ __ ____ ____" -In some contexts, threats to internal validity may not be a ____ _____ Examples: No other treatment was received (i.e. History) or Maturation (i.e. natural recovery) is unlikely

1. group of subjects 2. dependent variable 3. treatment 4. Re-assess, same -pretest, before, posttest, after 1. History 2. Maturation 3. Testing 4. Statistical Regression -internal validity, quasi-experimental -new or under-investigated treatments, investigation -Phase 1 Clinical Trial -major concern

Who are the Practitioners of Pseudoscience? Some possibilities include: 1. The Imposter -Not a ______, but want others to think so -Uses _____ to exploit others' credulity (our readiness to believe that something is real or true) for own self-gain 2. The Fool -Not a scientists, but foolishly _____ otherwise -Expression of sincere _____ and _____ to fool others 3. The Manipulator -May or may not be a _____ -Strong need to _____ and _____ certain beliefs -Uses whatever tools possible to _____ those beliefs 4. The Sincere Pretender -May or may not be a scientist, but "_____" to be one -Is sincere and reasonable, but lacks ____ and ____ 5. The Failed Scientist -Is a scientist, but fails to ___ ___ -Uses ____ to exploit others' credulity for own self-gain 6. The Failed Practitioner Ex: Dr. Oz

1. scientist -deceit 2. believes -sincere beliefs and intention 3. scientist -promote and bolster -spread 4. wants -knowledge and skill 5. be one -fraud

Between-subjects Designs: Examples Randomized Pretest-Posttest Control Group Characteristics 1. Obtain ____ appropriate for study 2. ____ ____ subjects to two groups R O1 X1 O2 R O3 O4 3. Assess both groups on ____ ____ 4. Each group is exposed to different _____ of independent variable Example: Treatment vs. no treatment 5. Assess both groups on ____ ____ again, once treatment reached defined endpoint -Note: This design also serves as prototype for "____ ____ ____ ____" or RCT Threats to Internal Validity -Are there possible threats to internal validity? -In other words, does random assignment and addition of control group account for Selection Bias? History? Maturation? Testing? Statistical Regression? The design controls or accounts for: 1. _____ _____ 2. _______ 3. ______ ______ 4. _____ (limited) 5. _____ (limited) Limitations: -History: If history factor ____ ____ one group and not the other -Testing: Controls for Testing, but not interaction effect of _____X_____ -Attrition: If ____ ____ of subjects across groups Considerations -No-treatment control groups present obvious ____ ____ Example: Withholding treatment from people seeking help -____ _____ may minimize this concern Example: "You may be randomly assigned to no-treatment group and can withdraw at any time-- okay?" -But potential problem because even if consent to participate, may still ___ ___ afterwards (e.g., see Onslwo et al. (1994)- notes for Pretest-posttest Possible Solutions 1. ____ _____ control group: -Participants in no-treatment control group put on "____ ____" and will receive treatment, once first treatment group is completed 2. ____ _____ design -"Control" group receives treatment, just not the _____ treatment (to be discussed later)

1. subjects 2. Randomly assign 3. dependent variable 4. levels 5. dependent variable -Randomized Controlled Trial Design 1. Selection Bias 2. Maturation 3. Statistical Regression 4. HIstory (limited) 5. Testing (limited) -differentially affects -Testing x Treatment -differential loss -ethical concerns -Informed consent -drop out afterwards 1. Waiting list, waiting list 2. Treatment comparison, experimental

Between-subjects Designs: Examples Randomized Treatment Comparison Characteristics 1. Obtain ___ appropriate for study 2. ___ ___ subjects to two groups R O1 X1 O2 R O3 X2 O4 3. Assess both groups on ____ ____ 4. Each group is exposed to different ____ of _____ variable Example: Tx 1 vs. Tx 2 5. Assess both groups on ____ _____ again, once treatments reach defined endpoint 6. Compare ____ ____ between each treatment group -Note: Usually minimizes ____ _____ because all participants receive treatment in some form Considerations Two important considerations include: 1. What is the purpose of comparing treatments? 2. How might participants' knowledge of treatment condition influence outcome? What is the purpose of comparing treatments? -Treatment comparison depends on study's _____ -In other words, what do you want to know about the "experimental treatment?" We'll look at six possibilities 1. Standard Practice -Experimental treatment developed because believed to be improvement on "____ ____" or "___ ____" -Treatment comparison is with whatever approach is defined as "___ _____" or "____ ____" -Thus, is experimental treatment as ___ as, or _____ than, what is already available? 2. Theoretically Opposite Treatment -Experimental treatment developed on basis of different ____ ____ of disorder and how it ____ be managed -Treatment comparison is with whatever approach is defined as _____ ____ -Thus, which treatment and, by extension, which theory is the best view of disorder and how it should be managed? 3. Same Treatment with Add-ons -Consider first: Treatments often come in "packages" consisting of multiple components believed essential to treatment change -Example: Treatment packages for misarticulation often include multiple components such as: Client must: 1. Differentiate between error and correct production of sound 2. Produce correct sound in isolation or nonsense syllables 3. Produce correct sound in words and sentences 4. Produce sound in conversation within and beyond clinic -Treatment package adds ____ _____ that may provide greater benefit -Treatment comparison is between same treatments, but one has ____-___ -Thus, is treatment with add-ons as _____ as, or ____ than, treatment without add-ons? 4. Same Treatment with Subtractions -Treatment package subtracts or removes components because may be unnecessary -Treatment comparison is between same treatments, but one has ____ components -Thus, is treatment with subtractions as ____ as, or better than, treatment with no subtractions? 5. Same Treatment with Adjustments -Treatment package adjusts certain parameters to determine ____ manner of ____ treatment -Example: Treatment presented in different formats: Intensive (every week day for 2 weeks) vs. Spaced (once/twice a week for 10 weeks) -Treatment comparison is between same treatments, but one has _____ -Thus, is treatment with adjustments as beneficial as, or better than, treatment without adjustments?

1. subjects 2. Randomly assign 3. dependent variable 4. levels of independent 5. dependent variable 6. outcome differences -ethical concerns Considerations -question 1. "standard practice" or "best available" -standard practice" or "best available" -good , better 2.theoretical view, should -theoretically opposite 3. additional components -add-ons -beneficial, better 4. fewer -beneficial, better 5. optimal, administrating -adjustments

Within-Subjects Design: Crossover 3. Crossover Characteristics 1. Compare two different _____ across same group of subjects R O1 X1 O2 X2 O3 R O4 X2 O5 X1 O6 2. All subjects exposed to Treatment ___, then to Treatment ___ BUT add two important elements -First, ____ ____ subjects to each group (R) -Second, ______ order of treatments Example Which hearing aid is better? Brand A or Brand B? All subjects in Group 1 try: 1. Brand A for several weeks 2. Then Brand B for several weeks All subjects in Group 2 try: 1. Brand B for several weeks 2. Then Brand A for several weeks Compare subject's hearing aid preference Best test results → best hearing aid Concerns -Possible threats to internal validity -Random assignment and counterbalancing account for: _______ _________ _____ _____ ______ _____ ______ _____ and ____ effects However, similar to A-B-A withdrawal designs: -Some dependent variables, such as those associated with _____ (e.g., language), may not be applicable to this design -Because once skill is achieved, it is unlikely that skill will dissipate, thus _____ is certain Example: Once child learns rule for plurals, unlikely to lose that skill

1. treatments 2. 1, 2 -randomly assign -counterbalance Concerns -History Maturation Statistical regression Testing Selection bias Carryover and Order effects -skills -carryover

Within-Subjects Design: Multi-treatment Characteristics 1. Compare two different _____ across ____ group of subjects -O1 X1 O2 X2 O3 2. All subjects exposed to _____ ____, then to _____ __ Potential strength: -Different effects of two treatments can be more easily discerned when based on ____ ____ rather than different subjects Example: Which hearing aid is better: Brand A or B? All subjects wear each brand: 1. First Brand A for several weeks 2. Then Brand B for several weeks -Compare subject's hearing aid preference -Best test results → best hearing aid Considerations What are potential problems with multi-treatment design? 1. 2. 1. Carryover Effect -Occurs when ____ _____ alters behavior observed in _____ ____ -Problem: Differences between Treatment 1 and 2 are not due to "real" differences, but due to ____ effect on 2 Example: Previous treatment "changes" subject and those changes carry over to subsequent treatment 2. Order Effect -Occurs when treatments occur at ____ ____ ____ ____ (e.g., early vs. late) -Problem: Differences between Treatment 1 and 2 are not due to "real" differences, but due to 1 simply occurring at a ____ ____ than 2 Examples: -Learning: Subjects learn how to perform experimental task better over time (e.g., Testing) -Fatigue: Subjects deteriorate simply because they become less interested or tired (e.g., Maturation) Solution How can investigator control for these weaknesses? -________: All subjects receive both treatments but in different _____ Counterbalancing Example: Group 1: Treatment 1 first, Treatment 2 second Group 2: Treatment 2 first, Treatment 1 second -Treatment that is consistently best across both groups is "winner" or is demonstration of "real" effect

1. treatments, same 2. Treatment 1, Treatment 2 -same subjects Considerations 1.Carryover Effect 2.Order Effect 1. Carryover Effect -previous treatment, subsequent treatment -1's 2. Order Effect -different points in time -different time Solutions -Counterbalancing -orders

3. Between-Subjects Design Characteristics 1. Comparison between ___ or _____ groups -R O1 X1 O2 -(different group) R O3 X2 O4 2. Groups differ on _____ of _____ variable -Example: Tx 1 vs. Tx 2 3. All groups assessed on same _____ variable 4. ____ or _____ assessment of dependent variable over time -Assessment intervals are ____ and ____ before study by the design Example: assess once before treatment and once after treatment Control for Error Variance Two sources of error variance: 1. ____ _____ (i.e., threats to internal validity) 2. _____ _____ (i.e., reliability or validity issues) How do between-subjects designs control for error variance? -Problem: To determine effect of independent variable, groups must be as _____ as possible so only variable that is different is presence or absence of _____ variable -Recall: Within-subjects designs control for this concern by using _____ as their own controls -In contrast, between-subjects designs face special challenge because if groups being compared differ from each other in important ways, then possible ____ ____ -Thus making it difficult to determine if: Difference between groups is due to ____ of _____ _____ (i.e., treatment) OR that groups were ____ to begin with -Solution: Between-subjects designs overcome this problem by implementing procedure called _____ _____ that reduces likelihood of Selection Bias -But sometimes Random Assignment is confused with Random Selection -So, let's look at Random Selection first

1. two or more 2. levels, independent 3. dependent 4. Continuous or repeated -specified and set Control for Error Variance 1. Extraneous factors 2. Measurement Error -similar, independent -subjects -Selection Bias -manipulation of independent variable, different -Random Assignment

4. Multiple Baseline Design Characteristics 1. Establish 2 or more baselines across: a.) b.) c.) Examples -a.) behaviors: ___ ____ for different misarticulations -b.) settings: Same Tx for misarticulations at ____ and ____ -c.) persons: Same Tx for misarticulations for _____ __, __, ___ 2. Coordinate phase changes across each baseline at ___ ___ __ ___ 3. Clinic-friendly because does not require ____ of _____ to demonstrate effects Possible Concerns -How does Multiple Baseline design rule out threats to internal validity? First, remember that: -A-B design (simple phase change) has ____ internal validity -But, A-B-A-B design replicates ____ ___ __, thus controls for these threats -Multiple Baseline rules out threats in similar way by ____ ___ ___ on more than one ____ Example: -Baseline 2 is control for Baseline ___ -Baselines 3 is control for Baselines ___ and ___ -Thus, possible extraneous variables can be ____ ___ or _____ ____ Considerations -BUT, there are important considerations for this to hold up: 1. _____ number of baselines -required is ___ -____ is better -More than three is even more powerful, but increasingly impractical due to ____ ____ (i.e., too much to keep track of!) 2. Phase changes should be repeated only when a ___ ___ ____ has been shown in the preceding phase change 3. Baseline changes must be ______ of each other -In other words, phase change must occur only _____ a baseline, NOT ______ baselines Example: what is the possible problem if you saw this series of changes? -It appears that when Tx was introduced to Participant 1, changes occurred simultaneously across the other participants suggesting some extraneous factor may be occurring besides Tx

a.) behaviors b.) settings c.) persons Examples -a.) Same Tx b.) school and home c.) clients 1, 2, etc. 2. different points in time 3. withdrawal of treatment Possible Concerns -weak -first phase change -replication phase changes , baseline Example - 1 -2 and 1 -ruled out or accounted for Considerations 1. Minimum -two -three -assessment burden 2. reasonably clear effect 3. independent -WITHIN, between

3. A-B-A-B Design Characteristics 1. A Phase: If baseline is acceptable, then introduce 2. ___ ____: If clear change emerges, then introduce 3. ____ ____: If change reverts to baseline, then introduce 4. ___ ____: Return to B phase -In other words, return to ____ ___ (independent variable) as specified in ___ phase Possible Outcomes and Implications -What are three possible outcomes for the second B phase? What are the implications of these outcomes? 1. Improvement -Change _____ with expected treatment effect Implications? 1. Clear indication that independent variable is ___ ___ 2. Continue with treatment until ____ ____ 2. Deterioration -Change ____ of expected treatment effect Implications? 1. Independent variable did not have ____ ___, 1st time effect was a ____ 2. Or there are potent ___ ___ at work 3. _____ treatment 3. No Improvement -Trend or level seen during ______ continues during ______ Implications? 1. Independent variable is not ____ ___ or is ____ 2. Wait and see if there is a ___ ___ 3. Introduce ____ or ___ _____ Possible Concerns -What are the possible threats to internal validity of the A-B-A-B design? Or to think of it another way: -Does additional A-B to A-B design allow investigator to claim that independent variable is most likely the reason for the study's finding and not an extraneous factor? A-B-A-B: Threats to Internal Validity -Satisfying ____ of ____ accounts for threats to internal validity -Demonstrating ____ ____ and replicating first ____ ____ controls for alternative explanations -Basic logic: Multiple phase changes replication minimizes ____ as alternative hypothesis

p2. B Phase 3. A Phase 4. B Phase -same treatment, B 1. Improvement -consistent 1. causal variable 2. successful conclusion 2. Deterioration -opposite 1. expected effect, fluke 2. counter factors 3. Withdraw 3. No Improvement -Baseline (A Phase), Treatment (B Phase) 1. causal variable or is impotent 2. delayed effect 3. modified or new treatment -logic of design -experimental control , AB phase -chance