ENS 305 Midterm B
Uncontrolled trial
interventio "arm" only pre post differences-statistical analysis no control causal criterial no randomization with no control group
Case studies
"Descriptive" No statistical analysis (inferential) 1 or more 'cases' Vs. testimonials (obvious bias) Starting point/First Step/Pilot Data Causality Criteria Could technically be there but cannot generalize just keep it descriptive no external validity numbers are too low
Positive control
(optional) a different ingredient/intervention which is know to have the desired effect Example: old drug reduces cholesterol Basically proves that it is possible to get desired effect No ceiling/flooring effect Use something that you know will get the outcome you want Positive control lower cholesterol Possible to get desired effect treatment we don't know what actually is the effect and its new so that why w are testing/investigating it positive= has been established before treatment/ positive control-have same effect
The effect of a decaffeinated green tea extract formula on fat oxidation - Roberts et al., 2015
=What study design? double-blind, placebo-controlled, parallel design -Why this study design? Establish causality -What is the risk for misinterpretation? False positive Why, differences in subjects? Outside of study? -do you 'accept' study outcomes-> unbalanced, small N, unknown confounders
Causal Relationships within BRE
-Establishing Cause and effect 3 main criteria -Correspondence or association Do things happen together (e.g. change together) -Precedence Time order what comes first cause or effect Ex. Strength training results vs. intervention ( intervention comes before effect) -Other/alternative explanations Spurious relationships Confounders Can never account for all ( eliminate spurious effects and account for all confounders)
Parallel group vs. cross-over group
=Each subject is own control ('tied') confounders cancel Other confounders don't Randomization: Other confounders, Learning effect =Carry-over effect Wash-out Considerations Long enough Time commitment Drop out
Cross-sectional
Assessment of cross section of population at a specific point in time Observational (no intervention) No pre-post measure or follow or backtrack Identify associations (covariances/correlations)-Starting point Often sophisticated stats to account for many confounders Causality Criteria correspondence "Missing" Precedence Randomization, control group, ↑ chance of confounders STUDY DESIGN COMMONLY MISUSED Wrongfully used to back up causal inference People who did not do the study Need to look at primary source did all the fancy stats only association of variables missing cause and effect no randomization, control or pre or post measurements Correlation is not causation (one moment in time)
Controls negative
Basic idea - everything is the same except the I/T isolate effect of I/T (any difference or change must be due to I/T) no spurious effects possible placebo Sugar pill; Sham operation; Attention (Stretching) Issues E.g. Sugar taste close to what treatment group is getting Not perfect placebo ex. Apple vs. muffin
10,000 People from a population of interest are randomized into two groups. The mean height of the people in group 1 and group 2 should be what? It should always be 0 Group 1 will be taller because it contains more men and men on average are taller than women The difference between group 1 and group 2 should approximately average out to -1 Cannot know what the mean height is but it should be about the same for both groups
Cannot know what the mean height is but it should be about the same for both groups
Follow up
Does the effect persist after the clinical intervention is over? Example: exercise intervention to loose body fat Do people keep fat off after intervention stops Maintenance phase After complex/expensive/time intensive intervention have simple procedures in place to maintain change Example: Exercise intervention to lower blood sugar Intervention: supervised aerobic and resistance training 3 times a week for 6 month Maintenance: information booklets, regular physicals, reminders
Expert opinin
Editorials Comments Fyi ≠ position statements ≈ reviews Causality Criteria Inductive inference (≈content validity) lowest end of the pyramid not really a study design more clinical expertise
Best Research Evidence
Focus on Best Research Evidence to inform: Clinical, field or private practice -where and how to find it peer reviewed research articles Specific info, Need to know article structure
cross over design
Get treatment first and half of subjects cross over and get control and other half got control first and then crosses over and gets treatment Compare each subject in both conditions compare each they will cancel out Other confounders anything related to subject behaviors diet habits fitness level genetics should cancel out cross over has wash out period go back to baseline measurements before starting on other half
crossover design
Get treatment first and half of subjects cross over and get control and other half got control first and then crosses over and gets treatment Compare each subject in both conditions compare each they will cancel out Other confounders anything related to subject behaviors diet habits fitness level genetics should cancel out wash out period measurements get back to baseline before switching over to other side may take long
Sample Selection Enrollment
Ideally measure entire population of interest impossible Sample Consideration Independent sampling generalize (external validity) Reduce potential confounders/error variability The more differences the more possibility for confounder Homogenous sample Animal/In vitro: Clones/cultures/inbred strains Humans - focus on subpopulations Balance external validity and homogenous sample Avoid ceiling/flooring effect
basic comparison
Still T/I vs. control How to get there Average pre values for each condition Average post values for each condition Then same as parallel group design What is the change pre-post in each condition What is the difference between conditions
Analysis
Many different statistical techniques Inferential statistics (SLO Part 4) External validity Confirm correspondence Account for other explanations ('error' variability) T-test: two groups ANOVA: multiple groups Regression: Covarinace, Correlation, scatterplot
top of pyramid evidence
Massive meta-analysis (N=158 studies) Active learning increases student performance in science, engineering, and mathematics • Freeman et al., 8410-8415 | PNAS | June 10, 2014 | vol. 111 | no. 23 "average examination scores improved by about 6% in active learning" "students in classes with traditional lecturing were 1.5 times more likely to fail" "results hold across the STEM disciplines" "greatest effects are in small (n ≤ 50 students) classes"
Can one make a causal inference form this study? No, because it does not look at the association of the independent and dependent variable Yes, because it has a large sample size so randomization should work No, because one cannot tell whether physical function outcomes or tea consumption happened first Yes, because it accounted for so many confounders that other explanations and spurious effect must be eliminated
No, because one cannot tell whether physical function outcomes or tea consumption happened first dont know which came first, physical function dependent, tea consumption independent, cause comes before no time order, eliminate other explanations
Can one make a causal inference form this study? No, because there is nothing in the design that would balance out confounders Yes, because it does not interfere with voluntary behavior (not component of causal inference) No, because one cannot tell whether health outcomes or tea consumption happened first Yes, because it has a control/reference group without the 'exposure' for comparison (not randomly assigned)
No, because there is nothing in the design that would balance out confounders eliminate confounders by randomization which cohort doesnt have, cant make casual inference
Nothinng is perfect
Potential issues: Sources of error ; e.g. Ceiling effect p.384 Quantitative vs. Qualitative research p.393 ff Small sample size , type 1, 2 error p.385 Sensitivity/specificity p.392
Non randomized controlled trial
Quasi experimental design assignment to group not random example: SCI yes or no How does it effect spinal cord injury cant go around hurting peoples spines unethical cant randomized Step below randomized control
Time Series
Sequence of Values over time Analyze trends/forecast/incidences Trend change Bike lane example no comparison group confounders all over the place
Analyzing
identify components of concepts (parts of the whole) and logical sequence/ connection (activities)
Case-control/reference study
Subjects with outcome compare (exposure) to those (control) w/o outcome Observational longitudinal Retrospective Direction of inquiry is retrospective going back wards people with disease look at how they exposed Still have time order No randomization and not protected by confounders Retrospective analysis in particular lends itself to confirmatory bias General problem when searching evidence for something You already believe or know something (or want to) and you look for things that make sense and support your view Forget limitations/overlook other info Poor rational thinking poor decision making Human nature Protection: knowledge & higher order thinking skills Main goal of education (college) questionairres about past history, past medical history, past behavior, get information quickly
Reviewed articles
Summary of multiple individual studies Not a single experiment 2 main types Systemic review-Comprehensive/exhaustive review of all (high quality) evidence Preferably RCTs Identify and weigh evidence Evaluation Issue of Result 'counting' Be aware of publication bias 20:3 go published that it did work didnt publish the 3 that didnt Meta-analysis-Less emphasis on background/mechanisms weigh evidence objectively Not just counting Should address publication bias Ideally Systematic Review plus statistical analysis Line present larger the square more subjects less variability or narrower the line or shorter 95% confidence and less variability Larger square and shorter line given more weight
Green Tea extract can augment fat oxidation ('burning') during moderate intensity exercise. The mean half-life of the main active component in green tea (EGCG) is 2-5 h for healthy males. If the washout in an RCT with a crossover design is too short what would happen? The overall Outcome would indicate that the EGCG effect is a false positive. The overall Outcome would indicate that the EGCG effect is larger (i.e. amplified) than it truly is. The overall Outcome would indicate that the EGCG effect is contrary (i.e. inverse) to what the hypothesis is. The overall Outcome would indicate that the EGCG effect is a false negative.
The overall Outcome would indicate that the EGCG effect is larger (i.e. amplified) than it truly is.
Cohort study
Using a specific sample (cohort) of interest to identify associations Longitudinal Prospective Observational just look at it don't Maniuplate anyting Causality Criteria Precedence Correspondence-relationship between cause and effect "Missing" randomization designed control group ↑ chance of confounders Using a specific sample (cohort) of interest to identify associations Longitudinal Prospective Observational just becuase it is listed lower on pyramid doesnt mean it is not an effective study design cheaper not bringing people in lab or detailed intervention, more ethical follow overlong period of time
Tea consumption and physical function in older adults - Ng et al., 2014
What study design? Cross sectional Why this study design? Accommodate large N, real life (voluntary) behavior, immediate results What is the risk for misinterpretation? Causal inference Spurious effect False positive Why? unaccounted for variable
Green tea consumption and mortality due to cardiovascular disease, cancer, and all causes in Japan: Ohsaki Study - Kuriyama et al. 2006
What study design? prospective cohort study Why this study design? Accommodate large N, real life (voluntary) behavior, long term outcomes What is the risk for misinterpretation? Spurious effect False positive (CVD), false negative (cancer) Why differences in 'control' reference groups
Can one make a causal inference form this study? Yes, if one accepts that all confounders balanced out No, because it is not a crossover design Yes, because it is a RCT No, because it does not have a positive control
Yes, if one accepts that all confounders balanced out Positive control- doesn't factor in causal inference related to ceiling and flooring effect cross over and parallel have randomization
A RCT with a crossover design should protect against what (independent of randomization)? The confounding effect of age in a exercise training study The confounding effect of the weather in a hydration study All of the above None of the above
all of the above
A study is investigating the effects of kinesio tape on elbow joint stability. What is a potential confounder? muscle strength (of muscles around the elbow joint) Skill of the person doing the movement that is tested All of the above None of the above
all of the above
I conclude the following: All studies have advantages and disadvantages and no one study can provide a definitive answer Even with the best available evidence one can never be 100% sure that one thing causes another and one has to keep an open mind Life contains uncertainties so one cannot rely on simple answers but has to make a comprehensive evaluation of the best available information All of the above
all of the above
What are advantages of the study design employed in this study? Accommodate a large sample Does not interfere with what people do 'normally' Can get relative quick results All of the above None of the above
all of the above
What are advantages of the study design employed in this study? Accommodate a large sample Does not interfere with what people do 'normally' Can follow long term outcomes All of the above None of the above
all of the above, study type cohort take sample and follow over long period of time and see if they are exposed or not
Carry over effect No confounders, treatment success, control negative Not enough 'wash out' to reset to baseline decline during control after cross-over What is the effect? A = a false positive B = a false negative C = amplified D = inverse
amplified
Green Tea extract can augment fat oxidation ('burning') during moderate intensity exercise. The mean half-life of the main active component in green tea (EGCG) is 2-5 h for healthy males. How long should the wash-out in a RCT with a cross over design be? Around 3.5 hours Around 5 hours At least 10 hours At least a couple of weeks
at least a couple of weeks
blinding
avoid (un)intentional bias (spurious effect) objectivity Bias could be alternative explanation Single:Subject does not know which they are getting treatment vs. control Placebo (e.g. sugar pill) =Double:Subject and tester don't know =Triple:Subject, tester and analyzer do not know
Evidence Based Practice
best research evidence patient values and preferences clinical expertise -Personal trainer Extreme workouts vs. injury risk -AT Kaatsu/BFO to reduce muscle atrophy post ACL? -PT Front foot running style to reduce stress on knee =Coach/TM Proper warm up to prevent injury/maximize performance (PAP) =Management Branding for sports team/ membership retention strategies
Creating/ Synthesis
build new structure/ understanding/ concept from previous (activities)
Ultimate Question:
do you 'accept' study outcomes?
Randomized Control trial/clinical trial
experimental establish cause -> effect anything below randomized control trial lack component for cause and effect new intervention/ treatment: ex. new exercise technique-> fitness, new fat burning supplement-> health hypothesis- new treatment will have beneficial effect Steps: select relevant subjects, randomize into a treatment and control group, apply treatment or control/placebo, measure outcome =Randomization and control (neg.) should eliminate other explanations -treatment should correspondent with hypothesized effect. do pre-measurement before intervention and post- measurement before analysis Causal inference: correspondence, precedence, other explanations all 3 criteria for causal inference What happens if weight isn't the same- different weight at baseline, control doesn't change, treatment side flooring effect super low already not gonna lose a lot of weight if start at 80lBs or weight training study ceiling effect its possible to have the same effect
Researchers plus reviewers plus editors will point out every possible confounder and its potential effect?
false
What is the greatest risk of misinterpretation for this study? False positive False Negative Amplified effect Inverse effect
false positive
for ____________test-retest; intervention-control, whatever-whatnot) consistent errors/effects do not influence the outcome
general rule of differences
Which two key consideration of sample selection are inversely related? consistent vs. random error homogenous sample vs external validity Ceiling vs flooring effect All of the above
homogenous sample vs external validity
Understanding
making sense out of information, restate in own words (MC test)
BRE in standard scientific article structure where to find
methods: who and What was measured and how-> protocol/design to study Part 3- causal inference
Confounders
misinterpretation if randomization and or control do not work -find them in Limitation section of article - usually at end of discussion - false positive- looks like there is an effect of intervention but there is not confounder causes effect ( type 1) -false negative- looks like there is no effect of intervention but there really is in both intervention and control( type 2) -amplified effect- effect of treatment seems larger than it is -inverse effect- effects seem opposite of hypothesis treatment ends up worse than control in something that was suppose to enhance outcome eliminate confounders by negative control to see what happens from pre to post measurements could use positve control but not its main purpose unbalanced groups, even rows has nagative effect same magnitude but opposite direction of TE= confounders in intervention group, odd rows confounders in control group confounders has positive effect same magnitude and direction of TE two main confounder: genetics and environment, If something will happen on both sides the constant error on both sides of cross over design it will cancel out Unbalanced more old people than the other group will effect study results or outcome
If a confounder has the same effect in the control and the treatment group will it change the difference between the two groups Yes No Cannot be determined from this information Depends on whether the control group is a positive control
no
What are advantages of the study design employed in this study? Accommodate a large sample Does not interfere with what people do 'normally' Due to the design certain confounders such as age cancel out even when unbalanced All of the above None of the above
none of the above parallel group, bring people into lab and do intervention hard to accommodate large samples age doesn't change
parallel groups
parallel design- accumulates in liver and then give control right away there is still treatment in body Standard drug trial want drug to wash out of system right away supplement stays in body for 3 days then wash out period for 3 days some motor skills are hard to unlearn so cant use cross over design
Find BRE
peer reviewed research/ scientific journal articles nature, science, new england journal of medicine, medicine and Science in Sports and Exercise ( ACSM) archives of Physical Mediicine and Rehabilitation (ACRM) -search engines: pubmed.com, google scholar -other primary sources- dissertations, thesis, books Article behind paywall go to library article access or ILL https://illiad.sdsu.edu/illiad/ Acute effect of fast walking on postprandial blood glucose control in type 2 diabetes
BRE primary source of information
peer reviewed research/scientific journal articles the article/ script go through peer review before its published journal send it to expert in field that aren't bias and ensure objectivity have no connection towards writer get best evaluation
Ceiling effect
power strength training occur when a high proportion of subjects in a study have maximum scores on the observed variable reduce variability
BRE Secondary sources (questionable)
questionable not coming from person who did research or experiment, doesn't give enough space to what's really going on magazines, news, websites
Flooring effect
reduces variability arises when a data-gathering instrument has a lower limit to the data values it can reliably specify data starts out low and results are even lower active people or young people will have low response and come right back down wont have large effect diabetes
Remembering/Knowledge
repeat/recall words, facts, information, formulas (MC test)
External validty
sample from entire population how much data can take from sample and apply it to everyone homogeneous sample people are similar together (weight loss, rehab, or weight training effected by one specific age or one specific gender) cant generalize to everyone high homogeneous =less confounders and low external validity
What are the chances that all confounders are completely and perfectly balanced out?
slim to none
Randomization
subjects assigned to groups completely at random differences other than intervention balance out confounder=0 balance out-> zero effect-> similar to what type of error (consistent) not effected by consistent error= reliability (m and R were not effected by consistent error) Large sample sizes related to confounders should balance out and compare two balance out and difference= 0 Consequences of confounders and randomization didn't work sample size wasn't large enough (TE+EC)-(EP+EC)=TE+EP and EC or effect of confounders cancels out randomization balances out same in each group, no time order or correspondence
BRE avoid Bro science
telephone game effect abs everyday
The study (statistically) controlled for these potential confounders "age, gender, education, housing type, co-morbidities, hospitalization, arthritis and hip fracture, gDS depression score, MMSe cognitive score, body mass index, creatinine, serum albumin, haemoglobin, physical activities score and coffee consumption" and still found a positive effect for tea on physical function. What can we conclude? Tea causes good physical function There is no spurious effect This could be a false positive All of the above None of the above
this could be a false positive can't say spurious effect because can't control all confounders better to say good association not tea causes better physical function there was a positive effect
Application
use concept (abstract) in new situation without being prompted (activities)
Evaluation
weighs information based on analysis/ understanding/ knowledge and arrives at conclusion/ decision (activities)
Who will ultimately have to apply their background knowledge to identify confounders (and their effect) that are of interest to you?
you yourself, as a qualified professional with a B.S. in Kinesiology