OT 518 Unit 1 (Quantitative Research)

bimodal distribution

-2 peaks

positive and negative skew

-mean, median, and mode are different -positive skew: tail is to the right of the peak -negative skew: tail is to the left of the peak -present data using median and interquartile range

types of distribution

-normal distribution -negative and positive skew -bimodal -the distribution determines how data is presented and the type of analysis that is done (e.g., mean and SD or interquartile range)

continuous data

-numeric data -can take any value -interval data -ex: height, weight, age, temperature

types of observational studies

1. cohort 2. case-control 3. cross-sectional 4. ecological

types of analytic studies

1. experimental studies (the intervention is under control of the researcher) 2. observational studies (the researcher simply observes the participants at one point in time (cross-sectional) or over time (longitudinal))

trajectory of experimental studies

1. feasibility or pre-pilot ("How could we use yoga as a treatment?") 2. pilot studies ("What is the preliminary effect of with a few patients?") 3. efficacy studies ("Does yoga have a statisically significant effect on a larger group or patients in an identical setting?") 4. effectiveness studies ("What is the effect of implementing yoga for all patients seen by a physician?") 5. monitoring ("Does yoga continue to have good effects after adopted by as a common treatment? Any adversee effects?")

AOTA levels of evidence

-Level 1: Randomized control trial (RCT) -Level 2: Multiple groups, nonrandomized (cohort study, case-control study, non-randomized control trial) -Level 3: Single group, nonrandomized (pre-post test) -Level 4: Descriptive (single case study, case series) -Level 5: Descriptive study, narrative review

phases of traditional clinical trials

-Phase 1: Piloting -Phase 2: Efficacy -Phase 3: Effectiveness -Phase 4: Comfirmatory or surveillance

uncontrolled trials

-Phase I and II clinical trials -no control or comparison group -descriptive pre-post test design where the investigator controls access to the treatment

presentation of data

-continuous variables: means and standard deviations for symmetric and normal distribution (parametric); median and range for non-normal distribution (skewed/non-parametric) -categorical data: counts and percentages

discrete data

-countable, ordered numeric -smooth transition from one value to the next -interval data -ex: number of students, number of strokes

nominal measurement

-counts of like things -group all objects (all inclusive) -groupings are mutually exclusive -comparisons are percentages

intention-to-treat (ITT)

-data analyzed based on original random assignments regardless of the treatment subjects actually received -objective: avoid misleading artifacts such as non-random attrition or crossover -rationale: estimates the effects of allocating an intervention in the real world, not just the effects for the subgroup who adhere to treatment -requires outcomes data on all participants -when missing data occurs, outcomes must be input based on assumptions about the missing subjects (this has its own issues)

pros and cons of retrospective studies

-data is already available -often involves large numbers -definitions may change over time -follow-up times may be inconsistent

appraisal: population

-demographics (age, gender, race) -clinical setting (lab vs. clinic, diagnosis or patient group) -community -country -inclusion/exclusion criteria -how the sample was recruited

appraisal: statistical tests

-determine if test is appropriate -statistical significance -trends vs. null results -t test -Fisher's Exact -Chi2 -ANOVA -Kaplan-eier -Kappy -Correlation -Odds Ratio -Regression -Wilcoxon Rank/Sum

what makes up a clinical scenario

-diagnosis/clinical population (patient, organization/population) -clinical context of the scenario (rational/reason) -clear statement of a clinical dilemma/decision that needs to be addressed

range

-difference between the largest observation and the smallest observation

bias and study design

-each analytic study design has particular types of bias to which it's most vulnerable -case-control studies: selection bias (knowledge of exposure status influences identification of diseased and non-diseased study subjects), recall bias (knowledge of disease status influences the determination in of exposure status) -cohort studies: loss to follow-up (attrition; greatest danger), selection bias (in retrospective studies; especially when comparing evolution of care)

reducing bias

-ensure both groups are "outcome free" or at least equivalent on the outcome at the outset -Who makes the determination of the outcome status? What role does the expecation play? -ensure that outcome is related temporally to the intervention, the instrument is reliable and valid with this client population, the instrument has been demonstrated to be sensitive to change, and the assessment is conducted in a standardized way

p-value

-level of significance (denoted by "alpha") -gives us a way of comparing different groups -most commonly used alpha is 0.05 -If p-value is small (less than alpha = <0.05), there's a lower chance of seeing the effects due only to chance -probability of the found difference due to chance is low; can reject null hypothesis

appraisal: outcome

-main goal of the study (related to the phase of the clinical trial) -outcome measures (primary vs. secondary vs. tertiary, objective vs. subjective, proximal vs. distal)

non-respondent bias

-non-respondents to a survey often differ from respondents -volunteers also differ from non-volunteers, late respondents from early respondents, and study dropouts from those who complete the study -also called response bias -systematic error due to difference in characteristics between those who choose to participate in a study and those who do not

Type I error

-null hypothesis is true, but reject null hypothesis -incorrectly reject null hypothesis when it's true -claim a difference exists that doesn't exist

cross-sectional studies

-observational studies designed to collect data on an outcome and exposure/treatment variables of interest at one point in time -a snapshot in time -objective: prevalence

within-group design

-one group of subjects is tested under all conditions and each subject acts as their own control -treatment effects are associated with differences observed within a subject across treatment conditions, rather than between subjects across randomized trials advantages: -control for potential influence of individual differences (e.g., age, gender, IQ, etc.) -using subjects as their own control provides the most equivalent comparison group possible disadvantages: -potential for practice effects or learning effect -carryover effects: when subjects are exposed to multiple treatment conditions

case-control studies

-outcome oriented -a type of retrospective observational study where people who had an intervention are compared to people who have not -conducted after outcome has occurred (looking back in time) -objective: to identify variables that may predict the condition that you're trying to understand

statistical tests for continuous numerical data

-paired t-test -student's t-test (independent samples) -Analysis of Variance (ANOVA) -note: for these tests, data must be normally distributed

appraisal: comparison/control

-placebo vs. control vs. comparative effectiveness -an intervention form, so it needs all the same information

appraisal: evaluation

-population -allocation -results -summary

power

-probability of correctly rejecting the null hypothesis and conclude that the alternative hypothesis is true -the larger the sample, the greater the power -from hypothesis, correctly claim that difference really exists -prior to study, need to determine minimum sample size to detect difference/effect -need to look at a prior study to calculate power

observational studies

-prospective: observations made looking forward -retrospective: observations are made from existing data

controlled trials

-randomized control trial (RCT) (pretest-posttest control group design; factorial design; randomzied block design) -quasi-randomized trial (pretest-posttest control group design; factorial design; randomzied block design) -non-randomized trial (one-way repeated measures design)

variance

-reflects the variation of the distribution within a full set of scores -the average distance of all data points to the mean

Bonferroni adjustment for p-value

-several comparisons/analyses were being made on the same sample, which may risk inflate the value of α (α error) if each test is performed at the same 0.05 criterion -to reduce Type I error (claim a difference that does not exist), the level of significance is divided by the number of comparisons

standard deviation

-spread of the data of mean (population-based) in a sample

z-score

-tells you the likelihood that a value has occured -a value from converting a data point to a "standardized" value -correponds to the standard deviation from the mean -use z-table to convert z-score into a percentile

appraisal: PICO question/hypothesis

-the PICO question the author(s) asked -"reverse" of our PICO process (where on the spectrum of clinical traits does it sit) -notice primary and secondary outcomes

internal validity

-the extent to which the results reflect the truth about what happened just within the study -selection bias (randomization, allocation) -performance bias (blinding of participants and personnel) -detection bias (blinding of outcome assessment staff) -attrition bias (e.g., acceptable drop-out, compliance with study methods, etc.) -reliable/standardized outcome measures -appropriate statistical tests -coherent conclusions based on statistical findings

threats to internal validity

1. allocation bias (unequal distribution of participant characteristics across groups) 2. measurement bias (systematic error from inaccurate measurment/classification of subjects on study variabes) 3. recall bias (caused by differences in accuracy of recalling past events by cases and controls) 4. placebo effect (individuals' expectations to get better may lead to changes in outcome) 5. maturation bias (changes from the natural growth process or the natural healing process may alter findings)

evidence pyramind (bottom to top)

1. expert opinion 2. case series/case studies 3. case control studies 4. cohort studies 5. randomized control trials (RCTs) 6. critically appraised individual articles 7. critically appraised topics (syntheses) 8. systematic reviews and meta-analyses

The Triple Aim

1. improved health (better outcomes) 2. efficient high quality care (value-based care) 3. improve the patient's experience (increased transparency, public reporting, accountability)

EBP components

1. research evidence 2. clinical expertise 3. information from the practice context 4. patient values and circumstances

threats to external validity

1. sampling bais (unless sampling method ensures that all members of the "universe" or reference population have the same probability of inclusion in the sample, bias is possible) 2. ecological validity bias (the environment may differ from the real world)

effect size

-Cohen's d -an attempt to quantify how much of a difference the treatment group had in comparison to the control group -standardized mean differences effect size: small = 0.20, medium = 0.50, large = 0.80 -but no universally accepted definition of a large efect size -quality of research design is more important than effect size -in meta-analysis, calculate the effect size for each study, weigh them, and then compare those effect sizes across the studies

types of experimental design

between-group comparison: -prestest-posttest control group design -factorial design -randomzed block deisgn within-group comparison: -one-way repeated measures design -crossover design

value-based healthcare

reimbursement based on: -healthcare providers' achieved rates of pre-specified patient outcomes -adherence to patient-centered scientifically grounded best practice guidelines

observational study

the researcher simply observes the participants at one point in time (cross-sectional) or over time (longitudinal)

appraisal: timeline for data collection

-how often/when data was collected -if there was follow-up

randomized controlled trial (RCT)

-also called randomized clinical trial (RCT), clinical trial, randomized trial, intervention trials, health care trials -randomly assigns subject to a study group -assigns interventions (exposures) to subjects -examines associations between exposures and outcomes -other key features to help reduce potential for bias: blinding, sample size justification, statistical analyses with minimized number of covariates -exposures/interventions are manipulated by researcher -participants are randomly allocated by a process equivalent to the flip of a coin to either one intervention or another -control group is incorporated within the design -the only design to test causal relationship (the 2 groups are identical except for intervention, so any differences in outcome are, in theory, attributable to the intervention)

Type II error

-alternative hypothesis is true, but accept null hypothesis -incorrectly accept null hypothesis when alternative hypothesis is true -conclude there's no effect when there is -more likely to make Type II errors than Type I errors

ANOVA

-applied when 3 or more group means are compared -unlike t-test based on t-statistic, ANOVA based on the F statistic, which is a ratio of between groups treatment effects to within-group variability -unlike variance used in t-test, ANOVA calculates sum of squares (SS) to show the variability of scores within a sample

critical appraisal

-assess the validity and relevance of the evidence you find -validity: how likely the results are to be "truth" -relevance: the importance and usefulness of the findings

pretest-posttest design

-assign participants to group -pretest -intervention -posttest at end of intervention

randomization

-based on chance in which participants of a clinical study are assigned to comparison/control and treatment groups -the researcher doesn't know which treatment is better -from what's known at the time, any one of the treatments chosen could benefit the participant -minimizes the differences among groups by equally distributing people with particular characeristics among all trial groups -enables statements of causation since only difference between the groups is the intervention -randomization increases internal validity, reduces potential for bias

appraisal: conclusions

-based on results of the statistical tests -what authors make of their findings -possible impact of study on practice

pros and cons of prospective studies

-can identify "diagnosis" and confounders clearly at outset -intervention can be more clearly documented -temporal sequence of onset and outcome

single subject design

-case study -evaluate the method on observable behavior (outcome) through repeated measures over time -study of a single person (or a few people) over time -measurement of outcome (DV) under multiple conditions (at least 2) -each subject serves as his/her own treatment and control condition -subject may alternate between different treatment conditions -longitudinal or time-series studies -ongoing measurements at multiple times -not good for looking at the effects of multiple attributes or pre-existing variables that aren't under control of the researcher -not good for looking at effects on things that can't be measured repeatedly and reliably

interquartile range (IQR)

-categorizes data which has been organized in order from lowest to highest values -creates 4 equal groups (same number of samples) of observations and determines the cut points -looks at 0-25%, 25-50%, 50-75%, and 75-100%

normal distribution

-central tendency -symmetric, smooth, and bell-shaped curve -mean, median, and mode have the same value -curve can be very narrow or wide, but it's always symmetric -present data using mean and standard deviation

context of healthcare reform

-changing demographcis -escalating costs -poor patient outcomes

statistical tests for categorical data

-chi-squared test -Fisher Exact Test -nonparametric study (analyzie frequencies, proportions) -sum of differences in each cell from what's observed vs. what's to be expected

concurrent control group

-clients in another department or similar organization (at the same time)

historical control group

-clients in the previous 6 months -use "self" as a control

resolving confouding

-cohort studies may eliminate participants with the confounding variable -may match the 2 groups on the counfounders (may be expensive in terms of data loss) -adjust for confounders by controlling for the level of the confounder via regression analysis -randomization (ensures that potential confounding factors, known or unknown, are evenly distributed among the study groups) -restriction (restricts admission to the study to a certain category of a confounder) -matching (equal representation of subjects with certain confounders among study groups) can overcome a great deal of confounding.

appraisal: intervention

-components of each intervention (manualized or new protocol) -frequency and duration -provider/interveners -equipment or devices -number of participants (N); determines external validity of study; power calculations; important for contextualizing null results

confounding

-error in the interpretation of what may be an accurate measurement -creates false associations between treatment and outcome, or masks true associations (impact results/outcome) -3rd factor, potentially controllable -consequence: estimated association isn't the same as true effect -actual data may be correct but the subsequent effect attributed to the exposure of interest is actually caused by something else -cofounders distort effect of treatment on outcome (i.e., interpretation of results) -creates effect that intervention is (or isn't) effective when really what's being obseved is the effect of the confounder -can cause overestimation or underestimation of the true association, and may even change the direction of the observed effect -a "true" confounder must have a relationship with the exposure, have a relationship with the outcome even in the absence of the exposure, not be on the causal pathway, and have an uneven distribution in comparison groups -not always easy to recognized confounders -a practical way to achieve this is to analyze data with and without controlling for potential confounders; if estimate of association differs remarkably when controlled for variable, it's a confounder and should be controlled for -a factor can confound an association only if it differs between the study groups -common confounder: co-intervention

bias

-error in the measurement of a variable -creates false associations between treatment and outcome, or mask true associations (impact results/outcome) -systematic error (incorrect measurement) in study design and implementation, fatal flaw in study -potential sources of bias should be eliminated or mninimized through rigorous design considerations and meticulous conduct of a study

paired t-tests

-evaluates whether the mean difference in paired scores is different from zero (null hypothesis) -same participants in each group (ex: pretest/posttest) -indicates that the study probably didn't have a control group -data needs to normally distributed

EBP

-evidence-based practice -a process where by research evidence, clinical knowledge, and reasoning are used to make decisions that are effective for a specific client -1. research evidence -2. clinical knowledge -3. patient values and circumstances -4. information from the practice context

cohort studies

-exposure-oriented -resarcher observes the intervention but does not control it -following a gorup of subjects (cohort) who are likely to develop a certain outcome -prospective: cohort is identified and followed over time to see what happens -retrospective: cohorts are defined from a previous point in time and information is collected about the outcome -benefits: can provide data concerning the timing of the outcome, can define possible causal outcomes (correlations), can capitalize on larger data sets (do a lot of statistical analyses) -but can't define causation (not an RCT)

value-based payment

-facility or provider payments are tied to performance on defined outcomes -targeted outcomes have a strong evidence-based interventions -poor performance is tied to financial penalties -objective is to incentivize providers to deliver high quality care

asking clinical questions

-focus time and effort -get high-yield search strategies -clarify your question -helps to communicate with others -more likely to find an answer

confidence interval (CI)

-gives an estimated range of values which is likely to include an unknown population parameter -the estimated range is calculated from a given set of sample data -assumes normal distribution -usually calculate from CI using +/- 2 SDs (95% CI) -population mean = sample mean +/- 2 SD -small CI = similar results between groups

standard error

-gives us an estimate of the spread of the variable for the entire population -versus the standard deviation, which gives the variable for a sample

ordinal measurement

-group all things (all inclusive) -groupings are mutually exclusive -arrangement of groupings suggest an inherent order -but the "distance" between groupings is unknown -comparisons are distributions (e.g., medians, quartiles)

hypothesis testing

-hypothesis: essential part of analytic studies; born out of statistical analysis -analytic science is inherently skeptical (always starts assuming no effect)

factorial design

-include 2 or more independent variables -subjects randomly assigned to various combination of levels of the 2 variables

student's t-test

-independent sample t-tests -evaluates whether 2 group means are different from each other -different participants in each group -equal or unequal variances need to be considered -indicates that the study probably had a control group (more robust findings) -data needs to normally distributed

non-randomized design

-intervention is under control of the researcher -study participants aren't randomized (allocation is usually arbitrary) -pre-post designs are the most commonly designed

one-way repeated measures design

-just 1 group -baseline measurement -get intervention/condition 1 -measurement -intervention/condition 2 -disadvantage: the order -advantage: determine feasibility (inform the bigger picture)

external validity

-the extent to which the results reflect the truth about what may happen outside this study, but in similar situations -phase of research -internal validity: do we trust the results at all, and do they apply to sitautions outside the study? -sample size: more is better -sample characteristics (similarity to target population, reflects your population) -setting of the study (similarity to your setting, differences to your setting) -protocol of study (does intervention align with clinical environment, unrealistic for real practice situations (resources (time, money, space), equipment, staffing))

PICO: population of interest

-the features of the people that are of most importance -demographics (e.g., age, gender, race, ethnicity) -diagnosis -disability status

experimental study

-the intervention is under the control of the researcher

PICO: intervention

-the intervention you're questioning; be specific -treatment -exposure to disease -risk behavior

research design

-the master plan specifying the methods and procedures for collecting and analyzing the needed information -the choice of design depends on objectives of the research, how much is known about the problem, and practibility/feasibility

median

-the middle-most observation of ordered data

mode

-the most frequently occurring observation

interval measurement

-the objects are ordered; there is equal distance between the points on the scale -the "distance" between groupings is known -comparisons are meaningful in terms of amounts -you can make predications based on these amounts -continuous or discrete data

mean

-the sum of observational scores, divided by the number of observations

randomized block design

-used when an extraneous factor might influence differences between groups -in order to control for this effect, this variable is build tinto the design as an IV -control for a variable that may impact an outcome differently -"blocking variable" (can't change this variable) -Ex: separting men and women, and each group gets the same treatment (gender blocking)

PICO: outcome of interest

-what is the helath status or behavior that you're interested in achieving

PICO: comparison

-what your intervention can be compared to -no therapy/placebo -standard therapy -alternate therapy -absence of disease -absence of a risk factor

ordinal data

-with a natural order -categorical data -ex: socioeconomic status (SES), ranking in a race (1st, 3rd, 6th person in race), government pay scale

crossover design

-with only 2 levels of an independent variable are repeated, a preferred method to control or order effects is to counterbalance the treatment conditions so that their order is systematically varied -to see if order matters -ex: half subjects received A before B, and the other half received B before A

nominal data

-without a natural order -categorical data -ex: gender, race and ethnicity, handedness

steps in EBP

1. Ask (questions) 2. Acquire (search) 3. Appraise (critical appraisal) 4. Apply (integrate into practice) 5. Evaluate (evaluate 1-4 and revise)

PICO

1. Population/patient/problem 2. Intervention/exposure 3. Comparison 4. Outcome

PICO question template

In (P) ____, what is the effect of (I) ____ on (O) _____ compared with (C) ______?