Ethical Issues in Clinical Research, Principles of Measurement, Concepts of Measurement Reliability, Concepts of Measurement Validity, Evidence-Based Practice, Sampling and Validity in Experimental Designs, Principles of Clinical Trials, Quasi-Experi...
Design survey questionnaire quote
"Anybody can write down a list of questions and photocopy it, but producing worthwhile and generalizable data from questionnaires needs careful planning and imaginative design."
Construct Validity of Causes and Effects
"Concerns the theoretical conceptualizations of the intervention and response variables and whether these have been developed sufficiently to allow reasonable interpretation and generalization of their relationship."
Repeated Measures design
"One group of subjects is tested under all conditions and each subject acts as their own control." and other term "Within-Subjects design"
Experimental bias (2)
-Hawthorne effect: the subject does better because he is being observed -Experimenter effect: the experimenter's dress and behavior affect the subject
Internal validity-single group threats (when one group is tested) (5)
-History -Maturation -Attrition -Testing -Instrumentation
Instrumentation
-Measurement errors -Example: observers, stopwatch timer, calibration of the device
Example of Cluster Sampling
-National Health and Nutrition Examination Survey -Multi-stage cluster sampling
Calculated value of t: numerator and denominator
-Numerator is the difference between means -Denominator represents the pooled variance within groups
Observe trends and patterns in the data to evaluate variability of the response over time
-Obscured in group studies where responses are taken pre and post treatment
Random errors (2)
-Occur in many directions and unpredictable -Due to chance (human fatigue, inattention, mechanical inaccuracy, simple mistakes)
Threats to Validity: -No study is ___ -Remember to discuss ___ and ___ -It is difficult to apply the near-perfect set up of a lab to the ___ ___ -Address ____ ____ in your study design and methods
-Perfect -Generalizability and limitations -Clinical setting -Potential threats
Measurements with RCT (2)
-Pre-intervention (before the program) -Post-intervention (after the program)
Split-Middle Line (2)
1. A median line in the baseline phase 2. Extended to B phase to test null hypothesis that 50% of intervention scores and below split-middle line
Restrictions on Interpretation of Linear Regression Analysis (3)
1. Cannot go beyond values of X that we used to create the regression line 2. Is used to predict score for a new sample, not to predict scores we already know 3. Linear regression analysis is not useful if the graph is nonlinear
Snowball Sampling (2)
1. Chain referrals 2. Used in sensitive topics or rare traits
Self-report outcome measures and physical performance measures are used in clinical research to assess: (2)
1. Change after interventions 2. Diagnostic and prognostic accuracy
Conclusion (4)
1. Chi-Square measures association between two categorical variables 2. The purpose of chi-squared is to test the difference between observed frequencies (O) and the frequencies expected by chance (E) 3. The purpose of goodness-of-fit test compared observed frequencies to a theoretical or known distribution
Controlling Inter-Subject Differences Random assignment can help balance the distribution. Other ways to control: (5)
1. Choose homogenous samples on subject traits 2. Block by attribute as an independent variable 3. Match subjects based on characteristics 4. Use the subjects as their own control 5. Analysis of covariance (ANCOVA)
Contributing to Evidence-Based Practice (3)
1. Clarify understanding of patient perspectives and needs 2. Understand provider and caregiver practices 3. Inform translational research (moving scientific discoveries quickly into practice), understanding why certain interventions do not get used or incorporated into usual care
Where do research questions come from? (3)
1. Clinical experience e.g. knowledge, experience and curiosity 2. Clinical theory 3. Professional published literature:
Coding (2)
1. Closed choice responses 2. Open-ended responses
Make a list of resources (5)
1. Cochrane-for SRs-why reinvent the wheel? 2. PEDro for RCTs, SRs, and Clinical Practice Guidelines (CPGs) 3. Databases: PubMed, CINAHL, EBSCO 4. APU library all databases 5. Grey literature
Three Parts to Analyzing Survey Results
1. Coding 2. Summarizing survey data 3. Missing data
Characteristics of PubMed Advanced Search (5)
1. Combined search terms 2. Alternative terms 3. Provides history of the search 4. Shows hits with each term 5. Fosters refining the search
What boolean operators are used for? (3)
1. Combining terms 2. Alternative terms 3. Excluding terms
Design of Surveys (10)
1. Come up with a research question (PICO) 2. Add guiding questions 3. Come up with hypotheses 4. Create an outline 5. Review existing instruments 6. Design your instrument 7. General to specific questions, 11 or 12-point font, color paper 8. Preliminary draft to colleagues 9. Pilot test on 5-10 participants from the population of interest 10. Choose a sample from the accessible population
The next slides cover (3)
1. Commonly reported Pearson product-moment where both X and Y are on interval or ratio scales 2. Also Spearman rho for ranked data (ordinal) 3. Correlation methods for dichotomous data
History Definition and Example
1. Confounding effects of special events outside of the intervention 2. Example: workout at the gym before muscle testing
Types of Nonprobability Sampling (4)
1. Convenience sampling 2. Quota sampling 3. Purposive sampling 4. Snowball sampling
Interpreting Correlation Coefficients (7)
1. Correlation vs. comparison 2. Correlation looks at the covariance of X and Y 3. Comparison (aka means comparison) looks at the difference between the distributions 4. Causation and correlation: NO 5. Correlation does not imply causation 6. Correlation demonstrates that two variables are related 7. Causation is established under controlled experimental conditions and implies one variable caused an effect on another
Kruskal-Wallis Analysis of Variance (three groups or more) (3)
1. Corresponds to a one-way ANOVA 2. Uses x^2 (df=k-1) 3. Multiple comparison procedures can be applied
Calculated value of t compared to critical value of t (2)
1. Critical value depends on directional or non-directional hypothesis 2. Degrees of freedom based on the sample sizes
Retrospective Research (3)
1. Data collected in the past e.g. how many people were treated using US for tendinitis in the past 20 years in a Los Angeles PT department 2. Medical records, databases, surveys 3. No direct control of the variables under study
Prospective Research (3)
1. Data collected in the present 2. Follows subjects as they are being treated 3. More reliable than retrospective studies because of the potential for greater control data collection methods and understanding of study protocol
Outliers (4)
1. Data points that lie outside the cluster of scores 2. Examine using a scatter plot 3. May be an extreme score due to a small sample size or may result from measurement error or from other extraneous factors 4. Need to determine if outliers should be retained or discarded
Target Behavior: Choosing (5)
1. Define the behavior 2. Assess relative stability 3. Define the intervention 4. Define a measure 5. Decide on the number of targets, interventions, measures
Two measures of variability (general)
1. Dispersion of scores 2. Differences that exist among a set of scores
Grey literature (2)
1. Dissertations and Theses-Unpublished Studies 2. Fact sheets by condition-specific organizations
Systematic replication (2)
1. Done after direct replication 2. Repeat the study in different environments
Multiple comparison procedures can be applied (2)
1. Dunn's multiple comparison 2. Mann-Whitney U test with a Bonferroni correction
Repeated Measures ANOVA (4)
1. Each subject tested under all experimental conditions 2. Also called within-subjects design 3. Means represent different treatment conditions rather than different groups 4. Simplest example is the one-way repeated measures design, which has one independent variable
Data Analysis (7)
1. Graph the data 2. Look at the patterns in A and B phases 3. Split the baseline from the intervention phases 4. Look for a linear trend line in the intervention phase 5. Split middle line: find the median of the baseline phase, then extrapolate the line into the intervention phase to test the null hypothesis 6. Look at the outcomes vs. the MCID, MDC or SEM 7. Additional Statistical Analyses can be found in Portney, 2020, Chapter 18
History (5)
1. Greatest threat to internal validity 2. The longer you wait between measures-->introduce new ingredients to the pot of stew 3. How do you know you have an award-winning recipe? 4. Work hard to address threats to validity in your study 5. Work hard to scrutinize for threats to validity in studies you read
Questions to Consider in Selecting a Design (5)
1. How many independent variables are being tested? eg Exercise and education 2. How many groups of subjects are being tested? eg treatment and control group 3. How will subjects be assigned to groups? Randomly using a computer generated number 4. How often will observations of responses be made? Eg Physical therapy-every treatment session 5. What is the temporal sequence of interventions and measurements? Research-baseline and end of treatment
Simple linear regression (3)
1. How well one variable (X) predicts the second variable (Y) 2. Independent variable, X 3. Dependent variable, Y
Experimental Design (3)
1. Large samples with control groups 2. A few measures 3. Averages and group generalizations may miss individual characteristics
What is a statistic in descriptive statistics?
A descriptive index computed from sample data
Define directional hypothesis
A directional (or one tailed hypothesis) states which way you think the results are going to go
Cluster sampling
A form of probability sampling in which large sub-groups (clusters) are randomly selected first, and then smaller units from these clusters are successively chosen
What do face-to-face interviews provide in regards to informed consent?
A form to sign
Example of Snowball Sampling
A friend tells a friend
Assumptions for Regression Analysis
A horizontal band is good, it means the assumptions for linear regression have been met
What is the measures of variability of the coin rotation test?
A less variable than B
What may you give in analysis?
A narrative description of the results
Define non-directional hypothesis
A non-directional (or two tailed hypothesis) simply states that there will be a difference between the two groups/conditions but does not say which will be greater/smaller, quicker/slower
Experimental Design Definition
A structure for evaluation a cause and effect relationship between a set of independent and dependent variables
Cluster random assignment
A study that is exploring the benefits of exercise for patients admitted in medical ward with COVID-19 in Huntington Hospitial -A Ward versus B Ward
Define sample
A subgroup of the population. Serves as the reference group to estimate characteristics of and to draw conclusions about the population
Define frequency distribution
A table of rank ordered scores that counts the number of times each value occurred (frequency)
Define variable
A variable is a property that can differentiate individuals or objects. It represents an attribute that can have more than one value
What is a standardized score relative to the mean?
A z-score
Validity and Reliability Dart Board
A: Scores are reliable, not valid B: Scores show random error, average validity C: Scores are not reliable, not valid D: Scores are both reliable and valid
Examples of Different Variances
A: Variance between groups No variance within groups B: Equal variance within groups C: Greater variance within groups D: Unequal variance within groups
Example of previously stated
A: n=10 B: n=50
Comparison of Group Means
ANOVA looks at distance of each group mean from the grand mean (total mean)
Define scales
An ordered system based on a series of questions or items which provides an overall rating representing the degree the respondent possesses a value, characteristic or attitude
What type of analysis is done in chi-square?
Analysis of proportion of frequencies
What does one-way repeated measure?
Analysis of variance
What two things must not be identified of subjects?
Anonymity and individual's names
Example of convergent validity
Anxiety Inventory for Respiratory (AIR) and HAD scale
Example of: 1. To describe the quality or quantity of a given variable
Anxious, IQ, ROM, MMT
Define open-ended questions
Ask respondents to answer in their own words provides their feelings and opinions
Define close-ended questions
Ask respondents to select and answer from among several fixed choices
Internal consistency
Aspects within a test measure the same thing, example: Anxiety Inventory Respiratory Disease scale
What does chi-square measure?
Association between two categorical variables
Example of Rank Correlations
Association between verbal and reading comprehension
Example of Partial Correlations
Association of age (X) and hospital stay (Y), with the effect of function removed (Z)
What is inferential statistics based on?
Assumptions on how the sample represents the population
Where are the specific aims and objectives found?
At the end of the introduction
What type of categorical data is seen in the analysis?
Averages and frequencies
Example of a One-Way ANOVA
Change in elbow pain-free ROM with four modalities: Rest, Ice, Splint, Nonsteroidal anti-inflammatory drugs (NSAID)
Example of: 4. To evaluate a patient's progress
Change in gait distance after treatment
What should you always allow subjects to do?
Change their mind
Testing effect
Changes occur because of the test
Group work
Check final slide for research study and questions
Linear Regression Example
Check slides
Example of a group of individuals followed over time
Children who are born in 2000 in US, exposed to tobacco smoke and not exposed in 20 years follow-up developing lung disease
Example of: 3. To make a choice on treatment approach
Choosing one treatment over another based on comparative research
What type of scale is the nominal scale?
Classificatory
Define clinical research
Clinical research is a structured process of investigating facts and theories and exploring connections, with the purpose of improving individual and public health
Superiority Trials
Clinical trials that seek evidence in favor of a new treatment
What questions should you code in analysis?
Close-ended questions
What serves as a detailed reference for writing systematic reviews?
Cochrane Handbook for Systematic Reviews of Interventions
What measure of variability is this? Another way to describe interval/ratio data Useful for comparing measures that have different units Ratio of the standard deviation to the mean Expressed as a % Cannot use negative numbers
Coefficient of variation
Example of therapeutic trial
Cold therapy in reducing swelling after an injury
What type of responses are in the analysis section?
Collate responses
Example of Purposive Sampling
Collect subjects from each class based on GPA
Example of Quota Sampling
Collect volunteers from each DPT class till quota is met for each class
What does a meta-analysis do?
Combines the selected studies in a quantitative analysis and provides an overall estimate of the intervention effect
Time series design: What type of interventions?
Community based
Define Quasi-Experimental research
Comparative research approach in which subjects cannot be randomly assigned to groups or control groups are not used
What is the purpose of goodness of fit test?
Compares observed frequencies to a theoretical or known distributions
Simple random assignment
Computer generated numbers, tossing a coin
Which Type of Validity does this purpose align with? The extent to which the target test correlates with a reference standard taken at relatively the same time.
Concurrent validity
Which type of validity does this purpose align with? Establishes the ability of an instrument to measure the dimensions and theoretical foundation of an abstract construct.
Construct validity
Which Type of Validity does this purpose align with? Establishes that the multiple items that make up a questionnaire, inventory, or scale adequately sample the universe of content that defines the construct being measured.
Content Validity
What does content validity evolve out of?
Content validity evolves out of the process of planning and constructing a test
Part of construct validity is based on what?
Content validity; define the universe of content that defines the construct
How long does Delphi Survey continue?
Continues until responses are consistent with the previous round
Advantage of Repeated Measures design
Controls for individual differences
Power and Effect Size (continued)
Conventional effect sizes:
Which type of validity does this purpose align with? The extent to which a test correlates with other tests of closely related constructs.
Convergent validity
Example of correlation
Correlation between grip strength and falls, both a function of conditioning
Example of exploratory/observational study?
Correlation e.g. height and weight
Reporting reliability
Correlation or association?
Mann-Whitney U test (two groups)
Corresponds to an independent t-test
What type of arithmetic properties are in nominal scales?
Counting, percentages
Arithmetic properties of ordinal scales
Counting, percentages, medial values are useful
How do you address rater bias?
Covering the instrument readings, having another team member give the intervention
Which Type of Validity does this purpose align with? Establishes the correspondence between a target test and a reference or "gold" standard measure of the same construct.
Criterion-related validity
Both SR and MA do what?
Critically appraise research studies
LR: Current or History?
Current
Frequencies represent individual counts
Data represents actual number of persons, objects, or events in the category
What are P-values a good report for and bad report for?
P-values are not the best report of correlation, better for F tests and t-tests, ANOVAs.
Examples of variables (3)
P: men vs. women, kids vs. adults, exercisers vs. sedentary I/C: running vs. biking O: Rating of Perceived Exertion vs. VO2 max
A literature review will look at the variables of the ____
PICO
____ ____ are keywords for the literature search
PICO terms
Example of using a truncation symbol
PTSD AND music* would find articles with the terms PTSD and music/musical/musician/musicians/musicality in them
Examples of dependent variables
Pain, range of motion
Define Delphi Survey
Panel of experts complete a questionnaire to ID their opinions
Parametric versus nonparametric tests
Parametric test use: interval, ratio data e.g. ROM Nonparametric test use: Quality of life e.g. Likert scale Descriptive test: nominal, rank order e.g. gender
Define single blind study
Participants are unaware of group assignment but the researchers do
Random Assignment: Process of assigning subjects to group
Participants have an equal chance of being assigned to any group
Total Variance in a One-Way ANOVA
Partitioning variance into components -Between groups (SSb) and within groups error (SSe)
Maturation Definition and Example
Passage of time changes people and their responses Example: child development tests
What does qualitative research focus on? Interview using?
Patient's experiences, feelings and attitudes. It uses a semi-structured interview using open-ended questions. Eg tell me about your shoulder problems
An example of discrimination between two conditions of interest
Patients with ACL injury attend an outpatient PT department and the therapist performed a screening using the Thessaly test and confirmed with diagnostic tool with MRI as (Gold standard)
Example of variables that may confound results or interfere with interpretation of the findings?
Patients with Cancer with OA
Example of Inclusion Criteria
Patients with diagnosis of OA, older than 60 years old, male and female
Example of ceiling and floor effects and definition
Patients with knee problems using pain scale everyone (most) scores '0' floor effects or everyone scores '10' ceiling effects' In other words, a floor effect which is not being able to show small differences when patients are near the bottom of the scale
Time series design: What does it look for?
Patterns or trends
How is test-retest reliability reported?
Pearson's r, Spearman's rho, ICC's
Example of historical control
People who were studied for pain study 20 years ago, as a control group
What can you report in analysis?
Percentages
What measure of variability is this: ́A score's position within a distribution ́ Data is divided into 100 equal portions
Percentile
What is the purpose based on?
Proportion of frequencies in each category
Two examples of longitudinal studies
Prospective and retrospective
What should you do to obtain informed consent in telephone interviews?
Provide full information at start of call to obtain verbal consent
When to use...
Proxy decision maker (e.g. children, individuals with severe Alzheimer disease)
Falls under Exploratory analysis
Purpose is to measure the relationship between two measured variables
Example of looking at content validity
Quality of life is important in adult patients with chronic illnesses. Which outcome measure has a better content validity? -Lower Extremity Functional Scale (LEFS) or 36-item short form health survey (SF-36)
What do we need to describe what we see?
Quantitative numbers
What are the two ways that values can be measured?
Quantity (height, weight) Qualitatively (gender, hair color)
What measure of variability is this: ́Distribution of scores is divided in four parts ́ 25%=Q1, 50%=Q2, 75%=Q3 Q3-Q1=the middle 50% range Q2 is the median
Quartile
What type of symbol is a wildcard?
Question mark ?
What r value is considered important?
R closer to 2.0 is considered important because it allows comparison across categories
What is the gold standard for developmental research?
RCT
Examples of direct variables
ROM (using goniometer), leg length (tape measure)
Examples of ratio scales
ROM, height, weight -50# is half of 100#
Randomized controlled trials: ____ assignment
Random
True experiment design:
Random assignment
What does QED lack?
Random assignment or comparison groups, or both
What type of control trial is the gold standard?
Randomized control trial (RCT)
Example of explanatory/experimental study
Randomized controlled trials e.g. Experimental and control group
What measure of variability is this: ́Difference between highest and lowest values in a distribution ́ Use minimum and maximum scores ́ Limited as it does not reflect the dispersion between the two scores
Range
Interval scale: _____ order
Rank
What do numbers indicate in ordinal scale?
Rank order
Example of prior:
Rate of Perceived Exertion between genders during Submaximal Treadmill Testing
What is intrarater reliability affected by?
Rater bias; when the rater remembers prior scores or is biased by knowledge of which intervention was given
What type of data and scale is VAS considered?
Ratio data and a continuous scale
What type of bias can occur in surveys?
Recall bias
What does content validity refer to?
Refers to the adequacy that the complete universe of content is sample by a test's items
Define precision variables
Refers to the exactness of a measure. In statistic, it shows the exact decimal places
Define statistical regression and example
Refers to the natural effect of participants at extreme ends of a measure falling in a certain direction just due to the passage of time rather than the effect of an intervention -Example: High and low scores
What does a predictive correlation study use?
Regression, statistical analysis technique to predict behavior or response
Example of investigating intervention effects
Regular exercise may lead to increase in physical fitness
Example of: 3. Can we make useful and accurate predictions about a patient's future function or status based on the outcome of a test?
Regular physical exercise improving physical fitness or reducing overweight in one year time
Define diagnostic accuracy
Relates to the ability of a test to discriminate between the target condition and health
Example of Pearson Correlation Coefficient
Relationship between ambulation and cognitive function
What does a correlation study look at?
Relationship or association between variables
What should you look at in analysis if possible?
Reliability and validity
Define reliability
Reliability is the "extent to which a measurement is consistent with results upon repeated application
What should follow-up communication consist of?
Reminders to facilitate an increased return rate and thanking the participants
What type of measure is used in single-subject designs?
Repeated measurement
Define primary sources
Reports provided directly by the investigator, such as journal articles
Examples of primary sources
Research articles e.g. Randomized Controlled Trials and systematic reviews
Example of treatment and setting
The exercise program works in a hospital setting. Validity is threatened when you try to apply it to an outpatient setting
What is the Generalizability Theory?
The idea that not all error is random; some error components can be attributed to other sources, such as rater or test occasion.
Define face validity
The implication that an instrument appears to test what is intended to test
What does the box represent in the box plot?
The interquartile range
In content validity, what must the items adequately represent?
The items must adequately represent the full scope of the construct being studied
Purpose of the Chi-Square Test
The purpose is to test the difference between observed frequencies (O) and frequencies expected by chance (E)
What else has reliability in the measurement system?
The rater
What is a predictive correlation study based on?
The relationship of the behavior and other variables
Define the alternative hypothesis
The research hypothesis
Define interviews
The researcher asks respondent specific questions and records the answers
What does the research hypothesis state?
The researchers true expectation of the results, guiding the interpretation of outcomes and conclusions
Examples of a One-Way Repeated Measures ANOVA
The same sample (n=9) tested the elbow flexor strength in three forearm positions
Define minimally clinically important difference (MCID)
The smallest difference that signifies an important difference in a patient's condition
What is a z-score relative to a mean?
The standard deviation above or below
Define the null hypothesis
The statistical or test hypothesis
Double-blind study
The subjects and the investigators are blinded to the intervention allocation
Define index test
The test that is being validated against a known gold standard
Uniform distribution
The total sample is divided equally across categories
Treatment and selection example
The treatment works on dancers who are 18-35 years old. Validity is threatened if you try to say there is a cause-effect in 50-year-old dancer
What does the 0.05 come from?
The z-score
Experimental group
Research manipulates variables and outcomes to control for confounding effects
How much does research misconduct occur?
Research misconduct occurs to a substantial degree among biomedical researchers from both industry and universities (Godecarchle et al., 2018).
What do individuals need to be respected with from the beginning to end of study?
Respect
What kind of a study is case-control studies?
Retrospective epidemiological study
Examples of secondary srources
Review articles and textbooks
What type of information can be on a systematic review?
Rigorous and comprehensive secondary information
What does MA only include?
Statistical synthesis of the data
Analysis of covariance (ANCOVA)
Statistical technique used to compare groups while controlling for the effect of an extraneous variable
What does Rasch analysis do?
Statistically manipulates ordinal data to create a linear measure on an interval scale
Define PubMed Clinical Queries
Strategies for targeting search
Quota Sampling
Stratification used to obtain representative proportions of specific subgroups
Presence of association does not imply a causal relationship
Strong relationship between X and Y may be a function of a third variable
Are most interviews structured or un-structured?
Structured
Example of alternative hypothesis
There is a difference in the treatment of pain in young adults with chronic shoulder problems with heat therapy compared with cold therapy
Example of null hypothesis
There is no difference in the treatment of pain in young adults with chronic shoulder problems with heat therapy compared with cold therapy
What must there be to be a qualitative research method?
There must be a developing theory, understanding, or explaining of an observed phenomenon
Example of non-directional hypothesis
There will be a difference in heat therapy compared with cold therapy in the treatment of knee problems in young adults
Example of diagnostic trials
Thessaly test in the diagnosis of anterior cruciate ligament injury
What do these requirements provide for researchers?
They provide a framework to guide researchers and IRBs in their assessments of individual clinical research protocols
What do medical subject headings provide?
They provide a hierarchy of key terms related to search that can be used to search with varying level of specificity
In content validity, what should the number of items that address each component reflect?
They should reflect the relative importance of that component
National Research Act (1974)
This law requires the development of a full research proposal that identifies the importance of of the study, obtaining informed consent and review by an institutional review board (IRB).
Example of: 2. The instrument taking the measurement
Imprecise instrument
Disproportional sampling Example
In a group of 500 male dancers and 2500 female dancers, the sample had 25 male and 75 femal dancers
Where must data be stored?
In a secured environment
Define likelihood ratio (LR)
In diagnostic testing, the ratio indicating the usefulness of the test ruling in or ruling out a condition
Where may informed consent be included?
In the cover letter
Threat to Statistical Conclusion Validity (4)
Inappropriate use of statistical procedures for analyzing data leading to invalid conclusions -low statistic power -violated assumptions -high error rate -low reliability and high variance
Demographics
Include questions that can characterize the sample of respondents
How do you define the target population?
Inclusion and exclusion criteria
Selection of studies for review require?
Inclusion and exclusion criteria
What are the intervals in ordinal scale?
Inconsistent or not known
Define Positive predictive value (PPV+)
Indicates the probability that individuals with a positive test truly have the disease
Define variables
Individual, object, or environmental characteristics
Define confounding effects
Influences on the independent variable and its relationship to the dependent variable by extraneous variables
Example of blocking variable used during random assignment
Inpatient orthopedic hospital setting: Ward A patients receive (New intervention) versus Ward B patients (Standard intervention)
Test-retest reliability
Instrument between occasions/time intervals, usually 7 to 10 days
Define historical research
Intended to document events in the past as a way of understanding why they occurred and how they have influenced current practice
What type of reliability is usually done in single-subject designs?
Inter-rater reliability
Correlation Matrix (example)
Intercorrelations of variables related to evidence-based clinical practice in physical therapist students
What accounts for both correlation and agreement?
Intraclass Correlation Coefficient (ICC)
What are the two types of extraneous variables?
Intrinsic and Extrinsic
What should you keep track of in analysis?
Invalid responses and unanswered questions for consideration in statistical analysis and final report
Define time series design
Involves observation measurement in a defined period of time
In content validity, what should the test not contain?
Irrelevant items
Correlation looks at the covariance of X and Y
Is a change in X proportional to a change in Y?
Paired T-Test
Is for comparing two means for correlated samples or repeated measures
Define the intervention question
Is it applicable to addressing the target
Retrospective epidemiological study
Is useful in studying conditions with long latency or incubation periods
Define discrete variables
It can only be measured in separate units and that can not be measured in intervals of less than 1
Clinical prediction rule
It indicates the likelihood of a condition being present based on a set of criteria
Define stem and leaf plot
It is a grouped frequency distribution is like a histogram on its side, but with individual values.' It shows a pattern of distribution of a continuous variable
What is rater bias influenced by?
It is influenced by subjective scoring
How is a criterion-referenced test interpreted?
It is interpreted according to a fixed standard that represents an acceptable level of performance
Challenges with Cross-Sectional Research
It may not be possible to know whether the presumed cause (exposure) truly preceded the outcome
What does validity relate to?
It relates to alignment of the measurement with a targeted construct; i.e., can inferences be made?
What does reliability relate to?
It relates to the consistency of a measurement E.g. consistent results during repeated assessment
How do frequency distributions display data?
It summarizes it
What should you let the patient know in regards to answering?
It's okay to refuse to answer
Example of looking at expected relationships
Lean body mass and bone density during puberty
Practice Effects
Learning effect with repeating a task over time
Case Studies: ____ rigorous type of study design
Least
QER: ___ control
Less
Larger sample more or less variable?
Less variable
How does construct and content validity apply to the Holy Spirit?
Like the wind; you can see the effect in people's lives but not actually see a physical being
Generalizability in Q-sort?
Limited because subjects are not randomly chosen
Length of the survey
Long surveys less likely to be completed
Experimental research definition
Looks at the effect of one or more variables on an outcome (Cause and effect)
Skewed to the right
Lots of scores at the low went, not many at the high end
Greater sampling error leads to what accurate mean?
Lower
Is nominal the lowest or highest level of measurement?
Lowest
Example of a criterion-referenced test?
Lung function test (LFT): LFT with forced expiratory volume<70% predicted with obstructive lung disease
Define minimal clinically important difference (MCID)
MCID is the smallest difference in a measured variable that signifies an important rather than trivial difference a measurement E.g. housebound patient able to walk to the shops
Define minimal detectable change (MDC)
MDC is the amount of change in a variable that must be achieved to reflect a true difference, the smallest amount of change that passes the threshold of error E.g. the change shows statistically significant benefit, may not be a change in patients daily activities
Example of dichotomous variable
Male or female
Examples of ordinal scales
Manual muscle test (0-5), balance; good>fair>poor, pain: 0-10 pain scale
Example of prefacing sensitive questions
Many people forget to take their medication from time to time. Do you ever forget to take your medication?
Blinding is also referred to as
Masking
_____ if the number represents a known quantity (numerals)
Mathematical
Assumption that no more than 20% of the cells contain an expected frequency less than 5
May collapse the table to combine categories or increase sample size
What is usually given with continuous data?
Mean
Mean of the sample=
Mean of the popullation
What does measures of central tendency reference? (3)
Mean, Median, Mode
Give the mean, median, and mode (general): Example: In the city of Bradbury, CA, the for sale homes are listed as $1.5, $2, $2, $2.1, $2.5, $3, $5, $11, and $68 million dollars
Mean: high because of the 68 mil home Mode: Low because most of the houses are lower priced Median: Usually reported because it is unaffected by the extreme home
Define random assignment (random allocation)
Means that each subject has an equal chance of being selected to any of the group Eg experimental or control group
Example of knowing what exactly made the difference?
Measure ROM of the knee contracture over 5 hot pack and massage sessions in the clinic
Define variance
Measure of variability between scores
What is measurement a basis for?
Measurement is a basis for making decisions or drawing conclusions
What does semantic differential measure?
Measures attitudes and feelings on a continuum between two extreme opposites, usually a 7-point scale
What two measures do we typically need?
Measures of central tendency and variability
Define parameters in descriptive statistics
Measures of population characteristics
What is not affected by outliers or extreme scores?
Median
What is the horizontal line at in a box plot?
Median
What is in the middle in skewed distributions?
Median between mode and mean in skewed distributions
Examples of active variables that are manipulated
Medication or placebo
Median
Midpoint in a list of scores
Define blinding
Minimizes observation bias by ensuring those involved in the study are unaware of a subject's group assignment
What do the whiskers show in a box plot?
Minimum and maximum scores
When does misuse occur?
Misuse occurs because of incorrect use of treatments or diagnostic tests that can result in no benefit or harm, such as providing medications for elderly patients that are innapropriate
Example of a wildcard
Mobili?ation will return mobilization and mobilisation
What is usually given with categorical data
Mode
ICC 0.50 to 0.75
Moderate reliability
Convenience sampling
Most common method in which subjects are chosen on the basis of availability
Time series design: Measures?
Multiple measures before and after treatment
Define One-Way Repeated Measures Design Over Time
Multiple measures of the dependent variable take over prescribed time intervals
What are the categories in nominal scale?
Mutually exclusive
What is a great source of secondary data?
National Center for Health Statistics
What does rater reliability need?
Needs to be consistent for you to be confident for the results of the data
Another name for combining terms?
Nesting
What does the research question open?
New investigations with defined variables and design strategies to address issues in a population with certain disorders
Can you generalize the findings of your case to the general population?
No
Is face validity the same as content validity?
No
Is it correct to say that there is a 95% probability that the population mean falls within an obtained confidence interval?
No
One-Way Repeated Measures Design Over Time: Comparison group?
No comparison group
One-Group Pretest-Posttest Design: Comparison group?
No comparison group, which limits internal and external validity
Repeated measures design: Comparison group?
No comparison group, which limits internal and external validity
Interaction between respondents in Delphi survey?
No interaction
Levels of evidence (1-5)
No longer have subcategories 1a, 1b, 2a, 2b
What is the difference between Non-Equivalent Pretest-Posttest Control Group Design and pretest-posttest experimental design?
No random assignment, uses intact groups
One-Way Repeated Measures Design Over Time: Randomization?
No randomization
Too specific PICO
No results
When Ho is true
No treatment effect, the error of the variance gets larger and the t-ratio will be smaller
Interval scale: ___ ___ zero
No true
___ true ____ in ordinal scale
No true zero
Disproportional sampling random?
No; the men have a greater chance of being selected as there are fewer to choose from
What type of scales are used in chi-square?
Nominal or ordinal scales
QER: ____ groups
Non-equivalent
Non-probability samples=
Non-random selection
What happens when you take on a new patient with new baseline data?
Nonconcurrent multiple baseline design
What may be included as a choice in branching?
None of the above as a choice
What can be used when the assumptions are not met?
Nonparametric statistics
What can nonparametric statistics be used with?
Nonrandom samples, when there is no HOV, when the participants in the sample are too varied
What type of design is one-group pretest posttest design?
Not randomized, therefore quasi-experimental design
Do we collect a sampling distribution of the population?
Not really
What does quantitative research use?
Numbers to quantify outcomes. They are numerical data, statistical analysis, close-ended
____ are given to both quantitative and qualitative values
Numerals
How are numerals used? (Example)
Numerals are used as labels, with no quantitative meaning, such as coding data on an opinion scale
What is the Visual Analogue Scale typically confused with?
Numerical Pain Rating Scale (1-10) which is an ordinal scale
H0 in Goodness of Fit
Observed proportion in sample does not differ from the proportion in the expected population
Measurement error
Observed score=true score±error component
Example of Repeated Measures Design
Older patients with shoulder pain receiving different levels of strength of steroid injection e.g. 100ml, 200ml, 300ml
When does overuse occur?
Overuse occurs when treatments or tests are provided even when potential harms exceed likely benefits, such as unnecessary surgeries or mammograms for older women
What can quality issues come from? (3)
Overuse, underuse, or misus of treatments or tests
Example of predictive validity
The HAD scale test is performed at the evaluation of patients with chronic lung disease. The HAD scale is given at baseline 8 weeks after pulmonary rehabilitation. The HAD is both valid and reliable. The score is good for the patients who got better after the rehabilitation with low anxiety score. Those patients with low anxiety score walked for longer-distances compared with high anxiety score. Thus, the HAD is a good predictor of exercise capacity
How many members must the IRB have?
The IRB must have at least five members with diverse backgrounds that facilitate complete and adequate review process
McNemar Test for Correlated Samples
The McNemar test is the version of chi-square used with a repeated measures design when data are categorical
Example of concurrent validity
The Thessaly test is much cheaper than an MRI. The tuning fork test was used on battlefields to triage for fractures as there were no X-ray machines
Define construct validity
The ability of an instrument to measure an abstract construct and the degree to which the instrument reflects the theoretical components of the construct
Define descriptive research
Documents "the factors that describe characteristics, behaviors, and conditions of individuals and groups" e.g. Socio-demographic characteristic: age, gender, smoking status
What are the types of primary sources that can be used in historical research?
Documents, diaries, letters, photographs or videotapes, eyewitness accounts, procedure manuals, published histories, meeting minutes or direct recordings of events
Example of test of independence
Does a relationship exist between depression and dementia? Yes or No?
Attrition
Dropout
Why is a power analysis done during a study?
During study planning to estimate sample size -A priori analysis
Run-in period
During which all subjects receive a placebo treatment. To demonstrate that subjects can adhere to regimen are then randomized to be part of the study
Define cumulative
Each item represents and increasing amount of the attribute measured
Random assignment
Each subject has an equal chance of being assigned to any group
Therapeutic trials
Effect of an intervention
Power and Effect Size
Effect size indices for the ANOVA -eta squared (n^2) -Cohen's f Values are based on results of ANOVA
Example of: 2. Can the test evaluate the magnitude or quality of a variable or the degree of change from one time to another?
Efficacy of physical therapy in reducing pain in patients with osteoarthritis
Errors in Hypothesis Testing: What is the decision
Either correct or not correct
CD: Participants are randomized to a treatment sequence
Either intervention followed by a control or controll followed by an intervention
A-B-A Design
Endurance (6MWT) in a child with cerebral palsy -Intervention: Aquatic-based aerobic exercise program
Concealed Allocation
Ensures group assignment is done without knowledge of those involved in the experimental process. Minimizes bias in group formation
Interval scale: ____ intervals
Equal
Ratio scale: ____ intervals
Equal
Define equipoise
Equipoise means that there is genuine uncertainty about a treatment's effectiveness, supporting testing it on humans.
Example of: 3. The variability of the characteristic being measured
Error often occur due to physiological changes e.g. blood pressure
Differences between more than two means
Error variance within groups
Belmont Report (1979)
Established the rules and regulations that govern research efforts of the Department of Health and Human Services (DHHS) and the Food and Drug Administration. The office of human research protection (OHPR) is the administrative arm of DHHS that is responsible for implementing and regulations and providing guidance to those who conduct human studies.
Define predictive validity
Establishes the outcome of the target test can be used to predict a future criterion score or outcome
Define Standard Error of the Mean
Estimate of the population standard deviation
Confidence Interval for Independent T-Test
Estimates the difference between population means
Confidence Interval for Paired T-Test
Estimates the population mean difference
What should investigators be trained appropriately to understand?
Ethics, law as well as social norms and responsible citizen when conducting research
Stratified random assignment
First dividing the subjects into strata and then within each stratum randomly assigning them to groups
Example of Consecutive Sampling
Flyer is sent to recruit volunteers e.g. DPT students for research -Beware of volunteer bias
PICO: ___ is important in clinical research
Focus
EBP: ___ and ____ to the patient
Focused and specific
Question order
Follow thought process
Where are there challenges in branching?
For data analysis
Example of Likert Scale
For each statement below, please indicate whether you strongly agree (SA), Agree (A), are neutral (N), disagree (D), or Strongly Disagree Total the points for each item that represents the same thing, coded SA=5, A=4, N=3, D=2, SD=1
Chapter 2 of Capstone Paper
Full literature review
What is required with face to face with any increased risk?
Full review
Match subjects based on characteristics example
Gender -Not recommended for experimental design
Choose homogenous samples on subject traits example
Gender differences may influence the outcomes of the jump height study
What is Delphi survey good for?
Good for problem-solving practice issues in PT
Make a comprehensive list of what you want to look for
P-I-C-O terms and root words for search terms
T-Test for Independent Samples: Testing pinch strength
n=10 each group -Hypothesis is that the treatment group (hand splint) will pinch better than the control (no splint) group -Is this a directional or non-directional hypothesis -One-tailed or two? -Alpha=?
Reliability and validity of an instrument is
never perfect
CR: Generates ____ ____
new knowledge
Finding no effect in your study does not mean there is
no effect possible
ICC=0
no reliability
Cateogorical scales are
nominal
Historical control is ___ ____
non-concurrent, not happening at the same time
Mcnemar Test for Correlated Sample: A form
of chi-square for correlated samples
Where is the Cochrane Handbook for Systematic Reviews of Interventions free?
online: www.cochranelibrary.com
CD: Used to control for
order effects
Predicting _____ to draw conclusions about relationships and consider expect responses
outcomes E.g. smoking for 20 years may lead to lung cancer
Case series example
patients who received similar treatment and outcome
Informed consent ensures the need to respect individuals
personal autonomy
ICC<.5
poor reliability
CR: Encourages the improvement of ____ and ___ ____
practice and patient care
There are ____ properties under specific conditions and populations
psychometric
Specific aims or objectives: Describe the ___ of a project; what the study weeks to ___
purpose; accomplish
r^2 is a measure of proportion representing the total variance in Y scores that can be explained by x scores
r^2 is called the coefficient of determination
Blocking variable used during
random assignment to ensure that participants are similar across groups on this attribute
An ordinal scale is
ranked
Define effectiveness trials
real-world conditions (clinical setting in hospital) standardize the availability of the intervention in the study sample but do not go to extremes to reinforce implementation by providers or participation by patients. Eg all patients with neck problems
Justice
refers to fairness in the research process or the equitable distribution of the benefits and burdens. E.g. selection of subjects should not be discriminatory, drawn defined population
Autonomy
refers to self-determination and the capacity of individuals to make decisions affecting their lives and to act on those decisions. E.g. Children, cognitive problems
Beneficence
refers to the obligation to attend to the well-being of individuals. Those who engage in research may gain possible benefits and minimize harm. E.g. new knowledge and vaccine development
SAOO: Reflect the ___ or ___ that are intended
relationships; comparisons
LR: Relevant studies address the components of the ___ ___ or the ___ __
research question or the PICO question
Define secondary sources
reviews of studies presented by someone other than the original author
A confidence interval is an interval estimated based on
sample mean and standard error of the mean
The tendency for sample values to differ from population values is called
sampling error
Wording in Survey Questions: Preface
sensitive questions
A questionnaire is a ___ survey
structured
What are the three types of interviews?
structured, semi-structured, unstructured
Likert scale is what kind of a scale?
summative
External Validity
the extent the results of a study can be generalized beyond the internal specifications of the study sample
Define correlation
the measure of the degree of association between variables
Accessible population
the population of interest within reach
What must the IRB review?
the research proposals prior to implementation to ensure that the right of research subjects are protected.
When tossing a die.... An event is a ....
there are six possible outcomes single observable outcome
Null hypothesis
there is no difference
Quasi-experimental: think
think non-randomized design
The Common Rule
this rule codifies regulations adopted by many federal departments and agencies with a focus on institutional guidelines, informed consent, and review proposals for protection of participants.
Need a control group to control for the threat of ___, also known as ___ threat to validity
time, maturation
Control group: NO
treatment or placebo
Nonequivalent posttest-only control group design uses
uses existing groups instead of random assignment
Surveys are generally
valid
Psychometric properties
validity, reliability, MCID, MDC, sensitivity, specificity
CD: Considerations for
washout period
We cannot defend the null hypothesis,
we reject or do not reject
Target Behavior is chosen by
what matters most to the patient
CD: Should only be used
when condition is stable
Recall bias in case-control studies
Those with a condition may have better recall of exposures
Nonequivalent posttest-only control group design: threats due to
Threats due to confounding, selection biases, and attrition
How is ordinal data examined?
Through Rasch analysis
How is Semantic Differential examined?
Through factor analysis
How is previous exposure determined in case-control studies?
Through interview, questionnaires, chart review
One-Group Pretest-Posttest Design: What is the independent variable?
Time
One-Way Repeated Measures Design Over Time : What is the independent variable?
Time
Repeated measures design: IV?
Time
What is the IV in one-group pretest-posttest design
Time
What is the purpose of normative studies?
To describe typical or standard values for characteristics of a given population E.g. adult blood pressure 120/80 (as known standard)
Why is a power analysis done after a study?
To determine if a type II error was committed -Post Hoc Analysis
Basics of a Linear Regression
To predict an outcome or to explain the nature of an association based on shared variance
What is a truncation symbol used for?
To search different word forms
Too general PICO
Too many results on things not related to your real question
Define summative
Total score with each item contributing equal weight to the total
Example of Active controls
Treatment arm receiving both exercise therapy and education and control arm receiving education only for the treatment of anterior cruciate ligament injury
Ratio scale: ____ zero ____
True zero exists
Alternate forms
Two forms of same test; PT board exam
Example of outside factors affecting the posttest between different groups
Two groups of swimmer cross the English channel and time is the posttest. One group is followed by sharks
What does the independent samples t-test test the difference between
Two independent groups e.g. testing the range of movement of patients with knee injuries compared with healthy subjects without knee injuries
Define convergent validity
Two measures are believed to demonstrate the same underlying phenomenon
Example of discriminant validity
Two measures used to demonstrate high correlation Example: AIR and HAD scale
How many groups in RCT?
Two or more groups - Experimental group (receive new treatment) - Control group (placebo)
How do you exclude the retrieval of terms from your search?
Type 'Not'
Consecutive sampling
Type of convenience sampling in which all subjects are recruited who meet inclusion and exclusion criteria as they become available
How do you narrow a search in PubMed?
Type using "And"
How do you increase a search in PubMed?
Type using "or"
What is validity strengthened by in case studies?
Using validated and reliable outcome measures
Does Q-sort have good reliability?
Usually
Multiple Baseline Designs: -Concurrent?
Usually concurrent
Define discriminant validity
Validation assessing the degree to which an instrument yields different results when measuring two different constructs; that is the ability to discriminate between the constructs
Define validity
Validity relates to the confidence we have that our measurement tools are giving us accurate information about a relevant construct so that we can apply results in a meaningful way.
The strength of the relationship
Values between -1.0 and +1.0, where 0 is no relationship
What is reliability estimated based on?
Variance
What measure of variability is this? ́A measure of variation in a full set of scores ́ The deviation of each score from the mean ́ Subtract the mean from each score (X - X̅) ́ The sum of all the deviation scores will always be zero ́ Since there will be both (+) and (-) deviation scores, this is solved by squaring each deviation score ́ Then sum the squared deviation scores to get a sum of squares (SS) = Σ (X - X̅)2 Greater variability equals greater sum of squares (SS) ́ The SS is divided by n-1 to obtain the mean square (MS), which is the mean of the squared deviation scores, called the sample variance (s2) True measure of variabilitys2= SS/(n-1)= {Σ (X - X̅)2 }/(n-1)
Variance
How can you get to databases?
Via the APU library www.apu.edu/library -Find PubMed *Check slides for links*
Example of a measurement tool we can have confidence in
Visual analogue scale (0-10), measuring the level of pain
Example of intrarater reliability
Visual assessment of highest jump
Example of influence of subjective scoring
Visual movement analysis vs. jump mat reading
Example of Interrater reliability
Visual video analysis of the highest best quality jump
What is VAS?
a 100 mm line drawn, and the patient puts a hash mark to denote pain level, which is measured No pain__________________________________________________Terrible pain
3. Lowering alpha will require stronger evidence to show a significant decrease
Will decrease the change of a type I error, (remember type I error=where you say something is significant but it is not)
4. Much harder to find a significant difference
Will increase the chance you commit a type II error (remember type II error=where you say something is not significant but it is)
Correlation can only identify linear relationship
With curve correlation is low (r=.58)
How should you summarize responses in the analysis?
With descriptive statistics
Assumption with Random Errors
With more measures, "random errors cancel each other out, so the average score is a close estimate of the true score."
Repeated measures design
Within-subjects design -Subjects as their own control
Define research misconduct
a means of fabrication, falsification or plagiarism in proposing, performing or reviewing research, or in reporting
Define sensitivity
a measure of validity of a screening procedure, based on the probability that someone with a disease will test positive, true positive rate
Sensitivity=
a/a+c
Power of the study=
ability to find significant differences when they exist
Individuals must be.. (informed consent)
accurately informed (e.g. purpose, methods, risks, benefits)
What do you still need even if there is no physical contact or face-to-face?
an IRB
Research studies usually pick a ___ ___, manipulate some _____ vs. ____, and look at one or two ____
certain demographic; intervention; outcomes
CR: Provides potential ___
connections
SSD: "Clinically viable...
controlled experimental approach to a single case or several subjects."
Specificity=
d/b+d
EBP: Related to ____, ____, and/or ___
diagnosis, prognosis, and/or intervention
Controls
do not have the disorder (serve as comparison group)
LR: Contributes to understanding of what has been ___ and what is ___
done; missing
Wording in Survey Questions: Should beware of
double-barreled questions
CR: Provides _____ to justify treatments
evidence
Numerals are used to code data...
for analysis for qualitative values (0,1)
Probability is a system of rules
for analyzing a complete set of possible outcomes E.g. 1) When a flipping a coin, there are two possible outcomes
Wording in Survey Questions: Add
frequency and time
Threats to external validity occur when we try to
generalize outside of the study context and specifications
Case studies are not
generizable
What does systematic review report on?
latest, best evidence to help inform a clinical decision
Example of controlling for individual differences
leg length may affect the results of the Y-Balance test, so treatment effects are analyzed within each participant separately across treatment conditions
EBP: Weighing the ____ and ____ of the studies
level and quality
ICC>0.75
good reliability
Example of % of the total distribution
grades
Cases
have the disorder
Example of: 1. Is a test capable of discriminating among individuals with and without certain traits, diagnoses, or conditions?
healthy subjects versus patients
LR negative (-)
indicates how much the odds of the disease are decreased if the diagnostic test is negative
LR positive (+)
indicates how much the odds of the disease are increased if the diagnostic test is positive
Define negative predictive value (NPV-)
indicates the probability that individuals with a negative test truly do not have the disease
Reliability
inter-rater (two or more raters), intra-rater (one person)
Continuous scales have
interval or ratio values
Pearson's r
interval/ratio-level data
A jury cannot defend the innocent,
it decides guilty or not
When the effect of one or more attributes is studied,
it is not a true experiment
Skewed to the left
lots of scores at the high end, not many at the low end
Define fabrication
making up data or results and recording or reporting them. E.g. presenting data never occurred.
Define falsification
manipulating research materials, equipment or processes, or changing or omitting data or results such that the data are not accurately described in the research record.
Define exempt review
may be allowed for surveys, interviews, or studies existing records, provided that the data are collected in such a way the subjects cannot be identified.
Define specificity
measure of validity of a screening procedure, based on the probability that someone who does not have a disease will test negative, true negative rate
Define repeated measures design
multiple measures of the dependent variable (outcome measure) are taken
Elements of informed consent (12)
• Concise introduction • Confidentiality • Compensation • Contact information and request for more information • Consent statement • Purpose of the research • Procedures • Potential risks and discomforts • Potential benefits • Information on study outcomes • Signatures • Alternatives to participation 5 C's, 4 P's I, S, A
Actions of Data Misconduct (5)
• Dropping observations or data points from analyses based on a gut feeling they were inaccurate • Willfully distorting research results or data • Knowingly overlooking others' use of flawed data or methods • Inventing research data or cases • Failing to present data that contradict one's own previous research
Actions of Cutting Corners of Misconduct
• Inadequate monitoring of research projects due to work overload • Cutting corners in a hurry to complete a project • Continued unintentional carelessness in conducting research • Inappropriate or careless review of papers or proposals
Actions of Method Misconduct (3)
• Inadequate record keeping or data management related to research projects • Using inadequate or inappropriate research designs • Withholding key aspects of methodology in papers or proposals
Define selection bias in case-control studies
Cases and controls from otherwise similar population
Quasi experimental designs: Experimental
Cause and Effect-Quasi-Experimental
Example of: 1. The individual taking the measures
Lack of skill, not following the protocol
Consort and Pedro
Last 3 slides
Example of quantitative research
What is your level of pain today? Using a pain scale 0= no pain and 10= severe pain
Randomized block design: used when
concerned that an extraneous factor might influence the between group differences
Example of SAOO
"The purpose of this study is to examine the effect of ultrasound compared to hot pack in reducing pain in young adult tennis players with lateral epicondylitis"
Never say (2)
"the results of this study support there is no effect" "the null is true" just say reject or not reject
Frequency
# of occurrences of falls
Example of descriptive research
% of DPT students who sleep <8 hours/night
Z score=
(X- XJ)/s
Equation for z-score
(X-mean)/standard deviation
Z-ratio=
(XJ-μ)/s
To get the 95% interval, what is the z-score?
+-1.96
Grade: Degree of consistency of effect between or within studies: Scores
+1 Evidence of dose response across or within studies (or inconsistency across studies is explained by a dose response); also 1 point added if adjustment for confounders would have increased the effect size 0: All/most studies show similar results -1: Lack of agreement between studies (e.g. statistical heterogeneity between RCTs, conflicting results)
Declaration of Helsinki (1964)
- It highlights the concept of independent review of research protocols by a committee of individuals who are not associated with the proposed research project. - Declares that reports of research that has not been according to stated principles should not be accepted for publication.
Change scores used to:
-Demonstrate effectiveness of an intervention -Track the course of a disorder over time -Provide a context for clinical decision making
Nuremberg Code (1949)
-Emphasizes that every individual should voluntary consent to participate as a research subject. -Research should be conducted only by scientifically qualified person.
Researcher manipulates variables
-Active variables are manipulated -Attribute variables are observed, not manipulated
Interrater reliability
-Can 2 or more raters get the same measure? -How much variation exists between raters? -Do first intrarater reliability then perform interrater reliability? -Do interrater reliability with all raters at the same time, but independently?
Convential effect sizes for d (3)
-Small d=0.2 -Medium d=0.5 -Large d=0.8
Example of Stratified Random Sampling
-Study of depression as a risk factor for dementia -Proportional sampling for four strata
Two group Pretest-posttest design (6)
-Two groups --One treatment A group, one treatment B group -Random assignment -Pretest-->Intervention A or intervention B-->posttest -Usually done comparing a new treatment to a standard treatment -Done when it is not ethical or feasible to have a control group -Cause-effect between two treatments, not vs. no treatment
What is a truncation symbol usually?
-Usually an asterisk* -Some databases may use $ or #
Experimental Design Characteristics: -The most ____ form of scientific investigation -Supports a ___ relationship
-rigorous -cause-effect
Statistical power
1-beta -Power is the probability that a test will lead to rejection of the null hypothesis, or the probability of attaining statistical significance
Across behaviors (2) (Example)
1. 3 or more behaviors in the same patient Example: Down syndrome child, prelinguistic requesting, vocal imitation, commenting
What are the constant and predictable characteristics within a normal distribution? (3)
1. 68% of the scores are within +- 1 SD of the mean 2. 95% of the scores are within +-2 SD of the mean 3. 99% of the scores are within +-3 SD of the mean
Example of T-Test for Paired Samples (2)
1. 8 subjects are randomly selected, and each is tested for pelvic tilt without the pillow, then with the pillow 2. Each subject is measured under both conditions so this is a repeat measures design
Correlation of Dichotomies (4)
1. A dichotomy is a nominal variable with only two values 2. Usually represented as 0 or 1 in data 3. Can measure association if X or Y or both are dichotomous 4. Use a Chi-square test
Cohort studies (4)
1. A group of individuals followed over time 2. Study association between exposure and the outcome of interest 3. Also called a follow-up study 4. Either prospective or retrospective
Correlation (2)
1. A measure of association between two variables, X and Y 2. Does not imply causation
Retrospective study (3)
1. Difficulty in defining variables 2. Records many not be complete 3. Best for disorders with long latency periods
Examples of Withdrawal Designs (2)
1. A-B-A 2. A-B-A-B
Typical format for reporting your findings (6)
1. Abstract 2. Introduction 3. Methods 4. Results 5. Discussion and conclusion 6. References
The test statistic (3)
1. Different for different types of studies 2. Significance level usually alpha=0.05, but depends on the study, may need to subdivide if variables affect each other 3. Z-ratio
Level of significance
1. Alpha: probability of making a Type I error 2. How much error is acceptable?
Designs with Multiple Treatments (3)
1. Alternating Treatment Design (A-B-C-B-C) 2. Multiple Treatment Design (A-B-C-A) 3. Interactive Design (A-B-BC-B)
Examples of boolean operators (3)
1. And 2. Or 3. Not
Advantages of Nonparametric Statistics (3)
1. Appropriate for analysis of nominal or ordinal data (scales) 2. Tests hypotheses for group comparisons without normality or variance assumptions or the shape of the underlying population distribution 3. Data are reduced to ranks
One-Way ANOVA (3)
1. Appropriate for one-way design with one independent variable -Three or more levels 2. Comparison of between-group variance and within-group variance 3. Variance expresses as sum of square (SS)
What are the decisions that the IRB can make on a research proposal? (4)
1. Approve 2. Require modifications in 3. Defer approval 4. Deny approval
Data extraction (2)
1. Assessment of study quality 2. Data for meta-analysis
Adjustments for Small Samples (2)
1. Assumption that no more than 20% of the cells contain an expected frequency less than 5 2. Consider other approaches
Multiple Baseline Designs (4)
1. At least 3 data different sets of baseline data 2. Across behaviors 3. Across subjects 4. Across conditions
To compare more than 2 means within the same sample (2)
1. Based on an assumption that values of the outcome variable are independent of each other 2. Use of multiple t-tests will increase the chance of making a Type I error
Nonparametric analog of Pearson's r (2)
1. Based on ranked data 2. Ordinal data or non-normal distribution
What are the two design phases in single-subject designs?
1. Baseline phase (prior to treatment) 2. Intervention phase (during treatment)
Multiple Treatment Design (A-B-C-A) (3)
1. Baseline, Treatment B, Treatment C, Baseline 2. Treatment vs. control or treatment vs. treatment 3. "C" can be placebo
Alternating Treatment Design (A-B-C-B-C) (3)
1. Baseline, Treatment B, Treatment C, Treatment B, Treatment C 2. Treatment vs. control or treatment vs. treatment 3. "C" can be placebo
Length of phases (3)
1. Best to keep equal intervals for comparison 2. Usually 1 week 3. Graphing responses
Grading of Recommendations of Assessment, Development, and Evaluation (GRADE): Quality based on (4)
1. Blinding and allocation process 2. Follow-up and withdrawals 3. Sparse data 4. Other methodological concerns (e.g. incomplete reporting, subjective outcomes)
Physiological (1)
1. Blood pressure
Limited, N=1 (2)
1. But.. not unreasonable to consider this may work for a similar patient 2. Provides "real world insight"
Non probability samples (3)
1. Cannot estimate the degree of sampling error 2. Limits the generalizability of the outcome beyond the specific sample 3. Often done in clinical research because it is easier
What is done in descriptive statistics? (2)
1. Describe a population 2. Characterize data by shape, central tendency and variability
Correlation Coefficients (2)
1. Describe the strength and direction of a relationship between variables 2. How does one variable (X) go together with another variable (Y)?
Purposes of Qualitative Research (3)
1. Describing groups of people or social phenomena 2. Generating hypotheses that can be tested by further research 3. Developing theory to explain observed phenomena e.g. exploring the DPT students studying habits and strategies of preparing for exams
Research categories (3)
1. Descriptive 2. Exploratory 3. Experimental
Exploring relationships (2)
1. Descriptive research 2. Analytic research
Conclusion (3)
1. Descriptive research is the basis for any kind of research 2. Qualitative study generates research hypotheses and questions that may be a backbone for a quantitative study 3. Combining qualitative and quantitative (mixed research method) approach is the new trend in research
Case studies provides (3)
1. Detail on unusual patient conditions 2. Examples of innovative or creative therapies 3. Future research directives
Relevance of Measurement Scales (3)
1. Determination of which measures to use in clinical practice 2. Determination of which interpretations are meaningful 3. Statistical procedures
Types of Descriptive Research (5)
1. Developmental Research 2. Normative Studies 3. Qualitative Research 4. Descriptive surveys 5. Case studies
What can case studies be useful for? (7)
1. Diagnosis 2. Prognosis 3. Intervention 4. Application of theory to practice 5. Clinical measurement procedure 6. Administrative/educational processes 7. Risk management
Basics of Analysis of Variance (ANOVA) (5)
1. Differences between more than two means 2. Uses the F statistic 3. Based on parametric assumptions of normal distribution and equal variance among groups (Levene's test) 4. Measures between-group treatment effects as a ratio to within-group variability 5. Can be used in independent samples or repeat measures, must meet parametric test assumptions
The variability within each group (error variance) (2)
1. Error variance does not imply mistake, it indicates all sources of variability 2. Can be done
What are the 6 parts of writing good questions?
1. Every question should be answerable by every subject 2. Questions should be easy to answer 3. Consider recall of information 4. Consider if respondents will be honest 5. Try to use a variety of question types 6. Questions should generate varied responses
Probability Samples=Random Selection (4)
1. Everyone in the population has an equal chance of being selected 2. Reduces bias but not perfect 3. Samples are "considered" representative of the population-not actually are because there is a chance of variation 4. Very difficult to do. How can you have access to the whole population of interest?
Crossover design example (6)
1. Example: 20 patients with a urinary tract infection 2. Randomly assigned to 4 weeks in one of two groups --Treatment group of taking a cranberry pill --Placebo group takes a placebo pill 3. Pretest-->intervention-->posttest 4. Washout of 1 week 5. Switch groups 6. Pretest-->intervention-->posttest
Pearson-Product Movement Correlation Coefficient Example (4)
1. Example: n=12 normal infants, 30 weeks old 2. Tests on proximal (reaching) and distal (prehensile) skills 3. H0: p=0 There is no relationship between the two behaviors and Pearson's r=0 4. H1: p>0 There will be a + relationship, r>0
Uses of Observational Research (3)
1. Exploring relationships 2. Estimating risk 3. Investigate intervention effects
Effect size index, d (2)
1. Expresses the difference between two means in standard deviation units 2. May be referred to as the standardized mean difference (SMD)
How can an interview be conducted? (2)
1. Face to face 2. By phone
Correlation Concepts (3)
1. Falls under Exploratory analysis 2. Contrast to the mean difference studies we have looked at so far that examined how group A is different from Group B 3. Correlational studies examine the relationship between Group A and Group B and asks "Does Variable A increase with Variable B?"
Longitudinal research (3)
1. Follows a cohort and collects data over time with repeated measurements 2. Long-term documentation of the same subjects E.g. Five years follow-up DPT students after graduation of job satisfaction 3. It is important in establishing cause and effect, the time sequence of predictor and outcome variables is vitally important
Summarizing survey data (2)
1. Figures 2. Cross-tabulations
Interval recording (2)
1. For repetitive behavior 2. How many seizures in a given preset time interval (Monday, Tuesday)
Nonlinear regression (2)
1. For use when the relationship between X and Y is not a straight line 2. Example of a polynomial (quadratic) regression equation
Chi-Square Statistic Assumptions (2)
1. Frequencies represent individual counts 2. Categories are exhaustive and mutually exclusive
What is displayed in a coin rotation test? (2)
1. Frequency distribution 2. Cumulative percent
Professional published literature (3)
1. Gaps identified from a previous work 2. Conflicting findings (positive and negative) 3. Replication of previous studies with some modification e.g. sample size
Power-Efficiency (3)
1. Generally parametric tests are more powerful than nonparametric tests 2. Increasing sample size for a nonparametric test will increase the power of the test 3. Rule of thumb to estimate sample size for a nonparametric test: compute the needed sample size for a parametric test and add 15%
Four Methods to Display Frequency Distributions
1. Grouped frequency distribution 2. Histogram 3. Line plot (grouped data) 4. Stem-and-leaf plot
Correlation Matrix: Significance of r (5)
1. H0: r=0 There is no relationship between X and Y 2. H1: r>0 or r<0, or H1: r ≠ 0 There is a positive (or negative) relationship 3. The significance of a correlation coefficient does not mean the correlation coefficient represents a strong relationship 4. Always interpret the r in light of the sample size 5. Correlation coefficients are sensitive to sample size: with an increase in sample size, any observed correlation would likely be significant, even if r is too small to mean anything in terms of correlation. High in power even with a small sample. Risk of a Type I error
T-Test for Independent Samples (2)
1. HOV tested by Lavene's test, if p>.05, there are no significant differences between group variances 2. If Levene's test is p<0.05, then sample sizes and variance are unequal -In this case, we have to get (t) with separate variance calculations --A different calculation you don't need to know for this class
Factors need to be considered in Quasi-experimental design (5)
1. History of the design 2. Maturation (passing of time) 3. Testing procedure 4. Instrument accuracy 5. Replications of the study
What must you obtain for ethics in survey research? (2)
1. IRB review and approval 2. Informed consent
Explanatory (RCT) Trials
1. Ideal situations to control for confounding 2. Strict and broad exclusion criteria 3. Standardized treatment protocol
5 Steps of the Research Process
1. Identify the Research Question 2. Design the Study 3. Implement the Study 4. Analyze the Data 5. Disseminate Findings
Research Process (9)
1. Identify the research problem 2. Do a literature search and review 3. Redefine the research problem with clear variables and outcomes 4. State the purpose and hypotheses 5. Design the study protocol (participants, methods) 6. Collect and data 7. Analyze data 8. Interpret results 9. Report findings and suggest future studies
The Research Process (7)
1. Identify the research problem 2. Do a literature search and review 3. Redefine the research problem with clear variables and outcomes 4. State the purpose and hypotheses 5. Design the study protocol (participants, methods) 6. Collect and data 7. Analyze data 8. Interpret results 9. Report findings and suggest future studies
Four types of target behaviors
1. Impairment level 2. Functional level 3. Physiological 4. Subjective
Look for the linear trend line in the intervention phase and example
Calculate the slope values from two different times -Example: Day 9/Day 4
Control Group Types (3)
1. Inactive controls 2. Wait list controls 3. Active controls
Specifics with efficacy trials (4)
1. Incorporates random assignment to groups to minimize bias 2. Specific treatment protocol 3. Strict inclusion and exclusion criteria 4. Randomized controlled trial
Who should you consult about a COI? (2)
1. Independent person 2. Research ethics committee
Validity considerations (4)
1. Instrument and tests may have more than one type of validity 2. There are many ways to report validity 3. We need to pay attention for factors that affect validity 4. It is important to use a well-validated outcome measure to show the efficacy of an intervention
Target Behavior Measuring (4)
1. Instrumentation 2. Frequency 3. Interval Recording 4. Duration
What do the changes in outcome measures affect? (2)
1. Insurance reimbursement 2. Number of patient visits
Systematic collection of repeated measurements over time? (3)
1. Usually at frequent and regular intervals 2. Observe trends and patterns in the data to evaluate variability of the response over time 3. Response patterns can be modified as study progresses to get the most meaningful outcome
Strength of Correlations (2)
1. Interpretation affected by sample size, measurement error, types of variables being studied, and their application 2. Authors should provide rationale for interpretation
X and Y continuous variables
1. Interval or ratio scales 2. Normal distribution
What can systematic review be on? (3)
1. Interventions 2. Prognostic indicators 3. Diagnostic tools
What do surveys come in the form of? (2)
1. Interviews 2. Questionnaires
Examples of reliability of a measure (2)
1. Intra-rater reliability refers to the stability of data by one tester across two or more trials 2. Inter-rater reliability refers to the variation between two or more raters who measure the same subjects
Withdrawal Designs (2)
1. Introduces a second baseline period 2. May also include a second intervention period
Questions Addressed by Validity (3)
1. Is a test capable of discriminating among individuals with and without certain traits, diagnoses, or conditions? 2. Can the test evaluate the magnitude or quality of a variable or the degree of change from one time to another? 3. Can we make useful and accurate predictions about a patient's future function or status based on the outcome of a test?
Good questions to ask about a research question? (5)
1. Is the question important and answerable 2. Is the study feasible? 3. Is there sound rationale to justify the study? 4. What is the target population? 5. Is it ethical?
Analogy
1. Lighting and risk of being struck in a lightning-prone area: would you test it out once or many many times? -What are the chances with many trials? -Potential cumulative error is the same -You will say there is a significant difference when there really is none, and you will reject the null when you shouldn't -You will commit a Type I error
Generalizability (7)
1. Limited, N=1 2. How can we make the results stronger? 3. Direct replication 4. Systematic replication 5. Use of an individual patient 6. Clinical replication 7. Social validation
Prospective Longitudinal Study (3)
1. Long time period to consistently collect data 2. Exposure status determined at start, with follow-up to see if outcome develops 3. Confounding can occur over time from other events and conditions
Methods for Developmental Research (2)
1. Longitudinal 2. Cross-sectional
Observational Study Designs (2)
1. Longitudinal studies 2. Cross-sectional studies
Use the subjects as their own control (3)
1. Repeat measures and look at the response to different conditions within each subject 2. Subjects matched to themselves 3. Within subjects design
Selected Databases and Search Engines (4)
1. MEDLINE (PubMed) 2. CINAHL 3. Cochrane Database of Systematic Reviews 4. Physiotherapy Evidence Based (PEDro)
Search Strategy (2)
1. Make a comprehensive list of what you want to look for 2. Make a list of resources
T-Test for Paired Samples (4)
1. Matched pairs -Siblings, twins 2. Repeat measures -Subjects are their own control 3. More powerful, less variability
Cross-sectional research (2)
1. Measures a stratified group of subjects at one time point to draw conclusions about the population e.g. prevalence of obesity in children in US 2. Variables are measured concurrently, meaning that their time sequence cannot be confirmed
Challenges for cohort studies (4)
1. Misclassification of exposure 2. Bias 3. Attrition 4. Outcome may not occur in sufficient numbers -Not appropriate for studying rare conditions
Direct replication (1)
1. Repeat the study with several subjects or even the same subject
Similar to the Spearman correlation (3)
1. More conservative 2. Accounts for ties 3. Based on ranked data
Specifics with Effectiveness Trials
1. More generalizable 2. Treatment more reflective of practice 3. Inclusion and exclusion criteria not as strict, e.g. allows for comorbidities 4. Quasi experimental design
Pragmatic (PCT) trials (4)
1. More reflective of practice situations 2. Diverse patient populations with fewer exclusion criteria 3. Active controls 4. Treatment reflects practice and is less standardized
Tests of Independence (2)
1. Most common application of chi-square 2. Examines association between two categorical variables
What are some of the question formats that are asked in close ended questions? (8)
1. Multiple choice questions 2. Dichotomous questions 3. Check all that apply 4. Measuring intensity 5. Checklists 6. Measurement scales 7. Visual analog scales 8. Rank-order questions
Can you imply causation in regression? (4)
1. NO 2. Prediction of relationship 3. Not experimental 4. No causation
Uses the F statistic (2)
1. Named for Sir Ronald Fisher 2. Comparison of calculated F ratio to critical value of F
Clinical replication (2)
1. Repeat this in the clinic 2. With successful results reported in case reports, you have improved generalizability
Baseline Characteristics (3)
1. Repeated measurements during each phase 2. Demonstrate stability 3. Demonstrate trend
Strengthen by replication (3)
1. Repeating phases 2. Additional subjects 3. Comparing a second intervention
Limitations of the A-B Design (3)
1. No control comparison 2. Impossible to conclude causality 3. Strengthen by replication
Significance criterion (4)
1. No direct mathematical relationship between alpha and beta 2. Alpha and beta do affect each other 3. Lowering alpha will require stronger evidence to show a significant decrease 4. Much harder to find a significant difference
What are the four levels of measurement?
1. Nominal 2. Ordinal 3. Interval 4. Ratio
Assumptions of Non-Parametric Tests (3)
1. Non-parametric tests are not susceptible to outliers like parametric 2. Do not require restrictive assumptions 3. Can be used when you have non-normal interval or ratio level data
Spearman Rank Correlation Coefficient (2)
1. Nonparametric analog of Pearson's r 2. Symbol, rs
Assumption for Regression Analysis (4)
1. Normal distribution 2. Every observed value of Y for a given X will vary above and below the line 3. If we took many measures, the mean would fall on the regression line 4. Checking assumption sis done by analyzing residuals
Calculated value of t (2)
1. Numerator is the mean of the paired difference scores 2. Denominator represents the standard error of the difference scores
Social Threats (5)
1. Occur because the control and experimental group communicate with each other 2. Try to keep this from happening 3. When the group receiving no treatment learns from the treatment group and begins their own type of intervention 4. When the control group inadvertently receives extra care to equal out the fairness in treatment 5. When the control group becomes resentful
Systematic errors (2)
1. Occur in one direction 2. Predictable
Sampling bias (2)
1. Occurs when the individuals selected for a sample overrepresent or underrepresent certain population attributes e.g. Sample size 50 PT students (45 male and 5 female) 2. Can be conscious or unconcious
Tests hypotheses for group comparisons without normality or variance assumptions or the shape of the underlying population distribution (2)
1. Often used with small samples 2. Referred to as distribution-free
Across subjects (2) (Example)
1. One intervention given to several subjects 2. Example: Intense PT intervention for 4 patients with CP
Across conditions (2) (Example)
1. One patient in different environments Example: Patient with autism treated in the AM vs. PM, or in a loud clinic vs. a quiet one
Calculated value of t compared to critical value of t (2)
1. One tailed test 2. Degrees of freedom equals n-1
Threats to Construct Validity are Related to (3)
1. Operational Definitions 2. Length of the follow-up 3. Experimental bias
Multiple group threats (2)
1. Outside factors affect the posttest between different groups 2. The threats are the same as single group threats expect they affect one group over the other and threaten the validity of the study results
What do case studies develop?
A clinical knowledge base
• Case scenario : " A community survey on prevalence of domestic violence among secondary school students". 1. Who we should obtain the consent? students, parents, teachers or ministry of education. 2. If the results show that 50% of the students have been abused should we report them to the police. 3. I have decided to publish it. Can I send an abstract for presentation as part of the conference proceedings, and later submit similar abstract with the full text for publication. Is that redundant publication? 4. Can I submit the same paper in a different language?
1. Parents 2. Confidentiality? 3. Yes; it is not redundant 4. Yes
Who is blinded in studies? (3)
1. Participants 2. Care givers 3. Outcome accessors
National Institue of Health Clinical Research definition (3)
1. Patient oriented research 2. Epidemiologic and behiavioral studies 3. Outcomes research and health sciences research.
Examples of Correlation Matrix (2)
1. Pearson's r with n≥15, a moderate correlation r=0.45 would be considered significant, p<0.05. With n>60, even a small value like r=0.20 would be significant, p<0.05 2. "Children's Height Linked to Test Scores" N=14,000 children p<0.05, but r=0.11
Examples of Inactive controls (3)
1. Placebo 2. Sham 3. Attention control group
Recruitment (6)
1. Plan for subject recruitment from the accessible population 2. Not everyone invited will meet the selection criteria or agree to participate 3. Reporting subject participation through stages of the study 4. Relationship of sample size and the power of the study 5. How many patients do you need? For a pilot study, 15-20 patients 6. Randomized controlled trial: requires power calculation
Estimating population parameters (2)
1. Point estimate, e.g. mean 2. Confidence intervals
Regression of SBP on BMI for 10 Women (4)
1. Points fall in a linear pattern 2. Regression line has some error 3. (Y-Y hat) is the residual (difference between the actual and predicted values of Y) 4. Regression is "line of best fit", with smallest residuals
A measure of association between two variables, X and Y (2)
1. Positive relationship (X increases as Y increases) 2. Negative relationship (X increases as Y decreases)
Three next parts of reporting the findings?
1. Poster presentation 2. Platform presentation (5, 10, 20 min, or several of your studies as part of a full 50 minute education session) 3. Journal publication (JSPTR)
Variance (2)
1. Power is increased as variance (variability) in the data is decreased 2. This is accomplished in study design and sample size choice
Disadvantages of Repeated Measures (3)
1. Practice Effects 2. Carryover effects 3. Order effects
EBP: The expertise and experience of the clinician (2)
1. Practice pattern 2. Clinical setting
Quasi-experimental designs (6)
1. Pre-test and post-test design: Non-equivalent control group design 2. Single subject design 3. Time series design 4. Multiple baseline designs 5. Survey research: cross sectional design 6. Longitudinal survey designs (eg: cohort study)
Three parts of contacting the respondents
1. Prior to the survey 2. The cover letter 3. Follow-up communication
Two important concepts regarding inferential statistics
1. Probability 2. Sampling error
Four parts to sampling
1. Probability vs. non-probability sampling 2. Sampling errors 3. Response rates 4. Sample size
How do you collect this information? (3)
1. Prospective (e.g. birth cohort studies) or retrospective (administrative data set) 2. Longitudinal (e.g. data collected from the same sample over a repeated time) or cross sectional methods (e.g. data collected as specific point of time) 3. Surveys and secondary analysis (e.g. data collected from a previous study)
How does IRB protect subjects? (2)
1. Psychological risk 2. Confidentiality
Six Tips for Question Wording
1. Purposeful language 2. Avoid bias 3. Clarity 4. Avoid double-barreled questions 5. Frequency and time measures 6. Sensitive questions
Three keys for formatting the survey
1. Question order 2. Length of the survey 3. Demographics
What are the four parts of developing content for a survey?
1. Question writing 2. Expert review of draft questions 3. Pilot testing 4. Revisions
What are the two types of survey formats?
1. Questionnaires 2. Interviews
The square of the correlation coefficient, r^2 (2)
1. R squared 2. Values from 0.00 to 1.00
Impairment level (2)
1. ROM 2. MMT
Regression of SBP on BMI for 10 Women (continued) (2)
1. R^2=.75. Therefore, 75% of the variance in SBP can be predicted by knowing BMI 2. 25% of the variance of SBP is unexplained
Key Assumptions of Parametric Statistics
1. Random sampling 2. Variance in samples being compare are equivalent="Homogeneity of Variance (HOV) is tenable 3. Normal distribution 4. Data is interval or ratio
In true experiments, we have to check for (2)
1. Random sampling 2. Homogeneity of variance (HOV) (e.g. all comparison groups have the same variance) --Each sample has the same amount of variability
Conclusion (5)
1. Randomized controlled trial is the best way to measure the efficacy of an intervention 2. Single blind study (not controlled but can be carried out by a single researcher) 3. Crossover design usually done for rare diseases or maximize the number of potential subjects for the study 4. Repeated measures design are important to explore the long-term benefits of an intervention 5. Complex types of design require great deal of effort and huge amount of expenses to conduct
Cluster sampling (2)
1. Randomly select several large subgroups 2. Then randomly choose from the clusters
5 common measures of variability
1. Range (maximum to minimum) 2. Percentiles and Quartiles 3. Variance 4. Standard deviation 5. Coefficient of Variation
When do you sample the entire population? (3)
1. Rare disease e.g. when the population is very small? 2. Extensive resources and equipment to use 3. Very poor response rate
"Disproving" the null hypothesis (2)
1. Reject 2. Do not reject
Example of a Curvilinear Relationship (4)
1. Relationship between age and strength 2. Curvilinear pattern evident (quadratic trend) 3. Correlation can only identify linear relationship 4. Can perform trend analysis
How can a likert scale be coded?
Can be reverse coded
Recruitment (2)
1. Requires informed consent to participate, some people will say no 2. How many do you recruit for your study?
EBP: The patient's values (3)
1. Resources 2. Support system 3. Environment
Types of Observational Studies (9)
1. Retrospective 2. Longitudinal and Cross-Sectional Research 3. Correlation and Prediction 4. Cohort Studies 5. Evaluation of Causality in Observational Studies 6. Methodological Research: Reliability and Validity 7. Historical Research 8. Secondary Analysis
Check parametric assumptions (3)
1. Samples are randomly drawn from a original population with a normal distribution 2. Homogeneity of Variances (HOV) in the samples being compared are roughly equal 3. Data should be measured on the interval or ratio scales
Assumptions for Parametric Statistics (3)
1. Samples are randomly drawn from a parent population 2. Variances in the samples being compared are roughly equal 3. Data should be measured on the interval or ratio scales
Systematic Sampling (2)
1. Sampling at specified intervals (eg selecting every fifth patient coming to outpatient clinic for research study) 2. Starting point is random selection
Factors that influence Sample Representativeness (3)
1. Sampling procedure 2. Sampling size-how many? 3. Participation (response rate)
Where do we find evidence? (4)
1. Scientific peer-reviewed journals 2. Books and magazines 3. Government and professional websites (APTA) 4. Grey literature e.g. PhD theses, government documents, reports
Nonparametric Tests for Independent Samples (3)
1. Scores are ranked for all groups combined 2. Mann-Whitney U Test (two groups) 3. Kruskal-Wallis Analysis of Variance (three groups or more)
Assumptions for Correlation (6)
1. Scores represent the underlying population 2. Scores are normally distributed 3. Each subject as a score for both X and Y 4. X and Y are independent measures 5. X and Y values are observed, not controlled 6. Relationship between X and Y is linear, not curvilinear
Examples of Concealed Allocation (2)
1. Sealed envelopes 2. External service separate from the research institution
Challenges for case-control studies (5)
1. Selection bias 2. Observation bias e.g. when there is a systematic difference in the way information about the exposure is obtained 3. Recall bias 4. Confounding factor that may interfere with analytic interpretations of case control data 5. Matching cases and controls on other relevant factors, such as age or gender
Evaluating Methodologic Quality: What biases to look for? (4)
1. Selection bias in how groups were formed 2. Performance bias in how the intervention was given to each of the groups 3. Attrition bias is the loss of subjects: Did they perform an intention-to-treat analysis? 4. Detection bias if the outcomes assessed are different for different groups
Systematic review methods (4)
1. Selection of studies for review 2. Search strategy 3. Screening citations 4. Data extraction
Subjective (1)
1. Self-report outcome measures
Baseline data (2)
1. Serves as the control 2. Distinguishes Single-Subject design from case studies and experimental studies!
Multiple Regression (2)
1. Several independent variables are used to predict one dependent variable 2. R^2 gives the percentage of the total variance explained by all of the predictors in the model
Non-inferiority trial
A clinical trial designed to show that a new treatment is no better than and no worse than standard care
Four determinants of statistical power
1. Significance criterion 2. Variance 3. Sample size 4. Effect size
Social validity (2)
1. Significance of the intervention goals in the target population 2. Importance of the treatment's effects
Kendall's Tau-b (2)
1. Similar to the Spearman correlation 2. Alternate versions, tau-a and tau-c, can be used with categorical data
Random Assignment Strategies (5)
1. Simple random assignment 2. Block random assignment 3. Stratified random assignment 4. Cluster random assignment 5. Run-in period
Conclusion (5)
1. Simply to conduct and manageable 2. Less expensive to conduct 3. Generate hypothesis or research question(s) 4. Good to test the efficacy of an intervention eg pilot study 5. Be aware of their limitations not as robust as RCT
Conclusion (3)
1. Single subject design help us to understand the change in behavior (worse or better) following an intervention in a meaningful way 2. It provides us information about variability, level of change, and trend etc. 3. Generalizability is a big issue e.g. representativeness, setting and procedures
Two types of blinding
1. Single-blind study 2. Double-bline study
Single-Subject Design (3)
1. Smaller group--> "N of 1" or very small sample-->N=4 2. More measures: Observe change over ongoing treatment 3. Individual performance: Clinically-relevant EBP
Descriptive surveys (5)
1. Source of data about a specific group 2. Risk factors (e.g. smoking, drinking habits) 3. Characteristics (e.g. age, gender, socio-economic status) 4. Behavior or attitude (e.g. Exercise habits) 5. Overall picture
Functional level (2)
1. Stairs 2. Gait distance
How do we maximize reliability?
1. Standardize measurement protocols 2. Train raters 3. Calibrate and improve the instrument 4. Take multiple measurements 5. Choose a sample with a range scores 6. Pilot testing
More control for extraneous factors (3)
1. Standardize your protocol 2. Have a plan for lost or incomplete data 3. Blind the study
Statistical significance via p-values (2)
1. Statistical significance via p-values does not mean that low correlations are clinically important 2. The significant p-value just means the observed value is unlikely to be the result of chance
Use of an individual patient (2)
1. Strong evidence to that patient 2. May or may not generalize to different patients
Examples of One-Way Repeated Measures Design Over Time (2)
1. Study Sample-->Pretest-->Intervention-->Posttest 1-->Posttest 2-->Posttest 3 2. Study sample-->Pretest-->Intervention A-->Posttest 1-->Intervention B-->Posttest 2
A group of individuals followed over time (2)
1. Subjects classified by exposure status at start 2. All have potential for outcome condition
Normal curve (3)
1. Symmetrical 2. Most of the scores fall in the middle 3. Progressively fewer fall at the extremes
Measures of Central Tendency (3)
1. Symmetrical distribution 2. Positive skew 3. Negative skew
Bottom line (7)
1. T-tests compare how many means? only 2 2. More than two means compared? Don't use a t-test
Simple Random Sampling (3)
1. Table of random numbers 2. Computer selection 3. Sampling without replacement (if one draws a simple random sample such that no unit occurs more than one time in the sample e.g. labelling 1 to 1-0)
How should you go about picking the correct statistical analysis? (2)
1. Talk with a statistician or colleague who understands stats 2. Always double check
Levels of the Sampling Process (3)
1. Target population 2. Accessible Population 3. Sample
4 types of approaches to reliability testing
1. Test-retest reliability 2. Reliability 3. Alternate forms 4. Internal consistency
Example of Analysis of Covariance (5)
1. The APU faculty wants to compare two different classroom teaching strategies on clinical performance 2. Comparing the online learning group to the in-class learning group 3. One group has a higher GPA than the other and this may be affecting the regression lines 4. ANCOVA the GPA by taking both group's GPA and averaging it, then assigning the average to both groups, then adjust the regression lines 5. Effect is that GPA is out of the equation
Example of Rank Correlations: Conclusion (2)
1. The correlation between verbal and reading comprehension is strong and significant 2. Values for T (Kendall's tau) and Rs (Spearman's rho) are similar
Scales can be (2)
1. summative 2. cumulative
Intrarater reliability
Can the same rater measure consistently over multiple trials?
Effect Size (4)
1. The difference between sample means, or the estimated effect of independent variables, or strength of correlation/association between variables 2. The measure of the "degree to which the null hypothesis is false" 3. Important because statistical significance is not the same as treatment effect 4. A meaningful effect is tested by finding effect size
Comparing two means and looking for statistical significance is based on: (3)
1. The difference or degree of separation between two group means (effect size) 2. The variability within each group (error variance) 3. With two independent, randomly assigned groups
Several independent variables are used to predict one dependent variable (2)
1. The independent variables can include continuous or categorical variables 2. The dependent variables must be a continuous measure
Sources of Measurement error (3)
1. The individual taking the measures 2. The instrument taking the measurement 3. The variability of the characteristic being measured
Coefficient of Determination (3)
1. The square of the correlation coefficient, r^2 2. Indicates the proportion of the variance in X that can be explained by knowing the variance in Y 3. A measure of accuracy of the prediction of Y based on X
The Correlation Coefficient (3)
1. The strength of the relationship 2. Signs implies direction of the relationship 3. H0: population correlation=zero
Define continuous variables
Can theoretically take any value along continuum within a defined range, e.g. 1, 2, 3
Categories of PubMed Clinical Queries (5)
1. Therapy 2. Diagnosis 3. Etiology 4. Prognosis 5. Clinical prediction rule
Alternative hypothesis (2)
1. There is a difference 2. May be stated with or without direction
No control comparison (1)
1. Threats to internal validity-events occurring before and during the treatment period -History -Maturation -Testing effects
Writing a case study (steps) (11)
1. Title 2. Abstract 3. Background/Introduction 4. Literature 5. Case description 6. PICO and purpose 7. Methods 8. Evaluation, Assessment, Plan of CAre 9. Intervention 10. Results=Outcomes 11. Discussion/Conclusion
What are the six ways that a measurement is used?
1. To describe the quality or quantity of a given variable 2. To make absolute decisions 3. To make a choice on treatment approach 4. To evaluate a patient's progress 5. To discriminate between groups 6. To draw conclusions about predictive relationships between variables
Threats to External Validity (3)
1. Treatment and selection 2. Treatment and setting 3. Treatment and history
At least 2 groups in true experiment design
1. Treatment group 2. Comparison group
Pretest-posttest control group design (5)
1. Two or more groups: one treatment group, one control 2. Random assignment 3. Pretest-->intervention or placebo-->posttest 4. Scientific standard for establishing cause and effect 5. Example?
Potential errors in decision making (2)
1. Type I: mistakenly finding a difference 2. Type II: mistakenly finding no difference
Selection Criteria: Decide on (5)
1. Type of study 2. Levels of evidence (1-5) 3. Participants 4. Types of interventions 5. Types of outcome measures
Therefore we need to: (3)
1. Use outcome measures and physical performance measures in the clinic 2. Choose measures that are reliable and valid
Analysis of Covariance (2)
1. Used to minimize extraneous factors 2. Adjusts the group means by partitioning out the covariate that is creating large differences between groups
Random Numbers Table (2)
1. Used to select individuals from a numbered list 2. Can be generated by a computer
Correlation of Dichotomies: Phi coefficient (4)
1. Used when both X and Y are dichotomous 2. Can be calculated using the Pearson correlation 3. Example: A study examines the relationship between motor and verbal skill tests in 60 adults with traumatic brain injury with 1=pass, and 0=fail 4. Picture
Correlation of Dichotomies: Point Biserial Correlation Coefficient (5)
1. Used when one variable (X) is dichotomous and one variable (Y) is continuous 2. Also can be calculated with Pearson's correlation 3. Continuous scores on Y are classified in two series: those who score 0 on X and those who scored Y on X 4. Image: 5. Example: We want to know the relationship between elbow flexor spasticity in patients who have had a stroke on the right (1) or left (0)
One-ANOVA Hypothesis (5)
1. Used with one independent variable (IV) that has 3 or more levels 2. Null hypothesis 3. Alternative hypothesis 4. At least two means will differ 5. K is the number of levels of the IV
What can affect test-retest reliability? (2)
Carryover effect and testing effect
Consent elements (4)
1. Voluntary 2. Considerations for special populations 3. Free to withdraw 4. Informed consent form
Effect Size for Chi-Square (2)
1. W or PHI 2. Contingency Coefficient (C) 3. Cramer's V
Chapter 3 of Capstone Paper
Case study
Accuracy of Prediction (2)
1. When r is close to +/- 1.00 the regression line is considered a strong predictor 2. r^2 is a measure of proportion representing the total variance in Y scores that can be explained by x scores
Define questionnaires
A standardized survey, usually self-administered, that asks individuals to respond to a series of questions
Pearson-Product Moment Correlation (3)
1. X and Y continuous variables 2. Sample statistic, r 3. Population parameter, p (rho)
Consider Other Approaches (2)
1. Yates continuity correction is used when expected frequencies are too small in any one cell: Subtracting 0.5 from (O-E) for each cell 2. Fisher Exact Test
Linear Regression Equation (5)
1. a is the regression constant (Y intercept) 2. b is the regression coefficient (It is a slope of the line, which is the rate of change in Y for each on unit in X) 3. Y-hat is the predicted value of Y 4. H0: b=0 5. It means the slope of the line is horizontal, indicating no relationship between X and Y, where why Y is constant for all values of X
What are the two types of criterion validity
1. concurrent validity 2. predictive validity
Examples of Conflict of Interest
1. personal 2. commercial 3. political 4. academic 5. financial interest
What does the IRB consider? (4)
1. the scientific rigor of the proposal, 2. the competence of the investigators 3. the risk to subjects 4. the feasibility of the project
Example of precision variable
1.47
What does a z score of +-2.00 represent?
2 standard deviations, or 95.45% of the area under the curve
How many rounds of questionnaires are there in Delphi survey?
2-3 rounds
Types of Evidence for Validity
3 C's 1. Content Validity 2. Criterion-related validity 3. Construct validity
How long do you need to keep consent form safe?
7 years
What is the 95% interval called?
95% convidence interval.
What should reliability be at for clinical measures?
>0.90
Example of Percentile
A 92nd percentile on the PT licensure exam indicates the score was higher than 92% of those who took the same licensure exam
What is the normal distribution also known as?
A bell-shaped distribution or Gaussian distribution
Example of descriptive (qualitative) study
A case study eg. patient with tennis elbow
Define confidence intervals
A range of scores that likely contains the population parameter, with a certain level of confidence
Define probability sampling
A sample chosen using random selection methods
Disproportional sampling
A sample stratified on a particular variable, when the number of subjects within stratum are not proportional to the population size of the stratum
Define nonprobability sampling
A sample that was not selected using random selection
Systematic sampling
A sampling method in which persons are randomly chosen from unordered lists using a fixed sampling interval, such as every tenth person attending a PT outpatient program
Define survey
A set of questions that elicits quantitative or qualitative responses
What is a systematic review NOT?
A simple literature review, which is primarily narrative, descriptive, and does not include a systematic search of the current research literature
Define norm-referenced test
A standardized assessment designed to compare and rank individuals within a defined population
What is a z-score?
A standardized score relative to the mean
What is Q-sort used for?
Abstract ideas and theories
Diagnostic trials
Accuracy of diagnostic procedures
Example of Cumulative %
Add % for each score to all %'s below that score
Arithmetic properties of ratio scale
Addition, subtraction, multiplication, and division
Block by attribute as an independent variable example
Age
One-Group Pretest-Posttest Design: Who receives the treatment and what treatment?
All subjects receive the same treatment
Repeated measures design: what treatment is received and to who?
All subjects receive the same treatment
What is beyond the universe of content?
All theoretical context
Arithmetic Properties of Interval Scale
Allow subtraction and addition
What does the normal distribution allow for?
Allows for estimates of the population using sample data
Independent variable, X
Also known as predictor variable, explanatory variable or covariate
Dependent variable, Y
Also known as the response or outcome variable
Another name for the independent samples t-test
Also known as the unpaired t-test
Example of: 5. To discriminate between groups
Amputees as a result of gun shot vs. amputees as a result of diabetes
Define conflilct of interest
COI happens when researchers have interests that are not fully apparent and that may influence their judgements on what is published.
Why is GT relevant for development of standardization of measurement tools?
Because it provides a frame of reference for interpretation of reliability or areas for improvement in protocols
Why does sampling error occur?
Because the sample is not exactly representative of the population
Why do we need the type of information that normative studies supply?
Because they act as a basis for documenting the presence and severity of disease
What is limitations to the AB design?
Being able to control for threats to internal validity
Probability of making a Type II error
Beta
Completely randomized design
Between-subjects design -Subjects are randomly assigned to two or more individual groups
Define sampling bias
Bias that occurs when individuals who are selected for a sample overrepresent or underrepresent the underlying population characteristics
What is the content validity part of faith?
Bible, written books, movies, social media, praying together, sharing life, sharing about how Christ died as the one true sacrifice for our sins so we could know God and be filled with the Holy Spirit
Examples of nominal scales
Blood type (A, B), side of injury (Left, Right), gender (Male, Female)
Example of observed proportion not differing from proportion in expected population
Blood type in patients with venous thromboembolism
Examples of ratio scale
Body Mass Index=Weight divided by Height squared
Example of norm-referenced test
Body mass index (BMI)=BMI<20 'underweight' BMI>30 obese
Define double blind study
Both participants and researchers are unaware of group assignment
What is branching used for?
Branching is used to follow up specific answers with more detailed questions
Chapter 1 of Capstone Paper
Brief introduction to your major paper
How do you evaluate responses to assess patient's condition?
By documenting change or progress
Define inferential statistics
Decision-making process that allows us to estimate population characteristics from sample data
Where should you declare a COI if it is apparent there is one?
Declare it at the early stage of research; very important
Increase confidence by?
Decreasing precision (e.g. 99% confidence interval)
Block random assignment
Distributes subjects evenly among treatment groups within a small, even numbered subgroups or 'blocks' e.g. a group of four
Which type of validity does this purpose align with? The extent to which a test is uncorrelated with tests of distinct or contrasting constructs.
Divergent Validity
Known distribution
Divide the sample the way it is in the population
Intervention phase?
Do joint mobilizations and exercises, measure knee ROM after each of 5 sessions
What do you do when a linear regression analysis does not fit a line?
Do not jump to the assumption that there is no relationship Check for a curvilinear relationship! Polynomial regression (image) A parabolic curve with one turn
What does the statistical analysis called a power analysis depend on?
Depending on prior research, effect size, and best educated guess
Define sample
Derived from the accessible population
Quasi experimental designs: Descriptive
Describe Populations
Inclusion criteria
Describe the primary traits of the target and accessible populations that will qualify someone as a subject (subject with a respiratory disease, measured by spirometer)
What do you do to an existing variable to determine its value?
Describe the quantity of it Ex: Pain level score of 0= no pain Pain level score of 10=severe pain
Exclusion criteria
Describe those traits that would disqualify someone as a subject (healthy people with normal spirometry findings)
Descriptive research definition
Describes and classifies the sample or subject characteristics
Define developmental research
Description of developmental change and the sequencing of behaviors over time e.g. child development in the first year of life
Ordinal scale: Descriptive or True Number Values?
Descriptive
Strength of Correlations Chart
Do not quote these guidelines as absolute cutoff points!
What type of phases are in single-subject designs?
Design phases
Case report (case series) defined as
Detailed report of the symptoms, signs, diagnosis, treatment and follow-up one or more individual patients, usually describing interesting treatment options or uncommon conditions
Sample size
Determined through statistical analysis called a power analysis
Example of case control study?
Developing lung cancer and asbestos exposure 30 years ago
Methodoligical Research: Reliability and Validity
Development and testing of measuring instruments for use in research or clinical practice
The t-ratio
Difference between groups/Variability within groups
Potential confounder for historical controls?
Different characteristics between groups because of history or environment
What are the two measurements of variables?
Direct and Indirect
Conceptual Basis for Comparing Group Means: Decide
Directional or non-directional hypothesis
Define dichotomous variables
Discrete qualitative values with only two choices
Preventive trials
Evaluation of whether a procedure or agent reduces risk of developing a disease or disorder
Categories are exhaustive and mutually exclusive
Every subject can be assigned to only one appropriate category
What interventions are applied in one-group pretest-posttest design?
Everyone gets the same intervention. For example, older patients with knee problems receiving heat therapy for four weeks
What does the PICO question drive your search for?
Evidence to help you make sound clinical decisions for your patient
Exploratory research definition
Examines data for relationships
Example of a numeral's use
Example: '1' Strongly disagree to '5' Strong agree
Stratified Random Sampling
Example: DPT 1, DPT 2, DPT 3 -Proportional stratified sample subgroups based on a certain characteristic --Example: n=10% of each class --DPT 1=6, DPT 2=5, DPT 3=5
Example of Range
Example: range of test scores 98 (high)-70 (low)=28
What is an excellent response rate? And what is expected?
Excellent: 60-80% Expected: 30-60%
Examples of independent variables
Exercise therapy, ice therapy and laser therapy
What can you test a theory by looking at?
Expected relationships
What is QED similar to?
Experimental design
What does systematic review cover?
Exploratory and experimental research
Carryover effect
Exposure to multiple treatments without returning to baseline in between
Define exclusion criteria
Factors that would preclude someone from being a subject
Mistakenly finding no difference
False-negative
Type I Error and Significance: Mistakenly finding a difference
False-positive
Quasi experimental designs: Exploratory
Find relationships
Example of: 2. To make absolute decisions
Grade scale requirements to pass a class
Example of: 6. To draw conclusions about predictive relationships between variables
Grip strength predicts risk of mortality
For example:
Group 1 receiving modalities treatment (intervention A) and after washout receiving (intervention B) Exercise therapy
Example of experimental research
Group A will sleep 10 hours/night for 2 weeks vs. group B will sleep 6 hours/night and we will look at test scores over two weeks
Image of box plots
Group B more variable
Scores are ranked for all groups combined
H0: high and low ranks will be distributed in all groups
Example of adding frequency and time?
Has your low back pain limited you from performing your work more than 3 times in the past month?
Example of discrete variable
Heart rate is measured in bpm, not in a fraction of a beat
Example of directional hypothesis
Heat therapy is superior or much better compared to cold therapy in the treatment of knee problems in young adults
Independent samples t-test: assumption of
Homogeneity of variance
Example of exploratory research
Hours of sleep and exams scores
Question regarding threats to internal validity
How do you know what exactly made the difference?
Duration
How long the seizure lasts
Example of double-barreled questions
How many times a week do you jog or ride a stationary bicicyle
Diagnostic accuracy of any diagnostic procedure or a test gives us an answer to the following question:
How well this test discriminates between two conditions of interest (health. and disease)
Define logical positivism
Human experience is limited to logical and controlled relationships between specific measurable variables
What is preferred for reporting test-retest reliability?
ICCs are preferred as they take in both correlation and agreement
Literature Review (LR): ___ and ___ prior research
Identifies and analyzes
Stratified random sampling
Identifying relevant population characteristics and portioning members of a population into homogenous, non overlapping subsets, or strata, based on these characteristics
When should standard care be used as a comparison to new treatments?
If it is recognized as effective
When is a meta-analysis important?
If smaller studies are limited by too few participants to demonstrate a difference
When is repeated measures design good to use?
If the patient returns to baseline between interventions
How can case studies be valuable?
If there are no other studies to date that answer your PICO question
What can cause outliers?
If you can find a causal factor, then you can take out the outlier 1. Rater 2. Sample size-Subjects 3. Measurement 4. Instrument 5. Miscalculation
Example of skip pattern
If you chose no, skip to question 10
Face validity is a judgement by who after what?
Judgement by the users of a test after the test is developed
Acronym for the wording of constructing survey questions
KISS: Keep it simple stupid
Interval scale: ____ value
Known
Ratio scale: ____ value
Known
Goodness of Fit
One use of Chi-Square is to test Goodness of Fit to a given distribution (DF=K-1)
Single-blind study
Only the subjects, or the investigator or the measurement team is blinded to intervention allocation
Spearman's rho
Ordinal data
What does the results of clinical research affect?
Our clinical decisions and practice patterns
How is the outcome determined in a study association between exposure and the outcome of interest
Outcome determined over time
Chapter 4 of Capstone Paper
Overall discussion and conclusion as you look at both your literature and your case
Define Target Population
Overall group to which findings will be generalized E.g. Osteoarthritis (OA)
Intrinsic
Personal characteristics of the subject
Define Accessible Population
Persons who have an actual chance to be selected, who are available E.g. Patients with OA in Azusa
What do the cards have on them?
Phrases, statements, ideas, pictures
When should placebos be used
Placebos should only be used when the effectiveness or harm of the treatment is truly unknown.
PICO
Population or problem Intervention Comparison or control Outcomes
Parametric statistics estimate
Population parameters
Post HOC Analysis
Post hoc analyses are multiple comparison tests that follow a significant analysis of variance 1. Scheffe's test 2. Bonferroni's test 3. Tukey's test 4. Fisher's test
Random assignment strategies: potential benefits
Potential benefits to identify dropouts and refine the program
What may these factors due to findings?
Potentially variables that may confound the results or interfere with interpretation of the findings
Carryover effect
Practice effects: improved performance because you learned how to do the test better
Which type of validity does this purpose align with? The extent to which the target test can predict a future reference standard.
Predictive validity
Correlation and Causation
Presence of association does not imply a causal relationship
Testing Definition and Example
Pretesting or repeat testing changes the response Example: laterality recognition cards, coordination tests
What must be respected at all times?
Privacy
Define probability
Probability is the likelihood that any one event will occur, given all possible outcomes
Simple random sampling
Probability method selecting subject for a sample, where every subject in the population has an equal chance of being chosen
What are two problems with questionnaires?
Problem with misunderstanding the question and problem with the response rate
What is the central tendency for the coin rotation test?
Same for both groups
Convenience sample
Sample is pulled from the accessible population
What increases as the standard deviation of the mean decreases?
Sample size
Scores of GRADE
Score 0: No problems -1: Problem with 1 element -2: Problem with 2 elements -3: Problem with 3 or more elements
What does systematic reviews do in terms of a PICO question?
Search, appraise, summarize all pertinent evidence for all components of a PICO question
How fast can you run a coin rotation test?
Seconds
Define the behavior example
Seizures
Assess relative stability example
Seizures occur regularly vs. irregularly
Search strategy
Selection of databases and key terms used (e.g.) pain, exercise, osteoarthritis)
Example of sampling error
Selection of patients with and without the disease
What type of reporting occurs in surveys?
Self-report
Conclusion on Sensitivity
Sensitivity is the proportion of patients with the disease who test positive (+). If you test (-) on a highly sensitive test, you can rule out that you have the disease. (SnNOut=sensitivity, negative result rules out)
What does a broad search look at?
Sensitivity or the ability to identify all relevant studies
How many cases are in case studies?
Several cases
What two things should informed consent have at the end?
Signed and dated
What is Non-Equivalent Pretest-Posttest Control Group Design similar to?
Similar to pretest-posttest experimental design
Example of carryover effect
Single limb stance balance eyes open
Define Single-Subject Designs
Single-subject designs provide an alternative approach that allows us to draw conclusions about the effects of treatment based on the responses of a patient under controlled conditions
What may you use for persons to questions that are not applicable?
Skip patterns
Check Powerpoint for results
Slides 7-12
What can be added to the mix of a meta-analysis?
Smaller studies can be added and analyzed all together
Example of preventive trials
Smoking cessation program for smokers to reduce the risk of developing lung cancer
Example of a confounding factor
Smoking may lead to develop lung cancer without controlling for family history of genetics
Example of logical positivism
Some patients adhering to treatment while others not
Multiple Baseline Designs: Patient dropped?
Sometimes a patient is dropped from the study because the patient's baseline data is too different from others -Then you take a new patient with new baseline data
What are normative studies directed to?
Specific age group, gender, occupation, disability, culture
Cross-sectional
Specific point in time, looks at a cohort of developmental levels (Usually age) and describes differences between levels. It is regarded as a snap shot of the population. E.g. prevalence of depression during COVID-19 crisis
Conclusion on Specificity
Specificity is the proportion of patients without the disease who test negative (-). If you test (+) on a highly specific test, you can rule in that you have the disease. (SpPln=specificity, positive result rules in)
What does a narrow search look at?
Specificity or the ability to exclude irrelevant studies
What are wildcards used to check?
Spelling variations
Define dependability
Stability of data over time, the egree to which the study could be repeated with similar results
What measure of variability is this? ́Variability of scores around the mean Report the central tendency and the variability Usually ±1 s
Standard deviation
Control group
Standard treatment or placebo or no treatment
Another experimental group
Standard treatment/traditional care
What are standardized residuals used for?
Standardized residuals (R) are used to demonstrate the relative contribution of each cell to the overall value of chi-square
What does EBP start with?
Starts with a clinical question EG what is the best treatment for frozen shoulder?
Example of time series design
Study Sample -->1-->2-->3-->Intervention-->1-->2-->3
Test Order in One-group pretest-posttest design
Study Sample-->Pretest-->Intervention-->Posttest
What are medical subject headings? (MESH)
Subject headings developed by the National Library of Medicine
How does Q-sort work?
Subject sorts the card in rank order of 0 (low priority) to 10 (high priority)
Example of intrinsic variable
Subject talks too much
Instrumentation
Subjective outcome or physical performance measures
Randomized block design
Subjects are classified according to an attribute (blocking) variable and then randomized into groups Example: male, female
Define purposive sampling
Subjects are hand-picked by a researcher based on specific criteria
Define historical control
Subjects from previous research studies that serve as controls for experimental subjects in a subsequent study
What is considered informed consent?
Submission of questionnaire
Mean
Sum of scores divided by the number of scores
What type of trials are included in statistical hypothesis testing?
Superiority and non-inferiority trials
Example of a variable
Survey assigning DPT students Gender as a variable: '1' to represent female and '2' to represent male
Types of measurement error (2)
Systematic and random
What type of collection of these measurements?
Systematic collection of repeated measurements over time
Innapropriate use of Multiple T-Tests
T-Tests are most commonly used and most misused
What may you use to report results in analysis?
Table for reporting results
How do you go from variance to standard deviation and vice versa?
Take the square root of variance to get the standard deviation or square the standard deviation to get the variance
How do stems and leaves related in a stem and leaf plot?
Take the stem and add a leaf
Example of systematic error
Taking a measure with a device that has not been zeroes
Example of criterion validity
Taking the Thessaly test and comparing it to the 'gold standard' of an MRI for a meniscus tear for the same patients. If the correlation (r) is high, then the Thessaly test is a good predictor of meniscus tear
A-B Design
Target behavior is measured repeatedly across both the baseline and intervention phases
What is Q-sort?
Technique using cards
Examples of indirect variables
Temperature (thermometer), muscle contraction using (EMG)
Example of extrinsic variable
Temperature control of the room breaks down
Example of interval scale
Temperature; 0 degrees celsius is not the absence of heat
Sampling error
Tendency for sample values to differ from actual population values
Example of testing effect
Testing hamstring extensibility
Example of cluster sampling
Testing work life balance in DPT students -Randomly select 10 PT programs in California -Randomly select a class -Randomly select students from that class
Correct Interpretation of a 95% confidence interval
That if we were to repeat sampling many times, 95% of the time our confidence interval would contain the true population mean
What is systematic review standardized through?
The Cochrane Collaboration
The Process of Evidence Based Practice
The Five A's 1. Ask a relevant question 2. Acquire evidence to inform decisions 3. Appraise the research 4. Apply the findings to clinical practice 5. Assess the success of the process in applying the evidence to a patient.
Example of treatment and history
The effect of diet on lowering cholesterol is different on today's society than 50 years ago. Then, the focus was on diet change among non-exercisers. Today, there is more media on fitness and health so it is likely a study today will have a different sample of the population
What are measures of central tendency in reference to?
The center of the distribution
What is correlation a function of?
The co-variation of data (how much one variable varies with another
What does qualitative research describe?
The complex nature of humans and how individuals perceive their experiences within a specific social context
Define independent variable
The condition, intervention, or characteristic that will predict or cause a given outcome
Define evidence based practice
The conscientious, explicit and judicious use of current best evidence in making decisions about the care of the individual patient. It means integrating individual clinical expertise with the best available external clinical evidence from systematic research
What is important to pick correctly in analysis?
The correct statistical analysis
Partial Correlations
The correlation between X and Y, with the effect of a third variable, Z, statistically removed
Define criterion validity
The degree to which the outcome of one instrument, the target test, can be used as a substitute measure for an established reference standard criterion test. Can be tested as concurrent or predictive validity
What can the PICO question drive?
The development of the research question and clinical research
What does a z-ratio reflect?
The difference between an individual sample mean and the population mean
What does the z-score reflect?
The difference between an individual sample score and the sample mean
Define sampling error
The difference between an observed statics from a sample and the population parameter
Mode
The most frequent number
Prevalence
The number of cases of a disease at a given point in time expressed as a proportion of the population at risk
What does frequency represent in a stem and leaf plot?
The number of numbers
Purpose of an operational definition?
The operational definition defines variables as they are to be used within a study, including detail of measurements and manipulation of variables
Define dependent variable
The outcome variable... presumed to vary depending on the independent variable
Interpreting probability values
The p value is the probability of finding an effect as big as the one observed when the null hypothesis is true
Interpretation of Lumbar Pillow Study
The population means could be negative, positive or even zero. When zero is contained within a confidence interval, means are not significantly different
Define Inclusion Criteria
The primary traits of the target and accessible populations that will make someone eligible to be a participant
Define measurement
The process of assigning numerals to variables to represent quantities of characteristics according to certain rules
Example of operational definition
Ultrasound therapy for superficial knee problems (method of measurement, including tools and procedures used to obtain measurements)
What does underuse occur?
Underuse occurs when treatments or tests are not applied despite their having been shown to be effective, such as restrictions on use of MRIs that could be informative because of expense
Nonequivalent posttest-only control group design: use for
Use for exploratory purposes NOT explanatory
What does EBP involve?
Use of evidence to inform clinical practice and decisions
Define efficacy trials
Use strict inclusion and exclusion criteria to enroll a defined homogenous patient population EG adult patients with tennis elbow aged 18-25 years
Define secondary analysis
Use the same database to re-examine variables and answer questions other than what was looked at prior
A form of Chi-Square for correlated samples
Used when variables are not independent. Subjects either act as their own control or are matched
What is nonparametric statistics used with?
Usedwithnominalorordinaldata
What does the 95% confidence interval mean?
We are confident that 95% of all scores will fall there, and any score out of these bounds is significantly different
Hypothesis testing: -Much like a jury decides, guilty or not guilty
We decide if the findings are significant or not significant
Example of the measure of the "degree to which the null hypothesis is false"
We expect no difference in strength between two groups, the effect size is 0. We find a difference of 20 pounds, so the true effect size is 20
How do we measure change?
We measure with self-report outcome measures and physical performance measures
Waht do we do instead?
We take into account our sample size and standard error of the mean for our sample
Example of continuous variables
Weight measured in pounds, distance walked in meters
In analysis, what may you need to do to subscaled items?
Weight them
What should be carefully monitored of the subjects?
Welfare
Outcome
What do you want to measure that is also important to your patient?
Estimating risk
What factors (exposures) may increase or decrease the likelihood of developing a condition (outcome)? For example: cigarette smoking may lead to develop lung cancer
Question regarding reliability and experimental control
What if your patient started exercising at the gym during the intervention period?
Comparison
What is another intervention you are interested in? It can be the standard, or well-known intervention or something novel.
Example of construct validity
What is faith as a construct? What is the Holy Spirit as a construct
Patient or Problem
What is the best general description for your patient, if you had to make a general category?
Invervention
What is the intervention of interest?
Define a measure question
What type of seizure and how will you record it?
When are standardized residuals used?
When Chi-Square is significant
Question writing: when should you use existing instruments?
When applicable
What about agreement?
When do you want to know how much agreement there is between measures?
Define sampling error
When samples differ from the population
When is a paired t-test used?
When the same subjects exposed to both conditions
Non parametric statistics
When the statistical conditions do not meet the key assumptions
When are historical controls used?
When there are ethical concerns over having a control group
Example of observation bias
When there is a systematic difference in the way information about the exposure is obtained
Define concurrent validity
When two measures are taken at relatively the same time. Most often used when the target test is considered more efficient than the gold standard and therefore, can be used instead of the gold standard
Are questionnaires anonymous?
Yes
Can interviews be unstructured?
Yes
Does case reports with an exempt status require informed consent?
Yes
Are construct validities challenging to do?
Yes; they are challenging to do as the constructs are not "real" or directly observable.
Example of a population
You are working in PT treatment clinic for knee osteoarthritis (OA), the population would be all the people in California with OA
What can you do once an instrument has test-retest reliability?
You can use it to test for changes in a patient, but you have to account for carryover and testing effects in your study protocol
What is the construct validity part of faith?
You can't see it as an object but you can see it demonstrated in a person's life e.g. loving and caring, compassionate
What happens when you draw random samples enough times?
You get a normal distribution curve=sampling distribution of the means
Not enough participants?
You may have limited power and fail to see a significant difference when there really is one (type II error)
Type I Error
You say there is a significant difference but there really is not (False positive). We reject the null hypothesis, when the null hypothesis is in fact true. Eg: wrongly significant for alpha=0.05
Type II error
You say there is no significant difference but there really is (False negative). We accept the null hypothesis, when the null hypothesis is not true E.G. low sample size in the study
Example of reliability
You take your patient's heart rate 10 times. It may be different some of those times. It may be a true change. There may be random error (patient thought or fear of a dog and heart slowed, thought of test and heart sped up). All of this adds to the variance in a measure.
Define confidence interval
a "range of scores with specific boundaries that should contain the population mean"
Define plagiarism
appropriation of another person's ideas, processes, results, or words without giving appropriate credit or attribution and representing the work on one's work.
LR: Validates ___
assumptions
Skewed distribution
asymmetrical
EBP: Consideration of all of the current ____ ____
best evidence
Interrater
between raters
Random Assignment: Minimizes bias
by creating groups that are similar at the start of the trial
CR: Establishes relationships among ____ ____
clinical phenomena
Longitudinal
collecting data over an extended period of time
Drawing ______ to choose between courses of action
comparisons
Define expedited reviews
e.g. recoded data from older subject 18 years and older (deidentified), routine data in clinical practice.
Clinical Research (CR): Is a structured process of investigating ___ and ____
facts and theories
Sensitivity (another definition)
proportion of people with the disease who test (+)
Specificity (another definition)
proportion of people without the disease who test (-)
One-Group Pretest-Posttest Design: Analysis?
t-test for paired comparisons or nonparametric equivalents
Case studies generate
tests or theories
Define population
the aggregate of persons or objects that meet a specified set of criteria, and to whom we wish to generalize results of a study
When Ho is false
the treatment worked, the numerator will be larger and the denominator will be smaller a greater variance between groups, less variance within groups
Target population
the universe of all the population of interest=reference population
Intrarater
within a rater
What can you cross-tabulate in analysis?
yes/no and M/F
If you test (+) on a highly specific test,
you can rule in that you have the disease (SpPln=specificity, positive result rules in)
If you test (-) on a highly sensitive test,
you can rule out that you have the disease (SnNOut=sensitivity, negative result rules out)
Actions of Policy Misconduct (3)
• Circumventing or ignoring aspects of materials- handling requirements (e.g. biosafety, radioactive) • Circumventing or ignoring aspects of human-subjects research requirements (e.g. informed consent, ...) • Circumventing or ignoring aspects of animal-subjects research requirements
Actions of Outside Influence Misconduct
• Unauthorized use of confidential information in connection with one's own research • Not properly disclosing involvement in firms whose products are based on one's own research • Changing the results or conclusions of a study in response to pressure from funding source
Actions of Credit Misconduct
• Using another's words, data or ideas without giving due credit • Denying authorship credit to someone who has contributed substantively to a manuscript • Publishing, as original research, ones previously published data or results • Giving authorship credit to someone who has not contributed substantively to a manuscript
