Exam 1
flow of participants in an experimental study
# of volunteers--> exclude ineligible--># of eligible volunteers--> remove those who choose not to participate--> # randomly assigned--> remove drop outs and others lost to follow-up--> # that completed study--> exclude non-compliant participants--> final # included in analysis
Ways of Knowing: Authority
- accept the validity of information from a source that we judge to be an expert
uses of measurement
- describing quantity of an existing variable to determine its value - making decisions based on a criterion or standard of performance - drawing comparisons to choose between courses of action - evaluating responses to assess a patient's condition by documenting change or progress - discriminating among individuals who present different profiles related to their conditions or characteristics - predicting outcomes to draw conclusions about relationships and consider expected responses
covariates
-2 variables that 'vary' together -can be very difficult or even impossible to separate the direct effects of each on the outcome of interest
accuracy
-A description of how close a measurement is to the true value of the quantity measured -part of validity -improve by taking measurement multiple times
between-subjects design
-A research design in which different groups of participants are randomly assigned to experimental conditions or to control conditions (independent groups)
Intention to treat analysis
-An analytic strategy whereby participants are analyzed in the treatment group they were assigned regardless of whether they followed the study procedures completely -most rigorous way to look at data
repeated measures design
-An experiment using a within-groups design in which participants respond to a dependent variable more than once, after exposure to each level of the independent variable. -subjects used as their own control
pretest-posttest control group design
-An experiment using an independent-groups design in which participants are tested on the key dependent variable twice: once before and once after exposure to the independent variable -random assignment -aka parallel groups study -scientific standard for investigating cause and effect relationships
ways of knowing: tradition
-Culturally shared or widely held beliefs that people simply assume to be true -knowledge that is based on the premise that "that is just the way things are" or "that is just the way things work."
therapeutic trials
-Effect of intervention/treatment on specific disease/diagnosis -ex: drugs, surgery, PT
As treated analysis
-Patients are analyzed by the actual intervention/treatment received. If patients were in the active treatment group but did not take active treatment, the data would be analyzed as if they were in the placebo group.
performance bias
-Systematic differences between groups in terms of the actual care or treatment provided -stems from the research participant having knowledge of interventions
meta-analysis
-a procedure for statistically combining the results of many different research studies -mix of exploratory and explanatory
block randomization
-a process of randomization that first creates treatment blocks containing one random order of the conditions in the experiment; subjects are then assigned to fill each successive treatment block -even number of control and experimental -helps reduce outside factors (like weather, time of year)
discrete variable
-a quantitative variable that has either a finite number of possible values or a countable number of possible values -categories -ex: what sport you play, what year you graduated
research problem/question
-a question or issue that stimulates a response in the form of a structured scientific inquiry -must be observable, identifiable, clearly defined, and highly specific
within-subjects design
-a research design that uses each participant as his or her own control -ex: the behavior of an experimental participant before receiving treatment might be compared to his or her behavior after receiving treatment
diagnostic trials
-accuracy of diagnostic procedures
applied (clinical) research
-advances the development of new diagnostic tests, drugs, therapies and prevention strategies, answering questions with direct clinical application. -explanatory, exploratory, or descriptive
sequential clinical trials
-allows for continuous analysis of data -compares success of intervention for successive pairs of subjects -stopped as soon as the evidence is strong enough to detect a difference (may result in reduction of subjects needed) -an alternate approach to the RCT -specially constructed charts for visual analysis of outcomes
ways of knowing
-an assortment of methods used to acquire new knowledge -includes tradition, authority, experience, deductive/inductive reasoning, and scientific evidence
posttest-only control group design
-an experimental design in which the dependent variable is measured after the manipulation of the independent variable -unable to take pre-test measurements (pretest impractical, contraindicated, or potentially reactive)
translational research
-application of basic scientific findings to clinically relevant issues, and simultaneously, the generation of scientific questions based on clinical dilemmas -AKA "bench to bedside"
Levels of Evidence
-based on type of clinical question -relative hierarchy -level 1: systematic review -level 2: RCTs, observational studies with strong designs -level 3: study designs with poor control of bias (ex: retrospective cohorts) -level 4: descriptive studies (ex: case studies) -level 5: mechanistic reasoning
Polytomous variable
-can have multiple values -ex: a 5 point opinion scale
dichotomous variables
-can only take on 2 values -ex: yes or no
continuous variable
-can theoretically have any value along a continuum within a defined range -a quantitative variable that has an infinite number of possible values - a measurement -ex: inches, HR, weight in pounds
experimental research
-characterized by controlled manipulation of at least 1 independent variable by researchers -dependent variable collected under controlled conditions -statistical significance
where do research questions come from?
-clinical experience -clinical theory -professional literature (gaps, conflicts, replication)
elements of the informed consent form
-concise introduction -purpose of the research -procedures -potential risks and discomforts -potential benefits -information on study outcomes -alternatives to participation -confidentiality -compensation -contact info and request for more info -consent statement -signatures
scoping review
-conducted to discover new evidence on a subject, types of evidence available in a specific area of inquiry, or to determine missing areas in a body of literature -mix of descriptive and exploratory (main) research
Reliability
-consistency of measurement -stability over time
discussion-- Interpret
-contains interpretation of the study results and main findings -clearly states if the hypotheses were supported -explain any mechanism
types of validity
-content -criterion related -construct validity
ecological study
-correlational study at a population level -compare average disease and exposure in several populations
research hypothesis
-declarative statements that predicts the relationship between the independent and dependent variables -tested in a research study to provide evidence so the hypothesis may be accepted or rejected -null -directional vs nondirectional -simple vs complex
what variable goes on the y-axis?
-dependent variable -outcome variable -what you measure
phase 2 of clinical trials
-does the tx work? -tests on larger group of effected individuals -tests for efficacy, side effects
concealed allocation
-ensures group assignment is done without knowledge of those involved in the experimental process -minimizes bias in group formation -ex: sealed envelopes
content validity
-establishes that the multiple items that make up a questionnaire, inventory, or scale adequately sample the universe of content that defines the construct being measured
construct validity
-establishes the ability of an instrument to measure the dimensions and theoretical foundation of an abstract construct -includes convergent and divergent
criterion-related validity
-establishes the correspondence between a target test and a reference or "gold" standard measure of the same construct -includes concurrent and predictive validity
preventive trials
-evaluation of whether a procedure or agent reduces risk of developing a disease or disorder -ex: vaccine information
randomized control trial (RCT)
-examine outcomes in participants assigned to intervention or control group -gold standard for experimental research -provides the strongest evidence for determining whether or not clinical interventions work -random assignment to group -2 or more groups - pre and post measurements -ideal situations to control for confounding (strict and broad exclusion criteria, standardized treatment protocol) -NOT always the best design to answer a question (ex. effect of smoking)
explanatory research
-experimental -research concerned with identifying relationships among phenomena -looking at cause and effect -randomized controlled trials, pragmatic clinical trials, quasi-experiments, single-subject designs
research misconduct
-fabrication -falsification -plagiarism
phase 3 of clinical trials
-how does this tx compare with standard care? -tests on new and wider demographic -tests for long term effectiveness, comparisons
discussion-- Ideas
-implications for future research
characteristics of a good research question
-importance (new info) -ethical -feasible (sample size, available resources, scope, expertise)
control groups
-inactive (includes placebo, sham, and attention control group) -wait list -active
factorial designs for independent groups
-incorporates 2 or more independent variables -subjects randomly assigned to various combinations of the levels of the variables -larger samples needed -described according to the number of independent variables, number of levels within each factor
attribute variable
-independent variable -individual characteristic (ex. age, gender)
active variable
-independent variable -manipulated by researcher -assigned
What variable goes on the x-axis?
-independent variable -predictor, explanatory variables -induces or explains the change (what you think is causing the issue)
Minimal Detectable Change (MDC)
-indicates the amount of change required to exceed measurement variability -derived using a stable sample at 2 time points
stratified randomization
-individuals are grouped into strata and then randomized to one treatment group -strata may be age, level of injury, stage of disease, etc.
discussion-- Integrate
-integrate the findings of the current study in relation to previous research (supportive and non-supportive)
knowledge translation
-involves the adaptation of quality research into relevant priorities -includes the creation and application of knowledge -relates to the long-standing problem of underutilization of evidence
Phase 1 of clinical trials
-is the tx safe? -test drugs on healthy individuals -tests for safety, dosage, side effects
ways of knowing: deductive and inductive reasoning
-knowing through implying rules of inference -general to specific (deductive) -specific to general (inductive)
Ways of Knowing: Experience
-knowledge that we learn through direct experience or observation
barriers to research implementation
-lack of critical appraisal skills -access to the literature -lack of time -lack of resources -strategies to address barriers
analysis of factorial designs
-main effects of each independent variable -interaction effects -2/3 way analyses of variance commonly used (depends on number of independent variables)
blinding
-minimizes observation bias by ensuring those involved in the study are unaware of a subject's group assignments -aka masking
Pragmatic Trials (PCT)
-more reflective of practice situations -diverse patient populations with fewer exclusion criteria -active controls -treatment reflects practice and is less standardized
variables
-must be clearly defined and able to measure -may take on two or more values -may be continuous or discrete -independent, dependent, or control
non-inferiority trials
-newer interventions is "not worse" than usual care, can be used as an acceptable substitute -must specify margin
open-label trials
-no blinding of researchers or participants due to logistic or ethical reasons -both researchers and participants are aware of testing variables -may lead to bias (detection vs. perfomance)
generalizability theory
-not all error is random -some error components can be attributed to other sources, such as rater or test occasion
level of measurement: interval
-numbers have equal intervals, but no true zero -ex: calendar years, degrees Celsius or Fahrenheit
Levels of Measurement: Ordinal
-numbers indicate rank order -ex: MMT, function, pain
level of measurement: ratio
-numbers represent units with equal intervals, measured from true zero -ex: distance, age, time, decibels, weight
Levels of Measurement: Nominal
-numerals are category labels -ex: gender, blood type, diagnosis
Exploratory Research
-observational -Research conducted to gather more information about a problem or to make a tentative hypothesis more specific -find relationships -cohort studies, case-control studies, correlational and predictive research (diagnosis, prognosis), methodological research
per protocol analysis
-only subjects in a study who completed the study as per the protocol are analyzed
primary literature
-original research and/or new scientific discoveries -immediate results of research activities -often includes analysis of data collected in the field or laboratory -
crossover design
-participants are randomized to a treatment sequenced (intervention--> control OR control--> intervention) -used to control for order effects -should only be used when condition is stable -considerations for washout period
potential sources of measurement error
-person taking the measurements -the measuring instrument -variability in the characteristic
control variables
-plays a role in the explanatory variable, wish to omit the impact/effects of -used to test the possibility that an observed relationship between 2 variables has not been caused by the influence of another explanatory variable
discussion-- Identify
-point out study strengths -critique study limitations/ weaknesses
effects of repeated measures
-practice effects -carryover effects -order effects
Phases of clinical trials
-preclinical phase -phase 1 -phase 2 -phase 3 -phase 4 -early phases more drug related
nondirectional hypothesis
-predicts that the independent variable will have an effect on the dependent variable, but the direction of the effect is not specified -states there will be A difference
random assignment
-process of assigning subjects to groups -participants have an equal chance of being assigned to any group -helps spread out characteristics like age, weight, disease -minimizes bias by creating groups that are similar at the start of the trial -NOT random selection*
descriptive research
-qualitative -research methods that involve observing behavior to DESCRIBE that BEHAVIOR objectively and systematically -includes developmental research, normative research, descriptive surveys, case reports, historical research, and qualitative research
Random consent design
-random assignment strategies -Potential subjects are assigned randomly to groups before seeking consent to participate -Only those assigned to the experimental group are approached for consent
run-in period
-random assignment strategy -A period before a clinical trial is started when no treatment is given (placebo run-in period). -role in screening out ineligible or non-compliant participants, in ensuring that participants are in a stable condition, and in providing baseline observations
patient preference
-random assignment strategy -ask people if they are ok being in the control or experimental -may help compliance
strong internal validity design
-randomization controls for potential biases -threats not controlled for are attrition and differential social threats
research integrity
-research misconduct -conflicts of interest -peer review -retractions
how to develop research question rationale?
-review of literature -theoretical framework
tertiary literature
-summaries or condensed versions of materials -usually with references to primary or secondary sources -good place to look up facts or get a general overview of a subject
systematic review
-summarized findings from multiple studies of a specific clinical practice question or topic that recommend practice changes and future directions for research; one of the strongest sources of evidence for evidence-based practice -mix of exploratory (main) and explanatory
secondary literature
-summarizes and synthesizes primary literature -usually broader and less current than primary research
types of measurement error
-systematic -random
detection bias
-systematic differences between groups in how outcomes are determined -stems from researcher
validity
-the extent to which a test measures or predicts what it is supposed to -how well the results among the study participants represent true findings among similar individuals outside the study -accuracy
concurrent validity
-the extent to which the target test correlates with a reference standard taken at relatively the same time
Evidence Hierarchy
-the ranking of evidence sources according to the strength of the evidence -Top to bottom: systematic revies, RCTs, cohort studies, case-control studies, case series/reports, editorial/ expert opinion
Minimal Clinically Important Difference (MCID)
-the smallest amount of change required to produce clinically meaningful change -best estimated in a changing sample over time
types of clinical trials
-therapeutic trials -diagnostic trials -preventive trials
Structure of a written report- IMRAD
-title and abstract -Introduction -Methods -Results -Discussion -conclusion -references -supplementary materials -additional info
multiple factor experimental designs
-used to determine the effects of the interaction between multiple factors
elements of consent
-voluntary -considerations for special populations -free to withdraw -informed consent form
Phase 4 of clinical trials
-what else do we need to know? -continues to test for effectiveness and safety
discussion-- Implications
-who cares? -are there clinical or policy ramifications?
process for developing a research question
1) identify the problem 2) identify the rationale 3) identify the type of research
Clinical Research
A structured process of investigating facts and theories and exploring connections with the purpose of improving individual and public health
Which of the following is a true statement regarding reliability? A) A reliable examiner is one who assigns consistent scores to a patient's unchanging behavior. B) A reliable examiner is one who always assigns the same score no matter what the actual value is. C) A reliable instrument is one that does not perform with a predictable accuracy. D) A reliable instrument never provides an inaccurate measurement.
A) A reliable examiner is one who assigns consistent scores to a patient's unchanging behavior.
Random assignment is used in clinical trials in order to? A) Create groups that are similar at the start of the treatment period B) Ensure everyone in the accessible population has an equal chance of being selected to enter the study C) Maximize external validity of the study D) Ensure that the participants are blind to the independent variable
A) Create groups that are similar at the start of the treatment period
One advantage of a sequential clinical trial is? A) Decisions about effectiveness may be made before all the subjects are tested B) They are exempt from IRB approval C) All the data is analyzed at once at the end of the trial D) They require complex statistical testing
A) Decisions about effectiveness may be made before all the subjects are tested
A researcher wants to examine the effect of stretching exercise versus no stretching exercise on vertical jump height in college athletes. Participants were randomly assigned to the stretching or no stretching groups. The stretching group performed stretches of the lower extremity muscles and the no stretching group spent sat quietly for an equal amount of time as the other group. Vertical jump height was measured in centimeters for both groups before and immediately after the intervention period. This study is designed as a? A) Pretest-posttest control group design B) Multiple group control group design C) One-group pretest-posttest design D) Repeated measures design
A) Pretest-posttest control group design
Evidence-based practice includes all of the following, except? A) A general internet search B) Clinical expertise C) Patient values D) Clinical circumstances
A) a general internet search
The discussion section of an article should contain all of the following except: A) Conflicts of interest for authors. B) An overview of all findings. C) Limitations of the study. D) A review of literature to show supporting or conflicting findings.
A) conflicts of interest for authors
Manual muscle testing (MMT) reported on a 0 (zero) to 5 (normal) scale is an example of which scale of measurement? A) Ordinal B) Ratio C) Nominal D) Interval
A) ordinal
quasi-experimental design
An experiment that does not include either random assignment or a control group
research process step 4
Analyze data
Which of the following is the first step in developing a research question? A) Identifying the statistical test B) Identifying the problem C) Identifying the type of research D) Identifying the rationale
B) Identifying the problem
A physical therapist participates in a research study to determine the effect of noise level on the ability to perform a physical skill in this study, noise is what type of variable? A) Extraneous B) Independent C) Dependent D) Criterion
B) Independent
According to classical measurement theory which of the following statements is true regarding measurement error? A) If the same measurement is obtained three times in a row then there is no error B) It is equal to the difference between the true score and the observed score C) An observed score has no error associated with it D) Observed scores always overestimate the true score
B) It is equal to the difference between the true score and the observed score
An open access journal is characterized by all of the following except: A) It may provide immediate publication of an article for a fee. B) It will always require full peer review. C) It provides free full-text access to all. D) It may require authors to pay for a fee for publication.
B) It will always require full peer review.
A purpose of an experimental designs is? A) To describe populations B) To calculate the accuracy of diagnoses C) To determine cause and effect relationships D) To find predictive relationships between variables
C) To determine cause and effect relationships
A novice clinician uses a treatment technique that they learned from a continuing education course. This best describes which way of knowing? A) Experience B) Evidence-based practice C) Authority D) Tradition
C) authority
What type of study is used to test the hypothesis that a new treatment is at least as good as the current standard of care? A) Superiority trial B) Bioequivalence trial C) Non-inferiority trial D) Equivalence trial
C) non-inferiority trial
People with hip osteoarthritis are randomly assigned in a multigroup randomized trial to either an aquatic exercise group, a cycling exercise group, a strength training group or to a control group to assess the effect of the training condition on pain) Based on this description, this study has how many independent variables? A) Three B) Two C) One D) Four
C) one
case-control study
Compare exposure histories in people with disease (cases) and people without diseases (controls)
cohort study
Compare rates of new (incident) disease in people with different exposure histories or follow a population forward (prospective) or backward (retrospective) in time
piCo
Comparison or Control
A study investigated the effect of resistance exercise compared to flexibility exercise to improve activity for persons with knee pain. Participants are assigned randomly, by a masked research assistant, to one of the two exercise intervention groups. The participants are recruited from a group of volunteers and provide informed consent to participate in the study. Activity is measured at baseline and after the study by the six-minute walk test. In this study, the dependent variable is: A) The number of months since the participants were diagnosed with knee pain B) The informed consent form C) The type of exercise (resistance or flexibility) D) Activity as measured by six-minute walk test
D) Activity as measured by six-minute walk test
While reading the Methods section of a research report, a physical therapist notes the investigators used a repeated measures design. This type of experimental design is used to accomplish which of the following outcomes? A) Ensures that subjects with similar characteristics are assigned to different treatment groups. B) Selects a homogenous groups of subjects C) Keeps the subjects "blind" to the identity of the treatment group. D) Controls for the differences between subjects
D) Controls for the differences between subjects
Predictive validity is an example of which type of validity? A) Construct validity B) Face validity C) Content validity D) Criterion-related validity
D) Criterion-related validity
The use of an authorized surrogate decision maker who is committed to the well-being of a person who may have diminished capacity to provide consent best illustrates which of the following ethical principles? A) Justice B) Beneficence C) Respect for persons D) Non-maleficence
D) Respect for persons
In developing a clinical research study, what type of research design should be selected to look for a relationship in how clinical phenomena interact and does not involve manipulation of the variables? A) Explanatory B) Methodological C) Descriptive D) Exploratory
D) exploratory
the five 'I's of a good discussion
Integrate, interpret, implications, identify, ideas
pIco
Intervention (or independent predictor/ exposure)
Multidisciplinary research
Involves researchers from multiple disciplines independently doing research
picO
Outcomes
Pico
Population or Problem
complex hypothesis
a hypothesis describing the relationships among three or more variables
directional hypothesis
a prediction made by a researcher regarding a positive or negative change, relationship, or difference between two variables of a population
null hypothesis
a statement or idea that can be falsified, or proved wrong
equivalence trials
bioequivalence of treatment effect (mostly drug studies)
double blind
both participants and researchers are unaware of group assignment
respect for persons
broad concept that emphasizes voluntariness and autonomy
change in between MCD and MCID
change is 'real' but not likely to be meaningful to patient
Intradisciplinary research
collaboration between two or more disciplines
cross-sectional/ observational study
describe exposure and/or disease in a population
case series
describes a group of individuals with a disease
case study
describes an individual with a disease
Research process step 2
design the study
basic research
directed toward the acquisition of new knowledge
research process step 5
disseminate findings
change before the MDC
due to anticipated measurement error and variability
simple randomization
each individual is randomized to one treatment group
observed score
equals the difference of the true score and the error component
experimental study
examine outcome after an intervention
what happens when you EXCLUDE too many volunteers?
experiment not real world applicable
what happens when you INCLUDE too many volunteers?
external factors affect experiment
non-inferiority margin
how much difference between new and usual care would indicate "no worse" outcome
Research Process Step 1
identify the research question
research process step 3
implement the study
Parametric statistics
inferential statistical tests involving interval- or ratio-level data to make inferences about the population
nonparametric statistics
inferential statistics involving nominal- or ordinal-level data to make inferences about the population
superiority trials
newer information better than usual care
Levels of Measurement
nominal, ordinal, interval, ratio
threats to external validity
potential interaction of treatment and testing
simple hypothesis
predicts the relationship between a single dependent variable and a single independent variable
Interdisciplinary research
research within one discipline
transdisciplinary research
researchers creating new theories and methods that integrate knowledge/ goals/ philosophies/ perspectives from multiple disciplines
distributive justice
seeks to ensure that the benefits and burdens of research are equitable
Nonmaleficence
study should do no harm
convergent validity
the extent to which a test correlates with other tests of closely related constructs
divergent validity
the extent to which a test is uncorrelated with tests of distinct or contrasting constructs
predictive validity
the extent to which the target test can predict a future reference standard
beneficence
the study should do good
evidence based practice
use of best research evidence in conjunction with clinical expertise, patient values, and clinical circumstances to inform clinical decisions