EBPT II - Rest of Midterm Material

Compensatory rivalry

"We'll show them" attitude Exists when the study group not receiving the experimental treatment (i.e., intervention) feels disadvantaged, disappointed, or left out and decides to obtain a similar intervention on its own. It may or may not involve intentional competition or rivalry with the experimental group

Resentful Demoralization

"We're getting the short end of the stick so why bother" attitude This is almost the opposite of compensatory rivalry. Here, students in the comparison group know what the program group is getting. But here, instead of developing a rivalry, they get discouraged or angry and they give up (sometimes referred to as the "screw you" effect!). Unlike the previous two threats, this one is likely to exaggerate posttest differences between groups, making your program look even more effective than it actually is.

Threats to internal and external validity

- Are the investigator's conclusions correct? -Are the changes in the independent variable indeed responsible for the observed variation in the dependent variable? -Might the variation in the dependent variable be attributable to other causes?

Sources of Evidence for Secondary Literature

- Comprises the upper three levels of the pyramid (syntheses, synopsis, and information systems) - Syntheses (evidence-based reviews)- publications in which other authors have searched the literature on the topic in question and have appraised the retrieved articles - Synopses- consist of current best evidence combined with clinical expertise from an expert in the area

CAT Step 3: Appraise the Literature

- The retrieved articles are evaluated for their internal validity and external generalizability as well as their level of evidence - How close is the PICO of the study compared to the PICO of the clinical question and can the study assist with the final decision? - Methods section is reviewed to assess the validity of the study - Generalizability needs to be assessed - Is the population studied relevant to your PICO question?

CAT Step 1: Ask and Answerable Question

- What is the gap in the literature? - Take a clinical question and change its format so the literature search is based on this question -The question needs to be important to the following: •The patients' well-being •The clinicians' knowledge of needs •Of interest to the patient, clinician or learners •Likely to occur in clinical practice •Answerable in the time available

STAndards for the Reporting of Diagnostic accuracy studies (STARD)

-The statement consists of a checklist of 25 items and a flow diagram authors can use to improve the quality of reporting of studies of diagnostic accuracy -Articles following the STARD standards allow for a thorough review for potential bias in the results

Authors of a cervical spine manipulation study noted a statistically significant difference in quality of life scores between the intervention and control groups at baseline. What potential threat to research (internal) validity does this difference pose? A. Assignment B. Instrumentation C. Maturation D. Compensatory rivalry

A Assignment

Visual Analogue Scale (VAS)

A VAS is one of the simplest methods to assess the intensity of a subjective experience. A line is drawn, usually fixed at 100mm in length with word anchors on either end that represent extremes of the characteristic. KNOW: Scores obtained with VAS are generally treated as ratio level data

Delphi Survey

A panel of experts is asked to complete a series of questionnaires to identify their opinions There are several rounds of questionnaires (typically 2 or 3) Has great potential for planning and problem solving for a variety of practice issues Anonymity offered by this method will encourage honest responses from the panel

Functional Independence Measure (FIM)

A popular instrument for assessing function in rehab settings. The FIM is an 18-item scale designed to evaluate the amount of assistance a patient needs to accomplish activities of daily living (ADLs) The items measure both motor and cognitive functions Each item is scored on an ordinal scale from 1 (total assist) to 7 (total independence). *The larger/higher the total score, the less assistance the patient requires KNOW: Patient 1 is the most independent/more functional

Summative vs. Cumulative Scales

A summative scale is one that presents a total score with all items contributing equal weight to the total. **assessing one parameter - (ex berg balance only assesses balance) A cumulative scale demonstrates an accumulated characteristic, with each item representing an increasing amount of the attribute being measured

Likert Scale (how much you like something)

A summative scale, most often used to assess attitudes of values For each statement given below, please indicate whether you strongly agree (SA), agree (A), are neutral (N), disagree (D), or strongly disagree (SD) - SEE PICTURE

Review Questions

A survey is being developed to assess nurses' opinions and attitudes towards research. The following questions are being considered for the survey. Describe the problems with each of these questions and suggest how they could be improved. Do you do research? Yes____ No_____ (you could put this into the cumulative model ie. If yes go to question 2, if no go to question 3) How often to you read research journals? a)Seldom b)Often c)Frequently d)Every day **This needs more defining parameters. (i.e how often do you read research journals during your office hours)

A patient complains of numbness and tingling that travels from his wrist down to his thumb and index finger at rest. As a clinician you suspect carpal tunnel syndrome so during your initial evaluation, you include the Tinel sign provocative test. The Tinel sign special test has a -likelihood ratio of 0.3. If the patient tests negative for the Tinel sign provocative test, there is a good chance the patient _________have carpal tunnel syndrome due to this special test having a high_________? A) Does not; Sensitivity B) Does; Sensitivity C) Does not; Specificity D) Does; Specificity

A) Does not; Sensitivity Rationale: A provocative test that has a -LR less than or equal to 0.1 to 0.2 has a high sensitivity, testing negative means there is a high probability the patient doesn't have this condition. SnOUT: which helps to rule out a health condition.

Which of the following best represents the purpose of the STARD? A) Improve the quality of articles reporting on studies of diagnostic accuracy B) Create standards for diagnostic tools C) Ensuring diagnostic tools are valid D) Disprove the "gold standard" diagnostic tool that already exists

A) Improve the quality of articles reporting on studies of diagnostic accuracy Rationale: STARD stands for "Standards for the Reporting of Diagnostic accuracy studies." It is a checklist and flow diagram that can be used to improve the quality of articles reporting on studies of diagnostic accuracy. Using this allows for a more thorough review for potential bias in the results.

Tsa scanners at the airport have the ability to detect and ping potentially dangerous items. However, it will also ping several harmless items such as belts and jewelry. With regard to detecting dangerous items these scanners can be said to have_____. A) Low Specificity B) High Specificity C) Low Sensitivity D) No Sensitivity

A) Low Specificity Rationale: The scanner will most likely ping a harmless item than an actual harmful item. There will be many false positives. Therefore the scanner has a high sensitivity for detecting items, but a low specificity for harmful items.

A researcher is creating a new supplement that claims to enhance the attention span and alertness of young adult and adult students. She wants to test this drug to determine how effective it is by having half of the cohort take the actual encapsulated supplement while the other half takes a sugar pill. Which of the following best describes the type of research design that is being used for this study? A: Randomized Control Trial B: Quasi Experimental C: Single System Design D: Non-experimental

A: Randomized Control Trial Rationale: RCTs are used as an experimental design that randomly assigns participants into an experimental group or control group to reduce bias and to test the effectiveness of specific treatments or newer testing measures

Q-Sort

An analytic technique used to characterize attitude, opinions, or judgments of individuals through a process of comparative rank ordering The technique involves presenting an individual with a set of cards containing a series of written items such as statements, ideas, phrases, or pictures The individual is asked to sort the cards into piles according to some scaled criterion

Scales

An ordered system based on a series of questions or items that provide an overall rating that represents the degree to which a respondent possess a particular attitude, value, or characteristic Scales have been developed to measure attitudes, function, health and quality of life, pain, exertion and other physical, physiological and psychological variables

How Do I Actually Write a CAT?

Ask, Search, Appraise, Apply, and Evaluate 1.Asking a focused and answerable question that translates uncertainty to an answerable question 2.Searching for the best available evidence 3.Critically appraising the evidence for validity and clinical relevance 4.Applying the results to clinical practice 5.Evaluation of performance

For post-cardiac patients (1) in a critical care unit, does early mobilization (2) as compared to conservative treatment (3) improve hospital length of stay (4)? Identify the "I" component in PICO A. 1 B. 2 C. 3 D. 4

B. 2 Rationale: 1 is the population, 2 is the intervention, 3 comparisons, 4 is the outcome

Your patient presents with Googenberry disease and a glucose level of 50mg/dL. You find in an article that 90% of patients with Googenberry A disease have glucose levels in the same range as your patient (sensitivity) and that 15% of patients with Googenberry B disease have glucose levels in the same range as your patient (1-specificity). Determine the likelihood ratio for a positive test result in this scenario. A) A positive test result is 0.16 times as likely to be seen in someone with Googenberry A disease, as opposed to someone with Googenberry B disease. B) A positive test result is 0.16 times as likely to be seen in someone with Googenberry B disease, as opposed to someone with Googenberry A disease. C) A positive test result is 6 times as likely to be seen in someone with Googenberry A disease, as opposed to someone with Googenberry B disease. D) A positive test result is 6 times as likely to be seen in someone with Googenberry B disease, as opposed to someone with Googenberry A disease.

C) A positive test result is 6 times as likely to be seen in someone with Googenberry A disease, as opposed to someone with Googenberry B disease. Rationale: An LR+ is the probability that a person with the disease tested positive for the disease (true positive) divided by the probability that a person without the disease tested positive for the disease (false positive) and is calculated using the formula: LR+ = (sensitivity) / (1-specificity).

Which of the following best defines external validity? A) the extent to which the subjects know how the experiment will Impact themselves B) the extent to which the independent variable impacts the dependent variable C) the extent to which the results of a study can be generalized to the world at large D) the extent to which a study correctly measures what is being tested

C) the extent to which the results of a study can be generalized to the world at large Rationale: External validity refers to how well the outcome of study can be applied to other settings, people, situations, and time periods. Internal validity is how well a study establishes a cause and effect relationship between a treatment and outcome.

Which of the following is NOT a good question to ask to interpret if the results of the study are valid? A)What are the limitations of the current study? B)Did the study include between subject factors and/or within subject factors? C)Were the participants that we used brought into the study from the accessible population? D)What type of bias may have been generated by our recruitment process?

C)Were the participants that we used brought into the study from the accessible population? Rationale → Knowing if the participants from the accessible population doesn't show if they were brought into the study in a truly randomized fashion. Although it is likely that the participants did come from the accessible population, this may not have a huge impact on the results of the study.

Authors of a study investigating a new topical gel to facilitate chronic wound debridement indicate the potential of a maturation threat in their subjects. Which of the following statements is consistent with this type of threat? A. "A change in wound care nurse staffing occurred mid-way through the study." B. "Older subjects were disproportionately represented in the sample." C. "Some subjects may have improved due to the natural healing process." D. "Younger subjects were disproportionately represented in the control group."

C. "Some subjects may have improved due to the natural healing process."

Which of the following is NOT true regarding the levels of evidence according to type of research? A. It ranks poor-quality cohort studies above expert opinion B. It ranks systematic review as the highest level of evidence C. It classifies randomized control trials as level II evidence D. There are five total levels of evidence

C. It classifies randomized control trials as level II evidence Rationale: Randomized control trials are classified as level I evidence (level Ib)

A 25 year-old graduate student working on her thesis is searching for pre-appraised evidence. She decides to pull information from a systematic review, which level of the 6S hierarchy would this fall under? A. Studies B. Summaries C. Syntheses D. Synopses of Syntheses

C. Syntheses Rationale: A systematic review, e.g. Cochrane Library, essentially is a collection of articles "synthesized" into one.

A CPI in a neuro PT clinic assigns his students to read 3 research articles on new PT interventions used for post-stroke rehabilitation. He asks them to critically appraise the articles and write a report over the findings from the articles. What is this report considered? A. Synopsis B. Literature review C. Syntheses D. B and C

C. Syntheses Rationale: synopsis consists of best evidence combined with clinical expertise from an expert in the area. This excludes option C. Syntheses is where authors have searched the literature on the topic in question and have appraised the retrieved the articles, so C would be the correct answer.

Types of Scales

Categorical scales are based on nominal measurement How is the data expressed? Counts or percentages Continuous scales may be measured using interval or ratio values such as age, BP, or years of experience. (Intervals measured on the continuous scales) Scales are created so that a summary score can be obtained from a series of items *Structure scales on one dimension

Testing Threats related to study logistics

Change in the outcome that occurs as a result of a subject's increased familiarity with the testing procedure , or as a result of inappropriate cues provided by the tester Solution: Provide subjects with practice sessions Have testers use specific protocols, including scripts

Compensatory Rivalry or Resentful Demoralization Threat

Changes in behavior that occur as a result of subjects learning they are members of the control or comparison group Solution: Keep subjects separated Mask subjects to prevent knowledge of group assignment Ask all study participants to avoid behavior changes during the study time frame

Diffusion or Imitation of Treatments Threat

Changes in behavior that occur in the control or comparison group as a result of communication with the subjects in the experimental group about the interventions they are receiving Solution: Keep subjects separated Mask the subjects to prevent knowledge of group assignment Ask all study participants to avoid behavior changes during the study time frame

Research validity

Characterizes the extent to which a quantitative study of any type produces truthful results

Analysis of Survey Data

Collate responses and enter them into a computer Sort through responses and determine inclusion and exclusion Responses to closed-ended questions are coded Look at all parameters of the survey and assess accordingly

History Threats related to study logistics

Concurrent events occurring outside of the study that influence the study's outcome Solution: Randomize the assignment to groups Time the study to avoid the event(s) Perform statistical adjustment

Majority of Questions are Foreground Questions

Consist of the following components: •Patient's problem of interest •The main intervention (diagnostic test or treatment) that is going to be compared with the existing reference standard •The comparison intervention (diagnostic test or treatment) that is already identified •Outcome of interest

Positive predictive value (PPV) is defined as "the probability that someone with a positive test actually has the disease". Therefore, a test that has a high PPV can also be considered to be: A. Primary Literature B. Secondary Literature C. Specific D. Sensitive

D. Sensitive Rationale: Sensitive, because a test is considered sensitive if it is capable of correctly identifying somebody as having a disease with a positive test result.

"Believability": Quantitative Studies

Does the study focus on what the investigator wants to know? Does the study prevent unwanted influences from contaminating the study's outcomes? Can an alternative explanation for this study's results be identified? If the answer is yes... What are specific problems with the research design elements? What unwanted influences are introduced into the study as a result of these design problems? What alternative explanation(s) of the study's results are possible due to the introduction of these unwanted influences?

A study shows that exercise can decrease the mortality rate of a patient that is post heart transplant. If this study can be applied to a broader range of patients such as patients with heart disease, this study has a good what? A. Generalizability B. Internal Validity C. External Validity D. Reliability E. Both A and C F. All of the above

E. Both A and C Rationale: Generalizability are the results that apply to a broad range of patients in addition to those in the study. External validity is when the results can be applied to the world at large. If the study shows that exercise can decrease mortality in not only the post-transplant patients then it will have both generalizability and external validity.

Questionnaire Outline

Each designed item should relate back to at least one of the study's objectives Demographic information and its importance Guiding questions Preliminary draft

Double-Barreled Questions - Multiple questions within a question

Each question should be confined to a single idea. Surveys should avoid the use of double-barreled questions, using "or" or "and" to assess two things within a single question. Example: How many times a week do you jog or ride a stationary bicycle? AVOID THESE AT ALL COSTS ON SURVEYS

Frequency and Time Measures

Examples: How many alcoholic drinks do you consume each day? How many patients do you treat each day? Why may these questions be difficult to answer? The variables involved What would be a better way to phrase the questions above? You should provide a time frame for reference

A quantitative study's relevance is referred to by the term _____________

External validity

Generalizability vs Validity

Generalizability (applicability) means that the results apply (or are likely to apply) to the broad range of patients in addition to those in the study population Validity is the quality of being logically or factually sound

Informed Consent

Goes before the Institutional Review Board (IRB) Researchers must be able to demonstrate the protection of subjects from psychological risk and the guarantee of confidentiality Individuals who participate in face-to-face interviews can be given an informed consent form to sign in the presence of an interviewer and a witness

Threats Related to Study Logistics

History Instrumentation Testing

Threats to Research Validity

IMPORTANT The design problems that introduce unwanted influences into a quantitative study Imagine a study about school-based PT for children with spastic diplegia that is investigating the comparative effectiveness of task-specific functional training vs. a neurodevelopmental treatment (NDT) approach for normalization of gait patterns. 1.The two approaches are equally effective 2.The functional training approach is more effective than the NDT approach 3.The NDT approach is more effective than the functional training approach 4.Neither approach improves the subjects' gait patterns

Interviews

In an interview the researcher asks respondents scientific questions and records their answers for later analysis What interview setting do you anticipate will establish the best interviewer/interviewee rapport? Structured interview vs. unstructured interview

Instrumentation Threats related to study logistics

Inappropriate selection, application or function of techniques/instruments used to collect data in a study Solution: Select appropriate technique or instrument Ensure user training and practice Have testers use specific protocols Calibrate instruments

Hypotheses

Incorporated in addition to the guiding questions Important to direct statistical analyses and conclusions Are perceptions of career success different for male vs female PTs? Hypotheses examples: Men and women will differ with perceptions to career success. a)Men will rate themselves higher in career success compared to women. b)Men will report greater importance of salary and position compared with women

Believability is referred to as ___________

Internal validity

Internal Validity vs. External Validity

Internal validity - The degree to which the results are attributable to the independent variable and not some other rival explanation. Is the cause producing the effect? Is it trustworthy, is this the only cause? External validity - the extent to which the results of a study can be generalized

Threats to Research Validity in Quantitative Studies About Other Components of the Patient/Client Management Model:

Investigator Bias Narrow selection criteria Purposive Sampling Testing with prior knowledge of patient status

Measurement Criteria

Item difficulty- the position of items within the hierarchical scale Item fit- the extent to which the individual items conform to the uni-dimensional model. Item separation- reflects the spread of items, and person separation represents the spread of individuals

Attrition Threat

Loss of subjects resulting in reduction of sample size and/or inequality of group baseline characteristics Solution: Replace subjects, if appropriate and feasible Perform statistical adjustment

Describe the purpose of performing a Rasch analysis on a summative scale

Makes sure your survey design is of a hierarchical order (ex. Eating --> walking indoors --> climbing stairs)

Diffusion or Imitation of Treatment

May occur when subjects interact Either purposefully or unintentionally, these individuals may share aspects of the tx in their group that prompts changes in behaviors by members in a different group **Can lead to resentful demoralization or compensatory rivalry

Maturation Threat

Natural change in human behavior or function over the course of time that may influence the study's outcome Solution: Randomize the assignment to groups Time the study to minimize the effect Randomize the testing or treatment order

Selection Threat

Nonrepresentative sample that produces inaccurate results relative to the population of interest Solution: Recruit subjects from multiple locations Randomize the subject selection process

Statistical Regression to the Mean

Occurs when subjects enter a study with an extreme value for their baseline measure of the outcome of interest KNOW: Solutions include eliminating outliers from the baseline scores and take repeated baseline measures and average them to reduce the extremes

Compensatory Equalization of Treatments

Occurs when the individuals providing the interventions in the study purposefully or inadvertently supplement the activities of subjects in the control group to make up for what subjects in the experimental group are receiving. KNOW: Solution - mask the investigator and/or provide a clear and explicit protocol for intervention and administration including a script for instructions if needed

Study Relevance

Once a study's "credibility" has been evaluated, PTs must determine whether the findings are relevant to their situation. External validity is the term used to describe the usefulness of a quantitative study with respect to the "real world." Transferability is the analogous term for qualitative research designs.

Constructing Survey Questions

Open-ended questions ask respondents to answer in their own words Closed-ended questions provide multiple response choices (designed to include n/a or other option, don't know - should be exhaustive) What are some potential confounds of asking, "What types of exercise do you do regularly?" Not a good guided question because its too broad

Sources of Evidence for Primary Literature

Original articles (RCTs, cohort, case-control, cross-sectional studies)

Positive Predictive Value (PPV) vs. Negative Predictive Value (NPV)

PPV- the probability that someone with a positive test actually has the disease NPV- the probability that someone with a negative test does not have the disease

Compensatory Equalization of Treatments Threat related to investigators

Purposeful or inadvertent supplementation of the control or comparison group's activities that influences these subjects' performance Solution: Mask the investigators to prevent knowledge of group assignment Implement and enforce protocols for the interventions Ask all study participants to avoid discussing their activities with anyone

Believability: Qualitative Studies

Quantitative research designs assume that there is one objective truth that can be captured through properly designed and implemented methods The concept of believability appears inconsistent with the paradigm upon which qualitative research designs are based Triangulation- a method to confirm a concept or perspective generated through the qualitative research Reflexivity- a principle of qualitative designs that suggests researchers should reflect on and continually test their assumptions in each phase of the study.

Semantic Differential

Research has demonstrated that the adjective pairs used in this scale tend to fall along three underlying dimension which have been labeled evaluation, potency and activity Evaluation- good vs bad Potency- strong weak or cruel kind Activity- characterized by fast-slow, alive-dead etc The different concepts are labeled in the chart Lower scores reflect generally negative feelings toward the concept being assessed, and higher scores represent generally positive feelings

Sampling and Recruitment

Select your sample (e.g. stratified vs clustering) Contact with a cover letter 1.Purpose of study and its importance. Include any sponsorship information 2.Indicate why the respondent was chosen 3.Assure anonymous and confidential responses 4.Suggest the timing of the study 5.Provide details on how to complete the study 6.Thank the respondents and stress the importance of their responses for furthering your work 7.Sign the letter with all appropriate signatures

Threats Related to the Subjects

Selection (are we actually selecting from our target population within the accessible population) Study design solutions (multiple sites) Use random sampling techniques Problems with assignment to groups? Attrition (dropout or mortality) Maturation (changes over time that are internal to the subject)

Sensitivity and specificity

Sensitivity- the probability that the test will correctly identify someone with the disease as having the disease Specificity- the probability that the test will correctly identify someone with the disease as not having the disease •SPIN and SNOUT

Threats to external validity or transferability?

Setting differences, inadequate sample selection, time

Wording Questions

Simplicity is the key to a good questionnaire Language and slang? How many different sports do you participate in? Not a good question, because people may consider different things to be sports Do you participate in any of the following team sports on a regular basis as part of amateur league play? SEE PICTURE Wording question - Limited number of choices

Dealing with Sensitive Questions

Some people love to express their views while others may be hesitant Some questions may address social behaviors that have negative associations (e.g. smoking, sexual practices, drinking, illicit drug use) Compliance with medications? Following the doctor's orders? *Let the respondents know they can stop the survey at anytime *Ensure responses are anonymous

Branching Questions

Some question sequences try to follow up on specific answers with more detailed questions, using a technique called branching 1.Do you perform and clinical counseling activities? a)Noà skip to question 3 b)Yes 2. Approximately how many hours per week do you work as a consultant? ____

Rasch Analysis

Statistically manipulates ordinal data to create a linear measure on an interval scale *Be aware of the progression of the order of the entities on the scales. The items should progress from easy (eating) to most difficult (climbing stairs)

Questionnaires

Structured surveys that are self-administered using pen and paper or electronic formats What are some advantages to questionnaires? Disadvantages? Self-report- the researcher does not directly observe the respondent's behavior or attitudes but only records the respondent's report Recall bias- a tendency to alter past details

Statistical Regression to the Mean Threat

Subjects who start the study with extreme scores for the outcome measure change as a result of the mathematical tendency for scores to move toward the mean value Solution: Trim extreme data values from the study Aggregate repeated baseline measures

Threats to Construct Validity

The "believability" of research results also depends on a clear and adequate definition of the variables used in the study (construct validity) How do you measure a patient's socioeconomic status? 1.Salary 2.Assets (home, car) 3.Investments; and 4.Family income

The Rank-Order Question

The following are some of the reasons applicants choose to attend a particular school. Please rank them in terms of importance, from 1 (most important) to 5 (least important). ____ Location ____ Faculty reputation ____ Length of program ____ Affiliation with potential employers ____ Research opportunities

Pilot Testing and Revisions - initial set of experiments

The revised questionnaire should be pilot tested on a small representative sample (ideally from the target population) Monitor the time it takes for completion What to do if the survey includes multiple interviewers?

Design of Surveys

Time consuming What do you want to ask? The research question you want to answer Questions are not asked out of causal interest or curiosity, but because they reflect specific pieces of information that taken as a whole, will address the proposed research question. Guiding questions- objectives that delineate what the researcher is trying to find out How do PTs incorporate play into their practice? Do PTs assess play behaviors and what methods do they use? Are there differences between school-based and non-school-based settings that influence the role of play within pediatric PT?

Threats to Research Validity from Investigator Bias - Testing

Types of Studies affected - Diagnostic Tests Nature - Investigators apply the superior comparison test only to subjects who test positive on the diagnostic test of interest, may overestimate the usefulness of the diagnostic test Solutions - Apply the superior comparison test to all subjects regardless of the result of diagnostic test of interest Types of Studies affected - Diagnostic Tests, Clinical Measures, Prognostic factors, interventions, clinical prediction rules, outcomes Nature - Individuals responsible for measurement of subjects are influenced in their interpretation by knowledge of current subject status or prior test or measurement results, may produce inaccurate results due to investigator expectations Solutions - Conceal information about subject status or prior test results from individuals collection measures

Threats to Research Validity from Investigator Bias - Assignment

Types of Studies affected - Intervention Nature - Individuals responsible for enrolling subjects interfere with group assignment process, may create unbalanced groups prior to the start of the study Solutions - Create predetermined subject assignment list and conceal information about subject allocation

Threats to Research Validity from Investigator Bias - Selection

Types of studies affected - Diagnostic tests, clinical prediction rules related to diagnosis Nature - Use of subjects who do not represent the population of interest defined by the research question, limits ability to determine how well the diagnostic test or clinical prediction rule differentiates between or among different stages of a disorder of interest Solutions - Adequately define inclusion and exclusion criteria to create a sample that represents the spectrum of the disorder of interest Types of studies affected - Prognostic factors, clinical prediction rules related to prognosis Nature - Use of subjects that are "too healthy" or "too sick.", may result in misrepresentation of the timetable to recovery or adverse outcome, thereby undermining the usefulness of a prognostic factor Solutions - Enroll subjects at a common, early point in their condition Types of studies affected - Interventions, Outcomes Nature - Use of subjects who do not represent the population of interest defined by the research question, limits the ability to determine usefulness of the intervention for individuals with different levels of condition and/or different prognostic profiles Solutions - Adequately define inclusion and exclusion criteria to create a sample that represents the spectrum of the condition of interest

Assignment Threat

Unequal baseline characteristics of groups that might influence the study's outcome Solution: Randomize assignment of subjects to groups Perform statistical adjustment

Semantic Differential

Used to evaluate attitudes This method tires to measure the individual's feelings about a particular object or concept based on a continuum that extends between two extreme opposites How do you feel about natural childbirth? Good ___/___/ ___/ ___/ ___/ ___/ ___/ Bad How is this different than a likert scale? Based on opposite adjectives rather than agree vs. disagree

Cumulative Scales

Used to reflect increasing intensities/difficulty of the characteristics being measured Consider the following statements which were included in a self-assessment interview of elderly people concerning their functional health status. 1.I can go to the movies, church, or visiting without help. 2.I can walk up and down to the second floor without help. 3.I can walk half a mile without help. 4.I am not limited in any activities 5.I have no physical conditions or illnesses now. 6.I am still healthy enough to do heavy work around the house without help.

The Grid or Checklist

When a series of questions use the same format, a grid or checklist can provide a more efficient presentation For each of the following activities, please indicate the level of knee pain you have experienced during the past week: SEE PICTURE

Critically Appraised Topic (CAT)

•A standardized summary of research evidence organized around a clinical question, aimed at providing both a critique of the research and a statement of the clinical relevance of the results • •The five steps 1)Ask 2)Search 3)Appraise 4)Apply 5)Evaluate

Likelihood Ratios

•Both sensitivity and specificity are used in calculating the likelihood ratio for either a positive (LR+) or a negative (LR-) test •LR(+)= sensitivity/(1-specificity) •LR(-)= (1-sensitivity)/specificity •LRs can be used to calculate the odds of having a disease after the test results are known for the specific patient (posttest odds) •Multiply LR by specific patient's pretest odds of having disease •Pretest odds= patient's chance of having disease/patient's chance of not having the disease before the test

CAT Step 5: Evaluate

•Evaluate your findings and assess potential limitations •Level 3: Evidence-based reviews Search engines 1.The Cochrane Library 2.DynaMed 3.Sumsearch

The Final Question

•Expressed in a single, clear and focused sentence •In patients....how does...compare with... for the outcome(s) of....

CAT Step 2: Search for the Best Current Evidence

•Perform a thorough search of the literature •Select articles with a high level of evidence How do I do this? 1.Identify terms to fit the PICO question 2.Search for secondary sources 3.Search for primary sources

Critically Appraising Secondary Literature

•The main problems in secondary studies are the identification and then selection of primary studies to be included in the review •Two issues (1) the need for a broad search to ensure that as much of the available literature relevant to the question is retrieved and, (2) the selection of studies to abstract for inclusion in the review

Precision of Estimates

•The precision of each estimate depends on the number of people involved in the estimate •For sensitivity, this is the number of people classified as having the disease •Whereas for specificity this is the number of people who do not have the disease •Precision is reported in terms of a 95% CI

CAT Step 4: Apply

•The results from the chosen articles are interpreted with regard to the level of evidence, internal and external validity, and the similarity of the PICO of the article and the PICO of the current clinical question. •Results are applied to the patients' problem of interest •This step helps to improve clinical decision making •Diagnostic imaging is used to confirm a positive probability or rule out a negative probability

What does a CAT actually appraise?

•They critique the internal validity and external validity •Statistical rigor (methodology)

Interpreting the Results of a Study

•What type of bias may have been generated by our recruitment process? •Do you think there could have been a better recruitment process? If yes, please explain. •What are the limitations of the current study? •Did the study include between subject factors? Within subject factors? What were they? •What kind of statistical analysis did we use? •Were the participants that we used brought into the study in a truly randomized fashion? •Do you believe that using magazines as a teaching tool may help facilitate neuroscience content learning?