Research 2

Correlation

A measure of the degree of association among variables Function of covariation—extent to which one variable varies directory or indirectly with another variable Exploratory studies will provide evidence of a relationship that then can be tested in an experimental technique

Case Control Studies

A methodology used by selecting subjects based on whether or not they have a disorder Cases are those subjects with the disorder Controls are those subjects that are chosen as the comparison group without the disorder Look backwards in time to determine if groups differ in exposure histories or presence of characteristics that may increase the risk for the developing the condition Assumption that differences in exposure histories should explain why more cases than controls developed the outcome Results provide estimates that may support causal relationship between risk factors and disease Advantage Samples are easy to gather Disadvantage Uncertainty in temporal relationship between exposure and disease Proportion of cases and controls in study is not related to proportion of cases in population Selection of Cases Case definition: diagnostic and clinical criteria that identify someone as a case Once definition is established, criteria must be further defined Decide if new or existing cases should be included Population-based study: sample is obtained from general population of those with the disorder Affords greater generalizability, but too expensive and logically unfeasible Hospital-based study: sample is obtained from patients in a medical institution Selection of Controls Most difficult part is choosing control group Can be obtained from several sources Same hospital or institution of the cases From general population by random-digit dialing, lists, etc. Need to think about if other disorders (other than the case disorder) are represented Analysis Issues Must pay attention to bias in selection and classification of subjects Selection bias Subjects must be chosen regardless of exposure histories Self-selection bias can also occur Misclassification: random; will tend to minimize relationship between exposure and disease Observation bias: systematic difference in the way information about disease or exposure is obtained from study groups Interviewer bias: individual collecting data elicits/records/interprets information differently Recall bias: subjects who have experienced a particular disorder remember their exposure history differently than from those who are not affected

Prospective and Retrospective Cohorts

Can be either; based on where the researcher initiates entry into the exposure-disease cycle Prospective design Ability to control and monitor data collection and measure variables completely and accurately Most useful when disease occurs frequently Expensive and time consuming and subjects are often lost in follow-up Retrospective design Uses a cohort that has already been assembled Looks at how variables occurred over time in relation to specific outcomes Most efficient with diseases with long latency May have incomplete or inadequate data Selection of Subjects Must be appropriate for the research question Descriptive studies: sample must be representative of target population Analytic studies: two groups must be identified and must be enough subjects who eventually develop the disease Analysis Issues Will be influenced by misclassification of either exposure or disease Bias is less of a concern in prospective cohort studies than in case-control or retrospective studies Due to longitudinal nature of prospective cohort studies, attrition may be a concern Researchers must ensure good rapport with participants and be aggressive in maintaining a relationship

Theory Testing

Chooses specific variables for a study, based on expected relationships resulting from deductive hypotheses

Correlation and Predictive Studies

Correlational Study Purpose is to describe the nature of existing relationships Data will provide the rationale for clinical decisions or generation of hypotheses Predictive Study Predict the behavior/response based on the observed relationship between behavior and other variables Can be used to develop models for clinical decision making Useful for validation of diagnostic and prognostic information Used to validate measurement tools Regression is used to establish accuracy of prediction

Historical Research

Critical review of events, documents, literature, and other sources of data to understand how and why past events happened Can build on a foundation for interpreting current theory and practice Starts with determining a research question Once problem area chosen, framework formed to guide search for data Not just collection of facts or dates, but incorporates opinions, analyses, and inferences Sources of Historical Data Must be critical in accepting all sources Distinguish between primary and secondary sources Reliability and Validity Historian must be able to establish authenticity of data by subjecting material to external criticism Must also be subjected to internal criticism, which questions the truth/worth of material's content for the research question No scientifically rigorous procedure in doing this Synthesis of Historical Data Must decide how much information is needed to draw valid conclusions (not just because information cannot be found) Incorporates scientific logic into this process so interpretations are made as objectively as possible Must be alert on own biases Cause-and-effect statements cannot be made

Exploratory Research Observational

Data is collected in natural environment, without manipulation of variables May be considered descriptive or exploratory May involve prospective data collection or retrospective analysis of existing data Can be longitudinal or cross-sectional

Retrospective Research

Examination of data that have been collected in the past, often obtained from medical records, databases, or surveys Cannot control operational definitions or completeness in which data were collected Important source of information

Cohort Studies

Group of individuals who are followed together over time Most common is geographic cohorts, birth cohorts, historical cohorts , environmental cohorts, or developmental cohorts Cohort study Researcher selects a group of subjects who do not yet have the outcome of interest and follows them to see if they develop the disorder May be purely descriptive, but more often are analytical Useful when exposures are rare because cohort can be assembled and classified accordingly Effective for studying multiple disorders Advantage Ability to determine onset of the condition Temporal sequence can be determined Disadvantage Not useful for studying disorders that are uncommon in the population • Geographic cohorts: such as residents of a certain community • Birth cohorts: such as baby boomers • Historical cohorts: such as those who experience a specific common event:veterans of a specific war • Environmental cohorts: such as those who survived a natural disaster • Developmental cohorts: based on life changes such as getting married or moving into a nursing home

Prospective Research

Measured through direct recording in present Are more reliable than retrospective due to greater control of data collection and documentation of sequence of events

Methodological Research: Reliability & Validity

Methodological Research Development and testing of measuring instruments for use in research and practice These studies emphasize use of outcome instruments—establishing reliability and validity Purpose is to develop instruments that can then be used in further testing No instrument is absolutely reliable and valid, but must be tested under many conditions on a specific population Reliability Questions Focus on the rater and the tool itself Validity Questions Creating a new tool is an extensive process, time consuming, and expensive

Longitudinal Research

Researcher follows a group of subjects over time Repeated measurements taken over set intervals Allows researcher to describe patterns and suggest causal relationships between variables Practical difficulties Extended obligation to a single projects Long-term commitment of funds/resources Once study begins, changes can jeopardize the validity of all previous data Subjects cannot be replaced Compliance of subjects must be ensured Internal validity threats Testing effects Attrition Confounding variables

Secondary Analysis

Secondary Analysis Researcher uses existing database to re-examine variables and answer questions other than what was originally asked Advantages Minimal expense involved Ability to study large samples Elimination of data collection (most time consuming part) Disadvantage Lack of control over data collection process Unable to ensure quality of data Finding a Question to Fit the Data—or Finding Data to Fit the Question Look at data available to determine types of questions would fit Requires familiarity of data Searching for a database that will provide the needed information to answer a given research question Requires familiarity of databases Sources of Secondary Data Increasingly common because of availability of large computer data sets

Cross-Sectional Research

Study of stratified group of subjects at one point in time Draws conclusions about a population by comparing the characteristics of those strata Efficient Not threatened by testing or history effects Threatened by selection

Exploratory:

Systematic investigation of relationships between two or more variables Prediction of effect of one variable on another OR test relationships supported by clinical theory Usually guided by hypotheses to help structure measurements and interpretations of findings

Evaluating Causality in Observational Studies

Validity of statistical association between exposure and disease must be made under conditions that are not controlled Must first rule out bias influences, confounding and chance variation, and assess strength of cause and effect criteria Must establish: A time sequence Biologic credibility Consistency with other studies Prescience of a dose-response relationship Biologic credibility: should be able to postulate the mechanism by which the exposure might reasonably alter the risk of developing the disease Dose-response relationship: provides evidence for causality; this means that the severity of the disease can be associated with varying levels of the exposures

Diagram:

A: most desirable baseline; B: variable; C: accelerating; D: decelerating Accelerating or decelerating baseline indicates a change in the target behavior is occurring without intervention; may represent improvement or deterioration Trends in baseline data must be determined so that if treatments are given, the changes are a true change Often have to extend baseline or intervention phases to until stability is achieved Phases must be sufficient enough to capture any changes that will occur over time

Nonequivalent Posttest-Only Control Group Design

Also known as static group comparison Can include any number of treatment levels, with or without a control group Uses existing groups who have and have not received treatment Internal validity is severely threatened by selection bias and attrition Should only be used for exploratory research Does not show any casual relationships Analysis Discriminant analysis, analysis of covariance

Designs with Multiple Treatments

Alternating Treatment Design o Has rapid alternation of two or more interventions o Alternation can occur within at treatment session, session by session, or day by day o Can be used to compare treatments, control to a treatment o Data for each treatment is plotted on the same graph o Sequence effects are of concern and can be addressed through random ordering of treatments or by systematic counterbalancing o Advantages over an A-B-A design is that no withdrawal is necessary and data can be obtained quickly • Ex: Study on effect of visual feedback and mental practice on symmetrical weight bearing in three individuals with hemiparetic stroke. They used a multiple baseline design across subjects to compare strategies of visual feedback alone or feedback with mental practice on the proportion of body weight borne by the affected limb. Interventions were applied using an alternating treatment approach, with treatments systematically alternated between morning and afternoon OT sessions.

Sequential Clinical Trials

Approach to RCTs allowing for continuous analysis of data as it becomes available Instead of waiting until the end of the experiment Results accumulate as each subject is tested Leads to a substantial reduction in number of subjects needed for a study Not used a lot in rehabilitation research But could be based on convenience of design and applicability to research questions Used to compare a new treatment versus an old treatment Comparison between treatment A and B is evaluated choosing a preference of which is more clinically effective than the other Measuring Preferences Clinically meaningful difference between the two treatments The Sequential Chart Plotted chart to look at comparison of results Two types of charts Stopping Rules After each "little experiment" researcher decides if further research needs to be done Benefit is trial can be stopped as soon as it is clear that one treatment is superior to the other Considerations in Sequential Trials When the difference between the two treatments does not demonstrate a preference, then that pair of subjects is discarded Useful when treatment effect results are quick More readily fits with clinical research model Useful for studying qualitative outcomes using measures of preferences

Reliability

Assessed concurrently with data collection Report interrater reliability using a measure of percentage of agreement between observers

Pretest-Posttest Control Group Design

Basic structure of a RCT Compares two or more groups that are randomly assigned One group gets treatment other acts as control Both groups are tested before and after treatment- same time for both groups Considered the gold standard for establishing cause-and-effect relationships May have more than one dependent variable

Measuring Target Behaviors

Can reflect responses due to impairments, activity limitations, or measures of disability Assessment techniques vary (depending on the variables being measured) Ability to develop individualized measurement systems (for single-subject design) Most frequent ways to measure a behavior: Frequency, duration, and magnitude

Surveys

Composed of questions that are given to group Can be oral, written, or electronic Intended for generalization to a larger population OR descriptive of a group Many times focus on current practices, attitudes and values, or characteristics

Normative Studies

Describe typical/standard values for characteristics of a given population o Often directed toward a specific age group, gender, occupation, culture, or disability • Norms are expressed as a mean, with an average range • Importance of establishing the validity of normative values o Estimation of "normal" behavior/performance is often used as a basis for prescribing intervention or predicting future performance • Still need for normative studies as new measurement tools are developed • Samples : o Great potential for sampling bias when striving for establishment of standard values o Need a large sample size that is random, yet representative of the population o Target population should be clearly delineated o Replication is essential

Multi-Factor Designs for Independent Groups - Nested Design

Does not fit all multi-factor analysis, but is used when attribute variables cannot be crossed with all other levels of other variables Used in educational studies where classes are nested in schools or schools are nested in cities Analysis of Nested Designs Analysis of variance More complicated analysis, biostatistician

Efficacy vs. Effectiveness

Efficacy Benefit of an intervention as compared to the control or standard treatment Effectiveness Benefits of use of the treatment under "real world" conditions Just because findings of a RCT stated that a treatment works, does not necessarily mean clinicians find that it has the same effect when used in actual treatment sessions!

Clinical Trials

Experimental studies that look at the effect of interventions Frequently done on a large scale Therapeutic trials Effect of intervention/treatment on specific disease/diagnosis Preventive trials If procedure decreases risk of developing disease/disorder

Length of Phases

How long the phase should be It can be flexible, phases are not fixed It is desirable if they are given in relatively equal phases Must be a minimum of 3-4 data points in each phase Greater number of data points, more obvious trends are

Multi-Factor Designs for Repeated Measures - Two-Way Design with Two Repeated Measures

More than one IV Can be a 2 x 2 or even larger to become a 3 x 3 Analysis Two-way analysis of variance

Multi-Factor Designs for Independent Groups - Three-Way Factorial Design

Multiple comparisons Main effect of each IV Examine differences between the two intensities Difference between locations Interactions are examined, can occur in any differences in any of the 8 cells Advantage: "Real world" Disadvantage: Sample size must be very large Analysis of Factorial Designs Two-way or three-way analysis of variance

Time Series Design

Multiple measurements before and after intervention to examine patterns of behaviors Used often in studying community interventions and policy changes May be considered an extension of the one-group pretest-posttest design Most effective when serial data can be collected evenly (time) throughout the study Avoids confounding variables Most threats to internal validity are controlled for External validity is limited Analysis Use of graphic visual analysis Autoregressive integrated moving average Segmented regression analysis

One-Way Repeated Measures Design Over Time

Multiple measurements of DV are taken at specific times Intervention may be applied once or repeated in between measurements Time is IV No randomization of order of treatment and no comparison group Internal validity is threatened May be useful when time course of a disease is predictable Analysis One-way repeated measures analysis of variance

Choosing a Target Behavior

Must define specific target behavior that is being assessed Important considerations: Relative stability of behavior Establish a specific intervention May be influenced by assessment tools and clinical goals

Multigroup Designs

Nonequivalent Pretest-Posttest Control Group Design Historical Controls Nonequivalent Posttest-Only Control Group Design

Analysis of Pretest-Posttest Designs

Often analyzed using change scores With interval or ratio data: Unpaired t-test With ordinal data: Mann-Whitney U-test Analysis of covariance can be used to compare posttest scores Discriminant analysis can be used with two-factor design to distinguish between groups with more than one outcome measure

Multi-Factor Designs for Repeated Measures

One group of subjects is tested under all conditions, acting as his or her own control Considered a series of trials with each subject Also called within-subjects design Can only be used if outcome measure will revert back to baseline between interventions May not qualify as a true experiment Advantages Ability to control potential influences of individual differences Using subjects as their own control provides the most equivalent comparison group as possible Disadvantages Potential for practice effects (learning) Carryover effect

One-Group Pretest-Posttest Design

One set of repeated measurements taken before and after treatment on one group Measurement of change is based on difference between pretest and posttest scores IV is time, two levels being pretest and posttest Treatment is not the IV because all subjects get the treatment Weak due to not control group making it vulnerable to threats to internal validity External validity is also limited Can be used when previous research has been published with a control group with similar circumstances Also reasonable when experimental situation is isolated Analysis T-test for paired comparisons Wilcoxon signed-ranks test

One group Designs

One-Group Pretest-Posttest Design One-Way Repeated Measures Design Over Time Time Series Design

Constructing Survey Questions

Open-Ended Questions o Ask respondents to answer in their own words o Useful for finding out feelings and opinions, without biases or limits o Difficult to code and analyze Closed-Ended Questions o Select answers from several provided choices o Easily coded and increases uniformity across responses o Does not allow for personal viewpoints o Choices may overlook some important responses (may bias)

Experimental Designs

Provide a structure for evaluating cause-and-effect relationships

Quasi-Experimental Designs

RCTs are not always possible Good substitution as long as researchers document subject characteristics, controls with the research protocol, and uses blinding as much as possible Results may have potential biases of the sample, but can still provide important information Quasi-experimental designs are close to experimental design, but lack random assignment and/or a comparison group Often having nonequivalent groups Degree of control is decreased

Historical Controls

Research comparing treatments involving historical controls who had a different treatment at an earlier time Can be used if a researcher believes that a new intervention is more effective than standard care Should not be any patient in the literature, rather needs to be very specific Advantage Efficient Disadvantage Found positive effects from history controls that RCTs have not replicated Analysis T-test, Mann Whitney U test, chi square, multiple regression, logistic regression, or discriminant analysis

Posttest-Only Control Group Design

Same as pretest-posttest design but no pretest is given It is a true experimental design and can be expanded to include multiple levels of the intervention, with a control group Random assignment= strong internal validity Most useful when large number of subjects Can also be used when pretest is impractical or reactive

Nonequivalent Pretest-Posttest Control Group Design

Similar to pretest-posttest experimental design, except subjects are not randomly assigned Can be designed with one treatment group and one control group or with multiple of each Limited due to lack of randomization Some control over history, testing and instrumentation effects Pretest scores can be used to test assumption of initial equivalence Analysis Unpaired t-test, analysis of variance, analysis of covariance, analysis of variance with matching, Mann-Whitney U test

Single-Factor Designs for Repeated Measures - One-Way Repeated Measures Design

Simplest form of repeated measures design All subjects have all levels of the IV Order effects Must bring attention to biasing due to multiple-treatment conditions Randomize the order of presentation of each subject Latin Square Matrix composed of equal numbers of row and columns, used to randomized sequence Analysis: one-way analysis of variance

Designs for Independent Groups

Single-Factor Designs for Independent Groups Pretest-Posttest Control Group Design Posttest-Only Control Group Design Multi-Factor Designs for Independent Groups Factorial Design Two-Way Factorial Design Three-Way Factorial Design Randomized Block Design Nested Design

Repeated Measure Designs

Single-Factor Designs for Repeated Measures One-Way Repeated Measures Design Crossover Design Multi-Factor Designs for Repeated Measures Two-Way Design with Two Repeated Measures Mixed Design

Generalizations of Findings

Single-subject designs focus on clinical outcomes and provide data for clinical decision making. • While there may be proof that a treatment works on a single patient, it does not prove a treatment will work for others o It is possible to assume the treatment may work for other patients with similar characteristics • Results of single-subject designs may provide more "real world" understanding of individual responses than data from group studies. • Direct Replication o Necessary to demonstrate external validity o One successful single-subject study with three successful replications are considered sufficient that the results were not by chance • Systematic Replication o Should occur after direct replication is done o Used to demonstrate findings can be observed under different conditions than the initial study • Clinical Replication o Advanced replication procedure used to apply to multiple behaviors within actual practice o Can only be done after direct and systematic replication have been completed • Social Validation o Used to investigate the treatment effects within a social context o Important to assure interventions are appreciated and adopted o Whether or not interventions are perceived as useful and satisfactory o Not required to establish treatment effectiveness, it important aspect of treatment planning and decision making

Design Classifications

True Experimental Design Subjects are randomly assigned to 2 or more comparison groups Strongest design for evaluating casual relationships Randomized Control Trial (RCT) is gold standard of true experimental design Quasi-Experimental Design Lacks random assignment and/or comparison groups Appropriate when stronger designs are not feasible Accommodate limitations Used if true experimental design is unethical or impractical

Multi-Factor Designs for Independent Groups - Two-Way Factorial Design

Two IVs IVs are completely crossed, meaning every level of factor is represented at every level of other factors Effect of each IV is analyzed Interaction effect is analyzed to look at interaction of the two IVs Advantage Gives researcher information that cannot be obtained with single-factor design Enhances generalizability

Multi-Factor Designs for Repeated Measures - Mixed Design

Two IVs One repeated across all subjects The other randomized to independent groups Often used with attribute variables Can have more than two IVs Analysis Two-way analysis of variance

Two-Group Pretest-Posttest Design

Two experimental groups formed by random assignment Used when controlling the condition is not ethical or feasible Often used to compare a new treatment Appropriate when RQ addresses a difference between two interventions

Baseline Characteristics

Two important characteristics: stability and trend Most desirable is to show a constant level of behavior with very little variability Shows the target behavior is not changing When there is a variable baseline, can be a problem

Multi-Factor Designs for Independent Groups : Factorial Design

Two or more independent variables Groups randomly assigned to various combinations of levels of the two IVs Usually include two to three IVs As variables increase, so do number of experimental groups and increase in sample size Described based on dimensions (i.e. two-way has two IVs) Diagrammed using a matrix

Multigroup Pretest-Posttest Control Group Design

Used to compare several treatments to control conditions Allows for any number of independent variables Strong in internal validity

Multi-Factor Designs for Independent Groups - Randomized Block Design

Used when an attribute variable (blocking variable) is compared with an active IV Blocks of homogenous subjects are randomly assigned to treatment variables Blocking factor/variable must be related to DV Can involve more than one IV Generalization is limited based on definition of blocks Analysis of Randomized Block Designs Two-way analysis of variance, multiple regression, or discriminant analysis

Data Analysis in Single Subject Design

Visual Analysis (used most often) Focuses on the clinical significance of outcomes Analyzed based on within-phase and between-phase characteristics Within-phase can be described according to stability and trend Phase comparisons can only be made across adjacent phases. Level Value of the DV, or magnitude of performance during intervention Can also be described in terms of the mean Should always be shown on graph as raw data Trend Direction of change within a phase Can be accelerating or decelerating, stable or variable Slope The angle, or rate of change within the data Can only be determined in linear trends Celeration Line Used to estimate trend within a data series Specific procedures to calculate celebration line and its slope Note: data analysis of single-subject design is usually done with visual analysis. Arguments that analysis is unreliable or subjective

Selecting a Design

When an experimental design is appropriate, then to choose specific design, use the following six questions: 1. How many independent variables are being tested? 2. How many levels does each independent variable have, and are these levels experimental or control conditions? 3. How many groups of subjects are being tested? 4. How will subjects be assigned to groups? 5. How often will observations of responses be made? 6. What is the temporal sequence of interventions and measurements?

Single-Factor Designs for Repeated Measures -Crossover Design

When two levels of an IV are repeated in a systematically varied pattern Half the subjects have treatment A then B, and the other half get B then A Two subgroups are created then subjects are randomly assigned Should be used when subjects diagnosis will not change over time Useful when treatment effects are immediately reversible If there is treatment carryover, need to have a washout period Analysis Paired t-test Two-way analysis of variance Wilcoxon signed-rank test

Analysis of Posttest-Only Designs

With two groups: unpaired t-test with interval and ratio data With two groups: Mann-Whitney U-test with ordinal data More than two groups: one-way analysis of variance or Kurskal-Wallis analysis of variance by ranks Analysis of covariance can be used when covariate data of extraneous variables is present Can also use regression or discriminant analysis

Interval Recording:

breaking down the full measurement period into preset time intervals and determining if the behavior occurs or does not occur within each interval

Open-Ended Questions

difficult to code and analyze due to so many different responses can be obtained If misunderstood, they may elicit answers that are irrelevant to researcher's goal Can be effective in interview because the interviewer can clarify respondent's answers by follow-up questions Generally avoided in questionnaires, except when answers are fairly objective

Designing the instrument

first questions should pique interest or be neutral, and later sensitive questions should follow -Format should be "friendly" and not complicated or confusing; page should be uncluttered, printed in laser-quality print, aligned so it is easy to find the next question, font size should be a minimum of 11 or 12 pt, font should be simple to read Preliminary drafts: Panel helps to identify problems with questions, including wording and organization; panel should be given guiding questions Revision process should continue until the researcher is satisfied that the instrument is concise, clear, and serves intended purpose; while this process is time consuming it is important to help establish content validity

Duration:

how long a target behaviors last; can be measured as cumulative total duration or as duration of each individual occurrence of the behavior

Baseline phase:

is during no treatment; reflects natural state of targeted behavior over time Assumption is that baseline data reflects ongoing effects of background variables, thus when treatment begins the changes observed should be attributable to intervention Baseline data, essentially, provides a standard of comparison for evaluating the potential cause-and-effect relationship btwn tx and target behavior This approach is not appropriate for all types of interventions, especially those that are critical or life-threatening situations (so withholding tx is not harmful)

Quantitative Score:

many variables are measured using an instrument/assessment tool that provides a quantitative score—can provide a summary score, subscale score, or single test value Measurement must be chosen to reflect the element of performance that is of primary focus/concern Each measurement will have a different perspective of the target behavior—influencing how data will be interpreted

Frequency Count:

number of occurrences of the behavior within a set time interval or number of trials; can be expressed in percentages; can also translated into rates (rate is the number of times a behavior occurs within a specified time period)

Multiple Treatment Designs

o A variation of the withdrawal design o Application of one treatment (B) following baseline, the withdrawal of that treatment, then introduction of another treatment (C) An A-B-C-A design o Should be able to see differences in target responses across each phase of the study • Ex: Study on the axial weight loading on gait parameters in individuals with cerebellar ataxia. Five subjects with ataxic gait ambulated along a fixed walkway for five trials in each of four conditions: unweighted (baseline), with 10% body weight at the shoulders, with 10% body weight at the waist, and a final unweighted phase, creating an A-B-C-A design. Several gait parameters were measured, including velocity, cadence, step length, and double support time.

Interviews

o Asks respondents questions and record answers for analysis o Can be done in person or over the phone o Face-to-face found to be more effective in establishing rapport o Most are structured, with a standardized set of questions o Unstructured interviews, there is no fixed agenda o Interviews are conducted in respondent's natural setting (for the most part) 60-80% response rate is excellent, usually 30%- Advantages o In-depth analysis o Can probe responses o Can directly observe respondents' reactions Disadvantages o Cost and time o Need for personnel o Scheduling o Lack of anonymity

Research Question

o Guiding Questions o First step is to define purpose through series of guiding questions o Hypotheses o Surveys can be designed to examine relationships o Must specify hypotheses in addition to guiding questions o It is important to guide statistical analyses and conclusions o Questionnaire Outline o Once guiding questions and hypotheses are created, detailed outline should be made o Each question should add to a larger concept of the guiding questions o Demographic information can also be included o Research question: first need to delineate research question with reference to a target population o Surveys are appropriate when question requires obtaining information from subjects, rather than observing or measuring performance. o Guiding Questions: validity is a major concern in design of surveys o Questions are asked because they reflect certain pieces of information, that taken as a whole will address the proposed research question o Demographic information: needed to describe characteristics of respondents, compare characteristics, and to interpret how personal characteristics are related to subject's responses

Contacting Respondents

o Introduction should be given to describe purpose and how data will be used o How long it will take to complete o Due to low response rates, follow up should be planned Cover Letter o Must include a cover letter to orient the respondents and politely request participation o Should include: o Purpose and importance of study o Why respondent has been chosen o Ensure anonymity o Provide suggested timeframe o Ask for return response (with deadline) o Thank respondents with stress of importance of their response o Sign letter with name, degrees, and affiliation

Format of Case Studies

o Introduction with background literature on the patient problem o Theoretical or epidemiologic information o Full history of patient o Section on methods presents elements of the treatment plan, if any special assessments used, should be described o Results section includes the patient's responses and follow-up data o Discussion section provides interpretation of outcomes and conclusions o Any further research suggested • Generalizability o Most practical approach to research, but also least rigorous approach due to inherent lack of control and limited generalizability o Weak internal validity o Often concerned with rare cases, not representative of the "typical" patient o External validity is also limited • Case Reports in Epidemiology o Description of one or more people, documenting unique/unusual occurrence of medical condition o Presents a complete picture about characteristics of, exposure faced by, that person, resulting in presentation of hypothesis about the casual factors that might be the reason for the observed outcome o Do not provide sufficient control to allow for generalizations or causality conclusions

• Combining Qualitative and Quantitative Research

o Mixed methods o Qualitative and quantitative aspects can be combined within one study o The findings from each measure certain components of behavior and see how such measurements relate to the nature of the actual experience o Can increase validity of findings

Multiple Baseline Design Across Behaviors

o Monitors a minimum of three different (yet similar) behaviors in the same subject o Target behaviors are measured concurrently and continuously until baseline is achieved o Intervention is introduced chronologically across the different behaviors o Advantages Replication Experimental control • Example: Study on the effect of interactive teaching approach on the acquisition of prelinguistic skills in a child with Down syndrome and language delay. Focused on three specific components of communication: prelinguistic requesting, vocal imitation and commenting. They tallied the number of appropriate responses observed during each session. After an initial baseline, training addressed requesting toys within the environment, while other behaviors were monitored. Training strategies were later introduced for imitation and commenting behaviors in a staggered fashion.

Selecting a Sample

o Must choose a sample that is accessible o If possible, probability sample should be used o Can also use stratified and cluster sampling as well o Pilot testing: after pilot testing, researcher be revise and retest questionnaire until final instrument has an acceptable level of validity o Major concern during this process will be the length of the survey—long questionnaires are less likely to be completed o If a study involves interviews, there should be a formal interviewer training o Selecting a Sample- for interview surveys, respondents would typically be within a local geographic area o For phone interviews, can send advanced notice in the mail that phone call will be coming o For mail surveys, accessible population might be dispersed and may only be limited by availability of mailing addresses

Methods of Qualitative Research Collection

o Observation Often conducted prior to interviews Purpose is to identify people, interactions, influence of sociocultural context, and artifacts that may be studied in-depth to gather data relevant to the study After observation, field notes should be recorded about what was observed Can also be videotaped Essence of qualitative research is to look at the person's experience as lived by that individual Researcher becomes a participant in the active off the group so that their view point can be observed directly inherently biased o Interviews Form of direct contact between researcher and participant within natural environment Used to gather information using probing questions about participant's experiences and perceptions Should have a broad script to guide discussion, but must be flexible enough to probe further with follow-up questions based on a participant's responses Quality of data collected depends on the knowledge and skills of the interviewer • Data Analysis and Interpretation o An inductive process with constant interchange between data and theoretical conceptualization (both of reality) o Process of analysis is ongoing as data is collected o Data recorded through written memos and/or transcribed from audio or videotapes o Requires many hours of sifting, coding, and organizing • Sampling o Purposeful sampling-locating participants who will be effective informants and provide rich data o Theoretical sampling- based on need to collect data to further examine emerging categories and their relationships o Sample size is an important consideration—not all qualitative samples are small • Reliability and Validity o Have to determine the trustworthiness of data Credibility Internal Validity Truth I.V Transferability External validity Consistency Reliability o Techniques for Ensuring Trustworthiness of Qualitative Data Triangulation: concepts are confirmed using more than one source of data, more than one data collection method, or more than one set of researchers Audit Trail: a clear description and documentation of the thought processes used to interpret data; allows those who read the research to follow the investigator's logic

Withdrawal A-B-A Design (baseline, treatment, baseline)

o Replicates one baseline phase after the intervention phase o Shows that behavioral changes are only present during treatment phase o Internal validity is controlled due to decreased likelihood of confounding factors during both no treatment and treatment phases o Problem is that behavior must be reversible Which is often not the case • A-B-A-B Design (baseline, treatment, baseline, treatment) o Initial baseline data and ends on an intervention o Experimental control and clinical relevance strengthened o Advantages Provides two opportunities to evaluate treatment effectiveness Effects can be replicated during the two separate intervention phases controlling for internal validity o Limitations: Behaviors must be reversible to observe treatment effects. If target behavior does not return to original baseline (but stays level) still is possible to show change if there is further improvement during second intervention phase

Pilot Testing and Revisions

o Revised questionnaires should be pilot tested to small sample then interviews to determine if/when questions were unclear or misleading o Look for missing answers or inconsistencies o Monitor time it takes to complete questionnaire

Wording Questions

o Simplicity is key! o Should be written in common language for lowest education level o If English is not primary language can cause confusion o Avoid idioms or subtle cultural expressions o May have to translate into other languages o Should be clear and unambiguous o Questions that require interpretation are more likely to have misunderstood responses Double-Barreled Questions (2-step) o Avoid the use of double-barreled questions o Those than use "or", "and" and assess more than one thing in a single question o Better to have two questions Frequency and Time Measures (how many times smoke/drink) o Many times want to quantify behavior based on frequency and time o Should determine exactly what behavior is most relevant and provide timeframe for interpreting the question o Estimates of "typical" may tend to ignore extremes Dealing with Sensitive Questions o Often subject to recall bias o Remind respondents in introduction that they may refuse to answer any questions o Should be phrased to put respondent at ease

Questionnaires

o Structured surveys that are self-administered o Can be pen and paper or electronic Advantages o More efficient than interviews o Data can be gathered from a large sample (geographically) o Short period of time to gather data o Respondents can take time to choose answer o Provide anonymity o Useful to examine phenomena that cannot be observed Disadvantages o Potential misunderstanding of questions Distribution o Can be done via mail, electronically, survey software Return Rate o Often low o A response rate between 60-80% is considered excellent o A response rate from 30-60% is most common o Have to discard questionnaires that were incomplete or incorrectly filled out Self-Report o No direct observation occurs, rather respondent provides report o Potential for bias or inaccuracy of self-report Recall bias o Self-report measures are generally found to be valid

Format of Closed-Ended Questions

o Two common formats: brackets, circling answer o Dichotomous Responses o Yes or No o Range of Responses o When characteristic is on a continuum o Very Important to Not Important (including an option for Unsure) o Grid or Checklist Responses o Rank-Order Question o Present a series of responses as is asked to rank responses on an ordinal scale o Place order in terms of importance, from 1 (most important) to 5 (least important) o Branching (if say no, skip ques 3) o Saves time by avoiding questions that are irrelevant to a respondent

Stability:

reflects the consistency of response over time

Trend

slope; shows the rate of change in the behavior

Closed-Ended Questions

two basic considerations need to be made when writing closed-ended questions Responses should be exhaustive-include all possible responses that can be expected Response categories should be mutually exclusive- each choice should clearly represent a unique answer Should be a rationale for ordering response choices Can also write questions with more than one answer- but it is difficult to code multiple answers

Non-concurrent Multiple Baseline Design

• Can be used when concurrent observation of subjects is not possible due to availability • Researcher randomly selected length of several baselines (5, 10, 15 days) • When subjects, who meet the study criteria, become available subject is randomly assigned to a baseline length o Baseline data is then collected o As more subjects become available, are too randomly assigned to the remaining baselines • Ex: Study on the treatment of visual neglect in elderly patients with strokes. They used a non-concurrent A-B-A multiple baseline design to examine the effect of a scanning and cueing strategy on unilateral visual neglect as measured by a written inventory, the Star Cancellation Test (SCT). Seven patients, aged 60-85 years, were recruited from a stroke rehabilitation unit. Baselines were staggered between 3 and 4 weeks. Intervention and second baseline (withdrawal) phases each lasted 3 wks.

Descriptive Research

• Designed to describe factors that define characteristics, behaviors, and conditions of individuals and groups • Document nature of existing phenomena and describe how variables change over time o Generally structured around a set of guiding questions or research objectives to generate data/characterize a situation of interest o Can supply the foundation for classification of individuals, identifying relevant variables, or asking new research questions • May use prospective or retrospective data collection and longitudinal or cross-sectional methods • Includes: developmental research, normative research, qualitative research, and case studies

A-B design (Weakest)

• Does not control threats for internal validity well • To strengthen the control of the design • Must have evidence that treatment was responsible for the observed changes o Replication of effects Phases are repeated with withdrawing and reinstating baseline/treatment conditions Replicate effects across more than one subject The more often an effect can be replicated the stronger the design Less threats for internal validity

Developmental Research

• Includes description of developmental change and sequencing of behaviors in people over time • Have contributed to theoretical foundations of clinical practice o Ex. Gesell, Erikson, etc. • Can be characterized by method: longitudinal or cross-sectional o Longitudinal method is preferred o Cross-sectional provides a greater possibility of sampling large representative groups (assessments) • Provides invaluable source of information for correlational and experimental hypotheses

Case Studies

• Involve in-depth description of a person's condition or response to treatment o Can also focus on a group, institution, or other social unit • Case series is an expansion of a case study involving observations in several similar cases • Allows understanding of totality of a person's condition outcomes of care o Can explore the condition, emotions, thoughts, and past and present activities • Purposes of Case Studies o Understanding Unusual Patient Conditions o Providing Examples of Innovative or Creative Therapies o Generating and Testing Theory o Providing Future Research Directives • Understanding Unusual Patient Conditions: emphasizes unusual patient problems or diagnoses that present interesting clinical challenges; sometimes unique situations offer perspectives to support the implementation of guidelines for patient care • Providing Examples of Innovative or Creative Therapies: may focus on innovative approaches to treatment; changes in technology provide an excellent opportunity to demonstrate resourceful approaches to patient care; descriptive case studies can also be extremely helpful for understanding institutional culture and change • Generating and Testing Theory: especially helpful for demonstrating how clinical theories can be applied; can also offer insight into existing theories, providing opportunities to confirm or challenge specific hypotheses • Providing Future Research Directives: results can often highlight issues that require further inquiry, providing a rich source of research questions; ability to provide information that can be used to generative inductive hypotheses; eventually, after multiple documented cases, a conceptual framework forms, providing a basis for categorizing patients and for generating hypotheses that can be tested using exploratory or experimental methods

Qualitative Studies

• Linked to the philosophy of logical positivism • Researchers must first understand a person's perspective • Seeks to describe complex nature of humans and how individuals perceive their own experiences within a certain social context • Emphasizes understanding of the human experience • Methods allow for many simultaneous variables to be studied in a phenomena • Purpose: To examine experiences from a holistic approach that looks at the true nature of "reality as participants perceive it". Logical positivism: human experience is assumed to be limited to logical and controlled relationships between specific measurable variables

Natural History

• Longitudinal studies • Focuses on natural history of disease states • Important in future design of clinical trials and generation of hypotheses about etiology and progression of disease

Multiple Baseline Design Across Conditions

• One behavior is monitored on one individual • Treatment is given sequentially across two or more environmental conditions • Experimental control can only be achieved if targeted behavior is independent of the environment o Ex: Study on the effects of a computer-based program on communication functions in children with autism. Software was developed based on daily life activities in three settings. Five children from ages 8-12 were evaluated during play time, during food activities at breakfast or snack time, and during hygiene activities following snacks. Computer interaction was introduced on a staggered basis in each setting. Frequency of appropriate or inappropriate responses was measured with an A-B multiple baseline design.

Multiple Baseline Design Across Subjects

• One intervention applied to the same target behavior across three or more individuals who have common characteristics • Ex: Study on effectiveness of an intermittent intensive PT program for children with CP. Wanted to demonstrate that rest periods within the schedule of treatment could be well tolerated. Using an A-B multiple baseline design, baseline phases were staggered from 8 to 20 weeks, during which time the children received treatment 2x/wk. During the intervention phase, they received four treatments per week for 4wks, alternated with an 8-wk rest period. Changes in motor performance were assessed using the GMFM.

Perspectives in Qualitative Research

• Phenomenology o Looks to draw meaning from complex realities through analysis of first-person narrative materials o Experience is constructed within the person's social context • Ethnography o Study of attitudes, beliefs, and behaviors of a specific group of people within their own cultural milieu o Researcher becomes immersed in the subjects' way of life to further understand the cultural forces (those that shape behavior and feelings) • Grounded Theory o Allows researchers to develop a theory to explain what is observed o Researcher collects, codes, and analyzes data simultaneously o Concepts grow our of ongoing constant comparative analysis o Theory must come from the data

Two Core Elements - Single Subject Designs

• Repeated Measures o Systematically collect data repeating the same measurements over time o Frequently and set intervals o Required to look at trends and patterns over time o Can make observations and modify the design as study progresses • Design Phases o Two testing periods: baseline phase and intervention phase o Measured behavior occurs throughout both phases o Designs are plotted on a line graph o If one baseline and one treatment phase, design is called an A-B design

Design of Surveys

• Review of Existing Instruments o Look at other instruments to determine if they are adaptable to current study o Often can borrow from instruments or modify to fit another research study o Saves a lot of time o Designing the Instrument o Begin writing questions in groups based on categories o Should follow a logical sequence general to specific o Organization is very important o Preliminary Drafts o Distributed to panel to review the survey with feedback for revision given o Revisions should be made and then given back to panel

Descriptive Surveys

• Surveys are used to collect information about a specific group o To describe characteristics or risk factors • Purpose o Provide an overall picture of group's characteristics o May involved some correlational interpretations regarding association between specific variables

Multiple Baseline Designs

• When replication cannot be completed, multiple baseline design can be used o Allows for basic A-B format, but also with withdrawal variations • Has experimental control by having baseline data collected across a minimum of three data series • Once baseline is stable, intervention is applied for first series, while other baselines are continued • Once first series achieves stability (treatment), intervention is introduced to second series • Note: stability can be acceleration, deceleration, or level, as long as it is a constant trend • Process is repeated on a staggered basis for all remaining baselines • It is important that each baseline is independent (change is only observed with intervention and not during baselines) o Strengthens experimental control

Single Subject Designs

• While generalizations are important, sometimes group performance is not always relevant or able to predict individual performance • Single-subject designs are an alternative to draw conclusions based on effects of treatments using a single patient • Clinically viable and useful • Use a controlled experimental design o Require same attention to logical design and control as other experimental designs o IV= intervention o DV= patient response (target behavior) • Can be used to: o Compare several treatments o Compare components of treatments o Compare between treatment and no-treatment • Target behavior: observable, quantifiable, and a valid indicator of treatment effectiveness