Research Mid-term
In the context of data management, DATA refer to: pg. 548
*all the pieces of information that are collected from research participants to address a study's questions *all of the information that is gathered to monitor and manage study progress
Using techniques appropriately means that a researcher must: pg. 549
*understand the type of data needed for a given analytic strategy *understand the assumptions of the technique and how these can be determined *interpret results correctly
Limitations of cross sectional studies include:
-not appropriate for studying rare conditions or those with a short duration -cannot distinguish temporal sequence of cause & effect -may falsely classify person with health problem in remission as not having problem (b/c snapshot)
2 main factors influencing rigor of survey studies: p. 91
-potential nonresponse bias -potential response bias (unable to accurately recall info, interpret meaning of question wrong, response choices not reflective of respondent experience)
Advantages of cross sectional studies include:
-relatively easy -not a lot of time required -usually less expensive -provides snapshot of population
Criteria for selecting an instrument include: p. 542
-relevance & sufficiency of information gathered (is the approach relevant to the research question) -extent to which instrument conforms to standards of methodological rigor (literature evidence that quantitative instrument is reliable & valid or qualitative instrument is trustworthy) -logistical considerations (time required, impact on participants' health, etc., human & economic resources) -norms/criteria available for referencing -type of data instrument yields (is the numeric data sufficient for the kinds of analyses that will be needed to answer research question?)
What are the ways to decrease dispersion on the dependent variable due to individual differences?
1. select participants who are homogeneous 2. use a pretest-posttest experimental design as opposed to posttest design 3. use randomized matched subjects design
In ________ individuals are not randomly selected but groups are programs are selected and every member of that group or program is invited to participate. p. 521
Cluster sampling
______ can make a difference in occupational performance and volition. p. 516
Gender
T/F, it is possible for an experiment to have internal validity but no external validity.
True
__________ involves reporting a relationship when there really isn't one.
Type I error
The main advantage of stratified sampling is... p. 104
ensures each subgroup will be included appropriately in the sample and there will be more stability (less sampling error)
Cumulative Incidence... pg. 59
number of observed new cases during a specified period/total population at risk
Non-experimental group comparison designs are also known as?
quasi-experimental designs
The written form or guide to be used by the interviewer is called a ______. p. 95
questionnaire or interview schedule
The main purpose of ______ in quantitative research is to be able to accurately draw conclusions abut the population by studying the sample. p. 518
sampling
Bias is ______ rather than random error. 2 major sources of bias are selection bias and information bias.
systematic
Normative Research... pg. 61
*establishes usual or average values for specific variables *helpful for identifying and characterizing performance problems (how extreme person's deviation from norm) *norms serve as a basis for evaluating performance of a given individual by comparing that individual with what is typical for one with same characteristics *normative studies use random samples and ordinarily relay on large sample sizes
Documentation for a study should explain: pg. 550
*how to track what member of the research team has collected what data from which participants *when the data were collected *if any additional information needs to be collected *what the status of the data currently is (entered or not, where)
As a data management tool, a management plan and project timeline is helpful in the : pg. 556
*identifying the specific tasks that will need to be done over the course of the entire project *when these tasks need to be completed
While data is being prepared for analysis, the investigator will need to: pg. 564
*implement data checking and cleaning protocols *document corrections made to the data files *document the construction of derived variables *document the process of recoding variables *apply weights to the data, if necessary *transfer, if necessary, the data to the analyst *ensure that all files are archived for later use and sharing
2 types of secondary data
-aggregate: summary level data from applying statistical analyses to micro-level data sets (describe trends over specific time periods i.e census) -individual: separate info available for each person or member; sources include administrative (claims), operational (discharge reports), analytic (cost reports), medical records, primary data from previous studies
Considerations for using existing datasets:
-consider database, study methodology, study conclusions -select most appropriate database for your question(s) -obtain a thorough description of data & choose timely data source -talk to program staff affiliated with data or attend training in use of dataset -use computer information specialist -manage missing data
Nonprobability sampling methods: p. 521-522
-convenience (most problematic; use of volunteers or easily available subjects) -purposive (deliberate selection of individuals based on predetermined criteria) -snowball/network (initially identified subjects provide names of others who may meet the criteria) -quota (used when different proportions of subject types are needed so there is appropriate representation in sample that may not otherwise by attainable)
Determining sample size p. 522
-correlational > 30 -survey > 100 in major subgroups and 20-50 in minor subgroups -causal comparative or experimental >15
Most common factors considered when determining qualitative research participants: p. 526
-culture -geographic/organizational location -time or event related experience -personal experience or unique condition
4 steps to building an interview/questionnaire p. 95
-define & clarify variables -formulate questions -format questionnaire or interview schedule -pilot & revise questionnaire or interview schedule
Secondary analysis includes both ______ & _______ designs.
-descriptive: depicts distribution & frequency of health conditions, purpose is to identify populations at high risk and formulate hypotheses for further investigation -analytical: concerned with identifying or measuring effects of risk factors or health conditions and with testing hypotheses (most common are cross-sectional, case control, cohort)
Limitations associated with using existing databases p. 123-124
-difficult to know all details about secondary data -researcher doesn't have complete control over operationalizing variables -original study may lack theoretical framework -threats to reliability & validity -selection & information bias possible -secondary researcher has no choice in instruments -data may be out of data -lacks control conceiving, generating, recording data -data may be incomplete, erroneous, misleading
Qualitative methods to ensure reliability & validity
-engagement over time (build trust, insider view) -reflexivity (self reflection and reactions of people) -triangulation (using 2 or more strategies to collect same data) -member checking (investigator returns to participants to ask if data is accurate)
Questionnaire designs should ________ and mailed questionnaires must be ________. The cover should contain the title of the study, directions (distinguished from survey questions by italics, bold, etc.), name of sponsor/institution/principal investigator and the back cover should contain no questions. p. 97
-make the task of reading questions, following instructions and recording answers as easy as possible -clear & attractive to complete
Confounding is the mixing of the effects of extraneous variables under study. It may fully or partially account for the apparent association between variables or it may mask or hide and underlying true association. What 2 criteria must a variable meet to be considered confounding?
-must be a risk factor for the outcome under study or associated with the outcome but not necessary causal -must be associated with exposure under study but not a consequence of the exposure
Observational contexts include: p. 535
-natural (qualitative) -semistructured as in clinic (often used in OT) -standardized/laboratory (highest level of control over confounding factors)
Advantages & disadvantages of observational contexts include: p. 535-536
-natural: authentic & ecologically valid -semistructured: may simulate natural environment enough to increase ecological validity, may be applied uniformly but still some risk for confounding factors -standardized/laboratory: most control of environment but limits generalizability
In many cases a researcher may access data from national organizations via the internet. To access person level data from known sources normally requires these steps:
-negotiating administrative approval for access -undergoing an IRB protocol -assuring compliance with privacy regulations
Quantitative data can fall into these 4 categories: p. 530
-nominal (categorical) -ordinal (rank order i.e. 1-never 2-sometimes, etc.) -interval (continuous) -ratio (continuous, true zero point)
There is reason to question the reliability and validity of data collection procedures when: p. 531 RIGOR emphasizes dependability of reliability & validity at start of data collection
-not sufficient previous research on instrument -sample differs from those on which instrument has been studied -an aspect of how the instrument will be used varies from its standard use -new data collection procedure has been developed specifically for the study (reliability & validity unknown)
Methods of recording information from observations includes: p. 534
-observational guides (highly structured, quant or qual) -structured checklists (paper & pencil forms to indicate presence vs. absence of something, usually quant) -quantitative rating scales (ordinal i.e. Likert) -semistructured or unstructured note taking (usually qual but may be used in quant for anecdotal support) -electronic/digital recording
Survey questions may be asked in 2 forms: p. 96
-open (open-ended to elicit more detail-may be used in pilot to generate closed questions) -closed (provide options to be considered, may prompt memory) -normally both types are used in a survey
The three largest threats to external validity are: p. 518-519
-people (unusual subjects) -place/location (area of city with higher avg income) -time (depression study right after holidays)
Advantages of case-control studies
-quick and inexpensive -appropriate for evaluating rare health outcomes or long latency periods -multiple factors can be evaluated for a single disease
How can confounding be controlled in the design?
-restriction: narrow ranges of values of potential confounders -matching: by pairs or by frequency
3 key factors that influence survey study rigor: p. 102
-sampling -response rates -how survey is carried out
3 steps to plan & implement data collection: p. 541
-select instruments & procedures -develop collection plan -select & prepare personnel for data collection
4 types of individuals in any setting that are useful to qualitative researcher: p. 525
-social gadflies (well liked & lively) -constant observers (well established group members who will speak freely) -everyday philosophers (thoughtful and provide insight) -marginal people (do not feel like they belong to group and will often talk to outsiders about group as a result)
How can confounding be controlled in the study analysis?
-stratification: partitioning data according to levels of confounder -multivariate analysis: using math models to describe the effect of exposure and other factors that may be confounding
Advantages of cohort designs
-study factor/ind variable is observed on each subject before health outcome is observed -bias such as recall bias is excluded in exposure as researcher observes development of outcome rather than using memory -can study additional associations with outcome
What should the individual who participates in a study be called? p. 516
-subject for quantitative (denotes objectivity & passive nature) -participant for qualitative -informant for those giving researcher valuable info -respondent for survey research -stakeholder for participatory research where the investigation is designed to affect lives under study
3 strengths of multiple-baseline designs:
1. because of their design, there is no threat to internal validity. 2. they require no reversal or withdrawal of the intervention 3. they are useful when behaviors are not likely to be reversible.
When determining phase lengths, what factors should the researcher consider?
1. decide whether or not he/she intends to use statistical analysis 2. in situations where change is expected due to factors such as disease progression, the researcher should consider making the lengths of baseline and intervention comparable. 3. researcher should consider the variability in the expected data and the magnitude of the expected change, in cases where there is high variability and small but significant changes are expected, longer phases are advised
What are the three common interval-recording techniques?
1. momentary time sampling 2. partial interview recording 3. whole interview recording
What are the five factors a research design depends on?
1. prevention of type I error 2. prevention of type II error 3. external validity 4. resources available to researcher 5. the theoretical/clinical importance of the research question
Sources of type II error include:
1. small sample size 2. dispersion due to individual differences 3. dispersion due to measurement error 4. dispersion due to random events 5. robustness of independent variable 6. powerful statistical procedures
_____ calls and _____ mailings are usually attempted before considering a person a non-respondent. p. 104
10 (average) up to 15 3 and cover letter (6 face to face attempts)
Because researchers don't have control over the data (generated in the past), what must they be aware of?
2 existing sources of error: bias & confounding
What is bleeding?
A participant assigned to one condition actually experiences some or all of the comparison condition.
Another variation to the A-B design is the A-B-C ____________, which is characterized by having 3 or more phases with the criterion for success changing sequentially from one intervention phase to the next
A-B-C changing criterion design
In __________ design, the baseline if followed by an intervention, followed by a second intervention
A-B-C successive intervention design
In a Single-subject research design, What does A, B and C (and all other letters) represent
A=baseline, B=intervention phase, C (and all other letters)= additional interventions/conditions
_________ designs are used to compare effects of intervention and no intervention or can be used to compare 2 distinct interventions. These designs involve fast alternation of these 2 conditions.
Alternating-treatments design
Descriptive Case Studies
*in-depth descriptions of the experiences or behaviors of a particular individual or a series of individuals *typically undertaken to describe some new phenomena or document client's response to new intervention *differ from single-subject design in that there is no experimental manipulation of an independent variable *LEAST rigorous forms of research since they lack control and generalizability *can be useful for investigating new interventions that require substantial individualization or trail and error
To obtain good quality data, it is critical that an investigator can track participants through the course of study. This information ensures that all information: pg. 556
*is collected from participants at the right times *the data gathered from each participant are handled correctly *that everyone can be accounted for at the end of the study, including the individuals who were recruited but never actually participated, those who dropped out, or those who were lost to the study (moved, died)
Typical areas to include in a job description are: pg. 556
*job title *name of title of direct supervisor *summary of job character and purpose of the position *qualifications and/or specific skills required *specific list of job duties
At the time of project planning, the following data management tools and resources will need to be decided upon, developed, or at least initiated: pg. 562
*management plan and project timelines *job descriptions *equipment and supply needs (file cabinets, software) *submissions for human subjects protection (ethics review) and if necessary, HIPAA
During project initiation, the investigator must: pg. 563
*recruit and hire individuals who meet qualifications *train or arrange training for staff on protection of human subjects, use and protection of personal health information *decide what policies and procedures will be needed, and work with staff to prepare a policy and procedure manual *set up staff monitoring procedures that can be reasonably followed (weekly meetings) *develop systems to track participant recruitment and outcomes, including how ID numbers will be assigned *review data collection tools and ensure that their design will facilitate accurate data entry *test data entry systems, including data export protocols *set up and test data storage, security, and backup protocols
Information that is gathered to monitor and manage study progress includes: pg. 548
*recruitment processes and outcomes *progress on data collection, coding, entry, & cleaning *status of data storage and security *documentation of decisions that will make it easier for the investigator to describe what he/she has done
Developmental Research... pg. 61-62
*seeks to describe "patterns of growth or change over time within selected segments of a population" *characterizes typical grown and development (crawling, walking, talking, grasping, etc.) *seek to characterize the course of disease or disability over time *usually involves a cohort design in which a sample of participants is followed over time and repeated measures taken at certain intervals *is often accomplished through cross-section design
Data management is important for: pg. 550
*supporting a culture of consistent communication *Minimizing data collection and entry errors *facilitating data analysis and interpretation of findings *maximizing data quality by promoting internal validity (credibility) and reliability (dependability) of data *ensuring data security and confidentiality *ensuring compliance with relevant laws *facilitating the preparation of the research report, including identifying strengths and limitations *facilitating data archiving and sharing
The key contents of a quantitative codebook includes: pg. 560
*the list of variables *their names and labels *the coding for the different responses *it may also include: --information on origins of items --instructions on how to complete coding --how to code unusual responses --where individual data files re kept --who maintains passwords
The ultimate goal of data management is _____________________________. pg. 549
*to support the production of findings that are trustworthy and are useful to others.
In the context of data management, consistent communication includes: pg. 551
*verbal and written communication AMONG members of the team (all members including meetings, emails, etc.) *documentation of individual work and decisions (maintenance of study notebooks, lab books, etc)
Many researchers learn about data management by: pg. 548
*watching their mentors *talking to other researchers *picking up ideas at conferences *learning from their own mistakes
_______ ordinarily involve a small group of persons who represent the range of characteristics expected in the sample and may be used to define the study or evaluate questions. p. 101
Focus groups
What kind of interviews are comprised of a set of pre-established questions that follow strict administration and scoring rules, are used in quantitative research and mostly gather nominal & ordinal data, and mostly ask participants to select from among responses provided by interviewer?
Structured interviews (minimize interviewer bias, eliminate inaccuracies in scoring and easy to administrate) p. 536
_____ refers to the effort, inconvenience, pain, risk, and other factors that affect what will be required for or may be a consequence of data collection procedures; increases with time involved in data collection. p. 544
Subject burden -if high may lead to low participation rates, high dropout rates --> limits sample size (sampling bias) ---> limits representation of population
______ is considered equivalent to random sampling as long as there is no reoccurring pattern or order (every -nth person on a list of 1500 possible participants)
Systematic sampling p. 521
Actual data collection forms should never include the participant's name, but rather only the participant identifier. T or F pg. 558
True
For some studies, data entry can and will occur simultaneously with the data collection process. T or F pg. 563
True
It is important to think of a data management system as both a product as well as a dynamic process. T or F pg. 552
True
Most of the issues for data management need to be considered during the initial proposal development. T or F pg. 552
True
T/F Latinos & African Americans with disabilities are more likely to receive fewer comprehensive services and less culturally relevant services compared to white families. p. 516
True
T/F, as opposed to efficacy RCTs, effectiveness test whether the intervention works in typical clinical conditions
True
T/F, the A-B-A- or withdrawal design has stronger internal validity than A-B and A-B-C designs
True
Team meetings are an important strategy to facilitate communication and should include everyone on research team, even it the meeting doesn't address their specific area. T or F
True
The larger the project, the more sophisticated the data management plan will be. T or F
True
T/F When analyzing existing data, an investigator may usually apply for exempt IRB approval since no new data are collected.
True -still must seek approval from original researcher and bound to same confidentiality/privacy restrictions (sign a document of agreement & Certificate of Confidentiality)
_________ error is the failure to find and report a relationship when a relationship actually exists.
Type II
________ interviews are used in qualitative research, allow the discover of new information and establishment of trust but require substantial time, interviewing skills, and contextual knowledge.
Unstructured p. 537
________ use a straight line with labels to anchor each end and subjects mark a point on the line (generally 100mm for standard measurement).
Visual analogue scales
What is the primary weakness of alternating-treatment design?
Vulnerable to validity threats, interventions can influence each other.
Steps to sampling p. 516
a literature review will define the parameters of the population that will be important for a given study
What is a filter question? p. 96
a question used to determine is the respondent has knowledge/experience of an issue before asking him to answer more questions about the issue
Multiple-baseline designs can be either a.____b. ________ or c.________
a. across behaviors,b. across participants, c. across settings.
It is important to ensure Likert scales are _______.
balanced (equal agree/disagree choices)
The researcher needs to decide whether to explore specific experiences of a large number of individuals ______ or a greater range of experiences from a smaller number of individuals ______. p. 527
breadth depth
In _________, the purpose of the design is to find out if some specific variable from the past discriminates between people who today have a disease and people who do not.
case-control design (aka case-referent design or case comparison design)
The conditions of a _______ variable are qualitatively different from each other whereas the conditions of a ________ variable are quantitatively different from each other.
categorical, continuous
In a ____________ RCT, clinical sites are randomly assigned according to the conditions of the independent variable as opposed to randomly assigning individual participants.
cluster randomized control trial
Question sequencing is important - start out with a few easy questions, followed by main study questions which should be _______. p. 99
clustered together by topic
_________ _________ are one data management tool that helps track the many variables in a project and are relevant to quantitative and qualitative studies. pg. 560
code books
Quality control procedures for entering/coding survey data include: p. 107
codes should be clear to ensure coder reliability, use trained coders, write notes about which codes they are unsure of for a supervisor to check, possibly use a program that only allows certain entries for each question or a precoded form that can be electronically scanned (may not be financially feasible)
case-control or case-history studies are true retrospective studies where the researcher:
compares cases with controls and determines the outcome of interest by looking backwards in time to determine which factors may lead to a particular outcome; also uses odds ratio
Researchers may use _______ as an important source of data to extract a wide range of pre-existing information. p. 541
document/records review -mostly qualitative research for documents beyond medical records such as diaries, educational records, etc. but becoming difficult to use because of confidentiality issues
Once data approval issues have been decided the researcher can condor how the data will be obtained & stored. Steps need to be taken to...
ensure data are handled (retrieved & stored) a minimum number of times -procedure for how data will be located, coded & entered needs to be developed
Researchers graph data on either _____ graph paper or the standard behavior chart
equal interval graph paper
Cross-sectional designs, also referred to as prevalence studies aim to...
examine the extent to which a health behavior or condition exists among some group; most common measure of association is odds ratio which requires the calculation of a confidence interval
Survey response rates from highest to lowest p. 105
face-to-face, telephone, mail -no agreed upon minimum for an acceptable response rate -face-to-face & telephone have higher cooperation rates on evenings and weekends
Generality in single subjects studies can easily be achieved without repetition of the study?
false, generality is only achieved through replication in single subject studies
Audiotapes and notes are data recorded from _______, which is a group discussion conducted by an investigator serving as a moderator who guides the discussion by introducing questions. p. 540
focus groups (5-15 participants) -explore perceptions & attitudes -data can be collected from several people at once -minority opinions may be discouraged from speaking -used in quant (pilot or preliminary) and qualitative (primary research)
Face-to-face interviews are ideal for contacting ______ populations and they are usually completed with little or no missing data. p. 94
hard to reach populations (homeless, incarcerated)
Existing data may be used to examine estimates of variables such as:
hospitalizations, frequency of rehab visits, impairment & disability, measures of cost, risk factors, SES indicators, occupational indicators -sources of existing data may be found on pgs. 118-120
Why conduct a debriefing interview? p. 102
identify how the respondent experiences the questionnaire/interview and whether anything needs to be changed to improve accuracy of data yielded
The purpose of getting a representative sample is to ________. p. 518
increase the generalizability of the study to a broader population of similar populations (external validity)
Ethical considerations p. 545-546
informed consent, risk management, limiting subject burden -also maintaining rapport through study (empathic responding) & maintaining professional boundaries -do not accept money/gifts
When discussing quantitative procedures for data collection, we use the term _______ (refers to questionnaires, checklists, rating scales, measures) and for qualitative collection we use the term _____.
instruments strategies p. 531
In ___________analysis, dropouts are sought out for outcomes testing even if they discontinued the intervention they were assigned
intention-to-treat analysis
It may be helpful to use ranges (use boundaries with traditional rounded breaking points 4, not 4.5) when asking for responses to ______ questions. p. 101
interval scale -ex. age 10-20, income 30-50k
Secondary analysis -may lead researcher to ask more specific questions, generate hypotheses, engage in prospective research
involves the use of data that were gathered for previous research studies or other primary purposes (medical records, databases from surveys
In qualitative research, data coding involves applying ____________ to sections of text. pg. 560
labels
In some studies the quality of data collection is higher when data collectors share as many characteristics in common with participants as possible including:
language, age, racial/ethnic background, disability status (can facilitate rapport) p. 545
Survey questions should be written out in full, start on the ______ margin and be indented potentially using leader dots, response categories should be presented in a ______ list as opposed to ______ format, and all parts of the same question should appear on the same page. p. 98
left vertical horizontal
What components should mail survey cover letter include? p. 105
letterhead stationery, mailing date with name and address of respondent, why the study is important, why the answer's are important, assurance of confidentiality, and whom to contact with questions/concerns
An important ________________ of correlational studies with reference to making causal assertions is that, without experimental control, there is no way to rule out the possibility that two variables might be related by virtue of their association with a third, unobserved variable. pg 62
limitation
Ordinal responses (i.e. Likert scales) should be listed in _______. p. 101
logical order (avoid confusion and easier for data processing)
Advantages & disadvantages of different methods of data collection in survey
mail, direct, online, telephone, face-to-face -see attached chart from page 93
Documents that link the participant's name and identifier should be kept at a minimum. A ____ _____ is one tool to address this data management need. pg. 558
master file
With ______, a response is recorded if it occurs precisely at a predetermined moment
momentary time sampling
In _________ design, the same treatment variable is applied sequentially to separate behaviors in a single participant
multiple-baseline design across behaviors
With ________, one behavior is treated sequentially across matched participants
multiple-baseline design across participants
With____, the same behavior or behaviors are studied in several independent settings
multiple-baseline design across settings
___________ is the increase in type I error due to multiple statistcal tests on the same participants
multiplicity
The most commonly utilized form of observation in quantitative research is ______. This involves observing subjects and recording data on the variables of interest with little to no interaction with the subjects. p. 534
passive observation
Identifier information should be maintained in a _______________-____________ file. pg. 558
password-protected
Developing and implementing backup protocols as part of data management is essential. Two basic rules of backups: do them _________ and ____________. pg. 562
regularly; frequently
More complex variables may require the development of more questions. When several questions are asked about a variable in order to form a scale, the variable can be captured with greater ______. p. 96
reliability & validity
The ultimate goal of any research is to produce finding that others can trust and use. This goal is the same regardless of whether the study is quantitative, and the findings will be judged on their __________, _________, ___________, and ________, or whether the study is qualitative, and the findings will be judged on their ________, _________, __________, and _____________. pg. 548
reliability, validity, objectivity, generalizability dependability, credibility, neutrality, transferability
The dependability of research findings are all linked to whether the data collected are _______. Data can either be _________ (numerical) or _________ (narrative). p. 530
reliable & valid quantitative qualitative
In quantitative research, the roles of the observer include: p. 534-535
remain as objective as possible, prevent artificiality due to observer presence, use caution to ensure reliability & validity
Why are demographic type questions often placed at the end of a survey? p. 99
respondents see them as invasive and don't understand the relevance of the information at the beginning of the survey -not placed at end if used to screen eligibility to answer certain questions
The basic model underlying sampling theory and inferential statistics if simple random sample with replacement which means: p. 103
returning the respondent selected to the population after sampling so that he or she has an equal chance of being selected subsequently -not practical or desirable in common survey practice so virtually all are selected without replacement (often using systematic random selection)
General principles of survey formatting
see table p. 100
This type of interview may be used in quantitative research (coded later) or for more narrative accounts for qualitative research. They use pre-established schedules of open ended questions, require a high level of clinical judgment, allow for better rapport building and an in depth understanding but are long to administrate and vulnerable to interviewer bias.
semistructured interview p. 536-537
With __________ all the individuals in a defined target population have an equal and independent chance of being selected for a sample. p. 520
simple random sampling/sampling without replacement -method used to eliminate bias that makes sample unrepresentative of population
What is the primary weakness of multiple base line design studies?
they require more data collection time because of staggered start times for the intervention phases.
T/F Telephone interviews (along with mailed questionnaires) offer an advantage over face-to-face interviews when the sample is distributed over a large geographic area.
true
T/F, social validation is important because it is possible for an intervention to result in desirable changes in the dependent variable, bust still be identified by research participants or other stakeholders as being unacceptable
true
An advantage of factorial design?
Both interventions can be tested as well as the interaction between them
__________ provides a checklist of essential items that should be included in an RCT
CONSORT (Consolidated Standards of Reporting Trials)
_____________ studies have the advantage of eliminating the effects of sample bias on observed changes, but they are subject to the influence of historical events that are unrelated to the course of development. __________________ designs avoid the latter problem but are prone to cohort effects (effects that are unrelated to age and are due to some circumstance unique to a particular age or cohort. pg. 62
Cohort Cross-sectional
This kind of assessment typically measures aspects of physical, social, educational, or work-related settings in which subjects perform daily life occupations. p. 540
Contextual/environmental assessments
_____ is the critical process of selecting and adding pieces of information through interview and observation and analyzing it for further discovery. p. 528
Domain analysis
_______ is the effect of differences between two means or the degree of relationship between two variables in the results of a study. p. 522
Effect size (the smaller the effect size, the more subjects a study will need)
__________ has to do with generalizability, whether the results of a research project can be applied to the real world.
External Validity
T/F Piloting of the questionnaire is unnecessary and often wastes valuable research time. p. 101
FALSE - it is essential before the main study to determine if respondents understand questions and can respond -3 ways: focus groups, field pretesting, individual interviews
T/F, total double blind studies are possible in OT studies
False
T/F, Single subject studies have both strong internal and external validity?
False, it is strong in internal validity but weak in external validity
______ may be used to conduct a small number of interviews, receive feedback on the questionnaire, and address unforeseen problems before the main study. p. 101-102
Field pretesting
Use more paper for an uncluttered appearance and to avoid ______. p. 97
confusion & errors
2 kinds of sampling p. 519
probability nonprobability
Characterizing a Study Sample and Characteristics pg. 59
*All investigations include components of basic descriptive design through which participants and circumstances are characterized *Basic descriptive data are always collected, analyzed, and reported in any research *It is important to know key demographic variables (age, sex, race, or ethnicity) as well as conditions that make up a study (type of service provided, length of time service was provided, etc)
Univariate investigations are typically used to: pg. 58
*Characterize the sample/circumstances that make up study *Characterize a problem or phenomena *Document incidence/prevalence of health condition *Establish norms *Document developmental phenomena *Document Case Studies
Data Management is defined as the logistical, reflective, and behind-the-scenes processes and infrastructure that allow a researcher to: pg. 548
*Produce high-quality information to address the study questions *Describe how and what has been done during a study accurately and comprehensively
Incidence and Prevalence pg. 59
*Studies that address incidence/prevalence are referred to as epidemiological studies *usually conducted by public health/med research *Prevalence is a proportion of a total population who have a particular health-related condition *Incidence is concerned with how many persons have onset of a condition during a given span of time; refers to the number of new cases of a disease or disability in a population during the specified time period *Incidence/Prevalence calculated for subpopulations as well (HIV - first prevalent in males- then changed over years to include more women and children)
Correlational Research... pg. 62
*aims to demonstrate relationships between variables under study *sometimes referred to as exploratory, since it is frequently used to identify if variables are related *can serve as helpful first steps in sorting our causal relationships *serve as precursors to experimental studies *may or may not be a temporal sequence between the hypothesized predictor variables and the dependent variable of interest
Data management will facilitate study and data archiving that will make it easier for a research team to: pg. 551
*compare its finding to other similar studies *provide data to students for thesis or dissertation work based on secondary analyses *share data with other researchers who seek to build on the work that has already occurred
Descriptive Research... Pg. 58
*depicts naturally occurring events or characteristics of research participants *common in OT *includes studies that compare groups when there is not experimenter manipulation of an independent variable *takes advantage of naturally occurring events or available information in order to generate new insights through inductive processes *is a component of all research studies, since it is used to characterize subjects and other relevant circumstances that surround the research
Characterizing a Phenomenon or Problem pg. 59
*descriptive studies in OT often serve purpose of illuminating some phenomena or circumstance that is of interest to the field (developing children and sensory stimuli) *descriptive studies are often used in OT to characterize functional aspects of a disability (what types of occ challenges are faced by certain populations)
A system for tracking participants and their data involves a serious of individuals tools which include: pg. 557
*developing and assigning participant identifiers *creating and maintaining a master file *developing and using participant ID sheets
Issues during data collection can include; pg. 563
*document methods and responses to each recruitment strategy *document the outcome for each potential participant - who is in and who is out, and why *update policies and procedures as new situations emerge *document as decisions are made (e.g. how to code unusual response)
Developing a strong data management is an administrative process that supports rigorous research and involves: pg. 551
*documenting how all forms of data and project information will be collected, handled, stored, and prepared for use *developing tools and systems to operationalize the documentation *following through on the use of these tools and systems, refining and adding to them as the project evolves over time
2 principles of sampling in qualitative studies: p. 524-525
-appropriateness (identification of participants who will best inform researcher about topic - random selection is not effective sampling strategy) -adequacy (enough data will be available to provide rich description of topic)
Survey questions should NOT contain ______ (i.e. what was your experience of OT during your last hospital admission?) or ________. p. 96
-assumptions (assumes respondent recalls OT specifically) -2 components within one question
Limitations of cohort designs
-attrition rate -changes in criteria, practice standards or methods occurring over time
Before working with a data set, to help decrease threats to reliability and validity, the researcher needs to generate:
-carefully conceptualized research problem & question -identified variables required for investigation
How do investigators choose & justify their data collection procedures when planning research? p. 531
-choose instruments previously developed & investigated -spend substantial time developing instruments
Identification of data collection procedures includes the following strategies: p. 541-542
-examining research literature -corresponding with other researchers -consulting web based & published compilations of assessments
Reliability & validity of data collection in qualitative studies centers on: p. 532
-faithfully representing experience of participants -participants believing findings are accurate -comprehensive data (all perspectives represented) -accurately reflect experience in given context
Survey questions should: p. 96
-have face validity -ask things the respondent should know answer to -be clear & unambiguous, user friendly, not offensive
Survey research is characterized by collecting data using structured questions to elicit self-reported information via questionnaires, telephone or in person interviews. It is characterized by: p. 91
-identifying population and appropriately sampling -identifying research aims/question and systematically gathering information -developing statistical estimates that can generalized
Advantages of survey research p. 91
-large # of respondents -collect data on numerous variables -statistical manipulation w/ multiple uses of data set
Limitations of case-control studies
-various inherent biases -info on risk factors & confounding variables may be difficult to identify -difficult to assess whether health conditions causes exposure to a factor or factor causes health condition -difficult to identify & assemble case group representatives and identify control group
What are the 3 sources of data in every interview? p. 537
-verbal from subject -behavioral -proxy verbal (significant others, family, friends - corroborates or fills in gaps)
The alternating-treatment design has 3 strengths, what are they?
1. It does not require lengthy withdrawal of intervention 2. It requires less time for comparison to be made because a second baseline is not required 3. It is possible to proceed without formal baseline phase
There are two types of descriptive research: pg. 58
1. Univariate Research - data are collected on a single variable or a series of single variables and then characterized with descriptive statistics 2. Correlational Research - relationships between two or more variables are examined
(Retrospective) Cohort designs are also called longitudinal, incidence, follow-up or historical cohort studies. They may be prospective as well but this chap focuses on retrospective. They aim to:
determine incidence of health outcomes in persons who were exposed versus those who were not exposed to a certain factor or factors
Calculating sample size
p. 524
Another advantage of single subject study design
Because there is only one or a few participants, financial and time demands are realistic for practice setings
_______ are objective means of assessing any range of variables that involve physical, biological, or physiological functioning. p. 541
Biometry & physiological measures OT examples: grip strength, ROM, endurance
Data Management is done in conjunction with ________. pg. 549
design
A participant ___ ___________ is one data management tool that can address the dilemma of maintaining records. A major advantage of an ID sheet is that it can easily be removed from the participants' files when the info it contains is not germane to the activities of the research team. pg. 559
ID sheet
_________ are trains that qualify someone as a subject and _____ will prohibit the subject from being a candidate for the study. p. 518
Inclusion traits exclusion traits -good inclusion & exclusion criteria are specific and clearly identified
One advantage of a single subject study design
It does not require the withholding of treatment from a no-treatment control group because each participant serves as his/her own control
The most common form of a unidimensional (self) rating scale is the _______ and it most frequently uses a five category ordinal rating system. p. 538
Likert scale (ex. in OT: Medical Outcomes Survey Short-Form 36) -another ex. semantic differential (rate a concept on a series of bipolar adjectives as in free vs. constrained)
what is the difference between post-posttests and long-term follow up tests?
Long term follow-up tests occur after a period of no intervention
A disadvantage of factorial design
Many more subjects are needed to fill up all those groups and many more things can go wrong
_______ involves seeking individuals for qualitative research who have extremely difference experiences of the phenomenon being studied (find the broadest range of experiences), while homogenous selection seeks informants with the same experience. p. 526
Maximum variation
In __________ design, the researcher matches a sample that has a hypothesized risk factor to a sample that does not. Then the researcher picks a future point in time to see if the hypothesized risk factor truly predicts the disease or disability.
Multigroup cohort design (aka Cohort analytic study)
The ___________ of 1993 ensures that there is representation of women & minorities in clinical research studies unless there is a clear and compelling rational that their inclusion is inappropriate. p. 516
National Institute of Health Revitalization Act -sampling decisions should always carefully consider race/ethnicity & gender
In a ________subjects in one naturally occurring group receive an intervention and the subjects in a different naturally occurring group receive nothing.
Nonrandomized comparison group design
_________ are less likely to yield mistakes than using check boxes when coding and processing completed questionnaires and aligning response codes along the _____ margin helps to reduce errors.. p. 99
Numerical codes right
______ observation is commonly used in qualitative research when the investigator going the subjects and participates in the same discussion or activities as the subjects. p. 534
Participant
When patient identifiable information is needed to link data from two sources for analysis, who may the researcher need to apply to?
Patient Information Advisory Group set up under Section 60 of the Health and Social Care Act
The ______ is a self report method that encourages respondents to organize data into visual categories. p. 538
Q-sort
______ seeks to eliminate bias due to difference in the groups being compared. p. 520
Random assignment (often difficult in OT studies - may identify terms of subject allocation to groups prior to selecting subjects to control selection bias)
____ is the cornerstone of quantitative research and the best method of removing selection bias. p. 519-520
Randomization -may be used for subject selection and assignment
Methods of probability sampling include:
Randomization, stratified random, systematic, cluster
________ is typically concerned with ensuring the accuracy of information collected was not affected by extraneous circumstances, while _____ is concerned that the data collected actually represents the variable under study. p. 531
Reliability validity
__________ is the capacity of he dependent variable to show small but meaningful increments of change over time.
Responsiveness
_________ occurs when individuals are selected for a study who overrepresent or underrepresent certain population characteristics. p. 518
Sampling bias
________ represents the difference between the values obtained by the sample and the actual values that exist in the population. p. 519
Sampling error -larger the sample error, the less representative the sample is to the population and the lower the external validity -due to random error (happen by chance) or systematic error (flaw in sampling process = results in subjects differ dramatically from population; happens when using volunteers or groups that are available & convenient but different from intended population)
_________ are generally administered via a written form but are increasingly administered via the computer. p. 539
Self-reports -may have added benefit of being programmed to cue person at certain times to input data
_________ tests typical generate norm or criterion referenced scores. They are widely used & accepted as rigorous measures of outcomes but sometimes are too general or broad to answer detailed research questions. p. 539
Standardized examples: aptitude, motor proficiency, cognitive performance, AMPS
_______ refers to the likelihood of finding a significant difference between groups or association between variables when one exists. p. 522
Statistical power (typically (.80) -sample size, effect size, alpha level or level of significance (typically .05 or 5%) and the power or odds of observing a treatment effect when it occurs are the 4 majors factors
_________ is similar to random sampling but the selection is from identified subgroups in population (must be proportional to actual population). p. 521
Stratified random sampling
Each of the variables should be clearly defined both ______ & ______ and the appropriate level of data (nominal, ordinal, ratio, interval) desired should be identified. p. 95
conceptually operationally
Assigning identifiers ensure that participant names are kept confidential, but they also facilitate the data collection and management process by providing a ________________ ___________ that can be used to link files across the study. pg. 557
consistent variable
Correlational studies also can provide evidence that is _____________ or ______________ with causal assertions and thus they can provide important evidence for developing theoretical propositions that assert causal relationships between variables. gp. 62
consistent; inconsistent
Advantages of secondary analysis
cost effective, decreases subject burden, useful for exploring new research areas
___________ is a design strategy to counterbalance the dependent variables so that the dependent variables are experienced in different sequences
counterbalancing
In a __________design, the purpose is to compare different types of persons in terms of some immediately measurable dependent variable.
cross-sectional design (immediate ex post facto comparisons)
In a ______design, participants are randomly assigned to groups, but each group experiences both conditions of the independent variable
crossover design
The term _________ is used to refer to the information collected from participants, and ________ _________ is used to refer to the other types of information that a data management system must track. pg. 548
data project information
The term _______ __________ primarily relates to quantitative data and involves running frequency distributions and measures of central tendency and variability on each of the individual variables in the analysis file to check for missing or out of range values. pg. 560
data checking
The process of correcting data entry errors is referred to as __________ ______________. pg. 560
data cleaning
The _________ variable is the criterion used to compare the condition of the independent variable to each other.
dependent
Limitations of budget, time, and space lead to sample size & diversity limits and undermine the integrity of the study because they fail to provide _______. p. 515
necessary rigor
For variables that are ____________ or ____________ in nature, responses must be assigned a numerical value. This process is called ____________ ____________ pg. 559 & 560
nominal; ordinal data coding
A sample selected by a _______ may be useful for an exploratory or pilot study but caution should be exercised in generalizing from a ____ to a target population. p. 102
non probability sample
A_________does not predict which condition will end up with the superior outcome. It simply predicts that one will of the conditions will be superior.
nondirectional hypotheses
Even with a relatively high response rate, there is still a potential for _______ if the reason for nonresponse is related strongly to the survey topic. p. 104
nonresponse bias
Incidence Rate...
number of observed new cases during a specified period/total person-time (incidence rate characterized the number of new cases as a proportion of persons who were actually at risk during each of the identified segments of the total period)
A participant identifier is a __________ or _______________ code that is used in place of the participant's name on all documents pertaining to that individual. pg. 557
numeric; alphanumeric (e.g. A-001)
In quantitative research, data coding involves translating participant responses into ___________ values to facilitate _____________ analysis. pg. 559
numerical statistical
Quantitative data collection that focuses on human behavior, thought, attitude, or emotions primarily uses structured methods of collecting data such as: p. 530-531
observational rating scales, self report questionnaires, structured interviews, standardized tests -OT involves functional performance data collection -sometimes use biometric (kinematics of movement, brain activity)
Easy summary chart for retrospective (analytical) studies
p. 113
The other half of the systematic & information bias chart
p. 114-115
_________excludes drop-outs, with the rationale that inclusion of dropouts only causes random error and increases the chances of type II error
pre-protocol analysis
The sampling frame must be assessed for any _______ of elements that might coincide with the selection interval. p. 104
preexisting periodic arrangement of elements -avoid sample bias
The 2 ypes of tests that are highly prone to Type I error
pretest-postest no control group design and nonrandomized comparison group design
When does the researcher estimate phase lengths during the research process?
prior to implementation of the study
Interviews for _____ data may have the researcher assume a more formal role, while _____ interviews require the interviewer to assume a less structured role. p. 538
qualitative qualitative
In what tradition of data collection does the investigator start off with a research question and data collection methods in mind and then refine the process accordingly as data are collected? p. 532
qualitative data collection tradition
In a _____ study with random assignments to control and treatment groups, you will want a sufficiently large sample size to achieve statistical significance, whereas in _____ studies you would be likely to select a small number of participants to study in depth. p. 515
quantitative qualitative
The __________test us a conservative way to evaluate shifts in time-series data and is ideal because it is robust to serial dependency (i.e. correlation among successive responses in one individual)
randomization test
A___________ is an experiment wherein an important health outcome is the dependent variable, a clinical intervention is part of the independent variable, and research participants are recruited and randomly assigned over time as they become available.
randomized controlled trial (RCT)
Single subject research is also known as...
single-system research
The ___________ technique is used often used in visual analysis and can be used to describe data and predict outcomes given the rate of change
split-middle technique
Why should investigators minimize the amount of missing data? p. 530
statistical analyses are most rigorous when there are no or few missing data points
Observational approaches may be used to collect quantitative or qualitative data. It may be used to corroborate information obtained from other measures or when participants are unable to provide reliable self reports. Information typically sought includes: p. 534
subject characteristics, behavior, communication,
________ refers to those who are identified as eligible to participate in the study. p. 515
subject pool
Semantic differentials, Q-sorts, and visual analogue scales are typically used in the assessment of _______. p. 538
subjective experiences (difficult to describe verbally)
In qualitative research, the role of the observer involves: p. 535
sustaining interaction with participants over extended period of time to build trust and become familiar with phenomena being studied
The _____ is the population to which the researcher wants to generalize his or her intended findings. p. 517
target population (the sample would come from a subset of the target population) -- ensures validity
Cost of survey research
telephone more expensive than mail but cost less than face-face
Why/when do researchers use unstandardized questionnaires? p. 538-539
when existing measures do not address variables of interest and a new measure must be generated; potential to provide rough preliminary info
Qualitative data seeks to understand & report human behavior and experience, thus the concerns for data dependability are... p. 532
whether the data faithfully represent the circumstances as they are experienced by those under study (trustworthiness of data)
