Exam 2 methods of research

adjudication of outcomes

self report outcomes must be confirmed, create clear criteria for the outcome and blind treatment assignments

individual data sets

separate information is available for each subject: previous research studies, medical records, health care billing files, death certificates, large regional and national data sets

importance of blinding

serves to control co-interventions and biased outcomes, difficult to do sometimes due to techincal or ethical reasons such as difficult when assigned to educational dietary or exercise interventions

choice of outcome measurements in rct

should include several outcome measurements to increase the richness of the results and possibilities for secondary analyses, clinical outcomes provide the best evidence about whether and how to use treatments, intermediate markers- measurements that are related to the clinical outcome, can be considered surrogate markers for the clinical outcome, number of outcome variables- desirable to have several outcome variables that measure different aspects of the phenomena of interest

interviewing

skill of interviewer can have a substantial impact on quality of responses(need for standardizing the interview, should be written in language that resembles everyday speech, sometimes necessary to follow up or clarify meaning, probing can be standardized), can be conducted in person or by phone, computer assisted telephone interviewing, allows immediate range checks for values, disadvantages(must have phones, must be willing to be called)

research using existing data

speed and economy, settle for a variable that is not what would have been a preferred measure, quality of data may be poor, important confounders and outcomes might not have been recorded, investigator has little or no control over the data

conflict of interest

stock ownership, patent/licensing rights, speakers bureaus, consulting and advisory boards

payment to research participants

subjects deserve compensation for time and effort and for out of pocket expenses, also may be needed to enroll and retain subjects, payment raises ethical concerns about undue inducement to participate in research

exempt from IRB review

surveys, interviews or observations of public behavior unless subjects can be identified, disclosure could place subjects at risk for legal liability or damage their reputation, financial standing or employability, studies of existing records, data or specimens provided that samples exist and are publicly available, subjects cannot be identified, research on normal educational practices

elements of informed consent

this is research not medical care, research is always voluntary, the study will involve specified procedures, there are risks, there may or may not be personal benefit, there may be other options to consider, we will protect your confidentiality, well tell you about your rights related to injury, you can always ask questions, you are free to withdraw at any time, we may use your data for other future research

within group designs

time series design- single non randomized group with outcomes compared within each subjec during periods of different interventions, subject serves as own control to evaluate effect of treatment; disadvantage- lack of a concurrent control group, learning effects(participants do better on follow up tests because they learned from baseline), regression to the mean due to random variation, secular trends

best practices for survey

train and supervise interviewers, provide specifications, conduct demonstration interviews, involve real interviews, make it easy, be persistent, incentives

ethical issues with randomized clinical trials

treatment is determined by chance, ethical basis for assigning treatment by randomization is the judgement that both arms are in equipoise, principle of nonmaleficence- problematic to withhold from the control group therapies that are known to be effective

ecologic fallacy

type of faulty reasoning that sometimes is made in the interpretation of results that come from the analysis of aggregate data, mistake occurs when data that exist at a group or aggregate level are analyzed and interpretations are then made as though they automatically apply at the level of the individuals who make up those groups, results are more convincing when an association found in one data set can be duplicated in another collected using a different approach or different patient population

observational studies

typically designed to suggest a predictor may be a cause of outcome, can provide insights into the underlying biology of a disease, however not every association represents cause and effect there are other potential explanations such as chance(random error), bias (systematic error), effect-cause and confounding

IRB

a diverse group of scientists and non-scientists who are responsible to protect the rights and welfare of human subjects involved in research projects, review approve require modification or disapprove all research activities involving the use of human subjects, review proposed changes in approved research, review ongoing research at least annually to ensure appropriate protections for subjects

effect cause

cause-effect in reverse, real association, the outcome has caused the predictor, often a problem in cross sectional and case control studies, when no previous values are avaiable, case crossover studies if the timing of events is uncertain, less common in cohort studies, draw inferences from assessments of variables at different points in time, often unlikely on grounds of biologiv implausibility, reverse causality

vulnerable populations

children, pregnant women, fetuses, prisoners, cognitively impaired persons

systematic reviews

combine the results of multiple previous studies that have addressed a given research question to calculate a summary estimate of effect, identify completed studies that address a research question and evaluate the results of these studies to arrive at conclusions about a body of research, must include a complete written protocol before the study begins, protocol should include research question, methods for identifying all eligible studies,, methods for abstracting data from the studies, statistical methods, statistical aspects of systematic review is called meta-analyses, results can be no more reliable than the quality of the studies included

blocked randomization

commonly used technique to ensure number of subjects is equally distributed among study groups, randomization is done in blocks of predetermined size

equivalence trials

compare a new intervention to an existing standard of care, design may be the most ethical and clinically meaningful, often requires a larger sample size than placebo controlled trials

subgroup analyses

may be possible using data from all or some subset of the studies included in the systematic review, prone to producing misleading results due to subgroups being smaller than the entire trial population, may not be sufficient power to find important differences, investigators often examine results in a large number of subgroups increasing the likelihood of finding an effect by chance(planned subgroup analyses should be defined before the trial begins, test interactions for statistical significance to minimize misinterpretation)

IRB issues

may place undue emphasis on consent forms and fail to scrutinize the research design, review of scientific merit sometimes beyond the expertise of the IRB members and reviews left to funding agency, do not check whether research was actually carried out in accordance with approved protocols, sometimes lack the resources and expertise to adequately fulfill mission of protecting research subjects, IRB approval should be regarded only as a minimal ethical standard for research, judgement and character of the investigator most essential

scientific merit review

merit review ensures the feasibility and value of the research, procedures consistent with sound research, protection against unnecessary risk for subjects and researchers, ability to answer the research question, potentially important results

elements of a good systematic review

preferred reporting items for systematic reviews and meta analyses(PRISMA) is a tool to structure the reporting of searching and study selection; the research question, identifying completed studies, criteria for including and excluding studies, collecting data from eligible studies, presenting the findings clearly, meta analysis: statistics for systematic reviews

authorship

publication of scientific papers results in prestige, promotions and grants, eager to receive credit for publications, less eager to take responsibility for problems with published articles, honorary authorship- only trivial contributions to the paper, ghost authorship- authors who made substantial contributions to the paper but are not listed as authors, medical journals have set criteria for authorship( must make substantial contributions to the conception and design of the project or the data analysis and interpretation and the drafting and revising of the article

questionnaires vs interviews

questionnaires generally more efficient and uniform, interviews are best for complicated questions that require explanation or guidance, choice is dictated by practical considerations

federal criteria for approving human subjects research

risks to subjects are minimized, risks to subjects are reasonable in relation to anticipated benefits, selection of subjects is equitable, informed consent will be sound and appropriately documented, researchers will monitor the data to ensure safety, privacy and confidentiality will be maintained, additional safeguards will be included for vulnerable subjects

to improve adherence and follow up

screening visits(exclude participants who cannot meet hurdles), placebo run-in period, active drug rather than placebo for the run-in period(select subjects who tolerate and respond to intervention)

randomized trial

select a sample from the population, measure baseline variables, randomize participants, apple the interventions(one should be a blinded placebo), measure outcome variables during follow up(blinded to group assigments)

when to use close ended questions

when you want data that is rated or ranked, want subjects to use a prespecified set of response choices, prefer subjects answer in their own words, prefer to count the number of choices, researcher will report statistical data

Ruling out bias in the design phase

do the samples of study represent the population of interest, where did you get your study sample, do the measurements of the predictor variables represent the predictors of interest, do the measurements of the outcome variables represent the outcomes of interest

non randomized between group designs

does not control for influence of confounders

expedited review

IRB may allow certain research to undergo expedited review: single reviewer rather than a full committee, DHHS lists types of research that are eligible for expedited review, concept of "minimal risk"

ancillary studies

add measurements of a small number of variables to a study, often in a subset of the participants to answer a separate research question, investigator adds a small number of measurements to an existing study to answer a different research questions( advantages- fewer constraints, can be added to any type of study, banks of stored specimens; disadvantages- obtaining permission and separate informed consent, may be difficult to obtain access to the full database, authorship), can answer research question with little cost and effort, however cannot control the design but can specify a few key additional measurements, opportunities for these studies may be found in cohort studies or clinical trials

disadvantages to matching

additional time and expense, sampling strategy that is irreversible could build error into the study, need the appropriate matched analyses(especially multivariable techniques less familiar to most investigators), possibility of overmatching occurs when the matching variable is not a confounder

factorial design

allows 2 independent trials to be carried out for the price of one, limitations- possibility of interactions between the effects of treatments on the outcomes, reduce the power of these comparisons(only half of subjects could be included in each analyses), same study subjects must be appropriate for each intervention, multiple treatments may interfere with recruitment and adherence; allows experiments to have more than one independent variable, 2x2

between group designs

always include a group that receives an intervention to be tested and another than receives no active treatment(placebo) or a comparison treatment

frequency

always, very often, fairly often, sometimes, almost never, never

coping with confounders in the analysis phase

analytic methods for assessing the inderpendent contribution of predictor variables in observational studies: stratification, statistical adjustment, propensity scores

propensity scores

analytic technique that can be useful in observational studies of treatment efficacy, tool for controlling confounding by indication. predicted probability of confounder is calculated and assigned to each study subject or subjects can be matched by scores and outcomes compared between matched pairs

visual analog scale

another option for recording answers to close ended questions using lines or other drawings, subjects asked to mark a line at a spot, along a continuum that best represents response, lines are often 10cm long, score is distance from the lowest extreme; advantage- attractive because they rate characteristics on a continuous scale, ay be more sensitive to change than ratings based on categorical lists; some participants may find it difficult to understand, important to explain, give examples of how to answer the question

plagiarism

appropriating another persons ideas, results, or words without giving appropriate credit

adverse effects

ask broad, open ended question questions about all types of potential adverse effects

Belmont report

autonomy- informed consent, protection of vulnerable subjects, beneficence- maximize benefits while minimizing risks, justice- equitable selection

choice of control in rct

best control group receives no activve treatment in a way that can be blinded( compensates for any placebo effect), cleanest comparison between intervention and control groups occurs when there are no coninterventions, often not possible to withhold treatment, give standard care to all( increase the sample size required, minimizes potential differences in cointerventions, test whether the new intervention improves outcome when given in addition to standard care)

influence

big problem, moderate problem, small problem, very small problem, no problem

wording

can influence validity and reproducibility of the responses, must construct questions that are simple, free of ambiguity and encourage accurate,honest responses, clarity- concrete words over abstract preferred, simplicity- use simple, common words, avoid technical jargon, neutrality- avoid loaded words and stereotypes or questions that suggest there is a most desirable answer, sometimes useful to set a tone that permits the subjects to admit to undesirable behaviors, avoids biased words

randomizing

cornerstone of a clinical trial(factors which could confound an observed association will be distributed equally), consider special randomization techniques such as blocked randomization or stratified blocked randomization

equipoise

current evidence does not prove that either arm is superior

selecting the participants in rct

define the entry criteria, for rare events necessary to recruit subjects who are high risk, must have reliable estimates of the rate of the primary outcome; exclusion criteria due to unacceptable risk of placebo assignment, active treatment cannot ir is unlikely to be effective, unlikely to adhere to the intervention, unlikely to complete follow up, practical problems with participating in protocol; design an adequate sample size

endorsement

definitely true, true, dont know, false, definitely false

measuring baseline variables in rct

describe the participants, measure variables that are risk factors for the outcome or factors that can be used to define subgroups, measure baseline value of the outcome variable(must be measured in the same way that it is measured at the end), be parsimonious(focus on factors that are vital to the integrity of the study), establish banks of materials

strengths of survey research

describes the characteristics of a large population too large to observe directly, makes findings more generalizable, enables analysis of multiple variables, uniform measurement; good surveys- maximize response rates, obtain valid information, obtain reliable information, must ask good questions

specification

desing inclusion criteria that specify a value of the potential cnfounding variable and exclude everyone with a different value, disadvantage is that is limits generalizability, must not control too many or too narrowly

systematic reviews

different than literature reviews(literature reviews are not guided by transparently reported methodology and methods- carry an increased risk of bias because of subjective decision making, limited searching, how it was performed), research question meets the finer criteria in systematic reviews, identifying completed studies( comprehensive and unbiased search for completed studies, criteria for inclusion/exclusion(good rationale, established a priori, potentially eligible studies should be reviewed independently by 2 or more reviewers, publish reasons for excluding a study)

scales and scores to measure abstract variables

difficult to quantify abstract concepts, commonly measured by generating scores from a series of questions that are organized into a scale, likert scales are commonly used to quantify attitudes and behaviors, can compute an overall score by summing the scores for each item(composite score), internal consistency of a scale can be tested statistically, cronbachs alpha measures how well a set of items or variables measures a single unidimensional latent construct, when data have a multidimensional structure, cronbachs alpha will usually be low. technically speaking, cronbachs alpha is not a statistical test, it is a coefficient of reliability or conisistency, cronbachs alpha calculated from the correlations between scores on individual iteams, values of above .7 are usually acceptable, .8 or more is excellent, .5 or below unacceptable, lower values for internal consistency indicate that some items may be measuring different characteristics

avoid pitfalls

double-barreled questions( each question should contain only one concept), hidden assumptions(make assumptions that do not apply to all people), the question and answer do not match(need for grammatically correct question and responses)

advantages of matching

effective way to prevent confounding by constitutional factors, used to control confounders that cannot be measured and controlled in any other way, increase the precision of comparisons between groups, sampling convenience to narrow down a large number of potential controls and making it easier to select controls in a case control study

federal regulations

ensure that clinical research is conducted in an ethically acceptable manner, apply to all federally funded research in US including FDA in support of a new drug or device, can access regulations at OHRP or DHHS, regulations provide 2 main protections for human subjects IRB approval, informed consent

IRB mission

ensure that the research is ethically acceptable, welfare and rights of research participants are protected

cross over design

features of both within and between group designs; advantages- participant serves as own control,minimizing potential for confounding, increases statistical power, good choice when number of subjects is limited and carryover effects are judged not to be a problem; disadvantage- duration of study doubled, added complexity of analysis and interpretation by problem of carryover effects(can introduce a washout period)

stratified blocked randomization

further restrics chance imbalances, set of permuted blocks is generated for each combination of prognostic factors, having separate block randomization schemes for each combination of characteristics

monitoring clinical trials

harm and clear benefit- conditional power(likelihood of answering the research question given the results), stopping a trial- statistical tests, analyzing the results of a trial repeatedly increases probability of type 1 error- false positive(need to adjust for interim monitoring, adjusting alpha level)

methods of administering questionnaires

in person interview( standardizing the interview procedure from one interview to the next is key to reproducibility, best for complicated questions, probing must be standardized), questionnaires(generally more efficient and uniform way to administer simple questions, more economical than interviews), both affected by social desirability, susceptible to imperfect memory, computer assisted telephone interviewing, emailed questionnaires(advantages over snail mail provide data without a visit to office, disadvantages must have access and familiarity with the internets, questionnaires on websites/handheld electronic devices(clean data because answers can be automatically checked for missing and out of range values, errors pointed out to respondent, responses accepted only after errors corrected), direct measurement(accelerometry, sensors, blood levels of nutrients)

why do people participate in surveys

incentives, commitment, social proof group pressure, liking- comply when we like someone, authority, scarcity

dose response relationship

increasing levels of exposure are associated with either increasing or decreasing risk of the outcome

sensitivity analysis

indicate how sensitive the findings of the meta analysis are to certain decisions about the design of the systematic review or inclusion of certain studies

aggregate data sets

information is only available for groups of subjects, major advantage is availability, major drawback- associations are susceptible to confounding, groups tend to differ from each other in many ways, not all of which are causally related, associations don't necessarily hold for the individual, most appropriately used to test the plausibility of a new hypothesis or to generate new hypotheses

conflicts of interest

interests that might impair objectivity and undermine public trust, types of conflict of interest: dual roles for clinician investigators, financial conflicts; minimize conflicts of interest by blinding, independent DSMB, peer review process; disclose competing and conflicting interests, ban certain situations that lead to conflicts of interest

Informed consent

investigators must disclose information relevant to potential subjects decision about whether or not to participate in research: nature of research project, procedures of the study, risks and potential benefits and alternatives to participating in the study, procedures to maintain confidentiality, assurances that participation is voluntary, consent forms( avoid complicated sentences, should consider whether participants understand disclosed information), subjects who lack decision-making capacity( permission to participate should be obtained for a legally authorized representative, protocol should be subjected to additional scrutiny) additional requirements are to ensure that vulnerable people are not taken advantage of in research

consent process

involves a personal conversation, typically involves a written document(fact sheets, letter of invitation, formal consent document)

steps in assembling instruments for the study

make a list of variables(predictors, outcomes, potential confounders), prefer existing measures if suitable( assemble a file of instruments that are available for measuring each variable, focus on major variables, may be necessary to delete, change, add items), compose a new instrument if necessary( 1st draft should include more questions about the topic than will be included in the final instrument, check for clarity and how a particular question might be misinterpreted, identify words that can be confusing or misunderstood, avoid jargon, complex questions might need to be split into 2 or more questions simplify, ask experts in questionnaire design to review), revise and shorten the set of instruments for the study(usually collect more data than will be analyzed, creates immense respondent burden), pretest(test for instrument clarity and timing, conduct pilot studies), validate(choose questions that have face validity, establish content and construct validity, compare with gold standard or other validated instruments, validation process is time consuming and expensive)

fabrication

making up results and recording or reporting them

falsification

manipulating research materials, equipment or procedures or changing or omitting data or results to that the research record misrepresents the actual findings

IRB approval

risks to participants are minimized, risks are reasonable in relation to anticipated benefits and importance of knowledge that may be expected to result, selection of participants is equitable, informed consent sought from participants or their legally authorized representatives, confidentiality is adequately maintained

close ended questions

more common, form the basis for most standardized measures; produce standardized data than can be analyzed statistically, ask subjects to choose from one or more preselected answers; advantages: quicken, easier to analyze, answers have a better chance of being more reliable or consistent over time, list of possible answers helps clarify the meaning of the question; disadvantages: do not allow subjects to express their own answers, potential responses may not include subjects real response( can be minimized by conducting a pre-test), when a single response is desired, set of possible responses should be mutually exclusive; when the question allows more than one answer, allowing subjects to mark all that apply not ideal, does not force subjects to consider individual responses, may include an answer that does not apple to subjects overlooks an item, better to ask subjects to mark each response as yes or no

comparison

much more often than others, somewhat more than others, about the same as others, somewhat less than others, much less than others

choice of intervention in RCT

must consider several issues: intensity, duration and frequency of the intervention(keeping in mind safety and effectiveness), feasibility of blinding, whether to treat with one or combination of interventions, generalizability to the way the treatment will be used in practiceness, balance between effectiveness and safety, may decide to compare several promising doses in one study, combination of drugs or therapies

IRB members

must include community members and individuals knowledgeable about legal and ethical issues concerning research, KUMC human subjects committee includes researchers, community members, including representatives from faith organizations and individuals with legal training

intensity

none, very milk,milk,moderate, severe

spurious associations

not a real association, association is due to change or bias, to reduce random error increase sample size and precision, calculate p values and confidence intervals, interpret them in the contect of prior evidence; to reduce systematic error consider differences between the research question and the study plan(population/subjects, phenomena/meausrements), obtain additional data to see if potential biases have occurred, check consistencies with other studies,

ruling out bias in the analysis phase

obtain additional information to estimate the magnitude of the potential bias, use multiple controls. blind interviewer, use a different measure, look at other studies

respect for persons

obtain informed consent, protect participants with impaired decision making capacity, maintain confidentiality

HIPAA permissions

obtain written authorization from the patient or meet one of the following criteria: de-identified data, HIPAA waiver, limited data set or data use agreement, activities that are preparatory to research, research on decedents

group or cluster randomization

permits efficient studies of naturally occurring clusters, units of randomization and analysis are groups not individuals(public health approaches)

ethical concerns of previously collected data

potential for significant discoveries, offers no physical risks to participants, requiring consent can make the research impossible to carry out or prohibitively expensive, breaches of confidentiality may occur, religious reasons, do not want gene identification for predisposition of certain diseases or behaviors, genetic research- privacy must be respected, breaching confidentiality is justified only when the breach allows 3rd parties to take actions that are highly likely to reduce a significant risk of serious harm

stages in testing new therapies

preclinical-studies in cell cultures and animals; phase i- unblinded, uncontrolled studies in a few volunteers to test safety, phase ii-small randomized blinded trials to test tolerability and intensity/dose, phase iii- large randomized blinded trials to test the effect of the therapy on clinical outcomes, phase iv- large trials or observational studies conducted after the therapy has been FDA approved to assess rate of serious side effects and evaluate therapeutic uses

setting the time frame

questions about average behavior can be asked in 2 ways: asking about usual or typical behavior, counting actual behaviors during a period of time, usual or typical behavior: report things most commonly done, ignore extremes, choosing the time period involves trade offs: best length of time depends on the characteristic, focusing on recent and brief period of time, may improve subjects recall and ability to average, however may not be typical(holidays), when asking about actual behavior, the longer the time period, more difficult to remember, tend to bias their answer toward recent modes of behavior

formatting(to improve flow of survey)

questions concerning major subject areas should be grouped together, begin with emotionally neutral questions, more sensitive questions placed in the middle, questions about personal characteristics at end of instrument, for each set of questions, instructions must indicate clearly how to respond, possible answers to closed ended questions should be lined up vertically and preceded by boxes or brackets rather than open blanks, may wish to follow up certain questions with more detailed ones, best accomplished by branching question( subjects answer initial question to determine if they are directed to answer additional questions or skip ahead to later questions, branching saves time and allows subjects to avoid irrelevant or redundant questions), online surveys(generally clearer and easier for respondents to complete, use skip logic, must be carefully validated during pretesting phase)

justice

requires that the benefits and burdens of research be distributed fairly(vulnerable populations should not be targeted for research if other populations would also be suitable), requires equitable access to the benefits of research

principle of beneficence

research design must be scientifically sound, risks of research be acceptable in relation to likely benefits, screen potential participants to exclude those likely to suffer adverse effects, use fata from procedures that are carried out in course of clinical care, monitoring participants for adverse effects

evidence favoring causality

results are consistent in studies of various designs, strength of the association smaller p values reduce likelihood of confounding, dose response relation provides positive evidence for causality, biologic plausibility-important consideration for drawing causal inference

formatting of questionnaires

use a cover letter to describe purpose of study, how the data will be used, use examples on how to complete the survey

guidelines for asking closed questions

use a meaningful scale(one that makes sense in terms of the surveys objectives), balance all responses(balanced with 2 endpoints mean the opposite of one another and the intervals between the points on the scale are about equal, use a neutral response category only if it is valid(neutral category is either a middle point or no opinion or don't know, some researchers believe that neutral choices provide subjects with an excuse for not answering, pretest), use 5- to 7-point rating scales(adequate for majority of surveys that use ordered responses, self administered questionnaires and telephone interviews should probably use 4 or 5 responses, in person interviews should use visual aids for scales with 5 or more points on them), put the negative end of the scale first(especially for questions that are potentially embarassing or about socially undesirable behaviors or attitudes, very important in face to face interviews, less important in anonymous and mail surveys, if the survey deals with a problem that the subjects thinks is important, direction of scale may not count at all), keep questionnaires uncluttered and easy to complete(tell subjects how and where to mark responses, emphasize any special tasks or requirements in the question, highlight important aspects within the question, avoid questions with skip patterns in self administered questionnaires, organize responses so that they are readable)

secondary data analysis

use of existing data to investigate research questions other than the main ones for which the data were originally gathered, greatly reduces the time and cost of doing research, provides little of no control over the study population, design, or measurements, large community based data sets are useful for studying the effectiveness and utilization of an intervention and discovering rare adverse events

meta analysis

use of statistical methods to summarize the results of these studies, calculating summary effect estimates and variance, statistical tests of heterogeneity, statistical estimates of publication bias, summary effect estimate and confidence interval-provide the best estimates of treatment effect based upon all available evidence, tests for examining heterogeneity- examine if the results of the individual studies are similar enough to combine, heterogeneity of treatment effect refers to systematic differences between the results of studies that cannot be attributed simply through chance), assessment of publication bias- this is one of the most important biases which occurs in systematic reviews, assessment of publication bias- this is one of the most important biases which occurs in systematic reviews(funnel plots)

matching

used in case control studies, selects cases and controls with matching values of the confounding variables, preserves generalizability, usually done individually(pairwise matching), can also do frequency matching matched as a group for each level of a variable the number of those with that amount could be counted, can also be used in multiple cohort designs

open ended questions

useful when it is important to hear what subjects have to say in their own words, often used in exploratory research, can form the basis for more structure items later, disadvantages: require qualitative methods to code and analyze responses, more time, subjective judgement; advantages: useful in getting unanticipated answers, describing the world as the subjects sees it rather than how the researcher does, sometimes subjects prefer to state their views in their own words, sometimes provides quotable materials

pilot clinical trials

vary from a brief test of feasibility to a full scale study with hundreds of participants; focused primarily on feasibility, time required, cost(recruitment, willingness to accept randomization, comply with protocol), provide parameters to allow accurate estimation of sample size(effect size- the effect of the intervention on the main outcome, statistical variability, "screening bias" likely occurs because persons who fit the enrollment criteria for a clinical trial and agree to be randomized are healthier than general population- crucial to determine the rate of the outcome in the placebo group), follow up and adherence to protocol, if substantial number of subjects do not adhere to protocol or are lost to follow up results likely to underpowered or biased, important to achieving complete follow up)

confounding

when a 3rd factor causes an association, real association, a variable that is associated with the predictor variable and a cause of the outcomes, confuses the interpretation

when to use open ended questions

when subjects own words are essential, when subjects capable to provide answers in own words, prefer to ask only open questions because the choices are unknown, researcher has skills to analyze subjects comments, can handle responses that appear infrequently, researcher will provide individual or grouped verbal responses