FCS 5310 Research

Data Collection: Focus Group Duration

- 90 minutes to 2 hours

Data Collection: Focus Group

- About 6, but never more than 12 participants - Researchers will still collect some quantitative information from patients, even if just participant characteristics. - Researcher serves as observer and guides discussion while recording and taking notes - Uses focus group guides

Pragmatism

- Arises out of actions, situations, and consequences - These researchers focuses on the research problem and question and use all approaches available to understand the problem - Commonly a worldview for mixed methods researcher - Includes different aspects of worldviews

Ethical issues to anticipate Before Study

- Association ethics standards (ex. Dietetics code of ethics) - Gain IRB approval - Gain approval from participants and site - Sites may have conflicted interests - Negotiation of authorship for publication

Research Problem

- Comes from a void in the literature - Issue or concern that needs to be addressed - Topics that have been neglected in the literature

Convergent Design

- Include both quantitative and qualitative data collection and analysis methods(no specific order), merges results from both methods and then interpret results to compare - First analyze the qualitative data, then analyze the quantitative data, last used a mixed methods data analysis - Joint Display: final procedure involves merging the two forms of data in a table or graph

Characteristics of Mixed Methods

- Involves collecting both quantitative and qualitative data, integrating the two forms of data, and using distinct designs that may involve philosophical assumptions and theoretical frameworks - The integration of qualitative and quantitative data yields additional insight beyond ability of either quan/qual data solely

Post-positivist

- More quantitative than qualitative - Want to focus on cause and effect; scientific method - Deterministic - Reductionistic → reduce the ideas into small, discrete variables that are testable - Tests theories that have already been developed

Ethical issues to anticipate: Collecting data

- Must respect site and disrupt as little as possible - All participants must receive same treatment - Deception - Power imbalances and exploitation of participants - Participants might be considered "used" - Harmful information collection

Exploratory Sequential

- Phase one: Qualitative data collection and analysis - Phase Two: Identify Feature For Testing (new instrument, new experimental activities, new variable) INTEGRATION - Phase Three: Quantitatively test the Feature designed - Then interpret results and discuss how the test improves the results

Explanatory Sequential

- Phase one: Starts with quantitative data collection and analysis, then identify results for follow up; - Phase two uses qualitative data collection and analysis; then interpret results and explain how qualitative better explains quantitative data - INTEGRATION comes from qualitative data and results explanation of initial quantitative results

Survey

- Same thing as a questionnaire - Provides numeric description of trends, attitudes, or opinions of a population - May be descriptive questions, questions about the relationships between variables, or questions about predictive relationships between variables over time - Different ways to administer: interviewer administered, participant administered, online, mailed, in-person, telephone, app, etc.

Ethical issues to anticipate: Analyzing data

- Siding with participants - Disclosing only positive results - maintenance of privacy of data

Transformative worldview

- Strives to advocate for underrepresented populations - Address issues surrounding social justice, discrimination, and opression - Are usually critical theorists (feminists, marxists, racial and ethnic minorities, etc.) - Intertwined with politics and political change → participatory action - Participatory action = participants may help to design questions, collect data, analyze information, or reaps the benefits of the research - Welcomes collaboration

Inferential Stats

- Trying to reach conclusions that extend beyond collected data a. Types of statistical tests that allow you to assess the relationship between variables (correlations, regressions, etc) i. Risk Ratio ii. Odds Ratio iii. ANOVAs iv. MANOVAs v. t-test vi. Factor analysis

Social Constructivists/ Constructivists

- Typically seen as qualitative rather than quantitative - Seek to understand more about the world - Focuses on complexity rather than narrowing to single idea - Relies on the participants' views regarding the situation being studied (usually use open-ended questions to learn participant's views) - The views of participants are usually formed through a historical and cultural construct.

Descriptive Statistics

- Used to describe the basic features of data in study; Provide information about data being analyzed; described by type, distribution, location, variation in categories as nominal, ordinal, continuous. a. Data is typically grouped as low, medium, and high. May include frequencies (how many males and females enrolled in study), means of standard deviation,

What is multi-methods?

- Uses different research methods within a single research approach (compatible data collection or analysis methods) (e.g. using observations and in-depth interviews in the same study → both more qualitative research methods) - Within methods triangulation - Part of the development of mixed methods research

Under which conditions might a qualitative research approach be more useful than a quantitative approach?

- When there is not much known about the research topic. The purpose is to explore. - To gain an understanding of opinions, motivations, or underlying reasons for an occurrence - To understand or explore a complex situation - To develop a theory

Data Collection: Observation and Two Types

- research takes notes on behavior and activities at research site(s) - Participant: Have participants observe and then discuss - Nonparticipant: Someone from the research team goes somewhere to observe the environment

Validity

Accuracy

Theory

An underlying explanation of the way that variables relate to one another

Characteristic of Quantitative Research

Approach for testing objective theories by examining the relationship among variables Final report has set structure Data is analyzed using statistical software/procedures Deductive Closed ended questions and responses Controlling (alternative explanations are controlled; controlled for bias) Focused on generalizable results Uses instruments to measure variables (scales, surveys)

External validity

Assuming there is a relationship in this study, do conclusions hold for other populations, settings, times (contexts)? Gets at generalizability; threats to it: When experimenters draw incorrect inferences fro the sample data to other persons, other settings, and past or future situations

Internal validity

Assuming there is a relationship in this study, is the relationship a casual one? Cause & effect; Threats to it are: experimental procedures, treatments, or experiences of the participants that threaten the researcher's ability to draw correct inferences from the data about the population in an experiment

External validity threats: Interaction of Selection and Treatment:

Does the program's impact only apply to this particular group, or is it also applicable to other individuals with different characteristics?

Internal Validity threat: History (unplanned events that occur during the intervention that may influence the outcome

Evaluators should do their best to identify any external events or changes that may impact their program results (media coverage, policies, major events, etc.); can be reduced by limiting follow up time.

Characteristics of Qualitative Research

Exploring and understanding the meaning individuals/groups ascribe to a social or human problem Flexible structure Emerging study design Inductive Open ended questions and responses Observing Not focused on generalizability Researcher is one of the main instruments

Generalizability

Extension of research findings and conclusions from a study conducted on a sample population to population at large

Ethical issues to anticipate: Reporting, sharing, and storing data

Falsifying authorship, evidence, data, finding, and conclusions - Plagiarism - Disclosure of harmful info - Being vague - Duplicate publications - Conflict of interest and ethical compliance needs to be reported - Must state who owns data

Sample

Group of people objects, or items that are taken from a larger population for measurement. Should be representative of population to ensure generalizability

External validity threats:- Interaction of Testing and Treatment:

If your design included a pretest, would your results be the same if implemented without a pretest? - Interaction of Setting and Treatment: How much of your results are impacted by the setting of your program, and could you apply this program within a different setting and see similar results?

Ethical issues to anticipate: Beginning

Need research problem that will benefit participants - disclosure of study purpose - Pressure participants for informed consent - Must be sensitive to the needs of vulnerable population

Respect for persons

People are autonomous individuals (free will and agency), people have the right to make their own decisions, vulnerable populations need more protection

Describe and explain how to develop an interview (or focus group) guide or protocol.

Steps: o Determine your population ▪ Who can provide rich data about the topic you are looking for and who is willing to spend the time in an interview o Determine the research questions that will be used ▪ Open ended questions ▪ Arrange questions from broad to narrow ▪ Questions should be simple and asked one at a time ▪ Ask "how" questions instead of "why" questions → this gets at processes ▪ Develop probes → elicit more detailed and elaborate responses; provide the researcher with more details or for more information ▪ Have opening questions that put the participant at ease o Create an introductory and closing paragraph ▪ Introduction: describes the purpose of the study and will prompt the researcher to collect informed consent ▪ Closing: thank the participant for their time and wrap up any final questions/remarks

Internal validity

The cause and effect relationship within a study (can you rule out alternative explanations)

External validity threats:- Multiple Treatment Threats:

The program may exist in an ecosystem that includes other programs. Can the results seen with the program be generalized to other settings without the same program-filled environment?

Inferential statistics

Trying to reach conclusions that extend beyond collected data; more than one variable at a time

Construct a statement about how a researcher would use a theoretical lens in a mixed methods study utilizing a social science framework.

Understanding changes needed for a marginalized group Researchers studying food insecurity on campus may already have an idea of how it affects students negatively. This is likely to affect the type of data they collect and how they ask their questions? (just a guess)

informed consent

an ethical principle that research participants be told enough to enable them to choose whether they wish to participate

research question

an interrogatory statement describing the variables and population of the research study

Reflexivity

analyzing and critically considering our own role in, and effect on, our research

Convenience sample

anyone that is willing to participate in a study (standing outside of HEB and asking people to participate)

External validity

are the outcomes of your study generalizable? Are they accurate when applied to other settings - example: Is the sample an accurate representation of the population?

Data Collection: Focus Group Strengths

can gather a lot of information from multiple people in a short amount of time

Data Collection: Focus Group Limitations

can sometimes be too many people to get a true understanding of a situation (cannot go as in-depth), at risk for social acceptability bias

Continuous variable

can take any numerical value and be measured; can be meaningfully divided into smaller increments

Causation

casual relationship (requires much more evidence) comes from experimental designs

Confounding Bias

caused by mixing the effect of the exposure of primary interest with extraneous risk factors (confounding variables/confounders)

Deductive

based on logic, preconceived idea about the relationship of variables; developing a hypothesis based on existing theory, then designing research strategy to test hypothesis

Selection Bias

bias introduced by the selection of individuals, groups or data for analysis in such a way that proper randomization is not achieved, thereby ensuring that the sample obtained is not representative of the population intended to be analyzed

Internal validity threats: participant attitude/motivation

blind participants to the study hypothesis

Population/target population

broader group of people to whom you intend to generalize the results of your study

Secondary data collection

data that is collected by someone other than the user. Common sources of secondary data for social science include censuses, information collected by government departments, organizational records and data that was originally collected for other research purposes.

Descriptive notes

describing what you are observing. This includes describing sites, smells, noise, time, setting, etc.

Grounded Theory

design of inquiry from sociology in which the researcher derives a general, abstract theory of a process, action, or interaction grounded in the views of participants. This process involves using multiple stages of data collection and the refinement and interrelationship of categories of information. Inductive approach → developing a theory following observation

Beneficence

do good, do no harm, the potential benefits should outweigh the potential harms, participants have the right to be aware (need to be fully informed of the study to make an autonomous decision)

Data Collection: Observation Strength

does not rely on the accounts of other individuals, can collect data in the environment it is occurring, do not have to rely on people's willingness to participate

Research Methods

involve the forms of data collection, analysis, and interpretation that researchers propose for their studies

Reflexive Notes

involves examining and consciously acknowledging the assumptions and preconceptions you bring into the research and that therefore shape the outcome → writing down what you are thinking and feeling during an observation; How are your experiences influencing your observations

Longitudinal study

involves repeated observations of the same variables over short or long periods of time

Test-retest reliability:

is used to assess the degree to which the participants' performance is repeatable, i.e., how consistent their sum scores are across time

Briefly explain qualitative analysis with coding and interpretation.

● Finding themes ● Takes a long time ● Often multiple places ● After coding, look for patterns and overarching themes ● Researchers will develop a code book based on existing literature and by looking through the data to identify themes → this code book will be used to code themes within participant transcripts → inputting these codes into a qualitative software will allow for researchers to look through transcripts with more ease

a. As a qualitative researcher, the person doing the study becomes the instrument of inquiry. How would you propose to discuss the researcher's role in a qualitative study?

● Subjectivity ● This requires preparation and training for researcher ● It is important for the researcher to be aware of their own perspective and worldview they bring to the research as these things affect how you collect, analyze, and interpret data

Operational variable

refer to how you will define and measure a specific variable

Reliability

refers to whether scores to items on an instrument are consistent and stable over time; Consistency within test administration and scoring

theoretical lens

requires one to develop culturally relevant programs by using resource models such as the precede/proceed model

Primary Data Collection

research approaches, contact methods, sampling plan, research instruments

Experimental study

researcher manipulates one variable (IV) to test its effect on another variable (DV) and controls for all other possible affecting variables

Dependent variable

response variable/outcome

Inductive

starts with observations and theories proposed toward end of search for patterns from observation and development of theories

Random sample:

subset of individuals randomly selected by researchers to represent an entire group as a whole

Cultural humility

the ability to maintain an interpersonal stance that is other oriented (Open to the others norms/customs) in relation to aspects of cultural identity that are most important to the person

Cultural competence/competency

the ability to understand, communicate with and effectively interact with people across other cultures

Ethics

the principles of right and wrong that guide an individual in making decisions

Data Collection: in-depth interview Limitations

time consuming (interview and transcribing interview), need more training (researcher must learn how to conduct an interview, potential participant bias depending on recruitment

Internal validity threats:instrumentation (changes in the instrument, observers, scorers)

training on how to use instruments and conduct many pre-test collections

Confounding variable

variable that affects the dependent variable that is not the independent variable

Independent variable

variable that is manipulated in experimental studies; a variable those variation doesn' depend on another

Cross-sectional study

-Type of observational study that analyzes data from a population or subset at a specific-point in time

a. What are >3 reasons for doing mixed methods research? Mixed methods can be used for many purposes.

1. Developing better measurement tools 2. Understanding experimental results by incorporating perspectives of individuals 3. Developing an understanding of needed changes for a marginalized group 4. Understand the need for an impact of a program, intervention, or policy

External validity threats:Interaction of History and Treatment

: An oversimplification here may be to say how "timeless" is this program. Could you get the same results received today in a future setting, or was there something specific to this time point (perhaps a major event) that influenced its impact.

Mediating variable

A variable that is influenced by the independent variable and inturn, influences the dependent variable (transmit the effect of an independent variable on a dependent variable)

Longitudinal survey:

Collects data at multiple points in time b. Strengths: Looks at trends; High validity c. Limitations: Time-consuming, high risk for dropout/attrition; Less reliability; More expensive; Typically require large sample size d. Useful for wanting to learn about sequential events that lead to an outcome of interest

Experimental Design

a. Participant and design: Describe the procedures for recruiting participants and selection process used b. Procedures & measures: discuss the instruments used and the experimental design c. Analysis: what type of statistical analyses will be used (report descriptive statistics, indicate what inferential statistics will be used)

Descriptive statistics

Used to describe the basic features of data in study; Provides simple summaries about sample and measures used; one variable at a time

Ethnography

a design of inquiry coming from anthropology and sociology in which the researcher studies the shared patterns of behaviors, language, and actions of an intact cultural group in a natural setting over a prolonged period of time. Data collection often involves observations and interviews. Inductive approach → using observations to develop a theory or describe participants views

Bias (from an epidemiological/biostatistics perspective)

a systematic source of error

Cross sectional survey:

a. Collects data at one point in time b. Strengths: Allows researchers to look at several variables at once; inexpensive; fast; less respondent burden, good for generating hypotheses c. Limitations: Does not get at cause and effect, subject to bias (many confounding variables; recall bias; social acceptability bias); cannot study rare diseases or diseases with short time span d. Useful when wanting to explore relationships between hypothesis-generating studies, mechanisms, trends, and patterns over time; Useful in epidemiological research (looking at risk factors and disease prevalence) e. Example: NHANES

Justice

fairness to all people without discrimination, equity (everyone has access to what they need → picture of people with a bike that fits their needs)

Data Collection: in-depth interview Strengths

gather rich information and gain an understanding of individual perspectives, interviewers can establish rapport with participants

Information Bias

happens when key information is either measured, collected, or interpreted inaccurately.

For experimental designs, know identify weaker (not recommended) and stronger (recommended) study designs (e.g., post-test only, pre-test/post-test, pre-test/post-test with comparison group).

i. (In order of strength) Pre-test/post-test with comparison group → pre-test/post-test → post-test

Ways to work ish with vulnerable populations

i. Individuals from vulnerable populations may have a reduced ability to make decisions about research participation and may have increased risk associated with research participation ii. Researchers may conduct observational research with these populations as a way to minimize risk for participants; Researchers may also develop a research plan that describes how they will attempt to minimize risks to this population; Can develop an informed consent process that is rigorous and fully explains the research study (using visual aids, consent document with lower-literacy, verbally explain informed consent document); Researchers can consistently ask for consent throughout the research process and not just at the beginning of a study/interview

a. Review process for creating a visual model of theory

i. Latent constructs will be represented by a circle and observed variables represented by squares ii. arrows indicate direction of relationship iii. Moderating variables are indicated by drawing an arrow pointed at the arrow between the independent and dependent variables iv. Each factor = simple shape (boxes >ovals) v. reads left to right vi. Upstream factors to the left, Downstream to the right

Pre/Post Test

i. Measures are taken before and after a treatment is implemented; includes only 1 group; No comparison ii. Strengths: Provides at least some evidence for effect; More internal validity iii. Limitations: Hard to interpret; Less external validity (not generalizable) iv. Useful when: Ethical reasons RCT cannot happen; Conducting an educational intervention (school) or with psychological test (used in a counseling setting)

Quasi-experimental

i. No randomization to groups ii. Strengths: can evaluate an intervention in vulnerable populations iii. Limitation: Lack of randomization (not as strong of a research outcome), may introduce more confounding variables, cannot infer causality iv. Useful when: cannot randomize groups (dietary intervention you know will help participants → e.g. must determine who will receive nutrition education via intervention and the rest of the participants still receive the education as a handout); Experimenting a life saving cancer drug (All participants may receive the experimental drug)

RCT

i. Randomly assigns subjects to more than one group ii. Strength: The gold standard for determining effect, less confounding variables due to design iii. Limitations: Expensive, time-consuming, invasive, cannot answer all research questions with RCT because of specificity required iv. Useful when: Testing efficacy or effectiveness of an intervention, treatment, or exposure

a. What are the characteristics of mixed methods designs (in general)?

i. Uses a blend of quantitative and qualitative designs, depending on the goals of the research

a. What is integration? What is the benefit of "mixing" qualitative and quantitative data?

i. comparing different perspectives drawn from quantitative and qualitative data ii. explaining quantitative results with a qualitative follow up data collection and analysis iii. developing a more complete understanding of changes needed for marginalized groups iv. evaluating both the processes and outcomes of a program, experimental intervention, or policy decision.

Conceptual variable

ideas about what needs to be measured

Association

non-causal relationship; general relationship between two random variables

Purposive sample:

non-probability sample that's based on characteristics of a population and objective of study

What are strategies researchers can use in qualitative research to support the reliability of their conclusions?

o Training o Interview guide o Constructive criticism o Review transcripts, coding, and research methods o Check transcripts to make sure no mistakes o Consistent coding ▪ Cross-check codes developed by diff researchers o Coordinate communication among coders ▪ Regular documented meeting ▪ Share in analysis

What are strategies researchers can use in qualitative research to support the validity of their conclusions?

o Triangulate different data sources of info by examining evidence from the sources and used to build it into a coherent justification for themes. o Use member checking to determine accuracy of qualitative findings; ▪ Researcher takes parts of polished product to determine if participants feel they are accurate o Use rich, detailed description to convey findings o Clarify bias researcher brings to study o Present negative or discrepant info that runs counter to the themes o Spend prolonged time in field, more exposure to participants, more accurate findings will be o Use peer debriefing o Use external auditor

Data Collection: Observation Limitation

observer bias, time consuming, "hawthorn effect" (people perform better when they know they are being observed), may not increase understanding of the reasoning behind peoples' behavior

Data Collection: in-depth interview

one single participant - Intentional conversation that covers topics and includes probes to obtain rich data about perspective, experience, etc. - Verbal exchange in which an interviewer attempts to acquire information and gain an understanding of interviewee - Semi structured interviews/in-depth interviews - Can be face to face, phone, or in person

Research Approach

plans and procedures for research that span the steps from broad assumptions to detailed methods of data collection, analysis, and interpretation

Who are people of vulnerable populations

prisoners, pregnant women, impoverished individuals, elderly, infants and children, persons with impared decisional capacity, disadvantaged groups; They have decreased freewill to make informed choices and have decreased capability for self-protection against risk

IRB Purpose

protects human participants' rights by reviewing and approving research procedures to ensure that the study minimizes risk to the participants and that the researchers do not harm people; Required an approved informed consent document that informs the participant of the research procedures

IUCAC Purpose

protects research animals from undue harm; oversees the care and wellbeing of animals used for research purposes and ensures that researchers are complying with legal standards set forth by the Animal Welfare Act

Sampling strategies List

random sample purposive sample convenience sample

Internal validity threats:selection bias

randomization