Research Methods Quiz2

Next steps after accumulating processed data

"Blocks" of information may be compared across individuals, or groups of individuals; compared with other blocks; and ultimately, themes will emerge

Key-informant interviews

-An interview conducted with individuals considered to be influential, prominent, and/or well-informed people in an organization or community; or -Individuals who are selected from a larger sample for more in-depth discussion

Focus-group interviews

-An interview platform wherein a group of people are able to discuss their varied perspectives on a given topic, fostering expressions of their perspectives through the creation of a supportive environment -Typically a homogeneous group with a specific area of experience, between 4 to 12 participants per group with interviews lasting 1 to 2 hours

Participation

-An overall approach to inquiry and a data-gathering method; demands firsthand involvement in the social world chosen for study -Not to "become" a member, but to experience first-hand reality for informants without preconception -Data is collected from journals and field notes of reflections and observations

In-depth interviews

-In this research method, the purpose is to probe the ideas of the interviewees and obtain the most detailed information available regarding a particular event, issue, or context -Studies vary with respect to their a priori structure and in the latitude the interviewee has in responding to questions

four main methods for gathering qualitative data

-Participation in the setting -Direct observation -In-depth interviewing -Analyzing written documents and material objects

Interviewer training

-Technical skills on how to ask the interview questions and the use of follow-up probes to elicit more detailed explanations -Appropriate behaviors to gain access to a population -Interpersonal behaviors to demonstrate during recruitment and data collection

Observation

-This research method requires careful watching, listening, and recording of events, behaviors, and objects in the social setting chosen for study -When exclusively using observation methods, the investigator's role is a passive, unobtrusive presence -Participation and observation studies involve both roles to some degree, as well as further immersion into a phenomenon's setting

Steps of sampling

1) define population of interest 2) consider the unit of analysis 3) identify a performance site 4) develop a sampling approach 5) implement sampling procedure

Social desirability bias/Hawthorne effect

A bias that occurs when subjects change their behavior because they know they are being observed (can occur during participation or observation)

Meta-Analysis

A comprehensive study that summarizes the findings of several individual studies to answer scientific (and clinical) questions. Includes statistical data that reflects a best estimate of the cumulative evidence for the explanation of a mechanism or outcome of study. Represents the highest level of evidence in support of a phenomenon or outcome, particularly when conducted on randomized experimental trials. (different than systematic review)

2)Data collection [ma]

A comprehensive, sensitive, and extensive bibliographic search strategy is developed to compile possible reports Key words and variables determine sources of potentially relevant reports The inclusion criteria are defined and eligibility criteria are set for the meta-analysis Construct definitions distinguish relevant from irrelevant studies

Nonexperimental designs

A correlational study that is purely descriptive or observational, in which variables that are thought to be associated with one another are analyzed statistically (Shadish, Cook, & Campbell, 2002) Many threats to validity exist

Reflexivity

A deliberate and systematic process of self-examination used in qualitative research -In observational and participation studies, these are the only data that will be collected and analyzed -The reactions and views of the qualitative researcher have the potential to color the data collection and analysis processes

Probabilistically equivalent

A descriptive distinction of two or more groups, which are not equal in terms of all variables; however, any differences between the two groups are likely to be based on chance rather than on some systematic pattern of variation

Prolonged engagement in the field

The period of time spent in the field observing the phenomenon of interest, which varies according to: The nature of the inquiry and its scope The design of the study The time available to the investigator The time available to the research participants themselves Ideally, this should continue until saturation: The point in the data collection period when the researcher is gaining little or no new information

What are the steps involved in conducting a meta-analysis?

The steps involved in planning, conducting, and reporting a meta-analysis include: -Problem formulation, including identification of the problem and formation of the research questions -Data collection, through a comprehensive, sensitive, and extensive search strategy, including keywords, inclusion, and exclusion criteria -Data evaluation and coding, including assessment of included studies, data identification and quantification, and recognition of characteristics of interest -Analysis and interpretation, including statistical procedures and interpretation of results -Reporting results into a final report

Phenomenology

The study of individuals' own unique, first-person, conscious experience.

Triangulation

The use of two or more strategies to collect and/or interpret or analyze information, increasing the accuracy (or trustworthiness) of data gathered [May utilize peer debriefing] A process by which multiple analysts may discuss areas of disagreement and controversy, allowing for contrary views to arise and receive careful review

Theory building

This allows the construction of explanations that are grounded in the qualitative data and include mechanisms for defining relationships among data elements, including original text, codes, memos, and documents Metadata elements can be imported and linked to one another through semantic relationships, which can evolve into formal analytic products such as conceptual models

Tagging and retrieval

This creates hypertext "tags," whereby noncontiguous text or images are electronically linked to one another Links can also be made between one text segment and another, either within a document or between one document and another This retrieves all text segments labeled with a particular code, so that passages on a given topic or theme can be grouped and viewed together

Codebook development

This provides support for the creation of a list of codes to be applied to textual data, as well as a code directory with code definitions or criteria Some software has advanced codebook features, with separate fields for the code name, a brief definition, a full definition, inclusion criteria, exclusion criteria, and examples

Interrater reliability

This typically requires separate and redundant coding of data The coding may then be: Analyzed through quantitative statistics for agreement between the two sets of codes Reconciled through discussion and consensus of investigators Pre- and post-reconciliation statistics may be calculated for the coding to determine interrater reliability

quasi-experimental design

a study in which subjects have controlled exposure to an iv; however they are NOT RANDOM to a condition group (they remain in single group or assigned to a group according to criteria)

Management of bias

being aware of: -Overreliance on accessible research participants or favoring of more dramatic events and statements involving research participants and the context of study -Biasing effects produced by the presence of the investigator in the research site, that is, the Hawthorne effect -Biases stemming from the influence of the participants and the research site on the investigator

threats to external validity

biased sample regarding treatment nonreplicable study/lab context nonreplibacle events experienced by participants

Step 2 of sampling: unit of analysis

classification level of the individual variable of interest -a subject(child, patient, etc), setting(clinic, hospital), or cohort(family, class)

threats to construct validity

confounding variables expectancy effects interaction effects testing anxiety

Step 1 of sampling: define pop of interest

facilitated through literature review research problem will help set parameters for pop of interest

threats to internal validity

history maturation practice effects regression to the mean selection bias attrition measurement approach containment effects ethical/social confounds related to inequalities

random assignment vs random selection

random assignment: random allocation and participants are unaware of group they are in random selection: random allocation and participants are aware of the group they're in

Hawthorne effect

refers to a type of reactivity in which individuals modify an aspect of their behavior in response to their awareness of being observed. (most challenging threat to internal validity)

examples of validity

statistical conclusion validity internal validity construct validity external validity

experimental design

subjects are assigned to one or two or more condition groups (thereby have controlled exposure to the iv) through RANDOM assignment

Step 5 of sampling: implement procedures

subjects should be recruited according to the plan established during the development of approach

research problem

succinctly stated, derived from practice, informed by the existing scientific literature, innovative and of high potential impact

statistical conclusion validity

the extent to which an iv and dv are related coronary. relevant measure include: effect size, statistical power, statistical significance

construct validity

the extent to which generalizations may be made regarding a particular concept or phenomenon based on what is being observed. includes the degree to which a construct is operationalized in psychometric research. a cause and effect relationship demonstrated between an intervention and an outcome in a study.

external validity

the extent to which we can generalize an inference to different persons, settings, and times

internal validity

the extent to which we may infer that: the explanation for a particular phenomenon is true a specific action/intervention has caused effect highly dependent on research design

Operational multiplicity

the fact that a particular concept or construct may be operationally defined in a variety of ways across different studies

Retaining voice

the informants' own words and voices (through the original data) should be utilized to communicate the themes as much as possible What is heard and thought by the researcher is also important and should be closely represented

Step 3 of sampling: identify performance site

the location in which a research study is done direct impact on study and sampling approach

Transcribing

the process of exactly converting audio information into a written record; hand-written notes should be entered into the record as well Transcriptions should record laughs, hesitations, interruptions, emphases on particular words and phrases, and other speech patterns as much as possible, alongside the actual words of the conversation

research question

the specific knowledge that is being sought through a study. should be narrowed, focused, and specific. intend to generate knowledge to help close gap within scientific literature

Types of qualitative designs

Case-study—A study that examines the effects of an intervention on a single individual over time with no control Grounded-theory—A theoretical understanding of a phenomenon by extracting meaning from social interactions (Miller & Fredericks, 1999) Phenomenology—A method of investigation in which an individual's significant subjective experience is described through first-person narrative, without reference to theory or knowledge from other discipline perspectives Ethnographic designs—A method of investigation that examines the societal and cultural context that shapes meaning and behavior for a cultural group, from an insider perspective, according to the perspectives of members of that group

Interpretive approach

Categorizing the meaning of words/topics used

Numeric approach

Counting the frequency of occurrence of a topic/word

specific aims

ways in which investigators will approach knowledge discovery within a study. derive from research question(s) and are similarly precise. describe how study will approach the research question, the value, and impact. should be clear and explicitly stated in proposal/paper

Project-based methods

Daily operational procedures that help to minimize errors in all processes related to data collection, data storage, and data management

Metadata

Data about data, including new text and/or graphic products created by the researcher to represent key themes, constructs, and relationships that emerge from and are applied to a body of qualitative data

Choosing an appropriate research design depends on multiple criteria:

Knowledge—The extent of knowledge of the participants and problem under study Impact—The anticipated value, impact, and sustainability of the research question, hypothesis, or proposed outcome Feasibility—The practicality and feasibility of locating and retaining participants in the study Environment—The available environment/s for research Resources—The available resources (money, staff, and time) for research Mentor approval—The importance of adequate support from mentors, collaborators, and institutions involved Time frame—The acceptability of the expected time frame for completion of the study (to the participants, the researchers, the mentors/collaborators/involved institutions, and the funding agencies) Safety and ethics—Assurance that the design will support a safe and ethical process for those involved

4)Analysis and interpretation [ma]

Statistical procedures Interpretation of results

What are the key features of qualitative analysis?

The process of qualitative data interpretation and analysis involves examination of words, thus standing in contrast to the objective quantitative approach toward analysis of numerical data. Qualitative analysis is inductive, allowing research findings to emerge from the raw data that has been collected without the restraints imposed by stricter methodologies or pre-established theories. The goal of qualitative analysis is not to come up with conclusions that could be generalized to populations, but rather one that describes a particular sample or small group in-depth. Qualitative analysis begins during data collection and continues after data collection is completed, in a back-and-forth pattern. This analytic tradition creates a process that may be seen as both flexible and rigorous.

What are the four types of validity to consider when choosing a research design?

A goal of any research study is to answer a question of interest while maintaining the highest level of validity feasible. Validity establishes the degree of rigor with which a study creates evidence, which may truthfully and accurately capture or measure a phenomenon of interest. The primary types related to choosing a quantitative research design include: Statistical conclusion validity—This is the extent to which an independent variable and dependent variable are related or covary (Shadish, Cook, & Campbell, 2002). Internal validity—This is the extent to which we may infer that (1) an explanation for a particular phenomenon is true or (2) a specific action or intervention has caused an effect (Shadish et al., 2002). Construct validity—This is the extent to which generalizations may be made regarding a particular concept or phenomenon based on what is observed in an actual research study. External validity—This is the extent to which generalizations may be made regarding an inference to different persons, settings, and times (Shadish et al., 2002).

Computer-assisted qualitative data analysis (CAQDA)

A high-tech (i.e., computerized) method for qualitative data analysis Allows for high-technology data storage and management (as opposed to low-technology data storage and management) where data is managed/organized via computerized software

Member checks

A process whereby investigators check out their assumptions and emerging interpretations about the data with the original stakeholders who provided the information

Proximity connectors

A search logic that analyzes the spatial relationships between coded segments

Boolean connectors

A search logic that uses a form of algebraic terms to connect search terms, allowing more control over the information retrieved than when using individual search terms

Descendency search

A search that uses author names and highly similar study topics to build a compendium of related studies

Ancestry method

A search where the reviewer retrieves information by tracking citations from one study or research report to another

Semantic connectors

A semantic relationship indicator that specifies a hierarchical or other type of relationship between codes

Index codes

A set of conceptual labels that indicate places in a text where one can retrieve information on a particular topic. (then complied to code directory)

one-group posttest-only design

A single group of participants are assigned to a treatment group, with measures being taken after the intervention is administered to determine if there is an effect on the desired outcome

Audit trails

A systematically maintained set of documentation; it allows a researcher the means of managing record-keeping and encouraging reflexivity about a project and its goals, which typically includes: All data generated in the study Explanations of all concepts and models that shaped the study design Explanations of procedures used in data collection and analysis Notes about technical aspects of data collection and analysis as well as decisions taken throughout the study to refine data collection procedures and interpretations Personal notes and reflections Copies of all instruments and interview protocols used to collect study data

1)Problem formulation [ma]

A topic that can be addressed by meta-analysis is selected A research problem must be formed, including: Definitions of relevant conceptual and operational variables A rationale for relating the variables to one another The independent variable or treatment must first be defined conceptually Conceptual definitions must be linked to empirical reality A clearly stated research question, including detailed operational descriptions of how the concepts will be defined, must be proposed (this will determine what studies will be included)

When calculating a sample size, which of the following statements is incorrect regarding the relationship among variables?

A. As the sample size increases, the statistical power increases. B. If variation in outcome decreases, the statistical power increases. C. If variation in outcome increases, the sample size needs to decrease.*** D. If the effect size decreases, the sample size needs to increase.

Which of the following would notbe included as a threat to construct validity?

A. Expectancy effects B. Interaction effects C. Maturation D. Testing anxiety

Theoretical frameworks often serve as the basis for developing research questions, specific aims, and hypotheses. Which of the following would not be an outcome of explicitly stating the theoretical basis for a study?

A. It would allow for logical deduction of expected observations and study predictions. B. It would allow for assumptions to be made within a study that have previously been validated through existing practice theory evidence. C. It would ground the statistical precision of a study's methods. D. It would produce research that is increasingly logical and transparent. ANSWER C

(rp) Step 3: Compare and contrast data sources utilizing a particular code

After reading and coding one particular data source and selecting one code, compare and contrast other data sources utilizing that code (cross-case analysis) The comparison and contrasting allows for clarifying what is meant by a particular data unit or coded block of text Repeat the process, comparing and contrasting each data source to others until all cases in a sample have been analyzed

An inductive process:

Allows research findings to emerge from the raw data that has been collected, without the restraints imposed by stricter methodologies or preestablished theories

The limitations of using meta-analysis include:

Alternate conceptualizations of included treatment variables may rival the one offered Quantitative synthesis of results from multiple studies may create an illusion of statistical objectivity that is not justified by the data obtained from the review The potential for nonindependence of hypotheses may affect the results of inferential statistical procedures used to analyze the data The role of inferential statistical procedures in the data analysis stage of meta-analysis is controversial

Unstructured interviews

An interview guided by a short list of general questions, wherein the interviewer respects how the interviewee frames and structures the responses, providing probing questions when necessary

Semistructured interviews

An interview guided by a structure, using a combination of fixed-response and open-ended questions

Structured interviews

An interview, wherein the interviewer asks questions exactly as written, which may include fixed-response and open-ended questions

Written reports should include:

An introduction to the topic or group under study, including a discussion of the research purpose and the relevance of the research A literature review or "background" section, illustrating past research on the topic, including a discussion of any gaps in this past literature that might be filled by the current analyses. A methods section that details the sample, data collection and processing procedures, analytic strategies, and potential biases/limitation of the research A findings section detailing one or a few broad findings and all the specific nuances found within these findings; for both broad and specific sets of findings, numerous examples of original data should be presented and explained in full A discussion and/or conclusion section, summarizing what was found in the results and the implications of the findings Any tables, charts, graphs, or other appendices that make the discussion of methods or findings clearer to the reader References to past research on the topic

Determining the audience

General audiences will focus on findings and themes Academic/research audiences will have an interest in the methodology and rigor

Why is it important that a research question be grounded in a theoretical framework?

Given the general rule that research should be grounded in theory, the respective attributes of a research question should appropriately reflect that theory as it serves as a foundation for the methodological approach of the study. The use of an established theory allows for logical deduction of expected observations and study predictions that may be made from a given research question. Furthermore, the concepts and postulates of that theory will allow for assumptions to be made within a study that have previously been validated through existing practice theory evidence. These studies will have a broader impact, given the larger body of evidence upon which they are built, and are likely to be more logical and transparent given the established theoretical principles employed.

Assuming you have unrestricted entry into a community-based setting, what data collection approach would be most likely to yield detailed, accurate data about the lived experiences of participants?

If both the setting and the resources for a study allow for participant observation, then likely no other data collection method would produce such rich and detailed information. Participant observation allows qualitative researchers to see how things really are (observation) and also check in (participation) with knowledgeable insiders who can confirm, or disconfirm, the researchers' emergent insights, understandings, and explanations as they experience the social phenomenon firsthand.

Assuming you have an unlimited amount of time to spend with participants who are highly talkative and have the capacity for self-reflection, which approach to interviewing would be the most desirable as a data collection approach? Which would be the most desirable in a study with limited time or less-talkative participants?

If circumstances afford an investigator with an unlimited amount of time to spend with participants who are highly talkative and have the capacity for self-reflection, then an in-depth interview would be an ideal data collection approach. This allows an investigator to obtain the most detailed information possible about the phenomenon of interest, which may be a particular event, issue, or context. By contrast, if time is limited or participants are less talkative, a focus group might be an appropriate design. This allows an investigator to gather data in a short amount of time, and the group format allows participants to hear one another's responses and contribute their own responses in light of what others have said in a supportive environment.

What are the main considerations in choosing an appropriate design for a given study question?

In choosing a research design, an investigator must consider several factors, such as the existing extent of knowledge regarding the topic, the anticipated impact of the study, the feasibility of obtaining/maintaining a sample, the availability of the study environment, any needed resources, the availability of mentor support/approval, the feasibility of the time frame, and the safety and the ethics involved in the study. An investigator must consider any and all of these factors for choosing a study design, as different designs will allow for different amounts of accommodation for these various issues. The ultimate goal is to choose the most rigorous methodology that may be accomplished within the scope of these design considerations. Ultimately, this will afford the study more potential for validity within its methodology and findings.

Coding and retrieving

In order to ensure that the quality of information is maintained during a qualitative study, the process of coding and retrieving requires: The creation of a code directory with clear operational definitions Coding, analyzing, and/or interpreting the data, which must take place in a systematic and consistent way

What are the key methodological considerations in quantitative versus qualitative approaches to sampling?

In quantitative research, the main concerns of sampling are whether the sample represents the target population, whether compared groups are equivalent, and whether sample size is large enough to achieve statistically significant results.In qualitative studies, the researcher seeks participants who will best inform the researcher about the topic under inquiry and purposefully samples until the topic is saturated. The number of participants is not as critical as selecting participants who can ensure richness of information and depth of understanding.

How does a specific aim differ from a research question? Provide an example of a specific aim.

In research, specific aims are derived from research questions, and specify the purpose of a study and how it intends to accomplish what it proposes to do. These aims should be clear and succinct, explicitly stating how methods and evidence collected with a study will serve to address the specific research question. Inherently, they describe the methodological approach of a study. Examples from the text include: "To test the effects of an intervention that helps families manage distressingbehaviors in family members with dementia" (Gitlin, Winter, Dennis, Hodgson, &Hauck, 2010, p. 1465). "To investigate the effects of a token—a behavior therapy technique for controllingdrooling in children with cerebral palsy and mild intellectual disability" (Sethy,Mokashi, & Hong, 2011). "To follow the rehabilitation outcomes of war fighters who sustained combat amputations in Operation Enduring Freedom or Operation Iraqi Freedom"(Melcer, Walker, Galarneau, Belnap, & Konoske, 2010). "To provide a description of the experiences of dinnertime and bedtime routines and rituals in Australian families with a young child with an Autism Spectrum Disorder (ASD), as well as common challenges experienced" (Marquenie, Rodger, Mangohig, & Cronin, 2011).

Why is it important to define inclusion and exclusion criteria for one's research pool beforebeginning a study?

Inclusion criteria are traits that a researcher has identified as characterizing the population. They serve as the criteria that qualify someone as a subject or participant in the study. Exclusion criteria are characteristics that will prohibit a subject from being an appropriate candidate for the study. These are typically factors that could potentially confound the results of the study (Portney & Watkins, 2008). Careful consideration must be given to both of these sets of criteria, as they will quite literally define the sample that may be collected for a research investigation. This decision should be well informed by existing literature within a given topic, building upon that evidence and reducing the potential for study limitations.

Sample approach (subjects who are actually participating) Sample size- the # of participants

Inclusion criteria—Traits that the researcher has identified as characterizing a population Exclusion criteria—Characteristics that will prohibit the subject from being an appropriate candidate for the study subject pool-possible candidates

Advantages of Meta-Analysis

Increased statistical power Obtaining an estimate of the magnitude of experimental effects Greater insight into the nature of relationships among variables The ability to objectively explore contradictions in a group of studies

What are the advantages and disadvantages of meta-analysis as a research method in occupational therapy?

Light and Pillemer (1984) have identified four specific advantages in the use of meta- analysis to integrate studies: They produce increased statistical power by creating a much larger sample of study. They produce an estimate of the magnitude of experimental effects. They produce greater insight into the nature of relationships among variables. They objectively explore contradictions in a group of studies. Meta-analyses also have limitations, which must be recognized when applying their findings to occupational therapy practice. Meta-analysis involves assumptions that, if not made clear, may lead readers to misinterpretation. Variance among treatment variables for included studies may threaten the conceptualizations utilized. Further, readers may have a false sense of statistical objectivity, rather than one indicated by data obtained through the review. Finally, nonindependence may affect the results of inferential procedures used.

Methodological actions to enhance data trustworthiness include:

Management of bias Interviewer training Prolonged engagement in the field Reflexivity Triangulation Member checks Audit trails Project-based methods

Memoing

Memoing includes notes and memos through which investigators can record impressions about the data, methodological insights, and developing interpretations or theories These can be ordered, sorted, and displayed in special ways

What are the role and purpose of meta-analysis in OT research and practice?

Meta-analysis represents the highest level of evidence with regard to the evaluation of a phenomenon or outcome. Evidence-based practice among clinical and medical professions (including occupational therapy) relies on means by which research may be translated into practice. Toward this effort, a meta-analysis is a comprehensive study that summarizes the findings of several individual studies to answer these questions. This outcome may validate (or invalidate) a proposed intervention, which is critical for determining accountable therapeutic practices. Validated, and thereby evidence-based, interventions not only demonstrate proof to justify best practices for clients, but also communicate efficacy among allied health-care professionals and third-party reimbursement agencies

Memos

Methodological and analytic notes from which explanations and findings are built should be written within the data during the active reading process as analytic memos: Reflective notes through which investigators may organize their thoughts about data Memos may be used for generating discussion, critique, and new insights during investigator meetings, allowing for organic and iterative interaction with data

5)Reporting results [ma]

Methodological limitations of both original studies and the meta-analysis should be highlighted, and any recommendations should be followed by practical and evidence-based advice. A proposal for future research should be included

Of the four primary approaches to collecting data in qualitative research, which would be the most appropriate in a setting in which direct interaction with study participants was not encouraged?

Of the primary approaches for qualitative data collection, both direct observation and analysis of written documents allow for nonobtrusive methods. Observation entails careful watching, listening, and recording of events, behaviors, and objects in the social setting chosen for study. This is composed into detailed nonjudgmental, concrete descriptions of what has been observed. While methods for observation may not involve any direct interaction, there still might be some confounding effect caused by the presence of the observer. In cases where this would limit the design of a study, analysis of written documents might be the best approach. All data collection for this method is through content analysis of any form of communication or text.

What are two additional quantitative research designs that are commonly used in OT research, and what are their strengths and limits?

Other types of studies that may be feasibly conducted by occupational therapists include other types of quasi-experimental designs, such as the two-group posttest-only design and the one-group posttest-only design. In the two-group posttest-only design, there is the benefit of having a control comparison group. With only the posttest assessment, though, there are increased threats to validity that may confound the findings. This might include effects of history, maturation, or other time-related factors that might account for some of the variance. In the one-group posttest-only design, there is significantly less rigor and the investigator (and individuals reviewing the study) must make several assumptions regarding the participants' status before the intervention. Furthermore, there is no control to make comparisons. This design might be beneficial if information regarding the group's status may be obtained retroactively, or if the group wasn't accessible for a pretest assessment.

two group posttest only design

Participants are assigned to either a treatment group or a no-treatment control group, with measures being taken for both groups after the intervention is administered to determine if they differ on the desired outcome

Two-group posttest-only randomized experiment

Participants are randomly assigned to either a treatment group or a no-treatment control group, with measures being taken for both groups after the intervention is administered to determine if they differ on the desired outcome This design relies on groups being probabilistically equivalent

Qualitative researchers strive to preserve the original format of their data so that readers of their analyses can see or experience for themselves as much of the participants' world as possible

Patterns or common expressions of people's perceptions or understandings of their world Meanings they attach to aspects of the settings in which they live or the behaviors in which they engage Reasons why people think particular things Ways in which people account for or come to particular actions Methods for organizing their day-to-day situations

Steps in Meta-Analysis

Problem formulation Data collection Data evaluation and coding Analysis and interpretation Reporting results

How can you demonstrate an inductive approach to qualitative analysis?

Qualitative analysis uses an inductive approach, wherein themes emerge from the raw data that is collected. This should be collected in an "unbiased" manner, not defined in accordance with pre-established theories or constructs. Miles et al. (2014, p. 10) suggest that the researcher attempt to capture data from an insider's perspective, "through a process of deep attentiveness, of empathetic understanding..., and of suspending or 'bracketing' preconceptions about the topic under discussion." A researcher using an inductive naturalistic approach is not only centrally concerned with the identification of key themes and findings, but also with epistemology, or what counts as knowledge.

What are the major steps involved in qualitative data analysis?

Qualitative research is an iterative process (fluid and nonlinear), wherein some data are collected, interim analyses are performed, and research instruments are modified before further data are collected and analyzed. The key procedures involved in the qualitative data analysis process include active reading, writing analytic memos, index coding/conceptual labeling, and producing code directories. An audit trail, in which study design, data collection and management procedures, and analytic decisions are carefully documented, should be utilized because of the dynamic nature of the research.

What are the advantages of gaining access to a performance site in the early stages of a study? Portray a scenario in which this did not happen and describe a possible consequence

Qualitative sampling often requires gaining access to a performance site, wherein a researcher is afforded an entry point into a population of interest. Recognizing that aqualitative investigator's role is to gain a depth of information on the population of interest,without a sample altering his or her behaviors in the investigator's presence, researchers often have to establish a rapport of trust within that community. For instance, if a qualitative investigator wants to gain a better understanding of the perceived barriers to medical care for an underserved or marginalized population (e.g., a substance abuse recovery group, or a homeless shelter), they might initially be met with mistrust and data might either be not forthcoming or even false. An investigator will likely be unsuccessful if he or she is perceived as an outsider to a community, or a member of a group that is directly related to the area of interest for a study (i.e., the medical community). Rather, qualitative researchers should strive to gain access to a population of interest within the initial stages of a study to have the greatest likelihood of success.

What are common examples of sampling approaches in qualitative research?

Qualitative sampling requires an investigator to strategically determine who would make the best participants for a study, and common approaches include: Maximum variation—seeking individuals who have extremely different experiences of the phenomenon being studied. The researcher is seeking to find the broadest range of experiences, information, and/or perspectives possible. Homogenous selection—seeking informants who have the same experiences. This strategy is used for exploring a particular phenomenon in depth, rather than examining all the variations of which it is an instance. Theory-based selection—seeking only individuals who exemplify a particular theoretical construct for the purpose of expanding the current understanding of a theory. This strategy focuses on a particular concept and seeks to explore its meaning in depth. Confirming or disconfirming cases—seeking informants who will support or challenge an emerging interpretation. Extreme/Average examples of a phenomenon—seeking individuals in each of these

What are some of the quality assurance measures that are specific to qualitative data analysis?

Quality assurance measures that are specific to qualitative data analysis include methods for both data coding and retrieving, as well as toward interrater reliability. With regard to data coding during a study, the creation of a code directory with clear operational definitions is an important first step in ensuring that the coding process is rigorous and reliable. Particularly when multiple investigators are coding, analyzing, and/or interpreting the data, it is important that these activities take place in a systematic and consistent way (Carey & Gelaud, 2008). For improving interrater reliability, the coding process may utilize redundant coding, wherein the degree of agreement may be determined between the two independently coded sections. In addition to, or independent of, the quantitative approach to determining interrater reliability, a study may use a process of reconciliation for the redundant coding (i.e., coming to consensus where discrepancy occurs).

What are the more common sampling approaches in quantitative research, and what would the corresponding performance sites be?

Quantitative research involves two types of sampling strategies: probability and nonprobability. Probability sampling means that each member or element of the population can theoretically have an equal chance of being selected for the sample. Common probability sampling strategies include: Random sampling, wherein all the individuals in a defined target population have an equal and independent chance of being selected for a sample. This is often prohibitive for OT-related investigations, as there is a bias in the population of individuals receiving therapeutic (intervention-based) services, although this may be done through survey studies where individuals within the community may be randomly selected to receive a survey soliciting feedback about a topic of research interest. Stratified random sampling, where random selection is taken from identified subgroups within the population so that representation from the stratifiedcategories is proportional to that category's proportion of the whole population.This would require sample sites that have proportional access to all subgroups that are represented within the broad community. This may include factors such as age or years of experience with a specific area of research interest. Systematic sampling, which is considered equivalent to random sampling as long as there is no reoccurring pattern or order in the listing (Portney & Watkins, 2008). This technique utilizes a systematic equation for selecting individuals randomly. Again, this involves a large subject pool, and may be most appropriate for survey sampling. Cluster sampling, which involves selection of groups or programs, wherein every member of that group or program is invited to participate in the study, rather than specific individuals. This type of sampling might occur in situations where programs (such as therapy centers or inpatient hospitals) are sampled to investigate regional differences in treatments or incidence and prevalence studies. Nonprobability sampling, otherwise known as nonrandom sampling, includes: Convenience, which is the use of volunteers or easily available subjects. This can include any type of recruitment sites (academic or clinical sites), as by definition that sample is one of convenience. Purposive sampling, which is the deliberate selection of individuals by the researcher based on certain predetermined criteria (usually stated as inclusion and exclusion criteria). This form of sampling may occur in traditional clinical, community, or academic sites, as long as individuals meet the recruitment criteria. Snowball or network sampling, in which initially identified subjects provide names of others who may meet the study criteria. This may involve sample sites for which it is difficult to obtain access (because of anonymity, or limited access to the population); recruitment is primarily conducted through referral from insiders within that community. Quota sampling, in which different proportions of subject types are needed so that there is appropriate representation in the sample that may not be attainable with purposive or convenience sampling. Recruitment would require a sampling site where there is sufficient representation of a population, consistent with its broad demographics.

(research process) Step 1: Perform a formal analysis or interpretation of one data source

Read a data source multiple times Gain recognition of concepts, events, feelings, behaviors, and patterns for that source Once familiar, begin coding the data, coming up with brief terms, marking blocks of information Disaggregate the data, breaking the data down into manageable segments, and identify or name those segments Code for the purpose of explaining or developing an understanding, rather than at a descriptive level (if research is more conceptual, coding may be more focused) Inscribe social discourse Look beyond the words and descriptions of interview subjects to understand more fundamental conceptualizations of how people experience what they do; this is a deliberate act to ensure that interactional details are fully captured Don't code everything; rather, code everything relevant to the research question Codes may be used to essentially flag patterns or themes that rest within the data

What is the role of the research problem in shaping a subsequent study? Where does one begin in attempting to define such a problem?

Research occurs within an existing context of literature; this literature establishes a foundation of knowledge and offers prescribed theories, which relate concept and postulates. Inherently, there are gaps of knowledge or contradictions (between theory and knowledge, or experience with knowledge) that exist within this body of evolving evidence. In identifying those gaps or contradictions within the literature, an investigator is defining a problem, or, more specifically, a research problem that may be addressed with further study. Research problems should be succinctly stated, derived from practice, informed by the existing scientific literature, innovative, and of high potential impact. Once a research problem has been identified, its scope may be narrowed and refined into a specific research question. This question is one that may be appropriately addressed within the range of a research study.

When is a statement of a research hypothesis required in a research study? Under what circumstances would a hypothesis not be required?

Research studies may include one, multiple, or no hypotheses within their respective proposals or articles. A hypothesis is a statement of expected results from a study. For quantitative studies, a hypothesis (or multiple) would likely be included if the study uses inferential statistics or when the study is comparing a dependent variable across two or more groups. Quantitative studies that are exploratory or descriptive might not have a specified hypothesis, rather including only a research question. Qualitative studies likely will only include a hypothesis if it is a descriptive or exploratory study.

How to report the analysis

Researchers should try to keep their reports of qualitative analysis as simple as possible; this means presenting very few themes or findings in each report (A name or label for the category The authors' description of the meaning attached to the category A quotation from the raw text or original data to elaborate on the meaning of the category and to show the type of text coded into the category)

(rp) Step 2: Select a particular code for further analysis

Select a particular code (and thus a particular set of data units or patterns among data units) for further analysis Pick a code based on the data source or research question that began their project (rather than relying of previous studies, or existing preconceptions) Pay attention to codes/patterns that simply emerge out of the data source themselves through critical reflection Focus on one pattern or code at a time

Threats to validity for quasi-experimental designs

Selection bias—Participants self-selecting into a study or condition out of self-interest; or a researcher's bias in assigning certain participants to a specific condition Experimenter expectancy—The confounding impact that is caused by the psychological knowledge that a subject is receiving a treatment on part of the examiner

How is validity defined differently in qualitative research?

Similar to quantitative research, validity for qualitative research is the degree with which a study creates evidence, which may truthfully and accurately capture or measure a phenomenon of interest. With qualitative information, this is indicated through the study's authenticity andgroundedness with data collection and analysis. Specifically, "whether participants' perspec- tives have been genuinely represented in the research (authenticity) [and] (w)hether the findingsare coherent in the sense that they 'fit' the data and social context from which they were derived"(Fossey, Harvey, McDermott, & Davidson, 2002, p. 723).

What is a methodological limitation in qualitative sampling approaches?

Similar to quantitative sampling, qualitative sampling may impose limitations on a study. Typically, this is because of the method of purposeful sampling employed. There is generally a tradeoff between breadth and depth of quality of information obtained, which may be directly attributable to the sampling methods. A researcher needs to decide whether to explore specific experiences of a large number of individuals (seeking breadth) or a greater range of experiences from a smaller number of individuals (depth) (Patton, 2014). Investigators should be able to justify why all participants were selected, interviewed, and/or observed, as well as recognize any limitations this imposes on their specific study.

What are applied examples of the five steps in the sampling process?

Step 1: Define the Population of InterestWithin the given scope of a research area and question, a population of study should be defined by the existing body of literature relevant to that topic. For example, if a researcher were to be conducting a study on cognitive impairment found in individuals with early-onsetAlzheimer's disease, existing studies within the literature would help to narrow down and specify who within this population of interest would be appropriate for sampling given the nature of a proposed study.Step 2: Consider the Unit of AnalysisResearch studies may use varying levels of data analysis, indicating the specific unit of study under investigation (e.g., individual/s, inpatient facilities, teams/groups of individuals, social classes). For a study investigating the functional deficits experienced by individuals with early-onset Alzheimer's disease, an individual unit might be the client with the diagnosis of Alzheimer's disease, or it might be the individual's collective family or household.Step 3: Identify a Performance SiteThe performance site is the location in which one ultimately conducts the research study. It is required for sampling purposes; therefore, careful consideration must be demonstrated by the researcher when choosing one. The specific site (or sites) allows for entry into a population of individuals. A researcher must have a strong professional relationship with a site where he or she is proposing to conduct sampling. For instance, if investigating a population of individuals with early-onset Alzheimer's disease, sampling might occur in aninpatient unit, or within the home setting (as provided access through a community organization).Step 4: Develop a Sampling ApproachSubstantive thought must be given toward the development of a sampling approach, as it will serve to define the rigor of a study. In quantitative research, researchers should use theoretically defined methods of sampling that are required for making inferences about the population of interest and that are assumed by more powerful parametric, and even some of the nonparametric, statistics. In qualitative studies, sampling is designed to be less rigid, but it is not any less important. For a study investigating a population of individuals with early-onset Alzheimer's disease, quantitative studies would require planning of the sample size,efforts toward random selection, and potential sample stratification. Qualitative studies would require investigators to propose an approach, which might change or be adapted depending on the nature of the data being collected. Step 5: Implement the Sampling Procedures Finally, investigators must implement their procedures, documenting the process to allow for replicability and demonstrate the rigor of the approach. Any proposed study (such as one on early-onset Alzheimer's disease) should demonstrate careful execution of their sampling procedure's plan and document the process throughout.

(rP) Step 4: Draw some general conclusions about what a coding strategy or arising data pattern means

Stop specific cross-case comparisons and look for broad patterns, such as a synthesis or thematic narrative that illuminates the overall theme Ask yourself: What am I going to call this theme as I move into a reporting stage? What are the examples of variations under the umbrella of this coded theme, and how can I explain them? Why do these variations matter? How am I going to define the nuances I found within it as I looked at different data units and in different data sources? What are examples of the commonalities across all data units I have labeled with a given theme, and how do I explain them? Why do they matter? What is the significance of this theme overall for my research topic, and how might my analysis of this particular topic/theme add to previous literature on this subject?

3)Data evaluation and coding [ma]

Study quality assessment Data identification and quantification Characteristics of interest

What is the process by which a research question is defined? How are gaps in the existing knowledge discovered?

The aim of research is to address gaps that exist in literature (that may be stated as a research problem) through the production of evidence toward new knowledge. These gaps might be identified through clinical experience, through review of the literature, or through previous research experience. Once identified, substantial consideration should be given toward the production of a specific research question. This is because it will frame the scope of the entire investigation, including the methodology employed. When determining a research question, an investigator might: Review the existing literature to determine what is already known, and how that knowledge was produced. Consult with others regarding the topic of interest and the proposed research question, including consultation with experts within the area (or previous researchers), clinicians/consumers of the knowledge, or policymakers/grant funders. Appraise and review the research question with feedback.

What is an example of a quantitative research design that appropriately balances experimental rigor with feasibility and ethics?

The challenge with choosing a research design is balancing the experimental rigor that is hoped to be achieved with the feasibility and ethics involved with conducting a study. While experimental trials may be considered the highest-level design for achieving experimental validity, the feasibility of conducting a randomized assignment of the sample is a technically daunting task. Furthermore, establishing a comparison control group raises several ethical considerations when it comes to withholding some level of treatment (or administering a placebo). The quasi-experimental design allows for increased flexibility when conducting a study. First, a quasi-experimental design allows for convenience sampling, which facilitates the sampling approach. Second, this design allows for more options with control groups, which are more feasible/flexible to conduct, and allows for maintaining ethical standards. For instance, the one-group pretest-posttest designs may be readily done in many circumstances. This design directly allows for comparison between the pretest and posttest, although there is not a control group to limit other threats to validity. Furthermore, using only one group might be more ethically sound if it is believed that the intervention is of benefit to the study participants and withholding it would be unethical.

What are the various approaches to defining effect size in meta-analysis?

Three types of effect sizes are used in most meta-analyses conducted in OT research: The standardized mean difference o Identified through a d-index, which represents a standardized group contrast on a continuous measure, it is commonly used for designs where two groups or conditions are compared. The odds ratioo The odds ratio indicates the odds of success in the treatment group relative to the odds of success in the control group. The correlation coefficiento This is identified through a r-index, which represents the strength of association between two continuous measures.

What are three threats to validity that accompany the use of quasi-experimental designs?

While quasi-experimental designs are much more feasible for accomplishing and supporting ethical practices, there are a number of threats to validity that are inherent to them. These threats include (but are not limited to) maturation, practice effects, and attrition. Maturation refers to the extent with which individuals (subjects) change with time, and their response toward measures utilized during a study will reflect these inherent changes. Practice effects refer to the extent that individuals within studies receive repeated measures, learn from their prior experiences, and change their behaviors during subsequent administrations. Attrition refers to the extent that certain individuals will discontinue participation with a study because of a number of factors; these factors might be significant within a study if the reasons that people are discontinuing are directly related to the variables within the study.

Conceptual labels

Words or short phrases serve as a "tag" for segments of text, categorically describing information the segments contain. Through the inductive process of active reading, conceptual labels become more systematic and consolidated, developing into index codes

Text-based searches

Words or strings of text can be located in the original data This allows for word counts, or calculations of specific word frequency

hypothesis

a direct prediction of the anticipated findings of a study. rest research question(s) and offer a definitive prediction of study findings or outcomes (may have one, multiple, or none/explanatory)

data processing

includes making permanent records of interviews, field notes, or other documents that count as data (Rubin & Rubin, 2011) Data processing should begin simultaneously with data collection, commencing the back-and-forth nature of collection and analysis and allowing early themes to arise Data should be stored and replicated so that the originals are never altered

difference between inductive and deductive reasoning

inductive reasoning aims at developing a theory while deductive reasoning aims at testing an existing theory. Inductive reasoning moves from specific observations to broad generalizations, and deductive reasoning the other way around. inductive more qualitative, deductive more quantitative research

threats to statistical validity

low statistical power, small sample size improper use of statistical test high # of studied variables limits to reliability measure utilized limits to reliability interventions performed variations in study context variations in study participants

Step 4 of sampling: develop sample approach

plan should be devised that specifies how sampling will be conducted reflect the study design, including statistical methods sample size should be determined