Research Methods, Design and Analysis (chapters 7-12)

reversal design

a type of single-subject design that involves repeated alternations between a baseline period and a treatment period (test scores -> praise -> test scores)

what are the advantage of recruiting research participants over the internet

access to a larger and more diverse sample

natural setting

collection of data in the field at the site where participants experience the issue or problem under study. do not bring individuals into a lab, nor do they typically send out instruments for individuals to complete. up-close information gathered by talking directly to people and seeing them behave and act within their context. Face-to face interaction, often over time

Time-Series Design

A quasi-experimental design with repeated measurements of the dependent variable before and after the treatment condition is introduced. Comparison is accomplished through the participants' own pretreatment responses. The treatment effect is determined through the counterfactual.

when should we conduct power analysis

in advance to estimate sample size needed if an effect of the size you are looking for actually exists

selection-regression bias

might exist if the two groups are from different populations, such as the experimental treatment group from a population of individuals without a reading disability

Therapeutic Criterion

demonstration that the treatment condition has eliminated a disorder or has improved everyday functioning

how does recruitment of research participants made

depends on type of participant needed

process evalutation

determines whether program reaches target population (staff training working)

the probability of correctly rejecting a null hypothesis is what

false by a specified amount

selection-attrition bias

pretest allows examination of the nature of attrition to see if there is a difference between those that drop out or do not complete the experiment and those that do

Single-case research designs

research design in which a single participant or a single group of individuals is used to investigate the influence of a treatment condition

what are the disadvantage of recruiting research participants over the internet

some people dont have access to it representativeness

time series designs

-With multiple data points before and after treatment is introduced -Doesn't eliminate the history threat -Assessment of a treatment effect is based on the assumption that the pattern of pretreatment responses would continue in the absence of the treatment

Functions of the control group

1) Serves as a source of comparison 2) a control for rival hypotheses.

Issues With ABA Design

1. Ends with a baseline condition, and from the standpoint of a therapist, this not really desirable, since the benefits of the treatment condition is denied. [Addressed with ABAB design] 2. If a return to baseline does not occur with all dependent variables, it might be the result of a carryover effect across phases. [Addressed by Multiple-Baseline Design] 3. Withdrawal vs. Reversal. Withdrawal. Reversal design can be more time-consuming.

making causal inference from quasi-experiment requires...

1. cause and effect must covary 2. cause must preced effect 3. rival hypotheses must be implausible

design components that are usually available to a researcher

1. control or comparison groups (0,1, 1+) 2. pretest (0,1,1+) 3. posttest (1, 1+) 4. within-participants and/or between participants independent variables 5. inclusion of 1+ theoretically interesting independent variables 6. measurement of 1+ theoretically interesting dependent variables

principles used to rule out rival explanations in quasi-experiments

1. identification and study of plausible threats to internal validity 2. control by design 3. Coherent pattern matching

when writing a procedure for a qualitative proposal what should you consider?

1. identify the specific design that you will be using and provide references to the literature that discusses the approach 2. provide some background information about the design, such as its decline origin, the applications of it, and a brief definition of it 3. discuss why it Nisan appropriate strategy to use in the proposed study 4. identify how the use of the design will shape many aspects of the design process, such as the title, the problem, the research questions, the data collection and analysis and report write-up

Selection and Additive/Interaction Threats to the internal validity of a nonequivalent comparison group design

1. selection bias 2. selection-attrition bias 3. selection-maturation bias 4. selection instrumentation bias 5. selection-regression bias 6. selection-history bias

levels of quantitative data analysis

1. the more general procedure in analyzing the data 2. the second would be the analysis steps embedded within specific qualitative designs

Most Common Design

A Time-Series Design.

Counterbalancing

A control method that only applies to studies where all participants recieve all levels of the IV. This technique controls sequencing effects.

Matching by Holding Variables Constant

A matching technique that creates a homogenous sample by having participants that only have a certain amount or type of extraneous variable.

Matching by Yoked Control

A matching technique that takes the experimental participant's outcome and gives the same outcome to the control participant.

Single-Case Research Designs

A research design in which a single participant or group of participants to investigate the influence of a treatment. Typically used with single participants. There is no random assignment and there is no control group and is different from a case-study.

Order Effect

A sequencing effect that arises from the order in which the treatment conditions are administered.

One-Group Posttest-Only Design (Weak #1

A single group of research participants is measured on a dependent variable after having undergone an experimental treatment.)

ABA Design

A single-case design with three separate conditions. There is one baseline condition, one treatment condition, and then another baseline. Is the most basic single-case research design. The response to the treatment condition is compared to baseline responses recorded before and after treatment.

Within Participants Design

A type of experimental design where all research participants receive all treatments.

The only way to equate groups

Assign participants randomly to the two groups

One-Group Pretest-Posttest Design (Weak #2)

Design in which a treatment condition is interjected between a pretest and posttest of the dependent variable. (Improves the one-group posttest design by adding a pretest to measure the dependent variable, before the treatment.

Randomized Controlled Trial (RCT)

Designs with random assignment to the experimental and control groups.

Reversal Design

During the third phase of the ABA design, during (the second A condition), the behavior is applied to an alternative but incompatible behavior so that a reversal in behavior is produced. Is used less frequently.

Withdrawal Design

During the third phase of the ABA design, the treatment condition is removed. This design is used more frequently.

Matching

Equating participants in mainly nonexperimental research on one or more known variables.

Between-participants (Strong)

Groups are composed of different people, the participants are exposed to different experimental conditions, participants are randomly assigned to the groups.

Method of Difference

If groups are equal on every variable except for one, then that variable is the cause of the difference between the groups.

Issues With Time-Series Design

If only one pretreatment and posttreatment measure is taken, then history and maturation effects may be present, and so it is best to obtain multiple pretreatment and posttreatment responses.

Cyclic variations

Incorporate the cycle in all phases Or the results must be replicated across participants that are at different stages of the cyclic variation Or include participants not affected by the cyclical variation

Posttest-Only Design with Nonequivalent Groups (Weak #3)

Performance of an experimental group is compared with that of an nonequivalent control group at the posttest.

Experimental group (Treatment Group)

Research participants that receive some level of the independent that is intended to produce an effect.

Matching by Building the Extraneous Variable into the Research Design

Sometimes called "blocking", by taking an extraneous variable and treat it as you would an IV. Recommended if your intrested in the difference produced by the various levels of the extraneous variable.

When To Use Single-Case Research Designs

Sometimes you are not able to use large groups of participants/individuals, or that you may need to assess the effect of a treatment on a single individual.

Individual Matching (Subject matching)

Takes a sample of research participants, measures each one, and each is matched with a second participant of the same matching variable(s), then the pair are randomly assigned to the various treatment conditions.

A Condition

The baseline condition in which the target behavior is recorded in its freely occurring state or to the presentation of the treatment. It is later reintroduced, with the treatment condition being withdrawn and is used to see if the behavior reverts to baseline.

Reversal

The behavior reverts back to its pretreatment level, and is crucial for demonstrating that the experimental condition produced the effect, rather than extraneous variables.

B Condition

The experimental condition in which the treatment is deliberately imposed to try to alter the target behavior, and is typically continued for an interval equivalent to the original baseline period or until some substantial and stable change occurs in the behaviors being observed.

Matching Variable

The extraneous variable used in matching.

Random Assignment

The most important and most powerful of all the control methods. A probabilistic control technique designed to equate experimental groups at the start of an experiment on all extraneous variables, both known and unknown.

Research Design

The outline, plan or strategy used to investigate the research problem

Counterfactual

The pattern of pretreatment responses would continue in the absence of the treatment.

Research Goal

To use the strongest design that is possible, ethical and feasible for your research question.

To achieve internal validity. . . .

We must eliminate potential rival hypotheses via control techniques and a control group.

Diferential Influence

When the influence of an extraneous variable is different for the various groups.

Statistical Control

When you take a quantitative matching variable, and leave it in its natural unit and then enter it into the study during data analysis.

single case design

a design in which only one participant is used (ABA design)

program evaluation

a form of applied research that includes a number of research activities designed to evaluate programs from planning to completion (Common Core)

Control Group

a group of research participants that do not receive the active level of the independent variable (they receive zero amount or some sense a standard value amount)

nonequivalent comparison group design

a quasi-experimental design in which the results obtained from nonequivalent experimental and control groups are compared -lack of random assignment leads to participants in control and experimental groups to not be equivalent which can affect DV -operate as rival hypotheses

one-group posttest-only design

a quasi-experimental design that has no control group and no pretest comparison; a very poor design in terms of internal validity

quasi-experimental design

a research design in which an experimental procedure is applied but all extraneous variables are not controlled

Selection bias

b/c groups are nonequivalent, there will always be a potential selection bias. However, the pretest allows the exploration of the possible size and direction of the bias on any variables measured at pretesting

whats the convenience sample weakness

cant really compare to the general population

baseline

condition of an experiment in which the independent variable is not present (test scores)

narrative research

employsrestorying the participant's stories using structural devices, such as plot, setting, activities, climax and denouement

ABAB Designs

extension to ABA design to include reintroduction of the treatment condition--solution to ABA design Baseline--treatment--baseline--treatment

outcome evaluation

focuses on the end result of the program (did drug use reduce)

whats important no matter how participants are recruited

important to report how participants were selected and assigned

reflexivity

in qualitative research, the inquirer reflects about how their role in the study and their personal background, culture, and experiences hold potential for shaping their interpretations, such as the themes they advance and the meaning they ascribe to the data. More than merely advancing biases and values in the study, but how the background of the researchers actually may shape the direction of the study

is generic power a thing

no

methods section for proposal for qualitative research

partly requires education readers as to the intent of qualitative research, mentioning specific designs, carefully reflecting on the role othe researcher plays in the study, drawing from an ever-expanding list of types of data sources, using specific protocols for recording data, analyzing the information through multiple steps of analysis and mentioning approaches for documenting the accuracy or validity of the data collected.

inductive and deductive data analysis

qualitative researchers build their patterns, categories, and themes from the bottom up by organizing the data into increasingly more abstract units of information. Researcher work back and forth between themes and database until they establish a comprehensive set of themes. then they look back at their data from the themes to determine if more evidence can support each theme or whether they need to gather additional information.

researcher as key instrument

qualitative researchers collect data themselves through examining documents, observing behavior or interviewing participants. they may use a protocol - an instrument for collecting data- but the researchers are the ones who actually gather the information. do not tend to use or rely on questionnaires or instruments developed by other researchers.

holistic account

qualitative researchers try to develop a complex picture of the problem or issue under study. involves reporting multiple perspectives, identifying the many factors involved in a situation, and generally sketching the larger picture that emerges

multiple sources of data

qualitative researchers typically gather multiple forms of data, such as interviews, observations, documents, and audiovisual information rather than rely on a single data source. then the researchers review all of the data, make sense of it, and organize it into categories or themes that cut across all of the data sources.

qualitative methods

rely on text and image data, have unique steps in data analysis and draw a diverse design.

ABA Designs

response to treatment condition is compared to baseline responses recorded before and after treatment Baseline--treatment--baseline; if you take away the treatment and the effect is gone, it reveals that the treatment works Demonstration of treatment effectiveness requires return to baseline

Institutional Animal care and use committee (IACUC)

reviews research protocols for studies using animals determines if ethical

Institutional Review Board (IRB)

reviews research protocols for studies using humans primary concern is participant welfare

what are the factors that can influence power

sample size alpha level (acceptable chance of falsely rejecting null hypothesis) effect size (the magnitude of the relationships between IV AND DV in a population)

Changing-Criterion Design

single-case design in which a participant's behavior is gradually shaped by changing the criterion for success during successive treatment periods

Multiple-Baseline Design

single-case design in which the treatment condition is successively administered to several target participants, target outcomes, or target settings

Interaction Design

single-case design used to identify interaction effects Similar to factorial design in that you look at each treatment (IV) individually (main effects) and combined (interaction)

Experimental Criterion

single-case research, repeated demonstration that a behavioral change occurs when the treatment is introduced Non-overlap of performance between treatment and baseline phases or different trends of behavior between the 2 phases Replicating treatment effect over time

Subjective evaluation method

social validation method where others' are asked if they perceive a change in the participant's behavior

design componenets

structures and procedures used in constructing research designs

quasi-experimental design

studies effect of IV where controls cannot be implemented (Adderall study)

multiple baseline design

the effectiveness of the treatment is demonstrated when a behavior changes only after the manipulation is introduced (smoking design)

participant's earnings

the entire qualitative research process, the researcher keeps a focus on learning the meaning that the participants hold about the problem or issue, not the meaning that the researcher bring to the research or that writers express in the literature.

the more power

the more significant of a study

needs assessment

the process of identifying and prioritizing the needs of issue (type of drug abused by kids)

emergent design

the research process for qualitative researchers is emergent. the initial plan for research cannot be tightly prescribed, and some or all phases of the process may change or shift after the researcher enters the field and begins to collect data Key is to learn about the problem from the participants and to address the research to obtain that information.

selection-maturation bias

this might exist if one group of participants becomes more experienced, tired, or bored than those in the other group

selection-instrumentation bias

this might exist if the nonequivalent groups of participants start at different points on the pretest, particularly if the measuring instrument does not have equal intervals

program theory assessment

to figure out what the program should actually do, how to address problems (finding homeless people jobs)

Carry-over effect

use short treatment condition phases or use multiple-baseline design

efficiency assessment

weighs the program's benefits and effectiveness in relation to its costs, to determine whether it is an efficient method for addressing the problem (is program worth it?)

whats a convenience sample of students

where they pull certain groups that will work will for their own experiment