Module 4: Research Methods - Study Design

Strengths of a cohort study

A retrospective cohort study usually costs less than a prospective study. It may also be the only viable method for studying events that have occurred in the more distant past. A study of how counseling and support services affect post-foster care disposition among children who have been in foster care since infancy would be well suited to a retrospective cohort design.

Cohort Study Strengths

-The major strength of this study is the ability to establish causality. The temporal sequence of the variables is known, because they are measured as they occur. This eliminates or greatly minimizes the problem of recall and recall bias. -Multiple risk factors can be examined in a cohort study. EX: Injury, witness violence, and death of a fellow soldier can all attribute (and be risk factors) for GAD -Multiple outcomes may also be studied. EX: GAD, Depression & PTSD can be measured in order to determine if the risk factors cause them

Different Types of Cases

-Treated Cases who are individuals who are receiving treatment for the outcome. -Community Cases are selected from persons who are not receiving treatment.

Postest Only with Non-equivalent Groups Study

A posttest only design with non-equivalent groups is used when it is not possible to randomize participants. The dependent variable is measured only after the introduction is introduced to one group. An existing group that is similar to the experimental group is used as the comparison group, which does not receive the intervention. The analysis is conducted by comparing the dependent variable of the two groups at the posttest. If data can be obtained on the dependent variable and potential confounding variables, the two groups can be compared pretest to see if they were similar on these variables. However, these data, if they are even available, are measured retrospectively.

Trend Study

A type of cross-sectional study that consists of two or more cross-sectional studies conducted at specified intervals using the same methods. A different sample is selected for each time point of data collection. The same variables are measured at each time point. The reason this is a type of cross-sectional study is because data are collected only once from each study participant. EX: A study designed to examine gender composition of MSW graduates between 1950 and 2010 is an example of a trend study.

Randomization vs Random Selection

Both randomization and random sampling involve randomness, but they are used in very different ways. Randomization is used to make the experimental and control groups comparable in order to increase internal validity. Random selection is used to increase external validity.

Social Artifacts

Could be a newspaper, song lyrics, or television show used as a source of information for a unit of analysis

What are the strengths of a case control study?

Feasibility when studying a rare outcome and the ability to include multiple potential risk factors in the same study.

Cohort Study Timing

Most cohort studies are prospective. The data are collected as events occur, following study participants forward in real time. In a retrospective cohort study, all of the events have already occurred. The researcher collects data on these events retrospectively. The timing of a retrospective cohort study is also forward, because the researcher still starts by collecting data on the risk factor and then on the outcomes, even though both have already occurred.

3 Pre experimental designs:

One Shot Case Study One Group Pretest-Posttest Postest Only with Non-equivalent Groups

There are 3 categories of experimental designs:

Pre-experimental designs Quasi-experimental designs Experimental designs

Some other threats to internal validity in a cross sectional design?

Recall bias History (if long time to collect data) Maturation (if long delay from time of selection to time of data collection) Cross-sectional studies are very inefficient for studying a rare condition or disease. EX: If you were interested in studying whether certain stressors, such as child abuse, were risk factors for schizophrenia, a cross-sectional study design would be a poor choice. Schizophrenia is a rare disease. And you would need thousands of study participants to have enough with schizophrenia to make it possible to assess whether child abuse and schizophrenia are associated. A trend study consists of two or more cross-sectional studies conducted at specified intervals using the same methods.

unit of analysis

Refers to the entity being studied. The units that we study in order to describe a population or a phenomena The dependent variable usually shows who the unit of analysis is. The unit of analysis may also be a geopolitical unit, such as a city, a state, or a country. The source of information might be a public official or census data or government reports. When the unit of analysis is a social artifact-- such as a newspaper, song lyrics, or television show-- the source of data is the artifact. EX: An individual, family, parent-child diad, or the gang. Social service agency, school, church. Country. Social artifact such as magazines. EX: A study to determine whether the social adjustment of children who live in single-parent families is different from children who live in two-parent families may collect data about the parents and siblings, but the unit of analysis is the individual child. Usually, the dependent variable makes it clear who or what the unit of analysis is. In this example the dependent variable is social adjustment of the child. If the purpose of the study is to determine the strength of the parent-child bond, the unit of analysis is the parent-child dyad because the dependent variable is about both the parent and child.

Other weaknesses of a Cohort Study

Resources (time and money) Attrition (people dropping out over time) Not appropriate for studying rare risk factors Outcomes with long latency (not cost effective for these)

Threats to Internal Validity in Cohort Studies

Selection bias may result in a sample that is not representative of the target population. Instrumentation change may be a problem if different ways of measuring the same variables are used at different time points. This would make it difficult to compare these variables over time. If data are collected too frequently, testing could affect study measures. History could be a problem if external events that occurred during the course of the study affect the study participants on variables important to the study. This may be an even bigger problem if the external event affects those with the risk factor differently than those without the risk factor. Finally, maturation may adversely affect the ability to establish causality. Study participants may be changing merely because time is passing.

What is another threat to internal validity in a cross sectional design?

Selection bias. If the study participants are not representative of the target population, this may affect the ability to provide an accurate description of the target population. It may also affect the relationships between variables.

Case Control Study

Study Design = Case-Control Study Time Dimension = Cross Sectional Directionality = Backwards Timing = Retrospective

Cohort Study

Study Design = Cohort Study Time Dimension = Longitudinal Directionality = Forward Timing = Usually Prospective / May be Retrospective

Cross Sectional Study

Study design = cross-sectional study Time dimension = cross-sectional Directionality = non directional Timing = retrospective -Can be used to describe 1 variable -Can be used to demonstrate a relationship between 2 or more variables -Multiple dependent and independent variables can be studied EX: we can only conclude that being injured and generalized anxiety disorder are related to each other.

Some other threats to internal validity in a case control study?

Temporal Ambiguity Recall bias Selection bias History (if long time to collect data) Maturation (if long delay from time of selection to time of data collection)

What is one of the main threats to internal validity in a cross sectional design?

Temporal ambiguity. Causality cannot be established because all of the variables are measured at the same time. And we cannot accurately establish the temporal order of these variables. In the study of soldiers, it may be that being injured caused generalized anxiety disorder. EX: Soldier Injury > GAD but can also be GAD > Increase chances of Injury

Number and Frequency of Time Points in a Cohort Study

The duration of a cohort study, the time between each time point of data collection, and the number of times data are collected depends on the purpose of the study and the variables that are measured. Studies may last several months or several years or more with data collected only at the beginning and end of the study or at multiple times during the study period. The frequency of follow up measurements is determined by the risk factors and outcomes studied. It also depends on how often the study participants will tolerate having data collected and whether using the same measure repeatedly over very short periods of time might result in measurement bias. Feasibility is also a consideration.

When data are collected in a cohort study:

The hypothesized risk factor (IV) and outcome variables (DV) are measured at time one. At each subsequent time point, the outcome variable is measured. If the risk factor is something that can change over time, that is also measured at each time point. EX: if the risk factor was ethnicity, this would only need to be measured at time one. EX1: If the risk factor is something that can change over time, that is also measured at each time point. If the risk factor was income, this would need to be measured at each time point. The study participants are usually selected from a single study population based on one or more common characteristics that are not related to the study risk factors.

case control study design

The purpose of this study is to determine whether a variable is a risk factor for annuity outcome. This design is very useful when the outcome is rare. Study participants are selected based on the dependent variable known as the outcome. EX: In the study of whether child abuse is a risk factor for schizophrenia, participants are selected based on whether or not they have a diagnosis of schizophrenia. There are actually two samples selected in a case-control study. One sample is selected from a study population of individuals who have the outcome. These study participants are the cases. The other sample is selected from a study population that does not have the outcome. These study participants are the controls. Both study populations are selected from the SAME clinic. Have a cross-sectional time dimension. Data are collected only once from each study participant.

cohort study design

The purpose of this study is to identify risk factors for the onset of a condition or disorder or factors that affect the progression of a condition or disorder. The participants are followed over time to see who develops the outcome or dependent variable. This study design gets its name because the same group of study participants, a cohort, is followed over time. Data are collected at least two-time points for each participant but data may be collected at more than two-time points for each participant.

Unit of Analysis & Source of Information

The unit of analysis may not always be the source of information. If you are observing the individual then the unit of analysis IS the source of information. The source of information may also be the parent, the pediatrician, or the medical record. The unit of analysis is determined first. And then, the source and method of data collection are determined.

Random Assignment or Randomization

They refer to the process by which study participants are assigned at random to different groups in an experiment. It means that the researcher does not use any judgment in assigning study participants to the experimental or control groups.

Sources of Data/Information

When the unit of analysis is a group, an organization, an agency, or a program, the source of data may be the director. Members of the group or staff may also be sources of data. The researcher may also collect data from records maintained by the group or organization, such as client records or annual reports.

Time Dimension

When the unit of analysis is observed at only one point in time, the study has a cross-sectional time dimension. When the unit of analysis is observed at two or more time points, the study has a longitudinal time dimension. **Do not confuse the time dimension of a study with how long it takes to conduct the study. It is possible for a cross-sectional study to take many months to conduct if the study sample is large and in-person interviews are conducted with each study participant. It is also possible for a longitudinal study to be conducted in a very short time frame.

One Shot Case Study

a single group of study participants is measured on the dependent variable after receiving the intervention. Analysis is conducted by comparing the data obtained from study participants with what would be expected if there were no intervention. This design does not control for any threats to internal validity. All of the potential threats to internal validity are present in this design. This is an extremely weak design because it is impossible to know whether there was any change caused by the intervention.

One Group Pretest-Posttest Design

a single group of study participants is measured on the dependent variable after receiving the intervention. The data are analyzed by comparing the pretest data with the posttest data. Pretest means the measurement made prior to the introduction of the intervention, and posttest means the measurement made after the introduction of the intervention. This design allows the researchers to be sure that the pretest is measured prior to the intervention and the posttest follows exposure to the intervention. There are several threats to internal validity when using this study design. While it is possible to establish the temporal order of the intervention and the dependent variable, it is not possible to rule out confounding. There may be alternate factors other than the intervention that caused the change in the dependent variable. It is not possible to know whether the intervention caused any change observed in the dependent variable, or if this was due to some other variable, including one that is related to one of these threats to internal validity.

Controls should:

be as similar as possible to the cases with the only difference being that they do not have the outcome. EX: Schizophrenic case-groups were chosen from same clinic and geographic area.

Retrospective Study

both risk factor (IV) and outcome (DV) occur before the onset of the study. Data are gathered on events that have already occurred. The data are based on recall by participants or other informants or from records. Advantage of measuring variables this way: the study is less time-consuming because all of the phenomena to be measured have already occurred. Disadvantage: the measurement of variables relies on the memory of participants and may be affected by recall or recall bias. Some phenomena change over time and may not be the same at the time of the study as when they originally occurred.

Non-directional Study

both the risk factor (IV) and outcome (DV) are measured at the same time.

What are are three main types of observational study designs?

cross-sectional, case control, and cohort.

Random Selection

is NOT synonymous to random assignment or randomization. It is related to how study participants are selected to be in the study sample.

Data Analysis in a Cohort Study:

is conducted by comparing participants who have the risk factor with those who do not have the risk factor. If a higher percent of participants with the risk factor develop the outcome or have a higher level of the outcome variable, this provides evidence that the hypothesized risk factor causes the outcome. EX: we can conclude that being injured is a risk factor for generalized anxiety disorder. because the forward directionality of the study allows us to establish causality, because we know the temporal order of the variables.

Directionality

is the sequence in which the risk factor (IV) and outcome (DV) are observed or measured. This refers to how the study is conducted and not when the events actually occurred. The directionality of a study may be forward, backward, or non-directional. This determines the ability to distinguish the sequence in which the variables occurred, which is an important criterion for establishing causality.

cross-sectional study design

provides an accurate description of the target population if the study has a representative sample and uses valid measures. Cross-sectional studies are also used to examine relationships between variables and to generate etiologic hypotheses. Once a relationship between two variables is established, researchers can hypothesize about the causal relationship between the two variables and design a study to test these causal hypotheses. Participants are selected if they meet the study eligibility criteria. The researcher does not need to know anything about the independent variable or the dependent variable when selecting participants. In this study, the design is cross-sectional as well as the time dimension. EX: A researcher is conducting a cross-sectional study in order to learn whether the experiences of soldiers who had served in a combat zone are associated with having an anxiety disorder. The independent variables are sustaining an injury, witnessing violence, and the death of someone in their unit with whom they were close. The dependent variable is generalized anxiety disorder. The study sample is selected from individuals who served in Iraq or Afghanistan and who are now back in the US. Data are collected only once from each study participant. The data collection instrument, which may be an interview, a self-administered questionnaire, or another method of data collection, includes measures for all of the variables in the study. The researcher interviews each soldier participating in the study by telephone. The interview includes questions about their experiences in the combat zone for the independent variables and a measure that assesses whether the soldier has a diagnosis of generalized anxiety disorder for the dependent variable.

Observational research

refers to studies in which no treatment or intervention is tested. The researcher observes study participants without doing anything to them. EX: conducting an interview, a self-administered questionnaire, or watching participants perform an activity.

Timing

refers to the chronological relationship between the onset of the study and the actual occurrence of the risk factor (IV) and the outcome (DV) -- in other words, when things are measured in relation to when they actually occur.

In a cohort study participants are selected based on:

same age group or having the same occupation. Even those who have the dependent variable (If the purpose of the study is to learn who develops the outcome, then only individuals who currently don't have the condition or disorder will be selected to participate in the study). EX: the study participants may be selected because they are age 65 and older or because they are social workers.

control group

the group that does not receive the experimental treatment, the study participants have been randomly assigned to that group and is used as a comparison group.

Prospective Study

the researcher observes both the risk factor (IV) and the outcome (DV) after the onset of the study. Data are gathered as they occur. Advantage of measuring variables this way: the problem of recall is eliminated or minimized. Disadvantage: this can be more time-consuming.

Backward Directionality Study

the researcher starts by measuring the outcome (DV) variable and then observes the risk factor (IV) that has already occurred.

Forward Directionality Study

the researcher starts with observation of the risk factor (IV) and then observes the onset or change in the outcome (DV) variable.