Methods Of Research Final
To what errors of observation and recording do you need to be sensitive?
1. Running chronological descriptions of everything!!! When taking notes on these things always try to be concrete; i.e. describe events! Don't "Abstract" what you think is going on. You can abstract and analyze later. What this means is don't draw inferences from what you observe. Rather, record what you observe as you actually observe it. (Especially at the Beginning) Your own feelings and reactions to what you do observe, which can indicate bias (lack of objectivity) on the part of the researcher! (B/c of Abbreviation) Previously forgotten (and/or unrecorded) events, which are often brought back to mind when recording today's events. Patterns and norms of behavior that you do observe. [This typically occurs; however, much, much later in the process.] The KEY is: Always write as much detail as possible!)
What are the two different tests that we discussed in class that a researcher can use (minimally) to test a hypothesis? Know what level of measurement (for the dependent variable) applies to each test.
1. The 1st Analysis for testing a research Hypothesis -Nominal or Ordinal DV you could get frequency distributions (N=2) of the Dep. Var., w/ each distribution calculated within each of the categories (i.e. attributes) of the IND. VAR. to test the hypothesis. -Done with computer through "tabulation or cross tabulation." I.E. Chi square -Crosstabs = INDEPVARNAME by DEPVARNAME (i.e. gender BY attitude towards death penalty) (SS,S,D,SD) "Mles have more supportive attTDP than females." 2. Interval or Ratio DV in hypothesis you could get meanS (X's with dash on top of X) of the DV, with each mean calculated within each of the categories (i.e. attributes) of the IV to test variable -Done with computer with test of mean (or average) differences; i.e. t-test -means tables= DepVarName ($) BY IndDepVar(gender)
What are some of the most important threats to internal validity in experiments?
1ST - Internal Validity & Threats to Internal Validity (AKA Sources of INValidity). A. History (H): Events may occur during the course of the experiment, which may alter how the research subjects behave or think (e.g. Sugar added to the diets of the children). B. Maturation or Aging (A): Research subjects my mature and/or change because of "Aging" over the course of the experiment (especially if it takes a long time). C. Testing (T): Sometimes the influence of testing and retesting (i.e. pretesting and posttesting) may be problematic because it may influence the research subjects (e.g. maybe the research subjects figure out what the experiment is about because of the pretest and it changes them) D. Experimental Mortality (M): In some experiments, research subjects may drop out of the sample (and those that drop out may be very different from those that remain). E. Unmeasured Differences between the experimental and control groups (U): Even with methods of establishing equivalency the groups may still be (hopefully only a little) different.
How should you process and analyze the data from field research?
2nd: Analyze the data! - While there are no exact rules here, what you are looking for is: Norms & patterns of behavior Deviations from the norms and patterns observed All of which will help to create classifications of different types of behavior (as well as the extent of behaviors!) P - In doing so, over time, you go back and forth between observing, and drawing tentative conclusions, to more observations, which revise your conclusions, to more observations, which leads to more verifiable patterns. Eventually, patterns of behaviors will arise from the people that you are studying. Some of them will do things that others will not. Some will do these behaviors allot, others not so much at all. Eventually the variation in the behaviors that you do observe (Effects) will be explained by other concepts (Causes) that you will witness with the behaviors occurrences, which will help you to develop ₮heory (based on your observations)!
What is the function of "probe" when asking survey questions?
A Probe is a survey technique that allows the interviewer to: Derive a response to a closed-ended response category question, and Derive a fuller response to an open-ended response question
What is the "target population"?
All the people in the population of a CERTAIN place/dwelling that you want to GENERALIZE to.
• At what step of the wheel of science does the distribution of surveys take place; i.e. where they are actually handed out to the respondents in the sample, are filled by the respondents, and are then collected by the researchers?
At step #3 of the wheel of science.
How does the measurement of "behaviors" influence measurement validity?
Behaviors: (actions): what units do- action/movement/motion. (i.e. "isms"; prejudism) actually observed behaviors (i.e. Field Research) are more VALID than self-reports of behaviors (i.e. surveys) "-" (i.e. Observing a group of Juvenile Delinquents and their behavior). *What you observe people do is MORE VALID than what they say they do. *CONTENT VALIDITY INCREASES VALIDITY
Understand the basics of the "classic true experiment?" Understand how the two other "true' experiments are related to the classic true experiment.
Classic True Experiment: Two groups, random assignment, posttest measure only. Solomon four-group design repeats the classical design but adds groups that are not pretested
What are the strengths and weaknesses of each mode of field research?
Complete Observer: ID Unknown & Not a Full Participant Simplest mode of observation [e.g. Real Study/Obs.] This method has fewer ethical concerns; as long as you are in a public place there is no real risk of invasion of privacy. But the risk with this one is that the full appreciation of the phenomena being observed will not be captured. PROBLEM: The researcher lacks feedback and explanation from the research subjects that they are observing. Therefore, there is a bit of "speculation" going on by the researcher in terms of what they do and/or don't observe. Feedback and explanation can strengthen Validity of Measurement; especially for the Attitudes and/or Opinions of the Research Subjects. Related; another real potential problem is: Selective Perception: The researcher "observing" what they expected to find! This is why F.R. is considered weak on: Measurement Rel.: Observer as Participant: ID known & Not a Full Participant e.g. studying a motorcycle gang, or inner city urban gangs, or crack cocaine distribution, or burglary rings, etc. The Observer as Participant role is one of the more powerful methods of observation because there are many groups that you could not study if you did not reveal that you were conducting social science research; i.e. there are some groups that you probably cannot "infiltrate". Participant as Observer: ID known & Full Participant Participant as Observer where once again your identity as a researcher is known but you also fully take part in the group's activities (that you are studying) and you are actually a member of the group. e.g. over-eaters anonymous, AA, or already a member of an inner-city gang, or a motorcycle gang, or are a prisoner/guard in prison, etc. PROBLEMS with 1. Observer as Participant: ID known & Not a Full Participant & 2. Participant as Observer: ID known & Full Participant Potential Problems where your "ID is known": One is that the people that you are studying might alter their behavior; e.g. they may downplay some of the things that they typically do (e.g. involvement in crime) or; on the other hand, they may "sensationalize" and do more than they normally do. Second, the group may try to get the researcher (that they know is studying them) to see things from their point of view (e.g. Lofton - "seduction"). This may alter the researcher's objectivity. P - You are always to a certain extent an outsider (when your ID is known)! Complete Participant: ID Unknown & Full Part -Researcher acts as a full member of the group; i.e. they engage in all activities as a normal part of the group. However, they do not tell the people that are being studied that they are actually being studied. This is akin to: Infiltration! In other words, you do not identify yourself as a researcher, nor do you tell them the purpose of your research. P - This mode is only taken when the researcher believes that revealing their I.D. will greatly bias their observations (e.g. Alfred's study of the Satanic Cult in San Francisco!) P - Being a Complete Participant does not always solve the problem of influencing the group. You can still have "reactivity effects" even when your I.D. is Unknown
Which one has the most ethical concerns?
Complete Participant: ID Unknown & Full Part
What is survey research?
Data collection used in social science research (i.e. to make observations), so we will need to be considerate of sampling issues as well. Focus on survey more than any others because they are "fact of life"
How do the different types of field research observation modalities differ with each other?
Depends on whether the researcher: -ID is: "known or unknown," and whether the -Researcher is: "not full participant or a full participant" 1. Complete observer, 2. Observer as Participant, 3. Complete Participant, & 4. Participant as observer
What are the four different modes of observation?
Depends on whether the researcher: ID is: "known or unknown," and whether the Researcher is: "not full participant or a full participant" 1. Complete observer, 2. Observer as Participant, 3. Complete Participant, & 4. Participant as observer
What is the difference between descriptive and explanatory analyses?
Descriptive statistics look at a single variable one at a time; e.g. in terms of percentages, means, medians, standard deviations, etc. Explanatory analyses typically meet the three steps for establishing causality, so they begin with correlations but typically go on to look at the effect of X1 on Y, controlling for X2, X3, and X4. That helps to demonstrate non-spuriousness! That is why they are the most common and most powerful reason for conducting social science research!
What is the difference between internal and external validity?
Experimental outcome actually reflect influence of the experimental stimulus/treatment- INTERNAL ("Are we correct?; "True Cause") Results of the experiments are generalizable to the "real" (i.e. non-experimental) social world. "Is the sample representative?"
What types of studies are experiments particularly well suited for?
Experiments are especially well suited for research projects involving: limited concepts well-defined concepts, hypothesis testing.
3 TYPES OF SURVEYS
Face to face is best method • 1. SELF-ADMNISTERED QUESTIONNAIRES: Here the"respondent" is going to fill-in and answer the survey questions him or herself; i.e. there is no "interviewer"! • Monitored Method: Here you give the respondent the survey. Sit with them as they fill it out. Answer questions if they arise. Then, when they are done, collect the survey, move on to the next respondent. • "Take-Home" Method: Here you give the respondent the survey to"take home", with sufficient instructions for filling out the survey. Give them a few days to fill it out. Then, pick up the survey from them. • Ensure in some manner that the respondent has not skipped any questions b/c there is no interviewer! • 2. Mail Survey: Here you mail the survey to your respondents. Hopefully, they will fill them out (completely). Then, hopefully, they will mail them back. • SURVEY & LETTER OF EXPLANATION • Issues to Consider for Mail Surveys ♣ Mail Distribution and Return (...the Response Rate) • Survey • Letter of Explanation (e.g. "Cover Sheet) • Self-Addressed Envelope • Postage ♣ Survey (Itself) Short & Clear. Use attractive layout. Not too fine of print. ♣ Monitoring return rates with charts • Data charts • Strongly support, strongly oppose • Follow-Ups (Reminder Mailings) o 1 Initial Mailing - must have letter of explanation or introduction o 2 Follow-Ups (Important b/c the Response Rate!) ♣ A New Survey ♣ A (New) Letter of Encouragement • The higher the response rate the higher external validity • 3. FACE-TO-FACE INTERVIEWS: Here the "interviewer" asks the Survey questions and, once the respondent answers, the interviewer records the respondent's response
What are some of the basic statistics used in univariate analysis?
Frequency Distribution, a Mode, or a Mean YOU NEED to KNOW LEVEL OF MEASUREMENT for each variable when deciding to use the 3 basic statistics. Level of Measurement: Nominal: No Order(i.e. dem., rep., ind., other.,) Ordinal: Order (i.e. SA,A,D,SD) Interval: Numbers make sense(#'s are in order) Ratio: " " " and true zero point You can use a Frequency Distribution for any level of measurement; however, they are not very useful for interval/ratio variables because they have so many attributes! Only use Means ( X ) for interval/ratio variables; NOT APPROPRIATE FOR NOMINAL/ORDINAL Modes (Mo) are very useful for nominal and ordinal level variables. IF VARIABLE IS NOT RATIO, REPORT MODE***
• Dr. Anderson is doing an evaluation of a program. To do so he has two groups that are not the same to use. He measures his dependent variable for both groups at time one (t1) and also measures the same dependent variable for both groups at time two (t2). He makes sure that the date of the program that he is evaluating takes place in only one of the groups and it takes place after time one (t1) and before time two (t2). What type of a design is Dr. Anderson using in his evaluation of the program?
He is using a non-equivalent control group design
What is the major disadvantage of experimental methods?
In fact, the major disadvantage of Exp. Methods is: the difficulty in assessing their Generalizability. But, (again) they are generally considered Weak!
What is meant by internal and external validity (in experiments)?
Internal Validity refers to the possibility that the conclusion drawn from experimental results may not accurately reflect what has gone on in the experiment itself. The term external validity implies the determination of whether the casual relationship observed in the study can be generalized or not.
What is field research?
It is qualitative, carried out completely in "natural" settings, and Measure/Observe actual behaviors...
Why is field research a more inductive approach?
Norms & patterns of behavior Deviations from the norms observed Allows you to create classification of diff. types of behaviors
How does it differ from the other methods we studied this semester?
Observed behaviors are MORE VALID then self-report. The study of people acting in the natural course of their daily lives.
What are the basic differences between a standardized Question and an Open-ended Question?
Open-Ended: not "fill-in-the-blank", like blank pages of paper to fill. -Strengths: very ideographic (details), BUT its WEAKNESS is people will ramble. -MORE VALID LESS RELIABLE Standardized: Use very short, simple questions that contain "close-ended" response categories. -MORE RELIABLE LESS VALID (i.e. Strongly Agree, Agree, Disagree, S.Disagree)
What are randomization, matching, and pretesting in experiments?
Randomization is a technique for assigning experimental participants to experimental and control groups randomly. Matching is a process in which participants are matched on the basis of their similarities on one or more variables, and one member of the pair is assigned to the experimental group and the other to the control group. Pretesting (especially on the Dependent Variable.): Sometimes "equivalency" is attained by pretesting the two groups, especially on the DV and then you ensure through matching that the two groups start out the same on the Outcome variable.
What are some of the ways that Field Researchers supplement their observations?
Reorganize and create Multiple Files out of your Field Notes! Each file is organized in a different way! 1. Biographical File: A file on each individual that is being studied in an overall Biographical File 2. Activities File: A file on each activity in an overall Activities File 3. Analytic File (Later): This includes groups of notes by ideas of what you think is going on: Patterns and norms of behaviors that you do observe. Eventually you will explain the observed behaviors with other concepts (to develop ₮heory)! P - You need to update your Files from your Field Notes! Maybe once a month, if the study goes that long, update your files from your field notes. This will provide a mechanism of feedback between your observations and your development of ₮heory.
What step of the wheel of science is the distribution and return of surveys conducted at?
SURVEY's are at STEP 3!!!! Who and what can be studied with survey research? A) What: almost anything; Aspects: -Characteristics: (i.e. facts) what units are(gender, race, age) [+: STRONG] -Attitudes(i.e. orientations): What units think/feel/believe(opinions: SA,A,D,SD) [o: neutral] -Behaviors(i.e. actions): What units do(discriminate) [-=Weak] *single indicator = strong; multiple indictor= more valid content validity* B) WHO: Units of Analysis: individuals(MICRO)
Why are multivariate statistics considered strong(er) for Explanatory Research?
Sometimes it is just as important to test spuriousness. (A # of variables.) X1----Y (.05 & true) Controlling for variables related to DV which might account for the relationship between X1 and Y!
What step of the wheel of science does "data analysis" (ultimately) represent?
Step #4; Hypothesis Testing (i.e. empirical generalization) Comparing observed data to you theoretically derived hypotheses. (however, we are going to have to wait a little while for the test b/c to test hypothesis MINIMALLY you must use bi-variate statistics. Reasons/Purpose: description, exploration, explanation
At what step of the wheel of science does the distribution of surveys take place; i.e. where they are actually handed out to the respondents in the sample, are filled by the respondents, and are then collected by the researchers?
Step 3
Survey methods: step 3
Survey Methods are one of the Data Collection "Ӎ"ethods that we can use in social science research (i.e. to make Observations) • The question of "Internal Validity" refers to whether the independent variable (X) is a true cause of the dependent variable (Y) • Surveys are stronger on external validity, and weaker on internal validity. • 3 steps for establishing causality o Covariation o Temporal order o Demonstration of non-spuriousness • Surveys allow us to 1) explore, 2) describe, and 3) explain most of the phenomena that we are interested in studying. • Surveys always use individuals as "Units of Observation". People fill out surveys!; i.e we get our observations from people. It is their "observations" that they provide us. o When we do this we create rates for macro units from initial observations • surveys are often seen as the best method for gathering information on larger groupings (or [dare we say] "Populations") • Open ended question are more valid, less reliable • Closed ended less valid , more reliable • Open ended is field research There are at least two ways that Surveys can help us with this situation. 1. Surveys often use: Question Standardization: They use very short, simple questions that contain "Closed Ended" response categories. 2. Surveys often use: (Probability) Sampling, which help to produce samples that are "Representative" of the "Population".
How are the different methods (e.g. surveys, experiments, field research, etc.) related to internal and external validity?
Survey: Weaker on internal validity and stronger on external validity (because there IS repr. sample). F.R. stronger on internal validity and weaker on external validity (because of NO repr. sample) Experiments: Stronger on internal validity and weaker on external validity (because it is difficult to assess their external validity: generalizability and considered weak because of NO representative sample. *if something increases internal validity then it decreases external validity, and vice versa*
Why are bivariate statistics considered weak for Explanatory Research?
Testing whether the relationship between 2 variables is statistically significant (or not). Only satisfy the first 2 steps out of 3 to establish causality. DOES NOT test NON-SPURIOUSNESS
How should you record observations in field research?
The KEY is: Always write as much detail as possible!) 1. Record of your observations; i.e. these are your detailed notes about what you have observed. They are your most prized possessions. You should keep multiple copies of them and protect them in some fashion. These are your "data"! 2. Record what you observe as you observe it and/or as close to when you saw it: do not rely on your memory as much as possible.
Which of the following is the most common type of "self-administered" questionnaire?
The mail survey.
What are the essential aspects of a true experiment? Be able to briefly explain why each is important.
There are 3 essentials of a true experiment! 1.You have clearly defined independent and dependent variables. IMP: Because the researcher must be able to control (and induce) the Independent Variable into the situation and Observe (i.e. measure) the Dependent Variable 2. You must have 2 Groups: An Experimental Group and a Control Group. IMP: Because to determine if the Independent variable had an effect you will compare a measure/Observation of the DV for the Experimental Group to a measure/Observation of the DV for the Control Group. Why? Because: The Experimental group IS exposed to the Stimulus/Treatment (Yes). The Control group is not exposed to the Stimulus/Treatment (No). 3. You must have a method for establishing Equivalency of the Experimental and Control Groups. [Equivalency can be attained through A) Random Assignment, and/or B) Matching (on IV's related to the DV; Not the Stimulus), and/or C) Pretesting (sometimes, especially on the Dependent Variable).] IMP: Because if the Experimental Group and Control Group start out different than that could be the cause of the differences between the 2 groups (that you eventually do observe). In fact, the key to the successful use of a "Control" group rests in ensuring that the two groups start out the same!
What are Hawthorne effects?
This occurs when a test subject changes the way they act or behave when they know they are being involved in a study. One way to control this is to have a control group and provide them with special treatment. This will give both groups special attention. Researchers could also try and not inform the subjects that they are in an experiment.
What is a standardized regression coefficient?
USE in MULTIVARIATE "beta"- allows analysis to determine if the relationship between X1 and Y is statistically significant (i.e. non-spuriousness) even after controlling for the influences of X2, X3, X4, etc. (in a multivariate Analysis) It allows to compare the magnitudes (or the impact) that each of the independent variables has on the dependent variable.
What is the difference between "units of analysis" and "units of observation"?
Units of Analysis with survey method are individuals (but not always study/analyze people with survey methods) Units of OBSERVATION: surveys ALWAYS USE THIS. People fill out surveys! (i.e. we get our observations from people. It is their "observations" that they provide us. I.E: Cities with higher unemployment rates have higher crime rates than cities with lower unemployment rates. -Units of OBS: ALWAYS: individuals (micro) -Units of Analysis: Cities/MACRO -Dependent Variable: Higher Crime SURVEYS ARE OFTEN SEEN AS THE BEST METHOD FOR GATHERING INFO ON LARGER GROUPINGS ("POPULATIONS).
What is the difference between univariate, bivariate, and multivariate analyses?
Univariate: 1 variable/purpose: Descriptive A. Frequency Distribution: This Univariate Statistic will provide the number of respondents in each category (or "attribute") of a single variable and (usually) the percentage of respondents in each category of a single variable. Central Tendency: This type of univariate statistic involves a number that summarizes what is typical (or most common) about the sample (e.g. Median, Standard Deviation). B. Mode (Mo): The category (or "value") with the most cases; i.e. it has more "N's" than any other category (or value). e.g. Sophomore and "Very Afraid" (p. 1 Above) Or "Average" C. Mean ( X ): The "sum" of the "Values" for all cases, divided by the total N. . __________ (ONLY FOR INTERVAL/RATIO VARIABLE) 2 VARIABLES (i.e. x1---------->Y) purpose: +/- "weak" A. nominal or ordinal DV-"Tabulation" B. Interval or Ratio DV- Test of mean differences Does not allow to demonstrate non-spuriousness. Bivariate statistics satisfy (only) the first 2 (of the 3) steps necessary to establish CAUSALITY. (covariation & Est. Temp. Order) Explanatory research is most common & most powerful reason for conducting social science research. Bivariate stats. are what you could use MINIMALLY to test hypothesis. (2) A number of variables/ explanatory research and demonstration of non-spuriousness Sometimes it is important to demonstrate spuriousness Purpose: Explanatory +:STRONG Causality: 1.Covariation (x1---->Y).05 2.Est. Temporal Order(X1------>Y) 3. Demonstrate non-spuriousness (i.e. x1,x2,x3,x4 ------>Y) b/c the relationship might be spuriousness and b/c they DO allow you to demonstrate non-spuriousness.
How do you interpret the magnitude of the effect for standardized regression coefficients?
Use in MULTIVARIATE Example: "neutralization is always more powerful than Moral Beliefs in Crime causation." If we could figure out how to measure "neutralization" and "moral beliefs." We could test two different theories head-to-head. We want to CONTROL FOR other variables related to the DEPENDENT VARIABLE (in order to demonstrate non-spuriousness) Control variables to demonstrate non-spuriousness (.05&true) DV: involvement in crime
What are the strengths and weaknesses of field research generally?
WEAKNESS: Measurement Reliability "-" Gen. (Ext. Val.) "-" (b/c no Representative Sample) *Hypotheses are not rigorously tested. Strength: Ideographic (Very detailed) Open Ended - Much More Valid than categories!!! Validity of Measurement "+" Internal Validity (Can be strong) Social Processes (can be studied!) because FR is conducted over time e.g. Differential Association, Labeling Theory, Learning Theories, etc. Context (and how the setting can influence behavior) can be documented INDUCTIVE APPROACH
Make sure to understand what X, O, P and R stand for in experiments (and quasi-experiments).
X stands for the experimental stimulus (treatment) IV O stands for the Outcome (i.e. posttest) O=Observation (e.g. Measurement of DV) P stands for the Pretest (="O") R stands for Random assignment (as a method of establishing equivalency) T stands for a Testing effect (which can result from a Pretest)
What should you keep in mind in order to be a critical consumer of research? To what issues of theory construction and methodology should you pay close attention to when reading social research?
You should keep in mind the research question because ultimately, we are going to use it to determine how well the study has achieved its goal. We're going to ask a question then choose a method for answering the question, (hopefully the best method) then we are going to get the results and interpret what actually they actually mean, and actually answer the research question.
What are we trying to do with probability sampling?
help produce samples that are representative of the population.
What is the VIP fundamental principal in regards to the role of the interviewer in Face to Face Interviewers ?
o A fundamental principal in regard to the role of the interviewer is that the questions are typically "standardized" and that they mean the same thing to each respondent. This is important for both validity and reliability of measurement. o Role of interviewer is a recording device o Response rate is higher, external validity will be higher
What are the rules of interviewing ?
o Appearance and demeanor- the way a respondent looks and acts play a part in result. o Dress similar to respondent. Act in a pleasant and respectful manner o Be familiar with questionnaire- practice before you go out there. o Follow question wording -Don't change wording . o Record responses exactly
Behavioral Data:
o Behaviors (Actions): What Units do - Action/Motion/Movement o Observed behaviors are more valid o Actually Observed behaviors (e.g. Field Research.) are more VALID than self-reports of behaviors (e.g. Surveys) o What you observe people doing are more valid than what they say they do.
What is the difference between internal and external validity?
o Internal validity ♣ Are we correct that the independent variable is a true cause of the dependent variable? ♣ Experiments are strong on internal validity ♣ Behaviors (Actions): What Units do - Action/Motion/Movement ♣ Pretest posttest design is the weakest design for demonstrating non-spuriousness ♣ Strong with experiments and can be strong with field research Weak with surveys ♣ Experiments are strong with reliability ♣ If the Internal Validity is a Significant (.05) and True predictor of the Dependent Variable then the relationship IS Internally Valid; i.e. the relationship between X1 and Y is A) not only significant but B) it is also a true (non-spurious) relationship! o External Validity ♣ Generalizabilty ♣ What we are trying to do here is to select a Sample in such a manner that the aggregate characteristics of the sample approximate the "Parameters" that actually exist in the Population. ♣ Probability sampling is deriving a representative sample then we can generalize the results back to the population ♣ Parameter: A summary description of how a Variable (or statistic) is distributed in the Population; e.g. Md. Family Income, X Yrs. Of Ed ♣ If the "Sample" is Representative of the Population then results derived from analyzing the sample data will be generalizable to the "Population", from which the Sample was drawn. ♣ Surveys are strong ♣ Field research is weak ♣ Experiments are weak ♣ Natural selection (equal chance) ♣ Surveys are stronger on external validity, and weaker on internal validity.
How does the use of multiple times series improve confidence that a given intervention had its intended effects?
• Having a Complex Multiple Time Series • Trend 1 O O O O O O O O O O O • Trend 2 w/ X O O O O O X O O O O O • Trend 3 w/ X & X removed O O O O O X O O O O • X Cause -- O Effect TRUE?
What would help to make you more and more confident with quasi-experimental designs?
• Having a Complex Multiple Time Series • Trend 1 O O O O O O O O O O O • Trend 2 w/ X O O O O O X O O O O O • Trend 3 w/ X & X removed O O O O O X O O O O • X Cause -- O Effect TRUE?
What are the common quasi-experimental designs to do evaluation research?
• Non-equivalent control group (NO RANDOM ASSIGNMENT) • (long) Single Time Series (ONE GROUP) i) O O O [O X O] O O O O ii) An example of a short time single time series (a Pretest-Posttest Design) • Multiple Time Series (2 GROUPS at least) i) O O O O O X O O O O O ii) O O O O O O O O O O EXPECT GROUP 1 TO CHANGE NOT GROUP 2
Why is a long-time series valuable in evaluation research?
• PROBLEM: (w/ Single Time Series): Did something else cause the change in the trend? • ** With 1 Group NOT real confident that "X" caused change • SOLUTION: IT IS HELPFUL TO HAVE THE STOPPAGE (OR THE REMOVAL) OF AN INTERVENTION (TO ALSO EVALUATE EFFECTIVENESS) reverse logic
STRENGTHS AND WEAKNESSES OF SURVEY RESEARCH
• Strengths • Reliability • Useful in Describing Characteristics of Large Populations • Broad Range of Information Can Be Collected (Which Makes Survey • Research Flexible During Analyses) • Large Samples are Possible • `Generalizable • Weaknesses • Validity of Measurement is Sometimes Weak • Standardization May Yield Superficial Measurement of Concepts • "Context" (and the Influence of Context) is Difficult to Capture • Surveys are Inflexible in the Field • Processes are Hard to Study
What is the major purpose of evaluation research?
• To assess the impact of Social Interventions • D.P, B.C's Laws
How does it differ from classic experimental research?
• independent and dependent variables • pretesting and post testing • experimental/control groups
