Political Science 15
The most common case of hypothesis testing occurs when we...
Just have a hypothesis that a parameter should be positive or negative. This is commonly seen in regression where we anticipate a positive or negative relationship between the independent variable and the dependent variable, but we don't have a specific number in mind. In this situation our null hypothesis is 0 - we can look at the slope coefficient to see if it is positive or negative so we test to rule out zero.If we can rule out zero we call that regression slope statistically significant.
Laboratory vs. Field
Laboratory: Take place in a completely controlled environment. Field: Take place in the real world.
Sample Size
Larger samples will lead to more accurate inferences than smaller samples. The size of the population does not change the required sample size for a given level of accuracy. There are diminishing returns to accuracy as sample size increases. Most samples trade off accuracy for reduced cost.
Central Tendencies
Mean, Median, Mode
4 Kinds of Research Designs
(1) (True) Experiments (2) Natural Experiments (3) Quasi-Experiments (4) Observational Studies
There are four necessary (but not sufficient) conditions that tell us if one variable has a causal effect on another variable
(1) A correlation between the dependent and independent variables (2) Temporal ordering, or observing our hypothesized cause happening before our hypothesized effect. (3) A plausible causal mechanism, or a reasonable story as to why the independent variable should cause the dependent variable to change (4) Ruling out alternative explanations through careful research design
3 Possible Observed Relationships Between A and B
(1) Causality: A -> B (2) Reversed Causality: B -> A (3) Spurious Relationship: C / / A B
Threats to External Validity
(1) testing interaction effects (people act differently because they are being observed) (2) unrepresentative Subjects (i.e. undergraduate subjects versus all voters) (3) spurious measures (the treatment only works in the experimental setting)
Threats to Internal Validity
(Example: Traffic Deaths and Speeding in CT) History: better weather in 1956? Maturation: long term downward trend in death rate due to improving roads, better medical care? Testing: Did the pretest (studying 1955 deaths) make people more careful? Instrumentation: Did we change the way we measure traffic fatalities (i.e. count/don't count pedestrians?) Instability: Do fatalities vary widely across years? Regression to the Mean: Was 1955 just an unlucky year?
Treatment
(The independent variable) is applied to the treatment group, and the values of the dependent variable are compared across groups. Alternative explanations for for any differences observed should be ruled out by the assignment to groups.
Natural Experiment
-treatment and control groups -"as-if" randomization -NO control over the experiment
Quasi-Experiments
-treatment and control groups -NO randomization -NO control over the treatment
"True" Experiment
A Controlled Randomized Experiment -treatment and control groups -randomization -control over the treatment
Selection Effects are a common cause of what?
A common cause of spurious relationships.
Confidence Intervals
A confidence interval is a range that we feel is X% likely to cover the true population parameter. We construct a 95% CI by taking the critical values of a hypothesis test with a 5% level of significance multiplying those values by the standard error, and adding/subtracting them from the test statistic. For example: a sample mean = 30, standard deviation = 2, and critical values = ±2, our 95% CI is [26,34]. Confidence levels are very common in public opinion polls.
Valid Measure
A measure is valid if it actually captures the concept you are interested in. GNP growth is probably a valid measure of economic progress in a country, but it is probably not a valid measure of human rights.
Variance
A measure of dispersion for interval level variables.
R^2
A measure of goodness of fit of a regression line. With one independent variable R^2 is simply the correlation (r) squared. R^2 is always between 0 and 1 with higher numbers indicating a better fit of the regression line to the data. It is interpreted as the proportion of variance in the dependent variable explained by the independent variable.
Statistic
A number that we calculate in our sample to approximate the parameter that we care about. We then use statistical inference to infer something about the population parameter using the statistic we calculated in our sample.
Unit Nonresponse
A potential observation doesn't wind up in our data - (example: some people refuse to take an exit poll after the 2004 election)
How to Standardize a Variable
A variable can be standardized by subtracting off the mean and dividing by the standard deviation. This restates the value of the variable in terms of the number of standard deviations each observation is from the mean value of the variable. This is known as a z-score.
z scores (from lecture)
A z-score is a measure of how many standard deviations a particular observation is above or below the mean. We subtract the mean from the observation and divide by the standard deviation.
Item Nonresponse
An observation is missing some data - (example: a country doesn't keep statistics on some ethnic groups)
Operational Definitions
Before we can test our hypothesis we have to establish operational definitions for our variables. This is the next step after defining concepts in our hypothesis. Operational definitions are very specific. Defining a concept means taking a vague term like "democracy" and provide a more precise definition, like "a country with respect for free speech is one that has no laws forbidding demonstrations or speech against the government."
Causal-Process Observation
Can sometimes rule out competing explanations by pointing out they are not plausible.
Deterministic Causality
Changes in the independent variable always cause changes in the dependent variable.
Probabilistic Causality
Changes in the independent variable usually cause changes in the dependent variable.
Data Coding
Coding is the process of assigning numerical values to the values of your variable. Nominal: codes are just indications of the category Ordinal: codes are indications of ordering Interval/Ratio: codes are the actual numerical value
Content Validity
Does the measure capture all aspects of the concept - for example, does a measure of Democracy capture competitive elections, respect for the rule of law, etc?
Median
The "middle" value of a variable (if we arranged the data from lowest to highest values, the median is in the middle.) It is usually used with ordinal level data.
Statistical Inference Depends on...
Drawing a sample of the population that is large enough to make reliable inferences from.
Control Variables
Could C cause A -> B? In an experimental setting we can randomize so that C is not systematically related to A and B. In observational studies we include C as a control variable.
Two-Tailed Tests
Cut off some probability in each tail of a distribution. Some hypothesis tests are one tailed and only cut off probability in one tail. Most tests are two tailed.
Face Validity
Does the measure appear to capture the content you care about?
Fundamental Problem of Causal Inference
Each observation has only one value on the independent variable at any given time, so it is impossible to know with complete certainty if things would have been different if the independent variable was different.
Goodness of Fit
Even if 3 different scatterplots all have the same equation, it is likely that one will fit better on the line in the center than the other two.
Histograms
For interval level data a histogram is useful
Scatterplots
For interval level data, scatterplots are a good way to examine relationships between variables.
Bar Graphs
For nominal and ordinal level data bar graphs work well
Interval Measurement
Has actual numerical values that have meaning (example: percent of GDP spent on national defense).
Correlation
If we believe A -> B then we should see a relationship between A and B. Positive Correlation: larger values of A are associated with larger values of B. Negative Correlation: larger values of A are associated with larger values of B. Correlation does not prove causality: "Correlation is not causation."
Controlling for Alternatives
If we believe A -> B, then we should have a plausible explanation for why this is so. Could be guided by theory (deductive approach) or simply a logical explanation (inductive approach).
Temporal Ordering
If we believe A -> B, then we should see a change in A first, and then a responding change to B. This does not prove causality. Making this mistake is the POST HOC FALLACY
Statistically Significant
If we reject that the null hypothesis = 0, we say that variable is statistically significant - that is, we can reject the null hypothesis that it has no effect.
T-distribution/T-tests
In most cases we do not know the variance of the sampling distribution, so we must estimate it. This means we switch from using a normal distribution to a t-distribution, and thus the actual hypothesis tests we usually see in the social sciences are t-tests.
Cross-Level Inference
In some cases collecting data on the proper unit of analysis is not possible. Attempting to make inferences about one unit of analysis with data from another unit of analysis is known as cross-level inference. One example is the ecological fallacy: making a mistaken conclusion about individual level behavior based on group level data.
Changes in Sample Size
Increasing sample size reduces the variance of the sampling distribution, making our estimates of the population parameter more accurate.
Correlations
Measure the relationship between two interval level variables. Correlations always fall between -1 and 1. Positive correlations indicate a positive relationship, negative correlations indicate a negative relationship. No relationship gives a 0 correlation, but a 0 correlation doesn't necessarily mean no relationship because correlations only capture linear relationships: y = a + b*x
Multiple Regression Part 2 (From Lecture)
Multiple regression is when you have more than one independent variable (independent variable plus control variable.) y = a +bx + cz a is the intercept or (constant term) this tells us what the dependent variable (y) is expected to be when all independent variables (x and z) are equal to zero. b and c are the slope coefficients. These tell us how the value of the dependent variable is expected to change as the value of one independent variable increases by 1 unit, holding the other variable constant.
Levels of Measurement
Nominal Ordinal Interval Ratio
Null and Alternative Hypotheses
Null Hypothesis: the number we will actually test. H0 Null means there is no difference between our hypothesized value and the true population parameter. For more general hypothesis we set the null hypothesis to be 0. The alternative hypothesis is simply that the null hypothesis is incorrect. We designate this Ha
Ordinal Measurement
Ordered categories (example: a 5 point survey question on ideology running from most to least liberal) - all we know is that something has more of one category than another
Sample Selection
Our goal is to draw a random sample. In a random sample, every element in the population has an equal chance to be in the sample. It is very hard to draw truly random samples. Random digit dialing (RDD), but what about people with no phones? Web surveys, but what about people with no internet access? Biased samples will lead to biased inferences about the population.
Medians are more resistant to...
Outliers!! (One or a few extreme values in the data). Example: housing prices where there are a bunch of regular homes and mansion - the median home price will be a lower number than the mean. Watch for outliers as a sign of a problem with the data.
Social Sciences: Usually Deterministic or Probabilistic?
Probabilistic!
Measures of Dispersion
Quartiles are related to the median - again, line our data up from lowest to highest values, and look at the value of the observations 1/4, 1/2, and 3/4 of the way along the line. Can also do this with other fractions: Quintiles (1/5), Deciles (1/10), Percentiles (1/100).
Multiple Regression
Regression with more than one independent variable is known as multiple regression. Multiple regression is a powerful technique because it allows researchers to simultaneously consider multiple explanations. Typically these regressions have the dependent and independent variables from the hypothesis as well as a number of control variables.
Example of an Ecological Fallacy
Rich people tend to vote Republican, but rich states tend to vote Democratic.
Bivariate Frequency Distributions
Show how frequently combinations of values between two variables occur. They are used with nominal, ordinal, and grouped interval level variables.
Other Observational Studies
Simply observing the world and trying to make causal inferences, without experimental control or special cases with treatment and control groups. -NO treatment and control groups -NO randomization -NO control over the treatment We use statistical controls (control variables) to rule out alternative explanations.
Our Hypotheses are really...
Statements about our population parameters. Example: "The relationship between IMF loans and political instability is positive." We are really saying that a regression slope or correlation is positive.
Regression to the Mean
Statistically, extreme values on a variable tend to be followed by less extreme values.
"As-If" Random Assignment
Subjects do not self-select into treatment and control groups. Assignment to treatment and control groups is plausibly uncorrelated with alternative explanations. Lower on internal validity than if we had truly random assignment.
Testing Hypotheses
Suppose we have a sample statistic, and we know the sample size (n), and we have some estimate o the variance in the population (theta). Our hypothesis provides a guess at the population parameter we care about. Using the normal distribution, we can then calculate the probability that we would have obtained the same statistic we have if the hypothesis was correct.
Evaluating Reliability
Test-retest Method: measure some concept in a population at time 1, and then go back and measure the same concept in the same population at time 2. Alternative form Method: Measure the same concept with two different methods at 2 different times. Split-halves Method: Measure the same concept with 2 different methods at the same time. Inter-coder Reliability: Have 2 different people measure the same concept, compare their answers.
Internal Validity
The ability to determine if there is a causal relationship between the independent and dependent variables in the study.
External Validity
The ability to generalize the results of the study to the population we really care about.
Significance Level
The amount of probability we cut off in the tail of our distribution around our null hypothesis is the significance level. This is the probability we reject our null hypothesis if it is, in fact, true. It is standard in the social sciences to set the significance level to 5%. That is, we usually cut off the last 5% in the tails of the distribution as too unlikely to think the null hypothesis is correct. This makes the probability of a Type 1 Error 5%.
Parameters
The characteristics of the population that we are specifically interested in. It is what our hypothesis is about. It could be a number (the mean education in the U.S.) or a relationship (the relationship between education and voter turnout).
Critical Values
The cut off for the significance level. Usually at 5%. We usually reject our hypothesis if it falls outside of the critical value on either side of the distribution around the null hypothesis.
Treatment Effect
The difference in the dependent variable we would see under two different values of the independent variable.
Sampling Distributions
The distribution of estimates we would get if we drew a large number of samples from the population and calculated our statistics in each sample. Sampling distributions are almost always normal distributions.
Example of confidence level intervals in public opinion polls
The level of support for candidate X is 50%, ± 3%. The range 47% to 53% is a 95% confidence interval, meaning there is a 95% chance the candidates true level of support falls in that range.
Mode
The most commonly occurring value of a variable. It is usually used with nominal level data.
p-value
The results of hypothesis tests which tell us how much probability is beyond the test statistic in the tail of the sampling distribution. p-values of less than 0.05 are usually regarded as indicating statistical significance. A p-value of less than 0.05 usually means you reject the null hypothesis.
The Central Limit Theorem
The sample statistics from random samples of a population will be normally distributed around the population parameter with variance theta^2/n
Substantive Significance
The size of the effect of the independent variable on the dependent variable. A variable could be statistically significant, but have such a tiny substantive effect that it is unimportant in the real world.
Standard Deviation
The square root of the variance.
Standard Errors
The standard error of a sample statistic is just our estimate of the standard deviation of the sampling distribution of that statistic.
Standard Errors In Regression
The standard error on a regression coefficient will grow smaller both as sample size increases and as the variance on that coefficient's variable increases.
Mean
The sum of the values of a variable divided by the number of observations. It is usually used with interval level data.
Degrees of Freedom
The t distribution is similar to the normal, but more spread out to account for the additional uncertainty that comes from estimating theta. This will make our critical values slightly larger. We say the t-distribution is distributed with n-1 degrees of freedom, where n is sample size. More degrees of freedom mean more information was used to determine our distribution. As sample size increases, the t approximates the normal.
Population
The universe of things we are interested in studying. For causal inference usually very broad (i.e. all countries that exist now and will ever exist.)
Z-score
The value of the variable in terms of the number of standard deviations each observation is from the mean value of the variable.
Dispersion/Standard Deviation (From Lecture)
The variance of a variable is the sum of the squared differences between each value of that variable and the mean, divided by N - 1 (where N is the amount of variables within the data set). We square the differences so that positive and negative differences don't cancel out. We divide by N - 1 to get a (conservative) estimate of the mean dispersion of the variable.
Linear regression
These trend lines are known as regression lines. They are perhaps the most common way to examine the relationship between dependent and independent variables in the social sciences. Trend lines can be fit in more than two dimensions, allowing researchers to consider the effect of alternative explanations.
Reliable Measure
To what extent would our measure yield the same results if we went out and collected more data? The more consistent the results, the higher the reliability. Example: An NGO might measure the number of people killed in a genocide - this is a valid measure (it captures the scale of the atrocity), but it won't be reliable (there will be a lot of variation, error, uncertainty, conflicting reports).
Types of Errors
Type 1 Error: When we reject a null hypothesis that is true. Type 2 Error: When we fail to reject a null hypothesis that is false.
Univariate Frequency Distribution
Univariate frequency distributions show how frequently each value of a variable occurs. They are used with nominal, ordinal, and grouped interval level variables. (Grouped interval level just means we pick ranges for the interval level variable - for example, how many people between $10,000 and $20,000, etc.)
Nominal Measurement
Unordered categories (example: a survey of race) - all we know is the categories are different.
Statistical Inference
Using statistical information we can observe to make inferences about some things we can't observe.
Bimodal/Multimodal
Variables with more than one mode.
When we test our hypothesis using the data in our sample we are unlikely to get...
We are unlikely to get exactly the number we hypothesized. Our task is to determine if our sample statistic differs from our hypothesis because of sampling variability or because the hypothesis is wrong.
Population Sample
We can almost never observe the entire population so we draw a sample from the population. We calculate sample statistics using the sample We then make an inference about the population parameters using these sample statistics.
Ordinary Least Squares
We fit regression lines by minimizing the sum of the squared distances between each data point and the line.
Hypothesis Testing With a t... When do we calculate a regression line?
When the null hypothesis is 0! Example: We hypothesize that IMF loans cause more political instability. There is no specific number there like how much instability, so our null hypothesis is 0.
Interpretation of Regression Results
a is the intercept (or constant term). This tells us what the value of the dependent variable (y) is expected to be when the independent variable (x) is equal to zero. b is the slope coefficient. This tells us how the value of the dependent variable is expected to change as the independent variable increases by one unit.
Characteristics of Experiments
random assignment treatment and control groups control over the value of the independent variable
Equation for a Regression Line
y = a + b*x + e y : our dependent variable x : our independent variable a : the intercept of the regression line b : the slope of the regression line e : random error
z score calculation WITH standard deviation
z = (xbar - µ) /( δ - √n)
