2.9 Correlation and Regression

estimated variance of prediction error

s = standard error. Sf is the standard deviation of the prediction error, and is the square root of the calculation image. This number is used to calculate the prediction interval with a specified critical t-value.

describe limitations of regression analysis.

- Parameter Instability - regression relations can change over time (just as correlations can). - Public knowledge of regression relationships may negate their future usefulness. - If any of the assumptions to the classic normal linear regression model are violated.

calculate and interpret a confidence interval for the predicted value of the dependent variable

...

calculate the predicted value for the dependent variable, given an estimated regression model and a value for the independent variable

...

formulate a null and alternative hypothesis about a population value of a regression coefficient, and determine the appropriate test statistic and whether the null hypothesis is rejected at a given level of significance;

...

calculate and interpret the coefficient of determination

Coefficient of determination (R-squared) determines how well the independent variable explains variation in the dependent variable. The coefficient of determination is the fraction of the total variation that is explained by the regression.

describe limitations to correlation analysis

Correlation measures linear association, thus two variables can have a strong nonlinear relation and still have very low correlation (think parabola). Outliers can also skew correlation coefficients. Spurious correlation has been defined as: correlation between two variables that reflects chance relationships in a particular data set, correlation induced by a calculation that mixes each of two variables with a third, and correlation between two variables arising not from a direct relation between them but from their relation to a third variable.

describe the use of analysis of variance (ANOVA) in regression analysis, interpret ANOVA results, and calculate and interpret the F-statistic

In regression analysis, we use ANOVA to determine the usefulness of the independent variable or variables in explaining variation in the dependent variable. The F-statistic tests whether all the slope coefficients in a linear regression are equal to 0. In a regression with one independent variable, this is a test of the null hypothesis H0: b1 = 0 against the alternative hypothesis Ha: b1 ≠ 0. F-test replicates the t-test and thus is not used often in regressions with just one independent variable. If the independent variable explains little of the variation in the dependent variable, the value of the F-statistic will be very small.

calculate and interpret the standard error of estimate

SEE measures the uncertainty that actual observations are much farther from the fitted regression line. SEE looks much like the formula for computing a standard deviation, except that n - 2 appears in the denominator instead of n -1 (two parameters are being estimated)

calculate and interpret a sample covariance and a sample correlation coefficient, and interpret a scatter plot;

Sample Covariance = average value of the product of deviations of observations on two random variables from their sample means. Sample Correlation Coefficient = covariance of two variables divided by the product of their sample standard deviations. Scatter plots show linear relationships between data.

describe the assumptions underlying linear regression, and interpret regression coefficients

Six classic normal linear regression model assumptions: 1. The relationship between the dependent variable, Y, and the independent variable, X is linear in the parameters b0 and b1. This requirement means that b0 and b1 are raised to the first power only and that neither b0 nor b1 is multiplied or divided by another regression parameter (as in b0/b1, for example). The requirement does not exclude X from being raised to a power other than 1. 2. The independent variable, X, is not random.22 3. The expected value of the error term is 0: E(ε) = 0. 4. The variance of the error term is the same for all observations: , i = 1,...,n. 5. The error term, ε, is uncorrelated across observations. Consequently, E(εiεj) = 0 for all i not equal to j.23 6. The error term, ε, is normally distributed. The slope coefficient equals Cov(Y, X)/Var(X)

distinguish between the dependent and independent variables in a linear regression

This equation states that the dependent variable (variable you are attempting to explain), Y, is equal to the intercept, b0, plus a slope coefficient, b1, times the independent variable (variable you are using to explain changes in Y), X, plus an error term, ε. The error term represents the portion of the dependent variable that cannot be explained by the independent variable. We refer to the intercept b0 and the slope coefficient b1 as the regression coefficients.

formulate a test of the hypothesis that the population correlation coefficient equals zero, and determine whether the hypothesis is rejected at a given level of significance

This test determines the significance of the correlation coefficient. Magnitude of r (correlation coefficient) needed to reject the null hypotheses (H0: ρ = 0) decreases as sample size n increases. Another way to make this point is that sampling from the same population, a false null hypothesis H0: ρ = 0 is more likely to be rejected as we increase sample size, all else equal.

calculate and interpret a confidence interval for a regression coefficient

where b1 is the slope coefficient (b0 - or intercept - could b e used as well), tc is the critical t value, and sb1 is the standard error. In a regression with one independent variable, there are two estimated parameters, the intercept term and the coefficient on the independent variable, thus n - 2 degrees of freedom are used.