Chapter 6
Stepwise regression
1.Stepwise regression is another approach to selecting subset regression models. This is actually a collection of related methods that are designed to work effectively with large data sets. (book definition) 2.A 'best' regression model is sometimes developed in stages. A list of several potential explanatory variables are available and this list is repeatedly searched for variables which should be included in the model. The best explanatory variable is used first, then the second best, and so on. This procedure is known as stepwise regression. (online definition)
Sample correlation coefficient, r
A correlation coefficient is a number between -1 and 1 which measures the degree to which two variables are linearly related. If there is perfect linear relationship with positive slope between the two variables, we have a correlation coefficient of 1; if there is positive correlation, whenever one variable has a high (low) value, so does the other. If there is a perfect linear relationship with negative slope between the two variables, we have a correlation coefficient of -1; if there is negative correlation, whenever one variable has a high (low) value, the other has a low (high) value. A correlation coefficient of 0 means that there is no linear relationship between the variables. There are a number of different correlation coefficients that might be appropriate depending on the kinds of variables being studied. (Refer to equation 6.37)
Variance inflation factor
A measure of the amount of multicollinearity in a set of multiple regression variables. The presence of multicollinearity within the set of independent variables can cause a number of problems in the understanding the significance of individual independent variables in the regression model. Using variance inflation factors helps to identify multicollinearity issues so that the model can be adjusted.(equation 6.60 of the book)
Residual sum of squares
A statistical technique used to measure the amount of variance in a data set that is not explained by the regression model. The residual sum of squares is a measure of the amount of error remaining between the regression function and the data set. A smaller residual sum of squares figure represents a regression function which explains a greater amount of the data.
Unbiased estimators
An Estimator is said to be unbiased if the Expected Value of the parameter is equal to the true value of the parameter.
Studentized residuals
Another scaled residual, the studentized residual, is very useful in multiple regression. The studentized residual scales the usual least squares residual by dividing it by its exact standard error.
Significance of regression
DID NOT FIND ANY DEFINITION IN THE BOOK.
Standard errors of model coefficients
In general, Standard error is the standard deviation of the values of a given function of the data (parameter), over all possible samples of the same size. (In the book, Standard errors of model coefficients has been explained by equations 6.18 and 6.19 which we do not need to even memorize. lolll)
Residuals
Residual (or error) represents unexplained (or residual) variation after fitting a regression model. It is the difference (or left over) between the observed value of the variable and the value suggested by the regression model. We use the residuals to provide information about the adequacy of the fitted model.
Standardized residuals
Standardized residuals are the residuals divided by their estimated standard errors. They are used to adjust for the fact that different residuals have different variances.
Simple linear regression
The case of simple linear regression considers a single regressor or predictor x and a dependent or response variable Y.
Response variable
The response variable is the variable of interest that we wish to make a prediction about using a set of explanatory variables.
t-tests on regression coefficients
This is section 6.2.2. There is no unique definition for it in the book.