8 - Linear Regression Pt. 2

What does a systematic approach to building good regression model seek to find?

A significant model that has the highest adjusted R2.

What does multiple linear regression use?

Several explanatory variables to predict the outcome of a response variable.

What are explanatory variables?

Independent variables (X)

What should be included in a good regression model?

Only significant independent variables.

What should be noted about variables with large p-values?

Although they're not statistically significant, it could be simply the result of a sampling error and a modeler may wish to keep it.

Why can strong correlations among the independent variables be problematic?

Because the independent variable change in unison.

What does the adjusted R2 reflect?

Both the number of independent variables and the sample size and may either increase or decrease when an independent variable is added or dropped.

How do we test for interactions?

By defining a new variable as the product of the two variables, and testing whether tis variable is significant, leading to an alternative model.

How do you mitigate overfitting?

By using good logic, intuition, theory, and parsimony.

What is the first step in the systematic approach to building good regression models?

Construct a model with all available independent variables. Check for significance of the independent variables by examining the p-values.

What is the fourth step in the systematic approach to building good regression models?

Continue until all variables are significant.

What may indicate multicolinearity?

Correlations between independent variables exceeding an absolute value of 0.7.

What are response variables?

Dependent variables, outcome variables, (Y)

What is the principle of parsimony?

Good models are as simple as possible.

What do statisticians recommend if interactions are significant?

First order terms should be kept in the model, regardless of their p-values.

What is overfitting?

Fitting a model too closely to the sample data at the risk of not fitting it well to the population in which we're interested.

What is the second step in the systematic approach to building good regression models?

Identify the independent variable having the largest p-value that exceeds the chosen level of significance

What do additional variables do?

Increase R2 and therefore help to explain a larger proportion of the variation.

What happens when an independent variable is added to a regression model?

It will always result in a new R2 that is greater than or equal to the original.

How do you determine if a variable is significant?

It's not clear so rather than dropping all insignificant variables at once, take a structured approach.

What should guide your model?

Logic. In many applications, behavioral, economic, or physical theory might suggest that certain variables should belong in a model.

What is correlation?

Measures the linear relationship between pairs of variables.

What does a multiple linear regression model have?

More than one independent variable.

What does regression analysis require?

Numerical data.

What is the third step in the systematic approach to building good regression models?

Remove the variable identified in the previous step from the model and evaluate the adjusted R2. (Don't remove all variables with p-values that exceed alpha at the same time, just one at a time)

What does an increase in adjusted R2 indicate?

That the model has improved.

What are partial regression coefficients?

The estimated regression coefficients

What do partial regression coefficients represent?

The expected change in the dependent variable when the associated independent variable is increased by one unit while the other values of all other independent variables are held constant.

What does a higher absolute value of the correlation mean?

The greater the strength of the relationship.

What happens if the absolute value of the t statistic is less than one?

The standard error will decrease and the adjusted R2 will increase if the variable is removed.

What happens if the absolute value of t is greater than one?

The standard error will increase and the adjusted R2 will decrease if the variable is removed.

What is a better indicator than correlation and multicollinearity?

The variance inflation factor.

What happens if too many terms are added to multiple regression models?

Then the model may not adequately predict other values from the population.

How do you use regression with categorical data?

They can be included as independent variables but must be coded numeric with dummy variables.

What should be noted about the independent variables selected?

They should make sense in attempting to explain the dependent variable.

How is the best regression model often identified?

Through experimentation and trial and error.

What is the goal of explanatory modeling?

To explain the relationship between predictors (explanatory variables) and target.

What is the model goal?

To fit the data well and understand the contribution of explanatory variables to the model.

What is a key goal of regression analysis?

To isolate the relationship between each independent variable and the dependent variable.

What is the goal of predictive modeling?

To predict target values in other data where we have predictor values, but not target values.

What happens when a categorical variable has more than two levels?

We need to add n-1 additional variables to the model.

When does multicollinearity occur?

When independent variables in a regression model are correlated.

When may curvilinear models be appropriate?

When scatter charts or residual plots show nonlinear relationships.

When does an interaction occur?

When the effect of one variable is dependent on another variable.