ch 7

Overfitting

Fitting a model too closely to sample data, resulting in a model that does not accurately reflect the population.

training set

the data set used to build the candidate models

coefficient of determination

A measure of the goodness of fit of the estimated regression equation. It can be interpreted as the proportion of the variability in the dependent variable y that is explained by the estimated regression equation.

least squares method

A procedure for using sample data to find the estimated regression equation.

point estimator

A single value used as an estimate of the corresponding population parameter.

Regression Analysis

A statistical procedure used to develop an equation showing how the variables are related.

best subsets

A variable selection procedure that constructs and compares all possible models with up to a specified number of independent variables.

dummy variable

A variable used to model the effect of categorical independent variables in a regression model; generally takes only the value zero or one.

confidence interval

An estimate of a population parameter that provides an interval believed to contain the value of the parameter at some level of confidence.

confidence level

An indication of how frequently interval estimates based on samples of the same size taken from the same population using identical sampling techniques will contain the true value of the parameter we are estimating.

prediction interval

An interval estimate of the prediction of an individual y value given values of the independent variables.

stepwise selection

An iterative variable selection procedure that considers adding an independent variable and removing an independent variable at each step.

backward elimination

An iterative variable selection procedure that starts with a model with all independent variables and considers removing an independent variable at each step.

forward selection

An iterative variable selection procedure that starts with a model with no variables and considers adding an independent variable at each step.

cross-validation

Assessment of the performance of a model on data other than the data that were used to generate the model.

Leave-one-out cross-validation

Method of cross-validation in which candidate models are repeatedly fit using n-1 observations and evaluated with the remaining observation.

Holdout method

Method of cross-validation in which sample data are randomly divided into mutually exclusive and collectively exhaustive sets, then one set is used to build the candidate models and the other set is used to compare model performances and ultimately select a model.

k-fold cross-validation

Method of cross-validation in which sample data set are randomly divided into k equal- sized, mutually exclusive and collectively exhaustive subsets. In each of k iterations, one of the k subsets is used to evaluate a candidate model that was constructed on the data from the other k-1 subsets.

extrapolation

Prediction of the mean value of the dependent variable y for values of the independent variables x 1 ,​ x 2 ,..., x q that are outside the experimental range.

multiple linear regression

Regression analysis involving one dependent variable and more than one independent variable.

simple linear regression

Regression analysis involving one dependent variable and one independent variable.

piecewise linear regression model

Regression model in which one linear relationship between the independent and dependent variables is fit for values of the independent variable below a prespecified value of the independent variable, a different linear relationship between the independent and dependent variables is fit for values of the independent variable above the prespecified value of the independent variable, and the two regressions have the same estimated value of the dependent variable (i.e., are joined) at the prespecified value of the independent variable.

interaction

Regression modeling technique used when the relationship between the dependent variable and one independent variable is different at different values of a second independent variable.

t-test

Statistical test based on the Student's t probability distribution that can be used to test the hypothesis that a regression parameter β j is zero; if this hypothesis is rejected, we conclude that there is a regression relationship between the jth independent variable and the dependent variable.

validation set

The data set used to compare model forecasts and ultimately pick a model for predicting values of the dependent variable.

multicollinearity

The degree of correlation among independent variables in a regression model.

residual

The difference between the observed value of the dependent variable and the value predicted using the estimated regression equation; for the i th observation, the i th residual is y i - y ^ i

estimated regression

The estimate of the regression equation developed from sample data by using the least squares method. The estimated multiple linear regression equation is y ^ = b 0 + b 1 x 1 + b 2 x 2 +...+ b q x q .

knot

The prespecified value of the independent variable at which its relationship with the dependent variable changes in a piecewise linear regression model; also called the breakpoint or the joint.

hypothesis testing

The process of making a conjecture about the value of a population parameter, collecting sample data that can be used to assess this conjecture, measuring the strength of the evidence against the conjecture that is provided by the sample, and using these results to draw a conclusion about the

Statistical inference

The process of making estimates and drawing conclusions about one or more characteristics of a population (the value of one or more parameters) through analysis of sample data drawn from the population.

experimental region

The range of values for the independent variables x 1 ,​ x 2 ,..., x q for the data that are used to estimate the regression model.

interval estimation

The use of sample data to calculate a range of values that is believed to include the unknown value of a population parameter.

dependent variable

The variable that is being predicted or explained. It is denoted by y and is often referred to as the response.

independent variables

The variable(s) used for predicting or explaining values of the dependent variable. It is denoted by x and is often referred to as the predictor variable.