Practice Quiz Module 4 Part III

Which of the following dataset is used for trying out different models and selecting a best model? validation data training data testing data

validation data

Every hyperparameter, if set poorly, can have a huge negative impact on training, and so all hyperparameters are about equally important to tune well. True or False?

False

If searching among a large number of hyperparameters, you should try values in a grid rather than random values, so that you can carry out the search more systematically and not rely on chance. True or False?

False

Why do we hate to use Validation set approach ? We lose big amount of Time by Training the Model on it We lose big amount of Data by not training the model on it

We lose big amount of Data by not training the model on it

To tune our hyper parameters we use ............... LOOCV Binary Search Grid Search

Grid Search

Which of the following dataset is used for final evaluation of the chosen model? training data testing data validation data

testing data

Which of the following dataset is used for building or fitting a model? testing data training data validation data

training data

If you have 10,000,000 examples, how would you split the train/validation/test set? 33% train, 33% validation, 33% test 98% train, 1% validation, 1% test

98% train, 1% validation, 1% test

Which of the following is correct use of cross validation? Selecting variables to include in a model Comparing predictors Selecting parameters in prediction function Correct answer: All of the mentioned

All of the mentioned

You trained a model and found out the error on your training data is zero. Which of the following is the reason for that? Model may probably be underfitting Model may probably be overfitting

Model may probably be overfitting

An ML scientist has built a decision tree model using scikit-learn with 1,000 trees. The training accuracy for the model was 99.2% and the test accuracy was 70.3%. Should the Scientist use this model in production? Yes, because it is not generalizing well on the test set Yes, because it is generalizing well on the training set No, because it is generalizing well on the training set No, because it is not generalizing well on the test set

No, because it is not generalizing well on the test set

What is the primary reason that one might want to pick random search over grid search when performing hyperparameter optimization? Deterministic policies are unable to handle uncertainty in the data Random search can explore more parameter space than grid search Random search leave smaller unexplored regions than grid searches Random search can be parallelized, whereas grid search cannot

Random search leave smaller unexplored regions than grid searches

A Machine Learning Engineer is creating a regression model for forecasting company revenue based on an internal dataset made up of past sales and other related data. What metric should the Engineer use to evaluate the ML model? Precision Cross-entropy log loss Root Mean squared error (RMSE) Sigmoid

Root Mean squared error (RMSE)

During hyperparameter search, whether you train one model or train a lot of models is largely determined by: The presence of local minima (and saddle points) in your neural network The number of hyperparameters you have to tune Whether you use batch or mini-batch optimization The amount of computational power you can access

The amount of computational power you can access

Suppose you have picked the parameter for a model using 10-fold cross validation (CV). Which of the following is the best way to pick a final model to use and estimate its error? Average all of the 10 models you got; use the average CV error as its error estimate Train a new model on the full data set, using the parameter you found; use the average CV error as its error estimate Average all of the 10 models you got; use the error the combined model gives on the full training set Pick any of the 10 models you built for your model; use its error estimate on the held-out data

Train a new model on the full data set, using the parameter you found; use the average CV error as its error estimate

You can save the trained machine learning model using Scikit-Learn's joblib. True False

True

You need to write monitoring code to check your system's live performance at regular intervals and trigger alerts when it drops.

True

Leaving 50 % of the Data to validate the model on it is considered as LOPCV Validation Set Approach LOOCV K Fold Cross Validation

Validation Set Approach

What does CV stand for in GridSearchCV and why? cross-validation: once we found the best parameters we estimate the model performance through cross-validation on the full data contribution value: we estimate how much each parameter contributes to the model generalization performance cross-validation: the score of each combination of parameters on the grid is computed by using an internal cross-validation procedure circular values: we do a permutation of all the possible parameter value combinations

cross-validation: the score of each combination of parameters on the grid is computed by using an internal cross-validation procedure

Which of the following are the main ways to fix under-fitting? feed the training algorithm with better features gather more training data select a more powerful model reduce the constraints on the model

feed the training algorithm with better features select a more powerful model reduce the constraints on the model