Introduction to Machine Learning Part 3

Which of the following accurately describes the two main types of ensemble learning methods?

Aggregation of heterogeneous learners and homogeneous learners

Which of the following statements is NOT correct about the AdaBoost? - The weak learners are ALL decision stumps - Each stump depends on the previous tree's errors rather than being independent. - All the instances (training observations) get the SAME weight during the entire AdaBoosting process. - Misclassified observations are assigned HIGHER weights

All the instances (training observations) get the SAME weight during the entire AdaBoosting process.

Which of the following accurately describes bagging and boosting?

Bagging creates multiple copies of the training data and applies the weak learner to each copy to obtain multiple weak models, while boosting uses the original training data and iteratively creates multiple models by using a weak learner.

Which of the following models is not using a decision tree as a weak learner? - AdaBoost - Gradient Boosting Machine - XGBoost - Decision Trees

Decision Trees (DTs)

What are the two main types of unsupervised learning algorithms?

Dimension reduction algorithm and Clustering techniques

What is the main characteristic of unsupervised learning in machine learning?

Does not use labeled data (no target variable)

What are some benefits of using dimension reduction techniques, such as PCA, in machine learning?

Easier data visualization, pattern identification, and reduced overfitting

Which of the following statements accurately describes ensemble learning?

Ensemble learning involves combining the predictions from a collection of models to improve accuracy.

Feature importance refers to techniques that assign a score to input features based on how useful they are at predicting a target variable.

Feature importance refers to techniques that assign a score to input features based on how useful they are at predicting a target variable.

How does Principal Component Analysis (PCA) order the eigenvectors according to their usefulness in explaining the total variance in the initial data?

From highest to lowest eigenvalues

From the Elements of Statistical Learning (aka the Bible of Machine Learning and btw our main textbook): Trees have one aspect that prevents them from being the ideal tool for predictive learning, namely -------------

Inaccuracy

Which of the following statements is NOT correct about the Gradient Boosting Machines (GBM)? - In gradient boosting, each weak learner corrects its predecessor's error. - Unlike AdaBoost, the weights of the training instances are not tweaked, instead, each predictor is trained using the residual errors of predecessor as labels. - Unlike AdaBoost, each tree can be larger than a stump. - It can only be applied to classification problems.

It can only be applied to classification problems.

Which of the following statements is true about XGBoost?

It is a customized version of gradient boosting decision trees focusing on performance and speed.

How does Principal Component Analysis (PCA) differ from linear regression in terms of their objectives and methodologies?

PCA focuses on reducing dimensions and explaining variance in data, while linear regression predicts target variables based on input feature

Which of the following statements accurately describes random forests?

Random forests combine the simplicity of decision trees with flexibility by creating a bootstrapped dataset, resulting in a vast improvement in accuracy.

What is the main difference between random forests and bagging?

Random forests use a modified tree learning algorithm that inspects a random subset of the features, while bagging inspects the entire feature space.

What is the relationship between the Proportion Variance Explained (PVE) values of all the principal components in a dataset?

The PVE values sum to one.

What is a potential drawback of using dimension reduction techniques like PCA in machine learning?

The dimension reduction process can be difficult to interpret

What do the eigenvectors of the covariance matrix represent in the context of Principal Component Analysis (PCA)?

The directions of the new feature space

In the context of Principal Component Analysis, what is the relationship between the first and second principal components and the variance in the data?

The first principal component captures the highest variance, while the second principal component is orthogonal to the first and captures the second highest variance.

Which of the following is not a hyperparameter in random forest? - The number of observations in the same neighborhood (K) - The number of trees to use (B) - The minimum size of each node (or leaf) - The maximum depth of each tree

The number of observations in the same neighborhood (K)

What information does a scree plot provide in the context of Principal Component Analysis (PCA)?

The proportion of total variance in the data explained by each principal component

How do eigenvalues contribute to the interpretation of the data in PCA?

They explain the variance of the data along the new feature axes.

What is the goal of Principal Component Analysis (PCA) in terms of projection errors and spread?

To minimize projection errors and maximize spread

What is the primary goal of dimension reduction in machine learning?

To represent a dataset with many correlated features by a smaller set of features that still effectively describe the data

Random forest is an example of ensemble machine learning algorithm meaning that a group of weak learners (individual trees) come together and build a strong learner (the forest) with a better performance in general. True or False

True

What is a potential advantage of using XGBoost over other gradient boosting frameworks?

XGBoost can handle missing data more efficiently

What is a potential disadvantage of using XGBoost?

XGBoost has a higher risk of overfitting compared to other frameworks.

Which of the followings is not an advantage of a Random Forest model? - Handles missing values and categorical data - Handles large dataset with high dimensionality - Used both for regression and classification - easy to interpret and determine variable significance

easy to interpret and determine variable significance

Out-of-bag error is defined as:

the proportion of out-of-bag samples that were incorrectly classified