Data Mining Exam 1: Lecture 4

A drawback of using validation set is that there will be less data available for training True False

True

Which of the following options is/are true for K-fold cross-validation? A. Increase in K will result in higher time required to cross validate the result. B. Higher values of K will result in higher confidence on the cross-validation result as compared to lower value of K C. If K=N, then it is called Leave one out cross validation, where N is the number of observations. D. All of the above.

D. All of the above

Validation Set

Divide training data into 2 parts: Training set & Validation Set

A classifier trained on less training data is less likely to overfit. True False

False

Decision tree is learned by minimizing information gain. True False

False

Training error is more important than test error when evaluating the performance of a model. True False

False

Identifying the best attributes in a decision tree:

We look at the ID3 heuristic which splits attributes based on their entropy

Overfitting

When the model is too complex, training error is small but test error is large

Underfitting

When the model is too simple, both training and test errors are large

Ensemble Classifiers List:

-Boosting -Bagging -Random Forests

Base Classifiers List:

-Decision tree based methods -Rule-based methods -Nearest-neighbor -Neural Networks -Naive Bayes and Bayesian Belief Networks -Support Vector Machines

Classification Errors include

-Training errors (apparent errors) -Test errors (errors committed on the test set) -Generalization errors (expected error of a model over random selection of records from same dist.)

Cross Validation

-partition data into k disjoint subsets k-fold: train on k-1 partitions, test on remaining one -Leave-one-out: k=n

What is a decision tree?

An inductive learning task - using particular facts to make more generalized conclusions A predictive model based on a branching series of boolean tests

Below are the 8 actual values of a target variable. [0,0,0,1,1,1,1,1] What is the correct way to compute the entropy of the target variable?

B. -5/8 log2(5/8) + (-3/8 log2(3/8))

Which one of the following statements regarding decision trees is NOT correct? A. Decision Tree is an inductive learning task that uses particular facts to make more generalized conclusions. B. Decision trees work more effectively with continuous attributes. C. The time for building a tree may be higher than another type of classifier. D. The trees may suffer from error propagation.

B. Decision trees work more effectively with continuous attributes.

Classification Techniques include

Base Classifiers Ensemble Classifiers

Decision trees can be represented as rules instead of graphically

But, this may be much harder to read

Which of the following is NOT true when identifying the best attributes for Decision Tree? A. Choose the best attribute(s) to split the remaining instances and make that attribute a decision node B. ID3 heuristic can help to determine the best attribute C. ID3 splits based on attributes with the highest entropy D. ID3 must use discrete (or discretized) attributes

C. ID3 splits based on attributes with the highest entropy

Entropy is minimized when

all values of the target attribute are the same. Ex: if we know that commute time will always be short, then entropy = 0

Entropy is

the measure if disinformation

ID3 splits on attributes with the lowest entropy and we calculate the entropy for all values of an attribute as

the weighted sum of subset entropies

Entropy is maximized when

there is an equal chance of all values for the target attribute (i.e. the result is totally random and equally likely)

We can also measure information gain

which is inversely proportional to entropy