decision trees pt 3 , neural networks, and whatever is missing

Pataasin ang iyong marka sa homework at exams ngayon gamit ang Quizwiz!

•precision (

% of predicted positives that are actually positives)

•recall

(% of positives falling in the predicted positive class)

•log-likelihood

(probability of observing the data given the model)

•Design of a learning element is affected by -Which components of the performance element are to be learned

-What feedback is available to learn these components -What representation is used for the components

Weakness of K means clustering

Weaknesses: Sensitive to initialization: K-means is sensitive to the initial placement of centroids, which can lead to different results for different initializations. This can be mitigated by running the algorithm multiple times with different initializations. Assumes spherical clusters: K-means assumes that the clusters are spherical in shape and have equal variance, which may not always be the case in real-world datasets. Requires predefined number of clusters: K-means requires the user to specify the number of clusters to be created, which can be challenging in situations where the optimal number of clusters is not known in advance. Can converge to local optima: K-means can converge to local optima, which may not be the global optimum. To mitigate this, multiple runs with different initializations are recommended.

kmeans algo

1. Choose the number of clusters k 2. Select k random points from the data as centroids 3. Assign all the points to the closest cluster centroid 4. Recompute the centroids of newly formed clusters

How many distinct decision trees with n Boolean attributes? = number of Boolean functions

= number of distinct truth tables with 2n rows = 2^2^n

Bernoulli Distribution

A Bernoulli distribution is a discrete probability distribution for a Bernoulli trial — a random experiment that has only two outcomes (usually called a "Success" or a "Failure"). For example, the probability of getting a heads (a "success") while flipping a coin is 0.5. The probability of "failure" is 1 - P (1 minus the probability of success, which also equals 0.5 for a coin toss).

naive bayes cons

Assumes independence of features: Limited expressiveness: Naive Bayes is a linear classifier, which means it cannot capture complex relationships between features. Sensitivity to input data: Naive Bayes is sensitive to input data and can be affected by outliers and noise in the data. Cannot handle continuous features: Naive Bayes assumes that the features are discrete and categorical, which can limit its usefulness in some applications. May require a large number of examples: In some cases, Naive Bayes may require a large number of examples to accurately estimate the probabilities of different classes.

expected value for a random value

E[X]=p variance VAR[X]=p(1-p)

IG E(parent|energy) 30 instances. 16 gym 14 dont

Eparent=(-16/30log2(16/30))-(14/30log2(14/30))

gini impurity• suppose a particular split separates the data into two groups:• group A: 1 (12) and 0 (5)

GI_A=1-(12/17)^2-(5/17)^2

•To implement Choose-Attribute in the DTL algorithm

IG which feature should be root/after split

Train" model with stochastic gradientdescent for neural netowkr s

In standard Gradient Descent, the objective is to find the minimum of a cost function by iteratively updating the model parameters in the opposite direction of the gradient of the cost function. In contrast, SGD updates the model parameters for each training example (or a small subset of training examples) in a random fashion, rather than using the entire dataset to compute the gradient at each step.

Polytomous logistic regression (PLR) is a statistical technique used for modeling relationships between a categorical dependent variable with more than two categories and one or more independent variables.

It is an extension of binary logistic regression, which is used for modeling binary outcomes (i.e., outcomes with two categories) PLR models the relationship between the dependent variable and the independent variables by estimating separate sets of coefficients for each category of the dependent variable. The coefficients for each category are then compared to estimate the odds of one category compared to another, with a reference category usually chosen for comparison..

kmeans clustering

K-means clustering is a popular unsupervised machine learning algorithm used for clustering similar data points into a fixed number of groups or clusters. It is a simple and fast algorithm The algorithm works by iteratively assigning each data point to the nearest cluster centroid, and then updating the centroid based on the mean of the assigned points. The process continues until the cluster assignments no longer change or a maximum number of iterations is reached.

maximum likelihood for Bernoulli: the maximum likelihood method can be used to estimate the unknown parameter p, which represents the probability of success in a single trial

L(p|x) = p^k * (1-p)^(n-k) where k is the number of successes in the sample and (n-k) is the number of failures

The main advantage of SGD over Gradient Descent is its computational efficiency, especially for large-scale datasets. By updating the parameters for each training example (or a small subset of examples),

SGD can converge much faster than Gradient Descent, as it takes advantage of the stochastic nature of the training data to avoid getting stuck in local minima. However, the resulting trajectory may be more noisy than the one obtained from the full batch method.

Naive Bayes pros

Simple and easy to implement: Fast and efficient: Good performance: Handles irrelevant features: Can work with small datasets: Naive Bayes can work well with small datasets, as it can handle missing data and can perform well with a small number of examples.

trengths/weaknesses• simple to compute, method is easy to explain• need to specify K, which may be unknown• only finds convex clusters STRENGTHS:

Strengths: Scalability: K-means is a relatively simple and fast algorithm that can be applied to large datasets. Ease of use: K-means is easy to implement and can be applied to a wide range of problems, making it a popular choice for clustering tasks. Clusters are well separated: K-means tends to produce well-separated clusters, which can make it easy to interpret and understand the results.

Maximum Likelihood Trainin

Tends to overfit • Not defined if d > n • Feature selection

clustering

want to group data, may not necessarily havelabels

Naïve Bayes Prediction •Usually

add a small constant (e.g. 0.5) to avoid divide by zero problems and to reduce bi

back propagation starts with output and works backward

adjust wts of neural network so minimze different bet predicted output and actual out

Training a model on a dataset refers to the process of using a set of input data (features)

along with their corresponding output labels to teach the model to make accurate predictions.

The decision tree algorithm starts with the entire training dataset at the root node, and recursively splits the data into smaller subsets based on the values of the input features. At each node, the algorithm selects the feature

and the threshold value that best separates the data into different classes. The process continues until the subsets are pure, meaning that all instances in a subset belong to the same class, or until a stopping criterion is met.

Learning curve = % correct on test set

as a function of training set size

Maximum likelihood is a statistical method used to

estimate the parameters of a probability distribution that best explain the observed data. The maximum likelihood method involves finding the set of parameter values that maximize the likelihood function, which is a function that measures the goodness of fit between the observed data and the theoretical probability distribution

RBF nEURAL nETWORK

for function approximation

GLM stands for Generalized Linear Models, which is a class of regression models that extends the linear regression framework to

handle non-normal response variables and non-constant variance. GLMs can handle response variables that follow different distributions such as binomial, Poisson, and gamma.

In machine learning, learning refers to the process of training a model on a dataset

in order to make accurate predictions on new, unseen data. The goal of learning is to enable the model to capture the underlying patterns and relationships in the data, so that it can generalize well to new data.

Patrons has the highest IG of all attributes and so

is chosen by the DTL algorithm as the root

To find the maximum likelihood estimate of the parameters, we need to

maximize the likelihood function with respect to θ. This is often done by taking the logarithm of the likelihood function and then differentiating it with respect to the parameters to obtain the maximum likelihood estimates. The resulting estimates are the values of θ that make the observed data most likely under the assumed probability distribution.

rationale for using neural network

nonlineraity uncertainity complexity parallel/distributed processing fault tolerance

CNN is a neural network with some convolutional layers(and some other layers). A convolutional layer has a

numberof filters that does convolutional operation

n simple terms, a Naive Bayes classifier assumes that the presence of a particular feature in a class is unrelated to the presence of any other feature.

or example, a fruit may be considered to be an apple if it is red, round, and about 3 inches in diameter. Even if these features depend on each other or upon the existence of the other features, all of these properties independently contribute to the probability that this fruit is an apple and that is why it is known as 'Naive'.

Then, we use the glm() function to fit a logistic regression model, where response is the binary response variable and

predictor1 and predictor2 are the two predictor variables.

An important part of every Bernoulli trial is that each action must be independent. That means the

probabilities must remain the same throughout the trials; each event must be completely separate and have nothing to do with the previous event.


Kaugnay na mga set ng pag-aaral

Peds Exam Three Practice Questions

View Set

Chapter 1 A Framework for Maternal and Child Health Nursing

View Set

AI/Machine Learning (15) AND Cognition/Emotion/Therapy (16)

View Set

Chapter 11: the gallbladder and biliary system (practice test)

View Set

The Copyeditor's Handbook Glossary of Copyediting Terms

View Set