Support Vector Machine (SVM)

Kernel Function

A kernel function is a mathematical function used SVM to transform the input data into a higher dimensional space, where it becomes possible to find a linear or non-linear decision boundary to separate the classes.

Sigmoid kernel

A kernel function that maps the data into a higher-dimensional feature space using a sigmoid function. It is used for binary classification problems.

Polynomial kernel

A kernel function that transforms the data into a higher-dimensional feature space by creating polynomial combinations of the original features.

Radial basis function (RBF) kernel

A non-linear kernel function that maps the data into a higher-dimensional feature space based on the distance between instances. This is the most widely used kernel function for non-linear data.

Linear kernel:

A simple kernel function that performs a linear transformation of the input data. It is used when the data is linearly separable in the original feature space.

Transforming data into a higher dimensional space will be computationally expensive. What is the solution?

Kernal trick

In real life, in many cases, we don't have linearly separable data. How do we use SVM for 'not linearly separable' data?

Kernel Function

The most commonly used kernel functions:

Linear Kernel Polynomial kernel Radial basis function (RBF) kernel Sigmoid kernel: A

Disadvantages of SVM (2)

SVM can be computationally expensive and may not be suitable for large datasets or real-time applications.

Advantages of SVM (2)

SVM is a robust algorithm that works well for many types of data and is particularly useful for problems where the data is not linearly separable in the original feature space. It is also a versatile algorithm, capable of performing both binary and multi-class classification, as well as regression tasks.

What are the most important data points in the SVM model?

Support vectors

Which is the Minus-Plane?

The purple line

Which is the Plus-Plane?`

The yellow line

Kernel trick

a computational shortcut that allows the algorithm to perform complex non-linear transformations of the input data into a higherdimensional feature space without actually computing the transformation explicitly. • Instead, it looks at the relations between data points.

The support vectors play a crucial role in determining the ___________, as ______________.

accuracy of the SVM model, as the optimal hyperplane is built around them.

Any change in the position of the support vectors will result in a ...

change in the position of the decision boundary

The main goal of SVM is to...

find the optimal hyper plane (decision boundary) that separates the class.

SVM have a very clever way to use a __________ without ______________________.

huge number of features requiring nearly as much computation as seems to be necessary

The kernel function returns the_______________________., where the data can be _______________________.

inner product of the two instances in the transformed space linearly or non-linearly separated.

Support Vectors are the data points that ...

lie on the margin or closest to the decision boundary

The optimal hyperplane is the one that ...

maximizes the margin between the classes

The kernel function returns the inner product of the two instances in the transformed space, where the data can be linearly or non-linearly separated. • This enables SVMs to _________________ without _____________________.

perform complex non-linear classification and regression tasks actually computing the explicit mapping of the input data into a higher dimensional space.

The kernel trick is a computational shortcut that allows the algorithm to _________________________ into a ________________ without___________________. • Instead, it looks at the _______________

perform complex non-linear transformations of the input data higherdimensional feature space actually computing the transformation explicitly. relations between data points.

The key idea behind the kernel trick is to ____________________ with a _________________________________

replace the inner product between two vectors in the original input space kernel function that measures the similarity between two instances.

Support vectors are used to...

to define the decision boundary that separates the classes

Key idea behind the kernel trick

to replace the inner product between two vectors in the original input space with a kernel function that measures the similarity between two instances.

A kernel function is a mathematical function used in SVM to _________________________, where it ____________________, to ______________.

transform the input data into a higher dimensional space becomes possible to find a linear or non-linear decision boundary separate the classes.