Scribd
Upload
117 views

Support Vector Machine

Support Vector Machine (SVM) is a supervised machine learning algorithm used for both classification and regression problems. It finds the optimal separating hyperplane that maximizes the margin between two classes of data points. The data points that lie closest to this hyperplane are called the support vectors, and they are important for defining the nature of the separation. There are linear and non-linear versions of SVM depending on whether the classes can be separated by a straight line or more complex decision boundaries are required.

Uploaded by

Prashant Sahu
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
117 views

Support Vector Machine

Support Vector Machine (SVM) is a supervised machine learning algorithm used for both classification and regression problems. It finds the optimal separating hyperplane that maximizes the margin between two classes of data points. The data points that lie closest to this hyperplane are called the support vectors, and they are important for defining the nature of the separation. There are linear and non-linear versions of SVM depending on whether the classes can be separated by a straight line or more complex decision boundaries are required.

Uploaded by

Prashant Sahu
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 12

Support Vector Machine

SVM
Support Vector Machine or SVM is one of the most popular Supervised
Learning algorithms, which is used for Classification as well as Regression
problems.
However, primarily, it is used for Classification problems in Machine
Learning.
The goal of the SVM algorithm is to create the best line or decision
boundary that can segregate n-dimensional space into classes so that we
can easily put the new data point in the correct category in the future.
This best decision boundary is called a hyperplane.
SVM

SVM chooses the extreme


points/vectors that help in
creating the hyperplane. These
extreme cases are called as
support vectors, and hence
algorithm is termed as Support
Vector Machine. Consider the
below diagram in which there are
two different categories that are
classified using a decision
boundary or hyperplane:
Example
• SVM can be understood with the example that we have used in the KNN classifier.
• Suppose we see a strange cat that also has some features of dogs, so if we want a model that can accurately
identify whether it is a cat or dog, so such a model can be created by using the SVM algorithm.
• We will first train our model with lots of images of cats and dogs so that it can learn about different features of
cats and dogs, and then we test it with this strange creature.
• So as support vector creates a decision boundary between these two data (cat and dog) and choose extreme cases
(support vectors), it will see the extreme case of cat and dog.
• On the basis of the support vectors, it will classify it as a cat. Consider the below diagram:

SVM algorithm can be used for Face detection, image classification, text categorization, etc.
Types Of SVM
SVM can be of two types:
•Linear SVM: Linear SVM is used for linearly separable data, which means if a dataset can be classified into two
classes by using a single straight line, then such data is termed as linearly separable data, and classifier is used
called as Linear SVM classifier.
•Non-linear SVM: Non-Linear SVM is used for non-linearly separated data, which means if a dataset cannot be
classified by using a straight line, then such data is termed as non-linear data and classifier used is called as Non-
linear SVM classifier.
Hyperplane and Support Vectors in the SVM
algorithm:
Hyperplane: There can be multiple lines/decision boundaries to segregate the classes in n-dimensional space, but
we need to find out the best decision boundary that helps to classify the data points. This best boundary is known
as the hyperplane of SVM.
The dimensions of the hyperplane depend on the features present in the dataset, which means if there are 2
features (as shown in image), then hyperplane will be a straight line. And if there are 3 features, then hyperplane
will be a 2-dimension plane.
We always create a hyperplane that has a maximum margin, which means the maximum distance between the
data points.

Support Vectors:
The data points or vectors that are the closest to the hyperplane and which affect the position of the hyperplane are
termed as Support Vector. Since these vectors support the hyperplane, hence called a Support vector.
How does SVM works? Linear SVM
The working of the SVM So as it is 2-d space so by
algorithm can be understood by just using a straight line, we
using an example. Suppose we can easily separate these
have a dataset that has two tags two classes. But there can
(green and blue), and the dataset be multiple lines that can
has two features x1 and x2. We separate these classes.
want a classifier that can classify Consider the below image:
the pair(x1, x2) of coordinates in
either green or blue. Consider
the below image:
Optimal Hyperplane
Hence, the SVM algorithm helps to find the best line or decision boundary; this best boundary or region is called
as a hyperplane. SVM algorithm finds the closest point of the lines from both the classes. These points are called
support vectors. The distance between the vectors and the hyperplane is called as margin. And the goal of SVM
is to maximize this margin. The hyperplane with maximum margin is called the optimal hyperplane.
Non-linear SVM
If data is linearly arranged, then we can separate it by using a straight line, but for non-linear data, we cannot
draw a single straight line. Consider the below image:

So to separate these data points, we need to add one


more dimension. For linear data, we have used two
dimensions x and y, so for non-linear data, we will
add a third dimension z. It can be calculated as:
z=x2 +y2

By adding the third dimension, the


sample space will become as image
given here:
Non-linear SVM
So now, SVM will divide the Since we are in 3-d Space, hence it is looking like a plane
datasets into classes in the parallel to the x-axis. If we convert it in 2d space with z=1, then
following way it will become as:
Kernel
•All values for z would be positive always because z is the squared sum of both x and y
•In the original plot, green circles appear close to the origin of x and y axes, leading to lower
value of z and blue triangle relatively away from the origin result to higher value of z.

In the SVM classifier, it is easy to have a linear hyper-plane between these two classes. But,
another burning question which arises is, should we need to add this feature manually to have a
hyper-plane. No, the SVM algorithm has a technique called the kernel trick

The SVM kernel is a function that takes low dimensional input space and transforms it to a higher
dimensional space i.e. it converts not separable problem to separable problem.
It is mostly useful in non-linear separation problem.
Simply put, it does some extremely complex data transformations, then finds out the process to
separate the data based on the labels or outputs you’ve defined.

When we look at the hyper-plane in original input space it


looks like a circle:
Advantage and Disadvantage
•Pros:
• It works really well with a clear margin of separation
• It is effective in high dimensional spaces.
• It is effective in cases where the number of dimensions is greater than the
number of samples.
• It uses a subset of training points in the decision function (called support
vectors), so it is also memory efficient.
•Cons:
• It doesn’t perform well when we have large data set because the required
training time is higher
• It also doesn’t perform very well, when the data set has more noise i.e.
target classes are overlapping
• SVM doesn’t directly provide probability estimates, these are calculated
using an expensive five-fold cross-validation. It is included in the related
SVC method of Python scikit-learn library.

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy