Skip to main content

The most powerful kernel trick of SVM

 The kernel trick is most important and powerful technique of SVM .

Linear VS Non-Linear dataset

linear vs non-linear figure

Problem Statement

Currently we have learn how to apply SVM algorithm at linear datasets, but what if we have non linear dataset.

Solution of Problem

Solution is kernel trick.


Kernel Trick

The Kernel trick is trick where we add  many SVMS  models by bagging,voting,stacking and boosting or we can use SVM class to implement it.


Implementation

To implement it follow code given below-
    
    from sklearn.svm import SVC
    svc=SVC()
    svc.fit(X_train,y_train)
    svc.score(X_test,y_test)
  

Comments

Popular posts from this blog

Loss function

Loss function Loss function is Quantity helps to find loss of our model when ever it is greater models perform poor else model perform good. Today we learnt about  Loss function in regression in other class we discus  Loss function in classification There are Several Loss function in regression MSE(Mean Squared Error) MAE(Mean Absolute Error) RMSE(Root Mean Squared Error) R2 Score MSE MSE formula --> 𝛴 (y i -p) 2 Note - MSE is effected by outlier MAE MAE formula --> Σ | y i -p | RMSE RMSE formula --> √Σ (y i -p) 2 R2 Score R2 Score formula --> 1-RSS/TSS RSS=Sum of Square of Residual TSS=Total Sum of Square

Mathematics of Ridge regression

 Mathematics of Ridge regression Let's get started with "What do we have to do?" In such case of simple linear regression(OLS).b=y_mean-m*x_mean and m=∑(Xi-X_mean)(yi-y_mean)/ ∑(Xi-X_mean) 2 In such a case of ridge we derive everything again but we don't derive b because in ridge we reduce m not b will be the same as well. 1. Choose a loss function.In this case I choose L=𝜮(yi-p) 2 2. Calculate dL/dm= ∑(Xi-X_mean)(yi-y_mean)/ ∑(Xi-X_mean) 2 +λ to minimize loss Then m= ∑(Xi-X_mean)(yi-y_mean)/ ∑(Xi-X_mean) 2 +λ And in this case, you clearly see if we increase λ m decrease but it affects b also but not that much And λ is a hyperparameter in whole Regularization