机器学习笔记 ---- Support Vector Machines

Support Vector Machines

1. Cost Function

J(θ)=C[∑i=1my(i)cost1(θTx(i))+(1−y(i))cost0(θTx(i))]+12∑j=1nθ2j$$J(\theta )=C\left[\sum _{i=1}^m{y}^{(i)}{\text{cost}}_1({\theta }^T{x}^{(i)})+(1-y^{(i)}){\text{cost}}_0({\theta }^T{x}^{(i)})\right]+\frac{1}{2}\sum _{j=1}^n{\theta }_j^2$$

2. Hypothesis

f(x)={10if θTx>=0otherwise$$f(x)=\{\begin{array}{ll}1& \text{if }{\theta }^T\text{x>=0}\\ 0& \text{otherwise}\end{array}$$

3. Margin of SVM

4. Kernels

Define landmarks l$l$



Using f$f$ and θ$\theta$ when making predictions..

5. How to Get Landmarks

One way is to use the first m training examples.

6. The Effects of Parameters in SVM

1) For C$C$
Large C : λ small, low bias, high variance
Small C : λ big, high bias, low variance
2) For σ2${\sigma }^2$
Large σ2${\sigma }^2$ : high bias, low variance
Small σ2${\sigma }^2$ : low bias, high variance

7. Choice of Kernal

Need to satisfy Mercer’s Theorem.
1) No kernal (Linear Kernal)
when n is large/ n is small && m is large
2) Gaussian Kernal
when n is small, m is intermediate
Need to use feature scaling before using!
3) Other Alternative Choices:
Polynomial Kernal, String Kernal, Chi-Square Kernal, Intersection Kernal…