李宏毅ML lecture-15 unsupervised Learning Neighbor Embedding

李宏毅ML lecture-15 unsupervised Learning Neighbor Embedding

• Manifold Learning(流体学习)
• Locally Linear Embedding (LLE)
• Laplacian Eigenmaps
• T-distributed Stochastic Neighbor Embedding (t-SNE)

t-SNE

Manifold Learning(流体学习)

非线性降维

Locally Linear Embedding (LLE)

假设一个点可以由他周围的点线性组成,周围的点要足够近才有效.
存在一个权重描述这个事情.

当把$x$x降维之后变为$z$z,降维的function没有特别定义
保持权重不变
$w_{i,j}$wi,j​已知后,找到一组集合$\{z\}${z},使得下面这个公式最小.

对相关的邻居的数量敏感.

邻居太少无法挖掘出之间的相关关系.
邻居太多,过拟合.

Lawrence K. Saul, Sam T. Roweis, “Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifolds”, JMLR, 2013

Laplacian Eigenmaps

半监督学习如下:


无监督学习如下:

Belkin, M., Niyogi, P. Laplacian eigenmaps and spectral techniques for embedding and clustering. Advances in neural information processing systems . 2002

T-distributed Stochastic Neighbor Embedding (t-SNE)

Locally Linear Embedding (LLE): [Alpaydin, Chapter 6.11]
Laplacian Eigenmaps: [Alpaydin, Chapter 6.12]
t-SNE
Laurens van der Maaten, Geoffrey Hinton, “Visualizing Data using t-SNE”, JMLR, 2008
Excellent tutorial: https://github.com/oreillymedia/t-SNE-tutorial