Learning eigenfunctions links spectral embedding and kernel {PCA}
| Type of publication: | Article |
| Citation: | Bengio-eigen-NC2004 |
| Journal: | Neural Computation |
| Volume: | 16 |
| Number: | 10 |
| Year: | 2004 |
| Pages: | 2197--2219 |
| URL: | http://www.iro.umontreal.ca/~l... |
| Abstract: | In this paper, we show a direct relation between spectral embedding methods and kernel PCA, and how both are special cases of a more general learning problem, that of learning the principal eigenfunctions of an operator defined from a kernel and the unknown data generating density. Whereas spectral embedding methods only provided coordinates for the training points, the analysis justifies a simple extension to out-of-sample examples (the Nyström formula) for Multi-Dimensional Scaling, spectral clustering, Laplacian eigenmaps, Locally Linear Embedding (LLE) and Isomap. The analysis provides, for all such spectral embedding methods, the definition of a loss function, whose empirical average is minimized by the traditional algorithms. The asymptotic expected value of that loss defines a generalization performance and clarifies what these algorithms are trying to learn. Experiments with LLE, Isomap, spectral clustering and MDS show that this out-of-sample embedding formula generalizes well, with a level of error comparable to the effect of small perturbations of the training set on the embedding. |
| Userfields: | topics={HighDimensional,Kernel,Unsupervised},cat={J}, |
| Keywords: | |
| Authors | |
| Added by: | [ADM] |
| Total mark: | 0 |
|
Attachments
|
|
|
Notes
|
|
|
|
|
|
Topics
|
|
