[BibTeX] [RIS]
Learning eigenfunctions links spectral embedding and kernel {PCA}
Type of publication: Article
Citation: Bengio-eigen-NC2004
Journal: Neural Computation
Volume: 16
Number: 10
Year: 2004
Pages: 2197--2219
URL: http://www.iro.umontreal.ca/~l...
Abstract: In this paper, we show a direct relation between spectral embedding methods and kernel PCA, and how both are special cases of a more general learning problem, that of learning the principal eigenfunctions of an operator defined from a kernel and the unknown data generating density. Whereas spectral embedding methods only provided coordinates for the training points, the analysis justifies a simple extension to out-of-sample examples (the Nyström formula) for Multi-Dimensional Scaling, spectral clustering, Laplacian eigenmaps, Locally Linear Embedding (LLE) and Isomap. The analysis provides, for all such spectral embedding methods, the definition of a loss function, whose empirical average is minimized by the traditional algorithms. The asymptotic expected value of that loss defines a generalization performance and clarifies what these algorithms are trying to learn. Experiments with LLE, Isomap, spectral clustering and MDS show that this out-of-sample embedding formula generalizes well, with a level of error comparable to the effect of small perturbations of the training set on the embedding.
Userfields: topics={HighDimensional,Kernel,Unsupervised},cat={J},
Keywords:
Authors Bengio, Yoshua
Delalleau, Olivier
Le Roux, Nicolas
Paiement, Jean-François
Vincent, Pascal
Ouimet, Marie
Added by: [ADM]
Total mark: 0
Attachments
  • bengio_eigenfunctions_nc_2004....
Notes
    Topics