Learning the Density Structure of High-Dimensional Data

Yoshua Bengio

We claim and present arguments to the effect that a large class of manifold learning algorithms that are essentially local will suffer from at least four generic problems associated with noise in the data, curvature of the manifold, dimensionality of the manifold, and the presence of many manifolds with little data per manifold. This suggests non-local manifold learning algorithms which attempt to discover shared structure in the tangent planes at different positions. A criterion for such an algorithm is proposed and experiments estimating a tangent plane prediction function are presented, showing its advantages with respect to local manifold learning algorithms.