Theory of Principal Curves
Self-consistent principal curves
- Trevor Hastie
and
Werner Stuetzle
define principal curves as "self-consistent" smooth curves which
pass through the "middle" of a d-dimensional probability distribution
or data cloud. They present some theoretical results, give an
algorithm
for the construction of principal curves and compare them to
other generalizations of principal components. They illustrate
the use of principal curves on two
applications.
- Tom Duchamp and
Werner Stuetzle show that
HS principal curves are
saddle points of the distance function (MSE) which explains why
cross-validation is not a viable method for choosing the complexity of principal
curve estimates. In a twin paper, they analytically calculate the
HS principal curves of
several planar distributions.
- Filip Mulier and Vladimir Cherkassky points out the similarities
between the Self-Organizing Map algorithm and principal curves.
Regularized principal curves
- The principal curve definition of
Balázs Kégl,
Adam Krzyzak,
Tamás Linder,
and
Ken Zeger
has the advantage that principal curves always exist if the distribution
has finite second moments. The new definition also makes it possible to carry out
a theoretical analysis of learning principal curves from training
data. Based on their definition, they develop the
Polygonal Line Algorithm which
compares favorably with previous methods.
- S. Sandilya and Sanjeev Kulkarni show that,
similarly to principal curves with a length constraint,
principal curves with bounded turn always exist if the distribution
has finite second moments. The new definition has the advantage that it refers
more directly to the objective function minimized by the
Polygonal Line Algorithm than the
theoretical model
considered by Kégl et. al.
- Alex Smola, Robert Williamson,
Sebastian Mika, and
Bernhard Schölkopf
consider regularized principal manifolds in general, and
obtain learning rates for certain types of regularization
operators.
Parametric approaches
- Robert Tibshirani
gives an alternative
definition of a principal curve, based on a mixture model. Estimation is carried
out through an EM algorithm.
- Kui-yu Chang and
Joydeep Ghosh use a parametric model
to extend principal curves to higher-dimensional principal manifolds,
and compare the new method to the generative topographic mapping algorithm.
Other approaches
- Rather then using self-consistency,
Pedro Delicado
generalizes another property of the first principal component to define
principal curves.