PLearn 0.1
Public Member Functions | Static Public Member Functions | Public Attributes | Static Public Attributes | Protected Member Functions | Static Protected Member Functions | Protected Attributes | Private Types | Private Member Functions
PLearn::ManifoldKNNDistribution Class Reference

K nearest neighbors density estimator that takes into accound the local manifold structure. More...

#include <ManifoldKNNDistribution.h>

Inheritance diagram for PLearn::ManifoldKNNDistribution:
Inheritance graph
[legend]
Collaboration diagram for PLearn::ManifoldKNNDistribution:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 ManifoldKNNDistribution ()
 Default constructor.
virtual real log_density (const Vec &x) const
 Return log of probability density log(p(y)).
virtual real survival_fn (const Vec &y) const
 Return survival function: P(Y>y).
virtual real cdf (const Vec &y) const
 Return cdf: P(Y<y).
virtual void expectation (Vec &mu) const
 Return E[Y].
virtual void variance (Mat &cov) const
 Return Var[Y].
virtual void generate (Vec &y) const
 Return a pseudo-random sample generated from the distribution.
virtual void resetGenerator (long g_seed) const
 Reset the random number generator used by generate() using the given seed.
virtual void forget ()
 (Re-)initializes the PDistribution in its fresh state (that state may depend on the 'seed' option) and sets 'stage' back to 0 (this is the stage of a fresh learner!).
virtual void train ()
 The role of the train method is to bring the learner up to stage == nstages, updating the train_stats collector with training costs measured on-line in the process.
virtual string classname () const
virtual OptionListgetOptionList () const
virtual OptionMapgetOptionMap () const
virtual RemoteMethodMapgetRemoteMethodMap () const
virtual ManifoldKNNDistributiondeepCopy (CopiesMap &copies) const
virtual void build ()
 Simply call inherited::build() then build_().
virtual void makeDeepCopyFromShallowCopy (CopiesMap &copies)
 Transforms a shallow copy into a deep copy.

Static Public Member Functions

static string _classname_ ()
static OptionList_getOptionList_ ()
static RemoteMethodMap_getRemoteMethodMap_ ()
static Object_new_instance_for_typemap_ ()
static bool _isa_ (const Object *o)
static void _static_initialize_ ()
static const PPathdeclaringFile ()

Public Attributes

PP< GenericNearestNeighborsknn_manifold
 Nearest neighbors search algorithms for local manifold structure estimation.
PP< GenericNearestNeighborsknn_density
 Nearest neighbors search algorithms for density estimation from ellipsoid's volume.
int manifold_dimensionality
 Dimensionality of the manifold.
real min_sigma_square
 Minimum variance in all directions on the manifold.
bool center_around_manifold_neighbors
 Indication that the estimation of the manifold tangent vectors should be made around the knn_manifold neighbors' mean vector, not around the test point.
bool use_gaussian_distribution
 Indication that a Gaussian distribution should be used as the knn_manifold nearest neighbors distribution, instead of the uniform in the ellipsoid.
PP< PDistributiondensity_learner
 Generic density learner for knn_density nearest neighbors.

Static Public Attributes

static StaticInitializer _static_initializer_

Protected Member Functions

void computeLocalPrincipalComponents (const Vec &x, Vec &eig_values, Mat &eig_vectors) const

Static Protected Member Functions

static void declareOptions (OptionList &ol)
 Declares the class options.

Protected Attributes

Mat nearest_neighbors_manifold
 Nearest neighbors for manifold estimation.
Vec nearest_neighbors_manifold_vec
 Nearest neighbors for manifold estimation as a vector.
Mat nearest_neighbors_density
 Nearest neighbors for density estimation.
Vec nearest_neighbors_density_vec
 Nearest neighbors for density estimation as a vector.
Mat eig_vectors
 Manifold local tangent vectors.
Vec eig_values
 Manifold local standard deviations.
Mat Ut
 SVD temporary variables.
Mat V
Vec S
 SVD temporary variable.
Vec eig_vectors_projection
 Projection on the local tangent vectors.
Vec neighbors_mean
 Mean vector of neighbors.
Vec test_minus_mean
 Difference between test point and neighbors_mean;.
VMat density_learner_train_set
 The density learner training set.

Private Types

typedef UnconditionalDistribution inherited

Private Member Functions

void build_ ()
 This does the actual building.

Detailed Description

K nearest neighbors density estimator that takes into accound the local manifold structure.

Definition at line 55 of file ManifoldKNNDistribution.h.


Member Typedef Documentation

Reimplemented from PLearn::UnconditionalDistribution.

Definition at line 57 of file ManifoldKNNDistribution.h.


Constructor & Destructor Documentation

PLearn::ManifoldKNNDistribution::ManifoldKNNDistribution ( )

Default constructor.

Definition at line 56 of file ManifoldKNNDistribution.cc.


Member Function Documentation

string PLearn::ManifoldKNNDistribution::_classname_ ( ) [static]

Reimplemented from PLearn::UnconditionalDistribution.

Definition at line 51 of file ManifoldKNNDistribution.cc.

OptionList & PLearn::ManifoldKNNDistribution::_getOptionList_ ( ) [static]

Reimplemented from PLearn::UnconditionalDistribution.

Definition at line 51 of file ManifoldKNNDistribution.cc.

RemoteMethodMap & PLearn::ManifoldKNNDistribution::_getRemoteMethodMap_ ( ) [static]

Reimplemented from PLearn::UnconditionalDistribution.

Definition at line 51 of file ManifoldKNNDistribution.cc.

bool PLearn::ManifoldKNNDistribution::_isa_ ( const Object o) [static]

Reimplemented from PLearn::UnconditionalDistribution.

Definition at line 51 of file ManifoldKNNDistribution.cc.

Object * PLearn::ManifoldKNNDistribution::_new_instance_for_typemap_ ( ) [static]

Reimplemented from PLearn::UnconditionalDistribution.

Definition at line 51 of file ManifoldKNNDistribution.cc.

StaticInitializer ManifoldKNNDistribution::_static_initializer_ & PLearn::ManifoldKNNDistribution::_static_initialize_ ( ) [static]

Reimplemented from PLearn::UnconditionalDistribution.

Definition at line 51 of file ManifoldKNNDistribution.cc.

void PLearn::ManifoldKNNDistribution::build ( ) [virtual]

Simply call inherited::build() then build_().

Reimplemented from PLearn::UnconditionalDistribution.

Definition at line 120 of file ManifoldKNNDistribution.cc.

References PLearn::UnconditionalDistribution::build(), and build_().

{
    // ### Nothing to add here, simply calls build_().
    inherited::build();
    build_();
}

Here is the call graph for this function:

void PLearn::ManifoldKNNDistribution::build_ ( ) [private]

This does the actual building.

Reimplemented from PLearn::UnconditionalDistribution.

Definition at line 130 of file ManifoldKNNDistribution.cc.

References center_around_manifold_neighbors, eig_values, eig_vectors, eig_vectors_projection, PLearn::PLearner::inputsize_, knn_density, knn_manifold, manifold_dimensionality, min_sigma_square, neighbors_mean, PLERROR, PLWARNING, PLearn::TVec< T >::resize(), PLearn::TMat< T >::resize(), PLearn::PLearner::train_set, use_gaussian_distribution, Ut, and V.

Referenced by build().

{
    if(min_sigma_square < 0)
        PLERROR("In ManifoldKNNDistribution::build_(): min_sigma_square should be"
                " >= 0.");

    if(!knn_manifold)
        PLERROR("In ManifoldKNNDistribution::build_(): knn_manifold must be"
                " provided.");

    if(!knn_density)
        PLERROR("In ManifoldKNNDistribution::build_(): knn_density must be"
                " provided.");

    if(inputsize_ > 0)
    {
        if(manifold_dimensionality > inputsize_)
            manifold_dimensionality = inputsize_;
        if(manifold_dimensionality < 1)
            PLERROR("In ManifoldKNNDistribution::build_(): manifold_dimensionality"
                    " should be > 0.");
        eig_vectors.resize(manifold_dimensionality,inputsize_);
        eig_values.resize(manifold_dimensionality);
        Ut.resize(inputsize_,inputsize_);
        V.resize(knn_manifold->num_neighbors,knn_manifold->num_neighbors);
        eig_vectors_projection.resize(manifold_dimensionality);
        neighbors_mean.resize(inputsize_);
    }

    if(train_set)
    {
        knn_manifold->setTrainingSet(train_set,true);
        knn_density->setTrainingSet(train_set,true);
        
        knn_manifold->train();
        knn_density->train();
    }

    if(use_gaussian_distribution && !center_around_manifold_neighbors)
    {
        PLWARNING("In ManifoldKNNDistribution::build_(): when using "
                  "use_gaussian_distribution=true, center_around_manifold_neighbors"
                  "must be true too. Setting center_around_manifold_neighbors=true...");
        center_around_manifold_neighbors = true;
    }
}

Here is the call graph for this function:

Here is the caller graph for this function:

real PLearn::ManifoldKNNDistribution::cdf ( const Vec y) const [virtual]

Return cdf: P(Y<y).

Reimplemented from PLearn::PDistribution.

Definition at line 180 of file ManifoldKNNDistribution.cc.

References PLERROR.

{
    PLERROR("cdf not implemented for ManifoldKNNDistribution"); return 0;
}
string PLearn::ManifoldKNNDistribution::classname ( ) const [virtual]

Reimplemented from PLearn::UnconditionalDistribution.

Definition at line 51 of file ManifoldKNNDistribution.cc.

void PLearn::ManifoldKNNDistribution::computeLocalPrincipalComponents ( const Vec x,
Vec eig_values,
Mat eig_vectors 
) const [protected]
void PLearn::ManifoldKNNDistribution::declareOptions ( OptionList ol) [static, protected]

Declares the class options.

Reimplemented from PLearn::UnconditionalDistribution.

Definition at line 66 of file ManifoldKNNDistribution.cc.

References PLearn::OptionBase::buildoption, center_around_manifold_neighbors, PLearn::declareOption(), PLearn::UnconditionalDistribution::declareOptions(), density_learner, knn_density, knn_manifold, manifold_dimensionality, min_sigma_square, and use_gaussian_distribution.

{
    declareOption(ol, "knn_manifold", &ManifoldKNNDistribution::knn_manifold,
                  OptionBase::buildoption,
                  "Nearest neighbors search algorithms for local manifold structure"
                  "estimation.");

    declareOption(ol, "knn_density", &ManifoldKNNDistribution::knn_density,
                  OptionBase::buildoption,
                  "Nearest neighbors search algorithms for density "
                  "estimation from ellipsoid\n"
                  "volume.");

    declareOption(ol, "manifold_dimensionality", 
                  &ManifoldKNNDistribution::manifold_dimensionality,
                  OptionBase::buildoption,
                  "Dimensionality of the manifold.");

    declareOption(ol, "min_sigma_square", 
                  &ManifoldKNNDistribution::min_sigma_square,
                  OptionBase::buildoption,
                  "Minimum variance in all directions on the manifold. This value"
                  "is added\n"
                  "to the estimated covariance matrix.");

    declareOption(ol, "center_around_manifold_neighbors", 
                  &ManifoldKNNDistribution::center_around_manifold_neighbors,
                  OptionBase::buildoption,
                  "Indication that the estimation of the manifold tangent vectors\n"
                  "should be made around the knn_manifold neighbors' mean vector,\n"
                  "not around the test point."
                  );

    declareOption(ol, "use_gaussian_distribution", 
                  &ManifoldKNNDistribution::use_gaussian_distribution,
                  OptionBase::buildoption,
                  "Indication that a Gaussian distribution should be used as the\n"
                  "knn_manifold nearest neighbors distribution, instead of the\n"
                  "uniform in the ellipsoid."
                  );

    declareOption(ol, "density_learner", 
                  &ManifoldKNNDistribution::density_learner,
                  OptionBase::buildoption,
                  "Generic density learner for knn_manifold nearest neighbors."
                  );

    // Now call the parent class' declareOptions().
    inherited::declareOptions(ol);
}

Here is the call graph for this function:

static const PPath& PLearn::ManifoldKNNDistribution::declaringFile ( ) [inline, static]

Reimplemented from PLearn::UnconditionalDistribution.

Definition at line 138 of file ManifoldKNNDistribution.h.

:
    //#####  Protected Options  ###############################################
ManifoldKNNDistribution * PLearn::ManifoldKNNDistribution::deepCopy ( CopiesMap copies) const [virtual]

Reimplemented from PLearn::UnconditionalDistribution.

Definition at line 51 of file ManifoldKNNDistribution.cc.

void PLearn::ManifoldKNNDistribution::expectation ( Vec mu) const [virtual]

Return E[Y].

Reimplemented from PLearn::PDistribution.

Definition at line 188 of file ManifoldKNNDistribution.cc.

References PLERROR.

{
    PLERROR("expectation not implemented for ManifoldKNNDistribution");
}
void PLearn::ManifoldKNNDistribution::forget ( ) [virtual]

(Re-)initializes the PDistribution in its fresh state (that state may depend on the 'seed' option) and sets 'stage' back to 0 (this is the stage of a fresh learner!).

Reimplemented from PLearn::UnconditionalDistribution.

Definition at line 197 of file ManifoldKNNDistribution.cc.

{}
void PLearn::ManifoldKNNDistribution::generate ( Vec y) const [virtual]

Return a pseudo-random sample generated from the distribution.

Reimplemented from PLearn::PDistribution.

Definition at line 203 of file ManifoldKNNDistribution.cc.

References PLERROR.

{
    PLERROR("generate not implemented for ManifoldKNNDistribution");
}
OptionList & PLearn::ManifoldKNNDistribution::getOptionList ( ) const [virtual]

Reimplemented from PLearn::UnconditionalDistribution.

Definition at line 51 of file ManifoldKNNDistribution.cc.

OptionMap & PLearn::ManifoldKNNDistribution::getOptionMap ( ) const [virtual]

Reimplemented from PLearn::UnconditionalDistribution.

Definition at line 51 of file ManifoldKNNDistribution.cc.

RemoteMethodMap & PLearn::ManifoldKNNDistribution::getRemoteMethodMap ( ) const [virtual]

Reimplemented from PLearn::UnconditionalDistribution.

Definition at line 51 of file ManifoldKNNDistribution.cc.

real PLearn::ManifoldKNNDistribution::log_density ( const Vec x) const [virtual]

Return log of probability density log(p(y)).

Reimplemented from PLearn::PDistribution.

Definition at line 211 of file ManifoldKNNDistribution.cc.

References center_around_manifold_neighbors, PLearn::columnMean(), computeLocalPrincipalComponents(), density_learner, density_learner_train_set, eig_values, eig_vectors, eig_vectors_projection, i, PLearn::PLearner::inputsize_, j, knn_density, knn_manifold, PLearn::TMat< T >::length(), PLearn::TVec< T >::length(), Log2Pi, manifold_dimensionality, PLearn::max(), min_sigma_square, PLearn::mypow(), PLearn::PLearner::n_examples, nearest_neighbors_density, nearest_neighbors_density_vec, nearest_neighbors_manifold, nearest_neighbors_manifold_vec, neighbors_mean, Pi, PLearn::pl_gammln(), pl_log, PLearn::pownorm(), PLearn::product(), PLearn::substract(), test_minus_mean, PLearn::TVec< T >::toMat(), and use_gaussian_distribution.

{

    real ret = 0;
    if(density_learner)
    {
        knn_manifold->computeOutput(y,nearest_neighbors_manifold_vec);
        nearest_neighbors_manifold = 
            nearest_neighbors_manifold_vec.toMat(
                knn_manifold->num_neighbors,inputsize_);
        density_learner_train_set = VMat( nearest_neighbors_manifold );
        density_learner_train_set->defineSizes(inputsize_,0);
        density_learner->setTrainingSet(density_learner_train_set,true);
        density_learner->train();
        density_learner->log_density(y);
        ret = density_learner->log_density(y) + 
            pl_log((real)knn_manifold->num_neighbors)-pl_log((real)n_examples);
    }
    else if(use_gaussian_distribution)
    {
        computeLocalPrincipalComponents(y,eig_values,eig_vectors);
        if(!center_around_manifold_neighbors)
            columnMean(nearest_neighbors_manifold,neighbors_mean);

        // Compute log-normalization constant
        ret = - inputsize_ *0.5 *Log2Pi;
        for(int i=0; i<manifold_dimensionality; i++)
            ret -= 0.5 * pl_log(eig_values[i]+min_sigma_square);
        ret -= (inputsize_-manifold_dimensionality)*0.5*pl_log(min_sigma_square);
        substract(y,neighbors_mean,test_minus_mean);
        product(eig_vectors_projection,eig_vectors,test_minus_mean);
        for(int j=0; j<eig_values.length(); j++)
            ret -= mypow(eig_vectors_projection[j],2) * 
                (1/(eig_values[j]+min_sigma_square) 
                 - 1/min_sigma_square) ;
        ret -= pownorm(test_minus_mean,2)
            /min_sigma_square;
        ret += pl_log((real)knn_manifold->num_neighbors)-pl_log((real)n_examples);
    }
    else
    {
        computeLocalPrincipalComponents(y,eig_values,eig_vectors);

        // Find volume of ellipsoid defined by eig_values, eig_vectors and
        // min_sigma_square that covers all the nearest_neighbors found by knn_density
        knn_density->computeOutput(y,nearest_neighbors_density_vec);
        nearest_neighbors_density = 
            nearest_neighbors_density_vec.toMat(knn_density->num_neighbors,inputsize_);
        nearest_neighbors_density -= y;
        real max = -1;
        real scaled_projection=0;
        for(int i=0; i<nearest_neighbors_density.length(); i++)
        {
            scaled_projection = 0;
            product(eig_vectors_projection,eig_vectors,nearest_neighbors_density(i));
            for(int j=0; j<eig_values.length(); j++)
                scaled_projection += mypow(eig_vectors_projection[j],2) * 
                    (1/(eig_values[j]+min_sigma_square) 
                     - 1/min_sigma_square) ;
            scaled_projection += pownorm(nearest_neighbors_density(i),2)
                /min_sigma_square;
            if(max < scaled_projection)
                max = scaled_projection;
        }
        
        // Compute log-volume of the ellipsoid: pi
        real log_vol = 0.5 * inputsize_ * pl_log(scaled_projection);
        for(int i=0; i<manifold_dimensionality; i++)
            log_vol += 0.5 * pl_log(eig_values[i]+min_sigma_square);
        log_vol += (inputsize_-manifold_dimensionality)*0.5*pl_log(min_sigma_square);
        log_vol += 0.5*inputsize_*pl_log(Pi) - pl_gammln(0.5*inputsize_+1);
        
        ret = pl_log((real)knn_density->num_neighbors)-pl_log((real)n_examples)-log_vol;
    }
    return ret;
}

Here is the call graph for this function:

void PLearn::ManifoldKNNDistribution::makeDeepCopyFromShallowCopy ( CopiesMap copies) [virtual]
void PLearn::ManifoldKNNDistribution::resetGenerator ( long  g_seed) const [virtual]

Reset the random number generator used by generate() using the given seed.

Definition at line 316 of file ManifoldKNNDistribution.cc.

References PLERROR.

{
    PLERROR("resetGenerator not implemented for ManifoldKNNDistribution");
}
real PLearn::ManifoldKNNDistribution::survival_fn ( const Vec y) const [virtual]

Return survival function: P(Y>y).

Reimplemented from PLearn::PDistribution.

Definition at line 324 of file ManifoldKNNDistribution.cc.

References PLERROR.

{
    PLERROR("survival_fn not implemented for ManifoldKNNDistribution"); return 0;
}
void PLearn::ManifoldKNNDistribution::train ( ) [virtual]

The role of the train method is to bring the learner up to stage == nstages, updating the train_stats collector with training costs measured on-line in the process.

Reimplemented from PLearn::PDistribution.

Definition at line 333 of file ManifoldKNNDistribution.cc.

{}
void PLearn::ManifoldKNNDistribution::variance ( Mat cov) const [virtual]

Return Var[Y].

Reimplemented from PLearn::PDistribution.

Definition at line 339 of file ManifoldKNNDistribution.cc.

References PLERROR.

{
    PLERROR("variance not implemented for ManifoldKNNDistribution");
}

Member Data Documentation

Reimplemented from PLearn::UnconditionalDistribution.

Definition at line 138 of file ManifoldKNNDistribution.h.

Indication that the estimation of the manifold tangent vectors should be made around the knn_manifold neighbors' mean vector, not around the test point.

Definition at line 79 of file ManifoldKNNDistribution.h.

Referenced by build_(), computeLocalPrincipalComponents(), declareOptions(), and log_density().

Generic density learner for knn_density nearest neighbors.

Definition at line 87 of file ManifoldKNNDistribution.h.

Referenced by declareOptions(), log_density(), and makeDeepCopyFromShallowCopy().

The density learner training set.

Definition at line 173 of file ManifoldKNNDistribution.h.

Referenced by log_density(), and makeDeepCopyFromShallowCopy().

Manifold local standard deviations.

Definition at line 160 of file ManifoldKNNDistribution.h.

Referenced by build_(), log_density(), and makeDeepCopyFromShallowCopy().

Manifold local tangent vectors.

Definition at line 158 of file ManifoldKNNDistribution.h.

Referenced by build_(), computeLocalPrincipalComponents(), log_density(), and makeDeepCopyFromShallowCopy().

Projection on the local tangent vectors.

Definition at line 166 of file ManifoldKNNDistribution.h.

Referenced by build_(), log_density(), and makeDeepCopyFromShallowCopy().

Nearest neighbors search algorithms for density estimation from ellipsoid's volume.

Definition at line 67 of file ManifoldKNNDistribution.h.

Referenced by build_(), declareOptions(), log_density(), and makeDeepCopyFromShallowCopy().

Nearest neighbors search algorithms for local manifold structure estimation.

Definition at line 63 of file ManifoldKNNDistribution.h.

Referenced by build_(), computeLocalPrincipalComponents(), declareOptions(), log_density(), and makeDeepCopyFromShallowCopy().

Dimensionality of the manifold.

Definition at line 70 of file ManifoldKNNDistribution.h.

Referenced by build_(), computeLocalPrincipalComponents(), declareOptions(), and log_density().

Minimum variance in all directions on the manifold.

This value is added to the estimated covariance matrix.

Definition at line 74 of file ManifoldKNNDistribution.h.

Referenced by build_(), declareOptions(), and log_density().

Nearest neighbors for density estimation.

Definition at line 154 of file ManifoldKNNDistribution.h.

Referenced by log_density(), and makeDeepCopyFromShallowCopy().

Nearest neighbors for density estimation as a vector.

Definition at line 156 of file ManifoldKNNDistribution.h.

Referenced by log_density(), and makeDeepCopyFromShallowCopy().

Nearest neighbors for manifold estimation.

Definition at line 150 of file ManifoldKNNDistribution.h.

Referenced by computeLocalPrincipalComponents(), log_density(), and makeDeepCopyFromShallowCopy().

Nearest neighbors for manifold estimation as a vector.

Definition at line 152 of file ManifoldKNNDistribution.h.

Referenced by computeLocalPrincipalComponents(), log_density(), and makeDeepCopyFromShallowCopy().

Mean vector of neighbors.

Definition at line 168 of file ManifoldKNNDistribution.h.

Referenced by build_(), computeLocalPrincipalComponents(), log_density(), and makeDeepCopyFromShallowCopy().

SVD temporary variable.

Definition at line 164 of file ManifoldKNNDistribution.h.

Referenced by computeLocalPrincipalComponents(), and makeDeepCopyFromShallowCopy().

Difference between test point and neighbors_mean;.

Definition at line 170 of file ManifoldKNNDistribution.h.

Referenced by log_density(), and makeDeepCopyFromShallowCopy().

Indication that a Gaussian distribution should be used as the knn_manifold nearest neighbors distribution, instead of the uniform in the ellipsoid.

Definition at line 84 of file ManifoldKNNDistribution.h.

Referenced by build_(), declareOptions(), and log_density().

SVD temporary variables.

Definition at line 162 of file ManifoldKNNDistribution.h.

Referenced by build_(), computeLocalPrincipalComponents(), and makeDeepCopyFromShallowCopy().


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines