PLearn 0.1
Public Member Functions | Static Public Member Functions | Public Attributes | Static Public Attributes | Static Protected Member Functions | Protected Attributes | Private Types | Private Member Functions | Private Attributes
PLearn::KernelProjection Class Reference

#include <KernelProjection.h>

Inheritance diagram for PLearn::KernelProjection:
Inheritance graph
[legend]
Collaboration diagram for PLearn::KernelProjection:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 KernelProjection ()
 Default constructor.
virtual Vec getEigenvalues ()
 Return the eigenvalues of this learner.
virtual void build ()
 Simply calls inherited::build() then build_().
virtual void makeDeepCopyFromShallowCopy (CopiesMap &copies)
 Transforms a shallow copy into a deep copy.
virtual string classname () const
virtual OptionListgetOptionList () const
virtual OptionMapgetOptionMap () const
virtual RemoteMethodMapgetRemoteMethodMap () const
virtual KernelProjectiondeepCopy (CopiesMap &copies) const
virtual int outputsize () const
 Returns the size of this learner's output, (which typically may depend on its inputsize(), targetsize() and set options).
virtual void forget ()
 (Re-)initializes the PLearner in its fresh state (that state may depend on the 'seed' option) And sets 'stage' back to 0 (this is the stage of a fresh learner!).
virtual void train ()
 The role of the train method is to bring the learner up to stage==nstages, updating the train_stats collector with training costs measured on-line in the process.
virtual void computeOutput (const Vec &input, Vec &output) const
 Computes the output from the input.
virtual void computeCostsFromOutputs (const Vec &input, const Vec &output, const Vec &target, Vec &costs) const
 Computes the costs from already computed output.
virtual TVec< string > getTestCostNames () const
 Returns the names of the costs computed by computeCostsFromOutpus (and thus the test method).
virtual TVec< string > getTrainCostNames () const
 Returns the names of the objective costs that the train method computes and for which it updates the VecStatsCollector train_stats.
virtual void setTrainingSet (VMat training_set, bool call_forget=true)
 Overridden to forward to the kernel.

Static Public Member Functions

static string _classname_ ()
static OptionList_getOptionList_ ()
static RemoteMethodMap_getRemoteMethodMap_ ()
static Object_new_instance_for_typemap_ ()
static bool _isa_ (const Object *o)
static void _static_initialize_ ()
static const PPathdeclaringFile ()

Public Attributes

bool compute_costs
bool free_extra_components
int ignore_n_first
Ker kernel
real min_eigenvalue
int n_comp
int n_comp_for_cost
string normalize
Vec eigenvalues
Mat eigenvectors

Static Public Attributes

static StaticInitializer _static_initializer_

Static Protected Member Functions

static void declareOptions (OptionList &ol)
 Declares this class' options.

Protected Attributes

int n_comp_kept
int n_examples
 Learnt number of examples obtained from train_set when doing setTrainingSet.
bool first_output
 A boolean indicating we haven't performed any output yet.
Vec last_input
 The last input given when computing costs.
Vec last_output
 The last output computed when computing costs.

Private Types

typedef PLearner inherited

Private Member Functions

void build_ ()
 This does the actual building.

Private Attributes

Vec k_x_xi
 Global storage to save memory allocations.
Mat result
Mat used_eigenvectors

Detailed Description

Definition at line 53 of file KernelProjection.h.


Member Typedef Documentation

Reimplemented from PLearn::PLearner.

Reimplemented in PLearn::Isomap, PLearn::KernelPCA, PLearn::LLE, and PLearn::SpectralClustering.

Definition at line 58 of file KernelProjection.h.


Constructor & Destructor Documentation

PLearn::KernelProjection::KernelProjection ( )

Default constructor.

Definition at line 53 of file KernelProjection.cc.


Member Function Documentation

string PLearn::KernelProjection::_classname_ ( ) [static]

Reimplemented from PLearn::PLearner.

Reimplemented in PLearn::Isomap, PLearn::KernelPCA, PLearn::LLE, and PLearn::SpectralClustering.

Definition at line 71 of file KernelProjection.cc.

OptionList & PLearn::KernelProjection::_getOptionList_ ( ) [static]

Reimplemented from PLearn::PLearner.

Reimplemented in PLearn::Isomap, PLearn::KernelPCA, PLearn::LLE, and PLearn::SpectralClustering.

Definition at line 71 of file KernelProjection.cc.

RemoteMethodMap & PLearn::KernelProjection::_getRemoteMethodMap_ ( ) [static]

Reimplemented from PLearn::PLearner.

Reimplemented in PLearn::Isomap, PLearn::KernelPCA, PLearn::LLE, and PLearn::SpectralClustering.

Definition at line 71 of file KernelProjection.cc.

bool PLearn::KernelProjection::_isa_ ( const Object o) [static]

Reimplemented from PLearn::PLearner.

Reimplemented in PLearn::Isomap, PLearn::KernelPCA, PLearn::LLE, and PLearn::SpectralClustering.

Definition at line 71 of file KernelProjection.cc.

Object * PLearn::KernelProjection::_new_instance_for_typemap_ ( ) [static]

Reimplemented from PLearn::Object.

Reimplemented in PLearn::Isomap, PLearn::KernelPCA, PLearn::LLE, and PLearn::SpectralClustering.

Definition at line 71 of file KernelProjection.cc.

StaticInitializer KernelProjection::_static_initializer_ & PLearn::KernelProjection::_static_initialize_ ( ) [static]

Reimplemented from PLearn::PLearner.

Reimplemented in PLearn::Isomap, PLearn::KernelPCA, PLearn::LLE, and PLearn::SpectralClustering.

Definition at line 71 of file KernelProjection.cc.

void PLearn::KernelProjection::build ( ) [virtual]

Simply calls inherited::build() then build_().

Reimplemented from PLearn::PLearner.

Reimplemented in PLearn::Isomap, PLearn::KernelPCA, PLearn::LLE, and PLearn::SpectralClustering.

Definition at line 137 of file KernelProjection.cc.

References PLearn::PLearner::build(), and build_().

Referenced by PLearn::SpectralClustering::build(), PLearn::LLE::build(), and PLearn::KernelPCA::build().

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::KernelProjection::build_ ( ) [private]

This does the actual building.

Reimplemented from PLearn::PLearner.

Reimplemented in PLearn::Isomap, PLearn::KernelPCA, PLearn::LLE, and PLearn::SpectralClustering.

Definition at line 146 of file KernelProjection.cc.

References first_output, last_input, n_comp, n_comp_kept, and PLearn::TVec< T >::resize().

Referenced by build().

{
    if (n_comp_kept == -1) {
        n_comp_kept = n_comp;
    }
    first_output = true;  // Safer.
    last_input.resize(0);
}

Here is the call graph for this function:

Here is the caller graph for this function:

string PLearn::KernelProjection::classname ( ) const [virtual]

Reimplemented from PLearn::Object.

Reimplemented in PLearn::Isomap, PLearn::KernelPCA, PLearn::LLE, and PLearn::SpectralClustering.

Definition at line 71 of file KernelProjection.cc.

void PLearn::KernelProjection::computeCostsFromOutputs ( const Vec input,
const Vec output,
const Vec target,
Vec costs 
) const [virtual]

Computes the costs from already computed output.

Implements PLearn::PLearner.

Definition at line 158 of file KernelProjection.cc.

References PLearn::abs(), compute_costs, PLearn::diff(), PLearn::dot(), PLearn::endl(), kernel, last_input, last_output, PLearn::TVec< T >::length(), MISSING_VALUE, n_comp_for_cost, PLearn::perr, PLearn::pownorm(), PLearn::TVec< T >::resize(), and PLearn::TVec< T >::subVec().

{
    if (!compute_costs)
        return;
    // fs_squared_norm_reconstruction_error (see getTestCostNames).
    real k_x_x = kernel->evaluate(input, input);
    real fs_norm;
    if (n_comp_for_cost > 0) {
        // Only take the 'n_comp_for_cost' first components.
        fs_norm = pownorm(output.subVec(0, n_comp_for_cost));
    } else {
        fs_norm = pownorm(output);
    }
    costs.resize(2);
    if (last_input.length() == 0) {
        last_input.resize(input.length());
        last_output.resize(output.length());
        last_input << input;
        last_output << output;
        costs[1] = MISSING_VALUE;
    } else {
        real k_x_y = kernel->evaluate(input, last_input);
        real fs_dotp;
        if (n_comp_for_cost > 0) {
            // Only take the 'n_comp_for_cost' first components.
            fs_dotp = dot(output.subVec(0, n_comp_for_cost), last_output.subVec(0, n_comp_for_cost));
        } else {
            fs_dotp = dot(output, last_output);
        }
        last_input.resize(0);
        real diff = k_x_y - fs_dotp;
        costs[1] = diff * diff;
    }
    costs[0] = abs(k_x_x - fs_norm);
    if (k_x_x - fs_norm < -1e-5) {
        // TODO Remove this later after making sure it didn't happen.
        perr << "Negative error: " << k_x_x - fs_norm << " (k_x_x = " << k_x_x << ", fs_norm = " << fs_norm << ")" << endl;
    }
}                                

Here is the call graph for this function:

void PLearn::KernelProjection::computeOutput ( const Vec input,
Vec output 
) const [virtual]

Computes the output from the input.

Reimplemented from PLearn::PLearner.

Definition at line 202 of file KernelProjection.cc.

References eigenvalues, eigenvectors, PLearn::fast_exact_is_equal(), first_output, i, k_x_xi, kernel, n_comp_kept, n_examples, PLearn::norm(), normalize, outputsize(), PLASSERT, PLERROR, PLearn::TMat< T >::resize(), PLearn::TVec< T >::resize(), result, PLearn::rowSum(), PLearn::sqrt(), PLearn::TMat< T >::subMatRows(), and used_eigenvectors.

{
    PLASSERT( outputsize() > 0 );
    static real* result_ptr;
    if (first_output) {
        // Initialize k_x_xi, used_eigenvectors and result correctly.
        k_x_xi.resize(n_examples);
        used_eigenvectors = eigenvectors.subMatRows(0, n_comp_kept);
        result.resize(n_comp_kept,1);
        first_output = false;
    }
    // Compute the K(x,x_i).
    kernel->evaluate_all_i_x(input, k_x_xi);
    // Compute the output.
    rowSum(used_eigenvectors * k_x_xi, result);
    output.resize(n_comp_kept);
    result_ptr = result[0];
    if (normalize == "none") {
        real norm_coeff = sqrt(real(n_examples));
        for (int i = 0; i < n_comp_kept; i++) {
            output[i] = *(result_ptr++) / eigenvalues[i] * norm_coeff;
        }
    } else if (normalize == "unit_var") {
        for (int i = 0; i < n_comp_kept; i++) {
            output[i] = *(result_ptr++) / sqrt(eigenvalues[i]);
        }
    } else if (normalize == "unit_eigen") {
        output << result;
        output *= sqrt(real(n_examples));
    } else if (normalize == "unit_coord") {
        output << result;
        real norm = PLearn::norm(output,2);
        if (!fast_exact_is_equal(norm, 0))
            output /= norm;
    } else {
        PLERROR("In KernelProjection::computeOutput - Wrong value for 'normalize')");
    }
}    

Here is the call graph for this function:

void PLearn::KernelProjection::declareOptions ( OptionList ol) [static, protected]

Declares this class' options.

Reimplemented from PLearn::PLearner.

Reimplemented in PLearn::Isomap, PLearn::KernelPCA, PLearn::LLE, and PLearn::SpectralClustering.

Definition at line 76 of file KernelProjection.cc.

References PLearn::OptionBase::buildoption, compute_costs, PLearn::declareOption(), PLearn::PLearner::declareOptions(), eigenvalues, eigenvectors, free_extra_components, ignore_n_first, kernel, PLearn::OptionBase::learntoption, min_eigenvalue, n_comp, n_comp_for_cost, n_comp_kept, n_examples, normalize, PLearn::OptionBase::nosave, PLearn::redeclareOption(), and PLearn::PLearner::seed_.

Referenced by PLearn::SpectralClustering::declareOptions(), PLearn::LLE::declareOptions(), and PLearn::KernelPCA::declareOptions().

{

    // Build options.

    declareOption(ol, "kernel", &KernelProjection::kernel, OptionBase::buildoption,
                  "The kernel used to compute the Gram matrix.");

    declareOption(ol, "n_comp", &KernelProjection::n_comp, OptionBase::buildoption,
                  "Number of components computed.");

    declareOption(ol, "normalize", &KernelProjection::normalize, OptionBase::buildoption,
                  "The kind of normalization performed when computing the output\n"
                  " - 'none'      : classical projection on the eigenvectors\n"
                  " - 'unit_var'  : normalization to get unit variance on each coordinate\n"
                  " - 'unit_eigen': ignore the eigenvalues and do as if they were all 1\n"
                  " - 'unit_coord': coordinates are normalized so that they have norm 1\n");

    declareOption(ol, "min_eigenvalue", &KernelProjection::min_eigenvalue, OptionBase::buildoption,
                  "Any component associated with an eigenvalue <= min_eigenvalue will be discarded.");

    declareOption(ol, "compute_costs", &KernelProjection::compute_costs, OptionBase::buildoption,
                  "Whether we should compute costs or not.");

    declareOption(ol, "n_comp_for_cost", &KernelProjection::n_comp_for_cost, OptionBase::buildoption,
                  "The number of components considered when computing a cost (default = -1 means n_comp).");

    declareOption(ol, "free_extra_components", &KernelProjection::free_extra_components, OptionBase::buildoption,
                  "If set to 1, components computed but not kept won't be available after training.");

    declareOption(ol, "ignore_n_first", &KernelProjection::ignore_n_first, OptionBase::buildoption,
                  "Will ignore the first 'ignore_n_first' eigenvectors, if this option is > 0.");

    // Learnt options.

    declareOption(ol, "eigenvalues", &KernelProjection::eigenvalues, OptionBase::learntoption,
                  "The eigenvalues of the Gram matrix.");

    declareOption(ol, "eigenvectors", &KernelProjection::eigenvectors, OptionBase::learntoption,
                  "The eigenvectors of the Gram matrix.");

    declareOption(ol, "n_comp_kept", &KernelProjection::n_comp_kept, OptionBase::learntoption,
                  "The actual number of components actually kept in the output (we may discard\n"
                  "some because of low eigenvalues).");

    declareOption(ol, "n_examples", &KernelProjection::n_examples, OptionBase::learntoption,
                  "The number of points in the training set.");

    // Now call the parent class' declareOptions
    inherited::declareOptions(ol);

    // Hide unused options.

    redeclareOption(ol, "seed", &KernelProjection::seed_, OptionBase::nosave,
                    "No seed used here.");

}

Here is the call graph for this function:

Here is the caller graph for this function:

static const PPath& PLearn::KernelProjection::declaringFile ( ) [inline, static]

Reimplemented from PLearn::PLearner.

Reimplemented in PLearn::Isomap, PLearn::KernelPCA, PLearn::LLE, and PLearn::SpectralClustering.

Definition at line 141 of file KernelProjection.h.

KernelProjection * PLearn::KernelProjection::deepCopy ( CopiesMap copies) const [virtual]

Reimplemented from PLearn::PLearner.

Reimplemented in PLearn::Isomap, PLearn::KernelPCA, PLearn::LLE, and PLearn::SpectralClustering.

Definition at line 71 of file KernelProjection.cc.

void PLearn::KernelProjection::forget ( ) [virtual]

(Re-)initializes the PLearner in its fresh state (that state may depend on the 'seed' option) And sets 'stage' back to 0 (this is the stage of a fresh learner!).

Reimplemented from PLearn::PLearner.

Reimplemented in PLearn::Isomap, PLearn::KernelPCA, PLearn::LLE, and PLearn::SpectralClustering.

Definition at line 244 of file KernelProjection.cc.

References eigenvalues, eigenvectors, PLearn::endl(), first_output, last_input, n_comp, n_comp_kept, n_examples, PLearn::pout, PLearn::TVec< T >::resize(), PLearn::PLearner::stage, and PLearn::PLearner::verbosity.

Referenced by PLearn::SpectralClustering::forget(), PLearn::LLE::forget(), and PLearn::KernelPCA::forget().

{
    stage = 0;
    if (verbosity > 1)
        pout << "forget: n_comp_kept = " << n_comp_kept << endl;
    n_comp_kept = n_comp;
    if (verbosity > 1)
        pout << "forget: n_comp_kept = " << n_comp_kept << endl;
    n_examples = 0;
    first_output = true;
    last_input.resize(0);
    // Free memory.
    eigenvectors = Mat();
    eigenvalues = Vec();
}

Here is the call graph for this function:

Here is the caller graph for this function:

virtual Vec PLearn::KernelProjection::getEigenvalues ( ) [inline, virtual]

Return the eigenvalues of this learner.

Definition at line 128 of file KernelProjection.h.

{return eigenvalues;}
OptionList & PLearn::KernelProjection::getOptionList ( ) const [virtual]

Reimplemented from PLearn::Object.

Reimplemented in PLearn::Isomap, PLearn::KernelPCA, PLearn::LLE, and PLearn::SpectralClustering.

Definition at line 71 of file KernelProjection.cc.

OptionMap & PLearn::KernelProjection::getOptionMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Reimplemented in PLearn::Isomap, PLearn::KernelPCA, PLearn::LLE, and PLearn::SpectralClustering.

Definition at line 71 of file KernelProjection.cc.

RemoteMethodMap & PLearn::KernelProjection::getRemoteMethodMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Reimplemented in PLearn::Isomap, PLearn::KernelPCA, PLearn::LLE, and PLearn::SpectralClustering.

Definition at line 71 of file KernelProjection.cc.

TVec< string > PLearn::KernelProjection::getTestCostNames ( ) const [virtual]

Returns the names of the costs computed by computeCostsFromOutpus (and thus the test method).

Implements PLearn::PLearner.

Definition at line 263 of file KernelProjection.cc.

References PLearn::TVec< T >::append(), and compute_costs.

Referenced by getTrainCostNames().

{
    TVec<string> t;
    if (!compute_costs)
        return t;
    // Feature space squared norm reconstruction error:
    // | K(x,x) - ||output||^2 |
    t.append("fs_squared_norm_reconstruction_error");
    // Feature space dot product reconstruction squared error:
    // ( K(x,y) - <output_x,output_y> )^2
    t.append("fs_dotp_reconstruction_squared_error");
    return t;
}

Here is the call graph for this function:

Here is the caller graph for this function:

TVec< string > PLearn::KernelProjection::getTrainCostNames ( ) const [virtual]

Returns the names of the objective costs that the train method computes and for which it updates the VecStatsCollector train_stats.

Implements PLearn::PLearner.

Definition at line 280 of file KernelProjection.cc.

References getTestCostNames().

{
    return getTestCostNames();
}

Here is the call graph for this function:

void PLearn::KernelProjection::makeDeepCopyFromShallowCopy ( CopiesMap copies) [virtual]
int PLearn::KernelProjection::outputsize ( ) const [virtual]

Returns the size of this learner's output, (which typically may depend on its inputsize(), targetsize() and set options).

Implements PLearn::PLearner.

Definition at line 305 of file KernelProjection.cc.

References n_comp_kept.

Referenced by computeOutput().

{
    return n_comp_kept;
}

Here is the caller graph for this function:

void PLearn::KernelProjection::setTrainingSet ( VMat  training_set,
bool  call_forget = true 
) [virtual]

Overridden to forward to the kernel.

Reimplemented from PLearn::PLearner.

Definition at line 313 of file KernelProjection.cc.

References kernel, PLearn::VMat::length(), n_examples, PLERROR, and PLearn::PLearner::setTrainingSet().

Referenced by PLearn::IsomapTangentLearner::build_(), and PLearn::KPCATangentLearner::train().

                                                                         {
    inherited::setTrainingSet(training_set, call_forget);
    n_examples = training_set->length();
    // Save the dataset in the kernel, because it may be needed after we reload
    // the learner.
    if (kernel)
    {
        kernel->specify_dataset = training_set;
        kernel->build();  
    }
    else
        PLERROR("KernelProjection::setTrainingSet: You cannot use setTrainingSet without a kernel set");
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::KernelProjection::train ( ) [virtual]

The role of the train method is to bring the learner up to stage==nstages, updating the train_stats collector with training costs measured on-line in the process.

Implements PLearn::PLearner.

Definition at line 330 of file KernelProjection.cc.

References eigenvalues, PLearn::eigenVecOfSymmMat(), eigenvectors, first_output, PLearn::flush(), free_extra_components, ignore_n_first, kernel, PLearn::TMat< T >::length(), PLearn::TVec< T >::length(), min_eigenvalue, n_comp, n_comp_kept, n_examples, PLWARNING, PLearn::pout, PLearn::PLearner::report_progress, PLearn::TMat< T >::resize(), PLearn::TVec< T >::resize(), PLearn::PLearner::stage, PLearn::TMat< T >::subMatRows(), PLearn::TVec< T >::subVec(), PLearn::PLearner::verbosity, and PLearn::TMat< T >::width().

Referenced by PLearn::KPCATangentLearner::train(), and PLearn::IsomapTangentLearner::train().

{
    if (stage == 1) {
        PLWARNING("In KernelProjection::train - Learner has already been trained");
        return;
    }
    Mat gram(n_examples,n_examples);
    // (1) Compute the Gram matrix.
    if (report_progress) {
        kernel->report_progress = true;
    }
    clock_t time_for_gram = clock();
    kernel->computeGramMatrix(gram);
    time_for_gram = clock() - time_for_gram;
    if (verbosity >= 3) {
        pout << flush;
    }
    // (2) Compute its eigenvectors and eigenvalues.
    eigenVecOfSymmMat(gram, n_comp + ignore_n_first, eigenvalues, eigenvectors);
    if (ignore_n_first > 0) {
        eigenvalues = eigenvalues.subVec(ignore_n_first, eigenvalues.length() - ignore_n_first);
        eigenvectors = eigenvectors.subMatRows(ignore_n_first, eigenvectors.length() - ignore_n_first);
    }
    
    n_comp_kept = eigenvalues.length(); // Could be different of n_comp.
    // (3) Discard low eigenvalues.
    int p = 0;
    while (p < n_comp_kept && eigenvalues[p] > min_eigenvalue)
        p++;
    n_comp_kept = p;

    // (4) Optionally remove the discarded components.
    if (free_extra_components) {
        eigenvalues.resize(n_comp_kept);
        eigenvectors.resize(n_comp_kept, eigenvectors.width());
    }
    // All done!
    first_output = true;
    stage = 1;
}

Here is the call graph for this function:

Here is the caller graph for this function:


Member Data Documentation

Reimplemented from PLearn::PLearner.

Reimplemented in PLearn::Isomap, PLearn::KernelPCA, PLearn::LLE, and PLearn::SpectralClustering.

Definition at line 141 of file KernelProjection.h.

Definition at line 88 of file KernelProjection.h.

Referenced by computeCostsFromOutputs(), declareOptions(), and getTestCostNames().

A boolean indicating we haven't performed any output yet.

Definition at line 77 of file KernelProjection.h.

Referenced by build_(), computeOutput(), forget(), and train().

Definition at line 89 of file KernelProjection.h.

Referenced by declareOptions(), and train().

Global storage to save memory allocations.

Definition at line 61 of file KernelProjection.h.

Referenced by computeOutput(), and makeDeepCopyFromShallowCopy().

The last input given when computing costs.

Definition at line 79 of file KernelProjection.h.

Referenced by build_(), computeCostsFromOutputs(), forget(), and makeDeepCopyFromShallowCopy().

The last output computed when computing costs.

Definition at line 80 of file KernelProjection.h.

Referenced by computeCostsFromOutputs(), and makeDeepCopyFromShallowCopy().

Definition at line 94 of file KernelProjection.h.

Referenced by computeCostsFromOutputs(), and declareOptions().

Definition at line 71 of file KernelProjection.h.

Referenced by build_(), computeOutput(), declareOptions(), forget(), outputsize(), and train().

Learnt number of examples obtained from train_set when doing setTrainingSet.

Reimplemented from PLearn::PLearner.

Definition at line 72 of file KernelProjection.h.

Referenced by computeOutput(), declareOptions(), forget(), setTrainingSet(), and train().

Definition at line 62 of file KernelProjection.h.

Referenced by computeOutput(), and makeDeepCopyFromShallowCopy().

Definition at line 63 of file KernelProjection.h.

Referenced by computeOutput(), and makeDeepCopyFromShallowCopy().


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines