PLearn 0.1
Public Member Functions | Static Public Member Functions | Public Attributes | Static Public Attributes | Static Protected Member Functions | Protected Attributes | Private Types | Private Member Functions
PLearn::LLC Class Reference

#include <LLC.h>

Inheritance diagram for PLearn::LLC:
Inheritance graph
[legend]
Collaboration diagram for PLearn::LLC:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 LLC ()
 Default constructor.
virtual void build ()
 Simply calls inherited::build() then build_().
virtual void makeDeepCopyFromShallowCopy (CopiesMap &copies)
 Transforms a shallow copy into a deep copy.
virtual string classname () const
virtual OptionListgetOptionList () const
virtual OptionMapgetOptionMap () const
virtual RemoteMethodMapgetRemoteMethodMap () const
virtual LLCdeepCopy (CopiesMap &copies) const
virtual int outputsize () const
 Returns the size of this learner's output, (which typically may depend on its inputsize(), targetsize() and set options).
virtual void forget ()
 (Re-)initializes the PLearner in its fresh state (that state may depend on the 'seed' option) And sets 'stage' back to 0 (this is the stage of a fresh learner!).
virtual void train ()
 The role of the train method is to bring the learner up to stage==nstages, updating the train_stats collector with training costs measured on-line in the process.
virtual void computeOutput (const Vec &input, Vec &output) const
 Computes the output from the input.
virtual void computeCostsFromOutputs (const Vec &input, const Vec &output, const Vec &target, Vec &costs) const
 Computes the costs from already computed output.
virtual TVec< std::string > getTestCostNames () const
 Returns the names of the costs computed by computeCostsFromOutpus (and thus the test method).
virtual TVec< std::string > getTrainCostNames () const
 Returns the names of the objective costs that the train method computes and for which it updates the VecStatsCollector train_stats.

Static Public Member Functions

static string _classname_ ()
static OptionList_getOptionList_ ()
static RemoteMethodMap_getRemoteMethodMap_ ()
static Object_new_instance_for_typemap_ ()
static bool _isa_ (const Object *o)
static void _static_initialize_ ()
static const PPathdeclaringFile ()

Public Attributes

int knn
PP< PLearnermixture
int n_comp
real regularization
bool train_mixture

Static Public Attributes

static StaticInitializer _static_initializer_

Static Protected Member Functions

static void declareOptions (OptionList &ol)
 Declares this class' options.

Protected Attributes

Vec mixture_output
 Used to store the output of the underlying mixture.
Mat L
int sum_of_dim

Private Types

typedef PLearner inherited

Private Member Functions

void build_ ()
 This does the actual building.

Detailed Description

Definition at line 51 of file LLC.h.


Member Typedef Documentation

typedef PLearner PLearn::LLC::inherited [private]

Reimplemented from PLearn::PLearner.

Definition at line 55 of file LLC.h.


Constructor & Destructor Documentation

PLearn::LLC::LLC ( )

Default constructor.

Definition at line 55 of file LLC.cc.

    : sum_of_dim(-1),
      knn(5),
      n_comp(1),
      regularization(0),
      train_mixture(true)
{}

Member Function Documentation

string PLearn::LLC::_classname_ ( ) [static]

Reimplemented from PLearn::PLearner.

Definition at line 67 of file LLC.cc.

OptionList & PLearn::LLC::_getOptionList_ ( ) [static]

Reimplemented from PLearn::PLearner.

Definition at line 67 of file LLC.cc.

RemoteMethodMap & PLearn::LLC::_getRemoteMethodMap_ ( ) [static]

Reimplemented from PLearn::PLearner.

Definition at line 67 of file LLC.cc.

bool PLearn::LLC::_isa_ ( const Object o) [static]

Reimplemented from PLearn::PLearner.

Definition at line 67 of file LLC.cc.

Object * PLearn::LLC::_new_instance_for_typemap_ ( ) [static]

Reimplemented from PLearn::Object.

Definition at line 67 of file LLC.cc.

StaticInitializer LLC::_static_initializer_ & PLearn::LLC::_static_initialize_ ( ) [static]

Reimplemented from PLearn::PLearner.

Definition at line 67 of file LLC.cc.

void PLearn::LLC::build ( ) [virtual]

Simply calls inherited::build() then build_().

Reimplemented from PLearn::PLearner.

Definition at line 111 of file LLC.cc.

References PLearn::PLearner::build(), and build_().

Here is the call graph for this function:

void PLearn::LLC::build_ ( ) [private]

This does the actual building.

Reimplemented from PLearn::PLearner.

Definition at line 120 of file LLC.cc.

References mixture_output, PLearn::TVec< T >::resize(), and sum_of_dim.

Referenced by build().

{
    // ### This method should do the real building of the object,
    // ### according to set 'options', in *any* situation. 
    // ### Typical situations include:
    // ###  - Initial building of an object from a few user-specified options
    // ###  - Building of a "reloaded" object: i.e. from the complete set of all serialised options.
    // ###  - Updating or "re-building" of an object after a few "tuning" options have been modified.
    // ### You should assume that the parent class' build_() has already been called.
    if (sum_of_dim > 0)
        mixture_output.resize(sum_of_dim);
}

Here is the call graph for this function:

Here is the caller graph for this function:

string PLearn::LLC::classname ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 67 of file LLC.cc.

void PLearn::LLC::computeCostsFromOutputs ( const Vec input,
const Vec output,
const Vec target,
Vec costs 
) const [virtual]

Computes the costs from already computed output.

Implements PLearn::PLearner.

Definition at line 136 of file LLC.cc.

{
    // No cost to compute.
}                                
void PLearn::LLC::computeOutput ( const Vec input,
Vec output 
) const [virtual]

Computes the output from the input.

Reimplemented from PLearn::PLearner.

Definition at line 145 of file LLC.cc.

References L, mixture, mixture_output, n_comp, PLearn::product(), and PLearn::TVec< T >::resize().

{
    output.resize(n_comp);
    // As in the train method, we assume the mixture has just the nice output
    // we need.
    mixture->computeOutput(input, mixture_output);
    product(output, L, mixture_output);
}    

Here is the call graph for this function:

void PLearn::LLC::declareOptions ( OptionList ol) [static, protected]

Declares this class' options.

Reimplemented from PLearn::PLearner.

Definition at line 72 of file LLC.cc.

References PLearn::OptionBase::buildoption, PLearn::declareOption(), PLearn::PLearner::declareOptions(), knn, L, PLearn::OptionBase::learntoption, mixture, n_comp, regularization, sum_of_dim, and train_mixture.

{
    // ### For the "flags" of each option, you should typically specify  
    // ### one of OptionBase::buildoption, OptionBase::learntoption or 
    // ### OptionBase::tuningoption. Another possible flag to be combined with
    // ### is OptionBase::nosave

    // Build options.

    declareOption(ol, "knn", &LLC::knn, OptionBase::buildoption,
                  "Number of neighbors used to compute local reconstruction weights.");

    declareOption(ol, "mixture", &LLC::mixture, OptionBase::buildoption,
                  "A mixture of local dimensionality reducers.");

    declareOption(ol, "n_comp", &LLC::n_comp, OptionBase::buildoption,
                  "Number of components computed.");

    declareOption(ol, "regularization", &LLC::regularization, OptionBase::buildoption,
                  "A regularization coefficient (to use if crash in the eigensystem, but assume the consequences).");

    declareOption(ol, "train_mixture", &LLC::train_mixture, OptionBase::buildoption,
                  "Whether the mixture should be trained or not.");

    // Learnt options.

    declareOption(ol, "L", &LLC::L, OptionBase::learntoption,
                  "The matrix of factors (bias and linear transformation for each neighborhood).");

    declareOption(ol, "sum_of_dim", &LLC::sum_of_dim, OptionBase::learntoption,
                  "Must be equal to mixture->outputsize().");

    // Now call the parent class' declareOptions.
    inherited::declareOptions(ol);
}

Here is the call graph for this function:

static const PPath& PLearn::LLC::declaringFile ( ) [inline, static]

Reimplemented from PLearn::PLearner.

Definition at line 115 of file LLC.h.

LLC * PLearn::LLC::deepCopy ( CopiesMap copies) const [virtual]

Reimplemented from PLearn::PLearner.

Definition at line 67 of file LLC.cc.

void PLearn::LLC::forget ( ) [virtual]

(Re-)initializes the PLearner in its fresh state (that state may depend on the 'seed' option) And sets 'stage' back to 0 (this is the stage of a fresh learner!).

Reimplemented from PLearn::PLearner.

Definition at line 157 of file LLC.cc.

References L, PLearn::TMat< T >::resize(), PLearn::PLearner::stage, and sum_of_dim.

{
    stage = 0;
    sum_of_dim = -1;
    L.resize(0,0);
}

Here is the call graph for this function:

OptionList & PLearn::LLC::getOptionList ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 67 of file LLC.cc.

OptionMap & PLearn::LLC::getOptionMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 67 of file LLC.cc.

RemoteMethodMap & PLearn::LLC::getRemoteMethodMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 67 of file LLC.cc.

TVec< string > PLearn::LLC::getTestCostNames ( ) const [virtual]

Returns the names of the costs computed by computeCostsFromOutpus (and thus the test method).

Implements PLearn::PLearner.

Definition at line 167 of file LLC.cc.

{
    static TVec<string> noCost;
    return noCost;
}
TVec< string > PLearn::LLC::getTrainCostNames ( ) const [virtual]

Returns the names of the objective costs that the train method computes and for which it updates the VecStatsCollector train_stats.

Implements PLearn::PLearner.

Definition at line 176 of file LLC.cc.

{
    static TVec<string> noCost;
    return noCost;
}
void PLearn::LLC::makeDeepCopyFromShallowCopy ( CopiesMap copies) [virtual]

Transforms a shallow copy into a deep copy.

Reimplemented from PLearn::PLearner.

Definition at line 185 of file LLC.cc.

References PLearn::PLearner::makeDeepCopyFromShallowCopy(), and PLERROR.

{
    inherited::makeDeepCopyFromShallowCopy(copies);

    // ### ex:
    // deepCopyField(trainvec, copies);

    // ### Remove this line when you have fully implemented this method.
    PLERROR("LLC::makeDeepCopyFromShallowCopy not fully (correctly) implemented yet!");
}

Here is the call graph for this function:

int PLearn::LLC::outputsize ( ) const [virtual]

Returns the size of this learner's output, (which typically may depend on its inputsize(), targetsize() and set options).

Implements PLearn::PLearner.

Definition at line 199 of file LLC.cc.

References n_comp.

{
    return n_comp;
}
void PLearn::LLC::train ( ) [virtual]

The role of the train method is to bring the learner up to stage==nstages, updating the train_stats collector with training costs measured on-line in the process.

Implements PLearn::PLearner.

Definition at line 207 of file LLC.cc.

References PLearn::endl(), PLearn::fillItSymmetric(), PLearn::generalizedEigenVecOfSymmMat(), i, in, knn, L, PLearn::TMat< T >::length(), PLearn::VMat::length(), mixture, mixture_output, n, n_comp, PLearn::PLearner::nstages, PLearn::openString(), PLASSERT, PLearn::PStream::plearn_ascii, PLWARNING, PLearn::pout, PLearn::product(), regularization, PLearn::regularizeMatrix(), PLearn::TVec< T >::resize(), PLearn::PLearner::stage, PLearn::TMat< T >::subMatRows(), sum_of_dim, train_mixture, PLearn::PLearner::train_set, PLearn::transposeProduct(), and PLearn::PLearner::verbosity.

{
    PLASSERT( mixture );
    if (stage >= nstages) {
        PLWARNING("In LLC::train - Learner has already been trained, skipping training");
        return;
    }
    if (verbosity >= 2)
        pout << "Computing local reconstruction weights" << endl;
    PP<ReconstructionWeightsKernel> reconstruct = new ReconstructionWeightsKernel();
    reconstruct->knn = knn + 1; // +1 because it includes the point itself.
    reconstruct->build();
    reconstruct->setDataForKernelMatrix(train_set);
    int n = train_set->length();
    Mat lle_mat(n,n);
    reconstruct->computeLLEMatrix(lle_mat); // Fill lle_mat with W + W' - W' W.
    for (int i = 0; i < n; i++)
        lle_mat(i,i) = lle_mat(i,i) - 1;    // lle_mat = - (I - W') * (I - W)
    if (train_mixture) {
        if (verbosity >= 2)
            pout << "Training mixture" << endl;
        mixture->setTrainingSet(train_set);
        mixture->train();
    }
    // Obtain the number of components in the mixture (= the number of 'experts').
    // We assume here the mixture has a 'n_components' option.
    int n_comp_mixture;
    string mixture_n_components = mixture->getOption("n_components");
    openString(mixture_n_components, PStream::plearn_ascii) >> n_comp_mixture;
    // Obtain the dimension of each expert in the mixture.
    // We assume here the mixture has a 'outputsizes' option which is a TVec<int>
    // containing the outputsize of each expert.
    TVec<int> dimension;
    string mixture_outputsizes;
    PStream in = openString(mixture_outputsizes, PStream::plearn_ascii);
    in >> dimension;
    sum_of_dim = n_comp_mixture;
    for (int k = 0; k < dimension.length(); k++)
        sum_of_dim += dimension[k];
    mixture_output.resize(sum_of_dim);
    // Compute the output of the mixture for all elements in the training set.
    // The output must be a vector of size 'sum_of_dim' which is the concatenation
    // of the output of each expert in the mixture, each weighted by its
    // responsibility r_k (that can depend on x, and such that sum_k r_k = 1),
    // and with a bias (= r_k) added as the first dimension of each expert.
    if (verbosity >= 2)
        pout << "Computing mixture outputs" << endl;
    Mat U(n, sum_of_dim);
    mixture->useOnTrain(U);
    if (verbosity >= 2)
        pout << "Building the generalized eigenvector system" << endl;
    Mat B(sum_of_dim, sum_of_dim);
    transposeProduct(B, U, U);
    B /= real(1.0 / n);           // B = 1/n U' U
    Mat A(sum_of_dim, sum_of_dim);
    Mat tmp(n, sum_of_dim);
    product(tmp, lle_mat, U);
    // A = - U' (I - W') (I - W) U (because we want the smallest eigenvalues).
    transposeProduct(A, U, tmp);
    tmp = Mat(); // Free memory.
    fillItSymmetric(A); // A and B should be already symmetric, but it may be safer
    fillItSymmetric(B); // to ensure it.
    if (verbosity >= 2)
        pout << "Solving the generalized eigensystem" << endl;
    Vec eigen_val;
    Mat eigen_vec;
    if (regularization > 0)
        regularizeMatrix(B, regularization);
    generalizedEigenVecOfSymmMat(A, B, 1, n_comp + 1, eigen_val, eigen_vec);
    // Ignore the smallest eigenvalue (should be 0).
    if (verbosity >= 5)
        pout << "Smallest eigenvalue: " << eigen_val[0] << endl;
    L = eigen_vec.subMatRows(1, eigen_vec.length() - 1);
    if (verbosity >= 2)
        pout << "Training is over" << endl;
    stage = 1;
}

Here is the call graph for this function:


Member Data Documentation

Reimplemented from PLearn::PLearner.

Definition at line 115 of file LLC.h.

Definition at line 75 of file LLC.h.

Referenced by declareOptions(), and train().

Mat PLearn::LLC::L [protected]

Definition at line 66 of file LLC.h.

Referenced by computeOutput(), declareOptions(), forget(), and train().

Definition at line 76 of file LLC.h.

Referenced by computeOutput(), declareOptions(), and train().

Vec PLearn::LLC::mixture_output [mutable, protected]

Used to store the output of the underlying mixture.

Definition at line 60 of file LLC.h.

Referenced by build_(), computeOutput(), and train().

Definition at line 77 of file LLC.h.

Referenced by computeOutput(), declareOptions(), outputsize(), and train().

Definition at line 78 of file LLC.h.

Referenced by declareOptions(), and train().

Definition at line 67 of file LLC.h.

Referenced by build_(), declareOptions(), forget(), and train().

Definition at line 79 of file LLC.h.

Referenced by declareOptions(), and train().


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines