PLearn 0.1
Public Member Functions | Static Public Member Functions | Public Attributes | Static Public Attributes | Static Protected Member Functions | Protected Attributes | Private Types | Private Member Functions
PLearn::ModulesLearner Class Reference

Trains an OnlineLearningModule wrt the cost of a CostModule. More...

#include <ModulesLearner.h>

Inheritance diagram for PLearn::ModulesLearner:
Inheritance graph
[legend]
Collaboration diagram for PLearn::ModulesLearner:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 ModulesLearner ()
 Default constructor.
virtual int outputsize () const
 Returns the size of this learner's output, (which typically may depend on its inputsize(), targetsize() and set options).
virtual void forget ()
 (Re-)initializes the PLearner in its fresh state (that state may depend on the 'seed' option) and sets 'stage' back to 0 (this is the stage of a fresh learner!).
virtual void train ()
 The role of the train method is to bring the learner up to stage==nstages, updating the train_stats collector with training costs measured on-line in the process.
virtual void computeOutput (const Vec &input, Vec &output) const
 Computes the output from the input.
virtual void computeCostsFromOutputs (const Vec &input, const Vec &output, const Vec &target, Vec &costs) const
 Computes the costs from already computed output.
virtual TVec< std::string > getTestCostNames () const
 Returns the names of the costs computed by computeCostsFromOutpus (and thus the test method).
virtual TVec< std::string > getTrainCostNames () const
 Returns the names of the objective costs that the train method computes and for which it updates the VecStatsCollector train_stats.
virtual string classname () const
virtual OptionListgetOptionList () const
virtual OptionMapgetOptionMap () const
virtual RemoteMethodMapgetRemoteMethodMap () const
virtual ModulesLearnerdeepCopy (CopiesMap &copies) const
virtual void build ()
 Finish building the object; just call inherited::build followed by build_()
virtual void makeDeepCopyFromShallowCopy (CopiesMap &copies)
 Transforms a shallow copy into a deep copy.

Static Public Member Functions

static string _classname_ ()
static OptionList_getOptionList_ ()
static RemoteMethodMap_getRemoteMethodMap_ ()
static Object_new_instance_for_typemap_ ()
static bool _isa_ (const Object *o)
static void _static_initialize_ ()
static const PPathdeclaringFile ()

Public Attributes

PP< OnlineLearningModulemodule
 Module to train.
PP< CostModulecost
 Cost function.
string hessian_estimation
 Estimation of the second-order terms.

Static Public Attributes

static StaticInitializer _static_initializer_

Static Protected Member Functions

static void declareOptions (OptionList &ol)
 Declares the class options.

Protected Attributes

Vec output
 stores module's output
Vec d_output
 stores the gradient wtr the output
Vec d2_output
 stores the diagonal of hessian wtr the output
Vec costs
 stores the costs

Private Types

typedef PLearner inherited

Private Member Functions

void build_ ()
 This does the actual building.

Detailed Description

Trains an OnlineLearningModule wrt the cost of a CostModule.

The CostModule provides the output gradient to train the OnlineLearningModule. In order to stack layers, you can use StackedModulesModule, and in order to compute several costs, you can use CombinedCostsModule.

Definition at line 57 of file ModulesLearner.h.


Member Typedef Documentation

Reimplemented from PLearn::PLearner.

Definition at line 59 of file ModulesLearner.h.


Constructor & Destructor Documentation

PLearn::ModulesLearner::ModulesLearner ( )

Default constructor.

Definition at line 54 of file ModulesLearner.cc.

References PLearn::PLearner::random_gen.

    : hessian_estimation( "none" )
{
    random_gen = new PRandom();
}

Member Function Documentation

string PLearn::ModulesLearner::_classname_ ( ) [static]

Reimplemented from PLearn::PLearner.

Definition at line 52 of file ModulesLearner.cc.

OptionList & PLearn::ModulesLearner::_getOptionList_ ( ) [static]

Reimplemented from PLearn::PLearner.

Definition at line 52 of file ModulesLearner.cc.

RemoteMethodMap & PLearn::ModulesLearner::_getRemoteMethodMap_ ( ) [static]

Reimplemented from PLearn::PLearner.

Definition at line 52 of file ModulesLearner.cc.

bool PLearn::ModulesLearner::_isa_ ( const Object o) [static]

Reimplemented from PLearn::PLearner.

Definition at line 52 of file ModulesLearner.cc.

Object * PLearn::ModulesLearner::_new_instance_for_typemap_ ( ) [static]

Reimplemented from PLearn::Object.

Definition at line 52 of file ModulesLearner.cc.

StaticInitializer ModulesLearner::_static_initializer_ & PLearn::ModulesLearner::_static_initialize_ ( ) [static]

Reimplemented from PLearn::PLearner.

Definition at line 52 of file ModulesLearner.cc.

void PLearn::ModulesLearner::build ( ) [virtual]

Finish building the object; just call inherited::build followed by build_()

Reimplemented from PLearn::PLearner.

Definition at line 136 of file ModulesLearner.cc.

References PLearn::PLearner::build(), and build_().

Here is the call graph for this function:

void PLearn::ModulesLearner::build_ ( ) [private]

This does the actual building.

Reimplemented from PLearn::PLearner.

Definition at line 84 of file ModulesLearner.cc.

References cost, costs, d2_output, d_output, hessian_estimation, PLearn::PLearner::inputsize(), PLearn::PLearner::inputsize_, PLearn::lowerstring(), module, output, outputsize(), PLERROR, PLearn::PLearner::random_gen, PLearn::TVec< T >::resize(), and PLearn::PLearner::targetsize().

Referenced by build().

{
    // hessian estimation
    string h_est = lowerstring( hessian_estimation );
    if( h_est == "none" || h_est == "" )
        hessian_estimation = "none";
    else if( h_est == "diag" )
        hessian_estimation = h_est;
    else if( h_est == "simpler_diag" )
        hessian_estimation = h_est;
    else
        PLERROR( "ModulesLearner::buildOptions(): hessian_estimation\n"
                 "value '%s' is unknown.\n", hessian_estimation.c_str() );

    if( hessian_estimation == "diag" )
        cost->estimate_simpler_diag_hessian = false;
    else
        cost->estimate_simpler_diag_hessian = true;

    // Assign random_gen to module and cost, unless they already have one
    if( !(module->random_gen) )
    {
        module->random_gen = random_gen;
        module->forget();
    }
    if( !(cost->random_gen) )
    {
        cost->random_gen = random_gen;
        cost->forget();
    }

    // if train_set is not set, we don't know inputsize nor targetsize
    if( inputsize_ >= 0 ) // we don't use inputsize() because it crashes if <0
    {
        module->input_size = inputsize();
        module->build();

        output.resize( outputsize() );
        d_output.resize( outputsize() );
        if( hessian_estimation != "none" )
            d2_output.resize( outputsize() );

        cost->input_size = outputsize();
        cost->target_size = targetsize();
        cost->build();

        costs->resize( cost->output_size );
    }
}

Here is the call graph for this function:

Here is the caller graph for this function:

string PLearn::ModulesLearner::classname ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 52 of file ModulesLearner.cc.

Referenced by train().

Here is the caller graph for this function:

void PLearn::ModulesLearner::computeCostsFromOutputs ( const Vec input,
const Vec output,
const Vec target,
Vec costs 
) const [virtual]

Computes the costs from already computed output.

Implements PLearn::PLearner.

Definition at line 287 of file ModulesLearner.cc.

References cost.

{
    cost->fprop( output, target, costs );
}
void PLearn::ModulesLearner::computeOutput ( const Vec input,
Vec output 
) const [virtual]

Computes the output from the input.

Reimplemented from PLearn::PLearner.

Definition at line 282 of file ModulesLearner.cc.

References module.

{
    module->fprop( input, output );
}
void PLearn::ModulesLearner::declareOptions ( OptionList ol) [static, protected]

Declares the class options.

Reimplemented from PLearn::PLearner.

Definition at line 60 of file ModulesLearner.cc.

References PLearn::OptionBase::buildoption, cost, PLearn::declareOption(), PLearn::PLearner::declareOptions(), hessian_estimation, and module.

{
    declareOption(ol, "module", &ModulesLearner::module,
                  OptionBase::buildoption,
                  "The module to train");

    declareOption(ol, "cost", &ModulesLearner::cost,
                  OptionBase::buildoption,
                  "The cost module");

    declareOption(ol, "hessian_estimation",
                  &ModulesLearner::hessian_estimation,
                  OptionBase::buildoption,
                  "Estimation of the second-order terms. One of:\n"
                  "  - \"none\": using only first-order derivative for"
                  " update,\n"
                  "  - \"diag\": estimating the diagonal of the hessian,\n"
                  "  - \"simpler_diag\": positive estimation of the diagonal\n"
                 );

    // Now call the parent class' declareOptions
    inherited::declareOptions(ol);
}

Here is the call graph for this function:

static const PPath& PLearn::ModulesLearner::declaringFile ( ) [inline, static]

Reimplemented from PLearn::PLearner.

Definition at line 137 of file ModulesLearner.h.

:
    //#####  Protected Options  ###############################################
ModulesLearner * PLearn::ModulesLearner::deepCopy ( CopiesMap copies) const [virtual]

Reimplemented from PLearn::PLearner.

Definition at line 52 of file ModulesLearner.cc.

void PLearn::ModulesLearner::forget ( ) [virtual]

(Re-)initializes the PLearner in its fresh state (that state may depend on the 'seed' option) and sets 'stage' back to 0 (this is the stage of a fresh learner!).

(Re-)initialize the PLearner in its fresh state (that state may depend on the 'seed' option) and sets 'stage' back to 0 (this is the stage of a fresh learner!)

A typical forget() method should do the following:

  • initialize a random number generator with the seed option
  • initialize the learner's parameters, using this random generator
  • stage = 0

Reimplemented from PLearn::PLearner.

Definition at line 168 of file ModulesLearner.cc.

References PLearn::TVec< T >::clear(), cost, costs, d2_output, d_output, module, output, PLearn::PLearner::random_gen, PLearn::PLearner::seed_, and PLearn::PLearner::stage.

{

    random_gen->manual_seed( seed_ );

    // reset temporary vectors
    output.clear();
    d_output.clear();
    if( d2_output )
        d2_output.clear();
    costs.clear();

    // reset module and cost
    module->forget();
    cost->forget();

    stage = 0;
}

Here is the call graph for this function:

OptionList & PLearn::ModulesLearner::getOptionList ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 52 of file ModulesLearner.cc.

OptionMap & PLearn::ModulesLearner::getOptionMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 52 of file ModulesLearner.cc.

RemoteMethodMap & PLearn::ModulesLearner::getRemoteMethodMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 52 of file ModulesLearner.cc.

TVec< string > PLearn::ModulesLearner::getTestCostNames ( ) const [virtual]

Returns the names of the costs computed by computeCostsFromOutpus (and thus the test method).

Implements PLearn::PLearner.

Definition at line 295 of file ModulesLearner.cc.

References cost.

{
    // Return the names of the costs computed by computeCostsFromOutputs
    return cost->costNames();
}
TVec< string > PLearn::ModulesLearner::getTrainCostNames ( ) const [virtual]

Returns the names of the objective costs that the train method computes and for which it updates the VecStatsCollector train_stats.

Implements PLearn::PLearner.

Definition at line 301 of file ModulesLearner.cc.

References cost.

{
    // Return the names of the objective costs that the train method computes
    // and for which it updates the VecStatsCollector train_stats
    return cost->costNames();
}
void PLearn::ModulesLearner::makeDeepCopyFromShallowCopy ( CopiesMap copies) [virtual]

Transforms a shallow copy into a deep copy.

Reimplemented from PLearn::PLearner.

Definition at line 143 of file ModulesLearner.cc.

References cost, costs, d2_output, d_output, PLearn::deepCopyField(), PLearn::PLearner::makeDeepCopyFromShallowCopy(), module, and output.

{
    inherited::makeDeepCopyFromShallowCopy(copies);

    // deepCopyField(trainvec, copies);

    deepCopyField(module, copies);
    deepCopyField(cost, copies);
    deepCopyField(output, copies);
    deepCopyField(d_output, copies);
    deepCopyField(d2_output, copies);
    deepCopyField(costs, copies);
}

Here is the call graph for this function:

int PLearn::ModulesLearner::outputsize ( ) const [virtual]

Returns the size of this learner's output, (which typically may depend on its inputsize(), targetsize() and set options).

Implements PLearn::PLearner.

Definition at line 158 of file ModulesLearner.cc.

References module.

Referenced by build_().

{
    // Compute and return the size of this learner's output (which typically
    // may depend on its inputsize(), targetsize() and set options).
    if( module )
        return module->output_size;
    else
        return -1;
}

Here is the caller graph for this function:

void PLearn::ModulesLearner::train ( ) [virtual]

The role of the train method is to bring the learner up to stage==nstages, updating the train_stats collector with training costs measured on-line in the process.

Implements PLearn::PLearner.

Definition at line 196 of file ModulesLearner.cc.

References classname(), cost, costs, d2_output, d_output, PLearn::VMat::getExample(), hessian_estimation, PLearn::PLearner::initTrain(), PLearn::PLearner::inputsize(), PLearn::VMat::length(), module, PLearn::PLearner::nstages, output, PLearn::PLearner::report_progress, PLearn::sample(), PLearn::PLearner::stage, PLearn::PLearner::targetsize(), PLearn::tostring(), PLearn::PLearner::train_set, and PLearn::PLearner::train_stats.

{
    // The role of the train method is to bring the learner up to
    // stage==nstages, updating train_stats with training costs measured
    // on-line in the process.

    /* TYPICAL CODE:

    static Vec input;  // static so we don't reallocate memory each time...
    static Vec target; // (but be careful that static means shared!)
    input.resize(inputsize());    // the train_set's inputsize()
    target.resize(targetsize());  // the train_set's targetsize()
    real weight;

    // This generic PLearner method does a number of standard stuff useful for
    // (almost) any learner, and return 'false' if no training should take
    // place. See PLearner.h for more details.
    if (!initTrain())
        return;

    while(stage<nstages)
    {
        // clear statistics of previous epoch
        train_stats->forget();

        //... train for 1 stage, and update train_stats,
        // using train_set->getExample(input, target, weight)
        // and train_stats->update(train_costs)

        ++stage;
        train_stats->finalize(); // finalize statistics for this epoch
    }
    */

    Vec input( inputsize() );
    Vec target( targetsize() );
    real weight;
    int nsamples = train_set->length();

    if( !initTrain() )
        return;

    PP<ProgressBar> pb;
    if( report_progress )
        pb = new ProgressBar( "Training " + classname() + " from stage "
                              + tostring(stage) + " to " + tostring(nstages),
                              nstages - stage );

    int initial_stage = stage;
    for( ; stage < nstages ; stage++ )
    {
        // clear stats of previous epoch
        train_stats->forget();
        for( int sample=0 ; sample < nsamples ; sample++ )
        {
            train_set->getExample( sample, input, target, weight );

            // fprop
            module->fprop( input, output );
            cost->fprop( output, target, costs );

            // bprop
            if( hessian_estimation != "none" ) // bbpropUpdate
            {
                cost->bbpropUpdate( output, target, costs[0],
                                    d_output, d2_output );

                module->bbpropUpdate( input, output, d_output, d2_output );
            }
            else // bpropUpdate
            {
                cost->bpropUpdate( output, target, costs[0], d_output );

                module->bpropUpdate( input, output, d_output );
            }

            train_stats->update( costs );
        }
        train_stats->finalize(); // finalize statistics for this epoch

        if(pb)
            pb->update( stage+1 - initial_stage );
    }
}

Here is the call graph for this function:


Member Data Documentation

Reimplemented from PLearn::PLearner.

Definition at line 137 of file ModulesLearner.h.

Vec PLearn::ModulesLearner::costs [mutable, protected]

stores the costs

Definition at line 163 of file ModulesLearner.h.

Referenced by build_(), forget(), makeDeepCopyFromShallowCopy(), and train().

Vec PLearn::ModulesLearner::d2_output [mutable, protected]

stores the diagonal of hessian wtr the output

Definition at line 160 of file ModulesLearner.h.

Referenced by build_(), forget(), makeDeepCopyFromShallowCopy(), and train().

Vec PLearn::ModulesLearner::d_output [mutable, protected]

stores the gradient wtr the output

Definition at line 157 of file ModulesLearner.h.

Referenced by build_(), forget(), makeDeepCopyFromShallowCopy(), and train().

Estimation of the second-order terms.

One of:

  • "none": using only first-order derivative for update,
  • "diag": estimating the diagonal of the hessian,
  • "simpler_diag": positive estimation of the diagonal

Definition at line 74 of file ModulesLearner.h.

Referenced by build_(), declareOptions(), and train().

Module to train.

Definition at line 65 of file ModulesLearner.h.

Referenced by build_(), computeOutput(), declareOptions(), forget(), makeDeepCopyFromShallowCopy(), outputsize(), and train().

Vec PLearn::ModulesLearner::output [mutable, protected]

stores module's output

Definition at line 154 of file ModulesLearner.h.

Referenced by build_(), forget(), makeDeepCopyFromShallowCopy(), and train().


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines