PLearn 0.1
Public Member Functions | Static Public Member Functions | Public Attributes | Static Public Attributes | Static Protected Member Functions | Protected Attributes | Private Types | Private Member Functions
PLearn::HyperLearner Class Reference

#include <HyperLearner.h>

Inheritance diagram for PLearn::HyperLearner:
Inheritance graph
[legend]
Collaboration diagram for PLearn::HyperLearner:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 HyperLearner ()
void setLearner (PP< PLearner > learner)
virtual void build ()
 Finish building the object; just call inherited::build followed by build_()
void setLearnerOptions (const TVec< string > &option_names, const TVec< string > &option_vals)
 sets the specified options to the specified values Will also detect if any option that is NOT listed in dont_restart_upon_change gets modified.
virtual void setTrainingSet (VMat training_set, bool call_forget=true)
 Forwarded to inner learner.
virtual void train ()
 Currently train_stats receives a single update, with the returned vector of the last strategy command...
virtual void forget ()
 Forwarded to inner learner.
virtual void finalize ()
 *** SUBCLASS WRITING: ***
TVec< string > getTrainCostNames () const
 Returns the getResultNames() of its last strategy command.
virtual string classname () const
virtual OptionListgetOptionList () const
virtual OptionMapgetOptionMap () const
virtual RemoteMethodMapgetRemoteMethodMap () const
virtual HyperLearnerdeepCopy (CopiesMap &copies) const
void makeDeepCopyFromShallowCopy (CopiesMap &copies)
 Transforms a shallow copy into a deep copy.
virtual void run ()
 Override this for runnable objects (default method issues a runtime error).
void auto_save ()
 Save the current HyperLearner in its expdir.

Static Public Member Functions

static string _classname_ ()
 Declares name and deepCopy methods.
static OptionList_getOptionList_ ()
static RemoteMethodMap_getRemoteMethodMap_ ()
static Object_new_instance_for_typemap_ ()
static bool _isa_ (const Object *o)
static void _static_initialize_ ()
static const PPathdeclaringFile ()

Public Attributes

PP< PTestertester
 The kind of train/test to perform for each combination of hyper-parameters.
TVec< string > option_fields
 learner option names to be reported in results table
TVec< string > dont_restart_upon_change
 list of options that do not require calling build() and forget() upon changing their value.
TVec< PP< HyperCommand > > strategy
 The strategy to follow to optimize hyper parameters.
string save_mode
bool provide_strategy_expdir
 should each strategy step be provided a directory expdir/Step#
bool save_final_learner
 should final learner be saved in expdir/final_learner.psave
bool save_strategy_learner
 should each Strat# learner be saved in expdir/Strat#final_learner.psave
bool reloaded
 needed for a warning
bool finalize_learner
 if true, we finalize the learner after training.

Static Public Attributes

static StaticInitializer _static_initializer_

Static Protected Member Functions

static void declareOptions (OptionList &ol)
 Declares this class' options.

Protected Attributes

PStream::mode_t save_mode_

Private Types

typedef EmbeddedLearner inherited

Private Member Functions

void build_ ()
 This does the actual building.

Detailed Description

Definition at line 55 of file HyperLearner.h.


Member Typedef Documentation

Reimplemented from PLearn::EmbeddedLearner.

Definition at line 57 of file HyperLearner.h.


Constructor & Destructor Documentation

PLearn::HyperLearner::HyperLearner ( )

Definition at line 77 of file HyperLearner.cc.

References PLearn::EmbeddedLearner::forward_test, and PLearn::EmbeddedLearner::provide_learner_expdir.

Referenced by auto_save().

    : save_mode_(PStream::plearn_ascii),
      save_mode("plearn_ascii"),
      provide_strategy_expdir(true),
      save_final_learner(true),
      save_strategy_learner(false),
      reloaded(false),
      finalize_learner(false)
{
    // Forward the 'test' method to the underlying learner.
    forward_test = true;
    // The default behavior is to let the PTester decide whether or not
    // to provide the underlying learner with an experiment directory.
    provide_learner_expdir = false;
}

Here is the caller graph for this function:


Member Function Documentation

string PLearn::HyperLearner::_classname_ ( ) [static]

Declares name and deepCopy methods.

Reimplemented from PLearn::EmbeddedLearner.

Definition at line 67 of file HyperLearner.cc.

: E[test1.E[mse]]");
OptionList & PLearn::HyperLearner::_getOptionList_ ( ) [static]

Reimplemented from PLearn::EmbeddedLearner.

Definition at line 67 of file HyperLearner.cc.

: E[test1.E[mse]]");
RemoteMethodMap & PLearn::HyperLearner::_getRemoteMethodMap_ ( ) [static]

Reimplemented from PLearn::EmbeddedLearner.

Definition at line 67 of file HyperLearner.cc.

: E[test1.E[mse]]");
bool PLearn::HyperLearner::_isa_ ( const Object o) [static]

Reimplemented from PLearn::EmbeddedLearner.

Definition at line 67 of file HyperLearner.cc.

: E[test1.E[mse]]");
Object * PLearn::HyperLearner::_new_instance_for_typemap_ ( ) [static]

Reimplemented from PLearn::EmbeddedLearner.

Definition at line 67 of file HyperLearner.cc.

: E[test1.E[mse]]");
StaticInitializer HyperLearner::_static_initializer_ & PLearn::HyperLearner::_static_initialize_ ( ) [static]

Reimplemented from PLearn::EmbeddedLearner.

Definition at line 67 of file HyperLearner.cc.

: E[test1.E[mse]]");
void PLearn::HyperLearner::auto_save ( )

Save the current HyperLearner in its expdir.

Definition at line 378 of file HyperLearner.cc.

References PLearn::diff(), PLearn::endl(), PLearn::PLearner::expdir, HyperLearner(), PLearn::PPath::isEmpty(), PLearn::Object::load(), PLearn::mvforce(), n, PLearn::perr, PLearn::Profiler::pl_profile_end(), PLearn::Profiler::pl_profile_start(), PLCHECK, PLERROR, PLearn::Object::save(), save_mode_, and PLearn::PLearner::verbosity.

Referenced by PLearn::HyperOptimize::optimize().

{
    Profiler::pl_profile_start("HyperLearner::auto_save");
    if(expdir.isEmpty())
        PLERROR("In HyperLearner::auto_save - we can't auto_save as"
                " we don't have any expdir");
    PPath f = expdir/"hyper_learner_auto_save.psave";
    PPath tmp=f+".tmp";

    if(verbosity>0)
        perr << "In HyperLearner::auto_save() - We save the hlearner"
             << endl;
    PLearn::save(tmp, this, save_mode_);

#ifdef BOUNDCHECK
    HyperLearner *n = new HyperLearner();
    PLearn::load(tmp,n);
    PLCHECK(PLearn::diff(n,this));
    delete n;
#endif

    mvforce(tmp,f);
    Profiler::pl_profile_end("HyperLearner::auto_save");
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::HyperLearner::build ( ) [virtual]

Finish building the object; just call inherited::build followed by build_()

Reimplemented from PLearn::EmbeddedLearner.

Definition at line 357 of file HyperLearner.cc.

References PLearn::EmbeddedLearner::build(), and build_().

Referenced by setTrainingSet().

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::HyperLearner::build_ ( ) [private]

This does the actual building.

Reimplemented from PLearn::EmbeddedLearner.

Definition at line 308 of file HyperLearner.cc.

References PLearn::EmbeddedLearner::learner_, PLearn::TVec< T >::length(), PLearn::PStream::parseModeT(), PLASSERT, save_mode, save_mode_, strategy, and tester.

Referenced by build().

{
    // ### This method should do the real building of the object,
    // ### according to set 'options', in *any* situation.
    // ### Typical situations include:
    // ###  - Initial building of an object from a few user-specified options
    // ###  - Building of a "reloaded" object: i.e. from the complete set of all serialised options.
    // ###  - Updating or "re-building" of an object after a few "tuning" options have been modified.
    // ### You should assume that the parent class' build_() has already been called.

    // Set the Tester's learner to point to the same as the HyperLearner's learner
    PLASSERT( tester );
    tester->learner = learner_;

    for(int commandnum=0; commandnum<strategy.length(); commandnum++)
        strategy[commandnum]->setHyperLearner(this);

    save_mode_ = PStream::parseModeT(save_mode);
}

Here is the call graph for this function:

Here is the caller graph for this function:

string PLearn::HyperLearner::classname ( ) const [virtual]

Reimplemented from PLearn::EmbeddedLearner.

Definition at line 67 of file HyperLearner.cc.

: E[test1.E[mse]]");
void PLearn::HyperLearner::declareOptions ( OptionList ol) [static, protected]

Declares this class' options.

Reimplemented from PLearn::EmbeddedLearner.

Definition at line 94 of file HyperLearner.cc.

References PLearn::OptionBase::buildoption, PLearn::declareOption(), PLearn::EmbeddedLearner::declareOptions(), dont_restart_upon_change, finalize_learner, PLearn::OptionBase::learntoption, PLearn::OptionBase::nosave, option_fields, provide_strategy_expdir, PLearn::redeclareOption(), reloaded, save_final_learner, save_mode, save_strategy_learner, PLearn::PLearner::seed_, strategy, and tester.

{
    declareOption(ol, "tester", &HyperLearner::tester, OptionBase::buildoption,
                  "A model for the kind of train/test to be performed for each \n"
                  "combination of hyper-parameters considered. \n"
                  "The tester's options are used as follows: \n"
                  "  - You don't need to provide tester.learner, it will be set from \n"
                  "    the HyperLearner's learner.\n"
                  "  - tester.statnames is the list of cost stats to be computed, \n"
                  "    reported in the results table, and used as criteria for the hyper-optimization \n"
                  "  - tester.splitter is the default splitter to use for validation (although this may be \n"
                  "    overridden locally in HyperOptimize, see help on HyperOptimize) \n"
                  "  - Similarly, no need to provide tester.learner tester.expdir, \n"
                  "    and tester.provide_learner_expdir. \n"
                  "    They will be set by HyperOptimize as it deems appropriate\n"
                  "    for each hyper-parameter combination it tries. \n"
                  "  - *If* HyperOptimize does set a non-empty expdir, then the kind of\n"
                  "    report and result files to be generated is taken from the remaining \n"
                  "    report_stats and save_... options of the tester \n");

    declareOption(ol, "option_fields", &HyperLearner::option_fields, OptionBase::buildoption,
                  "learner option names to be reported in results table");

    declareOption(ol, "dont_restart_upon_change", &HyperLearner::dont_restart_upon_change, OptionBase::buildoption,
                  "a list of names of options, used in the strategy, but that do not require \n"
                  "calling build() and forget() each time they are modified. \n"
                  "Note that this is almost always the case of the 'nstages' option \n"
                  "typically modified incrementally in an 'early stopping' fashion... \n");

    declareOption(ol, "strategy", &HyperLearner::strategy, OptionBase::buildoption,
                  "The strategy to follow to optimize the hyper-parameters.\n"
                  "It's a list of hyper-optimization commands to call"
                  " sequentially,\n"
                  "(mostly HyperOptimize, HyperSetOption and HyperRetrain"
                  " commands).\n");

    declareOption(ol, "provide_strategy_expdir", &HyperLearner::provide_strategy_expdir, OptionBase::buildoption,
                  "should each strategy step be provided a directory expdir/Step# to report its results");

    declareOption(ol, "save_final_learner", &HyperLearner::save_final_learner, OptionBase::buildoption,
                  "should final learner be saved in expdir/final_learner.psave");

    declareOption(ol, "save_strategy_learner", &HyperLearner::save_strategy_learner, OptionBase::buildoption,
                  "should final learner be saved in expdir/Strat#final_learner.psave");

    declareOption(ol, "save_mode", &HyperLearner::save_mode,
                  OptionBase::buildoption,
                  "The mode to use to save the file.");

    declareOption(
        ol, "finalize_learner", &HyperLearner::finalize_learner,
        OptionBase::buildoption,
        "Default false. If true, will finalize the learner after the training.");

    declareOption(ol, "reloaded", &HyperLearner::reloaded,
                  OptionBase::learntoption|OptionBase::nosave,
                  "Used to give a warning.");
    inherited::declareOptions(ol);

    // Hide some unused options.

    redeclareOption(ol, "seed", &HyperLearner::seed_, OptionBase::nosave,
                    "Not used.");

}

Here is the call graph for this function:

static const PPath& PLearn::HyperLearner::declaringFile ( ) [inline, static]

Reimplemented from PLearn::EmbeddedLearner.

Definition at line 112 of file HyperLearner.h.

HyperLearner * PLearn::HyperLearner::deepCopy ( CopiesMap copies) const [virtual]

Reimplemented from PLearn::EmbeddedLearner.

Definition at line 67 of file HyperLearner.cc.

: E[test1.E[mse]]");
void PLearn::HyperLearner::finalize ( ) [virtual]

*** SUBCLASS WRITING: ***

When this method is called the learner know it we will never train it again. So it can free resources that are needed only during the training. The functions test()/computeOutputs()/... should continue to work.

Reimplemented from PLearn::PLearner.

Definition at line 298 of file HyperLearner.cc.

References PLearn::PLearner::finalize(), PLearn::EmbeddedLearner::learner_, and tester.

{
    inherited::finalize();
    learner_->finalize();
    if(tester)tester->dataset=NULL;
}

Here is the call graph for this function:

void PLearn::HyperLearner::forget ( ) [virtual]

Forwarded to inner learner.

Reimplemented from PLearn::EmbeddedLearner.

Definition at line 285 of file HyperLearner.cc.

References i, PLearn::EmbeddedLearner::learner_, n, PLearn::TVec< T >::size(), PLearn::PLearner::stage, and strategy.

Referenced by setTrainingSet().

{
    learner_->forget();
    stage = 0;

    // Forward the forget to each command of the strategy.
    for (int i=0, n=strategy.size() ; i<n ; ++i)
        strategy[i]->forget();
}

Here is the call graph for this function:

Here is the caller graph for this function:

OptionList & PLearn::HyperLearner::getOptionList ( ) const [virtual]

Reimplemented from PLearn::EmbeddedLearner.

Definition at line 67 of file HyperLearner.cc.

: E[test1.E[mse]]");
OptionMap & PLearn::HyperLearner::getOptionMap ( ) const [virtual]

Reimplemented from PLearn::EmbeddedLearner.

Definition at line 67 of file HyperLearner.cc.

: E[test1.E[mse]]");
RemoteMethodMap & PLearn::HyperLearner::getRemoteMethodMap ( ) const [virtual]

Reimplemented from PLearn::EmbeddedLearner.

Definition at line 67 of file HyperLearner.cc.

: E[test1.E[mse]]");
TVec< string > PLearn::HyperLearner::getTrainCostNames ( ) const [virtual]

Returns the getResultNames() of its last strategy command.

Reimplemented from PLearn::EmbeddedLearner.

Definition at line 69 of file HyperLearner.cc.

References PLearn::TVec< T >::lastElement(), PLearn::TVec< T >::size(), and strategy.

Referenced by train().

{
    if (strategy.size() > 0)
        return strategy.lastElement()->getResultNames();
    else
        return TVec<string>();
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::HyperLearner::makeDeepCopyFromShallowCopy ( CopiesMap copies) [virtual]

Transforms a shallow copy into a deep copy.

Reimplemented from PLearn::EmbeddedLearner.

Definition at line 364 of file HyperLearner.cc.

References PLearn::deepCopyField(), dont_restart_upon_change, PLearn::EmbeddedLearner::makeDeepCopyFromShallowCopy(), option_fields, strategy, and tester.

Here is the call graph for this function:

void PLearn::HyperLearner::run ( ) [virtual]

Override this for runnable objects (default method issues a runtime error).

Runnable objects are objects that can be used as *THE* object of a .plearn script. The run() method specifies what they should do when executed.

Reimplemented from PLearn::Object.

Definition at line 331 of file HyperLearner.cc.

References PLearn::PLearner::expdir, PLearn::EmbeddedLearner::learner_, PLERROR, setTrainingSet(), tester, and train().

{
    if(!tester)
        PLERROR("No tester specified for HyperLearner.");

    if(!learner_ && !tester->learner)
        PLERROR("You must specify the underlying learner, \n"
                "either in the learner option of the HyperLearner \n"
                "or in the learne roption of the HyperLearner's tester \n");

    if(!tester->dataset)
        PLERROR("You must specify a dataset field in the tester");

    if(expdir=="")
        PLERROR("You should specify an experiment directory with the expdir option.");

    if(!learner_)
        learner_ = tester->learner;
    setTrainingSet(tester->dataset);
    train();    
}

Here is the call graph for this function:

void PLearn::HyperLearner::setLearner ( PP< PLearner learner) [inline]

Definition at line 89 of file HyperLearner.h.

Referenced by PLearn::HyperOptimize::optimize().

    { tester->learner = learner; learner_ = learner; }

Here is the caller graph for this function:

void PLearn::HyperLearner::setLearnerOptions ( const TVec< string > &  option_names,
const TVec< string > &  option_vals 
)

sets the specified options to the specified values Will also detect if any option that is NOT listed in dont_restart_upon_change gets modified.

If so, build() and forget() will be called on the learner

Definition at line 160 of file HyperLearner.cc.

References PLearn::TVec< T >::contains(), dont_restart_upon_change, i, PLearn::EmbeddedLearner::learner_, PLearn::TVec< T >::length(), and tester.

Referenced by PLearn::HyperOptimize::optimize(), and PLearn::HyperSetOption::optimize().

{
    // This function has been modified to call changeOptions on the learner
    // rather than a sequence of setOptions.  Learner options are accumulated
    // into a map suitable for calling changeOptions.  TESTER options are
    // still handled as a set of discrete setOptions.

    map<string,string> new_learner_options;
    int l = option_names.length();
    bool do_restart = false;
    for(int i=0; i<l; i++) {
        string optname = option_names[i];
        string newval = option_vals[i];

        // Decide if we should perform build after setting the options
        do_restart = do_restart ||
            ! dont_restart_upon_change.contains(optname);

        // Here, we incorporate the capability to change an option in the
        // tester (rather than the embedded learner) by prefixing the option
        // name with the special keyword TESTER.
        if (optname.substr(0,7) == "TESTER.") {
            // This is the old control flow for banging tester only
            optname = optname.substr(7);
            if (tester->getOption(optname) != newval)
                tester->setOption(optname, newval);
        }
        else {
            // New control flow: accumulate changes into the map
            if (learner_->getOption(optname) != newval)
                new_learner_options[optname] = newval;
        }
    }
    learner_->changeOptions(new_learner_options);
    if(do_restart)
    {
        learner_->build();
        learner_->forget();
    }
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::HyperLearner::setTrainingSet ( VMat  training_set,
bool  call_forget = true 
) [virtual]

Forwarded to inner learner.

Reimplemented from PLearn::EmbeddedLearner.

Definition at line 202 of file HyperLearner.cc.

References build(), forget(), PLWARNING, reloaded, PLearn::EmbeddedLearner::setTrainingSet(), and tester.

Referenced by run().

{

    //will set the training set to the sub learner
    inherited::setTrainingSet(training_set, call_forget);

    tester->dataset = training_set;
    if(tester->splitter)
        tester->splitter->setDataSet(training_set);

    if (call_forget)
    {
        if(reloaded)
            PLWARNING("In HyperLearner::setTrainingSet() - we were asked to"
                      " forget after having reloaded a previous version."
                      " To avoid doing this, in the PTester that includes this"
                      " HyperLearner, set call_forget_in_run = 0.");
        build();
        forget();
    }
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::HyperLearner::train ( ) [virtual]

Currently train_stats receives a single update, with the returned vector of the last strategy command...

Reimplemented from PLearn::EmbeddedLearner.

Definition at line 224 of file HyperLearner.cc.

References PLearn::endl(), PLearn::PLearner::expdir, finalize_learner, PLearn::EmbeddedLearner::getLearner(), getTrainCostNames(), PLearn::PLearner::getTrainingSet(), PLearn::PPath::isEmpty(), PLearn::PP< T >::isNull(), PLearn::EmbeddedLearner::learner_, PLearn::TVec< T >::length(), PLearn::PLearner::nstages, PLearn::perr, PLERROR, provide_strategy_expdir, PLearn::Object::save(), save_final_learner, save_mode_, save_strategy_learner, PLearn::EmbeddedLearner::setTrainStatsCollector(), PLearn::PLearner::stage, strategy, tester, PLearn::tostring(), PLearn::PLearner::train_stats, and PLearn::PLearner::verbosity.

Referenced by run().

{
    tester->dataset = getTrainingSet();
    Vec results;

    if (!train_stats)
        setTrainStatsCollector(new VecStatsCollector()); // set a dummy one if
    // none is set...

    train_stats->setFieldNames(getTrainCostNames());
    

    if(stage==0 && nstages>0)
    {
        for(int commandnum=0; commandnum<strategy.length(); commandnum++)
        {
            if(provide_strategy_expdir)
            {
                if(expdir!="")
                    strategy[commandnum]->setExperimentDirectory( expdir / ("Strat"+tostring(commandnum)) );
                else
                    strategy[commandnum]->setExperimentDirectory("");
            }

            if(verbosity>0)
                perr<<"HyperLearner: starting the optimization"<<endl;

            results = strategy[commandnum]->optimize();

            if(save_strategy_learner)
            {
                PPath strat_expdir=strategy[commandnum]->getExperimentDirectory();
                if(strat_expdir.isEmpty())
                    PLERROR("Cannot save the strategy model: no experiment directory has been set");
                if( getLearner().isNull() )
                    PLERROR("Cannot save final model: no final learner available");
                PLearn::save(strat_expdir+"final_learner.psave",*getLearner(), save_mode_);
            }
        }

        train_stats->update(results);

        if(finalize_learner)
            learner_->finalize();

        if(save_final_learner)
        {
            if(expdir=="")
                PLERROR("Cannot save final model: no experiment directory has been set");
            if( getLearner().isNull() )
                PLERROR("Cannot save final model: no final learner available");
            PLearn::save(expdir+"final_learner.psave",*getLearner(), save_mode_);
        }

        stage = 1;
    }
}

Here is the call graph for this function:

Here is the caller graph for this function:


Member Data Documentation

Reimplemented from PLearn::EmbeddedLearner.

Definition at line 112 of file HyperLearner.h.

list of options that do not require calling build() and forget() upon changing their value.

Definition at line 73 of file HyperLearner.h.

Referenced by declareOptions(), makeDeepCopyFromShallowCopy(), and setLearnerOptions().

if true, we finalize the learner after training.

Definition at line 85 of file HyperLearner.h.

Referenced by declareOptions(), and train().

learner option names to be reported in results table

Definition at line 72 of file HyperLearner.h.

Referenced by declareOptions(), PLearn::HyperOptimize::getResultsMat(), makeDeepCopyFromShallowCopy(), and PLearn::HyperOptimize::reportResult().

should each strategy step be provided a directory expdir/Step#

Definition at line 78 of file HyperLearner.h.

Referenced by declareOptions(), and train().

needed for a warning

Definition at line 81 of file HyperLearner.h.

Referenced by declareOptions(), and setTrainingSet().

should final learner be saved in expdir/final_learner.psave

Definition at line 79 of file HyperLearner.h.

Referenced by declareOptions(), and train().

Definition at line 77 of file HyperLearner.h.

Referenced by build_(), and declareOptions().

Definition at line 65 of file HyperLearner.h.

Referenced by auto_save(), build_(), and train().

should each Strat# learner be saved in expdir/Strat#final_learner.psave

Definition at line 80 of file HyperLearner.h.

Referenced by declareOptions(), and train().

The strategy to follow to optimize hyper parameters.

Definition at line 75 of file HyperLearner.h.

Referenced by build_(), declareOptions(), forget(), getTrainCostNames(), makeDeepCopyFromShallowCopy(), and train().


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines