PLearn 0.1
Public Member Functions | Static Public Member Functions | Public Attributes | Static Public Attributes | Static Protected Member Functions | Protected Attributes | Static Protected Attributes | Private Types | Private Member Functions
PLearn::UniformizeLearner Class Reference

#include <UniformizeLearner.h>

Inheritance diagram for PLearn::UniformizeLearner:
Inheritance graph
[legend]
Collaboration diagram for PLearn::UniformizeLearner:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 UniformizeLearner ()
virtual void build ()
 simply calls inherited::build() then build_()
virtual void makeDeepCopyFromShallowCopy (CopiesMap &copies)
 Transforms a shallow copy into a deep copy.
virtual string classname () const
virtual OptionListgetOptionList () const
virtual OptionMapgetOptionMap () const
virtual RemoteMethodMapgetRemoteMethodMap () const
virtual UniformizeLearnerdeepCopy (CopiesMap &copies) const
virtual int outputsize () const
 returns the size of this learner's output, (which typically may depend on its inputsize(), targetsize() and set options)
virtual void forget ()
 (Re-)initializes the PLearner in its fresh state (that state may depend on the 'seed' option) And sets 'stage' back to 0 (this is the stage of a fresh learner!)
virtual void train ()
 The role of the train method is to bring the learner up to stage==nstages, updating the train_stats collector with training costs measured on-line in the process.
virtual void computeOutput (const Vec &input, Vec &output) const
 Computes the output from the input.
virtual void computeCostsFromOutputs (const Vec &input, const Vec &output, const Vec &target, Vec &costs) const
 Computes the costs from already computed output.
virtual TVec< string > getTestCostNames () const
 Returns the names of the costs computed by computeCostsFromOutpus (and thus the test method)
virtual TVec< string > getTrainCostNames () const
 Returns the names of the objective costs that the train method computes and for which it updates the VecStatsCollector train_stats.
virtual void setTrainingSet (VMat training_set, bool call_forget=true)
 Declares the training set.
virtual TVec< string > getOutputNames () const
 Returns a vector of length outputsize() containing the outputs' names.

Static Public Member Functions

static string _classname_ ()
static OptionList_getOptionList_ ()
static RemoteMethodMap_getRemoteMethodMap_ ()
static Object_new_instance_for_typemap_ ()
static bool _isa_ (const Object *o)
static void _static_initialize_ ()
static const PPathdeclaringFile ()
static void computeRankMap (const Vec &v, int nquantiles, map< real, real > &rankmap)
 Sort v, and build a map mapping values to [0,1] rank.
static void computeWeightedRankMap (const Vec &v, int nquantiles, map< real, real > &rankmap, const Vec &weights)
static real mapToRank (real val, const map< real, real > &rankmap)
 Map non-missing value val to its [0,1] rank.

Public Attributes

TVec< string > which_fieldnames
TVec< intwhich_fieldnums
int nquantiles
bool raw_inputs_as_output

Static Public Attributes

static StaticInitializer _static_initializer_

Static Protected Member Functions

static void declareOptions (OptionList &ol)
 Declares this class' options.

Protected Attributes

int weight_field_index
TVec< string > input_field_names
TVec< map< real, real > > val_to_rank

Static Protected Attributes

static Vec v_no_missing
 Temporary storage vector used to store a copy without missing values.

Private Types

typedef PLearner inherited

Private Member Functions

void build_ ()
 This does the actual building.

Detailed Description

Definition at line 51 of file UniformizeLearner.h.


Member Typedef Documentation

Reimplemented from PLearn::PLearner.

Definition at line 54 of file UniformizeLearner.h.


Constructor & Destructor Documentation

PLearn::UniformizeLearner::UniformizeLearner ( )

Definition at line 48 of file UniformizeLearner.cc.

    :weight_field_index(-1),
     nquantiles(200),
     raw_inputs_as_output(false)
{
    // ...

    // ### You may or may not want to call build_() to finish building the object
    // build_();
}

Member Function Documentation

string PLearn::UniformizeLearner::_classname_ ( ) [static]

Reimplemented from PLearn::PLearner.

Definition at line 63 of file UniformizeLearner.cc.

OptionList & PLearn::UniformizeLearner::_getOptionList_ ( ) [static]

Reimplemented from PLearn::PLearner.

Definition at line 63 of file UniformizeLearner.cc.

RemoteMethodMap & PLearn::UniformizeLearner::_getRemoteMethodMap_ ( ) [static]

Reimplemented from PLearn::PLearner.

Definition at line 63 of file UniformizeLearner.cc.

bool PLearn::UniformizeLearner::_isa_ ( const Object o) [static]

Reimplemented from PLearn::PLearner.

Definition at line 63 of file UniformizeLearner.cc.

Object * PLearn::UniformizeLearner::_new_instance_for_typemap_ ( ) [static]

Reimplemented from PLearn::Object.

Definition at line 63 of file UniformizeLearner.cc.

StaticInitializer UniformizeLearner::_static_initializer_ & PLearn::UniformizeLearner::_static_initialize_ ( ) [static]

Reimplemented from PLearn::PLearner.

Definition at line 63 of file UniformizeLearner.cc.

void PLearn::UniformizeLearner::build ( ) [virtual]

simply calls inherited::build() then build_()

Reimplemented from PLearn::PLearner.

Definition at line 110 of file UniformizeLearner.cc.

References PLearn::PLearner::build(), and build_().

Here is the call graph for this function:

void PLearn::UniformizeLearner::build_ ( ) [private]

This does the actual building.

Reimplemented from PLearn::PLearner.

Definition at line 104 of file UniformizeLearner.cc.

Referenced by build().

{
    
}

Here is the caller graph for this function:

string PLearn::UniformizeLearner::classname ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 63 of file UniformizeLearner.cc.

virtual void PLearn::UniformizeLearner::computeCostsFromOutputs ( const Vec input,
const Vec output,
const Vec target,
Vec costs 
) const [virtual]

Computes the costs from already computed output.

virtual void PLearn::UniformizeLearner::computeOutput ( const Vec input,
Vec output 
) const [virtual]

Computes the output from the input.

void PLearn::UniformizeLearner::computeRankMap ( const Vec v,
int  nquantiles,
map< real, real > &  rankmap 
) [static]

Sort v, and build a map mapping values to [0,1] rank.

Missing values in v are ignored.

Definition at line 280 of file UniformizeLearner.cc.

References PLearn::TVec< T >::append(), PLearn::TVec< T >::hasMissing(), i, PLearn::is_missing(), PLearn::TVec< T >::length(), nquantiles, PLearn::TVec< T >::resize(), PLearn::sortElements(), and v_no_missing.

Referenced by train().

{
    v_no_missing.resize(v.length()); // Allocate enough memory.
    if (!v.hasMissing())
        v_no_missing << v;
    else {
        v_no_missing.resize(0);
        for (int i = 0; i < v.length(); i++)
            if (!is_missing(v[i]))
                v_no_missing.append(v[i]);
    }
    rankmap.clear();
    int max_index = v_no_missing.length() - 1;
    sortElements(v_no_missing);
    rankmap[v_no_missing[0]] = 0;
    rankmap[v_no_missing[max_index]] = 1;
    for(int k=1; k<nquantiles; k++)
    {
        real rank = real(k)/real(nquantiles);
        int pos = int(round(rank * max_index));
        real val = v_no_missing[pos];
        if(rankmap.find(val) == rankmap.end())
            rankmap[val] = rank;
    }
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::UniformizeLearner::computeWeightedRankMap ( const Vec v,
int  nquantiles,
map< real, real > &  rankmap,
const Vec weights 
) [static]

Definition at line 308 of file UniformizeLearner.cc.

References PLearn::TMat< T >::appendRow(), PLearn::TMat< T >::column(), PLearn::TVec< T >::hasMissing(), i, PLearn::is_missing(), PLearn::TMat< T >::length(), PLearn::TVec< T >::length(), nquantiles, PLearn::TMat< T >::resize(), and PLearn::sortRows().

Referenced by train().

{
    int l= v.length();

    Mat vw(0, 2);

    if (!v.hasMissing())
    {
        vw.resize(l,2);
        vw.column(0) << v;
        vw.column(1) << weights;

    }
    else 
    {
        Vec vvw(2);
        for (int i = 0; i < l; i++)
            if (!is_missing(v[i]))
            {
                vvw[0]= v[i];
                vvw[1]= weights[i];
                vw.appendRow(vvw);
            }
    }

    

    rankmap.clear();
    int max_index = vw.length() - 1;
    sortRows(vw, TVec<int>(1,0));

    for (int i = 1; i < l; i++)
        vw(i,1)+= vw(i-1,1);

    rankmap[vw(0,0)] = 0;
    rankmap[vw(max_index,0)] = 1;
    real totw= vw(max_index,1);

    for(int k=1, i= 0; k<nquantiles; ++k)
    {
        real rank = real(k)/real(nquantiles);
        real qw= totw*rank;
        while(vw(i,1) < qw)
            ++i;

        real val = vw(i,0);

        rank= vw(i,1)/totw;

        if(rankmap.find(val) == rankmap.end())
            rankmap[val]= rank;
    }
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::UniformizeLearner::declareOptions ( OptionList ol) [static, protected]

Declares this class' options.

Reimplemented from PLearn::PLearner.

Definition at line 65 of file UniformizeLearner.cc.

References PLearn::OptionBase::buildoption, PLearn::declareOption(), PLearn::PLearner::declareOptions(), input_field_names, PLearn::OptionBase::learntoption, nquantiles, raw_inputs_as_output, val_to_rank, which_fieldnames, and which_fieldnums.

{
    // ### Declare all of this object's options here
    // ### For the "flags" of each option, you should typically specify  
    // ### one of OptionBase::buildoption, OptionBase::learntoption or 
    // ### OptionBase::tuningoption. Another possible flag to be combined with
    // ### is OptionBase::nosave

    // ### ex:
    // declareOption(ol, "myoption", &UniformizeLearner::myoption, OptionBase::buildoption,
    //               "Help text describing this option");
    // ...

    //build
    
    declareOption(ol, "which_fieldnames", &UniformizeLearner::which_fieldnames, OptionBase::buildoption,
                  "The names of the fields to uniformize.\n"
                  "If both which_fieldnames and which_fieldnums are empty, all fields are normalized.");
    declareOption(ol, "which_fieldnums", &UniformizeLearner::which_fieldnums, OptionBase::buildoption,
                  "The indexes of the fields to uniformize. Leave this option empty if you specify which_fieldnames.\n"
                  "If both which_fieldnames and which_fieldnums are empty, all fields are normalized.");
    declareOption(ol, "nquantiles", &UniformizeLearner::nquantiles, OptionBase::buildoption,
                  "How many intervals to use to divide the sorted values");

    declareOption(ol, "raw_inputs_as_output", &UniformizeLearner::raw_inputs_as_output, OptionBase::buildoption,
                  "If true, raw inputs are appended to uniformized outputs for all uniformized fields.");

    //learnt

    declareOption(ol, "val_to_rank", &UniformizeLearner::val_to_rank, OptionBase::learntoption,
                  "Remembers mapping between a few values and their [0,1] ranking.");

    declareOption(ol, "input_field_names", &UniformizeLearner::input_field_names, OptionBase::learntoption,
                  "Remembers the names of the input fields.");

    // Now call the parent class' declareOptions
    inherited::declareOptions(ol);
}

Here is the call graph for this function:

static const PPath& PLearn::UniformizeLearner::declaringFile ( ) [inline, static]

Reimplemented from PLearn::PLearner.

Definition at line 118 of file UniformizeLearner.h.

UniformizeLearner * PLearn::UniformizeLearner::deepCopy ( CopiesMap copies) const [virtual]

Reimplemented from PLearn::PLearner.

Definition at line 63 of file UniformizeLearner.cc.

void PLearn::UniformizeLearner::forget ( ) [virtual]

(Re-)initializes the PLearner in its fresh state (that state may depend on the 'seed' option) And sets 'stage' back to 0 (this is the stage of a fresh learner!)

Reimplemented from PLearn::PLearner.

Definition at line 148 of file UniformizeLearner.cc.

References PLearn::PLearner::stage.

{
    stage = 0; // untrained
}
OptionList & PLearn::UniformizeLearner::getOptionList ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 63 of file UniformizeLearner.cc.

OptionMap & PLearn::UniformizeLearner::getOptionMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 63 of file UniformizeLearner.cc.

TVec< string > PLearn::UniformizeLearner::getOutputNames ( ) const [virtual]

Returns a vector of length outputsize() containing the outputs' names.

Default version returns ["out0", "out1", ...] Don't forget name should not have space or it will cause trouble when they are saved in the file {metadatadir}/fieldnames

Reimplemented from PLearn::PLearner.

Definition at line 435 of file UniformizeLearner.cc.

References input_field_names, n, and outputsize().

{
    int n = outputsize();
    TVec<string> outnames(n);
    int nk= which_fieldnums.size();

    for(int k= 0; k < nk; ++k)
        outnames[k]= string("uniformized_")+input_field_names[which_fieldnums[k]];
    for(int k= nk; k < n; ++k)
        outnames[k]= input_field_names[k-nk];

    return outnames;
}

Here is the call graph for this function:

RemoteMethodMap & PLearn::UniformizeLearner::getRemoteMethodMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 63 of file UniformizeLearner.cc.

TVec< string > PLearn::UniformizeLearner::getTestCostNames ( ) const [virtual]

Returns the names of the costs computed by computeCostsFromOutpus (and thus the test method)

Implements PLearn::PLearner.

Definition at line 422 of file UniformizeLearner.cc.

{
    static TVec<string> nocosts;
    return nocosts;
}
TVec< string > PLearn::UniformizeLearner::getTrainCostNames ( ) const [virtual]

Returns the names of the objective costs that the train method computes and for which it updates the VecStatsCollector train_stats.

Implements PLearn::PLearner.

Definition at line 428 of file UniformizeLearner.cc.

{
    static TVec<string> nocosts;
    return nocosts;
}
void PLearn::UniformizeLearner::makeDeepCopyFromShallowCopy ( CopiesMap copies) [virtual]

Transforms a shallow copy into a deep copy.

Reimplemented from PLearn::PLearner.

Definition at line 117 of file UniformizeLearner.cc.

References PLearn::deepCopyField(), input_field_names, PLearn::PLearner::makeDeepCopyFromShallowCopy(), val_to_rank, which_fieldnames, and which_fieldnums.

Here is the call graph for this function:

real PLearn::UniformizeLearner::mapToRank ( real  val,
const map< real, real > &  rankmap 
) [static]

Map non-missing value val to its [0,1] rank.

Definition at line 366 of file UniformizeLearner.cc.

References PLearn::is_missing(), and PLASSERT.

{
    PLASSERT( !is_missing(val) );
    real minv = rankmap.begin()->first;
    if(val<=minv)
        return 0;
    real maxv = rankmap.rbegin()->first;
    if(val>=maxv)
        return 1;
    map<real,real>::const_iterator high = rankmap.upper_bound(val);
    map<real,real>::const_iterator low = high; --low;

    real rank = low->second + (val-low->first)*(high->second-low->second)/(high->first-low->first);
    return rank;
}

Here is the call graph for this function:

int PLearn::UniformizeLearner::outputsize ( ) const [virtual]

returns the size of this learner's output, (which typically may depend on its inputsize(), targetsize() and set options)

Implements PLearn::PLearner.

Definition at line 131 of file UniformizeLearner.cc.

References PLearn::PLearner::inputsize(), PLearn::TVec< T >::length(), raw_inputs_as_output, PLearn::TVec< T >::size(), which_fieldnames, and which_fieldnums.

Referenced by getOutputNames().

{
    int nk= 0;
    if(raw_inputs_as_output)
    {
        nk= which_fieldnames.length();
        if(nk == 0)
            nk= which_fieldnums.size();
        if(nk == 0)//no field specified: uniformize all
            nk= inputsize();
    }
    return nk+inputsize();
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::UniformizeLearner::setTrainingSet ( VMat  training_set,
bool  call_forget = true 
) [virtual]

Declares the training set.

Then calls build() and forget() if necessary. Also sets this learner's inputsize_ targetsize_ weightsize_ from those of the training_set. Note: You shouldn't have to override this in subclasses, except in maybe to forward the call to an underlying learner.

Reimplemented from PLearn::PLearner.

Definition at line 156 of file UniformizeLearner.cc.

References PLearn::VMat::getFieldIndex(), PLearn::PLearner::getTrainingSet(), input_field_names, PLearn::TVec< T >::length(), PLERROR, PLearn::TVec< T >::resize(), PLearn::PLearner::setTrainingSet(), PLearn::TVec< T >::size(), PLearn::PLearner::train_set, weight_field_index, which_fieldnames, and which_fieldnums.

{
    inherited::setTrainingSet(training_set, call_forget);
    VMat dataset = getTrainingSet();

    if(dataset->weightsize() > 1)
        PLERROR("In UniformizeLearner::setTrainingSet: Only one weight supported.");

    if(train_set->weightsize() > 0)
        weight_field_index= dataset->fieldIndex(dataset->weightFieldNames()[0]);
    
    input_field_names.resize(dataset->inputsize());
    input_field_names << dataset->inputFieldNames();

    int nk = which_fieldnames.length();
    if(nk==0)
        nk = which_fieldnums.size();
    else
    {
        which_fieldnums.resize(nk);
        for(int k=0; k<nk; k++)
            which_fieldnums[k] = train_set->getFieldIndex(which_fieldnames[k]);
    }

    if(nk == 0)//no field specified, uniformize all.
    {
        nk= train_set->inputsize();
        which_fieldnums.resize(nk);
        for(int k= 0; k < nk; ++k)
            which_fieldnums[k]= k;
    }
}

Here is the call graph for this function:

void PLearn::UniformizeLearner::train ( ) [virtual]

The role of the train method is to bring the learner up to stage==nstages, updating the train_stats collector with training costs measured on-line in the process.

Implements PLearn::PLearner.

Definition at line 192 of file UniformizeLearner.cc.

References computeRankMap(), computeWeightedRankMap(), PLearn::VMat::getColumn(), PLearn::VMat::getFieldIndex(), PLearn::VMat::length(), PLearn::TVec< T >::length(), nquantiles, PLearn::TVec< T >::resize(), PLearn::TVec< T >::size(), PLearn::PLearner::stage, PLearn::PLearner::train_set, val_to_rank, weight_field_index, which_fieldnames, and which_fieldnums.

{
    // The role of the train method is to bring the learner up to stage==nstages,
    // updating train_stats with training costs measured on-line in the process.

    /* TYPICAL CODE:      
       static Vec input  // static so we don't reallocate/deallocate memory each time...
       static Vec target
       input.resize(inputsize())    // the train_set's inputsize()
       target.resize(targetsize())  // the train_set's targetsize()
       real weight

       if(!train_stats)  // make a default stats collector, in case there's none
       train_stats = new VecStatsCollector()

       if(nstages<stage) // asking to revert to a previous stage!
       forget()  // reset the learner to stage=0

       while(stage<nstages)
       {
       // clear statistics of previous epoch
       train_stats->forget() 
          
       //... train for 1 stage, and update train_stats,
       // using train_set->getSample(input, target, weight)
       // and train_stats->update(train_costs)
          
       ++stage
       train_stats->finalize() // finalize statistics for this epoch
       }
    */

    if(stage==0) // untrained
    {
        int nk = which_fieldnames.length();
        if(nk==0)
            nk = which_fieldnums.size();
        else
        {
            which_fieldnums.resize(nk);
            for(int k=0; k<nk; k++)
                which_fieldnums[k] = train_set->getFieldIndex(which_fieldnames[k]);
        }

        if(nk == 0)//no field specified, uniformize all.
        {
            nk= train_set->inputsize();
            which_fieldnums.resize(nk);
            for(int k= 0; k < nk; ++k)
                which_fieldnums[k]= k;
        }


        int l = train_set->length();

        bool weighted= train_set->weightsize() == 1;

        static Vec colw;
        if(weighted)
        {
            colw.resize(l);
            train_set->getColumn(weight_field_index, colw);
        }

        static Vec colv;
        colv.resize(l);
      
        val_to_rank.resize(nk);
        for(int k=0; k<nk; k++)
        {
            train_set->getColumn(which_fieldnums[k],colv);
            if(weighted)
                computeWeightedRankMap(colv, nquantiles, val_to_rank[k], colw);
            else
                computeRankMap(colv, nquantiles, val_to_rank[k]);
        }
        stage = 1; // trained
    }
}

Here is the call graph for this function:


Member Data Documentation

Reimplemented from PLearn::PLearner.

Definition at line 118 of file UniformizeLearner.h.

Definition at line 76 of file UniformizeLearner.h.

Referenced by computeRankMap(), computeWeightedRankMap(), declareOptions(), and train().

Definition at line 78 of file UniformizeLearner.h.

Referenced by declareOptions(), and outputsize().

Temporary storage vector used to store a copy without missing values.

Definition at line 59 of file UniformizeLearner.h.

Referenced by computeRankMap().

Definition at line 66 of file UniformizeLearner.h.

Referenced by declareOptions(), makeDeepCopyFromShallowCopy(), and train().

Definition at line 61 of file UniformizeLearner.h.

Referenced by setTrainingSet(), and train().


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines