PLearn 0.1
Public Member Functions | Static Public Member Functions | Public Attributes | Static Public Attributes | Protected Member Functions | Static Protected Member Functions | Protected Attributes | Private Types | Private Member Functions
PLearn::AddLayersNNet Class Reference

#include <AddLayersNNet.h>

Inheritance diagram for PLearn::AddLayersNNet:
Inheritance graph
[legend]
Collaboration diagram for PLearn::AddLayersNNet:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 AddLayersNNet ()
 Default constructor.
virtual void build ()
 Simply calls inherited::build() then build_().
virtual void makeDeepCopyFromShallowCopy (CopiesMap &copies)
 Transforms a shallow copy into a deep copy.
virtual string classname () const
virtual OptionListgetOptionList () const
virtual OptionMapgetOptionMap () const
virtual RemoteMethodMapgetRemoteMethodMap () const
virtual AddLayersNNetdeepCopy (CopiesMap &copies) const
Vec getHiddenUnitsActivation (int layer)
 Return the current activations of a given added hidden layer.
Mat getHiddenWeights (int layer)
 Return the hidden weights for a given hidden layer.
Mat getOutputHiddenWeights (int layer)
 Return the weights going out of a given added hidden layer.

Static Public Member Functions

static string _classname_ ()
static OptionList_getOptionList_ ()
static RemoteMethodMap_getRemoteMethodMap_ ()
static Object_new_instance_for_typemap_ ()
static bool _isa_ (const Object *o)
static void _static_initialize_ ()
static const PPathdeclaringFile ()

Public Attributes

TVec< intadd_hidden
string added_hidden_transfer_func
TVec< intparts_size

Static Public Attributes

static StaticInitializer _static_initializer_

Protected Member Functions

virtual void initializeParams (bool set_seed=true)
 Overridden to account for the added layers.
virtual void buildPenalties (const Var &hidden_layer)
 Fill the costs penalties.

Static Protected Member Functions

static void declareOptions (OptionList &ol)
 Declares this class' options.

Protected Attributes

TVec< intreal_parts_size
 This vector contains the 'real' parts' sizes (there is not '-1').
VarArray hidden_layers
 Contains the added hidden layers.
VarArray hidden_weights
 Contains the weights for the hidden layers added.

Private Types

typedef NNet inherited

Private Member Functions

void build_ ()
 This does the actual building.

Detailed Description

Definition at line 52 of file AddLayersNNet.h.


Member Typedef Documentation

Reimplemented from PLearn::NNet.

Definition at line 56 of file AddLayersNNet.h.


Constructor & Destructor Documentation

PLearn::AddLayersNNet::AddLayersNNet ( )

Default constructor.

Definition at line 56 of file AddLayersNNet.cc.


Member Function Documentation

string PLearn::AddLayersNNet::_classname_ ( ) [static]

Reimplemented from PLearn::NNet.

Definition at line 73 of file AddLayersNNet.cc.

OptionList & PLearn::AddLayersNNet::_getOptionList_ ( ) [static]

Reimplemented from PLearn::NNet.

Definition at line 73 of file AddLayersNNet.cc.

RemoteMethodMap & PLearn::AddLayersNNet::_getRemoteMethodMap_ ( ) [static]

Reimplemented from PLearn::NNet.

Definition at line 73 of file AddLayersNNet.cc.

bool PLearn::AddLayersNNet::_isa_ ( const Object o) [static]

Reimplemented from PLearn::NNet.

Definition at line 73 of file AddLayersNNet.cc.

Object * PLearn::AddLayersNNet::_new_instance_for_typemap_ ( ) [static]

Reimplemented from PLearn::NNet.

Definition at line 73 of file AddLayersNNet.cc.

StaticInitializer AddLayersNNet::_static_initializer_ & PLearn::AddLayersNNet::_static_initialize_ ( ) [static]

Reimplemented from PLearn::NNet.

Definition at line 73 of file AddLayersNNet.cc.

void PLearn::AddLayersNNet::build ( ) [virtual]

Simply calls inherited::build() then build_().

Reimplemented from PLearn::NNet.

Definition at line 109 of file AddLayersNNet.cc.

References PLearn::NNet::build(), build_(), PLearn::NNet::do_not_change_params, and PLearn::NNet::initialization_method.

{
    // We ensure that weights are not filled with random numbers, in order to be
    // able to compare with a classical NNet using the same seed.
    string initialization_method_backup = initialization_method;
    bool do_not_change_params_backup = do_not_change_params;
    initialization_method = "zero";
    do_not_change_params = true;
    inherited::build();
    initialization_method = initialization_method_backup;
    do_not_change_params = do_not_change_params_backup;
    build_();
}

Here is the call graph for this function:

void PLearn::AddLayersNNet::build_ ( ) [private]

This does the actual building.

Reimplemented from PLearn::NNet.

Definition at line 126 of file AddLayersNNet.cc.

References add_hidden, added_hidden_transfer_func, PLearn::TVec< T >::append(), PLearn::NNet::buildCosts(), PLearn::NNet::buildFuncs(), PLearn::NNet::buildOutputFromInput(), PLearn::NNet::buildTargetAndWeight(), PLearn::NNet::do_not_change_params, PLearn::NNet::hidden_layer, hidden_layers, hidden_weights, PLearn::NNet::hiddenLayer(), i, initializeParams(), PLearn::NNet::input, PLearn::PLearner::inputsize_, PLearn::TVec< T >::isEmpty(), PLearn::TVec< T >::length(), PLearn::VarArray::makeSharedValue(), PLearn::VarArray::nelems(), PLearn::NNet::optimizer, PLearn::NNet::output, PLearn::NNet::params, PLearn::NNet::paramsvalues, parts_size, PLERROR, real_parts_size, PLearn::TVec< T >::resize(), PLearn::NNet::sampleweight, PLearn::subMat(), PLearn::NNet::target, PLearn::tostring(), and PLearn::vconcat().

Referenced by build().

{
    // ### This method should do the real building of the object,
    // ### according to set 'options', in *any* situation. 
    // ### Typical situations include:
    // ###  - Initial building of an object from a few user-specified options
    // ###  - Building of a "reloaded" object: i.e. from the complete set of all serialised options.
    // ###  - Updating or "re-building" of an object after a few "tuning" options have been modified.
    // ### You should assume that the parent class' build_() has already been called.

    // Don't do anything if we do not have an inputsize.
    if (inputsize_ < 0)
        return;
    if (parts_size.isEmpty() || add_hidden.isEmpty())
        PLERROR("In AddLayersNNet::build_ - You must fill both 'parts_size' and 'add_hidden'");
    if (parts_size.length() != add_hidden.length())
        PLERROR("In AddLayersNNet::build_ - 'parts_size' and 'add_hidden' must have the same length");
    int n_parts = parts_size.length();
    int count_parts_size = 0;
    bool found_minus_one = false;
    int minus_one_index = -1;
    for (int i = 0; i < n_parts; i++) {
        if (parts_size[i] >= 0) {
            count_parts_size += parts_size[i];
        } else if (parts_size[i] == -1) {
            if (found_minus_one) {
                PLERROR("In AddLayersNNet::build_ - There can be only one '-1' in 'parts_size'");
            } else {
                // There is a '-1'.
                found_minus_one = true;
                minus_one_index = i;
            }
        } else {
            // There is a negative value that is not -1, that should not happen.
            PLERROR("In AddLayersNNet::build_ - Wrong value for parts_size[%d]: %d", i, parts_size[i]);
        }
    }
    if (count_parts_size > inputsize_)
        PLERROR("In AddLayersNNet::build_ - The sum of all parts size (%d) exceeds the inputsize (%d)", count_parts_size, inputsize_);
    if (found_minus_one) {
        real_parts_size.resize(parts_size.length());
        real_parts_size << parts_size;
        real_parts_size[minus_one_index] = inputsize_ - count_parts_size;
    } else {
        real_parts_size = parts_size;
        if (count_parts_size != inputsize_)
            PLERROR("In AddLayersNNet::build_ - The sum of all parts size (%d) is less than inputsize (%d)", count_parts_size, inputsize_);
    }

    // Now we redo the graph of variables, even if there is no added layer
    // (because the weights are not initialized in the parent class, since
    // 'initialization_method' is forced to 'zero' at build time).
  
    params.resize(0);

    // Create a Var for each part.
    VarArray input_parts(n_parts);
    int index = 0;
    for (int i = 0; i < n_parts; i++) {
        input_parts[i] = subMat(input, index, 0, real_parts_size[i], 1);
        input_parts[i]->setName("input_part_" + tostring(i));
        index += real_parts_size[i];
    }

    // Add the required hidden layers.
    hidden_layers.resize(n_parts);
    hidden_weights.resize(n_parts);
    for (int i = 0; i < n_parts; i++) {
        if (add_hidden[i] > 0) {
            Var weights = Var(1 + real_parts_size[i], add_hidden[i], ("w_added_" + tostring(i)).c_str());
            hidden_layers[i] = hiddenLayer(input_parts[i], weights, added_hidden_transfer_func);
            hidden_weights[i] = weights;
            params.append(hidden_weights[i]);
        } else {
            hidden_layers[i] = input_parts[i];
        }
    }

    // Create the concatenated "input" to the regular NNet.
    Var concat_input = vconcat(hidden_layers);

    Var hidden_layer;
    Var before_transfer_func;

    // Build main network graph.
    buildOutputFromInput(concat_input, hidden_layer, before_transfer_func);

    // Build target and weight variables.
    buildTargetAndWeight();

    // Build costs.
    buildCosts(output, target, hidden_layer, before_transfer_func);

    // Shared values hack...
    if (!do_not_change_params) {
        if(paramsvalues.length() == params.nelems())
            params << paramsvalues;
        else
        {
            paramsvalues.resize(params.nelems());
            initializeParams();
            if(optimizer)
                optimizer->reset();
        }
        params.makeSharedValue(paramsvalues);
    }

    // Build functions.
    buildFuncs(input, output, target, sampleweight, NULL);

}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::AddLayersNNet::buildPenalties ( const Var hidden_layer) [protected, virtual]

Fill the costs penalties.

Reimplemented from PLearn::NNet.

Definition at line 241 of file AddLayersNNet.cc.

References add_hidden, PLearn::affine_transform_weight_penalty(), PLearn::TVec< T >::append(), PLearn::NNet::bias_decay, PLearn::NNet::buildPenalties(), hidden_weights, i, PLearn::TVec< T >::length(), parts_size, PLearn::NNet::penalties, PLearn::NNet::penalty_type, and PLearn::NNet::weight_decay.

                                                          {
    inherited::buildPenalties(hidden_layer);
    if (hidden_weights.length() != parts_size.length())
        // The hidden weights have not yet been correctly initialized.
        return;
    for (int i = 0; i < parts_size.length(); i++) {
        if (add_hidden[i] > 0 && (weight_decay > 0 || bias_decay > 0)) {
            penalties.append(affine_transform_weight_penalty(hidden_weights[i], weight_decay, bias_decay, penalty_type));
        }
    }
}

Here is the call graph for this function:

string PLearn::AddLayersNNet::classname ( ) const [virtual]

Reimplemented from PLearn::NNet.

Definition at line 73 of file AddLayersNNet.cc.

void PLearn::AddLayersNNet::declareOptions ( OptionList ol) [static, protected]

Declares this class' options.

Reimplemented from PLearn::NNet.

Definition at line 78 of file AddLayersNNet.cc.

References add_hidden, added_hidden_transfer_func, PLearn::OptionBase::buildoption, PLearn::declareOption(), PLearn::NNet::declareOptions(), and parts_size.

{
    // ### For the "flags" of each option, you should typically specify  
    // ### one of OptionBase::buildoption, OptionBase::learntoption or 
    // ### OptionBase::tuningoption. Another possible flag to be combined with
    // ### is OptionBase::nosave

    // Build options.

    declareOption(ol, "parts_size", &AddLayersNNet::parts_size, OptionBase::buildoption,
                  "The size of each part. '-1' can be used to specify this part's size should\n"
                  "be such that all inputs are considered ('-1' can thus only appear once).");

    declareOption(ol, "add_hidden", &AddLayersNNet::add_hidden, OptionBase::buildoption,
                  "Specify for each part how many hidden units we want to add.");

    declareOption(ol, "added_hidden_transfer_func", &AddLayersNNet::added_hidden_transfer_func, OptionBase::buildoption,
                  "The transfer function for the added hidden layers.");

    // Learnt options.

    // declareOption(ol, "myoption", &AddLayersNNet::myoption, OptionBase::learntoption,
    //               "Help text describing this option");

    // Now call the parent class' declareOptions.
    inherited::declareOptions(ol);
}

Here is the call graph for this function:

static const PPath& PLearn::AddLayersNNet::declaringFile ( ) [inline, static]

Reimplemented from PLearn::NNet.

Definition at line 120 of file AddLayersNNet.h.

:

AddLayersNNet * PLearn::AddLayersNNet::deepCopy ( CopiesMap copies) const [virtual]

Reimplemented from PLearn::NNet.

Definition at line 73 of file AddLayersNNet.cc.

Vec PLearn::AddLayersNNet::getHiddenUnitsActivation ( int  layer)

Return the current activations of a given added hidden layer.

This is not a copy, so they should not be modified.

Definition at line 256 of file AddLayersNNet.cc.

References hidden_layers.

                                                     {
    return hidden_layers[layer]->value;
}
Mat PLearn::AddLayersNNet::getHiddenWeights ( int  layer)

Return the hidden weights for a given hidden layer.

This is not a copy, so they should not be modified.

Definition at line 263 of file AddLayersNNet.cc.

References hidden_weights.

                                             {
    return hidden_weights[layer]->matValue;
}
OptionList & PLearn::AddLayersNNet::getOptionList ( ) const [virtual]

Reimplemented from PLearn::NNet.

Definition at line 73 of file AddLayersNNet.cc.

OptionMap & PLearn::AddLayersNNet::getOptionMap ( ) const [virtual]

Reimplemented from PLearn::NNet.

Definition at line 73 of file AddLayersNNet.cc.

Mat PLearn::AddLayersNNet::getOutputHiddenWeights ( int  layer)

Return the weights going out of a given added hidden layer.

This is not a copy, so they should not be modified.

Definition at line 270 of file AddLayersNNet.cc.

References add_hidden, i, real_parts_size, and PLearn::NNet::w1.

                                                   {
    int count = 0;
    for (int i = 0; i < layer; i++)
        count += real_parts_size[i];
    return w1->matValue.subMatRows(count, add_hidden[layer]);
}
RemoteMethodMap & PLearn::AddLayersNNet::getRemoteMethodMap ( ) const [virtual]

Reimplemented from PLearn::NNet.

Definition at line 73 of file AddLayersNNet.cc.

void PLearn::AddLayersNNet::initializeParams ( bool  set_seed = true) [protected, virtual]

Overridden to account for the added layers.

Reimplemented from PLearn::NNet.

Definition at line 280 of file AddLayersNNet.cc.

References add_hidden, PLearn::NNet::fillWeights(), hidden_weights, i, PLearn::NNet::initializeParams(), PLearn::manual_seed(), PLearn::seed(), PLearn::PLearner::seed_, and PLearn::TVec< T >::size().

Referenced by build_().

                                                  {
    // TODO Remove later...
    if (set_seed) {
        if (seed_>=0)
            manual_seed(seed_);
        else
            PLearn::seed();
    }
    for (int i = 0; i < add_hidden.size(); i++)
        if (add_hidden[i] > 0)
            fillWeights(hidden_weights[i], true);
    inherited::initializeParams(false); // TODO Put this first later.
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::AddLayersNNet::makeDeepCopyFromShallowCopy ( CopiesMap copies) [virtual]

Transforms a shallow copy into a deep copy.

Reimplemented from PLearn::NNet.

Definition at line 297 of file AddLayersNNet.cc.

References add_hidden, PLearn::deepCopyField(), hidden_layers, hidden_weights, PLearn::NNet::makeDeepCopyFromShallowCopy(), parts_size, and real_parts_size.

Here is the call graph for this function:


Member Data Documentation

Reimplemented from PLearn::NNet.

Definition at line 120 of file AddLayersNNet.h.

Definition at line 83 of file AddLayersNNet.h.

Referenced by build_(), and declareOptions().

Contains the added hidden layers.

Definition at line 71 of file AddLayersNNet.h.

Referenced by build_(), getHiddenUnitsActivation(), and makeDeepCopyFromShallowCopy().

Contains the weights for the hidden layers added.

Definition at line 74 of file AddLayersNNet.h.

Referenced by build_(), buildPenalties(), getHiddenWeights(), initializeParams(), and makeDeepCopyFromShallowCopy().

This vector contains the 'real' parts' sizes (there is not '-1').

It is filled at build time from 'parts_size' and the training set inputsize.

Definition at line 68 of file AddLayersNNet.h.

Referenced by build_(), getOutputHiddenWeights(), and makeDeepCopyFromShallowCopy().


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines