PLearn 0.1
Public Member Functions | Static Public Member Functions | Public Attributes | Static Public Attributes | Static Protected Member Functions | Protected Attributes | Private Types | Private Member Functions
PLearn::RBMJointLLParameters Class Reference

Stores and learns the parameters between two layers of an RBM. More...

#include <RBMJointLLParameters.h>

Inheritance diagram for PLearn::RBMJointLLParameters:
Inheritance graph
[legend]
Collaboration diagram for PLearn::RBMJointLLParameters:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 RBMJointLLParameters (real the_learning_rate=0)
 Default constructor.
 RBMJointLLParameters (PP< RBMLLParameters > &the_target_params, PP< RBMLLParameters > &the_cond_params, real the_learning_rate=0)
 Constructor from two string prototymes.
virtual void setAsUpInput (const Vec &input) const
 Sets input_vec to input, target_given_cond and going_up to false.
virtual void setAsDownInput (const Vec &input) const
 Sets input_vec to input, target_given_cond to false, going_up to true.
virtual void setAsCondInput (const Vec &input) const
 Sets input_vec to input, and target_given_cond to true.
virtual void computeUnitActivations (int start, int length, const Vec &activations) const
 Computes the vectors of activation of "length" units, starting from "start", and concatenates them into "activations".
virtual void fprop (const Vec &input, Vec &output) const
 given the input, compute the output (possibly resize it appropriately)
virtual void bpropUpdate (const Vec &input, const Vec &output, Vec &input_gradient, const Vec &output_gradient)
 Adapt based on the output gradient: this method should only be called just after a corresponding fprop; it should be called with the same arguments as fprop for the first two arguments (and output should not have been modified since then).
virtual void forget ()
 reset the parameters to the state they would be BEFORE starting training.
virtual string classname () const
virtual OptionListgetOptionList () const
virtual OptionMapgetOptionMap () const
virtual RemoteMethodMapgetRemoteMethodMap () const
virtual RBMJointLLParametersdeepCopy (CopiesMap &copies) const
virtual void build ()
 Post-constructor.
virtual void makeDeepCopyFromShallowCopy (CopiesMap &copies)
 Transforms a shallow copy into a deep copy.

Static Public Member Functions

static string _classname_ ()
 optionally perform some processing after training, or after a series of fprop/bpropUpdate calls to prepare the model for truly out-of-sample operation.
static OptionList_getOptionList_ ()
static RemoteMethodMap_getRemoteMethodMap_ ()
static Object_new_instance_for_typemap_ ()
static bool _isa_ (const Object *o)
static void _static_initialize_ ()
static const PPathdeclaringFile ()

Public Attributes

PP< RBMLLParameterstarget_params
 RBMParameters between the target and the upper layer.
PP< RBMLLParameterscond_params
 RBMParameters between the conditioning input and the upper layer.

Static Public Attributes

static StaticInitializer _static_initializer_

Static Protected Member Functions

static void declareOptions (OptionList &ol)
 Declares the class options.

Protected Attributes

bool target_given_cond
 If true, we do not compute the up activations from a down example nor down activations from up example, but we compute the value of the "target" part of the down layer from the "conditioning" part, by summing over all the possible values of the up layer.
int target_size
 size of the target part of down layer
int cond_size
 size of the conditioning part of down layer
Vec out_act
 stores output activations

Private Types

typedef RBMLLParameters inherited

Private Member Functions

void build_units_types ()
 Builds up_units_types and down_units_types from the embedded RBMParameters.
void build_ ()
 This does the actual building.

Detailed Description

Stores and learns the parameters between two layers of an RBM.

Todo:
: yes
Deprecated:
Use ../RBMMixedConnection.h instead

Definition at line 56 of file RBMJointLLParameters.h.


Member Typedef Documentation

Reimplemented from PLearn::RBMLLParameters.

Definition at line 58 of file RBMJointLLParameters.h.


Constructor & Destructor Documentation

PLearn::RBMJointLLParameters::RBMJointLLParameters ( real  the_learning_rate = 0)

Default constructor.

Definition at line 52 of file RBMJointLLParameters.cc.

    : inherited(the_learning_rate)
{
}
PLearn::RBMJointLLParameters::RBMJointLLParameters ( PP< RBMLLParameters > &  the_target_params,
PP< RBMLLParameters > &  the_cond_params,
real  the_learning_rate = 0 
)

Constructor from two string prototymes.

Definition at line 57 of file RBMJointLLParameters.cc.

References build().

    : inherited( the_learning_rate ),
      target_params( the_target_params ),
      cond_params( the_cond_params )
{
    // We're not sure inherited::build() has been called
    build();
}

Here is the call graph for this function:


Member Function Documentation

string PLearn::RBMJointLLParameters::_classname_ ( ) [static]

optionally perform some processing after training, or after a series of fprop/bpropUpdate calls to prepare the model for truly out-of-sample operation.

THE DEFAULT IMPLEMENTATION PROVIDED IN THE SUPER-CLASS DOES NOT DO ANYTHING.

Reimplemented from PLearn::RBMLLParameters.

Definition at line 50 of file RBMJointLLParameters.cc.

OptionList & PLearn::RBMJointLLParameters::_getOptionList_ ( ) [static]

Reimplemented from PLearn::RBMLLParameters.

Definition at line 50 of file RBMJointLLParameters.cc.

RemoteMethodMap & PLearn::RBMJointLLParameters::_getRemoteMethodMap_ ( ) [static]

Reimplemented from PLearn::RBMLLParameters.

Definition at line 50 of file RBMJointLLParameters.cc.

bool PLearn::RBMJointLLParameters::_isa_ ( const Object o) [static]

Reimplemented from PLearn::RBMLLParameters.

Definition at line 50 of file RBMJointLLParameters.cc.

Object * PLearn::RBMJointLLParameters::_new_instance_for_typemap_ ( ) [static]

Reimplemented from PLearn::RBMLLParameters.

Definition at line 50 of file RBMJointLLParameters.cc.

StaticInitializer RBMJointLLParameters::_static_initializer_ & PLearn::RBMJointLLParameters::_static_initialize_ ( ) [static]

Reimplemented from PLearn::RBMLLParameters.

Definition at line 50 of file RBMJointLLParameters.cc.

void PLearn::RBMJointLLParameters::bpropUpdate ( const Vec input,
const Vec output,
Vec input_gradient,
const Vec output_gradient 
) [virtual]

Adapt based on the output gradient: this method should only be called just after a corresponding fprop; it should be called with the same arguments as fprop for the first two arguments (and output should not have been modified since then).

this version allows to obtain the input gradient as well

Since sub-classes are supposed to learn ONLINE, the object is 'ready-to-be-used' just after any bpropUpdate. N.B. A DEFAULT IMPLEMENTATION IS PROVIDED IN THE SUPER-CLASS, WHICH JUST CALLS bpropUpdate(input, output, input_gradient, output_gradient) AND IGNORES INPUT GRADIENT. this version allows to obtain the input gradient as well N.B. THE DEFAULT IMPLEMENTATION IN SUPER-CLASS JUST RAISES A PLERROR.

Reimplemented from PLearn::RBMLLParameters.

Definition at line 257 of file RBMJointLLParameters.cc.

References PLearn::TVec< T >::clear(), cond_size, PLearn::RBMLLParameters::down_units_bias, i, j, PLearn::RBMParameters::learning_rate, PLearn::multiplyAcc(), out_act, PLASSERT, PLearn::TVec< T >::resize(), PLearn::sigmoid(), PLearn::TVec< T >::size(), PLearn::TVec< T >::subVec(), target_size, PLearn::RBMParameters::up_layer_size, PLearn::RBMLLParameters::up_units_bias, w, and PLearn::RBMLLParameters::weights.

{
    PLASSERT( input.size() == cond_size );
    PLASSERT( output.size() == target_size );
    PLASSERT( output_gradient.size() == target_size );
    input_gradient.resize( cond_size );
    input_gradient.clear();

    //for( int k=0 ; k<target_size ; k++ )
    //    down_units_bias[k] -= learning_rate * output_gradient[k];
    multiplyAcc( down_units_bias.subVec(0, target_size),
                 output_gradient, -learning_rate );

    for( int i=0 ; i<up_layer_size ; i++ )
    {
        real* w = weights[i];
        real d_out_act = 0;
        for( int k=0 ; k<target_size ; k++ )
        {
            // dC/d(weights(i,k)+out_act[i])
            real d_z = output_gradient[k] * (-sigmoid(-w[k]-out_act[i]));
            w[k] -= learning_rate * d_z;

            d_out_act += d_z;
        }
        up_units_bias[i] -= learning_rate * d_out_act;

        for( int j=0 ; j<cond_size ; j++ )
        {
            real& w_ij = w[j+target_size];
            input_gradient[j] += d_out_act * w_ij;
            w_ij -= learning_rate * d_out_act * input[j];
        }
    }

}

Here is the call graph for this function:

void PLearn::RBMJointLLParameters::build ( ) [virtual]

Post-constructor.

The normal implementation should call simply inherited::build(), then this class's build_(). This method should be callable again at later times, after modifying some option fields to change the "architecture" of the object.

Reimplemented from PLearn::RBMLLParameters.

Definition at line 175 of file RBMJointLLParameters.cc.

References PLearn::RBMLLParameters::build(), build_(), and build_units_types().

Referenced by RBMJointLLParameters().

{
    // Begin by this, else inherited::build() will not work properly
    build_units_types();

    inherited::build();
    build_();
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::RBMJointLLParameters::build_ ( ) [private]

This does the actual building.

Reimplemented from PLearn::RBMLLParameters.

Definition at line 120 of file RBMJointLLParameters.cc.

References cond_params, cond_size, PLearn::RBMLLParameters::down_units_bias, PLearn::RBMLLParameters::down_units_bias_neg_stats, PLearn::RBMLLParameters::down_units_bias_pos_stats, PLearn::OnlineLearningModule::input_size, PLearn::merge(), PLearn::OnlineLearningModule::output_size, PLearn::TMat< T >::subMatColumns(), PLearn::TVec< T >::subVec(), target_params, target_size, PLearn::RBMLLParameters::up_units_bias, PLearn::RBMLLParameters::up_units_bias_neg_stats, PLearn::RBMLLParameters::up_units_bias_pos_stats, PLearn::RBMLLParameters::weights, PLearn::RBMLLParameters::weights_neg_stats, and PLearn::RBMLLParameters::weights_pos_stats.

Referenced by build().

{
    // The first part of weights correspond to the weights between the target
    // part and the up layer
    weights.subMatColumns( 0, target_size ) << target_params->weights;
    target_params->weights = weights.subMatColumns( 0, target_size );

    // Second part correspond to the weights between the cond and up layer
    weights.subMatColumns( target_size, cond_size ) << cond_params->weights;
    cond_params->weights = weights.subMatColumns( target_size, cond_size );

    // same thing for the statistics
    target_params->weights_pos_stats =
        weights_pos_stats.subMatColumns( 0, target_size );

    cond_params->weights_pos_stats =
        weights_pos_stats.subMatColumns( target_size, cond_size );

    target_params->weights_neg_stats =
        weights_neg_stats.subMatColumns( 0, target_size );

    cond_params->weights_neg_stats =
        weights_neg_stats.subMatColumns( target_size, cond_size );

    // Same thing for down units bias
    down_units_bias = merge( target_params->down_units_bias,
                             cond_params->down_units_bias );

    target_params->down_units_bias_pos_stats =
        down_units_bias_pos_stats.subVec( 0, target_size );

    cond_params->down_units_bias_pos_stats =
        down_units_bias_pos_stats.subVec( target_size, cond_size );

    target_params->down_units_bias_neg_stats =
        down_units_bias_neg_stats.subVec( 0, target_size );

    cond_params->down_units_bias_neg_stats =
        down_units_bias_neg_stats.subVec( target_size, cond_size );

    // The up layer units parameters are shared between the three RBMParameters
    up_units_bias = cond_params->up_units_bias;
    target_params->up_units_bias = up_units_bias;

    target_params->up_units_bias_pos_stats = up_units_bias_pos_stats;
    cond_params->up_units_bias_pos_stats = up_units_bias_pos_stats;

    target_params->up_units_bias_neg_stats = up_units_bias_neg_stats;
    cond_params->up_units_bias_neg_stats = up_units_bias_neg_stats;

    // sizes for fprop() and all OnlineLearningModules methods
    input_size = cond_size;
    output_size = target_size;
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::RBMJointLLParameters::build_units_types ( ) [private]

Builds up_units_types and down_units_types from the embedded RBMParameters.

Definition at line 94 of file RBMJointLLParameters.cc.

References cond_params, cond_size, PLearn::RBMParameters::down_units_types, out_act, PLERROR, PLearn::TMat< T >::resize(), PLearn::TVec< T >::resize(), target_params, target_size, PLearn::RBMParameters::up_units_types, and PLearn::RBMLLParameters::weights.

Referenced by build().

{
    if( !target_params || !cond_params )
        return;

    if( target_params->up_units_types != cond_params->up_units_types )
        PLERROR( "RBMJointLLParameters::build_units_types - \n"
                 "target_params->up_units_types should be equal to"
                 " cond_params->up_units_types\n"
                 "(\"%s\" != \"%s\").\n",
                 target_params->up_units_types.c_str(),
                 cond_params->up_units_types.c_str() );

    up_units_types = cond_params->up_units_types;
    down_units_types = target_params->down_units_types;
    down_units_types += cond_params->down_units_types;

    target_size = target_params->down_layer_size;
    cond_size = cond_params->down_layer_size;

    // to avoid "forget()" being called in RBMParameters::build_()
    weights.resize( int(up_units_types.length()),
                    int(down_units_types.length()) );
    out_act.resize( int(up_units_types.length()) );
}

Here is the call graph for this function:

Here is the caller graph for this function:

string PLearn::RBMJointLLParameters::classname ( ) const [virtual]

Reimplemented from PLearn::RBMLLParameters.

Definition at line 50 of file RBMJointLLParameters.cc.

void PLearn::RBMJointLLParameters::computeUnitActivations ( int  start,
int  length,
const Vec activations 
) const [virtual]

Computes the vectors of activation of "length" units, starting from "start", and concatenates them into "activations".

"start" indexes an up unit if "going_up", else a down unit.

Reimplemented from PLearn::RBMLLParameters.

Definition at line 223 of file RBMJointLLParameters.cc.

References i, j, PLearn::TVec< T >::length(), m, PLASSERT, PLearn::productAcc(), PLearn::softplus(), PLearn::TMat< T >::subMatColumns(), and w.

Referenced by fprop().

{
    if( target_given_cond )
    {
        PLASSERT( activations.length() == length );
        PLASSERT( start+length <= target_size );

        out_act << up_units_bias;
        Mat V = weights.subMatColumns( target_size, cond_size );
        // out_act = up_units_bias + V * input_vec
        productAcc( out_act, V, input_vec );

        // actY_i = B_i - sum_j softplus(-(W_ji + C_j + sum_k V_jk p(P_k)))
        //        = B_i - sum_j softplus(-(W_ji + out_act[j]) )
        for( int i=start ; i<start+length ; i++ )
        {
            real somme = down_units_bias[i];
            real *w = &weights[0][i];
            // step from one row to the next in weights matrix
            int m = weights.mod();

            for( int j=0; j< weights.length() ; j++, w+=m )
            {
                // *w = weights(j,i)
                somme -= softplus( -(*w + out_act[j]));
            }
            activations[i-start] = somme;
        }
    }
    else
        inherited::computeUnitActivations( start, length, activations );
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::RBMJointLLParameters::declareOptions ( OptionList ol) [static, protected]

Declares the class options.

Reimplemented from PLearn::RBMLLParameters.

Definition at line 70 of file RBMJointLLParameters.cc.

References PLearn::OptionBase::buildoption, cond_params, PLearn::declareOption(), PLearn::RBMLLParameters::declareOptions(), and target_params.

{
    // ### Declare all of this object's options here.
    // ### For the "flags" of each option, you should typically specify
    // ### one of OptionBase::buildoption, OptionBase::learntoption or
    // ### OptionBase::tuningoption. If you don't provide one of these three,
    // ### this option will be ignored when loading values from a script.
    // ### You can also combine flags, for example with OptionBase::nosave:
    // ### (OptionBase::buildoption | OptionBase::nosave)

    declareOption(ol, "target_params",
                  &RBMJointLLParameters::target_params,
                  OptionBase::buildoption,
                  "RBMParameters between the target and the upper layer");

    declareOption(ol, "cond_params", &RBMJointLLParameters::cond_params,
                  OptionBase::buildoption,
                  "RBMParameters between the conditioning input and the upper"
                  " layer");

    // Now call the parent class' declareOptions
    inherited::declareOptions(ol);
}

Here is the call graph for this function:

static const PPath& PLearn::RBMJointLLParameters::declaringFile ( ) [inline, static]

Reimplemented from PLearn::RBMLLParameters.

Definition at line 135 of file RBMJointLLParameters.h.

:
    //#####  Learned Options  #################################################
RBMJointLLParameters * PLearn::RBMJointLLParameters::deepCopy ( CopiesMap copies) const [virtual]

Reimplemented from PLearn::RBMLLParameters.

Definition at line 50 of file RBMJointLLParameters.cc.

void PLearn::RBMJointLLParameters::forget ( ) [virtual]

reset the parameters to the state they would be BEFORE starting training.

Note that this method is necessarily called from build().

Reimplemented from PLearn::RBMLLParameters.

Definition at line 299 of file RBMJointLLParameters.cc.

References PLearn::RBMLLParameters::clearStats(), cond_params, and target_params.

{
    if( target_params )
        target_params->forget();

    if( cond_params )
        cond_params->forget();

    clearStats();
}

Here is the call graph for this function:

void PLearn::RBMJointLLParameters::fprop ( const Vec input,
Vec output 
) const [virtual]

given the input, compute the output (possibly resize it appropriately)

Reimplemented from PLearn::RBMParameters.

Definition at line 213 of file RBMJointLLParameters.cc.

References computeUnitActivations(), PLearn::OnlineLearningModule::output_size, PLearn::TVec< T >::resize(), and setAsCondInput().

{
    // sets "input" as conditioning input, and compute "output"
    setAsCondInput( input );
    output.resize( output_size );
    computeUnitActivations( 0, output_size, output );
}

Here is the call graph for this function:

OptionList & PLearn::RBMJointLLParameters::getOptionList ( ) const [virtual]

Reimplemented from PLearn::RBMLLParameters.

Definition at line 50 of file RBMJointLLParameters.cc.

OptionMap & PLearn::RBMJointLLParameters::getOptionMap ( ) const [virtual]

Reimplemented from PLearn::RBMLLParameters.

Definition at line 50 of file RBMJointLLParameters.cc.

RemoteMethodMap & PLearn::RBMJointLLParameters::getRemoteMethodMap ( ) const [virtual]

Reimplemented from PLearn::RBMLLParameters.

Definition at line 50 of file RBMJointLLParameters.cc.

void PLearn::RBMJointLLParameters::makeDeepCopyFromShallowCopy ( CopiesMap copies) [virtual]

Transforms a shallow copy into a deep copy.

Reimplemented from PLearn::RBMLLParameters.

Definition at line 185 of file RBMJointLLParameters.cc.

References cond_params, PLearn::deepCopyField(), PLearn::RBMLLParameters::makeDeepCopyFromShallowCopy(), and target_params.

Here is the call graph for this function:

void PLearn::RBMJointLLParameters::setAsCondInput ( const Vec input) const [virtual]

Sets input_vec to input, and target_given_cond to true.

Definition at line 205 of file RBMJointLLParameters.cc.

References cond_size, PLearn::RBMParameters::going_up, PLearn::RBMParameters::input_vec, PLASSERT, PLearn::TVec< T >::size(), and target_given_cond.

Referenced by fprop().

{
    PLASSERT( input.size() == cond_size );
    input_vec = input;
    target_given_cond = true;
    going_up = false;
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::RBMJointLLParameters::setAsDownInput ( const Vec input) const [virtual]

Sets input_vec to input, target_given_cond to false, going_up to true.

Reimplemented from PLearn::RBMParameters.

Definition at line 199 of file RBMJointLLParameters.cc.

References PLearn::RBMParameters::setAsDownInput(), and target_given_cond.

Here is the call graph for this function:

void PLearn::RBMJointLLParameters::setAsUpInput ( const Vec input) const [virtual]

Sets input_vec to input, target_given_cond and going_up to false.

Reimplemented from PLearn::RBMParameters.

Definition at line 193 of file RBMJointLLParameters.cc.

References PLearn::RBMParameters::setAsUpInput(), and target_given_cond.

Here is the call graph for this function:


Member Data Documentation

Reimplemented from PLearn::RBMLLParameters.

Definition at line 135 of file RBMJointLLParameters.h.

RBMParameters between the conditioning input and the upper layer.

Definition at line 67 of file RBMJointLLParameters.h.

Referenced by build_(), build_units_types(), declareOptions(), forget(), and makeDeepCopyFromShallowCopy().

size of the conditioning part of down layer

Definition at line 158 of file RBMJointLLParameters.h.

Referenced by bpropUpdate(), build_(), build_units_types(), and setAsCondInput().

stores output activations

Definition at line 161 of file RBMJointLLParameters.h.

Referenced by bpropUpdate(), and build_units_types().

If true, we do not compute the up activations from a down example nor down activations from up example, but we compute the value of the "target" part of the down layer from the "conditioning" part, by summing over all the possible values of the up layer.

Definition at line 152 of file RBMJointLLParameters.h.

Referenced by setAsCondInput(), setAsDownInput(), and setAsUpInput().

RBMParameters between the target and the upper layer.

Definition at line 64 of file RBMJointLLParameters.h.

Referenced by build_(), build_units_types(), declareOptions(), forget(), and makeDeepCopyFromShallowCopy().

size of the target part of down layer

Definition at line 155 of file RBMJointLLParameters.h.

Referenced by bpropUpdate(), build_(), and build_units_types().


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines