PLearn 0.1
|
Stores and learns the parameters between two layers of an RBM. More...
#include <RBMJointGenericParameters.h>
Public Member Functions | |
RBMJointGenericParameters (real the_learning_rate=0) | |
Default constructor. | |
RBMJointGenericParameters (PP< RBMGenericParameters > &the_target_params, PP< RBMGenericParameters > &the_cond_params, real the_learning_rate=0) | |
Constructor from two string prototymes. | |
virtual void | setAsUpInput (const Vec &input) const |
Sets input_vec to input, target_given_cond and going_up to false. | |
virtual void | setAsDownInput (const Vec &input) const |
Sets input_vec to input, target_given_cond to false, going_up to true. | |
virtual void | setAsCondInput (const Vec &input) const |
Sets input_vec to input, and target_given_cond to true. | |
virtual void | computeUnitActivations (int start, int length, const Vec &activations) const |
Computes the vectors of activation of "length" units, starting from "start", and concatenates them into "activations". | |
virtual void | fprop (const Vec &input, Vec &output) const |
given the input, compute the output (possibly resize it appropriately) | |
virtual void | bpropUpdate (const Vec &input, const Vec &output, Vec &input_gradient, const Vec &output_gradient) |
Adapt based on the output gradient: this method should only be called just after a corresponding fprop; it should be called with the same arguments as fprop for the first two arguments (and output should not have been modified since then). | |
virtual void | forget () |
reset the parameters to the state they would be BEFORE starting training. | |
virtual string | classname () const |
virtual OptionList & | getOptionList () const |
virtual OptionMap & | getOptionMap () const |
virtual RemoteMethodMap & | getRemoteMethodMap () const |
virtual RBMJointGenericParameters * | deepCopy (CopiesMap &copies) const |
virtual void | build () |
Post-constructor. | |
virtual void | makeDeepCopyFromShallowCopy (CopiesMap &copies) |
Transforms a shallow copy into a deep copy. | |
Static Public Member Functions | |
static string | _classname_ () |
optionally perform some processing after training, or after a series of fprop/bpropUpdate calls to prepare the model for truly out-of-sample operation. | |
static OptionList & | _getOptionList_ () |
static RemoteMethodMap & | _getRemoteMethodMap_ () |
static Object * | _new_instance_for_typemap_ () |
static bool | _isa_ (const Object *o) |
static void | _static_initialize_ () |
static const PPath & | declaringFile () |
Public Attributes | |
PP< RBMGenericParameters > | target_params |
RBMParameters between the target and the upper layer. | |
PP< RBMGenericParameters > | cond_params |
RBMParameters between the conditioning input and the upper layer. | |
Static Public Attributes | |
static StaticInitializer | _static_initializer_ |
Protected Member Functions | |
virtual void | computeLinearUnitActivations (int i, const Vec &activations) const |
Computes the activations vector of unit "i", assuming it is linear "i" indexes a unit in "target" if "target_given_cond", else an up unit if "going_up", else a down unit. | |
virtual void | computeQuadraticUnitActivations (int i, const Vec &activations) const |
Computes the activations vector of unit "i", assuming it is quadratic "i" indexes a unit in "target" if "target_given_cond", else an up unit if "going_up", else a down unit. | |
Static Protected Member Functions | |
static void | declareOptions (OptionList &ol) |
Declares the class options. | |
Protected Attributes | |
bool | target_given_cond |
If true, we do not compute the up activations from a down example nor down activations from up example, but we compute the value of the "target" part of the down layer from the "conditioning" part, by summing over all the possible values of the up layer. | |
int | target_size |
size of the target part of down layer | |
int | cond_size |
size of the conditioning part of down layer | |
Vec | out_act |
stores output activations | |
Private Types | |
typedef RBMGenericParameters | inherited |
Private Member Functions | |
void | build_units_types () |
Builds up_units_types and down_units_types from the embedded RBMParameters. | |
void | build_ () |
This does the actual building. |
Stores and learns the parameters between two layers of an RBM.
Definition at line 56 of file RBMJointGenericParameters.h.
typedef RBMGenericParameters PLearn::RBMJointGenericParameters::inherited [private] |
Reimplemented from PLearn::RBMGenericParameters.
Definition at line 58 of file RBMJointGenericParameters.h.
PLearn::RBMJointGenericParameters::RBMJointGenericParameters | ( | real | the_learning_rate = 0 | ) |
Default constructor.
Definition at line 53 of file RBMJointGenericParameters.cc.
: inherited(the_learning_rate) { }
PLearn::RBMJointGenericParameters::RBMJointGenericParameters | ( | PP< RBMGenericParameters > & | the_target_params, |
PP< RBMGenericParameters > & | the_cond_params, | ||
real | the_learning_rate = 0 |
||
) |
Constructor from two string prototymes.
Definition at line 58 of file RBMJointGenericParameters.cc.
References build().
: inherited( the_learning_rate ), target_params( the_target_params ), cond_params( the_cond_params ) { // We're not sure inherited::build() has been called build(); }
string PLearn::RBMJointGenericParameters::_classname_ | ( | ) | [static] |
optionally perform some processing after training, or after a series of fprop/bpropUpdate calls to prepare the model for truly out-of-sample operation.
THE DEFAULT IMPLEMENTATION PROVIDED IN THE SUPER-CLASS DOES NOT DO ANYTHING.
Reimplemented from PLearn::RBMGenericParameters.
Definition at line 51 of file RBMJointGenericParameters.cc.
OptionList & PLearn::RBMJointGenericParameters::_getOptionList_ | ( | ) | [static] |
Reimplemented from PLearn::RBMGenericParameters.
Definition at line 51 of file RBMJointGenericParameters.cc.
RemoteMethodMap & PLearn::RBMJointGenericParameters::_getRemoteMethodMap_ | ( | ) | [static] |
Reimplemented from PLearn::RBMGenericParameters.
Definition at line 51 of file RBMJointGenericParameters.cc.
Reimplemented from PLearn::RBMGenericParameters.
Definition at line 51 of file RBMJointGenericParameters.cc.
Object * PLearn::RBMJointGenericParameters::_new_instance_for_typemap_ | ( | ) | [static] |
Reimplemented from PLearn::RBMGenericParameters.
Definition at line 51 of file RBMJointGenericParameters.cc.
StaticInitializer RBMJointGenericParameters::_static_initializer_ & PLearn::RBMJointGenericParameters::_static_initialize_ | ( | ) | [static] |
Reimplemented from PLearn::RBMGenericParameters.
Definition at line 51 of file RBMJointGenericParameters.cc.
void PLearn::RBMJointGenericParameters::bpropUpdate | ( | const Vec & | input, |
const Vec & | output, | ||
Vec & | input_gradient, | ||
const Vec & | output_gradient | ||
) | [virtual] |
Adapt based on the output gradient: this method should only be called just after a corresponding fprop; it should be called with the same arguments as fprop for the first two arguments (and output should not have been modified since then).
this version allows to obtain the input gradient as well
Since sub-classes are supposed to learn ONLINE, the object is 'ready-to-be-used' just after any bpropUpdate. N.B. A DEFAULT IMPLEMENTATION IS PROVIDED IN THE SUPER-CLASS, WHICH JUST CALLS bpropUpdate(input, output, input_gradient, output_gradient) AND IGNORES INPUT GRADIENT. this version allows to obtain the input gradient as well N.B. THE DEFAULT IMPLEMENTATION IN SUPER-CLASS JUST RAISES A PLERROR.
Reimplemented from PLearn::RBMGenericParameters.
Definition at line 309 of file RBMJointGenericParameters.cc.
References PLERROR.
{ PLERROR( "RBMJointGenericParameters::bpropUpdate() not implemented yet.\n" "If you only have linear units on up and down layer, you should\n" "consider using RBMJointLLParameters instead.\n" ); }
void PLearn::RBMJointGenericParameters::build | ( | ) | [virtual] |
Post-constructor.
The normal implementation should call simply inherited::build(), then this class's build_(). This method should be callable again at later times, after modifying some option fields to change the "architecture" of the object.
Reimplemented from PLearn::RBMGenericParameters.
Definition at line 183 of file RBMJointGenericParameters.cc.
References PLearn::RBMGenericParameters::build(), build_(), and build_units_types().
Referenced by RBMJointGenericParameters().
{ // Begin by this, else inherited::build() will not work properly build_units_types(); inherited::build(); build_(); }
void PLearn::RBMJointGenericParameters::build_ | ( | ) | [private] |
This does the actual building.
Reimplemented from PLearn::RBMGenericParameters.
Definition at line 128 of file RBMJointGenericParameters.cc.
References cond_params, cond_size, PLearn::RBMGenericParameters::down_units_params, PLearn::RBMGenericParameters::down_units_params_neg_stats, PLearn::RBMGenericParameters::down_units_params_pos_stats, PLearn::OnlineLearningModule::input_size, PLearn::merge(), PLearn::OnlineLearningModule::output_size, PLearn::TMat< T >::subMatColumns(), PLearn::TVec< T >::subVec(), target_params, target_size, PLearn::RBMGenericParameters::up_units_params, PLearn::RBMGenericParameters::up_units_params_neg_stats, PLearn::RBMGenericParameters::up_units_params_pos_stats, PLearn::RBMGenericParameters::weights, PLearn::RBMGenericParameters::weights_neg_stats, and PLearn::RBMGenericParameters::weights_pos_stats.
Referenced by build().
{ // The first part of weights correspond to the weights between the target // part and the up layer weights.subMatColumns( 0, target_size ) << target_params->weights; target_params->weights = weights.subMatColumns( 0, target_size ); // Second part correspond to the weights between the cond and up layer weights.subMatColumns( target_size, cond_size ) << cond_params->weights; cond_params->weights = weights.subMatColumns( target_size, cond_size ); // same thing for the statistics target_params->weights_pos_stats = weights_pos_stats.subMatColumns( 0, target_size ); cond_params->weights_pos_stats = weights_pos_stats.subMatColumns( target_size, cond_size ); target_params->weights_neg_stats = weights_neg_stats.subMatColumns( 0, target_size ); cond_params->weights_neg_stats = weights_neg_stats.subMatColumns( target_size, cond_size ); // Same thing for down units parameters down_units_params = merge( target_params->down_units_params, cond_params->down_units_params ); target_params->down_units_params_pos_stats = down_units_params_pos_stats.subVec( 0, target_size ); cond_params->down_units_params_pos_stats = down_units_params_pos_stats.subVec( target_size, cond_size ); target_params->down_units_params_neg_stats = down_units_params_neg_stats.subVec( 0, target_size ); cond_params->down_units_params_neg_stats = down_units_params_neg_stats.subVec( target_size, cond_size ); // The up layer units parameters are shared between the three RBMParameters up_units_params = cond_params->up_units_params; target_params->up_units_params = up_units_params; target_params->up_units_params_pos_stats = up_units_params_pos_stats; cond_params->up_units_params_pos_stats = up_units_params_pos_stats; target_params->up_units_params_neg_stats = up_units_params_neg_stats; cond_params->up_units_params_neg_stats = up_units_params_neg_stats; // sizes for fprop() and all OnlineLearningModules methods input_size = cond_size; output_size = target_size; }
void PLearn::RBMJointGenericParameters::build_units_types | ( | ) | [private] |
Builds up_units_types and down_units_types from the embedded RBMParameters.
Definition at line 95 of file RBMJointGenericParameters.cc.
References cond_params, cond_size, PLearn::RBMParameters::down_units_types, i, out_act, PLERROR, PLearn::TMat< T >::resize(), PLearn::TVec< T >::resize(), target_params, target_size, PLearn::RBMParameters::up_units_types, and PLearn::RBMGenericParameters::weights.
Referenced by build().
{ if( !target_params || !cond_params ) return; if( target_params->up_units_types != cond_params->up_units_types ) PLERROR( "RBMJointGenericParameters::build_units_types - \n" "target_params->up_units_types should be equal to" " cond_params->up_units_types\n" "(\"%s\" != \"%s\").\n", target_params->up_units_types.c_str(), cond_params->up_units_types.c_str() ); up_units_types = cond_params->up_units_types; down_units_types = target_params->down_units_types; target_size = target_params->down_layer_size; cond_size = cond_params->down_layer_size; for( int i=0 ; i<target_size ; i++ ) if( down_units_types[i] != 'l' ) PLERROR( "RBMJointGenericParameters::build_units_types - \n" "target_params->down_units_types[%d] should be 'l', is" " '%c'.\n", i, down_units_types[i] ); down_units_types += cond_params->down_units_types; // to avoid "forget()" being called in RBMParameters::build_() weights.resize( int(up_units_types.length()), int(down_units_types.length()) ); out_act.resize( int(up_units_types.length()) ); }
string PLearn::RBMJointGenericParameters::classname | ( | ) | const [virtual] |
Reimplemented from PLearn::RBMGenericParameters.
Definition at line 51 of file RBMJointGenericParameters.cc.
void PLearn::RBMJointGenericParameters::computeLinearUnitActivations | ( | int | i, |
const Vec & | activations | ||
) | const [protected, virtual] |
Computes the activations vector of unit "i", assuming it is linear "i" indexes a unit in "target" if "target_given_cond", else an up unit if "going_up", else a down unit.
Reimplemented from PLearn::RBMGenericParameters.
Definition at line 222 of file RBMJointGenericParameters.cc.
References i, j, PLearn::TVec< T >::length(), m, PLearn::matRowDotVec(), PLASSERT, PLearn::softplus(), PLearn::TMat< T >::subMatColumns(), and w.
{ if( target_given_cond ) { PLASSERT( activations.length() == 1 ); Mat V = weights.subMatColumns(target_size, cond_size); if (i==0) for (int j=0;j<weights.length(); j++) out_act[j] = up_units_params[j][0] + matRowDotVec(V, j, input_vec); // actY_i = B_i - sum_j softplus(-(W_ij + C_j + sum_k V_jk p(P_k))) real somme = down_units_params[i][0]; real* w = &weights[0][i]; // step from one row to the next in weights matrix int m = weights.mod(); for( int j=0; j< weights.length() ; j++, w+=m ) { // *w = weights(j,i) somme -= softplus( -(*w + out_act[j])); } activations[0] = somme; } else inherited::computeLinearUnitActivations(i, activations); }
void PLearn::RBMJointGenericParameters::computeQuadraticUnitActivations | ( | int | i, |
const Vec & | activations | ||
) | const [protected, virtual] |
Computes the activations vector of unit "i", assuming it is quadratic "i" indexes a unit in "target" if "target_given_cond", else an up unit if "going_up", else a down unit.
Reimplemented from PLearn::RBMGenericParameters.
Definition at line 261 of file RBMJointGenericParameters.cc.
References PLERROR.
{ if( target_given_cond ) PLERROR( "RBMJointGenericParameters::computeQuadraticUnitActivations" " -\n" "Cannot compute the activation of a quadratic unit in mode " "\"target_given_cond\".\n" ); else inherited::computeQuadraticUnitActivations(i, activations); }
void PLearn::RBMJointGenericParameters::computeUnitActivations | ( | int | start, |
int | length, | ||
const Vec & | activations | ||
) | const [virtual] |
Computes the vectors of activation of "length" units, starting from "start", and concatenates them into "activations".
"start" indexes an up unit if "going_up", else a down unit.
Reimplemented from PLearn::RBMGenericParameters.
Definition at line 274 of file RBMJointGenericParameters.cc.
References i, PLASSERT, PLERROR, and PLearn::TVec< T >::subVec().
Referenced by fprop().
{ string units_types; if( target_given_cond ) units_types = down_units_types.substr(0, target_size); else if( going_up ) units_types = up_units_types; else units_types = down_units_types; PLASSERT( start+length <= (int) units_types.length() ); int cur_pos = 0; // position index inside activations for( int i=start ; i<start+length ; i++ ) { char ut_i = units_types[i]; if( ut_i == 'l' ) { computeLinearUnitActivations( i, activations.subVec(cur_pos, 1) ); cur_pos++; } else if( ut_i == 'q' ) { computeQuadraticUnitActivations ( i, activations.subVec(cur_pos, 2) ); cur_pos += 2; } else PLERROR( "RBMJointGenericParameters::computeUnitActivations():\n" "value '%c' for units_types[%d] is unknown.\n" "Supported values are 'l' and 'q'.\n", ut_i, i ); } }
void PLearn::RBMJointGenericParameters::declareOptions | ( | OptionList & | ol | ) | [static, protected] |
Declares the class options.
Reimplemented from PLearn::RBMGenericParameters.
Definition at line 71 of file RBMJointGenericParameters.cc.
References PLearn::OptionBase::buildoption, cond_params, PLearn::declareOption(), PLearn::RBMGenericParameters::declareOptions(), and target_params.
{ // ### Declare all of this object's options here. // ### For the "flags" of each option, you should typically specify // ### one of OptionBase::buildoption, OptionBase::learntoption or // ### OptionBase::tuningoption. If you don't provide one of these three, // ### this option will be ignored when loading values from a script. // ### You can also combine flags, for example with OptionBase::nosave: // ### (OptionBase::buildoption | OptionBase::nosave) declareOption(ol, "target_params", &RBMJointGenericParameters::target_params, OptionBase::buildoption, "RBMParameters between the target and the upper layer"); declareOption(ol, "cond_params", &RBMJointGenericParameters::cond_params, OptionBase::buildoption, "RBMParameters between the conditioning input and the upper" " layer"); // Now call the parent class' declareOptions inherited::declareOptions(ol); }
static const PPath& PLearn::RBMJointGenericParameters::declaringFile | ( | ) | [inline, static] |
Reimplemented from PLearn::RBMGenericParameters.
Definition at line 134 of file RBMJointGenericParameters.h.
:
//##### Learned Options #################################################
RBMJointGenericParameters * PLearn::RBMJointGenericParameters::deepCopy | ( | CopiesMap & | copies | ) | const [virtual] |
Reimplemented from PLearn::RBMGenericParameters.
Definition at line 51 of file RBMJointGenericParameters.cc.
void PLearn::RBMJointGenericParameters::forget | ( | ) | [virtual] |
reset the parameters to the state they would be BEFORE starting training.
Note that this method is necessarily called from build().
Reimplemented from PLearn::RBMGenericParameters.
Definition at line 321 of file RBMJointGenericParameters.cc.
References PLearn::RBMGenericParameters::clearStats(), cond_params, and target_params.
{ if( target_params ) target_params->forget(); if( cond_params ) cond_params->forget(); clearStats(); }
given the input, compute the output (possibly resize it appropriately)
Reimplemented from PLearn::RBMParameters.
Definition at line 250 of file RBMJointGenericParameters.cc.
References computeUnitActivations(), PLearn::OnlineLearningModule::output_size, PLearn::TVec< T >::resize(), and setAsCondInput().
{ // sets "input" as conditioning input, and compute "output" setAsCondInput( input ); output.resize( output_size ); computeUnitActivations( 0, output_size, output ); }
OptionList & PLearn::RBMJointGenericParameters::getOptionList | ( | ) | const [virtual] |
Reimplemented from PLearn::RBMGenericParameters.
Definition at line 51 of file RBMJointGenericParameters.cc.
OptionMap & PLearn::RBMJointGenericParameters::getOptionMap | ( | ) | const [virtual] |
Reimplemented from PLearn::RBMGenericParameters.
Definition at line 51 of file RBMJointGenericParameters.cc.
RemoteMethodMap & PLearn::RBMJointGenericParameters::getRemoteMethodMap | ( | ) | const [virtual] |
Reimplemented from PLearn::RBMGenericParameters.
Definition at line 51 of file RBMJointGenericParameters.cc.
void PLearn::RBMJointGenericParameters::makeDeepCopyFromShallowCopy | ( | CopiesMap & | copies | ) | [virtual] |
Transforms a shallow copy into a deep copy.
Reimplemented from PLearn::RBMGenericParameters.
Definition at line 193 of file RBMJointGenericParameters.cc.
References cond_params, PLearn::deepCopyField(), PLearn::RBMGenericParameters::makeDeepCopyFromShallowCopy(), and target_params.
{ inherited::makeDeepCopyFromShallowCopy(copies); deepCopyField(target_params, copies); deepCopyField(cond_params, copies); }
void PLearn::RBMJointGenericParameters::setAsCondInput | ( | const Vec & | input | ) | const [virtual] |
Sets input_vec to input, and target_given_cond to true.
Definition at line 213 of file RBMJointGenericParameters.cc.
References cond_size, PLearn::RBMParameters::going_up, PLearn::RBMParameters::input_vec, PLASSERT, PLearn::TVec< T >::size(), and target_given_cond.
Referenced by fprop().
{ PLASSERT( input.size() == cond_size ); input_vec = input; target_given_cond = true; going_up = false; }
void PLearn::RBMJointGenericParameters::setAsDownInput | ( | const Vec & | input | ) | const [virtual] |
Sets input_vec to input, target_given_cond to false, going_up to true.
Reimplemented from PLearn::RBMParameters.
Definition at line 207 of file RBMJointGenericParameters.cc.
References PLearn::RBMParameters::setAsDownInput(), and target_given_cond.
{ target_given_cond = false; inherited::setAsDownInput( input ); }
void PLearn::RBMJointGenericParameters::setAsUpInput | ( | const Vec & | input | ) | const [virtual] |
Sets input_vec to input, target_given_cond and going_up to false.
Reimplemented from PLearn::RBMParameters.
Definition at line 201 of file RBMJointGenericParameters.cc.
References PLearn::RBMParameters::setAsUpInput(), and target_given_cond.
{ target_given_cond = false; inherited::setAsUpInput( input ); }
Reimplemented from PLearn::RBMGenericParameters.
Definition at line 134 of file RBMJointGenericParameters.h.
RBMParameters between the conditioning input and the upper layer.
Definition at line 67 of file RBMJointGenericParameters.h.
Referenced by build_(), build_units_types(), declareOptions(), forget(), and makeDeepCopyFromShallowCopy().
int PLearn::RBMJointGenericParameters::cond_size [protected] |
size of the conditioning part of down layer
Definition at line 157 of file RBMJointGenericParameters.h.
Referenced by build_(), build_units_types(), and setAsCondInput().
Vec PLearn::RBMJointGenericParameters::out_act [mutable, protected] |
stores output activations
Definition at line 160 of file RBMJointGenericParameters.h.
Referenced by build_units_types().
bool PLearn::RBMJointGenericParameters::target_given_cond [mutable, protected] |
If true, we do not compute the up activations from a down example nor down activations from up example, but we compute the value of the "target" part of the down layer from the "conditioning" part, by summing over all the possible values of the up layer.
Definition at line 151 of file RBMJointGenericParameters.h.
Referenced by setAsCondInput(), setAsDownInput(), and setAsUpInput().
RBMParameters between the target and the upper layer.
Definition at line 64 of file RBMJointGenericParameters.h.
Referenced by build_(), build_units_types(), declareOptions(), forget(), and makeDeepCopyFromShallowCopy().
int PLearn::RBMJointGenericParameters::target_size [protected] |
size of the target part of down layer
Definition at line 154 of file RBMJointGenericParameters.h.
Referenced by build_(), and build_units_types().