PLearn 0.1
|
This module outputs a linear combination of input ports. More...
#include <LinearCombinationModule.h>
Public Member Functions | |
LinearCombinationModule () | |
Default constructor. | |
void | fprop (const TVec< Mat * > &ports_value) |
Perform a fprop step. | |
virtual void | bpropAccUpdate (const TVec< Mat * > &ports_value, const TVec< Mat * > &ports_gradient) |
Perform a back propagation step (also updating parameters according to the provided gradient). | |
virtual void | forget () |
Reset the parameters to the state they would be BEFORE starting training. | |
virtual const TVec< string > & | getPorts () |
Return the list of ports in the module. | |
virtual string | classname () const |
virtual OptionList & | getOptionList () const |
virtual OptionMap & | getOptionMap () const |
virtual RemoteMethodMap & | getRemoteMethodMap () const |
virtual LinearCombinationModule * | deepCopy (CopiesMap &copies) const |
virtual void | build () |
Post-constructor. | |
virtual void | makeDeepCopyFromShallowCopy (CopiesMap &copies) |
Transforms a shallow copy into a deep copy. | |
Static Public Member Functions | |
static string | _classname_ () |
static OptionList & | _getOptionList_ () |
static RemoteMethodMap & | _getRemoteMethodMap_ () |
static Object * | _new_instance_for_typemap_ () |
static bool | _isa_ (const Object *o) |
static void | _static_initialize_ () |
static const PPath & | declaringFile () |
Public Attributes | |
Vec | weights |
### declare public option fields (such as build options) here | |
bool | adaptive |
whether to adapt the weights, and whether to clear them to 0 upon forget() | |
real | learning_rate |
learning rate if adapting the weights | |
TVec< string > | port_names |
all the input ports, followed by the output port | |
Static Public Attributes | |
static StaticInitializer | _static_initializer_ |
Static Protected Member Functions | |
static void | declareOptions (OptionList &ol) |
Declares the class options. | |
Private Types | |
typedef OnlineLearningModule | inherited |
Private Member Functions | |
void | build_ () |
This does the actual building. |
This module outputs a linear combination of input ports.
All the ports should have the same dimensions. The weights of the linear combination could either be learned or user-defined.
Definition at line 52 of file LinearCombinationModule.h.
typedef OnlineLearningModule PLearn::LinearCombinationModule::inherited [private] |
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 54 of file LinearCombinationModule.h.
PLearn::LinearCombinationModule::LinearCombinationModule | ( | ) |
Default constructor.
Definition at line 57 of file LinearCombinationModule.cc.
: adaptive(false), learning_rate(0) { }
string PLearn::LinearCombinationModule::_classname_ | ( | ) | [static] |
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 55 of file LinearCombinationModule.cc.
OptionList & PLearn::LinearCombinationModule::_getOptionList_ | ( | ) | [static] |
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 55 of file LinearCombinationModule.cc.
RemoteMethodMap & PLearn::LinearCombinationModule::_getRemoteMethodMap_ | ( | ) | [static] |
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 55 of file LinearCombinationModule.cc.
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 55 of file LinearCombinationModule.cc.
Object * PLearn::LinearCombinationModule::_new_instance_for_typemap_ | ( | ) | [static] |
Reimplemented from PLearn::Object.
Definition at line 55 of file LinearCombinationModule.cc.
StaticInitializer LinearCombinationModule::_static_initializer_ & PLearn::LinearCombinationModule::_static_initialize_ | ( | ) | [static] |
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 55 of file LinearCombinationModule.cc.
void PLearn::LinearCombinationModule::bpropAccUpdate | ( | const TVec< Mat * > & | ports_value, |
const TVec< Mat * > & | ports_gradient | ||
) | [virtual] |
Perform a back propagation step (also updating parameters according to the provided gradient).
The matrices in 'ports_value' must be the same as the ones given in a previous call to 'fprop' (and thus they should in particular contain the result of the fprop computation). However, they are not necessarily the same as the ones given in the LAST call to 'fprop': if there is a need to store an internal module state, this should be done using a specific port to store this state. Each Mat* pointer in the 'ports_gradient' vector can be one of:
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 167 of file LinearCombinationModule.cc.
References adaptive, PLearn::OnlineLearningModule::checkProp(), PLearn::dot(), i, PLearn::TMat< T >::isEmpty(), learning_rate, PLearn::TMat< T >::length(), PLearn::TVec< T >::length(), PLearn::multiplyAcc(), PLASSERT, weights, and PLearn::TMat< T >::width().
{ int n_ports = weights.length() + 1; PLASSERT( ports_value.length() == n_ports && ports_gradient.length() == n_ports); const TVec<Mat*>& input_grad = ports_gradient; Mat* output_grad = ports_gradient[n_ports-1]; if (output_grad && !output_grad->isEmpty()) { int mbs = output_grad->length(); int width = output_grad->width(); for (int i=0;i<n_ports-1;i++) { if (input_grad[i]) { PLASSERT(input_grad[i]->isEmpty() && input_grad[i]->width() == width); input_grad[i]->resize(mbs,width); multiplyAcc(*input_grad[i],*output_grad,weights[i]); } if (adaptive && learning_rate > 0) { Mat* input_i = ports_value[i]; PLASSERT(input_i); weights[i] -= learning_rate * dot(*output_grad,*input_i); } } } // Ensure all required gradients have been computed. checkProp(ports_gradient); }
void PLearn::LinearCombinationModule::build | ( | ) | [virtual] |
Post-constructor.
The normal implementation should call simply inherited::build(), then this class's build_(). This method should be callable again at later times, after modifying some option fields to change the "architecture" of the object.
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 113 of file LinearCombinationModule.cc.
References PLearn::OnlineLearningModule::build(), and build_().
{ inherited::build(); build_(); }
void PLearn::LinearCombinationModule::build_ | ( | ) | [private] |
This does the actual building.
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 83 of file LinearCombinationModule.cc.
References adaptive, PLearn::TVec< T >::fill(), i, learning_rate, PLearn::TVec< T >::length(), PLASSERT, PLCHECK, PLWARNING, port_names, PLearn::TVec< T >::resize(), PLearn::tostring(), and weights.
Referenced by build().
{ PLASSERT(weights.length()==0 || port_names.length()==0 || weights.length() + 1 ==port_names.length()); int n_ports=0; if (weights.length()!=0 && port_names.length()==0) // weights provided but not ports: give default port names { n_ports = weights.length() + 1; port_names.resize(n_ports); for (int i=0;i<n_ports-1;i++) port_names[i]="in_" + tostring(i+1); port_names[n_ports-1]="output"; } if (weights.length()==0 && port_names.length()!=0) // ports provided but not weights: initialize weights to 0 { n_ports = port_names.length(); weights.resize(n_ports - 1); weights.fill(0); if (!adaptive) PLWARNING("LinearCombinationModule::build: non-adaptive weights set to 0! the module will always output 0."); } PLCHECK( learning_rate >= 0 ); }
string PLearn::LinearCombinationModule::classname | ( | ) | const [virtual] |
Reimplemented from PLearn::Object.
Definition at line 55 of file LinearCombinationModule.cc.
void PLearn::LinearCombinationModule::declareOptions | ( | OptionList & | ol | ) | [static, protected] |
Declares the class options.
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 62 of file LinearCombinationModule.cc.
References adaptive, PLearn::OptionBase::buildoption, PLearn::declareOption(), PLearn::OnlineLearningModule::declareOptions(), learning_rate, and weights.
{ declareOption(ol, "weights", &LinearCombinationModule::weights, OptionBase::buildoption, "the weights of the linear combination: a vector with one element per input port\n"); declareOption(ol, "adaptive", &LinearCombinationModule::adaptive, OptionBase::buildoption, "whether to adapt the weights, if true they are cleared upon initialization (forget()).\n"); declareOption(ol, "learning_rate", &LinearCombinationModule::learning_rate, OptionBase::buildoption, "Learning rate to adapt the weights by online gradient descent.\n"); // Now call the parent class' declareOptions inherited::declareOptions(ol); }
static const PPath& PLearn::LinearCombinationModule::declaringFile | ( | ) | [inline, static] |
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 251 of file LinearCombinationModule.h.
:
//##### Protected Member Functions ######################################
LinearCombinationModule * PLearn::LinearCombinationModule::deepCopy | ( | CopiesMap & | copies | ) | const [virtual] |
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 55 of file LinearCombinationModule.cc.
void PLearn::LinearCombinationModule::forget | ( | ) | [virtual] |
Reset the parameters to the state they would be BEFORE starting training.
Note that this method is necessarily called from build().
Implements PLearn::OnlineLearningModule.
Definition at line 204 of file LinearCombinationModule.cc.
References adaptive, PLearn::TVec< T >::clear(), and weights.
Perform a fprop step.
Each Mat* pointer in the 'ports_value' vector can be one of:
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 134 of file LinearCombinationModule.cc.
References PLearn::OnlineLearningModule::checkProp(), PLearn::TMat< T >::clear(), i, PLearn::TMat< T >::isEmpty(), PLearn::TVec< T >::length(), PLearn::multiplyAcc(), PLASSERT, PLERROR, PLearn::TMat< T >::resize(), and weights.
{ int n_ports = weights.length() + 1; if ( n_ports < 2 ) // has build completed? there should be at least one input port + the output port PLERROR("LinearCombinationModule should have at least 2 ports (one input port and one output port)\n"); PLASSERT( ports_value.length() == n_ports ); // is the input coherent with expected nPorts const TVec<Mat*>& inputs = ports_value; Mat* output = ports_value[n_ports-1]; if (output) { PLASSERT( output->isEmpty() ); PLASSERT( inputs[0] ); int mbs = inputs[0]->length(); int width = inputs[0]->width(); output->resize(mbs, width); output->clear(); for (int i=0;i<n_ports-1;i++) { Mat* input_i = inputs[i]; if (!input_i || input_i->isEmpty()) PLERROR("In LinearCombinationModule::fprop - The %d-th input " "port is missing or empty", i); multiplyAcc(*output, *input_i, weights[i]); } } // Ensure all required ports have been computed. checkProp(ports_value); }
OptionList & PLearn::LinearCombinationModule::getOptionList | ( | ) | const [virtual] |
Reimplemented from PLearn::Object.
Definition at line 55 of file LinearCombinationModule.cc.
OptionMap & PLearn::LinearCombinationModule::getOptionMap | ( | ) | const [virtual] |
Reimplemented from PLearn::Object.
Definition at line 55 of file LinearCombinationModule.cc.
const TVec< string > & PLearn::LinearCombinationModule::getPorts | ( | ) | [virtual] |
Return the list of ports in the module.
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 242 of file LinearCombinationModule.cc.
References port_names.
{ return port_names; }
RemoteMethodMap & PLearn::LinearCombinationModule::getRemoteMethodMap | ( | ) | const [virtual] |
Reimplemented from PLearn::Object.
Definition at line 55 of file LinearCombinationModule.cc.
void PLearn::LinearCombinationModule::makeDeepCopyFromShallowCopy | ( | CopiesMap & | copies | ) | [virtual] |
Transforms a shallow copy into a deep copy.
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 123 of file LinearCombinationModule.cc.
References PLearn::deepCopyField(), PLearn::OnlineLearningModule::makeDeepCopyFromShallowCopy(), port_names, and weights.
{ inherited::makeDeepCopyFromShallowCopy(copies); deepCopyField(weights, copies); deepCopyField(port_names, copies); }
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 251 of file LinearCombinationModule.h.
whether to adapt the weights, and whether to clear them to 0 upon forget()
Definition at line 64 of file LinearCombinationModule.h.
Referenced by bpropAccUpdate(), build_(), declareOptions(), and forget().
learning rate if adapting the weights
Definition at line 66 of file LinearCombinationModule.h.
Referenced by bpropAccUpdate(), build_(), and declareOptions().
all the input ports, followed by the output port
Definition at line 69 of file LinearCombinationModule.h.
Referenced by build_(), getPorts(), and makeDeepCopyFromShallowCopy().
### declare public option fields (such as build options) here
the weights of the linear combination: one per input port
Definition at line 62 of file LinearCombinationModule.h.
Referenced by bpropAccUpdate(), build_(), declareOptions(), forget(), fprop(), and makeDeepCopyFromShallowCopy().