PLearn 0.1
|
#include <NetworkModule.h>
Public Member Functions | |
NetworkModule () | |
Default constructor. | |
virtual void | fprop (const Vec &input, Vec &output) const |
Basic fprop (not implemented). | |
virtual void | fprop (const TVec< Mat * > &ports_value) |
Compute the network's outputs by propagating each underlying module's computations through the network. | |
virtual void | bpropAccUpdate (const TVec< Mat * > &ports_value, const TVec< Mat * > &ports_gradient) |
Back-propagate gradient through the network, and update each module's internal parameters. | |
virtual const TVec< string > & | getPorts () |
Return the list of ports in the module. | |
virtual const TMat< int > & | getPortSizes () |
Return the size of each port. | |
virtual void | forget () |
Reset the parameters to the state they would be BEFORE starting training. | |
virtual string | classname () const |
virtual OptionList & | getOptionList () const |
virtual OptionMap & | getOptionMap () const |
virtual RemoteMethodMap & | getRemoteMethodMap () const |
virtual NetworkModule * | deepCopy (CopiesMap &copies) const |
virtual void | build () |
Post-constructor. | |
virtual void | makeDeepCopyFromShallowCopy (CopiesMap &copies) |
Transforms a shallow copy into a deep copy. | |
Static Public Member Functions | |
static string | _classname_ () |
static OptionList & | _getOptionList_ () |
static RemoteMethodMap & | _getRemoteMethodMap_ () |
static Object * | _new_instance_for_typemap_ () |
static bool | _isa_ (const Object *o) |
static void | _static_initialize_ () |
static const PPath & | declaringFile () |
Public Attributes | |
TVec< PP< OnlineLearningModule > > | modules |
TVec< PP< NetworkConnection > > | connections |
TVec< pair< string, string > > | ports |
bool | save_states |
Static Public Attributes | |
static StaticInitializer | _static_initializer_ |
Static Protected Member Functions | |
static void | declareOptions (OptionList &ol) |
Declares the class options. | |
Protected Attributes | |
TVec< PP< OnlineLearningModule > > | all_modules |
List of all modules, i.e. | |
TVec< PP< NetworkConnection > > | all_connections |
Similar to 'all_modules', but for connections in the network. | |
TVec< int > | fprop_path |
Ordered list of modules used when doing a fprop (the integer values correspond to indices in 'modules'). | |
TVec< int > | bprop_path |
Ordered list of modules used when doing a bprop (the integer values correspond to indices in 'modules'). | |
TVec< TVec< Mat * > > | fprop_data |
The i-th element is the list of Mat* pointers being provided to the i-th module in a fprop step. | |
TVec< TVec< int > > | fprop_data_idx |
Same as 'fprop_data', but the integer is the index of the matrix in the 'all_mats' vector. | |
TVec< TVec< int > > | fprop_toresize |
The i-th element is the list of matrices that need to be resized to empty matrices prior to calling fprop() on the i-th module in a fprop step. | |
TVec< TMat< int > > | fprop_toplug |
The i-th element is the list of correspondances between ports in this NetworkModule and ports in the i-th module of the fprop path. | |
TVec< TVec< Mat * > > | bprop_data |
The i-th element is the list of Mat* pointers being provided to the i-th module in a bprop step. | |
TVec< TVec< int > > | bprop_data_idx |
Same as 'bprop_data', but the integer is the index of the matrix in the 'all_mats' vector. | |
TVec< TVec< int > > | bprop_toresize |
The i-th element is the list of matrices that need to be resized to empty matrices prior to calling bpropUpdate() on the i-th module in a bprop step. | |
TVec< Mat > | all_mats |
A list of all matrices used to store the various computation results in the network (i.e. | |
TVec< string > | all_ports |
The list of ports, computed from the 'ports' option. | |
Private Types | |
typedef OnlineLearningModule | inherited |
Private Member Functions | |
void | build_ () |
This does the actual building. |
Definition at line 48 of file NetworkModule.h.
typedef OnlineLearningModule PLearn::NetworkModule::inherited [private] |
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 50 of file NetworkModule.h.
PLearn::NetworkModule::NetworkModule | ( | ) |
string PLearn::NetworkModule::_classname_ | ( | ) | [static] |
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 64 of file NetworkModule.cc.
OptionList & PLearn::NetworkModule::_getOptionList_ | ( | ) | [static] |
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 64 of file NetworkModule.cc.
RemoteMethodMap & PLearn::NetworkModule::_getRemoteMethodMap_ | ( | ) | [static] |
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 64 of file NetworkModule.cc.
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 64 of file NetworkModule.cc.
Object * PLearn::NetworkModule::_new_instance_for_typemap_ | ( | ) | [static] |
Reimplemented from PLearn::Object.
Definition at line 64 of file NetworkModule.cc.
StaticInitializer NetworkModule::_static_initializer_ & PLearn::NetworkModule::_static_initialize_ | ( | ) | [static] |
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 64 of file NetworkModule.cc.
void PLearn::NetworkModule::bpropAccUpdate | ( | const TVec< Mat * > & | ports_value, |
const TVec< Mat * > & | ports_gradient | ||
) | [virtual] |
Back-propagate gradient through the network, and update each module's internal parameters.
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 118 of file NetworkModule.cc.
References all_modules, bprop_data, bprop_path, bprop_toresize, PLearn::TMat< T >::clear(), PLearn::endl(), fprop_data, fprop_path, fprop_toplug, i, PLearn::TMat< T >::isEmpty(), j, PLearn::TMat< T >::length(), PLearn::TVec< T >::length(), PLASSERT, PLASSERT_MSG, PLearn::TMat< T >::resize(), and PLearn::TMat< T >::width().
{ // Reset internal gradients, either: // - to zero if no additional gradient is provided in 'ports_gradient' // - to the gradient provided in 'ports_gradient' otherwise PLASSERT( fprop_path.length() == bprop_path.length() ); for (int i = 0; i < bprop_path.length(); i++) { const TVec<int>& toresize = bprop_toresize[i]; int fprop_idx = fprop_data.length() - 1 - i; const TVec<Mat*>& f_data = fprop_data[fprop_idx]; for (int j = 0; j < toresize.length(); j++) { if (j == 0) { DBG_MODULE_LOG << "CLEAR: " << all_modules[bprop_path[i]]->classname() << endl; } int mat_idx = toresize[j]; DBG_MODULE_LOG << " grad = " << all_modules[bprop_path[i]]->getPortName(mat_idx) << endl; Mat* mat_toresize = bprop_data[i][mat_idx]; Mat* mat_tpl = f_data[mat_idx]; mat_toresize->resize(mat_tpl->length(), mat_tpl->width()); mat_toresize->clear(); } const TMat<int>& f_toplug = fprop_toplug[fprop_idx]; for (int j = 0; j < f_toplug.length(); j++) { int mod_idx = f_toplug(j, 1); Mat* current = bprop_data[i][mod_idx]; if (!current) // There is no gradient currently computed for this port, thus // nothing to initialize. continue; int this_idx = f_toplug(j, 0); Mat* provided = ports_gradient[this_idx]; if (provided) { // We are provided with some additional gradient, either as // input or output. If it is an output, there is nothing more // to do. However, if it is an input, we must use it to // initialize the gradient. if (!provided->isEmpty()) { PLASSERT( current->length() == provided->length() && current->width() == provided->width() ); PLASSERT( (*current)(0, 0) == 0 ); *current << *provided; } else { // Just for safety reason, crash here. It seems weird that // we may want to accumulate the gradient w.r.t. a port for // which a gradient is already computed internally (i.e. is // used in the bprop). If we want to allow this, care // should be taken as to what to do with the current value // of the gradient in 'provided' (in particular, do we use // it to update the internal paremeters?) PLASSERT( false ); } } } } // Backpropagate gradient and update parameters. for (int i = 0; i < bprop_path.length(); i++) { PP<OnlineLearningModule> module = all_modules[bprop_path[i]]; DBG_MODULE_LOG << "BPROP: " << module->name << endl; // First resize some gradient matrices, so that the gradient is // properly computed. const TVec<int>& toresize = bprop_toresize[i]; const TVec<Mat*> b_data = bprop_data[i]; for (int j = 0; j < toresize.length(); j++) { int mat_idx = toresize[j]; DBG_MODULE_LOG << " grad = " << module->getPortName(mat_idx) << endl; Mat* mat_toresize = b_data[mat_idx]; PLASSERT( mat_toresize->width() > 0 ); mat_toresize->resize(0, mat_toresize->width()); } // Plug in the matrices provided as parameters of this method. int fprop_idx = fprop_data.length() - 1 - i; const TVec<Mat*> f_data = fprop_data[fprop_idx]; const TMat<int>& f_toplug = fprop_toplug[fprop_idx]; for (int j = 0; j < f_toplug.length(); j++) { int this_idx = f_toplug(j, 0); int mod_idx = f_toplug(j, 1); Mat* provided = ports_gradient[this_idx]; if (provided) { Mat* current = b_data[mod_idx]; if (!current) { // We can directly plug in the provided matrix. b_data[mod_idx] = provided; } else { if (!provided->isEmpty()) { PLASSERT( !current->isEmpty() ); // This gradient is some external gradient we are // provided with. It has already been added in the step // where we reset (see above), thus there is nothing // more to do. TODO Note that it may be cleaner to // actually add it here to simplify the code above. } else { PLASSERT( current->isEmpty() ); // This gradient must be computed, but is already // computed somewhere here. We will copy it after the // update step. PLASSERT( false); // This should not happen (cf. above) } } } // Same for the fprop data since it is also given in argument. Mat* f_provided = ports_value[this_idx]; if (f_provided) { Mat* f_current = f_data[mod_idx]; if (!f_current) f_data[mod_idx] = f_provided; else { // 'f_provided' is thus the result of the computation, that // was copied from the value of 'f_current' at fprop time. // We must fill in 'f_current' with this value (once again, // this is not best for efficiency). f_current->resize(f_provided->length(), f_provided->width()); *f_current << *f_provided; } } } // Then perform the bpropUpdate step. module->bpropAccUpdate( fprop_data[fprop_data.length() - 1 - i], bprop_data[i]); // Restore the 'bprop_data' and 'fprop_data' parameters. for (int j = 0; j < f_toplug.length(); j++) { int this_idx = f_toplug(j, 0); int mod_idx = f_toplug(j, 1); Mat* provided = ports_gradient[this_idx]; if (provided) { Mat* current = b_data[mod_idx]; PLASSERT_MSG(current, "This should not happen: if we are " "provided with a matrix, it should have been plugged " "in here"); if (provided == current) // They are the same matrices: this means there used to be // a NULL matrix before. b_data[mod_idx] = NULL; else { // This should never happen. PLASSERT( false ); } } Mat* f_provided = ports_value[this_idx]; if (f_provided) { Mat* f_current = f_data[mod_idx]; if (f_current == f_provided) // This used to be a null pointer, that must be restored. f_data[mod_idx] = NULL; } } } }
void PLearn::NetworkModule::build | ( | ) | [virtual] |
Post-constructor.
The normal implementation should call simply inherited::build(), then this class's build_(). This method should be callable again at later times, after modifying some option fields to change the "architecture" of the object.
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 280 of file NetworkModule.cc.
References PLearn::OnlineLearningModule::build(), and build_().
{ inherited::build(); build_(); }
void PLearn::NetworkModule::build_ | ( | ) | [private] |
This does the actual building.
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 289 of file NetworkModule.cc.
References all_connections, all_mats, all_modules, all_ports, PLearn::TVec< T >::append(), PLearn::TMat< T >::appendRow(), bprop_data, bprop_data_idx, bprop_path, bprop_toresize, connections, PLearn::TVec< T >::fill(), PLearn::TVec< T >::first(), fprop_data, fprop_data_idx, fprop_path, fprop_toplug, fprop_toresize, i, j, PLearn::TVec< T >::lastElement(), PLearn::TVec< T >::length(), modules, PLearn::OnlineLearningModule::name, PLASSERT, PLASSERT_MSG, PLERROR, PLWARNING, PLearn::OnlineLearningModule::port_sizes, ports, PLearn::OnlineLearningModule::random_gen, PLearn::TMat< T >::resize(), PLearn::TVec< T >::resize(), save_states, PLearn::TVec< T >::size(), PLearn::test(), and PLearn::tostring().
Referenced by build().
{ // Initialize the 'all_modules' and 'all_connections' lists. all_modules.resize(modules.length()); all_modules << modules; all_connections.resize(connections.length()); all_connections << connections; if (save_states) { // Ensure all ports which are states are correctly saved. // This is done by adding connections to a NullModule in order to force // them being saved. int count_null = 0; for (int i = 0; i < modules.length(); i++) { const TVec<string>& mod_ports = modules[i]->getPorts(); for (int j = 0; j < mod_ports.length(); j++) { const string& port = mod_ports[j]; if (port.size() >= 5 && port.substr(port.size() - 5) == "state") { // Look for either an incoming or outgoing connection. string port_name = modules[i]->name + "." + port; bool has_a_connection = false; for (int k = 0; k < connections.length(); k++) if (connections[k]->destination == port_name || connections[k]->source == port_name) { has_a_connection = true; break; } if (!has_a_connection) { // Currently we create one NullModule / state, since // multiple incoming connections are not correctly // handled. PP<OnlineLearningModule> null; /* if (all_modules.length() == modules.length()) { // No null module added yet. */ null = new NullModule("_null_" + tostring(count_null++)); all_modules.append(null); /*} else { PLASSERT( all_modules.length() == modules.length() + 1 ); null = all_modules.lastElement(); }*/ all_connections.append( new NetworkConnection( modules[i], port, null, "null", false)); } } } } } // Forward random number generator to all underlying modules. if (random_gen) { for (int i = 0; i < all_modules.length(); i++) { if (!all_modules[i]->random_gen) { all_modules[i]->random_gen = random_gen; all_modules[i]->build(); all_modules[i]->forget(); } } } // Construct fprop and bprop paths from the list of modules and // connections. // First create a dictionary of modules. map<string, PP<OnlineLearningModule> > name_to_module; for (int i = 0; i < all_modules.length(); i++) { string module = all_modules[i]->name; if (name_to_module.find(module) != name_to_module.end()) { // There is already a module with the same name. For safety // reasons, we make it point to a NULL pointer to ensure we do not // accidentally use the wrong module. PLWARNING("In NetworkModule::build_ - There are two modules with " "the same name '%s': any connection attempting to use " "one of these modules will probably crash", module.c_str()); name_to_module[module] = NULL; } else name_to_module[module] = all_modules[i]; } // Initialize connections. for (int i = 0; i < all_connections.length(); i++) all_connections[i]->initialize(name_to_module); // Preprocess some convenience data structures from the list of // connections. // 'module_to_index' maps module pointers to their corresponding index in // the 'all_modules' list. map<const OnlineLearningModule*, int> module_to_index; for (int i = 0; i < all_modules.length(); i++) module_to_index[all_modules[i]] = i; // Analyze the list of ports. // The 'port_correspondances' lists, for each module, the correspondances // between the modules' ports and the ports of the NetworkModule. TVec< TMat<int> > port_correspondances(all_modules.length()); for (int i = 0; i < all_modules.length(); i++) port_correspondances[i].resize(0, 2); TVec<int> new_row(2); all_ports.resize(0); port_sizes.resize(0, 2); int n_ports = ports.length(); for (int i = 0; i < n_ports; i++) { const pair<string, string>& new_old = ports[i]; string new_name = new_old.first; const string& old_name = new_old.second; if (new_name.empty()) // Empty string: we use the full name of the form 'module.port'. new_name = old_name; all_ports.append(new_name); size_t dot_pos = old_name.find('.'); PLASSERT( dot_pos != string::npos ); string old_module_name = old_name.substr(0, dot_pos); string old_module_port = old_name.substr(dot_pos + 1); if ( name_to_module.find(old_module_name) == name_to_module.end()) PLERROR("NetworkModule::Can't find module with name %s\n",old_module_name.c_str()); new_row[0] = i; PP<OnlineLearningModule> old_module = name_to_module[old_module_name]; new_row[1] = old_module->getPortIndex(old_module_port); port_correspondances[module_to_index[old_module]].appendRow(new_row); new_row[0] = old_module->getPortLength(old_module_port); new_row[1] = old_module->getPortWidth(old_module_port); port_sizes.appendRow(new_row); } // The i-th element of 'in_connections' maps each port in the i-th module // to the connection that has it as destination (there may be only one). TVec< map<string, PP<NetworkConnection> > > in_connections; in_connections.resize(all_modules.length()); // The i-th element of 'out_connections' maps each port in the i-th module // to the connections that have it as source (there may be many). TVec< map<string, TVec< PP<NetworkConnection> > > > out_connections; out_connections.resize(all_modules.length()); // The 'inputs_needed' vector contains the number of inputs that must be // fed to a module before it can compute a fprop. TVec<int> inputs_needed(all_modules.length(), 0); // The 'compute_input_of' list gives, for each module M, the indices of // other modules that take an output of M as input. TVec< TVec<int> > compute_input_of(all_modules.length()); for (int i = 0; i < all_connections.length(); i++) { PP<NetworkConnection> connection = all_connections[i]; int src = module_to_index[connection->getSourceModule()]; int dest = module_to_index[connection->getDestinationModule()]; inputs_needed[dest]++; compute_input_of[src].append(dest); map<string, PP<NetworkConnection> >& in_conn = in_connections[dest]; if (in_conn.find(connection->getDestinationPort()) != in_conn.end()) { /* // The port has more than one incoming connection. Currently, we // only allow this to happen if the module is a NullModule (this is // for safety). If in the future we want more modules to allow // multiple incoming connections, we may get rid of this error. NullModule* test = dynamic_cast<NullModule*>( get_pointer(connection->getDestinationModule())); */ // TODO ACTUALLY this does not work, as the fprop/bprop paths may // not be correctly computed in the case of multiple incoming // connections. Until this is fixed, multiple incoming connections // are not allowed. bool test = false; if (!test) PLERROR("In LearningNetwork::build_ - A port may have only one " "incoming connection"); } in_conn[connection->getDestinationPort()] = connection; out_connections[src][connection->getSourcePort()].append(connection); } // The fprop and bprop paths can now be computed. fprop_path.resize(0); bprop_path.resize(all_modules.length()); bprop_path.fill(-1); TVec<bool> is_done(all_modules.length(), false); fprop_data.resize(0); fprop_data_idx.resize(0); bprop_data.resize(all_modules.length()); bprop_data_idx.resize(all_modules.length()); fprop_toresize.resize(0); bprop_toresize.resize(all_modules.length()); fprop_toplug.resize(0); // This is getting a little hackish here... If not enough memory is // allocated to store the work matrices, then when appending a new Mat to // 'all_mats', we will have to create a new Storage and copy previous data. // The problem is then that the pointers in 'fprop_data' and 'bprop_data' // will become invalid. So an easy fix is to allocate enough memory to // ensure we never have to resize the Storage later. int max_n_mats = 1000; all_mats.resize(max_n_mats); all_mats.resize(0); // A vector that stores the index of a module in the fprop path. TVec<int> module_index_to_path_index(all_modules.length(), -1); while (is_done.find(false) >= 0) { bool at_least_one_done = false; for (int i = 0; i < all_modules.length(); i++) { if (!is_done[i] && inputs_needed[i] == 0) { for (int j = 0; j < compute_input_of[i].length(); j++) inputs_needed[compute_input_of[i][j]]--; // Save any correspondance between ports of this module and // ports of the encapsulating NetworkModule. fprop_toplug.append(port_correspondances[i]); // Compute the list of matrices that must be provided to this // module when doing a fprop and bprop. TVec<string> mod_ports = all_modules[i]->getPorts(); map<string, PP<NetworkConnection> >& in_conn = in_connections[i]; map<string, TVec< PP<NetworkConnection> > >& out_conn = out_connections[i]; TVec<int> fprop_tores; TVec<int> bprop_tores; TVec<Mat*> fprop_mats; TVec<Mat*> bprop_mats; TVec<int> fprop_mats_idx; TVec<int> bprop_mats_idx; int n_in_conn = in_conn.size(); int n_in_conn_found = 0; for (int j = 0; j < mod_ports.length(); j++) { int count_inc_conn = in_conn.count(mod_ports[j]); if (count_inc_conn > 0) { // This port has an incoming connection: it is thus an // input, and the corresponding matrices for storing // its value and gradient are found by looking at the // source port of the connection. n_in_conn_found += count_inc_conn; PP<NetworkConnection> conn = in_conn[mod_ports[j]]; int src_mod = module_to_index[conn->getSourceModule()]; int path_index = module_index_to_path_index[src_mod]; int port_index = conn->getSourceModule()->getPortIndex( conn->getSourcePort()); if (port_index < 0) PLERROR("In NetworkModule::build_ - no port named" " \"%s\"\n" "in module \"%s\". Valid port names are:\n" "%s\n", conn->getSourcePort().c_str(), conn->getSourceModule()->name.c_str(), tostring(conn->getSourceModule() ->getPorts()).c_str() ); fprop_mats.append(fprop_data[path_index][port_index]); fprop_mats_idx.append( fprop_data_idx[path_index][port_index]); if (!conn->propagate_gradient) { // This connection does not propagate the gradient, // and thus we do not want to accumulate it. bprop_mats.append(NULL); bprop_mats_idx.append(-1); } else { int b_idx = all_modules.length() - 1 - path_index; bprop_mats.append(bprop_data[b_idx][port_index]); bprop_mats_idx.append( bprop_data_idx[b_idx][port_index]); bprop_tores.append(j); } PLASSERT_MSG( out_conn.find(mod_ports[j]) == out_conn.end(), "A port with an incoming connection cannot " "also have an outgoing connection" ); } else if (out_conn.find(mod_ports[j]) != out_conn.end()) { // This port has (at least) one outgoing connection: it // is thus an output, and it must be provided with // matrices to store its value (and gradient if the // connection propagates it). all_mats.append(Mat()); if (all_mats.length() > max_n_mats) PLERROR("In NetworkModule::build_ - Will need to " "increase 'max_n_mats'"); Mat* new_mat = &all_mats.lastElement(); fprop_mats.append(new_mat); fprop_mats_idx.append(all_mats.length() - 1); fprop_tores.append(j); // Ensure there exists a connection propagating the // gradient to this port. bool must_store_grad = false; const TVec< PP<NetworkConnection> >& out_j = out_conn[mod_ports[j]]; for (int k = 0; k < out_j.length(); k++) if (out_j[k]->propagate_gradient) { must_store_grad = true; break; } if (must_store_grad) { all_mats.append(Mat()); if (all_mats.length() > max_n_mats) PLERROR("In NetworkModule::build_ - Will need " "to increase 'max_n_mats'"); new_mat = &all_mats.lastElement(); bprop_mats.append(new_mat); bprop_mats_idx.append(all_mats.length() - 1); } else { // No connection propagating gradient to this port. bprop_mats.append(NULL); bprop_mats_idx.append(-1); } } else { // This port is not used (we do not provide its value, // and we do not care about obtaining it). fprop_mats.append(NULL); bprop_mats.append(NULL); fprop_mats_idx.append(-1); bprop_mats_idx.append(-1); } } if (n_in_conn != n_in_conn_found) PLERROR("In NetworkModule::build_ - Not all incoming " "connections were found for module '%s', please " "ensure all destination ports do exist", all_modules[i]->name.c_str()); module_index_to_path_index[i] = fprop_path.length(); // Update fprop path. PLASSERT( fprop_mats.length() == fprop_mats_idx.length() ); fprop_data.append(fprop_mats); fprop_data_idx.append(fprop_mats_idx); fprop_toresize.append(fprop_tores); fprop_path.append(i); // Update bprop path. PLASSERT( bprop_mats_idx.length() == bprop_mats.length() ); int bprop_idx = bprop_path.length() - fprop_path.length(); bprop_data[bprop_idx] = bprop_mats; bprop_data_idx[bprop_idx] = bprop_mats_idx; bprop_toresize[bprop_idx] = bprop_tores; bprop_path[bprop_idx] = i; is_done[i] = true; at_least_one_done = true; } } if (!at_least_one_done) { // Infinite loop: the algorithm cannot find a module that can be // propagated at this point. string err; for (int i = 0; i < all_modules.length(); i++) if (!is_done[i]) err += all_modules[i]->name + ", "; PLERROR("In NetworkModule::build_ - Cannot build propagation path " "due to the following modules waiting for incoming " "connections (there may be an infinite loop): %s", err.c_str()); } } PLASSERT( module_index_to_path_index.find(-1) == -1 ); }
string PLearn::NetworkModule::classname | ( | ) | const [virtual] |
Reimplemented from PLearn::Object.
Definition at line 64 of file NetworkModule.cc.
void PLearn::NetworkModule::declareOptions | ( | OptionList & | ol | ) | [static, protected] |
Declares the class options.
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 76 of file NetworkModule.cc.
References PLearn::OptionBase::buildoption, connections, PLearn::declareOption(), PLearn::OnlineLearningModule::declareOptions(), PLearn::OnlineLearningModule::input_size, modules, PLearn::OptionBase::nosave, PLearn::OnlineLearningModule::output_size, ports, PLearn::redeclareOption(), and save_states.
{ declareOption(ol, "modules", &NetworkModule::modules, OptionBase::buildoption, "List of modules contained in the network."); declareOption(ol, "connections", &NetworkModule::connections, OptionBase::buildoption, "List of connections between modules."); declareOption(ol, "ports", &NetworkModule::ports, OptionBase::buildoption, "A sequence of pairs of strings, where each pair is of the form\n" "\"P\":\"M.N\" with 'M' the name of an underlying module, 'N' one of\n" "its ports, and 'P' the name under which the NetworkModule sees this\n" "port. See the class help for an example. If 'P' is an empty string,\n" "then the port name will be 'M.N'."); declareOption(ol, "save_states", &NetworkModule::save_states, OptionBase::buildoption, "If set to 1, then any port of an underlying module whose name ends\n" "with 'state' will be automatically saved, even if it has no incoming\n" "or outgoing connection."); // Now call the parent class' declareOptions inherited::declareOptions(ol); // Hide unused parent options. redeclareOption(ol, "input_size", &NetworkModule::input_size, OptionBase::nosave, "Not used."); redeclareOption(ol, "output_size", &NetworkModule::output_size, OptionBase::nosave, "Not used."); }
static const PPath& PLearn::NetworkModule::declaringFile | ( | ) | [inline, static] |
NetworkModule * PLearn::NetworkModule::deepCopy | ( | CopiesMap & | copies | ) | const [virtual] |
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 64 of file NetworkModule.cc.
void PLearn::NetworkModule::forget | ( | ) | [virtual] |
Reset the parameters to the state they would be BEFORE starting training.
Note that this method is necessarily called from build().
Implements PLearn::OnlineLearningModule.
Definition at line 640 of file NetworkModule.cc.
References all_modules, i, PLearn::TVec< T >::length(), PLASSERT, and PLearn::OnlineLearningModule::random_gen.
{ // Forward forget to the underlying modules. for (int i = 0; i < all_modules.length(); i++) { PLASSERT( all_modules[i]->random_gen ); all_modules[i]->forget(); } }
Basic fprop (not implemented).
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 652 of file NetworkModule.cc.
References PLERROR.
{ PLERROR("In NetworkModule::fprop - Not implemented for single vector " "operations"); }
Compute the network's outputs by propagating each underlying module's computations through the network.
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 658 of file NetworkModule.cc.
References all_modules, PLearn::endl(), fprop_data, fprop_path, fprop_toplug, fprop_toresize, i, PLearn::TMat< T >::isEmpty(), j, PLearn::TMat< T >::length(), PLearn::TVec< T >::length(), PLASSERT, PLearn::TMat< T >::resize(), PLearn::TVec< T >::resize(), and PLearn::TMat< T >::width().
{ for (int i = 0; i < fprop_path.length(); i++) { PP<OnlineLearningModule> module = all_modules[fprop_path[i]]; DBG_MODULE_LOG << "FPROP: " << module->name << endl; // First resize some data matrices, so that the outputs are properly // computed. const TVec<int>& toresize = fprop_toresize[i]; const TVec<Mat*>& data = fprop_data[i]; for (int j = 0; j < toresize.length(); j++) { DBG_MODULE_LOG << " out = " << module->getPortName(toresize[j]) << endl; data[toresize[j]]->resize(0, 0); } // Then plug in the extra data provided in this method's fprop. const TMat<int>& toplug = fprop_toplug[i]; for (int j = 0; j < toplug.length(); j++) { int swap_1 = toplug(j, 0); Mat* provided = ports_value[swap_1]; if (!provided) // There is no provided matrix for this port, thus no extra // information to use or compute. continue; int swap_2 = toplug(j, 1); Mat* current = data[swap_2]; if (!provided->isEmpty()) { // A full matrix is provided. This means the data is readily // available (= input), and thus should not have been computed // anywhere else. PLASSERT( !current ); data[swap_2] = provided; } else { // An empty matrix is provided. This means we want to compute // this data. If it is already computed in the network, we do a // copy (may not be the most efficient way to do it though). If // it is not already computed, we can directly plug the // provided matrix. if (!current) { data[swap_2] = provided; } else { // This data should be something we compute. PLASSERT( current->isEmpty() ); } } } // Now perform fprop of the i-th module. module->fprop(data); // And restore 'data', in addition to copy any computed value that // needs to be retrieved. for (int j = 0; j < toplug.length(); j++) { int swap_1 = toplug(j, 0); Mat* provided = ports_value[swap_1]; if (!provided) continue; int swap_2 = toplug(j, 1); Mat* current = data[swap_2]; if (!provided->isEmpty()) { // This is either a matrix given as input, or directly plugged // in the network. In both cases, the old value in 'data' was a // null pointer. PLASSERT( current == provided ); data[swap_2] = NULL; } else { // This is an output we need to copy. PLASSERT( current && !current->isEmpty() ); provided->resize(current->length(), current->width()); *provided << *current; } } } }
OptionList & PLearn::NetworkModule::getOptionList | ( | ) | const [virtual] |
Reimplemented from PLearn::Object.
Definition at line 64 of file NetworkModule.cc.
OptionMap & PLearn::NetworkModule::getOptionMap | ( | ) | const [virtual] |
Reimplemented from PLearn::Object.
Definition at line 64 of file NetworkModule.cc.
const TVec< string > & PLearn::NetworkModule::getPorts | ( | ) | [virtual] |
Return the list of ports in the module.
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 736 of file NetworkModule.cc.
References all_ports.
{ return all_ports; }
Return the size of each port.
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 744 of file NetworkModule.cc.
References PLearn::OnlineLearningModule::port_sizes.
{ return port_sizes; }
RemoteMethodMap & PLearn::NetworkModule::getRemoteMethodMap | ( | ) | const [virtual] |
Reimplemented from PLearn::Object.
Definition at line 64 of file NetworkModule.cc.
void PLearn::NetworkModule::makeDeepCopyFromShallowCopy | ( | CopiesMap & | copies | ) | [virtual] |
Transforms a shallow copy into a deep copy.
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 752 of file NetworkModule.cc.
References all_connections, all_mats, all_modules, all_ports, bprop_data, bprop_data_idx, bprop_path, bprop_toresize, connections, PLearn::deepCopyField(), fprop_data, fprop_data_idx, fprop_path, fprop_toplug, fprop_toresize, i, PLearn::TVec< T >::length(), PLearn::OnlineLearningModule::makeDeepCopyFromShallowCopy(), modules, PLASSERT, ports, and PLearn::TVec< T >::resize().
{ inherited::makeDeepCopyFromShallowCopy(copies); deepCopyField(modules, copies); deepCopyField(connections, copies); deepCopyField(ports, copies); deepCopyField(all_modules, copies); deepCopyField(all_connections, copies); deepCopyField(fprop_path, copies); deepCopyField(bprop_path, copies); deepCopyField(fprop_data_idx, copies); deepCopyField(fprop_toresize, copies); deepCopyField(fprop_toplug, copies); deepCopyField(bprop_data_idx, copies); deepCopyField(bprop_toresize, copies); deepCopyField(all_mats, copies); deepCopyField(all_ports, copies); // Special code to handle the deep copy of 'fprop_data' and 'bprop_data'. // A better way may exist to do this! fprop_data = TVec< TVec<Mat*> >(fprop_data.length()); bprop_data = TVec< TVec<Mat*> >(bprop_data.length()); PLASSERT( fprop_data.length() == bprop_data.length() ); for (int i = 0; i < fprop_data.length(); i++) { TVec<Mat*>& fdata = fprop_data[i]; TVec<Mat*>& bdata = bprop_data[i]; const TVec<int>& fdata_idx = fprop_data_idx[i]; const TVec<int>& bdata_idx = bprop_data_idx[i]; fdata.resize(fdata_idx.length()); bdata.resize(bdata_idx.length()); for (int k = 0; k < fdata_idx.length(); k++) { int idx = fdata_idx[k]; if (idx == -1) fdata[k] = NULL; else fdata[k] = &all_mats[idx]; } for (int k = 0; k < bdata_idx.length(); k++) { int idx = bdata_idx[k]; if (idx == -1) bdata[k] = NULL; else bdata[k] = &all_mats[idx]; } } }
Reimplemented from PLearn::OnlineLearningModule.
Definition at line 96 of file NetworkModule.h.
TVec< PP<NetworkConnection> > PLearn::NetworkModule::all_connections [protected] |
Similar to 'all_modules', but for connections in the network.
Definition at line 113 of file NetworkModule.h.
Referenced by build_(), and makeDeepCopyFromShallowCopy().
TVec<Mat> PLearn::NetworkModule::all_mats [protected] |
A list of all matrices used to store the various computation results in the network (i.e.
the outputs of each module).
Definition at line 162 of file NetworkModule.h.
Referenced by build_(), and makeDeepCopyFromShallowCopy().
TVec< PP<OnlineLearningModule> > PLearn::NetworkModule::all_modules [protected] |
List of all modules, i.e.
the modules in the 'modules' option, possibly completed with an additional NullModule to save states when the 'save_states' option is true.
Definition at line 110 of file NetworkModule.h.
Referenced by bpropAccUpdate(), build_(), forget(), fprop(), and makeDeepCopyFromShallowCopy().
TVec<string> PLearn::NetworkModule::all_ports [protected] |
The list of ports, computed from the 'ports' option.
Definition at line 165 of file NetworkModule.h.
Referenced by build_(), getPorts(), and makeDeepCopyFromShallowCopy().
TVec< TVec<Mat*> > PLearn::NetworkModule::bprop_data [protected] |
The i-th element is the list of Mat* pointers being provided to the i-th module in a bprop step.
Definition at line 147 of file NetworkModule.h.
Referenced by bpropAccUpdate(), build_(), and makeDeepCopyFromShallowCopy().
TVec< TVec<int> > PLearn::NetworkModule::bprop_data_idx [protected] |
Same as 'bprop_data', but the integer is the index of the matrix in the 'all_mats' vector.
This is used to properly deep copy this object.
Definition at line 151 of file NetworkModule.h.
Referenced by build_(), and makeDeepCopyFromShallowCopy().
TVec<int> PLearn::NetworkModule::bprop_path [protected] |
Ordered list of modules used when doing a bprop (the integer values correspond to indices in 'modules').
Definition at line 121 of file NetworkModule.h.
Referenced by bpropAccUpdate(), build_(), and makeDeepCopyFromShallowCopy().
TVec< TVec<int> > PLearn::NetworkModule::bprop_toresize [protected] |
The i-th element is the list of matrices that need to be resized to empty matrices prior to calling bpropUpdate() on the i-th module in a bprop step.
The resizing is needed to ensure we correctly compute the desired gradients.
Definition at line 158 of file NetworkModule.h.
Referenced by bpropAccUpdate(), build_(), and makeDeepCopyFromShallowCopy().
Definition at line 56 of file NetworkModule.h.
Referenced by build_(), declareOptions(), and makeDeepCopyFromShallowCopy().
TVec< TVec<Mat*> > PLearn::NetworkModule::fprop_data [protected] |
The i-th element is the list of Mat* pointers being provided to the i-th module in a fprop step.
Definition at line 125 of file NetworkModule.h.
Referenced by bpropAccUpdate(), build_(), fprop(), and makeDeepCopyFromShallowCopy().
TVec< TVec<int> > PLearn::NetworkModule::fprop_data_idx [protected] |
Same as 'fprop_data', but the integer is the index of the matrix in the 'all_mats' vector.
This is used to properly deep copy this object.
Definition at line 129 of file NetworkModule.h.
Referenced by build_(), and makeDeepCopyFromShallowCopy().
TVec<int> PLearn::NetworkModule::fprop_path [protected] |
Ordered list of modules used when doing a fprop (the integer values correspond to indices in 'modules').
Definition at line 117 of file NetworkModule.h.
Referenced by bpropAccUpdate(), build_(), fprop(), and makeDeepCopyFromShallowCopy().
TVec< TMat<int> > PLearn::NetworkModule::fprop_toplug [protected] |
The i-th element is the list of correspondances between ports in this NetworkModule and ports in the i-th module of the fprop path.
The list is given by a matrix, where each row is a pair (p1, p2) with p1 the index of the port in this NetworkModule, and p2 the index of the corresponding port in the i-th module during fprop.
Definition at line 143 of file NetworkModule.h.
Referenced by bpropAccUpdate(), build_(), fprop(), and makeDeepCopyFromShallowCopy().
TVec< TVec<int> > PLearn::NetworkModule::fprop_toresize [protected] |
The i-th element is the list of matrices that need to be resized to empty matrices prior to calling fprop() on the i-th module in a fprop step.
The resizing is needed to ensure we correctly compute the desired outputs.
Definition at line 136 of file NetworkModule.h.
Referenced by build_(), fprop(), and makeDeepCopyFromShallowCopy().
Definition at line 55 of file NetworkModule.h.
Referenced by build_(), declareOptions(), and makeDeepCopyFromShallowCopy().
TVec< pair<string, string > > PLearn::NetworkModule::ports |
Definition at line 58 of file NetworkModule.h.
Referenced by build_(), declareOptions(), and makeDeepCopyFromShallowCopy().
Definition at line 60 of file NetworkModule.h.
Referenced by build_(), and declareOptions().