PLearn 0.1
Public Member Functions | Static Public Member Functions | Public Attributes | Static Public Attributes | Static Protected Member Functions | Protected Attributes | Private Types | Private Member Functions
PLearn::NetworkModule Class Reference

#include <NetworkModule.h>

Inheritance diagram for PLearn::NetworkModule:
Inheritance graph
[legend]
Collaboration diagram for PLearn::NetworkModule:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 NetworkModule ()
 Default constructor.
virtual void fprop (const Vec &input, Vec &output) const
 Basic fprop (not implemented).
virtual void fprop (const TVec< Mat * > &ports_value)
 Compute the network's outputs by propagating each underlying module's computations through the network.
virtual void bpropAccUpdate (const TVec< Mat * > &ports_value, const TVec< Mat * > &ports_gradient)
 Back-propagate gradient through the network, and update each module's internal parameters.
virtual const TVec< string > & getPorts ()
 Return the list of ports in the module.
virtual const TMat< int > & getPortSizes ()
 Return the size of each port.
virtual void forget ()
 Reset the parameters to the state they would be BEFORE starting training.
virtual string classname () const
virtual OptionListgetOptionList () const
virtual OptionMapgetOptionMap () const
virtual RemoteMethodMapgetRemoteMethodMap () const
virtual NetworkModuledeepCopy (CopiesMap &copies) const
virtual void build ()
 Post-constructor.
virtual void makeDeepCopyFromShallowCopy (CopiesMap &copies)
 Transforms a shallow copy into a deep copy.

Static Public Member Functions

static string _classname_ ()
static OptionList_getOptionList_ ()
static RemoteMethodMap_getRemoteMethodMap_ ()
static Object_new_instance_for_typemap_ ()
static bool _isa_ (const Object *o)
static void _static_initialize_ ()
static const PPathdeclaringFile ()

Public Attributes

TVec< PP< OnlineLearningModule > > modules
TVec< PP< NetworkConnection > > connections
TVec< pair< string, string > > ports
bool save_states

Static Public Attributes

static StaticInitializer _static_initializer_

Static Protected Member Functions

static void declareOptions (OptionList &ol)
 Declares the class options.

Protected Attributes

TVec< PP< OnlineLearningModule > > all_modules
 List of all modules, i.e.
TVec< PP< NetworkConnection > > all_connections
 Similar to 'all_modules', but for connections in the network.
TVec< intfprop_path
 Ordered list of modules used when doing a fprop (the integer values correspond to indices in 'modules').
TVec< intbprop_path
 Ordered list of modules used when doing a bprop (the integer values correspond to indices in 'modules').
TVec< TVec< Mat * > > fprop_data
 The i-th element is the list of Mat* pointers being provided to the i-th module in a fprop step.
TVec< TVec< int > > fprop_data_idx
 Same as 'fprop_data', but the integer is the index of the matrix in the 'all_mats' vector.
TVec< TVec< int > > fprop_toresize
 The i-th element is the list of matrices that need to be resized to empty matrices prior to calling fprop() on the i-th module in a fprop step.
TVec< TMat< int > > fprop_toplug
 The i-th element is the list of correspondances between ports in this NetworkModule and ports in the i-th module of the fprop path.
TVec< TVec< Mat * > > bprop_data
 The i-th element is the list of Mat* pointers being provided to the i-th module in a bprop step.
TVec< TVec< int > > bprop_data_idx
 Same as 'bprop_data', but the integer is the index of the matrix in the 'all_mats' vector.
TVec< TVec< int > > bprop_toresize
 The i-th element is the list of matrices that need to be resized to empty matrices prior to calling bpropUpdate() on the i-th module in a bprop step.
TVec< Matall_mats
 A list of all matrices used to store the various computation results in the network (i.e.
TVec< string > all_ports
 The list of ports, computed from the 'ports' option.

Private Types

typedef OnlineLearningModule inherited

Private Member Functions

void build_ ()
 This does the actual building.

Detailed Description

Definition at line 48 of file NetworkModule.h.


Member Typedef Documentation

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 50 of file NetworkModule.h.


Constructor & Destructor Documentation

PLearn::NetworkModule::NetworkModule ( )

Default constructor.

Definition at line 69 of file NetworkModule.cc.

                            :
    save_states(true)
{}

Member Function Documentation

string PLearn::NetworkModule::_classname_ ( ) [static]

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 64 of file NetworkModule.cc.

OptionList & PLearn::NetworkModule::_getOptionList_ ( ) [static]

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 64 of file NetworkModule.cc.

RemoteMethodMap & PLearn::NetworkModule::_getRemoteMethodMap_ ( ) [static]

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 64 of file NetworkModule.cc.

bool PLearn::NetworkModule::_isa_ ( const Object o) [static]

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 64 of file NetworkModule.cc.

Object * PLearn::NetworkModule::_new_instance_for_typemap_ ( ) [static]

Reimplemented from PLearn::Object.

Definition at line 64 of file NetworkModule.cc.

StaticInitializer NetworkModule::_static_initializer_ & PLearn::NetworkModule::_static_initialize_ ( ) [static]

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 64 of file NetworkModule.cc.

void PLearn::NetworkModule::bpropAccUpdate ( const TVec< Mat * > &  ports_value,
const TVec< Mat * > &  ports_gradient 
) [virtual]

Back-propagate gradient through the network, and update each module's internal parameters.

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 118 of file NetworkModule.cc.

References all_modules, bprop_data, bprop_path, bprop_toresize, PLearn::TMat< T >::clear(), PLearn::endl(), fprop_data, fprop_path, fprop_toplug, i, PLearn::TMat< T >::isEmpty(), j, PLearn::TMat< T >::length(), PLearn::TVec< T >::length(), PLASSERT, PLASSERT_MSG, PLearn::TMat< T >::resize(), and PLearn::TMat< T >::width().

{

    // Reset internal gradients, either:
    // - to zero if no additional gradient is provided in 'ports_gradient'
    // - to the gradient provided in 'ports_gradient' otherwise
    PLASSERT( fprop_path.length() == bprop_path.length() );
    for (int i = 0; i < bprop_path.length(); i++) {
        const TVec<int>& toresize = bprop_toresize[i];
        int fprop_idx = fprop_data.length() - 1 - i;
        const TVec<Mat*>& f_data = fprop_data[fprop_idx];
        for (int j = 0; j < toresize.length(); j++) {
            if (j == 0) {
                DBG_MODULE_LOG << "CLEAR: " <<
                    all_modules[bprop_path[i]]->classname() << endl;
            }
            int mat_idx = toresize[j];
            DBG_MODULE_LOG << "  grad = " <<
                all_modules[bprop_path[i]]->getPortName(mat_idx) << endl;
            Mat* mat_toresize = bprop_data[i][mat_idx];
            Mat* mat_tpl = f_data[mat_idx];
            mat_toresize->resize(mat_tpl->length(), mat_tpl->width());
            mat_toresize->clear();
        }
        const TMat<int>& f_toplug = fprop_toplug[fprop_idx];
        for (int j = 0; j < f_toplug.length(); j++) {
            int mod_idx = f_toplug(j, 1);
            Mat* current = bprop_data[i][mod_idx];
            if (!current)
                // There is no gradient currently computed for this port, thus
                // nothing to initialize.
                continue;
            int this_idx = f_toplug(j, 0);
            Mat* provided = ports_gradient[this_idx];
            if (provided) {
                // We are provided with some additional gradient, either as
                // input or output. If it is an output, there is nothing more
                // to do. However, if it is an input, we must use it to
                // initialize the gradient.
                if (!provided->isEmpty()) {
                    PLASSERT( current->length() == provided->length() &&
                            current->width() == provided->width() );
                    PLASSERT( (*current)(0, 0) == 0 );
                    *current << *provided;
                } else {
                    // Just for safety reason, crash here. It seems weird that
                    // we may want to accumulate the gradient w.r.t. a port for
                    // which a gradient is already computed internally (i.e. is
                    // used in the bprop). If we want to allow this, care
                    // should be taken as to what to do with the current value
                    // of the gradient in 'provided' (in particular, do we use
                    // it to update the internal paremeters?)
                    PLASSERT( false );
                }
            }
        }
    }

    // Backpropagate gradient and update parameters.
    for (int i = 0; i < bprop_path.length(); i++) {
        PP<OnlineLearningModule> module = all_modules[bprop_path[i]];
        DBG_MODULE_LOG << "BPROP: " << module->name << endl;
        // First resize some gradient matrices, so that the gradient is
        // properly computed.
        const TVec<int>& toresize = bprop_toresize[i];
        const TVec<Mat*> b_data = bprop_data[i];
        for (int j = 0; j < toresize.length(); j++) {
            int mat_idx = toresize[j];
            DBG_MODULE_LOG << "  grad = " << module->getPortName(mat_idx)
                           << endl;
            Mat* mat_toresize = b_data[mat_idx];
            PLASSERT( mat_toresize->width() > 0 );
            mat_toresize->resize(0, mat_toresize->width());
        }

        // Plug in the matrices provided as parameters of this method.
        int fprop_idx = fprop_data.length() - 1 - i;
        const TVec<Mat*> f_data = fprop_data[fprop_idx];
        const TMat<int>& f_toplug = fprop_toplug[fprop_idx];
        for (int j = 0; j < f_toplug.length(); j++) {
            int this_idx = f_toplug(j, 0);
            int mod_idx = f_toplug(j, 1);
            Mat* provided = ports_gradient[this_idx];
            if (provided) {
                Mat* current = b_data[mod_idx];
                if (!current) {
                    // We can directly plug in the provided matrix.
                    b_data[mod_idx] = provided;
                } else {
                    if (!provided->isEmpty()) {
                        PLASSERT( !current->isEmpty() );
                        // This gradient is some external gradient we are
                        // provided with. It has already been added in the step
                        // where we reset (see above), thus there is nothing
                        // more to do.  TODO Note that it may be cleaner to
                        // actually add it here to simplify the code above.
                    } else {
                        PLASSERT( current->isEmpty() );
                        // This gradient must be computed, but is already
                        // computed somewhere here. We will copy it after the
                        // update step.
                        PLASSERT( false); // This should not happen (cf. above)
                    }
                }
            }
            // Same for the fprop data since it is also given in argument.
            Mat* f_provided = ports_value[this_idx];
            if (f_provided) {
                Mat* f_current = f_data[mod_idx];
                if (!f_current)
                    f_data[mod_idx] = f_provided;
                else {
                    // 'f_provided' is thus the result of the computation, that
                    // was copied from the value of 'f_current' at fprop time.
                    // We must fill in 'f_current' with this value (once again,
                    // this is not best for efficiency).
                    f_current->resize(f_provided->length(),
                                      f_provided->width());
                    *f_current << *f_provided;
                }
            }
        }

        // Then perform the bpropUpdate step.
        module->bpropAccUpdate(
                fprop_data[fprop_data.length() - 1 - i], bprop_data[i]);

        // Restore the 'bprop_data' and 'fprop_data' parameters.
        for (int j = 0; j < f_toplug.length(); j++) {
            int this_idx = f_toplug(j, 0);
            int mod_idx = f_toplug(j, 1);
            Mat* provided = ports_gradient[this_idx];
            if (provided) {
                Mat* current = b_data[mod_idx];
                PLASSERT_MSG(current, "This should not happen: if we are "
                        "provided with a matrix, it should have been plugged "
                        "in here");
                if (provided == current)
                    // They are the same matrices: this means there used to be
                    // a NULL matrix before.
                    b_data[mod_idx] = NULL;
                else {
                    // This should never happen.
                    PLASSERT( false );
                }
            }
            Mat* f_provided = ports_value[this_idx];
            if (f_provided) {
                Mat* f_current = f_data[mod_idx];
                if (f_current == f_provided)
                    // This used to be a null pointer, that must be restored.
                    f_data[mod_idx] = NULL;
            }
        }
    }
}

Here is the call graph for this function:

void PLearn::NetworkModule::build ( ) [virtual]

Post-constructor.

The normal implementation should call simply inherited::build(), then this class's build_(). This method should be callable again at later times, after modifying some option fields to change the "architecture" of the object.

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 280 of file NetworkModule.cc.

References PLearn::OnlineLearningModule::build(), and build_().

Here is the call graph for this function:

void PLearn::NetworkModule::build_ ( ) [private]

This does the actual building.

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 289 of file NetworkModule.cc.

References all_connections, all_mats, all_modules, all_ports, PLearn::TVec< T >::append(), PLearn::TMat< T >::appendRow(), bprop_data, bprop_data_idx, bprop_path, bprop_toresize, connections, PLearn::TVec< T >::fill(), PLearn::TVec< T >::first(), fprop_data, fprop_data_idx, fprop_path, fprop_toplug, fprop_toresize, i, j, PLearn::TVec< T >::lastElement(), PLearn::TVec< T >::length(), modules, PLearn::OnlineLearningModule::name, PLASSERT, PLASSERT_MSG, PLERROR, PLWARNING, PLearn::OnlineLearningModule::port_sizes, ports, PLearn::OnlineLearningModule::random_gen, PLearn::TMat< T >::resize(), PLearn::TVec< T >::resize(), save_states, PLearn::TVec< T >::size(), PLearn::test(), and PLearn::tostring().

Referenced by build().

{
    // Initialize the 'all_modules' and 'all_connections' lists.
    all_modules.resize(modules.length());
    all_modules << modules;
    all_connections.resize(connections.length());
    all_connections << connections;
    if (save_states) {
        // Ensure all ports which are states are correctly saved.
        // This is done by adding connections to a NullModule in order to force
        // them being saved.
        int count_null = 0;
        for (int i = 0; i < modules.length(); i++) {
            const TVec<string>& mod_ports = modules[i]->getPorts();
            for (int j = 0; j < mod_ports.length(); j++) {
                const string& port = mod_ports[j];
                if (port.size() >= 5 &&
                    port.substr(port.size() - 5) == "state")
                {
                    // Look for either an incoming or outgoing connection.
                    string port_name = modules[i]->name + "." + port;
                    bool has_a_connection = false;
                    for (int k = 0; k < connections.length(); k++)
                        if (connections[k]->destination == port_name ||
                            connections[k]->source == port_name)
                        {
                            has_a_connection = true;
                            break;
                        }
                    if (!has_a_connection) {
                        // Currently we create one NullModule / state, since
                        // multiple incoming connections are not correctly
                        // handled.
                        PP<OnlineLearningModule> null;
                        /*
                        if (all_modules.length() == modules.length()) {
                            // No null module added yet.
                            */
                            null = new NullModule("_null_" +
                                    tostring(count_null++));
                            all_modules.append(null);
                        /*} else {
                            PLASSERT( all_modules.length() ==
                                        modules.length() + 1 );
                            null = all_modules.lastElement();
                        }*/
                        all_connections.append( new NetworkConnection(
                                    modules[i], port, null, "null", false));
                    }
                }
            }
        }
    }

    // Forward random number generator to all underlying modules.
    if (random_gen) {
        for (int i = 0; i < all_modules.length(); i++) {
            if (!all_modules[i]->random_gen) {
                all_modules[i]->random_gen = random_gen;
                all_modules[i]->build();
                all_modules[i]->forget();
            }
        }
    }

    // Construct fprop and bprop paths from the list of modules and
    // connections.

    // First create a dictionary of modules.
    map<string, PP<OnlineLearningModule> > name_to_module;
    for (int i = 0; i < all_modules.length(); i++) {
        string module = all_modules[i]->name;
        if (name_to_module.find(module) != name_to_module.end()) {
            // There is already a module with the same name. For safety
            // reasons, we make it point to a NULL pointer to ensure we do not
            // accidentally use the wrong module.
            PLWARNING("In NetworkModule::build_ - There are two modules with "
                    "the same name '%s': any connection attempting to use "
                    "one of these modules will probably crash",
                    module.c_str());
            name_to_module[module] = NULL;
        } else
            name_to_module[module] = all_modules[i];
    }

    // Initialize connections.
    for (int i = 0; i < all_connections.length(); i++)
        all_connections[i]->initialize(name_to_module);

    // Preprocess some convenience data structures from the list of
    // connections.

    // 'module_to_index' maps module pointers to their corresponding index in
    // the 'all_modules' list.
    map<const OnlineLearningModule*, int> module_to_index;
    for (int i = 0; i < all_modules.length(); i++)
        module_to_index[all_modules[i]] = i;

    // Analyze the list of ports.
    // The 'port_correspondances' lists, for each module, the correspondances
    // between the modules' ports and the ports of the NetworkModule.
    TVec< TMat<int> > port_correspondances(all_modules.length());
    for (int i = 0; i < all_modules.length(); i++)
        port_correspondances[i].resize(0, 2);
    TVec<int> new_row(2);
    all_ports.resize(0);
    port_sizes.resize(0, 2);
    int n_ports = ports.length();
    for (int i = 0; i < n_ports; i++) {
        const pair<string, string>& new_old = ports[i];
        string new_name = new_old.first;
        const string& old_name = new_old.second;
        if (new_name.empty())
            // Empty string: we use the full name of the form 'module.port'.
            new_name = old_name;
        all_ports.append(new_name);
        size_t dot_pos = old_name.find('.');
        PLASSERT( dot_pos != string::npos );
        string old_module_name = old_name.substr(0, dot_pos);
        string old_module_port = old_name.substr(dot_pos + 1);
        if ( name_to_module.find(old_module_name) == name_to_module.end())
            PLERROR("NetworkModule::Can't find module with name %s\n",old_module_name.c_str());

        new_row[0] = i;
        PP<OnlineLearningModule> old_module = name_to_module[old_module_name];
        new_row[1] = old_module->getPortIndex(old_module_port);
        port_correspondances[module_to_index[old_module]].appendRow(new_row);
        new_row[0] = old_module->getPortLength(old_module_port);
        new_row[1] = old_module->getPortWidth(old_module_port);
        port_sizes.appendRow(new_row);
    }

    // The i-th element of 'in_connections' maps each port in the i-th module
    // to the connection that has it as destination (there may be only one).
    TVec< map<string, PP<NetworkConnection> > > in_connections;
    in_connections.resize(all_modules.length());
    // The i-th element of 'out_connections' maps each port in the i-th module
    // to the connections that have it as source (there may be many).
    TVec< map<string, TVec< PP<NetworkConnection> > > > out_connections;
    out_connections.resize(all_modules.length());
    // The 'inputs_needed' vector contains the number of inputs that must be
    // fed to a module before it can compute a fprop.
    TVec<int> inputs_needed(all_modules.length(), 0);
    // The 'compute_input_of' list gives, for each module M, the indices of
    // other modules that take an output of M as input.
    TVec< TVec<int> > compute_input_of(all_modules.length());
    for (int i = 0; i < all_connections.length(); i++) {
        PP<NetworkConnection> connection = all_connections[i];
        int src = module_to_index[connection->getSourceModule()];
        int dest = module_to_index[connection->getDestinationModule()];
        inputs_needed[dest]++;
        compute_input_of[src].append(dest);
        map<string, PP<NetworkConnection> >& in_conn = in_connections[dest];
        if (in_conn.find(connection->getDestinationPort()) != in_conn.end()) {
            /*
            // The port has more than one incoming connection. Currently, we
            // only allow this to happen if the module is a NullModule (this is
            // for safety). If in the future we want more modules to allow
            // multiple incoming connections, we may get rid of this error.
            NullModule* test = dynamic_cast<NullModule*>(
                    get_pointer(connection->getDestinationModule()));
                    */
            // TODO ACTUALLY this does not work, as the fprop/bprop paths may
            // not be correctly computed in the case of multiple incoming
            // connections. Until this is fixed, multiple incoming connections
            // are not allowed.
            bool test = false;
            if (!test)
                PLERROR("In LearningNetwork::build_ - A port may have only one "
                        "incoming connection");
        }
        in_conn[connection->getDestinationPort()] = connection;
        out_connections[src][connection->getSourcePort()].append(connection);
    }
    // The fprop and bprop paths can now be computed.
    fprop_path.resize(0);
    bprop_path.resize(all_modules.length());
    bprop_path.fill(-1);
    TVec<bool> is_done(all_modules.length(), false);
    fprop_data.resize(0);
    fprop_data_idx.resize(0);
    bprop_data.resize(all_modules.length());
    bprop_data_idx.resize(all_modules.length());
    fprop_toresize.resize(0);
    bprop_toresize.resize(all_modules.length());
    fprop_toplug.resize(0);
    // This is getting a little hackish here... If not enough memory is
    // allocated to store the work matrices, then when appending a new Mat to
    // 'all_mats', we will have to create a new Storage and copy previous data.
    // The problem is then that the pointers in 'fprop_data' and 'bprop_data'
    // will become invalid. So an easy fix is to allocate enough memory to
    // ensure we never have to resize the Storage later.
    int max_n_mats = 1000;
    all_mats.resize(max_n_mats);
    all_mats.resize(0);
    // A vector that stores the index of a module in the fprop path.
    TVec<int> module_index_to_path_index(all_modules.length(), -1);
    while (is_done.find(false) >= 0) {
        bool at_least_one_done = false;
        for (int i = 0; i < all_modules.length(); i++) {
            if (!is_done[i] && inputs_needed[i] == 0) {
                for (int j = 0; j < compute_input_of[i].length(); j++)
                    inputs_needed[compute_input_of[i][j]]--;
                // Save any correspondance between ports of this module and
                // ports of the encapsulating NetworkModule.
                fprop_toplug.append(port_correspondances[i]);
                // Compute the list of matrices that must be provided to this
                // module when doing a fprop and bprop.
                TVec<string> mod_ports = all_modules[i]->getPorts();
                map<string, PP<NetworkConnection> >& in_conn =
                    in_connections[i];
                map<string, TVec< PP<NetworkConnection> > >& out_conn =
                    out_connections[i];
                TVec<int> fprop_tores;
                TVec<int> bprop_tores;
                TVec<Mat*> fprop_mats;
                TVec<Mat*> bprop_mats;
                TVec<int> fprop_mats_idx;
                TVec<int> bprop_mats_idx;
                int n_in_conn = in_conn.size();
                int n_in_conn_found = 0;
                for (int j = 0; j < mod_ports.length(); j++) {
                    int count_inc_conn = in_conn.count(mod_ports[j]);
                    if (count_inc_conn > 0) {
                        // This port has an incoming connection: it is thus an
                        // input, and the corresponding matrices for storing
                        // its value and gradient are found by looking at the
                        // source port of the connection.
                        n_in_conn_found += count_inc_conn;
                        PP<NetworkConnection> conn = in_conn[mod_ports[j]];
                        int src_mod = module_to_index[conn->getSourceModule()];
                        int path_index = module_index_to_path_index[src_mod];
                        int port_index = conn->getSourceModule()->getPortIndex(
                                conn->getSourcePort());
                        if (port_index < 0)
                            PLERROR("In NetworkModule::build_ - no port named"
                                    " \"%s\"\n"
                                    "in module \"%s\". Valid port names are:\n"
                                    "%s\n",
                                    conn->getSourcePort().c_str(),
                                    conn->getSourceModule()->name.c_str(),
                                    tostring(conn->getSourceModule()
                                             ->getPorts()).c_str()
                                    );
                        fprop_mats.append(fprop_data[path_index][port_index]);
                        fprop_mats_idx.append(
                                fprop_data_idx[path_index][port_index]);
                        if (!conn->propagate_gradient) {
                            // This connection does not propagate the gradient,
                            // and thus we do not want to accumulate it.
                            bprop_mats.append(NULL);
                            bprop_mats_idx.append(-1);
                        } else {
                            int b_idx = all_modules.length() - 1 - path_index;
                            bprop_mats.append(bprop_data[b_idx][port_index]);
                            bprop_mats_idx.append(
                                    bprop_data_idx[b_idx][port_index]);
                            bprop_tores.append(j);
                        }
                        PLASSERT_MSG(
                                out_conn.find(mod_ports[j]) == out_conn.end(),
                                "A port with an incoming connection cannot "
                                "also have an outgoing connection" );
                    } else if (out_conn.find(mod_ports[j]) != out_conn.end()) {
                        // This port has (at least) one outgoing connection: it
                        // is thus an output, and it must be provided with
                        // matrices to store its value (and gradient if the
                        // connection propagates it).
                        all_mats.append(Mat());
                        if (all_mats.length() > max_n_mats)
                            PLERROR("In NetworkModule::build_ - Will need to "
                                    "increase 'max_n_mats'");
                        Mat* new_mat = &all_mats.lastElement();
                        fprop_mats.append(new_mat);
                        fprop_mats_idx.append(all_mats.length() - 1);
                        fprop_tores.append(j);
                        // Ensure there exists a connection propagating the
                        // gradient to this port.
                        bool must_store_grad = false;
                        const TVec< PP<NetworkConnection> >& out_j =
                            out_conn[mod_ports[j]];
                        for (int k = 0; k < out_j.length(); k++)
                            if (out_j[k]->propagate_gradient) {
                                must_store_grad = true;
                                break;
                            }
                        if (must_store_grad) {
                            all_mats.append(Mat());
                            if (all_mats.length() > max_n_mats)
                                PLERROR("In NetworkModule::build_ - Will need "
                                        "to increase 'max_n_mats'");
                            new_mat = &all_mats.lastElement();
                            bprop_mats.append(new_mat);
                            bprop_mats_idx.append(all_mats.length() - 1);
                        } else {
                            // No connection propagating gradient to this port.
                            bprop_mats.append(NULL);
                            bprop_mats_idx.append(-1);
                        }
                    } else {
                        // This port is not used (we do not provide its value,
                        // and we do not care about obtaining it).
                        fprop_mats.append(NULL);
                        bprop_mats.append(NULL);
                        fprop_mats_idx.append(-1);
                        bprop_mats_idx.append(-1);
                    }
                }
                if (n_in_conn != n_in_conn_found)
                    PLERROR("In NetworkModule::build_ - Not all incoming "
                            "connections were found for module '%s', please "
                            "ensure all destination ports do exist",
                            all_modules[i]->name.c_str());
                module_index_to_path_index[i] = fprop_path.length();
                // Update fprop path.
                PLASSERT( fprop_mats.length() == fprop_mats_idx.length() );
                fprop_data.append(fprop_mats);
                fprop_data_idx.append(fprop_mats_idx);
                fprop_toresize.append(fprop_tores);
                fprop_path.append(i);
                // Update bprop path.
                PLASSERT( bprop_mats_idx.length() == bprop_mats.length() );
                int bprop_idx = bprop_path.length() - fprop_path.length();
                bprop_data[bprop_idx] = bprop_mats;
                bprop_data_idx[bprop_idx] = bprop_mats_idx;
                bprop_toresize[bprop_idx] = bprop_tores;
                bprop_path[bprop_idx] = i;

                is_done[i] = true;
                at_least_one_done = true;
            }
        }
        if (!at_least_one_done) {
            // Infinite loop: the algorithm cannot find a module that can be
            // propagated at this point.
            string err;
            for (int i = 0; i < all_modules.length(); i++)
                if (!is_done[i])
                    err += all_modules[i]->name + ", ";
            PLERROR("In NetworkModule::build_ - Cannot build propagation path "
                    "due to the following modules waiting for incoming "
                    "connections (there may be an infinite loop): %s",
                    err.c_str());
        }
    }
    PLASSERT( module_index_to_path_index.find(-1) == -1 );
}

Here is the call graph for this function:

Here is the caller graph for this function:

string PLearn::NetworkModule::classname ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 64 of file NetworkModule.cc.

void PLearn::NetworkModule::declareOptions ( OptionList ol) [static, protected]

Declares the class options.

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 76 of file NetworkModule.cc.

References PLearn::OptionBase::buildoption, connections, PLearn::declareOption(), PLearn::OnlineLearningModule::declareOptions(), PLearn::OnlineLearningModule::input_size, modules, PLearn::OptionBase::nosave, PLearn::OnlineLearningModule::output_size, ports, PLearn::redeclareOption(), and save_states.

{
    declareOption(ol, "modules", &NetworkModule::modules,
                  OptionBase::buildoption,
       "List of modules contained in the network.");

    declareOption(ol, "connections", &NetworkModule::connections,
                  OptionBase::buildoption,
       "List of connections between modules.");

    declareOption(ol, "ports", &NetworkModule::ports,
                  OptionBase::buildoption,
       "A sequence of pairs of strings, where each pair is of the form\n"
       "\"P\":\"M.N\" with 'M' the name of an underlying module, 'N' one of\n"
       "its ports, and 'P' the name under which the NetworkModule sees this\n"
       "port. See the class help for an example. If 'P' is an empty string,\n"
       "then the port name will be 'M.N'.");

    declareOption(ol, "save_states", &NetworkModule::save_states,
                  OptionBase::buildoption,
       "If set to 1, then any port of an underlying module whose name ends\n"
       "with 'state' will be automatically saved, even if it has no incoming\n"
       "or outgoing connection.");

    // Now call the parent class' declareOptions
    inherited::declareOptions(ol);

    // Hide unused parent options.

    redeclareOption(ol, "input_size", &NetworkModule::input_size,
                    OptionBase::nosave,
                    "Not used.");

    redeclareOption(ol, "output_size", &NetworkModule::output_size,
                    OptionBase::nosave,
                    "Not used.");

}

Here is the call graph for this function:

static const PPath& PLearn::NetworkModule::declaringFile ( ) [inline, static]

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 96 of file NetworkModule.h.

:

NetworkModule * PLearn::NetworkModule::deepCopy ( CopiesMap copies) const [virtual]

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 64 of file NetworkModule.cc.

void PLearn::NetworkModule::forget ( ) [virtual]

Reset the parameters to the state they would be BEFORE starting training.

Note that this method is necessarily called from build().

Implements PLearn::OnlineLearningModule.

Definition at line 640 of file NetworkModule.cc.

References all_modules, i, PLearn::TVec< T >::length(), PLASSERT, and PLearn::OnlineLearningModule::random_gen.

{
    // Forward forget to the underlying modules.
    for (int i = 0; i < all_modules.length(); i++) {
        PLASSERT( all_modules[i]->random_gen );
        all_modules[i]->forget();
    }
}

Here is the call graph for this function:

void PLearn::NetworkModule::fprop ( const Vec input,
Vec output 
) const [virtual]

Basic fprop (not implemented).

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 652 of file NetworkModule.cc.

References PLERROR.

{
    PLERROR("In NetworkModule::fprop - Not implemented for single vector "
            "operations");
}
void PLearn::NetworkModule::fprop ( const TVec< Mat * > &  ports_value) [virtual]

Compute the network's outputs by propagating each underlying module's computations through the network.

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 658 of file NetworkModule.cc.

References all_modules, PLearn::endl(), fprop_data, fprop_path, fprop_toplug, fprop_toresize, i, PLearn::TMat< T >::isEmpty(), j, PLearn::TMat< T >::length(), PLearn::TVec< T >::length(), PLASSERT, PLearn::TMat< T >::resize(), PLearn::TVec< T >::resize(), and PLearn::TMat< T >::width().

                                                       {
    for (int i = 0; i < fprop_path.length(); i++) {
        PP<OnlineLearningModule> module = all_modules[fprop_path[i]];
        DBG_MODULE_LOG << "FPROP: " << module->name << endl;

        // First resize some data matrices, so that the outputs are properly
        // computed.
        const TVec<int>& toresize = fprop_toresize[i];
        const TVec<Mat*>& data = fprop_data[i];
        for (int j = 0; j < toresize.length(); j++) {
            DBG_MODULE_LOG << "  out = " << module->getPortName(toresize[j])
                           << endl;
            data[toresize[j]]->resize(0, 0);
        }

        // Then plug in the extra data provided in this method's fprop.
        const TMat<int>& toplug = fprop_toplug[i];
        for (int j = 0; j < toplug.length(); j++) {
            int swap_1 = toplug(j, 0);
            Mat* provided = ports_value[swap_1];
            if (!provided)
                // There is no provided matrix for this port, thus no extra
                // information to use or compute.
                continue;
            int swap_2 = toplug(j, 1);
            Mat* current = data[swap_2];
            if (!provided->isEmpty()) {
                // A full matrix is provided. This means the data is readily
                // available (= input), and thus should not have been computed
                // anywhere else.
                PLASSERT( !current );
                data[swap_2] = provided;
            } else {
                // An empty matrix is provided. This means we want to compute
                // this data. If it is already computed in the network, we do a
                // copy (may not be the most efficient way to do it though). If
                // it is not already computed, we can directly plug the
                // provided matrix.
                if (!current) {
                    data[swap_2] = provided;
                } else {
                    // This data should be something we compute.
                    PLASSERT( current->isEmpty() );
                }
            }
        }

        // Now perform fprop of the i-th module.
        module->fprop(data);

        // And restore 'data', in addition to copy any computed value that
        // needs to be retrieved.
        for (int j = 0; j < toplug.length(); j++) {
            int swap_1 = toplug(j, 0);
            Mat* provided = ports_value[swap_1];
            if (!provided)
                continue;
            int swap_2 = toplug(j, 1);
            Mat* current = data[swap_2];
            if (!provided->isEmpty()) {
                // This is either a matrix given as input, or directly plugged
                // in the network. In both cases, the old value in 'data' was a
                // null pointer.
                PLASSERT( current == provided );
                data[swap_2] = NULL;
            } else {
                // This is an output we need to copy.
                PLASSERT( current && !current->isEmpty() );
                provided->resize(current->length(), current->width());
                *provided << *current;
            }
        }
    }
}

Here is the call graph for this function:

OptionList & PLearn::NetworkModule::getOptionList ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 64 of file NetworkModule.cc.

OptionMap & PLearn::NetworkModule::getOptionMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 64 of file NetworkModule.cc.

const TVec< string > & PLearn::NetworkModule::getPorts ( ) [virtual]

Return the list of ports in the module.

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 736 of file NetworkModule.cc.

References all_ports.

{
    return all_ports;
}
const TMat< int > & PLearn::NetworkModule::getPortSizes ( ) [virtual]

Return the size of each port.

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 744 of file NetworkModule.cc.

References PLearn::OnlineLearningModule::port_sizes.

{
    return port_sizes;
}
RemoteMethodMap & PLearn::NetworkModule::getRemoteMethodMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 64 of file NetworkModule.cc.

void PLearn::NetworkModule::makeDeepCopyFromShallowCopy ( CopiesMap copies) [virtual]

Transforms a shallow copy into a deep copy.

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 752 of file NetworkModule.cc.

References all_connections, all_mats, all_modules, all_ports, bprop_data, bprop_data_idx, bprop_path, bprop_toresize, connections, PLearn::deepCopyField(), fprop_data, fprop_data_idx, fprop_path, fprop_toplug, fprop_toresize, i, PLearn::TVec< T >::length(), PLearn::OnlineLearningModule::makeDeepCopyFromShallowCopy(), modules, PLASSERT, ports, and PLearn::TVec< T >::resize().

{
    inherited::makeDeepCopyFromShallowCopy(copies);

    deepCopyField(modules, copies);
    deepCopyField(connections, copies);
    deepCopyField(ports, copies);
    deepCopyField(all_modules, copies);
    deepCopyField(all_connections, copies);
    deepCopyField(fprop_path, copies);
    deepCopyField(bprop_path, copies);
    deepCopyField(fprop_data_idx, copies);
    deepCopyField(fprop_toresize, copies);
    deepCopyField(fprop_toplug, copies);
    deepCopyField(bprop_data_idx, copies);
    deepCopyField(bprop_toresize, copies);
    deepCopyField(all_mats, copies);
    deepCopyField(all_ports, copies);

    // Special code to handle the deep copy of 'fprop_data' and 'bprop_data'.
    // A better way may exist to do this!
    fprop_data = TVec< TVec<Mat*> >(fprop_data.length());
    bprop_data = TVec< TVec<Mat*> >(bprop_data.length());
    PLASSERT( fprop_data.length() == bprop_data.length() );
    for (int i = 0; i < fprop_data.length(); i++) {
        TVec<Mat*>& fdata = fprop_data[i];
        TVec<Mat*>& bdata = bprop_data[i];
        const TVec<int>& fdata_idx = fprop_data_idx[i];
        const TVec<int>& bdata_idx = bprop_data_idx[i];
        fdata.resize(fdata_idx.length());
        bdata.resize(bdata_idx.length());
        for (int k = 0; k < fdata_idx.length(); k++) {
            int idx = fdata_idx[k];
            if (idx == -1)
                fdata[k] = NULL;
            else
                fdata[k] = &all_mats[idx];
        }
        for (int k = 0; k < bdata_idx.length(); k++) {
            int idx = bdata_idx[k];
            if (idx == -1)
                bdata[k] = NULL;
            else
                bdata[k] = &all_mats[idx];
        }
    }
}

Here is the call graph for this function:


Member Data Documentation

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 96 of file NetworkModule.h.

Similar to 'all_modules', but for connections in the network.

Definition at line 113 of file NetworkModule.h.

Referenced by build_(), and makeDeepCopyFromShallowCopy().

A list of all matrices used to store the various computation results in the network (i.e.

the outputs of each module).

Definition at line 162 of file NetworkModule.h.

Referenced by build_(), and makeDeepCopyFromShallowCopy().

List of all modules, i.e.

the modules in the 'modules' option, possibly completed with an additional NullModule to save states when the 'save_states' option is true.

Definition at line 110 of file NetworkModule.h.

Referenced by bpropAccUpdate(), build_(), forget(), fprop(), and makeDeepCopyFromShallowCopy().

The list of ports, computed from the 'ports' option.

Definition at line 165 of file NetworkModule.h.

Referenced by build_(), getPorts(), and makeDeepCopyFromShallowCopy().

The i-th element is the list of Mat* pointers being provided to the i-th module in a bprop step.

Definition at line 147 of file NetworkModule.h.

Referenced by bpropAccUpdate(), build_(), and makeDeepCopyFromShallowCopy().

Same as 'bprop_data', but the integer is the index of the matrix in the 'all_mats' vector.

This is used to properly deep copy this object.

Definition at line 151 of file NetworkModule.h.

Referenced by build_(), and makeDeepCopyFromShallowCopy().

Ordered list of modules used when doing a bprop (the integer values correspond to indices in 'modules').

Definition at line 121 of file NetworkModule.h.

Referenced by bpropAccUpdate(), build_(), and makeDeepCopyFromShallowCopy().

The i-th element is the list of matrices that need to be resized to empty matrices prior to calling bpropUpdate() on the i-th module in a bprop step.

The resizing is needed to ensure we correctly compute the desired gradients.

Definition at line 158 of file NetworkModule.h.

Referenced by bpropAccUpdate(), build_(), and makeDeepCopyFromShallowCopy().

Definition at line 56 of file NetworkModule.h.

Referenced by build_(), declareOptions(), and makeDeepCopyFromShallowCopy().

The i-th element is the list of Mat* pointers being provided to the i-th module in a fprop step.

Definition at line 125 of file NetworkModule.h.

Referenced by bpropAccUpdate(), build_(), fprop(), and makeDeepCopyFromShallowCopy().

Same as 'fprop_data', but the integer is the index of the matrix in the 'all_mats' vector.

This is used to properly deep copy this object.

Definition at line 129 of file NetworkModule.h.

Referenced by build_(), and makeDeepCopyFromShallowCopy().

Ordered list of modules used when doing a fprop (the integer values correspond to indices in 'modules').

Definition at line 117 of file NetworkModule.h.

Referenced by bpropAccUpdate(), build_(), fprop(), and makeDeepCopyFromShallowCopy().

The i-th element is the list of correspondances between ports in this NetworkModule and ports in the i-th module of the fprop path.

The list is given by a matrix, where each row is a pair (p1, p2) with p1 the index of the port in this NetworkModule, and p2 the index of the corresponding port in the i-th module during fprop.

Definition at line 143 of file NetworkModule.h.

Referenced by bpropAccUpdate(), build_(), fprop(), and makeDeepCopyFromShallowCopy().

The i-th element is the list of matrices that need to be resized to empty matrices prior to calling fprop() on the i-th module in a fprop step.

The resizing is needed to ensure we correctly compute the desired outputs.

Definition at line 136 of file NetworkModule.h.

Referenced by build_(), fprop(), and makeDeepCopyFromShallowCopy().

Definition at line 55 of file NetworkModule.h.

Referenced by build_(), declareOptions(), and makeDeepCopyFromShallowCopy().

TVec< pair<string, string > > PLearn::NetworkModule::ports

Definition at line 58 of file NetworkModule.h.

Referenced by build_(), declareOptions(), and makeDeepCopyFromShallowCopy().

Definition at line 60 of file NetworkModule.h.

Referenced by build_(), and declareOptions().


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines