PLearn 0.1
Public Member Functions | Static Public Member Functions | Public Attributes | Static Public Attributes | Static Protected Member Functions | Private Types | Private Member Functions | Private Attributes
PLearn::Convolution2DModule Class Reference

Apply convolution filters on (possibly multiple) 2D inputs (images). More...

#include <Convolution2DModule.h>

Inheritance diagram for PLearn::Convolution2DModule:
Inheritance graph
[legend]
Collaboration diagram for PLearn::Convolution2DModule:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 Convolution2DModule ()
 Default constructor.
virtual void fprop (const Vec &input, Vec &output) const
 given the input, compute the output (possibly resize it appropriately)
virtual void fprop (const TVec< Mat * > &ports_value)
 New version.
virtual void bpropUpdate (const Vec &input, const Vec &output, Vec &input_gradient, const Vec &output_gradient, bool accumulate=false)
 Adapt based on the output gradient: this method should only be called just after a corresponding fprop; it should be called with the same arguments as fprop for the first two arguments (and output should not have been modified since then).
virtual void bpropAccUpdate (const TVec< Mat * > &ports_value, const TVec< Mat * > &ports_gradient)
 New version.
virtual void bbpropUpdate (const Vec &input, const Vec &output, Vec &input_gradient, const Vec &output_gradient, Vec &input_diag_hessian, const Vec &output_diag_hessian, bool accumulate=false)
 Similar to bpropUpdate, but adapt based also on the estimation of the diagonal of the Hessian matrix, and propagates this back.
virtual void forget ()
 reset the parameters to the state they would be BEFORE starting training.
virtual string classname () const
virtual OptionListgetOptionList () const
virtual OptionMapgetOptionMap () const
virtual RemoteMethodMapgetRemoteMethodMap () const
virtual Convolution2DModuledeepCopy (CopiesMap &copies) const
virtual void build ()
 Post-constructor.
virtual void makeDeepCopyFromShallowCopy (CopiesMap &copies)
 Transforms a shallow copy into a deep copy.

Static Public Member Functions

static string _classname_ ()
 optionally perform some processing after training, or after a series of fprop/bpropUpdate calls to prepare the model for truly out-of-sample operation.
static OptionList_getOptionList_ ()
static RemoteMethodMap_getRemoteMethodMap_ ()
static Object_new_instance_for_typemap_ ()
static bool _isa_ (const Object *o)
static void _static_initialize_ ()
static const PPathdeclaringFile ()

Public Attributes

int n_input_images
 ### declare public option fields (such as build options) here Start your comments with Doxygen-compatible comments such as //!
int input_images_length
 Length of each of the input images.
int input_images_width
 Width of each of the input images.
int n_output_images
 Number of output images to put in the output vector.
int kernel_length
 Length of each filter (or kernel) applied on an input image.
int kernel_width
 Width of each filter (or kernel) applied on an input image.
int kernel_step1
 Horizontal step of the kernels.
int kernel_step2
 Vertical step of the kernels.
TMat< intconnection_matrix
 Matrix of connections: it has n_input_images rows and n_output_images columns, each output image will only be connected to a subset of the input images, where a non-zero value is present in this matrix.
real start_learning_rate
 Starting learning-rate, by which we multiply the gradient step.
real decrease_constant
 learning_rate = start_learning_rate / (1 + decrease_constant*t), where t is the number of updates since the beginning
TMat< Matkernels
 Contains the kernels between input and output images.
Vec bias
 Contains the bias of the output images.
int output_images_length
 Length of the output images.
int output_images_width
 Width of the output images.
int input_images_size
 Size of the input images (length * width)
int output_images_size
 Size of the input images (length * width)
int kernel_size
 Size of the input images (length * width)

Static Public Attributes

static StaticInitializer _static_initializer_

Static Protected Member Functions

static void declareOptions (OptionList &ol)
 Declares the class options.

Private Types

typedef OnlineLearningModule inherited

Private Member Functions

void build_ ()
 This does the actual building.
void build_kernels ()
 Build the kernels.

Private Attributes

real learning_rate
int step_number
TVec< Matinput_images
TVec< Matoutput_images
TVec< Matinput_gradients
TVec< Matoutput_gradients
TVec< Matinput_diag_hessians
TVec< Matoutput_diag_hessians
Mat kernel_gradient
TMat< Matkernel_gradients
Mat squared_kernel
TVec< string > ports

Detailed Description

Apply convolution filters on (possibly multiple) 2D inputs (images).

Definition at line 51 of file Convolution2DModule.h.


Member Typedef Documentation

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 53 of file Convolution2DModule.h.


Constructor & Destructor Documentation

PLearn::Convolution2DModule::Convolution2DModule ( )

Member Function Documentation

string PLearn::Convolution2DModule::_classname_ ( ) [static]

optionally perform some processing after training, or after a series of fprop/bpropUpdate calls to prepare the model for truly out-of-sample operation.

THE DEFAULT IMPLEMENTATION PROVIDED IN THE SUPER-CLASS DOES NOT DO ANYTHING. in case bpropUpdate does not do anything, make it known THE DEFAULT IMPLEMENTATION PROVIDED IN THE SUPER-CLASS RETURNS false;

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 54 of file Convolution2DModule.cc.

OptionList & PLearn::Convolution2DModule::_getOptionList_ ( ) [static]

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 54 of file Convolution2DModule.cc.

RemoteMethodMap & PLearn::Convolution2DModule::_getRemoteMethodMap_ ( ) [static]

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 54 of file Convolution2DModule.cc.

bool PLearn::Convolution2DModule::_isa_ ( const Object o) [static]

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 54 of file Convolution2DModule.cc.

Object * PLearn::Convolution2DModule::_new_instance_for_typemap_ ( ) [static]

Reimplemented from PLearn::Object.

Definition at line 54 of file Convolution2DModule.cc.

StaticInitializer Convolution2DModule::_static_initializer_ & PLearn::Convolution2DModule::_static_initialize_ ( ) [static]

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 54 of file Convolution2DModule.cc.

void PLearn::Convolution2DModule::bbpropUpdate ( const Vec input,
const Vec output,
Vec input_gradient,
const Vec output_gradient,
Vec input_diag_hessian,
const Vec output_diag_hessian,
bool  accumulate = false 
) [virtual]

Similar to bpropUpdate, but adapt based also on the estimation of the diagonal of the Hessian matrix, and propagates this back.

If these methods are defined, you can use them INSTEAD of bpropUpdate(...) N.B. A DEFAULT IMPLEMENTATION IS PROVIDED IN THE SUPER-CLASS, WHICH JUST CALLS bbpropUpdate(input, output, input_gradient, output_gradient, out_hess, in_hess) AND IGNORES INPUT HESSIAN AND INPUT GRADIENT. this version allows to obtain the input gradient and diag_hessian

If these methods are defined, you can use them INSTEAD of bpropUpdate(...)

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 704 of file Convolution2DModule.cc.

References PLearn::backConvolve2D(), bpropUpdate(), PLearn::TVec< T >::clear(), connection_matrix, i, input_diag_hessians, input_images_length, input_images_size, input_images_width, PLearn::OnlineLearningModule::input_size, j, kernel_step1, kernel_step2, kernels, n_input_images, n_output_images, output_diag_hessians, output_images_length, output_images_size, output_images_width, PLearn::OnlineLearningModule::output_size, PLASSERT_MSG, PLERROR, PLearn::TVec< T >::resize(), PLearn::TVec< T >::size(), squared_kernel, PLearn::TVec< T >::subVec(), and PLearn::TVec< T >::toMat().

{
    // This version forwards the second order information, but does not
    // actually use it for the update.

    // Check size
    if( output_diag_hessian.size() != output_size )
        PLERROR("Convolution2DModule::bbpropUpdate: output_diag_hessian.size()"
                "\n"
                "should be equal to output_size (%i != %i).\n",
                output_diag_hessian.size(), output_size);

    if( accumulate )
    {
        PLASSERT_MSG( input_diag_hessian.size() == input_size,
                      "Cannot resize input_diag_hessian AND accumulate into it"
                    );
    }
    else
    {
        input_diag_hessian.resize(input_size);
        input_diag_hessian.clear();
    }

    // Make input_diag_hessians and output_diag_hessians point to the right
    // places
    for( int i=0 ; i<n_input_images ; i++ )
        input_diag_hessians[i] =
            input_diag_hessian.subVec(i*input_images_size, input_images_size)
                .toMat( input_images_length, input_images_width );

    for( int j=0 ; j<n_output_images ; j++ )
        output_diag_hessians[j] =
            output_diag_hessian.subVec(j*output_images_size,output_images_size)
                .toMat( output_images_length, output_images_width );

    // Propagates to input_diag_hessian
    for( int j=0 ; j<n_output_images ; j++ )
        for( int i=0 ; j<n_input_images ; i++ )
            if( connection_matrix(i,j) != 0 )
            {
                squared_kernel << kernels(i,j);
                squared_kernel *= kernels(i,j); // term-to-term product

                backConvolve2D( input_diag_hessians[i], squared_kernel,
                                output_diag_hessians[j],
                                kernel_step1, kernel_step2, true );
            }

    // Call bpropUpdate()
    bpropUpdate( input, output, input_gradient, output_gradient, accumulate );
}

Here is the call graph for this function:

void PLearn::Convolution2DModule::bpropAccUpdate ( const TVec< Mat * > &  ports_value,
const TVec< Mat * > &  ports_gradient 
) [virtual]

New version.

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 505 of file Convolution2DModule.cc.

References bias, PLearn::TMat< T >::clear(), connection_matrix, PLearn::convolve2Dbackprop(), decrease_constant, i, input_gradients, input_images, input_images_length, input_images_size, input_images_width, PLearn::TMat< T >::isEmpty(), j, kernel_gradients, kernel_length, kernel_step1, kernel_step2, kernel_width, kernels, learning_rate, PLearn::TMat< T >::length(), PLearn::TVec< T >::length(), PLearn::multiplyAcc(), n_input_images, n_output_images, PLearn::OnlineLearningModule::nPorts(), output_gradients, output_images, output_images_length, output_images_size, output_images_width, PLearn::Profiler::pl_profile_end(), PLearn::Profiler::pl_profile_start(), PLASSERT, PLCHECK_MSG, PLearn::OnlineLearningModule::port_sizes, PLearn::TMat< T >::resize(), start_learning_rate, step_number, PLearn::sum(), PLearn::TVec< T >::toMat(), and PLearn::TMat< T >::width().

{
    Profiler::pl_profile_start("Convolution2DModule::bpropAccUpdate");
    PLASSERT( ports_value.length() == nPorts()
              && ports_gradient.length() == nPorts() );

    Mat* input = ports_value[0];
    Mat* output = ports_value[1];
    Mat* input_grad = ports_gradient[0];
    Mat* output_grad = ports_gradient[1];

    // If we have output_grad and we want to update
    if( output_grad && !output_grad->isEmpty()
        && (!input_grad || input_grad->isEmpty() ) )
    {
        // If we have to compute input_grad
        bool compute_input_grad = false;
        if( input_grad )
            compute_input_grad = true;

        PLASSERT( input );
        PLASSERT( output );

        PLASSERT( input->width() == port_sizes(0,1) );
        PLASSERT( output->width() == port_sizes(1,1) );
        PLASSERT( output_grad->width() == port_sizes(1,1) );
        if( compute_input_grad )
            PLASSERT( input_grad->width() == port_sizes(0,1) );

        int batch_size = input->length();
        PLASSERT( output->length() == batch_size );
        PLASSERT( output_grad->length() == batch_size );

        learning_rate = start_learning_rate /
            (1.+decrease_constant*step_number);
        real avg_lr = learning_rate / batch_size;
        if( compute_input_grad )
            input_grad->resize(batch_size, port_sizes(0,1));

        // clear kernel gradient
        for( int i=0; i<n_input_images; i++ )
            for( int j=0; j<n_output_images; j++ )
                if( connection_matrix(i,j) != 0 )
                {
                    kernel_gradients(i,j).resize(kernel_length, kernel_width);
                    kernel_gradients(i,j).clear();
                }

        // TODO: optimize
        if( compute_input_grad )
            Profiler::pl_profile_start("convolve2Dbackprop");
        else
            Profiler::pl_profile_start("convolve2Dbackprop (update only)");

        for( int k=0; k<batch_size; k++ )
        {
            for( int i=0; i<n_input_images; i++ )
            {
                input_images[i] = (*input)(k)
                    .subVec(i*input_images_size, input_images_size)
                    .toMat(input_images_length, input_images_width);

                if( compute_input_grad )
                    input_gradients[i] = (*input_grad)(k)
                        .subVec(i*input_images_size, input_images_size)
                        .toMat(input_images_length, input_images_width);
            }

            for( int j=0; j<n_output_images; j++ )
            {
                output_images[j] = (*output)(k)
                    .subVec(j*output_images_size, output_images_size)
                    .toMat(output_images_length, output_images_width);
                output_gradients[j] = (*output_grad)(k)
                    .subVec(j*output_images_size, output_images_size)
                    .toMat(output_images_length, output_images_width);
            }

            for( int j=0; j<n_output_images; j++ )
                for( int i=0; i<n_input_images; i++ )
                    if( connection_matrix(i,j) != 0 )
                    {
                        if( compute_input_grad )
                            convolve2Dbackprop( input_images[i],
                                                kernels(i,j),
                                                output_gradients[j],
                                                input_gradients[i],
                                                kernel_gradients(i,j),
                                                kernel_step1, kernel_step2,
                                                true );
                        else
                            convolve2Dbackprop( input_images[i],
                                                output_gradients[j],
                                                kernel_gradients(i,j),
                                                kernel_step1, kernel_step2,
                                                true );
                    }
        }

        if( compute_input_grad )
            Profiler::pl_profile_end("convolve2Dbackprop");
        else
            Profiler::pl_profile_end("convolve2Dbackprop (update only)");

        for( int j=0; j<n_output_images; j++ )
        {
            for( int i=0; i<n_input_images; i++ )
                if( connection_matrix(i,j) != 0 )
                    multiplyAcc(kernels(i,j), kernel_gradients(i,j), -avg_lr);

            bias[j] -= avg_lr * sum( (*output_grad)
                .subMatColumns(j*output_images_size, output_images_size) );
        }
    }
    else if( !input_grad
             && output_grad && !output_grad->isEmpty() )
    {
        PLASSERT( input && !input->isEmpty() );
        PLASSERT( output && !output->isEmpty() );
    }
    else if( !input_grad && !output_grad )
    {
        PLASSERT( !input || !input->isEmpty() );
        PLASSERT( !output || !output->isEmpty() );
    }
    else
        PLCHECK_MSG( false, "Port configuration not implemented" );

    Profiler::pl_profile_end("Convolution2DModule::bpropAccUpdate");
}

Here is the call graph for this function:

void PLearn::Convolution2DModule::bpropUpdate ( const Vec input,
const Vec output,
Vec input_gradient,
const Vec output_gradient,
bool  accumulate = false 
) [virtual]

Adapt based on the output gradient: this method should only be called just after a corresponding fprop; it should be called with the same arguments as fprop for the first two arguments (and output should not have been modified since then).

this version allows to obtain the input gradient as well

Since sub-classes are supposed to learn ONLINE, the object is 'ready-to-be-used' just after any bpropUpdate. N.B. A DEFAULT IMPLEMENTATION IS PROVIDED IN THE SUPER-CLASS, WHICH JUST CALLS bpropUpdate(input, output, input_gradient, output_gradient) AND IGNORES INPUT GRADIENT. this version allows to obtain the input gradient as well N.B. THE DEFAULT IMPLEMENTATION IN SUPER-CLASS JUST RAISES A PLERROR.

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 441 of file Convolution2DModule.cc.

References bias, PLearn::TMat< T >::clear(), PLearn::TVec< T >::clear(), connection_matrix, PLearn::convolve2Dbackprop(), decrease_constant, i, input_gradients, input_images, input_images_length, input_images_size, input_images_width, PLearn::OnlineLearningModule::input_size, j, kernel_gradient, kernel_step1, kernel_step2, kernels, learning_rate, PLearn::multiplyAcc(), n_input_images, n_output_images, output_gradients, output_images_length, output_images_size, output_images_width, PLearn::OnlineLearningModule::output_size, PLASSERT_MSG, PLERROR, PLearn::TVec< T >::resize(), PLearn::TVec< T >::size(), start_learning_rate, step_number, PLearn::TVec< T >::subVec(), PLearn::sum(), and PLearn::TVec< T >::toMat().

Referenced by bbpropUpdate().

{
    // Check size
    if( input.size() != input_size )
        PLERROR("Convolution2DModule::bpropUpdate: input.size() should be\n"
                "equal to input_size (%i != %i).\n", input.size(), input_size);
    if( output.size() != output_size )
        PLERROR("Convolution2DModule::bpropUpdate: output.size() should be\n"
                "equal to output_size (%i != %i).\n",
                output.size(), output_size);
    if( output_gradient.size() != output_size )
        PLERROR("Convolution2DModule::bpropUpdate: output_gradient.size()"
                " should be\n"
                "equal to output_size (%i != %i).\n",
                output_gradient.size(), output_size);

    if( accumulate )
    {
        PLASSERT_MSG( input_gradient.size() == input_size,
                      "Cannot resize input_gradient AND accumulate into it" );
    }
    else
    {
        input_gradient.resize(input_size);
        input_gradient.clear();
    }

    // Since fprop() has just been called, we assume that input_images and
    // output_images are up-to-date
    // Make input_gradients and output_gradients point to the right places
    for( int i=0 ; i<n_input_images ; i++ )
        input_gradients[i] =
            input_gradient.subVec(i*input_images_size, input_images_size)
                .toMat( input_images_length, input_images_width );

    for( int j=0 ; j<n_output_images ; j++ )
        output_gradients[j] =
            output_gradient.subVec(j*output_images_size, output_images_size)
                .toMat( output_images_length, output_images_width );

    // Do the actual bprop and update
    learning_rate = start_learning_rate / (1+decrease_constant*step_number);
    for( int j=0 ; j<n_output_images ; j++ )
    {
        for( int i=0 ; i<n_input_images ; i++ )
            if( connection_matrix(i,j) != 0 )
            {
                kernel_gradient.clear();
                convolve2Dbackprop( input_images[i], kernels(i,j),
                                    output_gradients[j],
                                    input_gradients[i], kernel_gradient,
                                    kernel_step1, kernel_step2, true );

                // kernel(i,j) -= learning_rate * kernel_gradient
                multiplyAcc( kernels(i,j), kernel_gradient, -learning_rate ); // could be more efficiently done within the convolve2Dbackprop
            }
        bias[j] -= learning_rate * sum( output_gradients[j] );
    }

}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::Convolution2DModule::build ( ) [virtual]

Post-constructor.

The normal implementation should call simply inherited::build(), then this class's build_(). This method should be callable again at later times, after modifying some option fields to change the "architecture" of the object.

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 314 of file Convolution2DModule.cc.

References PLearn::OnlineLearningModule::build(), and build_().

Here is the call graph for this function:

void PLearn::Convolution2DModule::build_ ( ) [private]

This does the actual building.

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 182 of file Convolution2DModule.cc.

References bias, build_kernels(), PLearn::TMat< T >::column(), connection_matrix, PLearn::endl(), PLearn::TMat< T >::fill(), input_diag_hessians, input_gradients, input_images, input_images_length, input_images_size, input_images_width, PLearn::OnlineLearningModule::input_size, kernel_length, kernel_size, kernel_step1, kernel_step2, kernel_width, PLearn::TMat< T >::length(), n_input_images, n_output_images, PLearn::OnlineLearningModule::nPorts(), output_diag_hessians, output_gradients, output_images, output_images_length, output_images_size, output_images_width, PLearn::OnlineLearningModule::output_size, PLERROR, PLearn::OnlineLearningModule::port_sizes, ports, PLearn::TMat< T >::resize(), PLearn::TVec< T >::resize(), and PLearn::TMat< T >::width().

Referenced by build().

{
    MODULE_LOG << "build_() called" << endl;

    // Verify the parameters
    if( n_input_images < 1 )
        PLERROR("Convolution2DModule::build_: 'n_input_images' < 1 (%i).\n",
                n_input_images);

    if( input_images_length < 0 )
        PLERROR("Convolution2DModule::build_: 'input_images_length'<0 (%i).\n",
                input_images_length);

    if( input_images_width < 0 )
        PLERROR("Convolution2DModule::build_: 'input_images_width'<0 (%i).\n",
                input_images_width);

    if( n_output_images < 1 )
        PLERROR("Convolution2DModule::build_: 'n_output_images' < 1 (%i).\n",
                n_input_images);

    if( kernel_length < 0 )
        PLERROR("Convolution2DModule::build_: 'kernel_length'<0 (%i).\n",
                kernel_length);

    if( kernel_width < 0 )
        PLERROR("Convolution2DModule::build_: 'kernel_width'<0 (%i).\n",
                kernel_width);

    if( kernel_step1 < 0 )
        PLERROR("Convolution2DModule::build_: 'kernel_step1'<0 (%i).\n",
                kernel_step1);

    if( kernel_step2 < 0 )
        PLERROR("Convolution2DModule::build_: 'kernel_step2'<0 (%i).\n",
                kernel_step2);

    if( (input_images_length - kernel_length) % kernel_step1 != 0 )
        PLERROR("Convolution2DModule::build_:\n"
                "the difference (input_images_length - kernel_length) (%i)\n"
                "should be a multiple of kernel_step1 (%i).\n",
                (input_images_length - kernel_length), kernel_step1);

    if( (input_images_width - kernel_width) % kernel_step2 != 0 )
        PLERROR("Convolution2DModule::build_:\n"
                "the difference (input_images_width - kernel_width) (%i)\n"
                "should be a multiple of kernel_step2 (%i).\n",
                (input_images_width - kernel_width), kernel_step2);

    // Build the learntoptions from the buildoptions
    input_images_size = input_images_length * input_images_width;
    input_size = n_input_images * input_images_size;

    output_images_length = (input_images_length-kernel_length)/kernel_step1+1;
    output_images_width = (input_images_width - kernel_width)/kernel_step2+1;
    output_images_size = output_images_length * output_images_width;
    output_size = n_output_images * output_images_size;

    kernel_size = kernel_length * kernel_width;

    bias.resize(n_output_images);

    // If connection_matrix was not specified, or inconsistently,
    // make it a matrix full of ones.
    if( connection_matrix.length() != n_input_images
        || connection_matrix.width() != n_output_images )
    {
        connection_matrix.resize(n_input_images, n_output_images);
        connection_matrix.fill(1);
    }

    build_kernels();

    input_images.resize(n_input_images);
    output_images.resize(n_output_images);
    input_gradients.resize(n_input_images);
    output_gradients.resize(n_output_images);
    input_diag_hessians.resize(n_input_images);
    output_diag_hessians.resize(n_output_images);

    // port stuff
    ports.resize(2);
    ports[0] = "input";
    ports[1] = "output";

    port_sizes.resize(nPorts(), 2);
    port_sizes.column(0).fill(-1);
    port_sizes(0, 1) = input_size;
    port_sizes(1, 1) = output_size;
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::Convolution2DModule::build_kernels ( ) [private]

Build the kernels.

Definition at line 273 of file Convolution2DModule.cc.

References connection_matrix, forget(), i, j, kernel_gradient, kernel_gradients, kernel_length, kernel_width, kernels, PLearn::TMat< T >::length(), n_input_images, n_output_images, PLearn::TMat< T >::resize(), PLearn::TMat< T >::size(), squared_kernel, and PLearn::TMat< T >::width().

Referenced by build_().

{
    // If kernels has the right size, for all i and j kernel(i,j) exists iff
    // connection_matrix(i,j) !=0, and has the appropriate size, then we don't
    // want to forget them.
    bool need_rebuild = false;
    if( kernels.length() != n_input_images
        || kernels.width() != n_output_images )
    {
        need_rebuild = true;
    }
    else
    {
        for( int i=0 ; i<n_input_images ; i++ )
            for( int j=0 ; j<n_output_images ; j++ )
            {
                if( connection_matrix(i,j) == 0 )
                {
                    if( kernels(i,j).size() != 0 )
                    {
                        need_rebuild = true;
                        break;
                    }
                }
                else if( kernels(i,j).length() != kernel_length
                         || kernels(i,j).width() != kernel_width )
                {
                    need_rebuild = true;
                    break;
                }
            }
    }

    if( need_rebuild )
        forget();

    kernel_gradient.resize(kernel_length, kernel_width);
    kernel_gradients.resize(n_input_images, n_output_images);
    squared_kernel.resize(kernel_length, kernel_width);
}

Here is the call graph for this function:

Here is the caller graph for this function:

string PLearn::Convolution2DModule::classname ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 54 of file Convolution2DModule.cc.

void PLearn::Convolution2DModule::declareOptions ( OptionList ol) [static, protected]

Declares the class options.

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 77 of file Convolution2DModule.cc.

References bias, PLearn::OptionBase::buildoption, connection_matrix, PLearn::declareOption(), PLearn::OnlineLearningModule::declareOptions(), decrease_constant, input_images_length, input_images_width, PLearn::OnlineLearningModule::input_size, kernel_length, kernel_step1, kernel_step2, kernel_width, kernels, PLearn::OptionBase::learntoption, n_input_images, n_output_images, output_images_length, output_images_width, PLearn::OnlineLearningModule::output_size, PLearn::redeclareOption(), and start_learning_rate.

{
    // declareOption(ol, "myoption", &Convolution2DModule::myoption,
    //               OptionBase::buildoption,
    //               "Help text describing this option");

    declareOption(ol, "n_input_images", &Convolution2DModule::n_input_images,
                  OptionBase::buildoption,
                  "Number of input images present at the same time in the"
                  " input vector");

    declareOption(ol, "input_images_length",
                  &Convolution2DModule::input_images_length,
                  OptionBase::buildoption,
                  "Length of each of the input images");

    declareOption(ol, "input_images_width",
                  &Convolution2DModule::input_images_width,
                  OptionBase::buildoption,
                  "Width of each of the input images");

    declareOption(ol, "n_output_images", &Convolution2DModule::n_output_images,
                  OptionBase::buildoption,
                  "Number of output images to put in the output vector");

    declareOption(ol, "kernel_length", &Convolution2DModule::kernel_length,
                  OptionBase::buildoption,
                  "Length of each filter (or kernel) applied on an input image"
                  );

    declareOption(ol, "kernel_width", &Convolution2DModule::kernel_width,
                  OptionBase::buildoption,
                  "Width of each filter (or kernel) applied on an input image"
                  );

    declareOption(ol, "kernel_step1", &Convolution2DModule::kernel_step1,
                  OptionBase::buildoption,
                  "Horizontal step of the kernels");

    declareOption(ol, "kernel_step2", &Convolution2DModule::kernel_step2,
                  OptionBase::buildoption,
                  "Vertical step of the kernels");

    declareOption(ol, "connection_matrix",
                  &Convolution2DModule::connection_matrix,
                  OptionBase::buildoption,
                  "Matrix of connections:\n"
                  "it has n_input_images rows and n_output_images columns,\n"
                  "each output image will only be connected to a subset of"
                  " the\n"
                  "input images, where a non-zero value is present in this"
                  " matrix.\n"
                  "If this matrix is not provided, it will be fully"
                  " connected.\n"
                  );

    declareOption(ol, "start_learning_rate",
                  &Convolution2DModule::start_learning_rate,
                  OptionBase::buildoption,
                  "Starting learning-rate, by which we multiply the gradient"
                  " step"
                  );

    declareOption(ol, "decrease_constant",
                  &Convolution2DModule::decrease_constant,
                  OptionBase::buildoption,
                  "learning_rate = start_learning_rate / (1 +"
                  " decrease_constant*t),\n"
                  "where t is the number of updates since the beginning\n"
                  );

    declareOption(ol, "output_images_length",
                  &Convolution2DModule::output_images_length,
                  OptionBase::learntoption,
                  "Length of the output images");

    declareOption(ol, "output_images_width",
                  &Convolution2DModule::output_images_width,
                  OptionBase::learntoption,
                  "Width of the output images");

    declareOption(ol, "kernels", &Convolution2DModule::kernels,
                  OptionBase::learntoption,
                  "Contains the kernels between input and output images");

    declareOption(ol, "bias", &Convolution2DModule::bias,
                  OptionBase::learntoption,
                  "Contains the bias of the output images");


    // Now call the parent class' declareOptions
    inherited::declareOptions(ol);

    // Redeclare some of the parent's options as learntoptions
    redeclareOption(ol, "input_size", &Convolution2DModule::input_size,
                    OptionBase::learntoption,
                    "Size of the input, computed from n_input_images,\n"
                    "n_input_length and n_input_width.\n");

    redeclareOption(ol, "output_size", &Convolution2DModule::output_size,
                    OptionBase::learntoption,
                    "Size of the output, computed from n_output_images,\n"
                    "n_output_length and n_output_width.\n");
}

Here is the call graph for this function:

static const PPath& PLearn::Convolution2DModule::declaringFile ( ) [inline, static]

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 206 of file Convolution2DModule.h.

:
    //#####  Protected Member Functions  ######################################
Convolution2DModule * PLearn::Convolution2DModule::deepCopy ( CopiesMap copies) const [virtual]

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 54 of file Convolution2DModule.cc.

void PLearn::Convolution2DModule::forget ( ) [virtual]

reset the parameters to the state they would be BEFORE starting training.

Note that this method is necessarily called from build().

Implements PLearn::OnlineLearningModule.

Definition at line 639 of file Convolution2DModule.cc.

References bias, PLearn::TVec< T >::clear(), connection_matrix, i, j, kernel_length, kernel_width, kernels, n_input_images, n_output_images, PLWARNING, PLearn::OnlineLearningModule::random_gen, and PLearn::TMat< T >::resize().

Referenced by build_kernels().

{
    bias.clear();
    if( !random_gen )
    {
        PLWARNING( "Convolution2DModule: cannot forget() without random_gen" );
        return;
    }

    real scale_factor = 1./(kernel_length*kernel_width*n_input_images);
    kernels.resize( n_input_images, n_output_images );
    for( int i=0 ; i<n_input_images ; i++ )
        for( int j=0 ; j<n_output_images ; j++ )
        {
            if( connection_matrix(i,j) == 0 )
                kernels(i,j).resize(0,0);
            else
            {
                kernels(i,j).resize(kernel_length, kernel_width);
                random_gen->fill_random_uniform( kernels(i,j),
                                                 -scale_factor,
                                                 scale_factor );
            }
        }
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::Convolution2DModule::fprop ( const TVec< Mat * > &  ports_value) [virtual]

New version.

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 372 of file Convolution2DModule.cc.

References bias, connection_matrix, PLearn::convolve2D(), PLearn::TVec< T >::fill(), i, input_images, input_images_length, input_images_size, input_images_width, PLearn::TMat< T >::isEmpty(), j, kernel_step1, kernel_step2, kernels, PLearn::TVec< T >::length(), n_input_images, n_output_images, PLearn::OnlineLearningModule::nPorts(), output_images, output_images_length, output_images_size, output_images_width, PLearn::Profiler::pl_profile_end(), PLearn::Profiler::pl_profile_start(), PLASSERT, PLCHECK_MSG, PLearn::OnlineLearningModule::port_sizes, PLearn::TMat< T >::resize(), and PLearn::TVec< T >::toMat().

{
    Profiler::pl_profile_start( "Convolution2DModule::fprop" );
    PLASSERT( ports_value.length() == nPorts() );

    Mat* input = ports_value[0];
    Mat* output = ports_value[1];

    if( input && !input->isEmpty() && output && output->isEmpty() )
    {
        PLASSERT( input->width() == port_sizes(0,1) );

        int batch_size = input->length();
        output->resize(batch_size, port_sizes(1,1));

        Profiler::pl_profile_start( "convolve2D" );
        // TODO: optimize
        for( int k=0; k<batch_size; k++ )
        {
            for( int i=0; i<n_input_images; i++ )
                input_images[i] = (*input)(k)
                    .subVec(i*input_images_size, input_images_size)
                    .toMat(input_images_length, input_images_width);

            for( int j=0; j<n_output_images; j++ )
                output_images[j] = (*output)(k)
                    .subVec(j*output_images_size, output_images_size)
                    .toMat(output_images_length, output_images_width);

            for( int j=0; j<n_output_images; j++ )
            {
                output_images[j].fill( bias[j] );
                for( int i=0; i<n_input_images; i++ )
                    if( connection_matrix(i,j) != 0 )
                        convolve2D( input_images[i], kernels(i,j),
                                    output_images[j], kernel_step1,
                                    kernel_step2, true );
            }
        }
        Profiler::pl_profile_end( "convolve2D" );
    }
    else if (!input && !output)
    {
        // Nothing to do
    }
    else
        PLCHECK_MSG( false, "Unknown port configuration" );

    Profiler::pl_profile_end( "Convolution2DModule::fprop" );
}

Here is the call graph for this function:

void PLearn::Convolution2DModule::fprop ( const Vec input,
Vec output 
) const [virtual]

given the input, compute the output (possibly resize it appropriately)

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 340 of file Convolution2DModule.cc.

References bias, connection_matrix, PLearn::convolve2D(), PLearn::TVec< T >::fill(), i, input_images, input_images_length, input_images_size, input_images_width, PLearn::OnlineLearningModule::input_size, j, kernel_step1, kernel_step2, kernels, n_input_images, n_output_images, output_images, output_images_length, output_images_size, output_images_width, PLearn::OnlineLearningModule::output_size, PLERROR, PLearn::TVec< T >::resize(), PLearn::TVec< T >::size(), PLearn::TVec< T >::subVec(), and PLearn::TVec< T >::toMat().

{
    // Check size
    if( input.size() != input_size )
        PLERROR("Convolution2DModule::fprop: input.size() should be equal to\n"
                "input_size (%i != %i).\n", input.size(), input_size);
    output.resize(output_size);

    // Make input_images and output_images point to the right places
    for( int i=0 ; i<n_input_images ; i++ )
        input_images[i] =
            input.subVec(i*input_images_size, input_images_size)
                .toMat( input_images_length, input_images_width );

    for( int j=0 ; j<n_output_images ; j++ )
        output_images[j] =
            output.subVec(j*output_images_size, output_images_size)
                .toMat( output_images_length, output_images_width );

    // Compute the values of the output_images
    for( int j=0 ; j<n_output_images ; j++ )
    {
        output_images[j].fill( bias[j] );
        for( int i=0 ; i<n_input_images ; i++ )
        {
            if( connection_matrix(i,j) != 0 )
                convolve2D( input_images[i], kernels(i,j), output_images[j],
                            kernel_step1, kernel_step2, true );
        }
    }
}

Here is the call graph for this function:

OptionList & PLearn::Convolution2DModule::getOptionList ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 54 of file Convolution2DModule.cc.

OptionMap & PLearn::Convolution2DModule::getOptionMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 54 of file Convolution2DModule.cc.

RemoteMethodMap & PLearn::Convolution2DModule::getRemoteMethodMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 54 of file Convolution2DModule.cc.

void PLearn::Convolution2DModule::makeDeepCopyFromShallowCopy ( CopiesMap copies) [virtual]

Member Data Documentation

Reimplemented from PLearn::OnlineLearningModule.

Definition at line 206 of file Convolution2DModule.h.

Contains the bias of the output images.

Definition at line 106 of file Convolution2DModule.h.

Referenced by bpropAccUpdate(), bpropUpdate(), build_(), declareOptions(), forget(), fprop(), and makeDeepCopyFromShallowCopy().

Matrix of connections: it has n_input_images rows and n_output_images columns, each output image will only be connected to a subset of the input images, where a non-zero value is present in this matrix.

If this matrix is not provided, it will be fully connected.

Definition at line 90 of file Convolution2DModule.h.

Referenced by bbpropUpdate(), bpropAccUpdate(), bpropUpdate(), build_(), build_kernels(), declareOptions(), forget(), fprop(), and makeDeepCopyFromShallowCopy().

learning_rate = start_learning_rate / (1 + decrease_constant*t), where t is the number of updates since the beginning

Definition at line 97 of file Convolution2DModule.h.

Referenced by bpropAccUpdate(), bpropUpdate(), and declareOptions().

Definition at line 243 of file Convolution2DModule.h.

Referenced by bbpropUpdate(), build_(), and makeDeepCopyFromShallowCopy().

Length of each of the input images.

Definition at line 65 of file Convolution2DModule.h.

Referenced by bbpropUpdate(), bpropAccUpdate(), bpropUpdate(), build_(), declareOptions(), and fprop().

Size of the input images (length * width)

Definition at line 116 of file Convolution2DModule.h.

Referenced by bbpropUpdate(), bpropAccUpdate(), bpropUpdate(), build_(), and fprop().

Width of each of the input images.

Definition at line 68 of file Convolution2DModule.h.

Referenced by bbpropUpdate(), bpropAccUpdate(), bpropUpdate(), build_(), declareOptions(), and fprop().

Definition at line 247 of file Convolution2DModule.h.

Referenced by bpropUpdate(), build_kernels(), and makeDeepCopyFromShallowCopy().

Definition at line 248 of file Convolution2DModule.h.

Referenced by bpropAccUpdate(), and build_kernels().

Length of each filter (or kernel) applied on an input image.

Definition at line 74 of file Convolution2DModule.h.

Referenced by bpropAccUpdate(), build_(), build_kernels(), declareOptions(), and forget().

Size of the input images (length * width)

Definition at line 122 of file Convolution2DModule.h.

Referenced by build_().

Horizontal step of the kernels.

Definition at line 80 of file Convolution2DModule.h.

Referenced by bbpropUpdate(), bpropAccUpdate(), bpropUpdate(), build_(), declareOptions(), and fprop().

Vertical step of the kernels.

Definition at line 83 of file Convolution2DModule.h.

Referenced by bbpropUpdate(), bpropAccUpdate(), bpropUpdate(), build_(), declareOptions(), and fprop().

Width of each filter (or kernel) applied on an input image.

Definition at line 77 of file Convolution2DModule.h.

Referenced by bpropAccUpdate(), build_(), build_kernels(), declareOptions(), and forget().

Contains the kernels between input and output images.

Definition at line 103 of file Convolution2DModule.h.

Referenced by bbpropUpdate(), bpropAccUpdate(), bpropUpdate(), build_kernels(), declareOptions(), forget(), fprop(), and makeDeepCopyFromShallowCopy().

Definition at line 234 of file Convolution2DModule.h.

Referenced by bpropAccUpdate(), and bpropUpdate().

### declare public option fields (such as build options) here Start your comments with Doxygen-compatible comments such as //!

Number of input images present at the same time in the input vector

Definition at line 62 of file Convolution2DModule.h.

Referenced by bbpropUpdate(), bpropAccUpdate(), bpropUpdate(), build_(), build_kernels(), declareOptions(), forget(), and fprop().

Number of output images to put in the output vector.

Definition at line 71 of file Convolution2DModule.h.

Referenced by bbpropUpdate(), bpropAccUpdate(), bpropUpdate(), build_(), build_kernels(), declareOptions(), forget(), and fprop().

Definition at line 244 of file Convolution2DModule.h.

Referenced by bbpropUpdate(), build_(), and makeDeepCopyFromShallowCopy().

Definition at line 240 of file Convolution2DModule.h.

Referenced by bpropAccUpdate(), build_(), fprop(), and makeDeepCopyFromShallowCopy().

Length of the output images.

Definition at line 110 of file Convolution2DModule.h.

Referenced by bbpropUpdate(), bpropAccUpdate(), bpropUpdate(), build_(), declareOptions(), and fprop().

Size of the input images (length * width)

Definition at line 119 of file Convolution2DModule.h.

Referenced by bbpropUpdate(), bpropAccUpdate(), bpropUpdate(), build_(), and fprop().

Width of the output images.

Definition at line 113 of file Convolution2DModule.h.

Referenced by bbpropUpdate(), bpropAccUpdate(), bpropUpdate(), build_(), declareOptions(), and fprop().

Definition at line 253 of file Convolution2DModule.h.

Referenced by build_().

Definition at line 251 of file Convolution2DModule.h.

Referenced by bbpropUpdate(), build_kernels(), and makeDeepCopyFromShallowCopy().

Starting learning-rate, by which we multiply the gradient step.

Definition at line 93 of file Convolution2DModule.h.

Referenced by bpropAccUpdate(), bpropUpdate(), and declareOptions().

Definition at line 235 of file Convolution2DModule.h.

Referenced by bpropAccUpdate(), and bpropUpdate().


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines