PLearn 0.1
Public Member Functions | Static Public Member Functions | Public Attributes | Static Public Attributes | Static Protected Member Functions | Private Types | Private Member Functions
PLearn::HeterogenuousAffineTransformWeightPenalty Class Reference

Penalty associated to an affine transform with continuous and discrete input. More...

#include <HeterogenuousAffineTransformWeightPenalty.h>

Inheritance diagram for PLearn::HeterogenuousAffineTransformWeightPenalty:
Inheritance graph
[legend]
Collaboration diagram for PLearn::HeterogenuousAffineTransformWeightPenalty:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 HeterogenuousAffineTransformWeightPenalty ()
 Default constructor, usually does nothing.
 HeterogenuousAffineTransformWeightPenalty (Var input, VarArray weights, TVec< bool > the_input_is_discrete, real weight_decay, real bias_decay=0, string penalty_type="L2_square")
 Constructor initializing from input variables.
virtual void recomputeSize (int &l, int &w) const
 Recomputes the length l and width w that this variable should have, according to its parent variables.
virtual void fprop ()
 compute output given input
virtual void bprop ()
virtual string classname () const
virtual OptionListgetOptionList () const
virtual OptionMapgetOptionMap () const
virtual RemoteMethodMapgetRemoteMethodMap () const
virtual
HeterogenuousAffineTransformWeightPenalty
deepCopy (CopiesMap &copies) const
virtual void build ()
 Post-constructor.
virtual void makeDeepCopyFromShallowCopy (CopiesMap &copies)
 Transforms a shallow copy into a deep copy.

Static Public Member Functions

static string _classname_ ()
 HeterogenuousAffineTransformWeightPenalty.
static OptionList_getOptionList_ ()
static RemoteMethodMap_getRemoteMethodMap_ ()
static Object_new_instance_for_typemap_ ()
static bool _isa_ (const Object *o)
static void _static_initialize_ ()
static const PPathdeclaringFile ()

Public Attributes

TVec< boolinput_is_discrete
 Indication whether the input components are discrete.
real weight_decay_
 Weight decay parameter.
real bias_decay_
 Bias decay parameter.
string penalty_type_
 Type of weight decay penalty.

Static Public Attributes

static StaticInitializer _static_initializer_

Static Protected Member Functions

static void declareOptions (OptionList &ol)
 Declares the class options.

Private Types

typedef NaryVariable inherited

Private Member Functions

void build_ ()
 This does the actual building.

Detailed Description

Penalty associated to an affine transform with continuous and discrete input.

* HeterogenuousAffineTransformWeightPenalty *

Definition at line 53 of file HeterogenuousAffineTransformWeightPenalty.h.


Member Typedef Documentation

Reimplemented from PLearn::NaryVariable.

Definition at line 55 of file HeterogenuousAffineTransformWeightPenalty.h.


Constructor & Destructor Documentation

PLearn::HeterogenuousAffineTransformWeightPenalty::HeterogenuousAffineTransformWeightPenalty ( )

Default constructor, usually does nothing.

Definition at line 57 of file HeterogenuousAffineTransformWeightPenalty.cc.

    : weight_decay_(0.0), bias_decay_(0.0), penalty_type_("L2_square")
{}
PLearn::HeterogenuousAffineTransformWeightPenalty::HeterogenuousAffineTransformWeightPenalty ( Var  input,
VarArray  weights,
TVec< bool the_input_is_discrete,
real  weight_decay,
real  bias_decay = 0,
string  penalty_type = "L2_square" 
)

Constructor initializing from input variables.

Definition at line 62 of file HeterogenuousAffineTransformWeightPenalty.cc.

References build().

     : inherited(input & weights, 1, 1), input_is_discrete(the_input_is_discrete), 
       weight_decay_(weight_decay),bias_decay_(bias_decay),penalty_type_(penalty_type)
 //     : inherited(input & weights, input->length() != 1 ? weights[0]->width() : 1 , input->width() != 1 ? weights[0]->width() : 1)
 {
     build();
 }

Here is the call graph for this function:


Member Function Documentation

string PLearn::HeterogenuousAffineTransformWeightPenalty::_classname_ ( ) [static]
OptionList & PLearn::HeterogenuousAffineTransformWeightPenalty::_getOptionList_ ( ) [static]

Reimplemented from PLearn::NaryVariable.

Definition at line 55 of file HeterogenuousAffineTransformWeightPenalty.cc.

RemoteMethodMap & PLearn::HeterogenuousAffineTransformWeightPenalty::_getRemoteMethodMap_ ( ) [static]

Reimplemented from PLearn::NaryVariable.

Definition at line 55 of file HeterogenuousAffineTransformWeightPenalty.cc.

bool PLearn::HeterogenuousAffineTransformWeightPenalty::_isa_ ( const Object o) [static]

Reimplemented from PLearn::NaryVariable.

Definition at line 55 of file HeterogenuousAffineTransformWeightPenalty.cc.

Object * PLearn::HeterogenuousAffineTransformWeightPenalty::_new_instance_for_typemap_ ( ) [static]

Reimplemented from PLearn::Object.

Definition at line 55 of file HeterogenuousAffineTransformWeightPenalty.cc.

StaticInitializer HeterogenuousAffineTransformWeightPenalty::_static_initializer_ & PLearn::HeterogenuousAffineTransformWeightPenalty::_static_initialize_ ( ) [static]

Reimplemented from PLearn::NaryVariable.

Definition at line 55 of file HeterogenuousAffineTransformWeightPenalty.cc.

void PLearn::HeterogenuousAffineTransformWeightPenalty::bprop ( ) [virtual]

Implements PLearn::Variable.

Definition at line 144 of file HeterogenuousAffineTransformWeightPenalty.cc.

References bias_decay_, PLearn::TVec< T >::data(), PLearn::fast_exact_is_equal(), PLearn::Variable::gradientdata, i, input_is_discrete, j, PLearn::TVec< T >::last(), PLearn::TVec< T >::length(), n, penalty_type_, PLearn::Variable::row(), PLearn::sqrt(), PLearn::Variable::valuedata, PLearn::NaryVariable::varray, and weight_decay_.

{
    int n = varray[1]->width();
    int l = varray.length()-1;

    if (penalty_type_ == "L1_square")
    {
        real delta;
        if(!fast_exact_is_equal(bias_decay_, 0))
        {
            delta = 2*sqrt(valuedata[0]*bias_decay_)*gradientdata[0];
            for(int j=0; j<n; j++)
                if(varray.last()->valuedata[j] > 0)
                    varray.last()->gradientdata[j] += delta;
                else if(varray.last()->valuedata[j] < 0)
                    varray.last()->gradientdata[j] -= delta;
        }

        delta = 2*sqrt(valuedata[0]*weight_decay_)*gradientdata[0];
        for(int i=1; i<l; i++)
        {
            if(input_is_discrete[i-1])
            {
                int r = (int)varray[0]->valuedata[i-1];
                real* row = varray[i]->matValue.row(r).data();
                real* grow = varray[i]->matGradient.row(r).data();
                for(int j=0; j<n; j++)
                {
                    if(row[j] > 0)
                        grow[j] += delta;
                    else if( row[j] < 0)
                        grow[j] -= delta;
                }
                varray[i]->updateRow(r);
            }
            else
            {
                for(int j=0; j<n; j++)
                    if(varray[i]->valuedata[j] > 0)
                        varray[i]->gradientdata[j] += delta;
                    else if(varray[i]->valuedata[j] < 0)
                        varray[i]->gradientdata[j] -= delta;
            }
        }
    }
    else if (penalty_type_ == "L1")
    {
        real delta;
        if(!fast_exact_is_equal(bias_decay_, 0))
        {
            delta = bias_decay_*gradientdata[0];
            for(int j=0; j<n; j++)
                if(varray.last()->valuedata[j] > 0)
                    varray.last()->gradientdata[j] += delta;
                else if(varray.last()->valuedata[j] < 0)
                    varray.last()->gradientdata[j] -= delta;
        }

        delta = weight_decay_*gradientdata[0];
        for(int i=1; i<l; i++)
        {
            if(input_is_discrete[i-1])
            {
                int r = (int)varray[0]->valuedata[i-1];
                real* row = varray[i]->matValue.row(r).data();
                real* grow = varray[i]->matGradient.row(r).data();
                for(int j=0; j<n; j++)
                {
                    if(row[j] > 0)
                        grow[j] += delta;
                    else if( row[j] < 0)
                        grow[j] -= delta;
                }
                varray[i]->updateRow(r);
            }
            else
            {
                for(int j=0; j<n; j++)
                    if(varray[i]->valuedata[j] > 0)
                        varray[i]->gradientdata[j] += delta;
                    else if(varray[i]->valuedata[j] < 0)
                        varray[i]->gradientdata[j] -= delta;
            }
        }
    }
    else if (penalty_type_ == "L2_square")
    {
        if(!fast_exact_is_equal(bias_decay_, 0))
        {
            for(int j=0; j<n; j++)
                varray.last()->gradientdata[j] += 2*bias_decay_*varray.last()->valuedata[j]*gradientdata[0];
        }

        for(int i=1; i<l; i++)
        {
            if(input_is_discrete[i-1])
            {
                int r = (int)varray[0]->valuedata[i-1];
                real* row = varray[i]->matValue.row((int)varray[0]->valuedata[i-1]).data();
                real* grow = varray[i]->matGradient.row((int)varray[0]->valuedata[i-1]).data();
                for(int j=0; j<n; j++)
                {
                    grow[j] += 2*weight_decay_*row[j]*gradientdata[0];
                }
                varray[i]->updateRow(r);
            }
            else
            {
                for(int j=0; j<n; j++)
                    varray[i]->gradientdata[j] += 2*weight_decay_*varray[i]->valuedata[j]*gradientdata[0];
            }
        }
    }

}

Here is the call graph for this function:

void PLearn::HeterogenuousAffineTransformWeightPenalty::build ( ) [virtual]

Post-constructor.

The normal implementation should call simply inherited::build(), then this class's build_(). This method should be callable again at later times, after modifying some option fields to change the "architecture" of the object.

Reimplemented from PLearn::NaryVariable.

Definition at line 260 of file HeterogenuousAffineTransformWeightPenalty.cc.

References PLearn::NaryVariable::build(), and build_().

Referenced by HeterogenuousAffineTransformWeightPenalty().

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::HeterogenuousAffineTransformWeightPenalty::build_ ( ) [private]

This does the actual building.

Reimplemented from PLearn::NaryVariable.

Definition at line 297 of file HeterogenuousAffineTransformWeightPenalty.cc.

References PLearn::Variable::allowPartialUpdates(), i, input_is_discrete, PLearn::Variable::isVec(), PLearn::TVec< T >::length(), PLearn::lowerstring(), penalty_type_, PLERROR, PLWARNING, PLearn::TVec< T >::size(), PLearn::Variable::size(), PLearn::NaryVariable::varray, and PLearn::Variable::width().

Referenced by build().

{
    if(varray[0]->size() != varray.length()-2)
        PLERROR("In HeterogenuousAffineTransformWeightPenalty::build_(): The number of weight variables (%d) and input size (%d) is not the same", varray.length()-2, varray[0]->size());
    if(input_is_discrete.length() != varray[0]->size())
        PLERROR("In HeterogenuousAffineTransformWeightPenalty::build_(): input_is_discrete size (%d) and input size (%d) does not match", input_is_discrete.length(), varray[0]->size());
    if(!varray[0]->isVec())
        PLERROR("In HeterogenuousAffineTransformWeightPenalty::build_(): input should be a vector");
    for(int i=1; i<varray.length(); i++)
    {
        if(varray[i]->width() != varray[1]->width())
            PLERROR("In HeterogenuousAffineTransformWeightPenalty::build_(): %dth weight matrix has width %d, should be %d", i, varray[i]->width(), size());
        if(i<varray.length()-1 && input_is_discrete[i-1])
            varray[i-1]->allowPartialUpdates();
    }

    string pt = lowerstring( penalty_type_ );
    if( pt == "l1" )
        penalty_type_ = "L1";
    else if( pt == "l1_square" || pt == "l1 square" || pt == "l1square" )
        penalty_type_ = "L1_square";
    else if( pt == "l2_square" || pt == "l2 square" || pt == "l2square" )
        penalty_type_ = "L2_square";
    else if( pt == "l2" )
    {
        PLWARNING("In HeterogenuousAffineTransformWeightPenalty::build_(): L2 penalty not supported, assuming you want L2 square");
        penalty_type_ = "L2_square";
        }
    else
        PLERROR("In HeterogenuousAffineTransformWeightPenalty::build_(): penalty_type_ \"%s\" not supported", penalty_type_.c_str());
}

Here is the call graph for this function:

Here is the caller graph for this function:

string PLearn::HeterogenuousAffineTransformWeightPenalty::classname ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 55 of file HeterogenuousAffineTransformWeightPenalty.cc.

void PLearn::HeterogenuousAffineTransformWeightPenalty::declareOptions ( OptionList ol) [static, protected]

Declares the class options.

Reimplemented from PLearn::NaryVariable.

Definition at line 274 of file HeterogenuousAffineTransformWeightPenalty.cc.

References bias_decay_, PLearn::OptionBase::buildoption, PLearn::declareOption(), PLearn::NaryVariable::declareOptions(), input_is_discrete, penalty_type_, and weight_decay_.

{    
    declareOption(ol, "input_is_discrete", &HeterogenuousAffineTransformWeightPenalty::input_is_discrete,
                  OptionBase::buildoption,
                  "Indication whether each component of the input is discrete or not.");
    declareOption(ol, "weight_decay_", &HeterogenuousAffineTransformWeightPenalty::weight_decay_,
                  OptionBase::buildoption,
                  "Weight decay parameter.");
    declareOption(ol, "bias_decay_", &HeterogenuousAffineTransformWeightPenalty::bias_decay_,
                  OptionBase::buildoption,
                  "Bias decay parameter.");
    declareOption(ol, "penalty_type_", &HeterogenuousAffineTransformWeightPenalty::penalty_type_,
                  OptionBase::buildoption,
                  "Penalty to use on the weights.\n"
                  "Can be any of:\n"
                  "  - \"L1\": L1 norm,\n"
                  "  - \"L1_square\": square of the L1 norm,\n"
                  "  - \"L2_square\" (default): square of the L2 norm.\n");

    // Now call the parent class' declareOptions
    inherited::declareOptions(ol);
}

Here is the call graph for this function:

static const PPath& PLearn::HeterogenuousAffineTransformWeightPenalty::declaringFile ( ) [inline, static]

Reimplemented from PLearn::NaryVariable.

Definition at line 92 of file HeterogenuousAffineTransformWeightPenalty.h.

:
    //#####  Protected Member Functions  ######################################
HeterogenuousAffineTransformWeightPenalty * PLearn::HeterogenuousAffineTransformWeightPenalty::deepCopy ( CopiesMap copies) const [virtual]

Reimplemented from PLearn::NaryVariable.

Definition at line 55 of file HeterogenuousAffineTransformWeightPenalty.cc.

void PLearn::HeterogenuousAffineTransformWeightPenalty::fprop ( ) [virtual]

compute output given input

Implements PLearn::Variable.

Definition at line 75 of file HeterogenuousAffineTransformWeightPenalty.cc.

References bias_decay_, PLearn::TVec< T >::data(), PLearn::fast_exact_is_equal(), i, input_is_discrete, j, PLearn::TVec< T >::last(), PLearn::TVec< T >::length(), n, penalty_type_, PLearn::Variable::row(), PLearn::sqrt(), PLearn::square_f(), PLearn::sumabs(), PLearn::sumsquare(), PLearn::Variable::valuedata, PLearn::NaryVariable::varray, and weight_decay_.

{
    int n = varray[1]->width();
    int l = varray.length()-1;

    if (penalty_type_ == "L1_square")
    {
        if(!fast_exact_is_equal(bias_decay_, 0)) valuedata[0] = sqrt(fabs(bias_decay_))*sumabs(varray.last()->value);
        else valuedata[0] = 0;

        for(int i=1; i<l; i++)
        {
            if(input_is_discrete[i-1])
            {
                real* row = varray[i]->matValue.row((int)varray[0]->valuedata[i-1]).data();
                for(int j=0; j<n; j++)
                    valuedata[0] += sqrt(fabs(weight_decay_))*fabs(*row++);
            }
            else
            {
                for(int j=0; j<n; j++)
                    valuedata[0] += sqrt(fabs(weight_decay_))*fabs(varray[i]->valuedata[j]);
            }
        }
        valuedata[0] *= valuedata[0];
    }
    else if (penalty_type_ == "L1")
    {
        if(!fast_exact_is_equal(bias_decay_, 0)) valuedata[0] = bias_decay_*sumabs(varray.last()->value);
        else valuedata[0] = 0;

        for(int i=1; i<l; i++)
        {
            if(input_is_discrete[i-1])
            {
                real* row = varray[i]->matValue.row((int)varray[0]->valuedata[i-1]).data();
                for(int j=0; j<n; j++)
                    valuedata[0] += weight_decay_*fabs(*row++);
            }
            else
            {
                for(int j=0; j<n; j++)
                    valuedata[0] += weight_decay_*fabs(varray[i]->valuedata[j]);
            }
        }
    }
    else if (penalty_type_ == "L2_square")
    {
        if(!fast_exact_is_equal(bias_decay_, 0)) valuedata[0] = bias_decay_*sumsquare(varray.last()->value);
        else valuedata[0] = 0;

        for(int i=1; i<l; i++)
        {
            if(input_is_discrete[i-1])
            {
                real* row = varray[i]->matValue.row((int)varray[0]->valuedata[i-1]).data();
                for(int j=0; j<n; j++)
                    valuedata[0] += weight_decay_*square_f(*row++);
            }
            else
            {
                for(int j=0; j<n; j++)
                    valuedata[0] += weight_decay_*square_f(varray[i]->valuedata[j]);
            }
        }
    }

}

Here is the call graph for this function:

OptionList & PLearn::HeterogenuousAffineTransformWeightPenalty::getOptionList ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 55 of file HeterogenuousAffineTransformWeightPenalty.cc.

OptionMap & PLearn::HeterogenuousAffineTransformWeightPenalty::getOptionMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 55 of file HeterogenuousAffineTransformWeightPenalty.cc.

RemoteMethodMap & PLearn::HeterogenuousAffineTransformWeightPenalty::getRemoteMethodMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 55 of file HeterogenuousAffineTransformWeightPenalty.cc.

void PLearn::HeterogenuousAffineTransformWeightPenalty::makeDeepCopyFromShallowCopy ( CopiesMap copies) [virtual]

Transforms a shallow copy into a deep copy.

Reimplemented from PLearn::NaryVariable.

Definition at line 266 of file HeterogenuousAffineTransformWeightPenalty.cc.

References PLearn::deepCopyField(), input_is_discrete, and PLearn::NaryVariable::makeDeepCopyFromShallowCopy().

{
    inherited::makeDeepCopyFromShallowCopy(copies);

    deepCopyField(input_is_discrete, copies);
    //PLERROR("HeterogenuousAffineTransformWeightPenalty::makeDeepCopyFromShallowCopy not fully (correctly) implemented yet!");
}

Here is the call graph for this function:

void PLearn::HeterogenuousAffineTransformWeightPenalty::recomputeSize ( int l,
int w 
) const [virtual]

Recomputes the length l and width w that this variable should have, according to its parent variables.

This is used for ex. by sizeprop() The default version stupidly returns the current dimensions, so make sure to overload it in subclasses if this is not appropriate.

Reimplemented from PLearn::Variable.

Definition at line 70 of file HeterogenuousAffineTransformWeightPenalty.cc.

{    
    l = w = 1;
}

Member Data Documentation

Reimplemented from PLearn::NaryVariable.

Definition at line 92 of file HeterogenuousAffineTransformWeightPenalty.h.

Bias decay parameter.

Definition at line 65 of file HeterogenuousAffineTransformWeightPenalty.h.

Referenced by bprop(), declareOptions(), and fprop().

Indication whether the input components are discrete.

Definition at line 61 of file HeterogenuousAffineTransformWeightPenalty.h.

Referenced by bprop(), build_(), declareOptions(), fprop(), and makeDeepCopyFromShallowCopy().

Type of weight decay penalty.

Definition at line 67 of file HeterogenuousAffineTransformWeightPenalty.h.

Referenced by bprop(), build_(), declareOptions(), and fprop().

Weight decay parameter.

Definition at line 63 of file HeterogenuousAffineTransformWeightPenalty.h.

Referenced by bprop(), declareOptions(), and fprop().


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines