PLearn 0.1
Public Member Functions | Static Public Member Functions | Static Public Attributes | Protected Attributes | Private Types
PLearn::AffineTransformWeightPenalty Class Reference

Weight decay terms for affine transforms. More...

#include <AffineTransformWeightPenalty.h>

Inheritance diagram for PLearn::AffineTransformWeightPenalty:
Inheritance graph
[legend]
Collaboration diagram for PLearn::AffineTransformWeightPenalty:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 AffineTransformWeightPenalty ()
 Default constructor for persistence.
 AffineTransformWeightPenalty (Variable *affinetransform, real weight_decay, real bias_decay=0., string penalty_type="L2_square")
virtual string classname () const
virtual OptionListgetOptionList () const
virtual OptionMapgetOptionMap () const
virtual RemoteMethodMapgetRemoteMethodMap () const
virtual
AffineTransformWeightPenalty
deepCopy (CopiesMap &copies) const
virtual void recomputeSize (int &l, int &w) const
 Recomputes the length l and width w that this variable should have, according to its parent variables.
virtual void fprop ()
 Nothing to do by default.
virtual void bprop ()
 Nothing to do by default.

Static Public Member Functions

static string _classname_ ()
static OptionList_getOptionList_ ()
static RemoteMethodMap_getRemoteMethodMap_ ()
static Object_new_instance_for_typemap_ ()
static bool _isa_ (const Object *o)
static void _static_initialize_ ()
static const PPathdeclaringFile ()
static void declareOptions (OptionList &ol)
 Declare options (data fields) for the class.

Static Public Attributes

static StaticInitializer _static_initializer_

Protected Attributes

real weight_decay_
real bias_decay_
string penalty_type_

Private Types

typedef UnaryVariable inherited

Detailed Description

Weight decay terms for affine transforms.

Definition at line 54 of file AffineTransformWeightPenalty.h.


Member Typedef Documentation

Reimplemented from PLearn::UnaryVariable.

Definition at line 56 of file AffineTransformWeightPenalty.h.


Constructor & Destructor Documentation

PLearn::AffineTransformWeightPenalty::AffineTransformWeightPenalty ( ) [inline]

Default constructor for persistence.

Definition at line 65 of file AffineTransformWeightPenalty.h.

References PLearn::lowerstring(), PLERROR, and PLWARNING.

        : weight_decay_(0.0), bias_decay_(0.0), penalty_type_("L2_square")
    {
        string pt = lowerstring( penalty_type_ );
        if( pt == "l1" )
            penalty_type_ = "L1";
        else if( pt == "l1_square" || pt == "l1 square" || pt == "l1square" )
            penalty_type_ = "L1_square";
        else if( pt == "l2_square" || pt == "l2 square" || pt == "l2square" )
            penalty_type_ = "L2_square";
        else if( pt == "l2" )
        {
            PLWARNING("L2 penalty not supported, assuming you want L2 square");
            penalty_type_ = "L2_square";
        }
        else
            PLERROR("penalty_type_ \"%s\" not supported", penalty_type_.c_str());
    }

Here is the call graph for this function:

PLearn::AffineTransformWeightPenalty::AffineTransformWeightPenalty ( Variable affinetransform,
real  weight_decay,
real  bias_decay = 0.,
string  penalty_type = "L2_square" 
) [inline]

Definition at line 84 of file AffineTransformWeightPenalty.h.

References PLearn::lowerstring(), PLERROR, and PLWARNING.

        : inherited(affinetransform, 1,1),weight_decay_(weight_decay),bias_decay_(bias_decay),penalty_type_(penalty_type)
    {
        string pt = lowerstring( penalty_type_ );
        if( pt == "l1" )
            penalty_type_ = "L1";
        else if( pt == "l1_square" || pt == "l1 square" || pt == "l1square" )
            penalty_type_ = "L1_square";
        else if( pt == "l2_square" || pt == "l2 square" || pt == "l2square" )
            penalty_type_ = "L2_square";
        else if( pt == "l2" )
        {
            PLWARNING("L2 penalty not supported, assuming you want L2 square");
            penalty_type_ = "L2_square";
        }
        else
            PLERROR("penalty_type_ \"%s\" not supported", penalty_type_.c_str());
    }

Here is the call graph for this function:


Member Function Documentation

string PLearn::AffineTransformWeightPenalty::_classname_ ( ) [static]

Reimplemented from PLearn::UnaryVariable.

Definition at line 52 of file AffineTransformWeightPenalty.cc.

OptionList & PLearn::AffineTransformWeightPenalty::_getOptionList_ ( ) [static]

Reimplemented from PLearn::UnaryVariable.

Definition at line 52 of file AffineTransformWeightPenalty.cc.

RemoteMethodMap & PLearn::AffineTransformWeightPenalty::_getRemoteMethodMap_ ( ) [static]

Reimplemented from PLearn::UnaryVariable.

Definition at line 52 of file AffineTransformWeightPenalty.cc.

bool PLearn::AffineTransformWeightPenalty::_isa_ ( const Object o) [static]

Reimplemented from PLearn::UnaryVariable.

Definition at line 52 of file AffineTransformWeightPenalty.cc.

Object * PLearn::AffineTransformWeightPenalty::_new_instance_for_typemap_ ( ) [static]

Reimplemented from PLearn::UnaryVariable.

Definition at line 52 of file AffineTransformWeightPenalty.cc.

StaticInitializer AffineTransformWeightPenalty::_static_initializer_ & PLearn::AffineTransformWeightPenalty::_static_initialize_ ( ) [static]

Reimplemented from PLearn::UnaryVariable.

Definition at line 52 of file AffineTransformWeightPenalty.cc.

void PLearn::AffineTransformWeightPenalty::bprop ( ) [virtual]

Nothing to do by default.

Reimplemented from PLearn::UnaryVariable.

Definition at line 99 of file AffineTransformWeightPenalty.cc.

References PLearn::fast_exact_is_equal(), i, PLearn::multiplyAcc(), n, PLERROR, PLearn::sqrt(), PLearn::two(), and w.

{
    int l = input->length() - 1;
    if ( penalty_type_ == "L1_square" )
    {
        if (!input->matGradient.isCompact())
            PLERROR("AffineTransformWeightPenalty::bprop, L1_square penalty currently not handling non-compact weight matrix");
        int n=input->width();
        if (!fast_exact_is_equal(weight_decay_, 0))
        {
            real delta = 2*sqrt(valuedata[0]*weight_decay_)*gradientdata[0];
            real* w = input->matValue[1];
            real* d_w = input->matGradient[1];
            int tot = l * n; // Number of weights to update.
            for (int i = 0; i < tot; i++) {
                if (w[i] > 0)
                    d_w[i] += delta;
                else if (w[i] < 0)
                    d_w[i] -= delta;
            }
        }
        if(!fast_exact_is_equal(bias_decay_, 0))
        {
            real delta = 2*sqrt(valuedata[0]*bias_decay_)*gradientdata[0];
            real* d_biases = input->matGradient[0];
            real* biases = input->matValue[0];
            for (int i=0;i<n;i++) {
                if (biases[i]>0)
                    d_biases[i] += delta;
                else if (biases[i]<0)
                    d_biases[i] -= delta;
            }
        }
    }
    else if ( penalty_type_ == "L1")
    {
        if (!input->matGradient.isCompact())
            PLERROR("AffineTransformWeightPenalty::bprop, L1 penalty currently not handling non-compact weight matrix");
        int n=input->width();
        if (!fast_exact_is_equal(weight_decay_, 0) && l > 0)
        {
            real delta = weight_decay_ * gradientdata[0];
            real* w = input->matValue[1];
            real* d_w = input->matGradient[1];
            int tot = l * n; // Number of weights to update.
            for (int i = 0; i < tot; i++) {
                if (w[i] > 0)
                    d_w[i] += delta;
                else if (w[i] < 0)
                    d_w[i] -= delta;
            }
        }
        if(!fast_exact_is_equal(bias_decay_, 0) && l >= 0)
        {
            real delta = bias_decay_ * gradientdata[0];
            real* d_biases = input->matGradient[0];
            real* biases = input->matValue[0];
            for (int i=0;i<n;i++)
                if (biases[i]>0)
                    d_biases[i] += delta;
                else if (biases[i]<0)
                    d_biases[i] -= delta;
        }
    }
    else if (penalty_type_ == "L2_square" )
    {
        multiplyAcc(input->matGradient.subMatRows(1,l), input->matValue.subMatRows(1,l), two(weight_decay_)*gradientdata[0]);
        if(!fast_exact_is_equal(bias_decay_, 0))
            multiplyAcc(input->matGradient(0), input->matValue(0), two(bias_decay_)*gradientdata[0]);
    }
}

Here is the call graph for this function:

string PLearn::AffineTransformWeightPenalty::classname ( ) const [virtual]

Reimplemented from PLearn::UnaryVariable.

Definition at line 52 of file AffineTransformWeightPenalty.cc.

void PLearn::AffineTransformWeightPenalty::declareOptions ( OptionList ol) [static]

Declare options (data fields) for the class.

Redefine this in subclasses: call declareOption(...) for each option, and then call inherited::declareOptions(options). Please call the inherited method AT THE END to get the options listed in a consistent order (from most recently defined to least recently defined).

  static void MyDerivedClass::declareOptions(OptionList& ol)
  {
      declareOption(ol, "inputsize", &MyObject::inputsize_,
                    OptionBase::buildoption,
                    "The size of the input; it must be provided");
      declareOption(ol, "weights", &MyObject::weights,
                    OptionBase::learntoption,
                    "The learned model weights");
      inherited::declareOptions(ol);
  }
Parameters:
olList of options that is progressively being constructed for the current class.

Reimplemented from PLearn::UnaryVariable.

Definition at line 58 of file AffineTransformWeightPenalty.cc.

References bias_decay_, PLearn::OptionBase::buildoption, PLearn::declareOption(), penalty_type_, and weight_decay_.

Here is the call graph for this function:

static const PPath& PLearn::AffineTransformWeightPenalty::declaringFile ( ) [inline, static]

Reimplemented from PLearn::UnaryVariable.

Definition at line 103 of file AffineTransformWeightPenalty.h.

{ return new AffineTransformWeightPenalty(transformation, weight_decay, bias_decay, penalty_type); } 
AffineTransformWeightPenalty * PLearn::AffineTransformWeightPenalty::deepCopy ( CopiesMap copies) const [virtual]

Reimplemented from PLearn::UnaryVariable.

Definition at line 52 of file AffineTransformWeightPenalty.cc.

void PLearn::AffineTransformWeightPenalty::fprop ( ) [virtual]

Nothing to do by default.

Reimplemented from PLearn::UnaryVariable.

Definition at line 65 of file AffineTransformWeightPenalty.cc.

References PLearn::fast_exact_is_equal(), PLearn::sqrt(), PLearn::sumabs(), and PLearn::sumsquare().

{
    if (penalty_type_ == "L1_square")
    {
        if (input->length()>1)
            valuedata[0] = sqrt(fabs(weight_decay_))*sumabs(input->matValue.subMatRows(1,input->length()-1));
        else
            valuedata[0] = 0;
        if(!fast_exact_is_equal(bias_decay_, 0))
            valuedata[0] += sqrt(fabs(bias_decay_))*sumabs(input->matValue(0));

        valuedata[0] *= valuedata[0];
    }
    else if (penalty_type_ == "L1")
    {
        if (input->length()>1)
            valuedata[0] = weight_decay_*sumabs(input->matValue.subMatRows(1,input->length()-1));
        else 
            valuedata[0] = 0;
        if(!fast_exact_is_equal(bias_decay_, 0))
            valuedata[0] += bias_decay_*sumabs(input->matValue(0));
    }
    else if (penalty_type_ == "L2_square")
    {
        if (input->length()>1)
            valuedata[0] = weight_decay_*sumsquare(input->matValue.subMatRows(1,input->length()-1));
        else 
            valuedata[0] = 0;
        if(!fast_exact_is_equal(bias_decay_, 0))
            valuedata[0] += bias_decay_*sumsquare(input->matValue(0));
    }
}

Here is the call graph for this function:

OptionList & PLearn::AffineTransformWeightPenalty::getOptionList ( ) const [virtual]

Reimplemented from PLearn::UnaryVariable.

Definition at line 52 of file AffineTransformWeightPenalty.cc.

OptionMap & PLearn::AffineTransformWeightPenalty::getOptionMap ( ) const [virtual]

Reimplemented from PLearn::UnaryVariable.

Definition at line 52 of file AffineTransformWeightPenalty.cc.

RemoteMethodMap & PLearn::AffineTransformWeightPenalty::getRemoteMethodMap ( ) const [virtual]

Reimplemented from PLearn::UnaryVariable.

Definition at line 52 of file AffineTransformWeightPenalty.cc.

void PLearn::AffineTransformWeightPenalty::recomputeSize ( int l,
int w 
) const [virtual]

Recomputes the length l and width w that this variable should have, according to its parent variables.

This is used for ex. by sizeprop() The default version stupidly returns the current dimensions, so make sure to overload it in subclasses if this is not appropriate.

Reimplemented from PLearn::Variable.

Definition at line 54 of file AffineTransformWeightPenalty.cc.

{ l=1; w=1; }

Member Data Documentation

Reimplemented from PLearn::UnaryVariable.

Definition at line 103 of file AffineTransformWeightPenalty.h.

Definition at line 60 of file AffineTransformWeightPenalty.h.

Referenced by declareOptions().

Definition at line 61 of file AffineTransformWeightPenalty.h.

Referenced by declareOptions().

Definition at line 59 of file AffineTransformWeightPenalty.h.

Referenced by declareOptions().


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines