PLearn 0.1
|
This variables computes a max functions (softmax, log-softmax, hardmax, etc., determined by the field computation_type) on subvectors of the input, which lenght is defined by the field groupsizes. More...
#include <MultiMaxVariable.h>
Public Member Functions | |
MultiMaxVariable () | |
Default constructor, usually does nothing. | |
MultiMaxVariable (Variable *input, TVec< int > groupsizes, char computation_type='S') | |
Constructor. | |
MultiMaxVariable (Variable *input, int groupsizes, char computation_type='S') | |
virtual void | recomputeSize (int &l, int &w) const |
Recomputes the length l and width w that this variable should have, according to its parent variables. | |
virtual void | fprop () |
Nothing to do by default. | |
virtual void | bprop () |
Nothing to do by default. | |
virtual string | classname () const |
virtual OptionList & | getOptionList () const |
virtual OptionMap & | getOptionMap () const |
virtual RemoteMethodMap & | getRemoteMethodMap () const |
virtual MultiMaxVariable * | deepCopy (CopiesMap &copies) const |
virtual void | build () |
Post-constructor. | |
virtual void | makeDeepCopyFromShallowCopy (CopiesMap &copies) |
Transforms a shallow copy into a deep copy. | |
Static Public Member Functions | |
static string | _classname_ () |
MultiMaxVariable. | |
static OptionList & | _getOptionList_ () |
static RemoteMethodMap & | _getRemoteMethodMap_ () |
static Object * | _new_instance_for_typemap_ () |
static bool | _isa_ (const Object *o) |
static void | _static_initialize_ () |
static const PPath & | declaringFile () |
Public Attributes | |
TVec< int > | groupsizes |
### declare public option fields (such as build options) here Start your comments with Doxygen-compatible comments such as //! | |
char | computation_type |
int | groupsize |
Static Public Attributes | |
static StaticInitializer | _static_initializer_ |
Static Protected Member Functions | |
static void | declareOptions (OptionList &ol) |
Declares the class options. | |
Private Types | |
typedef UnaryVariable | inherited |
Private Member Functions | |
void | build_ () |
This does the actual building. | |
Static Private Member Functions | |
static void | softmax_range (Vec &x, Vec &y, int start, int length) |
static void | logSoftmax_range (Vec &x, Vec &y, int start, int length) |
static void | hardMax_range (Vec &x, Vec &y, int start, int length, bool value=false) |
static void | bpropSoftMax (Vec &gradientInput, Vec &gradient, Vec &variableValue, int start, int length) |
static void | bpropLogSoftMax (Vec &gradientInput, Vec &gradient, Vec &variableValue, int start, int length) |
static void | bpropHardMaxValue (Vec &gradientInput, Vec &gradient, Vec &variableValue, int start, int length) |
This variables computes a max functions (softmax, log-softmax, hardmax, etc., determined by the field computation_type) on subvectors of the input, which lenght is defined by the field groupsizes.
* MultiMaxVariable *
ex : if groupsizes = [1,2,3], and computation_type = 'S' (for softmax), and the input vector [1,2,3,4,5,6], the result will be [softmax([1]), softmax([2,3]), softmax([4,5,6])]
note : in that example matValue.width() of the variable must be 1+2+3=6
Definition at line 65 of file MultiMaxVariable.h.
typedef UnaryVariable PLearn::MultiMaxVariable::inherited [private] |
Reimplemented from PLearn::UnaryVariable.
Definition at line 67 of file MultiMaxVariable.h.
PLearn::MultiMaxVariable::MultiMaxVariable | ( | ) | [inline] |
Default constructor, usually does nothing.
Definition at line 87 of file MultiMaxVariable.h.
:computation_type('S'), groupsize(-1) {}
PLearn::MultiMaxVariable::MultiMaxVariable | ( | Variable * | input, |
TVec< int > | groupsizes, | ||
char | computation_type = 'S' |
||
) |
Constructor.
Definition at line 61 of file MultiMaxVariable.cc.
References build_().
: inherited(input, input->length(), input->width()), groupsizes(groupsizes), computation_type(computation_type) { build_(); }
PLearn::MultiMaxVariable::MultiMaxVariable | ( | Variable * | input, |
int | groupsizes, | ||
char | computation_type = 'S' |
||
) |
Definition at line 70 of file MultiMaxVariable.cc.
References build_().
: inherited(input, input->length(), input->width()), computation_type(computation_type), groupsize(groupsize) { build_(); }
string PLearn::MultiMaxVariable::_classname_ | ( | ) | [static] |
Reimplemented from PLearn::UnaryVariable.
Definition at line 57 of file MultiMaxVariable.cc.
OptionList & PLearn::MultiMaxVariable::_getOptionList_ | ( | ) | [static] |
Reimplemented from PLearn::UnaryVariable.
Definition at line 57 of file MultiMaxVariable.cc.
RemoteMethodMap & PLearn::MultiMaxVariable::_getRemoteMethodMap_ | ( | ) | [static] |
Reimplemented from PLearn::UnaryVariable.
Definition at line 57 of file MultiMaxVariable.cc.
Reimplemented from PLearn::UnaryVariable.
Definition at line 57 of file MultiMaxVariable.cc.
Object * PLearn::MultiMaxVariable::_new_instance_for_typemap_ | ( | ) | [static] |
Reimplemented from PLearn::UnaryVariable.
Definition at line 57 of file MultiMaxVariable.cc.
StaticInitializer MultiMaxVariable::_static_initializer_ & PLearn::MultiMaxVariable::_static_initialize_ | ( | ) | [static] |
Reimplemented from PLearn::UnaryVariable.
Definition at line 57 of file MultiMaxVariable.cc.
void PLearn::MultiMaxVariable::bprop | ( | ) | [virtual] |
Nothing to do by default.
Reimplemented from PLearn::UnaryVariable.
Definition at line 145 of file MultiMaxVariable.cc.
References bpropHardMaxValue(), bpropLogSoftMax(), bpropSoftMax(), computation_type, groupsizes, i, PLearn::UnaryVariable::input, PLearn::TVec< T >::length(), PLearn::TMat< T >::length(), PLearn::Variable::matGradient, PLearn::Variable::matValue, n, and PLERROR.
{ int k; Mat inputGradient = input->matGradient; int l = inputGradient.length(); Vec inputGradient_n; Vec value_n; Vec gradient_n; for(int n=0; n<l; n++) { inputGradient_n = inputGradient(n); value_n = matValue(n); gradient_n = matGradient(n); k=0; for(int i=0; i<groupsizes.length(); i++) { switch(computation_type) { //softmax case 'S': bpropSoftMax(inputGradient_n, gradient_n, value_n, k, groupsizes[i]); break; //log_softmax case 'L': //ici aussi j'ai tout copié, en changeant seulement les "bornes" de la sommation bpropLogSoftMax(inputGradient_n, gradient_n, value_n, k, groupsizes[i]); break; //hardmax_value case 'H': bpropHardMaxValue(inputGradient_n, gradient_n, value_n, k, groupsizes[i]); break; //hardmax case 'h': PLERROR("computation_type 'h' not implemented yet"); break; //random_softmax_value case 'R': PLERROR("computation_type 'R' not implemented yet"); break; //random_softmax case 'r': PLERROR("computation_type 'r' not implemented yet"); break; default : PLERROR("unable to bprop because of invalid computation_type"); } k+=groupsizes[i]; } } }
void PLearn::MultiMaxVariable::bpropHardMaxValue | ( | Vec & | gradientInput, |
Vec & | gradient, | ||
Vec & | variableValue, | ||
int | start, | ||
int | length | ||
) | [static, private] |
Definition at line 356 of file MultiMaxVariable.cc.
References i, PLearn::Variable::length(), and PLearn::sum().
Referenced by bprop().
{ real sum=0.; for(int i=start; i<start+length; i++) sum += gradient[i]; for(int i=start; i<start+length; i++) if(variableValue[i] != 0) gradientInput[i] += sum; }
void PLearn::MultiMaxVariable::bpropLogSoftMax | ( | Vec & | gradientInput, |
Vec & | gradient, | ||
Vec & | variableValue, | ||
int | start, | ||
int | length | ||
) | [static, private] |
Definition at line 346 of file MultiMaxVariable.cc.
References i, PLearn::Variable::length(), PLearn::safeexp(), and PLearn::sum().
Referenced by bprop().
{ real sum=0.; for (int i = start; i < start+length; i++) sum += gradient[i]; for (int i = start; i < start+length; ++i) gradientInput[i] += gradient[i] - sum * safeexp(variableValue[i]); }
void PLearn::MultiMaxVariable::bpropSoftMax | ( | Vec & | gradientInput, |
Vec & | gradient, | ||
Vec & | variableValue, | ||
int | start, | ||
int | length | ||
) | [static, private] |
Definition at line 327 of file MultiMaxVariable.cc.
References i, j, and PLearn::Variable::length().
Referenced by bprop().
{ //on parcout le gradient de notre vecteur for(int i=start; i<start+length; i++) { //et on rajoute un petit qqch pour chacun du gradient de l'input for(int j=start; j<start+length; j++) { //note ici jai juste copié ce quil y avait avant dans le bprob de softmaxVariable... if(i==j) gradientInput[i] += gradient[j]*variableValue[i]*(1.-variableValue[i]); else gradientInput[i] -= gradient[j]*variableValue[i]*variableValue[j]; } } }
void PLearn::MultiMaxVariable::build | ( | ) | [virtual] |
Post-constructor.
The normal implementation should call simply inherited::build(), then this class's build_(). This method should be callable again at later times, after modifying some option fields to change the "architecture" of the object.
Reimplemented from PLearn::UnaryVariable.
Definition at line 202 of file MultiMaxVariable.cc.
References PLearn::UnaryVariable::build(), and build_().
{ inherited::build(); build_(); }
void PLearn::MultiMaxVariable::build_ | ( | ) | [private] |
This does the actual building.
Reimplemented from PLearn::UnaryVariable.
Definition at line 248 of file MultiMaxVariable.cc.
References groupsize, groupsizes, i, PLearn::UnaryVariable::input, PLearn::PP< T >::isNotNull(), PLearn::TVec< T >::length(), PLERROR, PLearn::sum(), and PLearn::Var::width().
Referenced by build(), and MultiMaxVariable().
{ // ### This method should do the real building of the object, // ### according to set 'options', in *any* situation. // ### Typical situations include: // ### - Initial building of an object from a few user-specified options // ### - Building of a "reloaded" object: i.e. from the complete set of // ### all serialised options. // ### - Updating or "re-building" of an object after a few "tuning" // ### options have been modified. // ### You should assume that the parent class' build_() has already been // ### called. if (input.isNotNull() ) // otherwise postpone building until we have an input! { if (groupsizes.length() <= 0) { if (groupsize <= 0) PLERROR("Groupsize(s) not specified or invalid in MultiMaxVariable"); if (input->width() % groupsize != 0) PLERROR("Invalid groupsize in MultiMaxVariable"); TVec<int> vec(input->width()/groupsize, groupsize); groupsizes = vec; } else { int sum = 0; for(int i=0; i<groupsizes.length(); i++) sum += groupsizes[i]; if(sum != input->width()) PLERROR("Invalid groupsizes in MultiMaxVariable"); } } }
string PLearn::MultiMaxVariable::classname | ( | ) | const [virtual] |
Reimplemented from PLearn::UnaryVariable.
Definition at line 57 of file MultiMaxVariable.cc.
void PLearn::MultiMaxVariable::declareOptions | ( | OptionList & | ol | ) | [static, protected] |
Declares the class options.
Reimplemented from PLearn::UnaryVariable.
Definition at line 221 of file MultiMaxVariable.cc.
References PLearn::OptionBase::buildoption, computation_type, PLearn::declareOption(), PLearn::UnaryVariable::declareOptions(), groupsize, and groupsizes.
{ // ### Declare all of this object's options here. // ### For the "flags" of each option, you should typically specify // ### one of OptionBase::buildoption, OptionBase::learntoption or // ### OptionBase::tuningoption. If you don't provide one of these three, // ### this option will be ignored when loading values from a script. // ### You can also combine flags, for example with OptionBase::nosave: // ### (OptionBase::buildoption | OptionBase::nosave) // ### ex: declareOption(ol, "groupsizes", &MultiMaxVariable::groupsizes, OptionBase::buildoption, "this tells how to \"divide\" our diffrents inputs\nex: groupsizes = [1,2,3] says we divide our output like this :\n[x1],[x2,x3],[x4,x5,x6] and apply a maximum algorithm on each group separately"); declareOption(ol, "groupsize", &MultiMaxVariable::groupsize, OptionBase::buildoption, "shortcut if you want all groupsizes to be equals, for example if you set the value of this option to be 3, it will make groupsizes = [3,3,...,3]"); declareOption(ol, "computation_type", &MultiMaxVariable::computation_type, OptionBase::buildoption, "specifies what maximum algorithm should be used on our groups\n\'S\' = Softmax\n\'L\' = Log(Softmax)\n\'H\' = Hardmax*value\n\'h\' = hardmax\n\'R\' = random_Softmax*value\n\'r\' = random_Softmax"); // Now call the parent class' declareOptions inherited::declareOptions(ol); }
static const PPath& PLearn::MultiMaxVariable::declaringFile | ( | ) | [inline, static] |
Reimplemented from PLearn::UnaryVariable.
Definition at line 116 of file MultiMaxVariable.h.
:
//##### Protected Options ###############################################
MultiMaxVariable * PLearn::MultiMaxVariable::deepCopy | ( | CopiesMap & | copies | ) | const [virtual] |
Reimplemented from PLearn::UnaryVariable.
Definition at line 57 of file MultiMaxVariable.cc.
void PLearn::MultiMaxVariable::fprop | ( | ) | [virtual] |
Nothing to do by default.
Reimplemented from PLearn::UnaryVariable.
Definition at line 88 of file MultiMaxVariable.cc.
References computation_type, groupsizes, hardMax_range(), i, PLearn::UnaryVariable::input, PLearn::TVec< T >::length(), PLearn::TMat< T >::length(), logSoftmax_range(), PLearn::Variable::matValue, n, PLERROR, and softmax_range().
{ int k; Mat inputValue = input->matValue; Vec inputValue_n; Vec value_n; for(int n=0; n<inputValue.length(); n++) { k=0; inputValue_n = inputValue(n); value_n = matValue(n); for(int i=0; i<groupsizes.length(); i++) { switch(computation_type) { //softmax case 'S': // softmax(v.subVec(k,k+groupsizes[i]), value.subVec(k,k+groupsizes[i])); softmax_range(inputValue_n, value_n, k, groupsizes[i]); break; //log_softmax case 'L': // log_softmax(v.subVec(k,k+groupsizes[i]), value.subVec(k,k+groupsizes[i])); logSoftmax_range(inputValue_n, value_n, k, groupsizes[i]); break; //hardmax_value case 'H': hardMax_range(inputValue_n, value_n, k, groupsizes[i], true); break; //hardmax case 'h': hardMax_range(inputValue_n, value_n, k, groupsizes[i], false); break; //random_softmax_value case 'R': softmax_range(inputValue_n, value_n, k, groupsizes[i]); //TODO : RANDOM PLERROR("computation_type 'R' not fully implemented yet"); break; //random_softmax case 'r': softmax_range(inputValue_n, value_n, k, groupsizes[i]); //TODO : RANDOM PLERROR("computation_type 'r' not fully implemented yet"); break; default : PLERROR("invalid computation_type in MultiMaxVariable"); } k+=groupsizes[i]; } } }
OptionList & PLearn::MultiMaxVariable::getOptionList | ( | ) | const [virtual] |
Reimplemented from PLearn::UnaryVariable.
Definition at line 57 of file MultiMaxVariable.cc.
OptionMap & PLearn::MultiMaxVariable::getOptionMap | ( | ) | const [virtual] |
Reimplemented from PLearn::UnaryVariable.
Definition at line 57 of file MultiMaxVariable.cc.
RemoteMethodMap & PLearn::MultiMaxVariable::getRemoteMethodMap | ( | ) | const [virtual] |
Reimplemented from PLearn::UnaryVariable.
Definition at line 57 of file MultiMaxVariable.cc.
void PLearn::MultiMaxVariable::hardMax_range | ( | Vec & | x, |
Vec & | y, | ||
int | start, | ||
int | length, | ||
bool | value = false |
||
) | [static, private] |
Definition at line 310 of file MultiMaxVariable.cc.
References i, and PLearn::Variable::length().
Referenced by fprop().
{ int indMax=start; for(int i=start+1; i<start+length; i++) if(x[i] > x[indMax]) indMax = i; for(int i=start; i<start+length; i++) y[i] = 0; if(value) y[indMax] = x[indMax]; else y[indMax]=1; }
void PLearn::MultiMaxVariable::logSoftmax_range | ( | Vec & | x, |
Vec & | y, | ||
int | start, | ||
int | length | ||
) | [static, private] |
Definition at line 300 of file MultiMaxVariable.cc.
References i, PLearn::Variable::length(), PLearn::safeexp(), and PLearn::safelog().
Referenced by fprop().
{ real somme=0; for(int i=start; i<start+length; i++) somme += safeexp(x[i]); for(int i=start; i<start+length; i++) y[i] = x[i] - safelog(somme); }
void PLearn::MultiMaxVariable::makeDeepCopyFromShallowCopy | ( | CopiesMap & | copies | ) | [virtual] |
Transforms a shallow copy into a deep copy.
Reimplemented from PLearn::UnaryVariable.
Definition at line 208 of file MultiMaxVariable.cc.
References PLearn::deepCopyField(), groupsizes, and PLearn::UnaryVariable::makeDeepCopyFromShallowCopy().
{ inherited::makeDeepCopyFromShallowCopy(copies); // ### Call deepCopyField on all "pointer-like" fields // ### that you wish to be deepCopied rather than // ### shallow-copied. // ### ex: deepCopyField(groupsizes, copies); // ### If you want to deepCopy a Var field: // varDeepCopyField(somevariable, copies); }
Recomputes the length l and width w that this variable should have, according to its parent variables.
This is used for ex. by sizeprop() The default version stupidly returns the current dimensions, so make sure to overload it in subclasses if this is not appropriate.
Reimplemented from PLearn::Variable.
Definition at line 78 of file MultiMaxVariable.cc.
References PLearn::UnaryVariable::input, PLearn::Var::length(), and PLearn::Var::width().
void PLearn::MultiMaxVariable::softmax_range | ( | Vec & | x, |
Vec & | y, | ||
int | start, | ||
int | length | ||
) | [static, private] |
Definition at line 289 of file MultiMaxVariable.cc.
References i, PLearn::Variable::length(), PLERROR, and PLearn::safeexp().
Referenced by fprop().
{ real somme=0; for(int i=start; i<start+length; i++) somme +=safeexp(x[i]); if (somme == 0) PLERROR("trying to divide by 0 in softmax"); for(int i=start; i<start+length; i++) y[i] = safeexp(x[i])/somme; }
Reimplemented from PLearn::UnaryVariable.
Definition at line 116 of file MultiMaxVariable.h.
Definition at line 80 of file MultiMaxVariable.h.
Referenced by bprop(), declareOptions(), and fprop().
Definition at line 81 of file MultiMaxVariable.h.
Referenced by build_(), and declareOptions().
### declare public option fields (such as build options) here Start your comments with Doxygen-compatible comments such as //!
group sizes, ex: [2,3,4] = ([x1,x2],[x3,x4,x5],[x6,x7,x8,x9])
Definition at line 79 of file MultiMaxVariable.h.
Referenced by bprop(), build_(), declareOptions(), fprop(), and makeDeepCopyFromShallowCopy().