PLearn 0.1
Public Member Functions | Static Public Member Functions | Static Public Attributes | Protected Member Functions | Protected Attributes | Private Types
PLearn::SemiSupervisedProbClassCostVariable Class Reference

#include <SemiSupervisedProbClassCostVariable.h>

Inheritance diagram for PLearn::SemiSupervisedProbClassCostVariable:
Inheritance graph
[legend]
Collaboration diagram for PLearn::SemiSupervisedProbClassCostVariable:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 SemiSupervisedProbClassCostVariable ()
 Default constructor for persistence.
 SemiSupervisedProbClassCostVariable (Var prob_, Var target_, Var prior_, real flattenfactor=1.0)
virtual string classname () const
virtual OptionListgetOptionList () const
virtual OptionMapgetOptionMap () const
virtual RemoteMethodMapgetRemoteMethodMap () const
virtual
SemiSupervisedProbClassCostVariable
deepCopy (CopiesMap &copies) const
virtual void build ()
 Post-constructor.
virtual void recomputeSize (int &l, int &w) const
 Recomputes the length l and width w that this variable should have, according to its parent variables.
virtual void fprop ()
 compute output given input
virtual void bprop ()
virtual void symbolicBprop ()
 compute a piece of new Var graph that represents the symbolic derivative of this Var
virtual void rfprop ()
Varprob ()
Vartarget ()
Varprior ()

Static Public Member Functions

static string _classname_ ()
 SemiSupervisedProbClassVariable.
static OptionList_getOptionList_ ()
static RemoteMethodMap_getRemoteMethodMap_ ()
static Object_new_instance_for_typemap_ ()
static bool _isa_ (const Object *o)
static void _static_initialize_ ()
static const PPathdeclaringFile ()
static void declareOptions (OptionList &ol)
 Default constructor for persistence.

Static Public Attributes

static StaticInitializer _static_initializer_

Protected Member Functions

void build_ ()
 Object-specific post-constructor.

Protected Attributes

real flatten_factor
 the option
Vec raised_prob
 temporaries for avoiding re-computations in the bprop
real sum_raised_prob

Private Types

typedef NaryVariable inherited

Detailed Description

! This is a cost function for semi-supervised training of a probabilistic classifier. ! It takes three arguments: (prob,target,prior). ! The prob should be positive and summing to 1 (e.g. the output of a softmax). ! The target is either an integer (which indexes the output vector) or a missing value. ! The prior is a vector of probabilities (they don't have to be normalized) which ! act as a prior when the target is not provided. ! There are two types of cases: when the target is given (supervised case) ! and when the target is missing (unsupervised case). ! The cost value is computed as follows: ! ! If target is not missing: ! cost = - log(prob[target]) ! Else ! cost = - (1/flatten_factor) log sum_i (prior[i] * prob[i])^flatten_factor ! ! The only option of this class is the hyper-parameter "flatten_factor" ! If prior is an empty Var, then prior[i]=1 is assumed. ! ! N.B. ! - prior can be understood as the supervised target for supervised examples ! (then the above cost becomes the classical max. likelihood, = log prob_{target}) ! - for unsupervised examples prior embodies the information available on the target; ! reasonable choices are: ! - all 1's (recommanded) ! - prior relative frequencies (conditional, if some other information can be used, heuristic) ! - output of the model trained on supervised examples only (?? not clear from theory) ! In the latter two cases it might be desirable to modulate the "strength" of the ! prior probabilities by raising them to some power. Note that prior[i] do not ! have to be normalized but they should be non-negative. ! - when flatten_factor=1 the gradient on unsupervised examples (prior=const) is 0 ! - increasing "flatten_factor" increases the confidence given to the model current prediction on unsupervised examples ! - this cost may be very sensitive to INITIALIZATION: ! - it would be wise to start from purely supervised training (i.e. a=1, or skip the unsupervised examples) ! - and gradually increase it (<=5, say) e.g. a=1, then a=1.1, then a=1.5, then a=2. ! ! N.B2: THE IMPLEMENTATION OF THIS CLASS CHEATS AND DOES NOT COMPUTE THE GRADIENTS ! WITH RESPECT TO THE target AND prior INPUTS, AS SUCH GRADIENTS SHOULD NOT BE USED. !

Definition at line 92 of file SemiSupervisedProbClassCostVariable.h.


Member Typedef Documentation

Reimplemented from PLearn::NaryVariable.

Definition at line 94 of file SemiSupervisedProbClassCostVariable.h.


Constructor & Destructor Documentation

PLearn::SemiSupervisedProbClassCostVariable::SemiSupervisedProbClassCostVariable ( ) [inline]

Default constructor for persistence.

Definition at line 106 of file SemiSupervisedProbClassCostVariable.h.

PLearn::SemiSupervisedProbClassCostVariable::SemiSupervisedProbClassCostVariable ( Var  prob_,
Var  target_,
Var  prior_,
real  flattenfactor = 1.0 
)

Definition at line 56 of file SemiSupervisedProbClassCostVariable.cc.

References build_().

    : inherited(prob_ & target_ & (VarArray)prior_,1,1), flatten_factor(ff)
{
    build_();
}

Here is the call graph for this function:


Member Function Documentation

string PLearn::SemiSupervisedProbClassCostVariable::_classname_ ( ) [static]

SemiSupervisedProbClassVariable.

Reimplemented from PLearn::NaryVariable.

Definition at line 54 of file SemiSupervisedProbClassCostVariable.cc.

OptionList & PLearn::SemiSupervisedProbClassCostVariable::_getOptionList_ ( ) [static]

Reimplemented from PLearn::NaryVariable.

Definition at line 54 of file SemiSupervisedProbClassCostVariable.cc.

RemoteMethodMap & PLearn::SemiSupervisedProbClassCostVariable::_getRemoteMethodMap_ ( ) [static]

Reimplemented from PLearn::NaryVariable.

Definition at line 54 of file SemiSupervisedProbClassCostVariable.cc.

bool PLearn::SemiSupervisedProbClassCostVariable::_isa_ ( const Object o) [static]

Reimplemented from PLearn::NaryVariable.

Definition at line 54 of file SemiSupervisedProbClassCostVariable.cc.

Object * PLearn::SemiSupervisedProbClassCostVariable::_new_instance_for_typemap_ ( ) [static]

Reimplemented from PLearn::Object.

Definition at line 54 of file SemiSupervisedProbClassCostVariable.cc.

StaticInitializer SemiSupervisedProbClassCostVariable::_static_initializer_ & PLearn::SemiSupervisedProbClassCostVariable::_static_initialize_ ( ) [static]

Reimplemented from PLearn::NaryVariable.

Definition at line 54 of file SemiSupervisedProbClassCostVariable.cc.

void PLearn::SemiSupervisedProbClassCostVariable::bprop ( ) [virtual]

Implements PLearn::Variable.

Definition at line 124 of file SemiSupervisedProbClassCostVariable.cc.

References grad, PLearn::Variable::gradientdata, i, n, prob(), raised_prob, sum_raised_prob, and target().

{
    real target_value = target()->valuedata[0];
    int n=prob()->size();
    real* dprob=prob()->gradientdata;
    real* p=prob()->valuedata;
    if (finite(target_value)) // supervised case
    {
        int t = int(target_value);
        for (int i=0;i<n;i++)
            if (i==t && p[t]>0)
                dprob[i] += -gradientdata[0]/p[t];
    }
    else // unsupervised case
    {
        for (int i=0;i<n;i++)
            if (p[i]>0)
            {
                real grad = - gradientdata[0]*raised_prob[i]/(p[i]*sum_raised_prob);
                if (finite(grad))
                    dprob[i] += grad;
            }
    }
}

Here is the call graph for this function:

void PLearn::SemiSupervisedProbClassCostVariable::build ( ) [virtual]

Post-constructor.

The normal implementation should call simply inherited::build(), then this class's build_(). This method should be callable again at later times, after modifying some option fields to change the "architecture" of the object.

Reimplemented from PLearn::NaryVariable.

Definition at line 63 of file SemiSupervisedProbClassCostVariable.cc.

References PLearn::NaryVariable::build(), and build_().

Here is the call graph for this function:

void PLearn::SemiSupervisedProbClassCostVariable::build_ ( ) [protected]

Object-specific post-constructor.

This method should be redefined in subclasses and do the actual building of the object according to previously set option fields. Constructors can just set option fields, and then call build_. This method is NOT virtual, and will typically be called only from three places: a constructor, the public virtual build() method, and possibly the public virtual read method (which calls its parent's read). build_() can assume that its parent's build_() has already been called.

Reimplemented from PLearn::NaryVariable.

Definition at line 70 of file SemiSupervisedProbClassCostVariable.cc.

References flatten_factor, PLearn::Variable::isScalar(), PLearn::TVec< T >::length(), PLearn::Variable::length(), PLERROR, raised_prob, PLearn::TVec< T >::resize(), PLearn::TVec< T >::size(), and PLearn::NaryVariable::varray.

Referenced by build(), and SemiSupervisedProbClassCostVariable().

{
    if (varray.size() >= 3 && varray[0] && varray[1] && varray[2]) {
        // varray[0], varray[1] and varray[2] are (respectively) prob_, target_ and prior_ from constructor
        if (varray[2]->length()>0 && varray[0]->length() != varray[2]->length())
            PLERROR("In SemiSupervisedProbClassCostVariable: If prior.length()>0 then prior and prob must have the same size");
        if (!varray[1]->isScalar())
            PLERROR("In SemiSupervisedProbClassCostVariable: target must be a scalar");
        raised_prob.resize(varray[0]->length());
    }
    if (flatten_factor <= 0)
        PLERROR("In SemiSupervisedProbClassCostVariable: flatten_factor must be positive, and even > 1 for normal use.");
}

Here is the call graph for this function:

Here is the caller graph for this function:

string PLearn::SemiSupervisedProbClassCostVariable::classname ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 54 of file SemiSupervisedProbClassCostVariable.cc.

void PLearn::SemiSupervisedProbClassCostVariable::declareOptions ( OptionList ol) [static]

Default constructor for persistence.

Reimplemented from PLearn::NaryVariable.

Definition at line 85 of file SemiSupervisedProbClassCostVariable.cc.

References PLearn::OptionBase::buildoption, PLearn::declareOption(), PLearn::NaryVariable::declareOptions(), and flatten_factor.

Here is the call graph for this function:

static const PPath& PLearn::SemiSupervisedProbClassCostVariable::declaringFile ( ) [inline, static]

Reimplemented from PLearn::NaryVariable.

Definition at line 109 of file SemiSupervisedProbClassCostVariable.h.

{ return varray[0]; }
SemiSupervisedProbClassCostVariable * PLearn::SemiSupervisedProbClassCostVariable::deepCopy ( CopiesMap copies) const [virtual]

Reimplemented from PLearn::NaryVariable.

Definition at line 54 of file SemiSupervisedProbClassCostVariable.cc.

void PLearn::SemiSupervisedProbClassCostVariable::fprop ( ) [virtual]

compute output given input

If target is not missing: cost = - log(prob[target]) Else cost = - (1/flatten_factor) log sum_i (prior[i] * prob[i])^flatten_factor

Implements PLearn::Variable.

Definition at line 95 of file SemiSupervisedProbClassCostVariable.cc.

References flatten_factor, i, n, PLERROR, PLearn::pow(), prior(), prob(), raised_prob, PLearn::safeflog(), PLearn::Variable::size(), sum_raised_prob, target(), and PLearn::Variable::valuedata.

{
    real target_value = target()->valuedata[0];
    int n=prob()->size();
    real* p=prob()->valuedata;
    if (finite(target_value)) // supervised case
    {
        int t = int(target_value);
        if (t<0 || t>=n)
            PLERROR("In SemiSupervisedProbClassCostVariable: target must be either missing or between 0 and %d incl.\n",prob()->size()-1);
        valuedata[0] = -safeflog(p[t]);
    }
    else // unsupervised case
    {
        sum_raised_prob=0;
        real* priorv = prior()->valuedata;
        for (int i=0;i<n;i++)
        {
            raised_prob[i] = pow(priorv[i]*p[i],flatten_factor);
            sum_raised_prob += raised_prob[i];
        }
        valuedata[0] = - safeflog(sum_raised_prob)/flatten_factor;
    }
}

Here is the call graph for this function:

OptionList & PLearn::SemiSupervisedProbClassCostVariable::getOptionList ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 54 of file SemiSupervisedProbClassCostVariable.cc.

OptionMap & PLearn::SemiSupervisedProbClassCostVariable::getOptionMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 54 of file SemiSupervisedProbClassCostVariable.cc.

RemoteMethodMap & PLearn::SemiSupervisedProbClassCostVariable::getRemoteMethodMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 54 of file SemiSupervisedProbClassCostVariable.cc.

Var& PLearn::SemiSupervisedProbClassCostVariable::prior ( ) [inline]

Definition at line 121 of file SemiSupervisedProbClassCostVariable.h.

Referenced by fprop().

{ return varray[2]; }

Here is the caller graph for this function:

Var& PLearn::SemiSupervisedProbClassCostVariable::prob ( ) [inline]

Definition at line 119 of file SemiSupervisedProbClassCostVariable.h.

Referenced by bprop(), and fprop().

{ return varray[0]; }

Here is the caller graph for this function:

void PLearn::SemiSupervisedProbClassCostVariable::recomputeSize ( int l,
int w 
) const [virtual]

Recomputes the length l and width w that this variable should have, according to its parent variables.

This is used for ex. by sizeprop() The default version stupidly returns the current dimensions, so make sure to overload it in subclasses if this is not appropriate.

Reimplemented from PLearn::Variable.

Definition at line 91 of file SemiSupervisedProbClassCostVariable.cc.

{ l=1; w=1; }
void PLearn::SemiSupervisedProbClassCostVariable::rfprop ( ) [virtual]

Reimplemented from PLearn::Variable.

Definition at line 156 of file SemiSupervisedProbClassCostVariable.cc.

References PLERROR.

{
    PLERROR("SemiSupervisedProbClassCostVariable::rfprop() not implemented");
}
void PLearn::SemiSupervisedProbClassCostVariable::symbolicBprop ( ) [virtual]

compute a piece of new Var graph that represents the symbolic derivative of this Var

Reimplemented from PLearn::Variable.

Definition at line 150 of file SemiSupervisedProbClassCostVariable.cc.

References PLERROR.

{
    PLERROR("SemiSupervisedProbClassCostVariable::symbolicBprop() not implemented");
}
Var& PLearn::SemiSupervisedProbClassCostVariable::target ( ) [inline]

Definition at line 120 of file SemiSupervisedProbClassCostVariable.h.

Referenced by bprop(), and fprop().

{ return varray[1]; }

Here is the caller graph for this function:


Member Data Documentation

Reimplemented from PLearn::NaryVariable.

Definition at line 109 of file SemiSupervisedProbClassCostVariable.h.

the option

Definition at line 98 of file SemiSupervisedProbClassCostVariable.h.

Referenced by build_(), declareOptions(), and fprop().

temporaries for avoiding re-computations in the bprop

Definition at line 101 of file SemiSupervisedProbClassCostVariable.h.

Referenced by bprop(), build_(), and fprop().

Definition at line 102 of file SemiSupervisedProbClassCostVariable.h.

Referenced by bprop(), and fprop().


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines