PLearn 0.1
Public Member Functions | Static Public Member Functions | Public Attributes | Static Public Attributes | Static Protected Member Functions | Private Types | Private Member Functions | Private Attributes
PLearn::MeanImputationVMatrix Class Reference

provides mean imputation for missing variables More...

#include <MeanImputationVMatrix.h>

Inheritance diagram for PLearn::MeanImputationVMatrix:
Inheritance graph
[legend]
Collaboration diagram for PLearn::MeanImputationVMatrix:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 MeanImputationVMatrix ()
 MeanImputationVMatrix (VMat the_source, real the_number_of_train_samples=0.0, bool call_build_=true)
virtual void build ()
 Simply calls inherited::build() then build_().
virtual void makeDeepCopyFromShallowCopy (CopiesMap &copies)
 Transforms a shallow copy into a deep copy.
virtual void getNewRow (int i, const Vec &v) const
 Must be implemented in subclasses: default version returns an error.
Vec getMeanVector ()
virtual string classname () const
virtual OptionListgetOptionList () const
virtual OptionMapgetOptionMap () const
virtual RemoteMethodMapgetRemoteMethodMap () const
virtual MeanImputationVMatrixdeepCopy (CopiesMap &copies) const

Static Public Member Functions

static string _classname_ ()
 MeanImputationVMatrix.
static OptionList_getOptionList_ ()
static RemoteMethodMap_getRemoteMethodMap_ ()
static Object_new_instance_for_typemap_ ()
static bool _isa_ (const Object *o)
static void _static_initialize_ ()
static const PPathdeclaringFile ()

Public Attributes

PP< PDistributiondistribution
string distribution_access_to_target
real number_of_train_samples
VMat mean_source

Static Public Attributes

static StaticInitializer _static_initializer_

Static Protected Member Functions

static void declareOptions (OptionList &ol)
 Declares this class' options.

Private Types

typedef SourceVMatrix inherited

Private Member Functions

void build_ ()
 This does the actual building.
void computeMeanVector ()

Private Attributes

bool obtained_inputsize_from_source
bool obtained_targetsize_from_source
bool obtained_weightsize_from_source
Vec cond_mean
 Vector used to store the conditional mean of the missing values given the observed value, when the 'distribution' option is set.
Vec variable_mean
Vec tmp_target
 Temporary storage vector for a target.

Detailed Description

provides mean imputation for missing variables

Definition at line 56 of file MeanImputationVMatrix.h.


Member Typedef Documentation

Reimplemented from PLearn::SourceVMatrix.

Definition at line 58 of file MeanImputationVMatrix.h.


Constructor & Destructor Documentation

PLearn::MeanImputationVMatrix::MeanImputationVMatrix ( )
PLearn::MeanImputationVMatrix::MeanImputationVMatrix ( VMat  the_source,
real  the_number_of_train_samples = 0.0,
bool  call_build_ = true 
)

Definition at line 82 of file MeanImputationVMatrix.cc.

References build_().

                                                              :
    inherited(the_source, call_build_),
    obtained_inputsize_from_source(false),
    obtained_targetsize_from_source(false),
    obtained_weightsize_from_source(false),
    distribution_access_to_target("train_only"),
    number_of_train_samples(the_number_of_train_samples)
{
    if (call_build_)
        build_();
}

Here is the call graph for this function:


Member Function Documentation

string PLearn::MeanImputationVMatrix::_classname_ ( ) [static]

MeanImputationVMatrix.

Reimplemented from PLearn::SourceVMatrix.

Definition at line 69 of file MeanImputationVMatrix.cc.

OptionList & PLearn::MeanImputationVMatrix::_getOptionList_ ( ) [static]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 69 of file MeanImputationVMatrix.cc.

RemoteMethodMap & PLearn::MeanImputationVMatrix::_getRemoteMethodMap_ ( ) [static]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 69 of file MeanImputationVMatrix.cc.

bool PLearn::MeanImputationVMatrix::_isa_ ( const Object o) [static]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 69 of file MeanImputationVMatrix.cc.

Object * PLearn::MeanImputationVMatrix::_new_instance_for_typemap_ ( ) [static]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 69 of file MeanImputationVMatrix.cc.

StaticInitializer MeanImputationVMatrix::_static_initializer_ & PLearn::MeanImputationVMatrix::_static_initialize_ ( ) [static]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 69 of file MeanImputationVMatrix.cc.

void PLearn::MeanImputationVMatrix::build ( ) [virtual]

Simply calls inherited::build() then build_().

Reimplemented from PLearn::SourceVMatrix.

Definition at line 162 of file MeanImputationVMatrix.cc.

References PLearn::SourceVMatrix::build(), and build_().

Here is the call graph for this function:

void PLearn::MeanImputationVMatrix::build_ ( ) [private]

This does the actual building.

Reimplemented from PLearn::SourceVMatrix.

Definition at line 171 of file MeanImputationVMatrix.cc.

References computeMeanVector(), distribution, distribution_access_to_target, PLearn::VMatrix::inputsize_, PLearn::VMat::length(), PLearn::VMatrix::length_, mean_source, number_of_train_samples, obtained_inputsize_from_source, obtained_targetsize_from_source, obtained_weightsize_from_source, PLASSERT, PLERROR, PLearn::SourceVMatrix::setMetaInfoFromSource(), PLearn::SourceVMatrix::source, PLearn::VMatrix::targetsize_, PLearn::VMatrix::updateMtime(), PLearn::VMatrix::weightsize_, PLearn::VMat::width(), and PLearn::VMatrix::width_.

Referenced by build(), and MeanImputationVMatrix().

{
    if (source) {
        string error_msg =
            "In MeanImputationVMatrix::build_ - For safety reasons, it is forbidden to "
            "re-use sizes obtained from a previous source VMatrix with a new source "
            "VMatrix having different sizes";
        length_ = source->length();
        width_ = source->width();
        if(inputsize_<0) {
            inputsize_ = source->inputsize();
            obtained_inputsize_from_source = true;
        } else if (obtained_inputsize_from_source && inputsize_ != source->inputsize())
            PLERROR(error_msg.c_str());
        if(targetsize_<0) {
            targetsize_ = source->targetsize();
            obtained_targetsize_from_source = true;
        } else if (obtained_targetsize_from_source && targetsize_ != source->targetsize())
            PLERROR(error_msg.c_str());
        if(weightsize_<0) {
            weightsize_ = source->weightsize();
            obtained_weightsize_from_source = true;
        } else if (obtained_weightsize_from_source && weightsize_ != source->weightsize())
            PLERROR(error_msg.c_str());

        setMetaInfoFromSource();
        updateMtime(mean_source);
        
        computeMeanVector();

        // Train the user-provided distribution if needed.
        if (distribution) {
            distribution->setPredictorPredictedSizes(0, -1);
            if (distribution->stage == 0) {
                // Currently not implemented for a limited number of training
                // samples, but it should not be too difficult to do it.
                PLASSERT( number_of_train_samples == 0 );
                VMat the_train_source = mean_source ? mean_source : source;
                // Redefine sizes to train on the whole data.
                the_train_source = new ForwardVMatrix(the_train_source);
                the_train_source->defineSizes(the_train_source->width(), 0, 0, 0);
                distribution->setTrainingSet(the_train_source);
                distribution->train();
            }
        }
    } else {
        // Restore the original undefined sizes if the current one had been obtained
        // from the source VMatrix.
        if (obtained_inputsize_from_source) {
            inputsize_ = -1;
            obtained_inputsize_from_source = false;
        }
        if (obtained_targetsize_from_source) {
            targetsize_ = -1;
            obtained_targetsize_from_source = false;
        }
        if (obtained_weightsize_from_source) {
            weightsize_ = -1;
            obtained_weightsize_from_source = false;
        }
    }

    // Check valid values.
    if (distribution_access_to_target != "train_only" &&
        distribution_access_to_target != "none")
        PLERROR("In MeanImputationVMatrix::build_ - Invalid value for option "
                "'distribution_access_to_target'");
}

Here is the call graph for this function:

Here is the caller graph for this function:

string PLearn::MeanImputationVMatrix::classname ( ) const [virtual]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 69 of file MeanImputationVMatrix.cc.

void PLearn::MeanImputationVMatrix::computeMeanVector ( ) [private]

Definition at line 301 of file MeanImputationVMatrix.cc.

References PLearn::computeMean(), PLearn::VMatrix::length(), PLearn::VMat::length(), mean_source, number_of_train_samples, PLASSERT, PLearn::TVec< T >::resize(), PLearn::SourceVMatrix::source, variable_mean, PLearn::VMat::width(), PLearn::VMatrix::width(), and PLearn::VMatrix::width_.

Referenced by build_().

{
    VMat the_mean_source;
    if (mean_source) {
        PLASSERT( mean_source->width() == source->width() );
        the_mean_source = mean_source;
    } else
        the_mean_source = source;
   
    PLASSERT( the_mean_source );

    int length = the_mean_source->length();
    int width = width_;
    PLASSERT( width = the_mean_source->width() );
    variable_mean.resize(width);
    if (number_of_train_samples > 0.0)
    {
        if (number_of_train_samples >= 1.0)
            length = (int) number_of_train_samples;
        else
            length = (int) ((double) length * number_of_train_samples);
        if (length < 1)
            length = 1;
        if (length > the_mean_source->length())
            length = the_mean_source->length();
    }
    VMat sub_source = the_mean_source;
    if (length != the_mean_source->length())
        sub_source = new SubVMatrix(sub_source, 0, 0,
                                    length, sub_source->width());
    computeMean(sub_source, variable_mean);
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::MeanImputationVMatrix::declareOptions ( OptionList ol) [static, protected]

Declares this class' options.

Reimplemented from PLearn::SourceVMatrix.

Definition at line 99 of file MeanImputationVMatrix.cc.

References PLearn::OptionBase::buildoption, PLearn::declareOption(), PLearn::SourceVMatrix::declareOptions(), distribution, distribution_access_to_target, PLearn::OptionBase::learntoption, mean_source, number_of_train_samples, obtained_inputsize_from_source, obtained_targetsize_from_source, obtained_weightsize_from_source, and variable_mean.

{

    declareOption(ol, "number_of_train_samples",
                  &MeanImputationVMatrix::number_of_train_samples,
                  OptionBase::buildoption,
        "If equal to zero, all the underlying dataset samples are used to\n"
        "compute the variable means. If it is a fraction between 0 and 1,\n"
        "this proportion of the samples will be used. If greater than or\n"
        "equal to 1, the integer portion will be interpreted as the number\n"
        "of samples to use.");

    declareOption(ol, "mean_source",
                  &MeanImputationVMatrix::mean_source,
                  OptionBase::buildoption,
        "If specified, this VMat will be used to compute the means instead\n"
        "of the 'source' option.");

    declareOption(ol, "distribution",
                  &MeanImputationVMatrix::distribution,
                  OptionBase::buildoption,
        "If provided, this conditional distribution will provide a way to\n"
        "compute the conditional mean given observed values. Otherwise, the\n"
        "empirical mean will be used. This distribution is trained on the\n"
        "'source' (or 'mean_source') VMat unless it already has a stage > 0.\n"
        "Whether this distribution has access to the target value or not\n"
        "depends on the 'distribution_access_to_target' option.\n");

    declareOption(ol, "distribution_access_to_target",
                  &MeanImputationVMatrix::distribution_access_to_target,
                  OptionBase::buildoption,
        "Used only when a distribution is specified, and modifies the way\n"
        "target values are given to the distribution:\n"
        "- train_only: only the first 'number_of_train_samples' are given\n"
        "              access to their target (if this number is 0, all\n"
        "              samples are given access to their target)\n"
        "- none      : no sample is given access to its target\n");

    declareOption(ol, "variable_mean", &MeanImputationVMatrix::variable_mean,
                                       OptionBase::learntoption,
        "The vector of variable means observed from the source matrix.");

    declareOption(ol, "obtained_inputsize_from_source",
                  &MeanImputationVMatrix::obtained_inputsize_from_source,
                  OptionBase::learntoption,
        "Set to 1 when the inputsize was obtained from the source matrix.");

    declareOption(ol, "obtained_targetsize_from_source",
                  &MeanImputationVMatrix::obtained_targetsize_from_source,
                  OptionBase::learntoption,
        "Set to 1 when the targetsize was obtained from the source matrix.");

    declareOption(ol, "obtained_weightsize_from_source",
                  &MeanImputationVMatrix::obtained_weightsize_from_source,
                  OptionBase::learntoption,
        "Set to 1 when the weightsize was obtained from the source matrix.");

    inherited::declareOptions(ol);
}

Here is the call graph for this function:

static const PPath& PLearn::MeanImputationVMatrix::declaringFile ( ) [inline, static]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 109 of file MeanImputationVMatrix.h.

MeanImputationVMatrix * PLearn::MeanImputationVMatrix::deepCopy ( CopiesMap copies) const [virtual]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 69 of file MeanImputationVMatrix.cc.

Vec PLearn::MeanImputationVMatrix::getMeanVector ( )

Definition at line 293 of file MeanImputationVMatrix.cc.

References variable_mean.

{
    return variable_mean;
}
void PLearn::MeanImputationVMatrix::getNewRow ( int  i,
const Vec v 
) const [virtual]

Must be implemented in subclasses: default version returns an error.

Reimplemented from PLearn::SourceVMatrix.

Definition at line 257 of file MeanImputationVMatrix.cc.

References cond_mean, distribution, distribution_access_to_target, PLearn::TVec< T >::fill(), PLearn::TVec< T >::hasMissing(), PLearn::is_missing(), j, PLearn::TVec< T >::length(), MISSING_VALUE, number_of_train_samples, PLASSERT, PLearn::TVec< T >::resize(), PLearn::SourceVMatrix::source, PLearn::TVec< T >::subVec(), tmp_target, and variable_mean.

{
    PLASSERT( source );
    source->getRow(i, v);

    if (v.hasMissing()){
        if (distribution) {
            Vec target;
            bool restore_target = false;
            if ((number_of_train_samples > 0 && i >= number_of_train_samples &&
                 distribution_access_to_target == "train_only") ||
                distribution_access_to_target == "none")
            {
                tmp_target.resize(source->targetsize());
                target = v.subVec(source->inputsize(), source->targetsize());
                tmp_target << target;
                target.fill(MISSING_VALUE);
                restore_target = true;
            }
            distribution->missingExpectation(v, cond_mean);
            int k = 0;
            for (int j = 0; j < v.length(); j++)
                if (is_missing(v[j]))
                    v[j] = cond_mean[k++];
            if (restore_target)
                target << tmp_target;
        } else
            for (int j = 0; j < v.length(); j++)
                if (is_missing(v[j]))
                    v[j] = variable_mean[j];
    }
}

Here is the call graph for this function:

OptionList & PLearn::MeanImputationVMatrix::getOptionList ( ) const [virtual]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 69 of file MeanImputationVMatrix.cc.

OptionMap & PLearn::MeanImputationVMatrix::getOptionMap ( ) const [virtual]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 69 of file MeanImputationVMatrix.cc.

RemoteMethodMap & PLearn::MeanImputationVMatrix::getRemoteMethodMap ( ) const [virtual]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 69 of file MeanImputationVMatrix.cc.

void PLearn::MeanImputationVMatrix::makeDeepCopyFromShallowCopy ( CopiesMap copies) [virtual]

Transforms a shallow copy into a deep copy.

Reimplemented from PLearn::SourceVMatrix.

Definition at line 243 of file MeanImputationVMatrix.cc.

References cond_mean, PLearn::deepCopyField(), distribution, PLearn::SourceVMatrix::makeDeepCopyFromShallowCopy(), mean_source, tmp_target, and variable_mean.

Here is the call graph for this function:


Member Data Documentation

Reimplemented from PLearn::SourceVMatrix.

Definition at line 109 of file MeanImputationVMatrix.h.

Vector used to store the conditional mean of the missing values given the observed value, when the 'distribution' option is set.

Definition at line 68 of file MeanImputationVMatrix.h.

Referenced by getNewRow(), and makeDeepCopyFromShallowCopy().

Definition at line 78 of file MeanImputationVMatrix.h.

Referenced by build_(), declareOptions(), and getNewRow().

Definition at line 79 of file MeanImputationVMatrix.h.

Referenced by build_(), computeMeanVector(), declareOptions(), and getNewRow().

Definition at line 62 of file MeanImputationVMatrix.h.

Referenced by build_(), and declareOptions().

Definition at line 63 of file MeanImputationVMatrix.h.

Referenced by build_(), and declareOptions().

Definition at line 64 of file MeanImputationVMatrix.h.

Referenced by build_(), and declareOptions().

Temporary storage vector for a target.

Definition at line 73 of file MeanImputationVMatrix.h.

Referenced by getNewRow(), and makeDeepCopyFromShallowCopy().


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines