PLearn 0.1
Public Member Functions | Static Public Member Functions | Public Attributes | Static Public Attributes | Protected Member Functions | Static Protected Member Functions | Protected Attributes | Private Types | Private Member Functions
PLearn::KNNImputationVMatrix Class Reference

The first sentence should be a BRIEF DESCRIPTION of what the class does. More...

#include <KNNImputationVMatrix.h>

Inheritance diagram for PLearn::KNNImputationVMatrix:
Inheritance graph
[legend]
Collaboration diagram for PLearn::KNNImputationVMatrix:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 KNNImputationVMatrix ()
 Default constructor.
virtual string classname () const
virtual OptionListgetOptionList () const
virtual OptionMapgetOptionMap () const
virtual RemoteMethodMapgetRemoteMethodMap () const
virtual KNNImputationVMatrixdeepCopy (CopiesMap &copies) const
virtual void build ()
 Simply calls inherited::build() then build_().
virtual void makeDeepCopyFromShallowCopy (CopiesMap &copies)
 Transforms a shallow copy into a deep copy.

Static Public Member Functions

static string _classname_ ()
 Declares name and deepCopy methods.
static OptionList_getOptionList_ ()
static RemoteMethodMap_getRemoteMethodMap_ ()
static Object_new_instance_for_typemap_ ()
static bool _isa_ (const Object *o)
static void _static_initialize_ ()
static const PPathdeclaringFile ()

Public Attributes

VMat full_source
int knn
VMat neighbors
int n_train_samples
bool report_progress

Static Public Attributes

static StaticInitializer _static_initializer_

Protected Member Functions

virtual void getNewRow (int i, const Vec &v) const
 Fill the vector 'v' with the content of the i-th row.

Static Protected Member Functions

static void declareOptions (OptionList &ol)
 Declares the class options.

Protected Attributes

Mat imputed_input
 Input parts with imputed missing values, for all samples that had some missing value in 'source'.
TVec< intsample_index_to_imputed_index
 The i-th element is the index in 'imputed_input' of the imputed input for the i-th sample in 'source'.

Private Types

typedef SourceVMatrix inherited

Private Member Functions

void build_ ()
 This does the actual building.

Detailed Description

The first sentence should be a BRIEF DESCRIPTION of what the class does.

Place the rest of the class programmer documentation here. Doxygen supports Javadoc-style comments. See http://www.doxygen.org/manual.html

Todo:
Write class to-do's here if there are any.
Deprecated:
Write deprecated stuff here if there is any. Indicate what else should be used instead.

Definition at line 57 of file KNNImputationVMatrix.h.


Member Typedef Documentation

Reimplemented from PLearn::SourceVMatrix.

Definition at line 59 of file KNNImputationVMatrix.h.


Constructor & Destructor Documentation

PLearn::KNNImputationVMatrix::KNNImputationVMatrix ( )

Default constructor.

Definition at line 73 of file KNNImputationVMatrix.cc.

                                          :
    knn(5),
    n_train_samples(-1),
    report_progress(true)
{}

Member Function Documentation

string PLearn::KNNImputationVMatrix::_classname_ ( ) [static]

Declares name and deepCopy methods.

Reimplemented from PLearn::SourceVMatrix.

Definition at line 68 of file KNNImputationVMatrix.cc.

OptionList & PLearn::KNNImputationVMatrix::_getOptionList_ ( ) [static]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 68 of file KNNImputationVMatrix.cc.

RemoteMethodMap & PLearn::KNNImputationVMatrix::_getRemoteMethodMap_ ( ) [static]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 68 of file KNNImputationVMatrix.cc.

bool PLearn::KNNImputationVMatrix::_isa_ ( const Object o) [static]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 68 of file KNNImputationVMatrix.cc.

Object * PLearn::KNNImputationVMatrix::_new_instance_for_typemap_ ( ) [static]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 68 of file KNNImputationVMatrix.cc.

StaticInitializer KNNImputationVMatrix::_static_initializer_ & PLearn::KNNImputationVMatrix::_static_initialize_ ( ) [static]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 68 of file KNNImputationVMatrix.cc.

void PLearn::KNNImputationVMatrix::build ( ) [virtual]

Simply calls inherited::build() then build_().

Reimplemented from PLearn::SourceVMatrix.

Definition at line 119 of file KNNImputationVMatrix.cc.

References PLearn::SourceVMatrix::build(), and build_().

Here is the call graph for this function:

void PLearn::KNNImputationVMatrix::build_ ( ) [private]

This does the actual building.

Reimplemented from PLearn::SourceVMatrix.

Definition at line 128 of file KNNImputationVMatrix.cc.

References PLearn::TMat< T >::appendRow(), PLearn::computeMean(), PLearn::TVec< T >::fill(), full_source, PLearn::VMat::getExample(), PLearn::TVec< T >::hasMissing(), i, imputed_input, PLearn::is_missing(), j, knn, PLearn::TMat< T >::length(), PLearn::TVec< T >::length(), PLearn::VMat::length(), PLearn::mean(), n_train_samples, neighbors, PLASSERT, report_progress, PLearn::TMat< T >::resize(), PLearn::TVec< T >::resize(), sample_index_to_imputed_index, PLearn::SourceVMatrix::setMetaInfoFromSource(), PLearn::SourceVMatrix::source, PLearn::VMatrix::updateMtime(), and PLearn::VMat::width().

Referenced by build().

{
    if (!source)
        return;

    PLASSERT( full_source );
    PLASSERT( neighbors || full_source->length() == source->length() );
    PLASSERT( full_source->width() == source->width()  );

    updateMtime(source);
    updateMtime(full_source);
    updateMtime(neighbors);

    VMat candidates;
    if (neighbors)
        candidates = full_source ? full_source : source;
    else
        candidates = full_source;
   
    if (n_train_samples > 0)
        candidates = new SubVMatrix(candidates, 0, 0, n_train_samples,
                                                      candidates->width());

    // Prepare nearest neighbor learner.
    PP<ExhaustiveNearestNeighbors> nn_learner =
        new ExhaustiveNearestNeighbors();
    nn_learner->num_neighbors = candidates->length();
    nn_learner->copy_target = false;
    nn_learner->copy_index = true;
    nn_learner->build();

    if (!neighbors) {
        nn_learner->setTrainingSet(candidates);
        nn_learner->train();
    }

    // Compute global mean.
    Vec global_mean;
    PLearn::computeMean(candidates, global_mean);

    // Perform actual missing values imputation.
    Vec input, target, output, input_nn;
    if (neighbors)
        output.resize(neighbors->width());
    real weight;
    imputed_input.resize(0, source->inputsize());
    Vec imputed_row(source->inputsize());
    sample_index_to_imputed_index.resize(source->length());
    sample_index_to_imputed_index.fill(-1);
    PP<ProgressBar> pb;
    if (report_progress)
        pb = new ProgressBar("Imputing missing values", source->length());
    for (int i = 0; i < source->length(); i++) {
        source->getExample(i, input, target, weight);
        if (input.hasMissing()) {
            if (neighbors)
                neighbors->getRow(i, output);
            else
                nn_learner->computeOutput(input, output);
            for (int k = 0; k < input.length(); k++)
                if (is_missing(input[k])) {
                    int j = 0;
                    int count_neighbors = 0;
                    real mean = 0;
                    while (count_neighbors < knn && j < output.length()) {
                        int neighbor_index = int(round(output[j]));
                        if (neighbors && full_source)
                            full_source->getExample(neighbor_index, input_nn,
                                                    target, weight);
                        else
                            source->getExample(neighbor_index, input_nn,
                                               target, weight);
                        if (!is_missing(input_nn[k])) {
                            mean += input_nn[k];
                            count_neighbors++;
                        }
                        j++;
                    }
                    if (count_neighbors > 0) {
                        // Found some neighbors with an observed value for
                        // variable 'k'.
                        mean /= count_neighbors;
                    } else {
                        mean = global_mean[k];
                    }
                    imputed_row[k] = mean;
                } else
                    imputed_row[k] = input[k];
            imputed_input.appendRow(imputed_row);
            sample_index_to_imputed_index[i] = imputed_input.length() - 1;
        }
        if (pb)
            pb->update(i + 1);
    }

    // Obtain meta information from source.
    setMetaInfoFromSource();
}

Here is the call graph for this function:

Here is the caller graph for this function:

string PLearn::KNNImputationVMatrix::classname ( ) const [virtual]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 68 of file KNNImputationVMatrix.cc.

void PLearn::KNNImputationVMatrix::declareOptions ( OptionList ol) [static, protected]

Declares the class options.

Reimplemented from PLearn::SourceVMatrix.

Definition at line 82 of file KNNImputationVMatrix.cc.

References PLearn::OptionBase::buildoption, PLearn::declareOption(), PLearn::SourceVMatrix::declareOptions(), full_source, knn, n_train_samples, neighbors, and report_progress.

{
    declareOption(ol, "knn", &KNNImputationVMatrix::knn,
                             OptionBase::buildoption,
        "Number of nearest neighbors considered.");

    declareOption(ol, "neighbors", &KNNImputationVMatrix::neighbors,
                                   OptionBase::buildoption,
        "Optional VMat that, if specified, contains in element (i,j) the\n"
        "j-th nearest neighbor of sample i, either in 'source' or (if\n"
        "provided) in 'full_source'.");

    declareOption(ol, "full_source", &KNNImputationVMatrix::full_source,
                                     OptionBase::buildoption,
        "If 'neighbors' is not provided, this is the same dataset as\n"
        "'source', but with no missing values.\n"
        "Otherwise, this is another dataset, possibly with missing values,\n"
        "that corresponds to the neighbors indexed in 'neighbors'.");

    declareOption(ol, "n_train_samples",
                  &KNNImputationVMatrix::n_train_samples,
                  OptionBase::buildoption,
        "If > 0, only samples in the first 'n_train_samples' will be\n"
        "considered candidate nearest neighbors.");

    declareOption(ol, "report_progress",
                  &KNNImputationVMatrix::report_progress,
                  OptionBase::buildoption,
        "Whether or not to display a progress bar.");

    // Now call the parent class' declareOptions
    inherited::declareOptions(ol);
}

Here is the call graph for this function:

static const PPath& PLearn::KNNImputationVMatrix::declaringFile ( ) [inline, static]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 84 of file KNNImputationVMatrix.h.

:

KNNImputationVMatrix * PLearn::KNNImputationVMatrix::deepCopy ( CopiesMap copies) const [virtual]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 68 of file KNNImputationVMatrix.cc.

void PLearn::KNNImputationVMatrix::getNewRow ( int  i,
const Vec v 
) const [protected, virtual]

Fill the vector 'v' with the content of the i-th row.

v is assumed to be the right size. ### This function must be overridden in your class

Reimplemented from PLearn::SourceVMatrix.

Definition at line 230 of file KNNImputationVMatrix.cc.

References PLearn::TVec< T >::hasMissing(), i, imputed_input, PLearn::VMatrix::inputsize_, PLASSERT, sample_index_to_imputed_index, PLearn::SourceVMatrix::source, and PLearn::TVec< T >::subVec().

{
    source->getRow(i, v);
    if (v.hasMissing()) {
        int idx = sample_index_to_imputed_index[i];
        PLASSERT( idx >= 0 );
        v.subVec(0, inputsize_) << imputed_input(idx);
    }
}

Here is the call graph for this function:

OptionList & PLearn::KNNImputationVMatrix::getOptionList ( ) const [virtual]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 68 of file KNNImputationVMatrix.cc.

OptionMap & PLearn::KNNImputationVMatrix::getOptionMap ( ) const [virtual]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 68 of file KNNImputationVMatrix.cc.

RemoteMethodMap & PLearn::KNNImputationVMatrix::getRemoteMethodMap ( ) const [virtual]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 68 of file KNNImputationVMatrix.cc.

void PLearn::KNNImputationVMatrix::makeDeepCopyFromShallowCopy ( CopiesMap copies) [virtual]

Transforms a shallow copy into a deep copy.

Reimplemented from PLearn::SourceVMatrix.

Definition at line 243 of file KNNImputationVMatrix.cc.

References PLearn::deepCopyField(), full_source, imputed_input, PLearn::SourceVMatrix::makeDeepCopyFromShallowCopy(), neighbors, and sample_index_to_imputed_index.

Here is the call graph for this function:


Member Data Documentation

Reimplemented from PLearn::SourceVMatrix.

Definition at line 84 of file KNNImputationVMatrix.h.

Definition at line 64 of file KNNImputationVMatrix.h.

Referenced by build_(), declareOptions(), and makeDeepCopyFromShallowCopy().

Input parts with imputed missing values, for all samples that had some missing value in 'source'.

Definition at line 97 of file KNNImputationVMatrix.h.

Referenced by build_(), getNewRow(), and makeDeepCopyFromShallowCopy().

Definition at line 65 of file KNNImputationVMatrix.h.

Referenced by build_(), and declareOptions().

Definition at line 67 of file KNNImputationVMatrix.h.

Referenced by build_(), and declareOptions().

Definition at line 66 of file KNNImputationVMatrix.h.

Referenced by build_(), declareOptions(), and makeDeepCopyFromShallowCopy().

Definition at line 68 of file KNNImputationVMatrix.h.

Referenced by build_(), and declareOptions().

The i-th element is the index in 'imputed_input' of the imputed input for the i-th sample in 'source'.

A value of -1 means this sample has no missing value.

Definition at line 102 of file KNNImputationVMatrix.h.

Referenced by build_(), getNewRow(), and makeDeepCopyFromShallowCopy().


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines