PLearn 0.1
Public Member Functions | Static Public Member Functions | Public Attributes | Static Public Attributes | Private Types | Private Member Functions | Private Attributes
PLearn::CovariancePreservationImputationVMatrix Class Reference

#include <CovariancePreservationImputationVMatrix.h>

Inheritance diagram for PLearn::CovariancePreservationImputationVMatrix:
Inheritance graph
[legend]
Collaboration diagram for PLearn::CovariancePreservationImputationVMatrix:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 CovariancePreservationImputationVMatrix ()
virtual ~CovariancePreservationImputationVMatrix ()
virtual void build ()
 Simply calls inherited::build() then build_().
virtual void makeDeepCopyFromShallowCopy (CopiesMap &copies)
 Does the necessary operations to transform a shallow copy (this) into a deep copy by deep-copying all the members that need to be.
virtual void getExample (int i, Vec &input, Vec &target, real &weight)
 Default version calls getSubRow based on inputsize_ targetsize_ weightsize_ But exotic subclasses may construct, input, target and weight however they please.
virtual real get (int i, int j) const
 This method must be implemented in all subclasses.
virtual void put (int i, int j, real value)
 This method must be implemented in all subclasses of writable matrices.
virtual void getSubRow (int i, int j, Vec v) const
 It is suggested that this method be implemented in subclasses to speed up accesses (default version repeatedly calls get(i,j) which may have a significant overhead).
virtual void putSubRow (int i, int j, Vec v)
 It is suggested that this method be implemented in subclasses of writable matrices to speed up accesses (default version repeatedly calls put(i,j,value) which may have a significant overhead)
virtual void appendRow (Vec v)
 This method must be implemented for matrices that are allowed to grow.
virtual void insertRow (int i, Vec v)
 This method must be implemented for matrices that are allowed to grow.
virtual void getRow (int i, Vec v) const
 These methods do not usually need to be overridden in subclasses (default versions call getSubRow, which should do just fine)
virtual void putRow (int i, Vec v)
virtual void getColumn (int i, Vec v) const
 Copies column i into v (which must have appropriate length equal to the VMat's length).
VMat getCovarianceFile ()
virtual string classname () const
virtual OptionListgetOptionList () const
virtual OptionMapgetOptionMap () const
virtual RemoteMethodMapgetRemoteMethodMap () const
virtual
CovariancePreservationImputationVMatrix
deepCopy (CopiesMap &copies) const

Static Public Member Functions

static void declareOptions (OptionList &ol)
 Declares this class' options.
static string _classname_ ()
 CovariancePreservationImputationVMatrix.
static OptionList_getOptionList_ ()
static RemoteMethodMap_getRemoteMethodMap_ ()
static Object_new_instance_for_typemap_ ()
static bool _isa_ (const Object *o)
static void _static_initialize_ ()
static const PPathdeclaringFile ()

Public Attributes

VMat train_set
 A referenced train set.

Static Public Attributes

static StaticInitializer _static_initializer_

Private Types

typedef ImputationVMatrix inherited

Private Member Functions

void build_ ()
 This does the actual building.
void createCovarianceFile ()
void loadCovarianceFile ()
void computeCovariances ()
real computeImputation (int row, int col) const
real computeImputation (int row, int col, Vec input) const

Private Attributes

int train_length
int train_width
int train_inputsize
int train_targetsize
int train_weightsize
int train_row
Vec train_input
TVec< string > train_field_names
PPath train_metadata
int source_length
int source_width
int source_inputsize
int source_targetsize
int source_weightsize
PPath covariance_file_name
VMat covariance_file
int indj
int indk
Mat n_obs
Mat sum_xj
Mat sum_xj_xk
Vec mu
Mat cov

Detailed Description

Definition at line 56 of file CovariancePreservationImputationVMatrix.h.


Member Typedef Documentation

Reimplemented from PLearn::ImputationVMatrix.

Definition at line 58 of file CovariancePreservationImputationVMatrix.h.


Constructor & Destructor Documentation

PLearn::CovariancePreservationImputationVMatrix::CovariancePreservationImputationVMatrix ( )

Definition at line 57 of file CovariancePreservationImputationVMatrix.cc.

{
}
PLearn::CovariancePreservationImputationVMatrix::~CovariancePreservationImputationVMatrix ( ) [virtual]

Definition at line 61 of file CovariancePreservationImputationVMatrix.cc.

{
}

Member Function Documentation

string PLearn::CovariancePreservationImputationVMatrix::_classname_ ( ) [static]
OptionList & PLearn::CovariancePreservationImputationVMatrix::_getOptionList_ ( ) [static]

Reimplemented from PLearn::ImputationVMatrix.

Definition at line 55 of file CovariancePreservationImputationVMatrix.cc.

RemoteMethodMap & PLearn::CovariancePreservationImputationVMatrix::_getRemoteMethodMap_ ( ) [static]

Reimplemented from PLearn::ImputationVMatrix.

Definition at line 55 of file CovariancePreservationImputationVMatrix.cc.

bool PLearn::CovariancePreservationImputationVMatrix::_isa_ ( const Object o) [static]

Reimplemented from PLearn::ImputationVMatrix.

Definition at line 55 of file CovariancePreservationImputationVMatrix.cc.

Object * PLearn::CovariancePreservationImputationVMatrix::_new_instance_for_typemap_ ( ) [static]

Reimplemented from PLearn::Object.

Definition at line 55 of file CovariancePreservationImputationVMatrix.cc.

StaticInitializer CovariancePreservationImputationVMatrix::_static_initializer_ & PLearn::CovariancePreservationImputationVMatrix::_static_initialize_ ( ) [static]

Reimplemented from PLearn::ImputationVMatrix.

Definition at line 55 of file CovariancePreservationImputationVMatrix.cc.

void PLearn::CovariancePreservationImputationVMatrix::appendRow ( Vec  v) [virtual]

This method must be implemented for matrices that are allowed to grow.

Reimplemented from PLearn::VMatrix.

Definition at line 120 of file CovariancePreservationImputationVMatrix.cc.

References PLERROR.

{
  PLERROR("In CovariancePreservationImputationVMatrix::appendRow not implemented");
}
void PLearn::CovariancePreservationImputationVMatrix::build ( ) [virtual]

Simply calls inherited::build() then build_().

Reimplemented from PLearn::ImputationVMatrix.

Definition at line 75 of file CovariancePreservationImputationVMatrix.cc.

void PLearn::CovariancePreservationImputationVMatrix::build_ ( ) [private]

This does the actual building.

Reimplemented from PLearn::ImputationVMatrix.

Definition at line 151 of file CovariancePreservationImputationVMatrix.cc.

References PLearn::isfile(), and PLERROR.

{
    if (!train_set || !source) PLERROR("In CovariancePreservationImputationVMatrix::train set and source vmat must be supplied");
    train_length = train_set->length();
    if(train_length < 1) PLERROR("In CovariancePreservationImputationVMatrix::length of the number of train samples to use must be at least 1, got: %i", train_length);
    train_width = train_set->width();
    train_targetsize = train_set->targetsize();
    train_weightsize = train_set->weightsize();
    train_inputsize = train_set->inputsize();
    if(train_inputsize < 1) PLERROR("In CovariancePreservationImputationVMatrix::inputsize of the train vmat must be supplied, got : %i", train_inputsize);
    source_width = source->width();
    source_targetsize = source->targetsize();
    source_weightsize = source->weightsize();
    source_inputsize = source->inputsize();
    if (train_width != source_width) PLERROR("In CovariancePreservationImputationVMatrix::train set and source width must agree, got : %i, %i", train_width, source_width);
    if (train_targetsize != source_targetsize) PLERROR("In CovariancePreservationImputationVMatrix::train set and source targetsize must agree, got : %i, %i", train_targetsize, source_targetsize);
    if (train_weightsize != source_weightsize) PLERROR("In CovariancePreservationImputationVMatrix::train set and source weightsize must agree, got : %i, %i", train_weightsize, source_weightsize);
    if (train_inputsize != source_inputsize) PLERROR("In CovariancePreservationImputationVMatrix::train set and source inputsize must agree, got : %i, %i", train_inputsize, source_inputsize);
    train_field_names.resize(train_width);
    train_field_names = train_set->fieldNames();
    source_length = source->length();
    length_ = source_length;
    width_ = source_width;
    inputsize_ = source_inputsize;
    targetsize_ = source_targetsize;
    weightsize_ = source_weightsize;
    declareFieldNames(train_field_names);
    train_metadata = train_set->getMetaDataDir();
    covariance_file_name = train_metadata + "covariance_file.pmat";
    cov.resize(train_width, train_width);
    mu.resize(train_width);
    if (!isfile(covariance_file_name))
    {
        computeCovariances();
        createCovarianceFile();
    }
    else loadCovarianceFile();
}

Here is the call graph for this function:

string PLearn::CovariancePreservationImputationVMatrix::classname ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 55 of file CovariancePreservationImputationVMatrix.cc.

void PLearn::CovariancePreservationImputationVMatrix::computeCovariances ( ) [private]

Definition at line 227 of file CovariancePreservationImputationVMatrix.cc.

References PLearn::is_missing(), and PLearn::ProgressBar::update().

{
/*
    We need to populate the matrix of COV for all combinations of input variables
    we need in one pass to populate 4 matrices of dxd:
    n(j,k) the number of samples where x(i, j) and x(i, k) are simultaneously observed.
    sum_x(j)(k) the sum of the x(i, j) values where x(i, j) and x(i, k) are simultaneously observed.
    sum_x(j)_x(k) the sum of the x(i, j)*x(i, k) values where x(i, j) and x(i, k) are simultaneously observed.
    we can the calculate mu(k) = sum_x(k, k)/n(k, k)
    COV(j, k) = (sum_x(j)_x(k) - sum_x(j)(k) * mu(k) - sum_x(k)(j) * mu(j) + mu(k) * mu(j)) (1 / n(j,k))
    All we need after is the COV matrix to impute values on missing values.
    
*/
    n_obs.resize(train_width, train_width);
    sum_xj.resize(train_width, train_width);
    sum_xj_xk.resize(train_width, train_width);
    train_input.resize(train_width);
    n_obs.clear();
    sum_xj.clear();
    sum_xj_xk.clear();
    mu.clear();
    cov.clear();
    ProgressBar* pb = 0;
    pb = new ProgressBar("Computing the covariance matrix", train_length);
    for (train_row = 0; train_row < train_length; train_row++)
    {
        train_set->getRow(train_row, train_input);
        for (indj = 0; indj < train_width; indj++)
        {
            for (indk = 0; indk < train_width; indk++)
            {
                if (is_missing(train_input[indj]) || is_missing(train_input[indk])) continue;
                n_obs(indj, indk) += 1.0;
                sum_xj(indj, indk) += train_input[indj];
                sum_xj_xk(indj, indk) += train_input[indj] * train_input[indk];
            }
        }
        pb->update( train_row ); 
    }
    delete pb;
    for (indj = 0; indj < train_width; indj++)
    {
        mu[indj] = sum_xj(indj, indj) / n_obs(indj, indj); 
    }
    for (indj = 0; indj < train_width; indj++)
    {
        for (indk = 0; indk < train_width; indk++)
        {
            cov(indj, indk) = sum_xj_xk(indj, indk) - sum_xj(indj, indk) * mu[indk] - sum_xj(indk, indj) * mu[indj];
            cov(indj, indk) = (cov(indj, indk) /  n_obs(indj, indk)) + mu[indk] * mu[indj];
        }
    }
}

Here is the call graph for this function:

real PLearn::CovariancePreservationImputationVMatrix::computeImputation ( int  row,
int  col 
) const [private]

Definition at line 281 of file CovariancePreservationImputationVMatrix.cc.

{
    Vec input(source_width);
    source->getRow(row, input);
    return computeImputation(row, col, input);
}
real PLearn::CovariancePreservationImputationVMatrix::computeImputation ( int  row,
int  col,
Vec  input 
) const [private]

Definition at line 288 of file CovariancePreservationImputationVMatrix.cc.

References PLearn::is_missing().

{
    real sum_cov_xl = 0;
    real sum_xl_square = 0;
    for (int indl = 0; indl < source_width; indl++)
    {
        if (is_missing(input[indl])) continue;
        sum_cov_xl += cov(indl, col) * (input[indl] - mu[indl]);
        sum_xl_square += (input[indl] - mu[indl]) * (input[indl] - mu[indl]);
    }
    if (sum_xl_square == 0.0) return mu[col];
    return mu[col] + sum_cov_xl / sum_xl_square;
}

Here is the call graph for this function:

void PLearn::CovariancePreservationImputationVMatrix::createCovarianceFile ( ) [private]

Definition at line 190 of file CovariancePreservationImputationVMatrix.cc.

{
    covariance_file = new FileVMatrix(covariance_file_name, train_width + 1, train_field_names);
    for (indj = 0; indj < train_width; indj++)
    {
        for (indk = 0; indk < train_width; indk++)
        {
            covariance_file->put(indj, indk, cov(indj, indk));
        }
    }
    for (indk = 0; indk < train_width; indk++)
    {
        covariance_file->put(train_width, indk, mu[indk]);
    }
}
void PLearn::CovariancePreservationImputationVMatrix::declareOptions ( OptionList ol) [static]

Declares this class' options.

Reimplemented from PLearn::ImputationVMatrix.

Definition at line 65 of file CovariancePreservationImputationVMatrix.cc.

References PLearn::OptionBase::buildoption, PLearn::declareOption(), and train_set.

{

  declareOption(ol, "train_set", &CovariancePreservationImputationVMatrix::train_set, OptionBase::buildoption, 
                "A referenced train set.\n"
                "The covariance imputation is computed with the observed values in this data set.\n");

  inherited::declareOptions(ol);
}

Here is the call graph for this function:

static const PPath& PLearn::CovariancePreservationImputationVMatrix::declaringFile ( ) [inline, static]

Reimplemented from PLearn::ImputationVMatrix.

Definition at line 120 of file CovariancePreservationImputationVMatrix.h.

CovariancePreservationImputationVMatrix * PLearn::CovariancePreservationImputationVMatrix::deepCopy ( CopiesMap copies) const [virtual]

Reimplemented from PLearn::ImputationVMatrix.

Definition at line 55 of file CovariancePreservationImputationVMatrix.cc.

real PLearn::CovariancePreservationImputationVMatrix::get ( int  i,
int  j 
) const [virtual]

This method must be implemented in all subclasses.

Returns element (i,j).

Implements PLearn::VMatrix.

Definition at line 96 of file CovariancePreservationImputationVMatrix.cc.

References PLearn::is_missing().

{ 
  real variable_value = source->get(i, j);
  if (is_missing(variable_value)) computeImputation(i, j);
  return variable_value;
}

Here is the call graph for this function:

void PLearn::CovariancePreservationImputationVMatrix::getColumn ( int  i,
Vec  v 
) const [virtual]

Copies column i into v (which must have appropriate length equal to the VMat's length).

Reimplemented from PLearn::VMatrix.

Definition at line 142 of file CovariancePreservationImputationVMatrix.cc.

References PLearn::is_missing(), and PLearn::TVec< T >::length().

{  
  source-> getColumn(i, v);
  for (int source_row = 0; source_row < v->length(); source_row++)
    if (is_missing(v[source_row])) v[source_row] = computeImputation(source_row, i);
}

Here is the call graph for this function:

VMat PLearn::CovariancePreservationImputationVMatrix::getCovarianceFile ( )

Definition at line 222 of file CovariancePreservationImputationVMatrix.cc.

{
    return covariance_file;
}
void PLearn::CovariancePreservationImputationVMatrix::getExample ( int  i,
Vec input,
Vec target,
real weight 
) [virtual]

Default version calls getSubRow based on inputsize_ targetsize_ weightsize_ But exotic subclasses may construct, input, target and weight however they please.

If not a weighted matrix, weight should be set to default value 1.

Reimplemented from PLearn::VMatrix.

Definition at line 87 of file CovariancePreservationImputationVMatrix.cc.

References PLearn::is_missing(), and PLearn::TVec< T >::length().

{
  source->getExample(i, input, target, weight);
  for (int source_col = 0; source_col < input->length(); source_col++)
  {
    if (is_missing(input[source_col])) input[source_col] = computeImputation(i, source_col, input);
  }  
}

Here is the call graph for this function:

OptionList & PLearn::CovariancePreservationImputationVMatrix::getOptionList ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 55 of file CovariancePreservationImputationVMatrix.cc.

OptionMap & PLearn::CovariancePreservationImputationVMatrix::getOptionMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 55 of file CovariancePreservationImputationVMatrix.cc.

RemoteMethodMap & PLearn::CovariancePreservationImputationVMatrix::getRemoteMethodMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 55 of file CovariancePreservationImputationVMatrix.cc.

void PLearn::CovariancePreservationImputationVMatrix::getRow ( int  i,
Vec  v 
) const [virtual]

These methods do not usually need to be overridden in subclasses (default versions call getSubRow, which should do just fine)

Copies row i into v (which must have appropriate length equal to the VMat's width).

Reimplemented from PLearn::VMatrix.

Definition at line 130 of file CovariancePreservationImputationVMatrix.cc.

References PLearn::is_missing(), and PLearn::TVec< T >::length().

{  
  source-> getRow(i, v);
  for (int source_col = 0; source_col < v->length(); source_col++)
    if (is_missing(v[source_col])) v[source_col] = computeImputation(i, source_col, v);
}

Here is the call graph for this function:

void PLearn::CovariancePreservationImputationVMatrix::getSubRow ( int  i,
int  j,
Vec  v 
) const [virtual]

It is suggested that this method be implemented in subclasses to speed up accesses (default version repeatedly calls get(i,j) which may have a significant overhead).

Fills v with the subrow i lying between columns j (inclusive) and j+v.length() (exclusive).

Reimplemented from PLearn::VMatrix.

Definition at line 108 of file CovariancePreservationImputationVMatrix.cc.

References PLearn::is_missing(), and PLearn::TVec< T >::length().

{  
  source->getSubRow(i, j, v);
  for (int source_col = 0; source_col < v->length(); source_col++) 
    if (is_missing(v[source_col])) v[source_col] = computeImputation(i, source_col + j);
}

Here is the call graph for this function:

void PLearn::CovariancePreservationImputationVMatrix::insertRow ( int  i,
Vec  v 
) [virtual]

This method must be implemented for matrices that are allowed to grow.

Reimplemented from PLearn::VMatrix.

Definition at line 125 of file CovariancePreservationImputationVMatrix.cc.

References PLERROR.

{
  PLERROR("In CovariancePreservationImputationVMatrix::insertRow not implemented");
}
void PLearn::CovariancePreservationImputationVMatrix::loadCovarianceFile ( ) [private]

Definition at line 206 of file CovariancePreservationImputationVMatrix.cc.

{
    covariance_file = new FileVMatrix(covariance_file_name);
    for (indj = 0; indj < train_width; indj++)
    {
        for (indk = 0; indk < train_width; indk++)
        {
            cov(indj, indk) = covariance_file->get(indj, indk);
        }
    }
    for (indk = 0; indk < train_width; indk++)
    {
        mu[indk] = covariance_file->get(train_width, indk);
    }
}
void PLearn::CovariancePreservationImputationVMatrix::makeDeepCopyFromShallowCopy ( CopiesMap copies) [virtual]

Does the necessary operations to transform a shallow copy (this) into a deep copy by deep-copying all the members that need to be.

This needs to be overridden by every class that adds "complex" data members to the class, such as Vec, Mat, PP<Something>, etc. Typical implementation:

  void CLASS_OF_THIS::makeDeepCopyFromShallowCopy(CopiesMap& copies)
  {
      inherited::makeDeepCopyFromShallowCopy(copies);
      deepCopyField(complex_data_member1, copies);
      deepCopyField(complex_data_member2, copies);
      ...
  }
Parameters:
copiesA map used by the deep-copy mechanism to keep track of already-copied objects.

Reimplemented from PLearn::ImputationVMatrix.

Definition at line 81 of file CovariancePreservationImputationVMatrix.cc.

References PLearn::deepCopyField().

Here is the call graph for this function:

void PLearn::CovariancePreservationImputationVMatrix::put ( int  i,
int  j,
real  value 
) [virtual]

This method must be implemented in all subclasses of writable matrices.

Sets element (i,j) to value.

Reimplemented from PLearn::VMatrix.

Definition at line 103 of file CovariancePreservationImputationVMatrix.cc.

References PLERROR.

{
  PLERROR("In CovariancePreservationImputationVMatrix::put not implemented");
}
void PLearn::CovariancePreservationImputationVMatrix::putRow ( int  i,
Vec  v 
) [virtual]

Reimplemented from PLearn::VMatrix.

Definition at line 137 of file CovariancePreservationImputationVMatrix.cc.

References PLERROR.

{
  PLERROR("In CovariancePreservationImputationVMatrix::putRow not implemented");
}
void PLearn::CovariancePreservationImputationVMatrix::putSubRow ( int  i,
int  j,
Vec  v 
) [virtual]

It is suggested that this method be implemented in subclasses of writable matrices to speed up accesses (default version repeatedly calls put(i,j,value) which may have a significant overhead)

Reimplemented from PLearn::VMatrix.

Definition at line 115 of file CovariancePreservationImputationVMatrix.cc.

References PLERROR.

{
  PLERROR("In CovariancePreservationImputationVMatrix::putSubRow not implemented");
}

Member Data Documentation

Reimplemented from PLearn::ImputationVMatrix.

Definition at line 120 of file CovariancePreservationImputationVMatrix.h.

Definition at line 111 of file CovariancePreservationImputationVMatrix.h.

Definition at line 104 of file CovariancePreservationImputationVMatrix.h.

Definition at line 103 of file CovariancePreservationImputationVMatrix.h.

Definition at line 105 of file CovariancePreservationImputationVMatrix.h.

Definition at line 106 of file CovariancePreservationImputationVMatrix.h.

Definition at line 110 of file CovariancePreservationImputationVMatrix.h.

Definition at line 107 of file CovariancePreservationImputationVMatrix.h.

Definition at line 100 of file CovariancePreservationImputationVMatrix.h.

Definition at line 98 of file CovariancePreservationImputationVMatrix.h.

Definition at line 101 of file CovariancePreservationImputationVMatrix.h.

Definition at line 102 of file CovariancePreservationImputationVMatrix.h.

Definition at line 99 of file CovariancePreservationImputationVMatrix.h.

Definition at line 108 of file CovariancePreservationImputationVMatrix.h.

Definition at line 109 of file CovariancePreservationImputationVMatrix.h.

Definition at line 96 of file CovariancePreservationImputationVMatrix.h.

Definition at line 95 of file CovariancePreservationImputationVMatrix.h.

Definition at line 91 of file CovariancePreservationImputationVMatrix.h.

Definition at line 89 of file CovariancePreservationImputationVMatrix.h.

Definition at line 97 of file CovariancePreservationImputationVMatrix.h.

Definition at line 94 of file CovariancePreservationImputationVMatrix.h.

A referenced train set.

The covariance imputation is computed with the observed values in this data set.

Definition at line 64 of file CovariancePreservationImputationVMatrix.h.

Referenced by declareOptions().

Definition at line 92 of file CovariancePreservationImputationVMatrix.h.

Definition at line 93 of file CovariancePreservationImputationVMatrix.h.

Definition at line 90 of file CovariancePreservationImputationVMatrix.h.


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines