PLearn 0.1
|
#include <CovariancePreservationImputationVMatrix.h>
Public Member Functions | |
CovariancePreservationImputationVMatrix () | |
virtual | ~CovariancePreservationImputationVMatrix () |
virtual void | build () |
Simply calls inherited::build() then build_(). | |
virtual void | makeDeepCopyFromShallowCopy (CopiesMap &copies) |
Does the necessary operations to transform a shallow copy (this) into a deep copy by deep-copying all the members that need to be. | |
virtual void | getExample (int i, Vec &input, Vec &target, real &weight) |
Default version calls getSubRow based on inputsize_ targetsize_ weightsize_ But exotic subclasses may construct, input, target and weight however they please. | |
virtual real | get (int i, int j) const |
This method must be implemented in all subclasses. | |
virtual void | put (int i, int j, real value) |
This method must be implemented in all subclasses of writable matrices. | |
virtual void | getSubRow (int i, int j, Vec v) const |
It is suggested that this method be implemented in subclasses to speed up accesses (default version repeatedly calls get(i,j) which may have a significant overhead). | |
virtual void | putSubRow (int i, int j, Vec v) |
It is suggested that this method be implemented in subclasses of writable matrices to speed up accesses (default version repeatedly calls put(i,j,value) which may have a significant overhead) | |
virtual void | appendRow (Vec v) |
This method must be implemented for matrices that are allowed to grow. | |
virtual void | insertRow (int i, Vec v) |
This method must be implemented for matrices that are allowed to grow. | |
virtual void | getRow (int i, Vec v) const |
These methods do not usually need to be overridden in subclasses (default versions call getSubRow, which should do just fine) | |
virtual void | putRow (int i, Vec v) |
virtual void | getColumn (int i, Vec v) const |
Copies column i into v (which must have appropriate length equal to the VMat's length). | |
VMat | getCovarianceFile () |
virtual string | classname () const |
virtual OptionList & | getOptionList () const |
virtual OptionMap & | getOptionMap () const |
virtual RemoteMethodMap & | getRemoteMethodMap () const |
virtual CovariancePreservationImputationVMatrix * | deepCopy (CopiesMap &copies) const |
Static Public Member Functions | |
static void | declareOptions (OptionList &ol) |
Declares this class' options. | |
static string | _classname_ () |
CovariancePreservationImputationVMatrix. | |
static OptionList & | _getOptionList_ () |
static RemoteMethodMap & | _getRemoteMethodMap_ () |
static Object * | _new_instance_for_typemap_ () |
static bool | _isa_ (const Object *o) |
static void | _static_initialize_ () |
static const PPath & | declaringFile () |
Public Attributes | |
VMat | train_set |
A referenced train set. | |
Static Public Attributes | |
static StaticInitializer | _static_initializer_ |
Private Types | |
typedef ImputationVMatrix | inherited |
Private Member Functions | |
void | build_ () |
This does the actual building. | |
void | createCovarianceFile () |
void | loadCovarianceFile () |
void | computeCovariances () |
real | computeImputation (int row, int col) const |
real | computeImputation (int row, int col, Vec input) const |
Private Attributes | |
int | train_length |
int | train_width |
int | train_inputsize |
int | train_targetsize |
int | train_weightsize |
int | train_row |
Vec | train_input |
TVec< string > | train_field_names |
PPath | train_metadata |
int | source_length |
int | source_width |
int | source_inputsize |
int | source_targetsize |
int | source_weightsize |
PPath | covariance_file_name |
VMat | covariance_file |
int | indj |
int | indk |
Mat | n_obs |
Mat | sum_xj |
Mat | sum_xj_xk |
Vec | mu |
Mat | cov |
Definition at line 56 of file CovariancePreservationImputationVMatrix.h.
typedef ImputationVMatrix PLearn::CovariancePreservationImputationVMatrix::inherited [private] |
Reimplemented from PLearn::ImputationVMatrix.
Definition at line 58 of file CovariancePreservationImputationVMatrix.h.
PLearn::CovariancePreservationImputationVMatrix::CovariancePreservationImputationVMatrix | ( | ) |
Definition at line 57 of file CovariancePreservationImputationVMatrix.cc.
{ }
PLearn::CovariancePreservationImputationVMatrix::~CovariancePreservationImputationVMatrix | ( | ) | [virtual] |
Definition at line 61 of file CovariancePreservationImputationVMatrix.cc.
{ }
string PLearn::CovariancePreservationImputationVMatrix::_classname_ | ( | ) | [static] |
CovariancePreservationImputationVMatrix.
Reimplemented from PLearn::ImputationVMatrix.
Definition at line 55 of file CovariancePreservationImputationVMatrix.cc.
OptionList & PLearn::CovariancePreservationImputationVMatrix::_getOptionList_ | ( | ) | [static] |
Reimplemented from PLearn::ImputationVMatrix.
Definition at line 55 of file CovariancePreservationImputationVMatrix.cc.
RemoteMethodMap & PLearn::CovariancePreservationImputationVMatrix::_getRemoteMethodMap_ | ( | ) | [static] |
Reimplemented from PLearn::ImputationVMatrix.
Definition at line 55 of file CovariancePreservationImputationVMatrix.cc.
Reimplemented from PLearn::ImputationVMatrix.
Definition at line 55 of file CovariancePreservationImputationVMatrix.cc.
Object * PLearn::CovariancePreservationImputationVMatrix::_new_instance_for_typemap_ | ( | ) | [static] |
Reimplemented from PLearn::Object.
Definition at line 55 of file CovariancePreservationImputationVMatrix.cc.
StaticInitializer CovariancePreservationImputationVMatrix::_static_initializer_ & PLearn::CovariancePreservationImputationVMatrix::_static_initialize_ | ( | ) | [static] |
Reimplemented from PLearn::ImputationVMatrix.
Definition at line 55 of file CovariancePreservationImputationVMatrix.cc.
void PLearn::CovariancePreservationImputationVMatrix::appendRow | ( | Vec | v | ) | [virtual] |
This method must be implemented for matrices that are allowed to grow.
Reimplemented from PLearn::VMatrix.
Definition at line 120 of file CovariancePreservationImputationVMatrix.cc.
References PLERROR.
{ PLERROR("In CovariancePreservationImputationVMatrix::appendRow not implemented"); }
void PLearn::CovariancePreservationImputationVMatrix::build | ( | ) | [virtual] |
Simply calls inherited::build() then build_().
Reimplemented from PLearn::ImputationVMatrix.
Definition at line 75 of file CovariancePreservationImputationVMatrix.cc.
{ inherited::build(); build_(); }
void PLearn::CovariancePreservationImputationVMatrix::build_ | ( | ) | [private] |
This does the actual building.
Reimplemented from PLearn::ImputationVMatrix.
Definition at line 151 of file CovariancePreservationImputationVMatrix.cc.
References PLearn::isfile(), and PLERROR.
{ if (!train_set || !source) PLERROR("In CovariancePreservationImputationVMatrix::train set and source vmat must be supplied"); train_length = train_set->length(); if(train_length < 1) PLERROR("In CovariancePreservationImputationVMatrix::length of the number of train samples to use must be at least 1, got: %i", train_length); train_width = train_set->width(); train_targetsize = train_set->targetsize(); train_weightsize = train_set->weightsize(); train_inputsize = train_set->inputsize(); if(train_inputsize < 1) PLERROR("In CovariancePreservationImputationVMatrix::inputsize of the train vmat must be supplied, got : %i", train_inputsize); source_width = source->width(); source_targetsize = source->targetsize(); source_weightsize = source->weightsize(); source_inputsize = source->inputsize(); if (train_width != source_width) PLERROR("In CovariancePreservationImputationVMatrix::train set and source width must agree, got : %i, %i", train_width, source_width); if (train_targetsize != source_targetsize) PLERROR("In CovariancePreservationImputationVMatrix::train set and source targetsize must agree, got : %i, %i", train_targetsize, source_targetsize); if (train_weightsize != source_weightsize) PLERROR("In CovariancePreservationImputationVMatrix::train set and source weightsize must agree, got : %i, %i", train_weightsize, source_weightsize); if (train_inputsize != source_inputsize) PLERROR("In CovariancePreservationImputationVMatrix::train set and source inputsize must agree, got : %i, %i", train_inputsize, source_inputsize); train_field_names.resize(train_width); train_field_names = train_set->fieldNames(); source_length = source->length(); length_ = source_length; width_ = source_width; inputsize_ = source_inputsize; targetsize_ = source_targetsize; weightsize_ = source_weightsize; declareFieldNames(train_field_names); train_metadata = train_set->getMetaDataDir(); covariance_file_name = train_metadata + "covariance_file.pmat"; cov.resize(train_width, train_width); mu.resize(train_width); if (!isfile(covariance_file_name)) { computeCovariances(); createCovarianceFile(); } else loadCovarianceFile(); }
string PLearn::CovariancePreservationImputationVMatrix::classname | ( | ) | const [virtual] |
Reimplemented from PLearn::Object.
Definition at line 55 of file CovariancePreservationImputationVMatrix.cc.
void PLearn::CovariancePreservationImputationVMatrix::computeCovariances | ( | ) | [private] |
Definition at line 227 of file CovariancePreservationImputationVMatrix.cc.
References PLearn::is_missing(), and PLearn::ProgressBar::update().
{ /* We need to populate the matrix of COV for all combinations of input variables we need in one pass to populate 4 matrices of dxd: n(j,k) the number of samples where x(i, j) and x(i, k) are simultaneously observed. sum_x(j)(k) the sum of the x(i, j) values where x(i, j) and x(i, k) are simultaneously observed. sum_x(j)_x(k) the sum of the x(i, j)*x(i, k) values where x(i, j) and x(i, k) are simultaneously observed. we can the calculate mu(k) = sum_x(k, k)/n(k, k) COV(j, k) = (sum_x(j)_x(k) - sum_x(j)(k) * mu(k) - sum_x(k)(j) * mu(j) + mu(k) * mu(j)) (1 / n(j,k)) All we need after is the COV matrix to impute values on missing values. */ n_obs.resize(train_width, train_width); sum_xj.resize(train_width, train_width); sum_xj_xk.resize(train_width, train_width); train_input.resize(train_width); n_obs.clear(); sum_xj.clear(); sum_xj_xk.clear(); mu.clear(); cov.clear(); ProgressBar* pb = 0; pb = new ProgressBar("Computing the covariance matrix", train_length); for (train_row = 0; train_row < train_length; train_row++) { train_set->getRow(train_row, train_input); for (indj = 0; indj < train_width; indj++) { for (indk = 0; indk < train_width; indk++) { if (is_missing(train_input[indj]) || is_missing(train_input[indk])) continue; n_obs(indj, indk) += 1.0; sum_xj(indj, indk) += train_input[indj]; sum_xj_xk(indj, indk) += train_input[indj] * train_input[indk]; } } pb->update( train_row ); } delete pb; for (indj = 0; indj < train_width; indj++) { mu[indj] = sum_xj(indj, indj) / n_obs(indj, indj); } for (indj = 0; indj < train_width; indj++) { for (indk = 0; indk < train_width; indk++) { cov(indj, indk) = sum_xj_xk(indj, indk) - sum_xj(indj, indk) * mu[indk] - sum_xj(indk, indj) * mu[indj]; cov(indj, indk) = (cov(indj, indk) / n_obs(indj, indk)) + mu[indk] * mu[indj]; } } }
real PLearn::CovariancePreservationImputationVMatrix::computeImputation | ( | int | row, |
int | col | ||
) | const [private] |
Definition at line 281 of file CovariancePreservationImputationVMatrix.cc.
{ Vec input(source_width); source->getRow(row, input); return computeImputation(row, col, input); }
real PLearn::CovariancePreservationImputationVMatrix::computeImputation | ( | int | row, |
int | col, | ||
Vec | input | ||
) | const [private] |
Definition at line 288 of file CovariancePreservationImputationVMatrix.cc.
References PLearn::is_missing().
{ real sum_cov_xl = 0; real sum_xl_square = 0; for (int indl = 0; indl < source_width; indl++) { if (is_missing(input[indl])) continue; sum_cov_xl += cov(indl, col) * (input[indl] - mu[indl]); sum_xl_square += (input[indl] - mu[indl]) * (input[indl] - mu[indl]); } if (sum_xl_square == 0.0) return mu[col]; return mu[col] + sum_cov_xl / sum_xl_square; }
void PLearn::CovariancePreservationImputationVMatrix::createCovarianceFile | ( | ) | [private] |
Definition at line 190 of file CovariancePreservationImputationVMatrix.cc.
{ covariance_file = new FileVMatrix(covariance_file_name, train_width + 1, train_field_names); for (indj = 0; indj < train_width; indj++) { for (indk = 0; indk < train_width; indk++) { covariance_file->put(indj, indk, cov(indj, indk)); } } for (indk = 0; indk < train_width; indk++) { covariance_file->put(train_width, indk, mu[indk]); } }
void PLearn::CovariancePreservationImputationVMatrix::declareOptions | ( | OptionList & | ol | ) | [static] |
Declares this class' options.
Reimplemented from PLearn::ImputationVMatrix.
Definition at line 65 of file CovariancePreservationImputationVMatrix.cc.
References PLearn::OptionBase::buildoption, PLearn::declareOption(), and train_set.
{ declareOption(ol, "train_set", &CovariancePreservationImputationVMatrix::train_set, OptionBase::buildoption, "A referenced train set.\n" "The covariance imputation is computed with the observed values in this data set.\n"); inherited::declareOptions(ol); }
static const PPath& PLearn::CovariancePreservationImputationVMatrix::declaringFile | ( | ) | [inline, static] |
Reimplemented from PLearn::ImputationVMatrix.
Definition at line 120 of file CovariancePreservationImputationVMatrix.h.
CovariancePreservationImputationVMatrix * PLearn::CovariancePreservationImputationVMatrix::deepCopy | ( | CopiesMap & | copies | ) | const [virtual] |
Reimplemented from PLearn::ImputationVMatrix.
Definition at line 55 of file CovariancePreservationImputationVMatrix.cc.
This method must be implemented in all subclasses.
Returns element (i,j).
Implements PLearn::VMatrix.
Definition at line 96 of file CovariancePreservationImputationVMatrix.cc.
References PLearn::is_missing().
{ real variable_value = source->get(i, j); if (is_missing(variable_value)) computeImputation(i, j); return variable_value; }
Copies column i into v (which must have appropriate length equal to the VMat's length).
Reimplemented from PLearn::VMatrix.
Definition at line 142 of file CovariancePreservationImputationVMatrix.cc.
References PLearn::is_missing(), and PLearn::TVec< T >::length().
{ source-> getColumn(i, v); for (int source_row = 0; source_row < v->length(); source_row++) if (is_missing(v[source_row])) v[source_row] = computeImputation(source_row, i); }
VMat PLearn::CovariancePreservationImputationVMatrix::getCovarianceFile | ( | ) |
Definition at line 222 of file CovariancePreservationImputationVMatrix.cc.
{ return covariance_file; }
void PLearn::CovariancePreservationImputationVMatrix::getExample | ( | int | i, |
Vec & | input, | ||
Vec & | target, | ||
real & | weight | ||
) | [virtual] |
Default version calls getSubRow based on inputsize_ targetsize_ weightsize_ But exotic subclasses may construct, input, target and weight however they please.
If not a weighted matrix, weight should be set to default value 1.
Reimplemented from PLearn::VMatrix.
Definition at line 87 of file CovariancePreservationImputationVMatrix.cc.
References PLearn::is_missing(), and PLearn::TVec< T >::length().
{ source->getExample(i, input, target, weight); for (int source_col = 0; source_col < input->length(); source_col++) { if (is_missing(input[source_col])) input[source_col] = computeImputation(i, source_col, input); } }
OptionList & PLearn::CovariancePreservationImputationVMatrix::getOptionList | ( | ) | const [virtual] |
Reimplemented from PLearn::Object.
Definition at line 55 of file CovariancePreservationImputationVMatrix.cc.
OptionMap & PLearn::CovariancePreservationImputationVMatrix::getOptionMap | ( | ) | const [virtual] |
Reimplemented from PLearn::Object.
Definition at line 55 of file CovariancePreservationImputationVMatrix.cc.
RemoteMethodMap & PLearn::CovariancePreservationImputationVMatrix::getRemoteMethodMap | ( | ) | const [virtual] |
Reimplemented from PLearn::Object.
Definition at line 55 of file CovariancePreservationImputationVMatrix.cc.
These methods do not usually need to be overridden in subclasses (default versions call getSubRow, which should do just fine)
Copies row i into v (which must have appropriate length equal to the VMat's width).
Reimplemented from PLearn::VMatrix.
Definition at line 130 of file CovariancePreservationImputationVMatrix.cc.
References PLearn::is_missing(), and PLearn::TVec< T >::length().
{ source-> getRow(i, v); for (int source_col = 0; source_col < v->length(); source_col++) if (is_missing(v[source_col])) v[source_col] = computeImputation(i, source_col, v); }
void PLearn::CovariancePreservationImputationVMatrix::getSubRow | ( | int | i, |
int | j, | ||
Vec | v | ||
) | const [virtual] |
It is suggested that this method be implemented in subclasses to speed up accesses (default version repeatedly calls get(i,j) which may have a significant overhead).
Fills v with the subrow i lying between columns j (inclusive) and j+v.length() (exclusive).
Reimplemented from PLearn::VMatrix.
Definition at line 108 of file CovariancePreservationImputationVMatrix.cc.
References PLearn::is_missing(), and PLearn::TVec< T >::length().
{ source->getSubRow(i, j, v); for (int source_col = 0; source_col < v->length(); source_col++) if (is_missing(v[source_col])) v[source_col] = computeImputation(i, source_col + j); }
This method must be implemented for matrices that are allowed to grow.
Reimplemented from PLearn::VMatrix.
Definition at line 125 of file CovariancePreservationImputationVMatrix.cc.
References PLERROR.
{ PLERROR("In CovariancePreservationImputationVMatrix::insertRow not implemented"); }
void PLearn::CovariancePreservationImputationVMatrix::loadCovarianceFile | ( | ) | [private] |
Definition at line 206 of file CovariancePreservationImputationVMatrix.cc.
{ covariance_file = new FileVMatrix(covariance_file_name); for (indj = 0; indj < train_width; indj++) { for (indk = 0; indk < train_width; indk++) { cov(indj, indk) = covariance_file->get(indj, indk); } } for (indk = 0; indk < train_width; indk++) { mu[indk] = covariance_file->get(train_width, indk); } }
void PLearn::CovariancePreservationImputationVMatrix::makeDeepCopyFromShallowCopy | ( | CopiesMap & | copies | ) | [virtual] |
Does the necessary operations to transform a shallow copy (this) into a deep copy by deep-copying all the members that need to be.
This needs to be overridden by every class that adds "complex" data members to the class, such as Vec
, Mat
, PP<Something>
, etc. Typical implementation:
void CLASS_OF_THIS::makeDeepCopyFromShallowCopy(CopiesMap& copies) { inherited::makeDeepCopyFromShallowCopy(copies); deepCopyField(complex_data_member1, copies); deepCopyField(complex_data_member2, copies); ... }
copies | A map used by the deep-copy mechanism to keep track of already-copied objects. |
Reimplemented from PLearn::ImputationVMatrix.
Definition at line 81 of file CovariancePreservationImputationVMatrix.cc.
References PLearn::deepCopyField().
{ deepCopyField(train_set, copies); inherited::makeDeepCopyFromShallowCopy(copies); }
This method must be implemented in all subclasses of writable matrices.
Sets element (i,j) to value.
Reimplemented from PLearn::VMatrix.
Definition at line 103 of file CovariancePreservationImputationVMatrix.cc.
References PLERROR.
{ PLERROR("In CovariancePreservationImputationVMatrix::put not implemented"); }
Reimplemented from PLearn::VMatrix.
Definition at line 137 of file CovariancePreservationImputationVMatrix.cc.
References PLERROR.
{ PLERROR("In CovariancePreservationImputationVMatrix::putRow not implemented"); }
It is suggested that this method be implemented in subclasses of writable matrices to speed up accesses (default version repeatedly calls put(i,j,value) which may have a significant overhead)
Reimplemented from PLearn::VMatrix.
Definition at line 115 of file CovariancePreservationImputationVMatrix.cc.
References PLERROR.
{ PLERROR("In CovariancePreservationImputationVMatrix::putSubRow not implemented"); }
Reimplemented from PLearn::ImputationVMatrix.
Definition at line 120 of file CovariancePreservationImputationVMatrix.h.
Definition at line 111 of file CovariancePreservationImputationVMatrix.h.
Definition at line 104 of file CovariancePreservationImputationVMatrix.h.
Definition at line 103 of file CovariancePreservationImputationVMatrix.h.
Definition at line 105 of file CovariancePreservationImputationVMatrix.h.
Definition at line 106 of file CovariancePreservationImputationVMatrix.h.
Definition at line 110 of file CovariancePreservationImputationVMatrix.h.
Definition at line 107 of file CovariancePreservationImputationVMatrix.h.
Definition at line 100 of file CovariancePreservationImputationVMatrix.h.
Definition at line 98 of file CovariancePreservationImputationVMatrix.h.
Definition at line 101 of file CovariancePreservationImputationVMatrix.h.
Definition at line 102 of file CovariancePreservationImputationVMatrix.h.
Definition at line 99 of file CovariancePreservationImputationVMatrix.h.
Definition at line 108 of file CovariancePreservationImputationVMatrix.h.
Definition at line 109 of file CovariancePreservationImputationVMatrix.h.
TVec<string> PLearn::CovariancePreservationImputationVMatrix::train_field_names [private] |
Definition at line 96 of file CovariancePreservationImputationVMatrix.h.
Definition at line 95 of file CovariancePreservationImputationVMatrix.h.
Definition at line 91 of file CovariancePreservationImputationVMatrix.h.
Definition at line 89 of file CovariancePreservationImputationVMatrix.h.
Definition at line 97 of file CovariancePreservationImputationVMatrix.h.
Definition at line 94 of file CovariancePreservationImputationVMatrix.h.
A referenced train set.
The covariance imputation is computed with the observed values in this data set.
Definition at line 64 of file CovariancePreservationImputationVMatrix.h.
Referenced by declareOptions().
Definition at line 92 of file CovariancePreservationImputationVMatrix.h.
Definition at line 93 of file CovariancePreservationImputationVMatrix.h.
Definition at line 90 of file CovariancePreservationImputationVMatrix.h.