PLearn 0.1
|
Generate samples from a mixture of two gaussians. More...
#include <NeighborhoodConditionalMean.h>
Public Member Functions | |
NeighborhoodConditionalMean () | |
Default constructor. | |
int | outputsize () const |
SUBCLASS WRITING: override this so that it returns the size of this learner's output, as a function of its inputsize(), targetsize() and set options. | |
void | train () |
*** SUBCLASS WRITING: *** | |
void | computeOutput (const Vec &, Vec &) const |
*** SUBCLASS WRITING: *** | |
void | computeCostsFromOutputs (const Vec &, const Vec &, const Vec &, Vec &) const |
*** SUBCLASS WRITING: *** | |
TVec< string > | getTestCostNames () const |
*** SUBCLASS WRITING: *** | |
TVec< string > | getTrainCostNames () const |
*** SUBCLASS WRITING: *** | |
virtual string | classname () const |
virtual OptionList & | getOptionList () const |
virtual OptionMap & | getOptionMap () const |
virtual RemoteMethodMap & | getRemoteMethodMap () const |
virtual NeighborhoodConditionalMean * | deepCopy (CopiesMap &copies) const |
virtual void | build () |
Finish building the object; just call inherited::build followed by build_() | |
virtual void | makeDeepCopyFromShallowCopy (CopiesMap &copies) |
Transforms a shallow copy into a deep copy. | |
Static Public Member Functions | |
static string | _classname_ () |
static OptionList & | _getOptionList_ () |
static RemoteMethodMap & | _getRemoteMethodMap_ () |
static Object * | _new_instance_for_typemap_ () |
static bool | _isa_ (const Object *o) |
static void | _static_initialize_ () |
static const PPath & | declaringFile () |
Public Attributes | |
VMat | test_train_input_set |
### declare public option fields (such as build options) here Start your comments with Doxygen-compatible comments such as //! | |
VMat | test_train_target_set |
The corresponding target vectors. | |
int | number_of_test_samples |
The number of test samples at the beginning of the test train concatenated sets. | |
int | number_of_train_samples |
The number of train samples in the reference set to compute the % of missing. | |
TVec< string > | target_field_names |
The vector of names of the field to select from the target_set as target for the built training files. | |
PPath | train_covariance_file_name |
PPath | test_train_covariance_file_name |
TVec< int > | various_ks |
The vector of various Ks to experiment with. Values must be between 1 and 100. | |
Vec | deletion_thresholds |
The vector of thresholds to be tested for each of the various Ks. | |
string | experiment_name |
The name of the group of experiments to conduct. | |
TVec< string > | missing_indicator_field_names |
The field names of the missing indicators to exclude when we experiment without them. | |
PP< PTester > | experiment_template |
The template of the script to conduct the experiment. | |
Static Public Attributes | |
static StaticInitializer | _static_initializer_ |
Static Protected Member Functions | |
static void | declareOptions (OptionList &ol) |
Declares the class options. | |
Private Types | |
typedef PLearner | inherited |
Private Member Functions | |
void | build_ () |
This does the actual building. | |
void | computeNeighborhood () |
void | experimentWithVariousKs () |
void | createMasterHeaderFile () |
void | getMasterHeaderRecords () |
void | updateMasterHeaderRecords (int row, int col) |
Private Attributes | |
ProgressBar * | pb |
VMat | train_covariance_file |
CovariancePreservationImputationVMatrix * | train_covariance_vmatrix |
VMat | train_covariance_vmat |
Vec | train_covariance_vector |
VMat | test_train_covariance_file |
CovariancePreservationImputationVMatrix * | test_train_covariance_vmatrix |
VMat | test_train_covariance_vmat |
Vec | test_train_covariance_vector |
PPath | test_train_neighborhood_file_name |
BallTreeNearestNeighbors * | test_train_neighborhood_learner |
VMat | test_train_neighborhood_file |
Vec | test_train_neighborhood_vector |
PPath | master_header_file_name |
VMat | master_header_file |
int | master_header_length |
int | master_header_width |
int | master_header_row |
int | master_header_col |
TVec< string > | master_header_names |
Mat | master_header_records |
int | to_deal_with_k |
string | to_deal_with_target |
int | to_deal_with_ind |
NeighborhoodImputationVMatrix * | test_train_neighbor_imputation_vmatrix |
VMat | test_train_neighbor_imputation_vmat |
VMat | test_train_neighbor_imputation_file |
Vec | test_train_neighbor_imputation_vector |
Experimentation * | experimentation_learner |
Generate samples from a mixture of two gaussians.
Definition at line 62 of file NeighborhoodConditionalMean.h.
typedef PLearner PLearn::NeighborhoodConditionalMean::inherited [private] |
Reimplemented from PLearn::PLearner.
Definition at line 64 of file NeighborhoodConditionalMean.h.
PLearn::NeighborhoodConditionalMean::NeighborhoodConditionalMean | ( | ) |
string PLearn::NeighborhoodConditionalMean::_classname_ | ( | ) | [static] |
Reimplemented from PLearn::PLearner.
Definition at line 51 of file NeighborhoodConditionalMean.cc.
OptionList & PLearn::NeighborhoodConditionalMean::_getOptionList_ | ( | ) | [static] |
Reimplemented from PLearn::PLearner.
Definition at line 51 of file NeighborhoodConditionalMean.cc.
RemoteMethodMap & PLearn::NeighborhoodConditionalMean::_getRemoteMethodMap_ | ( | ) | [static] |
Reimplemented from PLearn::PLearner.
Definition at line 51 of file NeighborhoodConditionalMean.cc.
Reimplemented from PLearn::PLearner.
Definition at line 51 of file NeighborhoodConditionalMean.cc.
Object * PLearn::NeighborhoodConditionalMean::_new_instance_for_typemap_ | ( | ) | [static] |
Reimplemented from PLearn::Object.
Definition at line 51 of file NeighborhoodConditionalMean.cc.
StaticInitializer NeighborhoodConditionalMean::_static_initializer_ & PLearn::NeighborhoodConditionalMean::_static_initialize_ | ( | ) | [static] |
Reimplemented from PLearn::PLearner.
Definition at line 51 of file NeighborhoodConditionalMean.cc.
void PLearn::NeighborhoodConditionalMean::build | ( | ) | [virtual] |
Finish building the object; just call inherited::build followed by build_()
Reimplemented from PLearn::PLearner.
Definition at line 130 of file NeighborhoodConditionalMean.cc.
{ // ### Nothing to add here, simply calls build_(). inherited::build(); build_(); }
void PLearn::NeighborhoodConditionalMean::build_ | ( | ) | [private] |
This does the actual building.
Reimplemented from PLearn::PLearner.
Definition at line 140 of file NeighborhoodConditionalMean.cc.
References PLearn::endl(), and PLERROR.
{ MODULE_LOG << "build_() called" << endl; if (train_set) { for (int iteration = 1; iteration <= 1; iteration++) { cout << "In NeighborhoodConditionalMean, Iteration # " << iteration << endl; computeNeighborhood(); experimentWithVariousKs(); train(); } PLERROR("In NeighborhoodConditionalMean: we are done here"); } }
string PLearn::NeighborhoodConditionalMean::classname | ( | ) | const [virtual] |
Reimplemented from PLearn::Object.
Definition at line 51 of file NeighborhoodConditionalMean.cc.
void PLearn::NeighborhoodConditionalMean::computeCostsFromOutputs | ( | const Vec & | input, |
const Vec & | output, | ||
const Vec & | target, | ||
Vec & | costs | ||
) | const [virtual] |
*** SUBCLASS WRITING: ***
This should be defined in subclasses to compute the weighted costs from already computed output. The costs should correspond to the cost names returned by getTestCostNames().
NOTE: In exotic cases, the cost may also depend on some info in the input, that's why the method also gets so see it.
Implements PLearn::PLearner.
Definition at line 518 of file NeighborhoodConditionalMean.cc.
{}
void PLearn::NeighborhoodConditionalMean::computeNeighborhood | ( | ) | [private] |
Definition at line 156 of file NeighborhoodConditionalMean.cc.
References PLearn::VMatrix::defineSizes(), PLearn::endl(), PLearn::isfile(), PLERROR, and PLearn::ProgressBar::update().
{ /* prepare correlation based versions of datatset: we have to write a VMatrix for that use the ball tree nearest neighbor to build a ball tree using train only, with unknown it would be too long find the 100 nearest neighbors of samples in train and test in order from the closest to the furthest now we can create a neighborhood imputation for K from 1 up to 100 averaging the observed values of the the k closest input vectors. If there is no observed values in the k closest, we have to use something else: mean of the covariance preservation imputationof the the k closest input vectors. */ cout << "In NeighborhoodConditionalMean:" << endl; cout << endl << "****** STEP 1 ******" << endl; cout << "The first thing to do is to impute an initial value the the missing values in order to be able" << endl; cout << "to compute distance between samples." << endl; cout << "This step uses the CovariancePreservationVMatrix to do that." << endl; cout << "The Covariance PreservationVMatrix creates a covariance_file in the metadata of the source file" << endl; cout << "if it is not already there." << endl; cout << "The file is kept in train_imputed_with_covariance_preservation.pmat." << endl; if( train_covariance_file_name == "" ) PLERROR("In NeighborhoodConditionalMean::computeNeighborhood() train_covariance_file_name must not be empty",train_covariance_file_name.c_str()); if (isfile(train_covariance_file_name)) { train_covariance_file = new FileVMatrix(train_covariance_file_name); train_covariance_file->defineSizes(train_covariance_file->width(), 0, 0); cout << train_covariance_file_name << " already exist, we are skipping this step." << endl; } else { train_covariance_vmatrix = new CovariancePreservationImputationVMatrix(); train_covariance_vmatrix->source = train_set; train_covariance_vmatrix->train_set = train_set; train_covariance_vmatrix->build(); train_covariance_vmat = train_covariance_vmatrix; train_covariance_file = new FileVMatrix(train_covariance_file_name, train_covariance_vmat->length(), train_covariance_vmat->fieldNames()); train_covariance_file->defineSizes(train_covariance_vmat->width(), 0, 0); pb = new ProgressBar("Saving the train file imputed with the covariance preservation", train_covariance_vmat->length()); train_covariance_vector.resize(train_covariance_vmat->width()); for (int train_covariance_row = 0; train_covariance_row < train_covariance_vmat->length(); train_covariance_row++) { train_covariance_vmat->getRow(train_covariance_row, train_covariance_vector); train_covariance_file->putRow(train_covariance_row, train_covariance_vector); pb->update( train_covariance_row ); } delete pb; } cout << endl << "****** STEP 2 ******" << endl; cout << "We do the same thing with the test_train dataset" << endl; cout << "using the covariance file created at the previous step." << endl; cout << "The file is kept in test_train_imputed_with_covariance_preservation.pmat." << endl; if( test_train_covariance_file_name == "" ) PLERROR("In NeighborhoodConditionalMean::computeNeighborhood() test_train_covariance_file_name must not be empty",test_train_covariance_file_name.c_str()); if (isfile(test_train_covariance_file_name)) { test_train_covariance_file = new FileVMatrix(test_train_covariance_file_name); test_train_covariance_file->defineSizes(test_train_covariance_file->width(), 0, 0); cout << test_train_covariance_file_name << " already exist, we are skipping this step." << endl; } else { test_train_covariance_vmatrix = new CovariancePreservationImputationVMatrix(); test_train_covariance_vmatrix->source = test_train_input_set; test_train_covariance_vmatrix->train_set = train_set; test_train_covariance_vmatrix->build(); test_train_covariance_vmat = test_train_covariance_vmatrix; test_train_covariance_file = new FileVMatrix(test_train_covariance_file_name, test_train_covariance_vmat->length(), test_train_covariance_vmat->fieldNames()); test_train_covariance_file->defineSizes(test_train_covariance_vmat->width(), 0, 0); pb = new ProgressBar("Saving the test_train file imputed with the covariance preservation", test_train_covariance_vmat->length()); test_train_covariance_vector.resize(test_train_covariance_vmat->width()); for (int test_train_covariance_row = 0; test_train_covariance_row < test_train_covariance_vmat->length(); test_train_covariance_row++) { test_train_covariance_vmat->getRow(test_train_covariance_row, test_train_covariance_vector); test_train_covariance_file->putRow(test_train_covariance_row, test_train_covariance_vector); pb->update( test_train_covariance_row ); } delete pb; } cout << endl << "****** STEP 3 ******" << endl; cout << "We this initial imputation, we find the 100 nearest neighbors of each sample in the test_train dataset." << endl; cout << "Their indexes are kept in the neighborhood_file of the test_train dataset metadata." << endl; cout << "The BallTreeNearestNeighbors learner is used to build a tree with the train set" << endl; cout << "in order to speed up the identification of the 100 nearest neighbors of the test_train dataset." << endl; test_train_neighborhood_file_name = test_train_covariance_file_name + ".metadata/neighborhood_file.pmat"; if (isfile(test_train_neighborhood_file_name)) { test_train_neighborhood_file = new FileVMatrix(test_train_neighborhood_file_name); cout << test_train_neighborhood_file_name << " already exist, we are skipping this step." << endl; } else { test_train_neighborhood_learner = new BallTreeNearestNeighbors(); test_train_neighborhood_learner->setOption("rmin", "1"); test_train_neighborhood_learner->setOption("train_method", "anchor"); test_train_neighborhood_learner->setOption("num_neighbors", "100"); test_train_neighborhood_learner->setOption("copy_input", "0"); test_train_neighborhood_learner->setOption("copy_target", "0"); test_train_neighborhood_learner->setOption("copy_weight", "0"); test_train_neighborhood_learner->setOption("copy_index", "1"); test_train_neighborhood_learner->setOption("nstages", "-1"); test_train_neighborhood_learner->setOption("report_progress", "1"); test_train_neighborhood_learner->setTrainingSet(train_covariance_file, true); test_train_neighborhood_learner->train(); test_train_neighborhood_file = new FileVMatrix(test_train_neighborhood_file_name, test_train_covariance_file->length(), 100); test_train_covariance_vector.resize(test_train_covariance_file->width()); test_train_neighborhood_vector.resize(100); pb = new ProgressBar("Saving the test_train file with the index of the 100 nearest neighbors", test_train_covariance_file->length()); for (int test_train_neighborhood_row = 0; test_train_neighborhood_row < test_train_covariance_file->length(); test_train_neighborhood_row++) { test_train_covariance_file->getRow(test_train_neighborhood_row, test_train_covariance_vector); test_train_neighborhood_learner->computeOutput(test_train_covariance_vector, test_train_neighborhood_vector); test_train_neighborhood_file->putRow(test_train_neighborhood_row, test_train_neighborhood_vector); pb->update( test_train_neighborhood_row ); } delete pb; } }
void PLearn::NeighborhoodConditionalMean::computeOutput | ( | const Vec & | input, |
Vec & | output | ||
) | const [virtual] |
*** SUBCLASS WRITING: ***
This should be defined in subclasses to compute the output from the input.
Reimplemented from PLearn::PLearner.
Definition at line 517 of file NeighborhoodConditionalMean.cc.
{}
void PLearn::NeighborhoodConditionalMean::createMasterHeaderFile | ( | ) | [private] |
Definition at line 359 of file NeighborhoodConditionalMean.cc.
References PLearn::tostring().
{ master_header_length = target_field_names.length() * 2; master_header_width = various_ks.length(); master_header_names.resize(master_header_width); master_header_records.resize(master_header_length, master_header_width); master_header_records.clear(); for (master_header_col = 0; master_header_col < master_header_width; master_header_col++) master_header_names[master_header_col] = "K_" + tostring(master_header_col); master_header_file = new FileVMatrix(master_header_file_name, master_header_length, master_header_names); for (master_header_row = 0; master_header_row < master_header_length; master_header_row++) for (master_header_col = 0; master_header_col < master_header_width; master_header_col++) master_header_file->put(master_header_row, master_header_col, 0.0); }
void PLearn::NeighborhoodConditionalMean::declareOptions | ( | OptionList & | ol | ) | [static, protected] |
Declares the class options.
Reimplemented from PLearn::PLearner.
Definition at line 63 of file NeighborhoodConditionalMean.cc.
References PLearn::OptionBase::buildoption, PLearn::declareOption(), deletion_thresholds, experiment_name, experiment_template, missing_indicator_field_names, number_of_test_samples, number_of_train_samples, target_field_names, test_train_covariance_file_name, test_train_input_set, test_train_target_set, train_covariance_file_name, and various_ks.
{ declareOption(ol, "test_train_input_set", &NeighborhoodConditionalMean::test_train_input_set, OptionBase::buildoption, "The concatenated test and train input vectors with missing values."); declareOption(ol, "test_train_target_set", &NeighborhoodConditionalMean::test_train_target_set, OptionBase::buildoption, "The corresponding target vectors."); declareOption(ol, "number_of_test_samples", &NeighborhoodConditionalMean::number_of_test_samples, OptionBase::buildoption, "The number of test samples at the beginning of the test train concatenated sets."); declareOption(ol, "number_of_train_samples", &NeighborhoodConditionalMean::number_of_train_samples, OptionBase::buildoption, "The number of train samples in the reference set to compute the % of missing."); declareOption(ol, "target_field_names", &NeighborhoodConditionalMean::target_field_names, OptionBase::buildoption, "The vector of names of the field to select from the target_set as target for the built training files."); declareOption(ol, "train_covariance_file_name", &NeighborhoodConditionalMean::train_covariance_file_name, OptionBase::buildoption, "The path to the file train set where missing value are imputed by the covariance preservation algo."); declareOption(ol, "test_train_covariance_file_name", &NeighborhoodConditionalMean::test_train_covariance_file_name, OptionBase::buildoption, "The path to the file test_train set where missing value are imputed by the covariance preservation algo."); declareOption(ol, "various_ks", &NeighborhoodConditionalMean::various_ks, OptionBase::buildoption, "The vector of various Ks to experiment with. Values must be between 1 and 100."); declareOption(ol, "deletion_thresholds", &NeighborhoodConditionalMean::deletion_thresholds, OptionBase::buildoption, "The vector of thresholds to be tested for each of the various Ks."); declareOption(ol, "experiment_name", &NeighborhoodConditionalMean::experiment_name, OptionBase::buildoption, "The name of the group of experiments to conduct."); declareOption(ol, "missing_indicator_field_names", &NeighborhoodConditionalMean::missing_indicator_field_names, OptionBase::buildoption, "The field names of the missing indicators to exclude when we experiment without them."); declareOption(ol, "experiment_template", &NeighborhoodConditionalMean::experiment_template, OptionBase::buildoption, "The template of the script to conduct the experiment."); inherited::declareOptions(ol); }
static const PPath& PLearn::NeighborhoodConditionalMean::declaringFile | ( | ) | [inline, static] |
Reimplemented from PLearn::PLearner.
Definition at line 116 of file NeighborhoodConditionalMean.h.
:
//##### Protected Member Functions ######################################
NeighborhoodConditionalMean * PLearn::NeighborhoodConditionalMean::deepCopy | ( | CopiesMap & | copies | ) | const [virtual] |
Reimplemented from PLearn::PLearner.
Definition at line 51 of file NeighborhoodConditionalMean.cc.
void PLearn::NeighborhoodConditionalMean::experimentWithVariousKs | ( | ) | [private] |
Definition at line 273 of file NeighborhoodConditionalMean.cc.
References PLearn::VMatrix::defineSizes(), PLearn::endl(), PLearn::isfile(), PLERROR, PLearn::tostring(), and PLearn::ProgressBar::update().
{ /* We control the experiments using a master header file giving the status for each ks. If the file is not there, we create it. An experiment directory is created for each ks to eexperiment with various level of variable deletion. */ cout << endl << "****** STEP 4 ******" << endl; cout << "We now prepare experimentation at various levels of Ks, the number of neighbors between 1 and 100." << endl; cout << "The first thing is to load the master header file from the test_train_imputed_with_covariance_preservation.pmat metadata." << endl; cout << "If it is not there, the file is created." << endl; train_set->lockMetaDataDir(); master_header_file_name = test_train_covariance_file_name + ".metadata"; master_header_file_name += "/Experiment/" + experiment_name + "/"; master_header_file_name += "neighborhood_header.pmat"; if (!isfile(master_header_file_name)) createMasterHeaderFile(); else getMasterHeaderRecords(); cout << "With the master header data, we can choose which K to experiment with." << endl; for (master_header_row = 0; master_header_row < master_header_length; master_header_row++) { for (master_header_col = 0; master_header_col < master_header_width; master_header_col++) if (master_header_records(master_header_row, master_header_col) <= 0.0) break; if (master_header_col < master_header_width) break; } if (master_header_row >= master_header_length) { train_set->unlockMetaDataDir(); //reviewGlobalStats(); PLERROR("In NeighborhoodConditionalMean: we are done here"); } to_deal_with_k = various_ks[master_header_col]; to_deal_with_target = target_field_names[master_header_row / 2]; to_deal_with_ind = master_header_row % 2; cout << "Next target to deal with: " << to_deal_with_target << endl; cout << "Next experiment missing indicator: " << to_deal_with_ind << endl; cout << "Next k (number of neighbors) to experiment with: " << to_deal_with_k << endl; updateMasterHeaderRecords(master_header_row, master_header_col); train_set->unlockMetaDataDir(); cout << endl << "****** STEP 5 ******" << endl; cout << "We perform the imputaton with the selected number of neighbors." << endl; cout << "The resulting file is loaded in memory to be passed to the experimentation script." << endl; test_train_neighbor_imputation_vmatrix = new NeighborhoodImputationVMatrix(); test_train_neighbor_imputation_vmatrix->source = test_train_input_set; test_train_neighbor_imputation_vmatrix->reference_index = test_train_neighborhood_file; test_train_neighbor_imputation_vmatrix->reference_with_missing = train_set; test_train_neighbor_imputation_vmatrix->reference_with_covariance_preserved = train_covariance_file; test_train_neighbor_imputation_vmatrix->number_of_neighbors = to_deal_with_k; test_train_neighbor_imputation_vmatrix->build(); test_train_neighbor_imputation_vmat = test_train_neighbor_imputation_vmatrix; test_train_neighbor_imputation_file = new MemoryVMatrix(test_train_neighbor_imputation_vmat->length(), test_train_neighbor_imputation_vmat->width()); test_train_neighbor_imputation_file->defineSizes(test_train_neighbor_imputation_vmat->width(), 0, 0); test_train_neighbor_imputation_file->declareFieldNames(test_train_neighbor_imputation_vmat->fieldNames()); test_train_neighbor_imputation_vector.resize(test_train_neighbor_imputation_vmat->width()); pb = new ProgressBar("Loading the test_train file imputed with the selected # of neighbors", test_train_neighbor_imputation_vmat->length()); for (int test_train_neighbor_imputation_row = 0; test_train_neighbor_imputation_row < test_train_neighbor_imputation_vmat->length(); test_train_neighbor_imputation_row++) { test_train_neighbor_imputation_vmat->getRow(test_train_neighbor_imputation_row, test_train_neighbor_imputation_vector); test_train_neighbor_imputation_file->putRow(test_train_neighbor_imputation_row, test_train_neighbor_imputation_vector); pb->update( test_train_neighbor_imputation_row ); } // ::PLearn::save(header_expdir + "/" + deletion_threshold_str + "/source_names.psave", source_names); delete pb; cout << endl << "****** STEP 6 ******" << endl; cout << "We are now ready to launch the experimentation for this k." << endl; cout << "The Experimentation program will build learners for the specified deletion thresholds." << endl; experimentation_learner = new Experimentation(); experimentation_learner->save_files = 0; experimentation_learner->experiment_without_missing_indicator = to_deal_with_ind; experimentation_learner->target_field_name = to_deal_with_target; experimentation_learner->missing_indicator_field_names = missing_indicator_field_names; experimentation_learner->experiment_name = experiment_name; experimentation_learner->number_of_test_samples = number_of_test_samples; experimentation_learner->number_of_train_samples = number_of_train_samples; experimentation_learner->reference_train_set = train_set; experimentation_learner->target_set = test_train_target_set; experimentation_learner->experiment_template = experiment_template; experimentation_learner->deletion_thresholds = deletion_thresholds; experimentation_learner->experiment_directory = test_train_covariance_file_name + ".metadata"; experimentation_learner->experiment_directory += "/Experiment/" + experiment_name + "/"; experimentation_learner->experiment_directory += "K_" + tostring(to_deal_with_k); experimentation_learner->setTrainingSet(test_train_neighbor_imputation_file); }
void PLearn::NeighborhoodConditionalMean::getMasterHeaderRecords | ( | ) | [private] |
Definition at line 373 of file NeighborhoodConditionalMean.cc.
References PLERROR.
{ master_header_file = new FileVMatrix(master_header_file_name, true); master_header_length = master_header_file->length(); master_header_width = master_header_file->width(); if (master_header_length != target_field_names.length() * 2) PLERROR("In NeighborhoodConditionalMean: master header file length and target_field_names do not agree"); if (master_header_width != various_ks.length()) PLERROR("In NeighborhoodConditionalMean: master header file width and various_ks do not agree"); master_header_records.resize(master_header_length, master_header_width); for (master_header_row = 0; master_header_row < master_header_length; master_header_row++) for (master_header_col = 0; master_header_col < master_header_width; master_header_col++) master_header_records(master_header_row, master_header_col) = master_header_file->get(master_header_row, master_header_col); }
OptionList & PLearn::NeighborhoodConditionalMean::getOptionList | ( | ) | const [virtual] |
Reimplemented from PLearn::Object.
Definition at line 51 of file NeighborhoodConditionalMean.cc.
OptionMap & PLearn::NeighborhoodConditionalMean::getOptionMap | ( | ) | const [virtual] |
Reimplemented from PLearn::Object.
Definition at line 51 of file NeighborhoodConditionalMean.cc.
RemoteMethodMap & PLearn::NeighborhoodConditionalMean::getRemoteMethodMap | ( | ) | const [virtual] |
Reimplemented from PLearn::Object.
Definition at line 51 of file NeighborhoodConditionalMean.cc.
TVec< string > PLearn::NeighborhoodConditionalMean::getTestCostNames | ( | ) | const [virtual] |
*** SUBCLASS WRITING: ***
This should return the names of the costs computed by computeCostsFromOutputs.
Implements PLearn::PLearner.
Definition at line 519 of file NeighborhoodConditionalMean.cc.
References PLearn::TVec< T >::append().
{ TVec<string> result; result.append( "MSE" ); return result; }
TVec< string > PLearn::NeighborhoodConditionalMean::getTrainCostNames | ( | ) | const [virtual] |
*** SUBCLASS WRITING: ***
This should return the names of the objective costs that the train method computes and for which it updates the VecStatsCollector train_stats.
Implements PLearn::PLearner.
Definition at line 525 of file NeighborhoodConditionalMean.cc.
References PLearn::TVec< T >::append().
{ TVec<string> result; result.append( "MSE" ); return result; }
void PLearn::NeighborhoodConditionalMean::makeDeepCopyFromShallowCopy | ( | CopiesMap & | copies | ) | [virtual] |
Transforms a shallow copy into a deep copy.
Reimplemented from PLearn::PLearner.
Definition at line 109 of file NeighborhoodConditionalMean.cc.
References PLearn::deepCopyField().
{ deepCopyField(test_train_input_set, copies); deepCopyField(test_train_target_set, copies); deepCopyField(number_of_test_samples, copies); deepCopyField(number_of_train_samples, copies); deepCopyField(target_field_names, copies); deepCopyField(test_train_covariance_file_name, copies); deepCopyField(train_covariance_file_name, copies); deepCopyField(various_ks, copies); deepCopyField(deletion_thresholds, copies); deepCopyField(experiment_name, copies); deepCopyField(missing_indicator_field_names, copies); deepCopyField(deletion_thresholds, copies); inherited::makeDeepCopyFromShallowCopy(copies); }
int PLearn::NeighborhoodConditionalMean::outputsize | ( | ) | const [virtual] |
SUBCLASS WRITING: override this so that it returns the size of this learner's output, as a function of its inputsize(), targetsize() and set options.
Implements PLearn::PLearner.
Definition at line 516 of file NeighborhoodConditionalMean.cc.
{return 0;}
void PLearn::NeighborhoodConditionalMean::train | ( | ) | [virtual] |
*** SUBCLASS WRITING: ***
The role of the train method is to bring the learner up to stage==nstages, updating the stats with training costs measured on-line in the process.
TYPICAL CODE:
static Vec input; // static so we don't reallocate/deallocate memory each time... static Vec target; // (but be careful that static means shared!) input.resize(inputsize()); // the train_set's inputsize() target.resize(targetsize()); // the train_set's targetsize() real weight; if(!train_stats) // make a default stats collector, in case there's none train_stats = new VecStatsCollector(); if(nstages<stage) // asking to revert to a previous stage! forget(); // reset the learner to stage=0 while(stage<nstages) { // clear statistics of previous epoch train_stats->forget(); //... train for 1 stage, and update train_stats, // using train_set->getSample(input, target, weight); // and train_stats->update(train_costs) ++stage; train_stats->finalize(); // finalize statistics for this epoch }
Implements PLearn::PLearner.
Definition at line 500 of file NeighborhoodConditionalMean.cc.
{ /* PP<ExplicitSplitter> explicit_splitter = new ExplicitSplitter(); explicit_splitter->splitsets.resize(1,2); explicit_splitter->splitsets(0,0) = output_file; explicit_splitter->splitsets(0,1) = train_test_file; cond_mean = ::PLearn::deepCopy(cond_mean_template); cond_mean->setOption("expdir", targeted_metadata + "/TreeCondMean/dir/" + to_deal_with_name); cond_mean->splitter = new ExplicitSplitter(); cond_mean->splitter = explicit_splitter; cond_mean->build(); Vec results = cond_mean->perform(true); */ }
Definition at line 388 of file NeighborhoodConditionalMean.cc.
{ master_header_records(row, col) += 1.0; master_header_file->put(row, col, master_header_records(row, col)); master_header_file->flush(); }
Reimplemented from PLearn::PLearner.
Definition at line 116 of file NeighborhoodConditionalMean.h.
The vector of thresholds to be tested for each of the various Ks.
Definition at line 88 of file NeighborhoodConditionalMean.h.
Referenced by declareOptions().
The name of the group of experiments to conduct.
Definition at line 90 of file NeighborhoodConditionalMean.h.
Referenced by declareOptions().
The template of the script to conduct the experiment.
Definition at line 94 of file NeighborhoodConditionalMean.h.
Referenced by declareOptions().
Definition at line 174 of file NeighborhoodConditionalMean.h.
Definition at line 164 of file NeighborhoodConditionalMean.h.
Definition at line 160 of file NeighborhoodConditionalMean.h.
Definition at line 159 of file NeighborhoodConditionalMean.h.
Definition at line 161 of file NeighborhoodConditionalMean.h.
TVec<string> PLearn::NeighborhoodConditionalMean::master_header_names [private] |
Definition at line 165 of file NeighborhoodConditionalMean.h.
Definition at line 166 of file NeighborhoodConditionalMean.h.
Definition at line 163 of file NeighborhoodConditionalMean.h.
Definition at line 162 of file NeighborhoodConditionalMean.h.
The field names of the missing indicators to exclude when we experiment without them.
Definition at line 92 of file NeighborhoodConditionalMean.h.
Referenced by declareOptions().
The number of test samples at the beginning of the test train concatenated sets.
Definition at line 78 of file NeighborhoodConditionalMean.h.
Referenced by declareOptions().
The number of train samples in the reference set to compute the % of missing.
Definition at line 80 of file NeighborhoodConditionalMean.h.
Referenced by declareOptions().
Definition at line 146 of file NeighborhoodConditionalMean.h.
The vector of names of the field to select from the target_set as target for the built training files.
Definition at line 82 of file NeighborhoodConditionalMean.h.
Referenced by declareOptions().
Definition at line 151 of file NeighborhoodConditionalMean.h.
Definition at line 84 of file NeighborhoodConditionalMean.h.
Referenced by declareOptions().
Definition at line 154 of file NeighborhoodConditionalMean.h.
Definition at line 153 of file NeighborhoodConditionalMean.h.
CovariancePreservationImputationVMatrix* PLearn::NeighborhoodConditionalMean::test_train_covariance_vmatrix [private] |
Definition at line 152 of file NeighborhoodConditionalMean.h.
### declare public option fields (such as build options) here Start your comments with Doxygen-compatible comments such as //!
The concatenated test and train input vectors with missing values.
Definition at line 74 of file NeighborhoodConditionalMean.h.
Referenced by declareOptions().
Definition at line 172 of file NeighborhoodConditionalMean.h.
Definition at line 173 of file NeighborhoodConditionalMean.h.
Definition at line 171 of file NeighborhoodConditionalMean.h.
NeighborhoodImputationVMatrix* PLearn::NeighborhoodConditionalMean::test_train_neighbor_imputation_vmatrix [private] |
Definition at line 170 of file NeighborhoodConditionalMean.h.
Definition at line 157 of file NeighborhoodConditionalMean.h.
Definition at line 155 of file NeighborhoodConditionalMean.h.
BallTreeNearestNeighbors* PLearn::NeighborhoodConditionalMean::test_train_neighborhood_learner [private] |
Definition at line 156 of file NeighborhoodConditionalMean.h.
Definition at line 158 of file NeighborhoodConditionalMean.h.
The corresponding target vectors.
Definition at line 76 of file NeighborhoodConditionalMean.h.
Referenced by declareOptions().
Definition at line 169 of file NeighborhoodConditionalMean.h.
Definition at line 167 of file NeighborhoodConditionalMean.h.
string PLearn::NeighborhoodConditionalMean::to_deal_with_target [private] |
Definition at line 168 of file NeighborhoodConditionalMean.h.
Definition at line 147 of file NeighborhoodConditionalMean.h.
Definition at line 83 of file NeighborhoodConditionalMean.h.
Referenced by declareOptions().
Definition at line 150 of file NeighborhoodConditionalMean.h.
Definition at line 149 of file NeighborhoodConditionalMean.h.
CovariancePreservationImputationVMatrix* PLearn::NeighborhoodConditionalMean::train_covariance_vmatrix [private] |
Definition at line 148 of file NeighborhoodConditionalMean.h.
The vector of various Ks to experiment with. Values must be between 1 and 100.
Definition at line 86 of file NeighborhoodConditionalMean.h.
Referenced by declareOptions().