PLearn 0.1
Public Member Functions | Static Public Member Functions | Public Attributes | Static Public Attributes | Static Protected Member Functions | Private Types | Private Member Functions | Private Attributes
PLearn::Experimentation Class Reference

Generate samples from a mixture of two gaussians. More...

#include <Experimentation.h>

Inheritance diagram for PLearn::Experimentation:
Inheritance graph
[legend]
Collaboration diagram for PLearn::Experimentation:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 Experimentation ()
 Default constructor.
int outputsize () const
 SUBCLASS WRITING: override this so that it returns the size of this learner's output, as a function of its inputsize(), targetsize() and set options.
void train ()
 *** SUBCLASS WRITING: ***
void computeOutput (const Vec &, Vec &) const
 *** SUBCLASS WRITING: ***
void computeCostsFromOutputs (const Vec &, const Vec &, const Vec &, Vec &) const
 *** SUBCLASS WRITING: ***
TVec< string > getTestCostNames () const
 *** SUBCLASS WRITING: ***
TVec< string > getTrainCostNames () const
 *** SUBCLASS WRITING: ***
virtual string classname () const
virtual OptionListgetOptionList () const
virtual OptionMapgetOptionMap () const
virtual RemoteMethodMapgetRemoteMethodMap () const
virtual ExperimentationdeepCopy (CopiesMap &copies) const
virtual void build ()
 Finish building the object; just call inherited::build followed by build_()
virtual void makeDeepCopyFromShallowCopy (CopiesMap &copies)
 Transforms a shallow copy into a deep copy.

Static Public Member Functions

static string _classname_ ()
static OptionList_getOptionList_ ()
static RemoteMethodMap_getRemoteMethodMap_ ()
static Object_new_instance_for_typemap_ ()
static bool _isa_ (const Object *o)
static void _static_initialize_ ()
static const PPathdeclaringFile ()

Public Attributes

int save_files
 ### declare public option fields (such as build options) here Start your comments with Doxygen-compatible comments such as //!
int experiment_without_missing_indicator
 If set to 1, the missing_indicator_field_names will be excluded from the built training files.
string target_field_name
 The name of the field to select from the target_set as target for the built training files.
TVec< string > missing_indicator_field_names
 The field names of the missing indicators to exclude when we experiment without them.
string experiment_name
 The name of the group of experiments to conduct.
int number_of_test_samples
 The number of test samples at the beginning of the train set.
int number_of_train_samples
 The number of train samples in the reference set to compute the % of missing.
VMat reference_train_set
 The train and valid set with missing values to compute the % of missing.
VMat target_set
 The data set with the targets corresponding to the train set.
Vec deletion_thresholds
 The vector of the various deletion threshold to run this experiment with.
PPath experiment_directory
 The path in which to build the directories for the experiment's results.
PP< PTesterexperiment_template
 The template of the script to conduct the experiment.

Static Public Attributes

static StaticInitializer _static_initializer_

Static Protected Member Functions

static void declareOptions (OptionList &ol)
 Declares the class options.

Private Types

typedef PLearner inherited

Private Member Functions

void build_ ()
 This does the actual building.
void experimentSetUp ()
void createHeaderFile ()
void getHeaderRecord ()
void updateHeaderRecord (int var_col)
void setSourceDataset ()
void reviewGlobalStats ()

Private Attributes

int main_row
int main_col
int main_length
int main_width
Vec main_input
TVec< string > main_names
PPath main_metadata
int main_fields_selected_col
TVec< string > main_fields_selected
int fields_width
int fields_col
int target_row
int target_col
int target_length
int target_width
Vec target_input
TVec< string > target_names
int header_width
Vec header_record
TVec< string > header_names
string header_expdir
PPath header_file_name
PP< FileVMatrixheader_file
int to_deal_with_total
int to_deal_with_next
real deletion_threshold
string deletion_threshold_str
int test_length
int test_width
Vec test_record
VMat test_file
int train_valid_length
int train_valid_width
Vec train_valid_record
VMat train_valid_file
VMat source_set
TVec< string > source_names
PP< PTesterexperiment

Detailed Description

Generate samples from a mixture of two gaussians.

Definition at line 54 of file Experimentation.h.


Member Typedef Documentation

Reimplemented from PLearn::PLearner.

Definition at line 56 of file Experimentation.h.


Constructor & Destructor Documentation

PLearn::Experimentation::Experimentation ( )

Default constructor.

Definition at line 61 of file Experimentation.cc.

{
}

Member Function Documentation

string PLearn::Experimentation::_classname_ ( ) [static]

Reimplemented from PLearn::PLearner.

Definition at line 56 of file Experimentation.cc.

OptionList & PLearn::Experimentation::_getOptionList_ ( ) [static]

Reimplemented from PLearn::PLearner.

Definition at line 56 of file Experimentation.cc.

RemoteMethodMap & PLearn::Experimentation::_getRemoteMethodMap_ ( ) [static]

Reimplemented from PLearn::PLearner.

Definition at line 56 of file Experimentation.cc.

bool PLearn::Experimentation::_isa_ ( const Object o) [static]

Reimplemented from PLearn::PLearner.

Definition at line 56 of file Experimentation.cc.

Object * PLearn::Experimentation::_new_instance_for_typemap_ ( ) [static]

Reimplemented from PLearn::Object.

Definition at line 56 of file Experimentation.cc.

StaticInitializer Experimentation::_static_initializer_ & PLearn::Experimentation::_static_initialize_ ( ) [static]

Reimplemented from PLearn::PLearner.

Definition at line 56 of file Experimentation.cc.

void PLearn::Experimentation::build ( ) [virtual]

Finish building the object; just call inherited::build followed by build_()

Reimplemented from PLearn::PLearner.

Definition at line 130 of file Experimentation.cc.

{
    // ### Nothing to add here, simply calls build_().
    inherited::build();
    build_();
}
void PLearn::Experimentation::build_ ( ) [private]

This does the actual building.

Reimplemented from PLearn::PLearner.

Definition at line 140 of file Experimentation.cc.

References PLearn::endl(), PLERROR, and PLearn::save().

{
    MODULE_LOG << "build_() called" << endl;
    if (train_set)
    {
        for (int iteration = 1; iteration <= train_set->width(); iteration++)
        {
            cout << "In Experimentation, Iteration # " << iteration << endl;
            experimentSetUp();
            train();
            ::PLearn::save(header_expdir + "/" + deletion_threshold_str + "/source_names.psave", source_names);
        }
        PLERROR("In Experimentation::build_() we are done here");
    }
}

Here is the call graph for this function:

string PLearn::Experimentation::classname ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 56 of file Experimentation.cc.

void PLearn::Experimentation::computeCostsFromOutputs ( const Vec input,
const Vec output,
const Vec target,
Vec costs 
) const [virtual]

*** SUBCLASS WRITING: ***

This should be defined in subclasses to compute the weighted costs from already computed output. The costs should correspond to the cost names returned by getTestCostNames().

NOTE: In exotic cases, the cost may also depend on some info in the input, that's why the method also gets so see it.

Implements PLearn::PLearner.

Definition at line 493 of file Experimentation.cc.

{}
void PLearn::Experimentation::computeOutput ( const Vec input,
Vec output 
) const [virtual]

*** SUBCLASS WRITING: ***

This should be defined in subclasses to compute the output from the input.

Reimplemented from PLearn::PLearner.

Definition at line 492 of file Experimentation.cc.

{}
void PLearn::Experimentation::createHeaderFile ( ) [private]

Definition at line 318 of file Experimentation.cc.

References PLearn::tostring().

{ 
    header_record.clear();
    header_names.resize(header_width);
    for (int header_col = 0; header_col < header_width; header_col++) 
        header_names[header_col] = tostring(deletion_thresholds[header_col] + 0.005).substr(0,4);
    header_file = new FileVMatrix(header_file_name, 1, header_names);
    header_file->putRow(0, header_record);
}

Here is the call graph for this function:

void PLearn::Experimentation::declareOptions ( OptionList ol) [static, protected]

Declares the class options.

Reimplemented from PLearn::PLearner.

Definition at line 68 of file Experimentation.cc.

References PLearn::OptionBase::buildoption, PLearn::declareOption(), deletion_thresholds, experiment_directory, experiment_name, experiment_template, experiment_without_missing_indicator, missing_indicator_field_names, number_of_test_samples, number_of_train_samples, reference_train_set, save_files, target_field_name, and target_set.

{
    declareOption(ol, "save_files", &Experimentation::save_files,
                  OptionBase::buildoption,
                  "If set to 1, save the built train and test files instead of running the experiment.");
    declareOption(ol, "experiment_without_missing_indicator", &Experimentation::experiment_without_missing_indicator,
                  OptionBase::buildoption,
                  "If set to 1, the missing_indicator_field_names will be excluded from the built training files.");
    declareOption(ol, "target_field_name", &Experimentation::target_field_name,
                  OptionBase::buildoption,
                  "The name of the field to select from the target_set as target for the built training files.");
    declareOption(ol, "missing_indicator_field_names", &Experimentation::missing_indicator_field_names,
                  OptionBase::buildoption,
                  "The field names of the missing indicators to exclude when we experiment without them.");
    declareOption(ol, "experiment_name", &Experimentation::experiment_name,
                  OptionBase::buildoption,
                  "The name of the group of experiments to conduct.");
    declareOption(ol, "number_of_test_samples", &Experimentation::number_of_test_samples,
                  OptionBase::buildoption,
                  "The number of test samples at the beginning of the train set.");
    declareOption(ol, "number_of_train_samples", &Experimentation::number_of_train_samples,
                  OptionBase::buildoption,
                  "The number of train samples in the reference set to compute the % of missing.");
    declareOption(ol, "reference_train_set", &Experimentation::reference_train_set,
                  OptionBase::buildoption,
                  "The train and valid set with missing values to compute the % of missing.");
    declareOption(ol, "target_set", &Experimentation::target_set,
                  OptionBase::buildoption,
                  "The data set with the targets corresponding to the train set.");
    declareOption(ol, "deletion_thresholds", &Experimentation::deletion_thresholds,
                  OptionBase::buildoption,
                  "The vector of the various deletion threshold to run this experiment with.");
    declareOption(ol, "experiment_directory", &Experimentation::experiment_directory,
                  OptionBase::buildoption,
                  "The path in which to build the directories for the experiment's results.");
    declareOption(ol, "experiment_template", &Experimentation::experiment_template,
                  OptionBase::buildoption,
                  "The template of the script to conduct the experiment.");

    inherited::declareOptions(ol);
}

Here is the call graph for this function:

static const PPath& PLearn::Experimentation::declaringFile ( ) [inline, static]

Reimplemented from PLearn::PLearner.

Definition at line 110 of file Experimentation.h.

:
    //#####  Protected Member Functions  ######################################
Experimentation * PLearn::Experimentation::deepCopy ( CopiesMap copies) const [virtual]

Reimplemented from PLearn::PLearner.

Definition at line 56 of file Experimentation.cc.

void PLearn::Experimentation::experimentSetUp ( ) [private]

Definition at line 156 of file Experimentation.cc.

References PLearn::endl(), PLearn::isfile(), PLERROR, PLearn::tostring(), and PLearn::ProgressBar::update().

{ 
    // initialize primary dataset
    main_row = 0;
    main_col = 0;
    main_length = train_set->length();
    main_width = train_set->width();
    main_names.resize(main_width);
    main_names << train_set->fieldNames();
    if (train_set->hasMetaDataDir()) main_metadata = train_set->getMetaDataDir();
    else if (experiment_directory == "") PLERROR("In Experimentation::experimentSetUp() we need one of experiment_directory or train_set->metadatadir");
         else main_metadata = experiment_directory;
    if (experiment_without_missing_indicator > 0)
    {
        fields_width = missing_indicator_field_names.size();
        main_fields_selected.resize(main_width - fields_width);
        for (fields_col = 0; fields_col < fields_width; fields_col++)
        {
            for (main_col = 0; main_col < main_width; main_col++)
            {
                if (missing_indicator_field_names[fields_col] == main_names[main_col]) break;
            }
            if (main_col >= main_width) PLERROR("In Experimentation::experimentSetUp() no field with this name in input dataset: %", (missing_indicator_field_names[fields_col]).c_str());
        }
        main_fields_selected_col = 0;
        for (main_col = 0; main_col < main_width; main_col++)
        {
            for (fields_col = 0; fields_col < fields_width; fields_col++)
            {
                if (missing_indicator_field_names[fields_col] == main_names[main_col]) break;
            }
            if (fields_col < fields_width) continue;
            main_fields_selected[main_fields_selected_col] = main_names[main_col];
            main_fields_selected_col += 1;
        }
    }
    
    // initialize target dataset
    target_row = 0;
    target_col = 0;
    target_length = target_set->length();
    target_width = target_set->width();
    target_names.resize(target_width);
    target_names << target_set->fieldNames();
    if (target_length != main_length) PLERROR("In Experimentation::experimentSetUp() target and main train datasets should have equal length");
    for (target_col = 0; target_col < target_width; target_col++)
    {
        if (target_field_name == target_names[target_col]) break;
    }
    if (target_col >= target_width) PLERROR("In Experimentation::experimentSetUp() no field with this name in target dataset: %", target_field_name.c_str());
    
    // initialize the header file
    cout << "initialize the header file" << endl;
    reference_train_set->lockMetaDataDir();
    if (experiment_directory == "") header_expdir = main_metadata + "/Experiment/" + experiment_name;
    else header_expdir = experiment_directory;
    header_expdir += "/" + target_field_name;
    if (experiment_without_missing_indicator > 0) header_expdir += "/no_ind/";
    else header_expdir += "/ind/";
    header_file_name = header_expdir + "header.pmat";
    if (deletion_thresholds.length() <= 0)
    {
        deletion_thresholds.resize(20);
        for (int header_col = 0; header_col < 20; header_col++) deletion_thresholds[header_col] = (real) to_deal_with_next / 20.0;
    } 
    header_width = deletion_thresholds.length();
    header_record.resize(header_width);
    if (!isfile(header_file_name)) createHeaderFile();
    else getHeaderRecord();
    
    // choose deletion threshold to experiment with
    cout << "choose deletion threshold to experiment with" << endl;
    to_deal_with_total = 0;
    to_deal_with_next = -1;
    for (int header_col = 0; header_col < header_width; header_col++)
    {
        if (header_record[header_col] != 0.0) continue;
        to_deal_with_total += 1;
        if (to_deal_with_next < 0) to_deal_with_next = header_col;
    }
    if (to_deal_with_next < 0)
    {
        reference_train_set->unlockMetaDataDir();
        reviewGlobalStats();
        PLERROR("In Experimentation::experimentSetUp() we are done here");
    }
    deletion_threshold = deletion_thresholds[to_deal_with_next];
    deletion_threshold_str = tostring(deletion_threshold + 0.005).substr(0,4);
    cout << "total number of thresholds left to deal with: " << to_deal_with_total << endl;
    cout << "next thresholds to deal with: " << deletion_threshold << endl;
    updateHeaderRecord(to_deal_with_next);
    reference_train_set->unlockMetaDataDir();
    
    // build the train and test sets
    setSourceDataset();
    cout << "source data set width: " << source_set->width() << endl;
    main_input.resize(source_set->width());
    source_names.resize(source_set->width());
    source_names << source_set->fieldNames();
    source_names.resize(source_set->width() + 1);
    source_names[source_set->width()] = target_field_name;
    
    // load test data set
    ProgressBar* pb = 0;
    test_length = number_of_test_samples;
    test_width = source_set->width() + 1;
    test_file = new MemoryVMatrix(test_length, test_width);
    test_file->defineSizes(test_width - 1, 1, 0);
    test_record.resize(test_width);
    pb = new ProgressBar( "loading the test file for threshold: " + deletion_threshold_str, test_length);
    for (main_row = 0; main_row < test_length; main_row++)
    {
        source_set->getRow(main_row, main_input);
        for (main_col = 0; main_col < source_set->width(); main_col++) test_record[main_col] = main_input[main_col];
        test_record[source_set->width()] = target_set->get(main_row, target_col);
        test_file->putRow(main_row, test_record);
        pb->update( main_row );
    }
    delete pb;
    
    // load training and validation data set
    train_valid_length = main_length - test_length;
    train_valid_width = source_set->width() + 1;
    train_valid_file = new MemoryVMatrix(train_valid_length, train_valid_width);
    train_valid_file->defineSizes(train_valid_width - 1, 1, 0);
    train_valid_record.resize(train_valid_width);
    pb = new ProgressBar( "loading the training and validation file for threshold: " + deletion_threshold_str, train_valid_length);
    for (main_row = test_length; main_row < main_length; main_row++)
    {
        source_set->getRow(main_row, main_input);
        for (main_col = 0; main_col < source_set->width(); main_col++) train_valid_record[main_col] = main_input[main_col];
        train_valid_record[source_set->width()] = target_set->get(main_row, target_col);
        train_valid_file->putRow(main_row - test_length, train_valid_record);
        pb->update( main_row - test_length );
    }
    delete pb;
    
    // save files if requested
    if (save_files <= 0) return;
    VMat save_test = new FileVMatrix(header_expdir + "/" + deletion_threshold_str + "/test.pmat", test_length, test_width);
    save_test->declareFieldNames(source_names);
    pb = new ProgressBar( "saving the test file for threshold: " + deletion_threshold_str, test_length);
    for (main_row = 0; main_row < test_length; main_row++)
    {
        test_file->getRow(main_row, test_record);
        save_test->putRow(main_row, test_record);
        pb->update( main_row );
    }
    delete pb;
    VMat save_train_valid = new FileVMatrix(header_expdir + "/" + deletion_threshold_str + "/train_valid.pmat", train_valid_length, train_valid_width);
    save_train_valid->declareFieldNames(source_names);
    pb = new ProgressBar( "saving the training and validation file for threshold: " + deletion_threshold_str, train_valid_length);
    for (main_row = 0; main_row < train_valid_length; main_row++)
    {
        train_valid_file->getRow(main_row, train_valid_record);
        save_train_valid->putRow(main_row, train_valid_record);
        pb->update( main_row );
    }
    delete pb;
    PLERROR("In Experimentation::experimentSetUp() we are done here");
}

Here is the call graph for this function:

void PLearn::Experimentation::getHeaderRecord ( ) [private]

Definition at line 328 of file Experimentation.cc.

References PLERROR, and PLearn::tostring().

{ 
    header_file = new FileVMatrix(header_file_name, true);
    if (header_width != header_file->width()) 
        PLERROR("In Experimentation::getHeaderRecord() the existing header file does not match the deletion_thresholds width)");
    header_names = header_file->fieldNames();
    for (int header_col = 0; header_col < header_width; header_col++) 
        if (header_names[header_col] != tostring(deletion_thresholds[header_col] + 0.005).substr(0,4))
            PLERROR("In Experimentation::getHeaderRecord() the existing header file names does not match the deletion_thresholds values)");;
    header_file->getRow(0, header_record);
}

Here is the call graph for this function:

OptionList & PLearn::Experimentation::getOptionList ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 56 of file Experimentation.cc.

OptionMap & PLearn::Experimentation::getOptionMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 56 of file Experimentation.cc.

RemoteMethodMap & PLearn::Experimentation::getRemoteMethodMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 56 of file Experimentation.cc.

TVec< string > PLearn::Experimentation::getTestCostNames ( ) const [virtual]

*** SUBCLASS WRITING: ***

This should return the names of the costs computed by computeCostsFromOutputs.

Implements PLearn::PLearner.

Definition at line 494 of file Experimentation.cc.

References PLearn::TVec< T >::append().

{
    TVec<string> result;
    result.append( "MSE" );
    return result;
}

Here is the call graph for this function:

TVec< string > PLearn::Experimentation::getTrainCostNames ( ) const [virtual]

*** SUBCLASS WRITING: ***

This should return the names of the objective costs that the train method computes and for which it updates the VecStatsCollector train_stats.

Implements PLearn::PLearner.

Definition at line 500 of file Experimentation.cc.

References PLearn::TVec< T >::append().

{
    TVec<string> result;
    result.append( "MSE" );
    return result;
}

Here is the call graph for this function:

void PLearn::Experimentation::makeDeepCopyFromShallowCopy ( CopiesMap copies) [virtual]

Transforms a shallow copy into a deep copy.

Reimplemented from PLearn::PLearner.

Definition at line 113 of file Experimentation.cc.

References PLearn::deepCopyField().

Here is the call graph for this function:

int PLearn::Experimentation::outputsize ( ) const [virtual]

SUBCLASS WRITING: override this so that it returns the size of this learner's output, as a function of its inputsize(), targetsize() and set options.

Implements PLearn::PLearner.

Definition at line 491 of file Experimentation.cc.

{return 0;}
void PLearn::Experimentation::reviewGlobalStats ( ) [private]

Definition at line 389 of file Experimentation.cc.

References PLearn::endl(), PLearn::isfile(), PLearn::VMat::length(), PLearn::load(), PLearn::right(), and PLearn::tostring().

{
    cout << "There is no more variable to deal with." << endl;
    bool missing_results_file = false;
    for (int header_col = 0; header_col < header_width; header_col++)
    {
        deletion_threshold = deletion_thresholds[header_col];
        deletion_threshold_str = tostring(deletion_threshold + 0.005).substr(0,4);
        PPath expdir = header_expdir + "/" + deletion_threshold_str;
        PPath train_valid_results_file_name = expdir + "/Split0/LearnerExpdir/Strat0results.pmat";
        PPath test_results_file_name = expdir + "/global_stats.pmat";
        PPath source_names_file_name = expdir + "/source_names.psave";
        if (!isfile(train_valid_results_file_name))
        {
            cout << "Missing training and validation results for threshold " << deletion_threshold_str << endl;
            missing_results_file = true;
        }
        if (!isfile(test_results_file_name))
        {
            cout << "Missing test results for threshold " << deletion_threshold_str << endl;
            missing_results_file = true;
        }
        if (!isfile(source_names_file_name))
        {
            cout << "Missing variable selected saved file for threshold " << deletion_threshold_str << endl;
            missing_results_file = true;
        }
    }
    if (missing_results_file) return;
    cout << endl << endl;
    cout << "Results for experiment " << experiment_name << endl;
    cout << "       The file used for this experiment was " << main_metadata << endl;
    cout << "       The target used was " << target_field_name << endl;
    if (experiment_without_missing_indicator > 0) cout << "       The experiment was carried without missing indicators" << endl;
    else cout << "       The experiment was carried with missing indicators" << endl;
    cout << endl << endl;
    cout << "           number                                                                  " << endl;
    cout << "             of                                                                    " << endl;
    cout << " deletion variable   weigth    train    valid     test     test       std      test" << endl;
    cout << "threshold selected    decay     mse      mse      mse      cse       error     cle " << endl;
    cout << endl;
    cout << fixed << showpoint;
    real best_valid_mse_threshold = -1.0;
    real best_valid_mse_value;
    for (int header_col = 0; header_col < header_width; header_col++)
    {
        deletion_threshold = deletion_thresholds[header_col];
        deletion_threshold_str = tostring(deletion_threshold + 0.005).substr(0,4);
        PPath expdir = header_expdir + "/" + deletion_threshold_str;
        PPath train_valid_results_file_name = expdir + "/Split0/LearnerExpdir/Strat0results.pmat";
        PPath test_results_file_name = expdir + "/global_stats.pmat";
        PPath source_names_file_name = expdir + "/source_names.psave";
        ::PLearn::load(source_names_file_name, source_names);
        VMat train_valid_results_file = new FileVMatrix(train_valid_results_file_name);
        VMat test_results_file = new FileVMatrix(test_results_file_name);
        int train_valid_last_row = train_valid_results_file->length() - 1;
        real weight_decay = train_valid_results_file->get(train_valid_last_row, 2);
        real train_mse =    train_valid_results_file->get(train_valid_last_row, 3);
        real valid_mse =    train_valid_results_file->get(train_valid_last_row, 4);
        if (best_valid_mse_threshold < 0.0)
        {
            best_valid_mse_threshold = deletion_threshold;
            best_valid_mse_value = valid_mse;
        }
        else if (valid_mse < best_valid_mse_value)
        {
            best_valid_mse_threshold = deletion_threshold;
            best_valid_mse_value = valid_mse;
        }
        real test_mse =     test_results_file->get(1, 0);
        real test_cse =     test_results_file->get(1, 2);
        real test_cse_std = test_results_file->get(1, 3);
        real test_cle =     test_results_file->get(1, 4);
        cout << setiosflags(ios::right) << setw(9) << deletion_threshold_str << "   "
             << setw(4) << source_names.size() << "    "
             << setw(6) << weight_decay << " "
             << setw(6) << train_mse << " "
             << setw(6) << valid_mse << " "
             << setw(6) << test_mse << " "
             << setw(6) << test_cse << "+/-"
             << setw(6) << test_cse_std << " "
             << setw(6) << test_cle << endl;
    }
    cout << endl << endl;
    cout << "       Based on the validation mse, the suggested threshold is " << best_valid_mse_threshold << endl;
    cout << endl << endl;
}

Here is the call graph for this function:

void PLearn::Experimentation::setSourceDataset ( ) [private]

Definition at line 346 of file Experimentation.cc.

References PLearn::VariableDeletionVMatrix::build(), PLearn::VariableDeletionVMatrix::complete_dataset, PLearn::VariableDeletionVMatrix::deletion_threshold, PLearn::SelectColumnsVMatrix::fields, PLearn::VariableDeletionVMatrix::number_of_train_samples, PLearn::VariableDeletionVMatrix::remove_columns_with_constant_value, PLearn::SourceVMatrix::source, PLearn::VariableDeletionVMatrix::train_set, and PLearn::VMat::width().

{
    VMat selected_train_set = train_set;
    VMat selected_reference_set= reference_train_set;
    if (experiment_without_missing_indicator > 0)
    {
            SelectColumnsVMatrix* new_train_set = new SelectColumnsVMatrix();
            new_train_set->source = train_set;
            new_train_set->fields = main_fields_selected;
            selected_train_set = new_train_set;
            selected_train_set->build();
            selected_train_set->defineSizes(selected_train_set->width(), 0, 0);
            SelectColumnsVMatrix* new_reference_set = new SelectColumnsVMatrix();
            new_reference_set->source = reference_train_set;
            new_reference_set->fields = main_fields_selected;
            selected_reference_set = new_reference_set;
            selected_reference_set->build();
            selected_reference_set->defineSizes(selected_reference_set->width(), 0, 0);
    }
    if (deletion_threshold <= 0.0)
    {
        source_set = selected_train_set;
        return;
    }
    VariableDeletionVMatrix* new_set = new VariableDeletionVMatrix();
    // VMat: The data set with all variables to select the columns from.
    new_set->complete_dataset = selected_train_set;
    // VMat: The train set in which to compute the percentage of missing values.
    new_set->train_set = selected_reference_set;
    // double: The percentage of non-missing values for a variable above which, the variable will be selected.
    new_set->deletion_threshold = deletion_threshold;
    // bool: If set to 1, the columns with constant non-missing values will be removed.
    new_set->remove_columns_with_constant_value = 1;
    // double: If equal to zero, all the train samples are used to calculated the percentages and constant values.
    // If it is a fraction between 0 and 1, this proportion of the samples will be used.
    // If greater or equal to 1, the integer portion will be interpreted as the number of samples to use.
    new_set->number_of_train_samples = number_of_train_samples;
    // int: The row at which, to start to calculate the percentages and constant values.
    new_set->start_row = 0;
    source_set = new_set;
    source_set->build();
}

Here is the call graph for this function:

void PLearn::Experimentation::train ( ) [virtual]

*** SUBCLASS WRITING: ***

The role of the train method is to bring the learner up to stage==nstages, updating the stats with training costs measured on-line in the process.

TYPICAL CODE:

  static Vec input;  // static so we don't reallocate/deallocate memory each time...
  static Vec target; // (but be careful that static means shared!)
  input.resize(inputsize());    // the train_set's inputsize()
  target.resize(targetsize());  // the train_set's targetsize()
  real weight;
  
  if(!train_stats)   // make a default stats collector, in case there's none
      train_stats = new VecStatsCollector();
  
  if(nstages<stage)  // asking to revert to a previous stage!
      forget();      // reset the learner to stage=0
  
  while(stage<nstages)
  {
      // clear statistics of previous epoch
      train_stats->forget(); 
            
      //... train for 1 stage, and update train_stats,
      // using train_set->getSample(input, target, weight);
      // and train_stats->update(train_costs)
          
      ++stage;
      train_stats->finalize(); // finalize statistics for this epoch
  }

Implements PLearn::PLearner.

Definition at line 477 of file Experimentation.cc.

References PLearn::deepCopy().

{
    PP<ExplicitSplitter> explicit_splitter = new ExplicitSplitter();
    explicit_splitter->splitsets.resize(1,2);
    explicit_splitter->splitsets(0,0) = train_valid_file;
    explicit_splitter->splitsets(0,1) = test_file;
    experiment = ::PLearn::deepCopy(experiment_template);
    experiment->setOption("expdir", header_expdir + "/" + deletion_threshold_str);
    experiment->splitter = new ExplicitSplitter();
    experiment->splitter = explicit_splitter;
    experiment->build();
    Vec results = experiment->perform(true);
}

Here is the call graph for this function:

void PLearn::Experimentation::updateHeaderRecord ( int  var_col) [private]

Definition at line 340 of file Experimentation.cc.

{ 
    header_file->put(0, var_col, 1.0);
    header_file->flush();
}

Member Data Documentation

Reimplemented from PLearn::PLearner.

Definition at line 110 of file Experimentation.h.

Definition at line 171 of file Experimentation.h.

Definition at line 172 of file Experimentation.h.

The vector of the various deletion threshold to run this experiment with.

Definition at line 84 of file Experimentation.h.

Referenced by declareOptions().

Definition at line 186 of file Experimentation.h.

The path in which to build the directories for the experiment's results.

Definition at line 86 of file Experimentation.h.

Referenced by declareOptions().

The name of the group of experiments to conduct.

Definition at line 74 of file Experimentation.h.

Referenced by declareOptions().

The template of the script to conduct the experiment.

Definition at line 88 of file Experimentation.h.

Referenced by declareOptions().

If set to 1, the missing_indicator_field_names will be excluded from the built training files.

Definition at line 68 of file Experimentation.h.

Referenced by declareOptions().

Definition at line 153 of file Experimentation.h.

Definition at line 152 of file Experimentation.h.

Definition at line 165 of file Experimentation.h.

Definition at line 167 of file Experimentation.h.

Definition at line 166 of file Experimentation.h.

Definition at line 164 of file Experimentation.h.

Definition at line 163 of file Experimentation.h.

Definition at line 162 of file Experimentation.h.

Definition at line 143 of file Experimentation.h.

Definition at line 150 of file Experimentation.h.

Definition at line 149 of file Experimentation.h.

Definition at line 146 of file Experimentation.h.

Definition at line 144 of file Experimentation.h.

Definition at line 148 of file Experimentation.h.

Definition at line 147 of file Experimentation.h.

Definition at line 142 of file Experimentation.h.

Definition at line 145 of file Experimentation.h.

The field names of the missing indicators to exclude when we experiment without them.

Definition at line 72 of file Experimentation.h.

Referenced by declareOptions().

The number of test samples at the beginning of the train set.

Definition at line 76 of file Experimentation.h.

Referenced by declareOptions().

The number of train samples in the reference set to compute the % of missing.

Definition at line 78 of file Experimentation.h.

Referenced by declareOptions().

The train and valid set with missing values to compute the % of missing.

Definition at line 80 of file Experimentation.h.

Referenced by declareOptions().

### declare public option fields (such as build options) here Start your comments with Doxygen-compatible comments such as //!

If set to 1, save the built train and test files instead of running the experiment.

Definition at line 66 of file Experimentation.h.

Referenced by declareOptions().

Definition at line 185 of file Experimentation.h.

Definition at line 184 of file Experimentation.h.

Definition at line 156 of file Experimentation.h.

The name of the field to select from the target_set as target for the built training files.

Definition at line 70 of file Experimentation.h.

Referenced by declareOptions().

Definition at line 159 of file Experimentation.h.

Definition at line 157 of file Experimentation.h.

Definition at line 160 of file Experimentation.h.

Definition at line 155 of file Experimentation.h.

The data set with the targets corresponding to the train set.

Definition at line 82 of file Experimentation.h.

Referenced by declareOptions().

Definition at line 158 of file Experimentation.h.

Definition at line 177 of file Experimentation.h.

Definition at line 174 of file Experimentation.h.

Definition at line 176 of file Experimentation.h.

Definition at line 175 of file Experimentation.h.

Definition at line 170 of file Experimentation.h.

Definition at line 169 of file Experimentation.h.

Definition at line 182 of file Experimentation.h.

Definition at line 179 of file Experimentation.h.

Definition at line 181 of file Experimentation.h.

Definition at line 180 of file Experimentation.h.


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines