PLearn 0.1
Public Member Functions | Static Public Member Functions | Public Attributes | Static Public Attributes | Static Protected Member Functions | Private Types | Private Member Functions | Private Attributes
PLearn::AnalyzeFieldStats Class Reference

Generate samples from a mixture of two gaussians. More...

#include <AnalyzeFieldStats.h>

Inheritance diagram for PLearn::AnalyzeFieldStats:
Inheritance graph
[legend]
Collaboration diagram for PLearn::AnalyzeFieldStats:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 AnalyzeFieldStats ()
 Default constructor.
int outputsize () const
 SUBCLASS WRITING: override this so that it returns the size of this learner's output, as a function of its inputsize(), targetsize() and set options.
void train ()
 *** SUBCLASS WRITING: ***
void computeOutput (const Vec &, Vec &) const
 *** SUBCLASS WRITING: ***
void computeCostsFromOutputs (const Vec &, const Vec &, const Vec &, Vec &) const
 *** SUBCLASS WRITING: ***
TVec< string > getTestCostNames () const
 *** SUBCLASS WRITING: ***
TVec< string > getTrainCostNames () const
 *** SUBCLASS WRITING: ***
virtual string classname () const
virtual OptionListgetOptionList () const
virtual OptionMapgetOptionMap () const
virtual RemoteMethodMapgetRemoteMethodMap () const
virtual AnalyzeFieldStatsdeepCopy (CopiesMap &copies) const
virtual void build ()
 Finish building the object; just call inherited::build followed by build_()
virtual void makeDeepCopyFromShallowCopy (CopiesMap &copies)
 Transforms a shallow copy into a deep copy.

Static Public Member Functions

static string _classname_ ()
static OptionList_getOptionList_ ()
static RemoteMethodMap_getRemoteMethodMap_ ()
static Object_new_instance_for_typemap_ ()
static bool _isa_ (const Object *o)
static void _static_initialize_ ()
static const PPathdeclaringFile ()

Public Attributes

int min_number_of_samples
 ### declare public option fields (such as build options) here Start your comments with Doxygen-compatible comments such as //!
int max_number_of_samples
 The maximum number of samples used to train the learner.
VMat targeted_set
 The train and test data sets with the target field.
PP< PTestercond_mean_template
 The template of the script to learn the conditional mean.
TVec< string > fields
 The field name of the variable to be analyzed.

Static Public Attributes

static StaticInitializer _static_initializer_

Static Protected Member Functions

static void declareOptions (OptionList &ol)
 Declares the class options.

Private Types

typedef PLearner inherited

Private Member Functions

void build_ ()
 This does the actual building.
void analyzeVariableStats ()
void createHeaderFile ()
void getHeaderRecord ()
void updateHeaderRecord (int var_col)
void reviewGlobalStats ()

Private Attributes

int main_width
TVec< string > main_names
StatsCollector main_stats
PPath main_metadata
TVec< intmain_ins
real main_total
real main_missing
real main_present
int targeted_length
int targeted_width
Vec targeted_input
TVec< string > targeted_names
StatsCollector targeted_stats
PPath targeted_metadata
real targeted_missing
PPath header_file_name
VMat header_file
Vec header_record
int fields_col
int fields_width
TVec< intfields_selected
int to_deal_with_total
int to_deal_with_next
real to_deal_with_value
string to_deal_with_name
int ind_next
int output_length
int output_width
int output_col
string output_path
TVec< string > output_names
Vec output_vec
TVec< intoutput_variable_src
VMat output_file
int train_test_length
string train_test_path
TVec< inttrain_test_variable_src
VMat train_test_file
PPath results_file_name
VMat results_file
int results_length
real results_nstages
real results_mse
real results_std_err
PPath test_output_file_name
VMat test_output_file
int test_output_length

Detailed Description

Generate samples from a mixture of two gaussians.

Definition at line 54 of file AnalyzeFieldStats.h.


Member Typedef Documentation

Reimplemented from PLearn::PLearner.

Definition at line 56 of file AnalyzeFieldStats.h.


Constructor & Destructor Documentation

PLearn::AnalyzeFieldStats::AnalyzeFieldStats ( )

Default constructor.

Definition at line 60 of file AnalyzeFieldStats.cc.


Member Function Documentation

string PLearn::AnalyzeFieldStats::_classname_ ( ) [static]

Reimplemented from PLearn::PLearner.

Definition at line 55 of file AnalyzeFieldStats.cc.

OptionList & PLearn::AnalyzeFieldStats::_getOptionList_ ( ) [static]

Reimplemented from PLearn::PLearner.

Definition at line 55 of file AnalyzeFieldStats.cc.

RemoteMethodMap & PLearn::AnalyzeFieldStats::_getRemoteMethodMap_ ( ) [static]

Reimplemented from PLearn::PLearner.

Definition at line 55 of file AnalyzeFieldStats.cc.

bool PLearn::AnalyzeFieldStats::_isa_ ( const Object o) [static]

Reimplemented from PLearn::PLearner.

Definition at line 55 of file AnalyzeFieldStats.cc.

Object * PLearn::AnalyzeFieldStats::_new_instance_for_typemap_ ( ) [static]

Reimplemented from PLearn::Object.

Definition at line 55 of file AnalyzeFieldStats.cc.

StaticInitializer AnalyzeFieldStats::_static_initializer_ & PLearn::AnalyzeFieldStats::_static_initialize_ ( ) [static]

Reimplemented from PLearn::PLearner.

Definition at line 55 of file AnalyzeFieldStats.cc.

void PLearn::AnalyzeFieldStats::analyzeVariableStats ( ) [private]

Definition at line 133 of file AnalyzeFieldStats.cc.

References PLearn::TVec< T >::clear(), createHeaderFile(), PLearn::endl(), fields, fields_col, fields_selected, fields_width, getHeaderRecord(), header_file_name, header_record, ind_next, PLearn::is_missing(), PLearn::isfile(), PLearn::TVec< T >::length(), PLearn::VMat::length(), main_metadata, main_missing, main_names, main_present, main_stats, main_total, main_width, PLearn::manual_seed(), max_number_of_samples, PLearn::StatsCollector::n(), PLearn::StatsCollector::nmissing(), output_col, output_file, output_length, output_names, output_path, output_variable_src, output_vec, output_width, PLERROR, PLearn::TVec< T >::resize(), reviewGlobalStats(), PLearn::shuffleElements(), PLearn::TVec< T >::size(), targeted_input, targeted_length, targeted_metadata, targeted_names, targeted_set, targeted_width, to_deal_with_name, to_deal_with_next, to_deal_with_total, to_deal_with_value, PLearn::PLearner::train_set, train_test_file, train_test_length, train_test_path, train_test_variable_src, PLearn::ProgressBar::update(), updateHeaderRecord(), and PLearn::VMat::width().

Referenced by build_().

{ 
    // initialize primary dataset
    int main_length = train_set->length();
    main_width = train_set->width();
    Vec main_input;
    main_input.resize(main_width);
    main_names.resize(main_width);
    main_names << train_set->fieldNames();
    main_metadata = train_set->getMetaDataDir();
    
    // validate the field instructions
    fields_width = fields.size();
    fields_selected.resize(main_width);
    fields_selected.clear();
    for (fields_col = 0; fields_col < fields_width; fields_col++)
    {
        int main_col;
        for (main_col = 0; main_col < main_width; main_col++)
        {
            if (fields[fields_col] == main_names[main_col]) break;
        }
        if (main_col >= main_width) 
            PLERROR("In AnalyzeFieldStats::analyzeVariableStats() no field with this name in input dataset: %s", (fields[fields_col]).c_str());
        fields_selected[main_col] = 1;
    }
    
    // initialize targeted datasets
    cout << "initialize train_test datasets" << endl;
    targeted_length = targeted_set->length();
    targeted_width = targeted_set->width();
    targeted_input.resize(targeted_width);
    targeted_names.resize(targeted_width);
    targeted_names << targeted_set->fieldNames();
    targeted_metadata = targeted_set->getMetaDataDir();
    
    // initialize the header file
    cout << "initialize the header file" << endl;
    train_set->lockMetaDataDir();
    header_record.resize(main_width);
    header_file_name = targeted_metadata + "/TreeCondMean/header.pmat";
    if (!isfile(header_file_name)) createHeaderFile();
    else getHeaderRecord();
    
    // choose variable to build a conditionnal function for
    cout << "choose variable to build a conditionnal function for" << endl;
    TVec<int> indices;
    to_deal_with_total = 0;
    to_deal_with_next = -1;
    for (int main_col = 0; main_col < main_width; main_col++)
    {
        if (header_record[main_col] != 2.0) continue;
        to_deal_with_total += 1;
        if (to_deal_with_next < 0) to_deal_with_next = main_col;
    }
    if (to_deal_with_next < 0)
    {
        train_set->unlockMetaDataDir();
        reviewGlobalStats();
        PLERROR("AnalyzeFieldStats::analyzeVariableStats() we are done here");
    }
    to_deal_with_name = main_names[to_deal_with_next];
    cout << "total number of variable left to deal with: " << to_deal_with_total << endl;
    cout << "next variable to deal with: " << main_names[to_deal_with_next] << endl;
    updateHeaderRecord(to_deal_with_next);
    train_set->unlockMetaDataDir();
    
    // find the available targeted records for this variable
    ProgressBar* pb = 0;
    main_stats = train_set->getStats(to_deal_with_next);
    main_total = main_stats.n();
    main_missing = main_stats.nmissing();
    main_present = main_total - main_missing;
    indices.resize((int) main_present);
    ind_next = 0;
    pb = new ProgressBar( "Building the indices for " + to_deal_with_name, main_length);
    for (int main_row = 0; main_row < main_length; main_row++)
    {
        to_deal_with_value = train_set->get(main_row, to_deal_with_next);
        if (is_missing(to_deal_with_value)) continue;
        if (ind_next >= indices.length()) 
            PLERROR("AnalyzeFieldStats::analyzeVariableStats() There seems to be more present values than indicated by the stats file");
        indices[ind_next] = main_row;
        ind_next += 1;
        pb->update( main_row );
    }
    delete pb;
    
    // shuffle the indices.
    manual_seed(123456);
    shuffleElements(indices);
    
    // initialize output datasets
    output_length = (int) main_present;
    if (output_length > max_number_of_samples) output_length = max_number_of_samples;
    output_width = 0;
    for (int main_col = 0; main_col < main_width; main_col++)
    {
        if (header_record[main_col] != 1) output_width += 1;
    }
    output_variable_src.resize(output_width);
    output_names.resize(output_width);
    output_vec.resize(output_width);
    output_path = main_metadata + "condmean_" + to_deal_with_name + ".pmat";
    output_col = 0;
    for (fields_col = 0; fields_col < fields_width; fields_col++)
    {
        int main_col;
        for (main_col = 0; main_col < main_width; main_col++)
        {
            if (fields[fields_col] == main_names[main_col]) break;
        }
        if (main_col >= main_width) 
            PLERROR("In AnalyzeFieldStats::analyzeVariableStats() no field with this name in input dataset: %s", (fields[fields_col]).c_str());
        if (fields_col != to_deal_with_next && header_record[main_col] != 1)
        {
            output_variable_src[output_col] = main_col;
            output_names[output_col] = fields[fields_col];
            output_col += 1;
        }
    }
    output_variable_src[output_col] = to_deal_with_next;
    output_names[output_col] = to_deal_with_name;
    output_file = new MemoryVMatrix(output_length, output_width);
    output_file->declareFieldNames(output_names);
    output_file->defineSizes(output_width - 1, 1, 0);
    
    //Now, we can build the training file
    pb = new ProgressBar( "Building the training file for " + to_deal_with_name, output_length);
    for (int main_row = 0; main_row < output_length; main_row++)
    {
        train_set->getRow(indices[main_row], main_input);
        for (output_col = 0; output_col < output_width; output_col++)
        {
            output_vec[output_col] = main_input[output_variable_src[output_col]];
        }
        output_file->putRow(main_row, output_vec);
        pb->update( main_row );
    }
    delete pb;
    
    // initialize train_test datasets
    train_test_length = targeted_length;
    train_test_variable_src.resize(output_width);
    train_test_path = targeted_metadata + "targeted_" + to_deal_with_name + ".pmat";
    output_col = 0;
    for (fields_col = 0; fields_col < fields_width; fields_col++)
    {
        int main_col;
        for (main_col = 0; main_col < targeted_width; main_col++)
        {
            if (fields[fields_col] == targeted_names[main_col]) break;
        }
        if (main_col >= targeted_width) 
            PLERROR("In AnalyzeFieldStats::analyzeVariableStats() no field with this name in targeted dataset: %s", (fields[fields_col]).c_str());
        if (fields_col != to_deal_with_next && header_record[main_col] != 1)
        {
            train_test_variable_src[output_col] = main_col;
            output_col += 1;
        }
    }
    train_test_variable_src[output_col] = to_deal_with_next;
    train_test_file = new MemoryVMatrix(train_test_length, output_width);
    train_test_file->declareFieldNames(output_names);
    train_test_file->defineSizes(output_width - 1, 1, 0);
    
    //Now, we can build the targeted file
    pb = new ProgressBar( "Building the targeted file for " + to_deal_with_name, train_test_length);
    for (int main_row = 0; main_row < train_test_length; main_row++)
    {
        targeted_set->getRow(main_row, targeted_input);
        for (output_col = 0; output_col < output_width; output_col++)
        {
            output_vec[output_col] = targeted_input[train_test_variable_src[output_col]];
        }
        train_test_file->putRow(main_row, output_vec);
        pb->update( main_row );
    }
    delete pb;
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::AnalyzeFieldStats::build ( ) [virtual]

Finish building the object; just call inherited::build followed by build_()

Reimplemented from PLearn::PLearner.

Definition at line 108 of file AnalyzeFieldStats.cc.

References PLearn::PLearner::build(), and build_().

{
    // ### Nothing to add here, simply calls build_().
    inherited::build();
    build_();
}

Here is the call graph for this function:

void PLearn::AnalyzeFieldStats::build_ ( ) [private]

This does the actual building.

Reimplemented from PLearn::PLearner.

Definition at line 118 of file AnalyzeFieldStats.cc.

References analyzeVariableStats(), PLearn::endl(), PLERROR, train(), PLearn::PLearner::train_set, and PLearn::VMat::width().

Referenced by build().

{
    MODULE_LOG << "build_() called" << endl;
    if (train_set)
    {
        for (int iteration = 1; iteration <= train_set->width(); iteration++)
        {
            cout << "In AnalyzeFieldStats, Iteration # " << iteration << endl;
            analyzeVariableStats();
            train();
        }
        PLERROR("AnalyzeFieldStats::build_() we are done here");
    }
}

Here is the call graph for this function:

Here is the caller graph for this function:

string PLearn::AnalyzeFieldStats::classname ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 55 of file AnalyzeFieldStats.cc.

void PLearn::AnalyzeFieldStats::computeCostsFromOutputs ( const Vec input,
const Vec output,
const Vec target,
Vec costs 
) const [virtual]

*** SUBCLASS WRITING: ***

This should be defined in subclasses to compute the weighted costs from already computed output. The costs should correspond to the cost names returned by getTestCostNames().

NOTE: In exotic cases, the cost may also depend on some info in the input, that's why the method also gets so see it.

Implements PLearn::PLearner.

Definition at line 433 of file AnalyzeFieldStats.cc.

{}
void PLearn::AnalyzeFieldStats::computeOutput ( const Vec input,
Vec output 
) const [virtual]

*** SUBCLASS WRITING: ***

This should be defined in subclasses to compute the output from the input.

Reimplemented from PLearn::PLearner.

Definition at line 432 of file AnalyzeFieldStats.cc.

{}
void PLearn::AnalyzeFieldStats::createHeaderFile ( ) [private]

Definition at line 314 of file AnalyzeFieldStats.cc.

References fields_selected, header_file, header_file_name, header_record, main_missing, main_names, main_present, main_stats, main_total, main_width, min_number_of_samples, PLearn::StatsCollector::n(), PLearn::StatsCollector::nmissing(), targeted_missing, targeted_set, targeted_stats, and PLearn::PLearner::train_set.

Referenced by analyzeVariableStats().

{ 
    for (int main_col = 0; main_col < main_width; main_col++)
    {
        targeted_stats = targeted_set->getStats(main_col);
        targeted_missing = targeted_stats.nmissing();
        main_stats = train_set->getStats(main_col);
        main_total = main_stats.n();
        main_missing = main_stats.nmissing();
        main_present = main_total - main_missing;
        if (fields_selected[main_col] < 1) header_record[main_col] = 1;                  // delete column, field not selected
        else if (targeted_missing <= 0) header_record[main_col] = 0;                     // nothing to do
        else if (main_present < min_number_of_samples) header_record[main_col] = 1;      // delete column
        else header_record[main_col] = 2;                                                // build tree
    }
    header_file = new FileVMatrix(header_file_name, 1, main_names);
    header_file->putRow(0, header_record);
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::AnalyzeFieldStats::declareOptions ( OptionList ol) [static, protected]

Declares the class options.

Reimplemented from PLearn::PLearner.

Definition at line 69 of file AnalyzeFieldStats.cc.

References PLearn::OptionBase::buildoption, cond_mean_template, PLearn::declareOption(), PLearn::PLearner::declareOptions(), fields, max_number_of_samples, min_number_of_samples, and targeted_set.

{

    declareOption(ol, "min_number_of_samples", &AnalyzeFieldStats::min_number_of_samples,
                  OptionBase::buildoption,
                  "The minimum number of samples required to train the learner.");
    declareOption(ol, "max_number_of_samples", &AnalyzeFieldStats::max_number_of_samples,
                  OptionBase::buildoption,
                  "The maximum number of samples used to train the learner");
    declareOption(ol, "targeted_set", &AnalyzeFieldStats::targeted_set,
                  OptionBase::buildoption,
                  "The train and test data sets with the target field.");
    declareOption(ol, "cond_mean_template", &AnalyzeFieldStats::cond_mean_template,
                  OptionBase::buildoption,
                  "The template of the script to learn the conditional mean.");
    declareOption(ol, "fields", &AnalyzeFieldStats::fields,
                  OptionBase::buildoption,
                  "The vector of fields to consider by names.");

    inherited::declareOptions(ol);
}

Here is the call graph for this function:

static const PPath& PLearn::AnalyzeFieldStats::declaringFile ( ) [inline, static]

Reimplemented from PLearn::PLearner.

Definition at line 96 of file AnalyzeFieldStats.h.

:
    //#####  Protected Member Functions  ######################################
AnalyzeFieldStats * PLearn::AnalyzeFieldStats::deepCopy ( CopiesMap copies) const [virtual]

Reimplemented from PLearn::PLearner.

Definition at line 55 of file AnalyzeFieldStats.cc.

void PLearn::AnalyzeFieldStats::getHeaderRecord ( ) [private]

Definition at line 333 of file AnalyzeFieldStats.cc.

References fields_selected, header_file, header_file_name, header_record, main_missing, main_present, main_stats, main_total, main_width, min_number_of_samples, PLearn::StatsCollector::n(), PLearn::StatsCollector::nmissing(), and PLearn::PLearner::train_set.

Referenced by analyzeVariableStats().

{ 
    header_file = new FileVMatrix(header_file_name, true);
    header_file->getRow(0, header_record);
    for (int main_col = 0; main_col < main_width; main_col++)
    {
        if (header_record[main_col] == 0) continue;
        if (header_record[main_col] == 2) continue;
        if (header_record[main_col] == 1 && fields_selected[main_col] < 1) continue;
        if (header_record[main_col] == 1)
        {
            main_stats = train_set->getStats(main_col);
            main_total = main_stats.n();
            main_missing = main_stats.nmissing();
            main_present = main_total - main_missing;
            if (main_present >= min_number_of_samples) header_record[main_col] = 2;
            continue;
        }
    }
}

Here is the call graph for this function:

Here is the caller graph for this function:

OptionList & PLearn::AnalyzeFieldStats::getOptionList ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 55 of file AnalyzeFieldStats.cc.

OptionMap & PLearn::AnalyzeFieldStats::getOptionMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 55 of file AnalyzeFieldStats.cc.

RemoteMethodMap & PLearn::AnalyzeFieldStats::getRemoteMethodMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 55 of file AnalyzeFieldStats.cc.

TVec< string > PLearn::AnalyzeFieldStats::getTestCostNames ( ) const [virtual]

*** SUBCLASS WRITING: ***

This should return the names of the costs computed by computeCostsFromOutputs.

Implements PLearn::PLearner.

Definition at line 434 of file AnalyzeFieldStats.cc.

References PLearn::TVec< T >::append().

{
    TVec<string> result;
    result.append( "MSE" );
    return result;
}

Here is the call graph for this function:

TVec< string > PLearn::AnalyzeFieldStats::getTrainCostNames ( ) const [virtual]

*** SUBCLASS WRITING: ***

This should return the names of the objective costs that the train method computes and for which it updates the VecStatsCollector train_stats.

Implements PLearn::PLearner.

Definition at line 440 of file AnalyzeFieldStats.cc.

References PLearn::TVec< T >::append().

{
    TVec<string> result;
    result.append( "MSE" );
    return result;
}

Here is the call graph for this function:

void PLearn::AnalyzeFieldStats::makeDeepCopyFromShallowCopy ( CopiesMap copies) [virtual]
int PLearn::AnalyzeFieldStats::outputsize ( ) const [virtual]

SUBCLASS WRITING: override this so that it returns the size of this learner's output, as a function of its inputsize(), targetsize() and set options.

Implements PLearn::PLearner.

Definition at line 431 of file AnalyzeFieldStats.cc.

{return 0;}
void PLearn::AnalyzeFieldStats::reviewGlobalStats ( ) [private]

Definition at line 359 of file AnalyzeFieldStats.cc.

References PLearn::endl(), fields_selected, header_file, header_record, PLearn::isfile(), PLearn::left(), PLearn::VMat::length(), main_missing, main_names, main_present, main_stats, main_total, main_width, PLearn::StatsCollector::n(), PLearn::StatsCollector::nmissing(), results_file, results_file_name, results_length, results_mse, results_nstages, results_std_err, targeted_metadata, test_output_file, test_output_file_name, test_output_length, and PLearn::PLearner::train_set.

Referenced by analyzeVariableStats().

{ 
    cout << "There is no more variable to deal with." << endl;
    for (int main_col = 0; main_col < main_width; main_col++)
    {
        if (header_record[main_col] == 0)
        { 
            cout << setiosflags(ios::left) << setw(30) << main_names[main_col];
            cout << " : no missing values for this variable in the targeted files." << endl;
            continue;
        }
        if (header_record[main_col] == 1 && fields_selected[main_col] < 1)
        {
            cout << setiosflags(ios::left) << setw(30) << main_names[main_col];
            cout << " : field not selected." << endl;
            continue;
        }
        if (header_record[main_col] == 1)
        {
            main_stats = train_set->getStats(main_col);
            main_total = main_stats.n();
            main_missing = main_stats.nmissing();
            main_present = main_total - main_missing;
            cout << setiosflags(ios::left) << setw(30) << main_names[main_col];
            cout << " : field deleted, only " << setw(6) << main_present << " records to train with." << endl;
            continue;
        }
        results_file_name = targeted_metadata + "/TreeCondMean/dir/" + main_names[main_col] + "/Split0/LearnerExpdir/Strat0results.pmat";
        if (!isfile(results_file_name))
        {
            header_file->put(0, main_col, 2.0);
            cout << setiosflags(ios::left) << setw(30) << main_names[main_col];
            cout << " : missing results file." << endl;
            continue;
        }
        test_output_file_name = targeted_metadata + "/TreeCondMean/dir/" + main_names[main_col] + "/Split0/test1_outputs.pmat";
        if (!isfile(test_output_file_name))
        {
            header_file->put(0, main_col, 2.0);
            cout << setiosflags(ios::left) << setw(30) << main_names[main_col];
            cout << " : missing test output file." << endl;
            continue;
        }
        results_file = new FileVMatrix(results_file_name);
        results_length = results_file->length();
        results_nstages = results_file->get(results_length - 1, 2);
        results_mse = results_file->get(results_length - 1, 6);
        results_std_err = results_file->get(results_length - 1, 7);
        test_output_file = new FileVMatrix(test_output_file_name);
        test_output_length = test_output_file->length();
        cout << setiosflags(ios::left) << setw(30) << main_names[main_col];
        cout << " : tree built with " << setw(2) << (int) results_nstages << " leaves, "
             << setw(6) << test_output_length << " test output records found, "
             << "performance: " << setiosflags(ios::fixed) << setprecision(4) << results_mse
             << " +/- " << setiosflags(ios::fixed) << setprecision(4) << results_std_err << endl;
    }
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::AnalyzeFieldStats::train ( ) [virtual]

*** SUBCLASS WRITING: ***

The role of the train method is to bring the learner up to stage==nstages, updating the stats with training costs measured on-line in the process.

TYPICAL CODE:

  static Vec input;  // static so we don't reallocate/deallocate memory each time...
  static Vec target; // (but be careful that static means shared!)
  input.resize(inputsize());    // the train_set's inputsize()
  target.resize(targetsize());  // the train_set's targetsize()
  real weight;
  
  if(!train_stats)   // make a default stats collector, in case there's none
      train_stats = new VecStatsCollector();
  
  if(nstages<stage)  // asking to revert to a previous stage!
      forget();      // reset the learner to stage=0
  
  while(stage<nstages)
  {
      // clear statistics of previous epoch
      train_stats->forget(); 
            
      //... train for 1 stage, and update train_stats,
      // using train_set->getSample(input, target, weight);
      // and train_stats->update(train_costs)
          
      ++stage;
      train_stats->finalize(); // finalize statistics for this epoch
  }

Implements PLearn::PLearner.

Definition at line 417 of file AnalyzeFieldStats.cc.

References cond_mean_template, PLearn::deepCopy(), output_file, targeted_metadata, to_deal_with_name, and train_test_file.

Referenced by build_().

{
    PP<ExplicitSplitter> explicit_splitter = new ExplicitSplitter();
    explicit_splitter->splitsets.resize(1,2);
    explicit_splitter->splitsets(0,0) = output_file;
    explicit_splitter->splitsets(0,1) = train_test_file;
    PP<PTester> cond_mean = ::PLearn::deepCopy(cond_mean_template);
    cond_mean->setOption("expdir", targeted_metadata + "/TreeCondMean/dir/" + to_deal_with_name);
    cond_mean->splitter = new ExplicitSplitter();
    cond_mean->splitter = explicit_splitter;
    cond_mean->build();
    Vec results = cond_mean->perform(true);
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::AnalyzeFieldStats::updateHeaderRecord ( int  var_col) [private]

Definition at line 354 of file AnalyzeFieldStats.cc.

References header_file.

Referenced by analyzeVariableStats().

{ 
    header_file->put(0, var_col, 3.0);
}

Here is the caller graph for this function:


Member Data Documentation

Reimplemented from PLearn::PLearner.

Definition at line 96 of file AnalyzeFieldStats.h.

The template of the script to learn the conditional mean.

Definition at line 72 of file AnalyzeFieldStats.h.

Referenced by declareOptions(), makeDeepCopyFromShallowCopy(), and train().

The field name of the variable to be analyzed.

Definition at line 74 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats(), declareOptions(), and makeDeepCopyFromShallowCopy().

Definition at line 145 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats().

Definition at line 146 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats().

Definition at line 142 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats(), createHeaderFile(), and getHeaderRecord().

Definition at line 152 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats().

Definition at line 131 of file AnalyzeFieldStats.h.

Definition at line 130 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats().

Definition at line 128 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats(), createHeaderFile(), and reviewGlobalStats().

The maximum number of samples used to train the learner.

Definition at line 68 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats(), declareOptions(), and makeDeepCopyFromShallowCopy().

### declare public option fields (such as build options) here Start your comments with Doxygen-compatible comments such as //!

The minimum number of samples required to train the learner.

Definition at line 66 of file AnalyzeFieldStats.h.

Referenced by createHeaderFile(), declareOptions(), getHeaderRecord(), and makeDeepCopyFromShallowCopy().

Definition at line 155 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats().

Definition at line 160 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats(), and train().

Definition at line 153 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats().

Definition at line 157 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats().

Definition at line 156 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats().

Definition at line 159 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats().

Definition at line 158 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats().

Definition at line 154 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats().

Definition at line 166 of file AnalyzeFieldStats.h.

Referenced by reviewGlobalStats().

Definition at line 165 of file AnalyzeFieldStats.h.

Referenced by reviewGlobalStats().

Definition at line 167 of file AnalyzeFieldStats.h.

Referenced by reviewGlobalStats().

Definition at line 169 of file AnalyzeFieldStats.h.

Referenced by reviewGlobalStats().

Definition at line 168 of file AnalyzeFieldStats.h.

Referenced by reviewGlobalStats().

Definition at line 170 of file AnalyzeFieldStats.h.

Referenced by reviewGlobalStats().

Definition at line 137 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats().

Definition at line 135 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats().

Definition at line 140 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats(), reviewGlobalStats(), and train().

Definition at line 141 of file AnalyzeFieldStats.h.

Referenced by createHeaderFile().

Definition at line 138 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats().

The train and test data sets with the target field.

Definition at line 70 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats(), createHeaderFile(), declareOptions(), and makeDeepCopyFromShallowCopy().

Definition at line 139 of file AnalyzeFieldStats.h.

Referenced by createHeaderFile().

Definition at line 136 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats().

Definition at line 172 of file AnalyzeFieldStats.h.

Referenced by reviewGlobalStats().

Definition at line 171 of file AnalyzeFieldStats.h.

Referenced by reviewGlobalStats().

Definition at line 173 of file AnalyzeFieldStats.h.

Referenced by reviewGlobalStats().

Definition at line 151 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats(), and train().

Definition at line 149 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats().

Definition at line 148 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats().

Definition at line 150 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats().

Definition at line 164 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats(), and train().

Definition at line 161 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats().

Definition at line 162 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats().

Definition at line 163 of file AnalyzeFieldStats.h.

Referenced by analyzeVariableStats().


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines