PLearn 0.1
Public Member Functions | Static Public Member Functions | Public Attributes | Static Public Attributes | Protected Member Functions | Static Protected Member Functions | Protected Attributes | Static Protected Attributes | Private Types | Private Member Functions
PLearn::InfiniteMNISTVMatrix Class Reference

VMatrix that uses the code from "Training Invariant Support Vector Machines using Selective Sampling" by Loosli, Canu and Bottou (JMLR 2007), to generate "infinite" stream (i.e. More...

#include <InfiniteMNISTVMatrix.h>

Inheritance diagram for PLearn::InfiniteMNISTVMatrix:
Inheritance graph
[legend]
Collaboration diagram for PLearn::InfiniteMNISTVMatrix:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 InfiniteMNISTVMatrix ()
 Default constructor.
 ~InfiniteMNISTVMatrix ()
virtual string classname () const
virtual OptionListgetOptionList () const
virtual OptionMapgetOptionMap () const
virtual RemoteMethodMapgetRemoteMethodMap () const
virtual InfiniteMNISTVMatrixdeepCopy (CopiesMap &copies) const
virtual void build ()
 Simply calls inherited::build() then build_().
virtual void makeDeepCopyFromShallowCopy (CopiesMap &copies)
 Transforms a shallow copy into a deep copy.

Static Public Member Functions

static string _classname_ ()
 RowBufferedVMatrix.
static OptionList_getOptionList_ ()
static RemoteMethodMap_getRemoteMethodMap_ ()
static Object_new_instance_for_typemap_ ()
static bool _isa_ (const Object *o)
static void _static_initialize_ ()
static const PPathdeclaringFile ()

Public Attributes

bool include_test_examples
 Indication that the test examples from the MNIST dataset should be included.
bool include_validation_examples
 Indication that the validation set examples (the last 10000 examples from the training set) should be included in this VMatrix.
bool random_switch_to_original_training_set
 Indication that the VMatrix should randomly (from time to time) provide an example from the original training set instead of an example from the global dataset.
real proportion_of_switches
 Proportion of switches to the original training set.
int seed
 Seed of random number generator.
real input_divisor
 Value that the inputs should be divided by.
PP< PRandomrandom_gen
 Random number generator.
string test_images
 File path of MNIST test images.
string test_labels
 File path of MNIST test labels.
string train_images
 File path of MNIST train images.
string train_labels
 File path of MNIST train labels.
string fields
 File path of MNIST fields information.
string tangent_vectors
 File path of MNIST transformation tangent vectors.

Static Public Attributes

static StaticInitializer _static_initializer_

Protected Member Functions

virtual void getNewRow (int i, const Vec &v) const
 Fill the vector 'v' with the content of the i-th row.

Static Protected Member Functions

static void declareOptions (OptionList &ol)
 Declares the class options.

Protected Attributes

unsigned char * image

Static Protected Attributes

static mnistproblem_t * dataset = 0
static int n_pointers_to_dataset = 0

Private Types

typedef RowBufferedVMatrix inherited

Private Member Functions

void build_ ()
 This does the actual building.

Detailed Description

VMatrix that uses the code from "Training Invariant Support Vector Machines using Selective Sampling" by Loosli, Canu and Bottou (JMLR 2007), to generate "infinite" stream (i.e.

INT_MAX sized set) of samples from the MNIST dataset. The samples are obtained by applying some class-invariante transformations on the original MNIST dataset.

Definition at line 64 of file InfiniteMNISTVMatrix.h.


Member Typedef Documentation

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 66 of file InfiniteMNISTVMatrix.h.


Constructor & Destructor Documentation

PLearn::InfiniteMNISTVMatrix::InfiniteMNISTVMatrix ( )
PLearn::InfiniteMNISTVMatrix::~InfiniteMNISTVMatrix ( )

Member Function Documentation

string PLearn::InfiniteMNISTVMatrix::_classname_ ( ) [static]

RowBufferedVMatrix.

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 58 of file InfiniteMNISTVMatrix.cc.

OptionList & PLearn::InfiniteMNISTVMatrix::_getOptionList_ ( ) [static]

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 58 of file InfiniteMNISTVMatrix.cc.

RemoteMethodMap & PLearn::InfiniteMNISTVMatrix::_getRemoteMethodMap_ ( ) [static]

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 58 of file InfiniteMNISTVMatrix.cc.

bool PLearn::InfiniteMNISTVMatrix::_isa_ ( const Object o) [static]

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 58 of file InfiniteMNISTVMatrix.cc.

Object * PLearn::InfiniteMNISTVMatrix::_new_instance_for_typemap_ ( ) [static]

Reimplemented from PLearn::Object.

Definition at line 58 of file InfiniteMNISTVMatrix.cc.

StaticInitializer InfiniteMNISTVMatrix::_static_initializer_ & PLearn::InfiniteMNISTVMatrix::_static_initialize_ ( ) [static]

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 58 of file InfiniteMNISTVMatrix.cc.

void PLearn::InfiniteMNISTVMatrix::build ( ) [virtual]

Simply calls inherited::build() then build_().

Reimplemented from PLearn::VMatrix.

Definition at line 243 of file InfiniteMNISTVMatrix.cc.

References PLearn::VMatrix::build(), and build_().

Here is the call graph for this function:

void PLearn::InfiniteMNISTVMatrix::build_ ( ) [private]

This does the actual building.

Reimplemented from PLearn::VMatrix.

Definition at line 183 of file InfiniteMNISTVMatrix.cc.

References dataset, PLearn::VMatrix::extrasize_, fields, include_test_examples, include_validation_examples, PLearn::VMatrix::inputsize_, PLearn::VMatrix::length_, PLERROR, random_gen, seed, tangent_vectors, PLearn::VMatrix::targetsize_, test_images, test_labels, train_images, train_labels, PLearn::VMatrix::updateMtime(), PLearn::VMatrix::weightsize_, and PLearn::VMatrix::width_.

Referenced by build().

{
    random_gen->manual_seed(seed);

    if( !InfiniteMNISTVMatrix::dataset )
    {
        char* test_images_char = new char[test_images.size()+1];
        char* test_labels_char = new char[test_labels.size()+1];
        char* train_images_char = new char[train_images.size()+1];
        char* train_labels_char = new char[train_labels.size()+1];
        char* fields_char = new char[fields.size()+1];
        char* tangent_vectors_char = new char[tangent_vectors.size()+1];
        
        strcpy(test_images_char,test_images.c_str());
        strcpy(test_labels_char,test_labels.c_str());
        strcpy(train_images_char,train_images.c_str());
        strcpy(train_labels_char,train_labels.c_str());
        strcpy(fields_char,fields.c_str());
        strcpy(tangent_vectors_char,tangent_vectors.c_str());
        
        InfiniteMNISTVMatrix::dataset = create_mnistproblem(
            test_images_char,
            test_labels_char,
            train_images_char,
            train_labels_char,
            fields_char,
            tangent_vectors_char);
        if( !InfiniteMNISTVMatrix::dataset )
            PLERROR("In InfiniteMNISTVMatrix(): could not load MNIST dataset");
    }


    if( include_test_examples )
        if( include_validation_examples )
            length_ = INT_MAX;
        else
            // Might be removing more samples than need, but we have so many anyways...
            length_ = INT_MAX - ((INT_MAX-10000)/50000)*10000 + 1;
    
    else
        if( include_validation_examples )
            length_ = INT_MAX - 10000+1;
        else
            // Might be removing more samples than need, but we have so many anyways...
            length_ = INT_MAX - ((INT_MAX-10000)/50000)*10000 - 10000 + 1;

    inputsize_ = 784;
    targetsize_ = 1;
    weightsize_ = 0;
    extrasize_ = 0;
    width_ = 785;

    // ### You should keep the line 'updateMtime(0);' if you don't implement the 
    // ### update of the mtime. Otherwise you can have an mtime != 0 that is not valid.
    updateMtime(0);
    //updateMtime(filename);
    //updateMtime(VMat);
}

Here is the call graph for this function:

Here is the caller graph for this function:

string PLearn::InfiniteMNISTVMatrix::classname ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 58 of file InfiniteMNISTVMatrix.cc.

void PLearn::InfiniteMNISTVMatrix::declareOptions ( OptionList ol) [static, protected]

Declares the class options.

Reimplemented from PLearn::VMatrix.

Definition at line 122 of file InfiniteMNISTVMatrix.cc.

References PLearn::OptionBase::buildoption, PLearn::declareOption(), PLearn::VMatrix::declareOptions(), fields, include_test_examples, include_validation_examples, input_divisor, proportion_of_switches, random_switch_to_original_training_set, seed, tangent_vectors, test_images, test_labels, train_images, and train_labels.

{
     declareOption(ol, "include_test_examples", &InfiniteMNISTVMatrix::include_test_examples,
                   OptionBase::buildoption,
                   "Indication that the test examples from the MNIST dataset should be included.\n"
                   "This option is false by default. If true, these examples will be the first"
                   "10000\n"
                   "of this VMatrix.\n");

     declareOption(ol, "include_validation_examples", &InfiniteMNISTVMatrix::include_validation_examples,
                   OptionBase::buildoption,
                   "Indication that the validation set examples (the last 10000 examples from the\n"
                   "training set) should be included in this VMatrix.\n");     

     declareOption(ol, "random_switch_to_original_training_set", 
                   &InfiniteMNISTVMatrix::random_switch_to_original_training_set,
                   OptionBase::buildoption,
                   "Indication that the VMatrix should randomly (from time to time) provide\n"
                   "an example from the original training set instead of an example\n"
                   "from the global dataset.\n");     

     declareOption(ol, "proportion_of_switches", &InfiniteMNISTVMatrix::proportion_of_switches,
                   OptionBase::buildoption,
                   "Proportion of switches to the original training set.\n");     

     declareOption(ol, "seed", &InfiniteMNISTVMatrix::seed,
                   OptionBase::buildoption,
                   "Seed of random number generator.\n");

     declareOption(ol, "input_divisor", &InfiniteMNISTVMatrix::input_divisor,
                   OptionBase::buildoption,
                   "Value that the inputs should be divided by.\n");     

     declareOption(ol, "test_images", &InfiniteMNISTVMatrix::test_images,
                   OptionBase::buildoption,
                   "File path of MNIST test images.\n");     

     declareOption(ol, "test_labels", &InfiniteMNISTVMatrix::test_labels,
                   OptionBase::buildoption,
                   "File path of MNIST test labels.\n");     

     declareOption(ol, "train_images", &InfiniteMNISTVMatrix::train_images,
                   OptionBase::buildoption,
                   "File path of MNIST train images.\n");     

     declareOption(ol, "train_labels", &InfiniteMNISTVMatrix::train_labels,
                   OptionBase::buildoption,
                   "File path of MNIST train labels.\n");     

     declareOption(ol, "fields", &InfiniteMNISTVMatrix::fields,
                   OptionBase::buildoption,
                   "File path of MNIST fields information.\n");     

     declareOption(ol, "tangent_vectors", &InfiniteMNISTVMatrix::tangent_vectors,
                   OptionBase::buildoption,
                   "File paht of MNIST transformation tangent vectors.\n");

    // Now call the parent class' declareOptions
    inherited::declareOptions(ol);
}

Here is the call graph for this function:

static const PPath& PLearn::InfiniteMNISTVMatrix::declaringFile ( ) [inline, static]

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 126 of file InfiniteMNISTVMatrix.h.

:
    //#####  Protected Options  ###############################################
InfiniteMNISTVMatrix * PLearn::InfiniteMNISTVMatrix::deepCopy ( CopiesMap copies) const [virtual]

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 58 of file InfiniteMNISTVMatrix.cc.

void PLearn::InfiniteMNISTVMatrix::getNewRow ( int  i,
const Vec v 
) const [protected, virtual]

Fill the vector 'v' with the content of the i-th row.

'v' is assumed to be the right size.

Implements PLearn::RowBufferedVMatrix.

Definition at line 91 of file InfiniteMNISTVMatrix.cc.

References PLearn::TVec< T >::data(), dataset, i, image, include_test_examples, include_validation_examples, input_divisor, PLearn::VMatrix::inputsize_, j, PLearn::TVec< T >::last(), proportion_of_switches, random_gen, and random_switch_to_original_training_set.

{
    int i_dataset;
    if( include_test_examples )
        if( include_validation_examples )
            i_dataset = i;
        else
            if( i < 10000)
                i_dataset = i;
            else
                i_dataset = i + ((i-10000)/50000)*10000;
    else
        if( include_validation_examples )
            i_dataset = i+10000;
        else
            i_dataset = i + (i/50000)*10000 + 10000;

    if( random_switch_to_original_training_set && 
        random_gen->uniform_sample() < proportion_of_switches )
        i_dataset = (i_dataset % 50000)+10000;

    image = compute_transformed_vector_in_place(InfiniteMNISTVMatrix::dataset, i_dataset, image);

    unsigned char* xj=image;
    real* vj=v.data();
    for( int j=0; j<inputsize_; j++, xj++, vj++ )
        *vj = *xj/input_divisor;
    
    v.last() = InfiniteMNISTVMatrix::dataset->y[ (i_dataset<10000) ? i_dataset : 10000 + ((i_dataset - 10000) % 60000) ];
}

Here is the call graph for this function:

OptionList & PLearn::InfiniteMNISTVMatrix::getOptionList ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 58 of file InfiniteMNISTVMatrix.cc.

OptionMap & PLearn::InfiniteMNISTVMatrix::getOptionMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 58 of file InfiniteMNISTVMatrix.cc.

RemoteMethodMap & PLearn::InfiniteMNISTVMatrix::getRemoteMethodMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 58 of file InfiniteMNISTVMatrix.cc.

void PLearn::InfiniteMNISTVMatrix::makeDeepCopyFromShallowCopy ( CopiesMap copies) [virtual]

Transforms a shallow copy into a deep copy.

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 249 of file InfiniteMNISTVMatrix.cc.

References image, PLearn::RowBufferedVMatrix::makeDeepCopyFromShallowCopy(), n_pointers_to_dataset, and PLWARNING.

{
    inherited::makeDeepCopyFromShallowCopy(copies);

    // ### Call deepCopyField on all "pointer-like" fields
    // ### that you wish to be deepCopied rather than
    // ### shallow-copied.
    // ### ex:
    // deepCopyField(trainvec, copies);

    PLWARNING("InfiniteMNISTVMatrix::makeDeepCopyFromShallowCopy is not totally implemented. Need "
              "to figure out how to deep copy the \"dataset\" variable (mnistproblem_t*).\n");
    InfiniteMNISTVMatrix::n_pointers_to_dataset++;
    image = (unsigned char*)malloc(EXSIZE);
}

Here is the call graph for this function:


Member Data Documentation

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 126 of file InfiniteMNISTVMatrix.h.

mnistproblem_t * PLearn::InfiniteMNISTVMatrix::dataset = 0 [static, protected]

Definition at line 138 of file InfiniteMNISTVMatrix.h.

Referenced by build_(), getNewRow(), and ~InfiniteMNISTVMatrix().

File path of MNIST fields information.

Definition at line 107 of file InfiniteMNISTVMatrix.h.

Referenced by build_(), and declareOptions().

unsigned char* PLearn::InfiniteMNISTVMatrix::image [mutable, protected]

Indication that the test examples from the MNIST dataset should be included.

This option is false by default. If true, these examples will be the first 10000 of this VMatrix.

Definition at line 74 of file InfiniteMNISTVMatrix.h.

Referenced by build_(), declareOptions(), and getNewRow().

Indication that the validation set examples (the last 10000 examples from the training set) should be included in this VMatrix.

Definition at line 78 of file InfiniteMNISTVMatrix.h.

Referenced by build_(), declareOptions(), and getNewRow().

Value that the inputs should be divided by.

Definition at line 92 of file InfiniteMNISTVMatrix.h.

Referenced by declareOptions(), and getNewRow().

Proportion of switches to the original training set.

Definition at line 86 of file InfiniteMNISTVMatrix.h.

Referenced by declareOptions(), and getNewRow().

Random number generator.

Definition at line 95 of file InfiniteMNISTVMatrix.h.

Referenced by build_(), getNewRow(), and InfiniteMNISTVMatrix().

Indication that the VMatrix should randomly (from time to time) provide an example from the original training set instead of an example from the global dataset.

Definition at line 83 of file InfiniteMNISTVMatrix.h.

Referenced by declareOptions(), and getNewRow().

Seed of random number generator.

Definition at line 89 of file InfiniteMNISTVMatrix.h.

Referenced by build_(), and declareOptions().

File path of MNIST transformation tangent vectors.

Definition at line 109 of file InfiniteMNISTVMatrix.h.

Referenced by build_(), and declareOptions().

File path of MNIST test images.

Definition at line 99 of file InfiniteMNISTVMatrix.h.

Referenced by build_(), and declareOptions().

File path of MNIST test labels.

Definition at line 101 of file InfiniteMNISTVMatrix.h.

Referenced by build_(), and declareOptions().

File path of MNIST train images.

Definition at line 103 of file InfiniteMNISTVMatrix.h.

Referenced by build_(), and declareOptions().

File path of MNIST train labels.

Definition at line 105 of file InfiniteMNISTVMatrix.h.

Referenced by build_(), and declareOptions().


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines