PLearn 0.1
|
VMatrix that uses the code from "Training Invariant Support Vector Machines using Selective Sampling" by Loosli, Canu and Bottou (JMLR 2007), to generate "infinite" stream (i.e. More...
#include <InfiniteMNISTVMatrix.h>
Public Member Functions | |
InfiniteMNISTVMatrix () | |
Default constructor. | |
~InfiniteMNISTVMatrix () | |
virtual string | classname () const |
virtual OptionList & | getOptionList () const |
virtual OptionMap & | getOptionMap () const |
virtual RemoteMethodMap & | getRemoteMethodMap () const |
virtual InfiniteMNISTVMatrix * | deepCopy (CopiesMap &copies) const |
virtual void | build () |
Simply calls inherited::build() then build_(). | |
virtual void | makeDeepCopyFromShallowCopy (CopiesMap &copies) |
Transforms a shallow copy into a deep copy. | |
Static Public Member Functions | |
static string | _classname_ () |
RowBufferedVMatrix. | |
static OptionList & | _getOptionList_ () |
static RemoteMethodMap & | _getRemoteMethodMap_ () |
static Object * | _new_instance_for_typemap_ () |
static bool | _isa_ (const Object *o) |
static void | _static_initialize_ () |
static const PPath & | declaringFile () |
Public Attributes | |
bool | include_test_examples |
Indication that the test examples from the MNIST dataset should be included. | |
bool | include_validation_examples |
Indication that the validation set examples (the last 10000 examples from the training set) should be included in this VMatrix. | |
bool | random_switch_to_original_training_set |
Indication that the VMatrix should randomly (from time to time) provide an example from the original training set instead of an example from the global dataset. | |
real | proportion_of_switches |
Proportion of switches to the original training set. | |
int | seed |
Seed of random number generator. | |
real | input_divisor |
Value that the inputs should be divided by. | |
PP< PRandom > | random_gen |
Random number generator. | |
string | test_images |
File path of MNIST test images. | |
string | test_labels |
File path of MNIST test labels. | |
string | train_images |
File path of MNIST train images. | |
string | train_labels |
File path of MNIST train labels. | |
string | fields |
File path of MNIST fields information. | |
string | tangent_vectors |
File path of MNIST transformation tangent vectors. | |
Static Public Attributes | |
static StaticInitializer | _static_initializer_ |
Protected Member Functions | |
virtual void | getNewRow (int i, const Vec &v) const |
Fill the vector 'v' with the content of the i-th row. | |
Static Protected Member Functions | |
static void | declareOptions (OptionList &ol) |
Declares the class options. | |
Protected Attributes | |
unsigned char * | image |
Static Protected Attributes | |
static mnistproblem_t * | dataset = 0 |
static int | n_pointers_to_dataset = 0 |
Private Types | |
typedef RowBufferedVMatrix | inherited |
Private Member Functions | |
void | build_ () |
This does the actual building. |
VMatrix that uses the code from "Training Invariant Support Vector Machines using Selective Sampling" by Loosli, Canu and Bottou (JMLR 2007), to generate "infinite" stream (i.e.
INT_MAX sized set) of samples from the MNIST dataset. The samples are obtained by applying some class-invariante transformations on the original MNIST dataset.
Definition at line 64 of file InfiniteMNISTVMatrix.h.
typedef RowBufferedVMatrix PLearn::InfiniteMNISTVMatrix::inherited [private] |
Reimplemented from PLearn::RowBufferedVMatrix.
Definition at line 66 of file InfiniteMNISTVMatrix.h.
PLearn::InfiniteMNISTVMatrix::InfiniteMNISTVMatrix | ( | ) |
Default constructor.
Definition at line 60 of file InfiniteMNISTVMatrix.cc.
References image, n_pointers_to_dataset, and random_gen.
: include_test_examples(false), include_validation_examples(false), random_switch_to_original_training_set(false), proportion_of_switches(0.0), seed(1834), input_divisor(1.), test_images(TEST_IMAGES_PATH), test_labels(TEST_LABELS_PATH), train_images(TRAIN_IMAGES_PATH), train_labels(TRAIN_LABELS_PATH), fields(FIELDS_PATH), tangent_vectors(TANGVEC_PATH) /* ### Initialize all fields to their default value */ { InfiniteMNISTVMatrix::n_pointers_to_dataset++; random_gen = new PRandom(); image = (unsigned char*)malloc(EXSIZE); }
PLearn::InfiniteMNISTVMatrix::~InfiniteMNISTVMatrix | ( | ) |
Definition at line 80 of file InfiniteMNISTVMatrix.cc.
References dataset, image, and n_pointers_to_dataset.
{ InfiniteMNISTVMatrix::n_pointers_to_dataset--; if( InfiniteMNISTVMatrix::dataset && InfiniteMNISTVMatrix::n_pointers_to_dataset == 0 ) { destroy_mnistproblem(dataset); InfiniteMNISTVMatrix::dataset = 0; } free( image ); }
string PLearn::InfiniteMNISTVMatrix::_classname_ | ( | ) | [static] |
Reimplemented from PLearn::RowBufferedVMatrix.
Definition at line 58 of file InfiniteMNISTVMatrix.cc.
OptionList & PLearn::InfiniteMNISTVMatrix::_getOptionList_ | ( | ) | [static] |
Reimplemented from PLearn::RowBufferedVMatrix.
Definition at line 58 of file InfiniteMNISTVMatrix.cc.
RemoteMethodMap & PLearn::InfiniteMNISTVMatrix::_getRemoteMethodMap_ | ( | ) | [static] |
Reimplemented from PLearn::RowBufferedVMatrix.
Definition at line 58 of file InfiniteMNISTVMatrix.cc.
Reimplemented from PLearn::RowBufferedVMatrix.
Definition at line 58 of file InfiniteMNISTVMatrix.cc.
Object * PLearn::InfiniteMNISTVMatrix::_new_instance_for_typemap_ | ( | ) | [static] |
Reimplemented from PLearn::Object.
Definition at line 58 of file InfiniteMNISTVMatrix.cc.
StaticInitializer InfiniteMNISTVMatrix::_static_initializer_ & PLearn::InfiniteMNISTVMatrix::_static_initialize_ | ( | ) | [static] |
Reimplemented from PLearn::RowBufferedVMatrix.
Definition at line 58 of file InfiniteMNISTVMatrix.cc.
void PLearn::InfiniteMNISTVMatrix::build | ( | ) | [virtual] |
Simply calls inherited::build() then build_().
Reimplemented from PLearn::VMatrix.
Definition at line 243 of file InfiniteMNISTVMatrix.cc.
References PLearn::VMatrix::build(), and build_().
{ inherited::build(); build_(); }
void PLearn::InfiniteMNISTVMatrix::build_ | ( | ) | [private] |
This does the actual building.
Reimplemented from PLearn::VMatrix.
Definition at line 183 of file InfiniteMNISTVMatrix.cc.
References dataset, PLearn::VMatrix::extrasize_, fields, include_test_examples, include_validation_examples, PLearn::VMatrix::inputsize_, PLearn::VMatrix::length_, PLERROR, random_gen, seed, tangent_vectors, PLearn::VMatrix::targetsize_, test_images, test_labels, train_images, train_labels, PLearn::VMatrix::updateMtime(), PLearn::VMatrix::weightsize_, and PLearn::VMatrix::width_.
Referenced by build().
{ random_gen->manual_seed(seed); if( !InfiniteMNISTVMatrix::dataset ) { char* test_images_char = new char[test_images.size()+1]; char* test_labels_char = new char[test_labels.size()+1]; char* train_images_char = new char[train_images.size()+1]; char* train_labels_char = new char[train_labels.size()+1]; char* fields_char = new char[fields.size()+1]; char* tangent_vectors_char = new char[tangent_vectors.size()+1]; strcpy(test_images_char,test_images.c_str()); strcpy(test_labels_char,test_labels.c_str()); strcpy(train_images_char,train_images.c_str()); strcpy(train_labels_char,train_labels.c_str()); strcpy(fields_char,fields.c_str()); strcpy(tangent_vectors_char,tangent_vectors.c_str()); InfiniteMNISTVMatrix::dataset = create_mnistproblem( test_images_char, test_labels_char, train_images_char, train_labels_char, fields_char, tangent_vectors_char); if( !InfiniteMNISTVMatrix::dataset ) PLERROR("In InfiniteMNISTVMatrix(): could not load MNIST dataset"); } if( include_test_examples ) if( include_validation_examples ) length_ = INT_MAX; else // Might be removing more samples than need, but we have so many anyways... length_ = INT_MAX - ((INT_MAX-10000)/50000)*10000 + 1; else if( include_validation_examples ) length_ = INT_MAX - 10000+1; else // Might be removing more samples than need, but we have so many anyways... length_ = INT_MAX - ((INT_MAX-10000)/50000)*10000 - 10000 + 1; inputsize_ = 784; targetsize_ = 1; weightsize_ = 0; extrasize_ = 0; width_ = 785; // ### You should keep the line 'updateMtime(0);' if you don't implement the // ### update of the mtime. Otherwise you can have an mtime != 0 that is not valid. updateMtime(0); //updateMtime(filename); //updateMtime(VMat); }
string PLearn::InfiniteMNISTVMatrix::classname | ( | ) | const [virtual] |
Reimplemented from PLearn::Object.
Definition at line 58 of file InfiniteMNISTVMatrix.cc.
void PLearn::InfiniteMNISTVMatrix::declareOptions | ( | OptionList & | ol | ) | [static, protected] |
Declares the class options.
Reimplemented from PLearn::VMatrix.
Definition at line 122 of file InfiniteMNISTVMatrix.cc.
References PLearn::OptionBase::buildoption, PLearn::declareOption(), PLearn::VMatrix::declareOptions(), fields, include_test_examples, include_validation_examples, input_divisor, proportion_of_switches, random_switch_to_original_training_set, seed, tangent_vectors, test_images, test_labels, train_images, and train_labels.
{ declareOption(ol, "include_test_examples", &InfiniteMNISTVMatrix::include_test_examples, OptionBase::buildoption, "Indication that the test examples from the MNIST dataset should be included.\n" "This option is false by default. If true, these examples will be the first" "10000\n" "of this VMatrix.\n"); declareOption(ol, "include_validation_examples", &InfiniteMNISTVMatrix::include_validation_examples, OptionBase::buildoption, "Indication that the validation set examples (the last 10000 examples from the\n" "training set) should be included in this VMatrix.\n"); declareOption(ol, "random_switch_to_original_training_set", &InfiniteMNISTVMatrix::random_switch_to_original_training_set, OptionBase::buildoption, "Indication that the VMatrix should randomly (from time to time) provide\n" "an example from the original training set instead of an example\n" "from the global dataset.\n"); declareOption(ol, "proportion_of_switches", &InfiniteMNISTVMatrix::proportion_of_switches, OptionBase::buildoption, "Proportion of switches to the original training set.\n"); declareOption(ol, "seed", &InfiniteMNISTVMatrix::seed, OptionBase::buildoption, "Seed of random number generator.\n"); declareOption(ol, "input_divisor", &InfiniteMNISTVMatrix::input_divisor, OptionBase::buildoption, "Value that the inputs should be divided by.\n"); declareOption(ol, "test_images", &InfiniteMNISTVMatrix::test_images, OptionBase::buildoption, "File path of MNIST test images.\n"); declareOption(ol, "test_labels", &InfiniteMNISTVMatrix::test_labels, OptionBase::buildoption, "File path of MNIST test labels.\n"); declareOption(ol, "train_images", &InfiniteMNISTVMatrix::train_images, OptionBase::buildoption, "File path of MNIST train images.\n"); declareOption(ol, "train_labels", &InfiniteMNISTVMatrix::train_labels, OptionBase::buildoption, "File path of MNIST train labels.\n"); declareOption(ol, "fields", &InfiniteMNISTVMatrix::fields, OptionBase::buildoption, "File path of MNIST fields information.\n"); declareOption(ol, "tangent_vectors", &InfiniteMNISTVMatrix::tangent_vectors, OptionBase::buildoption, "File paht of MNIST transformation tangent vectors.\n"); // Now call the parent class' declareOptions inherited::declareOptions(ol); }
static const PPath& PLearn::InfiniteMNISTVMatrix::declaringFile | ( | ) | [inline, static] |
Reimplemented from PLearn::RowBufferedVMatrix.
Definition at line 126 of file InfiniteMNISTVMatrix.h.
:
//##### Protected Options ###############################################
InfiniteMNISTVMatrix * PLearn::InfiniteMNISTVMatrix::deepCopy | ( | CopiesMap & | copies | ) | const [virtual] |
Reimplemented from PLearn::RowBufferedVMatrix.
Definition at line 58 of file InfiniteMNISTVMatrix.cc.
Fill the vector 'v' with the content of the i-th row.
'v' is assumed to be the right size.
Implements PLearn::RowBufferedVMatrix.
Definition at line 91 of file InfiniteMNISTVMatrix.cc.
References PLearn::TVec< T >::data(), dataset, i, image, include_test_examples, include_validation_examples, input_divisor, PLearn::VMatrix::inputsize_, j, PLearn::TVec< T >::last(), proportion_of_switches, random_gen, and random_switch_to_original_training_set.
{ int i_dataset; if( include_test_examples ) if( include_validation_examples ) i_dataset = i; else if( i < 10000) i_dataset = i; else i_dataset = i + ((i-10000)/50000)*10000; else if( include_validation_examples ) i_dataset = i+10000; else i_dataset = i + (i/50000)*10000 + 10000; if( random_switch_to_original_training_set && random_gen->uniform_sample() < proportion_of_switches ) i_dataset = (i_dataset % 50000)+10000; image = compute_transformed_vector_in_place(InfiniteMNISTVMatrix::dataset, i_dataset, image); unsigned char* xj=image; real* vj=v.data(); for( int j=0; j<inputsize_; j++, xj++, vj++ ) *vj = *xj/input_divisor; v.last() = InfiniteMNISTVMatrix::dataset->y[ (i_dataset<10000) ? i_dataset : 10000 + ((i_dataset - 10000) % 60000) ]; }
OptionList & PLearn::InfiniteMNISTVMatrix::getOptionList | ( | ) | const [virtual] |
Reimplemented from PLearn::Object.
Definition at line 58 of file InfiniteMNISTVMatrix.cc.
OptionMap & PLearn::InfiniteMNISTVMatrix::getOptionMap | ( | ) | const [virtual] |
Reimplemented from PLearn::Object.
Definition at line 58 of file InfiniteMNISTVMatrix.cc.
RemoteMethodMap & PLearn::InfiniteMNISTVMatrix::getRemoteMethodMap | ( | ) | const [virtual] |
Reimplemented from PLearn::Object.
Definition at line 58 of file InfiniteMNISTVMatrix.cc.
void PLearn::InfiniteMNISTVMatrix::makeDeepCopyFromShallowCopy | ( | CopiesMap & | copies | ) | [virtual] |
Transforms a shallow copy into a deep copy.
Reimplemented from PLearn::RowBufferedVMatrix.
Definition at line 249 of file InfiniteMNISTVMatrix.cc.
References image, PLearn::RowBufferedVMatrix::makeDeepCopyFromShallowCopy(), n_pointers_to_dataset, and PLWARNING.
{ inherited::makeDeepCopyFromShallowCopy(copies); // ### Call deepCopyField on all "pointer-like" fields // ### that you wish to be deepCopied rather than // ### shallow-copied. // ### ex: // deepCopyField(trainvec, copies); PLWARNING("InfiniteMNISTVMatrix::makeDeepCopyFromShallowCopy is not totally implemented. Need " "to figure out how to deep copy the \"dataset\" variable (mnistproblem_t*).\n"); InfiniteMNISTVMatrix::n_pointers_to_dataset++; image = (unsigned char*)malloc(EXSIZE); }
Reimplemented from PLearn::RowBufferedVMatrix.
Definition at line 126 of file InfiniteMNISTVMatrix.h.
mnistproblem_t * PLearn::InfiniteMNISTVMatrix::dataset = 0 [static, protected] |
Definition at line 138 of file InfiniteMNISTVMatrix.h.
Referenced by build_(), getNewRow(), and ~InfiniteMNISTVMatrix().
File path of MNIST fields information.
Definition at line 107 of file InfiniteMNISTVMatrix.h.
Referenced by build_(), and declareOptions().
unsigned char* PLearn::InfiniteMNISTVMatrix::image [mutable, protected] |
Definition at line 140 of file InfiniteMNISTVMatrix.h.
Referenced by getNewRow(), InfiniteMNISTVMatrix(), makeDeepCopyFromShallowCopy(), and ~InfiniteMNISTVMatrix().
Indication that the test examples from the MNIST dataset should be included.
This option is false by default. If true, these examples will be the first 10000 of this VMatrix.
Definition at line 74 of file InfiniteMNISTVMatrix.h.
Referenced by build_(), declareOptions(), and getNewRow().
Indication that the validation set examples (the last 10000 examples from the training set) should be included in this VMatrix.
Definition at line 78 of file InfiniteMNISTVMatrix.h.
Referenced by build_(), declareOptions(), and getNewRow().
Value that the inputs should be divided by.
Definition at line 92 of file InfiniteMNISTVMatrix.h.
Referenced by declareOptions(), and getNewRow().
int PLearn::InfiniteMNISTVMatrix::n_pointers_to_dataset = 0 [static, protected] |
Definition at line 139 of file InfiniteMNISTVMatrix.h.
Referenced by InfiniteMNISTVMatrix(), makeDeepCopyFromShallowCopy(), and ~InfiniteMNISTVMatrix().
Proportion of switches to the original training set.
Definition at line 86 of file InfiniteMNISTVMatrix.h.
Referenced by declareOptions(), and getNewRow().
Random number generator.
Definition at line 95 of file InfiniteMNISTVMatrix.h.
Referenced by build_(), getNewRow(), and InfiniteMNISTVMatrix().
Indication that the VMatrix should randomly (from time to time) provide an example from the original training set instead of an example from the global dataset.
Definition at line 83 of file InfiniteMNISTVMatrix.h.
Referenced by declareOptions(), and getNewRow().
Seed of random number generator.
Definition at line 89 of file InfiniteMNISTVMatrix.h.
Referenced by build_(), and declareOptions().
File path of MNIST transformation tangent vectors.
Definition at line 109 of file InfiniteMNISTVMatrix.h.
Referenced by build_(), and declareOptions().
File path of MNIST test images.
Definition at line 99 of file InfiniteMNISTVMatrix.h.
Referenced by build_(), and declareOptions().
File path of MNIST test labels.
Definition at line 101 of file InfiniteMNISTVMatrix.h.
Referenced by build_(), and declareOptions().
File path of MNIST train images.
Definition at line 103 of file InfiniteMNISTVMatrix.h.
Referenced by build_(), and declareOptions().
File path of MNIST train labels.
Definition at line 105 of file InfiniteMNISTVMatrix.h.
Referenced by build_(), and declareOptions().