PLearn 0.1
Public Member Functions | Static Public Member Functions | Public Attributes | Static Public Attributes | Protected Member Functions | Static Protected Member Functions | Private Types | Private Member Functions | Static Private Attributes
PLearn::LIBSVMSparseVMatrix Class Reference

VMatrix containing data from a libsvm format file. More...

#include <LIBSVMSparseVMatrix.h>

Inheritance diagram for PLearn::LIBSVMSparseVMatrix:
Inheritance graph
[legend]
Collaboration diagram for PLearn::LIBSVMSparseVMatrix:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 LIBSVMSparseVMatrix ()
 Default constructor.
 LIBSVMSparseVMatrix (PPath filename, bool use_coarse_representation)
virtual string classname () const
virtual OptionListgetOptionList () const
virtual OptionMapgetOptionMap () const
virtual RemoteMethodMapgetRemoteMethodMap () const
virtual LIBSVMSparseVMatrixdeepCopy (CopiesMap &copies) const
virtual void build ()
 Simply calls inherited::build() then build_().
virtual void makeDeepCopyFromShallowCopy (CopiesMap &copies)
 Transforms a shallow copy into a deep copy.
virtual void getExample (int i, Vec &input, Vec &target, real &weight)
 Default version calls getSubRow based on inputsize_ targetsize_ weightsize_ But exotic subclasses may construct, input, target and weight however they please.
void getExamples (int i_start, int length, Mat &inputs, Mat &targets, Vec &weights, Mat *extra=NULL, bool allow_circular=false)
 Obtain a subset of 'length' examples, starting from 'i_start'.
virtual void getExtra (int i, Vec &extra)
 Complements the getExample method, fetching the the extrasize_ "extra" fields expected to appear after the input, target and weight fields Default version calls getSubRow based on inputsize_ targetsize_ weightsize_ and extrasize_.

Static Public Member Functions

static string _classname_ ()
 RowBufferedVMatrix.
static OptionList_getOptionList_ ()
static RemoteMethodMap_getRemoteMethodMap_ ()
static Object_new_instance_for_typemap_ ()
static bool _isa_ (const Object *o)
static void _static_initialize_ ()
static const PPathdeclaringFile ()
static VMat instantiateFromPPath (const PPath &filename)

Public Attributes

TVec< string > class_strings
 Strings associated to the different classes.
PPath libsvm_file
 File name of libsvm data.
bool use_coarse_representation
 Indication that a coarse (i.e.
TVec< Veclibsvm_input_data
 Input data.
TVec< Veclibsvm_extra_data
 Index of non-zero inputs.
Vec libsvm_target_data
 Target data.

Static Public Attributes

static StaticInitializer _static_initializer_

Protected Member Functions

virtual void getNewRow (int i, const Vec &v) const
 Fill the vector 'v' with the content of the i-th row.

Static Protected Member Functions

static void declareOptions (OptionList &ol)
 Declares the class options.

Private Types

typedef RowBufferedVMatrix inherited

Private Member Functions

void build_ ()
 This does the actual building.

Static Private Attributes

static VMatrixExtensionRegistrarextension_registrar

Detailed Description

VMatrix containing data from a libsvm format file.

Definition at line 52 of file LIBSVMSparseVMatrix.h.


Member Typedef Documentation

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 54 of file LIBSVMSparseVMatrix.h.


Constructor & Destructor Documentation

PLearn::LIBSVMSparseVMatrix::LIBSVMSparseVMatrix ( )

Default constructor.

Definition at line 65 of file LIBSVMSparseVMatrix.cc.

Referenced by instantiateFromPPath().

Here is the caller graph for this function:

PLearn::LIBSVMSparseVMatrix::LIBSVMSparseVMatrix ( PPath  filename,
bool  use_coarse_representation 
)

Definition at line 69 of file LIBSVMSparseVMatrix.cc.

References build().

                                                                                      :
    libsvm_file(filename),
    use_coarse_representation(use_coarse_representation)
{
    build();
}

Here is the call graph for this function:


Member Function Documentation

string PLearn::LIBSVMSparseVMatrix::_classname_ ( ) [static]

RowBufferedVMatrix.

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 63 of file LIBSVMSparseVMatrix.cc.

OptionList & PLearn::LIBSVMSparseVMatrix::_getOptionList_ ( ) [static]

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 63 of file LIBSVMSparseVMatrix.cc.

RemoteMethodMap & PLearn::LIBSVMSparseVMatrix::_getRemoteMethodMap_ ( ) [static]

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 63 of file LIBSVMSparseVMatrix.cc.

bool PLearn::LIBSVMSparseVMatrix::_isa_ ( const Object o) [static]

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 63 of file LIBSVMSparseVMatrix.cc.

Object * PLearn::LIBSVMSparseVMatrix::_new_instance_for_typemap_ ( ) [static]

Reimplemented from PLearn::Object.

Definition at line 63 of file LIBSVMSparseVMatrix.cc.

StaticInitializer LIBSVMSparseVMatrix::_static_initializer_ & PLearn::LIBSVMSparseVMatrix::_static_initialize_ ( ) [static]

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 63 of file LIBSVMSparseVMatrix.cc.

void PLearn::LIBSVMSparseVMatrix::build ( ) [virtual]

Simply calls inherited::build() then build_().

Reimplemented from PLearn::VMatrix.

Definition at line 196 of file LIBSVMSparseVMatrix.cc.

References PLearn::VMatrix::build(), and build_().

Referenced by LIBSVMSparseVMatrix().

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::LIBSVMSparseVMatrix::build_ ( ) [private]

This does the actual building.

Reimplemented from PLearn::VMatrix.

Definition at line 123 of file LIBSVMSparseVMatrix.cc.

References class_strings, d, PLearn::PStream::eof(), PLearn::VMatrix::extrasize_, PLearn::TVec< T >::find(), PLearn::PStream::getline(), i, PLearn::VMatrix::inputsize_, PLearn::PPath::isEmpty(), PLearn::VMatrix::length_, libsvm_extra_data, libsvm_file, libsvm_input_data, libsvm_target_data, PLearn::openFile(), PLearn::pl_isnumber(), PLERROR, PLearn::TVec< T >::push_back(), PLearn::PStream::raw_ascii, PLearn::removeblanks(), PLearn::TVec< T >::resize(), PLearn::PStream::skipBlanks(), PLearn::split(), PLearn::VMatrix::targetsize_, PLearn::toint(), PLearn::VMatrix::updateMtime(), use_coarse_representation, PLearn::VMatrix::weightsize_, and PLearn::VMatrix::width_.

Referenced by build().

{

    if(libsvm_file.isEmpty())
        return;

    // Read data
    PStream libsvm_stream = openFile(libsvm_file, PStream::raw_ascii);
    updateMtime(libsvm_file);
    libsvm_stream.skipBlanks();
    int input_index = 0;
    int largest_input_index = -1;
    int target_index = 0;
    int n_inputs = 0;
    string line;
    vector<string> tokens; 
    length_ = 0;
    width_ = -1;
    libsvm_input_data.resize(0);
    libsvm_extra_data.resize(0);
    libsvm_target_data.resize(0);
    while(!libsvm_stream.eof())
    {
        libsvm_stream.getline(line);
        line = removeblanks(line);
        libsvm_stream.skipBlanks();
        tokens = split(line,": ");
        
        // Get target
        target_index = class_strings.find(tokens[0]);
        if( target_index < 0){
            double d;
            if(pl_isnumber(tokens[0],&d) && ((double)((int)d))==d)
                target_index=(int)d;
            else
                PLERROR("In LIBSVMSparseVMatrix::build_(): target %s unknown and not an int",
                        tokens[0].c_str());
        }
        if( (tokens.size()-1)%2 != 0 )
            PLERROR("In LIBSVMSparseVMatrix::build_(): line %s has incompatible "
                    "format", line.c_str()); 
        libsvm_target_data.push_back(target_index);

        n_inputs = (tokens.size()-1)/2;
        Vec input_vec(n_inputs);
        Vec extra_vec(n_inputs);
        // Get inputs
        for( int i=0; i<n_inputs; i++)
        {
            input_index = toint(tokens[2*i+1])-1;
            extra_vec[i] = input_index;
            if( input_index > largest_input_index )
                largest_input_index = input_index;
            input_vec[i] = toreal(tokens[2*i+2]);
        }
        libsvm_input_data.push_back(input_vec);
        libsvm_extra_data.push_back(extra_vec);
        length_++;
    }

    // Set sizes
    if( inputsize_ < 0 ) inputsize_ = largest_input_index+1;
    if( targetsize_ < 0 ) targetsize_ = 1;
    if( weightsize_ < 0 ) weightsize_ = 0;
    if( use_coarse_representation )
        extrasize_ = 0;
    else
        extrasize_ = largest_input_index+1;
    if( width_ < 0 ) width_ = inputsize_ + targetsize_ + weightsize_;
}

Here is the call graph for this function:

Here is the caller graph for this function:

string PLearn::LIBSVMSparseVMatrix::classname ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 63 of file LIBSVMSparseVMatrix.cc.

void PLearn::LIBSVMSparseVMatrix::declareOptions ( OptionList ol) [static, protected]

Declares the class options.

Reimplemented from PLearn::VMatrix.

Definition at line 89 of file LIBSVMSparseVMatrix.cc.

References PLearn::OptionBase::buildoption, class_strings, PLearn::declareOption(), PLearn::VMatrix::declareOptions(), libsvm_file, and use_coarse_representation.

{
    declareOption(ol, "class_strings", &LIBSVMSparseVMatrix::class_strings,
                  OptionBase::buildoption,
                  "Strings associated to the different classes. If not present we suppose classes are int.\n");

    declareOption(ol, "libsvm_file", &LIBSVMSparseVMatrix::libsvm_file,
                  OptionBase::buildoption,
                  "File name of libsvm data.\n");

    declareOption(ol, "use_coarse_representation", &LIBSVMSparseVMatrix::use_coarse_representation,
                  OptionBase::buildoption,
                  "Indication that a coarse (i.e. fixed length, filled with 0's) representation\n"
                  "of the data in the .libsvm file should be used.\n");

    //declareOption(ol, "libsvm_input_data", 
    //              &LIBSVMSparseVMatrix::libsvm_input_data,
    //              OptionBase::learntoption,
    //              "Input data.\n");
    //
    //declareOption(ol, "libsvm_extra_data", 
    //              &LIBSVMSparseVMatrix::libsvm_extra_data,
    //              OptionBase::learntoption,
    //              "Extra data.\n");
    //
    //declareOption(ol, "libsvm_target_data", 
    //              &LIBSVMSparseVMatrix::libsvm_target_data,
    //              OptionBase::learntoption,
    //              "Target data.\n");

    // Now call the parent class' declareOptions
    inherited::declareOptions(ol);
}

Here is the call graph for this function:

static const PPath& PLearn::LIBSVMSparseVMatrix::declaringFile ( ) [inline, static]

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 89 of file LIBSVMSparseVMatrix.h.

{
LIBSVMSparseVMatrix * PLearn::LIBSVMSparseVMatrix::deepCopy ( CopiesMap copies) const [virtual]

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 63 of file LIBSVMSparseVMatrix.cc.

void PLearn::LIBSVMSparseVMatrix::getExample ( int  i,
Vec input,
Vec target,
real weight 
) [virtual]

Default version calls getSubRow based on inputsize_ targetsize_ weightsize_ But exotic subclasses may construct, input, target and weight however they please.

If not a weighted matrix, weight should be set to default value 1.

Reimplemented from PLearn::VMatrix.

Definition at line 213 of file LIBSVMSparseVMatrix.cc.

References PLearn::VMatrix::getExample(), i, PLearn::VMatrix::length(), PLearn::VMatrix::length_, libsvm_input_data, libsvm_target_data, PLERROR, PLearn::TVec< T >::resize(), PLearn::VMatrix::targetsize_, and use_coarse_representation.

{
    if( use_coarse_representation )
        inherited::getExample(i,input,target,weight);
    else
    {
        if( i>= length_ || i < 0 )
            PLERROR("In LIBSVMSparseVMatrix::getExample(): row index should "
                    "be between 0 and length_-1");
        input.resize(libsvm_input_data[i].length());
        input << libsvm_input_data[i];
        target.resize(targetsize_);
        target[0] = libsvm_target_data[i];
        weight = 1;
    }
}

Here is the call graph for this function:

void PLearn::LIBSVMSparseVMatrix::getExamples ( int  i_start,
int  length,
Mat inputs,
Mat targets,
Vec weights,
Mat extra = NULL,
bool  allow_circular = false 
)

Obtain a subset of 'length' examples, starting from 'i_start'.

The 'extra' matrix is provided as a pointer so that it can be omitted without significant overhead. If the 'allow_circular' boolean parameter is set to 'true', then one may ask for a subset that goes beyond this VMat's length: in such a case, the rest of the subset will be filled with data found at the beginning of this VMat.

Reimplemented from PLearn::VMatrix.

Definition at line 230 of file LIBSVMSparseVMatrix.cc.

References PLERROR.

{
    PLERROR("In LIBSVMSparseVMatrix::getExamples(): not compatible with "
            "sparse inputs");    
}
void PLearn::LIBSVMSparseVMatrix::getExtra ( int  i,
Vec extra 
) [virtual]

Complements the getExample method, fetching the the extrasize_ "extra" fields expected to appear after the input, target and weight fields Default version calls getSubRow based on inputsize_ targetsize_ weightsize_ and extrasize_.

Reimplemented from PLearn::VMatrix.

Definition at line 237 of file LIBSVMSparseVMatrix.cc.

References i, PLearn::VMatrix::length(), PLearn::VMatrix::length_, libsvm_extra_data, PLERROR, PLearn::TVec< T >::resize(), and use_coarse_representation.

{
    if( use_coarse_representation )
        extra.resize(0);
    else
    {
        if( i>= length_ || i < 0 )
            PLERROR("In LIBSVMSparseVMatrix::getExample(): row index should "
                    "be between 0 and length_-1");
        extra.resize(libsvm_extra_data[i].length());
        extra << libsvm_extra_data[i];
    }
}

Here is the call graph for this function:

void PLearn::LIBSVMSparseVMatrix::getNewRow ( int  i,
const Vec v 
) const [protected, virtual]

Fill the vector 'v' with the content of the i-th row.

'v' is assumed to be the right size.

Implements PLearn::RowBufferedVMatrix.

Definition at line 76 of file LIBSVMSparseVMatrix.cc.

References PLearn::TVec< T >::clear(), PLearn::TVec< T >::data(), i, in, PLearn::VMatrix::inputsize_, j, PLearn::TVec< T >::length(), libsvm_extra_data, libsvm_input_data, libsvm_target_data, PLERROR, and use_coarse_representation.

{
    if( !use_coarse_representation )
        PLERROR("In LIBSVMSparseVMatrix::getNewRow(): not compatible with sparse representations. Use use_coarse_representation=true.");
    real* in = libsvm_input_data[i].data();
    real* ex = libsvm_extra_data[i].data();
    v.clear();
    for( int j=0; j<libsvm_input_data[i].length(); j++ )
        v[(int)round(ex[j])] = in[j];

    v[inputsize_] = libsvm_target_data[i];
}

Here is the call graph for this function:

OptionList & PLearn::LIBSVMSparseVMatrix::getOptionList ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 63 of file LIBSVMSparseVMatrix.cc.

OptionMap & PLearn::LIBSVMSparseVMatrix::getOptionMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 63 of file LIBSVMSparseVMatrix.cc.

RemoteMethodMap & PLearn::LIBSVMSparseVMatrix::getRemoteMethodMap ( ) const [virtual]

Reimplemented from PLearn::Object.

Definition at line 63 of file LIBSVMSparseVMatrix.cc.

static VMat PLearn::LIBSVMSparseVMatrix::instantiateFromPPath ( const PPath filename) [inline, static]

Definition at line 94 of file LIBSVMSparseVMatrix.h.

References LIBSVMSparseVMatrix(), and PLearn::VMatrix::VMat.

    {
        //By default use normal representation
        return VMat(new LIBSVMSparseVMatrix(filename, true));
    }

Here is the call graph for this function:

void PLearn::LIBSVMSparseVMatrix::makeDeepCopyFromShallowCopy ( CopiesMap copies) [virtual]

Member Data Documentation

Reimplemented from PLearn::RowBufferedVMatrix.

Definition at line 89 of file LIBSVMSparseVMatrix.h.

Strings associated to the different classes.

Definition at line 61 of file LIBSVMSparseVMatrix.h.

Referenced by build_(), declareOptions(), and makeDeepCopyFromShallowCopy().

Initial value:
    new VMatrixExtensionRegistrar(
        "libsvm",
        &LIBSVMSparseVMatrix::instantiateFromPPath,
        "libsvm format(good for sparce input).")

Definition at line 55 of file LIBSVMSparseVMatrix.h.

Index of non-zero inputs.

Definition at line 74 of file LIBSVMSparseVMatrix.h.

Referenced by build_(), getExtra(), getNewRow(), and makeDeepCopyFromShallowCopy().

File name of libsvm data.

Definition at line 64 of file LIBSVMSparseVMatrix.h.

Referenced by build_(), declareOptions(), and makeDeepCopyFromShallowCopy().

Input data.

Definition at line 71 of file LIBSVMSparseVMatrix.h.

Referenced by build_(), getExample(), getNewRow(), and makeDeepCopyFromShallowCopy().

Target data.

Definition at line 77 of file LIBSVMSparseVMatrix.h.

Referenced by build_(), getExample(), getNewRow(), and makeDeepCopyFromShallowCopy().

Indication that a coarse (i.e.

fixed length, filled with 0's) representation of the data in the .libsvm file should be used.

Definition at line 68 of file LIBSVMSparseVMatrix.h.

Referenced by build_(), declareOptions(), getExample(), getExtra(), and getNewRow().


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines