PLearn 0.1
Public Member Functions | Static Public Member Functions | Public Attributes | Static Public Attributes | Protected Member Functions | Static Protected Member Functions | Protected Attributes | Private Types | Private Member Functions
PLearn::PythonProcessedVMatrix Class Reference

Preprocess a source VMatrix using a Python code snippet. More...

#include <PythonProcessedVMatrix.h>

Inheritance diagram for PLearn::PythonProcessedVMatrix:
Inheritance graph
[legend]
Collaboration diagram for PLearn::PythonProcessedVMatrix:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 PythonProcessedVMatrix ()
 Default constructor.
virtual string classname () const
virtual OptionListgetOptionList () const
virtual OptionMapgetOptionMap () const
virtual RemoteMethodMapgetRemoteMethodMap () const
virtual PythonProcessedVMatrixdeepCopy (CopiesMap &copies) const
virtual void build ()
 Simply calls inherited::build() then build_().
virtual void makeDeepCopyFromShallowCopy (CopiesMap &copies)
 Transforms a shallow copy into a deep copy.

Static Public Member Functions

static string _classname_ ()
 Declares name and deepCopy methods.
static OptionList_getOptionList_ ()
static RemoteMethodMap_getRemoteMethodMap_ ()
static Object_new_instance_for_typemap_ ()
static bool _isa_ (const Object *o)
static void _static_initialize_ ()
static const PPathdeclaringFile ()

Public Attributes

string m_code
 The Python code snippet used to process the matrix.
map< string, string > m_params
 General-purpose parameters that are injected into the Python code snippet and accessible via the getParam/setParam functions.

Static Public Attributes

static StaticInitializer _static_initializer_

Protected Member Functions

virtual void getNewRow (int i, const Vec &v) const
 Fill the vector 'v' with the content of the i-th row.
PythonObjectWrapper getSourceRow (const TVec< PythonObjectWrapper > &args) const
 Function injected into the Python code to return a row in the source VMatrix.
PythonObjectWrapper getParams (const TVec< PythonObjectWrapper > &args) const
 Injected into the Python code to return m_params.
PythonObjectWrapper getParam (const TVec< PythonObjectWrapper > &args) const
 Injected into the Python code to return the m_param value associated to the given key (sole argument).
PythonObjectWrapper setParam (const TVec< PythonObjectWrapper > &args)
 Injected into the Python code to set the m_param value associated to the given key.
void compileAndInject ()
 If not already done, compile the Python snippet and inject the required stuff into the Python environment.

Static Protected Member Functions

static void declareOptions (OptionList &ol)
 Declares the class options.

Protected Attributes

PP< PythonCodeSnippetpython
 Actual Python environment.

Private Types

typedef SourceVMatrix inherited

Private Member Functions

void build_ ()
 This does the actual building.

Detailed Description

Preprocess a source VMatrix using a Python code snippet.

The Python code snippet (see the option 'code' below) must define the following functions:

from numarray import *

def getRow(row_no, source_row): """Return a numarray vector containing the processed row, given the original (source) row vector at the given row number."""

def getFieldNames(source_field_names): """Return the new fieldnames (after processing), given the original fieldnames. This function also determines the NEW width of the VMatrix."""

def getSizes(source_inputsize, source_targetsize, source_weightsize): """This function is optional. If defined, return a list of 3 integers containing, respectively, the new inputsize, targetsize and weightsize."""

def build(): """This function is optional. If defined, it is called on the build_() of the underlying C++ object. Its purpose is to allow matrix-wide processings to be performed and stored in the Python snippet internal variables. Use this, for instance, to compute matrix-wide means and variances for some normalization functions."""

The Python code snippet has, in turn, access to the following interface:

In addition, a map (from string to string) named "params" is defined as an option at the C++ level, and is made available to the Python code via two injected functions: getParam(name), and setParam(name,value). Both functions directly modify the C++ map, so that there is never any synchronization issues, and further enable Python to communicate back some 'side information' to C++. In addition, the injected function getParams() returns the entire map.

Definition at line 116 of file PythonProcessedVMatrix.h.


Member Typedef Documentation

Reimplemented from PLearn::SourceVMatrix.

Definition at line 118 of file PythonProcessedVMatrix.h.


Constructor & Destructor Documentation

PLearn::PythonProcessedVMatrix::PythonProcessedVMatrix ( )

Default constructor.

Definition at line 110 of file PythonProcessedVMatrix.cc.

    : inherited(),
      m_code(""),
      python()
{ }

Member Function Documentation

string PLearn::PythonProcessedVMatrix::_classname_ ( ) [static]

Declares name and deepCopy methods.

Reimplemented from PLearn::SourceVMatrix.

Definition at line 105 of file PythonProcessedVMatrix.cc.

OptionList & PLearn::PythonProcessedVMatrix::_getOptionList_ ( ) [static]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 105 of file PythonProcessedVMatrix.cc.

RemoteMethodMap & PLearn::PythonProcessedVMatrix::_getRemoteMethodMap_ ( ) [static]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 105 of file PythonProcessedVMatrix.cc.

bool PLearn::PythonProcessedVMatrix::_isa_ ( const Object o) [static]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 105 of file PythonProcessedVMatrix.cc.

Object * PLearn::PythonProcessedVMatrix::_new_instance_for_typemap_ ( ) [static]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 105 of file PythonProcessedVMatrix.cc.

StaticInitializer PythonProcessedVMatrix::_static_initializer_ & PLearn::PythonProcessedVMatrix::_static_initialize_ ( ) [static]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 105 of file PythonProcessedVMatrix.cc.

void PLearn::PythonProcessedVMatrix::build ( ) [virtual]

Simply calls inherited::build() then build_().

Reimplemented from PLearn::SourceVMatrix.

Definition at line 141 of file PythonProcessedVMatrix.cc.

References PLearn::SourceVMatrix::build(), and build_().

{
    // ### Nothing to add here, simply calls build_
    inherited::build();
    build_();
}

Here is the call graph for this function:

void PLearn::PythonProcessedVMatrix::build_ ( ) [private]

This does the actual building.

Reimplemented from PLearn::SourceVMatrix.

Definition at line 148 of file PythonProcessedVMatrix.cc.

References compileAndInject(), PLearn::VMatrix::declareFieldNames(), PLearn::VMatrix::inputsize_, PLearn::VMat::length(), PLearn::VMatrix::length_, PLASSERT, PLERROR, python, PLearn::SourceVMatrix::setMetaInfoFromSource(), PLearn::TVec< T >::size(), PLearn::SourceVMatrix::source, PLearn::VMatrix::targetsize_, PLearn::VMatrix::weightsize_, PLearn::VMat::width(), and PLearn::VMatrix::width_.

Referenced by build().

{
    // First step, compile the Python code
    compileAndInject();
    PLASSERT( python );
    PLASSERT( source );
    
    // Next ensure that we have some consistency
    if (! python->isInvokable("getRow"))
        PLERROR("PythonProcessedVMatrix: the Python code snippet must define the function "
                "'getRow'");
    if (! python->isInvokable("getFieldNames"))
        PLERROR("PythonProcessedVMatrix: the Python code snippet must define the function "
                "'getFieldNames'");

    // Set the global variables into the Python environment
    python->setGlobalObject("source_field_names", source->fieldNames());
    python->setGlobalObject("source_length",      source->length());
    python->setGlobalObject("source_width",       source->width());
    python->setGlobalObject("source_inputsize",   source->inputsize());
    python->setGlobalObject("source_targetsize",  source->targetsize());
    python->setGlobalObject("source_weightsize",  source->weightsize());

    // Get the new fieldnames (and the new width by the same token)
    TVec<string> fieldnames =
        python->invoke("getFieldNames", source->fieldNames()).as< TVec<string> >();
    length_ = source->length();
    width_  = fieldnames.size();
    declareFieldNames(fieldnames);
    
    // Obtain and set the new sizes if defined
    if (python->isInvokable("getSizes")) {
        TVec<int> sizes = python->invoke("getSizes",
                                       source->inputsize(),
                                       source->targetsize(),
                                       source->weightsize()).as< TVec<int> >();
        if (sizes.size() != 3)
            PLERROR("PythonProcessedVMatrix: the python function 'getSizes' must return\n"
                    "a vector of integers of size 3 (exactly), containing respectively\n"
                    "the new inputsize, targetsize and weightsize");
        inputsize_  = sizes[0];
        targetsize_ = sizes[1];
        weightsize_ = sizes[2];
    }
    
    // Finish building from source informations...
    setMetaInfoFromSource();

    // Finally, call the Python builder if one exists
    if (python->isInvokable("build"))
        python->invoke("build");
}

Here is the call graph for this function:

Here is the caller graph for this function:

string PLearn::PythonProcessedVMatrix::classname ( ) const [virtual]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 105 of file PythonProcessedVMatrix.cc.

void PLearn::PythonProcessedVMatrix::compileAndInject ( ) [protected]

If not already done, compile the Python snippet and inject the required stuff into the Python environment.

Definition at line 289 of file PythonProcessedVMatrix.cc.

References getParam(), getParams(), getSourceRow(), m_code, PLASSERT, python, and setParam().

Referenced by build_(), and makeDeepCopyFromShallowCopy().

{
    if (! python) {
        python = new PythonCodeSnippet(m_code);
        PLASSERT( python );
        python->build();
        python->inject("getSourceRow", this, &PythonProcessedVMatrix::getSourceRow);
        python->inject("getParams",    this, &PythonProcessedVMatrix::getParams);
        python->inject("getParam",     this, &PythonProcessedVMatrix::getParam);
        python->inject("setParam",     this, &PythonProcessedVMatrix::setParam);
    }
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::PythonProcessedVMatrix::declareOptions ( OptionList ol) [static, protected]

Declares the class options.

Reimplemented from PLearn::SourceVMatrix.

Definition at line 119 of file PythonProcessedVMatrix.cc.

References PLearn::OptionBase::buildoption, PLearn::declareOption(), PLearn::SourceVMatrix::declareOptions(), m_code, and m_params.

{
    declareOption(ol, "code", &PythonProcessedVMatrix::m_code,
                  OptionBase::buildoption,
                  "The Python code snippet used to process the matrix.  The functions\n"
                  "described in the class documentation must be provided.  Note that,\n"
                  "after an initial build(), changing this string calling build() again\n"
                  "DOES NOT result in the recompilation of the code.\n");

    declareOption(ol, "params", &PythonProcessedVMatrix::m_params,
                  OptionBase::buildoption,
                  "General-purpose parameters that are injected into the Python code\n"
                  "snippet and accessible via the getParam/setParam functions.  Can be\n"
                  "used for passing processing arguments to the Python code.\n");
    
    // Now call the parent class' declareOptions
    inherited::declareOptions(ol);
}

Here is the call graph for this function:

static const PPath& PLearn::PythonProcessedVMatrix::declaringFile ( ) [inline, static]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 148 of file PythonProcessedVMatrix.h.

:
    //#####  Protected Member Functions  ######################################
PythonProcessedVMatrix * PLearn::PythonProcessedVMatrix::deepCopy ( CopiesMap copies) const [virtual]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 105 of file PythonProcessedVMatrix.cc.

void PLearn::PythonProcessedVMatrix::getNewRow ( int  i,
const Vec v 
) const [protected, virtual]

Fill the vector 'v' with the content of the i-th row.

v is assumed to be the right size.

Reimplemented from PLearn::SourceVMatrix.

Definition at line 204 of file PythonProcessedVMatrix.cc.

References PLERROR, python, PLearn::TVec< T >::resize(), PLearn::TVec< T >::size(), PLearn::SourceVMatrix::source, PLearn::SourceVMatrix::sourcerow, and PLearn::VMat::width().

{
    // Get the correct row from the source matrix
    sourcerow.resize(source->width());
    source->getRow(i, sourcerow);

    // Process this row using Python
    Vec processed = python->invoke("getRow", i, sourcerow).as<Vec>();

    // Ensure it has the correct size and copy into v
    if (processed.size() != v.size())
        PLERROR("PythonProcessedVMatrix: the Python function 'getRow' returned a vector "
                "of the wrong length; at row %d, obtained length=%d, expected length=%d",
                i, processed.size(), v.size());

    v << processed;
}

Here is the call graph for this function:

OptionList & PLearn::PythonProcessedVMatrix::getOptionList ( ) const [virtual]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 105 of file PythonProcessedVMatrix.cc.

OptionMap & PLearn::PythonProcessedVMatrix::getOptionMap ( ) const [virtual]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 105 of file PythonProcessedVMatrix.cc.

PythonObjectWrapper PLearn::PythonProcessedVMatrix::getParam ( const TVec< PythonObjectWrapper > &  args) const [protected]

Injected into the Python code to return the m_param value associated to the given key (sole argument).

Returns None if not found.

Definition at line 258 of file PythonProcessedVMatrix.cc.

References m_params, PLERROR, and PLearn::TVec< T >::size().

Referenced by compileAndInject().

{
    if (args.size() != 1)
        PLERROR("PythonProcessedVMatrix::getParam: expected 1 argument; got %d",
                args.size());
    string key = args[0].as<string>();
    map<string,string>::const_iterator found = m_params.find(key);
    if (found != m_params.end())
        return PythonObjectWrapper(found->second);
    else
        return PythonObjectWrapper();        // None
}

Here is the call graph for this function:

Here is the caller graph for this function:

PythonObjectWrapper PLearn::PythonProcessedVMatrix::getParams ( const TVec< PythonObjectWrapper > &  args) const [protected]

Injected into the Python code to return m_params.

Definition at line 245 of file PythonProcessedVMatrix.cc.

References m_params, PLERROR, and PLearn::TVec< T >::size().

Referenced by compileAndInject().

{
    if (args.size() != 0)
        PLERROR("PythonProcessedVMatrix::getParams: expected 0 argument; got %d",
                args.size());

    return PythonObjectWrapper(m_params);
}

Here is the call graph for this function:

Here is the caller graph for this function:

RemoteMethodMap & PLearn::PythonProcessedVMatrix::getRemoteMethodMap ( ) const [virtual]

Reimplemented from PLearn::SourceVMatrix.

Definition at line 105 of file PythonProcessedVMatrix.cc.

PythonObjectWrapper PLearn::PythonProcessedVMatrix::getSourceRow ( const TVec< PythonObjectWrapper > &  args) const [protected]

Function injected into the Python code to return a row in the source VMatrix.

Definition at line 226 of file PythonProcessedVMatrix.cc.

References i, PLearn::VMat::length(), PLERROR, PLearn::TVec< T >::resize(), PLearn::TVec< T >::size(), PLearn::SourceVMatrix::source, PLearn::SourceVMatrix::sourcerow, and PLearn::VMat::width().

Referenced by compileAndInject().

{
    if (args.size() != 1)
        PLERROR("PythonProcessedVMatrix::getSourceRow: expected 1 argument; got %d",
                args.size());
    int i = args[0].as<int>();
    if (i < 0 || i >= source->length())
        PLERROR("PythonProcessedVMatrix::getSourceRow: desired row number %d is "
                "outside the range [0,%d).", i, source->length());

    sourcerow.resize(source->width());
    source->getRow(i, sourcerow);
    return PythonObjectWrapper(sourcerow);
}

Here is the call graph for this function:

Here is the caller graph for this function:

void PLearn::PythonProcessedVMatrix::makeDeepCopyFromShallowCopy ( CopiesMap copies) [virtual]

Transforms a shallow copy into a deep copy.

Reimplemented from PLearn::SourceVMatrix.

Definition at line 305 of file PythonProcessedVMatrix.cc.

References compileAndInject(), PLearn::SourceVMatrix::makeDeepCopyFromShallowCopy(), and python.

{
    inherited::makeDeepCopyFromShallowCopy(copies);

    // Need to recompile the Python code
    python = 0;
    compileAndInject();
}

Here is the call graph for this function:

PythonObjectWrapper PLearn::PythonProcessedVMatrix::setParam ( const TVec< PythonObjectWrapper > &  args) [protected]

Injected into the Python code to set the m_param value associated to the given key.

Always return None.

Definition at line 275 of file PythonProcessedVMatrix.cc.

References m_params, PLERROR, and PLearn::TVec< T >::size().

Referenced by compileAndInject().

{
    if (args.size() != 2)
        PLERROR("PythonProcessedVMatrix::setParam: expected 2 arguments; got %d",
                args.size());
    string key   = args[0].as<string>();
    string value = args[1].as<string>();
    m_params[key] = value;
    return PythonObjectWrapper();
}

Here is the call graph for this function:

Here is the caller graph for this function:


Member Data Documentation

Reimplemented from PLearn::SourceVMatrix.

Definition at line 148 of file PythonProcessedVMatrix.h.

The Python code snippet used to process the matrix.

The functions described in the class documentation must be provided. Note that, after an initial build(), changing this string calling build() again DOES NOT result in the recompilation of the code.

Definition at line 129 of file PythonProcessedVMatrix.h.

Referenced by compileAndInject(), and declareOptions().

General-purpose parameters that are injected into the Python code snippet and accessible via the getParam/setParam functions.

Can be used for passing processing arguments to the Python code.

Definition at line 136 of file PythonProcessedVMatrix.h.

Referenced by declareOptions(), getParam(), getParams(), and setParam().

Actual Python environment.

Definition at line 187 of file PythonProcessedVMatrix.h.

Referenced by build_(), compileAndInject(), getNewRow(), and makeDeepCopyFromShallowCopy().


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines