PLearn 0.1
|
Preprocess a source VMatrix using a Python code snippet. More...
#include <PythonProcessedVMatrix.h>
Public Member Functions | |
PythonProcessedVMatrix () | |
Default constructor. | |
virtual string | classname () const |
virtual OptionList & | getOptionList () const |
virtual OptionMap & | getOptionMap () const |
virtual RemoteMethodMap & | getRemoteMethodMap () const |
virtual PythonProcessedVMatrix * | deepCopy (CopiesMap &copies) const |
virtual void | build () |
Simply calls inherited::build() then build_(). | |
virtual void | makeDeepCopyFromShallowCopy (CopiesMap &copies) |
Transforms a shallow copy into a deep copy. | |
Static Public Member Functions | |
static string | _classname_ () |
Declares name and deepCopy methods. | |
static OptionList & | _getOptionList_ () |
static RemoteMethodMap & | _getRemoteMethodMap_ () |
static Object * | _new_instance_for_typemap_ () |
static bool | _isa_ (const Object *o) |
static void | _static_initialize_ () |
static const PPath & | declaringFile () |
Public Attributes | |
string | m_code |
The Python code snippet used to process the matrix. | |
map< string, string > | m_params |
General-purpose parameters that are injected into the Python code snippet and accessible via the getParam/setParam functions. | |
Static Public Attributes | |
static StaticInitializer | _static_initializer_ |
Protected Member Functions | |
virtual void | getNewRow (int i, const Vec &v) const |
Fill the vector 'v' with the content of the i-th row. | |
PythonObjectWrapper | getSourceRow (const TVec< PythonObjectWrapper > &args) const |
Function injected into the Python code to return a row in the source VMatrix. | |
PythonObjectWrapper | getParams (const TVec< PythonObjectWrapper > &args) const |
Injected into the Python code to return m_params. | |
PythonObjectWrapper | getParam (const TVec< PythonObjectWrapper > &args) const |
Injected into the Python code to return the m_param value associated to the given key (sole argument). | |
PythonObjectWrapper | setParam (const TVec< PythonObjectWrapper > &args) |
Injected into the Python code to set the m_param value associated to the given key. | |
void | compileAndInject () |
If not already done, compile the Python snippet and inject the required stuff into the Python environment. | |
Static Protected Member Functions | |
static void | declareOptions (OptionList &ol) |
Declares the class options. | |
Protected Attributes | |
PP< PythonCodeSnippet > | python |
Actual Python environment. | |
Private Types | |
typedef SourceVMatrix | inherited |
Private Member Functions | |
void | build_ () |
This does the actual building. |
Preprocess a source VMatrix using a Python code snippet.
The Python code snippet (see the option 'code' below) must define the following functions:
from numarray import *
def getRow(row_no, source_row): """Return a numarray vector containing the processed row, given the original (source) row vector at the given row number."""
def getFieldNames(source_field_names): """Return the new fieldnames (after processing), given the original fieldnames. This function also determines the NEW width of the VMatrix."""
def getSizes(source_inputsize, source_targetsize, source_weightsize): """This function is optional. If defined, return a list of 3 integers containing, respectively, the new inputsize, targetsize and weightsize."""
def build(): """This function is optional. If defined, it is called on the build_() of the underlying C++ object. Its purpose is to allow matrix-wide processings to be performed and stored in the Python snippet internal variables. Use this, for instance, to compute matrix-wide means and variances for some normalization functions."""
The Python code snippet has, in turn, access to the following interface:
In addition, a map (from string to string) named "params" is defined as an option at the C++ level, and is made available to the Python code via two injected functions: getParam(name), and setParam(name,value). Both functions directly modify the C++ map, so that there is never any synchronization issues, and further enable Python to communicate back some 'side information' to C++. In addition, the injected function getParams() returns the entire map.
Definition at line 116 of file PythonProcessedVMatrix.h.
typedef SourceVMatrix PLearn::PythonProcessedVMatrix::inherited [private] |
Reimplemented from PLearn::SourceVMatrix.
Definition at line 118 of file PythonProcessedVMatrix.h.
PLearn::PythonProcessedVMatrix::PythonProcessedVMatrix | ( | ) |
Default constructor.
Definition at line 110 of file PythonProcessedVMatrix.cc.
string PLearn::PythonProcessedVMatrix::_classname_ | ( | ) | [static] |
Declares name and deepCopy methods.
Reimplemented from PLearn::SourceVMatrix.
Definition at line 105 of file PythonProcessedVMatrix.cc.
OptionList & PLearn::PythonProcessedVMatrix::_getOptionList_ | ( | ) | [static] |
Reimplemented from PLearn::SourceVMatrix.
Definition at line 105 of file PythonProcessedVMatrix.cc.
RemoteMethodMap & PLearn::PythonProcessedVMatrix::_getRemoteMethodMap_ | ( | ) | [static] |
Reimplemented from PLearn::SourceVMatrix.
Definition at line 105 of file PythonProcessedVMatrix.cc.
Reimplemented from PLearn::SourceVMatrix.
Definition at line 105 of file PythonProcessedVMatrix.cc.
Object * PLearn::PythonProcessedVMatrix::_new_instance_for_typemap_ | ( | ) | [static] |
Reimplemented from PLearn::SourceVMatrix.
Definition at line 105 of file PythonProcessedVMatrix.cc.
StaticInitializer PythonProcessedVMatrix::_static_initializer_ & PLearn::PythonProcessedVMatrix::_static_initialize_ | ( | ) | [static] |
Reimplemented from PLearn::SourceVMatrix.
Definition at line 105 of file PythonProcessedVMatrix.cc.
void PLearn::PythonProcessedVMatrix::build | ( | ) | [virtual] |
Simply calls inherited::build() then build_().
Reimplemented from PLearn::SourceVMatrix.
Definition at line 141 of file PythonProcessedVMatrix.cc.
References PLearn::SourceVMatrix::build(), and build_().
{ // ### Nothing to add here, simply calls build_ inherited::build(); build_(); }
void PLearn::PythonProcessedVMatrix::build_ | ( | ) | [private] |
This does the actual building.
Reimplemented from PLearn::SourceVMatrix.
Definition at line 148 of file PythonProcessedVMatrix.cc.
References compileAndInject(), PLearn::VMatrix::declareFieldNames(), PLearn::VMatrix::inputsize_, PLearn::VMat::length(), PLearn::VMatrix::length_, PLASSERT, PLERROR, python, PLearn::SourceVMatrix::setMetaInfoFromSource(), PLearn::TVec< T >::size(), PLearn::SourceVMatrix::source, PLearn::VMatrix::targetsize_, PLearn::VMatrix::weightsize_, PLearn::VMat::width(), and PLearn::VMatrix::width_.
Referenced by build().
{ // First step, compile the Python code compileAndInject(); PLASSERT( python ); PLASSERT( source ); // Next ensure that we have some consistency if (! python->isInvokable("getRow")) PLERROR("PythonProcessedVMatrix: the Python code snippet must define the function " "'getRow'"); if (! python->isInvokable("getFieldNames")) PLERROR("PythonProcessedVMatrix: the Python code snippet must define the function " "'getFieldNames'"); // Set the global variables into the Python environment python->setGlobalObject("source_field_names", source->fieldNames()); python->setGlobalObject("source_length", source->length()); python->setGlobalObject("source_width", source->width()); python->setGlobalObject("source_inputsize", source->inputsize()); python->setGlobalObject("source_targetsize", source->targetsize()); python->setGlobalObject("source_weightsize", source->weightsize()); // Get the new fieldnames (and the new width by the same token) TVec<string> fieldnames = python->invoke("getFieldNames", source->fieldNames()).as< TVec<string> >(); length_ = source->length(); width_ = fieldnames.size(); declareFieldNames(fieldnames); // Obtain and set the new sizes if defined if (python->isInvokable("getSizes")) { TVec<int> sizes = python->invoke("getSizes", source->inputsize(), source->targetsize(), source->weightsize()).as< TVec<int> >(); if (sizes.size() != 3) PLERROR("PythonProcessedVMatrix: the python function 'getSizes' must return\n" "a vector of integers of size 3 (exactly), containing respectively\n" "the new inputsize, targetsize and weightsize"); inputsize_ = sizes[0]; targetsize_ = sizes[1]; weightsize_ = sizes[2]; } // Finish building from source informations... setMetaInfoFromSource(); // Finally, call the Python builder if one exists if (python->isInvokable("build")) python->invoke("build"); }
string PLearn::PythonProcessedVMatrix::classname | ( | ) | const [virtual] |
Reimplemented from PLearn::SourceVMatrix.
Definition at line 105 of file PythonProcessedVMatrix.cc.
void PLearn::PythonProcessedVMatrix::compileAndInject | ( | ) | [protected] |
If not already done, compile the Python snippet and inject the required stuff into the Python environment.
Definition at line 289 of file PythonProcessedVMatrix.cc.
References getParam(), getParams(), getSourceRow(), m_code, PLASSERT, python, and setParam().
Referenced by build_(), and makeDeepCopyFromShallowCopy().
{ if (! python) { python = new PythonCodeSnippet(m_code); PLASSERT( python ); python->build(); python->inject("getSourceRow", this, &PythonProcessedVMatrix::getSourceRow); python->inject("getParams", this, &PythonProcessedVMatrix::getParams); python->inject("getParam", this, &PythonProcessedVMatrix::getParam); python->inject("setParam", this, &PythonProcessedVMatrix::setParam); } }
void PLearn::PythonProcessedVMatrix::declareOptions | ( | OptionList & | ol | ) | [static, protected] |
Declares the class options.
Reimplemented from PLearn::SourceVMatrix.
Definition at line 119 of file PythonProcessedVMatrix.cc.
References PLearn::OptionBase::buildoption, PLearn::declareOption(), PLearn::SourceVMatrix::declareOptions(), m_code, and m_params.
{ declareOption(ol, "code", &PythonProcessedVMatrix::m_code, OptionBase::buildoption, "The Python code snippet used to process the matrix. The functions\n" "described in the class documentation must be provided. Note that,\n" "after an initial build(), changing this string calling build() again\n" "DOES NOT result in the recompilation of the code.\n"); declareOption(ol, "params", &PythonProcessedVMatrix::m_params, OptionBase::buildoption, "General-purpose parameters that are injected into the Python code\n" "snippet and accessible via the getParam/setParam functions. Can be\n" "used for passing processing arguments to the Python code.\n"); // Now call the parent class' declareOptions inherited::declareOptions(ol); }
static const PPath& PLearn::PythonProcessedVMatrix::declaringFile | ( | ) | [inline, static] |
Reimplemented from PLearn::SourceVMatrix.
Definition at line 148 of file PythonProcessedVMatrix.h.
:
//##### Protected Member Functions ######################################
PythonProcessedVMatrix * PLearn::PythonProcessedVMatrix::deepCopy | ( | CopiesMap & | copies | ) | const [virtual] |
Reimplemented from PLearn::SourceVMatrix.
Definition at line 105 of file PythonProcessedVMatrix.cc.
Fill the vector 'v' with the content of the i-th row.
v is assumed to be the right size.
Reimplemented from PLearn::SourceVMatrix.
Definition at line 204 of file PythonProcessedVMatrix.cc.
References PLERROR, python, PLearn::TVec< T >::resize(), PLearn::TVec< T >::size(), PLearn::SourceVMatrix::source, PLearn::SourceVMatrix::sourcerow, and PLearn::VMat::width().
{ // Get the correct row from the source matrix sourcerow.resize(source->width()); source->getRow(i, sourcerow); // Process this row using Python Vec processed = python->invoke("getRow", i, sourcerow).as<Vec>(); // Ensure it has the correct size and copy into v if (processed.size() != v.size()) PLERROR("PythonProcessedVMatrix: the Python function 'getRow' returned a vector " "of the wrong length; at row %d, obtained length=%d, expected length=%d", i, processed.size(), v.size()); v << processed; }
OptionList & PLearn::PythonProcessedVMatrix::getOptionList | ( | ) | const [virtual] |
Reimplemented from PLearn::SourceVMatrix.
Definition at line 105 of file PythonProcessedVMatrix.cc.
OptionMap & PLearn::PythonProcessedVMatrix::getOptionMap | ( | ) | const [virtual] |
Reimplemented from PLearn::SourceVMatrix.
Definition at line 105 of file PythonProcessedVMatrix.cc.
PythonObjectWrapper PLearn::PythonProcessedVMatrix::getParam | ( | const TVec< PythonObjectWrapper > & | args | ) | const [protected] |
Injected into the Python code to return the m_param value associated to the given key (sole argument).
Returns None if not found.
Definition at line 258 of file PythonProcessedVMatrix.cc.
References m_params, PLERROR, and PLearn::TVec< T >::size().
Referenced by compileAndInject().
{ if (args.size() != 1) PLERROR("PythonProcessedVMatrix::getParam: expected 1 argument; got %d", args.size()); string key = args[0].as<string>(); map<string,string>::const_iterator found = m_params.find(key); if (found != m_params.end()) return PythonObjectWrapper(found->second); else return PythonObjectWrapper(); // None }
PythonObjectWrapper PLearn::PythonProcessedVMatrix::getParams | ( | const TVec< PythonObjectWrapper > & | args | ) | const [protected] |
Injected into the Python code to return m_params.
Definition at line 245 of file PythonProcessedVMatrix.cc.
References m_params, PLERROR, and PLearn::TVec< T >::size().
Referenced by compileAndInject().
{ if (args.size() != 0) PLERROR("PythonProcessedVMatrix::getParams: expected 0 argument; got %d", args.size()); return PythonObjectWrapper(m_params); }
RemoteMethodMap & PLearn::PythonProcessedVMatrix::getRemoteMethodMap | ( | ) | const [virtual] |
Reimplemented from PLearn::SourceVMatrix.
Definition at line 105 of file PythonProcessedVMatrix.cc.
PythonObjectWrapper PLearn::PythonProcessedVMatrix::getSourceRow | ( | const TVec< PythonObjectWrapper > & | args | ) | const [protected] |
Function injected into the Python code to return a row in the source VMatrix.
Definition at line 226 of file PythonProcessedVMatrix.cc.
References i, PLearn::VMat::length(), PLERROR, PLearn::TVec< T >::resize(), PLearn::TVec< T >::size(), PLearn::SourceVMatrix::source, PLearn::SourceVMatrix::sourcerow, and PLearn::VMat::width().
Referenced by compileAndInject().
{ if (args.size() != 1) PLERROR("PythonProcessedVMatrix::getSourceRow: expected 1 argument; got %d", args.size()); int i = args[0].as<int>(); if (i < 0 || i >= source->length()) PLERROR("PythonProcessedVMatrix::getSourceRow: desired row number %d is " "outside the range [0,%d).", i, source->length()); sourcerow.resize(source->width()); source->getRow(i, sourcerow); return PythonObjectWrapper(sourcerow); }
void PLearn::PythonProcessedVMatrix::makeDeepCopyFromShallowCopy | ( | CopiesMap & | copies | ) | [virtual] |
Transforms a shallow copy into a deep copy.
Reimplemented from PLearn::SourceVMatrix.
Definition at line 305 of file PythonProcessedVMatrix.cc.
References compileAndInject(), PLearn::SourceVMatrix::makeDeepCopyFromShallowCopy(), and python.
{ inherited::makeDeepCopyFromShallowCopy(copies); // Need to recompile the Python code python = 0; compileAndInject(); }
PythonObjectWrapper PLearn::PythonProcessedVMatrix::setParam | ( | const TVec< PythonObjectWrapper > & | args | ) | [protected] |
Injected into the Python code to set the m_param value associated to the given key.
Always return None.
Definition at line 275 of file PythonProcessedVMatrix.cc.
References m_params, PLERROR, and PLearn::TVec< T >::size().
Referenced by compileAndInject().
{ if (args.size() != 2) PLERROR("PythonProcessedVMatrix::setParam: expected 2 arguments; got %d", args.size()); string key = args[0].as<string>(); string value = args[1].as<string>(); m_params[key] = value; return PythonObjectWrapper(); }
Reimplemented from PLearn::SourceVMatrix.
Definition at line 148 of file PythonProcessedVMatrix.h.
The Python code snippet used to process the matrix.
The functions described in the class documentation must be provided. Note that, after an initial build(), changing this string calling build() again DOES NOT result in the recompilation of the code.
Definition at line 129 of file PythonProcessedVMatrix.h.
Referenced by compileAndInject(), and declareOptions().
map<string,string> PLearn::PythonProcessedVMatrix::m_params |
General-purpose parameters that are injected into the Python code snippet and accessible via the getParam/setParam functions.
Can be used for passing processing arguments to the Python code.
Definition at line 136 of file PythonProcessedVMatrix.h.
Referenced by declareOptions(), getParam(), getParams(), and setParam().
PP<PythonCodeSnippet> PLearn::PythonProcessedVMatrix::python [protected] |
Actual Python environment.
Definition at line 187 of file PythonProcessedVMatrix.h.
Referenced by build_(), compileAndInject(), getNewRow(), and makeDeepCopyFromShallowCopy().