PLearn 0.1
|
This VMatrix takes a VMat of a sequence of symbolic elements (corresponding to a set of symbolic attributes) and constructs context rows. More...
#include <ProcessSymbolicSequenceVMatrix.h>
Public Member Functions | |
ProcessSymbolicSequenceVMatrix (bool call_build_=false) | |
Source VMat, from which contexts are extracted. | |
ProcessSymbolicSequenceVMatrix (VMat s, int l_context, int r_context, bool call_build_=true) | |
virtual void | build () |
Simply calls inherited::build() then build_(). | |
virtual void | makeDeepCopyFromShallowCopy (CopiesMap &copies) |
Transform a shallow copy into a deep copy. | |
virtual real | getStringVal (int col, const string &str) const |
returns value associated with a string (or MISSING_VALUE if there's no association for this string) | |
virtual string | getValString (int col, real val) const |
returns the string associated with value val for field# col. | |
virtual void | getValues (int row, int col, Vec &values) const |
Gives the possible values for a certain field in the VMatrix. | |
virtual void | getValues (const Vec &input, int col, Vec &values) const |
Gives the possible values of a certain field (column) given the input. | |
virtual void | getExample (int i, Vec &input, Vec &target, real &weight) |
Default version calls getSubRow based on inputsize_ targetsize_ weightsize_ But exotic subclasses may construct, input, target and weight however they please. | |
virtual PP< Dictionary > | getDictionary (int col) const |
Return the Dictionary object for a certain field, or a null pointer if there isn't one. | |
virtual string | classname () const |
virtual OptionList & | getOptionList () const |
virtual OptionMap & | getOptionMap () const |
virtual RemoteMethodMap & | getRemoteMethodMap () const |
virtual ProcessSymbolicSequenceVMatrix * | deepCopy (CopiesMap &copies) const |
Static Public Member Functions | |
static string | _classname_ () |
Declare name and deepCopy methods. | |
static OptionList & | _getOptionList_ () |
static RemoteMethodMap & | _getRemoteMethodMap_ () |
static Object * | _new_instance_for_typemap_ () |
static bool | _isa_ (const Object *o) |
static void | _static_initialize_ () |
static const PPath & | declaringFile () |
Public Attributes | |
int | n_left_context |
Number of elements at the left of the target element. | |
int | n_right_context |
Number of elements at the right of the target element. | |
int | conditions_offset |
Offset for the position of the element on which conditions are tested (default = 0). | |
bool | conditions_for_exclusion |
Indication that the specified conditions are for the exclusion (true) or inclusion (false) of elements in the VMatrix. | |
bool | full_context |
Indication that ignored elements of context should be replaced by the next nearest valid element. | |
bool | put_only_target_attributes |
Indication that the only target fields of the VMatrix rows should be the (target) attributes of the context's target element. | |
bool | use_last_context |
Indication that the last accessed context should be put in a buffer. | |
bool | exclude_missing_target_tokens |
Indication to exclude from the VMatrix the context centered around a token with missing value target fields. | |
TVec< TVec< pair< int, int > > > | conditions |
Conditions to be satisfied for the exclusion or inclusion (see conditions_for_exclusion) of elements in the VMatrix. | |
TVec< TVec< pair< int, string > > > | string_conditions |
Conditions, in string format, to be satisfied for the exclusion or inclusion (see conditions_for_exclusion) of elements in the VMatrix. | |
TVec< TVec< pair< int, int > > > | delimiters |
Delimiters of context. | |
TVec< TVec< pair< int, string > > > | string_delimiters |
Delimiters, in string format, of context. | |
TVec< TVec< pair< int, int > > > | ignored_context |
Elements to be ignored in context. | |
TVec< TVec< pair< int, string > > > | string_ignored_context |
Elements, in string format, to be ignored in context. | |
Static Public Attributes | |
static StaticInitializer | _static_initializer_ |
Protected Member Functions | |
virtual void | getNewRow (int i, const Vec &v) const |
Fill the vector 'v' with the content of the i-th row. | |
Static Protected Member Functions | |
static void | declareOptions (OptionList &ol) |
Declares this class' options. | |
Private Types | |
typedef SourceVMatrix | inherited |
typedef hash_map< int, int > | this_hash_map |
Private Member Functions | |
void | build_ () |
This does the actual building. | |
void | from_string_to_int_format (TVec< TVec< pair< int, string > > > &str_format, TVec< TVec< pair< int, int > > > &int_format) |
void | from_string_to_int_format (TVec< pair< int, string > > &str_format, TVec< pair< int, int > > &int_format) |
bool | is_true (const TVec< TVec< pair< int, int > > > &conditions, const Vec row) const |
void | fill_current_row (int i, int &cp, int &target_position) const |
Fills the current_row_i field, and returns the number of non missing values for the ith context. | |
Private Attributes | |
bool | fixed_context |
Indication that the context is of fixed length. | |
int | max_context_length |
Maximum length of a context. | |
int | n_attributes |
Number of attributes. | |
int | current_target_pos |
Position in the last context of the target element (symbol and attributes) | |
int | lower_bound |
Position in the source sequence of the first element in the last context. | |
int | upper_bound |
Position in the source sequence of the last element in the last context. | |
Vec | current_row_i |
Last context. | |
this_hash_map | current_context_pos |
Mapping from the position in the source sequence to the position in the last context. | |
TVec< int > | indices |
Indices of the selected contexts. | |
TVec< int > | left_positions |
Temporary fields. | |
TVec< int > | right_positions |
Vec | left_context |
Temporary fields. | |
Vec | right_context |
Vec | row |
Vec | element |
Vec | target_element |
Vec | subinput |
This VMatrix takes a VMat of a sequence of symbolic elements (corresponding to a set of symbolic attributes) and constructs context rows.
An example of a sequence of elements would be a sequence of words, with their lemma form and POS tag. This sequence is encoded using integers, and is represented by the source VMatrix such as each row is an element, and each column is a certain type of attribute (e.g. lemma, POS tag, etc.). The source VMat string mapping (functions getStringVal(...) and getValString(...)) contains the integer/string encoding of the symbolic data. The context rows can be of fixed length, or constrained by delimiter symbols. Certain rows can be selected/excluded, and certain elements can be excluded of a context according to some conditions on its attributes. The conditions are expressed as disjunctions of conjunctions. For example:
[ [ 0 : "person", 1 : "NN" ] , [ 0 : "kind", 2 : "plural" ] ]
is equivalent in C++ logic form to:
(fields[0]=="person" && fields[1]=="NN") || (fields[0]=="kind" && fields[2]=="plural")
Conditions can be expressed in string or int format. The integer/string mapping is used to make the correspondance. We call the 'target element' of a context the element around which other elements are collected to construct the context.
Definition at line 84 of file ProcessSymbolicSequenceVMatrix.h.
typedef SourceVMatrix PLearn::ProcessSymbolicSequenceVMatrix::inherited [private] |
Reimplemented from PLearn::SourceVMatrix.
Definition at line 89 of file ProcessSymbolicSequenceVMatrix.h.
typedef hash_map<int,int> PLearn::ProcessSymbolicSequenceVMatrix::this_hash_map [private] |
Definition at line 91 of file ProcessSymbolicSequenceVMatrix.h.
PLearn::ProcessSymbolicSequenceVMatrix::ProcessSymbolicSequenceVMatrix | ( | bool | call_build_ = false | ) |
Source VMat, from which contexts are extracted.
Definition at line 51 of file ProcessSymbolicSequenceVMatrix.cc.
References build_().
: inherited(call_build_), n_left_context(2), n_right_context(2), conditions_offset(0), conditions_for_exclusion(1), full_context(true), put_only_target_attributes(false), use_last_context(true), exclude_missing_target_tokens(false) /* ### Initialise all fields to their default value */ { if( call_build_ ) build_(); }
PLearn::ProcessSymbolicSequenceVMatrix::ProcessSymbolicSequenceVMatrix | ( | VMat | s, |
int | l_context, | ||
int | r_context, | ||
bool | call_build_ = true |
||
) |
Definition at line 63 of file ProcessSymbolicSequenceVMatrix.cc.
References build_(), n_left_context, and n_right_context.
: inherited(s, call_build_), conditions_offset(0), conditions_for_exclusion(1), full_context(true), use_last_context(true) /* ### Initialise all fields to their default value */ { n_left_context= l_context; n_right_context = r_context; if( call_build_ ) build_(); }
string PLearn::ProcessSymbolicSequenceVMatrix::_classname_ | ( | ) | [static] |
Declare name and deepCopy methods.
Reimplemented from PLearn::SourceVMatrix.
Definition at line 107 of file ProcessSymbolicSequenceVMatrix.cc.
OptionList & PLearn::ProcessSymbolicSequenceVMatrix::_getOptionList_ | ( | ) | [static] |
Reimplemented from PLearn::SourceVMatrix.
Definition at line 107 of file ProcessSymbolicSequenceVMatrix.cc.
RemoteMethodMap & PLearn::ProcessSymbolicSequenceVMatrix::_getRemoteMethodMap_ | ( | ) | [static] |
Reimplemented from PLearn::SourceVMatrix.
Definition at line 107 of file ProcessSymbolicSequenceVMatrix.cc.
Reimplemented from PLearn::SourceVMatrix.
Definition at line 107 of file ProcessSymbolicSequenceVMatrix.cc.
Object * PLearn::ProcessSymbolicSequenceVMatrix::_new_instance_for_typemap_ | ( | ) | [static] |
Reimplemented from PLearn::SourceVMatrix.
Definition at line 107 of file ProcessSymbolicSequenceVMatrix.cc.
StaticInitializer ProcessSymbolicSequenceVMatrix::_static_initializer_ & PLearn::ProcessSymbolicSequenceVMatrix::_static_initialize_ | ( | ) | [static] |
Reimplemented from PLearn::SourceVMatrix.
Definition at line 107 of file ProcessSymbolicSequenceVMatrix.cc.
void PLearn::ProcessSymbolicSequenceVMatrix::build | ( | ) | [virtual] |
Simply calls inherited::build() then build_().
Reimplemented from PLearn::SourceVMatrix.
Definition at line 388 of file ProcessSymbolicSequenceVMatrix.cc.
References PLearn::SourceVMatrix::build(), and build_().
{ inherited::build(); build_(); }
void PLearn::ProcessSymbolicSequenceVMatrix::build_ | ( | ) | [private] |
This does the actual building.
Reimplemented from PLearn::SourceVMatrix.
Definition at line 247 of file ProcessSymbolicSequenceVMatrix.cc.
References PLearn::TVec< T >::append(), PLearn::TVec< T >::clear(), conditions, conditions_for_exclusion, conditions_offset, current_context_pos, current_row_i, current_target_pos, delimiters, element, exclude_missing_target_tokens, fixed_context, from_string_to_int_format(), full_context, i, ignored_context, indices, PLearn::VMatrix::inputsize_, PLearn::is_missing(), is_true(), j, left_context, left_positions, PLearn::TVec< T >::length(), PLearn::VMatrix::length(), PLearn::VMat::length(), PLearn::VMatrix::length_, lower_bound, max_context_length, n_attributes, n_left_context, n_right_context, PLERROR, put_only_target_attributes, PLearn::TVec< T >::resize(), right_context, right_positions, row, PLearn::SourceVMatrix::source, string_conditions, string_delimiters, string_ignored_context, target_element, PLearn::VMatrix::targetsize_, PLearn::tostring(), PLearn::VMatrix::updateMtime(), upper_bound, PLearn::VMatrix::weightsize_, PLearn::VMat::width(), and PLearn::VMatrix::width_.
Referenced by build(), and ProcessSymbolicSequenceVMatrix().
{ if(!source) PLERROR("In SelectAttributeSequenceVMatrix::build_() : no source" " defined"); // defineSizes(source->inputsize(),source->targetsize(),source->weightsize()); pas bon car ecrase declare options n_attributes = source->width(); row.resize(n_attributes); element.resize(n_attributes); target_element.resize(n_attributes); max_context_length = 0; fixed_context = n_left_context >=0 && n_right_context >= 0; // from string to int format on ... // conditions from_string_to_int_format(string_conditions, conditions); string_conditions.clear(); string_conditions.resize(0); // delimiters from_string_to_int_format(string_delimiters, delimiters); string_delimiters.clear(); string_delimiters.resize(0); // ignored_context from_string_to_int_format(string_ignored_context, ignored_context); string_ignored_context.clear(); string_ignored_context.resize(0); // gathering information from source VMat indices.clear(); indices.resize(0); int current_context_length = 0; bool target_contains_missing = false; PP<ProgressBar> pb = new ProgressBar("Gathering information from source VMat of length " + tostring(source->length()), source->length()); for(int i=0; i<source->length(); i++) { source->getRow(i,row); if(is_true(delimiters,row)) { max_context_length = max_context_length < current_context_length ? current_context_length : max_context_length; current_context_length = 0; } else { if(!full_context || !is_true(ignored_context,row)) current_context_length++; } // Testing conditions for inclusion/exclusion if(i + conditions_offset >= 0 && i + conditions_offset < source->length()) { source->getRow(i+conditions_offset,row); if(exclude_missing_target_tokens) { target_contains_missing = false; int ni = source->targetsize()+source->inputsize(); for(int j=source->inputsize(); j<ni; j++) if(is_missing(row[j])) { target_contains_missing = true; break; } } if(!exclude_missing_target_tokens || !target_contains_missing) { if(is_true(conditions,row)) { if(!conditions_for_exclusion) indices.append(i); } else { if(conditions_for_exclusion) indices.append(i); } } } pb->update(i+1); } max_context_length = max_context_length < current_context_length ? current_context_length : max_context_length; if(n_left_context >= 0 && n_right_context >= 0) max_context_length = 1 + n_left_context + n_right_context; length_ = indices.length(); width_ = n_attributes*(max_context_length); if(inputsize_ < 0) inputsize_ = max_context_length * source->inputsize(); if(targetsize_ < 0 && put_only_target_attributes) targetsize_ = source->targetsize(); if(targetsize_ < 0 && !put_only_target_attributes) targetsize_ = max_context_length * source->targetsize(); if(weightsize_ < 0) weightsize_ = source->weightsize(); if(weightsize_ > 0) PLERROR("In ProcessSymbolicSequenceVMatrix:build_() : does not support weightsize > 0"); current_row_i.resize(n_attributes*(max_context_length)); current_target_pos = -1; current_context_pos.clear(); lower_bound = 1; upper_bound = -1; if(n_left_context < 0) { left_context.resize(width_); left_positions.resize(max_context_length); } else { left_context.resize(n_attributes*n_left_context); left_positions.resize(n_left_context); } if(n_right_context < 0) { right_context.resize(width_); right_positions.resize(max_context_length); } else { right_context.resize(n_attributes*n_right_context); right_positions.resize(n_right_context); } if(put_only_target_attributes) { width_ = source->inputsize()*max_context_length + source->targetsize(); } if(inputsize_+targetsize_ != width_) PLERROR("In ProcessSymbolicSequenceVMatrix:build_() : inputsize_ + targetsize_ != width_"); // Should we call: // setMetaInfoFromSource(); // ? updateMtime(source); }
string PLearn::ProcessSymbolicSequenceVMatrix::classname | ( | ) | const [virtual] |
Reimplemented from PLearn::SourceVMatrix.
Definition at line 107 of file ProcessSymbolicSequenceVMatrix.cc.
void PLearn::ProcessSymbolicSequenceVMatrix::declareOptions | ( | OptionList & | ol | ) | [static, protected] |
Declares this class' options.
Reimplemented from PLearn::SourceVMatrix.
Definition at line 142 of file ProcessSymbolicSequenceVMatrix.cc.
References PLearn::OptionBase::buildoption, conditions, conditions_for_exclusion, conditions_offset, PLearn::declareOption(), PLearn::SourceVMatrix::declareOptions(), delimiters, exclude_missing_target_tokens, full_context, ignored_context, n_left_context, n_right_context, put_only_target_attributes, string_conditions, string_delimiters, string_ignored_context, and use_last_context.
{ // ### Declare all of this object's options here // ### For the "flags" of each option, you should typically specify // ### one of OptionBase::buildoption, OptionBase::learntoption or // ### OptionBase::tuningoption. Another possible flag to be combined with // ### is OptionBase::nosave declareOption(ol, "n_left_context", &ProcessSymbolicSequenceVMatrix::n_left_context, OptionBase::buildoption, "Number of elements at the left of (or before) the target" " element.\n" "(if < 0, all elements to the left are included until a" " delimiter is met)\n"); declareOption(ol, "n_right_context", &ProcessSymbolicSequenceVMatrix::n_right_context, OptionBase::buildoption, "Number of elements at the right of (or after) the target" " element.\n" "(if < 0, all elements to the right are included until a" " delimiter is met)\n"); declareOption(ol, "conditions_offset", &ProcessSymbolicSequenceVMatrix::conditions_offset, OptionBase::buildoption, "Offset for the position of the element on which conditions" " are tested\n" "(default = 0)\n"); declareOption(ol, "conditions_for_exclusion", &ProcessSymbolicSequenceVMatrix::conditions_for_exclusion, OptionBase::buildoption, "Indication that the specified conditions are for the" " exclusion (true)\n" "or inclusion (false) of elements in the VMatrix\n"); declareOption(ol, "full_context", &ProcessSymbolicSequenceVMatrix::full_context, OptionBase::buildoption, "Indication that ignored elements of context should be" " replaced by the\n" "next nearest valid element\n"); declareOption(ol, "put_only_target_attributes", &ProcessSymbolicSequenceVMatrix::put_only_target_attributes, OptionBase::buildoption, "Indication that the only target fields of the VMatrix rows" " should be\n" "the (target) attributes of the context's target element\n"); declareOption(ol, "use_last_context", &ProcessSymbolicSequenceVMatrix::use_last_context, OptionBase::buildoption, "Indication that the last accessed context should be put in" " a buffer.\n"); declareOption(ol, "exclude_missing_target_tokens", &ProcessSymbolicSequenceVMatrix::exclude_missing_target_tokens, OptionBase::buildoption, "Indication to exclude from the VMatrix the context centered\n" "around a token with missing value target fields.\n"); declareOption(ol, "conditions", &ProcessSymbolicSequenceVMatrix::conditions, OptionBase::buildoption, "Conditions to be satisfied for the exclusion or inclusion" " (see\n" "conditions_for_exclusion) of elements in the VMatrix\n"); declareOption(ol, "string_conditions", &ProcessSymbolicSequenceVMatrix::string_conditions, OptionBase::buildoption, "Conditions, in string format, to be satisfied for the" " exclusion or\n" "inclusion (see conditions_for_exclusion) of elements in the" " VMatrix\n"); declareOption(ol, "delimiters", &ProcessSymbolicSequenceVMatrix::delimiters, OptionBase::buildoption, "Delimiters of context\n"); declareOption(ol, "string_delimiters", &ProcessSymbolicSequenceVMatrix::string_delimiters, OptionBase::buildoption, "Delimiters, in string format, of context\n"); declareOption(ol, "ignored_context", &ProcessSymbolicSequenceVMatrix::ignored_context, OptionBase::buildoption, "Elements to be ignored in context\n"); declareOption(ol, "string_ignored_context", &ProcessSymbolicSequenceVMatrix::string_ignored_context, OptionBase::buildoption, "Elements, in string format, to be ignored in context\n"); // Now call the parent class' declareOptions inherited::declareOptions(ol); }
static const PPath& PLearn::ProcessSymbolicSequenceVMatrix::declaringFile | ( | ) | [inline, static] |
Reimplemented from PLearn::SourceVMatrix.
Definition at line 309 of file ProcessSymbolicSequenceVMatrix.h.
ProcessSymbolicSequenceVMatrix * PLearn::ProcessSymbolicSequenceVMatrix::deepCopy | ( | CopiesMap & | copies | ) | const [virtual] |
Reimplemented from PLearn::SourceVMatrix.
Definition at line 107 of file ProcessSymbolicSequenceVMatrix.cc.
void PLearn::ProcessSymbolicSequenceVMatrix::fill_current_row | ( | int | i, |
int & | cp, | ||
int & | target_position | ||
) | const [private] |
Fills the current_row_i field, and returns the number of non missing values for the ith context.
Definition at line 523 of file ProcessSymbolicSequenceVMatrix.cc.
References PLearn::cp(), current_context_pos, current_row_i, current_target_pos, delimiters, element, PLearn::VMatrix::fill(), PLearn::TVec< T >::fill(), fixed_context, full_context, i, ignored_context, indices, is_true(), left_context, left_positions, PLearn::TVec< T >::length(), PLearn::VMat::length(), lower_bound, MISSING_VALUE, n_attributes, n_left_context, n_right_context, right_context, right_positions, PLearn::SourceVMatrix::source, PLearn::TVec< T >::subVec(), target_element, upper_bound, and use_last_context.
Referenced by getExample(), and getNewRow().
{ int target = indices[i]; // If the target element is a delimiter, do nothing: Hugo: may not be a good thing! source->getRow(target,target_element); /* if(is_true(delimiters,target_element)) { v.fill(MISSING_VALUE); if(use_last_context) { current_context_pos.clear(); current_row_i.fill(MISSING_VALUE); current_target_pos = -1; lower_bound = 1; upper_bound = -1; } return; } */ int left_added_elements = 0; int right_added_elements = 0; int left_pos = 1; int right_pos = 1; int p = 0; bool to_add = false; int this_lower_bound = target; int this_upper_bound = target; // Left context construction while(n_left_context < 0 || left_added_elements < n_left_context) { to_add = false; p = target-left_pos; if(p < 0 || p >= source->length()) break; // Verifying if context can be found in last context if(use_last_context && lower_bound <= p && upper_bound >= p) { if(current_context_pos.find(p) != current_context_pos.end()) { int position = current_context_pos[p]; element = current_row_i.subVec(n_attributes*position,n_attributes); if(!is_true(ignored_context,element)) { to_add = true; } } else { left_pos++; continue; } } // If not, fetch element in source VMat else { source->getRow(p,element); if(!is_true(ignored_context,element)) to_add = true; } if(is_true(delimiters,element)) break; if(to_add) { left_context.subVec(n_attributes*left_added_elements,n_attributes) << element; if(use_last_context) left_positions[left_added_elements] = p; this_lower_bound = p; left_added_elements++; } else if(!full_context) { left_context.subVec(n_attributes*left_added_elements,n_attributes).fill(MISSING_VALUE); if(use_last_context) left_positions[left_added_elements] = p; this_lower_bound = p; left_added_elements++; } left_pos++; } // Right context construction while(n_right_context < 0 || right_added_elements < n_right_context) { to_add = false; p = target+right_pos; if(p < 0 || p >= source->length()) break; // Verifying if context can be found in last context if(use_last_context && lower_bound <= p && upper_bound >= p) { if(current_context_pos.find(p) != current_context_pos.end()) { int position = current_context_pos[p]; element = current_row_i.subVec(n_attributes*position,n_attributes); if(!is_true(ignored_context,element)) { to_add = true; } } else { right_pos++; continue; } } // If not, fetch element in source VMat else { source->getRow(p,element); if(!is_true(ignored_context,element)) to_add = true; } if(is_true(delimiters,element)) break; if(to_add) { right_context.subVec(n_attributes*right_added_elements,n_attributes) << element; if(use_last_context) right_positions[right_added_elements] = p; this_upper_bound = p; right_added_elements++; } else if(!full_context) { right_context.subVec(n_attributes*right_added_elements,n_attributes).fill(MISSING_VALUE); if(use_last_context) right_positions[right_added_elements] = p; this_upper_bound = p; right_added_elements++; } right_pos++; } current_context_pos.clear(); current_target_pos = -1; lower_bound = this_lower_bound; upper_bound = this_upper_bound; //current_row_i.fill(MISSING_VALUES); // Constructing complete row cp = 0; if(fixed_context) // Adding missing value, for the case where the context is of fixed length if(n_left_context-left_added_elements>0) { current_row_i.subVec(cp*n_attributes,n_attributes*(n_left_context-left_added_elements)).fill(MISSING_VALUE); cp = n_left_context-left_added_elements; } // adding left context int temp = left_added_elements; while(temp > 0) { temp--; current_row_i.subVec(cp*n_attributes,n_attributes) << left_context.subVec(temp*n_attributes,n_attributes); if(use_last_context) current_context_pos[(int)left_positions[temp]] = cp; cp++; } // adding target element target_position = cp; current_row_i.subVec(cp*n_attributes,n_attributes) << target_element; if(use_last_context) { current_context_pos[target] = cp; current_target_pos = cp; } cp++; // adding right context int r=0; while(r<right_added_elements) { current_row_i.subVec(cp*n_attributes,n_attributes) << right_context.subVec(r*n_attributes,n_attributes); if(use_last_context) current_context_pos[(int)right_positions[r]] = cp; r++; cp++; } if(fixed_context && current_row_i.length()-cp*n_attributes > 0) { current_row_i.subVec(cp*n_attributes,current_row_i.length()-cp*n_attributes).fill(MISSING_VALUE); cp = current_row_i.length()/n_attributes; } }
void PLearn::ProcessSymbolicSequenceVMatrix::from_string_to_int_format | ( | TVec< TVec< pair< int, string > > > & | str_format, |
TVec< TVec< pair< int, int > > > & | int_format | ||
) | [inline, private] |
Definition at line 215 of file ProcessSymbolicSequenceVMatrix.h.
References i.
Referenced by build_().
{ int int_format_length = int_format.length(); int_format.resize(int_format_length + str_format.length()); for(int i=0; i<str_format.length(); i++) from_string_to_int_format(str_format[i], int_format[i+int_format_length]); }
void PLearn::ProcessSymbolicSequenceVMatrix::from_string_to_int_format | ( | TVec< pair< int, string > > & | str_format, |
TVec< pair< int, int > > & | int_format | ||
) | [inline, private] |
Definition at line 225 of file ProcessSymbolicSequenceVMatrix.h.
References i, PLearn::is_missing(), and PLERROR.
{ int int_format_length = int_format.length(); int_format.resize(int_format_length+str_format.length()); for(int i=0; i<str_format.length(); i++) { int_format[int_format_length+i].first = str_format[i].first; real value = source->getStringVal(str_format[i].first, str_format[i].second); if(is_missing(value)) PLERROR("In ProcessSymbolicSequenceVMatrix::" "from_string_to_int_format :\n" "%s not a valid string symbol for column %d\n", str_format[i].second.c_str(), str_format[i].first); int_format[int_format_length+i].second = (int)value; } }
PP< Dictionary > PLearn::ProcessSymbolicSequenceVMatrix::getDictionary | ( | int | col | ) | const [virtual] |
Return the Dictionary object for a certain field, or a null pointer if there isn't one.
Reimplemented from PLearn::SourceVMatrix.
Definition at line 486 of file ProcessSymbolicSequenceVMatrix.cc.
References PLearn::VMatrix::inputsize(), max_context_length, PLERROR, PLearn::SourceVMatrix::source, and PLearn::VMatrix::width_.
{ if(col < 0 || col >= width_) PLERROR("In ProcessSymbolicSequenceVMatrix::getDictionary() : invalid col %d, width()=%d", col, width_); if(source) { int src_col; if(col < max_context_length * source->inputsize()) src_col = col%source->inputsize(); else src_col = source->inputsize() + (col-max_context_length * source->inputsize())%source->targetsize(); return source->getDictionary(src_col); } return 0; }
void PLearn::ProcessSymbolicSequenceVMatrix::getExample | ( | int | i, |
Vec & | input, | ||
Vec & | target, | ||
real & | weight | ||
) | [virtual] |
Default version calls getSubRow based on inputsize_ targetsize_ weightsize_ But exotic subclasses may construct, input, target and weight however they please.
If not a weighted matrix, weight should be set to default value 1.
Reimplemented from PLearn::VMatrix.
Definition at line 731 of file ProcessSymbolicSequenceVMatrix.cc.
References PLearn::cp(), current_row_i, fill_current_row(), fixed_context, PLearn::VMatrix::inputsize(), PLearn::VMatrix::inputsize_, PLearn::TVec< T >::length(), PLearn::VMatrix::length_, n_attributes, PLERROR, put_only_target_attributes, PLearn::TVec< T >::resize(), PLearn::SourceVMatrix::source, PLearn::VMatrix::targetsize(), PLearn::VMatrix::targetsize_, PLearn::VMatrix::weightsize_, and PLearn::VMatrix::width_.
{ if(i<0 || i>=length_) PLERROR("In SelectAttributeSequenceVMatrix::getExample() : invalid row acces i=%d for VMatrix(%d,%d)",i,length_,width_); int target_position,cp; fill_current_row(i,cp,target_position); // Seperating input and output fields input.resize(cp*source->inputsize()); target.resize(put_only_target_attributes ? source->targetsize() : cp*source->targetsize()); if(weightsize_ == 0) weight = 1; else PLERROR("In ProcessSymbolicSequenceVMatrix::getExample(): weighsize() > 0 not implemented"); for(int t=0; t<cp*n_attributes; t++) { if(t%n_attributes < source->inputsize()) input[t/n_attributes * source->inputsize() + t%n_attributes] = current_row_i[t]; else if(put_only_target_attributes) { if(t/n_attributes == target_position) target[t%n_attributes - source->inputsize() ] = current_row_i[t]; } else target[t/n_attributes * source->targetsize() + t%n_attributes - source->inputsize() ] = current_row_i[t]; } if(fixed_context) { // Verify if inputsize and targetsize were redefined if(inputsize_ > input.length()) { input.resize(inputsize_); int it = 0; for(int t=cp*source->inputsize(); t<inputsize_; t++) input[t] = target[it++]; for(int t=0; t<targetsize_; t++) target[t] = target[it++]; target.resize(targetsize_); } else if(targetsize() > target.length()) { target.resize(targetsize_); int it = cp*source->inputsize()-inputsize_; for(int t=0; it<targetsize_; t++) target[it++] = target[t]; it = inputsize_; for(int t=0; it<input.length(); t++) target[t] = input[it++]; input.resize(inputsize_); } } return; }
void PLearn::ProcessSymbolicSequenceVMatrix::getNewRow | ( | int | i, |
const Vec & | v | ||
) | const [protected, virtual] |
Fill the vector 'v' with the content of the i-th row.
v is assumed to be the right size.
Reimplemented from PLearn::SourceVMatrix.
Definition at line 109 of file ProcessSymbolicSequenceVMatrix.cc.
References PLearn::cp(), current_row_i, PLearn::VMatrix::fill(), fill_current_row(), PLearn::VMatrix::inputsize(), PLearn::TVec< T >::length(), PLearn::VMatrix::length_, max_context_length, MISSING_VALUE, n_attributes, PLERROR, put_only_target_attributes, PLearn::TVec< T >::resize(), PLearn::SourceVMatrix::source, PLearn::TVec< T >::subVec(), and PLearn::VMatrix::width_.
{ if(i<0 || i>=length_) PLERROR("In SelectAttributeSequenceVMatrix::getNewRow() :\n" " invalid row acces i=%d for VMatrix(%d,%d)\n", i,length_,width_); current_row_i.resize(n_attributes*(max_context_length)); int cp,target_position; fill_current_row(i,cp,target_position); if(current_row_i.length()-cp*n_attributes > 0) current_row_i.subVec(cp*n_attributes,current_row_i.length()-cp*n_attributes).fill(MISSING_VALUE); // Seperating input and output fields for(int t=0; t<current_row_i.length(); t++) { if(t%n_attributes < source->inputsize()) v[t/n_attributes * source->inputsize() + t%n_attributes] = current_row_i[t]; else if(put_only_target_attributes) { if(t/n_attributes == target_position) v[max_context_length*source->inputsize() + t%n_attributes - source->inputsize() ] = current_row_i[t]; } else v[max_context_length*source->inputsize() + t/n_attributes * source->targetsize() + t%n_attributes - source->inputsize() ] = current_row_i[t]; } return; }
OptionList & PLearn::ProcessSymbolicSequenceVMatrix::getOptionList | ( | ) | const [virtual] |
Reimplemented from PLearn::SourceVMatrix.
Definition at line 107 of file ProcessSymbolicSequenceVMatrix.cc.
OptionMap & PLearn::ProcessSymbolicSequenceVMatrix::getOptionMap | ( | ) | const [virtual] |
Reimplemented from PLearn::SourceVMatrix.
Definition at line 107 of file ProcessSymbolicSequenceVMatrix.cc.
RemoteMethodMap & PLearn::ProcessSymbolicSequenceVMatrix::getRemoteMethodMap | ( | ) | const [virtual] |
Reimplemented from PLearn::SourceVMatrix.
Definition at line 107 of file ProcessSymbolicSequenceVMatrix.cc.
real PLearn::ProcessSymbolicSequenceVMatrix::getStringVal | ( | int | col, |
const string & | str | ||
) | const [virtual] |
returns value associated with a string (or MISSING_VALUE if there's no association for this string)
Reimplemented from PLearn::VMatrix.
Definition at line 394 of file ProcessSymbolicSequenceVMatrix.cc.
References PLearn::VMatrix::inputsize(), max_context_length, MISSING_VALUE, and PLearn::SourceVMatrix::source.
{ if(source) { int src_col; if(col < max_context_length * source->inputsize()) src_col = col%source->inputsize(); else src_col = source->inputsize() + (col-max_context_length * source->inputsize())%source->targetsize(); return source->getStringVal(src_col,str); } return MISSING_VALUE; }
returns the string associated with value val for field# col.
Or returns "" if no string is associated.
Reimplemented from PLearn::VMatrix.
Definition at line 409 of file ProcessSymbolicSequenceVMatrix.cc.
References PLearn::VMatrix::inputsize(), max_context_length, and PLearn::SourceVMatrix::source.
{ if(source) { int src_col; if(col < max_context_length * source->inputsize()) src_col = col%source->inputsize(); else src_col = source->inputsize() + (col-max_context_length * source->inputsize())%source->targetsize(); return source->getValString(src_col,val); } return ""; }
void PLearn::ProcessSymbolicSequenceVMatrix::getValues | ( | int | row, |
int | col, | ||
Vec & | values | ||
) | const [virtual] |
Gives the possible values for a certain field in the VMatrix.
Reimplemented from PLearn::SourceVMatrix.
Definition at line 424 of file ProcessSymbolicSequenceVMatrix.cc.
References indices, PLearn::VMatrix::inputsize(), PLearn::VMatrix::length_, max_context_length, PLERROR, PLearn::TVec< T >::resize(), PLearn::SourceVMatrix::source, and PLearn::VMatrix::width_.
{ if(row < 0 || row >= length_) PLERROR("In ProcessSymbolicSequenceVMatrix::getValues() : invalid row %d, length()=%d", row, length_); if(col < 0 || col >= width_) PLERROR("In ProcessSymbolicSequenceVMatrix::getValues() : invalid col %d, width()=%d", col, width_); if(source) { int src_col; if(col < max_context_length * source->inputsize()) src_col = col%source->inputsize(); else src_col = source->inputsize() + (col-max_context_length * source->inputsize())%source->targetsize(); source->getValues(indices[row],src_col, values); } else values.resize(0); }
void PLearn::ProcessSymbolicSequenceVMatrix::getValues | ( | const Vec & | input, |
int | col, | ||
Vec & | values | ||
) | const [virtual] |
Gives the possible values of a certain field (column) given the input.
Reimplemented from PLearn::SourceVMatrix.
Definition at line 441 of file ProcessSymbolicSequenceVMatrix.cc.
References PLearn::TVec< T >::fill(), PLearn::VMatrix::inputsize_, max_context_length, MISSING_VALUE, n_left_context, PLERROR, put_only_target_attributes, PLearn::TVec< T >::resize(), PLearn::SourceVMatrix::source, subinput, PLearn::TVec< T >::subVec(), and PLearn::VMatrix::width_.
{ if(col < 0 || col >= width_) PLERROR("In ProcessSymbolicSequenceVMatrix::getValues() : invalid col %d, width()=%d", col, width_); if(source) { int sinputsize = source->inputsize(); int stargetsize = source->targetsize(); int src_col,this_col; if(col < max_context_length * sinputsize) src_col = col%sinputsize; else src_col = sinputsize + (col-max_context_length * sinputsize)%stargetsize; // Fill subinput input part subinput.resize(sinputsize); if(col>=inputsize_) { if(put_only_target_attributes) this_col = n_left_context; else this_col = (col-inputsize_)/stargetsize; } else this_col = col/sinputsize; subinput << input.subVec(this_col*sinputsize,sinputsize); // Fill subinput target part subinput.resize(sinputsize+stargetsize); if(!put_only_target_attributes || col == inputsize_ || (col<inputsize_ && col/sinputsize == n_left_context)) { if(put_only_target_attributes) subinput.subVec(sinputsize,stargetsize) << input.subVec(inputsize_,stargetsize); else subinput.subVec(sinputsize,stargetsize) << input.subVec(inputsize_+this_col*stargetsize,stargetsize); } else { subinput.subVec(sinputsize,stargetsize).fill(MISSING_VALUE); } source->getValues(subinput,src_col,values); } else values.resize(0); }
bool PLearn::ProcessSymbolicSequenceVMatrix::is_true | ( | const TVec< TVec< pair< int, int > > > & | conditions, |
const Vec | row | ||
) | const [inline, private] |
Definition at line 244 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), and fill_current_row().
{ bool condition_satisfied; for(int i=0; i<conditions.length(); i++) { condition_satisfied = true; if(conditions[i].length() == 0) PLERROR("ProcessSymbolicSequenceVMatrix::is_true :\n" "conditions[%d] should not be empty\n", i); for(int j=0; j<conditions[i].length(); j++) { if(row[conditions[i][j].first] != conditions[i][j].second) { condition_satisfied = false; break; } } if(condition_satisfied) return true; } return false; }
void PLearn::ProcessSymbolicSequenceVMatrix::makeDeepCopyFromShallowCopy | ( | CopiesMap & | copies | ) | [virtual] |
Transform a shallow copy into a deep copy.
Reimplemented from PLearn::SourceVMatrix.
Definition at line 501 of file ProcessSymbolicSequenceVMatrix.cc.
References conditions, current_row_i, PLearn::deepCopyField(), delimiters, element, ignored_context, indices, left_context, left_positions, PLearn::SourceVMatrix::makeDeepCopyFromShallowCopy(), right_context, right_positions, row, string_conditions, string_delimiters, string_ignored_context, subinput, and target_element.
{ inherited::makeDeepCopyFromShallowCopy(copies); deepCopyField(conditions, copies); deepCopyField(string_conditions, copies); deepCopyField(delimiters, copies); deepCopyField(string_delimiters, copies); deepCopyField(ignored_context, copies); deepCopyField(string_ignored_context, copies); deepCopyField(current_row_i, copies); deepCopyField(indices, copies); deepCopyField(left_positions, copies); deepCopyField(right_positions, copies); deepCopyField(left_context, copies); deepCopyField(right_context, copies); deepCopyField(subinput, copies); deepCopyField(row, copies); deepCopyField(element, copies); deepCopyField(target_element, copies); }
Reimplemented from PLearn::SourceVMatrix.
Definition at line 309 of file ProcessSymbolicSequenceVMatrix.h.
TVec< TVec< pair<int,int> > > PLearn::ProcessSymbolicSequenceVMatrix::conditions |
Conditions to be satisfied for the exclusion or inclusion (see conditions_for_exclusion) of elements in the VMatrix.
Definition at line 171 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), declareOptions(), and makeDeepCopyFromShallowCopy().
Indication that the specified conditions are for the exclusion (true) or inclusion (false) of elements in the VMatrix.
Definition at line 152 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), and declareOptions().
Offset for the position of the element on which conditions are tested (default = 0).
Definition at line 148 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), and declareOptions().
this_hash_map PLearn::ProcessSymbolicSequenceVMatrix::current_context_pos [mutable, private] |
Mapping from the position in the source sequence to the position in the last context.
Definition at line 114 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), and fill_current_row().
Vec PLearn::ProcessSymbolicSequenceVMatrix::current_row_i [mutable, private] |
Last context.
Definition at line 111 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), fill_current_row(), getExample(), getNewRow(), and makeDeepCopyFromShallowCopy().
int PLearn::ProcessSymbolicSequenceVMatrix::current_target_pos [mutable, private] |
Position in the last context of the target element (symbol and attributes)
Definition at line 104 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), and fill_current_row().
TVec< TVec< pair<int,int> > > PLearn::ProcessSymbolicSequenceVMatrix::delimiters |
Delimiters of context.
Definition at line 178 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), declareOptions(), fill_current_row(), and makeDeepCopyFromShallowCopy().
Vec PLearn::ProcessSymbolicSequenceVMatrix::element [mutable, private] |
Definition at line 123 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), fill_current_row(), and makeDeepCopyFromShallowCopy().
Indication to exclude from the VMatrix the context centered around a token with missing value target fields.
Definition at line 167 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), and declareOptions().
Indication that the context is of fixed length.
Definition at line 94 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), fill_current_row(), and getExample().
Indication that ignored elements of context should be replaced by the next nearest valid element.
Definition at line 156 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), declareOptions(), and fill_current_row().
Elements to be ignored in context.
Definition at line 184 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), declareOptions(), fill_current_row(), and makeDeepCopyFromShallowCopy().
Indices of the selected contexts.
Definition at line 117 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), fill_current_row(), getValues(), and makeDeepCopyFromShallowCopy().
Vec PLearn::ProcessSymbolicSequenceVMatrix::left_context [mutable, private] |
Temporary fields.
Definition at line 123 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), fill_current_row(), and makeDeepCopyFromShallowCopy().
TVec<int> PLearn::ProcessSymbolicSequenceVMatrix::left_positions [mutable, private] |
Temporary fields.
Definition at line 120 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), fill_current_row(), and makeDeepCopyFromShallowCopy().
int PLearn::ProcessSymbolicSequenceVMatrix::lower_bound [mutable, private] |
Position in the source sequence of the first element in the last context.
Definition at line 107 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), and fill_current_row().
Maximum length of a context.
Definition at line 96 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), getDictionary(), getNewRow(), getStringVal(), getValString(), and getValues().
Number of attributes.
Definition at line 98 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), fill_current_row(), getExample(), and getNewRow().
Number of elements at the left of the target element.
If < 0, all elements to the left are included until a delimiter is met.
Definition at line 140 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), declareOptions(), fill_current_row(), getValues(), and ProcessSymbolicSequenceVMatrix().
Number of elements at the right of the target element.
If < 0, all elements to the right are included until a delimiter is met
Definition at line 144 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), declareOptions(), fill_current_row(), and ProcessSymbolicSequenceVMatrix().
Indication that the only target fields of the VMatrix rows should be the (target) attributes of the context's target element.
Definition at line 160 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), declareOptions(), getExample(), getNewRow(), and getValues().
Vec PLearn::ProcessSymbolicSequenceVMatrix::right_context [mutable, private] |
Definition at line 123 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), fill_current_row(), and makeDeepCopyFromShallowCopy().
TVec<int> PLearn::ProcessSymbolicSequenceVMatrix::right_positions [mutable, private] |
Definition at line 120 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), fill_current_row(), and makeDeepCopyFromShallowCopy().
Vec PLearn::ProcessSymbolicSequenceVMatrix::row [mutable, private] |
Definition at line 123 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), and makeDeepCopyFromShallowCopy().
TVec< TVec< pair<int,string> > > PLearn::ProcessSymbolicSequenceVMatrix::string_conditions |
Conditions, in string format, to be satisfied for the exclusion or inclusion (see conditions_for_exclusion) of elements in the VMatrix.
Definition at line 175 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), declareOptions(), and makeDeepCopyFromShallowCopy().
TVec< TVec< pair<int,string> > > PLearn::ProcessSymbolicSequenceVMatrix::string_delimiters |
Delimiters, in string format, of context.
Definition at line 181 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), declareOptions(), and makeDeepCopyFromShallowCopy().
TVec< TVec< pair<int,string> > > PLearn::ProcessSymbolicSequenceVMatrix::string_ignored_context |
Elements, in string format, to be ignored in context.
Definition at line 187 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), declareOptions(), and makeDeepCopyFromShallowCopy().
Vec PLearn::ProcessSymbolicSequenceVMatrix::subinput [mutable, private] |
Definition at line 123 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by getValues(), and makeDeepCopyFromShallowCopy().
Vec PLearn::ProcessSymbolicSequenceVMatrix::target_element [mutable, private] |
Definition at line 123 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), fill_current_row(), and makeDeepCopyFromShallowCopy().
int PLearn::ProcessSymbolicSequenceVMatrix::upper_bound [mutable, private] |
Position in the source sequence of the last element in the last context.
Definition at line 109 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by build_(), and fill_current_row().
Indication that the last accessed context should be put in a buffer.
Definition at line 163 of file ProcessSymbolicSequenceVMatrix.h.
Referenced by declareOptions(), and fill_current_row().